No title

ADVANCES IN IMAGING AND ELECTRON PHYSICS VOLUME 111 This Page Intentionally Left Blank Advances in Imaging and Ele...

Author: Peter W. Hawkes | Benjamin Kazan | Tom Mulvey

99 downloads 995 Views 6MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

ADVANCES IN IMAGING AND ELECTRON PHYSICS VOLUME 111

This Page Intentionally Left Blank

Advances in

Imaging and Electron Physics EDITED BY PETER W. HAWKES CEMES/Laboratoire d’Optique Electronique du Centre National de la Recherche Scientifique Toulouse, France

VOLUME 111

San Diego San Francisco New York Boston London Sydney Tokyo

This book is printed on acid-free paper. Copyright © 1999 by Academic Press All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system, without permission in writing from the publisher. The appearance of the code at the bottom of the first page of a chapter in this book indicates the Publisher’s consent that copies of the chapter may be made for personal or internal use of specific clients. This consent is given on the condition, however, that the copier pay the stated per-copy fee through the Copyright Clearance Center, Inc. (222 Rosewood Drive, Danvers, Massachusetts 01923), for copying beyond that permitted by Sections 107 or 108 of the U.S. Copyright Law. This consent does not extend to other kinds of copying, such as copying for general distribution, for advertising or promotional purposes, for creating new collective works, or for resale. Copy fees for pre-1998 chapters are as shown on the title pages: if no fee code appears on the title page, the copy fee is the same as for current chapters. 1076-5670/99 $30.00 ACADEMIC PRESS A Harcourt Science and Technology Company 525 B. St. Suite 1900, San Diego, California 92101-4495, USA http://www.apnet.com Academic Press 24 – 28 Oval Road, London NW1 7DX, UK http://www.hbuk.co.uk/ap/ International Standard Serial Number: 1076-5670 International Standard Book Number: 0-12-014753-X Typeset by Laser Words, Madras, India Printed in the United States of America 99 00 01 02 03 BB 9 8 7 6 5 4

3 2 1

CONTENTS

CONTRIBUTORS . . . . . . . . . . . . . . . . . . . . . . . PREFACE . . . . . . . . . . . . . . . . . . . . . . . . . FORTHCOMING CONTRIBUTORS . . . . . . . . . . . . . . . . . . .

vii ix xi

Number Theoretic Transforms and their Applications in Image Processing S. BOUSSAKTA AND A. G. J. HOLT I. Introduction . . . . . . . . . . . . . . . . . . . . . . II. Number Theoretic Transforms and Their Application to Convolutions . III. Application of Two-dimensional Fermat Number Transforms to the Calculation of 2-D Convolutions . . . . . . . . . . . . . . IV. 2-D NTTs of Periodic Functions and Their Applications . . . . . . V. Another Family of Number Theoretic Transforms using the Mersenne Numbers (NMNTs) . . . . . . . . . . . . . . . . . . . VI. Combination of 2-D NTTs Using the 2-D MRC Suitable for Parallel Image-Processing Applications . . . . . . . . . . . . . . . VII. Hardware Implementations . . . . . . . . . . . . . . . . VIII. Conclusions . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . List of Abbreviations and Symbols . . . . . . . . . . . . . Acknowledgments . . . . . . . . . . . . . . . . . . .

3 4 9 38 53 65 73 80 85 89 90

On the Electron-Optical Properties of the ZrO/W Schottky Electron Emitter M. J. FRANSEN, Th. L. VAN ROOY, P. C. TIEMEIJER, M. H. F. OVERWIJK, J. S. FABER, AND P. KRUIT I. II. III. IV. V. VI. VII.

Introduction . . . . . . . . . . . Electron Emission Theory . . . . . . Boersch Effect for Electron Emitters . . Experimental Methods and Systems . . Experiments on ZrO/W Schottky Emitters Applications to Other Emitters . . . . Conclusions . . . . . . . . . . . References . . . . . . . . . . .

v

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

92 95 116 119 128 156 162 164

vi

CONTENTS

The Size of Objects in Natural and Artificial Images LUIS ALVAREZ, YANN GOUSSEAU, AND JEAN-MICHEL MOREL I. II. III. IV. V. VI. VII. VIII.

Introduction . . . . . . . . . . . . . . . . . . . . Statistics of Natural Images: A Review . . . . . . . . . . Sizes of Sections in Natural Images . . . . . . . . . . . Size of Sections and the BV Norm of Natural Images . . . . The Dead Leaves Model . . . . . . . . . . . . . . . A Short Review of Texture Synthesis by Mathematical Methods . Some Principles for the Synthesis of Abstract Natural Images . . Appendix: Matheron’s Dead Leaves Model . . . . . . . . References . . . . . . . . . . . . . . . . . . . . Acknowledgements . . . . . . . . . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

168 170 175 186 192 210 228 235 239 242

. . . . . . .

244 254 259 263 281 312 323 325

. . . . . . . .

327 332 335 342 351 362 365 365

Reconstruction of Nuclear Magnetic Resonance Imaging Data from Non-Cartesian Grids GORDON E. SARTY I. II. III. IV. V. VI. VII.

Introduction . . . . . . . . . . . . . . . . . Gridding Reconstruction and Direct Reconstruction . . Natural k-Plane Coordinate Systems . . . . . . . . Integrating Curve Band-pass Operators . . . . . . . Curve Band-pass Operator Point-Spread Functions . . . Direct Reconstruction Using Natural k-Plane Coordinates Conclusion . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

.

An Integrated Approach to Computational Vision SIBEL TARI I. II. III. IV. V. VI.

Introduction . . . . . . . . . . . . . . . . . Review of the Edge-Strength Function . . . . . . . Geometry of the Edge-Strength Function . . . . . . Extraction and Representation of Perceptual Information . Shape Skeleton and Shape Segmentation (2-D Case) . . Coloring Nested Symmetries in Higher Dimensions . . Abstract . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

Index . . . . . . . . . . . . . . . . . . . . . . . . . .

367

CONTRIBUTORS

Numbers in parentheses indicate the pages on which the author’s contribution begins.

LUIS ALVAREZ (167), Departmento de Informatica y Sistemas, Universidad de Las Palmas, Campus de Tafira, Las Palmas, Spain, 35017. S. BOUSSAKTA (1), Section of Electrical and Electronic Engineering, University of Teeside, Middlesborough, Cleveland, United Kingdom, TS1 3BA. J. S. FABER (91), Philips Electron Optics/FEI, Eindhoven, The Netherlands. M. J. FRANSEN (91), Philips Analytical, Lelyweg 1, Almelo, The Netherlands, 7602 EA. YANN GOUSSEAU (167), Centre de Mathematiques et leurs Applications, ENS Cachan, 61 av. du President Wilson, Cachan Cedex, France, 94235. A. G. J. HOLT (1), Department of Electrical and Electronic Engineering, University of Newcastle, Newcastle upon Tyne, United Kingdom, NE1 7RU. P. KRUIT (91), Delft University of Technology, Department of Applied Physics, Delft, The Netherlands. JEAN-MICHEL MOREL (167), Centre de Mathematiques et leurs Applications, ENS Cachan, 61 av. du President Wilson, Cachan Cedex, France, 94235. M. H. F. OVERWIJK (91), Philips Analytical, Lelyweg 1, Almelo, The Netherlands, 7602 EA. GORDON E. SARTY (243), University of Saskatchewan, Department of Medical Imaging, Royal University Hospital, 103 Hospital Drive, Saskatoon, Saskatchewan, Canada, S7N 0W8.

vii

viii

CONTRIBUTORS

SIBEL TARI (327), Department of Engineering Sciences, Middle East Technical University, Ankara, Turkey, 06531. TH. L. VAN ROOY (91), Philips Analytical, Lelyweg 1, Almelo, The Netherlands, 7602 EA.

PREFACE Four of the the five contributions to this volume, some of which form short monographs on their topic, cover various aspects of image character, image manipulation and computer vision while the fifth is a detailed study of the Schottky electron emitter. We begin with a presentation of number-theoretic transforms and their utility in image processing by S. Boussakta and A.G.J. Holt. Both the basic theory and some account of the appropriate hardware are included, which makes this a self-contained study of an interesting area. A family of transforms based on the Mersenne numbers, introduced by the authors, is described fully. The literature on these questions is already extensive and the attractive features of these integer transforms are well known but they are only slowly gaining acceptance. I hope that this survey will lead more scientists to try them out. The second contribution, by M.J. Fransen, T.L. van Rooy, P.C. Tiemeijer, M.H.F. Overwijk, J.S. Faber amd P. Kruit, provides much new information about a surprisingly little studied type of electron source, the Schottky emitter consisting of a thin layer of zirconium oxide on tungsten. Although such emitters are widely used and their properties and advantages are known in general terms, there have been very few systematic studies and no reliable sets of measurements are available. “Are the present values of brightness and energy spread of the ZrO/W. . .emitter in the electron optical instruments in which these sources are employed in accordance with theoretical values?. . . What are the optimal values of the tip diameter, field strength, and emitter temperature for a certain application?. . .Is it possible to estimate the enegy width at all field strengths with a simple model?. . .What is the field on the emitter surface?” These are the questions to which the authors of this chapter provide answers and I have no doubt that it will rapidly become the standard reference for these emitters. Next, we have a study by L. Alvarez, Y. Gousseau and J.-M. Morel of scaling phenomena and image statistics, which provides a readable account of material that has hitherto been scattered over numerous publications. The seven sections have been organized in such a way that they can be read separately, although of course ‘the whole is greater than the sum of its parts.’ The chapter opens with a review of the statistics of natural images and continues with a discussion of the notion of size. The BV (bounded variation) norm is then introduced after which a long section explores the dead-leaves model. Next comes a short review of mathematical methods of performing texture synthesis and the seventh section presents some principles for the synthesis of abstract ix

x

PREFACE

images. The authors return to Matheron’s dead-leaves model in an appendix. The subject is thus very fully covered and here too, I am sure that students of these questions will frequently consult this authoritative survey. We then turn to a problem that has arisen recently in magnetic resonance imaging, where image acquisition is accelerated by replacing the traditional cartesian grid by more complicated patterns such as spirals or rosettes. How are the resulting data to be exploited? G.E. Sarty opens his chapter with a general introduction that will enable non-specialists to understand the nature of the problem and its practical importance. He then turns to gridding reconstruction and then moves on to k-plane coordinate systems. Two sections then explain the notions of curve bandpass operators and the associated point-spread functions and a final section describes direct reconstruction using natural k-plane coordinates. Although this may seem of rather specialist interest, it should be remembered that data are acquired in the spatial frequency plane in other areas too — radio astronomy is the obvious example — so the techniques presented are of direct interest beyond the world of MRI. The volume concludes with a short contribution by S. Tari on computer vision based on the edge-strength function. The relation between the human visual system and perception and the information gathered by a machine is examined and several ways of identifying shape and edges are examined. I thanks most warmly all the contributors to this volume for the time and trouble they have devoted to making their material accessible to readers less familiar with it and close with a list of chapters planned for future volumes. Peter W. Hawkes

FORTHCOMING CONTRIBUTIONS

D. Antzoulatos Use of the hypermatrix N. D. Black, R. Millar, M. Kunt, F. Ziliani and M. Reid Second generation image coding N. Bonnet Artificial intelligence and pattern recognition in microscope image processing G. Borgefors Distance transforms A. van den Bos and A. Dekker Resolution P. G. Casazza Frames J. A. Dayton Microwave tubes in space E. R. Dougherty and D. Sinha Fuzzy morphology J. M. H. Du Buf Gabor filters and texture analysis R. G. Forbes Liquid metal ion sources E. Förster and F. N. Chukhovsky X-ray optics A. Fox The critical-voltage effect M. Gabbouj Stack filtering W. C. Henneberger The Aharonov–Bohm effect M. I. Herrera and L. Bru´ The development of electron microscopy in Spain xi

xii


K. Ishizuka Contrast transfer and crystal images C. Jeffries Conservation laws in electromagnetics M. Jourlin and J. -C. Pinoli Logarithmic image processing E. Kasper Numerical methods in particle optics A. Khursheed Scanning electron microscope design G. Kögel Positron microscopy K. Koike Spin-polarized SEM W. Krakow Sideband imaging A. van de Laak-Tijssen, E. Coets and T. Mulvey Memoir of J. B. Le Poole L. J. Latecki Well-composed sets W. Li Vector transformation C. Mattiussi The finite volume, finite element and finite difference methods S. Mikoshiba and F. L. Curzo Plasma displays R. L. Morri Electronic tools in parapsychology J. G. Nagy Restoration of images with space-variant blur P. D. Nellist and S. J. Pennycook Z-contrast in the STEM and its applications M. A. O’Keef Electron image simulation


G. Nemes Phase-space treatment of photon beams B. Olstad Representation of image operators C. Passow Geometric methods of treating energy transport phenomena E. Petajan HDTV F. A. Ponce Nitride semiconductors for high-brightness blue and green light emission J. W. Rabalais Scattering and recoil imaging and spectrometry H. Rauch The wave-particle dualism D. Saldin Electron holography G. Schmahl X-ray microscopy J. P. F. Sellschop Accelerator mass spectroscopy S. Shirai CRT gun design methods T. Soma Focus-deflection systems and their applications I. Talmo Study of complex fluids by transmission electron microscopy J. Toulouse New developments in ferroelectrics T. Tsutsui and Z. Dechun Organic electroluminescence, materials and devices Y. Uchikawa Electron gun optics D. van Dyck Very high resolution electron microscopy

xiii

xiv


J. S. Villarrubia Mathematical morphology and scanned probe microscopy L. Vincent Morphology on graphs N. White Multi-photon microscopy J. B. Wilburn Generalized ranked-order filters C. D. Wright and E. W. Hill Magnetic force microscopy

ADVANCES IN IMAGING AND ELECTRON PHYSICS, VOL. 111

Number Theoretic Transforms and their Applications in Image Processing S. BOUSSAKTA1 AND A. G. J. HOLT2 1 Section

of Electrical and Electronic Engineering, University of Teesside, Middlesborough, Cleveland, TS1 3BA, U.K. 2 Department of Electrical and Electronic Engineering, University of Newcastle, Newcastle upon Tyne, NE1 7RU, U.K.

I. Introduction . . . . . . . . . . . . . . . . . . . . . .

3

II. Number Theoretic Transforms and Their Application to Convolutions . . .

4

A. Definition of NTTs . . . . . . . . . . . . . . . . . . .

4

B. General Constraints for Choosing NTTs for Digital Image and Signal Processing . . . . . . . . . . . . . . . . . . .

5

C. Choice of the Modulus and Kernel

6

. . . . . . . . . . . . .

1. Mersenne Transforms . . . . . . . . . . . . . . . . . 2. Fermat Number Transforms . . . . . . . . . . . . . . .

6 6

D. Considerations of Transforms and Word Lengths . . . . . . . . .

7

1. Word Length . . . . . . . . . . . . . . . . . . . .

8

E. Complex Convolution Using the NTTs . . . . . . . . . . . . III. Application of Two-Dimensional Fermat Number Transforms to the Calculation of 2-D Convolutions . . . . . . . . . . . . . . . . . . . 1. Definition of the Two-Dimensional Fermat Number Transform (2-D FNT) . . . . . . . . . . . . . . . . . 2. The Relationship between Transform Size and Modulus for 2-D FNT . . . . . . . . . . . . . . . . . . 3. Arithmetic Complexity of the 2-D Fermat Number Transforms 4. The Convolution Property of the 2-D FNT . . . . . . . 5. Examples . . . . . . . . . . . . . . . . . .

. . .

8 9 9

. . . .

9 10 10 11

A. Two-Dimensional Convolution with NTTs without Matrix Transpose and without Overlap . . . . . . . . . . . . . . . . . .

11

1. The Twiddle Factor ˛qs . . . . . . . . . . . . . . . .

16

a. Computation of the 2-D NTT . . . . . . . . . . . . . b. Example of 2-D Autoconvolution Obtained from Simulations . . .

19 20

B. Number Domain Analysis in Two Dimensions with Possible Applications . . . . . . . . . . . . . . . . . .

24

1. Simple Images in the 2-D NTT Domain . . . . . . . . . . . 2. Mathematical Development . . . . . . . . . . . . . . .

24 24

Volume 111 ISBN 0-12-014753-X

. . . .

. . . .

ADVANCES IN IMAGING AND ELECTRON PHYSICS Copyright © 1999 by Academic Press All rights of reproduction in any form reserved. ISSN 1076-5670/99 $30.00

2

S. BOUSSAKTA AND A. G. J. HOLT 3. Square Images . . . . . . . . . . . . . . . . . . . a. b. c. d. e. f. g. 4. 5. 6. 7. 8. 9.

Square Square Square Square Square Square Square

of of of of of of of

Side Side Side Side Side Side Side

D2 D3 D4 D5 D6 D7 D8

25

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

25 27 27 28 28 30 30

Rectangular Images . . . . . . . . . Summary of the Results . . . . . . . Generalization and Possible Applications . The Effect of Translation on the Zero Pattern Possible Applications . . . . . . . . Summary . . . . . . . . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

30 32 32 34 36 38

L L L L L L L

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

IV. 2-D NTTs of Periodic Functions and Their Applications . . . . . . . . A. Analysis of the 2-D FNT of Periodic Structures . . . . . . . . . Case 1: Tr D Tc . . . . . Generalization for 2-D FNT . . The Effect of Data Errors on 2-D Summary of the Results . . .

38 38

. . .

39 41 44 44

B. Applications . . . . . . . . . . . . . . . . . . . . .

46

C. Implementation of the Inspection Method . . . . . . . . . . .

49

1. 2. 3. 4.

1. 2. 3. 4.

. . . . FNT . .

. . . . . . . .

. . . . . . . .

. . . . . . . .

Implementation with F3 . . . . . . . . . . Implementation Using 1-D FNT . . . . . . . Number Theoretic Transform with Independent Length Summary . . . . . . . . . . . . . . .

. . . . . . . .

. . . . . . . .

.

. . . .

49 50 50 53

V. Another Family of Number Theoretic Transforms using the Mersenne Numbers (NMNTs) . . . . . . . . . . . . . . . .

53

1. Transform Definition . . . . . . . . . . . . . . . . .

53

a. The Forward Transform . . . . . . . . . . . . . . . b. The Inverse Transform . . . . . . . . . . . . . . .

54 54

2. Properties of the ˇ Function . . . . . . . . . . . . . . 3. Calculation of 1-D Convolutions . . . . . . . . . . . . . 4. Fast Algorithms for This Transform . . . . . . . . . . . .

54 55 56

a. Radix-2 Algorithm Derivation . . . . . . . . . . . . . b. Arithmetic Complexity . . . . . . . . . . . . . . .

56 57

B. The 2-D New Mersenne Number Transform (2-D NMNT) . . . . . .

58

1. 2. 3. 4.

. . . . . . . . . . and Modulus . . . . .

Definition of the 2-D Transform . . . . . . . . Relationship Between Transform Sizes and Modulus . . Calculation of 2-D Convolutions Using the 2-D NMNT . Multidimensional Transform . . . . . . . . . .

. . .

58 59 59 60

a. The M-D Forward Transform . . . . . . . . . . . . . b. The M-D Inverse Transform . . . . . . . . . . . . . .

60 61

5. The Calculation of Multidimensional Convolution

. . . . . . . .

. . . . . . . .

.

. . . . . . .

61

NUMBER THEORETIC TRANSFORMS AND THEIR APPLICATIONS

3

C. The Separable Two-Dimensional and Multidimensional Mersenne Transforms . . . . . . . . . . . . . . . . . .

62

1. Definition of the Separable Two-Dimensional Transform (2-D NMNT) . . . . . . . . . . . . . . . . . . . .

62

a. The Forward Transform . . . . . . . . . . . . . . . b. The Inverse Transform . . . . . . . . . . . . . . .

62 63

2. Arithmetic Complexity . . . . . . . . . . . . . . . . 3. The Calculation of 2-D Convolution . . . . . . . . . . . . 4. Generalization of This Transform . . . . . . . . . . . . .

63 63 64

a. The Forward Multidimensional Transform . . . . . . . . . b. The Inverse Multidimensional Transform . . . . . . . . .

64 64

5. Summary . . . . . . . . . . . . . . . . . . . . .

65

VI. Combination of 2-D NTTs Using the 2-D MRC Suitable for Parallel Image-Processing Applications . . . . . . . . . . . . . . . .

65

A. Introduction . . . . . . . . . . . . . . . . . . . . .

65

B. Two-Dimensional Composite Transform (2-D CNTT) . . . . . . .

66

C. Calculation of the 2-D Cyclic Convolution Using This Method . . . .

67

D. Two-Dimensional Mixed Radix Conversion (2-D MRC)

67

. . . . . .

E. Combination of the 2-D NMNT and 2-D FNT for Two Moduli . . . .

68

F. Advantages of This method . . . . . . . . . . . . . . . .

72

VII. Hardware Implementations . . . . . . . . . . . . . . . . . A. TTL-based Designs . . . . . . . . . . . . . . . . . . .

73 73

B. Pipelined Designs . . . . . . . . . . . . . . . . . . .

74

C. Vector Radix Method . . . . . . . . . . . . . . . . . .

74

VIII. Conclusions . . . . . . . . . . . . . . . . . . . . . .

80

References . . . . . . . . . . . . . . . . . . . . . . .

85

List of Abbreviations and Symbols . . . . . . . . . . . . . . .

89

Acknowledgments . . . . . . . . . . . . . . . . . . . .

90

I. INTRODUCTION This chapter briefly introduces the concepts of number theoretic transforms and their applications to 2-D convolutions. The use of NTTs for 2-D convolutions is discussed and a method for calculating them without matrix transpose and without overlap is explained with examples. Although the transform domain for 2-D NTTs is not as simple to interpret as for some other transforms, the 2-D FNT domain does have special patterns of zeros and nonzero values that could find applications. These are discussed for a number of simple images and also for 2-D periodic structures. The effects of small errors in the periodic data on the 2-D FNT are discussed and applications are considered. Another family of transforms based upon the Mersenne numbers is introduced together

4

S. BOUSSAKTA AND A. G. J. HOLT

with its separable form. The transform is extended to the multidimensional case. It is also shown how this transform can be combined with the 2-D FNT, using the 2-D mixed radix conversion, to provide extended dynamic range and great parallelism using simple and convenient moduli leading to an efficient method for parallel image-processing applications. Because of the somewhat inconvenient word lengths required by some NTTs, the use of VLSI, where the word length can be chosen to suit the design, has had a dramatic effect on circuit design for NTTs. A number of designs for pipeline FNT processors, including a vector radix structure, are discussed, and block diagrams for their operation are included. II. NUMBER THEORETIC TRANSFORMS AND THEIR APPLICATION TO CONVOLUTIONS Number theoretic techniques have been discussed by many authors, including Pollard [1971], and Rader [1972a, b], and their application to digital signal processing has been surveyed in an illuminating paper by Jullien [1991] and in other references [McClellan and Rader, 1970; Rabiner and Gold, 1975; Agarwal and Burrus, 1974a, b, c; King et al., 1989; Gregory and Krishnamurthy, 1984; Martens, 1983; Reed et al., 1977; Schroeder, 1990; Myers, 1990]. Here we briefly discuss some aspects of the theory of number theoretic transforms and their applications to image processing. The well-known fast Fourier transform (FFT) method for calculating the discrete Fourier transform (DFT) may involve significant amounts of roundoff error [Oppenheim and Weinstein, 1972; Meyer, 1989] and also require the storage or generation of the complex basis functions — i.e., sine and cosine — which must be rounded. Thus, exact results for the FFT are not possible on a digital machine. This encouraged the investigation of other transforms that retain the circular convolution property (CCP) and also reduce rounding errors and computational load. This search has led to the introduction of a new family of transforms defined in finite fields. These are known as number theoretic transforms (NTTs). A. Definition of NTTs Let F be a prime integer and ˛ be a root of unity of order N; then the NTT defined in the Galois field, GF[F] modulo F, is given by: Xk D

N1

xn ˛nk

mod F

nD0

k D 0, 1, 2, . . . , N 1

1

5


Since N and F are chosen to be relatively prime, N1 exists and an inverse can be defined: xn D 1/N

N1

Xk ˛nk

mod F

2

kD0

n D 0, 1, 2, . . . , N 1 NTTs in general have the cyclic convolution property and can be used to calculate the convolutions by the same method as the FFT. NTTs have some advantages over the FFT: 1. NTTs do not need any manipulation of the trigonometric functions (sine and cosine). 2. They are calculated modulo an integer and hence exact results are possible. 3. Multiplication-free transforms can be achieved through an appropriate choice of the transform parameters. B. General Constraints for Choosing NTTs for Digital Image and Signal Processing Using transforms defined in finite fields means that the problem of the accumulation of the round-off errors resulting from the use of the DFT is now solved. All arithmetic operations are in integer form carried out modulo some integer F, so no rounding and/or truncation errors occur, and because all results are residue reduced, no overflow occurs either. This makes exact results in the calculation of convolution and correlation possible, which is a desirable feature for any transform. Thus, no additional noise arising in the calculation is introduced by the transforms, which may be important in critical cases. If the desirable features of NTTs are considered, the ideal could be as follows: 1. Simple binary representation for the modulus simplifies arithmetic operations. 2. In order to avoid complex multiplications, the multiplication by powers of ˛ in the NTT should be as simple as possible. The case ˛ D 2 meets this condition. 3. The number of arithmetic operations required should be small. 4. The transform should have the cyclic convolution property. Besides these desirable features, the following conditions must be met in defining any NTTs:

6


1. N and F must be relatively prime so that N1 exists, ensuring that the inverse transform exists. 2. The kernel ˛ must be a root of unity of order N; i.e., N is the least positive integer such that ˛N D 1 mod F and ˛t 6D 1 for 0 < t < N. 3. In the case of F composite, (1 ˛t ) and F must be mutually prime for 0 < t < N, or N should divide the gcd(p1 1, p2 1, pk 1), where pi m2 mk are the prime factors of F (F D pm1 1 p2 Ð Ð Ð pk ) [Agarwal and Burrus, 1974a]. The condition that ˛ must be a root of unity of order N ensures the CCP and causes the rigid relationship between the modulus, the transform length, and the kernel ˛. This condition can be neglected when the NTTs are applied to applications other than convolution. C. Choice of the Modulus and Kernel The parameters that must be chosen for NTTs are the modulus F, the transform length N, and the basis function ˛. Frequently the choice starts by selecting an F suitable for the computation on binary processors (requiring a number of bits compatible with the processor used and a representation of few binary bits in order to facilitate the arithmetic operations), followed by the selection of suitable values for N and ˛. 1. Mersenne Transforms These transforms have as modulus the Mersenne primes F D 2p 1, with p being prime. This transform has the cyclic convolution property and can calculate cyclic convolutions. Although the Mersenne number transforms (MNTs) have simple arithmetic operations and kernel, the transform length p or 2p (corresponding to ˛ D 2 or 2, respectively) is not highly composite, making the application of fast algorithms impossible because p is prime [Rader, 1972a]. Transforms that solve these problems will be introduced in a later section. 2. Fermat Number Transforms The limitations of Mersenne transforms in terms of transform length flexibility and the lack of fast algorithms led Rader [1972a] and Agarwal and Burrus [1974a] to consider primes of the form Ft D 2b C 1 with b D 2t . This type of modulus, known as the Fermat numbers (Ft , t D 0, 1, 2, . . .), is suitable for NTTs. Transforms defined in these fields or rings are known as Fermat number transforms (FNTs). Only the first five Fermat numbers are known to be primes


7

(F0 , . . . , F4 ). For these cases we can define transforms up to Ft 1. For any Fermat numbers, it is always possible to define transforms for any N that is a power of 2 up to N D 2tC2 for composite Fermat numbers and for any N that is power of 2 less than Ft for prime Fermat numbers. These transforms can be implemented readily using radix 2, radix 4 [Brigham, 1974], split radix [Duhamel and Holmann, 1984], etc., decimation in time or in frequency FFT-type algorithms. It is desirable to have the multiplication by ˛ and its powers as simple as possible, preferably with ˛ D 2 or a power of 2. If ˛ D 2, then the multiplication by ˛ will reduce to bit shifts, giving the FNTs an advantage in speed and cost. It can be shown that 2 is a root of order N D 2tC1 mod Ft . Hence a transform with the simplest possible kernel exists in finite field defined modulo the Fermat numbers. Applications of the FNT are given in Siu and Constantinides [1983, 1984], Marshal and Soraghan [1988], Morandi et al. [1988], Boussakta and Holt [1988], Boussakta [1990], Smith [1983], Marir [1986], Marshal and Soragan [1988], Reed et al. [1977], Reed and Truong [1978], and Schroeder [1990]. D. Considerations of Transforms and Word Lengths The condition for NTTs to have the cyclic convolution property has created a rigid relationship between the modulus, the transform length, and the kernel chosen. In fact, the maximum length for a multiplierless FNT is 256. This is well suited for moderate-length convolutions and image filtering. However, a problem arises when a longer transform length is needed, e.g., if the sequences to be processed are naturally long. Several solutions are available. Agarwall and Burrus [1974b] used the scheme proposed by Rader [1975] of mapping one-dimensional convolution to two-dimensional. Other authors proposed the use of mixed-radix NTTs with 2 as a root of unity and composite transform lengths at the expense of using moduli with more complex arithmetic [Duhamel and Hollman, 1982] or at the expense of reducing the dynamic range [Robanov et al., 1998; Li and Peterson, 1990]. Another solution is to use a kernel that is different from 2, e.g., 3. Using this as the basis function all the transform lengths up to the modulus (for prime numbers) are possible and the transform length problem is considerably alleviated. As is shown in Agarwal and Burrus [1974c], values of N up to 65,536 are possible with ˛ D 3. Owing to the advances in fabrication technology of integrated circuits (ICs), the multiplication operation is not as great a problem as it once was, and fast implementation of FNTs with ˛ 6D 2 is possible in VLSI. Compared with the FFT, the FNT provides a simpler butterfly structure involving one single integer multiplier and two integer adders per butterfly against four real multipliers and six real adders in the case

8


of the FFT. The use of look-up tables can also remove the need for fast multiplications. It is also possible to use a kernel which is two or a power of two in most stages and general multiplications in the other stages. This was used by Agarwal and Burrus [1974c] and Reed and Truong [1978] and can be generalized to higher-order roots. Unusual-length NTTs [Parker and Benaissa, 1995] may be derived using recursive extensions of Rader’s theorem. See also Lee and Lu [1988] and Liu, Huizhu, and Lee [1991]. 1. Word Length With NTTs all operations are calculated modulo an integer F. Thus, an integer number is correctly presented only if its absolute value does not exceed F/2. When calculating the convolution of two sequences x(n) and h(n) in a finite field or a ring, the amplitude of x(n) and h(n) must be scaled so that the convolution result, jynj, does not exceed F/2 [Agarwal and Burrus, 1974a, c]. Using the polar representation of the Fermat prime field [Alfredsen, 1994] has shown that this representation provides suitable choices of the transform kernel for which multiplication by power of the kernel mod 2b C 1 are carried out by one modulo addition. As pointed out by P. R. Chevillat [1978], the signal-to-noise ratio (SNR) of an NTT filter shows no explicit dependence on the transform length as is the case for the FFT, where, owing to scaling and round-off error accumulation, doubling the length reduces the SNR by more than 3 dB. E. Complex Convolution Using NTTs The calculation of complex convolution (CC) arises in radar and sonar, for example. The calculation is usually performed in the complex field using the FFT. However, the same procedure can be evaluated in a finite ring of complex integers ZF , where all the arithmetic operations are performed as in a normal complex field with the real and imaginary parts evaluated modulo an integer F. Using NTTs to calculate the CC gives the same result as the conventional methods, provided the conditions for the dynamic range are met for both real and imaginary parts [Vegh and Leibovitz, 1976; Reed, 1975]. There are many proposed complex NTTs, such as the complex pseudo Fermat number (CPFNT), complex Mersenne (CMNT), and complex pseudo Mersenne (CPMNT) transforms, which are attractive for certain lengths [Nussbaumer, 1977] and complex NTTs based on quadratic residue number systems [Krishnan et al., 1986a, b]. We emphasize the FNTs here because such transforms have been tested in both hardware and software and found to be efficient. They can be used to compute the complex convolution [Nussbaumer, 1976].

9


III. APPLICATION OF 2-D FERMAT NUMBER TRANSFORMS TO THE CALCULATION OF 2-D CONVOLUTIONS The calculation of 2-D convolutions and correlations in the time domain is expensive in time even for relatively small filter sizes; hence transform methods are of interest. Multiplierless 2-D FNTs have been shown to be well suited for 2-D convolutions, where many restrictions imposed upon them in one dimension are either lifted or are of less effect. The reason for this, as was pointed out by Rader [1975] is that the 2-D filters are relatively small in size and magnitude. This is compatible with the 2-D FNT transform lengths which can be doubled using the method in [Boussakta and Holt, 1991]; hence the FNTs are found to be suitable for 2-D filtering [Vanser Kraats and Venetsanopoulos, 1982]; [Shakaff, Pajayakrit, and Holt, 1988; Marir, Shakaff, and Holt, 1985; Ahmed and Nundararajan, 1987; Boussakta, 1990; Marir, 1986]. In this section the application of 2-D NTTs to the calculation of 2-D convolutions and correlations and related problems encountered are discussed with examples. 1. Definition of the Two-Dimensional Fermat Number Transform (2-D FNT) The 2-D FNT of an image xm, n of dimension M ð N may be written as Xk, l D

M1 N1

nl xm, n ˛mk 1 ˛2

mod Ft

3

mD0 nD0

k D 0, 1, . . . , M 1,

l D 0, 1, . . . , N 1

Here ˛1 and ˛2 are roots of unity of order M and N, respectively. It is also possible to calculate the transform using one prime number along the rows and another prime number along the columns, although for practical reasons the modulus is usually kept the same for both dimensions. The inverse transform is given by xm, n D 1/MN

M1 N1

Xk, l˛mk ˛nl 1 2

mod Ft

4

kD0 lD0

m D 0, 1, . . . , M 1,

n D 0, 1, . . . , N 1

2. The Relationship Between Transform Size and Modulus for the 2-D FNT The condition under which 2-D NTTs have the 2-D cyclic convolution property creates a rigid relationship between the modulus, the transform size, and the kernel chosen. Table 1 shows the relationship between the transform size and the modulus. From ˛1 and ˛2 of Table 1, the values of ˛1 and ˛2 for any transform size that is a power of 2 up to Nmax ð Nmax can be calculated.

10

S. BOUSSAKTA AND A. G. J. HOLT TABLE 1 THE RELATIONSHIP BETWEEN TRANSFORM SIZE AND THE MODULUS FOR THE 2-D FNT.

Ft 28 C 1 D F3 216 C 1 D F4 232 C 1 D F5 264 C 1 D F6

N ð N for ˛1 D ˛2 D 2

N ð N forp ˛1 D ˛2 D 2

Nmax ð Nmax

˛ D ˛1 D ˛2 for Nmax ð Nmax

16 ð 16 32 ð 32 64 ð 64 128 ð 128

32 ð 32 64 ð 64 128 ð 128 256 ð 256

28 ð 28 216 ð 216 27 ð 27 28 ð 28

3 p3 p2 2

This is similar to a table for the 1-dimensional case (Agarwal and Burrus [1974c]) (© 1974, lEEE).

3. Arithmetic Complexity of the 2-D Fermat Number Transforms For certain transform sizes, the calculation of the 2-D Fermat number transforms will involve shifts and additions only. Therefore, the arithmetic complexity of this transform will depend on the transform sizes and the modulus chosen, as shown in Table 2. Note in Table 2 that the number of multiplications includes shifts and trivial multiplications by š1. 4. The Convolution Property of the 2-D FNT The two dimensional Fermat number transforms have the 2-D cyclic convolution property and hence can be used for the calculation of 2-D convolution/correlation as follows: yc m, n D 2-D INFNT f2-D FNT[xm, n] 2-D FNT[hm, n]g TABLE 2 NUMBER OF ARITHMETIC OPERATIONS NEEDED, FOR TRANSFORM SIZE N ð N,

Modulus

Nshift ðNshift for ˛1 D ˛2 D 2

Number of shifts for ˛1 D ˛2 D 2

Number of additions for ˛1 D ˛2 D 2

28 C 1 216 C 1 232 C 1 264 C 1

16 ð 16 32 ð 32 64 ð 64 128 ð 128

N2 log2 N N2 log2 N N2 log2 N N2 log2 N

2N2 log2 N 2N2 log2 N 2N2 log2 N 2N2 log2 N

BY THE

5

2-D FNT.

Nmax ðNmax ˛1 D ˛2 D 3

Number of Multiplications ˛1 D ˛2 D 3

Number of Additions ˛1 D ˛2 D 3

256 ð 256 216 ð 216

N2 log2 N N2 log2 N

2N2 log2 N 2N2 log2 N

Nshift ð Nshift Dpmaximum transform size involving shifts and additions only. For ˛1 D ˛2 D 2, the transform sizes are 2Nshift ð 2Nshift , but the number of shifts and adds are slightly higher. Nmax ð Nmax D maximum transform size involving multiplication.


where yc m, n D

M1 N1

xi, jhhm iiM , hn jiN

11

6

iD0 jD0

and h iM means modulo M. In the 2-D circular convolution, the indices of the rows and the columns are calculated modulo M and modulo N, respectively. The linear convolution of an M1 ð N1 image and an L1 ð L2 filter can be obtained through the circular counterpart only if M ½ M1 C L1 1 and N ½ N1 C L2 1

7

Thus, the two images should be padded with zeros to form an extended image, as follows: xm, n for m D 0, 1, 2, . . . , M1 1; n D 0, 1, 2, . . . , N1 1 xm, n D 0 for M1 m < M and N1 n < N 8 hm, n for m D 0, 1, 2, . . . , L1 1; n D 0, 1, 2, . . . , L2 1 hm, n D 0 for L1 m < M and L2 n < N 9 5. Examples Figure 1 presents an example of self-convolving a square image of constant intensity, shown in Figure 1(a). The sequences are sufficiently padded with zero values, so the well-known symmetric pyramid shape is the result. High intensity in the middle of the picture is presented in gray levels as white; dark indicates low intensity, as shown toward the edges in Figure 1(b). The array size is increased beyond the bounds set by Equation (7). This results in wraparound error, which is indicated by loss of the symmetric shape of the picture. (The top and right sides are missing, and the lower and left sides are in error; Figure 1(c).) Also, for the convolution calculated by means of 2-D NTTs to be equivalent to the ordinary convolution, the input data must be scaled so that the output must not exceed the modulus. Figure 2(a) shows a linear 2-D convolution verifying this condition, and Figure 2(b) shows an overflow indicated by varying intensity from white to black and vice versa. A. Two-Dimensional Convolution with NTTs without Matrix Transpose and without Overlap A problem arises in two-dimensional processing using the convolution method because of the large information content of the images (of the order of 1000 ð

12


(b)

(a)

(c)

FIGURE 1. Wraparound errors in 2-D convolution. (a) Square image of constant intensity. (b) Exact self-convolution. (c) Inexact self-convolution.

1000 discrete points). It may be necessary to determine the convolution of a filter with a large signal by successive convolutions of the filter with sections of the image. The well-known overlap-save technique for two-dimensional image processing requires that the image be divided into sections and the original image size extended with a frame of zeros, as described earlier. Each extended section is convolved with the filter, which has also been extended with zeros. Only a limited number of the results of these convolutions are retained as useful. This process is repeated for all the sections until the whole image is convolved. Hunt [1972] has determined the size of the optimum sections for use with this method.


(a)

13

(b)

FIGURE 2. Calculation of 2-D convolution using 2-D NTTs. (a) Exact linear convolution. (b) Inexact linear convolution due to integer overflow.

Hunt’s approach follows that of Helms [1967] in minimizing the quantity B D N1 N2 log2 N1 C log2 N2 /N1 L1 C 1N2 L2 C 1

10

Here N1 and N2 are the dimensions of the sections to be optimized and L1 and L2 are the filter dimensions. B is minimized and the results are tabulated. The constraint is that N1 and N2 are to be integer powers of 2. For a 30 ð 30 filter, Hunt’s table gives the optimum section as 512 ð 512 in dimension. Image arrays of the form 22u ð 22u may be convolved with a filter without recourse to sectioning. Also, the process may be computed without the need for matrix transposition [Marir, Shakaff, and Holt, 1985]. Consider images of 1024 ð 1024 pixels. The linear convolution y D x Ł h is exactly a circular convolution and can be computed using two forward NTTs and one inverse NTT if the relation N ½ L C H 1 is fulfilled, where N is the transform length, L is the length of the signal, and H is the filter length. The convolution becomes y D INV-NTT[NTTx NTTh] 11 The two-dimensional NTT can be written in its general form as follows: Xk, l D

N 1 1 N 2 1

1 hnli2 xm, n˛hmki ˛2 1

mod F

12

mD0 nD0

where h i1 stands for mod N1 and h i2 stands for mod N2 . Here ˛1 and ˛2 are roots of unity of order N1 and N2 , respectively. We assume a square input

14


matrix xm, n, where N1 D N2 D N, and let ˛1 D ˛2 D ˛, giving Xk, l D

N1 N1

xm, n˛hmki ˛hnli

mod F

13

mD0 nD0

The arrays that recur in this case are of the form 22u ð 22u and of the order of 1000 ð 1000 points. We treat the case u D 5, giving an image array of 1024 ð 1024. The value of u is arbitrary and the development is valid for any other value of u. With the dimensions defined, Equation (13) can be written as Xk, l D

1023 1023

xm, n ˛hmki ˛hnli

mod F

14

mD0 nD0

So far the value of ˛ has not been specified but must satisfy ˛1024 D 1 mod F4 and ˛t 6D 1 if 0 < t < 1024. We write Equation (14) in the form Xk, l D

1023

hmki

˛

mD0

1023

xm, n ˛hnli

mod F

15

nD0

k, l D 0, 1, . . . , 1023 Clearly the two-dimensional FNT is merely a one-dimensional FNT followed by another one-dimensional FNT. If it is assumed that the data are stored on a disk and randomly available in rows, then let the rows be first processed as follows. Let Gm, l D

1023

xm, n ˛hnli

mod F

16

nD0

m, l D 0, 1, . . . , 1023 Equation (16) can be written as Gm l D

1023

xm n ˛hnli

mod F

17

nD0

m, l D 0, 1, . . . , 1023 Equation (17) shows the one-dimensional nature of the transform to be computed.


Let

l D 32p C q n D 32r C s

15

p, q, r, s D 0, 1, 2, . . . , 31

Then Equation (17) becomes Gm p, q D

31 31

xm s, r˛h32psi ˛h32rqi ˛hqsi

mod F

18

sD0 rD0

because ˛h32pÐ32ri D ˛h1024pri D 1 mod F4 . Note that the submatrix xm s, r is made up of elements of xm n ordered columnwise. This is done for practical reasons and causes the result Gm p, q to be ordered in a rowwise fashion. Consider the following relations: ˛1024 D ˛32 32 D 1 mod F4

and 232 D 1 mod F4

Comparing the two relations, it is clear that if among all the ˛s that satisfy ˛1024 D 1 mod F4 , there is at least one that satisfies ˛32 D 2 mod F4 , then the 1024-length NTT will reduce to a 32-length FNT. In fact, there are 32 such ˛s. The first, ˛ D 5851, was found by a computer search. A list containing all these ˛s is given here. The list contains the values of ˛ that satisfy ˛32 D 2 mod F4 . These values are derived from ˛i D 5851 ð 2i mod F4 , i D 1, 2, . . . , 31. Values 5851 11702 23404 46808 28079 56158 46779 28021 56042 46547 27557 55114 44691 23845 47690 29843

of a 59686 53835 42133 18729 37458 9379 18758 37516 9495 18990 37980 10423 20846 41692 17847 35694

16


Equation (18), therefore, becomes Gm p, q D

31 31

xm s, r2ps 2rq ˛hqsi

mod F

19

sD0 rD0

which, apart from the twiddle factor ˛hqsi , is exactly a two-dimensional 32length FNT, provided it can be shown that the exponents ps and rq are computed modulo 32. 1. If ps < 32, then ps mod 32 D ps. 2. If ps D 32, then 2ps D 1 mod F4 D 2ps mod 32 . 3. If ps > 32, then ps D ps mod 32 C 32t t D integer and 2ps D 2ps mod 32 Ð 232t D 2ps mod 32 . Equation (19) is then written in the form Gmp,q D

31

2jpsj ˛hqsi

sD0

31

xm s, r2jrqj

mod F

20

rD0

p, q D 0, 1, . . . , 31 where j Ð j stands for mod 32. 1. The Twiddle Factor ˛qs The twiddle factor arises from the transformation of the one-dimensional process into a two-dimensional process and acts on the result of the first one-dimensional FNT only. It also creates two problems, one of storage and one of time. Referring to Equation (20), let Um s, q D

31

xm s, r2jrqj

mod F

21

rD0

s, q D 0, 1, . . . , 31 which represents a 32-length FNT. Next compute Ł Um s, q D Um s, q ˛hqsi

mod F

22

Equation (22) represents a pointwise multiplication of two 32 ð 32 matrices. Finally, 31 Ł Gm p, q D Um s, q2jpsj mod F 23 sD0

which represents another 32-length FNT.


17

Because the problems are associated with the twiddle factor ˛hqsi , consider how storage and time are reduced for that factor. There are 1024 sample values of ˛hqsi to be stored, but if it is noted that the matrix is symmetric, i.e. ˛hqsi D ˛hsqi , then savings in memory can be obtained at the price of a careful addressing scheme. The number of the sample values to be stored can be further reduced. The obvious reduction is made in the main diagonal, whenever ˛ii D ˛sq , with s 6D q. (For example, ˛2.2 D ˛1.4 . which is already stored) These cases and others bring the number of distinct values to be stored down to 339, which represents some 30% of the original space required. If each sample value (known prior to the computation of the two-dimensional FNT) has a 16-bit representation, only a 1 k ROM is necessary, with each ˛hqsi occupying two adjacent memory locations. This reduction in memory space is not matched by the same reduction in the number of multiplications by the twiddle factor; but if the time taken to compute a 32-length FNT is greater than or equal to the time needed to perform 32 multiplications, no time is wasted at all during the twiddling operation if parallel processing is used. After the first row of xm s, r has been transformed and while the second row is being processed, each element of the result of the previous operation is twiddled by ˛hqsi ; in other words, first compute Um 0, q D

31

x0, r2jrqj

mod F

24

rD0

q D 0, 1, . . . , 31. Then compute Um 1, q D

31

xm 1, r2jrqj

mod F

25

mod F

26

rD0

and

Ł Um 0, q D Um 0, q ˛hq.0i

until

31

xm 31, r2jrqj

mod F

27

UŁ m 30, q D Um 30, q ˛hq.30i

mod F

28

Um 31, q D

rD0

and

This means that the calculation of Equation (23) cannot be undertaken until UŁ m 31, q has been computed. But only UŁ m 31, 0 is needed in the first FNT Ł along the columns, and Um 31, 0 D Um 31, 0 ˛h31.0i mod F D Um 31, 0.

18


Thus, to compute Equation (23) for q D 0, the result of the last operation in Equation (22) is not required, and the two can be computed simultaneously. No twiddling occurs afterwards, and hence no time has been wasted. So far, it has been shown how the rows of a large input array can be transformed using a small-length FNT that needs no multiplications but only additions and word shifts (multiplication by powers of 2). The columns of the input array follow the same development, but because it was assumed that the data are available only in rows, the whole matrix must be transposed. If 32 rows of the input array can be accommodated in fast RAM, the whole twodimensional FNT of the 1024 ð 1024 image array can be computed using a 32-length FNT without transposing the array. Each row and each column of the 1024 ð 1024 matrix is broken up into a 32 ð 32 submatrix, where data are ordered columnwise. This is important, especially when processing the columns of the whole array. Consider the first column: when broken into a 32 ð 32 submatrix, the first row of this submatrix contains the elements of the initial column spaced 32 samples apart. Hence, if the input array is xm, n, m, n D 0, . . . , 1023, the first column would be broken up into the 32 ð 32 submatrix

AD

x0, 0 x1, 0 ...... x31, 0

x32, 0 x64, 0 . . . x992, 0 x33, 0 x65, 0 . . . x993, 0 ...... ......... x63, 0 x95, 0 . . . x1023, 0

29

The first row of A contains the elements of xm, 0 spaced 32 samples apart. Hence, if 32 rows of the input array are chosen such that each of the 1024 columns represents the first row of each of the 1024 submatrices, after processing each of the 32 rows read into fast memories, an extra 1024 32-length FNTs can be computed without extra input/output (I/O) operations. The twiddling operation following these 1024 FNTs consists of the multiplication of each 32-sample array by the first row of ˛hqsi . The next 32 rows of the input array also result in an extra 1024 FNTs, all of them being the second row of each 32 ð 32 submatrix. When all the 1024 rows of the input array have been read, processed and written back, not only will the NTT have been computed along the rows, but also a part of the NTT will have been done along the columns. Another pass on a different set of 32 rows will terminate the computation of the two-dimensional NTT. This development is shown shortly on a 4 ð 4 matrix. The number of I/O operations (each I/O operation consists of the transfer of 32 rows of data between disk and RAMs and back) is 32 for each dimension. An extra 31 I/O operations are needed at the end for the permutation of the rows of the result (it is assumed that the processing is done in place — i.e., the data are transferred back to the original position after computation).


19

Given enough fast memory, a large input image can be NTT transformed using a 32-length FNT without matrix transposing. Also, this fast memory allows a convolution to be computed without the need to use overlap sectioning methods, even when the filter dimensions are small. The two-dimensional convolution is computed using two forward NTTs and one inverse NTT. Thus, the two-dimensional NTT of the filter is computed and stored on a disk; the same disk will accommodate the two-dimensional NTT of the picture to be filtered. Both arrays are of dimension 1024 ð 1024. The fast memory is then used to accommodate 16 rows of each array, and the pointwise multiplication is computed. The result is then written back on to the disk while another pair of 16 rows is read into RAMs. The disk then contains the two-dimensional NTT of the convolution. The inverse two-dimensional NTT follows exactly the same development as the forward two-dimensional NTT. In fact, time-reversing every row (column) before computing the FNTs will enable the same algorithm and the same twiddle factors to be used. The advantages of this method are as follows: 1. The two-dimensional convolution using the overlap techniques is not efficient unless optimal sectioning is used, and referring to Hunt [1972], the optimal sections in this case must be of the order of 512 ð 512, which means that in this case eight times more memory would be needed: 2. Using this optimal sectioning, the advantage of using a small-length transform that has the attributes of the FNT is lost. 3. The method gives exact results (NTT property). a. Computation of the 2-D NTT. Suppose that the matrix

AD

a0 a4 a8 a12

a1 a5 a9 a13

a2 a6 a10 a14

a3 a7 a11 a15

30

is stored on a disk and is accessible by rows only. Suppose also that enough fast memory is available to accommodate two rows of A; then the two rows [a0 a1 a2 a3 ] and [a8 a9 a10 a11 ] are read into RAMs and each row is broken up into a 2 ð 2 submatrix, with data ordered columnwise: a0 a1

a2 a3

a8 a9

a10 a11

Each submatrix is NTT transformed to give b0 b2

b1 b3

b8 b10

b9 b11

20


Writing them back into a row format gives row 1

[b0 b1 b2 b3 ]

row 3

[b8 b9 b10 b11 ]

31

Doing the same for the other two rows of A yields row 2 [b4 b5 b6 b7 ]

row 4 [b12 b13 b14 b15 ]

32

Therefore, the NTT along the rows of A is given by B:

BD

b0 b4 b8 b12

b1 b5 b9 b13

b2 b6 b10 b14

b3 b7 b11 b15

and Bt D

b0 b1 b2 b3

b4 b5 b6 b7

b8 b9 b10 b11

b12 b13 b14 b15

33

To compute the NTT along the columns of B (rows of Bt ), the procedure would be row 1 of Bt row 3

[b0 b4 b8 b12 ] [b2 b6 b10 b14 ]

Breaking up into submatrices gives b0 b4

b8 b12

and

b2 b6

b10 b14

34

From Equation (34), it is clear that the rows to be transformed were already available in Equations 31 and 32 and hence would have been processed then, without the need to perform a matrix transposition. Referring to Eklundh’s fast algorithm for matrix transposing [Eklundh, 1972], with the same number of rows read into computer memory from a disk, the number of I/Os necessary to transpose a 4 ð 4 matrix is equal to 4. To compute a two-dimensional NTT, two matrix transpositions are needed, one to compute along the columns and the other to get back to the correct order, giving a total number of eight I/Os. The method described here uses only five I/Os: four to compute the two-dimensional NTT and one to permute the rows. b. Example of 2-D Autoconvolution Obtained from Simulations. Modulus D 28 C 1 D 257, ˛ D 35, array size D 4 ð 4 (padded with zeros to form 64 ð 64), magnitude D 2.


21

Input array: 2 2 2 2 0 0

xm, n D

2 2 2 2 0 0

2 2 2 2 0 0

0 0 0

2 2 2 2 0 0

0 0 0 0 0 0 .. .

0 0 0 0 0 0

0 0 0

... ... ... ... ... ...

0 0 0 0 64 0 0

35

... 0

64 In this case the two-dimensional FNT is given by

xk, l D

63 63

xm, n35hmki 35hnli

mod 257

36

mod 257

37

xn, m35hnli

mod 257

38

xm n35hnli

mod 257

39

mod 257

40

mod 257

41

mD0 nD0

D

63

35hmki

mD0

Gm, l D

63

63

xm, n35hnli

nD0

nD0

or Gm 1 D

63 nD0

Here h i stands for mod 64. If l D 8p C q and n D 8r C s then

Gm p, q D

7 7

xm s, r35h8psi 35h8rqi 35hqsi

sD0 rD0

but 358 mod 257 D 4; therefore Gm p, q D

7 7 sD0 rD0

xm s, r4jpsj 4jrqj 35hqsi

22


The following example shows the three steps involved in computing G0 p, q: 1.

7

U0 s, q D

x0 s, r4jrqj

mod 257

42

mod 257

43

mod 257

44

rD0

2. 3.

U0Ł s, q D U0 s, q ð 35hsqi 7

G0 p, q D

U0Ł s, q4jpsj

sD0

The first row of the 64 ð 64 matrix is x0 m D [2 2 2 2 0 0 . . . 0 0 0].

x0 s, r D

2 2 2 2 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

45

x0 m broken up into an 8 ð 8 sub-matrix.

U0 s, q D

2 2 2 2 0 0 0 0

2 2 2 2 0 0 0 0

2 2 2 2 0 0 0 0

2 2 2 2 0 0 0 0

2 2 2 2 0 0 0 0

2 2 2 2 0 0 0 0

2 2 2 2 0 0 0 0

2 2 2 2 0 0 0 0

46

FNT of x0 s, r computed along the rows.

˛hsqi D

1 1 1 1 1 1 1 1

1 35 197 213 2 70 137 154

1 197 2 137 251 236 135 85

1 213 137 36 135 28 29 169

1 2 251 135 165 237 214 170

1 70 236 28 237 119 99 219

Only 26 distinct values out of 64.

1 137 135 29 214 99 33 105

1 154 85 169 170 219 105 239

47


U0Ł s, q D

2 2 2 2 70 137 2 137 4 2 169 17 0 0 0 0 0 0 0 0 0 0 0 0

G0 p, q D

8 170 0 106 0 155 0 91

121 183 80 139 157 93 164 107

2 2 169 4 17 245 72 13 0 0 0 0 0 0 0 0

160 160 119 36 109 229 134 97

3 161 252 40 35 130 232 191

2 2 140 17 215 13 56 58 0 0 0 0 0 0 0 0

7 144 127 245 230 247 158 143

156 133 103 92 21 69 242 228

2 51 170 81 0 0 0 0 90 135 104 86 197 28 131 16

47 143 123 99 40 161 55 119

23

48

49

G0 p, q is the two-dimensional FNT of x0 s, r. next G0 1 D [8 121 160 3 7 156 90 47 170 183 160 . . . 228 16 119]

50

This method was used to compute the autoconvolution ym, n D xm, nŁ x m, n from Yk, l D jXk, lj2 :

ym, n D

4 8 12 16 12 8 4 0 0 .. .

8 16 24 32 24 16 8 0 0 .. .

12 24 36 48 36 24 12 0 0 .. .

16 32 48 64 48 32 16 0 0 .. .

12 24 36 48 36 24 12 0 0 .. .

8 16 24 32 24 16 8 0 0 .. .

4 8 12 16 12 8 4 0 0 .. .

0 0 0 0 0 0 0 0 0 .. .

0...0 0...0 0...0 0...0 0...0 0 . . . 0 64 0...0 0...0 0...0 .. .. . .

0

0

0

0

0

0

0

0 0...0

51

64 Images of approximate dimensions 1000 ð 1000 are common, so the 1024 ð 1024 two-dimensional NTT is not as peculiar as it might appear. Clearly the convolution can be computed efficiently using a single transform length, needing no matrix transposition and not using overlap sectioning techniques.

24


B. Number Domain Analysis in Two Dimensions with Possible Applications NTTs have been mainly applied to the calculation of convolutions (filtering) and correlations. Until recently no other applications were thought to exist, because of the residue number system, where the same quantities are congruent rather than equal. Hence the concept of magnitude, as in normal fields, does not hold. NTTs are calculated by retaining the residues only, which has led to difficulty in trying to relate the information content in the number domain generated by NTTs to the time representation of signals and images, such as that of the Fourier transform correspondence between frequency and time domain. The aim of this section is to ease that difficulty. First, the patterns that emerge in the 2-D NTTs of elementary images are shown. Also, the patterns of 2-D NTTs of periodic images are considered. Such patterns are regular in form, indicating that 2-D NTTs are, in fact, very well structured. This gives rise to the possibility of their application in pattern recognition. 1. Simple Images in the 2-D NTT Domain Although the transform domain for NTTs is not as simple to interpret as for some other transforms, the 2-D FNT domain for two-dimensional data does have special patterns of zero and nonzero values that could find applications. For objects such as squares, rectangles, and triangles, patterns of zeros appear in the 2-D FNT domain as a signature of the geometrical shapes [Marir, 1986; Boussakta, 1990]. In this section uses and limitations of these patterns are considered. Although the features of zeros can be formulated into equations, they are not unique. Thus, they do not characterize a unique shape. Additional pixels may be needed. However, where there is a need for classifying a closed set of objects, the zero pattern may offer a simple pattern classifier, which can be used for industrial detection problems, for example [Hollinghum, 1984; Wallace, 1988]. The zero patterns in the 2-D NTT domain are invariable to translation, and the 2-D NTT domain rotates with the rotation of the original picture. 2. Mathematical Development In the following development, xm, n is assumed to have a uniform intensity. The 2-D NTT transform defined in Equation (3) becomes Xk, l D xm, n

M1 N1 mD0 nD0

nl ˛mk 1 ˛2

mod F

52

25


3. Square Images For a square image of side L with i1 and i2 defining the position of the first pixel, Equation (52) becomes L10 1 L20 1

Xk, l D xm, n

nl ˛mk 1 ˛2

mod F

53

mDi1 nDi2

where L10 D L1 C i1 and L20 D L2 C i2 for a square of side L, L1 D L2 D L Xk, l D xm, n ˛1i1 k ˛2i2 l

L 1 1 L 2 1

nl ˛mk 1 ˛2

mod F

54

Xk, l D Ak, lBk, l

mod F

55

Ak, l D xm, n ˛1i1 k ˛2i2 k

mod F

56

mod F

57

mD0 nD0

where

Bk, l D

L 1 1 L 2 1

nl ˛mk 1 ˛2

mD0 nD0

Ak, l is in general different from zero and depends only on the intensity and the position of the first nonzero pixel in the image data. In real arithmetic Equation (57) can never be zero for a positive ˛1 and ˛2 . However, in finite fields it takes zero values for particular values of k and l [Boussakta, 1990]. This gives a regular pattern of zeros that can be used as a signature for the feature under consideration. We study simple objects; the results can be extended to more complicated shapes utilizing the linearity property of the 2-D NTTs for possible use in pattern recognition. a. Square Element of Side L D 2. For a square of side L D 2 and constant intensity (see Figure 3(a)), Equation (57) becomes Bk, l D

1 1

nl ˛mk 1 ˛2 mod F

58

mD0 nD0

To find the pattern of regular zeros, Equation (58) is set to equal zero. It is then solved modulo F as follows: Bk, l D 0

59

which is equivalent to 1 C ˛k1 1 C ˛l2 D 0

mod F

60

26


(a)

(b)

(c)

(d)

(e)

(f)

FIGURE 3. Square images of sides 2, 3, 4 and their 2-D FNTs. (a) Square image of side 2. (b) 2-D FNT transform of (a). (c) Square image of side 3. (d) 2-D FNT transform of (c). (e) Square image of side 4. (f) 2-D FNT transform of (e).


27

In finite fields Equation (60) has two solutions: k D M/2 and l D N/2

61

Thus, the NTT of this square contains one row and one column of zero values in the middle, which divide it into four parts, as shown in Figure 3(b). Note no particular modulus or transform length is specified, although in the following we use F3 with the transform dimensions M D N D 16 and ˛ D 2. Unless otherwise stated, both the image data and the 2-D FNT are displayed using 32 grey levels, ranging from black for zero intensity to white for the maximum intensity. b. Square of Side L D 3. Bk, l D

See Figure 3(c). Equation (57) becomes

2 2

ml ˛nk 1 ˛2

mod F

62

mD0 nD0

Bk, l D 0 mod F is equivalent to 1 21 1 C ˛k1 C ˛2k 1 1 C ˛2 C ˛2 D 0

mod F

63

Whatever the values of k and l, Equation (63) has no solution because neither 2l 1 (˛k1 C ˛2k 1 ) nor (˛2 C ˛2 ) can be equal to 1. Hence, there is no regular pattern of zeros (no whole column or row of zero values) in the NTT of a square image of size 3 ð 3. This is confirmed in Figure 3(d). However, if we look at the NTT as a whole, other elements can be selected as a signature of this square. For example, if the pattern of 2 is selected, it gives a regular pattern, forming a circle with one value at the center. c. Square of Side L D 4. Bk, l D

3 3

See Figure 3(e).

nl ˛mk 1 ˛2

mod F

64

mD0 nD0 3k 1 2l 3l Bk, l D 1 C ˛k1 C ˛2k 1 C ˛1 1 C ˛2 C ˛2 C ˛2 D 0

3k 1 C ˛k1 C ˛2k 1 C ˛1 D 0 for

3l 1 C ˛l2 C ˛2l 2 C ˛2 D 0 for

65

k D M2i C 1/2

and k D M2i C 1/4 and

mod F

66

l D N2j C 1/2

and l D N2j C 1/4

67

where i and j are integers such that k and l are less than M and N, respectively. For N D M D 16, the solutions are k, l D 4, 8, 12. Hence, the pattern of a

28


square image with side L D 4 divides the NTT image into 16 parts, as shown in Figure 3(f). d. Square of Side L D 5.

See Figure 4(a).

Bk, l D

4 4

ml ˛nk 1 ˛2 D 0

mod F

68

nD0 nD0

This equation has no solution; thus a square image of side L D 5 has no regular pattern of zeros, as shown in Figure 4(b). However, it has a special distribution of other intensities such as 2 that is different from all previous features. e. Square of Side L D 6. Bk, l D

5 5

See Figure 4(c). nl ˛mk 1 ˛2 D 0

mod F

69

mD0 nD0

D 1 C

˛3k 1 1

C

˛3l 2

2 2

nl ˛mk 1 ˛2 D 0

mod F

70

mD0 nD0

The double sum was shown to be different from zero. Thus, the regular zero values of Bk, l are the same as the zeros of: 3l 1 C ˛3k 1 1 C ˛2 D 0

mod F

71

which gives: k D M2i C 1/6 and l D N2j C 1/6

72

where i, j are integers such that k and l are also integers and less than M and N, respectively. For M D N D 16, there are two solutions, corresponding to k D 8 and l D 8

73

Thus, a square image of side L D 6 is characterized by one row and one column of zeros in the middle of the NTT image dividing it into four parts, which is the same as a square image of size 2 ð 2. This regular pattern of zeros is not unique, and other pixels are needed to make the difference. This does not become a problem because there is a clear difference in the images as a whole. Consider the dc values (the first pixels at k D 0 and l D 0). As shown in Figure 3(b) and Figure 4(d), these pixels are clearly different. Also,


(a)

(b)

(c)

(d)

(e)

29

(f)

FIGURE 4. Square images of sides 5, 6, 7 and their 2-D FNTs. (a) Square image of side 5. (b) 2-D FNT transform of part (a). (c) Square image of side 6. (d) 2-D FNT transform of part (c). (e) Square image of side 7. (f) 2-D FNT transform of part (e).

30


the distribution of other low-intensity pixels presented as black can result in a notable difference. Hence, a few extra pixels are sufficient to distinguish between these squares. f. Square of Side L D 7.

See Figure 4 (e).

Bk, l D

6 6

nl ˛mk 1 ˛2

mod F

74

mD0 nD0

Bk, l D 0 has no solution. Thus, as Figure 4(f) illustrates, there is no regular pattern of zeros in the NTT of this image. However, the pattern of 2 is well distributed, giving a diagonally parallel pattern of (2) that uniquely characterizes this shape, as shown in Figure 4(f). g. Square of Side L D 8. Bk, l D

See Figure 5(a). 7 7

nl ˛mk 1 ˛2 D 0

mod F

75

mD0 nD0

which is equivalent to 2k k 4l 2l l 1 C ˛4k 1 1 C ˛1 1 C ˛1 1 C ˛2 1 C ˛2 1 C ˛2 D 0

mod F

76

This equation has fourteen solutions, corresponding to k D M2i C 1/8

l D N2j C 1/8

k D M2i C 1/4

l D N2j C 1/4

k D M2i C 1/2

l D N2j C 1/2

78

If N ð M D 16 ð 16, then k D 2, 4, 6, 8, 10, 12, 14 l D 2, 4, 6, 8, 10, 12, 14 Therefore, seven rows and seven columns of a regular pattern of zeros are present in the NTT of a square image of size 8 ð 8. This is shown in Figure 5(b). The same development has been carried out for squares of other sizes. The results for these are shown in Figures 5–7. 4. Rectangular Images The same development for different rectangles shows that this pattern obeys the same rules as that of the squares. Taking M 6D N in the development


(a)

(b)

(c)

(d)

(e)

31

(f)

FIGURE 5. Square images of sides 8, 10, 11 and their 2-D FNTs. (a) Square image of side 8. (b) 2-D FNT transform of part (a). (c) Square image of side 10. (d) 2-D FNT transform of part (c). (e) Square image of side 11. (f) 2-D FNT transform of part (e).

32


presented previously will give the case for different rectangles. Thus, as for the square images, a rectangle with odd length and odd width has no zero pattern and a rectangle of length L and width W shares the same pattern of zeros along rows and along columns with the square of side L and the square of side W, respectively. In addition, a rectangle may have the same pattern of zeros as a square, which necessitates the comparison of other pixels to discern a difference. 5. Summary of the Results We may draw the following conclusions: 1. All squares having sides formed from odd numbers of pixels and all rectangles with odd lengths and odd widths have no pattern of zeros but a regular pattern of other intensities; as an example consider the value of 2. 2. Some patterns of zeros are common for several shapes, for example, the pattern of 2 ð 2, 6 ð 6, 10 ð 10, and 14 ð 14 or the pattern of 4 ð 4 and 12 ð 12. Therefore, other elements in the 2-D NTT domain should be selected to make the difference if needed. Few pixels are needed to distinguish between the patterns. For example, all cases previously stated as having similar 2-D NTTs have a different dc value, which may be used to identify each of them. Other intensities, such as the pattern of 2s, can also be used whenever the pattern of zeros is absent or there is a need for extra pixels to make the difference. 3. The pattern of zeros, as will be shown later, is invariable to translation and rotates with the rotation of the picture. In all the figures just given, the modulus is chosen to be F3 . The zero pattern is presented as a black background and the nonzero pattern is grey using 32 levels, which range from black to white. 6. Generalization and Possible Applications Because the NTTs are linear, generalization to images of higher complexity follows the same analysis. This is done for more complicated shapes, such as triangles and alphabetic characters [Boussakta, 1990; Marir, 1986]. In Figure 8(a) a triangle of equal sides is shown in the top left corner. Figure 8(b) shows only the pattern of zeros as white on a black background, which appears in the transform domain to form two triangles placed base to base. This shape is also diagonally symmetric. In Figure 8(c), the same triangle is displaced and reoriented. Figure 8(d) shows the pattern of zeros. It is clear that the same zero pattern exists (its orientation follows the input image), and the


(a)

(b)

(c)

(d)

(e)

33

(f)

FIGURE 6. Square images of sides 12, 13, 14 and their 2-D FNTs. (a) Square image of side 12. (b) 2-D FNT transform of part (a). (c) Square image of side 13. (d) 2-D FNT transformation of part (c). (e) Square image of side 14. (f) 2-D FNT transform of part (e).

34


(a)

(b)

(c)

(d)

FIGURE 7. Square images of sides 15 and 16 and their 2-D FNTs. (a) Square image of side 15. (b) 2-D FNT of part (a). (c) Square image of side 16. (d) 2-D FNT of part (c).

presence of the triangle can be detected by counting the few zero values. If the data are in diminished one representation, the detection of zero values in the 2-D FNT domain is easily done by testing only the most significant bit. This may be used, for example, in pattern recognition. 7. The Effect of Translation on the Zero Pattern If the squares or triangles presented so far are translated along the rows or the columns or in both directions, the coordinates will be changed from i1 and i2 to i1 C a and i2 C b, where a presents the translation along rows and b, along


(a)

(b)

(c)

(d)

(e)

(f)

35

FIGURE 8. Simple shapes and their 2-D FNTs. (a) A 3 ð 3 triangle. (b) Zero pattern in the 2-D FNT of part (a). (c) 3 ð 3 triangle displaced and reoriented. (d) Zero pattern in the 2-D FNT of part (c). (e) A square of side 4 translated into the middle. (f) Zero pattern of the 2-D FNT of part (e), which is the same as Figure 3(f).

36


columns. Then Equation (53) becomes Xk, l D xm, n

L 1 1

L 2 1

nl ˛mk 1 ˛2

mod F

79

mDi1 Ca nDi2 Cb 2 Cb Xk, l D xm, n ˛1ki1 Ca ˛li 2

L 1 1 L 2 1

nl ˛mk 1 ˛2

mD0 nD0

D A2 k, lBk, l where

80

lb ki1 li2 A2 k, l D xm, n ˛ka 1 ˛2 ˛1 ˛2 lb D Ak, l˛ka 1 ˛2

mod F

mod F 81

From Equation (81) we can see that translation is equivalent to multiplying each pixel by a corresponding factor: lb ˛ka 1 ˛2

mod F

82

However, the solution of the equation that gives rise to the pattern of zeros is unchanged. L 1 1 L 2 1 nl ˛mk mod F 83 Bk, l D 1 ˛2 mD0 nD0

which is the same as Equation (57). Hence the pattern of zeros that appears in the 2-D NTT domain remains unchanged. This is true for all kinds of images and all NTTs. Therefore, the method described here is insensitive to translation. This is an important feature for industrial inspection problems, for instance. Figure 8(e) shows a 4 ð 4 square translated to the middle, and Figure 8(f) shows its 2-D FNT transform, which has the same pattern of zeros as Figure 3(f). 8. Possible Applications The patterns of zeros show the behavior in the 2-D FNT domain with a view to the use of NTTs for purposes other than convolutions. The results may be important in pattern-recognition applications such as automatic isolated machine character recognition. In Figure 9(a), (c), and (e), letters are simulated; their 2-D NTTs are presented in Figure 9(b), (d), and (f), respectively, where only the zero values are displayed as white on different backgrounds. It is clear that the pattern of zeros is unique for all letters considered here. Counting all or some of the zero values can lead to the identification of each letter. Thus, the complexity of the process is reduced to the calculation of the 2-D FNT of each letter. It is not necessary to calculate and display the whole transform; only the zero values are needed.

37 FIGURE 9. Some characters and their 2-D FNT domain patterns. (a) Letter T. (b) Zero pattern in the 2-D FNT of the letter T. (c) Letter F. (d) Zero pattern in the 2-D FNT of the letter F. (e) Letter E. (f) Zero pattern in the 2-D FNT of the letter E.

38


9. Summary Although the number domain generated by the 2-D NTTs is not always simple to interpret, 2-D NTTs are meaningful. The study of some of the patterns of zeros that appear in the number domain for 2-D NTTs shows that, even though the patterns may be similar for some features, they might be used effectively in pattern-recognition problems where there is a need only for classifying a closed set of objects, such as those that occur in character recognition. If the process is complicated and the zero pattern might be not clear, other pixels in the NTT domain can be selected to show the difference between different patterns. This decision is problem dependent and can be made in the training session that precedes any pattern-recognition process [Hollingum, 1984; Wallace, 1988; Bongard, 1970]. IV. 2-D NTTS OF PERIODIC FUNCTIONS AND THEIR APPLICATIONS Some of the properties of the transform domain of the 2-D NTT and its possible applications are considered. Particular attention is given to the 2-D FNT and to data of 16 ð 16 pixel size. However, the results presented would apply to other picture sizes [Boussakta, 1990; Boussakta, Shakaff, Marir and Holt, 1988]. The transform domain for 2-D NTTs is not as simple to interpret as a frequency spectrum. However, for the periodic 2-D data patterns considered, the transform does have a well-structured form. A number of specific examples of 2-D FNT of various periodic patterns are considered, and both the data patterns and their FNTs are displayed. A general result is given for 2-D FNTs, showing the maximum number of nonzero values that appear in the transforms of given data patterns. These results are generalized for 2-D FNTs, and it is shown that similar results hold for other 2-D NTTs. For periodic 2-D data the transform contains many zero-valued entries: these are true zeros, because no rounding or truncation errors are introduced by the 2-D FNT calculation. It follows that the 2-D FNT computation of such data is simplified. The 2-D FNT of periodic data has a regular pattern, and any small imperfection in the data that interferes with its periodic nature can destroy the regularity of the 2-D FNT pattern. This is shown to be so for a number of examples where the effect of a small (one-pixel) change in otherwise periodic data produces a marked change in the resulting 2-D FNT patterns [Boussakta, 1990; Boussakta, Shakaff, Marir and Holt, 1988]. A. Analysis of the 2-D FNT of Periodic Structures Consider periodic patterns. The structure of 2-D FNT is formulated into simple equations, which lead to a general result. An image xm, n is periodic with period Tr ð Tc if the pixel xm, n is equal to the pixel xm C Tr , n C Tc , where Tr is the period along rows and Tc is the period along columns. An image of


39

period 8 ð 8 is shown in Figure 10(a). Although the following development assumes that ˛1 D ˛2 D 2, M D N D 16, and F D F3 D 257, the theory is applicable to any case where M and N are highly composite. 1. Case 1: Tr D Tc Let the period be Tr ð Tc D M/2 ð N/2 D 8 ð 8: xm, n D xm C 8, n C 8. The data are shown in Figure 10(a). The 2-D FNT of an image xm, n from Equation (3) is Xk, l D

15 15

xm, n2mk 2nl

mod F3

84

mD0 nD0

Interchanging the summations, performing them first with respect to m, Equation (84) can be written as Xk, l D

15

2nl

nD0

7

xm, n2mk C

mD0

15

xm, n2mk

mod F3

85

mD8

Because the image data are periodic along columns with Tr D 8, Equation (85) can be written as Xk, l D

15

2nl 1 C 28k

nD0

7

xm, n2mk

mod F3

86

mD0

Performing the same step along rows gives Xk, l D 1 C 28k 1 C 28l

7 7

xm, n2mk 2nl

mod F3

mD0 nD0

D Ak Ð Al

7 7

xm, n2mk 2nl

mod F3

87

Ak D 1 C 28k

mod F3

88

Al D 1 C 28l

mod F3

89

mD0 nD0

and

where Ak D 2 0 Al D 2 0

if k D 2i, i D 0, 1, 2, . . . , 7 mod F 3 otherwise if l D 2j, j D 0, 1, 2, . . . , 7 mod F 3 otherwise

90 91

40

S. BOUSSAKTA AND A. G. J. HOLT 1

2

3

4

5

6

7

8

1

2

3

4

5

6

7

8

2

3

4

5

6

7

8

9

2

3

4

5

6

7

8

9

3

4

5

6

7

8

9

1

3

4

5

6

7

8

9

1

4

5

6

7

8

9

3

2

4

5

6

7

8

9

3

2

5

6

7

8

9

2

4

6

5

6

7

8

9

2

4

6

6

7

8

9

1

2

3

4

6

7

8

9

1

2

3

4

7

8

9

3

6

9

2

1

7

8

9

3

6

9

2

1

8

9

1

2

4

6

8

1

8

9

1

2

4

6

8

1

1

2

3

4

5

6

7

8

1

2

3

4

5

6

7

8

2

3

4

5

6

7

8

9

2

3

4

5

6

7

8

9

3

4

5

6

7

8

9

1

3

4

5

6

7

8

9

1

4

5

6

7

8

9

3

2

4

5

6

7

8

9

3

2

5

6

7

8

9

2

4

6

5

6

7

8

9

2

4

6

6

7

8

9

1

2

3

4

6

7

8

9

1

2

3

4

7

8

9

3

6

9

2

1

7

8

9

3

6

9

2

1

8

9

1

2

4

6

8

1

8

9

1

2

4

6

8

1

(a)

67

0

61

0

40

0

221 0

0

0

245 0

177 0

84

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

144 0

110 0

117 0

205 0

105 0

181 0

87

0

250 0

0

0

0

0

0

0

0

0

0

0

44

0

192 0

243 0

144 0

159 0

115 0

29

0

239 0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

160 0

171 0

202 0

37

0

21

0

29

0

253 0

53

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

16

0

194 0

189 0

119 0

56

0

93

0

60

0

173 0

0

0

0

0

0

0

0

0

0

0

0

0

0]

26

0

164 0

150 0

134 0

95

0

247 0

54

0

4

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

173 0

205 0

36

0

30

0

26

0

183 0

222 0

17

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

8

0

102 0

188 0

36

0

181 0

38

0

10

0

38

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

(b)

FIGURE 10. Periodic image with Tr ð Tc D 8 ð 8 and its 2-D FNT. (a) Periodic image with Tr ð Tc D 8 ð 8. (b) Two-dimensional 2-D FNT of part (a).

41


Hence,

Xk, l D

4

7 7

if k D 2i, l D 2j,

xm, n2mk 2nl

mD0 nD0

i, j D 0, 1, 2, . . . , 7 otherwise

0

mod F3

92 Thus, period Tr ð Tc D 8 ð 8 leads to a maximum of 64 nonzero elements for an image of 16 ð 16. Figure 10(b) illustrates this. For other periods and cases when the images have different periodicities along rows and columns, the results are tabulated in Tables 3 and 4. 2. Generalization for 2-D FNT Let xm, n be any periodic image with period Tr ð Tc and Ft be any modulus. Then the 2-D FNT of the data is given by Xk, l D

M1 N1

xm, n2mk 2nl

mod Ft

mD0 nD0

D

M1 mD0

2mk

N1

xm, n2nl

mod Ft

93

nD0

TABLE 3 CASE 1 Tr D Tc . Maximum Number of Tr ð Tc

Equations of Nonzero Pixels 16

3 3

xm, n2mk 2nl

mod F3

Nonzero Pixels for k D 4i;

mD0 nD0

4ð4

i D 0, 1, 2, 3;

Xk, l D

l D 4j;

16

j D 0, 1, 2, 3 0

otherwise 64

2ð2

1 1

xm, n2mk 2nl

mod F3

for k D 8i;

mD0 nD0

Xk, l D

l D 8j; i, j D 0, 1 0

otherwise

4

42

S. BOUSSAKTA AND A. G. J. HOLT TABLE 4 CASE 2 Tr 6D Tc . 7 1

16

xm, n2mk 2nl

mod F3

mD0 nD0

2ð8

If k D 8i;

Xk, l D

i D 0, 1;

l D 2j;

16

j D 0, 1, . . . , 7 0

otherwise 8

7 3

xm, n2mk 2nl

mD0 nD0

4ð8

mod F3 if k D 4i;

Xk, l D

l D 2j

32

i D 0, 1, . . . , 3; j D 0, 1, . . . , 7 0

otherwise

32

3 1

xm, n2mk 2nl

mD0 nD0

2ð4

mod F3 if k D 8i;

Xk, l D

l D 4j;

8

i D 0, 1; j D 0, 1, . . . , 3 0

otherwise

Summing along rows gives Xk, l D

M1

Tc1

mk

2

mD0

xm, n2 C nl

N1

xm, n2nl

nDTc

nD0

CÐÐÐ C

2T c 1

xm, n2nl

mod Ft

94

nDNTc

because xm C Tr , n C Tc D xm, n. Equation (94) can be written as Xk, l D

M1

2mk 1 C 2Tc l C 22Tc l

mD0

CÐÐÐ C2

NTc l

T c 1 nD0

xm, n2nl

mod Ft

95


43

Summing along columns with respect to the period Tr , Equation (95) can be written as Xk, l D 1 C 2Tr k C 22Tr k C Ð Ð Ð C 2MTr k ð 1 C 2Tc l C 22Tc l C Ð Ð Ð C 2NTc l

T c 1 r 1 T

xm, n2mk 2nl

mod Ft

96

mD0 nD0

Factoring Equation (96) gives Xk, l D Ak Ð Al

T c 1 r 1 T

xm, n2mk 2nl

mod Ft

97

mD0 nD0

where log2 M/Tr

Ak D

1 C 2M/Ek ;

E D 2q

mod Ft

98

1 C 2N/Gl ;

G D 2p

mod Ft

99

qD1 log2 N/Tc

Al D pD1

From Equation (98) and Equation (99): Ak D Al D

M/Tr

0 N/Tc 0

if k D Mi/Tr ,

i D 0, 1, 2, . . . , Tr 1

otherwise if l D Nj/Tc ,

j D 0, 1, 2, . . . , Tc 1

otherwise

mod Ft 100 mod Ft 101

Therefore, M/Tr N/Tc

T c 1 r 1 T

xm, n2mk 2nl

mod Ft

mD0 nD0

Xk, l D

if k D Mi/Tr l D Nj/Tc i D 0, 1, 2, . . . , Tr 1 j D 0, 1, 2, . . . , Tc 1 0 otherwise

102

44


Theorem If an image xm, n is periodic with period Tr ð Tc , the number of nonzero pixels in its 2-D FNT is less than or equal to the product Tr ð Tc . These nonzero pixels are determined by the following equation: Xk, l D M/Tr N/Tc

T c 1 r 1 T

nl xm, n ˛mk 1 ˛2

mod Ft

mD0 nD0

with k D Mi/Tr ; l D Nj/Tc ; i D 0, 1, 2, . . . , Tr 1; j D 0, 1, 2, . . . , Tc 1. 103 The result in Equation (103) is derived for Fermat number transforms. It can be extended to other number theoretic transforms. 3. The Effect of Data Errors on 2-D FNT We have considered cases where the data are periodic. Now consider the effect on the 2-D FNT of small errors in the data that will produce departures from periodicity in one period of the data while all the other data remain periodic. Figure 11(a) shows an example where the data are all periodic with Tr D Tc D 2, except for a single pixel in row 2 column 5, where the expected data value 7 is replaced by the value 2, which may be regarded as an error. Clearly this single error disturbs the periodicity of the data pattern. If the actual data including the error were to be subtracted from the expected periodic data patterns, the error would be detected by the only nonzero value in the difference. However, this one difference pixel would be the only indication of the occurrence of an error. The effect of the single error upon the 2-D FNT transform domain is very much more obvious than it is in the nontransformed data. This is shown in Figure 11(b), which, in contrast to the transform of periodic data given in Figure 10(b), has no zero values at all; although a pattern can be detected in the transform elements, Figure 11(b) displays none of the regular structure with zeros that is apparent in the transforms of periodic data. The regular structure of the FNT has been completely altered by the single error that has been introduced into the data. It follows that, although the existence of small errors in a periodic pattern may be detected by eye after a subtraction between the actual and the expected or standard data patterns, it becomes much more obvious when the 2-D FNT of the data is observed. 4. Summary of the Results From the preceding development, the following results are noted: 1. The 2-D FNTs of periodic images are well structured. 2. The position and the values of the nonzero pixels can be precisely determined.

45


3

7

3

7

3

7

3

7

3

7

3

7

3

7

3

7

4

9

4

9

4

9

4

9

4

9

4

9

4

9

4

9

3

7

3

7

3

2

3

7

3

7

3

7

3

7

3

7

4

9

4

9

4

9

4

9

4

9

4

9

4

9

4

9

3

7

3

7

3

7

3

7

3

7

3

7

3

7

3

7

4

9

4

9

4

9

4

9

4

9

4

9

4

9

4

9

3

7

3

7

3

7

3

7

3

7

3

7

3

7

3

7

4

9

4

9

4

9

4

9

4

9

4

9

4

9

4

9

3

7

3

7

3

7

3

7

3

7

3

7

3

7

3

7

4

9

4

9

4

9

4

9

4

9

4

9

4

9

4

9

3

7

3

7

3

7

3

7

3

7

3

7

3

7

3

7

4

9

4

9

4

9

4

9

4

9

4

9

4

9

4

9

3

7

3

7

3

7

3

7

3

7

3

7

3

7

3

7

4

9

4

9

4

9

4

9

4

9

4

9

4

9

4

9

3

7

3

7

3

7

3

7

3

7

3

7

3

7

3

7

4

9

4

9

4

9

4

9

4

9

4

9

4

9

4

9

(a)

182 97

20

126 177 10

63

237 131 80

247 194 40

252 97

177 10- 63

217 5

194 40

252 97

217 200 160 237 131 80 126 177 10

63

160 237 131 80

247 194 40

252 97

217 5

20

126 177 10

63

5

160 237 131 80

247 194 40

252 97

20

126 177 10

63

217 5

80

247 194 40

252 97

20

60

97

126 177 10

63

217 69

63

217 5

20

247 194 40

20

160 237 131 80

237 131 80

247 194 40

177 10

63

217 5

194 40

252 97

217 5 20

160 126

160 237 131 80

247

20

126 177 10

63

217

160 237 131 80

247 194 40

252 97

126 177 10

217 5

160 237 131 80

247 194 40

252 97

63

160 237 131 247 194 40

252 97

20

126 177 10

20

126 177 10

63

217 5

252 97

160 237 131 80

247 194 40

20

126 177 10

63

217 5

5

160 237 131 80

247 194 40

252 97

20

126 177 10

63

217 5

80

247 194 40

252 97

63

217 5

160

20

126

160 237 131 80

247

20

126 177 10

63

217

252 97

160 237 131 80

247 194 40

20

126 177 10

63

217 5

160 237 131 80

247 194 40

252 97

20

160 237 131 126 177 10

(b)

FIGURE 11. (a) Periodic image where one pixel is in error. (b) Two-dimensional FNT, showing that the regular pattern is destroyed.

46


3. The amount of computation required to determine the 2-D FNT of periodic images can be drastically reduced by using the preceding results. For example, in the extremely redundant case of calculating an M ð N image with period Tr ð Tc , only Tr ð Tc pixels need to be computed instead of M ð N. In extreme cases, such as a periodic image of 128 ð 128 D 16, 384 pixels with period Tr ð Tc D 2 ð 2, the computation of 16,384 pixels is reduced to only 4 pixels. A slight imperfection in the period of the image data will destroy the highly regular pattern that arises due to the periodicity, and hence the presence of defects can be easily detected from the dramatic change in the 2-D FNT produced by even a one pixel error. B. Applications The 2-D FNTs can be used to detect defects, especially small defects, in periodic structures. This result would appear to be applicable to several detection problems. A possible application could be in the process for integrated or film circuits, where photomasks are used for fabrication steps. Typically, these photomasks possess a repeated or periodic pattern. Before being used the masks must be carefully inspected for any possible defects, which, if not detected, would result in the manufacture of faulty circuits. In this case, it may be possible to apply the 2-D FNT method to readily show the existence of a mask defect. The inspection process begins by taking a digitized image (of dimension M ð N) of the photomask followed by the computation of its 2-D FNT. The defect could be detected by the visual examination of the transforms of the ideal and faulty masks. This effect is illustrated in Figure 12(a) and (b), where the data simulating a very simple photomask as blocks of nine 1s and having a periodic structure are shown with their 2-D FNT. The transform clearly shows the regular pattern with small number of nonzero elements that is to be expected. Figure 13(a) shows the same simulated photomask data as in Figure 12(a), with the addition of a single error introduced at the boundary between two blocks of 1s. Figure 13(b) shows the 2-D FNT of the photomask with the error, and it is clear that it is drastically different from the 2-D FNT of the error-free mask in Figure 12(b). It follows that a single error in a photomask could readily be detected by eye when comparison is made between the 2-D FNTs of a standard mask and the one under inspection. Use can also be made of the fact that it will be necessary to compare only a few elements in the 2-D FNTs in order to detect the existence of an error. In Figure 12(b) the 2-D FNT of an error-free mask has many rows and columns of zero-valued elements.


47

(a)

(b)

FIGURE 12. (a) Perfect photomask of 64 ð 64. (b) 2-D FNT of the perfect photomask. The regular white pixels indicate nonzero data.

In order to detect the presence of the error in Figure 13(a), it is necessary to examine (automatically) only the element where row 7 and column 7 meet and note that its value is nonzero. The same method can, of course, be applied using any other zero-valued row and column in the 2-D FNT of the periodic, error-free photomask. Because some zero values do occur in the 2-D FNT of a mask containing an error (see Figure 13(b)), it would be advisable to check the values or the sum of a small number of elements on the same row and column where a zero element is expected, as at 7, 7 in the earlier example. The number of nonzero elements in the 2-D FNT is small, especially for the data with Tr ð Tc − M ð N, which is typically the case with a photomask.

48


(a)

(b)

FIGURE 13. (a) Defective photomask with just one pixel error. (b) 2-D FNT of part (a) showing a completely different FNT pattern of the defective mask.

Therefore, the detection process may require comparison of only a few 2-D FNT elements. Any differences between the 2-D FNT values being compared automatically imply a defective mask. If it is required, the precise location of the defect may be determined either by a subtraction operation between the data for the mask under test and a standard mask or by taking the inverse 2-D FNT of the difference between the 2-D FNT values for the standard and the faulty mask. The same result could, of course, be obtained by subtracting the data in Figure 12(a) from that in Figure 13(a). Note that in its present form, this defect-detection technique is a highly sensitive go/no-go test. If required, it


49

should be possible to reduce the sensitivity by, for example, grouping the data pixels before calculating the transform or by considering the value of the transform element at k D 0, l D 0, where this value can be constrained to be less than the modulus (for example, by thresholding the input image) and then checking that this value lies within some predefined limit. Referring back to Figures 12(b) and 13(b), comparing the two values at location k D 0, l D 0 indicates that the difference between the two images is 1, which is equal to the total error. The example quoted requires considerable further investigation and tests before it can be regarded as practical. It is not the aim of this chapter to investigate this application; however, it does indicate possible applications of number theoretic transforms that do not appear to have been explored before.

C. Implementation of the Inspection Method It is helpful to select a transform with a suitable length that can be implemented sufficiently fast and with small complexity. The 2-D FNT meets these requirements. The maximum transform size for p a multiplierless 2-D FNT, as shown in Table 1, is 256 ð 256, with ˛ D 2 and modulus F D F6 . This modulus needs 65 bits to be represented, which is much larger than required in the applications of pattern recognition or in industrial inspection, where the image is usually thresholded to an appropriate level (8 bits or less in many cases). For images up to 256 ð 256 the best modulus seems to be F3 or F4 . 1. Implementation with F3 The maximum multiplierless p transform length with the modulus F3 D 257 is 32, corresponding to ˛ D 2, and can be increased by using roots of unity of higher order. In fact, the transform length can be extended to 64, 128, and 256 if ˛ is chosen to be the fourth, the eighth, or the sixteenth root, which correspond to 35, 42, and 88, respectively. If the image is large, it would be segmented into small subimages and then each subimage inspected individually. The maximum transform length that can be attained is 256 for the modulus F3 D 257. This seems to be ideally suited for images up to 256 ð 256. However, this value of ˛ requires normal integer multiplications along the last three stages of the algorithm used to calculate the 2-D FNT, involving three integer multiplications per pixel. This can be done quickly by using a look-up table. In fact, in the finite field defined by the modulus F3 D 257, the number of elements is just 256. These elements must be multiplied by all powers of ˛, ranging from 0 to 127. Hence, with a look-up table, all

50


multiplications can be stored in a memory of size 32k bytes, which is fairly small. Thus, all multiplications are eliminated and replaced by a look-up table. 2. Implementation Using 1-D FNT The inspection process begins by taking a digitized image of dimension N ð M of the photomask or periodic image, followed by the computation of its 2D FNT. Because the inspection of the photomask is based on the pattern of zeros and not on the complete image, it is unnecessary to compute the whole transform. A partial computation of the 2-D FNT can lead to the same result. In Figure 14(a) and (b), the 1-D FNT along rows and along columns is computed and shows that the image is periodic along all rows and along all columns (using the result found in one dimension, the periodicity of each column can be easily detected). Figure 15(a) shows a photomask containing four errors. In Figure 15(b) the errors are indicated by rows of nonzero pixels showing the existence of these errors. The errors may lie anywhere in those rows. Thus, the location of the errors is narrowed and can be detected either by scanning those rows or by taking the 2-D FNT along columns, which leads to columns of nonzero pixels (Figure 15(c)), indicating the absence of periodicity along those columns. The errors are located at the intersection of the previous rows and these columns (the nonzero values are presented as white and the zero values as black background) [Boussakta, 1990]. 3. Number Theoretic Transform with Independent Length and Modulus Previously, the main concern has been 2-D FNTs that possess fast algorithms and can be computed with simple arithmetic operations. All operations are performed in a finite field with the arithmetic carried out modulo a Fermat prime, which requires a b C 1-bit representation. The pattern of zeros is true for all 2-D NTTs, where N is highly composite. For the detection of errors the convolution property, which limits transform lengths, is no longer needed. This allows other 2-D NTTs to be considered as candidate transforms for error detection. The one-dimensional number theoretic Walsh transform was introduced in Thomas, Larsen, and Keller [1983] and extended to two dimensions in Boussakta [1990], but, because of its lack of the cyclic convolution property, which is a major feature for a transform to be used in digital signal processing, this transform finds few applications. In this case the most important things are the transform itself and the transformed data rather than the convolution property. This type of transform has the advantage that it has no rigid relationship between the transform length and the modulus.


51

(a)

(b)

FIGURE 14. (a) 2-D FNT along the columns of a perfect photomask; the discontinuous white pattern indicates perfect periodicity along this dimension. (b) 2-D FNT along the rows of a perfect photomask; the discontinuous white pattern indicates perfect periodicity along the rows.

The two-dimensional number theoretic Walsh transform of length N ð N of an image data xm, n is given by Wk, l D 1/N

N1 N1

xm, nwk, mwl, n

mD0 nD0

k D 0, 1, 2, . . . , N 1 l D 0, 1, 2, . . . , N 1 where wk, m is the well-known Walsh function.

mod F

104

52


(b)

(a)

(c)

FIGURE 15. (a) A faulty photomask with four pixels in error. (b) The 2-D FNT along the rows of part (a), with four white bars locating the faults within the rows. (c) The 2-D FNT along the columns of part (a), with three white bars locating the faults within the columns.

The inverse transform is similar to the forward and is given by

xm, n D 1/N

N1 N1

Wk, lwk, mwl, n

kD0 lD0

m D 0, 1, 2, . . . , N 1 n D 0, 1, 2, . . . , N 1

mod F

105


53

The constant factor 1/N2 is split between the forward and the inverse transform to make them symmetric. This transform involves only additions and subtractions. The forward and the inverse Walsh transforms are taken modulo F; hence for the inverse to exist, F should be relatively prime to N (N is a power of two); the best choice for F is a modulus with efficient arithmetic such as Fermat and Mersenne primes. Equation (104) can be calculated by an efficient FFT-type algorithm involving N2 log2 N2 operations (operation means either one addition or one subtraction). The Fast Walsh transform can be computed by the same program used to compute the 2-D FNT. The only difference is that all powers of ˛ are set equal to one for the Walsh transform. The two transforms have a close relationship and each can be calculated from the other through a block diagonal transformation matrix [Boussakta and Holt, 1989]. 4. Summary The analysis of the 2-D FNTs of periodic structures leads to a rule that simplifies calculations and determines the position and values of nonzero elements in the transform domain, showing that 2-D NTTs are well structured. Some new applications of 2-D FNTs are suggested that indicate the number domain can be utilized for applications other than convolution. Integrated circuit designs and microprocessor implementations for 2-D FNTs should make this transform more readily available for use in systems. The transform-length constraint for the 2-D FNT is not a great problem; in fact, several solutions are proposed, allowing the processing of sufficiently large images while retaining the advantages of the FNT, such as fast computation and simple arithmetic. Other transforms, such as the number theoretic Walsh, may also be used. V. ANOTHER FAMILY OF NUMBER THEORETIC TRANSFORMS USING THE MERSENNE NUMBERS (NMNTS) So far only number theoretic transforms with Fourierlike structure have been considered; in this section another family of transforms is introduced. These transforms are defined modulo Mersenne numbers, but they have long transform lengths that are powers of 2 and FFT-type algorithms [Boussakta and Holt, 1992; Boussakta and Holt, 1994a; Boussakta and Holt, 1994b]. 1. Transform Definition The transform described here has the cyclic convolution property (CCP) and is defined modulo the Mersenne numbers. It is suitable for fast algorithms, with a convenient word length and arithmetic.

54


a. The Forward Transform. The 1-D forward transform of sequence xn of length N is defined as [Boussakta and Holt, 1995a] N1 Xk D xnˇnk 106 nD0

Mp

k D 0, 1, 2, . . . , N 1 where h iMp denotes mod MP , and Mp is Mersenne prime equal to 2p 1. ˇn D ˇ1 n C ˇ2 n

107

ˇ1 n D hRe˛1 C j˛2 n iMp and ˇ2 n D hIm˛1 C j˛2 n iMp Also

˛1 D šh2 iMp ; q

˛2 D šh3 iMp ; q

qD2

p2

108 109

In Equation (109) ˛1 and ˛2 are of order N D 2pC1 . For transform length N/d, where d is an integer power of 2, ˇ1 and ˇ2 are given by ˇ1 n D hRe˛1 C j˛2 d n iMp and ˇ2 n D hIm˛1 C j˛2 d n iMp

110

Re(Ð) and Im(Ð) denote real and imaginary parts of the enclosed term, respectively. The transform defined in Equation (106) uses a real kernel leading to a real transform. Also, this transform has the same form of inverse as its forward transform. b. The Inverse Transform. The form of the inverse transform is the same as that of the forward transform, except for a factor (1/N): N1 xn D 1/N Xkˇnk 111 kD0

Mp

n D 0, 1, 2, . . . , N 1 Because Mp is an odd number and N is a power of 2, the inverse of N always exists. Also the term 1/N could be split between the forward and the inverse transforms to make them both exactly the same form. 2. Properties of the ˇ Function Consider the relation between ˇÐ and the kernels of the Fourier and Hartley transforms in order to clarify the properties of ˇÐ D ˇ1 Ð C ˇ2 Ð.


55

As an example take the values of ˇ1 nk and ˇ2 nk calculated for N, Mp , ˛1 , ˛2 D 8, 31, 4, 4.

ˇ1 nk D

1 1 1 1 1 4 0 4 1 0 1 0 1 4 0 4 1 1 1 1 1 4 0 4 1 0 1 0 1 4 0 4

1 1 1 1 1 4 0 4 1 0 1 0 1 4 0 4 1 1 1 1 1 4 0 4 1 0 1 0 1 4 0 4

ˇ2 nk D

0 0 0 0 0 0 0 0 0 4 1 4 0 4 1 4 0 1 0 1 0 1 0 1 0 4 1 4 0 4 1 4 0 0 0 0 0 0 0 0 0 4 1 4 0 4 1 4 0 1 0 1 0 1 0 1 0 4 1 4 0 4 1 4

112

From these matrices it can be seen that ˇ1 Ð and ˇ2 Ð have the same sign and one and zero values as the cosine and sine in the DFT or DHT. Relations similar to those existing between the trigonometric functions casÐ, cosine, and sine, exist for ˇÐ, ˇ1 Ð, and ˇ2 Ð Table 5 gives the transform parameters for some different moduli and maximum transform lengths Nmax D 2p . From ˛1 and ˛2 of Table 5, the values of ˛1 and ˛2 for any transform length power of 2 to Nmax can be calculated. 3. Calculation of 1-D Convolutions Let xn and hn be two sequences of length N1 and N2 , respectively. Then their linear convolution is of length an N1 C N2 1 point sequence. The new transform of this length can be used to calculate the convolution. TABLE 5 TRANSFORM PARAMETERS FOR THE NMNT. Modulus (Mp ) 1 D 31 27 1 D 127 213 1 D 8,191 217 1 D 131,071 219 1 D 524,287

25

Nmax

˛1 (for Nmax )

˛2 (for Nmax )

32 128 8,192 131,072 524,288

5 5 5 5 5

10 22 5,381 94,889 46,561

56


yn D xnŁ hn D transformfXk Hev k C XN k Hod kg D transformfXk0Hkg

113

where is point-by-point multiplication, 0 is defined by Equation (113), and Hev k and Hod k stand for even and odd parts of Hk, respectively, and are given by Hev k D hfHk C HN kg/2iMp and

Hod k D hfHk HN kg/2iMp

114

The division by 2 in Equation (114) is a division modulo Mp 1/2 D 2p1 . Usually the filter impulse response hn is known a priori, and hence Equation (114) can be precalculated and stored. Also xn and hn must be scaled so that the convolution result jynj does not exceed Mp /2. 4. Fast Algorithms for This Transform The length of the new transform is a power of 2. Fast algorithms such as the radix-2, radix-4, and split radix [Rabiner and Gold, 1975; Duhamel and Hollman, 1982] could be applied for its evaluation. a. Radix-2 Algorithm Derivation. As an example, a radix-2 algorithm is derived. Dividing the input sequence into its odd and even parts, Equation (106) can be decomposed as Xk D hX2n k C X2 kiMp N/21 x2nˇ2nk X2n k D nD0

X2 k D

N/21

115 116 Mp

x2n C 1ˇ2n C 1k

nD0

117 Mp

ˇ2n C 1k D hˇ1 kˇ2nk C ˇ2 kˇ2nkiMp

118

X2n k can be identified as an N/2-point transform and Equation (117) can be written as N/21 N/21 X2 k D ˇ1 k x2n C 1ˇ2nkCˇ2 k x2n C 1ˇ2nk nD0

nD0

Mp

119

57


Combining Equations (115) and (119) leads to the recursive formula for this transform: Xk D hX2n k C X2nC1 kˇ1 k C X2nC1 N kˇ2 kiMp

120

k D 0, 1, 2, . . . , N 1 where X2nC1 k D

N/21

x2n C 1ˇ2nk

nD0

121 Mp

Combining the four points, Xk, XN/2 k, Xk C N/2, XN k gives an in-place radix-2 algorithm: Xk D hX2n k C [X2nC1 kˇ1 k C X2nC1 Nkˇ2 k]iMp 122 Xk C N/2 D hX2n k [X2nC1 kˇ1 k C X2nC1 N kˇ2 k]iMp123 XN/2 k D hX2n N/2 k C [X2nC1 N/2 C kˇ2 k X2nC1 N kˇ1 k]iMp

124

XN k D hX2n N/2 k [X2nC1 N/2 C kˇ2 k X2nC1 N kˇ1 k]iMp

125

b. Arithmetic Complexity. A single butterfly is shown in Figure 16. It calculates four points and requires four integer multiplications and six additions. Therefore, the calculation of the whole transform using a single butterfly involves N log2 N multiplications (including the multiplications by š1) and 3 2 N log2 N additions. Using different butterflies, the number of multiplications can be reduced quite significantly [Boussakta and Holt, 1995a]. In Table 6 the shifts are due to the fact that the stages of order three and higher include twiddle factor ˇ1 N/8 and ˇ2 N/8, which are both equal to 2p1/2 . TABLE 6 GIVES THE NUMBER OF OPERATIONS FOR DIFFERENT NUMBER OF BUTTERFLIES. Number of Butterflies Used 1 2 3 5

Number of Multiplications N log2 N N log2 N 2N C 2 N log2 N 3N C 4 N log2 N 4N C 8

Number of Shifts

Number of Additions

1 2N 2

3 2 N log2 N 3 2 N log2 N N C 1 3 3 2 N log2 N 2 N C 2 3 3 N log N 2 2 2N C 2

58

S. BOUSSAKTA AND A. G. J. HOLT X(k)

X(k)

X(N/2−k)

X(N/2−k) β1(κ)

X(N/2+k)

X(N/2+k)

−

β2(κ) β2(κ) β1(κ)

X(N−k)

−

−

X(N−k)

FIGURE 16. Four-point butterfly.

B. The 2-D New Mersenne Number Transform (2-D NMNT) The new transform can be readily extended to two-dimensional and multidimensional cases. 1. Definition of the 2-D Transform The transform pair is defined as follows. The 2-D forward transform of a 2-D input image xn, m of dimensions N ð N is defined as [Boussakta and Holt, 1995a]: N1 N1 xn, mˇnk, ml 126 Xk, l D nD0 mD0

Mp

k D 0, 1, 2, . . . , N 1,

l D 0, 1, 2, . . . , N 1

ˇn, m D ˇ1 n, m C ˇ2 n, m ˇ1 n, m D Re˛1 Cj˛2 nCm

Mp

127

and ˇ2 n, m D Im˛1 Cj˛2 nCm

128

Mp

˛1 and ˛2 are as defined for the 1 D case for transform dimensions N/d ð N/d; ˇ1 and ˇ2 are given by ˇ1 n, m D Re˛1 Cj˛2 d nCm

and ˇ2 n, m D Im˛1 Cj˛2 d nCm Mp 129 Re(Ð) and Im(Ð) denote the real and imaginary parts of the enclosed term, respectively. The two-dimensional inverse transform has the same form as that of the forward transform except of a factor (1/N2 ). N1 N1 xn, m D 1/N2 Xk, lˇnk, ml 130 Mp

kD0 lD0

n D 0, 1, 2, . . . , N 1,

Mp

m D 0, 1, 2, . . . , N 1

Because Mp is an odd number, the inverse of N2 always exists.


59

TABLE 7 THE RELATIONSHIP BETWEEN THE TRANSFORM SIZE AND THE MODULUS FOR THE 2-D NMNT. Modulus (Mp )

Nmax ð Nmax

˛1

˛2

25 ð 25 27 ð 27 213 ð 213 217 ð 217 219 ð 219

5 5 5 5 5

10 22 5,381 94,889 46,561

25 1 27 1 213 1 217 1 219 1

2. Relationship Between Transform Sizes and Modulus The choice of the modulus depends on the dynamic range and the transform size needed as shown in Table 7. From ˛1 and ˛2 of Table 7, the values of ˛1 and ˛2 for any transform-size power of 2 up to Nmax ð Nmax can be calculated. As an example, a number of different transform parameters for the modulus 213 1 are given in Table 8. 3. Calculation of 2-D Convolutions Using the 2-D NMNT Similar to the one-dimensional transform, the two-dimensional transform has the 2-D cyclic convolution property and hence can be used for the calculation of 2-D convolution/correlation for image-processing applications. yn, m D transform fXk, l Hev k, l C XN k, N l Hod k, lg D transformfXk, l0Hk, lg

131

TABLE 8 TRANSFORM PARAMETERS FOR THE MODULUS 213 1. Transform Sizes 8ð8 16 ð 16 32 ð 32 64 ð 64 128 ð 128 256 ð 256 512 ð 512 1024 ð 1024 2048 ð 2048 4096 ð 4096 8192 ð 8192

˛1 64 1,735 42 2,498 4,634 336 3,039 253 3,390 49 5

˛2 64 812 7,010 3,389 2,338 1,198 3,779 2,016 1,624 4,664 2,810

60


where Hev k, l and Hod k, l stand for even and odd parts of Hk, l, respectively, and are given by Hev k, l D hfHk, l C HN k, N 1g/2iMp and

Hod k, l D hfHk, l HN k, N 1g/2iMp

132

Finally, in order to ensure a meaningful result free from overflow effects, the input and filter array must be properly scaled such that yn, m < M

133

The procedure for the calculation of 2-D convolution/correlation using this transform is shown in Figure 17. 4. Multidimensional Transform The multidimensional forward and inverse transforms are defined next. a. The M-D Forward Transform. The multidimensional (M-D) forward transform of xn1 , n2 , . . . , nM of dimensions N ð N ð Ð Ð Ð N is defined as Xk1 , k2 , . . . , kM D

N1 N1 n1 D0 n2 D0

N1

...

xn1 , n2 , . . . , nM

nM D0

ð ˇn1 k1 , n2 k2 , . . . , nM kM

134 Mp

ki D 0, 1, 2, . . . , N 1,

i D 1, 2, . . . , M

ˇn1 , n2 , . . . , nM D ˇ1 n1 , n2 , . . . , nM C ˇ2 n1 , n2 , . . . , nM 135 M

ˇ1 n1 , n2 , . . . , nM D

ni

Re˛1 C j˛2 iD1

136 Mp

h(n,m) 2-D Transform H(k,l) x(n,m)

X(k,l) 2-D Transform

Y(k,l)

2-D Transform

y(n,m)

FIGURE 17. The 2-D convolution using the new 2-D transform.

61

NUMBER THEORETIC TRANSFORMS AND THEIR APPLICATIONS M

ˇ2 n1 , n2 , . . . , nM D

ni

Im˛1 C j˛2 iD1

137 Mp

For transform length N/d ð Ð Ð Ð ð N/d; ˇ1 and ˇ2 are given by M

ˇ1 n1 , n2 , . . . , nM D

ni

Re˛1 C j˛2 d iD1

138 Mp

M

ˇ2 n1 , n2 , . . . , nM D

Im˛1 C j˛2

ni

d iD1

139 Mp

b. The M-D Inverse Transform. given by:

The M-dimensional inverse transform is

xn1 , n2 , . . . , nM D

1/NM

N1 N1

...

k1 D0 k2 D0

N1 kM D0

Xk1 , k2 , . . . , kM

ð ˇn1 k1 , n2 k2 , . . . , nM kM Mp

ni D 0, 1, 2, . . . , N 1,

i D 1, 2, . . . , M

140

5. The Calculation of Multidimensional Convolution The M-D convolution property of this transform is given as: yn1 , n2 , . . . , nM D xn1 , n2 , . . . , nM Ł hn1 , n2 , . . . , nM D transform fXk1 , k2 , . . . , kM Hev k1 , k2 , . . . , kM C XNk1 , Nk2 , . . . , NkM Hod k1 , k2 , . . . , kM g D transform fXk1 , k2 , . . . , kM 0Hk1 , k2 , . . . , kM g 141 where Hev and Hod stand for even and odd parts of H , respectively.

62


C. The Separable Two-Dimensional and Multidimensional Mersenne Transforms (2-D NMNT) In some cases a separable transform, where the transform can be calculated using the row-column method, is preferable. One advantage in using a separable transform in two dimensions and more is the simplicity of using algorithms and programs developed for the 1-D case. Therefore, it is the aim of this section to present a separable version of the transform introduced in Equation (126). 1. Definition of the Separable Two-Dimensional Transform (2-D NMNT) Although the transform defined in Equation (126) is the true two-dimensional extension of the 1-D transform defined in Equation (106), Equation (126) is not separable because ˇn, m 6D ˇnˇm 142 when the row-column method is applied where the 2-D transform computation is broken into a series of 1-D transforms executed first along rows and then along the columns or vice versa. It is preferable to use a separable transform and hence define the 2-D transform as follows. a. The Forward Transform. The 2-D separable forward transform of xn,m of dimensions N ð N is defined as [Boussakta and Holt, 1995b] Xk, l D

N1 N1

xn, mˇnkˇml

nD0 mD0

143 Mp

k D 0, 1, 2, . . . , N 1,

l D 0, 1, 2, . . . , N 1

where ˇn D ˇ1 n C ˇ2 n

ˇm D ˇ1 m C ˇ2 m

144

ˇ1 n D hRe˛1 C j˛2 n iMp

ˇ1 m D hRe˛1 C j˛2 m iMp

145

ˇ2 n D hIm˛1 C j˛2 iMp

ˇ2 m D hIm˛1 C j˛2 iMp

146

n

m

for transform dimensions N/d ð N/d; ˇ1 and ˇ2 are given by ˇ1 n D hRe˛1 C j˛2 d n iMp ,

ˇ2 m D hIm˛1 C j˛2 d m iMp

Re(Ð) and Im(Ð) stand for real and imaginary parts, respectively.

147


63

b. The Inverse Transform. The transform defined in Equation (143) has a similar inverse defined as N1 M1 2 xn, m D 1/N Xk, lˇnkˇml 148 kD0 lD0

n D 0, 1, 2, . . . , N 1,

Mp

m D 0, 1, 2, . . . , M 1

Equation (143) and Equation (149) form a transform pair, and they are both separable and exact. 2. Arithmetic Complexity A successive calling of the 1-D transform applied along the rows and then along the columns using the row-column method can be used. For an image size of N ð N, the number of arithmetic operations using different numbers of butterflies to remove trivial multiplications is given in Table 9. Stages of order 3 and higher include twiddle factors ˇ1 N/8 and ˇ2 N/8, which are both equal to a power of 2 and can be calculated using shifts only [Boussakta and Holt, 1995a]. 3. The Calculation of 2-D Convolution The two-dimensional transform defined in Equations (143) and (148) has the 2D cyclic convolution property. Consequently, it can be used in the calculation of 2-D convolutions or 2-D correlations for image-processing purposes without the effect of rounding errors or truncation. Let Hev k, l D Hk, l C Hk, l/2

149

Hod k, l D Hk, l Hk, l/2

150

Haev k, l D Hk, l C Hk, l/2

151

Haod k, l D Hk, l Hk, l/2

152

TABLE 9 ARITHMETIC COMPLEXITY FOR THE 2-D SEPARABLE NEW MERSENNE TRANSFORM. Number of Butterflies Used 1 2 3 5

Number of Multiplications 3 2 2 N log2 N 3 2 2 N log 2 N 3N C 3N 2 3 2 9 2 2 N log2 N 2 N C 6N 3 2 2 2 N log2 N 6N C 12N

Number of Shifts

N2 4N

Number of Additions 7 2 2 N log2 N 7 2 2 N log 2 N 3N C 3N 2 7 2 9 2 2 N log2 N 2 N C 6N 7 2 2 2 N log2 N 5N C 8N

64


and A D Hev k, l C Haev k, l/2

153

B D Hev k, l Haev k, l/2

154

C D Hod k, l C Haod k, l/2

155

D D Hod k, l Haod k, l/2

156

The convolution is given by Yk, l D Xk, l A C Xk, l B C Xk, l C C Xk, l D mod Mp D Xk, l5Hk, l

157

In real applications, to reduce the number of arithmetic operations the quantities A, B, C, and D are precalculated. A listing of the program to calculate 2-D convolution using this transform is available upon request from the authors (S. Boussakta). 4. Generalization of this Transform The transform can be easily extended to the multidimensional case; owing to the separability of the kernel, it can be calculated using a 1-D transform applied along each direction. a. The Forward Multidimensional Transform. The M dimensional (M-D) forward transform of xn1 , n2 , . . . , nM of dimensions N ð N ð . . . ð N is defined as N1 N1 N1 M ... xn1 , n2 , . . . , nM ˇni ki 158 Xk1 , k2 , . . . , kM D n1 D0n2 D0

nM D0

ki D 0, 1, 2, . . . , N 1, ˇni D ˇ1 ni C ˇ2 ni

iD1

Mp

i D 1, 2, . . . , M 159

ˇ1 ni D hRe˛1 C j˛2 ni iMp

160

ˇ2 ni D hIm˛1 C j˛2 iMp

161

ni

˛1 and ˛2 are as defined for the 1-D case. b. The Inverse Multidimensional Transform. The form of the inverse transform is the same as that of the forward transform except for a factor

65


of (1/NM :

xn1 , n2 , . . . , nM D 1/NM

N1 N1

...

k1 D0k2 D0

ni D 0, 1, 2, . . . , N 1,

N1

M

Xk1 , k2 , . . . , kM

kM D0

ˇni ki iD1

i D 1, 2, . . . , M

Mp

162

D. Summary In this section another family of transforms was introduced. The transforms were defined modulo the Mersenne numbers, where arithmetic operations are simple; the transform length is long and equal to a power of 2, making it amenable to fast algorithms. It has the cyclic convolution property and hence can be used for the calculations of digital filtering without introducing additional processing noise. The transform was shown for the two-dimensional and multidimensional cases. It is worth mentioning that the emphasis is put on Mersenne primes in order to give long transform lengths with simple arithmetic and residue reduction; however, the theory could be extended to rings defined modulo Mersenne numbers and some other rings. VI. COMBINATION OF 2-D NTTS USING THE 2-D MRC SUITABLE FOR PARALLEL IMAGE-PROCESSING APPLICATIONS A. Introduction Two-dimensional digital convolution and correlation are widely used for digital image-processing applications such as image filtering, enhancement, and recognition. Because the number of arithmetic operations is very large and the demand for real-time, high-resolution images is ever increasing, their computation becomes extensive; hence the need for parallel image-processing algorithms for high speed and high throughput is becoming more and more important. It is the aim of this section to introduce a method for the calculation of 2-D convolution and correlation for image-processing applications. The technique combines a recently developed 2-D transform based on the Mersenne numbers with the 2-D Fermat number transform, using the 2-D mixed radix conversion. The resulting combination uses fast two-dimensional residue transforms, which can be implemented in parallel for high speed and high throughput rate [Boussakta, 1999; Boussakta and Holt, 1996].1 1

This section includes material reprinted with permission from Elsevier Science.

66


The advantage of this method is that it achieves a large dynamic range, great parallelism, and simple modulo arithmetic. The 2-D convolution and correlation are calculated modulo a large integer number, giving a sufficient dynamic range, but all calculations are performed modulo smaller Mersenne and Fermat numbers, where arithmetic operations are known to be much easier than other moduli (equivalent to 1’s complement). This combination provides both simple arithmetic operations and more balanced use of hardware and software. Thus it is suitable for parallel calculation of two-dimensional digital convolution/correlation. The 2-D NMNT and 2-D FNT were described in Sections III and V, respectively. B. Two-Dimensional Composite Transform (2-D CNTT) The transform sizes of the 2-D transforms in Equation (3) and Equation (143) are powers of 2. Also, both transforms have the 2-D CCP. Hence, using the 2D MRC, they can be combined for greater dynamic range, greater parallelism, and greater speed [Boussakta, 1999].2 Table 10 shows the most practical Fermat and Mersenne numbers which can be used for this method with the corresponding transform sizes. Depending on the dynamic range required, any set of numbers from Table 10 could be used and combined by this technique. The transform size must exist in each chosen modulus. When mixing Fermat and Mersenne numbers, the choice of the word length becomes more flexible and the difference in the number of bits between different moduli can be small, leading to more balanced parallel residue transforms. TABLE 10 USEFUL MERSENNE AND FERMAT NUMBERS.

Mp 2p 1 Mersenne Primes

Possible Transform Sizes Powers of 2 Using Mersenne Primes N ð N D 2i ð 2i , i p

25 1 27 1 213 1 217 1 219 1 231 1 261 1

N ð N D 2i ð 2i , i 5 N ð N D 2i ð 2i , i 7 N ð N D 2i ð 2i , i 13 N ð N D 2i ð 2i , i 17 N ð N D 2i ð 2i , i 19 N ð N D 2i ð 2i , i 31 N ð N D 2i ð 2i , i 61

2

Ft 22 C 1 Fermat Numbers

Possible Transform Sizes Using Fermat Numbers N ð N D 2i ð 2i , i 2t

24 C 1 28 C 1 216 C 1

N ð N D 2i ð 2i , i 4 N ð N D 2i ð 2i , i 8 N ð N D 2i ð 2i , i 16

232 C 1 264 C 1

N ð N D 2i ð 2i , i 7 N ð N D 2i ð 2i , i 8

t


67


For example, the set (27 1, 28 C 1) will give a dynamic range of nearly 15 bits and transform size up to 27 ð 27 , whereas the set (216 C 1, 217 1) will give a dynamic range of nearly 33 bits and a transform size up to 216 ð 216 . C. Calculation of the 2-D Cyclic Convolution Using this Method The transforms defined in Equation (3) and Equation (143) have the 2-D cyclic convolution property (2-D CCP); hence, to calculate the N ð N-length cyclic convolution modulo M denoted by yn, m, the N ð N cyclic convolution modulo, each mi in parallel, are calculated; then, using the 2-D MRC, the convolution yn, m, can be constructed, as shown in Figure 18, for two moduli. D. Two-Dimensional Mixed Radix Conversion (2-D MRC) The 2-D mixed radix conversion of yn, m is zL n, m, zL1 n, m, . . . , z1 n, m, where the zi n, ms satisfy Equation (163): L1

L2

mi C zL1 n, m

yn, m D zL n, m iD1

mi C Ð Ð Ð C z3 n, mm1 m2 iD1

C z2 n, mm1 C z1 n, m

163

In 2-D mixed radix conversion, the zi n, ms can be easily generated from the residue digits (in our case the 2-D residue convolutions) as follows: z1 n, m D hyn, mim1 D y1 n, m

z2 n, m D m11 yn, m z1 n, m m2

1 z3 n, m D m2 yn, m z1 n, m z2 n, m

164 165 166

m3

.. . H1(n, m) x1(n,m)

2-D IFNT

2-D FNT

y1(n, m) 2-D MRC

x(n,m)

y(n, m)

2-D NMNT

2-D NMNT

y2(n, m)

x2(n,m) H2(n, m) x1 (n) = x(n) mod Ft x2 (n) = x(n) mod Mp

FIGURE 18. The calculation of 2-D convolution using the new method using two moduli.

68

S. BOUSSAKTA AND A. G. J. HOLT H1(n, m) x1(n, m)

x2(n, m)

2-D CNTT

2-D CNTT

2-D CNTT

2-D CNTT

y1(n, m)

y2(n, m)

x(n, m)

x3(n, m)

2-D MRC

H2(n, m) 2-D CNTT

2-D CNTT

y(n, m)

y3(n, m)

H3(n, m) xn(n, m)

2-D NMNT

2-D CNTT

yn(n, m)

Hn(n, m) xi (n,m) = x(n,m) mod mi, i = 1,2,..n, mi is either a Mersenne or a Farmet number hi (n, m) = h(n, m) mod mi 2-D CNTT is either 2-D FNT or 2-D-NMNT

FIGURE 19. The calculation of 2-D convolution using the new method for n moduli.

From Equations (163) –(166), it can be seen that the calculation of the 2-D convolution depends only on smaller residue moduli mi ’s, which are in this case the popular Fermat and Mersenne numbers. Therefore, the calculation of the convolution modulo M can be carried out by performing the calculations with respect to two or more relatively prime moduli (m1 , m2 , . . . , mk ). Then the convolution modulo M D m1 ð m2 ð . . . ð mk is constructed using the 2-D MRC. This does not involve difficult and time-consuming arithmetic operations and residue reduction modulo M. The process of calculating a 2-D convolution and correlation using this method is shown in Figure 19 for n moduli.

E. Combination of the 2-D NMNT and 2-D FNT for Two Moduli For the purpose of illustration and without loss of generality, only two moduli are used in Table 11. From Table 11, it can be seen that larger transform blocks are obtainable when the 2-D new Mersenne number –based transform is used, as shown in Figure 18 for two moduli.

69

NUMBER THEORETIC TRANSFORMS AND THEIR APPLICATIONS TABLE 11 POSSIBLE TRANSFORM SIZES USING THIS METHOD. The Modulus (M)

Possible Transform Sizes Using This Method

27 128 C 1

N ð N D 2i ð 2i , i 7

213 1216 C 1

N ð N D 2i ð 2i , i 13

1216

C 1

N ð N D 2i ð 2i , i 16

231 1232 C 1

N ð N D 2i ð 2i , i 7

1264

N ð N D 2i ð 2i , i 8

217 261

C 1

Note that for greater dynamic range, more than two moduli can be combined. However the transform size will be limited by the modulus with the smallest transform size. Example Two moduli, m1 D F3 D 257 and m2 D M7 D 127, are chosen for this example. This choice gives a dynamic range of M D 257 ð 127 D 32,639 ³ 15 bits. The corresponding parameters are given in Table 12. Using these two moduli, a two-dimensional transform size up to N ð N D 128 ð 128 could be chosen. The input image is chosen to be a simple square padded with zeros to give a 16 ð 16 results as follows: 4 4 4 4 4 4 4 4 xn, m D 0 0 0 0 0 0 0 0

4 4 4 4 4 4 4 4 0 0 0 0 0 0 0 0

4 4 4 4 4 4 4 4 0 0 0 0 0 0 0 0

4 4 4 4 4 4 4 4 0 0 0 0 0 0 0 0

4 4 4 4 4 4 4 4 0 0 0 0 0 0 0 0

4 4 4 4 4 4 4 4 0 0 0 0 0 0 0 0

4 4 4 4 4 4 4 4 0 0 0 0 0 0 0 0

4 4 4 4 4 0 4 4 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

70


The 2-D filter is chosen to be the same: 4 4 4 4 4 4 4 4 hn, m D 0 0 0 0 0 0 0 0

4 4 4 4 4 4 4 4 0 0 0 0 0 0 0 0

4 4 4 4 4 4 4 4 0 0 0 0 0 0 0 0

4 4 4 4 4 4 4 4 0 0 0 0 0 0 0 0

4 4 4 4 4 4 4 4 0 0 0 0 0 0 0 0

4 4 4 4 4 4 4 4 0 0 0 0 0 0 0 0

4 4 4 4 4 4 4 4 0 0 0 0 0 0 0 0

4 4 4 4 4 4 4 4 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

The residue convolutions are calculated using the 2-D Fermat number and the 2-D new Mersenne-based transforms with a transform size of 16 ð 16. The 2-D residue convolution y1 n, m using the 2-D FNT with parameters (Ft , ˛, N ð N D 257, 2, 16 ð 16 is given by 16 32 48 64 80 96 112 128 y1 n, m D 112 96 80 64 48 32 16 0

32 64 96 128 160 192 224 256 224 192 160 128 96 64 32 0

48 96 144 192 240 31 79 127 79 31 240 192 144 96 48 0

64 128 192 256 63 127 191 255 191 127 63 256 192 128 64 0

80 160 240 63 143 223 46 126 46 223 143 63 240 160 80 0

96 192 31 127 223 62 158 254 158 62 223 127 31 192 96 0

112 224 79 191 46 158 13 125 13 158 46 191 79 224 112 0

128 256 127 255 126 254 125 253 125 254 126 255 127 256 128 0

112 224 79 191 46 158 13 125 13 158 46 191 79 224 112 0

96 192 31 127 223 62 158 254 158 62 223 27 31 192 96 0

80 160 240 63 143 223 46 126 46 223 143 63 240 160 80 0

64 128 192 256 63 127 191 255 191 127 63 256 192 128 64 0

48 96 144 192 240 31 79 127 79 31 240 192 144 96 48 0

32 16 0 64 32 0 96 48 0 128 64 0 160 80 0 192 96 0 224 112 0 256 128 0 224 112 0 192 96 0 160 80 0 128 64 0 96 48 0 64 32 0 32 16 0 0 0 0

Because the modulus 257 does not offer a sufficient dynamic range, the 2-D residue convolution calculated using the 2D FNT is in error. The pixels in error are highlighted in bold.

71

NUMBER THEORETIC TRANSFORMS AND THEIR APPLICATIONS TABLE 12 TRANSFORM PARAMETERS FOR COMBINING 257 (28 C 1) and 127(27 1).

Transform Sizes 8ð8 16 ð 16 32 ð 32 64 ð 64 128 ð 128

m1 D F3 D 257 ˛ 4 2 p 2 81 9

m2 D M7 D 127 ˛2 ˛1 8 106 102 49 5

8 24 30 34 22

The second 2-D residue convolution y2 n, m using the 2-D NMNT with parameters (Mp , ˛1 , ˛2 , N ð N D 127, 21, 24, 16 ð 16 is 16 32 48 64 80 96 112 1 y1 n, m D 112 96 80 64 48 32 16 0

32 48 64 80 96 112 1 112 96 80 64 48 32 16 0 64 96 1 33 65 97 2 97 65 33 1 96 64 32 0 96 17 65 113 34 82 3 82 34 113 65 17 96 48 0 1 65 2 66 3 67 4 67 3 66 2 65 1 64 0 33 113 66 19 99 52 5 52 99 19 66 113 33 80 0 65 34 3 99 68 37 6 37 68 99 3 34 65 96 0 97 82 67 52 37 22 7 22 37 52 67 82 97 112 0 2 3 4 5 6 7 8 7 6 5 4 3 2 1 0 97 82 67 52 37 22 7 22 37 52 67 82 97 112 0 65 34 3 99 68 37 6 37 68 99 3 34 65 96 0 33 113 66 19 99 52 5 52 99 19 66 113 33 80 0 1 65 2 66 3 67 4 67 3 66 2 65 1 64 0 96 17 65 113 34 82 3 82 34 113 65 17 96 48 0 64 96 1 33 65 97 2 97 65 33 1 96 64 32 0 32 48 64 80 96 112 1 112 96 80 64 48 32 16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Again, because the modulus 127 does not offer a sufficient dynamic range, the 2-D residue convolution calculated using the 2-D NMNT is in error. The pixels in error are highlighted in bold. Now applying the 2-D MRC to this example gives yn, m D m1 z2 n, m C z1 n, m D 257z2 n, m C z1 n, m

167

with z1 n, m D hyn, mim1 D y1 n, m

z2 n, m D m11 yn, m z1 n, m m2

1 D m1 y2 n, m y1 n, m

168

m2

D h85y2 n, m y1 n, mi127

169

72


Substituting y1 n, m and y2 n, m with their values in Equations (167), (168), and (169) gives the correct 2-D convolution result yn, m. 16

32

48

32

64

96 128 160 192 224

64

80

96 112

64

48

32

16 0

256 224 192 160 128

48

96 144 192 240 288 336

96

64

32 0

384 336 288 240 192 144

96

48 0

64 128 192 256 320 384 448

512 448 384 320 256 192 128

64 0

80 160 240 320 400 480 560

640 560 480 400 320 240 160

80 0

96 192 288 384 480 576 672

768 672 576 480 384 288 192

96 0

112 224 336 448 560 672 784 yn, m D

128 112

96

80

896 784 672 560 448 336 224 112 0

128 256 384 512 640 768 896 1024 896 768 640 512 384 256 128 0 112 224 336 448 560 672 784

896 784 672 560 448 336 224 112 0

96 192 288 384 480 576 672

768 672 576 480 384 288 192

96 0

80 160 240 320 400 480 560

640 560 480 400 320 240 160

80 0

64 128 192 256 320 384 448

512 448 384 320 256 192 128

64 0

48

96 144 192 240 288 336

384 336 288 240 192 144

96

48 0

32

64

96 128 160 192 224

256 224 192 160 128

96

64

32 0

16

32

48

64

80

128 112

0

0

0

0

0

96 112 0

0

0

0

96

80

64

48

32

16 0

0

0

0

0

0

0 0

From this example, it can be seen that the convolution is calculated modulo M D 257 ð 127, which has a large dynamic range, but arithmetic operations are carried out modulo 257 and 127, so they are simpler and faster. Also the residue convolutions yi n, m are not a problem if they overflow; the final result is correct provided there is no overflow modulo M. F. Advantages of This Method 1. The two-dimensional residue transforms are totally independent and hence can be calculated in parallel for high-speed processing and high throughput rate. 2. The method allows the dynamic range to be sufficiently large, but all operations are calculated using small moduli. 3. The residue moduli can be chosen to be the Mersenne and Fermat numbers, where arithmetic operations are known to be easier than when using other moduli. 4. The transform sizes of the residue transforms are equal to a power of 2; hence they are suitable for computation using fast algorithms such as radix-2 ð 2, radix-4 ð 4, split radix, etc. 5. The method is free from rounding and truncation errors.


73

VII. HARDWARE IMPLEMENTATIONS A. TTL-Based Designs The implementation of NTTs has been greatly influenced by developments in DSP hardware. Implementations by McClellan and Rader [1976] and Smith [1983] using the TTL circuits available at the time proved the operation of FNTs. Later both TTL and gate array devices were used [Shakaff, Pajayakrit, and Holt, 1988] in an investigation of the FNT as a block-mode imagefiltering tool. A block diagram of the overall system is shown in Figure 20. It consists of a single board, 16-bit MC68000 processor with VME interface, special-purpose 64-point FNT hardware, a modulo F4 multiplier, an auxiliary TMS32010 coprocessor also programmed to compute modulo F4 multiplications, fast memory banks for the transform process, and the essential image-capture, -storage, and -display hardware. The whole image-filtering process is controlled by the MC68000. This includes uploading the digitized image from the frame grabber, performing the overlap-save sectioning procedures, managing the movement of data blocks to and from the FNT hardware during the 2-D FNT computations, initiating the FNT and other hardware, controlling the multiplication process, and, finally, displaying the results. The hardware was microcoded TTL circuitry operated as a slave processor to a host MC68000. p It was designed to compute a 64-point transform using modulus F4 , with 2 as the basis function. A 17-bit word length is required, VME bus interface

Frame display

TV monitor

Decoder ‘start’

RAM

VME interface MC68000 PIA ports

Frame grabber Multiplier modulo F4

decoder

TMS 17-bit wide 32010 memory banks ‘done’

Memory banks control

‘Start FNT’ FNT hardware ‘End FNT’

Camera

FIGURE 20. Outline diagram of FNT used as block-mode image-filtering tool.

74


but arithmetic involving the extra bit is avoided by using the simplified binary operations described by Leibowitz [1976], with data being coded in the “diminished-1” representation. B. Pipelined Designs An NMOS VLSI circuit, a set of which can be cascaded to form a 32-bit FNT operation over F4 , is described in Towers, Pajayakrit, and Holt [1987] and Alfredsen [1996]. With the addition of a modulo F4 multiplier, a fast convolver/correlator can be constructed. The design comprises one complete section of a pipelined transformer and is novel in that it can be programmed to function at any point in a forward or inverse pipeline, thus allowing the construction of a pipelined convolver or correlator using identical chips. This overcomes the difficulty of fitting a complete pipeline onto one chip without resorting to the use of several designs. See also Truong et al. [1983] and Truong et al. [1982]. The pipeline sections described in Towers, Pajayakrit, and Holt [1987] were implemented in the NMOS technology available at the time. Later designs were based on the CMOS technology when it became available [Bouridane et al., 1989; Jullien, 1991]. The pipeline designs for FNTs described require multipliers modulo a Fermat number. Two VLSI designs for diminished-1 multiplication modulo a Fermat number are given in Benaissa et al. [1988a,b], Jullien, [1980], Sander, El-Guibaly, and Antinou [1993], and Ashur, Ibrahim, and Aggoun [1994]. A CMOS-based VLSI design giving a significant improvement in speed of operation and a higher data rate is described in Benaissa, Dlay, and Holt [1991a]. It is based on the use of three-input modulo F4 adders and uses the decimation-in-time algorithm. Using a 2.5-m process, it has predicted clock and sample rates of 16 MHz and 8 MHz, respectively, and can be cascaded to perform a 64-point FNT/IFNT. Alternative approaches to the problems of VLSI implementations of number theoretic transforms are given by Wigley and Jullien [1992], Nagpal, Jullien, and Miller [1983], Arambepola [1989], and Gudvangen [1997]. C. Vector Radix Method An effective algorithm for computing the 2-D FFT as an alternative to the conventional row-column decomposition has been presented by Rivard [1977] and Harris et al. [1977]. Known as the vector radix algorithm, it saves 25% of the complex multiplications and avoids the matrix-transpose operation. In this method the decimation is performed in both rows and columns simultaneously.

75


The 2-D FFT is decomposed successively into smaller 2-D FFTs until only trivial 2-D FFTs need to be evaluated. This algorithm can be applied to the 2-D FNT. An approach using the vector radix algorithm for the FNT and a CMOS VLSI design of a pipeline programmable vector radix FNT computational unit is described in Benaissa, Dlay, and Holt [1991b]. Only a programmable delay commutator chip is required to complete the 2-D convolution system. This allows a substantial reduction of the computational time required for image filtering via the FNT to be achieved at the expense of more hardware complexity. Figure 21 shows a block diagram for an N ð N pipeline vector radix 2 ð 2 FNT. It involves programmable variable-length delays, commutators, and 2-D radix 2 ð 2 FNT butterflies. An important feature of this architecture is that only two types of circuit are used, the butterfly and the variable-length delay. The data flow and interstage reordering of an 8 ð 8 pipeline vector radix FNT are illustrated in Figure 22; additional information is given in Figure 8 of the reference.3

2

ZN /2+N/2

ZN+1

OUTPUT

2

ZN /4

Butterfly

ZN/2

ZN

Commutator

ZN/4

Butterfly

Commutator

INPUT

2

ZN /2

Z1

2

ZN /4+N/4

FIGURE 21. N ð N pipeline vector radix 2 ð 2 butterfly.

Com mutat or 1

32

4

Butter fly 1

2

16 18

INPUT

9

18

36

Com mutat or 2

16

2

Butter fly 2

1

8

Com mutat or 3

8

Butter fly 3

1

9 OUTPUT

FIGURE 22. Data pattern through delay commutators for 8 ð 8 pipeline vector radix 2-D FNT.

3


76


The basic radix 2 ð 2 butterfly is shown in Figure 23. Because N is a power of 2, the decimation procedure requires log2 N stages, each stage consisting of N2 /4 butterflies and each butterfly involving three multiplications (shifts) and eight additions. Consequently the numbers of multiplications and additions performed in the computation of an N ð N radix 2 ð 2 FNT are Nmul D 34 N2 log2 N and Nadd D 2N2 log2 N, respectively. The number of multiplications is, therefore, 25% less than that required for a row-column decomposition (N2 log2 N), but the number of additions remains the same. In order to translate a DSP application into silicon, the most important decision to make is to select a suitable architecture. This selection depends mainly on the following points: first, on the modularity, parallelism, and data-flow mechanism of the application; second, on the throughput requirements; and third, on the feature size of the accessible technology. One of the most effective architectures for VLSI implementation of DSP applications is the pipeline scheme, due to its inherent modularity and suitability for high-throughput applications. Modulus F4 was chosen for convenient implementation. The computational unit is the main part of the 2-D transform. It was designed to function in both forward and inverse directions of the transform and according to the decimation stage of the vector radix algorithm under the control of externally supplied rank and direction signals. This approach allows a whole 2-D FNT butterfly section to be implemented on one chip, and a set of these are cascaded with delay commutator circuits and a multiplier to build a 2-D FNT convolver. A block diagram of the 2-D computational unit is shown in Figure 24; it consists of the basic circuits of an FNT butterfly. The arithmetic used is the diminished-1 arithmetic. The design can be divided into four major parts, as follows. 1. Addition/subtraction part: This part comprises four adders and four subtractors, implemented using diminished-1 arithmetic and two-input carrylook-ahead logic. 2. Multiplication by powers of 2 part: This part comprises three multipliers. Each multiplier is implemented with a modified barrel shifter capable of shifting left (positive) or right (negative) the input data under the control of an exponent generator driven by a programmable logic array finite-state machine (PLAFSM) based counter and a shifter. The exponent generator generates the exponents required in the multiplication, according to the different stages of the vector radix algorithm. The sequences of exponents for the three multipliers in the different stages of both the forward and inverse transforms using the vector radix algorithm are given in Tables 13, 14, and 15.

77


x (m, n)

X(k, l)

x (m, n+N/2)

X(k, l+N/2)

x (m+N/2, n)

X (k+N/2, l)

X (k+N/2, l+N/2)

x (m+N/2, n+N/2)

FIGURE 23. Basic radix 2 ð 2 butterfly.

INPUT

L04

L03 adder

adder

adder

L02 L01

adder

ENB

negator

SCAN

ENB

ENB

ENB

reg12

ENB

reg10

reg11

FSM Controler

negator

negator

reg13

reg6

RSTT

reg7

reg2

L01 ENB

ENB

reg8

reg3

ENB

reg9

L02

L04

reg4

L03 ENB

reg5

negator adder

adder

adder adder

reg15

reg14

RSTC

reg16

ENB

ENB

ENB

ENB

DIR2

reg17

ENBC1

ENBAR

OUTPUT

78

reg22

L01

reg26

ENB L02

reg23

ENB

reg18 reg19

ENB

ENBC3

FSM Controler

Multiply by powers of two

ENB

L04

reg24

ENBC2 ENB

reg20

Exp-gen


ENBC2 ENB

ENBC3

reg25

Exp-gen


Exp-gen

DIR1 RANK

reg21

ENBC2

FIGURE 24. Block diagram of a pipeline vector radix 2-D FNT computational unit.

reg1


79

3. Pipeline/testing registers: These registers divide the system into subblocks of reduced propagation delay, allowing a higher throughput. They were designed also to act as scan path registers during the testing mode. 4. The control part: This part controls all the operations of the chip. It consists of five programmable logic arrays (PLA) based finite-state machine (FSM) controllers. The first PLAFSM generates the LOAD and ENABLE signals used to multiplex the inputs and outputs in order to reduce the pin count. Because of multiplexing, the internal data rate is a quarter of the external clock and data rates. Therefore, the clocks of the different blocks in the chip are gated with the ENABLE signal, which controls all the clocked elements in the design. This PLAFSM can be reset by an external RESET LINE, and during testing mode it freezes the operation of the chip under the control of the external signal SCAN. Each of the three multipliers (barrel shifters) is controlled by a PLAFSM counter that generates the required sequence of exponents for the first stage in both forward and inverse directions. The exponents are then fed to a simple shifter to generate the sequences corresponding to the remaining ranks under the control of the externally supplied RANK signals. Each PLAFSM is clocked under the control of an ENABLE-COUNT (ENBC) signal generated by an additional PLAFSM. The use of a simple shifter and an extra PLAFSM significantly reduces the complexity of the PLAFSM counters. The last PLAFSM is used to generate the ENABLE-COUNT signals (ENBC1, ENBC2, ENBC3) required to control the three PLAFSM counters. As shown in Table 13, the sequence of exponents required for the first multiplier in the first stage consists of the iteration 16 times of the sequence f0, 1, . . . , 15g for the forward direction and its bit-reversed sequence for the inverse direction. Therefore, a direct use of a PLAFSM counter to generate all the exponents would result in a very large and slow — and, hence, impractical — PLAFSM. A more viable approach consists of using an additional PLAFSM that generates an ENABLE-COUNT (ENBC1 for mult 1) signal that controls the iteration of the much shorter sequence f0, 1, . . . , 15g 16 times, allowing the use of a much simpler PLAFSM counter. For the second multiplier Table 14 shows that the sequence of exponents is incremented only after each 16 cycles. Hence, ENBC2 is used to freeze the counting for each 16 cycles. For the third multiplier, Table 15 shows that the sequence of exponents is slightly more complicated than before. Not only are higher exponents required (up to 30), but also the sequencing starts from an incremented exponent after each 16 cycles. Therefore, ENBC3 controls the counting for each 16 cycles and the PLAFSM counter of the third multiplier is programmed to start counting after each 16 cycles from an incremented state.

80

S. BOUSSAKTA AND A. G. J. HOLT TABLE 13 EXPONENT SEQUENCES FOR THE FIRST MULTIPLIER

Rank Dir.

Sequence of Exponents

0

Forw. 0 1 Inv. 0 8

2 3 4 12

4 5 2 10

6 7 8 6 14 1

9 10 11 12 13 14 15. 16 Times 9 5 13 3 11 7 15. 16 Times

1

Forw. 0 2 Inv. 0 0

4 6 8 8

8 10 12 14 0 4 4 12 12 2

2 4 6 2 10 10

2

Forw. 0 4 Inv. 0 0

8 12 0 0

0 8

4 8

8 12 0 8 8 4

4 4

8 12 0 4 8 12. 16 Times 4 4 12 12 12 12. 16 Times

3

Forw. 0 8 Inv. 0 0

0 8 0 0

0 0

8 0

0 8 0 0 0 8

8 8

0 8

8 8

0 8

8 8

0 8

8. 16 Times 8. 16 Times

4

Forw. 0 0 Inv. 0 0

0 0 0 0

0 0

0 0

0 0 0 0 0 0

0 0

0 0

0 0

0 0

0 0

0 0

0 16 Times 0. 16 Times

8 10 12 14. 16 Times 6 6 14 14. 16 Times

The final layout of the vector radix computational unit section was designed in 2.5 m CMOS. The predicted clock rate was 18 MHz, and the estimated power consumption was 0.3 W. Due to multiplexing, the sample rate in a 16 ð 16-point impulse response overlap-save convolver would be 4.5 MHz. In order to complete the whole pipeline 2-D FNT, a design for a programmable delay commutator similar to that by Swartzlander and Hallnor [1985] was required. The design was programmable to take on different delay lengths according to the appropriate rank and direction of the pipeline. Table 16 shows the lengths of the delays required in a 32 ð 32-point forward and inverse vector radix pipeline transformer. It is clear that each pipeline section requires two programmable delay commutators, a prebutterfly and a postbutterfly chip. Consequently, the number of chips required to implement a 32 ð 32 point vector radix FNT transformer would be 5 ð 3 D 15. The CMOS designs discussed were for implementation with 2.5-μm technology. This work was carried out before the submicron technology became available. VIII. CONCLUSIONS Transforms defined in finite fields and rings with all arithmetic performed modulo an integer, which do not involve complex arithmetic or errors in calculation, have appreciable advantages for image processing and other applications. The 2-D NTTs are well suited to the calculation of two-dimensional convolutions and correlations. The choices of the transform size, N ð N, the modulus, F, and the basis function, ˛, used in the 2-D NTT have been widely

TABLE 14 EXPONENT SEQUENCES FOR THE SECOND MULTIPLIER. Sequence of Exponents for Forward Direction Rank 0

0 1 2 3 .. .

0 1 2 3 .. .

0 1 2 3 .. .

0 1 2 3 .. .

0 1 2 3 .. .

0 1 2 3 .. .

0 1 2 3 .. .

0 1 2 3 .. .

0 1 2 3 .. .

0 1 2 3 .. .

0 1 2 3 .. .

0 1 2 3 .. .

0 1 2 3 .. .

0 1 2 3 .. .

Sequence of Exponents for Inverse Direction 0 1 2 3 .. .

0 1 2 3 .. .

15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 Rank 1

0 2 4 .. .

0 2 4 .. .

0 2 4 .. .

0 2 4 .. .

0 2 4 .. .

0 2 4 .. .

0 2 4 .. .

0 2 4 .. .

0 2 4 .. .

0 2 4 .. .

0 2 4 .. .

0 2 4 .. .

0 2 4 .. .

0 2 4 .. .

0 2 4 .. .

0 2 4 .. .

14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14

81

Rank 2

0 4 8 .. .

0 4 8 .. .

0 4 8 .. .

0 4 8 .. .

0 4 8 .. .

0 4 8 .. .

0 4 8 .. .

0 4 8 .. .

0 4 8 .. .

0 4 8 .. .

0 4 8 .. .

0 4 8 .. .

0 4 8 .. .

0 4 8 .. .

0 4 8 .. .

0 4 8 .. .

12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 Rank 3

Rank 4

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. . . . . . . . . . . . . . . . . 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 0 0 8 .. .

0 0 8 .. .

0 0 8 .. .

0 0 8 .. .

0 0 8 .. .

0 0 8 .. .

0 0 8 .. .

0 0 8 .. .

0 0 8 .. .

0 0 8 .. .

0 0 8 .. .

0 0 8 .. .

0 0 8 .. .

0 0 8 .. .

0 0 8 .. .

0 0 8 .. .

14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12

0 8 0 .. .

0 8 0 .. .

0 8 0 .. .

0 8 0 .. .

0 8 0 .. .

0 8 0 .. .

0 8 0 .. .

0 8 0 .. .

0 8 0 .. .

0 8 0 .. .

0 8 0 .. .

0 8 0 .. .

0 8 0 .. .

0 8 0 .. .

0 8 0 .. .

0 8 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

8

8

8

8

8

8

8

8

8

8

8

8

8

8

8

8

8

8

8

8

8

8

8

8

8

8

8

8

8

8

8

8

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

TABLE 15 SEQUENCE OF EXPONENTS FOR THE THIRD MULTIPLIER. Sequence of Exponents for Forward Direction Rank 0

Rank 1

82

Rank 2

Rank 3

Rank 4

0 1 2 3 .. .

1 2 3 4 .. .

6 7 8 9 10 7 8 9 10 11 8 9 10 11 12 9 10 11 12 13 .. .. .. .. .. . . . . . 15 16 17 18 19 20 21 22 23 24 25 0 2 4 .. .

2 4 6 .. .

14

0

2 3 4 5 .. .

3 4 5 6 .. .

4 5 6 7 .. .

5 6 7 8 .. .

4 6 8 10 12 14 0 6 8 10 12 14 0 2 8 10 12 14 0 2 4 .. .. .. .. .. .. .. . . . . . . . 2 4 6 8 10 12 14

2 4 6 .. . 0

11 12 13 14 .. .

12 13 14 15 .. .

13 14 15 16 .. .

Sequence of Exponents for Inverse Direction 14 15 16 17 .. .

15 16 17 18 .. .

26 27 28 29 30

4 6 8 10 12 14 6 8 10 12 14 0 8 10 12 14 0 2 .. .. .. .. .. .. . . . . . . 2 4 6 8 10 12

0 4 8 12 0 4 8 12 0 4 8 12 0 4 8 12 4 8 12 0 4 8 12 0 4 8 12 0 4 8 12 0 8 12 0 4 8 12 0 4 8 12 0 4 8 12 0 4 .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. . . . . . . . . . . . . . . . . 12 0 4 8 12 0 4 8 12 0 4 8 12 0 4 8

0 16 8 24 4 20 12 28 2 18 10 26 6 22 14 30 16 8 24 4 20 12 28 2 18 10 26 6 22 14 30 1 8 24 4 20 12 28 2 18 10 26 6 22 14 30 1 17 Ð Ð Ð Ð Ð Ð Ð Ð Ð Ð Ð Ð Ð Ð Ð Ð .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. . . . . . . . . . . . . . . . . 30 1 17 9 25 5 21 13 29 3 19 11 27 7 23 15 0 8 4 .. .

0 8 8 4 4 12 12 2 2 10 10 6 6 14 14 8 4 4 12 12 2 2 10 10 6 6 14 14 0 0 4 12 12 2 2 10 10 6 6 14 14 0 0 8 8 .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. . . . . . . . . . . . . . . . 14 14 0 0 8 8 4 4 12 12 2 2 10 10 6 6 0 8 4 .. .

0 8 4 .. .

0 8 4 .. .

0 8 8 8 8 4 4 4 4 12 12 12 12 8 4 4 4 4 12 12 12 12 0 0 0 0 4 12 12 12 12 0 0 0 0 8 8 8 8 .. .. .. .. .. .. .. .. .. .. .. .. .. . . . . . . . . . . . . . 12 12 12 12 0 0 0 0 8 8 8 8 4 4 4 4

0 8 .. .

8 0 .. .

0 8 .. .

8 0 .. .

0 8 .. .

8 0 .. .

0 8 .. .

8 0 .. .

0 8 .. .

8 0 .. .

0 8 .. .

8 0 .. .

0 8 .. .

8 0 .. .

0 8 .. .

8 0 .. .

0 8 .. .

0 8 .. .

0 8 .. .

0 8 .. .

0 8 .. .

0 8 .. .

0 8 .. .

0 8 .. .

8 0 .. .

8 0 .. .

8 0 .. .

8 0 .. .

8 0 .. .

8 0 .. .

8 0 .. .

8 0 .. .

8

0

8

0

8

0

8

0

8

0

8

0

8

0

8

0

8

8

8

8

8

8

8

8

0

0

0

0

0

0

0

0

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0 0 0 .. .

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0


83

TABLE 16 DELAY LENGTHS FOR A PIPELINE 32 ð 32 POINT VECTOR RADIX TRANSFORMER. Forward Direction DIR D 0 Rank

0 1 2 3 4

Prebutterfly Delay Lengths 528, 264, 132, 66, 33,

512, 256, 128, 64, 32,

16 8 4 2 1

Inverse Direction (DIR D 1)

Postbutterfly Delay Lengths 8, 4, 2, 1, 0,

256, 128, 64, 32, 0,

264 132 66 33 0

Prebutterfly Delay Lengths 0, 1, 2, 4, 8,

0, 32, 64, 128, 256,

0 33, 66 132 264

Postbutterfly Delay Lengths 33, 32, 66, 64, 132, 128, 264, 256, 528, 512,

1 2 4 8 16

examined. The 2-D FNTs are shown to be suitable 2-D NTTs for the applications considered. These transforms have a highly composite transform length (power of two) and hence they are amenable to fast algorithms, have the capability of performing exact calculations, and maintain a simple arithmetic operation and hardware implementation if the kernel is properly selected. The transform size is limited if the 2-D FNT is used with ˛ D 2. Relaxing this condition for the whole transform or for some stages allows the transform length to be increased. Although the number domain generated by the NTTs is not always simple to interpret as in the case of the FFT, NTTs are meaningful and well structured. The study of some of the patterns of zeros that appear in the number domain for 2-D NTTs shows that, even though this pattern may be similar for some features, it might be used effectively in pattern-recognition problems where there is a need for classifying only a closed set of objects, such as occurs in character recognition. If the process is complicated and the zero pattern might be not secure, other pixels in the 2-D NTT domain can be selected to show the difference between different patterns. This is problem dependent and can be designed for the specific problem in the training session that precedes the pattern-recognition process. The 2-D NTTs offer an exceedingly sensitive method, whereby small errors in a nominally periodic pattern may be detected. The exact location of the pattern error may be determined by brief inspection of the transform. Also, it is not always necessary to calculate the complete transform. Analysis of the 2-D FNTs of periodic structures leads to a rule that simplifies the calculations and determines the position and values of nonzero elements in the transform domain, showing that 2-D NTTs are well structured. Some new applications of 2-D FNTs are suggested that indicate that the number domain can be utilized for applications other than convolution. Integrated circuit designs and microprocessor implementations for 2-D FNTs should make this transform more readily available for use in systems. The transform-size constraint for the 2-D FNT is not a great problem; in fact, several solutions are proposed, allowing the processing of sufficiently large

84


images while retaining the advantages of the FNT, such as fast computation and simple arithmetic. Other transforms, such as the number theoretic Walsh, may also be used in these applications. Another real transform defined modulo the Mersenne numbers, which provides long transform lengths equal to a power of 2, is introduced in Section V. This is done by dropping the condition that ˛ should be a power of 2 and using a new definition of NTTs that departs from the usual Fourierlike definition. This transform is suitable for fast algorithms. It has the cyclic convolution property and hence can be applied to the calculation of convolutions and correlations. The transform is extended to the separable case, making it a good candidate for the calculation of 2-D convolutions for image-processing applications. Section VI describes how the 2-D new transform is combined with the 2D FNT using the 2-D mixed radix conversion. The resulting transform uses fast, small residue transforms that can be implemented in parallel for high speed and high throughput rate. This method allows the combination of the Fermat and Mersenne number-based transforms with their moduli chosen to be conveniently close to one another, thus making more efficient use of the hardware and software than had previously been possible. The method is suitable for the calculation of 2-D convolutions and leads to increased dynamic range and to the convenient use of parallel operation. Also, all arithmetic operations are carried out modulo the Fermat and Mersenne numbers, which offer simple arithmetic. VLSI is an ideal medium for the implementation of transforms based upon NTTs and especially FNTs, because the moduli used are of the form 2b C 1, where b D 2t and so the word lengths required in the system are 2t C 1. The peculiar number of bits required presents no difficulty in a custom VLSI design but makes the use of standard 16- or 32-bit processors quite inefficient. A brief discussion is given in Section VII of VLSI implementations for 2-D FNTs for image processing. The pipeline structure that can alleviate the computational burden for implementing a row-column 2-D FNT is discussed. Another approach using the vector radix algorithm is described, and a CMOS VLSI design of a pipeline programmable vector radix 2-D FNT computational unit is presented. Only a programmable delay commutator chip is required to complete the 2-D convolution system. This is more efficient than the pipeline row-column structure. Thus, VLSI allows the 2-D FNT and other 2-D NTTs to be used in image-processing systems where high-accuracy and high-speed filtering is required. It is suggested that the 2-D NTTs offer a means of eliminating the noise arising from rounding and truncations in the use of other transforms and, using VLSI implementations, can provide the fast operations necessary for image processing. They are also used (for example, in Hsue [1995]) to avoid


85

round-off error in ill-conditioned problems arising in the solution of Toplitz systems of equations. REFERENCES Agarwal, R. C., and Burrus, C. S. (1974a). Number theoretic transforms to implement fast digital convolution, Proc. IEEE, 63(4), 550 – 560. Agarwal, R. C., and Burrus, C. S. (1974b). Fast one-dimensional digital convolution by multidimensional techniques, IEEE Trans. Acoust., Speech and Signal Processing ASSP-22(1), 1 – 10, Feb. Agarwal, R. C., and Burrus, C. S. (1974c). Fast convolution using Fermat number transforms with applications to digital filtering, IEEE Trans. Acoustics, Speech and Signal Processing ASSP-22(2), 87 – 97. Agarwal, R. C., and Burrus, C. S. (1975). Number theoretic transforms to implement fast digital convolution, Proc. IEEE 63(4), 550 – 560. Agarwal, R. C., and Cooley, J. W. (1974). New algorithms for digital convolution, IEEE Trans. Acoustics, Speech and Signal Processing ASSP-25(5), 392 – 410. Ahmed, M. O, and Nundararajan, D. (1987). A fast implementation of two-dimensional convolution algorithm for image processing applications, IEEE Trans. on Circuits and Syst. CAS-34, 577 – 579, May. Alfredsen, L. (1994). A fast Fermat number transform for long sequences, IEEE Seventh European Signal Processing Conference, pp. 1579 – 1581, September, Edinburgh, U.K. Alfredsen, L. (1996). VLSI architectures and arithmetic operations with applications to the Fermat number transform, Lenkoping Studies in Science and Technology (425), Linkoping, Sweden. Arambepola, B. (1989). VLSI architectures for convolver design using number theoretic transforms, Electron. Lett. 25(23), 1604 – 1606. Ashur, A. S., Ibrahim, M. K., and Aggoun, A. (1994). Area-time efficient diminished-1 multiplier for Fermat number transforms, Electron. Lett. 30(20), 1640 – 1641, Sept. Benaissa, M., Bouridane, A., Dlay S. S., and Holt, A. G. J. (1988). Diminished-1 multiplier for a fast convolver and correlator using the Fermat number transform, IEE Proc. G, Electron. Circuits and Syst. 135(5), 187 – 193. Benaissa, M., Dlay, S. S., and Holt, A. G. J. (1991a). CMOS VLSI design of a high-speed Fermat number transform-based convolver/correlator using three-input address, IEE Proc. G. 138(2), 182 – 190. Benaissa, M., Dlay, S. S., and Holt, A. G. J. (1991b). VLSI implementation issues for the 2-D Fermat number transform, Signal Processing 23, 257 – 272. Benaissa, M., Pajayakrit, A., Dlay, S. S., and Holt, A. G. J. (1988). VLSI design for diminished1 multiplication of integers modulo a Fermat number, IEE Proc. E, Computers and Digital Techniques 135(3), 161 – 164. Bongard, M., (1970). Pattern Recognition, New York, Washington: Spartan books. Bouridane, A., Pajayakrit, A., Dlay S. S., and Holt, A. G. J. (1989). CMOS VLSI Circuit of Pipeline sections for 32- and 64-point Fermat number transformers, Integration, the VLSI Journal 8, 51 – 64. Boussakta, S. (1990). Algorithms and development of the number theoretic and related fast transforms with applications, Ph.D. diss., University of Newcastle upon Tyne, U.K. Boussakta, S. (1999). A novel method for parallel image processing applications, Journal of Systems Architecture 45(11).

86


Boussakta, S., and Holt, A. G. J. (1988). Calculation of the discrete Hartley transform via the Fermat number transform using a VLSI chip, IEE Proc. G 135(3), 101 – 103. Boussakta, S., and Holt, A. G. J. (1989). Relationship between the Fermat number transform and the Walsh-Hadamard transform, IEE Proc. 136(Pt. G, no.4), 191 – 204, August. Boussakta, S., and Holt, A. G. J. (1991). Calculating linear from circular convolutions/correlations, Proc. Inst. of Acoustics, 13, (Pt. 9), 76 – 85. Boussakta, S., and Holt, A. G. J. (1992). New number theoretic transform, Electronics Letters 28(18), 1683 – 1684, August. Boussakta, S., and Holt, A. G. J. (1993a). New two-dimensional transform, Electronics Letters 29(11), 949 – 950. May 17. Boussakta, S., and Holt, A. G. J. (1993b). New fast symmetric long transform using Mersenne primes and its applications, Conference Proceedings of UDT, Undersea Defence Technology, Palais des Festivals et des Congres, Cannes, France, June 15 – 17, pp. 552 – 555. Boussakta, S., and Holt, A. G. J. (1994a). Filters based on a new transform, UDT, Conference Proceedings, Wembley, London, July 5 – 7, pp 314 – 319. Boussakta, S., and Holt, A. G. J. (1994b). Filtering employing a new transform, IEEE Proc. Conference OCEANS94, Brest, France, September 13 – 16, Vol. 1, pp 547 – 553. Boussakta, S., and Holt, A. G. J. (1995a). A new transform using the Mersenne Numbers, Proc. IEE, VIS. Image Signal Processing 142(6), 381 – 388, Dec. Boussakta, S., and Holt, A. G. J. (1995b). A new separable transform, Proc. IEE Vision, Image and Signal Processing 142(1), 27 – 30, February. Boussakta, S., and Holt, A. G. J. (1996). A novel combination of NTTs using the MRC, Signal Processing Journal 54, 94 – 98. Boussakta, S., Shakaff, A. Y., Marir, F., and Holt, A. G. J. (1988). Number theoretic transforms of periodic structures and their applications. IEE Proceedings 135, (Pt. G, no. 2), 83 – 96, April. Brigham, E. O. (1974). The Fast Fourier Transform. Englewood Cliffs, N.J.: Prentice Hall. Chevillat, P. R. (1978). Transform domain digital filtering with number theoretic transforms and limited wordlengths, IEEE Trans. Acoustics, Speech and Signal Processing ASSP-26(4), 284 – 290, Aug. Duhamel, P., and Hollman, H. (1982). Number theoretic transforms with 2 as a root of unity, Electronics Lett. 18, 978 – 980, Oct. 28. Duhamel, P., and Hollman, H. (1984). Split-radix FFT algorithm, Electronics Lett. (20), 14 – 16. Eklundh, J. O. (1972). A fast computer method for matrix transposing, IEEE Trans. C-21, 801 – 803. Gregory, R. T., and Krishnamurthy, E. V. (1984). Methods and Applications of Error-free Computation. New York: Springer-Verlag. Gudvangen, S. (1997). A class of sliding Fermat number transforms that admit a tradeoff between complexity and input-output, IEEE Trans. On Signal Processing, 45(12), 3094 – 3096, Dec. Harris, D. B., McClellan, J. H., Chan, D. S., and Scheussler, H. W. (1977). Vector radix fast Fourier transform, IEEE Int. Conf. ASSP, rec., 548 – 551. Helms, H. D. (1967). Fast Fourier transform method of computing difference equations and simulating filters, IEEE Trans. on Audio and Electroacoustics, AU-15(2), 85 – 90, June. Hollingum, J. (1984). Machine Vision: The Eyes of Automation. U.K.: IFS Publications. Hunt, B. R. (1972). Minimizing the computation time for using the technique of sectioning for digital filtering of pictures, IEEE Trans. on Computers (Nov), 1219 – 1222. Hsue, J. (1995). Fast algorithms for solving Toplitz systems of equations using number theoretic transforms, Signal Processing, 44(1), 89 – 101, June. Jullien. G. A. (1980). Implementation of multiplication modulo a prime number with applications to number theoretic transforms, IEEE Trans. Comput. C-29(10), 899 – 905, Oct. Jullien, G. A. (1991). Number theoretic techniques in digital signal processing, Advances in Electronics and Electron Physics, 80, 69 – 161.


87

King, R., Ahmadi, M., Gorgui-Naguib, R., Kwabwe, A., and Azimi-Sadjdi, M. (1989). Digital Filtering in One and Two Dimensions, London: Plenum Press. Krishnan, R., Jullien, G. A., and Miller, W. (1986a). Implementation of complex number theoretic transforms using quadratic residue number Systems, IEEE Trans. on Circuits and Systems CAS-33(8), 759 – 765, Aug. Krishnan, R., Jullien, G. A., and Miller, W. (1986b). The modified quadratic residue number system (MQRNS) for complex high-speed signal processing, IEEE Trans. on Circuits and Systems CAS-33(8), 325 – 327, Mar. Lee, S. C., and Lu, H. (1988). Fast convolution using generalised Fermat/Mersenne number transforms, Proc. IEEE, ICASSP, CH2561-9/88/0000-1910. Leibowitz, L. (1976). A simplified binary arithmetic for Fermat number transform, IEEE Trans. ASSP-24(5). 199 – 201, Oct. Li, W. and Peterson, A. M. (1990). FIR filtering by the modified Fermat number transform, IEEE Trans. Acoust., Speech, Signal Processing 38(9), 1641 – 1645, Sept. Liu, Huizhu, and Lee, C. (1991). A new approach to solve the sequence-length constraint problem in circular convolution using number theoretic transform, IEEE Trans. on Signal Processing 39(6), 1314 – 1321, June. Marir, F. (1986). The application of number theoretic transforms to two-dimensional convolutions and adaptive filtering, Ph.D. diss., University of Newcastle upon Tyne, U.K. Marir, F., Shakaff, A. Y., and Holt, A. G. J. (1985). Two-dimensional convolution using number theoretic transforms without matrix transposition and without overlap, IEE Proc. G, Electron. Circuits and Syst., 132(5), 211 – 215. Marshal, S., and Soraghan, J. J (1988). Shape analysis for object recognition using number theoretic transforms, IEEE Proc. International Conference on Acoustics, Speech and Signal Processing ICASSP.88, CH2561-9/88/0000-0964, pp. 964 – 967. Martens, J. B. (1983). Number theoretic transforms for the calculation of convolutions, IEEE Trans. Acoustics, Speech and Signal Processing ASSP-31(4), 969 – 978, Aug. McClellan, J. H., and Rader, C. M. (1970). Number Theory in Digital Signal Processing Englewood Cliffs, N.J.: Prentice Hall. McClellan, J. H., and Rader, C. M. (1976). Hardware realization of a Fermat number transform IEEE Trans., Acoust., Speech and Signal Processing ASSP-24, 216 – 225. Meyer, R. (1989). Error analysis and comparison of FFT implementation structures, IEEE Int. Conference, Acoustics, Speech and Signal Processing ICASSP.89, Glasgow, Scotland, 888 – 891. Morandi, C., Piazza, F., and Dolcetti, A. (1988). Image registration using Fermat transforms, Electronics Letters, 24(11), 678 – 680. May. Myers, D. G. (1990). Digital Signal Processing: Efficient Convolution and Transform Techniques, New York: Prentice Hall. Nagpal, H. K., Jullien, G. A., and Miller, W. C. (1983). Processor architectures for two-dimensional convolvers using a single multiplexed computational element with finite field arithmetic, IEEE Trans. on Computers C-32(11), 989 – 1001, Nov. Nussbaumer, H. J. (1976). Complex convolution via Fermat number transforms, IBM J. Res. Develop. 20, 282 – 284. Nussbaumer, H. J. (1977). Relative evaluation of various number theoretic transforms for digital filtering applications, IEEE Trans. on Acoustics, Speech and Signal processing ASSP-26, 83 – 93. Oppenheim, A. V., and Weinstein, C. J. (1972). Effects of finite register length in digital filtering and fast Fourier transform, Proc. IEEE 60(8), 957 – 976, Aug. Parker, M. G. and Benaissa, M. (1995). Unusual length number theoretic transforms using recursive extensions of Rader’s algorithm, IEE Proc. Vis. Image Signal Process 142(1), 31 – 34, Feb.

88


Pollard, J. M, (1971). The fast Fourier transform in a finite field, Mathematics of Computation 25(114), 365 – 374, Apr. Rabiner, L. R., and Gold, B. (1975). Theory and Application of Digital Signal Processing, Englewood Cliffs, NJ: Prentice-Hall. pp. 419 – 433. Rader, C. M. (1972a). Discrete convolution via Mersenne transforms, IEEE Trans. Comput. C21, 1269 – 1273, Dec. Rader, C. M. (1972b). The number theoretic DFT and exact discrete convolution, IEEE Arden House Workshop on Digital Signal Processing, Harriman, N.Y., Jan. 11. Rader, C. M. (1975). On the application of the number theoretic methods of high speed convolution to two-dimensional filtering, IEEE Trans. Circuits and Syst. CAS-22, 575, June. Reed, I. S. (1975). The use of finite fields to compute convolutions, IEEE Trans. on Information Theory IT-21(2), 203 – 208, March. Reed, I. S., and Truong, L. R. (1978). The fast decoding of Reed-Solomon codes using the Fermat transforms, IEEE Trans. on Information Theory IT-24(4), 497 – 500, July. Reed, I. S., Truong, T. K., Kwoh, Y. S., and Hall, E. L. (1977). Image processing by transforms over a finite field, IEEE Trans. on Computer C-26(9), 874 – 887, Sept. Rivard, G. E., (1977). Direct fast Fourier transform of bivariate functions, IEEE Trans. On Acous. Speech, Signal Process ASSP 25(3), 250 – 252, June. Robanov, N. S., Bovbel, E. I, Kucharchik, P. D., and Bodrov, V. J. (1998). The modified number theoretic transform over the direct sum of finite fields to compute the linear convolution, IEEE Trans. On Signal Processing 46(3), 813 – 817, March. Sander, S., El-Guibaly, F. and Antinou, A. (1993). Area efficient diminished-1 multiplier for Fermat number transforms, IEE Proc. G, Circuits Devices and Systems 140(3), 211 – 15, June. Schroeder, M. R. (1990). Number Theory in Science and Communication, New York: SpringerVerlag. Shakaff, A. Y. (1987). Practical implementation of the Fermat number transform with applications to filtering and image processing, Ph.D. diss., University of Newcastle upon Tyne, U.K. Shakaff, A. Y., Pajayakrit, A., and Holt, A. G. J. (1988). Practical implementations of blockmode image filters using the Fermat number transform on a microprocessor-based system, IEE Proc. G, Electron. Circuits and Syst. 135(4), 141 – 154. Siu, W. C., and Constantinides, A. G. (1983). Very fast discrete Fourier transform using number theoretic transform, IEE Proc. G, Circuit Theory and Applications 130(5), 201 – 204. Siu, W. C., and Constantinides, A. G. (1984). On the computation of discrete Fourier transform using Fermat number transform, IEE Proc. 130(5), 201 – 204, Feb. Smith, N. E. (1983). The application of number theoretic transforms to fixed coefficient and adaptive digital filters, Ph.D. diss., University of Newcastle upon Tyne, U.K. Swartzlander, E. E., and Hallnor, G., (1985). Frequency domain digital filtering with VLSI, VLSI and Modern Signal Processing, Englewood Cliffs, N.J.: Prentice Hall, 153 – 169. Szabo, N. S., and Tanaka, R. I. (1967). Residue Arithmetic and Its Applications to Computer Technology, NewYork: McGraw-Hill. Thomas, J. J, Larsen, G. N., and Keller, J. R. (1983). Number theoretic transforms with independent length and moduli, IEEE Trans. On Acoustics, Speech, and Signal Processing ASSP-31(1), 215 – 216, Feb. Towers, P. J., Pajayakrit, A., and Holt, A. G. J. (1987). Cascadable NMOS VLSI circuit for implementing a fast convolver using the Fermat number transform, IEE Proc. G, Electron. Circuits and Syst. 135(6), 57 – 66. Truong, T. K., Reed, I. S., Yeh, C. S., and Shao, H. M. (1982). Parallel VLSI architecture for a digital filter of arbitrary length using the Fermat number transforms. IEEE Proc. Int. Conference on Circuits and Computers ICCC-82, Sept. 28 – Oct. 1, New York, pp. 574 – 578.


89

Truong, T. K., Yeh, C. S., Reed, I. S., and Chang, J. J. (1983). VLSI design of number theoretic transforms for a fast convolution, Proceedings of IEEE International Conference on Computer Design, VLSI in Computing, ICCD, New York, 31, pp. 200 – 203. Vanser Kraats, R. H., and Venetsanopoulos, A. N. (1982). Hardware for two-dimensional digital filtering using Fermat number transforms, IEEE Trans. on Acoustics, Speech and Signal Processing ASSP-30(2), 155 – 161, April. Vegh, E., and Leibovitz, L. M. (1976). Fast complex convolution in finite rings, IEEE Trans. on Acoustics, Speech and Signal Processing, 343 – 344, Aug. Wallace, A. M. (1988). Industrial applications of computer vision since 1982, IEE Proceedings 135(Pt. E, no. 3), 117 – 133, May. Wigley, N. M., and Jullien, G. A. (1992). VLSI implementations of number theoretic concepts with applications in signal processing, Proc. SPIE, Int. Soc. Opt. Eng. 1770, 98 – 109.

LIST OF ABBREVIATIONS AND SYMBOLS CCP DFT FFT ˛ NTT F N Nmax FNT INFNT GF[F], ZF GF[F]C , Zcm gcd 2-D MNT Ft Mp DIF yL n, m yc n, m T Ns ð Ns 2-D FFT 2-D NTT 2-D MRC 2-D CCP 2-D FNT

Cyclic convolution property Pointwise multiplication Discrete Fourier transform Fast Fourier transform Nth root of unity, kernel, basis function Number theoretic transform Modulus Transform length Maximum transform length Fermat number transform Inverse FNT Finite field or ring Complex finite field or ring Greatest common divisor Two dimensional Mersenne number transform tth Fermat number pth Mersenne number Decimation in frequency Two-dimensional linear convolution Two-dimensional circular convolution Time period Number of nonzero values in 2-D NTT domain Two-dimensional Fast Fourier transform Two-dimensional number theoretic transform Two-dimensional mixed radix conversion Two-dimensional cyclic convolution property Two-dimensional Fermat number transform

90 M 2-D MNT 2-D NMNT ˇÐ, Ð ml ˛nk 1 ˛2 0


Overall modulus, the dynamic range (M D m1 ð m2 ð m3 ð Ð Ð Ð ð mk ) Two-dimensional Mersenne number transform Two-dimensional new Mersenne number transform Transform kernel for the 2-D NMNT Transform kernel for the 2-D FNT or the 2-D MNT An operation, defined in Equation (113) An Operation, define in Equation (157) ACKNOWLEDGEMENTS

The authors are pleased to acknowledge the support of research grants from SERC and MOD as well as financial support from the Ministry of Higher Education of Algeria and the Royal Thai Navy. They also thank their colleagues in the Department of Electrical and Electronic Engineering at Newcastle University for helpful discussions and Professor D.J. Kinniment for the use of VLSI design software. Also, Dr. S. Boussakta thanks the University of Teesside for allowing him to write this chapter. The authors have pleasure in acknowledging the help in checking sections of the proof by Dr. M. Benaissa and Dr. F. Mavir.


On the Electron-Optical Properties of the ZrO/W Schottky Electron Emitter M. J. FRANSEN1,2 , TH. L. VAN ROOY1 , P. C. TIEMEIJER1 , M. H. F. OVERWIJK1 , J. S. FABER3 , AND P. KRUIT2 1 Philips

Research Laboratories, Prof. Holstlaan 4, 5656 AA Eindhoven, The Netherlands. Philips Analytical, Lelyweg 1, 7602 EA Almelo, The Netherlands. 2 Delft University of Technology, Department of Applied Physics, Delft, The Netherlands. 3 Philips Electron Optics/FEI, Eindhoven, The Netherlands.

I. Introduction . . . . . . . . . . . . . . . . . . . . . . .

92

A. General Information . . . . . . . . . . . . . . . . . . .

92

B. The Schottky Emitter . . . . . . . . . . . . . . . . . .

93

II. Electron Emission Theory . . . . . . . . . . . . . . . . . .

95

A. Qualitative Description of the Emission Process

. . . . . . . . .

96

B. Calculation of Distribution Functions . . . . . . . . . . . . .

98

C. Emission at High Temperature and Low Field . . . . . . . . . .

100

1. Thermionic Emission . . . . 2. Schottky Emission . . . . . 3. Extended-Schottky Emission; The Approximation . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . Parabolic-Barrier . . . . . . . . . . . . .

100 101

D. Emission at Low Temperature and High Field . . . . . . . . . .

105

1. Thermal-Field Emission

. . . . . . . . . . . . . . . .

E. Bridging the Gap Between Thermionic and Field Emission . . . . . . 1. 2. 3. 4.

Determination of the Parameter Current Density . . . . . Total Energy Distribution . . Brightness . . . . . . .

III. Boersch Effect for Electron Emitters

Space . . . . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . . . . . . . . . . . .

A. Application of Jansen’s Theory in the Emitter Region

103 105 107 108 109 111 114 116

. . . . . . .

116

B. Knauer’s Model for Energy Broadening . . . . . . . . . . . .

119

IV. Experimental Methods and Systems . . . . . . . . . . . . . . .

119

A. Measurement of the Energy Distribution . . . . . . . . . . . .

120

B. Measurement of Brightness . . . . . . . . . . . . . . . .

124

C. Indirect Determination of Energy Distribution and Brightness . . . . . . . . . . . . . . . . . . . . . .

126

D. Measurement of Emitter Stability . . . . . . . . . . . . . .

127

Volume 111 ISBN 0-12-014753-X


92

M. J. FRANSEN, TH. L. VAN ROOY, et al.

V. Experiments on ZrO/W Schottky Emitters . . . . . . . . . . . . . A. Energy Spread of the Schottky Emitter 1. 2. 3. 4.

. . . . . . . . . . . .

Energy Spread of a 0.9 m Schottky Emitter . . . . . Energy Spread of a 0.3 m Schottky Emitter . . . . . Intermezzo: Energy Spread of the Heated Tungsten Cathode Boersch Effect for Schottky Emitters . . . . . . . .

128 128

. . . .

129 136 141 143

B. Brightness of the Schottky Emitter . . . . . . . . . . . . . .

147

C. Emission Stability . . . . . . . . . . . . . . . . . . .

151

. . . .

. . . .

. . . .

VI. Application to Other Emitters . . . . . . . . . . . . . . . . .

156

A. Electron-optical Parameters of a 0.5 m Schottky Emitter . . . . . .

157

B. HfO/W versus ZrO/W . . . . . . . . . . . . . . . . . .

158

VII. Conclusions . . . . . . . . . . . . . . . . . . . . . . .

162

References . . . . . . . . . . . . . . . . . . . . . . .

164

I. INTRODUCTION A. General Information One of the most successful high-brightness electron source types is the ZrO/W electron emitter, also known as Schottky emitter. It consists of a tungsten emitter covered with a thin layer of zirconiumoxide. This source is employed in many commercially available electron-beam instruments, such as electron microscopes [Otten, 1994] and electron beam lithography machines [Koek et al., 1993]. For a recent review of the Schottky emitter, see Swanson and Schwind [1997]. Remarkably, the parameters characterizing the source for use in electron-beam instruments, the spread in energy of the emitted particles and the reduced brightness, still had not been measured accurately, in spite of the valuable information these quantities give to the designer of electronoptical instruments. In the present work we intend to answer some questions concerning the electron-optical properties of the Schottky emitter. Are the present values of brightness and energy spread of the ZrO/W Schottky electron emitter in the electron-optical instruments in which these sources are employed in accordance with theoretical values? Currently, the accepted values for reduced brightness and energy spread of the Schottky emitter are 2 Ð 107 A/m2 sr Ð V [Otten, 1994] and 0.8 eV [Reimer, 1993], respectively. Are these values correct? Is it possible to optimize the source? What are the optimal values of the tip diameter, field strength, and emitter temperature for a certain application? As an example, let us consider lowvoltage scanning electron microscopy. The performance of such a microscope is limited by the aberrations of the objective lens [Joy and Joy, 1996], especially by the chromatic aberration. It seems advantageous to operate the

THE ELECTRON-OPTICAL PROPERTIES OF THE ZRO/W SCHOTTKY

93

emitter under such a condition that the energy spread is minimized. In this way, the contribution of the chromatic aberration is lower, and the optimum half opening angle of the beam can be somewhat larger. Such a choice, however, might have a negative influence on the brightness and thus on the current of the probe. With a good knowledge of the parameter dependencies of energy spread and brightness on the temperature of the emitter and the field strength on its surface, it is possible to optimize the source for such a system. Is it possible to estimate the energy width at all field strengths with a simple model? For an application in which high brightness is of prime importance, for instance, in electron-beam lithography, it is advantageous to operate the source at the highest extraction field possible. Up to now, there has been no simple analytical theory available to answer this question. The interpretation of experimental results is complicated by the influence of statistical electronelectron interactions on the energy spread and brightness. Is it possible to predict the order of magnitude of these effects for a given configuration? Swanson and Schwind indicate that longitudinal electron-electron interactions (Boersch effect) can cause a considerable energy broadening of several eV at high emission current. Can we separate the intrinsic energy spread resulting from the emission process from the additional broadening caused by the Boersch effect? Is there a simple model for an estimation of the energy broadening for a given emitter configuration? What is the field on the emitter surface? It is a generally accepted practice to use Schottky plots of experimental current-voltage data for determining the field factor ˇ D F/V describing the linear relation between the extraction voltage, V, and the field on the emitter surface, F. The implicit assumption that is made with this technique is that the emitter is operating in the Schottky emission regime. It is hard to judge the correctness of this assumption from a Schottky plot alone. Is there an alternative for this procedure? In general, can we characterize the Schottky electron source parameters in the electron-beam instrument in situ? The work described in this chapter is a part of a Ph.D. study [Fransen, 1999] on the electron-optical properties of three emitter types. Aside from the ZrO/W emitter described here, this study deals with the ultrasharp h111ioriented tungsten field emitter and the use of individual carbon nanotubes as monochromatic field emitters. B. The Schottky Emitter The remarkably stable field emission current from a tungsten emitter covered with a thin layer of zirconium (Zr) was reported by several groups investigating the physical nature of adsorption on metal surfaces [Shrednik, 1961; Fursei and Shakirova, 1966; Swanson and Crouser, 1969]. The adsorption of Zr on

94


W occurs preferentially on the (100)-face of tungsten. A flat facet is grown on top of an electrochemically etched h100i-oriented tungsten wire by heating the tip while applying an electric field. The combination of this large facet and the low work function of the adsorbed layer results in a stable emission current at moderate field strength. Noise phenomena observed with cold field emitters are reduced when using large emitting areas. It was soon recognized that this unique feature of Zr on W overcomes the intrinsic instabilities of a metallic field emitter. However, the size of a facet is a dynamic equilibrium between field and surface tension, and it is known that it can take several hours to reach this equilibrium, which is a drawback. At the end of this chapter we discuss the remaining, long-term emission current instabilities inherent to this source type in more detail. In early papers, in which the properties of the Zr layer on the emitter surface were studied, it was mentioned that “a poorly outgassed (Zr) source” [Collins and Blott, 1968; Swanson and Crouser, 1969] was necessary for the formation of the desired surface layer with low work function. Subsequent studies [Danielson and Swanson, 1979] revealed that the resulting layer was a compound of Zr, O, and W, with a work function of about 2.8 eV. For longtime operation of the source, it is necessary to keep the emitter at an elevated temperature of 1800 K, so that surface diffusion of fresh Zr over the emitter surface occurs. Because of this high temperature, the properties of the ZrO/W emitter are described using thermionic emission theory, in which the lowering of the potential barrier by the high electric field is taken into account. This lowering of the effective work function is known as the Schottky effect. We review the Schottky emission theory in Section II. The widespread use of this emission model to describe the properties of the ZrO/W electron emitter has led to the name Schottky emitter. At a temperature of 1800 K, thermionic emission from other tungsten faces becomes important as well. For this reason a screening electrode is usually mounted around the shank of the ZrO/W emitter, the suppressor. In Figure 1(a) a scanning electron microscope (SEM) image is shown of a ZrO/W emitter, manufactured by FEI and used in FEI/Philips electron microscopes. The emitter tip protrudes about 0.25 mm out of the suppressor. In Figure 1(b) the apex of the emitter is shown with a higher magnification, showing the flat facet. The faceted end-form of the Schottky emitter causes some effects unique for this emitter type; e.g., emission from the edges of the facet causes a bright ring around the uniform emission pattern for increased electric fields [Tuggle, 1984]. This ring is typically situated about 7° from the optical axis. In most applications only the central spot is used, by confining the beam to ¾ 4° with an aperture in the extraction electrode. Depending on the etching conditions and facet-formation procedure, emitters with a radius of curvature at the apex ranging from 0.1 m to 2 m can be


95

FIGURE 1. Scanning electron micrographs of a ZrO/W (Schottky) emitter employed in electron microscopes. (a) The emitter tip with the suppressor. (b) The tip itself. The flat facet at the apex of the emitter can clearly be recognized.

fabricated. Emitters with a low radius of curvature require a lower extraction voltage for the same emission current density. However, in applications where a high angular current density from the source is required, the field applied to the sharp emitter has to be higher to obtain the same angular current density, because the emitting surface is smaller. The choice of the optimum emitter radius depends strongly on the type of application; e.g., Philips transmission electron microscopes (TEMs) are equipped with emitters with an apex radius of 0.9 m, whereas for SEMs sharper tips in the order of 0.5 m are used. For special applications, such as miniaturized electron columns, even sharper emitters are used [Kim et al. 1997].

II. ELECTRON EMISSION THEORY The physical process of spontaneous electron emission from a metal into a vacuum depends on three parameters: the work function of the material under consideration, the temperature of the emitter, and the electric field strength on the emitter surface. Of these three, the last two can be chosen freely once the emitter material is chosen. Unfortunately, there is no general analytical expression describing the emission process for all possible work functions, temperatures, and fields. Approximations exist for two cases: high temperature and low field and low temperature and high field. In the transition regime between these two, no analytical theory is available. We start with a qualitative description of the approximations and review the equations for the emitted current density, total energy distribution, and reduced brightness in each regime of validity in subsequent sections. The last section is devoted to the limits of validity of the different emission regimes, and we discuss our efforts to describe the energy distribution in the gap between the thermionic

96


and field emission theories analytically. Expressions for the current density and brightness are then obtained with a simple numerical integration. A. Qualitative Description of the Emission Process In this section, we give a brief description of the physical model of spontaneous electron emission from a metallic emitter into vacuum. We assume that there is no band structure in the metal and use the free-electron approximation. Consider the interface between a conductor and vacuum at zero temperature T, drawn in the right-hand-side plot of Figure 2. In the metal, electrons occupy energy levels. At T D 0, the energy levels of the metal are filled up to the Fermi level, EF , and empty above. This distribution function is drawn in the left-hand-side plot of Figure 2 with a dashed line. Electrons are kept inside the metal because a potential difference exists between the metal and the vacuum: the work function W. This value depends on the material under consideration and its crystallographic orientation. It ranges from about 2.5 eV to 6 eV for pure metals [Lide, 1991–2]. Most metals have a work function around 4.5 eV, including h111i-oriented tungsten. The work function of a metal can be lowered with a suitable surface layer, such W = 2.8 eV

T = 1800 K

Energy relative to EF (eV)

ΔW

Un

z1

z2

0.6 V/nm

EF 1.2 V/nm metal 1

0 f(E,T)

−2

vacuum

0 2 Distance from metal vacuum interface (nm)

4

FIGURE 2. Simple model of the interface between metal and vacuum. The filling of energy levels in the metal is determined by the Fermi-Dirac distribution function, drawn at the left-hand side of the interface. At 0 K, the electron energy levels are filled up to the Fermi level, EF , and empty above. The difference, W, between the vacuum potential and the Fermi level is referred to as the work function.


97

as with ZrO, yielding 2.8 eV, or with HfO, which results in a work function of 3.0 eV. For nonzero temperatures, the step function drawn in Figure 2 smoothes out. The distribution of electrons with energy E and temperature T over the energy levels in the metal is given by the well-known Fermi-Dirac function: fE, T D

1 1 C exp[E EF C V0 /kb T]

1

with kb Boltzmann’s constant. It is drawn in Figure 2 on the left-hand side with a solid line for a temperature of 1800 K. The temperature dependence of the Fermi level is neglected. The parameter V0 determines the position of the zero level of the energy scale. We will choose the zero level either at the Fermi level, as in Figure 2, at the top of the barrier or at absolute zero energy, for ease of integration. At sufficiently high temperatures, electron states above the work function are occupied, and emission of electrons into vacuum becomes possible. This process is known as thermionic emission. The emission current from a given material can be greatly increased by the application of an electric field to the emitter surface, which causes an effective decrease of the work function. The field is generated by applying a suitable voltage difference between the emitting surface and a nearby electrode, commonly denoted as the extractor. In order to limit the size of the emitting area and to further increase the local field at the emitter surface, the emitter is generally shaped into a sharp point. The strong field has two effects on the emission process. The first effect can be seen directly in the plot on the right-hand side of Figure 2, where the function Vz — i.e., the potential as a function of the distance from the interface z — is drawn for two values of the electric field (0.6 V/nm and 1.2 V/nm). Vz is determined by the work function W, the effect of the electric field F, and the image charge potential. A work function of 2.8 eV is taken. The potential barrier, and thus the effective work function, is lowered with an amount 1W. This is known as the Schottky effect, and the enhanced emission is called Schottky emission. As a second effect, the width of the barrier becomes smaller for increasing fields. If the barrier width is sufficiently small, quantum mechanical tunneling of electrons through the potential barrier occurs. For high temperatures, when there is a substantial occupation of energy levels at the top of the barrier, electron tunneling will mainly occur there. The total emitted current is a combination of thermionic electrons originating from above the top of the barrier and electrons tunneling through the barrier. This emission regime is referred to as extended-Schottky emission. For low temperatures, tunneling of electrons occurs only around the Fermi level, because higher energy levels are unoccupied. This process is known as

98


field emission. The effect of higher temperatures on the tunneling current is included in the thermal-field emission model. Analytical expressions for the emission current have been published either for a high emitter temperature together with a weak electric field or for a high field at low temperature. We discuss these expressions in the next two sections. There is no overlap between both approaches: in the intermediate region, no simple analytical theory existed until now. We show that it is nevertheless possible to describe the energy distribution of the electrons in this transition region with a simple analytical expression and thus require only one integration step for obtaining the emission current density and the brightness. B. Calculation of Distribution Functions Here and in the following two sections we review the emission theory more quantitatively using some review books and papers, such as Principles of Electron Optics by Hawkes and Kasper [1989a, b], a cold field emitter review by Swanson and Bell of [1973] and a paper by Shimoyama and Maruse [1984] considering theoretical expressions for the maximum axial brightness. In this review, all quantities are in SI units. As a starting point for the discussion, we take Equation (44.8) from Hawkes and Kasper [1989a], defining the three-dimensional current density element d3 j as 2e d3 j D pfE, TDUn d3 p 2 mh3 In this relation, p is the three-dimensional impulse vector, fE, T describes the distribution of electrons in the metal, and DUn is the transmission coefficient, the wave-mechanical coefficient describing the fraction of electrons incident on the surface that can pass the potential barrier. We decompose the impulse vector p into a component normal to the surface, pn , and a transverse component, pt . The energy associated with these components is denoted as Un and Ut , respectively. In this chapter we use the terms normal energy and tangential energy instead of the correct definition, the energy associated with the normal (or tangential) component of the impulse vector. Note that the transmission coefficient is a function only of the normal energy, Un . In most practical situations, the curvature of the emitter surface can be neglected when considering effects close to the surface; consequently, the metal-vacuum interface is taken as plane. We define a Cartesian coordinate system with the z-axis normal to the emitter surface. We consider the current in the z-direction only; consequently the three-dimensional current density element simplifies to d3 jz . In the next sections we omit the z-index and simply write d3 j. The potential V is now a function of z only, and the normal and


99

transverse components of the energy Un and Ut are conserved. The following conservation laws apply: 1 2 p C p2y D Ut D constant 2m x 1 2 p C Vz D Un D constant 2m z Ut C Un D E D constant

3

where E is the total energy. We now introduce polar or cylindrical coordinates to obtain distributions of the normal energy, dj/dUn , the tangential energy, dj/dUt , or the total energy, dj/dE. Integration of these distribution functions yields the emission current density, j. The distribution dj/dE describes the total energy spread of the emitted beam. We express the momenta in polar coordinates: px D

2mE Vz sin cos

py D

2mE Vz sin sin

pz D

2mE Vz cos

4

With these equations, we calculate the volume element in momentum space at each plane z as d3 p D p2 dp d D m 2mE Vz sin dE d d

5

After substitution of Equation (5) into Equation (2), we can calculate the total energy distribution by integration over all emission angles (0 < < 2, 0 < < /2), once a suitable expression for the transmission coefficient is found. The tangential energy distribution dj/dUt is used to find an expression for the maximum brightness. Following Shimoyama and Maruse [1984], the reduced axial brightness, Br , of the source is defined in SI units as dj e 6 Br dUt Ut D0 We express the momenta in cylindrical coordinates: px D

2mUt cos

py D

2mUt sin

pz D

2mUn Vz

7

100


The volume element in momentum space at each plane z is given as d3 p D m2 p1 z dUn dUt d

8

After substitution of Equation (8) into Equation (2), the tangential energy distribution can be calculated by integration over and Un . In order to do so, we again need an expression for the transmission coefficient, DUn . The normal energy distribution, dj/dUn , can play a role in the situation when the beam coming from a flat emitter is limited with a narrow aperture in the extractor. For pointed cathodes however, where the electrons gain most of their kinetic energy in the vicinity of the emitter, such an aperture has to have an extremely small diameter in order to observe this effect, and it is not necessary to consider it in the remainder of this chapter. A general expression for the transmission coefficient requires the solution of the Schrödinger equation to obtain the tunnel chance of electrons through the barrier as a function of energy. For the barrier shapes encountered in field emission, this is possible only with numerical techniques [El-Kareh, Wolfe and Wolfe, 1977]. Several analytical approximations are known in literature for certain regimes of field and temperature. We discussed these regimes qualitatively in the first part of this section. In the next sections the analytical approximations are described in more detail. C. Emission at High Temperature and Low Field 1. Thermionic Emission For pure thermionic electron emission, the field F D 0. Only electrons occupying energy levels above the work function W can leave the metal. The origin of the energy scale is chosen at the top of the potential barrier. Vz becomes EF C W for z < 0 Vz D 9 0 for z ½ 0 The transmission factor DUn is simply DUn D

0 1

for Un < 0 for Un ½ 0

10

So, the emission current density, d3 j, is found to be ep cos 2 fE, TDUn d3 p m h3 4me ECW ' 3 E exp sin cos dE d d h kb T

d3 j D

11


101

in which fE, T, the Fermi-Dirac distribution describing the energy distribution of electrons in the metal, is simplified by omitting the unit term in the denominator of Equation (1). This approximation is allowed, because the only energies of interest are located around the top of the potential barrier, far away from the Fermi level. The total energy distribution for thermionic emission, djT /dE, is then easily calculated. The result is 4me ECW E exp dE djT D 3 h kb T

12

Integration over all energies (0 < E < 1) yields the emitted current density: 4me W 2 kb T exp jT D h3 kb T

13

The thermionic emission equation in this form was derivedpfor the first time by Dushman [1923], and corrected the empirical equation A T expc/T as found by Richardson [1902, 1912]. A plot of ln jT /T2 versus 1/T is generally referred to as a Richardson-Dushman plot. From the slope of this plot a value for the work function can be obtained. For the calculation of the maximum axial brightness of the thermionic emitter, the tangential energy distribution, djT /dUt , is required. We write d3 j as d3 j D

epz 2 U n C Ut C W exp m2 p1 z dUn dUt d m h3 kb T

14

and integrate over Un and : djT D

4me Ut C W k T exp b 3 h kb T

dUt

15

After substitution of this result into Equation (6) the thermionic maximum axial brightness becomes Br,T D

4me2 W k T exp b 3 h kb T

D

ejT kb T

16

Note that in a practical application of such a thermionic emitter, a weak field is applied in order to prevent the buildup of a space-charge cloud that causes a saturation of the current drawn from the source.

102


2. Schottky Emission When the field is increased, its influence on the effective work function cannot be neglected anymore. In order to calculate the potential distribution Vz, it is necessary to include the potential of image charges in the model. This yields a potential function made out of three components: the work function, W, the effect of the electric field, F, and the image charge potential: Vz D

0 W eFz e2 /16ε0 z

for z < 0 for z > 0

17

with ε0 the permittivity of free space. The zero of potential energy is taken at the Fermi level, EF . In Figure 2, this function is plotted for two fields: 0.6 V/nm and 1.2 V/nm. It can be seen that the effective barrier height is lowered as the field increases. The top of the potential barrier is located at zm D

e/16ε0 F

18

and Vm D Vzm D W 1W D W The origin of the energy scale for tion is again chosen at the top of given by 0 DUn D 1

e3 F/4ε0

19

the calculation of the total energy distributhe barrier. The transmission coefficient is for E < EF C Vm for E > EF C Vm

20

The calculation of the total energy distribution, the emission current density, the tangential energy distribution, and the brightness in the Schottky regime is completely analogous to that of the thermionic emission case considered earlier. The total energy distribution is found to be 4me E C Vm djS D E exp 3 h kb T

and the emission current density is given by 4me W 1W 2 kb T exp jS D h3 kb T

dE

D exp

21

1W jT kb T

22

The enhancement of the emission current with respect to the zero-field thermionic case resulting from the lowering of the potential barrier was first described by Schottky [1914, 1923] and is known as the Schottky effect. As long as


103

p electron tunneling can be neglected, plotting lnj as a function of F results in a straight line: the Schottky plot. We show an example of such a plot in Section IV. The tangential energy distribution is given by 4me Ut C W 1W djS D kb T exp 23 dUt 3 h kb T and the reduced axial brightness, Br,S , by Br,S D

4me2 W 1W k T exp b h3 kb T

D

ejS kb T

24

3. Extended-Schottky Emission; the Parabolic-Barrier Approximation When the field at the emitter surface is further increased, electron tunneling cannot be neglected anymore. Usually, the transmission coefficient is evaluated according to the Wentzel-Kramers-Brillouin (WKB) approximation as DUn D

1 1 C expG

25

with G the Gamov exponent: GUn D

2 h¯

z2

2mVz Un dz

26

z1

The limits z1 and z2 of the integral correspond to the intersection of Un with Vz, as indicated in Figure 2. The Gamov exponent can be integrated analytically. The resulting expression, however, contains elliptic integrals, which is not convenient for subsequent integrations. We discuss this further in the next section. In the present situation, it is sufficient to use an approximation that is valid only near the top of the potential barrier. The barrier function, Equation (17), is Taylor-approximated with a second-order polynomial, as first proposed by Christov [1966]: e2 Va z ³ Vm z zm 2 27 16ε0 zm3 This parabolic function is drawn in Figure 2 with a dashed line. Using this approximation, Equation (26) can be written in closed form, yielding: GUn D

Vm Un

28

104


with

D

h¯ p 4ε0 eF3 1/4 c1 F3/4 m

29

Using this Gamov exponent, the transmission coefficient can now also be used above the potential barrier, unlike Equation (26). We discuss the error that is introduced with this approach in Section II.E. With the choice of the origin of the energy scale at the top of the potential barrier, the current density, d3 j, can be integrated over all emission angles, yielding the total energy distribution in the parabolic-barrier approximation, djPBA : djPBA

E 4me ln 1 C exp D 3 h 1 C exp[E C W 1W/kb T]

dE 30

The total energy distribution in the extended-Schottky model, djES , is found after neglecting the unit term in the denominator of the Fermi-Dirac distribution function. This simplification is necessary in order to find an expression for the emission current density in closed form. Integration of djES from E D 1 to 1, using a tabulated integral [Gröbner and Hofreiter, 1961a], yields: jES D

4me W 1W 2 k T exp b 3 h kb T

q q D jS sin q sin q

31

with q D /kb T D c1 F3/4 /kb T. For low fields q approaches zero and the current density calculated with the extended-Schottky model reduces to the Schottky model. As the field increases and/or the temperature decreases, q rises and the current density increases due to the contribution of the tunnel effect. When q approaches unity Equation (31) diverges, due to the omission of the unit term in the denominator of the Fermi-Dirac distribution function, fE, T. A consequence of the significant number of electrons that tunnels through the potential barrier is that a Schottky plot loses its value. A Schottky plot of experimental current-voltage data will yield a line that is approximately straight, with a small upward curvature. The calculation of the transverse energy distribution, dj/dUt , and thus the maximum axial brightness in the extended-Schottky regime proceeds along the same lines as outlined previously. When, as before, the unit term in the denominator of the Fermi-Dirac distribution function is neglected, an analytical expression for the reduced axial brightness in the extended-Schottky model, Br,ES D

ejES kb T

32


105

can be derived. Use has been made of a tabulated integral [Gröbner and Hofreiter, 1961b]. Again, this equation is valid only for small values of q. In practice, an upper limit for q of 0.7 is used. For a combination of temperature, work function, and field that exceeds this value, no analytical theory exists. We pay more attention to this regime in the last part of this section. In the next section, analytical expressions for the energy distribution, current density, and brightness are obtained for low temperatures and high fields. D. Emission at Low Temperature and High Field 1. Thermal-Field Emission At room temperature or below, energy levels close to the top of the potential barrier are empty. Electrons can escape from the metal only by tunneling. The tunnel probability of electrons is largest for the highest occupied energy levels, close to the Fermi level, EF . We will now integrate the Gamov exponent, Equation (26), directly, yielding 4 2mW Un 3 vy 33 GUn D 3e¯hF with y D 1W/W Un . The function vy results from the inclusion of the image charge effect in the expression for the potential barrier, Equation (17). It contains elliptic integrals of the first and second kind, Kk and Ek: vy D

1 C y[Ek yKk]

34

p with k D 1 y/1 C y. In order to find an expression for the Gamov exponent that is simple enough to enable subsequent calculations, Equation (33) is Taylor-expanded around the Fermi level, EF , up to the first derivative. A new variable, D Un EF , is used for ease of calculation. This yields the following expression: GUn D c

d

35

with e¯hF p 2ty0 2mW W 4 p 2mW3 vy0 D b cD 3e¯hF d

dD

36 37

106


y0 D 1W/W, and the function ty D vy 23 y v0 y

38

Finally, the parameter b is given by bD

2vy0 3ty0

39

Both vy and ty are slowly varying functions. In a practical situation, b ³ 0.6, vy ³ 1 y 1.69 , and the function t obeys the relation ty ³ 1 C 0.1107y 1.33 [Hawkes and Kasper, 1989a]. For tungsten, with a work function of 4.5 eV, typical field strengths are on the order of 2–7 V/nm, which causes vy to vary between 0.8 and 0.45 and ty to vary between 1.07 and 1.03. Substitution of Equation (35) into Equation (25) yields a transmission function DUn given by DUn D

1 1 C expc /d

40

Because the emitted electrons originate mainly from the vicinity of the Fermi level, the transmission coefficient is simplified to expc C /d. This result enables further analytical evaluation of the distribution functions and the total emission current density. The current density element, d3 j, is written as d3 j D

E Vz 4me 3 h 1 C expE/kb T ð exp

E cos2 C Vz sin2 bW d

ð sin cos dE d d

41

We follow Kasper’s approach [Kasper, 1982] and evaluate the integral at a fixed z-value, at V D EF . By assuming that cos2 ³ 1, i.e., small values for the angle , the integral can be further simplified. E is chosen to be zero at the Fermi level. After performing the integrations over and , the following expression for the total energy distribution is obtained: expE/d 4me bW djTF E D dE 42 d exp 3 h d 1 C expE/kb T This expression exhibits a strong similarity with the total energy distribution in the extended-Schottky model, Equation (30) with the unit term in the


107

denominator of the distribution function omitted. In this case however, the transmission coefficient itself is simplified instead of fE, T. This is necessary in order to perform the integration over E analytically. Integrating over E finally yields p 4me 2 bW jTF D 43 d exp 3 h d sin p Again, use was made of a tabulated integral [Gröbner and Hofreiter, 1961c]. The parameter p D kb T/d is a measure for the additional number of electrons that is emitted from above the Fermi level. For emitters that are operated at room temperature or below, the factor p/ sin p is usually omitted, because it is close to unity. For practical values of the field strength, the resulting error is less than 6% at room temperature. The zero-temperature equation was first derived by Fowler and Nordheim [1928], who also showed that this equation corresponded well to the experimental results reported by Millikan and Lauritsen [1928]. A plot of lnj/F2 as a function of 1/F yields a straight line if T D 0, the so-called Fowler-Nordheim plot. It can be used to extract the emitting area, A D I/j, and the field factor, ˇ D F/V, from experimental I, V data. The effect of a nonzero temperature on the emission equation was investigated by Young [1959]. Again, this equation can be used only for p-values substantially below unity. In practice an upper limit of 0.7 is used. For combinations of field, work function, and temperature that exceed this value, no analytical theory exists. In the next section, we show that it is possible to describe this region with a theory that is partly analytical. With this theory, it is easy to study the behavior of the emission current, energy distribution and brightness in the transition region. The reduced brightness in the thermal-field regime can be calculated by substituting Equations (1), (40), and (8) into (2). Again, the unit term in the denominator of the transmission coefficient is neglected. A tabulated integral is used for the integration over Un [Gröbner and Hofreiter, 1961d]. The reduced brightness in the thermal-field emission regime, Bt,TF , is given by Br,TF D

ejTF d

44

Again, the resulting expression is valid for p < 0.7. E. Bridging the Gap Between Thermionic and Field Emission In this section we consider the consequences for the emission current density, total energy distribution, and brightness for a certain choice of the electric

108


field, F, the temperature, T, and the work function, W. At first, we investigate the limiting fields and temperatures for which the emission equations in the previous sections can be used. The discussion in this section qualitatively follows Swanson and Bell’s extensive review on metallic field emission [1973], but focuses on the transition regime between the extended-Schottky model and the thermal-field emission model. Up to now, the current density, energy spread, and brightness can be calculated only numerically. In the following, it is shown that the transition regime can well be described with an analytical expression for the energy spread, which can be easily integrated numerically in order to obtain the current density. 1. Determination of the Parameter Space In this section, the boundaries between the emission regimes, presented in the previous section, are discussed. We start with the transition from Schottky emission to extended-Schottky emission. We propose to define the critical field strength, Fc1 , between the Schottky regime and the extended-Schottky regime as the field at which 10% of the total emission current density is due to tunneling. In other words, q/ sinq D 1.1, i.e., q ³ 0.24, yielding

Fc1 D

p q mkb p h¯ 4 4ε0 e

4/3

T4/3 D 1.74 Ð 104 T4/3

[V/m]

45

This boundary value is thus a function of temperature only. Similarly, the upper limit for the field in the extended-Schottky regime is reached for q D 0.7, corresponding to Fc2 D 6.84 Ð 104 T4/3 [V/m] 46 The minimum value of the field for which the thermal-field emission model is valid is reached for p D 0.7, or p p 2kb Tty0 2mW Fc3 D ³ 1.06 Ð 106 WT [V/m] 47 e¯hp with W in eV and T in K. We take the boundary field Fc4 separating the thermal-field regime from the cold-field regime as the value for which less than 10% of the total emission current is due to emission above the Fermi level, EF . This means that p ³ 0.24 and p 48 Fc4 D 3.10 Ð 106 WT [V/m] again with W in eV.

em iss

io

n

re

gi

m e

109

ot

tk

y

109

W = 2.8 eV (ZrO/W emitter) e im eg r on ssi mi e e ld im fie eg d r l Co ion iss m e ld fie l a erm Th

gi Sc h

ot

tk

y

ex

em iss

te

io

n

nd

re

ed

Transition regime

m e

-S ch

Applied field (V/m)


108 100

1000 Cathode temperature (K)

FIGURE 3. Emission regimes as a function of field and temperature for an emitter work function of 2.8 eV.

In Figure 3 the limits of validity of the different regimes are sketched as a function of the electric field and emitter temperature. The work function of the cathode was taken to be W D 2.8 eV, characteristic for the ZrO/W emitter. The temperature axis of the plot runs from 100 K to 3680 K, the latter value being the melting temperature of tungsten. As an upper value of the field axis, the field F for which W D 1W is used, in other words: when the top of the potential barrier equals Ef [Swanson and Bell, 1973]. This map of the parameter space of the ZrO/W emitter can be used to investigate the effect of a change in the field and/or the emitter temperature. Between the analytical emission theories, a transition region exists between temperature-dominant emission and emission in which electron tunneling dominates. In the next paragraphs we investigate the possibility of finding a preferably simple expression to be used in this “forbidden” region. 2. Current Density In Figure 4 the emission current density, j, is shown as a function of the field strength, F, for the ZrO/W electron emitter with a work function of 2.8 eV. The emitter temperature is taken to be 1800 K. The dotted vertical lines correspond to the critical field values Fc that were defined in the previous section.

110

M. J. FRANSEN, TH. L. VAN ROOY, et al. 1016

Fc1

1015

Emission current density (A/m 2)

1014

Schottky regime

Extended-Schottky

Fc2

Fc3

Fc4

Parabolicbarrier app.

1013 1012

Thermalfield regime

1011 W = 2.8 eV

1010 109

(ZrO/W emitter)

108 107

T = 1800 K

T=0K

106 105

Cold field emission

104 108

109 Electric field strength (V/m)

1010

FIGURE 4. Calculated current density of the Schottky emitter as a function of the electric field strength. The work function is taken as 2.8 eV.

In the plot, the current density according to the Schottky theory, Equation (22), is shown with a solid line. In the extended-Schottky regime, the emission current increases more strongly than predicted by Schottky theory, as a result of electron tunneling through the top of the potential barrier. The current density in the extended-Schottky regime, Equation (31), is drawn with a dashed line. Above the critical field strength, Fc2 , it can be seen that the function diverges. Similarly, the current density in the thermal-field emission model, given by Equation (43) and drawn in Figure 4 with a dashed-dotted line, diverges when the field is below Fc3 . At T D 1800 K and W D 2.8 eV, the boundary between thermal-field and field emission is Fc4 D 9.3 Ð 109 V/m. It can be seen that the current density in the thermal-field emission regime meets the zero-temperature field-emission current density here (drawn with long dashes). In the regime between extended-Schottky and thermal-field, no analytical expression for the current density has been found. The expressions for the current density diverge as soon as p or q exceeds 0.7. The current density should, of course, be a smooth varying function of field and temperature. Theoretical values for the current density in the transition region have only been obtained numerically [El-Uareh, Wolfe, 1977]. Let us consider the approximations carried out in the previous sections in more detail. In order to find a simple expression for the emission current


111

density, we take the total energy distribution in the extended-Schottky model, Equation (30). It was necessary to omit the unit term in the denominator of the Fermi-Dirac distribution function to obtain Equation (31). It is clear that js ! 1 with increasing field strength. We can leave the unit term intact and integrate Equation (30) numerically. In Figure 4 we plot the resulting current density jPBA with a dashed–triple dotted line. Above q ³ 0.7 the current density calculated with the extended-Schottky model diverges. The numerical integration of Equation (30), however, yields a current density that rises steadily with the field and differs by only a factor of ¾1.5 from the current density in the thermal-field regime at fields exceeding Fc3 D 3.2 Ð 109 V/m. The accuracy of the current density jPBA in the parabolic-barrier approximation was further checked with a full numerical solution of the Schrödinger equation for calculating the transmission coefficient [Fransen et al., 1998]. The current density resulting from this numerical evaluation is drawn in Figure 4 with a dotted line. In the remainder of this chapter, we refer to the numerical integration of Equation (30) as a parabolic-barrier approximation and the full numerical evaluation of the Schrödinger equation, yielding an exact transmission coefficient, as a numerical model. It is interesting to compare the outcome of the full numerical calculation with the analytical approximations. The largest difference between both calculations is in the Schottky regime, where the current density calculated with the numerical model is about twice as high as the Schottky emission current, jS . The origin of this error results from the fact that in the Schottky regime the tunnel current is taken to be zero. The numerical calculation indicates that even in the Schottky regime tunneling takes place. In the transition regime, the parabolic-barrier approximation overestimates the current density by about 30%. From Figure 4 it is clear that the integration of Equation (30) yields a result that describes the emission current density in the transition regime quite reasonably. Let us discuss the use of Equation (30) for the total energy distribution in the transition regime. 3. Total energy distribution The physical interpretation of the transition region between the extendedSchottky regime and the thermal-field regime becomes clear if one considers the behavior of the total energy distribution as a function of the applied field. In Figure 5 we plot these energy distributions in the different regimes. All distributions are scaled to the same height. The temperature of the emitter is 1800 K. Figure 5(a) shows the total energy distribution at zero field, thermionic emission, given by Equation (12). The trailing edge of the spectrum is determined by the temperature of the emitter; the sharp leading edge results from the work function, taken as 2.8 eV (ZrO/W emitter). In Figure 5(b) the Schottky effect is shown, plotted using Equation (21): the effective barrier height is lowered

112


thermionic emission

(a)

Schottky emission (b)

extended-Schottky emission

(c)

parabolic-barrier approximation (d)

thermal-field emission (e) −2

−1

0

1

2

3

4

Energy relative to Fermi level (eV) FIGURE 5. Calculated total energy distributions, plotted for different emission regimes. As the field increases, the regime changes from (a) thermionic emission. (b) to Schottky, and (c) to extended-Schottky emission. (d) The changes in shape and position in the transition region are displayed. (e) The total energy distribution in the thermal-field regime is shown for comparison. The work function is W D 2.8 eV and the emitter temperature is 1800 K.


113

as a result of the field, and the spectrum shifts downwards in energy as a function of the field. The field ranges from 2.5 Ð 108 V/m to 1.25 Ð 109 V/m. Only the first value of the field is below the critical field Fc1 separating the Schottky regime from the extended-Schottky regime. In Figure 5(c) the same energy distributions are redrawn, but now they are plotted with the extendedSchottky equation for the total energy distribution, Equation (30), without the unit term in the denominator of the Fermi-Dirac distribution function. The effect of omitting the tunneling process on the energy spectrum is clear from a comparison of Figure 5(b) with 5(c). In Figure 5(d) we plot the total energy distribution in the parabolic-barrier approximation, Equation (30), in its full form. The field ranges from 2.5 Ð 108 V/m to 4 Ð 109 V/m in steps of 2.5 Ð 108 V/m. In a small range of fields, from 1.75 Ð 109 V/m to 2.75 Ð 109 V/m, the position and shape of the total energy distribution changes drastically. In this range, the peak of the distribution shifts rapidly from the top of the potential barrier, characteristic for emission that is mainly thermionic, toward the Fermi level, EF , where field emission is dominant. At this point in the discussion, the reader may question the accuracy of the parabolic-barrier approximation in describing the strong changes of the spectrum in the transition regime. In the previous section we demonstrated that the approximation is useful for the integral of the spectrum: the current density, jPBA . Let us now compare the full width at half maximum (FWHM) of the distributions calculated with the parabolic-barrier approximation with a numerical evaluation of the transmission coefficient and the total energy distribution [Fransen et al., 1998]. In Figure 6 the dependence of the FWHM on the electric field strength is plotted. The sudden increase in width of the energy distribution in the transition regime, as also observed in Figure 5(d), is in agreement with the full numerical calculation. For larger values of the work function, this increase in width is even more pronounced. Tungsten, having a work function of 4.5 eV, shows a maximum FWHM value of 2.8 eV at T D 1800 K and F D 2.4 Ð 109 V/m. The dependence of the FWHM on the field is described by the parabolicbarrier approximation within 6% of the numerical calculation for an emitter having a work function of 2.8 eV. As a further check of the accuracy of the parabolic-barrier approximation, we compare the position of the maximum of the energy distribution, calculated with the parabolic-barrier approximation, with that found by the full numerical calculation. In Figure 7 the peak position with respect to the Fermi level, EF , is plotted as a function of the electric field. The graph indicates that the parabolic-barrier approximation can safely be used in order to evaluate the position of peak maxima as a function of the field. We use the parabolic-barrier approximation in Sections V and VI.

114

M. J. FRANSEN, TH. L. VAN ROOY, et al. 1.2

FWHM energy spread (eV)

1.0

W = 2.8 eV

0.8

0.6

0.4 Parabolic-barrier approximation 0.2 Numerical calculation 0.0 108


1010

FIGURE 6. The FWHM of the calculated total energy distribution, plotted as a function of the electric field in the parabolic-barrier approximation. For comparison, the outcome of a full numerical calculation is also shown. 3 W = 2.8 eV

Position of maximum (eV)

2

1

0 Parabolic-barrier approximation Numerical calculation −1

108


1010

FIGURE 7. Calculated position of the peak of the total energy distribution as a function of the field strength resulting from the parabolic-barrier approximation and the numerical evaluation of the transmission coefficient.

115


4. Brightness Finally, we discuss the influence of temperature, field, and work function on the brightness of an emitter. Let us consider the reduced brightness as a function of the electric field. We consider two source types. The first one is a ZrO/W emitter, which has a work function of 2.8 eV and is operated at T D 1800 K. The second source is a tungsten field emitter with a work function of 4.5 eV, operated at room temperature. In Figure 8 we show these dependencies. Unfortunately, there is no analytical expression for the brightness in the transition region between extended-Schottky and thermal-field. However, a good approximation of the brightness in the transition regime is given by ejPBA 49 Br,PBA D kb T The emission current in the transition regime jPBA is obtained by integration of Equation (30) in its full form. In Figure 8, the brightness in the transition regime is plotted with a dashed line. It forms a smooth transition between the brightness in the extended-Schottky regime and the brightness in the thermalfield regime, as expected. 1014

Reduced brightness (A/m2sr V)

1012

1010

108

W = 2.8 eV T = 1800 K

W = 4.5 eV T = 293 K

106

104

102 108


1010

FIGURE 8. Calculated reduced brightness for the ZrO/W electron emitter and the coldtungsten field emitter.

116


III. BOERSCH EFFECT FOR ELECTRON EMITTERS At low beam currents, brightness and stability are uniquely determined by the source. At increasing currents, however, the influence of the repulsive forces between individual electrons in the beam becomes so strong that brightness is decreased and energy spread is increased. These two adverse effects on beam quality are generally referred to as Coulomb interactions. An estimate of the magnitude of the statistical Coulomb interactions in a charged particle beam instrument requires the use of Monte Carlo techniques for evaluation in the whole system, a tedious job. Recently, analytical approximations to the statistical Coulomb interactions were derived, valid for a large range of beam currents [Jansen, 1990]. A summary was written by Kruit and Jansen [1997]. With the expressions described there, it is possible to quickly evaluate the magnitude of the repulsive Coulomb forces in field-free beams. In our experiments, to be described later in this chapter, we observe a broadening of the energy spread of the beam as a result of the Coulomb repulsion effect. We therefore limit ourselves to the Coulomb forces that operate along the beam axis, the Boersch effect. In this section, we describe two analytical approximations to the statistical electron-electron interactions. It is outside the scope of this chapter to discuss the theory behind these approximations in full detail; consequently, only the essential equations for the Boersch effect are given. A. Application of Jansen’s Theory in the Emitter Region In the theory of Jansen [Jansen, 1990; Kruit and Jansen, 1997], an electronoptical instrument is modeled as a series of thin lenses, with drift spaces in between. In each drift space, a crossover is present. In a first approach, we simply take the cathode-anode region as field-free and assume that the emitted particles gain all energy very close to the source. We thus modify Jansen’s model by taking only the region of the beam after the crossover into account, so we have to divide his original expressions by 2. A schematic drawing of the beam geometry is shown in Figure 9: the crossover radius is denoted by rc and I is the current in the beam. The conical beam segment has a length L and a half-opening angle ˛. The potential V of the electrons is taken constant in the whole beam segment; in other words, the region close to the apex of the tip where the electrons gain most of their kinetic energy is not modeled. In the analytical approximations to the statistical Boersch effect in a narrow crossover [Jansen, 1990], four regimes are distinguished that can dominate the energy broadening: Gaussian, Holtsmarkian, Lorentzian, and pencil beam regime. In each regime a certain type of interaction dominates. The average broadening resulting from electron-electron interactions is given by 1E. Two


117

α

rc If

L V

FIGURE 9. Simple model for the estimation of the Boersch effect in the source region. The figure equals Figure 5 from Kruit and Jansen [1997], but here only the beam segment after the crossover is taken into account.

measures for the width of a distribution are used: the smallest full width containing 50% of the current (FW50) and the full width at half maximum (FWHM) of the energy distribution. After filling in parameters specific for electrons and considering that E D eV in our case, Equations (79–82) from Kruit and Jansen [1997] divided by 2, simplify to 1EFW50G D 208.1I1/2 V1/4 1EFW50H D 4.88 1EFW50L

for

rc '

2.26 Ð 109 [m],Ł ˛2 V

50

I2/3

51

1/3

rc ˛V1/3 I D 1.90 Ð 105 1/2 ˛V

1EFW50P D 1.29 Ð 1018

52

I2 L V

53

In these equations, the broadening is given in electronvolts (eV). For the FWHM energy broadening, the prefactors are 363.3, 7.23, 1.90 Ð 105 , and 2.25 Ð 1017 , respectively. In order to obtain a general expression for the energy broadening, the contributions can be added, using the following rule [Kruit and Jansen, 1997]: 1 1 1 1 1 D C C C 4 4 4 4 1EFW50 1EFW50G 1EFW50H 1EFW50L 1EFW50P

1/4

54

Ł Note that in Kruit and Jansen [1997] the dependency of Equation (79) on the current density is incorrectly given as I1/4 . This is a typing error; the graphical material is correct.

118


The correct power of the addition was determined by Jansen [1990] by comparison of the analytical expressions with a Monte Carlo simulation. The broadening can be plotted as a function of the emission current in a doublelogarithmic plot, which shows the region of validity of each term. As an example, we take the parameters for a ZrO/W Schottky emitter with an apex radius of curvature of 0.9 m. Figure 10 is plotted for an extraction voltage of 5 kV, a half-opening angle ˛ of 7° , and a facet radius rc D 0.12 m. The beam length, L, is 2.8 mm. The dotted lines are the FWHM broadening values obtained for the individual regimes, and the solid line is an addition of these contributions using Equation (54). The dashed line is an addition of the FW50 energy widths. In most practical applications, the current coming from the facet varies between 1 and 50 A, showing that the dominant regimes for the ZrO/W electron emitter are the Holtsmark regime and the Lorentz regime. Because the beam length, L, does not appear in the equations for these regimes, its magnitude is unimportant. A few years ago, an estimation of the energy broadening in the vicinity of a point source was carried out by Elswijk, van Rooy, and Schiller [1995].

nc

il b

ea

m

10.000

ssia Gau

1.000

an arki

0.100

an

tzi

n ore

ime

n reg

Pe

FWHM and FW50 energy spread (eV)

reg

im

e

100.000

me

regi

tsm Hol e im reg

L

0.010

0.001 0.1

1.0

10.0

100.0

1000.0

Emission current (10−6 A)

FIGURE 10. The four interaction regimes and their influence on the calculated energy broadening due to the Boersch effect as a function of the emitted current. The contribution to the FWHM energy spread for each regime is drawn with dotted lines. The solid line is the overall FWHM energy broadening. The dashed line shows the total broadening using the FW50 prefactors.


119

Elswijk calculated the energy broadening using Monte Carlo techniques and compared his results with an analytical model proposed by Loeffler [1969]. According to this model, the FWHM energy broadening is given (in eV) by 1E D 4.75 Ð 104

I ˛V1/2

55

a factor of 4 lower than the energy broadening in the Lorentzian regime given by Equation (52). Apart from the pre-factor, Elswijk’s equation can be seen as a subset of the equations given by Jansen. B. Knauer’s Model for Energy Broadening A weak point of the model described in the previous section is the assumption that the kinetic energy of the electrons is constant throughout the emitter region. A simple way to include the nonconstant velocity of the electrons in a model is to assume a spherical electric field around the emitter [Knauer, 1981; Modinos and Xanthakis, 1991]. In this case, the apex radius of curvature of the emitter tip, instead of the size of the crossover, rc , becomes the size parameter in the calculation. Knauer explicitly evaluates the magnitude of the correction to a calculation in which the electron velocity is taken as constant and finds that the energy broadening is a factor of 1.2 higher with the spherical acceleration field. The final expression for the energy broadening is quite similar to the energy broadening according to Jansen in the Holtsmark regime, Equation (51), except for the different angle dependency and prefactor: 1Ekn D 7.43

I2/3 1/3

V1/3 ˛4/3 rtip

56

Again, the broadening is given in eV. The parameter rtip is the radius of curvature of the emitter apex. The difference in dependency with Jansen’s equation for the Holtsmark regime on the half-opening angle ˛ is small. It will be difficult to determine the correct dependency with an experiment. With Knauer’s equation and the expressions from the previous section, we can investigate the relevance of these theories for describing the results of experiments. IV. EXPERIMENTAL METHODS AND SYSTEMS In this section we discuss the experimental setups and methods that we used for the determination of the energy distribution, reduced brightness, and the stability of a ZrO/W emitter.

120


A. Measurement of the Energy Distribution The energy of charged particles is usually analyzed by measuring the deflection of the particles in electrostatic or magnetic fields [Young and Kuyatt, 1968; Isaacson and Gomer, 1978]. Alternatively, retarding electric fields can also be used [Simpson, 1961; Bell and Swanson, 1979]. This last option is not viable for high-intensity beams, because the retardation close to zero velocity causes additional energy broadening by electron-electron interactions (Boersch effect). Magnetic deflection is, in general, used only in applications where the voltages on electrodes for electrostatic deflection would be impractically high, such as spectrometers for electron energy-loss spectroscopy in the transmission electron microscope. Energy analysis of electrons or ions with deflecting electrostatic fields is widely used in surface analysis techniques, such as Auger electron spectroscopy (AES) and X-ray photo-electron spectroscopy (XPS). The deflecting field can either be applied between two coaxial cylinders, such as in the cylindrical mirror analyzer, or between two cylindrical or spherical electrodes. The advantage of using spherical electrodes is that in this case the instrument is double-focusing: the focusing action occurs in the dispersive as well as in the nondispersive direction [Ballu, 1980]. Another advantage of spherical capacitor analyzers is that these instruments are commercially available. We used a VSW Class 150 electron energy analyzer [VSW, 1995], modified on a few points to improve the electron-optical resolution of the instrument. The analyzer was mounted on a dedicated UHV system. During experiments, the residual gas pressure in the vacuum system was about 2 Ð 108 Pa. The dispersive power of the analyzer increases when the incoming electrons are decelerated before entering the energy-separating region. This is done using a retarding electrostatic lens system. Because of the low energy of the electrons, it is necessary to apply -metal shielding for stray fields. In Figure 11 a schematic drawing of the energy analyzer is shown. The electron current per energy interval is determined by scanning the voltages on the halfspheres and measuring the transmitted electron current using a channeltron, which is an electron multiplier capable of detecting single electrons. The scan voltages on the deflector are controlled via a personal computer (PC). While scanning the voltages on the hemispheres, pulses of the channeltron are read out using an interface board and are stored on the PC. A retractable fluorescent screen is placed in front of the entrance lens of the analyzer. Electrons can enter the analyzer through a probe hole in the screen. The hole has a diameter of 2 mm. The distance between emitter and screen is about 10 cm. A viewport above the entrance lens column enables alignment of the emitter and the apertures with respect to the analyzer with the aid of a laser beam. The laser is mounted on top of the system. It can be aligned with respect to the analyzer with shift-and-tilt manipulators.


121

laser

deflector apertures

entrance lens

probe hole apertures fluorescent screen emitter module

FIGURE 11. Schematic drawing of the energy analyzer. The emitter module is placed opposite to the entrance lens, which decelerates the electrons before they enter the deflector. A channeltron behind the exit slit of the deflector is used to count individual electrons. Apertures along the beam path are used to control the beam current and/or the resolution of the spectrometer.

Let us take a closer look at the electron-optical resolution of such an analyzer. The resolution is determined by the size of the spot at the entrance and exit sides of the hemispheres, the half-opening angle of the beam, the initial velocity of the electrons, and the mean radius of the hemispheres. This last value cannot be easily changed; it is 150 mm in our case. Sets of apertures at the entrance and exit side of the deflector determine the beam size. For secondary electron emission spectroscopy, such as AES and XPS, the electron current is low and the apertures must be relatively large. For measurement of the energy distribution of high-brightness sources, the size of the apertures can be decreased, which improves the resolution. In our experiments, we mainly used circular apertures of 1.0, 0.3, and 0.1 mm. For alignment of the source with respect to the analyzer, we also used large slits with a diameter of 4 mm ð 12 mm. The electron energy analyzer should yield reliable results for a large range of currents. Especially at higher emission currents, the beam current in the retarding lens system and in the analyzer should still be low enough to prevent a broadening of the initial energy distribution by electron-electron interactions (Boersch effect). We therefore mounted a set of probe hole apertures in front

122


of the entrance lens system. The size of the probe hole apertures ranged from 5 m to 1 mm. As a result of this, the acceptance angle of the entrance lens changed, and thus the half-angle of the beam that enters between the hemispheres changes too. The electron-optical energy resolution, 1E, of such a spherical capacitor analyzer is a function of the pass energy, Ep , the half-opening angle of the beam entering the analyzer, ˛, the mean radius, R, between the two halfspheres, and the mean width of the entrance and exit apertures of the analyzer, d [Ballu, 1980]: d ˛2 1E D Ep C 57 2R 4 The opening angle ˛ depends on the lens magnification M and the acceptance angle of the entrance lens. This last parameter is determined by the size of the probe-hole aperture in front of the lens. The magnification of the entrance lens is about unity, so ˛ D E/Ep , with E the kinetic energy of the electrons entering the analyzer. For an analyzer aperture size of 0.3 mm, a probe hole aperture size of 0.4 mm, and a pass energy of 20 eV, the electron-optical resolution, using Equation (57), is 40 meV. The stability of the control electronics is of the same order of magnitude, so by adding both contributions quadratically, the analyzer broadening will be 57 meV, provided that the beam path is shielded adequately for stray fields. We determined the resolution of the analyzer with a method described by Young and Kuyatt [1968]. In this method, the energy distribution of a metallicfield emitter is measured at a low emitter temperature. The electron-energy distribution is now close to a step function, and so the high-energy tail of the energy distribution is almost absent. A broadening of the energy distribution as a result of the finite resolution of the analyzer can best be observed at the steepest edges in the spectrum. In our experimental setup, the emitter can be cooled to 77 K. Figure 12 shows such a spectrum, obtained from a cooled h111i-oriented tungsten field emitter, operated at 657 V and an emission current of 25 nA. The experimental spectrum is fitted with Equation (42). The fit is drawn with a dashed line. The solid line is a convolution of the fit with a Gaussian function, describing the energy broadening resulting from the finite resolution of the analyzer. We determine the resolution of the spectrometer from the sharp, temperature-controlled right edge of the spectrum. The best fit with the experimental data is found when the Gaussian broadening has a full width at half-maximum of 53 meV. This broadening is thus due to the complete experimental system — i.e., the power supply of the tip, the mechanical stability of the experimental setup, the influence of stray fields, and the electronics and electron optics of the analyzer.


123

101 Vext = 680 V

Counts (arb. units)

100 10−1 10−2 10−3

10−4 10−5 −3

−2

−1

0

1

Energy relative to Fermi level (eV) 1.2

Counts (arb. units)

1.0

0.8

0.6 FWHM = 0.27 eV 0.4

0.2

0.0 −3

−2 −1 0 Energy relative to Fermi level (eV)

1

FIGURE 12. Energy distribution of the electrons emitted by a cooled h111i-oriented tungsten field emitter, used for analyzer testing. The upper plot shows the spectrum on a logarithmic scale; the lower plot has a linear y-scale. The dashed line is a fit through the spectrum using Equation (42). The solid line is a convolution with a Gaussian function that describes the additional broadening resulting from the analyzer. From the fit with the temperature edge of the experimental data, we find that the analyzer resolution is 53 meV.

124


B. Measurement of Brightness The other important parameter for application of an electron source in an electron microscope is the reduced brightness, defined as 1I 1a!0 1 !0 1a1 Vbeam

Br D lim lim

58

in which 1I is the current in a beam segment with an infinitesimally small solid angle, 1 , and an infinitesimally small area, 1a, and measured at a beam potential Vbeam . In electron microscopy, the opening angles are small and 1 D ˛2 , with ˛ the half-opening angle. The brightness is a conserved quantity in an aberration-free electron-optical system, as long as electronelectron interactions are negligible. In an ideal system, its magnitude is thus determined by the source, so it is possible to measure the brightness on an arbitrary position in the beam, for instance, in the specimen plane of the transmission electron microscope. Several methods are described in the literature for determining the brightness of an electron source. The simplest method is a measurement of the current, area, and half-opening angle of the focused probe [Fransen et al., 1999; Lauer et al., 1985]. We used this method for a determination of the reduced brightness of a ZrO/W emitter with a radius of curvature of 0.9 m. Alternative methods exist for measuring emitter brightness, such as the use of an electron biprism [Speidel and Kurz, 1977; Lauer et al., 1985] or the use of the decay in intensity of Fresnel fringes near edges in a partly opaque, partly transparent specimen [van der Mast, 1975]. This last method, however, cannot be used directly in the TEM, as discussed in more detail by Fransen [1999]. We used a Philips CM300ST FEG transmission electron microscope, operated at 300 kV beam energy. Because the brightness is most important for probe-forming applications, the microscope is operated in the nanoprobe mode, i.e., the mode in which the condenser system of the microscope is optimized to form a very small demagnified image of the source on the specimen for a given beam current. A sketch of the instrument is shown in Figure 13. To determine its diameter, the focused probe was magnified on a CCD camera with the projection system of the microscope. The current in the probe is measured on the fluorescent screen of the microscope. The current readings from the screen are calibrated with a Faraday cup prior to the experiment. An aperture in the second lens of the condenser system determines the halfopening angle ˛ of the beam at the specimen position. This opening angle can be deduced from the size of the disc in the back-focal plane of the objective lens of the microscope. In determining these three parameters, the measurement of the probe size requires most care, because contributions resulting from lens aberrations and


125

Source plane

demagnifying optics α

Condensor system Specimen plane (without specimen)

magnifying optics

Projection system

Recording plane (fluorescent screen) CCD camera

FIGURE 13. Schematic drawing of the elements in a probe-forming TEM.

electron diffraction need to be taken into account. As long as these effects do not dominate, they can be corrected for. A convenient method for separating contributions from aberrations and diffraction from the contribution of the geometrical demagnified image of the source is the root-power-sum method [Barth and Kruit, 1996]. In short, the method proceeds as follows: the total probe size, dp , is a summation of four contributions. These are the geometrical probe size, dg , the spherical aberration size, ds , the chromatic aberration size, dc , and the electron diffraction size (Airy disk), ddif . The geometrical probe size, dg , is related to the source size by the demagnification of the system. The virtual source current density has a Gaussian distribution [Hawkes and Kasper, 1989b]. An important issue in the root-power-sum method is a proper the correct width measures for the contributions to the probe size. As shown by Barth and Kruit [1996], one should use the smallest full width containing 50% of the current in the distribution (FW50) instead of the commonly used ones. The FW50 contributions from aberrations and diffraction are given by ds D 0.177Cs ˛3 ,

dc D Kc Cc

1V ˛, Vbeam

and ddif D 0.54

˛

59

The aberration coefficients, Cs and Cc , of the SuperTwin objective lens of the microscope are 1.3 mm and 1.6 mm, respectively. Because the probe size is measured on the CCD, we have to take the aberrations from the condenser part as well as the objective part of the SuperTwin lens into account. We added both contributions linearly. At Vbeam D 300 kV, the electron wavelength D 1.97 pm. According to Barth and Nykerk [1999], the factor Kc depends on the ratio between the FW50 and the FWHM and varies around 0.4–0.6. In the transmission electron microscope, the chromatic aberration of the objective lens can be safely neglected at high acceleration voltages and small ˛.

126


Without this term, the contributions add up as dp D [d4s C d4dif 1.3/4 C dg1.3 ]1/1.3

60

The correct powers in this addition were derived by Barth and Kruit [1996] by comparison with ray tracing. Once dg , ˛, and I are found, the brightness of the beam can be calculated. In a SEM, a method similar to the one described in this section can be used. In this case the probe size must be determined by scanning the focused probe over a knife edge while recording the transmitted current. The method described in this section can also be used to determine if stray fields, mechanical vibrations, and transverse electron-electron interactions (trajectory displacement) limit the performance of an electron-beam instrument by determining brightness values at a range of extraction voltages and comparing them with theory. C. Indirect Determination of Energy Distribution and Brightness Section II showed that in an emission regime where a high emitter temperature is accompanied by a strong electric field, the emission properties of the emitter can change rapidly from Schottky emission to thermal-field emission. This results in a strong rise of the energy spread in the transition regime. As a result, it is essential to know the field at the emitter surface for an accurate calculation of the emission properties. The linear relation between the extraction voltage Vext and the electric field at the surface of the emitter F is expressed as F D ˇVext , in which the factor ˇ is a complicated function of several geometrical and electrical parameters. The task of estimating the energy spread and the maximum brightness for certain operating conditions reduces simply to a determination of the field factor ˇ and the emitter temperature. There are several possibilities for obtaining a value for ˇ. First, ˇ can be found from a calculation of the potential distribution in the space between emitter, suppressor, and anode. In principle, accurate results are possible by using suitable numerical methods [Kang et al., 1985]. However, it is known that the emitter shape may change during operation, which influences the field factor. It is, at least in principle, also possible to determine the field factor from current-voltage relations, similar to the use of the slope of the Fowler-Nordheim plot in the case of cold-field emission. In many cases [Swanson and Schwind, 1997 Kim et al., 1997], Schottky emission theory is applied, and thus p a value for ˇ is found from the slope of the plot of ln I as a function of Vext ; see


127

−12

−13

−14 ln(Icol)

1907 K 1850 K

−15

1780 K 1689 K

−16

1520 K 1478 K

−17 50

60

70 V0.5

80

90

FIGURE 14. Schottky plot of a ZrO/W emitter with an apex radius of curvature of 0.9 m, obtained for six different temperatures.

Equation (22). An example of such a plot is shown in Figure 14. Linear leastsquares fits through the data points yield ˇ-values of 1.2 š 0.1 Ð 105 m1 for the five highest temperatures. Only the value at 1478 K is significantly higher (2 Ð 105 m1 ). A closer examination of Figure 14 reveals that only the plot at 1907 K shows a true straight-line behavior; at the other temperatures an upward curvature of the experimental data points can be seen. This upward curvature is an indication that the use of the Schottky equation for the current density is incorrect and that the extended-Schottky model should be used instead. Swanson and Schwind [1997] propose a method of using the small upward curvature of such a plot for estimating the magnitude of the term q/ sin q in the emitted current density in the extended-Schottky model. We feel, however, that there are better methods available for determining the field factor ˇ. In Section V Equation (30) is fitted to series of experimental energy distributions, obtained as a function of the extraction voltage. From the shape and position of the spectra, it is possible to obtain an accurate value for ˇ. D. Measurement of Emitter Stability In order to measure the emission stability of electron emitters as a function of time, we used a simple vacuum system, as sketched in Figure 15. It contains

128

M. J. FRANSEN, TH. L. VAN ROOY, et al. mass spectrometer + vacuum gauge

retractable flu-screen Ti-sublimator

emitter collector

to ion getter pump

to turbomolecular pump

FIGURE 15. The experimental system used for monitoring electron emission as a function of time.

a ZrO/W emitter, together with its module. Opposite to the emitter module is a Faraday cup. The lens electrode of the module is used to focus the beam into the Faraday cup. The emission current was measured once every 10 s. A computer was used for logging the emission data to a file. We show some results in the next section.

V. EXPERIMENTS ON ZrO/W SCHOTTKY EMITTERS A. Energy Spread of the Schottky Emitter The first authors who studied the energy spread of the ZrO/W Schottky emitter in considerable detail were Bell and Swanson [1979]. In their paper, the authors present measurements on the total energy distribution, which show a considerable broadening at increasing emission current, resulting from the Boersch effect. The broadening is so large that it is hard to separate the intrinsic energy broadening resulting from the emission process from the electron-electron interactions. Energy broadening in the retarding-field analyzer used in Bell and Swanson’s experiments is probably the most important contributor to the large energy widths published. In the present work, we discuss energy distributions obtained with a deflection-type energy analyzer. In the first section, we describe our own experiments on a ZrO/W emitter with an apex radius of curvature of 0.9 m. We use the experimental results to find an accurate estimate of the field factor ˇ for this emitter-extractor system and separate the intrinsic energy broadening from the additional broadening resulting from electron-electron interactions. In the


129

second section, we evaluate the experimental energy distributions published by Kim et al., [1997a, b] who measured energy distributions of the electrons emitted from a ZrO/W emitter with an apex radius of curvature of 0.3 m. We now use the parabolic-barrier approximation to explain these experimental results and separate again the intrinsic energy broadening from the Boersch effect. Finally, we demonstrate the usefulness of the parabolic-barrier approximation for ˇ-determination on experimental results taken from a paper by Gadzuk and Plummer [1971], who measured the energy distribution of a tungsten field emitter as a function of the field at 1570 K. 1. Energy Spread of a 0.9 m Schottky Emitter In order to measure total energy distribution, we used the electron-energy analyzer discussed in Section IV.A. The Schottky emitter was mounted in an emitter module, which is shown in Figure 16, and placed in front of the entrance lens of the analyzer. The emitter was spotwelded onto a filament ribbon, which can be heated resistively. The heating current in the experiments ranged from 1.9 to 2.3 A, corresponding to emitter temperatures between 1200 and 1900 K, as determined with an optical pyrometer. Due to the wire resistance, the potential of the emitter tip will change slightly with varying heating current. The emitter is surrounded by three electrodes: the suppressor, the extractor, and the lens electrode. These electrodes are controlled with power supplies floating on the beam potential, Vbeam , 400 V in all experiments. During the experiments the suppressor was kept at a constant voltage of 500 V with respect to the emitter. The extractor voltage varied between 3000 and 6000 V, relative to the emitter voltage. The upper value of the extraction voltage was set by the range of the high voltage supplies used. The lens electrode is meant for keeping the illumination area on the fluorescent screen constant when changing the extraction voltage. In this setup, the extraction field on the emitter can be varied while the beam potential and, thus, the energy window of the analyzer remains constant. The emission current leaving the module, i.e., the beam current, is measured by observing the voltage drop over a resistor R at the grounded side of the beam voltage power supply. As a result of this, the potential of the module changes slightly when the emission current changes. In the experimental results presented here this effect is corrected for. The power supplies shown in Figure 16 are computer-controlled and, together with the computer-controlled electron energy analyzer, it is possible to obtain energy distributions for varying temperature and extraction voltage automatically. Using this setup, we measured energy distributions of a ZrO/W electron emitter with a radius of curvature of 0.9 m at nine different temperatures ranging from 1200 K to 1900 K and seven extraction voltages ranging from 3000 V to 6000 V.

130


FIGURE 16. Photograph and schematic drawing of the emitter module, mounted on a five-axis manipulator in order to measure the energy distribution. In the schematic drawing the electrical connections to the electrodes of the emitter module are shown.


131

All 63 spectra were fit with the total energy distributions given by the extended-Schottky model, Equation (30), in which the unit term in the denominator of the Fermi-Dirac distribution function is removed, and the full numerical evaluation of the transmission coefficient, discussed in Section II, using only two free parameters for the fit. The first parameter is the factor ˇ, describing the relation between the extraction voltage and the field on the emitter surface. We assume that this factor stays constant; in other words, we do not assume a change in emitter geometry during the experiment. We checked the validity of this assumption as follows. In scanning electron micrographs taken before and after the experiment, no significant changes in emitter shape were observed, even though the tip was cycled repeatedly from 1200 K to 1900 K while varying the electric field. Note that this assumption may not always be valid. We consider this problem in some more detail when we discuss measurements on the stability of the emitter. The second fit parameter is the offset from zero energy. For each temperature setting it is necessary to take a different offset value to account for the shift in voltage induced by the voltage drop of the heating supply of the emitter. Also, the value for the beam energy, 400 eV in this case, is included in this offset. Because this offset is not important in the following discussion, its value is not worth reporting. For the extended-Schottky model the best fit was obtained for ˇ D 1.9 Ð 105 m1 . With the numerical model a somewhat higher value of 2.2 Ð 105 m1 , accompanied by a slightly different offset, was necessary for a good fit. The difference between the two models is probably caused by the fact that the WKB transmission coefficient is overestimated below the top of the potential barrier in the extended-Schottky model, as compared to the transmission coefficient found directly by numerical integration. Let us compare these results with a determination of the field factor from a Schottky plot. The current-voltage measurements plotted in Figure 14 were recorded in between the energy distribution experiments described here. We found that ˇ D 1.2 Ð 105 m1 , a considerably lower value; note the sensitive dependency of the energy distribution on the field as plotted in Figure 5. It demonstrates that the Schottky plot cannot be used directly to estimate the field factor. As an example of the quality of the fits, consider Figure 17. The temperature of 1780 K, as measured with an optical pyrometer, is close to the optimum operating temperature of the Schottky emitter (1800 K). It is clear from the graph that the fit of the simulations to the experimental data is good, especially at low extraction voltages. We can check this by plotting the experimental data and the fitted theory logarithmically. The resulting curves are shown in Figure 18. The logarithmic plots show that far below the top of the barrier, the assumption of a parabolic barrier shape causes an overestimation of the emission current with respect to the numerical calculation.

132

M. J. FRANSEN, TH. L. VAN ROOY, et al. T = 1780 K

Experimental results

βnum = 2.2 105 m−1

Numerical calculation

βext = 1.9 105 m−1

extended-Schottky model

Vext = 6000 V

Vext = 5500 V

Vext = 5000 V

Vext = 4500 V

Vext = 4000 V

Vext = 3500 V

Vext = 3000 V

391

392

393 Kinetic energy (eV)

394

395

FIGURE 17. Energy distributions for a Schottky emitter with a radius of curvature of 0.9 m. The extraction voltages range from 3 kV to 6 kV, and the emitter temperature is 1780 K. The experimental spectra are fitted with the extended-Schottky model, using ˇ D 1.9 Ð 105 m1 (dashed line), and the numerical model, with ˇ D 2.2 Ð 105 m1 (thin line).

Both plots show that, at higher extraction voltages, the experimental energy distribution is broader than the simulated spectrum. We believe that this is due to electron-electron interactions. In order to check this assumption, we show spectra taken at the highest temperature in our experiment, 1907 K (Figures 19 and 20) and at the lowest temperature of 1210 K (Figures 21 and 22).

133

THE ELECTRON-OPTICAL PROPERTIES OF THE ZRO/W SCHOTTKY 106

Intensity (arb. units)

104

102

100

10−2

10−4 391

392


394

395

FIGURE 18. Figure 17 plotted logarithmically.

At 1907 K, the broadening of the experimental spectra with respect to the calculations becomes evident at medium extraction voltages, supporting the assumption that this effect is due to e-e interactions. This broadening is discussed in more detail in Section V.A.4. In contrast to this, at the lowest temperature we do not expect additional broadening. In this case, however, the value of q increases, and the accuracy of the extended-Schottky model for electron emission should decrease. Figures 21 and 22 show energy spectra obtained at 1210 K. Especially in the logarithmic plot, it is clear that the extended-Schottky model is not able to

134

M. J. FRANSEN, TH. L. VAN ROOY, et al. T = 1907 K βnum = 2.2 105 m−1

Experimental results Numerical calculation extended-Schottky model

βext = 1.9 105 m−1

Vext = 6000 V

Vext = 5500 V

Vext = 5000 V

Vext = 4500 V

Vext = 4000 V

Vext = 3500 V

Vext = 3000 V

391

392


394

395

FIGURE 19. Energy distributions for a Schottky emitter with a radius of curvature of 0.9 m. The emitter temperature is 1907 K. Note the deviation of the experimental energy spread from the simulations for increasing extraction voltages.

follow the experimental distribution far below the top of the potential barrier. In this region however, the intensity is low. In Figure 22 the energy distribution is drawn in the parabolic-barrier approximation, i.e., Equation (30) in its full form, with a dotted line, in order to show the difference. A comparison reveals that the difference for this range of fields occurs only in a region


135

106


104

102

100

10−2

10−4 391

392

393

394

395

Kinetic energy (eV)

FIGURE 20. Figure 19 plotted logarithmically.

where the intensity is low, so we can safely use Equation (31) for calculating the emission current density for this emitter. Note that this is not allowed at higher field strengths. In Figure 23 the full width at half maximum (FWHM) of each experimental spectrum is plotted as a function of the calculated emission current density. The solid lines in Figure 23 represent FWHM values for the same conditions as in the experiment, as calculated with the total energy distribution in the extended-Schottky model djES . At low current densities, the experimental FWHM values correspond reasonably to the calculated energy spreads. For current densities exceeding jES D 108 A/m2 , the experimental spread deviates

136



βnum = 2.2 105 m−1


βext = 1.9 105 m−1

extended-Schottky model

Vext = 6000 V

Vext = 5500 V

Vext = 5000 V

Vext = 4500 V

Vext = 4000 V

Vext = 3500 V

Vext = 3000 V

391

392


394

395

FIGURE 21. Energy distributions for a Schottky emitter with a radius of curvature of 0.9 m. The emitter temperature is 1210 K.

from the calculated FWHM values considerably, again due to the Boersch effect. 2. Energy Spread of a 0.3 m Schottky Emitter Energy spectra of a Schottky emitter with a radius of curvature of 0.3 m were published by the IBM group [Kim et al., 1997a, b]. These spectra exhibit large FWHM values accompanied with a strong change in the shape of


137

106


104

102

100

10-2

10-4 391

392


394

395

FIGURE 22. Figure 21 plotted logarithmically. The dotted line is the energy distribution calculated with Equation (30) in its full form, for comparison with the distributions calculated with the extended-Schottky model.

the spectra for increasing fields. Figure 24 shows experimental energy spectra, taken from Figure 3 of the paper by Kim et al. [1997a], at an emitter temperature of 1800 K. Let us analyze these spectra by fitting the parabolicbarrier approximation and the numerical model to these experimental spectra. Again, only two fit parameters are used: the field factor ˇ and an offset to the graphs accounting for the beam potential. In this case, it was necessary to use Equation (30) in its exact form. For the parabolic-barrier approximation, the best fit was found for ˇ D 8.2 Ð 105 m1 , and for the numerical model ˇ D 9.7 Ð 105 m1 yielded the best results. By fitting each spectrum with an

138


0.80

1907 K

FWHM Energy spread (eV)

1850 K 0.70

1780 K 1689 K

0.60

1520 K 1478 K

0.50

1400 K 1300 K

0.40

0.30 4 10

1210 K

105

106 107 Emission current density (A/m2)

108

109

FIGURE 23. FWHM energy widths for the Schottky emitter with a radius of curvature of 0.9 m, plotted as a function of the calculated current density using Equation (31) and ˇ D 1.9 Ð 105 m1 . The solid lines are calculated FWHM values. It can be seen that for current densities exceeding 108 A/m2 , the width of the calculated spectra is smaller than the width of the measured ones.

individual ˇ, the fit becomes slightly better, the necessary variation in ˇ being below 10%. Note that the difference between this value for ˇ and our own result from the previous section is not due to the variation in emitter radius only, but also to the dimensions of the emitter module, especially the distance between emitter and extractor. At the lowest fields in Figure 24 the fit of both models is in good agreement with the experimental results. For increasing fields, a symmetrical broadening of the experimental energy distribution with respect to the numerical model occurs. We did not succeed in explaining this effect with a different choice for ˇ. We assume, as before, that this additional broadening is due to electronelectron interactions. The IBM results at a lower emitter temperature, taken from Figure 5 of the paper by Kim et al. [1997a], support this conclusion. Figure 25 shows these results fitted with the parabolic-barrier approximation and the numerical model. A slightly different value for ˇ was necessary to obtain the best fit. The graphs illustrate the limit of usefulness of the parabolicbarrier approximation in representing the exact shape of the energy distribution: The peak shift is accurate, but the leading edge of the spectrum is distorted, because the parabolic approximation of the barrier becomes too inaccurate. Figures 6 and 7 showed, however, that two parameters of practical importance, the position of the spectrum and the width of the distribution, are well described by the energy distribution in the parabolic-barrier approximation. This fact is used to evaluate the experimental data from Kim et al. [1997b]. In this paper, a series of energy distributions is shown in which the width

THE ELECTRON-OPTICAL PROPERTIES OF THE ZRO/W SCHOTTKY T = 1800 K


βnum = 8.9 105 m−1


βPBA = 8.2 105 m−1

Parabolic-barrier approximation

139

Vext = 2500 V

Vext = 2400 V

Vext = 2300 V

Vext = 2000 V

Vext = 1500 V

995

996

997

998

999

1000

Kinetic energy (eV)

FIGURE 24. Energy distributions of a Schottky emitter with an apex radius of curvature of 0.3 m, taken from the paper by Kim et al. [1997a]. The experimental results were fitted with the parabolic-barrier approximation and the numerical model, both described in Section II.

of the distribution decreases for increasing extraction voltage. It can readily be shown with the parabolic-barrier approximation that this is a result of the emission process. Figure 26 shows experimental full width at half-maximum values of the energy distributions taken from the paper by Kim et al. [1997b]. The FWHM values calculated from the parabolic-barrier approximation are drawn with a straight line. The only parameter used for the fit is the field factor ˇ. Again, we find a good fit for ˇ D 8.2 Ð 105 m1 . For the highest emitter temperature, the experimental energy spread is larger than expected from theory, because of the Boersch effect. At the lowest temperature of 1330 K, it can be seen that the energy spread decreases for increasing field strength, consistent with theory. A similar plot can be made of the peak position of the distributions as a function of the field. Especially in the transition regime between thermionic and field emission, the position of the spectrum is sensitive to the field strength.

140


Experimental results Numerical calculation Parabolic-barrier approximation

Vext = 2000 V βnum = 9.7 105 m−1 βPBA = 8.5 105 m−1 Vext = 1900 V βnum = 9.0 105 m−1 βPBA = 8.1 105 m−1 Vext = 1800 V βnum = 9.0 105 m−1 βPBA = 7.8 105 m−1 Vext = 1590 V βnum = 9.0 105 m−1 βPBA = 7.6 105 m−1 Vext = 1000 V βnum = 9.5 105 m−1 βPBA = 7.7 105 m−1 995

996

997 998 Kinetic energy (eV)

999

1000

FIGURE 25. Energy distributions of a Schottky emitter with an apex radius of curvature of 0.3 m at 1450 K, taken from the paper by Kim et al. [1997a]. The same fitting procedure was used as in Figure 24. For the low-emission current density associated with lower emitter temperatures, it can be seen that the Boersch effect is absent.

Figure 27 compares the peak position of the spectra from the paper by Kim et al. [1997b] with the peak maxima calculated from the parabolic-barrier approximation. As an additional fit parameter, an offset from the beam potential to a position relative to the Fermi level was again used. For each emitter temperature we had to find an individual offset value, suggesting that the heater supply is located asymmetrically, similar to the authors’ experimental setup, sketched in Figure 16. It can be seen that the experimental results follow the sudden shift in position of the spectrum in the transition regime well. The results demonstrate that it should be possible to determine the field factor ˇ with a very simple energy analyzer — for instance, a retarding-field analyzer. Although the energy broadening resulting from electron-electron interactions (Boersch effect) in such an analyzer will be considerable, the position of the maximum of the distribution is rather insensitive to the symmetrical energy-broadening effect.



1.0

0.5

0.0 108

1.5 FWHM energy spread (eV)

1.5

T = 1800 K


1.0

0.5

0.0 108


0.5

1.5

T = 1450 K

1010

T = 1700 K

1.0

0.0 108

1010



1.5

141


1010

T = 1330 K

1.0

0.5

0.0 108


1010

FIGURE 26. Full width at half maximum of the energy distributions presented in the paper by Kim et al. [1997b]. The lines are calculated with Equation (30). The only fit parameter is the field factor ˇ, taken as 8.2 Ð 105 m1 .

3. Intermezzo: Energy Spread of the Heated Tungsten Cathode Experimental determinations of the total energy distribution in the transition regime between thermionic and field emission are rare in literature. Apart from the recent paper by Kim et al. [1997b], there is one earlier paper by Gadzuk and Plummer [1971], who measured the energy distribution of a tungsten emitter as a function of the field strength at an emitter temperature of 1570 K. In the paper, the authors show that the experimental results correspond well with theoretical spectra, in which a modified Gamov factor was used to describe the tunnel probability near the top of the barrier. With this method, it is necessary to integrate the Gamov factor numerically in order to obtain an energy distribution. The analytical expression found with the assumption of a parabolic barrier, however, is also well suited to explaining the measured energy width and position of the maximum for this emitter type, as will be demonstrated. Consider the full width at half maximum values and peak positions from Figure 2(a) of Gadzuk and Plummer’s paper. We plot these values in Figures 28 and 29. We fit the data points with the calculated FWHM using Equation (30). The work function is taken as 4.5 eV, as normal for tungsten. The only fit

3

T = 1800 K

2 1 0 −1 108 3

109 1010 Electric field strength (V/m) T = 1450 K

2 1 0 −1 108

109 1010 Electric field strength (V/m)

Position relative to Fermi level (eV)





142

3

T = 1700 K

2 1 0 −1 108 3

109 1010 Electric field strength (V/m) T = 1330 K

2 1 0 −1 108

109 1010 Electric field strength (V/m)

FIGURE 27. The position of the maximum of the energy distribution as a function of temperature and field. The lines are the positions calculated from Equation (30). The experimental values are taken from the paper by Kim et al. [1997b]. Again, the field factor ˇ is taken as 8.2 Ð 105 m1 . An additional fit parameter used was the offset from the beam energy to a position relative to the Fermi level.

parameter is the field factor ˇ. The best fit is found for ˇ D 1.85 Ð 106 m1 . The plot of the FWHM shows that the energy width can rise with a factor of five in the transition regime, as confirmed experimentally. The good correspondence of the results with theory, even at the highest currents, shows that additional broadening due to Boersch effect is negligible for this emitter in the range of fields considered. For emitters with a high work function, the sudden shift in peak position in the transition regime is even more pronounced than for the ZrO/W emitter. Again, the shift of the peak of the energy distribution can be used for a determination of the field factor ˇ. From a fit of the peak positions calculated with the aid of Equation (30) to the experimental data we find that ˇ D 1.9 Ð 106 m1 . It was necessary to apply a small shift of 0.15 eV to the experimental peak positions for a good fit in the y-direction of the plot. It shows that an evaluation of the peak shift as a function of the extraction voltage can well be used to extract a value for the field factor ˇ.


143


4

2

0 108


1010

FIGURE 28. Experimental FWHM energy widths of a tungsten emitter operated at 1570 K, taken from the paper by Gadzuk and Plummer [1971] and fitted with the theoretical widths calculated from Equation (30).

4. Boersch Effect for Schottky Emitters Let us take a closer look at the broadening of the experimental energy distributions for higher j. In Figure 19 the effect is clearly visible. It is necessary to check the assumption that this broadening is due to the Boersch effect. Section III discussed two of these approximations. The first one is a modification of Jansen’s theory for Boersch effect in a beam segment with a crossover [Kruit and Jansen, 1997], in which the velocity of the electrons is taken as constant. Secondly, the theory by Knauer [1981], who takes the acceleration of electrons in the vicinity of the emitter into account, was described. To compare the measured magnitude of the Boersch effect with Jansen’s expressions, we need to find values for the beam length, L, the emission current, I, the half-opening angle of the source, ˛, and the emitting area having a radius, rc . Note that rc is not equal to the emitter apex radius of curvature rtip that is used in Knauer’s model. These parameters were elucidated in Figure 9. The beam length, L, is simply the distance between the tip apex and the beam-limiting aperture in the extractor and equals 2.8 mm. We cannot use the total emitted current as a value for I, because it is well known that a large part of the current of a ZrO/W emitter is emitted in off-axis directions [Swanson and Schwind, 1997]. Instead, we estimate the current, If , in the

144



4

3

2

1

0

−1 108


1010

FIGURE 29. Experimental positions of the maximum of energy distributions of a tungsten emitter operated at 1570 K, taken from the paper by Gadzuk and Plummer [1971] and fitted with the theoretical positions calculated from Equation (30).

central spot coming from the flat facet on the apex of the emitter. In our total energy-distribution measurements we recorded the angular current density leaving the emitter module. The facet current If can be calculated from these data using the fact that the half-opening angle ˛ of the beam coming from the facet is 7° [Tuggle, 1984]. Finally, the crossover radius, rc , is calculated from the emission area A D If /jES . For q-values exceeding 0.7, jPBA must be used. From the experimental data we find that the crossover radius is close to 0.1 m. We take the mean value of our experimental data: rc D 0.12 m. In the analytical approximations to the statistical Boersch effect in a narrow crossover, Jansen distinguishes four regimes that can dominate the energy broadening: Gaussian, Holtsmarkian, Lorentzian, and pencil beam. Section III showed that the dominant regimes for the ZrO/W electron emitter are the Holtsmarkian regime and the Lorentzian regime. For a comparison of the results obtained from the calculations with experimental values, the results of Figure 23 are replotted in Figure 30, but now as a function of the extraction voltage. The four highest temperatures are plotted. The intrinsic energy spread is drawn with a solid line. The additional broadening is shown in the lower part of the graphs. The outcome of Jansen’s equations is drawn with a dashed line, whereas Knauer’s model is drawn with

1.0

T = 1689 K

0.5

0.0 2



1.5

4 6 Extraction voltage (kV)

1.5

1.0

T = 1850 K

0.5

0.0 2


8

145

1.5

1.0

T = 1780 K

0.5

0.0 2

8





8

1.5

1.0

T = 1907 K

0.5

0.0 2


8

FIGURE 30. Experimental (individual points) and calculated FWHM energy spread of the ZrO/W Schottky emitter with a radius of curvature of 0.9 m. The intrinsic energy spread is drawn with a solid line; the lower dashed and dotted curves are the energy-broadening curves in Jansen’s and Knauer’s models, respectively. The upper dashed and dotted lines are quadratic additions of the broadening values to the intrinsic energy distribution.

a dotted line. The additional energy broadening due to the Boersch effect is added quadratically to the energy width predicted by the emission theory and is also plotted with dashed and dotted lines. For the addition of the four components of Jansen’s broadening theory, we used the addition rule given in Section III. In Figure 31 the experimental FW50 energy spreads are compared with theoretical values, with and without the Boersch effect. It is clear from Figures 30 and 31 that both theories can be used to describe the magnitude of the Boersch effect in the vicinity of the ZrO/W emitter. The difference between Jansen’s theory and the experimentally observed broadening is less than 14%. With Knauer’s theory the largest difference is 18%. For sharper emitters, we found that Knauer’s model has a larger range of validity [Fransen, 1999]. We believe that the presence of large field gradients in the vicinity of very small emitters with an apex radius of curvature below 10 nm causes a misestimation of the correct broadening regime in Jansen’s model. More theoretical work has to be done on this subject.

146


FW50 energy spread (eV)

0.6 0.4 0.2 0.0 2 0.8


0.8

T = 1689 K


0.6 0.4 0.2 0.0 2


0.4 0.2

0.8

T = 1850 K

8

T = 1780 K

0.6

0.0 2

8



0.8


8

T = 1907 K

0.6 0.4 0.2 0.0 2


8

FIGURE 31. A repeat of Figure 30, but now the FW50 values are calculated as a function of the extraction voltage. Only the results from Jansen’s model are shown.

The accuracy of the estimation of the Boersch effect for the ZrO/W emitter can be tested further by using the results presented by Kim et al. [1997a] in combination with the analysis of these results in Section V.A.2. The estimation of the Boersch effect follows the same route as indicated in the previous paragraphs, with one minor difference: instead of jES , the current density in the parabolic-barrier approximation jPBA is used, obtained by numerical integration of Equation (30). The facet current If is calculated with the assumption that the half-angle of the emission cone coming from the facet is the same as in our own experiments (7° ). With these assumptions, we arrive at a facet radius, rc , of only 15 nm. The resulting FWHM energy spreads at 1800 K are plotted in Figure 32. Again, the energy broadening obtained with Knauer’s model is drawn with a dotted line, and Jansen’s model is drawn with a dashed line. The broadening is added quadratically to the intrinsic energy broadening. Also in this case the calculated FWHM values correspond well to the experimental energy spreads. The error in the estimation is below 10%. In this case, it can be seen that the intrinsic energy spread decreases when the extraction voltage exceeds 2.6 kV. However, the Boersch effect is so strong that this will not be visible in the experiment.

147

THE ELECTRON-OPTICAL PROPERTIES OF THE ZRO/W SCHOTTKY 1.5


T = 1800 K

1.0

0.5

0.0 1.0

1.5

2.0 Extraction voltage (kV)

2.5

3.0

FIGURE 32. A repeat of Figure 30, but now the Schottky emitter has an apex radius of curvature of 0.3 m. The data were taken from the paper by Kim et al. [1997a].

B. Brightness of the Schottky Emitter In spite of the importance of the reduced brightness for electron microscopy, experimental determinations of this quantity are rare in literature. We are aware of two previous attempts in which the brightness of the ZrO/W emitter was determined experimentally. Otten [1994] reported a reduced brightness of 2 Ð 107 A/m2 sr Ð V, measured in the TEM on an emitter with an apex radius of curvature of 0.9 m. Samoto et al. [1985] measured a value of 4 Ð 107 A/m2 sr Ð V. In their paper, an emitter with an apex radius of curvature of 0.8 m was used. In both cases the brightness was determined from probe-size and probe-current measurements. For the cold-field emitter, more results are available. The highest brightness value for a cold-field emitter measured in the electron microscope was reported by Speidel and Kurz [1997]. The brightness was calculated from the decrease in modulation depth of hologram fringes, produced with a biprism, as a function of the illumination angle. The emitter was operated at 2.5 kV, yielding an emission current of 1 A. The authors found, for a tungsten h310i-oriented tip operated at room temperature, that Br D 2 Ð 108 A/m2 sr Ð V. The experiment was repeated by Lauer, Hanszen, and Ade [1985] and Hanszen, Lauer, and Ade [1985] who determined a reduced brightness of 2 Ð 106 A/m2 sr Ð V with the

148


same method and 2 Ð 107 A/m2 sr Ð V from probe-size and probe-current measurements in STEM mode. According to these authors, the difference between the methods results from stray fields. It is difficult to estimate the magnitude of such disturbances, because for calculating the theoretical brightness it is necessary to know the field on the emitter surface. In the authors’ experiment an emitter was used with the same apex radius of curvature as for the energy-distribution measurements in a similar module. For a comparison of the experimental brightness values with theory we can use the same value for ˇ, 1.9 Ð 105 m1 . Let us describe our experimental results in some more detail. The current on the spot was measured on the fluorescent screen of the microscope. For extraction voltages that ranged from 2.75 kV to 4 kV, the probe current varied between 0.07 nA and 0.3 nA. From the size of the spot in the diffraction plane of the microscope, we found a half-opening angle of the beam of 6.2 mrad. The magnification of the projection system was 1.2 Ð 106 . In Figure 33(a) we show an enlarged image of the probe. In order to improve visibility in print, a gamma correction of 3 is applied to Figure 33(a). In Figure 33(b) a contour plot of the same data is shown, without gamma correction. The contours are plotted at 10%, 20%, . . ., of the maximum intensity. Between successive images, the beam is blanked in order to protect the camera. For short exposure times, the influence of the beam blanker is visible in the low-intensity asymmetric features in the contour plot. Its influence was minimized by taking the smallest cross section through the spot, perpendicular to the low-intensity tail. The direction of such a cross section is shown in Figure 33(b) with a dashed line. In Figure 34 the cross section is plotted. 150

100

Pixel number

Pixel number

150

50

100

50 0.5 nm

0

0

50 100 Pixel number

150

0

0

50 100 Pixel number

150

FIGURE 33. A typical CCD image of the focused probe (left), together with a contour plot (right). A gamma correction of 3 was used for the CCD image. The lines drawn in the contour plot correspond to intensities relative to the maximum of 10%, 20%, and so on. The asymmetries in the spot at low intensities are an artifact due to the beam blanker, visible at short exposure times of the camera. A one-dimensional cross section (denoted with a dashed line) is drawn through the profile in such a way that the asymmetries are minimized.

149


2500 FWHM = 0.38 nm

Intensity on CCD

2000

1500

1000

500

0 0.0

0.5

1.0 1.5 Position in specimen plane (nm)

2.0

FIGURE 34. The one-dimensional cross section along the dashed line through the contour plot of the spot shown in Figure 33. The Gaussian fitted through the experimental data points demonstrates that the simplification of taking the one-dimensional cross section of the beam is justified.

The spot image shown in Figure 33 is not purely a demagnified image of the source, but it is enlarged by the influence of lens aberrations and electron diffraction. These effects are corrected for with the root-power-sum method [Barth and Kruit, 1996] described in Section IV.B. In order to use this method, the two-dimensional FW50 of the focused probe must be found. The determination of this quantity is simplified considerably if the spot profile is Gaussian, because for a two-dimensional Gaussian beam, the FW50 equals the FWHM of a one-dimensional cross section. In Figure 34 a Gaussian curve has been fitted through the one-dimensional cross section of the spot shown in Figure 33. Although this method is an approximation of the correct way of determining the two-dimensional FW50 from Figure 33, the minor differences between the measured spot profile and the Gaussian fit indicate that, in this case, the error in using the FWHM of a one-dimensional cross-section is insignificant for the determination of dp . Typical 1-D FWHM spot sizes are about 0.4 nm. The geometrical probe size, dg , can now be calculated by inverting Equation (60). The reduced brightness can now be calculated with the use of the wellknown experimental definition [Barth and Kruit, 1996], in which the total current in the probe I is divided by the FW50 probe area, 0.25d2g , the solid angle, D ˛2 , and the acceleration voltage, V.

150


Reduced brightness [A/m2 sr V]

β = 1.9.105 m−1

108

107

106 2

3

4

5

Extraction voltage (kV)

FIGURE 35. Reduced brightness as a function of the extraction voltage. Measured data points are plotted without (ž) and with (4) root-power-sum correction. The maximum axial brightness predicted by the emission theory is plotted with a solid line. A field factor ˇ D 1.9 Ð 105 m1 is assumed.

In Figure 35 we plot the reduced brightness without (ž) and with (4) the root-power-sum correction as a function of the extraction voltage. It can be seen that the uncorrected experimental results are of the same order of magnitude as those reported by Otten [1994] and Samoto et al. [1985]. To calculate the theoretical reduced brightness as a function of the extraction voltage, F D ˇV is substituted into Equation (32), using the value for ˇ that was found in Section V.A.1. The resulting brightness values are drawn with a solid line in Figure 35. The experimentally determined values of the reduced brightness, corrected for lens aberrations and electron diffraction, are close to the maximum axial brightness in the extended-Schottky model. To our knowledge, this is the first quantitative confirmation of this equation. The close correspondence strongly suggests that the influence of external disturbances, like stray fields and mechanical vibrations, was negligible in this experiment. Furthermore, the fact that the agreement between the experimental results and the emission theory is good even at the highest extraction voltages demonstrates that transverse electron-electron interactions, which would decrease the experimental brightness, are negligible in this experiment. It may well be, however, that at higher emission currents, trajectory displacement is present [Tiemeijer, 1999].


151

Finally, the experimental brightness of the ZrO/W emitter in the transmission electron microscope is comparable with the results reported for cold field emitters in similar instruments.

C. Emission Stability One of the most important advantages of the Schottky emitter is the relatively large emitting area, combined with a high operating temperature. Because of this, impurities and inhomogeneities on the surface do not have a large influence on the total emission current and are quickly diffused away and annealed. However, it is well known that the shape of the emitter under an applied field may not be stable, because of the dynamic equilibrium between the surface tension on the emitter and the electric field [Swanson and Schwind, 1997]. A change in the electric field can cause a slow drift of the size and shape of the emitting facet, which may last until a new equilibrium state is found. The high temperature of the Schottky emitter helps to speed up this process considerably. We studied this effect in a small vacuum system, sketched in Figure 15, in which a module containing a ZrO/W emitter with a radius of curvature of 0.53 m was mounted (see Figure 16) facing a Faraday cup. Once every 10 s, the current leaving the emitter module and the current entering the Faraday cup are measured, and the data are stored in a personal computer. After the first 3 days, during which small changes to the operating parameters of the tip were made, the heating current was fixed to 2.4 A and the extraction voltage to 5.2 kV. During the startup, the emission pattern was recorded a few times. Figure 36 shows these micrographs. The concentric dark rings in the first micrographs are well known [Tuggle, 1984] and are thought to be due to the build-up process of a new facet. Indeed, after a few hours of emission, the concentric rings have disappeared, as demonstrated by Figure 36(c).

(a)

(b)

(c)

FIGURE 36. Emission patterns of the ZrO/W Schottky emitter during start-up. In the first patterns, dark concentric rings are visible.

152


We recorded the emission current leaving the module and the current entering the Faraday cup. Our simple data-logging system did not allow the temporal changes of the emission current to be observed while the experiment was running. A few Schottky plots were made at irregular intervals, and some more micrographs of the emission pattern were taken. In the micrographs, an evenly illuminated spot was visible. After 14 days of continuous operation, the emitter was stopped and the current leaving the emitter module was plotted. Figure 37 shows the emitter stability as a function of time. The results are surprising: after 3 days of stable operation, the emission current gradually rises, until a maximum is reached. Within a short time, the emission current drops from 10 to 5 A, after which the process repeats itself. As time goes by, the distance between the dips increases, and the duration of the effect increases as well. Figure 38 zooms in on these instabilities. The dips in the stability show a striking similarity when observed on a smaller time scale: all feature a decrease in current followed by one short-time dip, after which the current increases again. During this increase, more short-time depressions are observed. After a period of gradual increase, the current suddenly stabilizes, and the emitter resumes its desired mode of operation. The behavior just described is known in the literature as ring collapse [Swanson and Schwind, 1997]. A schematic drawing of the effect is shown in Figure 39. It was taken from page 16 of [Tuggle, 1984]. If the field on the emitter during operation is too low, it tends to reshape itself toward a larger radius of curvature. This means that material is transported from the center of the emitter toward the sides. The scanning electron micrographs shown in Figure 40 support this: after operation, the emitter radius has increased. Furthermore, the diameter of the cylindrical part of the emitter is larger, whereas the length of this part has become shorter. If we calculate the volume in each cylindrical part (ignoring the spherical end), we find that the amount of material in both volumes is the same. This confirms that material is transported from the apex of the tip toward the shank, causing a gradual increase in emitter radius. Note that this effect is not directly visible in the emission current: the field decreases, but the emitting area gets larger. The explanation of all the small features in Figure 38 still awaits further study. The optimum field strength, Fopt , that balances the surface tension, , is given by Fopt D

4k εo r

61

in which k is a form factor. According to Barbour et al. [1960], k D 0.5 for metallic emitters. The surface tension was estimated by Swanson and Schwind [1997] to be 1.41 N/m, yielding an optimal field strength of 8 Ð 108 V/m. Let

153


Current (μA)

10 8 6 4 2 0 0

6

12

18

24

30

36 Time (h)

42

48

54

60

66

72

0 72

78

84

90

96

102

108 Time (h)

114

120

126

132

138

144

150

156

162

168

174

180 Time (h)

186

192

198

204

210

216

222

228

234

240

246

252 Time (h)

258

264

270

276

282

288

294

300

306

312

318

324 Time (h)

330

336

342

348

354

360

12

Current (μA)

10 8 6 4 2

12

Current (μA)

10 8 6 4 2 0 144

12

Current (μA)

10 8 6 4 2 0 216

12

Current (μA)

10 8 6 4 2 0 288

FIGURE 37. The beam current leaving the ZrO/W Schottky emitter module, recorded over 15 days.

154


60

Current (including offset) (μA)

40

20

0 −60

−40

−20

0 Time (minutes)

20

40

60

FIGURE 38. The dips observed in Figure 37 plotted on a smaller time scale. An arbitrary offset is added to the emission current in order to separate the individual events from each other.


155

(a) (b) (c) (d) FIGURE 39. Schematic drawing of facet formation during emitter operation.

FIGURE 40. The tip apex of the Schottky emitter used for the stability experiment before (a) and after (b) operation. It is clear from the micrographs that the emitter radius has increased. Note that this effect is not directly visible in the emission current: the field decreases, but the emitting area gets larger.

us estimate the field strength in the stability experiment described here. The value of ˇ in this experiment is around 3.0 Ð 105 m1 . The procedure for estimating ˇ is described in the next section. At 5.2 kV, this means that F D 1.6 Ð 109 V/m. It is clear from the experimental data that even at this field strength emitter dulling occurs, indicating that Swanson’s estimate is too low. More work is necessary in order to quantify the relation between Fopt and r. Once such a relation is found for one emitter, the optimum field strength for other emitter radii is simply calculated with the aid of Equation (61). At this point in the discussion the reader may question the importance of the ring-collapse effect. A wrongly chosen extraction field may cause an increase of the emitter radius, but the emitting area gets larger as well, so the

156


emission current is not strongly influenced. However, during a ring collapse the properties of the beam change considerably. Figure 39(c) shows that just before the flat facet is restored, a small protrusion is left on the terrace. It is known from high-voltage theory that at these small protrusions, the field can locally be higher with a factor of three. Section II showed that, in the extendedSchottky regime and the transition regime, the energy spread depends strongly on the applied field. The energy spread of the total beam is, of course, the sum of the energy distribution of the electrons emitted by the protrusion and the distribution of the electrons emitted by the rest of the emitter area. The Schottky effect causes a shift between these distributions, and the resulting energy spread may be very large, which is detrimental for the performance of the electron microscope. Furthermore, the dependency of the emitted current on the extraction voltage changes strongly, and this is reflected in the slope of the Schottky plot. Attempts to determine the field factor ˇ will yield erroneous results. The slopes of the Schottky plots taken just before a collapse support this. From the Schottky plot taken after 24 h of emission, a value for ˇ of 1.8 Ð 105 m1 was determined, which seems to be rather low when compared to the value of 1.9 Ð 105 m1 for a blunter emitter, but one should realize that estimations of ˇ from Schottky plots tend to be too low. The Schottky plot recorded after 174 h yields ˇ D 7.2 Ð 105 m1 , and after 247 h, ˇ D 4.1 Ð 105 m1 . Both results were obtained just before a collapse occurred. Once again, this experiment shows that firm conclusions about the emitter properties cannot be drawn from a single Schottky plot. The ring-collapse effect can also influence experiments for determining the field factor ˇ from series of energy distributions. In fact, we also measured the energy distribution of the electrons emitted from a ZrO/W emitter with an apex radius of curvature of 0.5 m, as a function of temperature and field. Unlike the experiments discussed in Section V.A.1, not all spectra could be fitted with a single value for the field factor ˇ. Large energy widths, which could not be reproduced were sometimes measured. It is likely that the emitter shape changed during the experiment, in the same way as described before. It is clear that for reliable operation the field strength should be chosen in such a way that ring collapses do not occur. If chosen properly, periodic depressions in the emitted current and the large increase in energy spread accompanying a ring collapse are prevented.

VI. APPLICATION TO OTHER EMITTERS The previous section demonstrated the usefulness of the expressions for the energy distribution, brightness, and Boersch effect. These expressions can also


157

be used on other emitter shapes and types. We deal with two examples. The energy spread of a Schottky emitter with a smaller radius of curvature than the one previously measured, under similar operating conditions, is estimated. Then, the consequences of 0.9 m tip changing the ZrO coating to another one, for instance HfO, are described. A. Electron-optical Parameters of a 0.5 m Schottky Emitter The previous sections showed that the field on the emitter surface — and thus the intrinsic energy spread and brightness — can be determined accurately from experimentally determined energy distributions of Schottky emitters. Here, we use these results to see whether we can predict the effect of changes in the emitter parameters, such as the emitter radius. Let us consider the energy distribution of a Schottky emitter with a radius of curvature of 0.5 m. We assume that it is mounted in the same module as used for the 0.9 m emitter, so that, apart from its radius, all geometrical dimensions are the same. An empirical model for the influence of geometry changes on the field factor is given by Gomer [1961]. p In this model, originally derived by Charbonnier, ˇ D Krtip 1 and k D C 3 ˛x/rtip 0.13 , with x the emitter-anode distance and ˛ the half-angle of the cone-shaped emitter. Charbonnier found that the constant C equals 0.59 when ˛ is expressed in degrees. For the Schottky emitter with its additional suppressor electrode, the value of this constant is not correct, but the parameter dependency can be used to derive ˇ0.5 D r0.9 /r0.5 0.87 ˇ0.9 , yielding ˇ0.5 D 3.2 Ð 105 m1 . A more elaborate model, specifically designed for the Schottky emitter, is described by Swanson and Schwind [1997]. In this model, the dependency of ˇ on r is given as ˇ D Kr 0.76 , in which the constant K is a complicated function of the distances between suppressor, tip and anode, the ratio between anode voltage and suppressor voltage and the ratio between facet size and emitter radius. Using this equation, we find that ˇ0.5 D 3.0 Ð 105 m1 . With such an estimate for ˇ, the magnitude of the Boersch effect can be calculated with the aid of Jansen’s and Knauer’s equations. Again, the halfopening angle of the beam is assumed to be 7° , and the crossover radius, rc , is 59 smaller than found for the ZrO/W emitter with a radius of curvature of 0.9 m. Figure 41 shows the expected full width at half maximum energy spread as a function of a typical range of extraction voltages. Recent experiments by one of the authors [Tiemeijer, 1998] are in good agreement with these results. This value for the field factor can be used to estimate the reduced brightness of a ZrO/W emitter with an apex radius of curvature of 0.5 m. According to Figure 8, this may imply that a brightness of 6 Ð 109 A/m2 sr V is achievable for such an emitter, operated at 1800 K and 5 kV. Such a high value may, however, not be measured in practice, as a result of trajectory displacement

158

M. J. FRANSEN, TH. L. VAN ROOY, et al. 1.5


T = 1800 K

1.0

0.5

0.0 2

4

6

8


FIGURE 41. Full width at half maximum energy spread as a function of the extraction voltage, plotted for the ZrO/W emitter with an apex radius of curvature of 0.5 m. The solid line is the intrinsic energy spread, and the two dashed and dotted lines are the expected energy broadening values according to Jansen’s and Knauer’s theory, respectively.

at high emission-current densities. Recent calculations by one of the present authors [Tiemeijer, 1999], however, indicate that the trajectory displacement effect in the emitter region is small, even at high emission current densities. It is clear that more experimental work has to be done in order to verify the correctness of this value. B. HfO/W versus ZrO/W As another example, we consider the influence of a change of the work function of the emitter. A disadvantage of the ZrO/W electron emitter is the large thermionic emission current from the shank of the tip. These electrons, originating far off-axis, may be scattered elastically or inelastically inside the emitter module, and part of these may enter the electron-optical column with an energy lower than that of the primary beam. This effect has important consequences for the quality of the beam. Near the specimen, a weak intensity pattern with a fourfold symmetry can be observed around the central spot. As a solution for this, HfO instead of ZrO is proposed as a work function–lowering coating. As a result of the higher work function, the thermionic emission current will


159

be less intense. The work function of HfO is 3.0 eV, 0.2 eV higher than the ZrO/W emitter. In this section we study the consequences of the increase in work function on parameters such as current density, total energy distribution, Boersch effect, and brightness. For comparison, we use the field range in which the ZrO/W emitter is usually operated: 5 Ð 108 V/m for the 0.9 m radius of curvature TEM emitter at 3 kV and ¾ 2 Ð 109 V/m for the 0.3 m emitter at the highest-reported extraction voltage. For both emitters, a temperature of 1800 K is assumed. The parabolic-barrier approximation, Equation (30), is used for the calculation of current density and total energy distribution. We compare the two emitters for two situations: equal current density, i.e., a higher field for the HfO/W emitter, and equal field. Let us consider these two possibilities, in more detail, starting with the current density. Figure 42 plots the fraction between the ZrO/W and HfO/W current density. For thermionic and Schottky emission, this fraction is a constant, given by exp[WHfO/W WZrO/W /kB T] D 3.63. Thus, it can be concluded that the current emitted from off-axis positions is lowered by the same factor. At the apex of the tip, however, the field is higher, and a part of the current is due to tunneling through the top of the potential barrier. This tunnel process, however, is almost the same for both emitters, so the fraction of the loss in current is not as large when the field is kept constant, as can be seen in Figure 42.

jZrO/W/jHfO/W

4

2

0 108


1010

FIGURE 42. Fraction of the emission current density of the ZrO/W emitter and the HfO/W emitter, as a function of the electric field strength. In the Schottky regime, the fraction is a constant. As the importance of tunneling increases, the fraction becomes smaller.

160


We have determined the increase in field strength necessary for maintaining the same current density when changing the ZrO coating to HfO graphically from a plot of the current density as a function of the field strength. It turns out that the correction can be described simply as FHfO/W D FZrO/W C 2.8 Ð 108

62

with F in V/m. As an example, let us consider the 0.9 m radius of curvature ZrO/W emitter that was discussed in Section V.A.1. A typical extraction voltage for this emitter type is 4.5 kV and the corresponding field strength is 8.6 Ð 108 V/m, so the current would be lower by a factor of 3.63. In order to obtain the same current, the extraction voltage has to be raised to 5.97 kV. If the sharp ZrO/W emitter with its apex radius of curvature of 0.3 m, discussed in Section V.A.2, is replaced by an HfO/W emitter, the current would be lower by a factor of 3.34, for an extraction voltage of 2500 V and ˇ D 8.2 Ð 105 m1 . An increase in voltage of only 341 V is necessary to achieve the same current density. The shape of the total energy distribution of the HfO/W emitter is equal to that of the ZrO/W emitter in the Schottky and extended-Schottky regimes, as can be observed in Figure 43. In the transition regime between thermionic and


1.5

1.0

0.5

ZrO/W emitter HfO/W emitter 0.0 108


1010

FIGURE 43. FWHM energy distribution as a function of the electric field strength for the ZrO/W and HfO/W electron emitter.


161

field emission, the energy spread will be somewhat larger, due to the 0.2 eV larger distance between the Fermi level, EF , and the top of the potential barrier. If a ZrO/W emitter is changed for an HfO/W emitter and the emission current density should be the same, the energy spread of the latter is larger, at least at fields below 2 Ð 109 V/m. For the ZrO/W 0.9 m emitter at 4.5 kV, the FWHM energy spread is 0.49 eV. For the same current density, at 5.97 kV, the HfO/W energy spread is 0.57 eV. For an emitter with a lower radius of curvature, the field is higher, and the effect of a change in emitter material is more drastic. Consider a ZrO/W emitter with an apex radius of curvature of 0.3 m. At V D 2.5 kV, the intrinsic energy spread of the ZrO/W emitter is 1.14 eV. For the HfO/W emitter, this value increases to 1.25 eV. At 2.84 kV, the intrinsic energy spread of the HfO/W emitter is about 1.33 eV. For fields exceeding 2 Ð 109 V/m, the energy spread will decrease if the coating is changed from ZrO to HfO and the emission current is kept the same. Such a situation, however, will probably not occur because of the Boersch effect. To estimate the change in magnitude of the Boersch effect when the work function is changed, consider two emitter types, the 0.9 m and the 0.3 m Schottky emitter. Figure 44 shows a plot of the intrinsic energy width of the 0.9 m ZrO/W emitter and the HfO/W emitter with a solid line. Because the field strength is low for this emitter type, the intrinsic spread of both emitters is equal, and the Boersch effect is not so strong, especially for the HfO/W emitter, because the current density is low. The magnitude of the Boersch effect is calculated with Knauer’s theory and Jansen’s model for the energy broadening. The results produced by these theories are plotted with dotted and dashed lines, respectively. The resulting energy spread is obtained by adding the intrinsic energy width and the broadening quadratically. The parameters mentioned in Section V.A.4 are used: ˇ D 1.9 Ð 105 m1 , ˛ D 7° , rc D 0.12 m, and L D 2.8 mm. In this way, the only difference between the emitters is the magnitude of the facet current If . We make a worst-case estimation of the energy broadening and use Knauer’s theory. For an extraction voltage of 4.5 kV, the energy spread of the ZrO/W emitter is 0.53 eV. For the HfO/W emitter 0.49 eV is measured, due to the lower contribution by the Boersch effect. If the HfO/W emitter is operated at 5.97 kV in order to compensate for the loss in current, the energy width is 0.61 eV. For the sharp Schottky emitter with an apex radius of curvature of 0.3 m, the plot is somewhat different, as shown in Figure 45. Again, we use the parameters from Section V.A.4: ˇ D 8.2 Ð 105 m1 and rc D 15 nm. The intrinsic energy width of the ZrO/W emitter is represented by the lower solid line; the upper solid line is the intrinsic energy spread of the HfO/W emitter. The lower set of dashed and dotted lines corresponds to the sharp HfO/W emitter, calculated with Jansen’s and Knauer’s broadening model, respectively. For an extraction voltage of 2.5 kV we find with Knauer’s broadening model

162



1.5

1.0 ZrO/W HfO/W

0.5

0.0 2

4

6

8


FIGURE 44. Energy widths for the 0.9 m ZrO/W and HfO/W emitter. The intrinsic energy width is equal and drawn with a solid line. The dotted lines are calculated with Knauer’s model for energy broadening, and the dashed lines are done with Jansen’s theory.

a width of 1.38 eV for the ZrO/W emitter and 1.30 eV for the HfO/W emitter. An increase of the voltage in order to obtain the same current density with the HfO/W emitter yields a distribution having a width of 1.49 eV. Due to the lower current density of the hafniated emitter, its brightness will also be lower. The brightness is linearly dependent on the emission current density, so Figure 42 can be used directly for an estimation of change in brightness when a HfO/W emitter is employed instead of a ZrO/the emitter.

VII. CONCLUSIONS Let us return to the Introduction and see whether we have answered the questions that were posed there. We have shown that the generally accepted values for energy spread and brightness need some correction. It is possible to operate a ZrO/W electron emitter in such a way that one can achieve a brightness of 1.2 Ð 108 A/m2 sr Ð V combined with a full width at half maximum energy spread of 0.48 eV. An increase of the electric-field strength will result in a higher brightness, accompanied by an increase in energy spread. The calculations indicate, for

163

THE ELECTRON-OPTICAL PROPERTIES OF THE ZRO/W SCHOTTKY 2.0 ZrO/W


1.5

1.0

HfO/W

0.5

0.0 1.0

1.5

2.0 Extraction voltage (kV)

2.5

3.0

FIGURE 45. A repeat of Figure 44, but ZrO/W and HfO/W emitters with an apex radius of curvature of 0.3 m are now used.

instance, that it may be possible to operate a ZrO/W emitter with an apex radius of curvature of 0.5 m at a brightness of 6 Ð 109 A/m2 sr Ð V and an energy width of 1.2 eV. Currently, no experimental data are available on the transverse electron-electron interactions (trajectory displacement) in the emitter region, so it is not possible to say if such a high brightness value can be reached in practice. Recent calculations by one of the authors indicate, however, that the trajectory displacement effect in the emitter region is small. More experimental work is necessary in order to clarify this subject. The broadening of the energy distribution at high electric-field strengths is partly caused by the emission process itself and partly by statistical electronelectron interactions in the beam, the Boersch effect. These two effects can well be separated using simple analytical models. A comparison of these models with experimental results obtained in the authors’ experimental setup as well as results taken from literature allow confidence in using these models for optimizing Schottky electron emitters for certain applications. In order to optimize a ZrO/W emitter it is necessary to determine the field on the emitter surface accurately. It is possible to determine the field factor ˇ from an analysis of the peak position of the energy distribution as a function of the extraction voltage, measured with a simple energy analyzer. The authors have a strong preference for this method over the use of Schottky plots.

164


REFERENCES Ballu, Y. (1980). High resolution electron spectroscopy, supplement 13B of Advances in Electronics and Electron Physics. New York: Academic Press, pp. 257 – 381. Barbour, J. P., Charbonnier, F. M., Dolan, W. W., Dyke, W. P., Martin, E. E., and Trolan, J. K. (1960). Determination of the surface tension and surface migration constants for tungsten, Phys. Rev. 117(6), 1452 – 1459. Barth, J. E., and Nykerk, M.D. Dependence of the chromatic aberration spot size on the form of the energy distribution of the charged particles. (1999). Submitted for publication in the Proceedings of CPO5. Barth, J. E., and Kruit, P. (1996). Addition of different contributions to the charged particle probe size, Optik 101(3), 101 – 109. Bell, A. E., and Swanson, L. W. (1979). Total energy distributions of field-emitted electrons at high current density, Phys. Rev. B 19(7), 3353 – 3364. Christov, S. G. (1966). General theory of electron emission from metals, Phys. Stat. Sol. 17 (11), 11 – 26. Collins, R. A., and Blott, B. H. (1968). The adsorption and nucleation of zirconium on tungsten field emitters, Surface Science 10, 349 – 368. Danielson, L. R., and Swanson, L. W. (1979). High temperature coadsorption study of zirconium and oxygen on the W(100) crystal face. Surface Science 88, 14 – 30. Dushman, S. (1923). Electron emission from metals as a function of temperature, Phys. Rev. 21, 623 – 636. El-Kareh, A. B., Wolfe, J. C., and Wolfe, J. E. (1977). Contribution to the general analysis of field emission, J. Appl. Phys. 48(11), 4749 – 4753. Elswijk, H. B., van Rooy, Th. L., and Schiller, C. (1995). Energy broadening by Coulomb interactions of an electron beam emitted from a point source. J. Vac. Sci. Technol. B 13(3), 1037 – 1043. Fowler, R. H., and Nordheim, L. (1928). Electron emission in intense electric fields. Proc. Roy. Soc. London A 119, 173 – 181. Fransen, M. J. (1999). Towards high-brightness, monochromatic electron sources, Ph.D. diss. Delft University of Technology. Published by M. J. Fransen, ISBN 90-9012398-9. Fransen, M. J., Faber, J. S., van Rooy, Th. L., Tiemeijer, P. C and Kruit, P. (1998). Experimental evaluation of the extended-Schottky model for ZrO/W electron emission. J. Vac. Sci. Technol. B 16(4), 2063 – 2072. Fransen, M. J., Overwijk, M. H. F., and Kruit, P. (1999). Brightness measurements of a ZrO/W Schottky electron emitter in a Transmission Electron Microscope. Presented at IVESC 1998, accepted for publication in Applied Surface Science. Fursei, G. N., and Shakirova, S. A. (1966). Localization of field emission in small solid angles, Soviet Physics — Technical Physics 11(6), 827 – 832. Gadzuk, J. W., and Plummer, E. W. (1971). Energy distributions for thermal field emission. Phys. Rev. B 3(7), 2125 – 2129. Gomer, R. (1961). Field emission and field ionization. Cambridge, Mass.: Harvard University Press. Reprinted by the American Institute of Physics (2nd ed. 1993). Gröbner, W., and Hofreiter, N. (1961a). Integraltafel. II Teil: Bestimmte Integrale, 3rd ed. Wien: Springer-Verlag, p. 83, Equation 4a. Gröbner, W., and Hofreiter, N. (1961a). Integraltafel. II Teil: Bestimmte Integrale, 3rd ed., Wien: Springer-Verlag, p. 177, Equation 13d. Gröbner, W., and Hofreiter, N. (1961c). Integraltafel. II Teil: Bestimmte Integrale, 3rd ed. Wien: Springer-Verlag, p. 53, Equation 9.


165

Gröbner, W., and Hofreiter, N. (1961d). Integraltafel. II Teil: Bestimmte Integrale, 3rd ed. Wien: Springer-Verlag, p. 177, Equation 13e. Hanszen, K.-J., Lauer, R., and Ade, G. (1985). Possibilities of brightness measurements in nonrotationally symmetrical electron beams by measuring the current density and determining the decrease of contrast in electron interferograms or micrographs, Optik 71(2), 64 – 72. Hawkes, P. W., and Kasper, E. (1989a). Principles of electron optics Vol. 2: Applied geometrical optics. Chapter 44: Theory of electron emission. New York: Academic Press. Hawkes, P. W., and Kasper, E. (1989b). Principles of electron optics Vol. 2: Applied geometrical optics. Chapter 47: Brightness. New York: Academic Press. Isaacson, M., and Gomer, R. (1978). Extended range field emission spectroscopy. Appl. Phys. 15, 253 – 256. Jansen, G. H. (1990). Coulomb interactions in particle beams, suppl. 21 in Advances in electronics and electron physics. New York: Academic Press. Joy, D. C., and Joy, C. S. (1996). Low voltage scanning electron microscopy, Micron 27(3 – 4), 247 – 263, part VI. Kang, N. K., Tuggle, D. W., and Swanson, L. W. (1983). A numerical analysis of the electric field and trajectories with and without the effect of space charge for a field electron source. Optik 33(4), 313 – 331. Kasper, E. (1982). Field electron emission systems, volume 8 of Advances in Optical and Electron Microscopy. New York: Academic Press, pp. 207 – 260. Kim, H. S., Yu, M. L., Thomson, M. G. R., Kratschmer, E., and Chang, T. H. P. (1997a). Energy distributions of Zr/O/W Schottky electron emission, J. Appl. Phys. 81(1), 461 – 465. Kim, H. S., Yu, M. L., Thomson, M. G. R., Kratschmer, E., and Chang, T. H. P. (1997b). Performance of Zr/O/W Schottky emitters at reduced temperatures, J. Vac. Sci. Technol. B 15(6), 2284 – 2288. Knauer, W. (1981). Energy broadening in field emitted electron and ion beams, Optik 59(4), 335 – 354. Koek, B., Chisholm, Th., Davey, J. P., Romijn, J., and van Run, A.J. (1993). A Schottky-emitter electron source for wide range lithography applications. Jap. J. Appl. Phys. 32(Part 1, no. 12B), 5982 – 5987. Kruit, P., and Jansen, G. H. (1997). Handbook of charged particle optics. Chapter 7: Space charge and statistical Coulomb effects. New York: CRC Press. Lauer, R., Hanszen, K.-J., and Ade, G. (1985). Analyse des interferometrischen Verfahrens zur Messung des Richtstrahlwerts von Elektronenkanonen, number APh-24 in PTB-Bericht. Physikalisch-Technischen Bundesanstalt. Lide, D. R. (1991 – 2). CRC Handbook of Chemistry and Physics, 72nd ed. New York: CRC Press. Loeffler, K. H. (1969). Energy-spread generation in electron-optical instruments, Z. Angew. Phys. 27, 145 – 149. Millikan, R. A., and Lauritsen, C. C. (1928). Relations of field-currents to thermionic-currents. Proc. Nat. Acad. Sci. of the USA 14, 45 – 49. Modinos, A., and Xanthakis, J. P. (1991). Energy-broadening of field-emitted electrons due to Coulomb collisions, Surface Science 249, 373 – 378. Otten, M. T. (1994). Performance of the CM200 FEG, in Proceedings ICEM 13-Paris, 235 – 236. Reimer, L. (1993). Image Formation in Low-Voltage Scanning Electron Microscopy. SPIE Optical Engineering Press, Bellingham, Washington. Richardson, O. W. (1902). Negative radiation from hot platinum, Proc. Cambridge Philos. Soc. 11, 286 – 295. Richardson, O. W. (1912). The electron theory of contact electromotive force and thermoelectricity. Philos. Mag. XXIII, 263 – 278.

166


Samoto, N., Shimizu, R., Hashimoto, H., Tamura, N., Gamo, K., and Namba, S. (1985). A stable high-brightness electron gun with Zr/W tip for nanometer lithography. I. Emission properties in Schottky- and thermal field-emission regions, Jap. J. Appl. Phys. 24(6), 766 – 771. ¨ Schottky, W. (1914). Uber den Einfluss von Strukturwirkungen, besonders der Thomsonschen Bildkraft, auf die Elektronenemission der Metalle, Physik. Zeitschr. XV, 872 – 878. ¨ Schottky, W. (1923). Uber kalte und warme Elektronenentladungen, Zeit. Phys. 14, 63 – 106. Shimoyama, H., and Maruse, S. (1984). Theoretical considerations on electron optical brightness for thermionic, field and T-F emissions, Ultramicroscopy 15, 239 – 254. Shrednik, V. N. (1961). Investigation of atomic layers of zirconium on the faces of a tungsten crystal by means of electron and ion projectors. Soviet Physics — Solid State 3(6), 1268 – 1279. Simpson, J. A. Design of retarding field energy analyzers, Rev. Sci. Instrum. 32(12), 1283 – 1293. Speidel, R., and Kurz, D. (1977). Richtstrahlwertmessungen an einem Strahlerzeugungssystem mit Feldemissionskathode, Optik 49(2), 173 – 185. Swanson, L. W., and Bell, A. E. (1973). Recent advances in field electron microscopy of metals, volume 32 of Advances in Electronics and Electron Physics, Appendix I. New York: Academic Press, 296 – 304. Swanson, L. W., and Crouser, L. C. (1969). Angular confinement of field electron and ion emission, J. Appl. Phys. 40(12), 4741 – 4749. Swanson, L. W., and Schwind, G. A. (1997). Handbook of Charged Particle Optics. Chapter 2: A review of the ZrO/W Schottky cathode. New York: CRC Press. Tiemeijer, P. C. Unpublished experimental results. Tiemeijer, P. C. (Awaiting publication). Measurement of Coulomb interactions in an electron beam monochromator. Presented at TARA 1998, submitted to Ultramicroscopy. Tuggle, D. W. Emission characteristics and electron optical properties of the ZrO/W point cathode, Ph.D. diss., Oregon Graduate Center, 1984. van der Mast, K. D. A laser heated Schottky emission gun for electron microscopy, Ph.D. diss., Delft University of Technology, 1975. VSW (Vacuum Science Workshop). (1995). Ltd. Lyme Green Business Park, Macclesfield, Cheshire, U.K. Young, R. D. (1959). Theoretical total-energy distribution of field-emitted electrons, Phys. Rev. 113(1), 110 – 114. Young, R. D., and Kuyatt, C. E. (1968). Resolution determination in field emission energy analyzers, Rev. Sci. Instrum. 39(10), 1477 – 1480.


The Size of Objects in Natural and Artificial Images LUIS ALVAREZ1 , YANN GOUSSEAU2 , AND JEAN-MICHEL MOREL3 1 Departamento

de Informatica y Sistemas, Universidad de Las Palmas, Campus de Tafira, 35017 Las Palmas, Spain. [email protected] 2 Centre de Math´ ematiques et leurs Applications, ENS Cachan, 61 av. du Président Wilson, 94235 Cachan Cedex, France. [email protected] 3 Centre de Math´ ematiques et leurs Applications, ENS Cachan, 61 av. du Président Wilson, 94235 Cachan Cedex, France. [email protected]

I. Introduction . . . . . . . . . . . . . . . . . . . . . .

168

II. Statistics of Natural Images: A Review . . . . . . . . . . . . .

170

A. Motivations . . . . . . . . . . . . . . . . . . . . .

170

B. First-Order Statistics . . . . . . . . . . . . . . . . . .

171

C. Second-Order Statistics . . . . . . . . . . . . . . . . .

172

1. Covariance and Power Spectrum . . . . . . . . . . . . . 2. Other Second-Order Statistics . . . . . . . . . . . . . .

172 173

D. Linear Decomposition of Images . . . . . . . . . . . . . .

174

E. Scale Invariance in Natural Images

. . . . . . . . . . . . .

175

. . . . . . . . . . . . . .

175

. . . . . . . . . . . . . . . .

176

. . . . . . . . . . . . . . . . . .

177

III. Sizes of Sections in Natural Images A. The Distribution of Areas B. Digital Photographs

C. Other Types of Images

. . . . . . . . . . . . . . . . .

179

D. The Distribution of Boundary Lengths . . . . . . . . . . . .

184

E. The Distribution of Intercept Lengths . . . . . . . . . . . . . IV. Size of Sections and the BV Norm of Natural Images

. . . . . . . .

185 186

A. A Lower Bound for the BV Norm . . . . . . . . . . . . . .

186

B. Application to Natural Images . . . . . . . . . . . . . . .

190

V. The Dead Leaves Model . . . . . . . . . . . . . . . . . .

192

A. Random Closed and Compact Sets of IRn

. . . . . . . . . . .

193

B. The Dead Leaves Model . . . . . . . . . . . . . . . . .

194

C. The Geometric Covariogram . . . . . . . . . . . . . . . .

199

D. The Distribution of the Intercepts in the Dead Leaves Model . . . . .

201

E. The Stability of the Power Law . . . . . . . . . . . . . . .

205

F. The Size Distribution of the Relief Grains . . . . . . . . . . .

207

G. The Average Area After Occlusion: Numerical Experiments . . . . .

208

Volume 111 ISBN 0-12-014753-X


168

LUIS ALVAREZ, YANN GOUSSEAU, AND JEAN-MICHEL MOREL

VI. A Short Review of Texture Synthesis by Mathematical Methods A. The Normal Vectors Perturbation Method . . . . . . B. Solid Textures . . . . . . . . . . . . . . . C. Reaction-Diffusion Textures . . . . . . . . . . . D. Spot Noise Textures . . . . . . . . . . . . . E. Fractal Models . . . . . . . . . . . . . . . 1. Fractional Brownian Motion . . . . . . . . . . 2. Midpoints Displacement Method . . . . . . . . 3. The Spectral Synthesis Method . . . . . . . . . 4. Iterated Function Systems . . . . . . . . . . F. Random Fields Models . . . . . . . . . . . . 1. Markov Models . . . . . . . . . . . . . . 2. An Example: The Autobinomial Model . . . . . . 3. The Model of Gagalowicz and Ma . . . . . . . . G. Multiscale Synthesis Model . . . . . . . . . . . H. The FRAME Model . . . . . . . . . . . . . VII. Some Principles for the Synthesis of Abstract Natural Images . A. Structural Laws . . . . . . . . . . . . . . . B. Scale and Perspective Laws . . . . . . . . . . . C. Synthesis Rules and “Synthetic Worlds” . . . . . . . VIII. Appendix: Matheron’s Dead Leaves Model . . . . . . . A. Poisson Point Processes . . . . . . . . . . . . B. The Boolean Model . . . . . . . . . . . . . C. The Dead Leaves Model in IR2 . . . . . . . . . . References . . . . . . . . . . . . . . . . . . Acknowledgements . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . .

210 211 212 214 215 216 216 218 219 219 221 222 223 224 225 226 228 229 229 230 235 235 237 238 239 242

ABSTRACT This paper is partly written as a review, in which we discuss old and new image analysis and synthesis methods. We also introduce a new method for analyzing scaling phenomena in natural images and draw some consequences as to whether natural images belong to the space of functions with bounded variation. In some sense, our analysis computes the size distribution of objects in an image. By using the dead leaves model, we study the influence of occlusion on size distribution, and prove compatibility with our experimental results. I. INTRODUCTION There are many statistical studies of natural images, i.e., of pictures of scenes encountered in the surrounding world. Most of the studies are concerned with

THE SIZE OF OBJECTS IN NATURAL AND ARTIFICIAL IMAGES

169

first- or second-order statistics (through the power spectrum, the covariances, the cooccurrences) or with additive decompositions of images. Those statistical studies are partially matched by synthesis models, especially for texture synthesis, which try to compose an image either by imposing some second order statistics, or by adding (waveletlike) basis functions while maintaining some statistical requirements. Our aim in this paper is first to give an analytical account of those analysis and synthesis methods, in Sections II and VI, respectively. Then, we will try to complete the frame by introducing a new way of analyzing scaling phenomena in an image in Section III. We shall show that in natural images, there is a constant form for the size distribution. The definitions of sizes we consider are of several types: area, boundary length, and length of intersections with lines. An experimental program that we performed on many photographs of very diverse natural scenes indicates that the size distribution of homogeneous parts in images has a very peculiar form. We define in Section III what we mean by homogeneous parts, the connected components of image domains where contrast does not exceed a certain threshold. We can roughly define the size distribution of objects in an image as the size distribution of these parts. Our experiments show that the number of homogeneous parts as a function of their size s obeys the law cardfhomogeneous regions with size sg D

K s˛

where K is an image-dependent constant. When the size s denotes the area, in most photographs, ˛ is close to 2. As a consequence of the size power law, some information can be obtained about the “natural” function space for images, as shown in Section IV: we focus our attention on the space BV of functions with bounded variation. We are in a position to tell when a given image is not in this space, provided the observed size distribution model remains true at smaller (not observable) scales as well. In Section V, we present a powerful but scarcely used model, the dead leaves model of Matheron, and analyze the effect of occlusion on the size distribution of objects. There is no obvious equivalence between the way we define regions in our experimental program and regions in the dead leaves model. However, we show that, for some notions of size, the power law is preserved by occlusions in the dead leaves model. In order to straighten the link between the dead leaves model and our analysis, we also compute size distributions in synthetic images. We tried several principles for the synthesis, some being generalizations of the dead leaves model definition and some completely different, such as the exclusion principle. New kinds of textures can be generated by those models. We display several examples of “natural” and “nonnatural” abstract images in Section VII. For the purpose of simplicity, we tried to avoid any precise

170


references in this short introduction. Instead, each section is documented and can be read fairly independently.

II. STATISTICS OF NATURAL IMAGES: A REVIEW A. Motivations A digital, gray level image may be seen as the realization of a random vector of size M ð N taking values in a discrete set V D 1, . . . , C. For typical values such as M D N D C D 256, the number of possible realizations, CMN D 2524,288 , is huge. Obviously, “natural images”, i.e., digital photographs of natural scenes, form only a small subset of all possible realizations. Looking at random realizations of such vectors is enough to be convinced of this fact. Natural images are highly improbable events. It is therefore interesting to look for statistical characteristics of such images: what are the relationships between gray level values at distant pixels? Is it possible to define a probability law for natural images? In what follows, I is a digital image, for x 2 M ð N, Ix 2 V is the gray level value at pixel x, PrA stands for the probability of the event A, and E is the mathematical expectation. In addition to the legitimate interest in the complicated and fairly unknown structure of images, there are some strong motivations to investigate such statistics. The first one comes from biological vision and has initiated most of the work on the field. Indeed, it is likely that visual systems are adapted to the type of scenes they deal with (which may be very different from one species to another). H. Barlow [1961] suggested that the goal of sensory coding is to reduce redundancies in the input signals, so that studying those redundancies (for example, dependency between adjacent pixels) gives an insight to the way our brain is extracting informations from the huge amount of data it receives through the eyes. A second motivation arises from the Bayesian framework. Suppose we want to describe an image by a set of parameters fp1 , . . . , pn g. In order to estimate these parameters, given the image I, we make use of Baye’s rule PrIjfpi g Prfpi g Prfpi gjI D PrI which indeed presupposes the existence of a probability distribution Pr on images. This can for instance be useful for the denoising (through the search of a good approximation of the “true” image knowing a noisy, and possibly convolved, image) if we know characteristics of Pr. A third motivation is image compression. Imagine a very simple type of image where each realization is


171

either black or white; then only one bit will be needed to encode any image. More generally, information theory tells us that the compression rate is related to the entropy of the image I, EI D

PrI log PrI.

allI

Shannon’s source coding theorem asserts that in order to compress (many) images with almost no loss of information, the number of bits needed is almost equal to the entropy. Moreover trying to improve this bound leads to a drastic loss of information. Codes to achieve such a compression are given by information theory but are not feasible in practice, and the measure of loss may be poorly adapted to images. However, statistics of images may give an insight to the type of compression schemes that are to be used. We also show in the present paper (see Section IV) that statistics of images may account for the type of functional space to which images belong. The study of the statistics of images is not a new subject. It started with television signals studies [Kretzmer, 1952; Deriugin, 1956]. The subject has gained considerable interest during the last 10 years. In all studies, images are assumed to be translation invariant, allowing the relationships between pixels to depend on distance only, even though this assumption is not strictly observed. When dealing with statistics of natural images, two different approaches are possible: either assuming that a single image is large enough to evaluate parameters on it, by making ergodicity assumptions, or averaging the quantity under study over a large number of images. The first method enables us to study differences between images and may render easier a physical explanation, whereas the second is more robust and permits the estimation of outliers, which are not numerous enough in a single image. In order to study natural scenes, one may be interested in studying the light intensity arriving on the lens and thus in having a linear relationship between gray level and intensity. Moreover, one may want to have an histogram centered at 0 and invariant under multiplication of the gray levels by a constant. One way to achieve that is to study logI/I0 instead of I, where I0 is the mean gray level. B. First-Order Statistics The simplest thing we can study from an image is the gray level histogram, i.e. the distribution of gray levels in any pixel (under translation invariance each pixel is equivalent). Several authors [Ruderman and Bialek, 1994; Huang and

172


Mumford, 1999] found linear tails and a picked central value when plotting log(number of occurrences of gray level) versus log(gray level). They worked on images of logarithms of initial gray level values for the previously mentioned reasons. Those linear tails indicate that the histogram is not Gaussian, showing by the contraposal of the central limit theorem that values in different pixels are correlated. Moreover, the distribution is not symmetric, which could be due to the presence of large parts of sky in many images, as suggested in Huang and Mumford [1999]. However, looking at the histogram could be interesting for the purpose of biological studies when dealing with calibrated images, but it is not of much use in order to study the morphological structure of images. It is well known that we can strongly modify the histogram of an image without affecting its perception. C. Second-Order Statistics 1. Covariance and Power Spectrum In this section, I is assumed to be a continuous function from IR2 into IR. Under the assumption of translation invariance, the covariance of an image I is defined as (assuming the mean is zero) Cx D EI0Ix where 0 is an arbitrary origin and x a location in the image. If IO is the Fourier transform of the image (I is viewed as a continuous function belonging to L 2 and IO is a function of the complex variable ω), the power spectrum is defined as Pω D jIO ωj2 and we have the Wiener-Khintchin formula O Pω D Cω. It is well known that, when averaged over all directions, the power spectrum of natural images, 2 1 Ptei d ft D 2 0 has been experimentally found to have the form ft '

C t˛


173

with ˛ close to 2 [Deriugin, 1956; Field, 1987; Burton and Moorhead, 1987]. This phenomenon is also observed with the logarithm of images [Ruderman and Bialek, 1994]. It should be mentioned that the form of Pω is strongly dependent on the direction [Baddeley, 1997], and that there are large differences in the exponent ˛ of the power law from one image to another [Tolhurst et al., 1992]. Moreover, the Fourier transform is a nonlocal operation, and consequently the power spectrum may emphasize a particular aspect of the image, such as the presence of an edge. In the case of the presence of a single very sharp edge — for instance, at the horizon — the exponent of the power law gives no account of other statistics of the image; see Voss and Wyatt [1993]. A very interesting fact about this power law is that it is implied by scale invariance (as stressed by Field [1987]). Indeed, if there is the same energy for frequencies ω between t and t0 , t < jωj < t0 , and between at and at0 , at < jωj < at0 , then t0

ufu D

t

at0

u fu. at

But this must be true for all t and t0 so that fau D 1/a2 fu for all u, a, so that fu D C/u2 . It is worth noticing that the same reasoning about sounds leads to Pt D C/t, a property of the so-called 1/f-noise, which is also observed [Voss and Clarke, 1975]. Assuming this power law for the power spectrum and because of the WienerKhintchine relation, the correlation should be of the form Cx D A C B logjxj. It is worth mentioning that if Pω is a power law with an exponent ˛ 6D 2, then the covariance should be of the form Cx D A C Bjxj2˛ with the signs of A and B ensuring its decay when x increases. Those types of covariances have been observed, at least asymptotically [Roderman, 1994, 1997; Baddeley, 1997; Huang and Mumford, 1999]. 2. Other Second-Order Statistics The covariances of an image do not characterize the complete second-order statistics. Those statistics are fully described by the cooccurrence function: Ki, j, x D PrI0 D i and Ix D j

174


where x is a site in the image and i, j are gray levels. Gagalowicz and Ma [1985] made use of related quantities to synthesize textures, as explained in Section VI.F.3. As a particular case of cooccurrences Huang and Mumford [1999] look at the logarithm of the difference of gray levels between two adjacent pixels, working with the logarithm of the images. The observed distribution has a very peculiar form, with long tails. Once more this emphasizes the fact that images are non-Gaussian. The large tails of the distribution denote the presence of edges in the images. D. Linear Decomposition of Images Other types of studies deal with the decomposition of images into a linear combination of basis images. An image I is considered as a random vector of size M ð N and is written n ID an An 1

where fai g are random variables and fAi g are fixed images. The Ai ’s are determined by imposing some conditions on the ai ’s and are then used as a statistical description of the images. In the principal components analysis, the ai ’s are requested to be pairwise uncorrelated. The Ai ’s are obtained by looking for eigenvalues of the covariance matrix. For natural images, they are nonlocal and resemble a Fourier basis [Olshausen and Field, 1996; Bell and Sejnowski, 1997]. In the independent components analysis, the ai ’s must be pairwise independent. Such a decomposition may not exist, and the Ai ’s are approximated by numerical schemes. Bell and Sejnowski [1997] used a neural network approach to compute the independent components of natural images (actually on small 12 ð 12 patches extracted from four natural images) and obtained vectors that are localized and oriented and resemble edge-detector filters. Those vectors also resemble receptive fields of simple cells in the visual cortex, confirming the hypothesis of Barlow mentioned in the introduction. Another decomposition is studied in [Olshausen and Field, 1996]. The ai ’s should minimize ED I

2

ai Ai

C

S

ai

with i2 D Eai2 , a constant, and S some increasing real function. The idea is to find Ai ’s that cost as little as possible with respect to the cost function, S. As in the case of independent components analysis, the Ai ’s are localized and oriented.


175

E. Scale Invariance in Natural Images We do not attempt to give a precise definition of scale invariance, but we call so the property of images to have a zoom-invariant probability law. Scale invariance has been observed in several ways in natural images. First of all, the simple fact of being able to observe many common statistics on images representing arbitrarily chosen scenes (thus seen from arbitrary distance and with any size of objects) is a strong clue to scale invariance. Secondly, as we previously mentioned, the observed form of the power spectrum is the only possible one for scale invariant images. Moreover, many statistics have been found to be preserved when block-averaging images. This is the case of the histogram and covariances [Ruderman and Bialek, 1994; Ruderman, 1994] and is also the case for the gray value differences between adjacent pixels [Mumford, Zhu, and Gidas, 1997]. It must be emphasized that this is the case even though the procedures to obtain smaller images where very brutal: small filtering followed by subsampling. Actually, images are not exactly scale invariant. We already mentioned that the exponent of the power law decay of the power spectrum is not always 2. Moreover, when block-averaging the images, statistics are preserved by slightly renormalizing the image, i.e., by multiplying it by a constant at each blowdown, which is a way to measure the departure of images from scale invariance [Mumford, Zhu, and Gidas, 1997]. Our results about areas of sections also confirm this scale invariance, as is shown in Section III. III. SIZES OF SECTIONS IN NATURAL IMAGES As reviewed in the previous section, the known statistical properties of natural images are mainly first- or second-order statistics and statistics of additive decompositions of images. We have a different approach, working in the image domain on items that can have a straightforward visual interpretation. The goal of our experiments is to study the areas of homogeneous regions in digital images. We shall look at the area distribution of the connected components of bilevels of various digital photographs, and realize that those distributions are very similar one another. Indeed, the observed values are very close to functions A/area˛ , where ˛ is a real number around 2, and A is an image-dependent constant. That is, cardfhomogeneous regions with area ag D

A . a˛

We shall also study the distribution of the lengths of the boundaries of those connected components of bilevel sets and also observe a power law, with a

176


different exponent. Eventually, the same result is true for the distribution of the lengths of the intersections of those components with lines of fixed direction. From the first two of those statistics, we are in a position to tell when images are not in the space of functions with bounded variation, BV, as is shown in Section IV. A. The Distribution of Areas We now make clearer what we mean by homogeneous region of an image. We begin by equalizing the image histogram and uniformly quantifying it in the following way. We consider a digital image I of size H ð L, with G gray levels, and write Ii, j for the gray level at pixel i, j. Let k be an integer less than G. Let N1 be the first integer such that more than HL/k pixels have a gray level less than N1 ; then N2 , the first integer such that more than 2HL/k have a gray level less than N2 ; then N3 . N3 , . . . , Nk D G defined the same way, this sequence being possibly constant at some point. For l varying from 1 to k, let Il be the binary image with Il i, j D 1 if Ii, j 2 [Nl1 , Nl and Il i, j D 0 otherwise. We call those images k-bilevels of I. Each bilevel image represents a quantization level of the equalized image. Next, we look at the area histogram of the connected components of the bilevels. For s an integer varying from 0 to HL, let fs be the number of connected components with area s of the set of 1s pixels, in any of the k-bilevels of I. We consider both consider 4-connectivity (each pixel has 4 neighbors: up, down, right, left) and 8-connectivity (we add the diagonal neighbors, so that each pixel has 8 neighbors). We computed the function f on many digital photographs. We did not attempt to use a single source of images; the digitized images either are scanned photographs or are from a digital camera, with diverse optical systems and exposures. Those functions are of the form fs D C/s˛ , with C a constant and ˛ a real number close to 2, for values of s in a certain range and reasonable values of k (basically between 4 and 30). The observed fit is excellent, as can be seen from Figure 2, which actually corresponds to one of the worst cases we observed. For fixed k, we consider the set of points S D flogs, logfs, 0 s Tmax g where Tmax C 1 is the smallest value of s such that fs D 0. We perform a linear regression on this set S to find the straight line (in the log-log coordinates) glogi D A ˛ logi the closest to S in the least-squares sense and write E for the least-squares error. The choice to consider only the data


177

s Tmax is motivated by the fact that f takes mainly 0 and 1 values at some point, which makes difficult the fit of f to a reference function. Thus we minimize the energy E˛, A D

Tmax

log fi A C ˛ log i2 .

iD1

B. Digital Photographs We present the result for pictures having different scales and textures (see Figures 1–7) in Tables 1–7. The two texture images in Figures 6 and 7 are from the Brodatz’s album. The value of ˛ seems to be related to the amount of texture in the image; the more textured the image, the bigger the value of ˛. Typically, for photographs of natural scenes, the value of alpha is between 1.5 and 3 (the values close to 3 being reached for images as the baboon (Figure 3), which present textured areas), whereas for textures it is typically between 2.5 and 3.5. We also performed the linear regression on sets of points ST min D flogs, logfs, Tmin s Tmax g

FIGURE 1. Baobab, 665 ð 1024, 119 gray levels.

178

LUIS ALVAREZ, YANN GOUSSEAU, AND JEAN-MICHEL MOREL 105

number of c.c.

104

103

102

101

100 100

101

102

103

size of c.c.

FIGURE 2. Function f (area distribution) for the baobab image (Figure 1), k D 16, 8-connectivity, Tmax D 182.

FIGURE 3. Baboon, 512 ð 512, 224 gray levels.

for various values of Tmin to show that the fit of S to the power law was not forced by small areas only, and moreover that if the contribution of E mainly comes from the large areas, the value of ˛ computed with those large areas was close to the initial value. The results for the image of the baobab are shown in Table 3.

179


FIGURE 4. Airport, 510 ð 341, 256 gray levels. 105

number of c.c.

104

103

102

101

100 100

101 size of c.c.

102

FIGURE 5. Function f for the airport image (Figure 4), k D 16, 8-connectivity, Tmax D 92.

C. Other Types of Images In order to see whether the power law is in some sense characterizing digital photographs, we computed histograms of areas of bilevels for other types of images. We looked at noise images, white or correlated; text images; and synthesized images consisting of the superposition of simple shapes. White noise images, i.e., images in which the gray level values at distinct pixels are independent random variables, present a histogram of the form fs D expCs, with C a constant. We observed this fact on two different kinds of white noises: uniform and Gaussian. Thus the value of E is large for those images, and so is ˛, as presented in Table 8.

180


FIGURE 6. Texture1, 256 ð 256, 153 gray levels.

FIGURE 7. Texture2, 256 ð 256, 118 gray levels. TABLE 1 DIFFERENT VALUES OF THE QUANTIZATION NUMBER k FOR THE BAOBAB IMAGE, 8-CONNECTIVITY. AREA DISTRIBUTION IS fs D AS˛ , Tmax IS THE MAXIMAL CONSIDERED AREA. Image

k

˛

E

Tmax

A

Baobab Baobab Baobab Baobab

20 16 12 8

1.98 1.98 1.85 1.67

0.24 0.45 0.31 0.27

109 182 111 82

11.2 11.0 10.3 9.3

Text images produced by text editor do lead, as one would guess, to a histogram consisting of isolated peaks, whose height is not directly related to the value of s. Thus the fit to the power law is poor; see Figure 8 and Table 9. Concerning the synthesized images, we observed two distinct behaviors. First, if we consider superposition of simple shapes (disks, rectangles, polygons) with


181

TABLE 2 DIFFERENT VALUES OF k FOR THE BAOBAB IMAGE, 4-CONNECTIVITY. Image

k

˛

E

Tmax

A


20 16 12 8

2.15 2.06 1.96 1.86

0.32 0.38 0.30 0.40

146 160 106 133

11.7 11.4 10.8 10.1

TABLE 3 DIFFERENT VALUES OF Tmin FOR THE BAOBAB IMAGE, k D 16, 8-CONNECTIVITY. Image

Tmin

˛

E

A


5 10 20 50

2.04 2.07 2.09 2.08

0.44 0.44 0.46 0.50

11.3 11.4 11.5 11.5

TABLE 4 DIFFERENT VALUES OF k FOR THE BABOON IMAGE, 8-CONNECTIVITY. Image

k

˛

E

Tmax

A

Baboon Baboon Baboon Baboon

20 16 12 8

2.55 2.38 2.42 2.35

0.30 0.33 0.47 0.41

70 82 78 76

11.7 11.3 11.4 11.2

TABLE 5 DIFFERENT VALUES OF k FOR THE AIRPORT IMAGE (FIGURE 4), 8-CONNECTIVITY. Image

k

˛

E

Tmax

A

Airport Airport Airport Airport

20 16 12 8

1.95 1.93 1.79 1.83

0.43 0.47 0.38 0.35

92 92 95 63

9.7 9.6 9.0 8.7

various laws of sizes and possibly made of subobjects, the function fs is often well fitted by a power law, but with an ˛ close to 1. This is quite difficult to predict by calculations, and we refer to the section presenting the dead leaves model for the mathematical analysis of those phenomena. We present two results. In the first one we considered 50 images produced by the superposition of disks of the same size. Each disk has a random color uniformly drawn between 0 and 255, and its center follows a uniform law in the image (actually in a slightly enlarged

182

LUIS ALVAREZ, YANN GOUSSEAU, AND JEAN-MICHEL MOREL TABLE 6 DIFFERENT VALUES OF k FOR THE TEXTURE1 IMAGE (FIGURE 6), 8-CONNECTIVITY. Image

k

˛

E

Tmax

A

Texture1 Texture1 Texture1 Texture1

20 16 12 8

3.59 3.56 3.34 2.44

0.66 0.66 0.67 0.51

19 26 31 25

11.5 11.7 11.6 10.4

TABLE 7 DIFFERENT VALUES OF k FOR THE TEXTURE2 IMAGE (FIGURE 7), 8-CONNECTIVITY. Image

k

˛

E

Tmax

Slope

Texture2 Texture2 Texture2 Texture2

20 16 12 8

3.42 3.17 2.91 2.61

0.43 0.45 0.34 0.49

28 29 34 38

11.3 11.1 10.9 10.7

k D 16 Image Uniform noise Gaussian noise

FOR

TABLE 8 WHITE NOISES, 8-CONNECTIVITY. ˛

E

Tmax

A

5.00 4.70

0.93 0.88

11 11

13.5 13.4

FIGURE 8. Text, 703 ð 615, 2 gray levels.

image). In the section about the dead leaves model we used the same image to compute average sizes, but with a different analysis to avoid the union of objects and boundary problems. The values of fs for each s were then added over all images, and the function f fitted by a power law as before. We then performed the same analysis with disks having an area distributed according to a power law with exponent 2, and a radius between 20 and 80. See Table 10.


kD2 Image Text Text

FOR THE

183

TABLE 9 TEXT IMAGE (FIGURE 9).

Connectivity

˛

E

Tmax

A

8 4

0.84 1.70

1.15 0.81

17 59

4.0 8.1

TABLE 10 VARIOUS SYNTHESIZED IMAGES, 8-CONNECTIVITY, k D 16. Image Disks Disks, power law Eyes (Figure 9)

˛

E

Tmax

A

0.86 0.80 1.66

0.29 0.39 0.31

1000 807 686

9.0 6.9 12.4

Furthermore, if we do the same with more complicated figures, having holes and asperities, the observed law is a power law as well, but with an ˛ close to two. In the example presented in Figure 9, the basic shape is itself the connected component of the bilevel of a photograph, actually a part of a fly eye. It must be emphasized that discretization effects are very strong in these types of complicated shapes, but if we perform a morphological closing of size 1 pixel on the basic shapes, the results are not qualitatively affected, which tends to minimize the effect of discretization. Last but not least we present the results obtained when analyzing a correlated noise. We performed convolutions between white noises and a Gaussian 1 exp 2 x 2 C y 2 2

FIGURE 9. Dead leaves model, where all shapes have the same size, 900 ð 900, 256 gray levels.

184


TABLE 11 UNIFORM WHITE NOISE IMAGE, AFTER CONVOLUTION WITH A GAUSSIAN OF PARAMETER , k D 12, 8-CONNECTIVITY. Image convolved with a Gaussian Noise Noise Noise Noise

convolved convolved convolved convolved

with with with with

a a a a

Gaussian Gaussian Gaussian Gaussian

˛

E

Tmax

A

0.71 0.85 1.12 1.58

3.88 3.57 3.27 2.87

0.60 0.50 0.55 0.47

26 30 39 59

13.3 13.1 13.0 12.6

where is a variable parameter. This was done by multiplication in the frequency domain. Such a convolution can be seen as a crude approximation of the effect of an optical lens. The results we obtain for those images were similar to the ones for digital photographs of textures. We present the results obtained in the case of the white uniform noise in Table 11. We also tested the effect of the convolution with Bessel functions (Fourier transform of disks), and the results were very similar. Those “nonnatural images” lead to two remarks about the 1/area2 law. First, this law does not characterize natural images, even though a correlated noise looks similar to a natural texture, and the complicated synthesized images may look familiar to us. Second, the size law could be related to the way the optical photographic device captures the image, as suggested by the behavior of noise convolved with a Gaussian. More precisely, we observed that the convolution with a Gaussian increases the value of ˛ for images where the initial ˛ is small (such as text and synthetic images) whereas it tends to decrease its value when it is initially bigger than 2 (noises). Another, and more satisfactory explanation of this power law relies on the assumption that natural images are scale invariant so that all observed statistics should be scale (zoom) invariant. This simple assumption yields the 1/area2 law. Indeed, if we suppose that the total area occupied by regions having an area between A and A0 is the same as the total area occupied by regions with area between tA and tA0 , for all t, A, A0 , then the same reasoning as presented for the power spectrum (see Section I.C.1) leads to the power law with exponent 2. D. The Distribution of Boundary Lengths We performed exactly the same analysis on the boundary lengths of connected components of bilevels as we did before on areas of those components. As a discrete definition of the length of a discrete connected set S (8-connectivity), we chose to count the pixels not belonging to S that are neighbors of some pixel of S in the 4-connectivity sense. There are many other ways to define discrete boundary length. We tried several other methods that gave basically


185

the same results as the one we detail here. The notations k, E, A, Tmin , and Tmax refer to the same quantities as before; ˇ now stands for the exponent of the power law whose fit to the boundary length distribution is the best in the least-squares sense. We chose Tmin D 10, because some small values for the boundary lengths are attained only for regions touching the border of the image. The fit to the power law is again very good, and the exponent ˇ is usually between 2 and 3. We present the results for the two images of the baobab and the baboon in Tables 12 and 13. We note that ˇ ' 2˛ 1 accounts for connected components of bilevel sets satisfying on the average a decent isoperimetric ratio, c

area1/2 C. boundary length

This is not the case in general, except for some images of textures. E. The Distribution of Intercept Lengths We also studied the distribution of the lengths of the intersections of the bilevel sets with lines of fixed direction. This was motivated by the fact that this quantity, in an object-based model of images, is easily amenable to calculations (see Section V.D). We once more observed a power law, which, according to the previously mentioned model, is compatible with a power law hypothesis TABLE 12 BOUNDARY LENGTHS FOR THE BAOBAB IMAGE, WITH DIFFERENT VALUES OF THE QUANTIZATION NUMBER k. Image

k

ˇ

E

Tmax

A


20 16 12 8

2.39 2.37 2.26 2.23

0.34 0.35 0.25 0.35

176 154 125 111

13.2 13.0 12.3 11.6

TABLE 13 BOUNDARY LENGTHS FOR THE BABOON IMAGE, WITH DIFFERENT VALUES OF THE QUANTIZATION NUMBER k. Image

k

ˇ

E

Tmax

A


20 16 12 8

3.02 2.91 2.93 2.89

0.28 0.35 0.41 0.33

82 81 100 96

14.2 13.9 14.0 13.8

186

LUIS ALVAREZ, YANN GOUSSEAU, AND JEAN-MICHEL MOREL TABLE 14 INTERCEPT LENGTHS FOR THE BAOBAB IMAGE, WITH DIFFERENT VALUES OF THE QUANTIZATION NUMBER k. Image

k

ˇ

E

Tmax

A


20 16 12 8

2.72 2.60 2.55 2.33

0.42 0.37 0.47 0.44

88 77 118 124

13.3 13.0 13.1 12.6

TABLE 15 INTERCEPT LENGTHS FOR THE BABOON IMAGE, WITH DIFFERENT VALUES OF THE QUANTIZATION NUMBER k. Image

k

ˇ

E

Tmax

A


20 16 12 8

3.79 3.58 3.37 3.23

0.47 0.35 0.41 0.58

24 27 29 44

12.9 12.8 12.8 13.0

for the size distribution of objects, even if the model is not isotropic. See Section V.E. However, it should be mentioned that the result is strongly varying with the direction. The experiments we present here are concerned with the horizontal direction; see Tables 14 and 15. The observed values for the exponent of the power law are typically between 2 and 3.5.

IV. SIZE OF SECTIONS AND THE BV NORM OF NATURAL IMAGES The aim of this section is to give a computational tool to decide whether an image can belong to the space BV of functions with bounded variation. The BV assumption for natural images is far ranging, from image restoration [Rudin, 1987; Rudin, Osher, and Fatemi, 1992] to image compression.

A. A Lower Bound for the BV Norm We consider I 2 BV , a bounded image belonging to the space of functions with bounded variation [Ziemer, 1989; Evans and Gariepy, 1992] on a domain (e.g., rectangular) ² IR2 . For 2 IR, define the level set of I with level by I D fx, Ix ½ g.


187

Recall that a function is of bounded variation if, for almost every 2 IR, I is a set with finite perimeter and, denoting this perimeter by per I (for a precise definition of the perimeter and the essential boundary, refer to Evans and Gariepy [1992]), jjIjjBV D

per I d.

1

IR

(By the coarea formula, [Evans and Gariepy, 1992], we also have jjIjjBV D

jDIj.) In addition, by the classical isoperimetric inequality, we have for every set O with finite perimeter, perO ½ 21/2 O1/2

2

where O denotes the Lebesgue measure of O. In the following, we shall consider sections of the image. We always assume that the image I satisfies 0 Ix C. We first fix two parameters , , with 0 . For any n 2 IN, we consider the bilevel sets of I fx, C n 1 Ix < C ng D Cn1 InCn I. We call , -section of I any set that is a connected component of a bilevel set Cn1 InCn I for some n. We denote each one of them by S,,i for i 2 J, , a set of indices. Notice that the , -sections are disjoint and their union is the image domain , S,,i D .

3

i2J,

There are several ways to define the connected components of a set with finite perimeter, because such a set is defined up to a set with zero Lebesgue measure. We denote by H1 the one-dimensional Hausdorff measure, i.e., the length. In the following, we call a Jordan curve a simple closed curve of IR2 , i.e., the range of a continuous map c : [0, 1] ! IR2 , such that cs 6D ct for all 0 < s < t < 1, and c0 D c1. A Jordan curve defines two and only two connected components (in the usual sense) of IR2 nc[0, 1], one bounded and one unbounded. We shall say that a Jordan curve separates two points x and y if they do not belong to the same connected component of IR2 nc[0, 1]. One can prove [De Giorgi, 1954; Caselles and Morel, 1998] that a definition of connected components for a set with finite perimeter permits the following statements:

188


Theorem 1 (and definition) Let O be a set with finite perimeter. (i) The essential boundary of O consists, up to a set of zero H1 -measure, of a countable set of noncrossing simple rectifiable closed curves cj with finite length such that perO D 6j H1 cj (ii) Two points are in the same connected component of O if and only if for any representation of the essential boundary by a family of Jordan curves of the preceding kind, cj , they are not separated by one of the cj . (iii) With this definition, the perimeter of a set with finite perimeter is the sum of the perimeters of its connected components. We denote by Jn ² J, the set of indices of sections that are connected components of Cn1 InCn I. As an obvious consequence of Proposition 1, we have the following. Corollary 1 perCn1 InCn I D

perS,,i .

i2Jn

When A is a set with finite perimeter, we have [Evans and Gariepy, 1992] perA D jjllA jjBV where llA is the characteristic function of the set A. Lemma 1 If B ² A are two sets with finite perimeter, then perAnB perA C perB Proof

Indeed, by the subadditivity of the BV norm, we deduce from llAnB D llA llB

that perAnB perA C perB. In the following theorem, we analyze the statistics of sizes of sections. We fix , i.e., the overall contrast of considered sections and for each 0 , we count all sections S,,i that have an area between s and s C ds. In other terms we consider the integer cardfi, s jS,,i j s C dsg.


189

We average this number over all ’s in [0, ], and assume that this average number has a density f, s with respect to s. In other terms,

1

cardfi, s jS,,i j s C dsg d D f, s ds.

4

0

Theorem 2 Assume that there exists some > 0 such that (4) holds, i.e., the average number of sections with area s, for 0 , has a density f, s. Then there is a constant c, not depending on I, such that

jjIjjBV ½ c

s1/2 f, s ds.

5

0

Proof

Applying Corollary 1 and Lemma 1,

jjIjjBV D

perfx, Ix ½ g d IR

1 D 2 ½ D D

1 2 1 2 1 2

perfx, Ix ½ g d C IR

per In I d D IR 0

perfx, Ix ½ g d IR

1 2 n2Z

nC1

per In I d n

perCn1 InCn I d

n2Z

0

perS,,i d.

i2J,

By isoperimetric inequality (2), we therefore obtain

jjIjjBV ½ 1/2 0

jS,,i j1/2 d.

i2J,

Applying the Fubini-Tonelli theorem, some slicing, and Assumption (4), we get

jjIjjBV ½ 1/2

d

D 1/2

cardfi 2 J, , s jS,,i j s C dsgs1/2

0

0

0

d card fi 2 J, , s jS,,i j s C dsgs1/2

0

D 1/2

s1/2 f, s ds. 0

190


We can repeat the preceding analysis by assuming now that cardfi, p perS,,i p C dpg has an average density g˛, p with respect to p; i.e., 1

cardfi, p perS,,i p C dpg d D g, p dp.

6

0

Then we have the analog of Theorem 1 for the perimeters of sections. Theorem 3 Assume that there exists some > 0 such that (6) holds, i.e., the average number of sections with perimeter s, for 0 , has a density g, p. Then 1 C1 jjIjjBV ½ pg, p dp. 7 2 0 Proof

By proceeding as in the proof of Theorem 1, we have again jjIjjBV ½

1 2

0

perS,,i d.

i2J,

Applying again some slicing and Assumption (6) we get jjIjjBV ½ D

1 2 2

C1

d 0

cardfi 2 J, , p perS,,i p C dpgp

0 C1

pg, p dp. 0

B. Application to Natural Images In this section, we draw the consequences of Theorems 1 and 2 for the images analyzed in Section III. According to the results of this section, we can assume that the considered images satisfy C s˛ C g, p D ˇ p f, s D

8 9

for some constants ˛ > 0, ˇ > 0. This law has been experimentally checked for several values of D 256/k, k ranging from 8 to 20. We also checked that


191

the value of ˛ was almost not modified when the bilevels were not defined from gray level 0, but from some gray level less that 256/k (i.e., in the continuous model, for different values of ). By Theorem 1 we have

jjIjjBV ½ c 0

Cs1/2 ds D C1 s˛

if ˛ >

Cp dp D C1 pˇ

if ˇ > 2.

3 2

and in the same way, C1

jjIjjBV ½ c 0

Thus if we admit that (8) and (9) indeed hold for natural images when s ! 0, as is indicated by the experiments of Section IV.A, we determine that the considered images are not in BV if ˛ > 32 or ˇ > 2. Notice, however, that ˛ > 2, which happens for several of the considered images, is not compatible with a finite image area, since then s ds/s˛ D C1. As suggested to us by Vicent Caselles and Stéphane Mallat, this raises the question of whether the area is correctly measured by covering pixels. In fact, if a region is very ragged, the cardinality of covering pixels may be related to its perimeter as well, in which case the estimate of g, s is more reliable. We here point out that wavelet coefficients (see Meyer [1993] and Mallat [1997] for an introduction to wavelet decompositions) also give a way to decide whether or not an image belongs to the space BV. Let (ck ) be the wavelet coefficients of the image I, ordered in a nonincreasing sequence. Let us suppose that the wavelets have compact supports. We say that the ck ’s are in l1 if 6jck j < C1 and that they are in weak-l1 if there exists a constant C such that ck C/k. Obviously l1 is included in weak-l1 . It is quite easy to prove that if the ck are in l1 , then I is in BV. In the other direction, Cohen et al. [1998] recently proved that if I is in BV, then the ck ’s are in weak-l1 . Thus it is possible to decide whether an image belongs or not to BV by looking at its wavelet coefficients decay, except if they decrease as C/k, which often happens to be the case [Mallat, personal communication]. Moreover, it is worth noticing that the wavelet coefficients produced by the characteristic function of a simple shape already decrease as 1/k. We do not present here a precise comparison between the two criteria. Let us just mention that in the case of the baboon image (Figure 3), both methods agree: this image is not in BV. For the well-known image of Lena, our approach gives an ˛ of 1.9 (for k D 16), which suggests Lena being out of BV, whereas from the wavelet approach, the image is in BV.

192


V. THE DEAD LEAVES MODEL A tridimensional scene may be modeled as a group of solid objects occluding each other partially or fully, according to their respective positions and their position with respect to the observer. How, then, can the observer make inferences on the size of these partially hidden objects? And which ratio can we assume between the size of visible parts of objects and the size of the objects themselves? This question can be addressed in a statistical framework: given a size distribution for the objects before occlusion, how can we compute the size distribution of visible parts? In order to do so, we shall use a random shape model developed by Matheron [1967, 1968], in the framework of his theory, mathematical morphology. He called this model the dead leaves model. Its formalization requires some explanations, definitions, and mathematical results which are given exposed in Section V.A. This study has two motivations. The first one comes from phenomenology, which states that the occlusion phenomenon, so essential in human or animal life, does not seem to affect too much our perception of objects. In particular, Kanisza [1979] studied several perceptual strategies to detect occlusions but also to get rid of them by mental completion. This leads to the notion that with only the visible parts of objects, a layered view can be mentally reconstructed simply by interpolating their hidden parts. Even though gestaltists do not seem to have addressed the question, it is plain that such an interpolation (which Kanisza called amodal completion) can be performed only if a significant part of each object is under sight, large and significant enough to make extrapolation possible. A modal completion might be in many cases related to our prior knowledge of objects. However, Kanisza and his school proved that the completion takes place fairly independently from this prior knowledge, always giving a preference to constraints of a geometric nature, whether or not these lead to a common sense contradicting completion. The second motivation is the experimental program detailed in Section III. Even though the link between our definition of objects and the real visible parts of objects is not obvious, we can, at least in one case, derive a power law with the help of the model. The dead leaves model permits study of the influence of occlusion on the size distribution of a set of objects, with some restrictions: ž We shall address only stationary models, which do not take into account perspective for instance. It is possible to include perspective in the framework of the dead leaves model, but this will not be presented here. ž Objects will be assumed to spread out independently, which is of course an undue restriction for a real scene, where objects interact physically. ž Most of the theoretical results won’t deal directly with the area of visible objects but rather with the size distribution of their intersections with a straight line.


193

A. Random Closed and Compact Sets of IRn This section contains the prerequisites for the understanding of the dead leaves model. The interested reader should consult Matheron [1975]. For an introduction to the methods and applications of stochastic geometry, we refer to Stoyan, Kendall, and Mecke [1995]. We denote by , , K, respectively the sets of all closed, open and compact sets of IRn . In the following, we define a topology on and K and its associated -algebra of Borel sets. The choice for the topology is motivated by the following considerations: ž The set spaces thus obtained are separable, Hausdorff, and compact. ž The basic sets (open sets of sets, etc.) for this topology have an easy interpretation. ž The topology induced on Knf;g is metrizable with the Hausdorff metric. Let us set for any subset A of IRn , A A

D fF 2 /F \ A D ;g D fF 2 /F \ A 6D ;g.

In the same way, define subsets of K, KA , and KA . We define a topology T on , generated by the basis of open sets

K

, K 2 K;

G, G

2 .

It can be shown that , endowed with the topology T , is separable, Hausdorff, and compact. We can also endow K with a topology TK , a basis of which is KF , F 2 ; KG , G 2 . This topology is strictly finer than the topology induced on K by T . Define on K0 D Knf;g the Hausdorff distance dH by dH A, B D inffυ > 0/A ² Bυ and B ² Aυ g where

Aυ D fx 2 IRn /dx, A υg.

It can be shown that the topology TK on K0 and the topology induced by dH on K0 are equal (“point” ; is isolated in K). We denote by B and BK the Borel -algebras generated by T and TK , respectively. Definition 1 A random closed set (respectively a random compact set) of IRn is a measurable function from a probability space , T, P into , B (respectively into K, BK ).

194


We associate with a random closed set F its range measure F , defined on Borel sets B 2 B by F B D PF1 B. We write PrF \ A D ; D F A , PrF \ A 6D ; D F A , and so on. A first basic result for the study of random sets is the following. Theorem 4 Any random closed set F is characterized by its capacity function defined for every compact set K by GF K D PrF \ K D ;. Proof We have to show that if F1 and F2 are two closed random sets with the same capacity functions GF1 and GF2 , then they are equal. It is enough to prove that F1 and F2 satisfy F1 B D F2 B for every Borel set B in B . From the definition of the topology on , it is enough to have F1 K D F2 K and F1 G D F2 G for every K 2 K and every G 2 . The first condition boils down to the equality of the capacity functions. In addition, let G 2 . Then there exists a sequence of compact sets Kn such that Kn " G, which entails Kn " G . By continuity of the measure with respect to monotone sequences, we then have F1 Kn " F2 G . Now, Kn D Kn c and therefore 1 Kn D 1 F1 Kn D 1 F2 Kn D F2 Kn , which ends the proof. Definition 2 We shall say that a random closed set with capacity G is stationary if for every compact set K and every point x of IRn , one has GK D GK C x. We say that F is isotropic if it is stationary and for every rotation r, GK D GrK. We end this section with some notation that will be of use in the following. For any sets A and B, we set AL D fx, x 2 Ag A B D fx 2 IR2 /x C BL ² Ag A ý B D fx C y, x 2 A, y 2 Bg A B is called erosion of A by B, and A ý B is called dilation of A by B. B. The Dead Leaves Model The model we present here is a slight variation of a model introduced by Matheron [1968]. The original model was defined on all of IR2 , whereas we define it only on a window, but the result is locally identical. In Section VIII, we present the dead leaves model as introduced originally by Matheron. We consider a closed bounded domain in IR2 (a rectangle, for instance). Let Xi , i 2 IN (IN is the set of negative integers) be a family of independent


195

equally distributed random compact sets Xi . We assume that X0 is almost surely uniformly bounded, i.e., there exists R > 0 such that X0 ² B0, R almost surely. We also assume, for reasons that will become clear soon, that EX0 D > 0 for some disk D. We then consider a family xi , i 2 IN of random variables that are independently uniformly distributed in R D

C B0, R. We define a sequence of dead leaves in the following way. For each i 2 IN we call dead leaf at time i the random set xi C Xi . We then call visible part of the dead leaf i the set

xi C Xi n

xj C intXj i<j0

where intX is the interior of X. We then define the dead leaves model M to be the collection of all the visible part of the dead leaves i for i 2 IN. We also define a cell of M to be any connected component of a visible part. In the following, X0 will often be referred to as the grain of the model. In Figures 10, 11, and 12, we present some experimental realizations of dead leaves models for different choices of grain X0 . To simulate the model, we associate with each xi C Xi a different gray level uniformly distributed between 0 and 255. We already see from those examples that the behavior of the dead leaves model is strongly dependent on the shape of the leaves as well as on their size distribution. We also notice that the 3-D perception is much stronger in the case of elongated rectangles than in the case of disks.

FIGURE 10. Simulation of a dead leaves model, where the grain X0 is a disk with constant radius.

196

FIGURE 11. Simulation of a dead leaves model, where the grain X0 is a disk with an exponentially distributed radius.

FIGURE 12. Simulation of a dead leaves model, where the grain X0 is a rectangle with a rotation uniformly distributed in [0, 2], and with width and length following a uniform law between extremal values.

Lemma 2 Let K be a compact set of . Then the probability that K is contained in a given dead leaf xi C Xi is L EX0 K . R where is the Lebesgue measure. Proof K is in the dead leaf Xi if K ² xi C Xi , which is equivalent to xi C L ²X L i , i.e., xi 2 X L i K. It is easily checked that X L i K ² R . Since xi is K R uniformly distributed in , we deduce that L i K PrK ² xi C Xi D Prxi 2 X D

1 R

L i K dx Prx 2 X

R


D D D

1 R 1 E R

R

197

EllXL i K x dx

R

llXLi K x dx

L EX0 K EXL0 K D R R

10

L > 0, then K Corollary 2 If K ² , K measurable, is such that EX0 K is covered by some dead leaf with probability 1. Since for some disk D, we have EX0 D > 0, then is almost surely covered by a finite number of dead leaves. R L Proof Let a D EX0 K/

> 0. Then K is not completely covered by a leaf xi C Xi with probability 1 a < 1. Thus, since leaves are independent, K is not covered by any leaf with probability 0. Moreover, it is possible to cover with a finite number of the sets Yi D yi C D \ , with yi points of . Then each Yi is covered by some dead leaf at time ti with probability one, and thus at T D maxfti g, is covered by a finite number of leaves.

We are interested in the relation between the size of the grain X0 and the size of the visible parts of the dead leaves model, or the size of the cells. Next, we are going to prove a fundamental result which yields the probability for a compact set K ² to be included in a visible part of the dead leaves model. Theorem 5 Let M be the dead leaves model associated with X0 , as before. Then for any compact set K ² , the probability that K is included in a visible part of M is given by L EX0 K QK D . 11 L EX0 ý K Clearly, if K is connected, QK also denotes the probability that K is included in a cell of the model. Proof First, we note that the probability QK corresponds to the probability that for some i, K is included in the dead leaf xi C X and for i < j 0, K does not intercept any of the dead leaves xj C X. As the proof of Lemma 2 shows, in order to compute PrK \ xj C Xj D ;, we can work with Xj D X deterministic, xj uniformly distributed, and then take the expectation of the resulting formulas. The probability that the compact set K does not intercept L We remark the dead leaf xj C X is equal to the probability of xj 2 / X ý K. R L L that since X ² D0, R and K ² , X ý K ² . We deduce that

198


the probability in xj (for fixed X) that K does not meet xj C X is equal to L D1 PrK \ xj C X D ; D 1 Prxj 2 X ý K

L X ý K . R

Now, if Xj is a random compact set distributed as X0 , taking the expectation yields L EX0 ý K PrK \ xj C Xj D ; D 1 . 12 R We now consider the following disjoint events Ei , for i 2 IN: K is included in the dead leaf xi C Xi , and for i < j 0 the compact set K does not intercept the dead leaves xj C Xj . Since the leaves are independent, we have, using Lemma 2, i L L EX0 K EX0 ý K . 13 PrEi D 1 R R Making the sum over i 2 IN in (13) we obtain that the probability that K is contained in a visible part of the dead leaves covering is i L L EX0 K EX0 ý K QK D 1 R R i2IN which yields the announced result by summing the geometric sum, QK D

L EX0 K . L EX0 ý K

14

We end the section by remarking that if each leaf is assigned a value (color) in [0, C], C real, then a realization of the model can be seen as a function from to [0, C]. We may then wonder which functional space such a function belongs to. The characteristic function of a set with finite boundary length in

is in the space of functions with bounded variation, BV , and this will also be the case for almost every realization of the dead leaves model. Indeed, we saw previously that for almost every realization there exists an i0 < 0 such that the union of the leaves with indices i > i0 covers . This is also the case for the corresponding cells. Moreover, the sum of the length of the boundaries of those cells is obviously smaller than the sum of the lengths of the boundaries of their parent leaves. Now, if the parent leaves have boundaries whose length is bounded by a constant value B, the BV norm is finite and bounded by i0 Ł B for almost all realization. In particular, this is the case if the leaves are convex, since they are included in B0, R. We have thus proved the following:


199

Proposition 1 If X0 has uniformly bounded boundary length, then the dead leaves model M is in BV almost surely. The meaning of the result is strong and general. Even though we can have arbitrarily small realizations of X0 , these small leaves do not succeed in creating boundaries with infinite length, the reason being that the small leaves are steadily covered by larger ones. The next section introduces a tool in the study of compact sets that will be of use in the evaluation of the sizes of the cells of the dead leaves model. C. The Geometric Covariogram The geometric covariogram of a convex and compact set C of IR2 is the function C from IR2 to IR defined by C l, D C \ Cl , D C l, where Cl, is the set C C x, x being the point with polar coordinates (l, ), l > 0, 2 [0, . The C function has compact support and is equal to zero if l ½ 1 p? C, where p is the orthogonal projection on a line perpendicular to the direction , ? D C /2, and 1 is the Lebesgue measure in IR. In the following we assume that 1 p C > 0 for all (this will be the case with the leaves of M that have nonempty interiors). We are interested in the size distribution of the intersection of C with all lines with direction meeting C. We assume that the intersection of these random lines with a line perpendicular to is uniformly distributed in p C. We associate with each s 2 p C a line Ls of direction and meeting C. We denote by FC l, the distribution function of the size of the segments, defined by 1 fs, 1 Ls \ C ½ lg . FC l, D 1 p C Lemma 3 FC l, D Proof

1 p C \ Cl, . 1 p C

It is easily seen that Ls \ C \ Cl, D ;

iff 1 Ls \ C ½ l

which yields the result. The function FC is related to the function C by the next theorem.

200


Theorem 6 For any compact and convex set C in IR2 , 2 [0, , C ., is differentiable on [0, p? C, and ∂ C l, D 1 p C \ Cl, D 1 p CFC l, . ∂l Proof

15

[Matheron, 1975] For any fixed ,

ε1 p C \ Cl, ½ C \ Cl, nC \ ClCε, D C l, C lCε, ½ ε1 p C \ ClCε,

16

and ε1 p C \ Clε, ½ C \ Clε, nC \ Cl, D C lε, C l, ½ ε1 p C \ Cl, .

17

In order to establish the theorem, it is enough to prove the continuity of the function fl D 1 p C \ Cl, . For any l > 0, C ., is continuous from the left. Indeed, ln " l ) C \ Cln , # C \ Cl, (because the interception is an s.c.s. function) so that ln " l ) p C \ Cln , # p C \ Cl, (because p is continuous from K0 to K0 [Matheron, 1975, p. 69]). Therefore, we obtain 1 p C \ Cln , # 1 p C \ Cl, following the monotone continuity of 1 , so the left-continuity is proved. It is easily seen that fl is continuous from the right for any l such that C l, > 0, which ends the proof. We note that, in particular, the theorem states that C is a convex function. In what follows, we denote by C0 the derivative of C with respect to l. We can generalize the definition of the C function to a random compact set, using the expectation EC l, with respect to the probability law of C, when such a definition makes sense. In particular, we can define such an expectation when EC < C1, which is the case for the grain of the dead leaves model. For instance, we can consider a family of random compact sets fCr gr2I , depending — among other parameters — on r > 0 in such a way that Cr is a rescaling of C, Cr D rC, with a probability distribution fr. We then define the covariogram as C1

l, D E 0

Cr l, fr dr.


201

We are interested in the lengths of the intercepts of C, i.e., its intersections with lines of direction . We define FC l, D

EC0 l, EC0 0,

when this makes sense, i.e., when EC0 0, is finite and nonzero. As an easy consequence of Theorem 5, we have the following. Corollary 3 Assume that 0 < EC0 0, < C1. Then for any l 2 [0, p? C, EC l, 0 D EC0 l, so that FC l D

EC l, 0 . EC 0, 0

18

Proof We take the expectation with respect to the probability law of C of all members of relations (16) and (17). We notice that we can divide by ε and pass to the limit as ε ! 0. Indeed, we just have to prove that E1 p C \ Cl is continuous with respect to l. Now, we have seen in the proof of Theorem 5 that 1 p C \ Cl is continuous with respect to l. In addition, we obviously have 1 p C \ Cl 1 p C, which has a finite expectation by assumption. This proves the asked continuity by Lebesgue theorem. So, by dividing by ε and letting ε ! 0, we get the analog of (15), i.e., ∂ E l, D E1 p C \ Cl, D E1 p CFC l, . ∂l C

19

Thus in the following, where we will consider only covariograms for random convex sets, we will write C for EC and C0 for its derivative with respect to l. D. The Distribution of the Intercepts in the Dead Leaves Model We are interested in the relation between the size of the convex set X0 , the grain in the dead leaves model, and the size of the connected components of the partition of the plane provided by the dead leaves model M, the cells of M. In the following, as a notion of size of a set, we choose the set of sizes of its intersections with lines of direction , the so-called intercepts. We fix

202


, and we will sometimes omit it. Let x0 be the center of , the window in which the model has been defined. Let D be the straight line of direction 0 passing by x0 . Let Fxtess l be the repartition function of the length of the component of M \ D containing x0 . We define C1

Ftess l D C l

0 dFxtess u u

(the integral has actually finite bounds, because the model is bounded), according to the usual length biased sampling. The preceding integral is a 0 slight abuse. In fact, dFxtess /u D is a locally bounded nonnegative Radon 0 measure on (0, C1], since Fxtess l is nonincreasing, and the integral means Ftess l D C[l, C1]. Roughly, this definition means that we are interested in the length repartition of all the components of the intersection of the model with a straight line, whereas it is more likely that x0 be in a large component than in a small one. Besides, in order to avoid edge effects, we assume that x0 C X0 ² almost surely (we consider a large enough window). As we are going to see, Ftess l, is related to QK, , the probability that a segment K 2 on a line with direction and length l is included in a connected component of M (in this case K is connected, so that the probability that it is included in a visible part or a cell are the same). We write Ql, D QK, . We sometimes write Ql, Ftess l, when there is no ambiguity about the value of . Theorem 7 Let M be a dead leaves model associated with a convex X0 such that there exists a point x0 such that x0 C X0 ² almost surely. Then for any 2 [0, C1 y l x0 dFtess y, 20 Ql, D y l so that Ftess l, D

Q0 l, . Q0 0,

21

Proof Consider the following event Ey, dy: a segment contained in M \ D with length y y C dy contains x0 . The probability of Ey, dy is 0 dFxtess y. On the other hand, since M is stationary,1 the conditional probability that the segment of M \ D containing x0 also contains the whole segment [x0 l/2, x0 C l/2], knowing its length is y, is equal to the probability that a segment with length l and uniformly distributed center x 2 [y/2, y/2] be 1 M is not, strictly speaking, stationary, but we are considering only what happens inside the window .


203

wholly contained in [y/2, y/2]. Obviously, this last probability is y l/y. Since the events Ey, dy are disjoint, we conclude that C1

Ql D l

l y x0 dFtess y. y

D Ql lQ0 l and, therefore, by an easy calculaThis implies that tion using (20) that Ftess l D CQ0 l. But Ftess 0 D 1, and the theorem is proved. 0 l Fxtess

Next, we are going to formulate the relation between Ql, the probability for a segment of size l to be included in M, and the covariogram l associated with the grain X0 . We will write [0, l] for the segment of length l and direction , when is fixed. Theorem 8 Consider a dead leaves model whose grain X0 is convex. Then Ql, D Proof

22

According to Formula (11), we have Ql D

and

l, . 0, l 0 0,

EX0 [0, l] EX0 ý [0, l]

EX [0, l] D EX0 \ X0 l D l

because X is convex, and, again by convexity, EX0 ý [0, l] D EX0 C lE1 p X0 which is a directional version of Steiner formula. By Formula (18), we obtain EX0 ý [0, l] D 0 l 0 0

23

and the theorem follows. Theorem 9 Consider a dead leaves model whose grain X0 is convex. Then the size repartition Ftess l D Ftess l, of the connected visible intervals of the dead leaves model in direction is related to the size repartition FC l D FC l, of the leaves in direction by C1 h

Ftess l dl D

C1

1 2

1 C ah

FC l dl h

24

204


where aD Proof

0 0, . 0,

This is an easy consequence of Formulas (18), (21), and (22).

Corollary 4 The average size of the intercepts for the visible parts of leaves is twice as small as the for the leaves themselves, i.e., C1

ldFtess l, dl D

0

Proof

1 2

C1

ldFC l dl. 0

By Theorem 8, C1

Ftess l dl D

0

1 2

C1

FC l dl

25

0

and the result follows by integration by parts. If, in addition, we can make observations on the tessellation, then we can estimate Ftess and therefore deduce the value of a because, by (21) and (22), C1

Ftess l dl D

0

1 02 1 D D . Q0 0 2 0 00 2a

Then Formula (24) permits to deduce FC . This relation also implies that there is no invariant distribution function, i.e., no distribution of the grains such that Ftess D FC . We also get C1

C1

FC D

0

0

0 l 0 EX0 D 0 D 0 0 0 EUX0

because, by Cauchy formula, 2

E

pX0 d D EUX0

0

It seems nonetheless difficult to recover other characteristics of the dead leaves. Conversely, if we know the leaves model, then we can compute a, and, therefore, Ftess . Notice, however, that the cells of the model are not necessarily convex. Thus the connected components of the intersection of a given straightline with the tessellation are not independent: two intersections may belong to the same original leaf or even to the same cell.


205

E. The Stability of the Power Law In this section, we prove that two important quantities do follow a power law if the size distribution of objects does. The first one is the probability that two points at distance x are in the same visible part of an object, and the second one is the length distribution of intercepts, introduced in the previous section. The first of these results was proved, with a different formalism, by Ruderman [1997], who also experimentally observed the power law for the probability of two points belonging to the same object in natural images. The objects he considered were obtained by visual segmentation. We consider that an interesting quantity to study in images is the distribution of connected components of bilevels. We experimentally observed that the length distribution of intersections of those components with a line of fixed direction follows a power law (see Section III). The grain X0 is assumed to be a convex set. We assume that realizations of X0 are homothetics Xr D rX1 of a random compact X1 , according to the density function fr, and we fix . Moreover, we assume fr D 0 for r < Rmin or r > Rmax . Without loss of generality we assume Rmin D 1. Then we set l, r, D Xr l, D EXr \ Xrl, where the expectation is taken at fixed r. First of all, we remark that l, r, D EX \ r

Xrl,

D r E X1 \ X1l 2

l D r 1, , r 2

r ,

so that for any nonzero l, Rmax

X0 l, D

l, r, fr dr D l

3

Rmax /l 1/l

1

1 r 1, , flr dr. r 2

If we now assume that fr D Cr υ , i.e., that the size of objects follows a power law, we get X0 l, D l3υ Al, with Al, D

Rmax /l 1/l

1 r 2 1, , fr dr. r

206


1 Now, for l ½ 1, the previous integral is from 1 to Rmax /l, because 1, , r D 0 for r < 1. Eventually, if we suppose Rmax /l large enough and υ > 3 we get C1

Al, ³ A D

1 r 1, , fr dr r 2

1

so that X0 l, ³ Al3υ

26

with A independent of l. We also notice that if we allow objects not to be bounded, for instance, using Matheron’s original model (see Section VIII), then the previous approximation becomes an equality for l ½ 1. The probability Vl, that two points at distance l and in direction are included in the same visible part of the model is, according to Formula (11), Vl, D

EX0 fx, x C lg X0 l, D . EX0 ý fx, x C lg 2X0 0 X0 l,

Using Formula (26) we get, for large values of l (small enough compared to Rmax ), and υ > 3, Vl ³ Cl3υ . 27 If we average V over all orientations, we still have Vl ³ Cl3υ . It is then easy to show that, if each leaf is given a different color, the covariances of the model also follow a power law. This result was already obtained by Ruderman [1997]. He also experimentally observed this fact on images, using an object segmentation “by hand.” We now come to the study of Ftess l the repartition function of intercepts that was defined in Section V.D. This function has been observed to follow a power law in natural images (actually the density function is following a power law); see Section III. We recall that, following Formulas (21) and (22), Ftess l, D

X0 0 l, . X0 0 l, lX0 0 0,

Now, following the same reasoning as before, it is easy to see that if fr follows a power law Cr υ , and if υ > 3, then X0 0 l ³ cl2υ (as before either for fixed or an average in all directions). Therefore, for values of l much larger than 1 and much smaller than Rmax (hypothesis that can be removed if objects are unbounded) we get Ftess l ³ Cl2υ

28


207

which is indeed what we observe in Section III, a clue for a power law distribution of object sizes in the surrounding world. Moreover, our experiments on natural images gave values of υ slightly above 3 (the density functions have an exposent slightly above 2, see Section III.E). This emphasizes once more the fact that natural images are nearly scale invariant. Indeed, the limiting case υ D 3 correspond to a log behavior of the function V (still assuming Rmax /l is large), which, as we saw in the Section II.C.1, corresponds to the scale invariant case. F. The Size Distribution of the Relief Grains In this section we are interested in the size distribution of the grains that are completely visible after the occlusion process. More formally, those grains are the xi C Xi such that xi C Xi \ xj C Xj D ;,

i < j 0.

Following G. Matheron, those grains will be called relief grains. As before, we consider a dead leaves model on a window , with convex grains X0 . We further assume that X0 ² D0, R with probability 1, and to avoid edge effects we will suppose that the centers are uniformly picked in 2R D ý D0, 2R. Thus the result of Lemma 2 is valid for any compact K such that K D0, R D ; and K \ 6D ; which in particular is the case for any leaf that touches . Moreover, we suppose that X0 has area A and perimeter U following the density fA, U. Eventually, we assume that X0 is isotropic, which will enable us to apply the Steiner formula. Theorem 10 For a dead leaves model satisfying the preceding assumptions, thus with grains X0 having area A and perimeter U with density fA, U, the relief grains touching follow the density fr A, U D C

fA, U EUX0 U EX0 C A C 2

where C is a normalizing constant. Proof The probability that Xi is centered in , has area between A and A C dA, has perimeter between U and U C dU, and is not touched by any of the following leaves is EXi ý X0 jXi has area A and perimeter U i fA, UdA dU 1 2R 2R

208


which by the Steiner formula is fA, UdA dU 2R

1

UUX0 2 2R

i

EX0 C A C

.

We sum this relation over all i’s and get the density function of the relief grains up to a normalizing constant, which proves the theorem. The results about relief grains are computable because we know how to get the probability for a compact to be avoided by a leaf. On the other hand, we don’t know how to get the probability for a compact to be completely overlapped by the subsequent leaves, because this may be achieved in several stages, and thus we don’t know how to compute the proportion of leaves that disappear completely. G. The Average Area After Occlusion: Numerical Experiments As we saw on the three examples of Figures 10–12, both the shape of the grain and its size distribution have a strong influence on the visual flavour of the dead leaves model. In particular, the isoperimetric ratio of the grain seems to be of great importance. Contrarily to the average intercept, which is reduced to its half by occlusion whatever the parameters of the grain are, the average area after occlusion seems to follow more complicated rules. This quantity appeared to us as essential, since it gives a partial answer to the question: “how much do we see of objects?” In this section, we present some experimental results concerning the ratio between the average area of the the grain X0 , and the same quantity for the cells of the dead leaves model. We also checked, in a sense, the influence of discretization in our simulations by checking that the average intercept length for the cells was twice as small as the one for the grain, X0 . This experimental quantity is also of interest when the shape is not convex, in which case we do not have a theoretical result. We denote average area of the cells of M RS D average area of X0 and RL D

average length of intercepts for the cells of M . average length of intercepts for X0

The simulations of the model consist in sequentially superposing discrete shapes whose size follows a certain probability law and whose centers are uniformly drawn in the image, numerous enough to cover all of it. We worked on 900 ð 900 images. The averages for the grain X0 were computed in the


209

discrete case. In order to reduce boundary influence in the computations of the ratios, we take into account only cells falling in a small enough subimage. We studied only isotropic models (except for the horizontal squares) and compute segments only in the horizontal direction. Moreover, each experiment consists of 50 such images, over which the quantity under study is averaged. We present the results in the following cases: ž ž ž ž ž ž ž ž

Disks with radius of 20 pixels (diameter of 41 pixels) Disks with radius uniformly distributed between 15 and 40 pixels Disks with radius exponentially distributed between 5 and 50 pixels Squares with sides of 41 pixels parallel to the border of the image Squares with sides of 41 pixels, uniformly rotated Elongated rectangles of size 108 ð 12, uniformly rotated Arrows of fixed size, uniformly rotated T’s, uniformly rotated

In Table 16 we present RL and RS for those experiments. We notice that the preceding experimental results confirm the expected value of the ratio RL D 0.5 for the average length of segments and, therefore, the validity of our protocol. Moreover, it seems that the result remains true for nonconvex shapes. Concerning the average area ratio RS , and contrarily to the constant value of RL , the values are strongly dependent on two parameters: the size probability law and the shape of the grain. We basically observe two phenomena. The first one is that at constant shape for the grain, the more numerous the small objects, the bigger RS . This is illustrated in the case of disks with three different size laws and has been confirmed by other experiments. Secondly, the value of the isoperimetric constant for the grains seems to have TABLE 16 RL IN VARIOUS CASES. RS IS THE RATIO OF THE AVERAGE AREA AFTER AND BEFORE OCCLUSION. RL IS THE RATIO OF THE AVERAGE INTERCEPT LENGTH AFTER AND BEFORE OCCLUSION (PREDICTED BY THE THEORY TO BE 0.5).

RS

AND

Experiment Disks, same radius Disks, uniform radius Disks, exponential radius Horizontal squares Rotated squares Elongated rectangles Arrows T’s

RS

RL

0.257 0.283 0.339 0.275 0.23 0.08 0.10 0.061

0.51 0.51 0.52 0.51 0.51 0.53 0.52 0.54

210


a strong influence on RS . It seems that elongated or nonconvex shapes lead to a small value of RS . We claim that there must be a link between the decay of the average area by occlusion, and the decay of the average perimeter. This interaction between area and perimeter has already been observed in the case of the relief grains studied in the preceding section.

VI. A SHORT REVIEW OF TEXTURE SYNTHESIS BY MATHEMATICAL METHODS Research in texture analysis and synthesis has several aims. One of them, of a wide commercial relevance, is the ability to reproduce any visual aspect of the surrounding world by fast computational methods. A second aim is to understand how we perceive textures and discriminate them. This is the view point of psychophysics [Julesz, 1981,1986; Beck, 1983]. A third aim is the creation of mathematical models general enough to be able to synthesize both natural and new, artificial textures. All three aspects are closely related. The first one, however, seems to have of late adopted the “lazy solution,” which consists in storing in large databases all textures of interest for animation. The main problem of texture synthesis then becomes a mapping problem: how to map a given plane texture on the 3D surface of an object. We shall not develop this aspect here and refer to Heckbert [1986]. From the animation viewpoint, it must be emphasized that the superposition of two very simple synthesis methods is widely used: the normal perturbation method (explained in Section VI.A) followed by a rendering algorithm using an illumination model (see Kajiya [1986]). However, all textures obtained this way end up looking very much alike. Among the mathematical models that are candidates for being general texture synthesis models, some are, in fact, too specific in structure to permit a synthesis of all or even many textures, but they may give good results in some particular instances. This is the case with the models we shall first briefly review: fractional Brownian motion and the perturbation of normal vectors permit creation of rough surfaces; iterated function systems create ragged, textured self-similar shapes; solid texture methods create marblelike or woodlike textures; reaction-diffusion equations create regular patterns, appearing for instance on the skin of some animals and shells; spot noise methods create clouds or tissues, etc. These methods are generic and can also create textures that do not refer to any existing one. This is not a drawback at all! There are also much more specific methods simulating specified surfaces, such as particle systems for fires and plants [Reeves, 1983], physically based models for water surface [Fournier and Reeves, 1986], and tissues [Weil, 1986], etc.


211

We then present the only two really general models: Markov random fields and multiscale models. The first one models any texture by pointwise interactions (between pixels). This is a strong structural drawback: a privilege is given to the pixel scale. Long-distance interactions are not well modeled and subpixel interactions are not allowed. Thus Markov random fields, as general a formalism as they are, make it difficult to analyze and synthesize textures that have several significant scales. This probably explains the computational cost in identifying the Gibbs potentials of a given texture. Notice that in discrete grid models, even a good rotation invariance requires many nonlocal interactions between pixels. In fact, a recent paper by Heeger and Bergen [1995] seems to be a major breakthrough in texture analysis/synthesis. This progress is somewhat in agreement and inspired by psychophysical experiments showing the relevance for texture discrimination of the responses of textures to filter banks. See Bergen and Adelson [1988], Chubb and Landy [1991], and Malik and Perona [1990]. The Heeger and Bergen method and its generalizations [Simoncelli and Portilla, 1998; De Bonet and Viola, 1997] seem to be able to reproduce fairly any given texture in reasonable computational cost. At the end of the section we also present the Zhu and Mumford model, which merges the filter bank response method with a Gibbs field formalism.

A. The Normal Vectors Perturbation Method This is one of the simplest and most used methods for texturing a surface. It was introduced in Blinn [1978]. The idea is to start from a surface and then slightly modify the normal vectors to this surface before computing the lighting. To do so, we consider a small perturbation of the surface in the direction of the normal and then compute the effect on the normal vectors. We then consider the same initial surface, but with modified normals for the rendering. In the case where the surface is known through a parametric E equation Mu, v, with normal vectors n E u, v, and the deformed surface is E nu, v, then up to the first order in the deformation the new Mu, v C Fu, vE normals are E dF E dM dM dF n EC 3 n E3 n EC . du dv du dv Most of the time, F consists of a table of interpolated and filtered values (“bump map”). An easy way is to give random values to the elements of the table to be interpolated. This method is probably the most used in image synthesis, in conjunction with texture mapping. It is fast and avoids the storage of complicated surfaces, thus saving memory. The fact that only the normals and not the surface itself are modified is also one of the drawbacks of the

212


FIGURE 13. Normals are perturbed by the interpolation of a white noise.

method, because it creates artifacts at the junction of surfaces with different textures. We give a very simple example of the method in Figure 13, where we use a white noise to perturb the normals. The main idea of this texture model is to transform a smooth surface into a more irregular one, that looks more realistic. This is one of the leitmotivs of computer graphics, because all objects whose physical boundaries are modeled by primitives having a simple mathematical expression, if appropriate for man-made objects, look far too regular to mimic the appearance of natural objects. We here refer to constructive geometry (objects are modeled by a small number of geometric primitives linked together by logical operations), interpolation methods (starting from some points of an object, it is reconstructed by means of splines, Bezier curves, etc.), and elastically deformable models (objects are modeled by deformable surfaces subject to external constraints; see Terzopoulos et al. [1987]). Therefore, Fournier, Fussel, and Carpenter [1982] introduced a variation on constructive geometry to model such irregular objects as mountains, and Szeliski and Terzopoulos [1989] introduced a variation on deformable surfaces (thus also on interpolation methods) that permits the modeling of irregular objects by adding perturbations to the energies constraining the surface. It is impossible here to give a detailed account of all those methods, but the perturbation idea will often be encountered in the following.

B. Solid Textures The basic idea of solid texturing [Perlin, 1985; Peachey, 1985] is the following: each point of the three-dimensional Euclidean space is given a numerical value (or an array of values) and then a surface is textured by looking at the values corresponding to its boundary in this space. Those values can be used


213

to determine the gray level of a surface point, its color or its normal, that can then be used for rendering. The method is free of the distortion problems encountered when mapping textures to three-dimensional objects. The values at each point are determined by procedural methods, mainly variations around noises having good invariance properties, by translation, rotation, and possibly scaling (see Perlin [1985] and Peachey [1985]). Let us define the Noise function introduced in Perlin [1985]. At each point x of aZ2 , a rough grid where a is an integer larger than 1, Noise(x) is computed as a random value, and the values elsewhere, in the fine grid aZ2 , are obtained by regular interpolation (e.g., by splines) of the values in the rough grid. This function can then be used for a variety of texture synthesis. We can plainly use the value Noise(x) to determine the gray levels of the surface of the object. We can also use the values of the noise function to determine intensity or direction of vectors that will enable us to perturb the normals as in the Blinn model (Section VI.A). There are many more elaborate ways to use the 3-D information, and we mention just one of them. If we let each point x of IR3 have the value sinx, where x is the first coordinate of x, and use an appropriate look-up table (a table in which real values are related to colors), we can synthesize a texture looking like marble, but far too regular to be realistic. The idea is then to define a new function Turbulence from IR3 to IR, defined by Turbulencex D

k½0

Noise

x 2k 2k

(noises with lower and lower frequencies are superposed). The marble texture is then obtained by applying a look-up table to the values sinxCTurbulencex instead of sinx. Using similar methods, it is also possible to construct textures with approximately 1/f power spectrum. The results are impressive, and their generation is not too memory demanding, because they are synthesized by procedural methods. We show an example of such a marble solid texture in Figure 14. A variation on these methods, introduced in Perlin [1989], is the so called hypertexture model. It is a way to represent objects for which the texture is the result of an intricate 3-D shape, such as hair or turbulent fluids. An object is represented by a function D from IR3 into [0, 1], taking values 1 where the object is hard and values in (0, 1) in a region where the object is deformable. The soft part is deformed using various functions to give a textured aspect to the object. Since the first articles about solid textures, they have been widely used together with many variations. For instance, a recent article introduces new basis functions based on Voronoi cells [Worley, 1996].

214


FIGURE 14. Solid texture model: marble.

C. Reaction-diffusion Textures A texture is modeled as the result of local and nonlinear interactions, representing the diffusion of chemics (morphogens) in a medium, and the reactions of creation and vanishing of those chemics. This type of model was initially proposed by A. Turing for the purpose of understanding the patterns on the skins of animals. The interactions lead to differential equations. If CX, t stands for the concentration of morphogens at X, t (to which a texture control function will be associated), then the simplest model is: ∂C D a2 1C bC C R. ∂t Here a2 1C is the diffusion term, bC is the term for the disappearance of the morphogens in time, and R is an arbitrary function representing the creation of morphogens. In the case of anisotropic patterns (as with zebra’s stripes, for instance), the term a2 1C is replaced by an anisotropic diffusion operator a11

∂2 C ∂2 C ∂2 C C a22 2 . C 2a12 2 ∂x ∂x∂y ∂y

The coefficients may also depend on the location. For the purpose of texture synthesis, the function C is associated with a color map, or two such functions may be used simultaneously with a threshold in order to produce bicolor textures. We present a simple example with isotropic diffusion in Figure 15. Such textures are quite realistic, but they are limited to a restricted class. Two advantages of the method are the possibility to paste texture samples by applying limit conditions, but, also the possibility of directly synthesizing


215

FIGURE 15. Isotropic reaction-diffusion texture.

texture on surfaces that are not plane by modifying the equation, as explained in Turk [1991]. D. Spot Noise Textures This section presents a linear model for textures, based on the addition of translations of characteristic functions of sets. The method has been introduced by Van Wijk [1991] for the purpose of scientific visualization. We define a spot noise by fx D ai hx xi , where ai and xi are random variables, h is the characteristic function of a simple shape (disk, ellipse, . . .), x is a point, and f is the gray value in this point. We may have to rescale f to stay between minimum and maximum values. If the ai ’s are centered and if, on average, there are occurrences of Xi ’s by unit of surface, then one gets Pf ω D hai2 ijHωj2 where Pf is the power spectrum of f, and H the Fourier transform of h. This is of use in view of the analysis/synthesis of textures, but unfortunately it is not that easy to choose h. More precisely, if we know the power spectrum of a texture on a disk centered on the origin, it is not granted that the inverse Fourier transform of an approximation of that power spectrum will be positive, and it is even more difficult to find a simple shape whose transform will match the spectrum. The synthesis is achieved in the frequency domain. For instance, if we suppose that the ai ’s are constant and the xi ’s are uniformly distributed, we compute the discrete Fourier transform of the shape h and then uniformly randomize the phase at each point. We then come back to the image domain. This (up to the discretization) amounts to superposing copies of h all over

216


FIGURE 16. Spot noise: on the left the generating shape, on the right, the texture obtained by randomizing the phases.

the image and then normalizing the gray levels. We give some examples of synthesis in Figures 16 and 17. The control of the texture by the shape characteristics is far from being obvious, but we have the following heuristics: a symmetry of the shape will appear in the texture, the larger the shape the larger the correlations in the image. This method agrees with the hypothesis that a microtexture is characterized by its power spectrum only and that the phases of the Fourier transform are unimportant, a hypothesis that is easily checked by randomizing the phases of a microtexture. E. Fractal Models 1. Fractional Brownian Motion A one-dimensional fractional Brownian motion (fBm) is a stochastic process VH from IR to IR, with 0 < H < 1, such that its increments VH t1 VH t2 have a Gaussian distribution, centered and with variance proportional to t2 t1 2H . The case H D 12 is the case of the usual Brownian motion, which is the integral of a Gaussian white noise. If H > 12 , the increments are positively correlated, whereas they are negatively correlated if H < 12 , and this has a strong repercussion on the aspect of the graph of V. An important property of


217

FIGURE 17. Spot noise: generating shape on the left, generated texture on the right.

VH is the statistical self-affinity of its increments. Indeed VH t0 VH t0 C t and r 2H VH t0 VH t0 C rt have the same distribution. The realizations of the graphs of such processes are fractal in the sense that the Hausdorff dimension of the graph of VH (or its box-counting dimension, which in this case is the same) is 2 H with probability 1. We here stress the difference with a usual Brownian path for which the dimension is 2. In dimensions larger than 1 it is possible to define processes VH,n from IRn to IR, with increments from x1 to x2 proportional to jjx1 x2 jj2H , jj Ð jj being the Euclidean norm. The dimension of such a process is n C 1 H. Other sets of interest are the zero-sets of fBm. The zero-set of VH,n is the subset of IRn1 , fx/VH,n x D 0g. This set has dimension n H and, contrarily to fBm, is statistically self-similar; i.e., VH x0 VH x0 C x and r 1 VH x0 VH x0 C rx have the same distribution. Those processes do exhibit the kind of irregularity and relationships between scales one would expect from many natural objects, together with randomness. For instance, to represent a coast, the zero-set of a two-dimensional fBm will prove useful, because it provides a close curve, which is self-similar in agreement with the shape of a real coast line, with a dimension chosen through H. On the other hand, if one wants to represent a shape for which the closer we look the more regular it seems (as for some mountains), then a twodimensional fBm with H < 12 can be convenient. There is also another way to

218


use those processes, not taking them as a geometrical model, but as a model for the color levels of a texture. In Jardine [1993] a texture is synthesized by approximating a two-dimensional fBm (by the midpoints displacement method presented in the next section) and then associating with each value of the process a color value using a principal component analysis. More generally, many processes similar to fBm are widely used in computer graphics to perturb the normals or the color maps. The following two sections present some of the algorithms for the generation of approximates of fBm. A good reference concerning those algorithms is Saupe [1988]. 2. Midpoints Displacement Method The midpoints displacement method enables us to construct a pseudo fBm starting from a polygonal representation of a shape. This explains its success, considering the importance of geometric modeling in computer graphics. It was introduced for the purpose of computer graphics by Fournier, Fussel, and Carpenter [1982] and also gained some success being used as a way to produce mountains in the movie Star Trek II. In dimension 1, the idea is to start from a segment, then to divide it in two and to displace its center by a random vector, then to cut the two resulting segments in two and to displace their centers by a smaller quantity, etc. The amplitudes of the displacements at each iteration follow the proportionality law between Varxt1 xt2 and jt1 t2 j2H . The algorithm is the following: ž In IR2 equipped with a Euclidean basis x, t, we start from a line segment S0 whose endpoints have coordinates (0, T) and (0, T0 ), respectively, T and T0 being two random numbers following a centered normalized Gaussian law. ž Si are sets of segments ski recursively defined in the following way: for i D 0 to N, for each ski , we displace its center vertically by a random variable following a centered Gaussian law of variance 2iC1H . The set SiC1 then consists of the line segments thus obtained. The justification of this algorithm comes from the following formula, where BH is an fBm and u 2 [0, 1]: EBH ujBH 0 D 0, BH 1 D 1 D 12 u2H C 1 ju 1j2H so that it is equal to 12 if u D 12 . This method can then be applied to a triangle, a widely used primitive in geometrical modeling. We displace the middle of the sides as before and then


219

link them to the center of mass of the triangle, therefore obtaining three new triangles that can be modified, and so on. It is also possible to use squares on a grid, to use fictive centers of those squares, and to keep only points produced every two iterations. A serious problem arises with such a simulation, as stressed in Mandelbrot [1988]: we lose the stationarity of the fBm. Intuitively, this is the consequence of the immobility of the vertices of the triangles, which leads to creases. A solution consists in adding at each step a random perturbation not only to the centers of the line segments, but also to their endpoints. At each step, a random quantity is added to centers and endpoints, following the same law as the one used to displace centers in the previous algorithm. 3. The Spectral Synthesis Method The spectral synthesis method is based on the fact that an fBm with parameter H has a power spectrum (the square of the modulus of its Fourier transform) given by Sf D 1/jjfjj2HC2 (we assume here that the process is defined on a two-dimensional space). We therefore can build a Fourier decomposition whose coefficients are random variables with Ejai,j j2 D

i2

1 C j2 ˛

with ˛ D H C 1. Care must be taken with the symmetry conditions of the Fourier transform. The major drawback of the method is that it is not possible to adjust the data to some boundary conditions. We mention that if we choose the ai,j to be normally distributed with mean 0 and variance given by the preceding formula with ˛ D 2, we simulate the unique scale invariant Gaussian process [Mumford, Zhu, and Gidas, 1997]. Indeed fBms get closer to scale invariance as H tends to 0. 4. Iterated Function Systems This model represents an image as the attractor of a set of contracting affine functions and permits us to define a quite complicated object with few parameters. Therefore, the main application of this model, which is not presented here, is image compression. Some references about the use of iterated function schemes (IFS) in image synthesis are Demko, Hodges, and Naylor [1985], Barnsley et al. [1988], Barnsley [1988], and Barnsley, Elton, and Hardin [1989]. An IFS is a set (Ai , pi ), where the Ai are affine functions from IR2 to itself and the pi are strictly positive numbers such that pi D 1. Moreover, it is assumed that the set of transformations is contracting, i.e., each Ai has a

220


Lipschitz constant si such that p

si i < 1.

29

i

It is enough but not necessary that each Ai be contracting. Under this condition, we can associate with the IFS a unique compact set A and a unique measure (supported by A), P. In the case where there is only one function Ai , A is the fixed point of the application Ai , which is contracting from the set of compact sets of IR2 into itself for the Hausdorff distance d. The case where each Ai is contracting is the same. In fact, the fixed-point theorem holds for the function Ui Ai , which is also contracting on compact sets for d. The measure P has the following interpretation: if we apply the functions Ai to a point x with probability pi , then do the same with the point Ai x, and so on, the sequence of points thus obtained will visit all of A, visiting a region R with a frequency proportional to PR. The set is often, but not necessarily, a fractal set. For instance if we consider (in IR instead of IR2 but with the same definitions), A1 x D x/3, A2 x D x/3 C 2/3, p1 D p2 D 1/2, then the set A is the dyadic Cantor set, and the measure P is uniformly supported by A. To synthesize a digital image from (A, P), the set A is pixelized, and for each pixel R a gray level (or color array) proportional to PR is given. But of course we do not know (A, P), but Ai , pi . Thus: ž We consider a pixel grid. ž We start from a point x0 of A, (at least one of the Ai is contracting; otherwise Condition (29) is not true, and we can show that its fixed point belongs to A). ž We pick an Ai with probability pi and increment by 1 the gray level value of Ai x0 . ž We repeat the preceding operation (starting from the previous point) many times (many compared to the size of the grid). ž We rescale the obtained gray level values between 0 and 255. An ergodicity theorem ensures the validity of this method. Of course, we don’t know so far how to choose the Ai ’s in order to design an a priori fixed shape. For this task, the following result, the so-called collage theorem, is essential. We consider T a compact set. If dT, [i Ai T < ε then dT, A
m < 12 , be as defined in Theorem 2 and let

1 2

tF

E D K p

E

e2iKtÐEp ft dt

tI

Then we have as an equivalent definition of CK the following: CK D Ł K

and

271

RECONSTRUCTION OF NMR IMAGING DATA

Before proving the proposition, note that this is just the expression that K is the point spread function associated with CK when it is used to reconstruct an ideal NMR image. It is not immediately obvious that the rigorous definition of CK given in Theorem 2 is equivalent to an unrestricted convolution so it needs to be explicitly proved. As noted above, for 2 2 \ Lm2 0 2 ,

Proof

E D CK p

tF

E

E Kt e2iKtÐEp ft dt

tI

Because ² L , we can use Fubini’s theorem to write 2

E CK p tF

D tI

D D D

1

1

1

1

1

1

a, b e2iaKx tCbKy t da db e2ixKx tCyKy t ft dt

1

1

1

1

1

1

1

1

1

1

2

1

tF

a, b

tF

a, b

e2iKx t axCKy t by ft dt

da db

tI

e2iKx t xaCKy t yb ft dt

da db

tI tF

a, b

K x a, y bft dt

da db

tI

Thus CK D Ł K for 2 2 . Then, because 2 \ Lm2 0 2 is dense in Lm2 0 2 and convolution with K is bounded from Lm2 0 2 to Lm2 2 , the Hahn-Banach theorem may be used to conclude that CK D Ł K for all 2 Lm2 0 2 . Proposition 4 Let A be a bounded measurable subset of 2 . Let Em0 : L 2 A ! Lm2 0 2 , m0 > 0, be the extension mapping given by fp E if p E 2A Em0 fp E D 0 if p E 62 A Let m : Lm2 2 ! L 2 A, m < 0, be the restriction mapping given by the following: E D fp E if p E 2A m fp Then Em0 and m are continuous, and, as well, we have the following continuous imbeddings: Lm2 0 2 ! L 2 2 ! Lm2 2 when m0 > 0 and m < 0.

272 Proof

GORDON E. SARTY

To see that Em0 : L 2 A ! Lm2 0 2 , m0 > 0, is continuous, compute: jjEm0 fjj2L2

m0

2

0

D

jEm0 fp E j2 1 C jjp E jj2 m dE p

2

0

jfp E j2 1 C jjpjj E 2 m dE p

D A

sup1 C jjpjj E 2 m

0

jfpj E 2 dE p

p E 2A

A

2 m0

E jj D sup1 C jjp p E 2A

jjfjj2L2 A

To see that Lm2 0 2 , m0 > 0, is continuously embedded in L 2 2 , compute jjfjj2L2 2 D

jfp E j2 dE p 2 0

jfp E j2 1 C jjp E jj2 m dE p

2

D

jjfjj2L2 2 m0

To see that L 2 2 is continuously embedded in Lm2 2 , m < 0, compute jjfjj2Lm2 2 D

jfp E j2 1 C jjpjj E 2 m dE p 2

jfp E j2 dE p

2

D jjfjj2L2 2 To see that jj

m

2 m fjjL 2 A

: Lm2 2 ! L 2 A, m < 0, is continuous, compute D

j A

D 2

2

D

E j m fp

2

dE p

jA pf E p E j2 dE p jA pf E p E j2

1 infqE2A 1 C jjEqjj2 m

1 C jjp E jj2 m dE p infqE2A 1 C jjEqjj2 m 2

jA pf E pj E 2 1 C jjpjj E 2 m dE p


D

1 infqE2A 1 C jjEqjj2 m

273

jfp E j2 1 C jjpjj E 2 m dE p 2

1 jjfjj2Lm2 2 infqE2A 1 C jjEqjj2 m

E : [tI, tF ] ! Theorem 5 Let A be a bounded measurable subset of 2 and let K 2 be an injective C1 map that satisfies

c12
12 , then CK D

m

1 K TK

Em0 : L 2 A ! L 2 A

is continuous, where, as described in Proposition 4, Em0 : L 2 A ! Lm2 0 2 is the extension operator, m : Lm2 0 2 ! L 2 A is the restriction operator, and 1 are as defined in Theorem 2. K , TK , Proof

This theorem follows directly from Theorem 2 and Proposition 4.

The map used to describe the ideal MRI signal, the MRI map, may therefore be defined as 53 M D TK Em0 : L 2 A ! L 2 K, and the curve reconstruction operator, as RK D

m

1 K

: L 2 K, ! L 2 A

54

with the assurance that both maps are continuous and linear. C. Band-pass Operators as Integrals of Curve Band-pass Operators Next we show how to integrate the operators CK (which amounts to continuously adding the results of a continuum of MRI measurements) to obtain a band-limiting operator B. Then, if K circles the origin, we show how integrating the associated operators CK can lead to a resolution of the identity on L 2 A. Theorem 6 Let ϕE : [tI , tF ] ð [W1 , W2 ] ! 2 be a C1 map such that the restriction of ϕE to tI , tF ð[W1 ,W2 ] is a C1 diffeomorphism onto D ϕE tI , tF ð [W1 , W2 ]. Denote the components of ϕE as follows: ϕE t, ˇ D kx t, ˇ,

274

GORDON E. SARTY

ky t, ˇ. Suppose also that ϕE satisfies:

c1 ˇ
12 . Let CKˇ : L 2 A ! L 2 A be given by CK ˇ D 1 Kˇ

where

m

1 Kˇ TKˇ

Em0

: L 2 Kˇ , ! Lm2 2 , where dt D jJϕE t, ˇj dt, is given by 1 E Kˇ Sp

tF

D

E

Ste2ipE ÐKˇ t jJϕE t, ˇj dt

tI

and TKˇ : Wm0 2 ! L 2 Kˇ , is as given in Theorem 2, : Lm2 0 2 ! Wm0 2 is the (distributional) Fourier transform, and m : Lm2 0 2 ! L 2 A, 2 2 Em0 : L 2 A ! Lm 0 are the restriction and extension operators described in Proposition 4. Let B : L 2 A ! L 2 A be given by B D

m

1

Em0

where 1 : Wm0 2 ! Lm2 2 is the inverse Fourier transform. W Then B D W12 CKˇ dˇ strongly. Proof Let 2 L 2 A; then 2 L 1 A, so that Em0 2 L 1 2 . Also, since ϕE is a C1 diffeomorphism, the function j detDE ϕj jJϕE j 2 L 1 [tI , tF ] ð [W1 , W2 ] so that Fubini’s theorem may be used to justify the fourth equality in the following string of equalities: 1

Em0 x, y Em0 kx , ky e2ixkx Cyky dkx dky

D

D

W2 W1

D

W2 W1

tF tI tF tI

Em0 kx t, ˇ, ky t, ˇ e2ixkx t,ˇCyky t,ˇ jJϕE t, ˇj dt dˇ

1

1

1

1

Em0 a, b e2ikx t,ˇaCky t,ˇb da db

ð e2ixkx t,ˇCyky t,ˇ jJϕE t, ˇj dt dˇ

275

RECONSTRUCTION OF NMR IMAGING DATA 1

W2

D

1

W1

tF

1 1

e2ikx t,ˇaxCky t,ˇby

E a, b m0

tI

ð jJϕE t, ˇj dt da db dˇ 1

W2

D

1

W1

1 1

tF

Em0 a, b

e2iKˇx t,ˇaxCKˇy t,ˇby

tI

ð jJϕE t, ˇj dt da db dˇ W2

D

W1

1

1

1

1

W2

D

A

W1 W2

D

Em0 a, b Kˇ x a, y b da db

1 Kˇ TKˇ

W1

Em0 a, b Kˇ x a, y b da db

dˇ

dˇ

Em0 x, y dˇ

where tF

Kˇ p E D

E

e2iKˇ tÐEp jJϕE t, ˇj dt

tI

and the last equality (in the above string of equalities) is justified by ProposiW 1 E D W12 ˇK TˇK Em0 p E tion 3. Thus it has been shown that 1 Em0 p dˇ, and so

m

1

Em0 p E D

W2 m W1

1 Kˇ TKˇ

Em0 p E dˇ

because W2 m W1

1 Kˇ TKˇ

Em0 p E dˇ D

W2 m W1

1 Kˇ TKˇ

Em0 p E dˇ

is clear upon evaluating m on each side. W To see that W12 CKˇ dˇ is a well-defined Riemann integral, it needs to shown that ˇ 7! CKˇ is continuous with respect to the strong operator topology. This will be true if jjCKˇ CKˇ0 jjL2 A ! 0 for all 2 L 2 A whenever

276

GORDON E. SARTY

ˇ ! ˇ0 in [W1 , W2 ]. But, tF

CKˇ p E D

E ˇ t e2iKE ˇ tÐEp jJϕE t, ˇj dt E1 Ip K

tI tF

D

E1 Ip E ϕt, ˇ e2iEϕt,ˇÐEp jJϕE t, ˇj dt

tI

and, because Em0 has compact support for all 2 L 2 A, Em0 is a continE is continuous uous function, as are ϕE and jJϕE j, it is seen that ˇ 7! CKˇ p and so it is clear that jjCKˇ CKˇ0 jjL2 A ! 0 for all 2 L 2 A whenever ˇ ! ˇ0 in [W1 , W2 ]. From the definition of the Riemann integral as a generalized limit of sums and from the principle of uniform boundedness, it is true that

W2

W2

CKˇ dˇ D

W1

CKˇ dˇ W1

Because it has been shown that

m

1

Em0 x, y D

for each x, y 2 A, it can be concluded that in L A, where 2

that

m

1

W2 W1

m

1

W2 W1

CKˇ x, y dˇ

Em0 D

W2 W1

CKˇ dˇ

CKˇ dˇ is as defined above. Finally, it can be concluded

Em0 D

W2 W1

CKˇ dˇ as operators on L 2 A.

D. A Resolution of the Identity for Paley-Wiener Spaces Theorem 7 Let R D fkEu 2 2 j k ½ 0, uE 6D 0E fixedg be a ray in 2 and let E : [tI , tF ] ! 2 be a C1 map such that ϕE : [tI , tF ] ð 0 < c1 < c2 < 1. Let K 2 E E satisfies: 0, 1 ! is given by ϕE t, r D r Kt where K

c1
12 . Define the set rK D r K[t Let CrK : L 2 A ! L 2 A be given by CrK D

m

1 rK TrK

Em0

277


where

1 rK

is given by 1 E rK Sp

tF

D

E

Ste2ipE Ðr Kt jJϕE t, rj dt

tI

and TrK : Wm0 2 ! L 2 rK, r , where dr t D jJϕE t, rj dt, is as given in Theorem 2, : Lm2 0 2 ! Wm0 2 is the Fourier transform and, m : Lm2 2 ! L 2 A, Em0 : L 2 A ! Lm2 0 2 are the restriction and extension operators described in Proposition 4. Let B : L 2 A ! L 2 A be given by B D

1

m

Em0

where D ϕE [tI , tF ] ð [R1 , R2 ] for 0 < R1 < R2 and where Lm2 2 is the inverse Fourier transform. Then (a) B D

R2 R1

1

: Wm 2 !

CrK dr strongly.

(b) Also

1

D

CrK dr 0

strongly, where is the identity on L 2 A and 1

CrK dr s-lim s-lim

R2 !1

R1 !0

0

R2

CrK dr R1

Proof Part (a) follows directly from Theorem 6. To establish (b), note that each statement in the following string of equalities is true when evaluated at each 2 L 2 A and that the sequences implied by these statements all converge in L 2 A. Thus the principle of uniform boundedness allows7 D

m

D

m

D

m

1

Em0

1

2 Em0 1 s-lim s-lim ϕE tI ,tF ð[R1 ,R2 ]

D s-lim s-lim R1 !0

7

R2 !1

R1 !0

R2 !1

m

1

Em0

ϕE tI ,tF ð[R1 ,R2 ] Em0

The notation s-lim refers to convergence in the strong operator topology [Yosida, 1980].

278

GORDON E. SARTY

D s-lim s-lim BϕE tI ,tF ð[R1 ,R2 ] R1 !0

R2 !1

R2

D s-lim s-lim R1 !0

R2 !1

CrK dr R1

1

D

CrK dr 0

Theorem 7 uses the polar radial coordinate as the coordinate ˇ to compleE An immediate generalization ment the natural coordinate t given through K. 1 comes from any C map of [tI , tF ] ð 0, 1 to 2 that restricts to a diffeomorphism of tI , tF ð 0, 1 onto 2 nM, where M is a set of measure zero that satisfies

c1 ˇ
Mε , jjPN,K m PN,K jjLp A ε/3 and jjCK m CK jjLp A ε/3 from Equations (128) and (129). Furthermore, because of the uniform boundedness of PN,K , there exists Nε such that if N > Nε , jjPN,K m CK m jjLp A ε/3 from Equation (127), the preceding estimates remain valid for m > Mε . Therefore, jjPN,K CK jjLp A D jjPN,K PN,K m C PN,K m CK m C CK m CK jjLp A jjPN,K PN,K m jjLp A C jjPN,K m CK m jjLp A C jjCK m CK jjLp A ε This shows that CK D s-lim PN,K N!1

130

on L p A. In a similar manner it can be shown that as a mapping from L 1 [tI , tF ] to p L A, 131 RK D s-lim RN N!1

Next, we present a theorem about the Riemann sum approximation of the curve band-pass operator integration of Theorem 6. Theorem 20 Let ϕE : [tI , tF ] ð [W1 , W2 ] ! 2 be a C1 map such that the restriction of ϕE to tI , tF ð [W1 , W2 ] is a diffeomorphism onto D ϕE tI , tF ð


311

E ˇ : [tI , tF ] ! 2 be defined by K E ˇ t D ϕE t, ˇ. Let A be [W1 , W2 ]. Let K 2 E D a bounded measurable subset of and let BM be defined by BM p 6M E 1ˇ, where 1ˇ D W2 W1 /M and mD1 Cm p tF

E D Cm p

E

E ˇm t e2iEpÐKˇm t jJϕE t, ˇm j dt O K

tI

where ˇm 2 [W1 C m 11ˇ, W1 C m1ˇ]. Let B : L 2 A ! L 2 A be given by B D R 1 E1 I2 , where is the Fourier transform on L 1 2 and R, E1 , and I2 are restriction (from L 1 2 to L 2 A), extension (from L 1 to L 2 2 ), and inclusion (from L 2 A to L 1 A) operators, respectively. Then B p E D limM!1 BM p E . Proof

It follows from Theorem 6 that ˇ 7! Pˇ p E is continuous, where tF

E D Pˇ p

E

E ˇ t e2iEpÐKˇ t jJϕE t, ˇj dt O K

tI

so the result follows from the Riemann integration theorem. The next theorem shows how properly set-up, sampled, multiple-trajectory MRI data acquisitions may be combined to yield a nice approximation for the associated band-pass operator. Theorem 21 Let ϕE : [tI , tF ] ð [W1 , W2 ] ! 2 be a C1 map such that the restriction of ϕE to tI , tF ð [W1 , W2 ] is a C1 diffeomorphism onto D E ˇ : [tI , tF ] ! 2 be defined by K E ˇ t D ϕE t, ˇ. ϕE tI , tF ð [W1 , W2 ]. Let K 2 Let A be a bounded measurable subset of and let PNM be defined by M

PNM p E D

PmN p E 1ˇ

132

mD1

where 1ˇ D W2 W1 /M and N

PmN p E D

E

E ˇm tn e2iEpÐKˇm tn jJϕE tn , ˇm j 1t O K

nD1

where ˇm 2 [W1 C m 11ˇ, W1 C m1ˇ], 1t D tF tI /N, and tn 2 [tI C n 11t, tI C n1t]. Let B : L 2 A ! L 2 A be given by B D R

1

E1 I2

312

GORDON E. SARTY

where is the Fourier transform on L 1 2 and R, E1 , and I2 are restriction, extension, and inclusion operators. Then, for 2 L 2 A, E D lim lim PNM p E B p N!1 M!1

133

Define M

N

E D NM p

E

e2iEpÐKtn jJϕE tn , ˇm j 1t

mD1 nD1

Then PNM , as defined by Equation (132), can be written as E D PNM p

A

Eq NM p E qE dEq

134

Proof The statement of Equation (133) follows from the Riemann integration theorem. The equivalence of Equations (132) and (134) follows from the linearity of the integral. Also note that by mimicking the proof of Theorem 18, it can be shown that PNM : L 2 A ! L 2 A is continuous. The operators PNK and PNM associated with one or more sampled k-plane curves, respectively, may be described as sample band-pass operators. They filter the Paley-Wiener function9 through a zero-dimensional set in the kplane to produce the functions PN,K and PNM , which are still functions in L 2 A.

VI. DIRECT RECONSTRUCTION USING NATURAL k-PLANE COORDINATES The rigorous analysis of the operator CK and its integrability properties should convince the reader that the Jacobian determinant of the transformation from natural k-plane coordinates to Cartesian coordinates provides an excellent choice for the taper function. The next step is to examine the discrete approximation of the operators CK and B D ˇ CK dˇ for some specific cases. To recap the conclusions of the last section, reconstruction from a single-trajectory data acquisition can be obtained from the Riemann sum approximation to CK , via 9 Technically, is the Fourier transform of a Paley-Wiener function, but L 2 A is isomorphic to the classical Paley-Wiener function space via the Fourier transformation.

313


the single-shot NKPCRM 10 formula [Sarty, 1995] as Q

Stq e2iKx tq xn CKy tq ym jJtq , 0j

RSxn , ym D C

135

qD1

where C is a scaling constant that may be used to adjust the final greyscale range, or window, and xn and ym are the coordinates of the centers of the pixels of interest. Similarly, the multiple-shot NKPCRM formula [Sarty, 1997] is given by P

Q

RSxn , ym D C

Stq , ˇp e2iKx tq ,ˇp xn CKy tq ,ˇp ym jJtq , ˇp j

136

pD1 qD1

Note that the constants C in Equations (135) and (136) absorb the terms 1t and 1t1ˇ in Equations (46) and (39), respectively. The issue of a generalized sampling theorem now arises. For Cartesian coordinates, the aliasing behavior of undersampled reconstructions is well understood as a folding-in of the object from outside the field of view into the reconstruction. A good place to start understanding the aliasing properties of a reconstruction is to understand the associated PSF. A. Sample Band-pass Operators and Their Point-Spread Functions By substituting Stq D and Stq , ˇp D

1

1

1

1

1

1

1

1

E

E Ktq p E e2ipÐ dE p

E

E Ktq ,ˇp p E e2ipÐ dE p

into Equations (135) and (136), respectively, and interchanging the order of integration and summation, the associated PSFs may be seen as Q

x, y D C

e2iKx tq xCKy tq y jJtq , 0j qD1

10

Natural K-plane coordinate reconstruction method.

137

314

GORDON E. SARTY

for the single-shot NKPCRM and as P

Q

x, y D C

e2iKx tq ,ˇp xCKy tq ,ˇp y jJtq , ˇp j

138

pD1 qD1

for the multiple-shot NKPCRM. It is not yet clear what form a sampling theorem for arbitrarily sampled kspace data should take, but the relevant issues may be discerned by a detailed look at the PSFs. In the case of Cartesian coordinates, the familiar folding phenomenon that results from undersampling (see Figure 20) can be seen to follow from the periodicity of the PSF. In other words, the trigonometric sum P

Q

x, y D

e2iKx p,qxCKy p,qy

139

pD1 qD1

is periodic when fKx p, q, Ky p, qg is a finite grid uniform in each of the kx and ky directions, with a wavelength in each direction of D 1/ks , where ks is the smallest nonzero frequency11 sampled. For example, spin warp data are frequently sampled on a uniform square grid with a data sample spacing of 1k in each direction. The grid is usually centered in k-space so that the origin is

(a)

(b)

FIGURE 20. Undersampling spin warp data leads to the folding of image data from outside the field of view V D 1/1k, where 1k is the sample spacing. (a) A reconstruction of spin warp MRI data of a pumpkin with adequately spaced k-space samples. (b) A reconstruction of spin warp data spaced at 1k > 1/V, where V is the diameter of the pumpkin. 11

Note that the unit of frequency in the k-plane is cycles/meter if x and y are in meters.

315


not sampled. The lowest frequency sampled is then 1k/2, and the wavelength of the PSF is D 2/1k. The real part of the function of Equation (139) also has negative peaks at /2 with the standard spin warp k-space sampling scheme (see Figure 21). The negative peaks limit the field of view to V D /2 or V D 1/1k, a result known to every NMR imager but usually understood in terms of a Nyquist theorem and not in terms of the PSF. The resolution of the reconstructed image is determined by the area under the central peak. That is, the sharper and more concentrated the peak, the finer the resolution. The resolution is determined by the maximum radius in k-space that is sampled. In other words, the larger the region sampled, the finer the resolution. As P and Q in Equation (139) approach infinity, while keeping the region D [A, A] ð [A, A] a constant square, the PSF will approach the function x, y D 4A2

sin[2Ax] sin[2Ay] [2Ax] [2Ay]

140

in L 2 A as a consequence of Theorem 21 (see also Equation (117) with b D A and a D 0). Note that is no longer periodic on 2 and that 2 L 2 2 . One

150000 100000 50000 0 −50000 −100000 −150000 −0.4

−0.3 −0.2

−0.1

0 x

0.1

0.2

0.3

0.4

0 −0.1 −0.2 −0.3 −0.4

0.1

0.2

0.3

0.4

y

FIGURE 21. The PSF associated with the reconstruction of data sampled on a Cartesian grid in k-space (spin warp data) is periodic. If the object being imaged has a support larger than the distance between the peaks, then more than one peak will contribute to image reconstruction at a point and information from more than one small neighborhood of the object will be folded or added together. The wavelength of the PSF in the x- and y-directions is the distance between the positive peaks, D 2/1k. However, the negative peaks limit the field of view to 1/1k.

316

GORDON E. SARTY

can let A ! 2 as P,Q ! 1 and show that 2 L 2 A ! 2 L 2 2 . As P,Q ! 1, the peaks of the periodic separate more and more widely, and the function approaches the sinc tensor product . Regardless of the field of view imaged, the resolution of the reconstructed image can be taken as proportional to 1/A, one over half the length of the sides of the square covered by the k-space samples. This resolution approximation follows from the limit PSF and represents the region under its central peak. The first zeros of occur at x D š1/2A and y D š1/2A along a square centered at the origin and with sides of length 1/A. The number 1/A itself is too conservative an estimate of the resolution, which is closer to 1/2A when the Rayleigh criterion used. The Rayleigh criterion may be interpreted as the minimum offset that produces a ratio of the value of the peaks of two superimposed, offset PSFs to the value of the function between the two peaks of greater than 0.81 [Callaghan, 1991]. Obtaining a crisp image is then a matter of ensuring that the pixel size, V/Q (or V/P), is slightly larger than 1/2A. For the PSFs associated with discrete grids other than the Cartesian, it is still true that as P,Q ! 1, the trigonometric functions of Equations (137) and (138) approach the PSFs associated with the curve band-pass and ordinary band-pass operators, respectively. The convergence is valid with respect to the metric of a fixed function space L 2 A, and the limit function belongs to L 2 2 . For adequately sampled trajectories, the limit function may be used to estimate the resolution of the reconstructed image. However, the aliasing properties of undersampled reconstructions are more complicated than the folding that occurs with the Cartesian data. Computer simulations best illustrate what happens when non-Cartesian data are undersampled. B. Reconstructions of Undersampled Simulated MRI Data The mathematical phantom shown in Figure 22 provides a convenient image for the generation of a simulated MRI signal, as given by Equation (2). The phantom is composed of multiples of characteristic functions of ellipses. So a simulated MRI signal may be computed analytically as a superposition of the Fourier transforms of translated and rotated characteristic functions, E , of an ellipse. Specifically, let E x, y D

1 0

y2 x2 C 1 a2 b2 otherwise

if

141


317

FIGURE 22. The mathematical phantom used to generate the simulated MRI signals. It is a superposition of characteristic functions of circles. The x-axis is vertical, whereas the y-axis is horizontal, an important point to remember when looking at later figures. The scale of the image box shown is 0.2 m ð 0.2 m, and it is that scale that defines the field of view required to produce an alias-free image. In subsequent plots of the PSFs in later figures, the distances x and y are in meters.

Then E kx , ky D

abJ1 2

a2 kx2 C b2 ky2

a2 kx2 C b2 ky2

142

where J1 is the first-order Bessel function. If T x, y D E x Tx , y Ty , then OT kx , ky D e2iTx kx CTy ky OE x, y, and if R x, y D E x cos C y sin, x sin C y cos, then OR kx , ky D OE kx cos C ky sin, kx sin C ky cos. In what follows, all images were directly reconstructed on 64 ð 64 pixel grid and the taper functions used were those given by the Jacobian determinants associated with natural k-plane coordinates. The k-space was sampled to a radius that gave a PSF resolution comparable to the pixel’s spatial size. Note that it is possible to reconstruct the same data on a finer pixel grid by computing more values from Equations (135) or (136), but the PSF resolution won’t change. The equivalent trick with the FFT and Cartesian data is to zero-fill the k-space data by padding the data set at frequencies outside the sampled region with zeros. Figures 23 and 24 show the reconstructions of undersampled spiral data and the corresponding PSFs for 2, 4, 8, and 16 times undersampling, along with an adequately sampled reconstruction. Four interleaved spiral trajectories, each with ω D 32, were sampled and a total of 214 samples (212 per trajectory) were required to obtain an unaliased reconstruction. It should be noted that by using

318

GORDON E. SARTY

ω D 16, it is possible to obtain unaliased reconstructions from four interleaved spirals with a total of 213 samples; remarkably, the reconstructions and PSFs for 2, 4, 8, and 16 times undersampling are very similar to the results shown in Figures 23 and 24. By restricting ω1 to be equal to 1 in Equation (9), the petals of the rosette trajectory can be made to follow a largely radial direction, as shown in Figure 25. Such radially oriented excursions in k-space (ROSE) scans may have the advantage of being motion resistant in the sense that the artifact resulting from

FIGURE 23. From top to bottom, the images in part (a) show reconstructions of simulated k-space data from four interleaved Archimedean spirals at an adequately sampled rate of 2 14 total samples and with 213 and 212 total samples, which give undersampling by factors of 2 and 4, respectively. In (b) are the corresponding PSFs. Note how the PSF flat spot, which defines the field of view, shrinks and disappears with continued undersampling.

319

RECONSTRUCTION OF NMR IMAGING DATA 100000

50000

0

0.2 0.15

−50000 −0.2

−0.15

0.1 0.05 −0.1 −0.05

0 x

0.05

0.1

0.15

0.2

0 −0.05 −0.1 −0.15 −0.2

y

150000

100000

50000

0

−50000

−0.2

0.2 0.15 0.1 0.05 −0.15

−0.1

−0.05

0 x

(a)

0.05

0.1

0.15

0.2

0 −0.05 −0.1 −0.15 −0.2

y

(b)

FIGURE 24. From top to bottom, the images in part (a) show reconstructions of simulated k-space data from four interleaved Archimedean spirals undersampled with 211 and 210 total samples, which give undersampling by factors of 8 and 16, respectively. The corresponding PSFs (part (b)) now show radial spikes along the radii defined by the location of the k-space samples.

20

10

−20

−10

10

20

−10

−20 FIGURE 25. The ROSE k-plane trajectory follows a largely radial direction.

320

GORDON E. SARTY

object motion during data collection is less offensive than the artifact produced from the reconstruction of data acquired in other ways, such as Cartesian EPI, for example. Figures 26 and 27 show the reconstructions of undersampled ROSE data and the corresponding PSFs for 2, 4, 8, and 16 times undersampling, along with an adequately sampled reconstruction. Four interleaved ROSE trajectories with ω D 16 were used and a total of 214 samples were required to avoid aliasing.

FIGURE 26. From top to bottom, the images in part (a) show reconstructions of simulated k-space data from four interleaved ROSE trajectories at an adequately sampled rate of 2 14 total samples and undersampled with 213 and 212 total samples, which give undersampling by factors of 2 and 4, respectively. The corresponding trigonometric PSFs (part (b)) are shown on the right. Note how the flat spot that defines the field of view shrinks and finally inverts the central peak as the number of samples decreases.


321

FIGURE 27. From top to bottom, the images in part (a) show reconstructions of simulated k-space data from four interleaved ROSE trajectories undersampled at rates of 211 and 210 total samples, which give undersampling by factors of 8 and 16, respectively. The corresponding trigonometric PSFs are shown in part (b). Ring lobes have formed, and at 210 samples the central peak has inverted again.

Figures 28 and 29 show the reconstructions of undersampled sinusoidal data and the corresponding PSFs for 2, 4, 8, and 16 times undersampling, along with an adequately sampled reconstruction. A single sinusoid with ω D 64 was sampled, and the Jacobian determinant used for the taper function was the k-space velocity associated with the normal-segment coordinates. As with the ROSE reconstructions, 214 samples were required for an unaliased reconstruction. Note that the aliasing is similar to the folding found in Cartesian sampling. Also note that the folding occurs only in the amplitude direction of the sinusoid; the sampling 1k in the other direction is always less than the Nyquist value. The PSFs associated with the spiral and ROSE reconstructions show a clear flat spot that shrinks as the number of samples is decreased. The flat spot represents a zone of nearly complete destructive interference among the sinusoids

322

GORDON E. SARTY

FIGURE 28. From top to bottom, the images in part (a) show reconstructions of simulated sinusoidal k-space data at an adequately sampled rate of 214 total samples and with 213 and 212 total samples, which give undersampling by factors of 2 and 4, respectively. One sinusoidal trajectory was used, with the samples taken at equal 1t along the trajectory. The corresponding trigonometric PSFs are shown in part (b). As the number of samples decreases, the PSF degenerates along the y-direction.

that comprise the discrete PSF, and the diameter of the flat spot determines the field of view. The PSF associated with the sinusoidal acquisition has peaks all along the y-axis and is flat away from that axis. The grid used to compute the sine PSF for the figures doesn’t adequately show the structure of the function along the y-axis, which roughly consists of alternating positive and negative peaks. So while a numerical analysis of the PSFs associated with


323

FIGURE 29. From top to bottom, the images in part (a) show reconstructions of simulated k-space data from simulated sinusoidal k-space data undersampled with 211 and 210 total samples, which give undersampling by factors of 8 and 16, respectively. The PSFs shown in part (b) continue to degenerate in the y-direction as the number of samples along the k-space trajectory is reduced.

non-Cartesian sampled MRI data is a good place to start in formulating a general sampling theorem, it appears that a complete analytical understanding of the discrete PSF is required before it can be claimed that the reconstruction of such data is completely understood.

VII. CONCLUSION The reconstruction of MRI data sampled on non-Cartesian grids in k-space offers challenges not encountered with the reconstruction of data sampled on Cartesian grids. The FFT allows the efficient reconstruction of Cartesian sampled data, and its efficiency has been the motivating force behind the development of the gridding approach for the reconstruction of non-Cartesian sampled data, especially for spiral MRI.

324

GORDON E. SARTY

The direct reconstruction determined by summing data along k-space trajectories has been shown in this article to be intimately related to integral transforms that depend on the existence of local coordinates on the k-space that are naturally associated with the given k-space data-acquisition trajectories. The local coordinates are natural in the sense that acquisition time provides an obvious coordinate. The interleaf parameter from acquisition schemes that can be interleaved can provide the second natural k-space coordinate. The relationship between the integral transforms and their approximations required for practical image reconstruction has been explored from a function space point of view. That point of view has allowed definite statements to be made about the convergence properties of discrete operators and their corresponding discrete point-spread functions to their integral counterparts as the number of samples is increased. Thus, statements about the resolution of a discrete reconstruction scheme can be made through the analysis of the point-spread function associated with an integral reconstruction scheme. Gridding reconstruction has been shown to be an approximation of direct reconstruction, so that all the results obtained from a study of direct reconstruction, including those related to sampling issues, may be directly applied to gridding reconstruction. The computational inefficiency of direct reconstruction does not, therefore, detract from the value of the theorems about integral and discrete operators on Paley-Wiener spaces to practical situations. The mathematics underlying gridding reconstruction is approximately the same as that underlying direct reconstruction. Finally, the largest open issue in reconstructing non-Cartesian sampled MRI data is the formulation of a sampling theorem. In other words, how many samples are required for a generic sampling scheme to result in an unaliased reconstruction? The results of computer simulations show that a careful study of the trigonometric point-spread functions associated with the reconstructions may shed some light in the direction of a generalized sampling theorem. Non-Cartesian grids in k-space originally arose from an effort to increase the speed of MRI data acquisition while limiting the demands placed on the gradient field power systems. It turns out that differing k-space acquisition patterns provide advantages not seen in the signal model used in this article. For example, rosette and Lissajous patterns provide some spectral selectivity among chemically shifted nuclear spins or among slices in an object. The ROSE pattern may, like the older radial pattern, provide some resistance to motion artifact. Some patterns may even be conducive to combined spatial/temporal data acquisition for the production of real-time MRI movies. Although more sophisticated signal models are required for those situations, they will all be based on the signal model used here, and the basic theory of reconstruction will be as given in this article.


325

REFERENCES Bloch, F. (1946). Nuclear induction, Physical Rev. 70, 460 – 474. Bracewell, R. N., and Thompson, A. R. (1973). The main beam and ringlobes of an east-west rotation-synthesis array, Astrophysical. J. 182, 74 – 94. Brigham, O. E. (1974). The Fast Fourier Transform, Upper Saddle River, N.J.: Prentice Hall. Brouw, W. N. (1975). Aperture Synthesis, in Methods in Computational Physics, vol. 14: Radio Astronomy. New York: Academic Press, pp. 131 – 175. Callaghan, P. T. (1991). Principles of Nuclear Magnetic Resonance Microscopy. Oxford: Clarendon Press. Daubechies, I. (1992). Ten Lectures on Wavelets. Philadelphia: SIAM. Dutt, A., Gu, M., and Rokhlin, V. (1993). Fast Algorithms for Polynomial Interpolation, Integration and Differentiation, Tech. report YALEU/DCS/RR-977, Yale University, Department of Computer Science. Dutt, A., and Rokhlin, V. (1993). Fast Fourier transforms for nonequispaced data, SIAM J. of Sci. Comp. 14, 1368 – 1393. Dutt, A., and Rokhlin, V. (1995). Fast fourier transforms for nonequispaced data II, Appl. and Comp. Harmonic Anal. 2, 85 – 100. Beger, W. H., ed. (1997). CRC Standard Mathematical Tables. Boca Raton: CRC Press. Higgins, J. R. (1996). Sampling Theory in Fourier and Signal Analysis. New York: Oxford University Press. Hinshaw, W. A., and Lent, A. H. (1983). An introduction to NMR imaging: From the Bloch equation to the imaging equation, Proc. of the IEEE 71, 338 – 350. Hoge, R. D., Kwan, R. K. S., and Pike, G. B. (1997). Density compensation functions for spiral MRI, Mag. Resonance in Med. 38, 117 – 128. Jackson, J. I., Meyer, C. H., Nishimura, D. G., and Macovski, A. (1991). Selection of a convolution function for Fourier inversion using gridding, IEEE Trans. on Med. Imaging 10, 473 – 478. Lauterbur, P. C. (1973). Image formation by induced local interactions: Examples employing nuclear magnetic resonance, Nature 242, 190. Ljunggren, S. (1983). A simple graphical representation of Fourier-based imaging methods, J. of Mag. Resonance 54, 338 – 343. Maeda, A., Sano, K., and Yokoyama, T. (1988). Reconstruction by weighted correlation for MRI with time-varying gradients, IEEE Trans. on Med. Imaging 7, 26 – 31. Mansfield, P. (1977). Multi-planar image formation using NMR spin echos, J. of Physics C 10, L55. Meyer, C. H., Hu, B. S., Nishimura, D. G., and Macovski, A. (1992). Fast spiral coronary imaging, Mag. Resonance in Medicine 28, 202 – 213. Noll, D. C. (1995). Methodologic considerations for spiral k-space functional MRI, Int. J. of Imaging Syst. and Tech. 6, 175 – 183. Noll, D. C. (1997). Multishot rosette trajectories for spectrally selective MR imaging, IEEE Trans. on Medical Imaging 16, 372 – 377. Noll, D. C. Meyer, C. H., Pauly, J. M., Nishimura, D. G., and Macovski, A. (1991). A homogeneity correction method of magnetic resonance imaging with time-varying gradients, IEEE Trans. on Med. Imaging 10, 629 – 637. Noll, D. C., Pauly, J. M., Meyer, C. H., Nishimura, D. G., and Macovski, A. (1992). Deblurring for non-2D Fourier transform magnetic resonance imaging, Mag. Resonance in Med. 25, 319 – 333.

326

GORDON E. SARTY

Noll, D. C., Peltier, S. J., and Boada, F. E. (1998). Simultaneous multislice acquisition using rosette trajectories (SMART): A new imaging method for functional MRI, Mag. Resonance in Med. 39 709 – 716. O’Sullivan, J. (1985). A fast sinc function gridding algorithm for Fourier inversion in computer tomography, IEEE Trans. on Med. Imaging 4, 200 – 207. Reed M., and Simon, B. (1972). Methods of modern mathematical Physics, Vol. II. New York: Academic Press. Rosenfeld, D. (1998). An optimal and efficient new gridding algorithm using singular value decomposition, Mag. Resonance in Med. 40, 14 – 23. Sarty, G. E. (1995). Method for reconstruction for MRI signals, Canadian patent application 2,147,556 (April 21). Sarty, G. E. (1997). The natural k-plane coordinate reconstruction method for magnetic resonance imaging: Mathematical foundations, Int. J. of Imaging Syst. Tech. 8, 519 – 528. Schomberg, H., and Timmer, J. (1995). The gridding method for image reconstruction by Fourier transformation, IEEE Trans. on Med. Imaging 14, 596 – 607. Slichter, C. P., (1963). Principles of Magnetic Resonance. New York: Harper and Row. Sogge, C. D. (1993). Fourier Integrals in Classical Analysis. Cambridge, Mass.: Cambridge University Press. Stehling, M. K., Turner, R., and Mansfield, P. (1991). Echo-planar imaging: Magnetic resonance imaging in a fraction of a second, Science 251, 43 – 50. Tong R., and Cox, R. W. (1998). Reconstruct NMR images from arbitrary scanning trajectory using nonuniform fast Fourier transform, Proc. of ISMRM, 6th Scientific, Sydney, p. 2065. Tricomi, F. G. (1985). Integral Equations. New York: Dover Publications, Inc. Twieg, D. B. (1983). The k-trajectory formulation of the NMR imaging process with applications in analysis and synthesis of imaging methods, Med. Physics 10, 610 – 621. Vlaardingerbroek, M. T., and den Boer, J. A. (1996). Magnetic Resonance Imaging. Berlin: Springer-Verlag. Wolfram, S. (1991). Mathematica. Reading, Mass.: Addison-Wesley. Yosida, K. (1980). Functional Analysis. Berlin: Springer-Verlag.


An Integrated Approach to Computational Vision The Edge Strength Function and the Nested Symmetries SIBEL TARI Department of Engineering Sciences, Middle East Technical University, Ankara, Turkey, 06531

I. II. III. IV.

Introduction . . . . . . . . . . . . . . . Review of the Edge-Strength Function . . . . . . Geometry of the Edge-Strength Function . . . . . Extraction and Representation of Perceptual Information A. Local Symmetry Set . . . . . . . . . . . B. Partial Symmetries and the Nested Loci . . . . C. Illustrative Examples and Discussion . . . . . V. Shape Skeleton and Shape Segmentation (2-D Case) . VI. Coloring Nested Symmetries in Higher Dimensions . . A. 3-D Case . . . . . . . . . . . . . . . 1. Singular Locus . . . . . . . . . . . . Abstract . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

. . . . . . . . . . . . .

327 332 335 342 343 344 345 351 362 363 364 365 365

I. INTRODUCTION Visual perception has been a source of curiosity in many disciplines. Psychologists, psycho-physicists, and neurophysiologists have tried to find out how the visual stimuli — the light absorbed by the receptors in the retina — are transformed into perception of shapes. With the advance of digital computers, a new approach to perception called computational vision has been initiated. Computational vision is characterized by an interest in formally defining the perceptual process necessary to represent the world [Matlin and Foley, 1992; Marr, 1982]. Ultimately, a computational vision theorist wants to develop programs that can take in the sort of data sensed by the photoreceptors in the retina (a digital image) and process these data to extract useful information that yield representation of the sensed world. The goal is twofold: to build intelligent machines with the ability to see and to better understand human vision. An important feature of the computational approach is that it attempts to solve problems using general physical knowledge rather than using specific knowledge about the objects being seen at the time, in that the computational Volume 111 ISBN 0-12-014753-X


328

SIBEL TARI

approach differs from the information-processing approach to vision that is popular among computer people. Scientists working on vision, however, agree that recognition of an object such as a cat requires that we know what a cat is. Therefore, the work in computational vision has focused on the early stages of visual processing. It has always been the biggest challenge of computational vision to express intuitive shape properties in quantitative terms using a representation that places similar shapes in a neighborhood without fixing a metric or knowing what is meant by a similar shape, because such a knowledge depends on the context (of course, the representation should be computable from images). Representations based on algebraic invariants and techniques derived from signal processing fail to capture intuitive properties of flexible shapes surrounding us. All or part of the problem has been addressed by many leading researchers of computational vision [Blum, 1973; Marr, 1982; Latto, Mumford, and Shah, 1984; Mumford, 1987]; the need for an object-defined reference system is emphasized. In the late 1960s, motivated by the preceding problem, H. Blum [1967, 1973] searched for an alternative geometry that would be devised to work for flexible natural objects and would replace the conventional geometry developed in collaboration with mechanics and out of concern for properties such as congruence, similarity, and equivalence under planar projection. Inspired by the influential Gestalt school of psychology, he presented a pioneering work on shape representation. He called his transform the medial axis transform and the representation a medial axis, or shape skeleton. A medial axis transform is defined by an axis that is the locus of the centers and the radii of the inscribed maximal circles of the shape. The key point is that the shape is locally symmetric with respect to this axis. Following the work of Blum, a variety of shape representation methods, each of which directly or indirectly makes use of an axis about which the shape is locally symmetric, has been proposed. These methods include, among others, generalized cylinders [Binford, 1971; Marr and Nishiara, 1977]; generalized ribbons [Brooks, 1981]; smoothed local symmetries [Brady and Asada, 1984], cores [Burbeck and Pizer, 1994]; and shock grammar [Siddiqi and Kimia, 1996]. The use of local symmetry was established as the result of a collaborative effort of mathematicians, engineers and psychologists, in particular, the Gestalt psychologists. Since their introduction by Wertheimer [1923], Koffka [1935], and Köhler [1947], Gestalt laws have had significant impact on many areas of psychology, mostly on perception of form; they have been a major source of inspiration for the shape theories in computational vision. The law of Prägnanz [Koffka, 1935] of the Gestalt school refers to the tendency to perceive figures as good, simple, and stable. According to this law, some figures are better than others because they are simpler; a circle is a perfect figure. Gestalt theorists, however, failed

AN INTEGRATED APPROACH TO COMPUTATIONAL VISION

329

to provide quantitative measures, but later, H. Resnikoff [1989] showed that in a closed contour, the increase in information obtained by going from point A to point B is given by logjkB /kA j, where kA and kB are the respective curvatures at points A and B. Thus, information in a closed contour is concentrated at the points of extreme curvature (note that this is consistent with the Attneave and Arnoult [1956] experiments in the 1950s on human subjects). Therefore, the circle (maximum symmetry and uniform curvature) is the most economical shape for encoding in the memory. Leyton’s PISA [1988] (Process Inferring Symmetric Axis) is another symmetry-based representation in which a shape is interpreted as a circle deformed with indentations and protrusions. Leyton [1987] first argued the duality of local symmetry and curvature and used this duality to infer PISA. Thus, PISA measures how a shape deviates from a circle. The basic idea of interpretation of a shape as a collection of pieces that are themselves deformed versions of a perfect shape through indentations and protrusions (curvature extrema) is particularly explicit in superquadrics [Bajcsy and Solina, 1987] — parametrized forms that may be best described as lumps of clay deformed and glued — and in the entropy scale space [Kimia, Tannenbaum, and Zucker, 1995]. The underlying principle of the entropy scale space is the representation of a shape in a two-dimensional scale space by letting the shape evolve toward a more circular shape under the influence of a nonlinear wave-diffusion equation. The diffusion tends to smooth shapes and the reaction tends to form singularities. The singularities that arise during the evolution of the shape provide a multiscale shape skeleton. It appears that there is a close connection among local symmetry, curvature, and wave propagation. In fact, wave propagation is the source of inspiration for Blum’s [1967] method. He gave several constructions for shape skeletons. One of these constructions is known as a grassfire, in which the interior of the shape is filled with dry grass and a fire is set simultaneously at all points along the shape boundary. The fire front propagates with constant inward normal velocity. Then, at any time, its points are equidistant from the shape boundary. The axis of local symmetry or the shape skeleton is defined as the locus of quench points (singularities) of the advancing front. By keeping track of the time at which the front arrives at the skeleton points, it is possible to recover the original shape. Thus, no information is lost. A consequence of this fact is that if the shape boundary is noisy, the noise is also preserved in the skeleton and thus the representation is not robust with respect to noise (Figure 1). Many complicated strategies have been devised to prune noisy skeletons in order to arrive at the essential skeleton of shape [Ogniewicz, 1994]. An alternative systematic approach is to introduce smoothing or regularization in the grass fire itself, thus combining skeletonization and regularization in a single formulation.

330

SIBEL TARI

FIGURE 1. Blum’s shape skeletons for (a) a rectangle and (b) a rectangle with a small bump at the boundary.

A straightforward approach to regularization is to let the front move with a curvature-dependent inward normal velocity consisting of two components: a constant component (Blum’s grass fire [1967]) and a smoothing component proportional to the curvature. Then, the propagating fronts may be interpreted as the steps in the evolution of a shape boundary evolving toward a smoother form. The resulting Hamilton-Jacobi equation is difficult to solve, requiring computationally intensive and complicated techniques [Tari, Shah, and Pien, 1996; Osher and Sethian, 1988; Sethian, 1982]. In order to implement such a nonlinear evolution, a linear diffusion–based approach was recently formulated by Tari, Shah and Pien [1996, 1997]. To illustrate the idea, first consider Blum’s grass fire analogy. Think of time t as a function over the plane by setting tx, y as the time when the advancing fire front passes through the point x, y. The value of the function at point x, y is the Euclidean distance of the point to the shape boundary. Its level curves describe the propagation of the shape boundary, and the projection of its singularities constitute the shape skeleton (Figure 2). In place of the surface tx, y, they proposed the function vx, y, which is called the edge-strength function. Function v is smooth, equals 1 along the shape boundary, and decays rapidly away from the boundary. Thus 1 v may be thought of as a monotonic function of t. The surface 1 v looks like a smoother (monotonically scaled) version of the surface t; the level curves of 1 v, and thus the level curves of v, are smoothed analogs of the propagating fire fronts (Figure 3). (Notice that time is increasing in the direction of decreasing v.) The formulation involves a parameter, which controls the degree of smoothing. The edge-strength function is inspired by an approximation [Ambrosio and Tortorelli, 1992] of the famous Mumford-Shah segmentation functional [Morel and Solimini, 1996; Mumford and Shah, 1985, 1989]. Indeed, Tari, Shah, and Pien used Euler-Lagrange equations of the approximate functional (a set of coupled linear PDEs) to compute the edge-strength function from gray scale images.


331

FIGURE 2. Surface tx, y defined by equidistant contour lines.

FIGURE 3. Surface 1 v and the level curves of v.

What they demonstrated, however, is that the edge-strength function is much more than a technical device for applying gradient descent to Mumford-Shah functional. They showed that the level curves of the edge-strength function describe the evolution of the shape boundary by letting each point move with a curvature-dependent inward normal velocity, and the local geometry of the edge strength function provides information regarding the geometry of the shape. Later, Tari and Shah went further and presented a very efficient and easy method for extraction of this information from arbitrary dimensional images in the form of nested local symmetries [Tari and Shah, 1998]. The most important property of the method is that it extends naturally to real images (i.e., gray scale and color images). We have assumed to this point that the image segmentation problem is solved; the shape outline is already extracted and available. But this is not an easy task, especially if the image is noisy or has varying contrast. The crucial point is that the edge-strength function corresponding to the segmentation locus of a gray scale or color image can be computed without determining the segmentation locus itself.

332

SIBEL TARI

Thus, the method permits the extraction of local symmetries and the shape skeleton directly from raw (unprocessed) images, combining shape extraction and shape representation (two major challenges of computational vision) into a single integrated framework. The purpose of this chapter is to discuss this development in local symmetry extraction and its connections to the edge-strength function as determined by inhomogeneous diffusion and to the fronts moving with curvature-dependent speed. The chapter is organized as follows. In Section II, the edge-strength function is reviewed. In Section III, the geometry of the edge-strength function is examined and the behavior of its level curves is related to a front moving with a curvature-dependent velocity. Extraction of perceptual information in the form of nested local symmetry loci is discussed in Section IV; the idea is developed for 2-D case first and then generalized to an arbitrary dimension. Section V deals with the coloring or the classification of local symmetry points for 2-D case; the shape skeleton and the shape decomposition are computed; recovery of the object boundaries from the symmetry branches is discussed. In Section VI the constructions of Section V are extended to higher dimensions.

II. REVIEW OF THE EDGE-STRENGTH FUNCTION Mumford and Shah [1985] formulated the image-segmentation problem as a functional minimization via which a piecewise smooth approximation of the image and a set of discontinuity loci corresponding to object boundaries are to be recovered from a given noisy image. The functional is as follows: EMS D ˛

jjrujj2 dx dy C ˇ Rn0

u g2 dx dy C length0

1

R

where R ² 0,

2 > 0,

1 D 2 D 0, 1 D 0, 1 2 < 0, 1 D 0,

3 > 0

3 > 0

2 > 0,

3 > 0

3 > 0 2 3 < 0

The first case, a local minimum, corresponds to a shape center, such as the centroid in the prism example. The second and third cases correspond to


365

global shape symmetries, e.g., parallel boundaries. Neglecting the effect of diffusion, the second case occurs at the center, along the long axis of a square prism. The fourth and fifth cases are the saddle points, and they correspond to neck points where the shape breaks up. The former case is attained for a dumbbell-like shape. At the break point two parts touch at a single point. The latter case arises for a cylinder with a peanut-shaped cross-sectional area. Two separated parts, each of which is a cylinder, touch along a line perpendicular to the base of the cylinder. For the case of raw images, S0,3 also contains maximum points of v, namely, the points at which the eigenvalue with the largest magnitude is negative.

ABSTRACT A new development in local symmetry extraction and its connections to segmentation functionals and the fronts propagating with curvature-dependent speed are examined. The basic tool is a new distance function that attains its maximum value at the shape boundary and decays rapidly away from there. It is shown that the Hessian of the distance function captures perceptual information that can be extracted easily, efficiently, and robustly in the form of nested local shape symmetries at multiple scales. The most important property of the distance function is that it can be computed from unprocessed images; thus the segmentation loci of an image can be represented without the knowledge of the segmentation loci itself. The formulation is also valid for shapes in any dimension. REFERENCES Ambrosio, L., and Tortorelli, V. M. (1992). On the approximation of functionals depending on jumps by quadratic, elliptic functionals, Bol. Un. Mat. Ital. Attneave, F., and Arnoult, M. (1956). The quantitative study of shape and pattern perception, Psychology Bull. 53, 452 – 471. Bajcsy, R., and Solina, F. (1987). Three dimensional object representation revisited, IEEE Proc. of the Int. Conf. on Computer Vision. Binford, T. O. (1971). Visual perception by computer, IEEE Proc. Conference on Syst. Contr. Blum, H. (1973). Biological shape and visual science, J. of Theoretical Biology 38, 205 – 287. Blum, H. (1967). A Transformation for Extracting New Descriptors of Shape, in Models for the Perception of Speech and Visual Form (W. Wathen-Dunn, ed.). Cambridge, Mass.: MIT Press. Brady, M., and Asada, H. (1984). Smoothed local symmetries and their implementation, Int. J. of Robotics Research 3, 36 – 61. Brooks, R. A. (1981). Symbolic reasoning among 3-D models and 2-D images, AI 17, 285 – 348. Burbeck, C., and Pizer, S. (1994). Object representation by cores: Identifying and representing primitive spatial regions, UNC Tech Report, TR94-160.

366

SIBEL TARI

Kimia, B. B., Tannenbaum, A. R., and Zucker, S. W. (1995). Shapes, shocks, and deformations I: The components of shape and the reaction-diffusion space, Int. J. of Computer Vision. Koffka, K. (1935). Principles of Gestalt Psychology. New York: Harcourt Brace. Köhler, W. (1947). Gestalt Psychology: An Introduction to New Concepts in Modern Psychology. Liveright. Latto, A., Mumford, D., and Shah, J. (1984). The representation of shape, Proc. of the Workshop on Computer Vision Representation and Control. Leyton, M. (1987). Symmetry-curvature duality, CVGIP 38, 327 – 341. Leyton, M. (1988). A process grammar for shape, AI 34, 213 – 247. Marr, D. (1982). Vision. New York: Freeman. Marr, D., and Nishiara, K. (1977). Representation and recognition of the spatial organization of three dimensional shapes, Proc. of the Royal Soc. of London B200, 269 – 294. Matlin, M. W., and Foley, H. J. (1992). Sensation and Perception. Reading, Mass.: Allyn and Bacon. Morel, J. M., and Solimini, S. (1996). Variational Methods in Image Segmentation, Birkhäuser, Progress in Nonlinear Differential Equations and Their Applications. Mumford, D. (1987). The problem of robust shape descriptors, IEEE Proc. of the Int. Conf. on Computer Vision. Mumford, D., and Shah, J. (1985). Boundary detection by minimizing functionals, IEEE Proc. Conf. on Computer Vision Pattern Recognition. Mumford, D., and Shah, J. (1989). Optimal approximations by piecewise smooth functions and the associated variational problems, Comm. on Pure and Applied Math. XLII (5), 577 – 684. Ogniewicz, R. L. (1994). Skeleton-space: A multiscale shape description combining region and boundary information, IEEE Proc. of Conf. on Computer Vision Pattern Recognition. Osher, S., and Sethian, J. (1988). Fronts propagating with curvature dependent speed: Algorithms based on the Hamilton-Jacobi formulations, J. Comp. Physics, 79. Resnikoff, H. (1989). The Illusion of Reality. New York: Springer Verlag. Sethian, J. (1982). An analysis of flame propagation, Ph.D. diss., University of California Berkley, CPAM Report 79. Shah, J. (1991). Segmentation by nonlinear diffusion, II, IEEE Proc. Conf. on Computer Vision Pattern Recognition. Shah, J. (1992). Segmentation by nonlinear diffusion, II, IEEE Proc. Conf. on Computer Vision Pattern Recognition. Shah, J. (1996). Curve evolution and segmentation functionals: Application to color images, IEEE Proc. Int. Conf. on Image Processing. Shah, J., Pien, H., and Gauch, J. (1996). Recovery of surfaces with discontinuities by fusing shading and range data within a variational framework, IEEE Trans. in Image Proc. 5, 1243 – 1251. Siddiqi, K., and Kimia, B. (1996). Toward a shock grammar for recognition, IEEE Proc. of Conf. on Computer Vision Pattern Recognition. Tari, S., and Shah, J. (1998). Local symmetries of shapes in arbitrary dimension, IEEE Proc. Int. Conf. on Computer Vision. Tari, S., and Shah, J. (1997). Simultaneous segmentation of images and shapes, Vision Geom. Tari, S., Shah, J. and Pien, H. (1996). A computationally efficient shape analysis via level sets, IEEE Proc. Workshop in Math. Methods in Med. Image Anal. Tari, S., Shah, J., and Pien, H. (1997). Extraction of shape skeletons from grayscale images, Computer Vision Image Understanding 66, 133 – 146. Wertheimer, M. (1923). Untersuchungen zur Lehre von der Gestalt, II (translated as Laws of Organization in Perceptual Forms), in A Source Book of Gestalt Psychology, (W. D. Ellis, ed.). London: Routledge & Kegan Paul. pp. 71 – 88.

INDEX Apertures, 305 Artificial images. See Natural and artificial images Auger electron spectroscopy (AES), 120 Autobinomial model, 223 – 4 Baye’s rule, 170 Beam current, 129 Bloch equation, 247 Bochner’s theorem, 283 Boersch effect for electron emitters: Jansen’s theory, 116 – 19 Knauer’s model, 119 Boersch effect for Schottky emitters, 143 – 6 ˇ() function, 54 – 5, 126 – 7, 139 Brightness, 114 indirect measurement of, 126 – 7 measurement of, 124 – 6 of Schottky electron emitters, 147 – 51 Cauchy formula, 204 Chemical shift, 246 Complex convolution (CC), calculating, 8 Computational vision: defined, 327 edge-strength function, development of, 330 – 2 edge-strength function, geometry of, 335 – 42 edge-strength function, review of, 332 – 5 examples and discussion, 345 – 51 Gestalt laws and, 328 – 9 grass fire, 329 – 30 local symmetry set, 343 – 4 nested symmetries in higher dimensions, 362 – 5 partial symmetries and nested loci, 344 – 5 Process Inferring Symmetric Axis (PISA), 329 shape representation methods, 328 shape skeleton and segmentation, 351 – 62 wave propagation, 329 Coulomb interactions, 116 Current density, 109 – 11 Curve band-pass operator point-spread functions: curved k-plane trajectories and, 283 – 6 defined, 281 integration of, 298 – 306

as an inverse Fourier transform of a finite nonnegative Lebesgue measure, 281 – 3 lines, squares, and circles in k-space and, 286 – 98 reconstruction from discrete data samples and, 306 – 12 Curve band-pass operators, 263 band-pass operators as integrals of, 273 – 6 Fourier transforms restricted to curves in R2, 264 – 70 on Paley-Wiener spaces, 270 – 3 properties of, 278 – 81 resolution of the identity for Paley-Wiener spaces, 276 – 8 Curve reconstruction operator, 273 Cyclic convolution property (CCP), 4 – 5 Cyclic convolution property, (cont.) calculation of two-dimensional, 67 for 2-D FNTs, 10 – 11 Data errors, in 2-D FNT, 44 Deal leaves model: average area after occlusion, 208 – 10 Boolean model, 236 – 7 description of, 194 – 9, 231 – 9 distribution of intercepts in, 201 – 4 geometric covariogram, 199 – 201 grain of, 195 Lebesgue measure, 196 Poisson point processes, 231 – 6 random closed and compact sets of Rn , 193 in R3 , 238 – 9 restrictions of, 192 size distribution of grains, 207 – 8 stability of power law, 205 – 7 Direct/weighted reconstruction, 244, 257 – 8 via natural k-plane coordinates, 260 – 1 Discrete Fourier transform (DFT), 4 Distribution functions, calculation of, 98 – 100 Echo planar imaging (EPI), 250, 305 Edge-strength function development of, 330 – 2 geometry of, 335 – 42 review of, 332 – 5 Eklundh’s fast algorithm, 20

368

INDEX

Electron emission theory: brightness, 114 calculation of distribution functions, 98 – 100 current density, 109 – 11 description of emission process, 96 – 8 emission at high temperature and low field, 100 – 5 emission at low temperature and high field, 105 – 7 extended-Schottky emission, 97, 103 – 5 parameter space determination, 108 – 9 Schottky emission, 97, 102 – 3 thermal-field emission, 98, 105 – 7 thermionic emission, 97, 100 – 1 total energy distribution, 111 – 13 Electron emitters: See also Schottky electron emitters stability, measurement of, 127 – 8 Electron emitters, Boersch effect for: Jansen’s theory, 116 – 19 Knauer’s model, 119 Electron-energy analyzer, 120 – 2, 129 Energy distribution, total, 111 – 13 indirect measurement of, 126 – 7 measurement of, 120 – 3 measurement of, in heated tungsten cathode, 141 – 2 Euler-Lagrange equation, 333 – 5 Extended-Schottky emission, 97, 103 – 5 Extractor, 97

inspection method, implementation of, 49 – 53 nonzero pixels in, 44 relationship between transform size and modulus, 9 – 10 Fermi-Dirac function, 97, 104 Field emission, 98 Forward transform, 54, 60 – 2 multidimensional, 64 Fourier transform, 54 Fractal models: fractional Brownian motion, 216 – 18 iterated function systems, 219 – 21 midpoints displacement method, 218 – 19 spectral synthesis method, 219 Fractional Brownian motion, 216 – 18 FRAME model, 226 – 8 Fubini-Tonelli theorem, 189 Full width at half maximum (FWHM), 113, 117 – 19, 125, 135 – 6, 139, 146

Fast algorithms for Mersenne number transforms, 56 – 8 Fast Fourier transform (FFT), 4, 244, 254 – 6 Fast Walsh transform, 53 Fermat number transforms (FNTs), 6 – 7 Fermat number transforms, one-dimensional (1-D FNTs), 50 Fermat number transforms, two-dimensional (2-D FNTs): analysis of periodic structures, 38 – 46 applications, 46 – 9 arithmetic complexity, 10 combined with 2-D NMNT, 68 – 72 convolution property, 10 – 11 defined, 9 effect of data errors in, 44 examples, 11 generalization for, 41 – 4

Hahn-Banach theorem, 308 Hammersley-Clifford theorem, 222 Hartley transform, 54 Heine-Borel theorem, 265 Hermitian conjugation property, 286 – 7 Hilbert-Schmidt theorem, 280 Hilbert space, 263 – 4 Holtsmark regime, 118 – 19, 144 Hypertexture model, 213

Gamov exponent, 103, 141 Geometric covariogram, 199 – 201 Gestalt laws, 328 – 9 Gibbs field, 222 Gibbs sampler, 223, 226 – 7 Gradient echo, 248 Grass fire, 329 – 30 Gridding reconstruction, 244, 256 – 7 used to provide direct reconstruction, 258 – 9 Gyromagnetic ratio, 246

Inspection method, implementation of, 49 – 53 Intercepts in dead leaves model: distribution of, 201 – 4 Inverse discrete Fourier transform (IDFT), 257 – 8 point-spread functions and, 281 – 3 Inverse transform, 54, 61, 63 multidimensional, 64 – 5

INDEX Isoperimetric inequality, 187, 189 Iterated function systems, 219 – 21 Jansen’s theory, 116 – 19 Knauer’s model, 119 k-plane coordinate systems: direct reconstruction via, 260 – 1, 312 – 23 examples of, 261 – 2 multiple-shot, 313 – 14 normal segment, 259 – 60 reconstruction from single acquisition of data, 262 – 3 single-shot, 313 – 14 k-plane trajectories, point-spread functions and, 283 – 6 k-space, 247 – 53 point-spread functions and, 286 – 98 Larmor frequency, 246 Lebesgue measure, 196, 281 – 3 Linear decomposition, 174 Lissajous trajectories, 252 – 3, 305 Lorentz regime, 118, 144 Magnetic resonance imaging (MRI): generation of, 245 – 8 gradient limitations, 253 – 4 k-space, 247 – 53 reconstructions of undersampled simulated, 316 – 23 Magnetic resonance imaging, curve band-pass operator point-spread functions and: curved k-plane trajectories and, 283 – 6 defined, 281 integration of, 298 – 306 as an inverse Fourier transform of a finite nonnegative Lebesgue measure, 281 – 3 lines, squares, and circles in k-space and, 286 – 98 reconstruction from discrete data samples and, 306 – 12 Magnetic resonance imaging, curve band-pass operators and, 263 band-pass operators as integrals of, 273 – 6 Fourier transforms restricted to curves in R2 , 264 – 70 on Paley-Wiener spaces, 270 – 3 properties of, 278 – 81

369

resolution of the identity for Paley-Wiener spaces, 276 – 8 Magnetic resonance imaging, reconstruction of nuclear: on Cartesian grids, 244 conclusions, 323 – 4 direct/weighted, 244, 257 – 8 fast Fourier transform (FFT), 244, 254 – 6 gridding, 244, 256 – 7 gridding used to provide direct reconstruction, 258 – 9 natural k-plane coordinate systems, 259 – 63, 312 – 23 Magnetization, 246 Markov field, 222 – 3 Mersenne number transforms (MNTs), 6 calculation of 1-D convolutions, 55 – 6 fast algorithms for, 56 – 8 properties of the ˇ() function, 54 – 5 transform defined, 53 – 4 Mersenne number transforms, separable two-dimensional new (2-D NMNTs): calculation of 2-D convolutions, 63 – 4 defined, 62 – 3 forward and inverse transforms, 62 – 3 forward and inverse transforms, multidimensional, 64 – 5 Mersenne number transforms, two-dimensional new (2-D NMNTs): calculation of convolutions, 59 – 60 calculation of multidimensional convolutions, 61 combined with 2-D FNT, 68 – 72 defined, 58 multidimensional forward and inverse transforms, 60 – 1 relationship between transform sizes and modulus, 59 Meyer-Mallet scaling function, 301 – 2, 304, 306 Midpoints displacement method, 218 – 19 Monte Carlo techniques, 116 Multiscale synthesis model, 225 – 6 Mumford-Shah model, 330 – 3 Natural and artificial images: See also Deal leaves model; Textures size power law, 169, 205 – 7 statistical studies, 168 – 9

370 Natural images, principles for synthesis of abstract, 228 scale and perspective, 229 – 30 structural laws, 229 synthesis rules and synthetic worlds, 230 – 1 Natural images, sizes of sections in, 175 digital photographs, 177 – 9 distribution of areas, 176 – 7 distribution of boundary lengths, 184 – 5 distribution of intercept lengths, 185 – 6 lower bound for the BV norm and applications, 186 – 91 other types of images, 179 – 84 Natural images, statistics of covariance and power spectrum, 172 – 3 first-order, 171 – 2 linear decomposition, 174 motivations, 170 – 1 scale invariance, 175 second-order, 172 – 4 Natural k-plane coordinate systems. See k -plane coordinate systems Noise function, 213 Normal energy, 98 Normal-segment coordinates, 259 – 60 Normal vectors perturbation method, 211 – 12 Nuclear magnetic resonance. See Magnetic resonance imaging, reconstruction of nuclear Number domain analysis and two-dimensional number theoretic transforms (2-D NTTs): applications, 32, 34, 36 – 7 mathematical development, 24 rectangular images, 30, 32 simple images, 24 square images, 25 – 30 summary, 32, 37 – 8 zero pattern, 34, 36 Number theoretic transforms (NTTs): See also under type of advantages over FFT, 5 complex convolution, calculating, 8 conclusions, 80 – 5 constraints for selecting, 5 – 6 defined, 4 – 5 Fermat, 6 – 7 hardware implementations, 73 – 80

INDEX length considerations, 7 – 8 Mersenne, 6 pipelined designs, 74 TTL-based designs, 74 vector radix algorithm, 74 – 80 Number theoretic transforms, two-dimensional (2-D NTTs): computation of, 19 – 20 convolution without matrix transpose and overlap, 11 – 16 example of autoconvolution from simulations, 20 – 3 twiddle factor, 16 – 23 Number theoretic transforms (2-D NTTs), number domain analysis and two-dimensional: applications, 32, 34, 36 – 7 mathematical development, 24 rectangular images, 30, 32 simple images, 24 square images, 25 – 30 summary, 32, 37 – 8 zero pattern, 34, 36 Numerical model, 111 Nyquist theorems, 244, 255 Occlusion phenomenon, 192 One-dimensional Fermat number transforms (1-D FNTs), 50 One-dimensional number theoretic Walsh transform, 50 Overlap-save technique, 12 Paley-Wiener spaces, 270 – 3, 276 – 8 Parabolic-barrier approximation, 111, 113, 129, 137 – 9 Parameter space determination, 108 – 9 Periodic structures, analysis of 2-D FNT, 38 – 46 Phase gradient, 250 Pipelined designs, 74 Plancherel theorem, 268 Point-spread function (PSF), curve band-pass operator: curved k-plane trajectories and, 283 – 6 defined, 257, 281 flat spot, 321 – 2 integration of, 298 – 306 as an inverse Fourier transform of a finite nonnegative Lebesgue measure, 281 – 3

INDEX

371

lines, squares, and circles in k-space and, 286 – 98 reconstruction from discrete data samples and, 306 – 12 Point-spread functions, sample band-pass operators and, 313 – 16 Prägnanz, law of, 328 Process Inferring Symmetric Axis (PISA), 329

Spectral synthesis method, 219 Spin echo, 248 Spin-lattice relaxation, 247 Spin-spin relaxation, 247 Spin warp, 249 – 50 Spiraling imaging, 250 – 1 Spot noise textures, 215 – 16 Square images, 25 – 30 Steiner formula, 203, 207 – 8

Radix-2 algorithm 56 – 7 Random fields models, 221 autobinomial model, 223 – 4 gagalovicz and Ma model, 224 – 5 Markov models, 222 – 3 Rayleigh criterion, 316 Reaction-diffusion textures, 214 Read gradient, 249 Rectangular images, 30, 32 Richardson-Dushman plot, 101 Riemann integration theorem, 309, 310 – 11 Riemann sum approximation, 261, 263, 302, 306, 312 Riesz-Thorin theorem, 308 Ring collapse, 152, 156 Root-power-sum method, 125, 149 ROSE (radially oriented excursions in k-space), 318 – 21 Rosette imaging, 251

Tangential energy, 98 Textures, 210 fractal models, 216 – 21 FRAME model, 226 – 8 multiscale synthesis model, 225 – 6 normal vectors perturbation method, 211 – 12 random fields models, 221 – 5 reaction-diffusion, 214 solid, 212 – 13 spot noise, 215 – 16 Thermal-field emission, 98, 105 – 7 Thermionic emission, 97, 100 – 1 Tip angle, 247 Transmission coefficient, 98 TTL-based designs, 73 – 4 Turbulence, 213 Twiddle factor, 16 – 23 Two-dimensional composite transform (2-D CNTT), 66 Two-dimensional Fermat number transforms (2-D FNTs): analysis of periodic structures, 38 – 46 applications, 46 – 9 arithmetic complexity, 10 combined with 2-D NMNT, 68 – 72 convolution property, 10 – 11 defined, 9 effect of data errors in, 44 examples, 11 generalization for, 41 – 4 inspection method, implementation of, 49 – 53 nonzero pixels in, 44 relationship between transform size and modulus, 9 – 10 Two-dimensional mixed radix conversion (2-D MRC), 67 – 8 Two-dimensional new Mersenne number transforms (2-D NMNTs): calculation of convolutions, 59 – 60

Scale invariance, 175 Schottky effect, 94, 97, 102 Schottky electron emitters: basic description, 93 – 5 Boersch effect for, 143 – 6 brightness of, 147 – 51 effects of changing ZrO coating to HfO, 158 – 62 energy spread of, 128 – 46 energy spread of, with small radius of curvature, 157 – 8 measurement of total energy distribution in heated tungsten cathode, 141 – 2 stability of emission, 151 – 6 Schottky emission, 97, 102 – 3 extended, 97, 103 – 5 Schrödinger equation, 100, 111 Schwartz functions, 265, 281 Shannon source coding theorem, 171 Sobolev space, 263 – 4 Solid textures, 212 – 13 Spatial frequency domain. See k -space

372 Two-dimensional new Mersenne number transforms (2-D NMNTs): (cont.) calculation of multidimensional convolutions, 61 combined with 2-D FNT, 68 – 72 defined, 58 multidimensional forward and inverse transforms, 60 – 1 relationship between transform sizes and modulus, 59 Two-dimensional new Mersenne number transforms, separable (2-D NMNTs): calculation of 2-D convolutions, 63 – 4 defined, 62 – 3 forward and inverse transforms, 62 – 3 forward and inverse transforms, multidimensional, 64 – 5 Two-dimensional number theoretic transforms (2-D NTTs): computation of, 19 – 20 convolution without matrix transpose and overlap, 11 – 16 example of autoconvolution from simulations, 20 – 3 twiddle factor, 16 – 23

INDEX Two-dimensional number theoretic transforms (2-D NTTs), number domain analysis and: applications, 32, 34, 36 – 7 mathematical development, 24 rectangular images, 30, 32 simple images, 24 square images, 25 – 30 summary, 32, 37 – 8 zero pattern, 34, 36 Two-dimensional number theoretic Walsh transform, 51 – 3 Van der Corput theorem, 284 Vector radix algorithm, 74 – 80 Walsh function, 51 Wentzel-Kramers-Brillouin (WKB) approximation, 103 Wiener-Khintchin formula, 172 – 3 Work function, 96 – 7 X-ray photo-electron spectroscopy (XPS), 120 Zero pattern, 34, 36

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

Recommend Documents