Wavelets in Chemistry (Data Handling in Science and Technology)

DATA HANDLING IN SCIENCE AND TECHNOLOGY- VOLUME 22 Wavelets in Chemistry DATA HANDLING IN SCIENCE AND TECHNOLOGY- VOL...

Author: Beata Walczak

74 downloads 1471 Views 23MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

DATA HANDLING IN SCIENCE AND TECHNOLOGY- VOLUME 22

Wavelets in Chemistry

DATA HANDLING IN SCIENCE AND TECHNOLOGY- VOLUME 22

Wavelets in Chemistry

DATA HANDLING IN SCIENCE AND TECHNOLOGY Advisory Editors: B.G.M. Vandeginste and S.C. Rutan

Other volumes in this series: Volume 1 Volume 2 Volume 3 Volume 4 Volume 5 Volume 6

Volume 7 Volume 8 Volume 9 Volume 10 Volume 11 Volume 12 Volume 13 Volume 14 Volume 15 Volume 16 Volume 17 Volume 18 Volume 19

Volume 20A

Volume 20B

Volume 21 Volume 22

Microprocessor Programming and Applications for Scientists and Engineers, by R.R. Smardzewski Chemometrics: A Textbook, by D.L. Massart, B.G.M. Vandeginste, S.N. Deming, Y. Michotte, and L. Kaufman Experimental Design: A Chemometric Approach, by S.N. Deming and S.L. Morgan Advanced Scientific Computing in BASIC with Applications in Chemistry, Biology and Pharmacology, by P. Valk6 and S. Vajda PCs for Chemists, edited by J. Zupan Scientific Computing and Automation (Europe) 1990, Proceedings of the Scientific Computing and Automation (Europe) Conference, 12-15 June 1990, Maastricht, The Netherlands, edited by E.J. Karjalainen Receptor Modeling for Air Quality Management, edited by P.K. Hopke Design and Optimization in Organic Synthesis, by R. Carlson Multivariate Pattern Recognition in Chemometrics, illustrated by case studies, edited by R.G. Brereton Sampling of Heterogeneous and Dynamic Material Systems: Theories of Heterogeneity, Sampling and Homogenizing, by P.M. Gy Experimental Design: A Chemometric Approach (Second, Revised and Expanded Edition) by S.N. Deming and S.L. Morgan Methods for Experimental Design: Principles and Applications for Physicists and Chemists, by J.L. Goupy Intelligent Software for Chemical Analysis, edited by L.M.C. Buydens and P.J. Schoenmakers The Data Analysis Handbook, by I.E. Frank and R. Todeschini Adaption of Simulated Annealing to Chemical Optimization Problems, edited by J. Kalivas Multivariate Analysis of Data in Sensory Science, edited by T. Naes and E. Risvik Data Analysis for Hyphenated Techniques, by E.J. Karjalainen and U.P. Karjalainen Signal Treatment and Signal Analysis in NMR, edited by D.N. Rutledge Robustness of Analytical Chemical Methods and Pharmaceutical Technological Products, edited by M.W.B. Hendriks, J.H. de Boer, and A.K. Smilde Handbook of Chemometrics and Qualimetrics: Part A, by D.L. Massart, B.G.M. Vandeginste, L.M.C. Buydens, S. de Jong, P.J. Lewi, and J. Smeyers-Verbeke Handbook of Chemometrics and Qualimetrics: Part B, by B.G.M. Vandeginste, D.L. Massart, L.M.C. Buydens, S. de Jong, P.J. Lewi, and J. Smeyers-Verbeke Data Analysis and Signal Processing in Chromatography, by A. Felinger Wavelets in Chemistry, edited by B. Walczak

DATA HANDLING IN SCIENCE AND T E C H N O L O G Y - VOLUME 22

Advisory Editors: B.G.M. Vandeginste and S.C. Rutan

Wavelets in C h e m i s t ry edited Beata

by Walczak

Institute of Chemistry, Silesian University, 9 Szkolna Street, 40-006 Katowice, Poland

2000 ELSEVIER Amsterdam

- Lausanne - New York - Oxford - Shannon - Singapore - Tokyo

ELSEVIER SCIENCE PUBLISHERS B.V. Sara Burgerhartstraat 25 P.O. Box 211, 1000 AE Amsterdam, The Netherlands 92000 Elsevier Science B.V. All rights reserved. This work is protected under copyright by Elsevier Science, and the following terms and conditions apply to its use: Photocopying Single photocopies of single chapters may be made for personal use as allowed by national copyright laws. Permission of the Publisher and payment of a fee is required for all other photocopying, including multiple or systematic copying, copying for advertising or promotional purposes, resale, and all forms of document delivery. Special rates are available for educational institutions that wish to make photocopies for non-profit educational classroom use. Permission may be sought directly from Elsevier Science Rights & Permissions Department, PO Box 800, Oxford OX5 1DX, UK; phone: (+44) 1865 843830, fax: (+44) 1865 853333, e-mail: [email protected]. You may also contact Rights & Permissions directly through Elsevier's home page (http://www.elsevier.nl), selecting first 'Customer Support', then 'General Information', then 'Permissions Query Form'. In the USA, users may clear permissions and make payments through the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA; phone: (978) 7508400, fax: (978) 7504744, and in the UK through the Copyright Licensing Agency Rapid Clearance Service (CLARCS), 90 Tottenham Court Road, London WlP 0LP, UK; phone: (+44) 171 631 5555; fax: (+44) 171 631 5500. Other countries may have a local reprographic rights agency for payments. Derivative Works Tables of contents may be reproduced for internal circulation, but permission of Elsevier Science is required for external resale or distribution of such material. Permission of the Publisher is required for all other derivative works, including compilations and translations. Electronic Storage or Usage Permission of the Publisher is required to store or use electronically any material contained in this work, including any chapter or part of a chapter. Except as outlined above, no part of this work may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without prior written permission of the Publisher. Address permissions requests to: Elsevier Science Rights & Permissions Department, at the mail, fax and e-mail addresses noted above. Notice No responsibility is assumed by the Publisher for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products, instructions or ideas contained in the material herein. Because of rapid advances in the medical sciences, in particular, independent verification of diagnoses and drug dosages should be made. First edition 2000 Library of Congress Cataloging in Publication Data A catalog record from the Library of Congress has been applied for. ISBN: 0 444 50111 8 e The paper used in this publication meets the requirements of ANSI/NISO Z39.48-1992 (Permanence of Paper). Printed in The Netherlands.

PREFACE Wavelets seem to be the most efficient tool in signal denoising and compression. They can find unlimited numbers of applications in all fields of chemistry, where the instrumental signals are the source of information about the studied chemical systems or phenomena, and in all cases, when these signals have to be archived. The quality of the instrumental signals decides about the quality of answer to the basic analytical questions: how many components are in the studied systems, what are these components like and what are their concentrations? Efficient compression of the signal sets can drastically speed up further processing (such as, e.g. data visualization, modelling (calibration and pattern recognition), library search, etc.). Exploration of the possible applications of wavelets in analytical chemistry has just started and the proposed book about wavelet theory and about the already existing applications can significantly speed up this process. Presently wavelets are a hot issue in many different fields of science and technology. There are already many books about wavelets, but almost all of them are written by mathematicians, or by people involved in information science. Due to the fact that wavelet theory is quite complicated and different languages are involved in its presentation, these books are almost unreadable for chemists. Lack of the texts comprehensible to chemists seems to be a barrier and can be considered a reason why wavelets enter chemistry so slowly and so shyly. The book is written in the tutorial-like manner. We intended to gently introduce wavelets to an audience of chemists. Although the particular chapters are written by independent authors, we intended to cover all important aspects of wavelet theory and to present wavelet applications in chemistry and in chemical engineering. Basic concepts of wavelet theory, together with all important aspects of wavelet transforms, are presented in the first part of the book. This part is extensively illustrated with figures and simulated examples. The second part of this book consists of examples of wavelet applications in chemistry and in chemical engineering. Written by chemists for the chemists, this book can be of great help for all those involved in signals and data processing. All invited authors are the widely recognized experts in the field of chemometrics, with an unquestionable competence in the theory and practice of wavelets.

The book is addressed to analytical chemists, dealing with any type of spectral data (main interest: signal to noise enhancement and/or signal compression); organic chemists, involved in combinatorial chemistry (main interest: compression of instrumental signals); chemists involved in chemometrics (main interest: compression of the ill-posed data sets for the further preprocessing and data denoising); artificial intelligence fields (main interests: compression of any spectral libraries and speeding up library search); theoretical chemists (main interest: wavelets as a new family of basis functions with special properties); and engineers involved in process control (main interest: analysis of trends). Readers are expected to know basic terms of linear algebra and be familiar with the matrix notation. As a team of Contributors to this volume, we are well aware of certain repetitions occurring on its pages, which are hardly avoidable in case of massive joint enterprises of similar sort. There are, however, certain advantages of this situation as well, the main one being the enriching demonstration of the selected wavelet issues from different perspectives. Finally, may I allow myself to express my profound gratitude to all the Colleagues, whose experience, endurance and willingness to cooperate materialized in this volume, which hopefully will become a useful and up-to-date source textbook in the field of wavelets applied to chemistry. Beata Walczak

Katowice, November 1999

vii CONTENTS

PREFACE LIST OF CONTRIBUTORS

XV

PART I: THEORY CHAPTER 1 FINDING FREQUENCIES IN SIGNALS: THE FOURIER TRANSFORM (B. van den Bogaert) Introduction The Fourier integral Convolution Convolution and discrete Fourier Polynomial approximation and basis transformation The Fourier basis Fourier transform: Numerical examples Fourier and signal processing Apodisation CHAPTER 2 WHEN FREQUENCIES CHANGE IN TIME; TOWARDS THE WAVELET TRANSFORM (B. van den Bogaert) 1 Introduction 2 Short-time Fourier transform 3 Towards wavelets 4 The wavelet packet transform CHAPTER 3 FUNDAMENTALS OF WAVELET TRANSFORMS (Y. Mallet, O. de Vel and D. Coomans) Introduction Continuous wavelet transform Inverse wavelet transform Discrete wavelet transform Multiresolution analysis Fast wavelet transform Wavelet families and their properties Biorthogonal and semiorthogonal wavelet bases

3 4 5 8 9 13 18 22 28 33

33 35 40 53 57

57 59 63 65 65 74 76 79

viii CHAPTER 4 THE DISCRETE WAVELET TRANSFORM IN PRACTICE (O. de Vel, Y. Mallet and D. Coomans) 1 Introduction 2 Introduction to matrix theory 2.1 Patterned matrices 2.2 Matrix operations 2.3 Some matrix properties 3 Matrix representation of the discrete wavelet transform 3.1 The discrete wavelet transform for infinite signals 3.2 Discrete wavelet transform for signals with finite-length

85

85 85 86 88 89 91 91 97

CHAPTER 5 MULTISCALE M E T H O D S FOR DENOISING AND COMPRESSION (M.N. Nounou and B.R. Bakshi) 1 Introduction 2 Multiscale representation of signals using wavelets 3 Characterization of noise 3.1 Autocorrelation function 3.2 Power spectrum 3.3 Wavelet spectrum 4 Denoising and compression 4.1 Denoising and compression of data with Gaussian errors 4.2 Filtering of data with non-Gaussian errors 5 On-line multiscale filtering 5.1 On-line multiscale filtering of data with Gaussian errors 5.2 OLMS filtering of data with non-Gaussian errors 5.3 Hints for tuning the filter parameters in multiscale filtering and compression Conclusions

119

CHAPTER 6 WAVELET PACKET TRANSFORMS AND BEST BASIS ALGORITHMS (Y. Mallet, D. Coomans and O. de Vel) 1 Introduction 2 Wavelet packet transforms 2.1 What do wavelet packet functions look like? 3 Best basis algorithm

151

CHAPTER 7 JOINT BASIS AND JOINT BEST-BASIS FOR DATA SETS (B. Waiczak and D.L. Massart) 1 Introduction 2 Discrete wavelet transform and joint basis 3 Wavelet packet transform and joint best-basis

165

119 121 123 124 124 126 126 126 136 139 141 145 147 148

151 151 154 155

165 167 171

CHAPTER 8 THE ADAPTIVE WAVELET A L G O R I T H M FOR DESIGNING TASK SPECIFIC WAVELETS (Y. Mallet, D. Coomans and O. de Vel) 1 Introduction Higher multiplicity wavelets 2 m-Band discrete wavelet transform of discrete data 3 4 Filter coefficient conditions Factorization of filter coefficient matrices 5 6 Adaptive wavelet algorithm Criterion functions 7 Introductory examples of the adaptive wavelet algorithm 8 8.1 Simulated spectra 8.2 Mineral spectra Key issues in the implementation of the AWA 9

177

PART II: APPLICATIONS

203

CHAPTER 9 A P P L I C A T I O N OF WAVELET T R A N S F O R M IN PROCESSING C H R O M A T O G R A P H I C DATA (F.-t. Chau and A.K.-m. Leung) 1 Introduction Applications of wavelet transform in chromatographic studies 2 2.1 Baseline drift correction 2.2 Signal enhancement and noise suppression 2.3 Peak detection and resolution enhancement 2.4 Pattern recognition with combination of wavelet transform and artificial neural networks Conclusion

205

CHAPTER 10 APPLICATION OF WAVELET T R A N S F O R M IN E L E C T R O C H E M I C A L STUDIES (F.-t. Chau and A.K.-m. Leung) 1 Introduction 2 Application of wavelet transform in electrochemical studies 2.1 B-spline wavelet transform in voltammetry 2.2 Other wavelet transform applications in voltammetry 3 Conclusion

225

CHAPTER 11 APPLICATIONS OF WAVELET T R A N S F O R M IN S P E C T R O S C O P I C STUDIES (F.-t. Chau and A.K.-m. Leung) 1 Introduction

241

177 179 180 185 186 189 191 194 194 196 199

205 206 207 208 210 219 220

225 225 225 233 236

241

2 2.1 2.2 2.3 3 3.1 3.2 3.3 4 5

Applications of wavelet transform in infrared spectroscopy Novel algorithms for wavelet computation in IR spectroscopy Spectral compression with wavelet neural network Standardization of IR spectra with wavelet transform Applications of wavelet transform in ultraviolet visible spectroscopy Pattern recognition with wavelet neural network Compression of spectrum with wavelet transform Denoising of spectra with wavelet transform Application of wavelet transform in mass spectrometry Application of wavelet transform in nuclear magnetic resonance spectroscopy Application of wavelet transform in photoacoustic spectroscopy Conclusion

243 244 248 250 250 251 251 253 254 255 256 257

CHAPTER 12 APPLICATIONS OF WAVELET ANALYSIS TO PHYSICAL CHEMISTRY (H. Teitelbaum) 1 Introduction 2 Quantum mechanics 2.1 Molecular structure 2.2 Spectroscopy 3 Time-series 3.1 Chemical dynamics 3.2 Chemical kinetics 3.3 Fractal structures 4 Conclusion

263

CHAPTER 13 WAVELET BASES FOR IR LIBRARY COMPRESSION, SEARCHING AND RECONSTRUCTION (B. Walczak and J.P. Radomski) 1 Introduction Theory 2 2.1 Wavelet transforms 2.2 Compression of individual signals 2.3 Data set (library) compression 2.4 Compression ratio 2.5 Storage requirements 2.6 Matching criteria 2.7 The data 3 Results and discussion 3.1 Principal component analysis applied to IR data compression 3.2 Individual compression of IR spectra in wavelet domain 3.3 Joint basis and joint best-basis approaches to data set compression 3.4 Matching performance 4 Conclusions

291

263 264 264 273 274 274 279 282 285

291 292 292 292 293 294 295 296 296 297 297 298 303 305 308

CHAPTER 14 APPLICATION OF THE DISCRETE WAVELET TRANSFORMATION FOR ONLINE DETECTION OF TRANSITIONS IN TIME SERIES (M. Marth) 1 Introduction 2 Early transition detection 3 Application of the DWT 4 Results and conclusions

311

CHAPTER 15 CALIBRATION IN WAVELET DOMAIN (B. Walczak and D.L. Massart) 1 Introduction 2 Feature selection coupled with MLR 2.1 Stepwise selection 2.2 Global selection procedures Feature selection with latent variable methods 3 3.1 UVE-PLS 3.2 Feature selection in wavelet domain 4 Illustrative example 5 Conclusions

323

CHAPTER 16 WAVELETS IN P A R S I M O N I O U S FUNCTIONAL DATA ANALYSIS M O D E L S (B.K. Alsberg) 1 Introduction 2 Functional data analysis 2.1 From vectors to functions 2.2 Spline basis 2.3 Non-linear bases 2.4 Wavelet bases Methods for creating parsimonious models 3 3.1 The simple multiscale approach 3.2 The optimal scale combination (OSC) method 3.3 The masking method 3.4 Genetic algorithms 3.5 The dummy variables approach 3.6 Mutual information 3.7 Selecting large w coefficients 4 Regression and classification 4.1 Regression 4.2 Classification Example applications 5 5.1 Regression

351

311 311 315 319

323 324 324 325 326 328 331 333 347

351 352 354 355 357 358 361 362 366 367 369 369 372 373 375 375 377 38O 38O

xii 5.2 Classification 5.3 Conclusion

391 405

CHAPTER 17 MULTISCALE STATISTICAL PROCESS CONTROL AND MODEL-BASED DENOISING (B.R. Bakshi) 1 Introduction 2 Wavelets 3 General methodology for multiscale analysis, modeling, and optimization 4 Multiscale statistical process control 4.1 MSSPC methodology 4.2 MSSPC optimization 5 Multiscale denoising with linear steady-state models 5.1 Single-scale model-based denoising 5.2 Multiscale Bayesian data rectification 5.3 Performance of multiscale model-based denoising 6 Conclusions

411

CHAPTER 18 APPLICATION OF ADAPTIVE WAVELETS IN CLASSIFICATION AND REGRESSION (Y. Mallet, D. Coomans and O. de Vel) 1 Introduction 2 Adaptive wavelets and classification analysis 2.1 Review of relevant classification methodologies 2.2 Classification assessment criteria 2.3 Classification criterion functions for the adaptive wavelet algorithm 2.4 Explanation of the data sets 2.5 Results 3 Adaptive wavelets and regression analysis 3.1 Review of relevant regression methodologies 3.2 Regression assessment criteria 3.3 Regression criterion functions for the adaptive wavelet algorithm 3.4 Explanation of the data sets 3.5 Results

437

CHAPTER 19 WAVELET-BASED IMAGE COMPRESSION (O. de Vel, D. Coomans and Y. Mallett) 1 Introduction 2 Fundamentals of image compression 2.1 Performance measures for image compression 3 Image decorrelation using transform coding

457

411 412 414 415 416 418 422 422 425 430 433

437 437 437 440 440 442 444 448 448 450 452 452 453

457 459 461 462

xiii 3.1 3.2 3.3 4

The Karhunen-Loeve transform (KLT) The discrete cosine transform (DCT) Wavelet transform coding Integrated task-specific wavelets and best-basis search for image compression

462 463 465 473

CHAPTER 20 WAVELET ANALYSIS AND P R O C E S S I N G OF 2-D AND 3-D ANALYTICAL IMAGES (S.G. Nikolov, M. Woikenstein and H. Hutter) 1 Introduction 2 The 2-D and 3-D wavelet transform 3 Mathematical measures 4 Image acquisition 4.1 SIMS images 4.2 EPMA images Wavelet de-noising of 2-D and 3-D SIMS images 5 5.1 De-noising via thresholding 5.2 Gaussian and Poisson distributions 5.3 Wavelet de-noising of 2-D SIMS images 5.4 Wavelet de-noising of 3-D SIMS images Improvement of image classification by means of de-noising 6 6.1 Classification 6.2 Results Compression of 2-D and 3-D analytical images 7 7.1 Basics 7.2 Quantisation 7.3 Entropy coding 7.4 Results 8 Feature extraction from analytical images 8.1 Edge detection 8.2 Wavelets for texture analysis 9 Registration and fusion of analytical images 9.1 Image registration 9.2 Image fusion 10 Computation and wavelets 11 Conclusions

479

INDEX

551

479 482 487 487 487 488 488 488 491 491 496 502 5O2 503 506 506 509 509 509 513 513 521 526 526 535 540 542

This Page Intentionally Left Blank

XV

LIST OF CONTRIBUTORS B.K. Alsberg Department of Computer Science, University of Wales, Aberystw),th, Ceredigion SY23 3DB, UK e-mail: [email protected] Bhavik R. Bakshi Department of Chemical Engineering, The Ohio State University, 140 West 19th A venue, Columbus, OH 43210, USA e-mail: [email protected] Bas van den Bogaert Solvay SA, Rue de Ransbeek 310, DCRT/ACE, Industrial IT and Statistics, 1120 Brussels, Belgium e-mail." Bas. [email protected] Foo-tim Chau Department of Applied Biology and Chemical Technology, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong, People's Republic of China e-mail." BCFTCHA [email protected] Danny Coomans Statistics and Intelligent Data Analysis Group, School of Computer Science, Mathematics and Physics, James Cook University, Townsville, Queensland 4811, Australia e-mail: Danny. [email protected] H. Hutter Research Group on Physical Analysis and Computer Based Analytical Chemistry, Institute of Analytical Chemistry, Vienna University of Technology, Getreidemarkt 9/151, Vienna 1060, Austria e-mail: h. [email protected]

Alexander Kai-man Leung Department of Applied Biology and Chemical Technology, The Hong Kong Polytechnic UniversiO', Hung Hom, Kowloon, Hong Kong, People's Republic of China e-mail." kmleung(~fg702-6.abct.poO'u.edu.hk Yvette Mallet Statistics and Intelligent Data Analysis Group, School of Computer Science, Mathematics and Physics, James Cook UniversiO', Townsville, Queensland 4811, Australia e-mail: Yvette.Mallet~jcu.edu.au Michael Marth Freiburg Materials Research Center FMF, University of Freiburg, Germany D.L. Massart Pharmaceutical Institute, Vr(je Universiteit Brussel, Laarbeeklaan 103, B-1090 Brussels, Belgium e-mail: fabi@ vub. vub.ac.be Stavri G. Nikolov Image Communications Group, Centre for Communications Research, University of Bristol, Merchant Venturers Building, Woodland Road, Bristol BS8 1UB, UK e-mail.'stavri.nikolo v~ ~bristol.ac.uk Mohamed N. Nounou Department of Chemical Engineering, The Ohio State University, 140 West 19th Avenue, Columbus, OH 43210, USA Jan P. Radomski Interdisciplinary Center for Mathematical and Computational Modeling,

xvi

Warsaw University, Pawinskiego 5A, 02-106 Warsaw, Poland e-mail: [email protected] Heshel Teitelbaum Department of Chemistry, University of Ottawa, Ottawa, Ontario, Canada KIN 6N5 e-mail: [email protected] Olivier de Vel Statistics and Intelligent Data Anah'sis Group, School of Computer Science, Mathematics and Physics, James Cook University, Townsville, Queensland 4811, Australia e-mail." olivier.devel@ dsto.defence.gov.au

Beata Walczak blstitute q[ Chemistry, Silesian University, 9 Szkolna Street, 40-006 Kato~rice, Poland e-mail: beata(a tc3.ich.us.edu.pl M. Wolkenstein

Research Group on Physical Anah'sis and Computer Based Analytical Chemistry, hlstitute o1 Anah'tical Chemistry, Vienna Universit)" of Technoiog)', Getreidemarkt 9,/151, Vienna 1060, Austria e-mail. wolken(a mail.zserv.tuwien.ac.at

Part I

Theory


Wavelets in Chemistry Edited by B. Walczak 9 2000 Elsevier Science B.V. All rights reserved

CHAPTER 1 Finding Frequencies in Signals: The Fourier Transform Bas van den Bogaert Solvay SA, DCRT/ACE, Industrial IT and Statistics, Rue de Ransbeek 310, 1120 Brussels, Belgium

I Introduction This is a chapter on the Fourier transform. One may wonder: why speak of Fourier in a book on wavelets? To be honest, there are plenty of people that learn to use and appreciate wavelets without knowing about Fourier. You might be one of them. Yet, all those involved in the development of wavelets certainly knew Fourier, and as a consequence, wavelet literature is full of Fourier jargon. So, whereas you may not need to know Fourier to apply wavelets, you probably will need to know it in order to appreciate the literature. The goal of this chapter is to introduce Fourier in a soft way. Fourier has a rather bad reputation amongst chemists, the reputation of something highly mathematical and abstract. We will not argue with that. Part of Fourier is indeed inaccessible to the less mathematically inclined. Another part, however, is easy to grasp and apply. The discrete Fourier transform in particular, as one might use it in digital signal processing, has a simple basic structure and comprehensible consequences. It is also that part of Fourier that links well to the wavelet transform. The discrete wavelet transform, that is, the kind you are most likely to be using in the future. What makes these discrete transforms easy to understand is that they have a geometrical interpretation. In terms of linear algebra: they are basis transformations. Nevertheless, we will take a glance at pure and undiluted Fourier: the transform in integral form. Not that we need it, but it would be odd not to mention it. Moreover, useful notions from the Fourier integrals can be effectively used, if only loosely, for discrete Fourier.

2

The Fourier integral

Let us look the beast in the eye: +vc

F(m)-

/

f(t)e-i~~

'/

(1)

nt-OC

r(t) - ~

F(m) e+imtdm

(2)

--0(2

with i2 = - - 1 We have some function f of t, where t is often associated with time, so that we can think of f as a signal. Eq. (1) transforms f into F, where F is no longer a function in t, but in m. When we associate t with time, we may think of m as frequency, as the exponential may be written as: e -i~ - cos(rot) - i sin(rot)

(3)

Eq. (2) does the same thing as 1, but in the other direction. It takes F of m and transforms it into f of t. We see that in order to go in the other direction, the sign of the exponent has been swapped from minus to plus. Furthermore, there is a multiplication factor outside the integral. The factor is needed to get back to the same size if we were to go from f to F and back again to f. We could also have defined a set of Fourier integrals putting that factor in the first equation, or dividing it over both. Eqs (1) and (2) have everything to scare off chemists. There are integrals, complex numbers, and m is said to represent frequency, which leaves us pondering about the meaning of negative values for it. This is pure mathematics, it seems. Yet, this form of Fourier is not just a toy for mathematicians. It is useful for mathematical reasoning on models of the real world. Analytical solutions may be obtained for real-world problems. Useful or not, our present vision of the world becomes increasingly digital, we observe and manipulate the real world using digital tools that discretise it. Most often, signals are not continuous and infinitely long, they are discrete

and of finite length. Mathematics exist that allow travelling back and forth between the continuous and the discrete representation. When the continuous Fourier reasoning is to be used for our discrete data, the additional maths do not simplify things. Arguments that are compelling in continuous Fourier may get twisted upon translation to the digital domain. In fact, the discrete representation of Fourier analysis may seem better off without the burden of its continuous ancestor. However, it is possible to loosely apply continuous Fourier reasoning to discrete settings, reasoning that gives a feeling for what happens when one filters a signal, for instance. The most interesting example of such reasoning involves convolution, an operation that is ubiquitous in the domains where Fourier is used. It will be discussed in Section 3.

3

Convolution

We will introduce the concept of convolution using a simple example from systems analysis. We will take a small side step to introduce the basics of the system. Suppose we have a single reactor that has one flow going in and one going out as depicted in Fig. 1. Suppose the reactor is a so-called continuously stirred tank reactor, or CSTR. A CSTR is a well-known theoretical concept. In a CSTR, mixing is by definition perfect. As soon as some material arrives, it is instantaneously completely and homogeneously dispersed throughout the reactor. Imagine there is water flowing through. Now we spike the input with some ink. When the ink arrives at the reactor, we will immediately see it appear in the output. Not as strong as it was, but diluted to the volume of the reactor. After the initial jump, the colour of the output will gradually fade away, as the ink is washed out of the reactor. In the beginning, when the

i

J

CSTR

Fig. 1 A C S T R and its hnpuise response.

concentration is high, the material is washed away quickly and the concentration drops fast. As the concentration becomes lower, the rate at which the material leaves with the outflow becomes lower, i.e. the concentration drops more slowly. In short: the rate at which the concentration decreases is inversely proportional to the current concentration. This amounts to a simple differential equation. When we solve this equation we obtain a formula of the concentration profile in the output after a single spike in the input. This is called the impulse response of the CSTR. c(t)

-

(4)

c ( 0 ) e TM

where k, the time constant, depends on the ratio of flow to reactor volume, and c(0), the initial concentration, depends on the amount of ink introduced and, again, reactor volume. A high k means that the reactor is flushed rapidly and the concentration drops fast. Now, what would happen if we were to spike the input several times, with some time in between? As depicted in Fig. 2, the concentration profile in the output would be the sum of the responses to the individual spikes. When we cut through the output profile at some moment t we see that the contributions correspond to different positions on the basic impulse response. For the first spike, we are already on the tail of the response, for the last we are still close to the top. To get another view, we start by taking the mirror image of the impulse response. Its exponential slope will be to the left and its perpendicular edge will be to the right. The three contributions at time t are obtained by multiplying the input with the mirrored impulse response positioned at time t. In this example the input consists of the three impulses in the midst of zeros. Therefore, the multiplication leads to a sampling of three points from the (mirrored) impulse response. In general, the input is a

CSTR

Fig. 2 A series of impulses on a CSTR, the responses and their envelope.

continuous signal, which can be regarded as a series of infinitely closely spaced impulses of different amplitude, and there is an infinite number of contributions. In the example, the overall output signal at time t is the sum of the three products. In general, it is the integral of the product of two signals, namely the input and the mirrored impulse response positioned at time t. To obtain the entire output signal, we drag the mirrored impulse response over the input. At each position t of the impulse response, we multiply and sum to get the output signal in t. This dragging process is illustrated by Fig. 3. That operation is called convolution. Any input signal can be thought of as a series of impulses and the output signal will be the convolution of the impulse response and the input. In other words: if we know the impulse response of a system, we can derive what the output will be like given some input. The formal description of a convolution is the convolution integral: g(t)-

/

f(z)h(t-z)d~

(5)

--OO

where g(t) could be the output of a system with impulse response h(t) and input f(t). Impulse responses are usually relatively easy to come by, but the effect of a convolution is often difficult to picture without actually evaluating the convolution integral, which is seldom a simple task. This is where Fourier comes in. The convolution theorem states that a convolution in the t-domain is equivalent to a multiplication in the co-domain. What we need to do is Fourier transform the input and the impulse response. The product of these functions is the Fourier transform of the output. So if we want the output, we need to transform back.

Fig. 3 Every point of the envelope response can be seen as a multiplication of the input impulses and a mirrored impulse response.

This may not seem to be a simplification, but in many cases it is, and in the following, we will frequently use this property.

4

Convolution and discrete Fourier

In the discrete Fourier setting, the convolution theorem still holds, but with an important modification. The multiplication of discrete Fourier transforms corresponds to a convolution that is circular. One can imagine that the convolution described above, dragging some impulse response along the signal, gets into trouble when we have a finite set of data. At the beginning and the end of the signal, the shape we are dragging along it will stick out, as depicted in Fig. 4. The simplest solution is to exclude those sections of the signal in the output, i.e. to start the convolution on the position where the entire shape encounters signal, and to stop when the front of the shape meets the end of the signal. That would make the output signal shorter than the input. An alternative could be to simply sum the remaining products when, in some position, the shape sticks out. That would be equivalent to assuming that the signal is zero beyond the available data. In the CSTR example above that was a reasonable assumption, but in general it is not. Yet another way of solving the problem of missing data at the edges of the signal is to think of the signal as something that repeats itself. After the end

/ Fig. 4 Convolution at beginning o[ discrete signal. The impulse response is too long.

of the signal, we will suppose it starts over as at the beginning. Hence, before the start, we will suppose the signal behaved as it does at the end. This is a circular convolution, as depicted in Fig. 5. In a discrete convolution, either we lose part of the signal, or we deform that part. As long as the signal is long compared to the shape it is being convoluted with, we do not worry too much about the deformation. Under those circumstances, we will loosely use the convolution theorem, as if the circular aspect were not there.

5

Polynomial approximation and basis transformation

This section will elaborate the following ideas. The Fourier transform can be interpreted as a polynomial approximation of a signal, where the polynomial is a series of sines (and cosines) of increasing frequency. When the degree of the polynomial is high enough, the approximation will be perfect: we will accurately reproduce the entire signal. At that point, the polynomial can be seen as a basis for signal space, and the calculation of the coefficients boils down to a basis transformation. Suppose we have a set of n data points (xi, Yi), a calibration line, for example. The data are plotted in Fig. 6.

[_....._

Fig. 5 In a circular convolution, the signal is wrapped around to avoid the problem of an impulse response that is too long at the edges of the signal.

10

Fig. 6 Scatter plot of a set of x - y data.

We wish to describe y as a function of x. A straight line seems okay as a first approximation. In that case the model is a first-order polynomial: y = [30 + [31x + t;

(6)

where the error term t; describes the fact that the Yi will not perfectly fit our model, due to m e a s u r e m e n t error and to model insufficiency. We might want to add a quadratic term if we suspect curvature, i.e. go to a second-order polynomial: y - 130 + 131x + 132x2 + t:

(7)

Note that the e of Eq. (7) is not the same as in Eq. (6). If a second-order is not sufficient we try a third-order etc. If we use a polynomial of order n - 1, we are sure to perfectly describe the data. There would be no degrees of freedom left. Fig. 7 shows the first orders of a p p r o x i m a t i o n of the data of Fig. 6. In general, a perfect description is not what we aim for. As the responses have not been measured with infinite precision, a perfect description would go beyond describing the process we set out to observe. It would describe the m e a s u r e m e n t error as well. In a polynomial approximation, we would typically stop at an order well below the limiting n - 1. In other words, we suspect the higher-order terms to be representing noise. That is a general principle we will also encounter in Fourier. F o r the calculation of the coefficients in our polynomial model we use linear regression, i.e. a least squares projection of the data onto the model. This is very easy to write down in matrix notation. Our model becomes: y-

XI$+ t;

(8)

25

15 10

l

First order

Zero order

9 OoOoo

'~[

9"'.;~.

1

9o

51~176 O" 0 25 .

~~

01

5

.~ "

5

10

9

-

15

,

0

15

Second order . .

.

0

10

5

-

15

J

20

Tenth order

.,,,a'

25

"

20

9

10

0

5

10

15

20

Fig. 7 Polynomial approximation ol'orders O, 1, 2 and lO for the data in Fig. 6.

where y is the n-vector of responses Yl to y,,, t; the n-vector of residual errors, p the p-vector of the coefficients if the polynomial is of order p - 1 and X the n • p model matrix. In case of a second-order model, X can be constructed as"

X

-

l

1 xl x{/ 9

[

9

1

Xn

x n2

(9)

The coefficients [I are estimated using" -

( x T x ) - ' XVy

(10)

The matrix inversion is a crucial element. In the ideal situation, the columns of X are orthogonal. That means that x T x is diagonal and the matrix inversion boils down to simple divisions. We can go one step further and normalise those orthogonal columns, making XTX the identity matrix and allowing us to write" ~--XTy

(11)

Each coefficient 13j can be calculated independently as the inproduct of the response vector y and column j of X: n

[3J -- Z i=l

xi,jYi

(12)

12 If X is constructed simply by adding increasing powers of the basic x, as in Eq. (9), x T x is not diagonal. However, it is possible to construct polynomials in x that do result in'diagonal xTx, i.e. to construct orthogonal polynomials. A simple solution is the Chebychev series. If the xi are equidistant and we rescale them to {0, 1,... ,n}, the following series of polynomials is orthogonal: p0 = 1 Pl - x - ~ n

1

lk2 (n + 1) 2 - k 2 Pk+l - - P l P k - - ~ 4k 2 - 1 Pk-l

(1 < k 2, m E Z +, then the discrete wavelet transform is defined ~X2

FDWT(j, k) -- m j/2 /

f(t)~(m j - k) dt

j, k E Z

--3C

Typically, m -- 2 [4,5,6] so that, a = 2 -j

b = 2-Jk

in which case the mother wavelet is stretched or compressed by factors of two. Wavelets with m > 2 are sometimes referred to as higher multiplicity wavelets see Chapter 8. Our immediate discussion will however assume that m = 2 unless otherwise stated. The main difference between the continuous wavelet transform and the discrete wavelet transform (of continuous functions) is that the wavelet is stretched or dilated by 2 -j for some integer j, and translated by 2-Jk for some integer k. For example if j = 2, the children wavelets will be dilated by 2 -2 - 8 8 and translated by 1 k.

5

Multiresolution analysis

Multiresolution analysis (MRA) [7,8,9] provides a concise framework for explaining many aspects of wavelet theory such as how wavelets can be constructed [1,10]. M R A provides greater insight into the representation of functions using wavelets and helps establish a link between the discrete wavelet transform of continuous functions and discrete signals. M RA also allows for an efficient algorithm for implementing the discrete wavelet transform. This is called the fast wavelet transform and follows a pyramidal

66 scheme. Of course it should be stated that M R A still exists in the absence of wavelets, and that wavelets need not be associated with a multiresolution. However, the wavelets which we prefer to use, i.e. those with compact support (non-zero over a finite interval), will, in most instances be generated from an MRA. For these reasons it is desirable to have wavelets which satisfy the properties of a multiresolution. Consider the following example which presents some concepts that we will use when we explain the idea of a multiresolution analysis in greater detail. Example 2

Let V0 be a subspace that consists of all functions which are constant on unit intervals k _< t < k + 1 for any k E Z. These intervals are denoted by . . . , [ - 3 , - 2 ) , [-2, 1), [ - 1 , 0 ) , [0, 1), [1,2), [2, 3 ) , . . . An example of such a function is depicted in Fig. 7. You will notice that if we shift f(t) along by 1, then this function still remains in the same space, V0. Hence, if f(t) E V0, then f(t + 1) is also in V0. This property is called a shift invariance or a translation invariance property. Integer translates of any function remain in the same space - this is more generally stated: if f(t) E V0, then f ( t - k) E V0. Notice that if we rescale f(t) by a factor of 2, then this function will be constant on [~, k~__21).The function f(2t)is then in V1. If we translate f(2t) by half an integer, then this function remains in V1. This is demonstrating shift

r

9

-4-

9 " .~

0

-'2

6

.... ~

;~

t

Fig. 7 Example of a piecewise constant function, constant oll integer intervals.

67 invariance again, but at a different scale. Following this pattern, the functions in V2 are constant at [k,k]____!l), the functions in V3 are constant at [gk,k~___tl),and so on. Decreasing the resolution we can say that functions which change value at every second integer, i.e. are constant on [2k, 2(k + 1)] correspond to the space V_~. Note, from this example that the subspaces are nested i.e. V_~ c V0 c V I. Example 3

Fig. 8 depicts the scaling property for another piecewise constant function f(t) over an integer interval. You will observe that f(2 -1 t) is an element of the space V-1 and is "twice as stretched" as the function f(t) which is in the space V0. Conversely, f(2t) is an element of the space V I and is "twice as squashed" as the function f(t). Again the subspaces are nested i.e. V_I c V0 c V l.

f ( 2 1 t ) E V-t

bl

f(t)r

I

f(2t ) e Vl

i

i~

i/2

i

~,

Fig. 8 Piecewise constant functions in

V _ l , V O, V 1 .

68 Now that we have introduced some terminology we continue with the explanation of multiresolution analysis. As the name suggests, M RA allows us to represent functions at different resolutions, which can be likened to wavelets analysing functions through different size windows. A multiresolution divides the space of all square integrable functions L2(R) into a nested sequence of subspaces {Vj}jEz. Each subspace corresponds to a particular scale, just like the functions in Examples 2 and 3 are at different scales in V_ 1, V0 and V1. The subspaces corresponding to the different scales provide the key for representing functions from L 2(R) at different resolutions. The reason being, given some function f(t) E L2(R) then f(t) has pieces in each subspace. Let fvj denote the piece of f(t) deposited in Vj, then fv, is an approximation of f(t) at resolution 2J. We also define fv, to be an orthogonal projection of f(t) onto fvj. This implies that fv, will be the closest approximation of f(t) at resolution 2J, mathematically this is expressed V g(t) E Vj,

IIg(t)- f(t)ll > Ilfvj- f(t)ll.

The subspace Vj contains all the possible approximations of functions in LZ(R) at resolution 2J. For the subspaces to generate a multiresolution, they must satisfy some conditions. It has already been mentioned that the subspaces are nested, this means that Vj E Z, Vj C gj+l. That is, a function at a lower resolution can be represented by a function at a higher resolution. Since information about a function is lost as the resolution decreases, eventually the approximated function will converge to 0, i.e., limj_._~fvj- 0, and the intersection of all subspaces Vj is equal to {0}, or, ["lJ=~ j=_~ Vj - {0}. Conversely, as the resolution increases the approximated function gets progressively closer to the original function l i m j ~ f v j - f(t), and U j=~ vj is dense in L2(R) that is, the space L2(R) is a closure of the union of all subspaces Vj. Where do these subspaces come from? The subspaces {Vj} can be generated from each other by scaling the approximated functions in the appropriate subspace such that, f(t) E Vj ~:# f(2t) E Vj+,,

j E Z.

It can also be stated that integer translates of the approximated functions, remain in the same subspace: f(t) E V j ~ f ( 2 t - k ) E V j ,

j, k E Z .

69

Summarising, the sequence of subspaces {Vj}jc z is a multiresolution of L2(R) if the following conditions are satisfied:

l. Vj C Vj+l j=~ Vj - {0}, [.Jj=_~ j=~ vj is dense in L 2(R) 2. ["]j=_~ 3. f(t) E Vj r f(2t) E Vj+I, 4. f(t) E Vj r

f ( 2 t - k) E Vj,

jEZ j, k E Z

Theorem. If {Vj}jEz is a rnultiresolution of L2(R), then there exists a unique function ~(t) E LZ(R), called a scaling function such that {(~j.k(t) -- 2J/2(~(2Jt- k)} is an orthonormal basis of Vj [8].

Example 4 fv~(t)- ~k~-~ak4~(2t-k) constant.

is the part of f(t) in Vl, where ak is some

Example 5 If we wanted to construct a basis that could be used to represent any piecewise constant function in V0 a simple choice would be the box function (see Fig. 9)" 1 for0< t < 1 d~(t)- 0 otherwise

d)(t)

Fig. 9 The box function 4)(t).

70 This then implies that any function in Vj can be represented by a linear combination of the {~j.k(t)}. Hence, the orthogonal projection of f(t) E L2(R) into gj can be expressed as 3(2'

fvj(t) - ~

cj.k q~j.k(t).

--:3C

The coefficients Cj.k are called scaling coefficients. Since V0 c V~, 3O

4(t)- v'5

lk+(2t- k)

(4)

k=-

Eq. (4) is often referred to as the dilation equation.

Example 6 For the box function 1 0 - l l - l/x~2, thus ~ ( t ) - ~(2t)+ ~ ( 2 t - 1), this is clearly demonstrated in Fig. 10. So how do wavelets enter the picture? Wavelets are basis functions which can be used to represent the information lost in approximating a function at a lower resolution. This difference is called the detailed part of the function. We prefer that this error lies in the orthogonal complement of the Vj's. Consider the difference between approximating a function at resolution 2J and at 2j+l . This difference will lie in the orthogonal complement of Vj which is denoted by Wj such that,

vj+ - vj

4ff2t-l)

~(t) ~(2t)1

1

1

0

1

0

1/2

1

0

Fig. 10 Haar scaling basis functions.

1/2

1

71

In terms of the functions in the subspaces, then

fwj is

fVj+l -- fvj "j-fwj

the orthogonal projection of f(t) into Wj. Further decomposing fv, produces J fvj+I -- fvj_I -J-fwj_I 'J-fwj -- ~

fw i 9

i=-~

Then for some function f(t) we have DC

f(t) - fvj + ( f ( t ) - fvj) - fvj +

Zrwi i=j

and one can then understand how a multiresolution allows us to represent a function at various resolutions. Next, consider how we can represent each fwj. In order to represent the orthogonal projection of f(t) into Wj, it is convenient if we have an orthonormal basis for Wj, just as we had an orthonormal basis for Vj. It can be shown [8] that provided { ~ j . k ( t ) -- 2 j/2 ~ ( 2 J t -- k)} is an orthonormal basis for Vj then there will exist a wavelet basis {qt_j,k(t ) -- 2 j/2 -- q t ( 2 J t - k)} which spans Wj. Since W0 c V1, an expression for the wavelets can be obtained from a linear combination of the scaling functions in the space V1. That is oc

q,(t) - x/2 ~

h k ~ ( 2 t - k)

k=-oc

Example 7 The wavelet function 1

~(t)-~(Zt)-~(Zt-1)-

-1 0

0 7 + 11 B B ( 1 , 2 ) - {band(l, 3)} since 8 _< 3 + 12 BB(1,3) - {band(0, 6), band(0, 7)} since 13 > 7 + 2 when j = 2: BB(2, 0 ) -

{band(I,0), band(0, 3), band(0,4)} since 29 > 6 + 7 + 11

BB(2, 1) - {band(2, 1)} since 15 < 8 + 7 + 2 when j = 3: BB(3, 0) - {band(l, 0), band(0, 3), band(0, 4), band(2, 1)} since 43 > 6 + 7 + 11 + 15 One commonly used cost function particularly in data compression is entropy. If we let wj,~.i denote the ith wavelet packet coefficient band(j, ~) of the wavelet packet transform, then the entropy-like criterion for band(j,~) is defined as follows: E(j ' z ) - _ ~-'~,2j,z.k log W;z,k -~ k

where wj,~.k is the normalized wavelet packet coefficient obtained by ~ where ~ is the kth wavelet packet coefficient in band(j, z).

161

Let x1, x2,... , Xn be any n non-negative real numbers with a given sum, the

\2

log

i=1

is a measure of the scatter or dispersion of the values X I , X 2 , . . . , X n. The more similar the values the larger the value for E, the more concentrated the frequency is in one class, the smaller the value of entropy. It is also worthy to note that 0 • log(0) is defined to be zero.

Example 1 Compute E for each of the following: (a) (b) (c) (d) (e) (f) (g)

1,1,1,1,1,1 E = 1,1,1,2,2,2 E = 1,1,2,2,3,3 E -1,2,3,4,5,6 E = 1,1,1,3,3,3 E = 2,2,10,2,2,2 E = 1,1,1,1,1,6 E --

1.791759 1.599015 1.523619 1.443165 1.423695 0.7188009 0.567067

Example 2 Perform the wavelet packet transform using filter coefficients associated with the Haar wavelet and scaling functions, then, compute the wavelet packet coefficients associated with the best basis using the entropy cost function for the signal x = (0.0000, 0.0491,0.1951,0.4276, 0.7071,0.9415, 0.9808, 0.6716). The Haar low-pass filter coefficients are 10 - 11 - -~2' and the respective highpass filter coefficients are h0 - ~2' hi ~ . The wavelet packet coefficients resulting from the wavelet packet transform are displayed in Fig. 6 and the entropy cost table is displayed in Fig. 7. The best basis is shaded in grey. Another cost function is the threshold cost function which counts the number of coefficients greater than some prespecified value. For example, S-Plus [3] sets the threshold value (thresh) as the median of the absolute value of all the

162

0.0000 0.0347 0.3359 1.4046

0.0491 0.4403 1.6505 0.9296

0.1951 1.1658 0.2868 -0.2015

0.4276 1.1684 0.0018 0.2041

0.7071 0.0347 0.0917 -0.1274

0.9415 0.1644 -0.2718 -0.2571

0.9808 0.1658 0.1408 -0.1260

0.6716 -0.2187 -0.0374 0.0731

Fig. 6 An example of a wavelet packet transform.

1.5360 0.8977 0.2164 0.2785 0.3580

0.0981 0.0579 0.0590

0.1536 0.1071 0.0365 0.0281 0.0836 0.0276 0.0112

Fig. 7 Entropy cost table with best basils" shaded #1 grey.

coefficients in the wavelet packet transform. This cost function can also be useful for data compression. More formally, the threshold cost function for a band(j, ~) of wavelet packet coefficients is defined Threshold(j, ~) - ~

I(l~

> thresh)

k

where I = 1 if I~ > thresh and I = 0 otherwise. Other cost functions include "Stein's Unbiased Risk Estimate" and the Lp norm cost function. For more details we refer the reader to Chapter 5 or the reference [3]. As with the E criterion, the Threshold criterion is minimized since minimizing this criterion will result in a few large coefficients which is typically preferred for data compression. The best basis algorithm for the simulated signal shown in Fig. 2 is presented in Fig. 8. The cost function used here is entropy. It is interesting to mention that the same "best-basis" results for all four cost functions. This may not always be the case however. The next example performs the wavelet packet transform for a linear chirp signal. We show the coefficients in the wavelet packet transform in Fig. 9 and then the best-basis resulting from the entropy criterion in Fig. 10. The best-basis-resulting from the threshold criterion is presented in Fig. 11. Notice that for this example the best-basis changes with the cost function- hence the best in best-basis is really dependent on the cost function and the goal of the waveleteer!

163

Fig. 8 Best basis.for the simulated signal.

,_,+ !,.,,.,,,,~l,i,l.ll,~ll,,lllh,,~llllll~,,."~tllllllil~' ....i,..~r~ " ,,........ ~eve,~ Level 4

+....... ~iili:ili,~Liiii~++iii I,.................... ~,,,,i ................... fill+fill .................. i iiiii"i .... 11 ........ ill ..... 'ii ............... i .... l ....................... i '........................... ..... " II

....

"It ......

; ...........................

Level2

: ......

. ......

'l . . . .

~'

'"

.. ...........................

"

"

: ...............

'~'~

: .......

i .......

'

i .....

"..i .......

i!lii~l!~ii!~i~!ii!!i!~!i!i!i!i!i!..i :1.

:":'

":' " : + ' " : '

I

I

0.0

0.2

'"-' . . . . . .

-

.'":"'

'-':'""

? v-':,.

:

":

:

.

:

.." ':

I

I

I

0.4

0.6

0.8

'.:' :,

:

Fig. 9. Wavelet packet transform of a linear chirp signal.

.'

i

:

:

I

1.0

164

Fig. 10 Best basis for the lineal" chirp signal using the entrop)' cost function.

Fig. 11 Best basis for the linear chirp signal using the threshold cost function.

References 1. M. Wickerhauser, Adapted Wavelet Analysis from Theory to So[tware, AK Peters, (1994). 2. R. Coifman and V. Wickerhauser, Entropy-based Algorithms for Best-basis Selection, IEEE Transactions on Information Theoo', 38 (1992), 496-518. 3. A. Bruce and H. Gao, S + Wavelets User ~ Manual, Version 1.0, Seattle." StatSci, a Division of Mathsoft, (1994).


165

CHAPTER 7 Joint Basis and Joint Best-basis for Data Sets B. Walczak* and D.L. Massart ChemoA C, VUB-Fabi, Laarbeeklaan 103, B-1090 Brussels, Belgium

1

Introduction

In chemometrics we are very often dealing not with individual signals, but with sets of signals. Sets of signals are used for calibration purposes, to solve supervised and unsupervised classification problems, in mixture analysis etc. All these chemometrical techniques require uniform data presentation. The data must be organized in a matrix, i.e. for the different objects the same variables have to be considered or, in other words, each object is characterized by a signal (e.g. a spectrum). Only if all the objects are represented in the same parameter space, it is possible to apply chemometrics techniques to compare them. Consider for instance two samples of mineral water. If for one of them, the calcium and sulphate concentrations are measured, but for the second one, the pH values and the PAH's concentrations are available, there is no way of comparing these two samples. This comparison can only be done in the case, when for both samples the same sets of measurements are performed, e.g. for both samples, the pH values, and the calcium and sulphate concentrations are determined. Only in that case, each sample can be represented as a point in the three-dimensional parameter space and their mutual distances can be considered measures of similarity. Now suppose that the data are indeed organized in a matrix form. These can be, for instance, spectroscopic or chromatographic data, which contain a small number of objects and variables, a high number of objects but a small number of variables, a small number of objects but a high number of variables, or high numbers of both, objects and variables (see Fig. 1).

* Present address: On leave from Silesian University, 40-006 Katowice, Szkolna 9, Poland

166 (a)

(b)

(c)

(d)

_2

Fig. 1 Data sets with different dimensionalitv.

Depending on the type of data, different problems can be encountered. Of course, these problems are associated with the data processing methods applied. Consider for instance Principal Component Analysis (PCA), which is one of the most popular chemometrical methods. PCA can be used for many purposes, e.g. to reduce data dimensionality, to visualize data, to orthogonalize variables, to suppress data noise, etc. If the data set is small or flat (cases (a)-(c) on Fig. 1), very efficient PCA algorithms are available [1,2]. The problems start with a data set belonging to the case (d) in Fig. 1. If the data dimensionality exceeds 104, there is a problem. In other chemometrical techniques of data processing, such as Linear Discriminant Analysis (LDA), the problems occur not only due to big data sets (case d), but also in a case, where the number of variables exceeds that of objects (case c). The same is true for the Neural Networks (NN) approach. Numerous variables, which are very often strongly correlated, are not the best input to NN. In this situation, PCA is often used to reduce dimensionality and orthogonalize variables. New orthogonal features, PCs, speed up NN training and allow reduction of its architecture. This approach, although extremely popular, has a drawback, namely that PCs are global features, which means that local phenomena, that would allow data classification, are spread over numerous PCs. The respective final models are then in some way redundant and unstable. In this situation we would prefer to have local features, describing data variability [3]. The wavelet transform can help to solve such problems. It allows transformation of the data from the original domain (i.e. from the time or space domain) into the frequency-time, or the scale-time domain. New features, i.e. wavelet coefficients, describe local phenomena in a very efficient way and, moreover, allow data compression. As the reader may have already noticed,

167

in preceding chapters the wavelet transform was applied to the individual signals only. What is the difference between the compression of individual signals and of a data set as a whole? In the case of an individual signal, we are looking for the basis optimal for its compression, whereas while dealing with the data set, a joint basis for the compression of all signals has to be found. Only a joint basis allows the uniform data representation required for further processing by chemometrical techniques. Bases, which are optimal for the individual signals from a given set, can differ to a great extent. To have a uniform representation of the signals in the scale-time domain, a joint basis has to be found, which is optimal in the statistical sense for all the signals. In this chapter, we will present a methodology to select such a basis.

2

Discrete wavelet transform and joint basis

Let us consider the Discrete Wavelet Transform (DWT), applied to the set of m signals (e.g. spectra) of length n each, presented in the form of matrix X. If all signals are decomposed by DWT with the same filter and to the same decomposition level, they can be presented as m vectors of length n each in the timefrequency domain, forming matrix Z (see Fig. 2). The information content of signal 1

signal m

signal 2 n

U~I 1~,I+'1

::

n

::::::

. . . .

ill I

liiliilii!!!il!!11!11!1

I:iI"i!::i!111!:ii!:!-!~

n

t 2

n

I I~i !~+ii i +i'-~ ++

n

n

E~:.l::iiiiililiiii:i:+:ii

Fig. 2 Schematic representation of DTW decomposition o[+set o[signals.

168

both matrices, X and Z is exactly the same and so is also the data variance. The total variance of matrix X is equal to the total variance of matrix Z. Now we would like to compress the data in the time-frequency domain, without loss of an essential information about the data variability. This can be done, based on the variance of the wavelet coefficients (matrix Z). For each column of matrix Z, a variance can be calculated and in this way vector v (1 x n) is obtained, which contains n elements, each of them describing the variance of one column of matrix Z (see Fig. 3). The jth element of vector v is defined as: vj-Zi(zij-zj)2/m

fori-l,2

. . . . ,n and j -

1,2,...,m

(1) where: zij denotes the element of matrix Z, i numbers the rows of Z, j numbers the columns of Z, and z.j denotes the mean of the j-th column: zj-~izij/m

for i -

1,2 . . . . . n and j -

1.2 . . . . , m

(2)

In the matrix-vector notation used in Matlab, the vector v can be calculated as: v - (sum(Z - o n e s ( m , l ) , mean(Z )).^2))/m Of course, the sum of elements of vector v equals the total variance of the data. The elements of vector v can be sorted in decreasing order. The first n' coefficients, their sum exceeding a predefined percentage of variance, form the joint basis for all signals. time domain

wavelets domain n

n

n

compression

DWT

m

v

I

n

J---)~

varience vector

n'

compressed joint basis

Fig. 3 Schematic representation o f data compression in the joint basis.

169

Example A Near-Infrared data set, containing 1000 spectra of length 512 each, was decomposed by DWT (filter Daubechies No. 4) and the variance vector, v, calculated for the decomposed spectra, is presented in Fig. 4(a). As can easily be noticed, many elements of vector v are very small. The elements of v, ranked according to their values, are presented in Fig. 4(b). By changing the scale of Fig. 4(b), the first 100 elements of v can be visualized. The sum of all elements of vector v is equal to the data variance. For the data set studied, the total variance equals 3.6686. The cumulative sum of the elements of v, sorted in the decreasing order and expressed in percentage of the explained variance, is presented in Fig. 4(d). As can be noticed, the percentage of explained variance increases very rapidly for the first 20 coefficients, and then slowly approaches the limit of 100%. If the percentage of the explained variance needed is predefined as, e.g. 99%, then the 62 top coefficients are needed to describe the data set. The sum of these 62 top elements of vector v is equal to 3.6313, thus corresponding to 99% of the

(a)

(b)

1

1

0.5

~I

ol o

200 400 coefficients

0

(c)

200 400 sorted coefficients

(d)

0.01

0.005

0

0

50 sorted coefficients

100

00 2O 4O number of retained coefficients

Fig. 4 Elements o f the variance vector for the NIR data set (a): elements of variance vector sorted according to their amplitude (b), zoom on the top 100 elements o f variance vector sorted (c) and cumulative percentage o[the explained variance versus the number of the retained coefficients (d).

170

total variance (100* (3.6313/3.6686)= 99%). In other words, the data matrix Z with 512 columns can be compressed to a data matrix Z* with only 62 columns, corresponding to those of the top 62 elements of vector v. In Fig. 5, one randomly selected spectrum from the data set studied is shown, reconstructed with 10, 20 and 62 top wavelet coefficients from the selected basis. (a)

spectra 1

-

residuals ~/~

< 0.5

O0

200 460 variable number 0.1

(b) 1

0.05 < 0.5

(3

~

-0.0_~ -0.1 0

200 400 variable number

0.1

(c) 1 i ....

0.05

/
2. When m > 2, wavelets are referred to as higher multiplicity wavelets [7,8,9,10]. For higher multiplicity wavelets, there exists a single scaling function defined by ZX2

4'(t)- ~

Z

lkqb(mt- k)

k=-~c

which generates m -

1 wavelets ~c

~(~) (t) - v ~

~ k---

h~~)4'(mt - k)

z - 1. . . . ,m - 1

:xz

which have m - 1 corresponding sets of high-pass filter coefficients {h~z) }z=lm-1. The normalization constant ~ is used so that the wavelets form an orthonormal basis. We first consider redefining a multiresolution to cater for situations when functions are rescaled by a general factor rn > 2 and then show how the fast wavelet transform (or pyramidal algorithm) is performed for higher multiplicity wavelets. The sequence of closed subspaces {Vj}je z is an m-multiresolution of L2(R) if the following conditions are satisfied [11]" 1. 2. 3. 4.

Vj C Vj+l limj~_~ Vj - ['-'lgj - { 0 } , Vj - U V j is dense in L 2 (R) f(t) E Vj ~=~ f(mt) E Vj+l, j E Z f(t) E V j ~ f ( t - k ) E V j , j, k E Z

The subspace Vj contains all the possible approximations of functions in L2(R) at resolution mJ. The orthogonal projection of some function f(t) E L2(R) into Vj is written as oc

(t)

cj. *j.k(t)

k---

(~

where, d~j,k(t ) -- m j/2 Zk~C__ vc l k ~ ) ( m J t -- k). The orthogonal projection of f(t) into Wj is described by

180

m-1

fwj(t)- Z

Z

a(Z)'l'(Z) "j,k Vj,k (t)

z= 1 k=-oc,

Notice that the wavelet coefficients "j.k a(z) are also indexed by z. The function f(t) can be written as a linear combination of wavelet basis functions m-I

:~c

f(t)- Z

Z

oc

Z

a(z)'l'(z) "*j,kVj,k (t)

z= 1 j=-oc k=-oc

in what is often called the wavelet series representation of f(t). We reserve the term wavelet transform as the procedure which results in the computation of the scaling and wavelet coefficients. When m - 2, a pyramidal scheme is used for computing the scaling and wavelet coefficients. Likewise, a pyramidal algorithm can also be used for calculating the scaling and wavelet coefficients comprising the wavelet transform for higher multiplicity wavelets. That is, the scaling coefficients at some resolution are used to produce the scaling and wavelet coefficients at the next (lower) resolution. This is done as follows oc

Cj-l,i

Z Ik-miCj.k k=-oc OO

d~Z)l,i =

~ k---

3

.(z) nk_miCj.k oo

m-Band discrete wavelet transform of discrete data

Similar recursion formulae exist for computing the scaling and wavelet coefficients in the m-band DWT of discrete data as those derived for the DW'I of continuous functions using higher multiplicity wavelets. Recall that in th~ case of higher multiplicity wavelets there is one scaling function defined b3 one set of low-pass filter coefficients, and m - 1 wavelet functions which wer~ defined by m - 1 sets of high-pass filter coefficients. The DWT with higheJ multiplicity wavelets for continuous functions is likened to performing th~ DWT on discrete data using a filter system which contains one low-pass filte~ and m - 1 high-pass filters. The latter is referred to as an m-band DWT [10

181

of discrete data. For the m-band DWT, the down-sampling rate is by a factor of m. This corresponds to shifting the filter coefficients in each row of the filter matrices by m. This is explained further in the example presented next. A 3-band DWT for the spectrum x = (x0, xl . . . . ,x8) is shown in Fig. 1. There is one low pass and two high pass filters producing one set of scaling (or smoothed) coefficients and two sets of wavelet (or detailed) coefficients. As before, to go from one level to the next, only the scaling coefficients are filtered and the number of coefficients in each band is reduced by one third when moving from one level to the next. We have presented a transform with two levels ( n l e v = 2) Following the same notation as introduced earlier, band(j,r) will be referred to as the ~th band ~ E {0, 1 , . . . , m - 1} at the jth level j E { J , J - 1,... , J - nlev} of the DWT. The band at the top of the tree is band(2,0). At the next level the bands from left to right are referred to as band(I,0), band(I,1) and band(I,2). Similarly, the bands in the last level of the DWT are band(0,0), band(0,1) and band(0,2). Using the notation of Chapter 4, where the low- and high-pass filter coefficients are combined into one matrix, the 3-band DWT outlined in Fig. 1. Going from level 2 to level 1 is written fl -- P W ~ ~

Let us say that N f -

6, then the full matrix expression is written

Xo X!

X2

C2,0 C2,1 C2.2

X3 X4 C2,3 C2A

X5 X6 X7 X8 C2,5 C2.6 C2.7 C2.g

l

Fig. 1 A 3-hand discrete wavelet transform.

182

fCl, 0 '~

tl

0

0

0

0

0

0

0

0'~

el,1

0

0

0

1 0

0

0

0

0

Cl,2 (1) 1.o

0

0

0

0

0

0

1 0

0

0

1

0

0

0

0

0

0

0

d (1) 1,1

0

0

0

0

1 0

0

0

0

0

0

0

0

0

0

0

1 0

0

0

1 0

0

0

0

0

0

0

0

0

0

0

1 0

0

0

\0

0

0

0

0

0

0

1

/ lo h~l)

ll hl l)

12 h~ ')

13 14 h(')3 hi ')

h~2)

hl 2)

h?

h

0

0

0

0

0

(1) 1,2 (2) 1.o d (2) 1,1 A (2)

\ '~1,2 /

•

0 15 h~ i)

0

0

0

/ C2.0 '~

0

0

0

C2.1

2,

0

0

0

C2.2

12

14 h~ l)

15 h~ 1)

C2.3

o

13 h2~ ')

0

0

h~2)

h(l2)

h~?)

h~2)

h(2)"4 h~2)

C2.5

lo

11 hl 1)

12 h~ 1)

0 0

0 0

0 0

13 h3(1)

14 h~l)

15 h~l)

C2,6

h~ 1)

hl 2)

h~2)

0

0

0

h~2)

h~2)

h~2)

\c2.8 /

0

hi '-)

lo

ll

C2.4

C2,7

If we let A denote the matrix of filter coefficients with the first row containing the low-pass filter coefficients and the remaining m - 1 rows the sets of high pass filter coefficients and if Nf is the number of filter coefficients contained in each filter, then A will be an m • Nf matrix. A can be partitioned into m • m sub-matrices as follows A - (Ao Al ...Aq) Here, q is a non-negative integer such that q - (Nf/m) - 1. For our example, there were three filters (m -- 3), with each filter containing six filter coefficients ( N f - 6), hence q - 6 / 3 - 1 - 1 then 10 A

~

ll

) h l 1)

12

~0~12) h 1 2 ) h ~ 2)

13

14

h~2)

h~2)

l)

1(51 ) h5 )

183

could be expressed as A = (A0 A1) with lo 11 12 ) h~ 1) hl 1) h~ 1) Ao -h~2) hl 2) h~2) and 13 14 15 ) h~l) hi ') h~') h~2) h~2) h~2)

A1

An alternative way to describe the DWT is to introduce a convolution matrix for each of the low-pass and high-pass filtering operations. For the case m = 3 and N f - 6 as presented previously, the filter coefficient matrices which decomposed the original data at level 3 to the next lower level 2, would be represented as follows 10 ll 12 13 14 15 0 0 0 '~ 0 0 0 10 Ii 12 13 14 15 13 14 15 0 0 0 10 11 12

)

C2 -

o l,

--

0 h~ l)

D~2)

0

0

h~ 1) h~ l)

h l,

h~ l)

hl l)

0

0

0 0 0/

h~ 1) h~ l) 0

h~ l)

h~ l)

h; l)

h(l l)

h~ l)

0

0

/ h ~ 2 ) h l 2) h~2) h~2) h~1) h~2)

-

0

0

0

h~2) h~2) h~2)

0 /

h~2) h(12) h~2) h~2) h~2) h~2) 0

0

0

h~2) hl 2) h~2)

and the scaling and wavelet coefficients at level one in each of the bands would be calculated by Cl ~ C2c2

dl 1) - D~l)c2 dl 2) - D~2)c2 where 122 --(C2,0 C2.1 C2,2 C2,3 C2.4 C2.5 C2.6 C2.7 C2.8)T r --(Cl,0 Cl.1 Cl,2) T

184

dl 1 ) - (dl 1).0d(lll, dl')) T.2 d{2 ) - (d (2) d (2) A(2))T 1,0 1.1 "1.2 In general, the m-band DWT from some level j to the next lower level j-1 can be computed cj_j = Q g

d~Z_)I --DJZ)lCj_,

z--1,...,

m-

1.

In summation notation one has

Nf-1 ej-l,i- E lkej,mi+k. k=O Nf-1 dJZ)l,i -- E hkZ)Cj'mi+k k=O

for z--

1,..., m-

1.

Periodic boundary conditions have

Cj,k = Cj,mJ+k (Z) _ d!Z)+ k

,k

j.mJ

These operations can be considered equivalent to the discrete wavelet transform of a continuous function using higher multiplicity wavelets. Our applications involve performing the m-band D W T m > 2 for each object vector in a spectral data set containing n spectra each of dimension p. The wavelet (or scaling) coefficients produced from the DWT are used as features for some multivariate method. The m-band DWT has previously been described for a single data vector, but it is more convenient to redefine the transform using a slight change of notation. Let x[J](~) be a column vector containing the coefficients in band(j,r) of the DWT, so that for a given j, the scaling coefficients will be stored in x[J](0) and x[J](r) will be a vector of wavelet coefficients for ~ E {1,... , m - 1}. The D W T from level j to level j - 1 is then described by the matrix operations

x[J-1](0 ) -- Cjx[J](0) x I j - l l ( z ) - D~Z)x[J](0)

z-

1,... , m -

1.

185

The D W T from level j to level j then described by

1 for all spectra comprising a dataset is

X~_,](0) - CjX~I(0) x [ J - ' ] ( z ) - D~Z/x[3](0)

z-

1. . . . . m -

1.

where x[J](~), is the matrix containing the coefficients for the objects which would lie in band(j,~). Or more specifically, if x~](~) denotes the coefficients in band(j,~) obtained for object xi to level j then, this vector will form the ith column in x[J](~). The original data matrix would be represented by

xI,l(o). 4

Filter coefficient conditions

We have demonstrated that it is possible to obtain the discrete wavelet transform of both continuous functions and discrete data points without having to construct the scaling or wavelet functions. We only need to work with the filter coefficients. One may begin to wonder where the filter coefficients actually come from. Basically, wavelets with special characteristics such as orthogonality, can be determined by placing restrictions on the filter coefficients. The restrictions which are imposed on the filter coefficients so that an M R A and orthogonal wavelet basis exist are summarized as follows [6] 1. Orthogonality Z

A k A kT +i

_ ~5(0,9i)I ,

k

where 8(0, i) - 1 if i - 0 , and zero otherwise, I is the identity matrix. 2. The basic regularity condition

k

3. The Lawton matrix Mij - ~

lklk+j-mi k

must have 1 as a simple eigenvalue.

186 If more sophisticated wavelet and scaling functions are required, then more constraints need to be placed on the filter coefficients. We now consider in more detail the factorized form of a wavelet matrix, and show that A can be constructed from some set of normalized vectors, denoted by U l , . . . , Uq, and v. 5

F a c t o r i z a t i o n of filter coefficient matrices

Recall from Section 4, that the wavelet matrix A can be partitioned into m x m submatrices as follows A = (AoAl ... Aq). Provided that the orthogonality condition: ~-]k AkAk+iT --8(0, i) I is satisfied, the wavelet matrix can also be written in the factorized form [1] A = QFI F2.-" Fq. The symbol denotes the "polynomial p r o d u c t " which is defined by (BoB1 "'" Bp-I)~>(CoCICs-I) - (GoGI "'" Gp+s-2) with Gi - ~

BkCi-k k

The factors Fi -- (Rill - Ri)

where Ri is a projection matrix and Q -

~-~i Ai is an orthogonal matrix.

Example If m = 3 and q = 2 then A - (A0 Al A2) with each Aj having dimension 4 • 4 thus, A has size m • m(q + 1) - 3 • 9. Assuming the orthogonality condition is satisfied then A - Q~>FI~F2 = Q(R1 ]I - R1)~(R2 1 - R2) = [QR1R21Q(R1 - 2RIR2 + R 2 [ Q ( I -

R1)(I- R2)]

Our aim is to construct Q and each projection matrix Ri (for i - 1 , . . . , q). We first consider the representation of Q. The regularity condition ~-]k lk -- v/m, places a constraint on the first row of Q. This is equivalent to setting the first row of Q to (1/X/~)l~m where lm denotes an m x 1 column vector of ones.

187 The remaining m - 1 rows are constructed ensuring the orthogonality of Q is maintained. If the last m - 1 rows are calculated by (I - 2vvX)T 9 D Q will be orthogonal. Here, v represents a normalized vector, T is an upper triangular matrix with Tii = i - m and off-diagonal elements equal to 1. The symbol 9 indicates a form of element by element scalar multiplication across two matrices such that B 9 C = G ~ BijCij = G i j . This scalar product of T with some matrix D normalizes the rows of T. The m x m orthogonal matrix Q is partitioned as follows, Q-

((i (1/x/~)lT) -2vvT)T 9

"

Now for the projection matrices. A symmetric projection matrix of rank P can be written R - UU r where Um• is a matrix with orthonormal columns. For the wavelet matrix to be non-redundant we require r a n k ( R 1 ) < rank(R2) < . - - < rank(Rq). That is the individual ranks of the projection matrices form a monotonically increasing sequence [1]. For simplicity we set rank(Rl) = rank(R2) . . . . . rank(Rq) = 1 and Ri where

- - ui uT

uTui-

1.

Example The following example illustrates how A with m = 3 and q = 2 can be constructed. The example begins by defining the column vector v of length m - 1 and two columns vectors ul and u2 both of length m. Let v -- (-0.7918, -0.6107)1

Ul -- (-0.3873, -0.9097, 0.1497)v U2 - -

(-0.9062, 0.1674, 0.3884)a-

First, consider calculating the symmetric projectors R l u2uf.

R1--

0.1500 0.3523 -0.0580

0.3523 0.8276 -0.1362

-0.0580 ) -0.1362 0.0224

ulu~ and R2 =

188 and 0.8212 -0.1517 -0.3520

Re -

-0.1517 0.0280 0.0650

-0.3520\ 0.0650 0.1509

)

Now consider calculating Q. The first row of Q is (1/v/3, 1/v~, 1/x/~), and the remaining two rows are calculated by (I - 2vvr3)(T 9D) where (-2 0

T . D -

1 -1

__ ( - 0 . 8 1 6 5

0

l)(1/~/-6 1 l/v/2 0.4802 -0.7071

1/v~ l/q~

lj )

0.4802) 0.7071

i _ 2vvf _ (--0.2539 --0.9671) --0.96710.2541 which together give Q -

0.5774 0.2073 0.7896

0.5774 0.5802 -0.5745

0.5774 ) -0.7875 -0.2151

Now consider forming the wavelet matrix A. Using the factorized form of the wavelet matrix one has A -

Q(~Ft (~F2

R2) = [QR1R2IQ(RI - 2R1R2 + R2]Q(I- R l ) ( l - R2)].

= Q0(R1

[I -

R1)~(Retl

--

then substituting for Q, R~ and R2 one arrives at the following result for A. A-

0.1542 0.1690 -0.0430

-0.0285 -0.0312 0.0079

-0.0661 -0.0724 0.0184

0.1316 0.3027 0.8258

-0.0456 -0.1179 0.3569

0.2917 -0.2643 0.0069

-0.0198 -0.0451 -0.2488

0.6891 \ -0.5927 0.1234

-0.0285 -0.0312 0.0079

-0.0661) -0.0724 0.0184

where Ao -

0.1542 0.1690 -0.0430

0.6257 0.6566 -0.3336

)

189

A1

---

A2 -

0.1316 0.3027 0.8258 0.2917 -0.2643 0.0069

0.6257 0.6566 -0.3336 -0.0198 -0.0451 -0.2488

-0.0456 \ -0.1179 0.3569

)

0.6891 \ -0.5927 0.1234

)

We have now shown that A can be constructed from the normalized vectors u1,... , Uq and v. Initially, u l , . . . , Uq and v are randomly assigned elements from the uniform distribution. The optimization routine then proceeds to update the elements of these vectors so that some modelling criterion can be optimized.

6

Adaptive wavelet algorithm

In this section we summarize the adaptive wavelet algorithm for designing task specific wavelets. Fig. 2 summarizes the adaptive wavelet algorithm. Step 1 of the algorithm sets values for the parameters m, q, J0 and t0 and Step 2 initializes v and u l , . . . , Uq. Steps 3-6 go about constructing the filter coefficient matrix A, so the m-band DWT can be performed. Step 7 performs the DWT to level J0. Step 8 extracts the coefficients X[J~ and the multivariate criterion measure which we denoted by 3(x[J~ is calculated for the extracted data in Step 9. Step 10 assesses if the stopping criterion of the algorithm has been reached. The stopping criteria are discussed further at the end of this section. If the stopping criterion has not been reached, then the parameters v and {Ui}~i=l} are updated and the algorithm proceeds to Step 3. If some stopping criterion has been reached, then the algorithm proceeds to Step 10 where the Lawton matrix condition is verified. Provided Conditions 1 and 2 of Section 4 hold, then the Lawton matrix condition will not be satisfied for exceptional degenerate cases, thus the Lawton matrix is verified after the adaptive wavelet has been found. Finally, the multivariate statistical procedure can be performed using the coefficients XfJ~ The optimizer used in the adaptive wavelet algorithm is the default unconstrained MATLAB optimizer [12]. Before applying the adaptive wavelet algorithm, the m,q,j0 and t0 values need to be specified. There is no empirical rule for determining these parameters and more experimentation is required to find a suitable combination. We can however suggest some recommendations.

190

1. Set values for

m, q, j,,,~o,

q

V, Ill,..,tlq

Construct = ( R i I [ - Ri)

6. Set A =QOFt 0... OFq

7. Calculate X~J('r'~ ~

8. Calculate modelling criterion

~(xt~(to))

No

9~Yes matrix condition

Model 9

Fig. 2 The adaptive wavelet algorithm for designing task specific wavelets.

Choosing values for m and q." Since m determines the number of bands in the D W T and the down-sampling factor, we choose m such that p / m 0-j~ is an integer value. It is important to recall that m combines with q to determine the number of the filter coefficients (Nf = m(q + 1)). The larger the value for Nf the more parameters that are required to optimized. For this reason another constraint is placed on m so that Nf does not become too large. We constrain q for similar reasons. In this book we consider setting Nf -- 12 and Nf = 16.

191

Choosing values for Jo and ~o. 9 The parameters J0 and t0, simultaneously determine the band(j0, t0) and hence the coefficients X [j0](~0) for which optimization of the discriminant criterion is based. The coefficients X[i0](~0) are later used as inputs to the multivariate statistical method. The value for J0 determines the level of the D W T that the spectra are to be decomposed. A value for J0 should be chosen such that p/m (J-j~ which is the number of coefficients in band(j0, ~0), is suitable (not too large) for the multivariate procedure. For example, in classification the number of objects should be taken into consideration, since classifiers such as Bayesian linear discriminant analysis prefer that the number of variables be much less than the number of observational units. In our application we set the reduced dimensionality of the data set to be 8 or 16. 9 A value for ~0 is also required. To ensure the best J0 and ~0 combination, each of the appropriate values of j0 should be individually tested with each value of ~0. To reduce this computational burden, we have chosen to select ~0 as the band which gives the largest 3(x[J~ at initialization. It is recommended that if one suspects the basic shape of the data will be useful, then optimization over the scaling band may prove worthwhile.

7

Criterion functions

The adaptive wavelet algorithm outlined in Section 6 can be used for a variety of situations, and its goal is reflected by the particular criterion which is to be optimized. In this chapter, we apply the filter coefficients produced from the adaptive wavelet algorithm for discriminant analysis. It was stated earlier that the dimensionality is reduced by selecting some band(j0, t0) of wavelet coefficients from the discrete wavelet transform. It then follows that the criterion function will be based on the same coefficients i.e. X [j~ (%). If the filter coefficients are to be used for discriminatory purposes, then the criterion function should strive to reflect differences among classes. In this section three suitable discriminant criterion functions are described. These discriminant criterion functions are Wilk's lambda (3A), entropy (3E), and the cross-validated quadratic probability measure (~cvqpm)-

192

Wilks Lambda The Wilks' A criterion can be used to test the significance of the differences between group centroids [13]. A smaller value for A is preferred since this indicates a larger significance. Wilks' A is the ratio of the determinant of the within covariance matrix to the determinant of the total covariance matrix and is defined to be

A -ISwl

IS I ISwl ]Sw + SB] where the total covariance matrix S T - SB + Sw is the sum of the between (SB) and within (Sw) covariance matrix.

Entropy Saito and Coifman [14] discuss a cross-entropy measure which can be used to measure how differently vectors are distributed. Let ~(1) and ~(2) be vectors from classes 1 and 2 respectively. If the elements in ~(1) and ~(2) are nonnegative and sum to unity, then cross-entropy is defined by

Ecross

( ) P0 ~i(l) ;(1),~(2) -- Z ~i(1)log i=l ~i(2)

(1)

where Po-length(~(1))-length(~(2)) is the dimensionality of vectors. Eq. (1) is not symmetric, that ~s the measure of discrepancy for Ecross(~(1), g(2)) will be different to that for Ecross(~(2), ~(1))" For our purposes we prefer to use a symmetric criterion which is defined in [14] as

Esym(~(1),~(2)) -- Ecross(~(1),~(2))+ Ecross(~(2),~(1)) Measuring the distinctness of several vectors from different classes, involves

calculating Esym for each combination of vectors. Call this entropy measure the total entropy Etot. For example, the total symmetric entropy for ~(1), ~r and ~(3) is calculated as follows

Etot- Esym(~(1),~(2)) --[--Esym(~(1),~(3)) + Esym(~(2),~(3)) It is necessary to construct a single vector which in some way is representative of the classes, this could for instance be a mean vector. In [14], the repre-

193 sentative vector from each class is an energy vector. More specifically, define the class energy vector of the wavelet coefficients from band(j, 1:) as e [j] ('t)

(r)

diag(Xll)('r))(XP)/(1:))T --

r--1

const

.... '

R.

The denominator is a normalization constant. The numerator is simply the sum of squares of the wavelet coefficients from either the DWT or WPT which occur in the same position of the wavelet trees, where the DWT or WPT has been performed for objects belonging to the same class. The discriminatory criterion function is then [j] [j] (T) ''''' e (R) Lj] (1:)'~ -,~E \/e (r)(T)) -- Etot \(e (1) ]

--~ZEsvm(ell)(~). I

' e[J] (~))(r)

r:rr

Cross-validated quadratic probability measure ( CVQPM) The cross-validated quadratic probability measure (CVQPM) (see Chapter 12 for more details) assesses the trustworthiness of the class predictions made by the discriminant model. The CVQPM ranges from 0 to 1. Ideally, larger values of the QPM are preferred, since this implies the classes can be differentiated with a higher degree of certainty. The CVQPM criterion function based on a band of coefficients X t(~) would be defined as follows i -,~cvqpm ( ) ~(1:) x)

--- 1 ~

r )), - i ) . aQ ( (xpl

n i=l

where (xij ]

1

9

1

~ -, (r]X~r)()) /

~

R

.

2 I

r--I The posterior probability P(rJx) is computed as for Bayesian linear discriminant analysis [15]. That is, P(r}x) - p(xfr)P(r) p(x) where P(r) is the a priori probability of belonging to class r, p(x) the probability density of x and p ( x l r ) = (2rc)-P/21Swl -~ e x p [ - 0 . S ( x - ~ r ) Swl(X_ ~r)T] is the class probability density function which is assumed to follow a normal distribution.

194 8

Introductory examples of the adaptive wavelet algorithm

Section 8 applies the adaptive wavelet algorithm to two sets of data in an attempt to further illustrate the mechanics behind the procedures. The first set of data is simulated, whilst the second considers real spectra of various kinds of minerals. The classifier that we use is Bayesian linear discriminant analysis [15]. 8.1 Simulated spectra

The simulated data containing three classes were previously generated by [14] as follows: - - (6 + q).~[a,b](t) q- ~3(t) Class 1 x(2)t- (6-k- q).Z[a.b](t)(t- a ) / ( b - a)-k- ~;(l) Class 2

X(1)t

x(3),- (6 + rl).Z[a,b](t)(b- t ) / ( b - a) + ~(t)

Class 3

Here 11 and e ( j ) ~ N(0,1), Z[a.b](J)- 1 if j E [a,b] and zero otherwise, a ~ Uz(16, 32) and ( b - a) ~ Ul(32, 96) where Uz denotes the integer-valued uniform distribution. Each of the parameters from the normal and uniform distributions varies for each object. The training data set contains 300 spectra with equal class sizes and the testing data contains 3000 spectra also with equal class sizes. The dimensionality of the data (i.e. number of variables) is 128 of the simulated data is 128. Fig. 3 shows five sample spectra from each class of the training data. For this data we believe that the basic shape of the data will be useful for classification and the scaling band will therefore be considered as a possible candidate. Indeed, the scaling band produced the largest CVQPM of 0.9353 at initialization (see Fig. 4(a)). Thus, ~ = 0 was selected. A marginally smaller CVQPM was produced for band(2,1) at initialization followed by band(2,3) and then band(2,2). Upon termination of the algorithm, the discriminant measure for band(2,0) has further increased to 0.9641 and clearly produces a larger CVQPM than the remaining bands. To test the classification performance of the adaptive wavelet, the coefficients from each of the bands (at level 2) at initialization and at termination of the algorithm were used as inputs to the classifier. The results are summarized for both the training and test data in Table 1. At initialization the coefficients in band(2,0) gave the best classification rates closely followed by band(2,1). At completion the classification performance of band(2,0) has further improved,

195

_51 . . . . . . . . . 0

20 ,

10

40 ,

/

t

60 80 , index , ^ ,--, ~ _

s0 -5;

2b ..... .

[

' .~,~,

^ _a

6b

10

120 ,

class 2

8b index

_.'

100 ,

'

z_ o

_ -

.

'

'class3

'

index

Fig. 3 Simulated .wectra. (a)

Intialization l

"

"

Termination -

]

~o. ~.

1

J ]

~ 0.9

ro

ffSq 0 (b)

l!

1

2 3 band Intialization

0

l[

1 2 3 band Termination

1

Cy

0

1

2 band

3

0

1

2 band

3

Fig. 4 The C V Q P M for the coefficients at initialization and termination of the adaptive wavelet algorithm. Optimization was based on (a) the coefficients X~2](O) and (b) the coefficients ~ 2 ] ( 1 ) .

producing the most favourable results, band(2,1) gave the next best classification results. Since band(2,1) produced quite a competitive CVQPM at initialization, and promising classification results in the previous analysis, optimization over

196

Table 1. The percentage of correctly classified spectra, at initialization and termination of the adaptive wavelet algorithm. Optimization was based on the coefficients Xlel(0).

Initialization Termination

0 92.7 90.3 95.7 93.3

Train Test Train Test

1 89.7 88.4 81.7 80.8

2 58.7 53.8 72.7 67.0

3 67.0 66.7 53.0 50.4

3

~CVQPM ~CVQPM

this band was investigated. As presented in Fig. 4(b), band(2,1) now gives the largest CVQPM at termination, even larger than the scaling band. This is a general observation that can be m a d e - the band which the discriminant measure is calculated, will, in most instances produce the best discriminant measure at completion. The percentage of correctly classified spectra as displayed in Table 2 is also more favourable than the remaining bands, but not as favourable for the testing data as those produced when optimization was based on the scaling coefficients X[2](0) (see Table 1).

8.2 Mineral spectra We now apply the AWA to a mineralogical spectral data set. In this example we will investigate the performance of the various discriminatory criterion functions. These are 3A, 3E, 3CVQPM. The mineral data consist of five classes each representing the different minerals n a m e l y - amphilolite, calsilicate, granite, mica and soil. Both the training and test sets contains 20 spectra per class. The response is Log 1/reflectance, and this was measured for 512 wavelengths 1478, 1480,..., 2500 nm, hence the dimensionality of the data is

Table 2. The percentage of correctly classified spectra, at initialization and termination of the adaptive wavelet algorithm. Optimization was based on the coefficients Xi21(l). r Initialization Train Test Termination Train Test

0 92.7 90.3 86.0 87.6

1 89.7 88.4 96.3 90.6

2 58.7 53.8 61.3 60.5

3 67.0 66.7 61.0 58.8

~3

~CVQPM 3CVQPM

197 | 0 0 - -

r

..

'~~ 80I Amphilolite 60' " 1500 1900

l O 0

"

'

'

-

-

,

80

I,..,

6 0

2300

L

~

1500

_

,

.

1900

.

.

.

,. . . . . .

2300

8o

=

l_,Granite 1~/ I 60 ...................................... t 1500 1900 2300

1500

6 0

lOO~~..-

"

1900 '

.

.

.

.

2300

.~.-, ..

"~ 80 Soil 60~

~500

~9~

wavelength

23b0

Fig. 5 Five sample spectra ~'om each class o / t h e mineralogical data.

512. Fig. 5 shows five sample spectra from each class of the mineralogical data. In this example, the parameters m, q and J0 were set at 4, 3 and 3, respectively. Optimization was based on the coefficients X[3l(z) which gave the maximum 3(X[3](1:0)) at initialization where ~ E {0, 1,2,3}. The results for each of the criterion functions are displayed in Table 3. Here the classification rates of the individual bands at initialization and at completion of the algorithm are shown. Note that the same starting parameters for v, u~ and u2 have been used for the implementation involving the different modelling criteria, hence the same classification results occur at initialization for each of the criterion functions 3A, 3w, and 3CVQPM. The shading indicates which band optimization was based upon. For the Wilk's Lambda criterion, optimization was based on band(3,3), while the entropy criterion optimized over band(3,2). The CVQPM criterion optimized over the scaling band(3,0). Some features which we might expect from the adaptive wavelet algorithm, is that at termination, the band on which optimization was based would outperform the other bands, at least in

198

Table 3. The percentage of correctly classified spectra, using the coefficients Xl31(z) for ~= 0 , . . . , 3 at initialization and at termination of the adaptive wavelet algorithm. The discriminant criterion functions were Wilk's Lambda, symmetric entropy and the CVQPM.

Initialization Termination Termination Termination

r Train Test Train Test Train Test Train Test

0 97 90 98 91 97 86 100 96

1 96 90 96 89 94 89 98 92

2 97 91 95 88 94 90 96 89

3 97 88 100 90 97 87 95 87

~A, ~E, 3CVQPM 3A. 3E 3CVQPM

terms of the percentage of correctly classified training objects. This is the case with the CVQPM and entropy criterion, but is not so, for Wilks Lambda criterion. Overall, for the results presented in Table 2, the CVQPM seems to be performing the most adequately. It is the only criterion function which has improved the test classification rate from those obtained at initialization. One reason why the CVQPM, maybe outperforming the other criterion functions could be due to the fact that optimization and hence classification is based on scaling coefficients. So that a fair comparison could be made, the optimization routine using the Wilk's Lambda, and symmetric criterion functions was repeated, this time forcing optimization over the scaling band. These results are summarized in Table 4. Optimization over the scaling band did improve the results slightly for the Wilk's Lambda and symmetric entropy criterion, but these criterion functions were not able to improve upon the results previously obtained with the CVQPM criterion function. As demonstrated in this example the CVQPM criterion seems to be behaving more appropriately than the other two functions. This may be due to the cross validation being implemented as well as the probability-based measure. In applications to follow we consider the CVQPM function only. More ex-

199

Table 4. The percentage of correctly classified spectra, using the coefficients Xlal(z) for ~= 0 , . . . , 3 at initialization and at termination of the adaptive wavelet algorithm. Optimization was based o n X[31(0) and the discriminant criterion functions were Wilk's Lambda, symmetric entropy and the CVQPM.

Initialization Termination Termination Termination

Train Test Train Test Train Test Train Test

0 97 90 I00 91 ,96 92 100 96

1 96 90 95 89 94 90 98 92

2 97 91 96 86 85 76 96 89

3 97 88 96 90 91 87 95 87

3A,-~E,~CVQPM -~A. ~E ~CVQPM

amples of the AWA algorithm are presented in Chapter 12 where comparisons are made with the predefined filter coefficients.

9

Key issues in the implementation of the AWA

There are several items regarding the adaptive wavelet algorithm which warrant further discussion. These items are now considered separately.

Number of iterations. One can argue that using a prespecified number of iterations in the AWA (as we have done) does not necessarily allow for an optimal value to be found. This is quite a valid statement, but from a practical perspective it is more convenient. It is possible that with more extensive experimentation on real and simulated data, that a more suitable number of maximum iterations could be found. Local and global minima. If the AWA algorithm does converge to an optimal value prior to reaching the maximum number of iterations then one can query if it is indeed a local or global minima. As we have discussed previously, unless the problem is continuous and has only one optimal point, there can be no guarantee that a global optimal value has been found. One suggestion offered in [12] is that starting the optimization routine using different values for parameters at initialization may assist in overcoming this problem. Due to time constraints this was not

200 done for every model produced by the AWA. It was however trialled for a few settings where the criterion function did converge to the same optimal value. There is a need for more experimentation to be conducted with regards to the optimization part of the AWA. Constrained optimization versus unconstrained optimization. In the adaptive wavelet algorithm, it was possible to avoid using constraints which ensured orthogonality. This is due to some clever algebraic factorizations of the wavelet matrix for which much credit is due to [6]. However, one constraint which we have not discussed in very much detail is that the vectors v, u ~ , . . . , Uq are required to have unit length. This normalization procedure occurs during the optimization routine. An alternative strategy which could be employed, is to place constraints on these vectors requiring them to be normalized. Choosing the best (m, q, l, ~) settings. Selecting the (m, q, 1, ~) combination, involved trialling several suitable combinations of these values. Presently, it is unknown how one might be able to predetermine with any degree of certainty which setting combinations may produce more preferred results. In order to determine which settings are more preferable remains to be further explored. Validation without an independent test set. Each application of the adaptive wavelet algorithm has been applied to a training set and validated using an independent test set. If there are too few observations to allow for an independent testing and training data set, then cross validation could be used to assess the prediction performance of the statistical method. Should this be the situation, it is necessary to mention that it would be an extremely computational exercise to implement a full cross-validation routine for the AWA. That is, it would be too time consuming to leave out one observation, build the AWA model, predict the deleted observation, and then repeat this leave-one-out procedure separately. In the absence of an independent test set, a more realistic approach would be to perform cross-validation using the wavelet produced at termination of the AWA, but it is important to mention that this would not be a full validation.

References 1. R. Turcajov'a and J. Kautsky, Shift Products and Factorizations of Wavelet Matrices, Numerical Algorithms 8 (1994), 27-54.

201

2. R. Coifman and M. Wickerhauser, Entropy-Based Algorithms for Best Basis Selection, IEEE Transactions on Information Theory 38 (1992), 713-718. 3. B.A. Telfer, H.H. Szu, G.J. Dobeck, J.P. Garcia, H. Ko, A. Dubey, and N. Witherspoon, Adaptive Wavelet Classification of Acoustic and Backscatter and Imagery, Optical Engineering 33 (1994), 2192-2203. 4. H.H. Szu, B. Telfer, and S. Kadambe, Neural Network Adaptive Wavelets for Signal Representation and Classification, Optical Engineer#lg 31 ( 1992), 1907-1916. 5. W. Sweldens, The lifting scheme: a construction of second generation wavelets, Preprint Department of Mathematics, University of South Carolina (1994). 6. J. Kautsky and R. Turcajov'a, Adaptive wavelets for signal analysis: Proceedings of the sixth International Conference on Computer Analysis of h~lages and Patterns, Prague, Springer-Verlag (1995), 906-911. 7. J. Kautsky and R. Turcajov'a, Pollen Product Factorization and Construction of Higher Multiplicity Wavelets. L#1ear Algebra and its Applications 22 (1995), 241-260. 8. J. Kautsky, An Algebraic Construction of Discrete Wavelet Transforms, Applications of Mathematics 3 ( 1993), 169-193. 9. P. Steffen, P. Heller, R.A. Gopinath, and C.S. Burrus, Theory of Regular M-band Wavelet Bases, IEEE Transactions in Signal Processing, 41 (1993), 3497-3511. 10. P. Heller, H. Resnikoff and R. Wells, Jr, Wavelet Matrices and the Representation of Discrete Functions, In Wavelets- a Tutorial in Theory and Applications (C. Chui, Ed) Academic Press (1992). 11. R. Turcajov'a, Compactly Supported Wavelets and Their Generalizations." An Algebraic Approach, Ph.D. Thesis, The Flinders University of South Australia (1995). 12. Grace, Optimization Toolbox for Use with Matlab, The MathWorks, Inc., Natick, (1994). 13. M. Tatsuoka, Multivariate Analysis. Techniques for Educational and Po'chological Research, Wiley, New York (1971 ). 14. N. Saito and R.R. Coifman, Local Discriminant Bases, In Mathematical hnaging." Wavelet Applications in Signal and h~lage Processing H (A.F. Laine and M.A. Unser, Eds), Proc. SPIE, 2303 (1994). 15. G. McLachlan, Discriminant Anah'sis and Statistical Pattern Recognition, Wiley, New York (1992).


Part II Applications



205

CHAPTER 9 Application of Wavelet Transform in Processing Chromatographic Data Foo-tim Chau* and Alexander Kai-man Leung Department of Applied Biology and Chemical Technology, The Hong Kong Polytechnic UniversiO', Hung Horn, Kowloon, Hong Kong, People's Republic of China

1 Introduction The term chromatography is derived from Greek words meaning "colour" and "write" [1]. The name of this technique evolved from the earliest work of separating dyes or plant pigments on paper. Today, chromatography is used widely in analytical chemistry for the separation of compounds in sample mixtures. By adopting different chemical and physical properties, various chromatographic techniques and instruments have been developed for chemical analysis. Such techniques include paper chromatography, thin layer chromatography (TLC), gas chromatography (GC), liquid chromatography (LC), capillary electrophoresis (CE), supercritical fluid chromatography (SFC), ion chromatography (IC) and gel permeation chromatography (GPC). In the past, chromatography was used mainly for the separation of compounds. However, this situation has changed in the last decade. There was a tendency to combine different analytical techniques or instruments with chromatography for separation and characterization [2]. Examples include gas chromatography coupled with mass spectrometry (GC-MS) or Fourier transform infrared spectroscopy (GC-FTIR), liquid chromatography coupled with mass spectrometry (LC-MS), high performance liquid chromatography coupled with a diode array detector (HPLC-DAD), and capillary electrophoresis coupled with mass spectrometry (CE-MS) or a diode array detector (CE-DAD) [3]. In recent years, the development of wavelet transform (WT) theory in different fields of science has been growing very rapidly. The WT has two major characteristics, in that the basis functions of WT are localized in both the time and frequency domain, and there are a number of possible wavelet basis functions available. Such properties have attracted analytical chemists to

206 adopt WT in data analysis and signal processing in chromatography. Up to 1998, more than 120 publications reported the application of WT as a tool for data and signal processing (Table 1). So far, thirteen papers have reported on the adoption of WT in chromatographic data processing [4,5]. This Chapter brings to the attention of the international chemometrical community the results of the above research, originally largely published exclusively in Chinese.

2

Applications of wavelet transform in chromatographic studies

In chromatographic data analysis and signal processing, analytical chemists always face several problems such as noise suppression, signal enhancement, peak detection, resolution enhancement, and multivariate signal resolution [6]. Various chemometric methods have been proposed for tackling these problems, and give satisfactory results. Transformation techniques such as the Fourier transform, Laplace transform and Hartley transform have been utilized in chromatography for data processing [6]. Recently, the new mathematical technique WT has been introduced to help find answers to the above problems. In the following Sections, we describe selected major applications of WT in chromatography.

Table 1. Number of published papers from 1989 to 1998 that relate to the application of the wavelet transform in chemistry.

Year

Number ofpubl~hedpapers

1989 1990 1991 1992 1993 1994 1995 1996 1997 1998

1 0 0 2 5 6 4 22 50 34

Total

124

207

2.1 Baseline drift correction

Baseline drift is a very common problem in chromatographic studies. It is classified as a type of long-term noise and is defined as a change in the baseline position. This kind of drift is mainly caused by changes of temperature, or solvent programming and temperature effects on the detector [7]. In most cases, the drift is represented by a curve instead of a linear function. As a result, it induces errors in the determination of peak height and peak area, which are very important parameters for quantitative analysis. In practice, an artificial baseline is usually drawn beneath the peak (Fig. 1). The peak areas and heights determined will be either greater or smaller than the actual values, which depend on whether the true baseline has a convex or concave shape. Therefore, most analytical chemists prefer to find out the exact shape of the baseline and then to subtract it from the original raw chromatogram. Pan et al. [8] developed a wavelet-based method for correcting baseline drift in high performance liquid chromatography (HPLC). In general, the noise, 1.4

1.2

~o co o..

0.8

rr"

0.6

0.4

0.2

0

1

2

I

1

3

4

Retention Time (rain.)

._1

5

.l

6

I

7

8

Fig. 1 A simulated chromatogram with baseline drift is shown. The straight line drawn beneath the peak represents the artificial baseline for peak area measurement. (**)

208 chromatographic peaks, and the baseline are located, respectively, in the higher, middle and lowest frequency regions of the raw data. W T has an intrinsic property of enabling the resolution of a signal into higher and lower frequency parts. With proper use of this property, the correct baseline can be extracted from the raw data. Pan et al. proposed processing of the chromatogram with the Daubechies 06 wavelet function at an optimum resolution level, j. Then, zero values are assigned to the corresponding peak positions in Cj. After the inverse WT treatment of the signal, the reconstructed chromatogram at the resolution level J, Cj. baseline, is obtained, which represents the baseline for that chromatogram under study (Fig. 2(a)). Finally, a baseline-free chromatogram can be obtained by subtracting the baseline from the raw data (Fig. 2(b)). These workers applied this technique to resolve the baseline for an H P L C determination of a complex mixture of sixteen rare earth-elements and a satisfactory result was obtained. A similar technique was also adopted by Alsberg et al. [9] for baseline removal in a R a m a n spectroscopic study. 2.2 Signal enhancement and noise suppression

Noise suppression is a very common technique in chromatographic data processing. It aims to enhance an analytical signal to give a higher signal-tonoise ratio. Nowadays, many chromatographic instruments are controlled by computers and it has become a common practice to reduce the noise by 35000,- .

.

.

.

(a)

35ooor/

9

30000 ~

25000 .~ 20000

i

(b)

1

1 5ooo'-1

-50001 0

-

I ._ ~ ~ I .l l i0 20 30 40 50 60 t/s

J 70

,

'

i 50000

10 90 30 40 t/s

50 60

70

Fig. 2 (a) Curve 1 shows the signal of a mixture o[ sixteen rare-earth elements from an HPLC measurement and Curve 2 shows the baseline after wavelet treatment. (b) HPLC signal with baseline subtraction. Reproduced from reference [8] with the kind permission of Chinese Chemical Society.

209 employing digital processing methods such as filtering. Traditionally, analytical chemists favour the adoption of the Savitzky-Golay, Fourier, and Kalman filters for signal processing [10,1 1]. After the introduction of the WT technique to analytical chemistry, some workers found that the performance of WT is much better than that from the above mentioned filters in data denoising [ 12,13]. Shao et al. [14] reported the use of WT to smooth the HPLC signals of rare earth elements. Smoothing and de-noising are two different processes. Smoothing removes high frequency components of the transformed signal regardless of their amplitudes, whereas de-noising removes small-amplitude components of the transformed signal regardless of their frequencies [13]. The basic principle of WT smoothing is very simple. When a chromatogram in the digital form is treated with a proper wavelet function at the optimum resolution level j, the Cj thus produced represents the smoothed chromatogram, while Dj, Dj+I, K, Dj-1 represent the noises at various resolution levels (Fig. 3). In Shaos work, the original chromatograms were treated with the Harr wavelet function, and the smooth chromatogram Cj was employed for (a) 9 .

.

.

8

.

.

(b) '

9--

,

,

,

,

A

7

5

Dj_4 Dj_3 3 Dj- 2 2

Dj_I

0

-1 0

2

4 6 8 Retention T=me (m=n)

0

-1 0

2

4 6 8 Retenhon Time (mm.)

10

Fig. 3 Cj is a simulated chromatogram with white noise having a value of O.05. (a) shows the scale coefficients C at resolution level J tO J - 4 , and (b) shows the corresponding wavelet coefficients D at these resolution levels. The Daubechies DI6 wavelet function was adopted for the W T computation.

210 quantitative analysis. This result shows that WT can improve the signal-tonoise ratio and the detection limit for HPLC analysis. As compared with WT smoothing, de-noising via WT is another story. It requires one more step, thresholding, for removing noisy components from the wavelet coefficients D. Several methods have been proposed for discarding negligible coefficients or noise in the wavelet domain. These include absolute cut-off, relative energy, an entropy criterion and decreasing-rearrangements, fixed-percentages methods [15], and a universal thresholding algorithm [16]. In these methods, only coefficients with values greater than a pre-defined threshold value are retained. A zero value is assigned to those coefficients with magnitudes less than the threshold value. After the inverse WT treatment, a de-noised chromatogram is obtained. Mittermayr et al. [17] adopted this technique to process chromatographic data. These authors aimed to apply the German DIN 32645 standard for the determination of the detection limit of chromatographic data. Their results demonstrated that WT de-noising could improve the detection limit by up to a factor of three. They explained that the de-noising process could reduce the variance of the peak area and height, and the limit of detection is mainly determined by their variance. 2.3 Peak detection and resolution enhancement

Both peak detection and resolution enhancement are other problems encountered by analytical chemists in chromatographic studies. Obviously, the performance of each chromatographic system has limitations of its own, and consequently none of these systems is sufficiently universal to provide complete separation of excessively complex compound mixtures. As a result, peak overlapping always exists to a certain extent in the chromatogram. In this situation, we must resort to mathematical techniques to solve the problem. Usually, these problems are solved by using linear or non-linear regression analysis, curve-fitting techniques [18,19], derivative techniques [6], neural networks [20], statistical theory [21], and factor analysis [22]. The curve fitting technique is the most common method and has been widely available in most commercial software packages such as PeakFit (SPSS Inc.) and GRAMS/32 CurveFit (Galactic Industries Corporation) for chromatographic data processing. These packages allow the user to fit the chromatogram with a certain number of Gaussian and/or exponentially modified Gaussian functions [23] (Fig. 4). The parameters for these Gaussian functions are determined via linear or non-linear regression analysis. In the following sub-sections, techniques are given which use WT to handle peak-detection and resolution enhancement.

211 I

08

I

I

I

I

1

I

!

I

3

4

5

6

7

I

|

8

9

84

_>'06 ._ c-

04 i

~I

0

I

I

1

2

10

R e l e n l l o n T i m e (mJn.)

Fig. 4 Overlapped chromatographic peaks resoh'ed by the conventional curve-fitting method. The dotted lines represent the Gaussian functions ~'om curve-fitting treatment.

2.3.1 Derivative technique

The Derivative technique is another powerful method for resolving overlapping chromatographic peaks because it offers a higher apparent resolution of the differential data compared to the original data [24]. Although the technique is a useful tool for data analysis, it has a major drawback in increasing the noise level in higher-order derivative calculations [25]. Recently, our research group proposed a novel method which uses WT for approximate derivative calculations [26]. This method can enhance the signal-to-noise ratios through higher order derivative calculations and, at the same time, retain all the major properties of the conventional methods. An approximate first-derivative of an analytical signal can be expressed as the difference between the two scale coefficients C~_l, which are generated from any two different Daubechies wavelet functions. For example, a chromatographic signal X can be treated with two Daubechies wavelet functions of D2m and D2m, respectively with m and rh being any positive integer, and m-r ill. Then, the first derivative of X can be expressed as: X (1) ~ C J _ l , U 2 m -- CJ_l,O2rh

(1)

212

Eq. (1) can be applied to X (") again, to determine the approximate derivative at the next higher order. The approximate derivative calculation at a higher order can be generalized as: X (n)

,'~ C J _ I , D 2

m -r rh

m --CJ_l.D2th

and

n ~ 1

(2)

with C D2m.n (n) and C~!m_" being obtained from a WT treatment of X/n-I) at the _ , first resolution level with Daubechies wavelet functions D 2 m and D2m. In our studies, we have found that the signal-to-noise ratio value of the first derivative is highest with the use of Ds and D~8 wavelet functions. Fig. 5 shows a comparison between the conventional and wavelet methods for a signal with overlapping peaks. The first derivative obtained from the traditional method was smoothed with the Savitzky-Golay 17-points filter for the sec-

(a)

--

0t 0 0.02 .

9

4 .

.

.

6

8

.

lO ,

(b)

..........

"

_J

~9

c

001[ -0"02

9 ' 0 2 x 10 -`=

51

"

, 4 ",

"

"

"

d)

6

8

10

4 6 8 Retention Ttme/ (min.)

10

o.ol

0 ............

-

.

>., 0.02

~

-0.01 , 6

,"

, 8 "

1 10

-0.02

2

. 2 0 x 10 -3

.

. 4

.

(C) t

~o~wl L og ~ s 0

_.c_ 2

2

4 6 8 Retention Time / (min.)

10

0

2

Fig. 5 (a) A simulated chromatographic signal generated bl' overlapping of two Gaussiar, /'unctions with a white noise level of 0.001 being added (SNR = 500 for peak 1 ant, SNR = 250for peak 2 ) f o r WT derivative calculation. The first derivative (b), and the second derivative (c), of the signal. Fig. 5(a) was obtained b)' using the conventiona, method. Tke first derivative (d), and the second derivative (e), of the signal Fig. 5(a) Ira.! obtained by the proposed WT derivative method. (**)

213 ond derivative calculation (Fig. 5(c)). In the first-derivative plots (Figs 5(b) and (d)), both methods give the same results on the positions of the peak maximum and turning point. Moreover, in the second-derivative plots (Figs 5(c) and (e)), both give similar results on the peak-centre position. The major differences between the two methods were the signal-to-noise value on the first and second derivatives, and the number of coefficients of each derivative, as can be visualized from the plots. This wavelet-based derivative method can help analytical chemists to resolve overlapping chromatographic peaks at lower signal-to-noise ratios.

2.3.2 Wavelet coefficients method Shao and his co-workers developed another WT method to resolve overlapping chromatographic peaks [27-29]. They adopted the wavelet coefficients Dj for quantitative calculation. Fig. 6 shows the results of their study. The chromatograms of a mixture of benzene, methylbenzene and ethylbenzene at different concentrations are given in Fig. 6(a). Peak overlapping is observed in these chromatograms. After the wavelet treatment on one of the chromatograms, with the Harr wavelet function, the wavelet coefficients D at resolution levels J - 1 to J - 4 are depicted in Fig. 6(b). In this case, these workers found that Dj_3 is the best case for resolving the overlapped peaks and quantitative calculation. Fig. 6(c) shows the Dj_3 signals for all samples in Fig. 6(a). In order to determine the peak areas for individual components in Fig 6(c), the Dj_3 signals were baseline-corrected first, by linking the minimum points of every peak as baseline. After this treatment, three separated peaks can be identified from the baseline-corrected Dj_3 signal plot (Fig. 6(d)). Each peak corresponds to one of the components in the samples. Fig. 6(e) shows the calibration curves for the individual component in Fig. 6(d). Satisfactory results were obtained in their study for quantitative analysis. These authors have applied this method successfully for the quantitative analysis of plant hormones by HPLC with WT treatment [30] and found that better calibration curves were obtained when the chromatograms were processed with WT. 2.3.3 Multi-resolution and factor anal~'sis In modern chemical laboratories, new types of instrument known as hyphenated instruments, such as HPLC-DAD, GC-MS, GC-FTIR and CEDAD, have become very powerful tools for quantitative and qualitative analysis. These instruments can provide information from both chromatographic and spectroscopic studies at the same time. However, two-way data

214

(a)

1200000

(b)

1000000 z

-~

8o(x)oo

o o m 2~oco

-~00(~

......... 1.2

i ........

,! . . . . . . . . .

1.6

2.0

i.i, ...... 2.4

I., ...... 2.8

3.2

1.2

1.4

1.6

TIME(rain)

(c)

~;~.

'

2.0

2.2

2.4

2.6

2.8

3.0

3.2

TIME(rain)

2OOOOO 15oooo

1.8

25o0o0

/

(d) 2o0oo0

1ooooo z o p oL o

L'

~

'-

'

-~

o

--.,

-50000

i!!l,

Z 15OO00 _O l-.

5oooo

loe:x.no

,
2

(1)

0

The symbol 9 denotes the convolution operation between Nm-1 and Nm. The kth term of Nm is given by k

Nm(tk) -- Z Nm-, (tj)N1 (tk-j) j=0

(2)

with j >_ 0 [11]. The result is equivalent to the sum of products between coefficients in Nm-1 and Nm in a shifted manner. The mother wavelet function ~()v) may be expressed as 3m--2

qJ(t) - Z

qnNm( 2 k t - n)

(3)

n=0

with

qn-- 2m_-----lT n (m)j

N 2 m ( n - j + 1),

n - 0, 1, 2 , . . . , 3m - 2.

j=0

(4) In spline wavelet computation, optimization needs to be carried out on two parameters, namely the order of B-spline, m, and the truncation frequency or frequency scale, L, which represents the cut-off (or truncation) frequency value between the useful signal and noise. Details of the B-spline theory can be found in the references [12,13].

227

In 1995, Mo and Yan published their first paper on the application of WT in electroanalytical chemistry [14]. They developed a real-time continuous wavelet to de-noise signals from the staircase voltammetry. After a prolongation pre-treatment of the original voltammetric signal, the pre-processed signal is taken to act as the input signal of the filter system. Then, a detailed study is performed on the discrete value in the time-domain of the impulse response function for the wavelet filter. The real-time wavelet filter is set up by identifying the relationship between the prolongated signal and the original signal. By modifying the boundary condition of the input signal, the filter can improve the signal-to-noise ratio (SNR) and standard deviation of the post-processed signal. As shown in Fig. 1, the performance of the realtime wavelet filter in de-noising the voltammetric signals is very good. The authors [14] applied the new method successfully for real-time signal de-!

-iooo

oo

-9oo

6

~

e 9

9

3

2~

e o

o~

b)

_

!

-4~

I

-3~

!

-1~

l

0

I

1~

E/rnV (vs AglAgCt)

Fig. 1 (a) Experimental voltammogram from a solution conta&ing 1.0 • 10 -7 mol/l ZnS04 + l.O • 10 -3 mol/l K2S04 and the signal processed with the single side prolongation treatment. (b) 0.5 order deconvolution voltammogram from a solution containing 4.30 • 10 -5 mol/l K3Fe(C204)3 + 0.10 mol/l K2C204 + 0.010 mol/l H2C204 and the signal processed with the double side prolongation treatment. (e represents the original signal a n d - represents the processed signal with third order B-spline wavelet filter). Reproduced from reference [14] with kind permission of Science in China Press.

228

noising in investigating staircase voltammetry in ZnSO4-K2SO4 K3Fe(C204)3-KzC204- H2C204 systems.

and

In another two publications, Mo and his co-workers report a detailed study to optimize the order of B-spline wavelet basis, m, the truncation frequency L, the SNR, and sampling points, n, for voltammetry [13,15]. In these studies, the B-spline wavelet de-noising technique was employed to analyse the Ti(IV)-HzC204 reversible system [13,15], W(VI)-Mo(VI)-HAPP-KC103 absorption catalytic system [15] and ferrocene-LiC104-CH3CN system [16]. Fig. 2 shows the effect of different B-spline orders, m. The authors found that the third-order B-spline wavelet with m - 3 is the most suitable function to de-noise the voltammetric signals. This is because a very smooth voltammogram can be obtained (Fig. 2(b)). Furthermore, there is a minimum change in the peak width when compared to the theoretical voltammogram. Fig. 3 shows the effect of different truncation frequencies, L. When L is greater than 4, noise is embedded in the voltammograms. At L = 3, a smoothed voltammogram with minimum derivation from the theoretical peak potential, Ep, and peak current, lp, is observed (Fig. 3(d)). For the effect of SNR (from 1.0 to 0.15), they found that there is a large deviation in Ep and (n)

0.50

o.so

0.38

0.38

0.26

0.26

0.14

0.14

0.02

0.O2

o4-0.10 ,....

e,~-o, lO

(b)

> 0.50

0.48

0.26

0.36

0.14

0.24

0.02

0.12 o

-s

o

5

(E-s 1/2)NF/RT

~o

,

,

|

i

> 0.60

0.38

-0.10

(c)

0.00

i

(d)

-

-~o

,

-s

o

s

lo

(E-E 1/2)NF/RT

Fig. 2 Effect o f different B-spline order oJ~ (a) 177 = 2; (b), m = 3; (c), m = 4 and (d), the theoretical curve with S N R = 0.2, L = 3, n = 28. Reproduced f r o m reference [13] with kind permission o f Elsevier Science.

229 (a)

1.30

~-

o.8o

0.87

0.54

0.44

0.28

0.01

0.02

-0.42

-0.24

-0.85

.~-0.50

0.55

(c)

~

0.43

0.19

o.31

f

9

i

i

0.55

0.37

0.01

(b)

,

(d)

o. 19 0.07

-0.17 -0.35 -~o

-5

o

s

(E-E~/zlNF/RT

1o

-0.05 -~o

-5

0

5

(E-E~/2)NF/Rr

' "~o

Fig. 3 Effect o f different truncation li'equency q/] (a) L = 6" (b) L = 5," (c) L = 4 and (d) L = 3 with S N R = 0.2, in = 3, n = U. Reproducedji'om r~:/erence [ 1 3 ] with kind permission o/" Elsevier Science.

Ip when the SNR value is reduced. Finally, they tested the effect of the number of sampling points from 26 to 210. The deviations in Ep and Ip are reduced with a higher value of this parameter. The authors concluded that the third-order B-spline wavelet basis, and truncation frequency L = 3, are the optimum parameters for processing voltammetric signals. Recently, Mo's research group has developed a multi-filtering technique to process voltammetric signals with the B-spline wavelet [17, 18]. In this investigation, the B-spline wavelet analysis was adopted to decompose the signals into different low frequency components and noise. Occasionally, parts of useful information are filtered out together with noise. In order to extract useful information thoroughly, the noise contribution was treated as the original signal and processed with the B-spline wavelet method. The low frequency signals were then utilized to compensate the loss of information in the original signal after de-noising. In this work, the proposed method was tested with the simulated staircase voltammetric signals: Eqs (5)-(7) are the equations used in their simulation. The function is F(x) -

1

1 + exp(x)

(5)

230 and its semi-differential equation is Vk -- yO.5 __

d0.5 dxO.5

F(x)

9

(6)

Here, Vk, representing the potential, and x, can be derived from the currentfunction as follows: x-

( E - El/2) nF . RT

(7)

In Eq. (7), E and El/2 represent, respectively, the measured and half-wave potentials, while n, F, R, and T denote the number of electron transferred, the Faraday constant, the Universal Gas Constant, and temperature, respectively. The authors found that the relative errors in the peak height were less than 3%, and those of the peak potential were less than 10% when the SNR value was reduced to 0.1. This technique has been applied successfully to analyse the voltammetric signal of a Cd(II)-succinate-oxalate complex system [19,20]. In the past, the Fourier transform, spline function, and Kalman filter were the major techniques for data processing in electroanalytical chemistry [21]. Mo and his co-workers have also performed comparisons on the application of the Fourier transform (FT) and wavelet transform (WT) in voltammetric studies. The FT has a difficulty in filtering low frequency signals. Although this problem can be tackled by increasing the number of sampling points, this treatment may increase dramatically the number of computations [22,23]. Other alternatives such as lowering the truncated coefficient and operating a "quadratic filter" [24] can partly filter away the low frequency noise. However, these operations have a great effect on the original peak profile and reduce the height of the peak under study [23]. In general, the B-spline wavelet analysis performs better than the FT method in de-noising. When signals with singular points are processed with WT, these singular points affect the signal at both ends but do not spread to other data. For a signal with large differences at the boundaries, singular points will be observed which cause fluctuation of the signal at the boundaries after the de-noising treatment (Fig. 4). Bao et al. [23] performed a comparison on the differential pulse stripping voltammetric signal of a Pb(II)-KC1 system with both the Bspline wavelet and FT methods. Fig. 5 shows the result of the comparison. The signal processed by WT is smooth and very close to the real value.

231 0.50

(,) 0.40

(c) "~ 0.30 + ,~, 0.20 ./

(b)

!" "~ o.lo

&/ /

0.00

-0.I0

|

-lO

l

-S

~

-6

i

t

.4

i

-2

0

i 2

I 4

i 6

i

lO

g

% Fig. 4 Influence of singular points on FT and WT treatment. (a) Theoretical curve (solid line)" (b) Inverse FT curve (dotted line)" (c) the curved processed by B-spline WT (dashdot line). Reproduced from reference [23] with kind permission of American Chemical Society.

However, the signal processed by FT is vibrant and gives a lower value for the peak-height. The FT result leads to unavoidable errors in the analysis. B-Spline wavelet analysis is a very good technique for processing voltammetric signals, but it also suffers a drawback. For a signal with a low SNR value, peak-potential shifting is always observed in the de-noised signal [25]. Therefore, Mo and his co-workers developed a new procedure by combining

16:4

1

14i 12.A i

10. 8. 8

9

!~

. 9

_ .L_ 9 .gto

9

. ,oe. ., , L o .

9~

leel

~..

sg~

.......

9

~

"

9

.

9eo

4

2. 0 - I ,500

"-1o4~

"1.~[~

"1.200

-- | .100

--1.(~

V

Fig. 5 Results of the differential pulse stripping voltammetric signal of the Pb(II)-KCI system processed with (a) wavelet de-noising and {b) Fourier de-noising, respectively. Reproduced from reference [23] with kind permission of American Chemical Society.

232 both B-spline wavelet analysis and FT to process voltammetric signals [2528]. The combined algorithm can compensate the disadvantages of the Bspline wavelet analysis and FT. That is, FT filtration can keep the original peak position while WT can eliminate the low frequency noise signal. This new method involves decomposition of the original signal into discrete approximation and discrete details, in different density with the B-spline WT treatment. Then, the wavelet-transformed signals were processed further by the modified Fourier method [24,25] to obtain satisfying results. The conventional Fourier de-noising method usually involves the use of a multifunction of a rectangular filter function, fk, for both the real and the imaginary parts. The quantity fk is defined as: 1, fk--

k=0,1,...,i-

0,

i i+,

1

int( +')

(8)

with i, k and N representing the point of truncation, a running index, and the total number of data points, respectively. The symbol Int( ) denotes the integer function. In the modified Fourier de-noising method, the filter function is given as follows: 1 -(k/i)

fk--

O,

2

k=0,1,...,ik - i,i + 1 , . . . , I n t

1 2

"

(9)

This combined method has been applied successfully to process differentialpulse-stripping voltammetric signals of the Zn(II)-KC1 (Fig. 6) [25] and formaldehyde-acetaldehyde [26,27] systems. Very recently, Mo and his co-workers developed another combination technique to de-noise voltammetric signals. The spline wavelet and the Riemann-Liouville transform (RLT) were coupled together for the first time to filter random noise as well as extraneous currents [29-31]. Capacitative current is the most significant extraneous current. The RLT is a very effective method for removing some extraneous regularly changing currents, but it is not good for filtering random noise. It is an integral transformation employed in fraction calculus. Since the RLT procedure is quite complicated, details are not given here, but they can be found in references [29,31]. As with the combined B-spline wavelet and FT methods, the spline wavelet was applied first to filter random noise from the current signals. Then, the wavelet-processed current curves at different sampling times were treated

233

(a) 18 !

Zn

15 12 r

9 6 3

,

!

,

,

-1.1

-[.0

-0.9

-1.0

-0.9

,

-~1.4 -1.3 - 1 . 2

--

E/V (b) 18 I 15

o+,1

3

-1.4

!

,I

-1.3

-1.2

,

I

-1.1

E/V Fig. 6 Results of the (a) differential pulse stripping vohammetric signal of the Z n ( I I ) KCI system and (b) the background current signal of KCI solution that was de-noised with the combined B-spline wavelet and Fourier transform analysis. Reproduced from reference [25] with kind permission of Science in China Press.

with RLT to remove capacitative current from these current signals at every step. Fig. 7 shows the result of the combined spline wavelet and RLT on the Cd(II)- KNO3 system [29]. With this new method, the errors associated with the peak current were less than 5.0%, and those of the peak potential were less than 1.0%. 2.2 Other wavelet transform applications in v o l t a m m e t r y

Another research team has successfully applied WT in voltammetric studies. Specifically, WT was applied to process DPV signals [32], potentiometrictitration- [33], and oscillographic chronopotentiometric signals [34,35]. Chen et al. [32] proposed a new type of wavelet function, the difference of

234 (a)

( b ) 0.5 1.5 '

I

'

1.0 o

0.4 0.3 0.2

0.5

o.1 0.0

o.o

-.-0.5 --0.4 --0.5 --0.6 -0.7 -0.8

-o.1

-0.4 -0.5 -0.6 -.-0.7 --0.8

EAt versus SCE

Fig. 7 Results of the current curve of step voltammetric signal of the Cd(H)-KN03 system (a) without, and (b) after spline and RLT processing. Reproduced from reference [29] with kind permission of Royal Society of Chemisto'.

Gaussians (DOG), to process differential pulse voltammetric signals. The function is defined as" W ( t ) - exp ( - ~ ) -

1

~exp ( - ~-).

(10)

In DPV quantitative analysis, it is very difficult to measure the peak-height of a component in a sample with low concentration because it affects the linear detection range of the DPV system. So, these workers [32] employed the DOG wavelet function to transform the DPV signal, as generated from a highly concentrated Cu z+ solution, and to determine the scale parameter, a. Then, the DPV signals at other concentrations were transformed with a predetermined scale parameter. A new linear calibration curve was obtained by using the results deduced from the WT treatment, and this results in a lowering of the detection limit of the analysis. Wang et al. [33] reported a novel application of WT for potentiometric titration. They made use of the edge-detection property of WT to determine the end-point in the potentiometric titration. There are two common methods for end-point determination in potentiometric titration. The first one is achieved through direct graphical interpretation of the titration curve, using methods such as Behrend's-, Br6tter's-, and Tubbs' method [36]. The second is achieved through mathematical interpretation of coordinates of the recorded points using methods such as Gran's [37-39] and a derivative method [40]. Wang et al. [33] proposed the use of the maximum absolute value of the first-order differential function to determine the end-point for potentiometric titration. A second-order continuously differential spline function was chosen

235

in this work to process the titration curve obtained. After WT treatment on the curve, the maximum absolute value among the wavelet coefficients represents the end-point of that titration (Fig. 8). Oscillographic chronopotentiometry is a new type of electroanalytical technique, developed in the P. R. China [35]. This technique is based on the change of oscillographic signal on the cathode ray oscilloscope. Harr and Daubechies wavelet functions were employed by another group in the P. R. China to de-noise the oscillographic signals of Pb(II) ions in NaOH solution and multi-components systems such as Cu(II) and AI(III) ions in LiC1 solution and Cd(II) and In(III) in NaOH solution [34,35]. They found that this

(a)

(b) ]2

• NaOH

x N~OH

f

+ ,d,/

10

12

=...=..x

l

8

8

x

_- +~,/ ~

j

_ ~ x .,..xX

x~~"

I

x

- -4-

,!

d-

,4-

"4-

)

)

z,

5

o

10

15

~

§

-+

.4.-

I 20

I 25

]

o

t ....

5

+

+

l

!

1o

V.~./mL (c)

§ t

]5

20

Vx= x / m L (d)

]2

12

1o

I0

-

_l

25

_,l

30

• NaOH

f~...x

+ ~'f

7

1r

!

$

8

K~X

o

-X~-

-

4-

1

o

,,,x

+

J

~ .x....xx= x x ' ~

+

- x.......__x~ § !

lO

I

20

V),.oH/mL

+ _1

30

+

+ L

40

-+ _!

0

+

+ +4-::::-+++~ t ! l 1 10

20

t

+ I

30

V)~.o./mL

Fig. 8 Titration curves of (a) HCI," (b) HOAc" (c) 0 3 1 9 0 4 . and (d) H2C204 with NaOH and their discrete wavelet coefficients. Reproduced from reference [33] with kind permission of Higher Education Press.

236 method gives a significant reduction in the detection limit when compared with the classical signal-processing methods. Fang and Chen [41] proposed a new tool for processing electroanalytical signals with an adaptive wavelet filter based on the wavelet packet transform technique. They investigated the feature of WPT for white noise which is caused by random and irregular processes. The decomposition was performed with the B-spline wavelet of order 3. Their outcomes showed that the adaptive wavelet filter could be applied to a system with the interference originating from an existing power supply which is useful for the study of fast-electron-transfer processes. Fang et al. [42] also proposed a new algorithm for processing electroanalytical signals with data-length not equal to 2p, where P is an integer. Under this method, the signal S was broken down into two parts, namely Sa and Sb, with each part having a data length of 2p, as follows:

s /

r

Sa

SI~ $2~ $3~ $4~ $5, $6, $7, $8~..., Sn-5, Sn-4, Sn-3, Sn-2, Sn-l~ Sn

/

Sb

(11) Then, the wavelet de-noising process was performed separately on S~ and Sb, and the de-noised signals were then recombined to regenerate the signal in the original domain.

3

Conclusion

In conclusion, researchers in the P.R. China have applied WT successfully in electroanalytical studies, and we hope that this Chapter will introduce the works from the P.R. China to the other chemists around the world.

4

Acknowledgement

This work was supported by the Research Grant Council (RCG) of Hong Kong Special Administration Region (Grant No. HKP 45/94E) and the Research Committee of The Hong Kong Polytechnic University (Grant No. A020).

237

References 1. W.R. Heineman, Introduction to Electroanalytical Chemistry, In Chemical Instrumentation:A Systematic Approach: Third Edition (H.A. Strobel and W.R. Heineman, Eds) Wiley, New York, (1989), pp. 963-999. 2. J. Osteryoung, Introduction, In Handbook o/'hlstrumental Techniques for Analytical Chemistry (F. Settle, Ed) Prentice-Hall PTR, Upper Saddle River, N J, (1997), pp. 685-690. 3. S.D. Brown and R.S. Bear, Jr, Chemometric Techniques in Electrochemistry: a Critical Review, Critical Review of Analytical Chemistry, 24 (1993), 99-131. 4. Y.N. Ni and J.L. Bai, Applications of Chemometrics in Electroanalytical Chemistry, Chinese Journal of Analytical Chemistry, 24 (1996), 606-612 (in Chinese). 5. A.K.M. Leung, F.T. Chau and J.B. Gao, A Review on Applications of Wavelet Transform Techniques in Chemical Analysis: 1989-1997, Chemometric and Intelligence Laboratory Systems, 43 (1998), 165-184. 6. A.K.M. Leung, Wavelet Transform in Chemistry, http://fg702-6.abct.polyu.edu.hk/ ,~kmleung/wavelet.html, (accessed January 1999). 7. I. Daubechies, Ten Lectures on Wavelets, SIAM Press, Philadelphia, (1992). 8. C.K. Chui, An Introduction to Wavelets, Academic Press, New York, (1992), p. 49. 9. X.Q. Lu and J.Y. Mo, Wavelet Analysis as a New Method in Analytical Chemometrics, Chinese Journal of Analytical Chemistry, 24 (1996), 1100-1106 (in Chinese). 10. X.Q. Lu and J.Y. Mo, Spline Wavelet Multi-resolution Analysis for High-noise Digital Signal Processing in Ultraviolet-visible Spectrophotometry, Analyst (Cambridge, U.K.), 121 (1996), 1019-1024. 11. B.B. Hubbard, The World According to Wavelets. The Story q/'a Mathematical Technique in the Making, A.K. Peters, Wellesley, MA, (1996). 12. S. Sakakibara, A Practice of Data Smoothing by B-Spline Wavelets, In Wavelets Theory, Algorithms, and Applications (C.K. Chui, L. Montefusco and L. Puccio, Eds) Academic Press, San Diego, CA, (1994), pp. 179-196. 13. X.Y. Zou and J.Y. Mo, Spline Wavelet Analysis for Voltammetric Signal, Analytical Chimica Acta, 340 (1997). 115-121. 14. L. Yan and J.Y. Mo, Study on New Real-time Digital Wavelet Filters to Electroanalytical Signals, Chinese Science Bulletin, 40 (1995), 1567-1570 (in Chinese). 15. X.Y. Zou and J.Y. Mo, Spline Wavelet Analysis of Step Voltammetry Signal, Chemical Journal of Chinese University. 17 (1996), 1522-1527 (in Chinese). 16. X.Q. Lu, J.Y. Mo, J.W. Kang and J.Z. Gao, Method of Processing Discrete Data for Deconvolution Voltammetry II" Spline Wavelet Transformation, Analytical Letters, 31 (1998), 529-540. 17. X.Y. Zou and J.Y. Mo, Spline Wavelet Multifiltering Analysis. Chinese Science Bulletin, 42 (4) (1997), 382-385 (in Chinese). 18. X.Y. Zou and J.Y. Mo, Spline Wavelet Multifiltering Analysis. Chinese Science Bulletin (English Edition), 42 (8) (1997), 640-644. 19. X.Q. Lu and J.Y. Mo, Spline Wavelet Multifrequency Channel Filters for High Noise Digital Signal Processing in Voltammetry, Acta Science National University Sunyatseni, 36 (1997), 129-130.

238 20. X.Q. Lu and J.Y. Mo, Methods of Handling Discrete Data for Deconvolution Voltammetry (I): Wavelet Transform Smoothing, Chemical Journal of Chinese University, 18 (1997), 49-51 (in Chinese). 21. X.Q. Lu, J.Y. Mo, J.W. Kang and J.Z. Gao, Application of Signal Processing Method in Electroanalytical Chemistry, Chinese Journal of Analytical Chemisto', 26 (1998), 597-602 (in Chinese). 22. L.J. Bao, J.Y. Mo and Z.Y. Tang, Comparative Study on Signal Processing in Analytical Chemistry by Fourier and Wavelet Transforms, Acta Chimica Sinica, 55 (1997), 907-914 (in Chinese). 23. L.J. Bao, J.Y. Mo and Z.Y. Tang, The Application in Processing Analytical Chemistry Signals of a Cardinal Spline Approach to Wavelets, Analytical Chemistry, 69 (1997), 3053-3057. 24. E.E. Anbanel, J.C. Myland, K.B. Oldham and C.G. Zeski, Fourier Smoothing of Electrochemical Data Without the fast Fourier Transform, Journal of Electroanalysis Chemistry, 184 (1985), 239-255. 25. L.J. Bao and J.Y. Mo, A Modified Fourier Transform Method for Processing High Noise Level Electrochemical Signals, ChhTese Science Bulletin, 43 (1) (1998), 42-45 (in Chinese). 26. L.J. Bao, J.Y. Mo and Z.Y. Tang, The Application of Spline Wavelet and Fourier Transformation in Analytical Chemistry, Chemical Journal of Chinese UniversiO', 19 (1998), 193-197 (in Chinese). 27. L.J. Bao, J.Y. Mo and Z.Y. Tang, Combined Spline Wavelet and Fourier Transform Processing Analytical Chemistry Signal, Chemistry ill Hong Kong, 2 (1998), 53-58. (**) 28. L.J. Bao, Z.Y. Tang and J.Y. Mo, The Application of Spline Wavelet and Fourier Transform in Analytical Chemistry, In New Trends h~ Chemometrics, First International Conference on Chemonletrics #1 China, Zhangjiajie, China, October 17-22, 1997, (Y.Z. Liang, R. Nortvedt, O.M. Kvalheim, H.L. Shen, Eds) Hunan University Press, Changsha, (1997), pp. 197-198. 29. X.P. Zheng, J.Y. Mo and P.X. Cai, Simultaneous Application of Spline Wavelet and Riemann-Liouville Transform Filtration in Electroanalytical Chemistry, Analytical Communication, 35 (1998), 57-59. 30. X.P. Zheng and J.Y Mo, The Coupled Application of B-Spline Wavelet and RLT Filtration in Staircase Voltammetry, In New Trends #1 Chemometrics, First hlternational Conference on Chemometrics in ChhTa, Zhangiiajie, China, October 17-22, 1997 (Y.Z. Liang, R. Nortvedt, O.M. Kvalheim, H.L. Shen, Eds), Hunan University Press, Changsha, (1997), pp. 199-200. 31. X.P. Zheng and J.Y. Mo, Removal of Extraneous Signals in Step Voltammetry, Chinese Journal of Analytical Chemisto', 26 (1998), 679-683. 32. J. Chen, H.B. Zhong, Z.X. Pan and M.S. Zhang, Application of wavelet transform in differential pulse voltammetric data processing, Chinese Journal of Analytical Chemistry, 24 (1996), 1002-1006 (in Chinese) 33. H. Wang, Z.X. Pan, W. Liu, M.S. Zhang, S.Z. Si and L.P. Wang, The Determination of Potentiometric Titration End-points by using Wavelet Transform, Chemical Journal of Chinese University, 18 (1997), 1286-1290 (in Chinese).

239

34. J.B. Zheng, H.B. Zhong, H.Q. Zhang and D.Y. Yang, Application of Wavelet Transform in Retrieval of Useful Information from d2E/ dt 2 - t Signal, Chinese Journal of Analytical Chemisto', 26 (1998), 25-28 (in Chinese). 35. H.B. Zhong, J.B. Zheng, Z.X. Pan, M.S. Zhang and H. Gao, Investigation on Application of Wavelet Transform in Recovering Useful Information from Oscillographic Signal, Chemical Journal of Chinese University, 19 (1998), 547-549 (in Chinese). 36. K. Ren and A. Ren-Kurc, A New Numerical Method of Finding Potentiometric Titration End-points by Use of Rational Spline Functions. Talanta, 33 (1986), 641647. 37. G. Gran, Determination of the Equivalence Point in Potentiometric Titrations Part II, Analyst (London), 77 (1952), 661-671. 38. F.T. Chau, Introducing Gran Plots to Undergraduates, Ed. Chem, 27 (1990), 109110. 39. F.T. Chau, H.K. Tse, and F.L. Cheng, Modified Gran Plots of Very Weak Acids on a Spreadsheet, Journal of Chemical Editions, 67 (1990), A8. 40. W.R. Heineman and H.A. Strobel, Potentiometric Methods, In Chemical Instrumentation: A Systematic Approach: Third Edition (H.A. Strobel and W.R. Heineman, Eds), Wiley New York, (1989), pp. 1000-1054. 41. H. Fang and H.Y. Chen, Wavelet Analyses of Electroanalytical Chemistry Responses and an Adaptive Wavelet Filter, Anah'tical Chimica Acta, 346 (1997), 319325. 42. H. Fang, J.J. Xu and H.Y. Chen, A New Method of Extracting Weak Signal, Acta Chimica Sinica, 56 (1998), 990-993 (in Chinese).



241

CHAPTER 11 Applications of Wavelet Transform in Spectroscopic Studies Foo-tim Chau and Alexander Kai-man Leung Department of Applied Biology and Chemical Technology, The Hong Kong Polytechnic UniversiO', Hung Horn, Ko~rloon, Hong Kong, People's Republic of China

I

Introduction

The spectroscopic techniques, ultraviolet-visible (UV-VIS) spectroscopy, infrared (IR) spectroscopy, mass spectrometry (MS), nuclear magnetic resonance (NMR) spectroscopy, and photoacoustic (PA) spectroscopy, are widely used in analytical chemistry for both qualitative and quantitative analysis. Nowadays, most analytical instruments in modern laboratories are computerized, partly owing to the rapid development of advanced microelectronics technology. Digitized spectroscopic data can be exported from these instruments very easily for subsequent signal processing. Several types of technique are employed commonly in analytical chemistry for signal processing, and include data smoothing (denoising) and data compression. Data smoothing aims to remove high frequency components of the transformed signal regardless of their amplitudes, whereas data denoising aims to remove small amplitude components of the transformed signal regardless of their frequencies [1]. Both methods can increase the signal-to-noise ratio (S/ N) of signals by eliminating the noise or background via digital filters [2]. On the other hand, data compression aims at reducing the storage space and processing time during and after signal processing [3]. In chemical analysis, data compression is very important, especially in setting up digitized spectroscopic libraries [4]. Before 1989, Fourier transform (FT) and fast Fourier transform (FFT) were employed mainly by chemists to manipulate data from analytical studies [57]. After the publication of an important paper by Daubechies [8] in 1988, a new transformation algorithm called wavelet transform (WT) became a popular method in various fields of science and engineering for signal processing. This new technique has been demonstrated to be fast in computation, with localization and quick decay properties, in contrast to existin~ methods such as FFT. A few chemists have applied this new method foI

242

signal processing in chemistry, and satisfactory results have been obtained. Up to December 1998, more than 140 papers have been published in various fields of chemistry and chemical engineering [9~10]. Since few chemists are familiar with the wavelet theory and its application in chemistry, we shall present some specific applications of WT in analytical chemistry in this and the following chapters. We will focus our discussion on three major areas in analytical chemistry that include spectroscopy, chromatography, and electrochemical studies. The application of WT in chromatographic and electrochemical studies will be discussed in other chapters. In this chapter, selective applications of WT in UV-VIS, IR, MS, NMR and PA spectroscopies will be described. In spectroscopic measurement, the raw spectral data X are a combination of the true readings and noise in the discrete format. In order to extract the true readings from the raw data, a digital processing method such as filtering is commonly employed. In the past, a number of filters of various kinds have been developed in different fields of science and technology. However, only a few, such as the Savitzky-Golay, and Fourier and Kalman filters [11,12], are extensively used by chemists. These types of filters are implemented in most of the modern analytical instruments for data denoising. Since 1989, the development of wavelet theory has had a remarkable impact on analytical chemistry. Wavelet filters have been introduced into this area of chemistry for signal denoising. Recently, WT has been utilized to compress spectral data or to distinguish important properties from the acquired data. Generally, WT is superior to FT in many respects. In Fourier analysis, only sine and cosine functions are available as filters [13]. However, many wavelet filter families have been proposed. They include the Meyer wavelet, Coiflet wavelet, spline wavelet, the orthogonal wavelet, and Daubechies' wavelet [14,15]. Both Daubechies' and spline wavelets are widely employed in chemical studies. Furthermore, there is a well-known drawback in Fourier analysis (Fig. 1). Since the filters chosen for the Fourier analysis are localized in the frequency domain, the time-information is hidden after transformation. It is impossible to tell where a particular signal, for example as that shown in Fig. l(b), takes place [13]. A small frequency change in FT produces changes everywhere in the Fourier domain. On the other hand, wavelet functions are localized both in frequency (or scale) and in time, via dilations and translations of the mother wavelet, respectively. Both time and frequency information are maintained after transformation (Figs. l(c) and (d)).

243 (a) o~

(b) 1

lOO

E0.8

5O

#- 0.6 "6

~

8 ~_o.2 0

-50

1000

(s 8 1.5

2000 3000 Wavenumber / cm- 1

-100

4000

At resolution level J - 1

(d) ~

g

g

~_

~_ o

_.,/~J

l ..~

E

0.4

,ooo

~ooo

~ooo

Wavenumber / cm-~

,ooo

~

500 1000 1500 Arbitary Frequency At resolution level J - 2

2

o.s

o

1000

2000 3000 Wavenumber / crn-~

4000

Fig. 1 (a) Experimental; (b) Fourier tran,s.formed, and (c) ~ravelet-trans./brmed IR spectrum of benzoic acid. Spectra (c) and (d) were derived /i'onz (a/ with a Dauhechies DI6 wavelet.filter at resolution levels J - 1 and J - 2, re,v~ectivel.v.

Up to December 1998, more than 30 publications have reported spectroscopic studies with the use of a WT algorithm [9,10]. Within this work, WT has been utilized in three major areas that include data denoising, data compression, and pattern recognition. Two classes of wavelet algorithm namely discrete wavelet transform (DWT) and wavelet packet transform (WPT), have been commonly adopted in the computation. The former one is also known as the fast wavelet transform (FWT). The general theory on both FWT and WPT can be found in other Chapters of this book and some chemical journals [16-18], and is not repeated here. In the following sections, selected applications of WT in different spectral techniques will be described.

2

Applications of wavelet transform in infrared spectroscopy

Infrared spectroscopy plays an important role in the identification and characterization of chemicals and is used widely in modern laboratories. So

244 far, 14 publications have been reported which use WT in I R spectroscopy [9,10]. Besides data compression and data denoising, WT has been applied in some special areas such as wavelet neural networks, and standardization in IR studies. New computational algorithms involving WT will also be discussed in this section.

2.1 Novel algorithms for wavelet computation in IR spectroscopy Traditionally, the number of data points to be processed with WT must equal 2r' where P can equal any positive integer [19]. A data set with 2 r' data points can improve the computational efficiency. In real situations, it is not easy for a chemical instrument to generate exactly 2 P pieces of data. As in FFT, a series of zeros can be appended to one end or both ends of the original data set in order to bring the total length to the next power of 2. This method is called the zero padding method [20,21]. Alternatively, truncation of data at one or both ends of the original data, to the previous power of 2, may be adopted in some cases. In the WT treatment, the data length has another limitation. Owing to the generic nature of WT, the data length of the scale coefficient, Cj, and wavelet coefficient, Dj, must be the same after WT treatment at a particular level, j. So, the data length for a basis to be processed at the previous resolution level must be an even number. These constraints seriously limit the application of WT in signal processing. A novel algorithm called the coefficient position retaining (CPR) method has been introduced by our research group to improve the WT computation in IR spectroscopy [22]. This method can guarantee a smooth operation of FWT and WPT computation on spectral data with any data length. Suppose the original spectral data are represented by Cj, and J represents the highest level in the FWT computation. In this approach, if the data length, nc.j, of a scale coefficient Cj is an even number, FWT is applied as usual. Then, the scale and wavelet coefficients obtained at resolution level (j - 1) will have the number of coefficients nc.j-l(--nc.j/2) and nd.j_l( = no.j/2), respectively. On the other hand, if nc,j is an odd number, FWT is applied without using the last coefficient of Cj in the calculation. This coefficient is retained and transferred downward to the same position in the next higher resolution level. Then, it becomes the last coefficient of Dj_l at the next resolution level. As a result, the scale and wavelet coefficients will have nc.j-l(--nc.j/2) and nd,j-l((= nc.j/2)+ 1) elements respectively. Figs. 2(a)-(c) show a schematic diagram for applying the CPR algorithm in FWT to the data set with 1024, 1531, and 1023 data points, respectively. With the use of the CPR method, it

245

(1024)

1

I

H

L

J-1

Cj. I (512)

1

1

L

H

(255)

[ ~ii/~((256) ....";j;,] |,/:d ,,9:," f:,": " ":(

J-2

I

I

L

H

I; ++., ,t, o....

1

J-3

!:(128) l-(12s~:!

Coefficients for storage

Cj s

Dj

"

"

'S"

I

(a)

C1 (1531) I

I Applying CPR at position ,1. t53 I

H ~,

J- I

Cj.~ (765)

I

ApplyingCPR

H

at posit,on 765

~+~?'::!!i

+

C, ~

I

L

1

I L

J';:".,, D , ,

J-2

, . .'"

I

!t

l.c?+IV~I

J-3

(191) ~ ';(19|)

Coefficients for storage

' I)j '

:

--

F(~. 2a and h

,

!

(b)

246

J

Cj

(I023)

]

I

L

,

I

positi~1023

H ~

Applying

CPR

at

C j_!

(511)

I

L

I

H

i

Dosltion 511

Applying

CPR

al

J-2

i ion

,/-3

Coefficienls

for s l o r a g e 9

J-3

J,3

.

..; . . . . . . . .,

I

: .; ; ? ~ . . . ; i ! . ! . ' : . . : ; i ~ , / ; :

.,;'i~;;.-~?;i;.;

,' .. ,.,z,',:,.",>,: . . . . .> ..... , '..: ";"~,,">/'2> . .~.,.

9 . .... ,

. .

..

. /..,,.,,, ,.

l

".,.:".i":,~.~ : .':~ :.., ~:.~~i".;~i ''

(c)

Fig. 2 A schematic diagram showing the operation off the F W T method with a data length o f (a) N = 1024; (b) N = 1531 and (c) N = 1023, coupled with CPR treatment. The slanting line represents coefl?cients to be stored and the black region shows the position o/" the coefficient(s) to be archived in us#lg the CPR method.

can be guaranteed that the total data length remains unchanged and the quality of the reconstructed spectrum is not affected. This algorithm can also be applied to other chemical systems such as UV-VIS spectroscopy and chromatography for WT treatment. WT has been proposed as a new method for compressing spectra for storage and library searching in our study. In this kind of work, spectra are reconstructed from time to time from the compressed data. In order to maintain the quality of the reconstructed spectra, we have introduced another technique called the translation-rotation transformation (TRT) method [23] in the wavelet computation. In the FWT operation, the spectral data vector Cj needs to be extended periodically at the two extremes in the following manner: cextend { } j CJ.n-1, CJ.n, CJ.I, CJ.2;..., CJ.n-l~ CJ.n~ CJ.1; CJ.2~ CJ.n-l, - -

. . . ~

.

(1)

.

.

.

247

In the real situation, or in practice, the first data point, cj.~, and the last data point, Cj.n, at the two extremes do not have a common value. As a result, a small delay which results from discontinuity of the spectral data at the boundary will be observed at both ends of the reconstructed spectrum. Such a phenomenon is known as the side-lobe problem and causes deterioration of the quality of the reconstructed data (Fig. 3) [24]. The T R T algorithm involves subtraction of the data vector Cj by selected quantities B { = bl, b 2 , . . . , bk} to give the rotated array by cTRT

J.k

--

Cj.k -- bk

(2)

with

b k - cj.1 + (Cj,n- Cj ~)(k- 1)

(3)

n-1

(a)

(b)

(c)

1

~ 0.8 ~g

\

-~0.6

/

\

,j,

~0.7

~0.2 0 400

1000 2000 3000 4000 Wavenumber (cm 1 ) (d)

o~ 0 . 8

:'

,:~

'

/"

~ lit!l!, 1000 2000 3000 4000 Wavenumber (cm "1)

(f)

0.8

,!.1

0.4

0.6 3800

500 600 Wavenumber (cm n )

(e) l il

~o.~ 9 g

~

/ , ",.j

0.2 400

3900 4000 Wavenumber (crn1 )

1

0.9

--\

0.4

0

0.9

.5 0.8

~_ 0.4

g

~

1

'

500 600 Wavenumber (cm "1)

~

0.8

,...., E D

0.7 0.6 3800

3900 4000 Wavenumber (cm -1)

Fig. 3 (a) The reconstructed IR spectrum o/" benzoic acid that was pro~hwed./i'om the compressed data with the F W T and zero-padding method. (b) and (c) show the magnified plots of the reconstructed spectrum in (a). (d) The reconstructed IR spectrum o/benzoic acid that was produced from the compressed data using the FWT and TRT methods together. (e) and (f) show the magnified plots o/'the reconstructed spectrum in (d).

248 where k is a running index from 1 to n. After TRT treatment, both cJ,1 TRT and cTRT at the two extremes of Cj TRT are the same, and a smooth extension J.n vector can be observed. Figs. 3(d)-(f) show the result of the reconstructed IR spectrum of benzoic acid with TRT treatment. It is obvious that less sidelobes are observed compared to those without TRT treatment (Figs. 3(b) and (c)). Both CPR and TRT schemes are not limited to IR spectroscopic data only. They can be adopted in other areas of analytical chemistry for WT data processing.

2.2 Spectral compression with ~t'avelet neural net~vork The wavelet neural network (WNN), which is a combination of wavelet transform and neural network was proposed as a new algorithm for IR spectral data compression [25]. The neural network has been applied widely in chemistry [26]. It may be considered as a "black box" to transform mvariable inputs into n-variable outputs [27]. The network is formed by a group of neurons that are organized in different layer(s). Each neuron can accept m-variable inputs with different weighting factors which will be modified during a network training process. After the required computation, each neuron can deliver its own output from the current layer to the neurons at the next layer. This process is repeated on each layer until it reaches the output layer. Fig. 4(a) shows a typical single layer neural network with m inputs and 1 output. Each circle represents a neuron and has a particular weighting factor, w, which is usually derived from the sigmoidal transfer function (SF). The Z sign and S-shape symbol in each neuron represent respectively the summation operation and the sigmoidal transfer function. When spectral data X are applied to this neural network, a response or an output value, Ysv, is obtained through the following expression: n

YSF,i -- Z WiXi. i-1

(4)

As suggested in reference [25], the traditional sigmoidal function can be replaced with the Morlet wavelet basis function FDWT in neural network analysis (Fig. 4(b)). When a spectral data, X, is applied to this W N N system, a response or an output value YDWT is obtained as follows:

n

YDWT,i -- Z WiFDwT i=l

bi) ai

(5)

249

(a)

x (x,, x 2..... x~

(b)

x (x,, x 2..... x~

i F

W2

Yst (Ysr.l, YSF.2 ..... YSF.n)

(c)

YD~3- (YDv,q.I' YDW'r.2 ..... YDWT.n)

X (x,. x2..... xo)

( /

...

Wi

YDWT

Fig. 4 The architecture of (a) a single lal"er neural network with tile sigmoidal transfer [unction, as well as the wavelet neural network [br (b) IR spectral data compression, and (c) pattern recognition hi U V - V I S spectroscop.v.

and FDWT(X) -- COS(1.75X)exp(--x2/2).

(6)

In the above equation, wi, bi and ai denote the weighting factor, translation coefficient and dilation coefficient, respectively, for each wavelet basis. In Liu's work [25], the wavenumber and transmittance quantities of the IR spectrum were used as the input and target output values, respectively, of the network. Their proposed neural network consisted of a single layer network

250 with 49 neurons. With proper training, the weighting factor for each neuron and the required parameters in the wavelet function can be optimized. These authors adopted this WNN scheme to compress selected IR spectra: compression ratios of 50% and 80%, were reported when wavenumber intervals of 2.0 and 0.1 cm -1 , respectively, were used. Their work demonstrated that the original spectra can be represented and compressed by using the optimized W N N parameters, with the features of the I R spectra being well preserved.

2.3 Standardization of IR spectra with ~t'avelet transform Analytical chemists always face a problem in comparison of the performance between analytical instruments. There is no simple rule to justify which one is better because of the variations between the instrumental responses. In order to correct this, a standardization approach is generally adopted. However, a calibration model as developed on an instrument cannot be employed for the other instrument in the real situation. Walczak et al. [28] suggested a new standardization method for comparing the performance between two nearinfrared (NIR) spectrometers in the wavelet domain. In their proposed method, the NIR spectra from two different spectrometers were transformed to the wavelet domain at resolution level ( J - 1). Suppose c N IR1 and C NIR2 correspond to the NIR spectra from Instruments 1 and 2, respectively, in the wavelet domain. A univariate linear model is applied to determine the transfer parameters t between C NIRI_ and C NIR2"_I NIR1 --t f, NIR2 J - 1.n "n ~-'J - 1.n

(7)

where the quantity n represents the number of coefficients in C~_IRI and C~IR2. Once t is deduced from the standardization process, any sample spectrum from Instrument 2 in the wavelet domain can be transferred to the corresponding spectrum as acquired in Instrument 1, by: c

NIR2,new J - 1.n

--

~ /-~NIR2 tn~j_ 1.n"

(8)

Then, the standardized NIR spectrum can be obtained via inverse WT on cNIR2.new for subsequent data analysis The results [28] show that the proJ-l,n posed standardization method in the wavelet domain is superior to the traditional standardization methods.

3

Applications of wavelet transform in ultraviolet visible spectroscopy

Ultraviolet-visible spectroscopy is another technique that has been used extensively in analytical chemistry for characterization, identification and

251 quantitative applications [9,10]. They nition, data

analysis [29]. As compared with I R spectroscopy, only seven have been published which employ WT in UV-VIS spectroscopy can be classified into three major areas, namely pattern recogcompression, and data denoising.

3.1 Pattern recognition ~'ith ~t'avelet neural net~'ork Generally speaking, the term pattern recognition refers to the ability to assign an object to one of several possible categories according to the values of some measured parameters [30]. Classification of samples is one of the principal goals of pattern recognition, and can be achieved via unsupervised or supervised approaches [31]. Details of chemical applications with pattern recognition can be found in the literature [32,33]. As mentioned in the previous section, Liu and his co-workers adopted WNN for IR spectral data compression. Meanwhile, they also employed WNN in their UV-VIS spectroscopic studies [34,35]. These authors tried to determine the concentrations of molybdenum-tungsten mixture and amino acids mixtures simultaneously from the corresponding highly overlapped UV-VIS spectra with WNN. The architecture of WNN as used in their study is shown in Fig. 4(c). Mathematically, the network can be expressed as:

k n (Xi _ bi ) YDWT -- Z Wi Z XiFDwT i=l i=l ai

(9)

where k denotes the number of the wavelet basis used. The network parameters, wi, bi and ai are optimized to recognize an individual UV-VIS spectrum in the sample solution as well as possible. With a proper training of the network, the performance of WNN is better than the traditional backpropagation neural network. Besides, WNN has a higher ability to identify minor differences between UV-VIS spectra of individual components.

3.2 Compression of spectrum ~l'ith ~vavelet transfornl The advances in microelectronics have greatly enhanced mass storage capacity and processing speed. Archives of information of full spectra rather than only those of absorption peaks become more feasible. However, the demands of huge storage capacity are still somewhat prohibitive for highresolution spectra. Even if this problem can be resolved, the computer processing speed and the bandwidth of the telephone line or network is still

252 a limiting factor in data transmission. To tackle this, spectral data compression techniques play a very important role. When a spectrum is compressed by a certain method, two objectives should be met. First, a higher compression ratio should be achieved. Secondly, the reconstructed spectrum should have minimum distortion. The most commonly used compression technique in chemical studies is the Fourier transform and its variants. The advantages of FT are frequency localization, orthogonality, and the availability of fast numerical algorithms. Recently, researchers proposed to make use of WT for UV-VIS spectral data compression [17,24,36,37]. The compression process involves transformation of a spectrum to the wavelet domain. Then, the thresholding criterion is employed to select suitable coefficients in the wavelet domain, for storage. The spectrum can be restored to its original form via an inverse WT treatment on the selected (or compressed) data. Chau and his co-workers have proposed some wavelet-based methods to compress UV-VIS spectra [24,37]. In their work, a UV-VIS spectrum was processed with the Daubechies wavelet function, DI6. Then, all the Cj elements and selected Dj coefficients at different j resolution levels were stored as the compressed spectral data. A hard-thresholding method was adopted for the selection of coefficients from Dj. A compression ratio up to 83% was achieved. As mentioned in the previous section, the choice of mother wavelets is vast in WT, so one can select the best wavelet function for different applications. However, most workers restrict their choices to the orthogonal wavelet bases such as Daubechies' wavelet. Chau et al. chose the biorthogonal wavelet for UV-VIS spectral data compression in another study [37]. Unlike the orthogonal case, which needs only one mother wavelet tp(t), the biorthogonal one requires two mother wavelets, q~(t) and tb(t ), which satisfy the following biorthogonal property [38]: (Dj.k(t)t~l.m(t)dt -

{ 1 0

ifj- land kotherwise.

m.

(10)

In the biorthogonal WT method, two sets of low-pass filters, lk and ik, and high-pass filters, hk and hk, are adopted for signal decomposition and reconstruction stages, respectively. After applying biorthogonal WT to the UV-VIS spectrum, a set of scale and wavelet coefficients of Cj and Dj, Dj_I, K, D1 are obtained and are usually expressed in the floating-point representation. In order to enhance the storage efficiency, we have introduced two extra algorithms, namely the optimal bit allocation (OBA) algorithm [38] and variable length coding (VLC) algorithm [39] for data compression. As ~

253 compared with our previous work [24], the proposed biorthogonal WT method gives better performance.

3.3 Denoising of spectra )l'ith )~'avelet transform One of the main goals in analytical chemistry is to extract useful information from recorded data. However, the achievement of this goal is usually complicated by the presence of noise. In the past decades, a large number of digital filters has been developed in different fields of science and technology for the reduction of noise. In spite of the existence of diverse filters, only a few, such as the Savitzky-Golay, and Fourier, and Kalman filters [11,12] have been extensively used by chemists. Recently, WT has been identified as an effective method for denoising chemical data [1,40-42]. In UV-VIS spectroscopy, only two publications have been found which adopt WT as a denoising technique [43,44]. As stated in the previous section, most workers confine their wavelet functions in the Daubechies wavelet series only. For example, we have adopted the Daubechies wavelet function to denoise spectral data from a UV-VIS spectrophotometer [43]. In order to make use of the other available wavelet functions for chemical data analysis, Lu and Mo [44] suggested employing spline wavelets in their work for denoising UV-VIS spectra. The spline wavelet is another commonly used wavelet function in chemical studies. This function has been applied successfully in processing electrochemical signals [9,10] which will be discussed in detail in another chapter of this book. The mth order basis spline (B-spline) wavelet, Nm, is defined as follows [44]: 1

Nm(t) - Nm-1 (t) * N1 (t) -- / Nm-1 (t - x)dx

m > 2

(11)

O r

0

The symbol 9denotes a convolution operation between Nm-l and Nm. The kth term of Nm is given by k

Nm(tk) -- Z

Nm-1 (tj)Nl (tk-n)

(12)

n=0

with j _> 0 [13]. The result is equivalent to summing up the products between coefficients in Nm-1 and Nm in a shifted manner. The mother wavelet function W(t) may be expressed as

254

3m--2

W(t) -

~

q n N m ( 2 k t - n)

(13)

n=0

with qn= j:0

J

N , m ( n - j + 1) -

n - - 0 . 1 . 2 ..... 3 m - 2 .

(14)

In spline wavelet computation, two parameters, namely the order of B-spline, m, and truncation frequency, L, which represents the cut-off (or truncation) frequency value between the true signal and noise, need to be optimized. In Lu and Mo's study [44], they found that the best result for denoising UV-VIS spectra with high noise level was obtained with m = 3 and L = 4. Zhao and Wang [45] proposed a technique called the wavelet transform K-factor threewavelength method to determine simultaneously the concentrations of vanadium, molybdenum and titanium with UV-VIS spectroscopy. In their study, WT was adopted to denoise the spectra acquired. The concentrations of individual ions were determined from the UV-VIS spectra at three selected wavelengths.

4

Application of wavelet transform in mass spectrometry

In mass spectrometric studies, WT have been applied mainly in two areas including secondary ion mass spectrometry (SIMS) and instrumentation design. SIMS is a type of surface technique for trace analysis, determination of elemental composition, and the identity and concentrations of adsorbed species and elemental composition as a function of depth [46]. The application of wavelet denoising techniques to SIMS images has been studied by Grasserbauer et al. [47-50], and details about these studies are presented in another chapter of this book. With regard to instrumentation design, WT was applied to process real-time signals from the mass spectrometer. Shew [51] invented a new procedure for determining the relative ion abundances in ion cyclotron resonance mass spectrometry, by utilizing WT to isolate the intensity of a particular ion frequency as a function of position or time within the transient ion cyclotron resonance signal. In 1995, this new method was patented in the U.S. Shew explained that the WT intensity corresponding to the frequency of each ion species as a function of time can be fitted by an exponential decay curve. By

255 extrapolating these curves back in time to the end of the excitation phase, accurate values of the relative abundances of different ions within a sample can be deduced. An ion cyclotron resonance mass spectrometer with a Haar wavelet analysis module was thus set up. The result of Shew's work indicated that WT can provide high efficiency isolation of individual frequencies in the received signal corresponding to individual species. In another research study, Rying et al. [52] demonstrated the use of WT for automated run-torun detection of transient events, such as equipment faults, and automated extraction of features from time-dependent signals from a quadrupole mass spectrometer. These authors employed the wavelet analysis techniques to model MS signals for detection and control of run-to-run variability in a semiconductor process. Also, WT was utilized to transform the carrier gas (Ar +) and reaction by-product (H +) signals into the time-scale space for feature extraction, and statistical discrimination between nominal and induced fault process runs.

Application of wavelet transform in nuclear magnetic resonance spectroscopy Nuclear magnetic resonance (NMR) spectroscopy is one of the most powerful non-destructive techniques available for probing the structure of matter. So far only a few publications have related to the application of WT in N M R spectroscopy. In 1989, Guillemain et al. [53] were the first research group. They aimed at investigating how an appropriate use of WT could lead to an excellent estimation of the frequency of spectral lines in a signal and provide direct information on the time-domain features of these lines in N M R spectra. These authors reported seven applications of WT in N M R spectroscopy: these included the estimation of frequency- and amplitudemodulation laws in both simple and general cases, spectral line subtraction and re-synthesis, ridge extraction, addition of two sine waves, and of three exponentially decreasing sine waves. Recently, Neue [54] published another paper on an application of WT in dynamic N M R spectroscopy which could simplify the analysis of the free induction decay (FID) signal. Dynamic N M R spectroscopy is a technique used to measure rate parameters for a molecule [55]. The measured resonance frequencies represent the spatial coordinates of spins. Any motion, such as bond rotation and other molecular gymnastics, may change these frequencies as a function of time. The localization property of WT gives a better picture

256 of the nature of the underlying dynamic process in both the frequency and time domains. The third-order Battle-Lemari wavelets were employed for crystal rotation, and first-order kinetics, with NMR spectroscopy in their study. They concluded that WT will become a routine method for data analysis in NMR spectroscopy. A similar approach can also be found in a reference book by Hoch and Stern [56] who introduced WT as a new data processing technique for smoothing NMR data. Recently, Barache et al. [57] proposed the adoption of the continuous wavelet transform (CWT) for removal of a large spectral line and re-phasing the N M R signal influenced by eddy currents. In the NMR spectra of polymers or proteins, a large spectral line is always observed which masks some important small lines: CWT was employed in this situation to subtract this large component from the others. The authors also mentioned the application of CWT in pulsed magnetic-field-gradient NMR spectroscopy. Pulsed magnetic-field-gradient NMR is a standard technique for studying both diffusive and coherent molecular motions [57]. When there is a large change in the amplitude gradient, mechanical vibration and/or eddy currents will be induced both in the probe and in the magnet. These effects introduce large errors in measuring diffusion coefficients. CWT can remove such distortions by gradient switching, and simplify the analysis procedure.

6

Application of wavelet transform in photoacoustic spectroscopy

Photoacoustic (PA) spectroscopy is a combination of optical spectroscopy and calorimetry [58]. It is a technique for studying those materials that are unsuitable for the conventional transmission or reflection methodologies. It can be used to measure thermal and elastic properties of materials, to study chemical reactions, to measure the thickness of layers and thin films, and to perform a variety of other non-spectroscopic investigations. This technique can be applied to different types of inorganic, organic and biological materials in the gas-, liquid-, or solid phase. Nowadays, PA spectroscopy is mainly employed for material characterization [59]. Compared with other spectroscopic techniques, PA spectroscopy provides a non-destructive analysis and does not require any sample preparation. Recently, some researchers have tried to employ WT in processing signals from PA spectroscopy, and satisfactory results were obtained [60,61]. The major role of WT in PA spectroscopy is to filter simultaneously the noise and

257 the baseline in the PA spectra. The proposed method was applied to analyse PA spectra of degraded poly(vinyl chloride) (PVC) which is one of the most important commercial polymers. PA spectroscopy encountered some difficulties in characterizing the numbers of polyene sequences in degraded PVC [60]. In the initial stage of degradation, which is a very important process in the PVC industry, the absorption in the visible range is very weak [62]. As a result, the PA spectra of initially degraded PVC are often disturbed by the presence of noise at a high level. The irregular baseline also introduces difficulties in determining the PA absorption bands. In this situation, the zoomin and zoom-out capabilities of WT can help to extract both noise and baseline from the PA spectrum. After processing the PA spectrum with WT at a particular resolution level, j, the scale coefficients, Cj, represent the baseline while the wavelet coefficients {Dj, Dj+I. K. Dj_l } represent the noise.

7

Conclusion

In conclusion, WT has been applied successfully for data processing in various fields of spectroscopic studies. We can see that more W T - related applications in spectroscopy will be developed in the near future when more spectroscopists become aware of the unusual properties of wavelets. Raman spectroscopy, electronic spectroscopy, rotational spectroscopy, and vibrational spectroscopy could be new areas to be explored- and no application of WT has yet been reported in such areas.

8

Acknowledgement

This work was supported by Research Grant Council (RCG) of the Hong Kong Special Administration Region (Grant No. HKP 45/94E) and the Research Committee of The Hong Kong Polytechnic University (Grant No. A020).

References 1. V.J. Barclay, R.F. Bonner and I.P. Hamilton, Application of Wavelet Transforms to Experimental Spectra: Smoothing, DENOISING, and Data Set Compression, Analytical Chemisto', 69 (1997), 78-90. 2. P.D. Willson and T.H. Edwards, Sampling and Smoothing of Spectra, Applied Spectroscopic Review, 12 (1976), 1-81.

258

10. 11. 12. 13. 14. 15. 16.

17.

18. 19. 20.

21. 22.

K.M. keung and F.T. Chau, A Review on Signal Compression of Spectroscopic Data in Analytical Chemistry, Acta Phi"sits Chim. Sin., 13 (1997), 857-864. W.A. Warr, Computer-assisted Structure Elucidation- Library Search and Spectral Data Collections, Analytical Chemistry, 65 (1993), 1045A-1050A. L. Glasser, Fourier Transforms for Chemists Part I. Introduction to the Fourier Transform, Journal of Chemistry Education, 64 (1987), A228-A233. L. Glasser, Fourier Transforms for Chemists Part II. Fourier Transforms in Chemistry and Spectroscopy, Journal of Cllemistrv E~hication, 64 (1987), A260A266. L. Glasser, Fourier Transforms for Chemists Part III. Fourier Transforms in Data Treatment, Journal of Chemistry' E~hlcation, 64 (1987), A306-A313. I. Daubechies, Orthonormal bases of Compactly Supported Wavelets, Communications on Applied Pure Mathematics, 41 (1988), 909-996. A.K.M. Leung, F.T. Chau and J.B. Gao, A Review on Applications of Wavelet Transform Techniques in Chemical Analysis" 1989-1997, Chemometric Intelligent Laboratory System, 43 (1998), 165-184. A.K.M. Leung, Wavelet Transform in Chemistry, http" fg702-6.abct.polyu.edu.hk/ ~kmleung/wavelet.html, (accessed January 1999). S. Brown, T.B. Blank, S.T. Sum and L.G. Weyer, Chemometrics, Analytical Chemistry, 66 (1994), 315R-359R. S. Brown, S.T. Sum and F. Despangne, Chemometrics, Anal~'tical Chemisto', 68 (1996), 21R-62R. B.B. Hubbard, The World According to Wavelets. The Story of A Mathematical Technique in The Making, A K Peters, Wellesley, MA, 1996. C.K. Chui, An Introduction to Wavelets, Academic Press, Boston, MA, (1992). I. Daubechies, Ten Lectures on Wavelets, SIAM Press, Philadelphia, PA, (1992). B.K. Alsberg, A.M. Woodward and D.B. Kell, An Introduction to Wavelet Transforms for Chemometricians: A Time-frequency Approach, Chemometric Intellelligent Laboratory Systems, 37 (1997), 215-239. B. Walczak and D.L. Massart, Noise Suppression and Signal Compression using the Wavelet Packet Transform, Chemometric Intelligent Laboratory Systems, 36 (1997), 81-94. B. Walczak and D.L. Massart, Wavelets- Something for Analytical Chemistry, Trends Analytical Chemistry, 16 (1997), 451-463. S.G. Mallat, Multiresolution Approximation and Wavelets, Translational American Mathematical Society, 315 (1989) 69-88. F.R. Verdun, C. Giancaspro and A.G. Marshall, Effects of Noise, Time-domain Damping, Zero-filling and the FFT Algorithm on the "'Exact" Interpolation of Fast Fourier Transform Spectra, Applied Spectroscopy, 42 (1988) 715-721. N. Morrison, Introduction to Fourier Analysis, Wiley, New York, (1994), p. 388. A.K.M. Leung, F.T. Chau, J.B. Gao and T.M. Shih, Application of Wavelet Transform in Infrared Spectrometry" Spectral Compression and Library Search, Chemometric Intelligent Laboratory System, 43 (1998), 69-88.

259 23. J.W. Hayes, D.E. Glover, D.E. Smith and M.W. Overton, Some Observations on Digital Smoothing of Electroanalytical Data Based on the Fourier Transformation, Analytical Chemisto., 45 (1973), 277-284. 24. F.T. Chau, T.M. Shih, J.B. Gao and C.K. Chan, Application of the Fast Wavelet Transform Method to Compress Ultraviolet-Visible Spectra, Applied Spectroscopy, 50 (1996), 339-349. 25. W. Liu, J.P. Li, J.H. Xiong, Z.X. Pan and M.S. Zhang, The Compression of IR Spectra by Using Wavelet Neural Network, Chinese Science Bulletin (English Edition), 42 (10) (1997), 822-825. 26. J. Zupan and J. Gasteiger, Neural Networks: A New Method for Solving Chemical Problems or Just a Passing Phase? Anah'tical Chinl. Acta, 248 (1991), 1-30. 27. J. Zupan and J. Gasteiger, Neural Networks for Chemists, VCH, Weinheim, (1993). 28. B. Walczak, E. Bouveresse and D.L. Massart, Standardization of Near-infrared Spectra in the Wavelet Domain, Chemometric Intelligent Laboratory System, 36 (1997), 41-51. 29. H.H. Perkampus, UV-VIS Spectroscopy and Its Applications, Springer-Verlag, Berlin, (1992). 30. M.J. Adams, Chemometrics in Analytical Spectroscopy, The Royal Society of Chemistry, Cambridge, (1995), pp. 92-154. 31. R.G. Brereton, Chemometrics in Analytical Chemistry: a Review, Am~ivst, 112 (1987), 1635-1657. 32. D.D. Wolff and M.I.L. Parsons, Pattern Recognition Approach to Data Interpretation, Plenum, New York, (1983). 33. O. Strouf, Chemical Pattern Recognition, Research Studies Press, Letchworth, (1986). 34. W. Liu, Y.M. Wang, Z.X. Pan, W.L. Zhou and M.S. Zhang, Simultaneous Determination of Molybdenum and Tungsten using Wavelet Neural Network, Chinese Journal o['Anah'tical Chemistry', 25 (1997), 1189-1191 (in Chinese). 35. W. Liu, J.H. Xiong, H. Wang, Y.M. Wang, Z.X. Pan and M.S. Zhang, The Recognition of UV Spectra by Using Wavelet Neural Network, Chem. Journal o.[ Chinese UniversiO', 18 (1997), 860-863 (in Chinese). 36. F.T. Chau, J.B. Gao, T.M. Shih and J. Wang, Infrared Spectral Compression Procedure using the Fast Wavelet Transform Method, Applied Spectroscopic, 51 (1997), 649-659. 37. H.L. Ho, W.K. Cham, F.T. Chau and J.Y. Wu, Application of Biorthogonal Wavelet Transform to the Compression of Ultraviolet-visible Spectra, Comput. Chemisto', 23 (1999), 85-96. 38. M. Antonini, M. Barlatid, P. Mathieu and I. Daubechies, Image Coding using Wavelet Transform, IEEE Transactions on hnage Processing, 1 (1992), 205-218. 39. A. Fourier, SIGGRAPH "94 Course Notes, SIGGRAPH "94, July 24-28, Orlando, FL, Association for Computing Machinery. New York, (1994). 40. C.R. Mittermayr, S.G. Nikolov, H. Hutter and M. Grasserbauer, Wavelet Denoising of Gaussian Peaks: a Comparative study, Chemometric Intelligent Laboratory Systems, 34 (1996), 187-202.

260 41. C.R. Mittermayr, E. Rosenberg and M. Grasserbauer, Detection and Estimation of Heteroscedastic Noise by Means of the Wavelet Transform, Analytical Communnication, 34 (1997), 73-78. 42. S.N. Qian and H. Sun, Data-compression Method Based on Wavelet Transformation for Spectral Information, Spectroscopic. Spectral Analysis (Beijing), 16 (1996), 1-8. 43. J.B. Gao, F.T. Chau and T.M. Shih, Wavelet Transform Method for Denoising Spectral Data from UV-VIS Spectrophotometer, SEA Bulletin Mathematics, 20 (1996), 85-90. 44. X.Q. Lu and J.Y. Mo, Spline Wavelet Multi-resolution Analysis for High-noise Digital Signal Processing in Ultraviolet-visible Spectrophotometry, Analyst, ]2l (1996), 1019-1024. 45. K. Zhao and Z.H. Wang, Simultaneous Determination of Vanadium, Molybdenum and Titanium by Wavelet Transform K-factor-Three-Wavelength Method, Chinese Journal of Analytical Chemistry, 26 (1998), 620 (in Chinese). 46. H.A. Strobel and W.R. Heineman, Chemical Instrumentation A Systematic Approach, 3rd Ed., John Wiley & Sons, Inc., New York, (1989), pp. 824-829. 47. H. Hutter, C. Brunner, S.G. Nikolov, C. Mittermayer and M. Grasserbauer, Imaging Surface Spectroscopy for Two- and Three-Dimensional Characterization of Materials, Fresenius' Journal of Analytical Chemistr)', 355 (1996), 585-590. 48. S.G. Nikolov, H. Hutter and M. Grasserbauer, De-noising of SIMS Images via Wavelet Shrinkage, Chemometric Intelligent Laboratory System, 34 (1996), 263273. 49. M. Wolkenstein, H. Hutter, M. Grasserbauer, Wavelet Filtering for Analytical Data, Fresenius' Journal of Anah'tical Chemistry, 358 (1997a), 165-169. 50. M. Wolkenstein, H. Hutter, S.G. Nikolov and M. Grasserbauer, Improvement of SIMS Image Classification by Means of Wavelet Denoising, Fresenius' Journal o) Analytical Chemistry, 357 (1997b), 783-788. 51. S.L. Shew, Method and Apparatus for Determining Relative Ion Abundances in Mass Spectrometry Utilizing Wavelet Transforms, US Patent 5,436,477, July 25, Government Printing Office, Washington, DC, (1995). 52. E.A. Rying, R.S. Gyurcsik, J.C. Lu, G. Bilbro, G. Parsons and S.Y. Sorrell. Wavelet Analysis of Mass Spectrometry Signals for Transient Event Detection and Run-to-run Process Control, in Process Control, Diagnostics, and Modelling i~ Semiconductor Manufacturing, Electrochemical Society Proceedings, Vol. 97-9. Montreal, Canada, May 1997, (M. Meyyappan, D.J. Economou and S.W. Butle~ Eds), The Electrochemical Society, Inc., New Jersey, (1997), pp. 37-44. 53. P. Guillemain, R. Kronland-Martinet and B. Martens, Estimation of Spectral Line, with the Help of the Wavelet Transform- Application in NMR Spectroscopy, ir Wavelets and Applications, Proceedings of the Second International Conference or Wavelets and Their Applications, Marseilles, France, May 1989, (Y. Meyer Ed.) Springer-Verlag, Paris, 1992, pp. 38-60. 54. G. Neue, Simplification of Dynamic NMR Spectroscopy by Wavelet Transform Solid State Nuclear Magnetic Resonance, 5 (1996), 305-314.

261

55. W. Kemp, NMR in Chemistry A Multinuclear Introduction, Macmillan, London, (1986), pp. 158-168. 56. J.C. Hoch and A.S. Stern, NMR Data Processing, Wiley-Liss, Inc., New York, (1996), pp. 144-151. 57. D. Barache, J.P. Antoine and J.M. Dereppe, The Continuous Wavelet Transform, an Analysis Tool for NMR Spectroscopy, Journal o/" Magnetic Resonance, 128 (1997), 1-11. 58. A. Rosencwaig, Photoacoustics and Photoacoustic Spectroscopy, Wiley, New York, (1980), pp. 1-6. 59. M.L. McKelvy, T.R. Britt, B.L. Davis, J.K. Gillie. L.A. Lentz, A. Leugers, R.A. Nyquist and C.L. Putzig, Infrared spectroscopy, Anah'tical Chemisto', 68 (1996), 93R-160R. 60. J.J. Mao, Q.D. Su and M.S. Zhang, Wavelet Analysis Applied in Photoacoustic Spectroscopy, in New Trends in Chemometrics, First International Conference on Chemometrics in China, Zhangjiajie, P.R. China, 17-22 October, 1997, Y.Z. Liang, R. Nortvedt, O.M. Kvalheim, H.L. Shen and Q.S. Xu (Eds). Hunan University Press, Changsha, P.R. China, (1997), pp. 197-198. 61. J.J. Mao, P.Y. Sun, Z.X. Pan, Q.D. Su and M.S. Zhang, Wavelet Analysis on Photoacoustic Spectra of Degraded PVC, Fresenius' Journal of Analytical Chemistry, 361 (1998), 140-142. 62. A.A. Yassin and M.W. Sabaa, Degradation and Stabilization of Poly(vinyl chloride), Macromolecular Chemical Physics, C30 (1990), 491-558.



263

CHAPTER 12 Applications of Wavelet Analysis to Physical Chemistry Heshel Teitelbaum Department of Chemistry, University o[ Otta~'a, Ottawa, Ont., Canada K1N 6N5

1 Introduction The four cornerstones of physical chemistry are quantum mechanics, statistical mechanics, thermodynamics, and kinetics. Within these divisions we can recognize distinct subdivisions. Quantum mechanics, for example, can be divided into sections dealing with the solution of the Schr6dinger e q u a t i o n namely aspects which deal with the wavefunction and aspects which deal with the associated energy. The former gives us information about the atomic, molecular, and crystalline structure; whereas the latter gives us information about energy levels. Transitions among energy levels, of course, comprise the subject of spectroscopy, treated in its fundamental sense as a probe of the inner structure of molecules (in distinction from the applied or analytical sense which gives us qualitative and quantitative information about the substance, and which is treated elsewhere in this volume). Thermodynamics describes the equilibrium properties of substances such as bulk structure of solids, liquids and gases. Finally, for the purposes of this overview we consider chemical kinetics as one section of the broader field of ~'change", the other section being chemical dynamics. Physical chemistry is among the youngest of the scientific disciplines to which wavelet analysis has been applied. Thermodynamics and statistical mechanics, for example, are not yet represented. As such, only a portion of physical chemistry is actually addressed. However, common to all of the examples which we do have, is an ~'observed" signal or image which is a complicated function of either time, frequency or space. The use of wavelet analysis is geared to extracting patterned information buried in that signal. This can take the form of deconvolution, signal compression, signal denoising, or simulation by solving the differential equation which governs the observed phenomenon.

264

2

Quantum mechanics

2.1 Molecular structure

The distribution of electron density in space is a simple means of visualizing the structure of molecules. One needs to calculate the probability distribution, pi(z), using the wavefunction ~Pi pi('r) -- / ~ dr where dr is the generalized volume element, and ~Pi is the solution of the time-independent Schr6dinger equation+ H ~ i = EitlJi wherein H is the Hamiltonian operator of the molecular system, and Ei is the corresponding eigenvalue or energy of the ith stationary state. The Hamiltonian is a combination of the Laplace and potential operators. The potential energy operator accounts for the interactions between all electrons and nuclei. Often the nuclear motion can be ignored (Born-Oppenheimer approximation); however, one is still left with a pairwise-additive multi-electron interaction described by three co-ordinates for each electron. The resulting partial differential equation can only be solved by approximate numerical means for almost all problems of practical interest in chemistry because of the difficulty of numerically integrating multi-electron terms with inseparable co-ordinates [1]. One of the ways of simplifying the task is to recognize that the set of momentum co-ordinates, p, is conjugate to the set of position coordinates, r. Representing the Schr6dinger equation in terms of momentum co-ordinates transforms the problematical terms into one-centre terms, leaving the Hamiltonian invariant. Apart from this advantage, one also obtains a different visualization of the molecule in terms of electronic momenta, which would otherwise be averaged out in the usual representation. This presents an interesting approach which is amenable to analysis by wavelet transforms. This feature was first demonstrated by Fischer and Defranceschi [2,3]. For pedagogical purposes they chose, in their first study, to represent the typical molecular wavefunction, q~i, as a linear combination of atomic orbitals, (I)i (normally expressed in position space), weighted by coefficients ci, and then they expressed it in momentum space from which they were able to derive Fourier transforms, and compared the results with wavelet transforms. For example, the simplest basis function i.e. the l s Gaussian-type orbital,

265

(I)(x) -- (2at/rt)l/4e-Z~x-" is Fourier-transformed in momentum space to 1 / +(P)- x/~

(I)(x)e -ixp dx - (2~)-1/4e -p2/4

where x and p are the conjugate position and momentum co-ordinates respectively. It is the property of the Fourier transform which gives rise to the similarity in forms. Because of the inverse relation between x- and p- spaces the two representations are complementary [4]. One of the questions one would like to answer for atoms is in what position in space the momentum changes significantly. Unfortunately, Fourier transforms cannot give simultaneous information on position and momentum distributions. Physically this is because the momentum operator is basically a derivative, making the Fourier transform small when x is large, and conversely the momentum density large near the atomic nucleus. However the wavelet transform permits viewing both aspects simultaneously: and at the same time it can avoid the spurious non-physical oscillations in W which sometimes result from standard quantum chemistry programs [5]. Wavelets are a set of basis functions that are alternatives to the complex exponential functions of Fourier transforms which appear naturally in the momentum-space representation of quantum mechanics. Pure Fourier transforms suffer from the infinite scale applicable to sine and cosine functions. A desirable transform would allow for localization (within the bounds of the Heisenberg Uncertainty Principle). A common way to localize is to left-multiply the complex exponential function with a translatable Gaussian "window", in order to obtain a better transform. However, it is not suitable when r varies rapidly. Therefore, an even better way is to multiply with a normalized translatable and dilatable window, q/a.b(X)- atl/2q/([X- b]/a), called the analysing function, where b is related to position and 1/a is related to the complex momentum, qt(x) is the continuous wavelet mother function. 1 The transform itself is now

IThe reader should note a possible source of confusion. The traditional symbols for quantum mechanical wavefunctions and their component basis sets are the same as the wavelet and scaling functions used in wavelet analysis. In wavelet analysis one multiplies the function of interest, here the wavefunction, by the wavelet and scaling basis sets. In order to avoid confusion, we choose upper case W and 9 for quantum mechanical wavefunctions and lower-case ~ and q~ for wavelet and scaling functions.

266

F

Fo(a,b) - J O(x)q/a.b(X)*dx An important feature of wavelet analysis is to find the most appropriate mother function. This is not always obvious. The ranges of a and b are flexible, giving rise to continuous wavelets if unlimited [6], or orthonormal discrete wavelets if limited [7]. For the atomic orbital example, above, the authors demonstrated the effect of choosing as a mother function

~/a,b(X) --

2x

eX2/2

In that case the continuous wavelet transform becomes

(2) 1/4( 2a )3/2 Fo(a,b) - 2b

1 + 2a 2

e-b2/(l+2a):

which displays simultaneously both the momentum and position dependencies. This kind of visualization is useful, for example, when interpreting experimental electron momentum densities and spectroscopy [8,9]. (Note that Fo(a,b) will play, below, the role of a set of coefficients of an orthonormal basis set, which we shall see in an application of discrete wavelet analysis.) Further developments [3] lead naturally to improved solutions of the Schr6dinger equation, at least at the Hartree-Fock limit (which approximates the multi-electron problem as a one-electron problem where each electron experiences an average potential due to the presence of the other electrons.) The authors apply a continuous wavelet mother, qJ(x), to both sides of the Hartree-Fock equation, integrate and iteratively solve for the transform rather than for the wavefunction itself. In an application to the hydrogen atom, they demonstrate that this novel approach can lead to the correct solution within one iteration. For example, when one separates out the radial (one-dimensional)component of the wavefunction, the HartreeFock approximation as applied to the hydrogen atom's doubly occupied orbitals is, in spherical coordinates, 1 d20(x) 2

dx 2

O(x) Ix

=

O(x)

where ~ is the eigenvalue and 9 is the eigenfunction of interest. The transformed equation becomes

267

2 ~2 (q/a'b(X)dxx ) f ~ ( x ) li -- J" d 2dx

q/a'b(X)dx -- 8 f (I)(x)q/a.b(X)dx

The term on the right-hand side is obviously eF,; while the two terms on the left-hand side, can also each be written in terms of F,. As a trial function Fischer and Defranceschi used the following scaled and shifted mother wavefunction:

~/a.b(X)_ _~/ a3 2v/-~(x - b)

e

i,~-bt 2 2a2

in which case the transformed integro-differential equation becomes 3 ) -~

Fo(a,b) -

1 ~Fo(a,b) 2a aa b 2 p2

2 ~j(

+ --~ ~-, v, rCeD

0

dp

/

d[3

efr~-p

F,

a

,

[3

0

Now, instead of solving for the unknown pair, e and ~, one solves for the pair, e and F , . The authors tested the method on the 2p orbital of the hydrogen atom. Using the known eigenvalue of the trial Gaussian type function, O(x) -

xe -x2

they determined the corresponding first analytical approximation to F , . This was substituted into the right-hand side of the transformed Hartree-Fock equation to determine the next approximation to F , . Although, in principle, one could go on to determine a better value for e and then obtain a better F , , the authors stopped at the first iteration, since it already gave a result which was very close to the correct transform of the true solution, 21/2xe-I"l. Of course, the solution of the hydrogen atom problem is known. However, the implication is that the method will work also for more difficult problems with unknown solutions. In yet another development, rather than using continuous wavelet transforms, the same authors investigated the use of orthonormal wavelets in conjunction with the BCR algorithm in order to develop a Fast Wavelet

268

Transform useful for representing the H a r t r e e - F o c k operator for solving large chemical systems [3,10,11]. They chose the discrete wavelet basis sets described by Daubechies [12]"

L-I q/J'k(x) -- Z glq)J-1.2k+l(x)' 1=0

J -- 1. . . . . n

L-1 q)J'k(x) -- Z hlq)J -1.2k+l(x)' 1=0

j -- 1. . . . . n

where L is a limited number of coefficients. The actual coefficients, g and h, are related by gk -- (--1)khL-k-1,

k - 0 , . . . ,L - 1

with hk -- (q), q)-l,k). The value of j defines the coarseness of the scale. The wavelet transforms are

L-1 (I)(x)*j'k(X)dx - Z glCj-l'2k+l 1=0

dj'k -j q,k

m

L-1 (I)(x)q)J.k(X)dx -- Z hlCj-l.2k+l l=O

With an initial set of coefficients, C0.k, these recursion formulae lead to a complete set of coefficients. For hydrogen-like atoms the Schr6dinger equation is reduced to a one-dimensional eigenvalue problem in terms of the radial co-ordinate, as above. Starting with an initial guess to the wavefunction, (I)(x), the solution is obtained by iterative approximations involving the evaluation of the H a r t r e e - F o c k operator operating on ~(x), as above, and generating a better ~(x). What used to be a continuous function, F , , above, is now treated as a set of discrete coefficients, dj.k. When the H a r t r e e - F o c k operator is expressed (in non-standard form) as a sparse matrix, and it is applied to the set of transforms, dj.k and q.k, treated as a vector, the result is a set of transformed coefficients which can be used to calculate the system energy. It is found that, for several trial wavefunctions, (although the expectation values of the potential and kinetic energies are poor) the total energies are within 3% of the theoretical values. It is also shown that greater accuracy results from smaller discretization intervals, i.e. from a larger number of scales (at the expense of making the matrix denser and the calculation more costly). Furthermore, too many iterations lead to a spurious eigenvalue, which can be traced back to the influence of the Coulombic

269 singularity transforming into an undesireable pseudo potential. However, with care the method seems to hold promise for solving structures of complex molecules. In summary, the expected success of wavelet transforms for solving electronic structure problems in quantum mechanics are due to three important properties: (a) the ability to choose a basis set providing good resolution where it is needed, in those cases where the potential energy varies rapidly in some regions of space, and less in others; (b) economical matrix calculations due to their sparse and banded nature; and (c) the ability to use orthonormal wavelets, thus simplifying the eigenvalue problem. Although, in the above examples, it is Coulombic potentials that are involved, methods have also been developed which allow us to deal with arbitrary potentials. The procedure again involves transforming the Schr6dinger equation with the wavefunction expanded in terms of wavelets. Methods developed by Latto et al. [13], Beylkin [14], and Dahmen and Micchelli [15] then allow us to obtain the matrix elements of the potential operator, V, and of the kinetic energy operator, T, usable to evaluate the Hamiltonian matrix elements: (q/j.k]H]q/j,.k,). (q/j.k]H]q)j.k,) and (q)j.k]H]q)j.k,). For the kinetic energy components the procedure reduces to solving a set of coupled linear algebraic equations. This is simplified even further by the relationship due to the scaling between levels of detail. For example, consider the scaling functions, q~. If one scales successively by factors of 2 then one can show that (q0j.klTlq0j.k, } --2-J{q00.klTlq~0.k,), where T is a second derivative which can be applied analytically. For the terms involving potential energy, actual numerical quadrature is required. However, expansion of the potential function in terms of wavelets makes the procedure accurate and efficient. Thus,

J

V(x) j=0 k

k

Sweldens and Piessens [16] have demonstrated how to numerically calculate the expansion coefficients of the potential energy in terms of Daubechies basis functions. Modisette et al were the first to treat complicated potentials

270 using these orthonormal basis functions [17]. They examined the simple harmonic oscillator potential as well as a steep double well potential. Thus they have taken the first step in the transition from the description of atoms to that of molecules. In the harmonic oscillator case, for which V(y) cx 1/2y 2 and for which exact solutions are known, the authors were able to solve for the eigenvalues to an accuracy of one part in 1014, for maximum values of k-,~ 30. This could be obtained only when the Daubechies basis functions were very smooth, i.e. when L was as large as 20. The larger the scale of resolution the quicker the convergence with respect to k; but this came at the price of increased bandedness of the matrices. Note that in this special case, all terms < ~j.k]V >, are identically zero because of the property of vanishing moments for the wavelet functions, DC

J

" xn~(x)dx -- 0

--OC

where n is a positive integer. So the expansion of the potential only contains contributions from the scaling functions, which are easily evaluated. A better test of the Daubechies basis set's ability to handle complex problems is the pointy double well potential showing regions of rapid and slow dependence on y: V(y) o( -

1

1

V/(y- 5a)2+ a2

V/(y + 5a) 2 + a 2

Not only is there a near-singularity approaching that of a Coulombic centre, but there are two such centres, at y 4-5a. When a - 0.1 atomic units one could model, for example, the H~+ molecule ion. To rewrite the potential in terms of wavelets the authors chose y - 0 as the expansion origin, considered y to extend to 4-250 a, and subdivided the space into 100 equal intervals around the origin, thus taking 101 scaling functions. The wavelet expansion coefficients, though, decrease rapidly as y increases. Therefore, only six levels of resolution were needed (j -- 0-5), associated with spacings of 5a • 2j-5. The finer level wavelets concentrate in the wells. Focusing on the 5th excited energy level (which is influenced by both the rapidly as well as the slowly varying parts of the potential), the authors solved the Schr6dinger equation by the method described above. This ne-

271 cessitated expanding the quantum wavefunction in wavelets, generating and diagonalizing the Hamiltonian matrix, and determining the eigenvalue for different wavelet bases, retaining as many wavelets as necessary to maintain an accuracy of 1 part in 108 at a given level of resolution. Only 25 scaling functions were needed for the coarsest level of resolution. The procedure was then repeated for successively better levels of resolution until convergence at the exact result was reached. A total of 110 basis functions of all types were required for this part of the calculation. We conclude that once other families of wavelets are compared with the Daubechies set, and once criteria are developed for deciding on the origin and scale needed for expanding generalized potentials, the technique will result in accurate calculations of molecular energies. From a modest beginning with H-like atoms, to diatomic molecules, the field has now expanded to multidimensional problems. Here density functional theory is most appropriate, and initial studies have emerged (using nonorthonormal wavelets) [18]. Our abilities to calculate the electronic structure of multi-electron substances in cubic lattices [19] and molecular vibrations in four-atoms systems [20,21] have been extended by making full use of powerful parallel computers. The approach of Arias et al. [19] to determine the electronic structure of all the atoms in the periodic table is to expand functions, f, in three dimensions as a sum of scaling functions at the lowest resolution plus wavelet functions of all finer resolutions: Jnla\

f(r) -- Z Cjo'nq)Jo.n (r) ~- Z E dj.n q/j.n (r) n J=Jo n 9

.

As in the work of Fischer and Defranceschi described above, the Schr6dinger equation reduces to an eigenvalue problem. Beginning with a subspace containing scaling functions at scale J0, i.e. qb(2J"x- 1)~(2J"y- m)q~ (2J"z- n), space is subdivided into more and more detailed lattices of higher and higher resolution. Only the basis functions that have significant coefficients need to be retained. At distances far from the atomic core lower resolution is required. As the core is approached, where the electronic wavefunction oscillates rapidly, finer scales are added as needed, until the calculations converge or the desired accuracy is achieved. In the case of the hydrogen atom the authors find that, to reproduce the energy of the ls state within 2%, 7 scaling functions are needed for the region bounded by radii of 0.5 to 1 Bohr: while an additional 6 are needed for the region 0.25 to 0.5 Bohr: and another 6 for 0.125 to 0.25 Bohr, and another 6 for the innermost core. For greater ac-

272 curacy, they needed only to increase the size of the basis set modestly. As the atomic number increases, and the Coulomb potential becomes stronger, greater resolution is required; but it is found that only one scale need be added every time the atomic number doubles. Thus for uranium, rather than requiring a basis set of 108 plane waves as normally required, the wavelet approach requires only 67 basis functions. Using the same fixed basis set, the l s state's energy could be calculated to within 3% for all atoms of the periodic table. Interestingly, the same procedure as in the H atom can be used to calculate the energy of the molecular ion H +. The centres of the basis functions do not change as the atomic positions are varied. With no more than 167 basis functions the total energy is within 1% of the exact value for all arbitrary choices of atomic separations. When the need arises to suddenly change the basis set at a particular geometry, an extremely small discontinuity ~0.3 meV results. The same authors are continuing the development of the technique to describe the structure and energies of systems consisting of many electrons. They have succeeded in the case of an array of carbon atoms on a small cubic lattice using the local density approximation. The properties of wavelets as basis functions or as tools to visualize position and m o m e n t u m space simultaneously are only two of several. Others have been barely investigated. In particular, prospects for solving the time-dependent Schr6dinger equation [22] are exciting. Also, as has been emphasized by Calais [23], wavelets themselves can be treated as coherent states. The dilation/translation operation cited above, Illa.b(X) --1o[[l/2q/([X- b]/a), can be viewed as the application of a unitary operator, U(a,b), tp~,b(X) = U(a,b)~(x) In this case one can show that U(a,b)U(a', b') = U(a'a,b + a'b) and that if a' = 1/a and b' = b/a, then U(a,b) -1 - U(1/a, b/a) - U(a,b) + thus satisfying the same properties that e.g. the coherent states of a harmonic oscillator satisfy, U[p,q]+qU[p,q] = Q + ql U[p,q]+PU[p,q] - P + pl

273 where [p, q] are the co-ordinates in phase space, and P and Q are the position and momentum operators. Thus the wavelets, ~a.b(X), form a set of coherent states. Calais also discusses Zak's quantum mechanical kq-representation of Bloch electrons in electric and magnetic fields. It is another example of the concept of wavelet transforms since the translation and dilation operators which are involved, T(a) and ~(2rt/a) respectively, form a special case of the wavelet transform, U(a,b), i.e. X(na)f(x) = f ( x - na) ~(2rtk/a) - e -2rtkix/a The Zak transform, i.e. the combined operation on the set of Bloch functions, ~/kq(X), a(X -- q - na)e ikna

~/kq(X) -- ~//7-~ Z n

generates a family of coherent states where n and k are integers and 8 is the delta-function. Exploitation of this approach does not seem to have occurred yet in quantum chemistry.

2.2 Spectroscopy The electromagnetic spectrum of a molecule is essentially a representation of the probability, Jan,n]2 for transitions between any initial state, n, and any final state, n', as a function of the frequency, v, of incident radiation and time, t. In the limit of small perturbations by the radiation

4

>2

an,(t)an(t ) -- h2A2 < kIJn,l~l.lklJn

[E0(v)] "- sin2(rtAt)

where the transition moment integral, [LI.n,n,is defined as ~-In,n -- J" W~,(q)ll(q)Wn(q)dq

E ~ is the spatially dependent amplitude of the radiation's electric field, I1 the instantaneous dipole moment of the molecule, W the molecular wavefunction, A-- 8n,/h- 8n/h- v is the energy of the system state, and where the integration is carried out over all co-ordinate space q. If the wave functions of initial and final states are known, then the transition moment integral is easily evaluated. Con-

274 versely it is possible, in principle, to deconvolute the experimental ultrahigh resolution spectrum and obtain the wavefunctions, or at least the effective Hamiltonian. Quack and Jolicard have each written good reviews on the subject [24,25]. We note that the transition moment integral, [In,n, can be thought of as a transform. If the wavefunctions can be expanded as wavelets, as described above, then it should be possible to reformulate the problem and determine the effective potentials with less effort than usual. This appears to be an ideal approach, considering that the potential has regions of greater or lesser detail (in position space), and so does the spectral transform (in frequency space). The procedure of Wickerhauser for inverting complicated maps of large pparameter configuration space to a large d-dimensional measurement space may find important applications here, since the usual analysis requiring an effort of the order of d 3 can be replaced by one of the order of d 2 log d using Haar-Walsh wavelet transforms [26]. However, it seems that this approach has not yet been attempted. Instead, the application of wavelets to spectroscopy has surfaced differently, in terms of signal processing - imaging, decomposition, quantification, filtering, compression, and denoising. Spectroscopy in the ultraviolet [27], the infrared [28,29], the microwave [30], and radio (NMR) [31] regions of the electromagnetic spectrum have all been investigated. Secondary ion mass spectrometry (SIMS), as an imaging tool, has also profited from wavelet-transform processing [32,33]. Algorithms for fast decomposition and reconstruction of the spectra have been described by Depczynski et al. [34]; while those for denoising aspects have been described by Alsberg et al. [29] and by Wickerhauser [26]. These subjects are more fully described in the chapters on applied spectroscopy and on analytical chemistry.

3

Time-series

3.1 Chemical dynamics Quantum molecular dynamics is a natural offshoot of quantum mechanics whereby the fate of an encounter between atoms or molecules is determined by experiment or by numerical simulation. Simulation essentially involves solution of the Schr6dinger equation. The potential energy of interaction is assumed to have already been determined previously by variational techniques; and the initial wavefunctions (energy and geometrical structure) of

275

the reagents determined by techniques such as those described above. One can either solve the time-dependent Schr6dinger equation [22], or else solve the quantum scattering problem in terms of reaction probabilities using the time-independent Schr6dinger equation [34--44]. Alternatively, if the encounter is understood at its deepest level of detail (position and momenta of all atoms as a function of time), then one can invert the problem to determine the interaction potentials. Of course, the dynamics can be studied approximately, by solving the classical Hamiltonian equations of motion [45,46], instead of the Schr6dinger equation: and this is the procedure to which wavelet transforms have actually been applied. Essentially a time series of positions and velocities is analysed. The variations with time are complicated, displaying irregular regions of high and low frequency components, sometimes buried in noise. This localization is exactly what wavelet analysis is capable of addressing. Second, the method's feature of multi resolution analysis permits it to decompose observations into subsets, remove some, and thus act as a filter, especially of noise. Instead of space and momentum coordinates, as in the case of molecular structure, here we have time and frequency co-ordinates (as in the case of signal and image processing). The first applications of wavelet transforms to analyse time series in the field of chemical dynamics were those of Permann and Hamilton [47,48]. Their interest lay in modelling diatomic molecules, close to dissociation, perturbed by a photon. They modelled the reaction using the equation of motion for a forced and damped Morse oscillator, given by: d2x

P dt 2 =

-213D(e -~• -- e -213x) -- C ~dx - + (1 - ~x)e-~XF sin(rot)

where p is the reduced mass, x the atomic separation, [3 the Morse force constant, C the damping coefficient, F the forcing coefficient (representing photon electric field intensity), m the photon frequency and ~ describes the coupling of the photon to the molecular dipole moment. The initial value of x is set to correspond to large energies close to the dissociation limit, D; and the differential equation is solved numerically. This equation is nonlinear; and because the initial departures from equilibrium are ~large" and there is positive feedback between oscillator and photon, the solution results in oscillatory or chaotic behaviour [49], depending on the choice of parameters, C and F. For significant forcing and damping it would be trivial to observe the regimes when oscillation gives way to chattering and then to chaos. One traditional way of analysing for chaos is to draw phase plots of x versus dx/dt and check for banded limit cycles. However, it is not so clear how to

276 investigate the onset of chaos when C or F are small. The authors present an approach to magnify the effects by drawing phase plots involving higher derivatives. This assumes that the unstable phenomenon is not so small that it would be buried within the instability of the numerical integrator itself. Despite success, one is still unable to view the frequency spectrum in such plots by the traditional Fourier transform, because the frequency is highly and irregularly time dependent. In order to overcome this problem, the authors applied a wavelet analysis to decompose the observed x(t). (Similar procedures were simultaneously being developed at the time for chaotic phenomena in other fields [50].) The coefficients, C0.k, were chosen to mimic the sequence of data points obtained from the integration at multiples of the time interval, C0,k -- x(k~) Typically, the time scale was subdivided into 4096 units. The values of C0.k were considered to be coefficients of the basis set, q~j.k, such that the observable, at its finest level of detail, j - 0, could be interpreted as x(t) - ~

C0.k%,k(t) k

Various levels of approximation to this function can also be written. The fine approximations, Xj_l(t), can be written in terms of fuzzier approximations, xj(t)" xj_l ( t ) - x j ( t ) + Dj(t) - Z k

Cj.kq~j.k + Z

djkq/J -k

k

The difference in information between two successive degrees of approximation is the detail, Dj(t) at that level of resolution. Each cruder degree of detail has half of the data points of the finer detail. These terms could be determined using the recursion relations cj+l.k -- ~

hn-2kCj.n n

dj+l,k -- Z

gn-2kCj'n

n

and the properties of the scaling and wavelet basis sets, q~j,k(t) -- 2-J/2q~(2-Jt- k) %,k(t) -- 2-J/2qt(z-Jt- k)

277 where the scaling and wavelet mother functions as well as the high- and low-pass filters, h and g, were obtained from the fairly smooth 8-level expressions and values of Daubechies [51]. The authors focused on the details, Dj(t), in order to probe the chaotic behaviour. In principle, 12 details could be accessed (212= 4096); but in practice, D3, D4 and D5 showed all the interesting results. (Successive details had half the number of data points of the more resolved ones. Thus zeroes were added at the ends of the data sets to maintain the total of 4096 points.) In general, Dj increased in magnitude with increasing j; but only one choice of j was ideal for detecting the forcing frequency or its nonlinear effects on the oscillator. The real novelty of the studies, though, was to demonstrate the power of wavelet analysis as a pre-processing transformation for detecting buried information. Instead of using a single forcing frequency, the authors replaced the cosine function by three cosines, each with the same tiny amplitude but with different frequencies. The Fast Fourier Transform of x(t), even when amplified, could not reveal the presence of three separate frequencies, because their effect was buried in the relatively monstrous oscillation of the diatomic molecule. Although the appropriate details, D3 and D4, could successfully filter out the molecular oscillation, it was not obvious how many separate forcing frequencies were present. However, a combined FFT of the WT revealed the three frequencies most dramatically. Nine separate forcing terms could also be detected; however in this case, additional detail was observable, presumably due to bifurcations. In yet another interesting feature, the authors showed that by deleting those (few) parts of the Dj's which had unusually large amplitudes, the F F T - W T was clarified even more, without affecting the reconstruction of x(t). Subsequent to Permann and Hamilton's studies, Askar et al. applied wavelet analysis to two other physical problems: (a) deterministic motion of a threedimensional polymer; and (b) random motion of a one-dimensional chain [52]. In both cases the motion is ruled by differential equations which are numerically solved. The results are superimposed in case (a) by high-frequency deterministic components, and in case (b) additionally by noise. In the example of the motion of a 32-atom polymer of masses, m, subject to Newton's laws of motion, F(q) - m~2q/& 2 - - ~ V / ~ q

278 where the initial positions and velocities are fixed, the potential energy of interaction is given by

V - Z

Kij (bij - b~) 2 -+- Z

-t- Z

KijBk(0ijk -- 0i~k)2

KijTkl [1 - cos 3((~ijk 1 -- (~i~kl)]

q is the generalized set of 6 centre-of-mass co-ordinates, 29 torsional angles, qbiju, 30 bending angles, 0ijk, and 31 interatomic bond separations, bij. Superscripted co-ordinates denote equilibrium values. They, as well as the force constants, Kij, Kijk, and Kijkl, are also fixed. Starting with an initial helical conformation and a large impulsive velocity in one of the modes, the equations of motion were integrated, and all 96 co-ordinates were computed at fixed time intervals, kAt, where k - 1,2 . . . . . until the polymer rolled into a ball-like structure. The strategy was to represent the observable positions or angles, f(t), in terms of wavelet functions, as f(t) - Z

Z

j

dj'kl~/J.k(t)

k

(The authors decomposed the results, iterating up to k = 7 and j = 3.) At the finest resolution it was assumed that

fo(t) - ~

co,kq:)o,k(t)

k where the scaling coefficients, C0.k, were chosen as C0.k -- f(kAt), k - 1,2,... Using this as a starting point the authors evaluated the other scaling and wavelet coefficients from the recursion formulae

Cj+l,k -- Z

hn-2kCj.n n

dj+l,k -- Z

n

gn-2kCj'n

with the low- and high-pass filters, g and h, derived from the Daubechies wavelets using 8 coefficients [51], and with the basis sets given by q~j,k(t) -- 2-J/2q:)(z-Jt- k) ~j,k(t) -- 2 - J / 2 ~ ( z - J t - k) where the scaling and wavelet functions are also given by Daubechies [50]. The authors also made use of the wavelet transform for time-frequency analysis, as in Permann's work above. Noting that the wavelet,

279 ~j,k(t) ~ ( 2 - J t - k), is centred about t = 2Jk, they could focus on greater or lesser detail. The higher the value of j, the lower the frequency that the detailed wavelet coefficient could address. The multiple vibrations of bonds and gyrations of groups of atoms in space resulted in observables which varied in a chaotic manner. The torsional motions of two particular dihedral angles were e x a m i n e d - one for its small amplitude/low frequency plus large amplitude/high frequency character and the other for its large amplitude/low frequency plus small amplitude/high frequency character. The scaling coefficients, Cj,k, revealed the motions at lower and lower resolution as j increased; whereas the complementary detailed wavelet coefficients, dj.k, revealed the time history of the superimposed oscillations at poorer and poorer resolution as j increased, thus effectively demonstrating the filtering nature of the procedure. An interesting summary of the variability over all time steps was defined by the authors as E]dl.k . This quantity can identify those coordinates and their neighbours which are most "active".

12

This technique holds promise for addressing the holy grail of molecular dynamics calculations - the simulation of protein folding over long time scales- by using wavelets to perform the dynamical calculation itself without having necessarily to focus on fine-scale details. The same approach was taken for the second case of a polymer chain containing 16 atoms driven by random forces. To simplify matters, the bond lengths and angles were fixed. Only torsion was permitted, governed by a double well potential, V ( ~ ) - 7 ( 1 - ~2)2, where ~ represented the relative distances between nearest neighbouring atoms. In this case damping and stress terms were also included, as well as random forces which depended on the temperature, similar to the Langevin description of Brownian motion. A sudden heating at a predetermined time was found to cause the polymer to contract. The atomic motions were all rather equally random; however, the sudden change in randomness at the time of heating was easily detectable, not only among the scaling coefficients, but also among the wavelet coefficients, even at low resolution. In contrast, there were no features in the traditional Fourier transform which one could recognize as a detection of the sudden change in atomic behaviour.

3.2 Chemical kinetics Chemical kinetics may be considered to be the macroscopic version of chemical dynamics. Dynamics is concerned with determining the details of

280 an elementary chemical interaction; whereas kinetics averages out those details on a vastly longer time scale. From an a priori point of view kinetics makes use of information deduced by dynamical c a l c u l a t i o n s - the properties of molecular products averaged over time, over initial orientations of molecular reagents, as well as over their translational, rotational and vibrational energies - interpreted in terms of rate constants, k [53]. The rate of the elementary chemical reaction, A + BC--, AB + C, is thus expressed simply as

diAl

R a t e = d---~ = -k[A][BC] where [A] and [B] are the concentrations of species A and B respectively at time t. A complex chemical reaction is composed of several elementary steps, and they each contribute such terms to the overall rate of change of the concentration of each chemical species involved. Rate equations (treated as differential equations) are thus used in a phenomenological sense to describe observed time profiles of concentrations of reagents, intermediates and products. As long as the chemical mechanisms are not too complex, and as long as the chemical system is not too far from equilibrium, the mathematical problem is essentially linear, and solutions of sets of differential rate equations result in smooth variations of concentrations with time. Experimentalists have tended to expect this behaviour. Experimental signals could, at most, be processed for information content buried within noise. However, in recent years, it has been recognized that real chemical reactions are not so ideal. The oscillatory behaviour of the hydrogen/oxygen reaction has been known since 1936. Chaotic behaviour has been demonstrated for this reaction in 1988 [54], but has been well known for other reactions for a longer time [55-58]. The essential requirements for all chaotic behaviour is a nonlinear deterministic governing set of differential equations along with initial conditions which are far from equilibrium. In this sense, non-linear chemical kinetics can be considered to be another example of non-linear dynamics [49]. The essential observable features of chaotic systems is (a) a large frequency spectrum, (b) extreme sensitivity to initial conditions, and (c) extreme sensitivity to choice of parameters. Thus there could be a region of parameter space where chaos or oscillations can occur between two well-behaved regions [59]. Such situations can occur in such varied chemically driven areas as industrial processes or physiological reactions such as heart attacks [60]. It therefore becomes essential to know how to detect the onset of chaos. Traditional methods of analysis are very valuable [61]. However, because of nonlocal variability or because of small amplitude effects, techniques such as

281 Fourier analysis are not as useful. This is where wavelet analysis can make an impact. It appears that only one application of wavelet analysis to the study of chemical chaos has a p p e a r e d - that of Permann and Teitelbaum [62]. In an experimental study they found that CC13F, an erstwhile popular refrigerant, progressed homogeneously towards condensation via a nonlinear process. Electronic signals were generated by laser refractometry of a shock-compressed slug of gas passing by a stationary laser beam. The chemical reaction's history was spread out spatially, and could thus be probed by time-resolved refractometry. Generated signals were proportional to the rate of reaction (in this case production of dimers and trimers, etc. on the microsecond to millisecond time scale). Regular, oscillatory, and chaotic regimes were observed. It was desireable to determine the frequency spectrum of the chaotic signals. Fourier analysis proved to be frustrating because the transience of the signal introduced large uninteresting components (even when the signal is DC-shifted to decrease the amplitude of zero-frequency components). However, wavelet transformation proved to be very helpful. Digital signals were treated as a time series of 32,768 data points. Only every 8th point was retained and the 4096 remaining points were processed as described above for references [47,48]. Two observations were noted. First, decomposition of a millivolt signal into wavelet components allowed the researchers to modestly edit some of the lower numbered details (containing high frequency noise) and thus to reconstruct the signal with a signal-to-noise ratio improved by a factor of 10. This improvement revealed an underlying oscillation, not evident in the original signal. Second, the researchers could perform a Fast Fourier Transform of the wavelet details themselves in order to clearly reveal a crisp frequency spectrum, which in their case consisted of two frequencies, 62.5 and 94.2 kHz and their harmonics extending up to 1.3 MHz. It was also noted that a wavelet analysis of the original signal where every second point was retained, rather than every eighth point, was less effective in detecting all of the frequencies. Thus preliminary "smoothing" appears to be advantageous. With due care one can therefore extract the forcing frequencies which are characteristics of incipient chaos and determine if bifurcation is present. In addition, the Wavelet-Fast-Fourier-Transform technique essentially performs for an unrepeatable single-shot experiment the same task as a box-car averager does for repeatable experimental signals. The de-noising feature can also be applied to traditional linear kinetics. This was demonstrated by Fang and Chen for voltammetry [63]. Despite its use

282 in analytical chemistry, voltammetry is a technique which determines the current response to a sudden change in voltage and is designed, in principle, to determine the rate of electron transport in an electrochemical system. It is therefore representative of other techniques in kinetics which generate signals suffering from a finite level of white noise. Filtering by selecting an arbitrary frequency-cutoff after Fourier analysis is somewhat risky, in general. In addition, there are always the edge effects which introduce spurious Fourier components. In response, Fang and Cheng used Walczak's Wavelet Packet Transform procedure (for recognizing patterns in IR spectra by multi-resolution analysis) [64], followed by the application of an adaptive wavelet filter. Edge effects for a differential pulse polarogram could be minimized or eliminated by baseline subtraction prior to applying the wavelet transform. The procedure for low-pass filtering is to reconstruct signals from only those details which pass a test based on the magnitude of the power spectrum of the signal, subject to a minimization criterion. Although the procedure is more objective than that adopted by Permann, above, [62] there is still subjectivity in choosing the level of the pass criterion. The authors reported an improvement in signal-to-noise ratio of a factor of 6 with little distortion.

3.3 Fractal structures

Fractals are closely allied to chaos [65-69]. Instead of temporal stability/ instability, it is spatial regularity/irregularity which is involved. Although formally, fractal behaviour is not strictly a phenomenon of kinetics, it does appear quite often in materials science, and since it is also a phenomenon generated by nonlinear deterministic rules, as in the case of nonlinear kinetics, it is considered here in this section. Two applications of wavelet transforms to fractals have appeared. As will be seen below, they demonstrate the power of wavelet analysis for revealing the underlying deterministic rules (cf. rate equations) and for studying time-resolved chemical kinetics. Multifractals are characterized by a spectrum, f(cx), of singularities of magnitude, cx. This spectrum is often a narrow function of 0~ centred at cx- Do, where Do is termed the fractal dimension. When the fractal structure is globally self-similar, ie sequential magnifications of the structure are identical to the original structure, then f(cx) is sharply peaked with a unique value of a fractal dimension. Otherwise, there is a spectrum of dimensions, Dn (which decrease with increasing n). This number is essentially a summary of the effective number of dimensions needed to describe

283 the object's complexity; but it does not reveal anything about the spatial location of its singularities. Fourier analysis is limited, as usual, by its inability to localize frequency information. Thus wavelet analyses have been developed for analysing one-dimensional [70] and two-dimensional fractal images [71,72] and the analyses have been tested on several models: ThueMorse chains, period-doubling chains, snowflakes and diffusion-limited aggregates [73]. In the one-dimensional case Liviotti studied the formation of chains of "atoms" obeying simple deterministic rules: For the Thue-Morse (TM) chain, element A is replaced by the combination AB, while B is replaced by the combination BA. Starting with A, a line of elements is generated which doubles in length at every generation. The order of elements rapidly becomes chaotic. For the period doubling (PD) chain, the rules are: A is replaced by AB, while B is replaced by AA. Similar chaos results here. At least the TM chain could, for example, form the basis for a model description of polymer growth. A and B have lengths a and b, respectively. The author then defines a structure factor, Sh(q), -) 1

SN(q)

9

e~qXk

N k=0

where Xk is the position of the k-th element, and N is the number of elements. Long-range order appears as N increases, and SN displays sharp peaks whose magnitudes are proportional to N r, where 7 is related to the spectrum of fractal dimensions according to at = 2 ( 1 - 7 ) . To determine the spatial dependence of at the authors applied a "Mexican hat" wavelet function, ~(q), to SN:

h-~lim-slJ DC

T(s,u)-

ql (q -S yu)( q ) d q s

--O(2

The mathematical properties of fractals result in the following scaling relation T(2s,u) ~ Za-lT(s,u) which is satisfied by the relation T ( s , u ) = s~-1. Consequently a plot of In IT(s,u)] vs In s gives a straight line of slope (,15 10. 10

>'30[

30

20 30 y2 obsewed

40 y3 observed

50

20.

S

"~15 .~. .u_ ~10

0

2O

10 y4 observed

0

10 20 y5 obsewed

30

Fig. 4 The Yp,'edicted v e r s u s Y,h.......,.,,,1plots (/br the test set) according to the S M L R models.

The results reflect typical situations. While applying SMLR automatically, we can construct models with good predictive ability on condition that there are enough calibration samples, the spectra are not noisy and that no extrapolation outside the calibration domain is required. Otherwise less good results are obtained. In extreme case, as for y l, there is no possibility of constructing the SMLR model. The results of SMLR ought to be always carefully analysed and interpreted. Genetic Algorithm applied to the discussed data set leads to different subsets of selected variables. There are many different versions of GA, depending on the way reproduction, cross-over, etc., are performed. The algorithm used in our study, adapted from Leardi et al. [34,35], is particularly directed towards feature selection. In each GA run, a few subsets with similar responses are selected. Final solutions are then evaluated based on the RMSEP of an independent test set. Results are presented in Table 2 and in Figs. 5 and 6.

Table 2. RMSCV, RMSEP and the numbers of selected variables for modelling yl, y2, y3, y4 and y5 by GA-MLR.

yl

y2 y3 y4 yfi

RMSCV

RMSEP

S e l e c t e d variables

0.3245 0.05834 0.0596 0.04138 0.2563

1.208 0.1505 0.09227 0.05768 0.5951

32, 44, 56, l l5, l l6, 129, 147, 172, 232 61, 84, 92, 107, 116, 127, 140, 193, 197, 235 46, 70, 134, 169, 180 48, 83, 116, 139, 174, 181, 183, 190, 199, 235 6, 26, 39, 70, 114, 168, 188, 189, 242

337

yl

t " 0

100 200 vanable number y3

y2

0

I

100 200 wanablenumber

1:t 2

0

.

y4

.

1O0 200 ,,,an~le number y5

2

1O0 200 variable number

0

1O0 200 vanabte number

Fig. 5 Variables selected by GA .[or modelling y l , ),2, j'3, y4 and yS.

e~

20

>'10 ~

10

.

,

20 30 yl observed

!::J

~25~ 3O

20 c..,,i >,15 10. 10

~~~

. . 20 30 y2 observed

9

3O

.

40 50 y3 observed

30 "

2~ ~5

2005I"

O h

0

~ ' /

,

10 y4 observed

20

20

0

,

.

10 20 y5 observed

.

30

Fig. 6 The Yp,-eclicteclversus Y,,h.......,.,.,1plots ([or test set) according to the G A - M L R models.

338

Only for yl the predictive ability of the G A - M L R model is not satisfactory. For y2, y3, y4 and yS, the prediction of the models are excellent, but as one can notice, few of the selected variables are on the data baseline, which suggests that the models can prove unstable. In fact, this is a case with these models. If to the independent test set a small noise is added (simulated as randn(mt,nt)*0.001), the constructed models failed completely. Plots of Ypredicted v e r s u s Yobserved for the noisy test set, denoted as Xtn, are presented in Fig. 7. These results demonstrate the danger of working with few variables, which can, however, be overcome by applying the full-spectra models, or by applying GA to the compressed data set, containing the significant wavelet coefficients only. Results of PLS, i.e. of the full spectrum method, are presented in Table 3. The RMSCV and RMSEP values are much higher than the analogous values observed for the SMLR or G A - M L R models, but one can hope that the PLS models are more stable e.g. when instrumental problems occur. Still, one can try to lower model complexity by extracting relevant information from the original spectra. This can be done, for instance, by using the UVE-PLS or

8~

60

9

.

,

10

.

20 30 y1 observed

3O

4~I

2~

2~.j' 20

>.15

10

10

-

>'301

.

20 30 y2 observed

30

3O

~15

40

y3 observed ....

2o

-/ OL 0

0 10 y4 observed

20

0

10 20 y5 observed

30

Fig. 7 The Yp,'edicted v e r s u s Y(,h.......,,,,a f o r tlle test set contamined with white noise, Xtn, according to the G A - M L R models.

339

Table 3. RMSCV, RMSEP, number of latent variables (f) for the PLS and U V E - P L S models.

yl y2 y3 y4 y5

PLS f 7 6 6 6 6

RMSCV 0.5533 0.2559 0.1382 0.2581 0.5274

UVE-PLS f RMSCV 5 0.9290 5 0.5091 6 0.1320 4 0.5821 5 0.7745

RMSEP 1.0006 0.3393 0.2121 0.1409 0.9899

RMSEP 0.6406 0.3479 0.2145 0.3418 0.7740

RCE-PLS approaches. Results of PLS and UVE-PLS are presented in Table 3. As one can notice, elimination of uninformative variables leads in the majority of cases (i.e. in 4 out of 5) to the models with lower complexity. RMSEP for yl and y5 is decreased, for y2 and y3 is similar to the RMSEP of PLS models, and for y4 it is higher. The stability of regression coefficients and selected variables, are presented in Fig. 8. (a)

15

20

TI."_-m~l;:r_T ~

0

-20

0 200 400 600 real vanables random vanables ( b ) 20 ~

0 _='g .IQ

100 200 selected variables

0

,,

"

.

,

.

15

fii,ll~ ~ , - ~ 2 L ' ~ -,r~,~,,,l~, ...........

0

~ -20 i

-4O

0

0 200 400 6()0 real variables random~riables

(e)100 - 501 z . --

0 200 400 660 realvadebles random venables

0

1O0 200 coefficient number

Fig. 10 The variance vector for the NIR data set decomposed b)' D W T with the Daubechies [iher no. 4.

For the studied data set, 128 wavelet coefficients were identified as significant, and the remaining 128 (256- 128) as insignificant. If the data matrix W is compressed to the matrix Ws (30 x 128) then the PLS models are the same as those constructed for the original data, which shows that the coefficients removed from the matrix W are uninformative. In this case the only advantage of wavelet decomposition is that the data set was compressed [36]. Another possibility is to keep insignificant coefficients Wn and perform UVElike type of modelling, using these coefficients as irrelevant features to calculate the cut-off value of stability of the regression coefficients, associated with significant features. This type of modelling is called Relevant Component Extraction-PLS (RCE-PLS), in order to distinguish it from the U V E PLS approach. Results of RCE-PLS applied to the NIR data set are summarized in Table 4. The complexity of the RCE-PLS models is always lower than the complexity of the PLS models, whereas RMSEP is lower for yl, y3 and yS, and higher for y2 and y4. The Y p r e d i c t e d versus Yobserved plots for the test set according to the PLS, UVE-PLS and RCE-PLS models are presented in Fig. 11.

Table 4. Complexity (f), RMSCV and RMSEP for the RCE-PLS models.

yl y2 y3 y4 y5

RCE-PLS f 4 5 4 4 5

RMSCV 0.9681 0.3932 0.3319 0.5115 0.6424

RMSEP 0.5832 0.4714 0.1952 0.2301 0.7637

342

V1

PLS

,,

UVE.-PLS

~20[

~20>,~.

~20~.>,

10

10

10[ 1"0

2'0 3"0 y observed

v2

1"0

PLS .

30

30

"=

20 3'0 y observed

10

UVE-PLS

~ 25 15

~~

2o

y observed

u

3o 1~

PLS

>'15

2o

y observed

3o

10.

lo

2'o

y observed

UVE-PLS

50

3'o

RCE-PLS 50

,10

40

N 30

2~

y observed

p y4

I 5o 2%

PLS

2O ~15~

2'0 3'0 y observed RCE-PLS

~

I >'151

~

RCE-PLS

2O ,

~ao so

4o

y observed

ab

so

UVIE-PLS

20 T

4'o

y observed

RCE-PLS

i 10

-~

y observed

20

O~

PLS

3O

30-

10 y observed

0

UVE-PLS

10 y observed RCE-PLS

30 "

N 20 ._. ~-10

20

Oo lb 2b 20 y observed

so

o

lb

2"o

y observed

30

o

~b

2'0

y observed

Fig. 11 The Ypredicted versus Yoh......,,,,,t plots Jor the test sets according to the P L S , U V E P L S and R C E - P L S models.

343

While constructing the RCE-PLS models, we do not need to reconstruct the spectra at any step of the calibration procedure, but they can be reconstructed for visualization purposes. In Fig. 12, the original spectra (centered), spectra reconstructed with the relevant coefficients and spectra reconstructed with the irrelevant coefficients, together with the corresponding std spectra,

yl)

(a)

J~

~L -0.2[

1()0

0

(b)

"W 200

02[

(d) 0.1 0.05 0 0 0.1

-o~[;

"-- -'~[ "''1 260

0

lOO

(c)

-

,'m

0

1130

200

.......

0

100

200

(f) 0.1

-

100

(e)

. . . . . .

o

200

0

100

200

y2)

(a)

(d)

o.o: _>4 o.1

o

(b)

11~0

~L _o~F __ , 0

100

200

0

:

-.'~

_~k

'

200

_0.2[" o

"

_

~

~6o

100

200

100

200

0.05 db

L0 _=

200

01

(c) 0 2 '

100

(e)

_ A.~..

r

:,60

0 0

o.I

0.05 0 0

Fig. 12 (a) Original spectra (centered), (b) the relevant and (c) irrelevant components o/ the spectra and (d)-(e) the respective standard deviation (std) spectra jor modelling yl, y2, y3, 3'4 and yS.

344 y3)

(a)

02[_ _/,,l _o.Tr - 1130 l"'l 0 200

_o.f 0

100

y4)

~ 100

1

ii

~

_~1 100

200

1O0

200

0

100

200

0

100

200

(f)

(d) 0.1

" ~ 200

1130

-

0.05 0 0

200

~

0

0.05 0 "--'-"-"-~. 0 1O0

200

.....

-0.21. 0

(b)

(e)

C);

0

(a)

o.I

b)

(b)

(c)

(d)

(e)

o.~ 0

200

100

200

(r)

(c) 0 . 2 [

L

.o2r 0

0

"

-

~

- .......

-

100

-~'~

200

0.05 O~ ~ ' - "

0

,'7""~',

100

200

"

Fig. 12 (Contd.)

are presented. These figures well illustrate the difference between the UVEPLS and the RCE-PLS approaches. In UVE-PLS, variables are selected from the set of original variables, whereas selecting relevant features in the wavelet domain results in different weighting of the original variables. If the PLS, UVE-PLS or RCE-PLS models are applied to the test set slightly contamined with white noise (data set Xtn), they allow an acceptable prediction, thus giving evidence of their stability. RMSEP for Xtn, observed for

345 yS)

(a)

o.2[ 0

(b)

200

~I _o. l 0

(c)

100

"W

0

100

-0.2[ 0

|_

0

I00

200

0 0

I00

200

(e) 0.05

.

o.2[

(d) 0.1

200

__

16o

AL~

26o

0 . 1 ~ 0.05 0 0

I00

200

Fig. 12 (Contd.)

the G A - M L R , PLS, UVE-PLS and RCE-PLS models are summarized in Table 5, whereas the Ypredictedversus Yobserved plots are presented in Fig. 13. For data highly contaminated with noise also the difference between UVEPLS and RCE-PLS approaches becomes more evident. For illustrative purposes, in Fig. 14 the spectra (centered) of test set contaminated with high noise (simulated as randn*0.01, i.e. ten times higher, than for Xtn), the relevant and irrelevant components, extracted by RCE-PLS for modelling y4, are presented. As one can easily notice, the majority of noisy variables are properly identified and removed from the original noisy spectra.

Table 5. R M S E P for the test set contamined with white noise (Xtn) for the G A - M L R , PLS, U V E - P L S and R C E - P L S models.

yl y2 y3 y4 y5

GA-MLR 31.4029 2.1708 1.0911 0.9236 8.4285

PLS 0.8924 0.3428 0.2252 0.1639 0.9500

UVE-PLS 0.7401 0.3562 0.2211 0.4148 0.7581

RCE-PLS 0.4434 0.4898 0.2156 0.2556 0.5869

346 GA

PLS

30 ~ 20

(I)

-"920

"o

0

10 20 y5 observed

0

30

10 20 y5 observed

UVE-PLS

30

RCE-PLS

3O

.

~ 2o 0~

10 20 y5 observed

o

30

0

10 20 y5 observed

30

Fig. 13 The Ypredicted versus Yob. . . . . . . .,,,t plots Jor the test set contamined ~rith noise, Xtn, according to the G A - M L R , PLS, U V E - P L S and R C E - P L S models.

(a)

(d)

0.2 0 -0.2 0

0.05 100

200

(b)

(e)

0 0

100

20O

0 . 1 ~ 0.05 0

100

200

(c) 0.2~

o

....

"

~bo

260

(r)

0 0

0.1

100 .

0

100

200 .

.

.

200

Fig. 14 (a) The spectra (centered) of the test set contamined with noise, (b) the relevant and (c) the irrelevant components extracted by R C E - P L S for modelling y4.

347

5

Conclusions

To construct parsimonious multivariate models for highly correlated spectral data, one can extract all relevant information, present in data, and eliminate the irrelevant one. This can efficiently be done in the wavelet domain, where it is easy to distinguish between significant features and features associated with noise. The latter variables can be further used for discrimination of relevant and irrelevant features for data modelling. This approach usually leads to the decrease of model complexity and to increase of its stability.

References 1. B.G. Osborne, T. Fearn, P.H. Hindle, Practical NIR Spectroscop)" ~rith applications in Food and Beverage Analysis, Longman Group UK Limited, England, (1993). 2. N.R. Draper, H. Smith, Applied Regression Analysis, Wiley, New York, (1981). 3. H. Martens, T. Naes, Multivariate Calibration, Wiley, New York, (1991). 4. D. Jouan-Rimbaud, D. Massart, R. Leardi, O.E. de Noord, Genetic Algorithm as a tool for wavelength selection in multivariate calibration, Analytical Chemistry, 67 (1995), 4295-4301. 5. U. Horchner, J.H. Kalivas, Further Investigation on a Comparative Study of Simulated Annealing and Genetic Algorithm for Wavelength Selection, Anah'tical Chimica Acta, 311 (1995), 1-13. 6. M.J. Arcos, M.C. Ortiz, B. Villahoz, L.A. Sarabia, Genetic-Algorithm-Based Wavelength Selection in Multicomponent Spectrometric Determinations by PLS" Application on Indomethacin and Acemethacin Mixture, Analytical Chimica Acta, 339 (1997), 63-77. 7. G. Weyer, S.D. Brown, Application of New Variable Selection Techniques to Near Infrared Spectroscopy, Journal of Near Infrared Spectroscopy, 4 (1996), 163-174. 8. J.H. Kalivas (Ed.), Adaption q[ Simulated Annealing to Chemical Optimization Problems, Elsevier, Amsterdam, in press. 9. J.P. Brown, Measurement, Regression and Calibration, Clarendon Press, Oxford, (1993). 10. P. Salamin, H. Bartels, P. Forster, A Wavelength and Optimal Path Length Selection Procedure for Spectroscopic Multicomponent Analysis, Chemometrics and Intelligent Laboratory Systems, 11 ( 1991 ), 57-62. 11. A.J. Miller, Subset Selection in Regression, Chapman & Hall, New York, (1990). 12. J.G. Topliss, R.P. Edwards, Chance factors in Studies of Quantitative StructureActivity Relationships, Journal of Medical Chemistry, 22 (1979), 1238-1244. 13. S. Derksen, H.J. Keselma, Backward, Forward and Step Wise Automated Subset Selection Algorithms; Frequency of Obtaining Authentic and Noise Variables, British Journal of Mathematical and Statistical Psychoh~g)', 45 (1992), 265-282. 14. D.E. Goldberg, Genetic Algorithm in Search Optimisation and Machine Learning, Addison-Wesley, Reading, MA, (1989).

348 15. P.J.M. van Laarhoven, E.H.L. Aarts, Simulated Annealing: Theory and Applications, Reidel, Dordrecht, (1987). 16. D. Jouan-Rimbaud, D.L. Massart, O.E. de Noord, Random Correlation in Variable Selection for Multivariate Calibration with a Genetic Algorithm, Chemometrics and Intelligent Laboratory Systems, 35 (1996), 213-220. 17. I. Frank, Intermediate Least Squares Regression Method, Chemometrics and Intelligent Laboratory Systems, 1 (1987), 233-242. 18. A. Hoskuldsson, The H-principle on Modelling with Applications to Chemometrics, Chemometrics and Intelligent Laboratory Systems, 14 (1992), 139-153. 19. F. Lindgren, P. Geladi, S. Ranner, S. Wold, Journal of Chemometrics, 8 (1994), 349363. 20. F. Lindgren, P. Geladi, S. Ranner, S. Wold, Journal of Chemometrics, 9 (1995), 331342. 21. R. Wehrens, W.E. van der Linden, Bootstarping Principal Component Regression Models, Journal of Chemometrics, 11 (1997), 157-171. 22. F. Navarro-Villoslada, L.V. Perez-Arribas, M.E. Leon-Gonzalez, L.M. Polo-Diez, Selection of Calibration Mixtures and Wavelengths for Different Multivariate Calibration Methods, Analytical Chimica Acta, 313 (1995), 93-101. 23. C.H. Spiegelman, M.J. McShane, M.J. Goetz, M. Motamedi, Qin lci Yue, G.L. Cote, Theoretical Justification of Wavelength Selection in PLS Calibration" Development of a New Algorithm, Analytical Chemistry, 70 (1998), 35-44. 24. M. Forina, C. Casolino, C. Pizarro Millan, Iterative Predictor Weighting (IPW) PLS: A Technique for the Elimination of Useless Predictors in Regression Problems, Journal of Chemometrics, 13 (1999), 165-184. 25. M. Clark, R.D. Cramer II, The Probability of Chance Correlation Using Partial Least Squares (PLS), Quantum Struct-Acta Relat, 12 (1993), 137-145. 26. V. Centner, D.L. Massart, O.E. de Noord, S. de Jong, B.M. Vandeginste, C. Sterna, Elimination of Uninformative Variables for Multivariate Calibration, Analytical Chemistry, 68 (1996), 3851. 27. D. Jouan-Rimbaud, B. Walczak, R. Popi, O.E. de Noord, D.L. Massart, Application of wavelet transform to extract the relevant component from spectral data for multivariate calibration, Analytical Chemistry, 69 (1997), 4317-4323. 28. J. Rissanen, A Universal Prior for Integers and Estimation by Minimum Description Length, Analytical Statistics, 11 (1983), 416-431. 29. N. Saito, Simultaneous Noise Suppression and Signal Compression Using a Library of Orthonormal Bases and the Minimum Description Length Criterion, Wavelets in Geophysics, (eds. E. Foufoula-Georgiou and P.Kumar), Academic Press, New York, (1994). 30. B. Walczak, D.L. Massart, Noise Suppression and Signal Compression Using Wavelet Packet Transform, Chemometrics and hltelligent Laboratory Systems, 36 (1997), 81-94. 31. B.M. Wise, PLS Toolbox for Use with Matlab, version 1.4 (Eigenvector Technologies, West Richland, WA, USA). 32. R.W. Kennard, L.A. Stone, Computer Aided Design of Experiments, Technometrics, 11 (1969), 137-148.

349

33. H. van der Voet, Comparing the Predictive Accuracy of Models Using a Simple Randomization Test, Chemometrics and hTtelligent Laboratory Systems, 25 (1994), 313-323. 34. R. Leardi, R. Boggia, M. Terrile, Genetic Algorithms as a Strategy for Feature Selection, Journal of Chemometrics, 6 (1992), 267-281. 35. R. Leardi, Application of a Genetic Algorithm to Feature Selection Under Full Validation Conditions and to Outlier Detection, Journal o/" Chemometrics, 8 (1994), 65-79. 36. J. Trygg, S. Wold, PLS Regression on Wavelet Compressed NIR Spectra, Chemometrics and Intelligent Laboratory Systems, 42 (1998), 209-220.



351

C H A P T E R 16 Wavelets in Parsimonious Functional Data Analysis Models Bjorn K. Alsberg Department of Computer Science, University of Wales, Aberystwyth, Ceredigion SY23 3DB, UK e-mail." bka@ aber.ac.uk

I

Introduction

Occam's (or Ockham's) razor is a principle attributed to the 14th-century logician William of Occam which can be stated as follows" "Entities should not be multiplied unnecessarily". It is a principle commonly accepted as a sound working principle for construction of scientific knowledge. It means that if several possible hypotheses can explain an observed fact, then the one is chosen that has the minimum number of assumptions attached to it. Such hypotheses or models are often referred to as parsimonious. These models are often associated with the following properties: 1. 2. 3. 4.

Improved prediction. More general. Easier to understand. Few variables/parameters.

Reducing the model complexity often reduces the prediction error of a model [1], however this is not always true. It might be acceptable to sacrifice some of the prediction ability in favour of a less complex model. Historically, an example can be found from astronomy where the Ptolemian geocentric cosmology was replaced by the helio-centric Solar system. The first model suggested by Copernicus was based on circular planetary orbits which actually had higher prediction error compared to the Ptolemaic model. In spite of this the main idea of heliocentricity was eventually preferred (togethe~ with elliptic orbits) because of its simplicity, generality and explanator) power.

352 Improvement of understanding and more generality can often be attributed to the higher abstraction level of the model representation. For instance, a reaction coordinate can be seen to exist on a higher abstraction level than using all the coordinates for the different atoms involved in a reaction. The abstraction level indicates the degree of detail needed for the model representation. There is an inverse relationship between abstraction and the resolution of the representational detail. How can parsimonious models be constructed? There are several possible approaches, however in this chapter a combination of data compression and variable selection will be used. Data compression achieves parsimony through the reduction of the redundancy in the data representation. However, compression without involving information about the dependent variables will not be optimal. It is therefore suggested that variable selection should be performed on the compressed variables and not on the original variables which is the usual strategy. Variable selection has been applied with success in fields such as analytical chemistry [1-4], quantitative structureactivity relationships (QSAR) [5-8] and analytical biotechnology [9-1 1]. In this chapter, compression is achieved by assuming that the data profiles can be approximated by a linear combination of smooth basis functions. The bases used originate from the fast wavelet transform. The idea that data sets are really functions rather than discrete vectors is the main focus of functional data analysis [12-15] which forms the foundation for the generation of parsimonious models.

2

Functional data analysis

Spectra originating from infrared, Raman and ultraviolet spectroscopy are reasonably approximated by smooth functions. The degree of smoothness is defined by the continuity of the various derivatives of the function. A function f is said to be k-times continuously differentiable or C k if (ok/otk) f(t) is continuous for all points t E ~'. The traditional approach in multivariate data analysis and chemometrics is to consider the data profiles as discrete vectors where each sampled point along a spectroscopic profile is assigned a unique variable in the analysis. This is here referred to as the sampling point representation (SPR) [14]. SPR is often so simple and intuitive that it is sometimes difficult to see why any alternative representation should even be considered. One aspect of SPR is that information about

353 continuity between the different vector elements is lost. The apparent continuity is due to the fact that most people tend to organise the sampled data points to be meaningful, however this information is not contained explicitly in the representation itself. A simple experiment can demonstrate this: consider at data matrix with e.g. 100 spectra where each spectrum is described by 1000 sampled data points at different wavelengths. The aim is to use principal component analysis. The output loadings vectors from such an analysis will reflect the shapes of the input profiles. Another analysis is possible where we have randomly permutated the variable columns in the data matrix. ~ All the shapes of the spectral profiles are lost and this will also be reflected in the loadings profiles. However, mathematically the results from the two analyses are identical in the sense that it has produced the same eigenvector solution with the same eigenvalues. The only difference is a relabelling of the column (or row) vectors which do not have any effect on the convergence and solution of the PCA model. Thus, the indexing information we took for granted is not accessible to the PCA or any other multivariate method that does not take it into consideration explicitly. Functional data analysis [12,13,16] on the other hand does makes this information directly available for the multivariate methods by assuming each data object is a function rather than a discrete vector. The other aspect is that SPR is unnecessary memory demanding. The digital sampling density of spectroscopic and analytical instruments can often be adjusted by the experimenter. Both the real spectroscopic resolution and the digital sampling density will influence the actual number of variables (intensities at certain wavelengths) used to represent the spectral profile. There is a large redundancy in the data which can be attributed to the continuity of the data profile. In general, the smoother the data, the lesser number of bytes are needed to store it. By approximating the profiles as actual functions, it is possible to perform an efficient compression of the data set. Obviously, this is going to influence the complexity of the resulting calibration model. If the multivariate model needs to use additional parameters to handle this redundancy it will tend to mask the real underlying variations that are more interesting. A compression of the data profiles by a functional approximation can therefore be an efficient way to obtain a better understanding of deeper relationships.

1The same

also applies to the matrix rows.

354 2.1 From vectors to functions

This section sets the stage for some of the ideas discussed later. Going from equations with discrete vectors and matrices to equations with functions in its basic form is not difficult. Let us demonstrate these ideas by looking at an example where PCA is applied to functions. Let X be a matrix of continuous spectra. This means that the N rows in X are really functions such that X - [xl(t)T; x2(t)T'..." XN(t)T]. One way to find the principal components of X is to solve the eigenquation of the covariance matrix G = XX T. For the discrete case G can be written in terms of vector inner products Gij -- (xi Ixj)

( 1)

where xi is the ith row in X and xi is the jth row in X. For the discrete case we define the inner product between two vectors a and b as (alb)discrete -- ~

aibi

(2)

i

whereas in the case for inner product between two functions a(t) and b(t) we write f {a(t)lb(t))continuous - J a(t) b(t)dt.

(3)

The bracket notation is similar to the one used in quantum mechanics [17]. Basically, the summation signs are replaced with the corresponding integration signs in the equations for PCA (and other similar multivariate algorithms). Thus, the covariance matrix G = XX T has elements G i j - (xi(t)]xj(t))- f xi(t)xj(t)dt

(4)

G will here be an N x N matrix whereas the dual R = x T x is not a matrix, but a 2D function (also referred to as a kernel) R(u; v) - Z

xi(u)xi(v)

(5)

i

It is very common to represent the smooth functions xi(t) in a finite basis xi(t) - Z

cjOj(t) J

where ~)i(t) is the basis. Writing Eq. (4) in matrix notation we get

(6)

355

Oij - / Z Cik*k(t) Z Cj~*~(t) dt k r

- CBTBC T

(7)

where B = [~l (t); qb2(t):... ;qbK(t)]. For some bases the calculation of Uij = (~i]qbj) matrix will be easy. As we shall see below, the discrete wavelet transform from the Mallat algorithm produces an orthonormal basis which makes U equal to the identity matrix. For orthonormal bases no modification of the original multivariate algorithms is necessary and we can use the method directly on the basis of coefficients C. The conceptual relationship between function, sampled data and the coefficient space is shown in Fig. 1.

2.2 Spline basis Spline approximations of functions are a logical extension of using simple polynomials P k ( x ) - ~-]~-0 ci xi to fit a curve. It may be possible to find the coefficients ci to a kth degree polynomial that will fit in a least square sense a set of sampled points. However, these high degree polynomials are very unreliable for extrapolation and thus contain unrealistically large oscillations. Global polynomial functions are therefore poor at describing local features without using very large k. In spline theory, the idea is used that a function can be approximated by polynomials that are only valid over finite regions or segments. These seg-

Fig. 1 It Shows the basic idea behind the relationships between fimction space, sampled point space and coefficient space. All three different representations describe the shape of the observed spectrum.

356 ments are defined by points tj called knots. At the boundary between two regions the function has C k continuity. C k ~> 1 prevents the boundaries from introducing artificial sharp edges which would be detrimental to the approximation of smooth functions. To control the shape of the curve, control points or spline coefficients are used. For a uniform cubic parametric B-spline over a region i we have Qi(t) -- TMBCBs,

(8)

where T - [ ( t - t i ) 3 ( t - t i ) 2 ( t - t i ) 1 ] contains the polynomials of the parametric variable t, MB is the uniform B-spline basis matrix 1 MBsi -- ~

-1 3 --3

3 -6 0

-3 3 3

1 0 0

1

4

1

0

(9)

and CBsi is the geometry matrix which contains the control points

Pi-3,x CBsi --

Pi-3.y

Pi-2,x Pi-2,y Pi-l,x Pi,x

Pi-l,y Pi,y

Pi-3.z Pi-2.z Pi-l.z Pi,z

3 ~< i 1 then wT _

T, W T)

end if w T _ w T/(wvw)

t=

1/2

Ew

if a > 1 then t -- ~ ( t , T)

end if Calculate q and u if ( ~ i ( d t i ) z ) / t x t < c o n v then STOP = T R U E end if end while Calculate p (Storage of matrices) U p d a t e E and F end for

375 f~ is an operator that picks the k largest elements from the Iwl vector. 9 is an operator that takes one vector and a matrix and makes the vector orthogonal to the columns in the matrix. Let DPLS(X, y, k) be cross-validation VS-DPLS that selects the k largest ]wjl at each factor. The optimal selection of k (i.e. kopt) is the one with lowest PRESS value: for k = 1 to kmax do DPLSmodelk = DPLS(X. y, k) Store PRESS(k). end for Select model corresponding to min (PRESS) A note of caution should be made in connection with setting k = 1. In this case the selected regression model has full rank (i.e. the number of PLS factor is identical to the number of selected variables). The investigator should be careful not to use a model that might be unstable.

4

Regression and classification

4.1 Regression Multiscale regression or wavelet regression [60] is based on the simple idea that the mapping between the independent and dependent variables may involve different resolution levels. Most approaches to multivariate regression and classification only make use of the original data resolution in forming models. The multiscale approach enables the investigator to zoom in and out of the detail structures in the data. Let us now consider regression in general in terms of a matrix formulation of the fast wavelet transform.

The F W T basis matrix. The fast wavelet transform (FWT) can be formulated in terms of matrix algebra by storing each of the wavelet functions in the time/wavelength domain in a matrix B. This matrix contains all the translations and dilations of the wavelet necessary to perform a full transform. One common way to organise this matrix is to sort the sets of shifted basis

376 functions according to their scale. This means that we present all the basis functions that are shifted but have the same scale followed by the next higher (or lower) scale's shifted basis functions. This organisation is not chosen arbitrarily but is closely related to how Mallat's algorithm [39] for calculating the wavelet coefficients operates. The number of shifts along the x-axis depends on the value of the scale j. Assuming that the total number of elements in our data vector is M - 2J+~ the different scales are the integers from 0 to J. The shifting coefficient k has the integer values 0 to 2j-1 for each j value. The structure of the basis matrix B is as follows: -

B0 B~ .

B -

(35)

BJ-1 -

Bj

where each submatrix Bj has a diagonal dominant structure for scale j. is associated with the projection onto the lowest resolution (j=0) scaling function. Each basis matrix added is related to the direct sum between the corresponding detail spaces Wj mentioned earlier in the book. The largest submatrices correspond to the shortest scales (dominated by high-frequency components). B is orthonormal and the wavelet transform can be written as z - Bx

(36)

where z is the vector of wavelet coefficients and x is the vector containing the input data profile. Reconstruction is trivial x--BTz

(37)

In a typical chemical regression problem the X matrix contains the N spectra (as rows) and M wavelengths (as columns) where y is column vector of a component concentration. Assuming that Beers law holds we have that y-Xb

(38)

where b is the linear regression coefficient. To estimate b we need to find a generalised inverse such that I~- X+y

(39)

X + is the generalised inverse performed by some regression method (e.g. partial least squares regression). Inserting for X

377 X : ZB

(40)

where Z is the matrix of wavelet coefficients. Substituting this into equation 39 one gets I ~ - B + Z + y - B T Z + y - BVbw

(41)

where bw is the resulting B-coefficient regression vector from the direct analysis of the wavelet coefficient matrix Z. A simple postmultiplication of the basis matrix will convert this vector into the B-coefficient vector from the analysis of the raw data. This is of course related to the fact that the FWT basis is orthonormal. For wavelet transforms that do not satisfy this criterion the conversion back to the original domain is not equally straightforward.

4.2 Classification The idea of performing classification at different levels of resolution can be explained by using an analogy: assume we want to distinguish an elephant from a dog by analysing images taken at different distances. We know that the level of detail obtained from using a magnifying glass is not needed. In fact, it would probably be possible to distinguish these two animals from a distance of more than a kilometer (i.e. whether it is an elephant or not). At large distances we can only see the broad and overall features of these animals, but the detail would be sufficient for a classification. However, if also we want to distinguish between e.g. a wolf and a dog a higher level of detail is necessary. The concept is illustrated in Fig. 6. Looking at this figure we see that it is possible to present diagrammatically the way a discrimination with respect to spatial resolution between these objects works. By starting at the image at top left and moving to right and down, it becomes easier to resolve the image into different objects. Here we are not interested in the actual distance between the objects, but rather the actual shape. Thus, objects are grouped together that have the same size and shape. Such diagrams are here called scale dendrograms and have a similar interpretation as the classical dendrogram. The scale dendrogram efficiently summarises the separation of objects with respect to spectrum resolution. In this way it is possible to detect when certain patterns change significantly with the addition of a scale. It should be emphasised that the definition of similarity between the objects in the diagram will depend on the problem to solve.

378

Fig. 6 Classification at different levels of resolution. When the resolution possible to distinguish between the elephant and something that is as small want to discriminate between a dog and a wolf, we need higher resolution. side of the figure shows a scale dendrogram which is used to summarise properties of the classification of the objects.

is low it is still as a dog. I f we The right-hand the qualitative

Scale dendrograms can in principle be applied to both unsupervised and supervised classification, however in this chapter only examples from unsupervised classification are included.

Unsupervised classification- Cluster analysis. Unsupervised classification or cluster analysis is a way to find "natural patterns" in a data set. There are no independent "true" answers that can guide the classification and we are therefore restricted to construct a set of criteria or general rules that can highlight the "interesting" patterns in a data set. A data set consists usually of a set of objects that each are characterised by a feature vector x. To find patterns it is important to establish to what degree vectors are similar to each other. Similarity is often defined using a distance metric between object i and j Dij -- F(x(i), x (j))

(42)

where F in principle can be any function operating on the elements on the vector elements of the object vectors. Typically, F is a Minkowski distance

Dij -

Z [[Xik -- Xjk liP k-1

(43)

379 where M is the number of variables. When p = 2 this corresponds to the Euclidean metric. Such a metric for functions is written as: D ~ - <xi(t)[xj(t))

(44)

assuming xi(t) and xj(t) as functions. This metric gives each individual infinitisemal point along the two curves xi(t) and xj(t) equal contribution in the calculation. However, it may be that not all features of a curve should be be taken into consideration for the detection of clusters. At a certain scale two functions may be very similar, whereas irrelevant scale information makes them far apart in the space defined by Eq. (44). By using a similar approach as the previous sections, it is possible to include scale information in the calculation of the metric. A multiresolution formulation of Eq. (44) becomes: j D~j -

j

.

>~/2

c~~) + ~ d~(t)lc0~ + ~ dl(t ) 1=1

(45)

1=1

where c~k) is a constant and dlk)(t)is a detail function (from the wavelet coefficients) for scale 1. In general, one can construct distance matrices that emphasise on selected scales: Dij(v) -

c~i)

j + ~

i c~j) " v,d,(t)l + ~J v,dl(t)

1=1

/ ~/2 (46)

1=1

A systematic exploration of different vl parameters would be similar to the OSC approach (see above). However, here restrictions are placed upon the selection of v weights to produce distance matrices with increased resolution of the spectra (i.e. the simple multiscale approach): O}~ - 1/2

(47)

o } j l ) ~ ~

(49)

where ~ is the location of the Gaussian shape used, m is the frequency and 8 is the width of the Gaussian peak. if Fj(m) is the Fourier transform of spectrum j a set of filtered versions of the spectrum is made from Gj(m; ~) = Fj(m)Q(m; ~t),

~ E [~min, ~max]

(50)

The conceptual relation between classical, Fourier and multiscale cluster analysis is illustrated in Fig. 7.

5

Example applications

5.1 Regression 5.1.1 The simple multiscale approach Data set. This is one of several available test data set on the Internet [61]. The data set has been kindly provided by Dr. Windig and is a mixture of: 9 2-butanol; 9 methylene chloride; 9 methanol:

381 T m r , . ~

m

MULT'gSCALE/WAVELET O,LUb'TIBt ,~laU.YS~r.5

Fig. 7 Wavelet or multiscale cluster analysis can be seen as something in between classical cluster analysis on the time/wavelength domain and the Fourier cluster analysis in the .fi'equency domain.

9 dichloropropane; 9 acetone. The mixtures are measured by near infrared (NIR) spectroscopy at wavelenghts 1100-2498 nm. There are in total 140 spectra at 700 wavelengths. To enable analysis by FWT it is necessary to convert the data to length 2 n. A subset of the original data was extracted to make a total of 512 variables. The extracted data correspond to a window between variable no. 147 and 658, see Fig. 8. Only four of the five components are analysed (for display reasons only). For comparison, PLS regression was performed on the four components using all available wavelet coefficients (this is identical to using the original data). The prediction errors using A = {13, 11, 11, 17} PLS factors are 2:2%; 2"1%; 2:3% and 2:3%. To see the relation between the change of scale and PLS factors on the prediction error, it is instructive to plot the SEC calibration surfaces, see Figs. 9 and 10. An automatic determination of the optimal number of PLS factors for each scale is difficult without causing overfitting.

382 2.5

1.5

0.5

1O0

200

300

400

500

600

700

Fig. 8 The variables between the t~ro vertical bars were used #1 the analyses.

The interesting regions are zoomed to avoid large PRESS values to dominate the plot. Note the "rank ridge" at the left part of each plot. By inspecting the SEC surfaces, possible candidates for parsimonious models can be made, see Table 1. However, plotting the PLS-factor/error plots for selected scales is necessary to find better models. As Table 1 indicates, our first suggestions do have relatively high prediction errors compared to the models using all the wavelet coefficients. Fig. 11 plots the prediction errors with respect to PLS factors for selected scales. To achieve prediction errors comparable to those observed when using all the wavelet coefficients, we need to go to at least scale 5 and A = 10 PLS factors for all the components. Another important question is: how does the prediction model itself change with the resolution? When few scales are added we have very smooth representations of the spectra and it is expected that e.g. the B-coefficient regression vectors will display the same degree of smoothness as the input data. This can be seen in Fig. 12 which shows the PLS B-coefficient regression vectors for the different resolution levels. This type of plot can be used to get

383

Fig. 9 SEC surfaces for the four components.

a rough idea where the features important for the prediction may be located. However, it is not possible to obtain a precise localization since we are looking at whole scales rather than individual wavelet coefficients. In order to obtain more precise location of the features variable selection is required.

5.1.2 Variable selection by genetic algorithms In the remaining analyses only 2-butanol will be discussed. Variable selection using a genetic algorithm as described above was performed. A maximum population size of 64 and 100 generations was imposed. The mutation rate was set to 0.005 and the maximum number of PLS factors was 30. The GA routine used a double breeding cross-over rule. The cross-validation blocks were randomly selected. This was to ensure that the GA operation would not optimize by locking onto favourable structures in a fixed cross-validation operation. All calculations were carried out on MATLAB5 using an Alpha

384

Fig. 10 The SEC (RMSEP error) surfaces for the four components.

processor based machine with Ultrix operating system. The gaselctr routine in the PLS_Toolbox 4 performed the GA variable selection. The routine selected 249 variables (A = 14 PLS factors) with a prediction error of 2.1%. The PLS prediction error on the unseen validation set using all the variables is 2.1% (A = 13 PLS factors) for comparison. Thus, the prediction error is approximately the same with an almost 50% reduction in the total number of variables. A scalogram with the selected variables is shown in Fig. 13. Note that the whole of scales 2 and 3 are selected. 4Eigenvector Research, 830 Wapato Lake Road, Manson, WA 98831, USA.

385

Table 1. Parsimonious models suggested by Fig. 9. Compound 2-butanol Methylene chloride Methanol Dichloropropane

Scale 2 3 3 3

Error (validation set) 9.2 6.8 4.6 9.0

P L S factors 3 3 3 5

2-butanol

Methylene chloride

8 7

2

6

~6

~5

fl_ m 5 cO n4

rr4 3 5

2

4

6 8 PLS factors

2

10

Methanol

4

6 8 PLS factors

10

12

Dichloropropane

7

6

~5

fl_ m4 n-

3

rr 4

t 2

4

6 8 PLS factors

10

12

2

4

6 8 PLS factors

10

12

Fig. 11 PLS factor plots at di(ferent scales.[or the.[bur components. In spite of the fact that a significant model reduction has been accomplished, the results can be further improved. The next question to ask is whether all the selected variables really are necessary. It would be interesting to find which variables that seem to be very positive for the prediction ability of the PLS model. In order to shed more light on this problem, DVA was applied to the GA population of selected variables. The number of dummy variables tested for was set to be 60% of the total number of variables, (307) and the Student's t-test critical factor was chosen to produce a p-value of 0.0005.

386 2-butanol Pure s'pectrUm ". ~

I

Methylene chloride

Pu~ sp~trdm

|

0

200 400 B-coef bins

~ure spectrum

600

0

200 400 B-coef bins

600

Dichloropropane

Methanol

~

I Pure s ~ r u m

"o

tl) "o

"o "o

_.e

0

200 400 B-coef bins

600

0

200 400 B-coef bins

600

Fig. 12 This figure shows how the PLS B-coefficient regression vector for each component at different scales added. Note the importance of scales 3 and 4 which introduces localizable peak-like features at several places.

The threshold found using DVA produced 58 variables. These variables are plotted in a scalogram in Fig. 14. The prediction ability using these variables is 5.8% ( A = 14 PLS factors). Below, a comparison of this result with the TPW analysis is made.

5.1.3 The TPW approach The TPW variable selection is a more rapid method compared to both GA and GOLPE. A TPW selected 228 variables using A = 19 PLS factors. The result from the analysis is shown in Fig. 15. The prediction error on the unseen validation set is" 2.1%.

387

549

100

t~0

200

250 B~

3~

350

400

450

SO0

Fig. 13 GA variable selection o['2-butanol. Black areas indicate the (249) variables selected.

OVA on reoJIts Irom GA vanel:fle

eetecl~on

II II

7

~4

50

100

150

200

250

300

350

400

450

SO0

Fig. 14 The most important variables after D VA o f the GA-result. Here 58 variables are selected.

388

//

PLS ~.-varial:~ m~ction

4

50

100

150

20O

250

B~

3~0

350

400

450

500

Fig. 15 The results ~'om using TPW variable selection.

The next step was to use DVA on the results. The results from using DVA on TPW variable selection is shown in Fig. 16 where 61 variables (A = 14 PLS factors) are selected (RMSEP - 2.2%). The plot indicates regions at scale no. 2, 3, 5 and 7 as being in particular important. It is interesting to see that the region defined by bin no. 125-250 and scale 3 overlaps the region found by the GA variable selection procedure described above. By taking an intersection between the two wavenumberscale regions it is possible to see what regions they have in common, see Fig. 17. The prediction error using these 7 variables is 19.0% ( A - 4 PLS factors). Among these common variables a relatively narrow region at scale 5 is observed. In [62] the Simplisma method was used to find selective variables for the different components. Of interest are two wavenumbers: 2080 nm (bin no. 344) and 1716 nm (bin no. 163). Fig. 17 shows that this wavelet is close to the selective variable at 1716 nm. However, it is not selective for 2-butanol, but dichloropropane. Windig and Stephenson found that position 2080 nm is a selective variable for 2-butanol and the variable methods do select regions containing this position. In particular, in the results from the GA variable

389

I I II

OVA on TPW ~rial~e e d e d o n reeuHs

mm =

I

I

l

oo

50

100

150

200

250 B~

300

350

400

450

500

Fig. 16 DVA on the T P W variable selection results.

Common re~ons in both ~

and GA variab~ se~=bon us,rig DVA

I

'L

1

1~ ? " X . _ f ~ . _ _ _ _ j o

/

J

5o

too

15o

2oo

250

300

350

400

&50

500

Fig. 17 Important regions detected hv D VA applied to both T P W and GA variable selection. The pure spectrum of 2-butanol is includedjor comparison.

390 selection contains a narrow region at scale 6 which contains wavenumber 2080 nm. 5.1.4 Mutual information variable selection Mutual information as described earlier has been used on the results from the GA and TPW analyses. This means that the data matrix for each of the MI analyses is a binary "design matrix", i.e. the matrix indicating which variables have been selected during the multivariate modelling. The dependent yvariable is the prediction error. Thus the procedure is comparable to how DVA operates. The resolution factor (not related to wavelet resolution) for each run was set to 10. The cut-off threshold for the mutual information computed for each variable is determined by calculating the mutual information for a set of 306 dummy variables (60% of the 512 variables analysed). The results for the analysis are shown in Figs. 18 and 19. The prediction error for MI on the GA results has R M S E P = 2 . 5 % using 90 variables, ( A - 1 3 PLS factors) and MI on the TPW results has R M S E P = 2 . 8 % using 109 variables ( A - 11 PLS factors).

Iiij

II

Mutual in~orrnetion8nalys~ of Glk msulB

IIIII B

~4

450

~s

Fig. 18 MI analysis of the GA results.

391

5.2 Classification 5.2.1 The simple multiscale approach Cluster analysis. In this section, the application of the simple multiscale approach to cluster analysis is demonstrated. The masking method will also be used to localise important features. There are several possible cluster analysis algorithms, however only discriminant function analysis (DFA) will be used here. Before discussing the results from the simple multiscale analysis, this section will first present DFA and how it can applied to both unsupervised and supervised classification, followed by how the cluster properties ~ are measured at each resolution level. Discriminantfunction analysis. Discriminant function analysis (DFA) which is also referred to as canonical variates analysis is here the chosen cluster analysis method. DFA is usually used in a supervised mode, but can also be used in an unsupervised way. Here, the unsupervised mode is enabled by direct usage of the replicate information of object samples as classes. The effect of using DFA in this way is that it will reduce the within-replicategroup variance. DFA is in many ways similar to PCA. However, the

Mutual in~ermaliena n ~

ot ~

rlmui~

'1 II IIIII ] aII imm

II I i i

,,m

I 1~

1~

200

2'50

3NIO

3raO

4~

4~

Fig. 19 MI analysis of the TP W variable selection results.

392 eigenvectors are not pointing along directions corresponding to maximum variance in the data set. They point along directions which minimize the within-group variance and maximize the between-group variance. To accomplish this, the within-sample matrix sums of squares and cross products, W is computed together with T which contains the total sample matrix of sums of squares. The eigenvectors of the matrix (W -1 T - I ) correspond to the DFA latent variables. The eigenvectors are sorted according to the magnitude of the eigenvectors: ~1 > ~2 > ' ' " > ~r DFA scores are computed by projecting the data onto these eigenvectors. Due to the inversion step, DFA cannot handle collinearity and therefore can only analyse data matrices containing independent variables. A common way to accomplish this is to perform a PCA on the original data and only use the orthogonal scores vectors in the DFA routine. In the experiments performed the number of principal components used corresponds to 99.99% of the total variance. Using the same number of PCs for the different wavelet scales is not possible because data set reconstructed with very few scales are very smooth and have much fewer significant PCs. The DFA algorithm used here was implemented in MATLAB (The MathWorks, 24 Prime Park Way, Natick, Mass. 01760-1500, USA) following the description by Manly [63].

Measures of cluster structure. To test the simple multiscale approach to cluster analysis, the independent taxonomic information available for the different objects were used either directly or indirectly in the analyses. This is similar to the situation where a taxonomic expert is faced with a data set without the true classification information. In the process of determining "interesting clusters" the expert is expected to make use of his external knowledge in the assessment of the observed patterns. Thus, here the external class information is used to define a cluster. Having identified taxonomically relevant clusters, the next step is to measure how they relate to each other. The three properties measured for the two data set analysed were: 9 Overlap between the clusters. 9 Relative distances between the clusters. 9 Relative area of each cluster Note that relative rather than absolute areas and distances are used (i.e. compared to total area and maximum distance). The reason for this is that

393 the DFA spaces for the individual wavelet reconstructions have different magnitudes. When a smaller than the maximum number of scales are used in a wavelet reconstruction, variance is removed from the data set by making the spectra smoother. Thus, absolute DFA scores cannot be directly compared due to this. This is also the reason why the DFA score plots for the different wavelet reconstructions do not contain axes since they do not really convey any important information for the comparison. Having defined the objects contained in a cluster is not sufficient to describe properties like area and overlap. Here a very simple approach is employed since the number of significant DFA dimensions for both data sets is only 2. A cluster can be defined from the convex hull enclosing the objects. Computer algorithms for finding 2D convex hulls are readily available. A convex hull can be defined as follows: Let P1 and P: be two arbitrary points within the convex hull region. Then all points Pj falling on a straight line from Pl to P2 must also be within the convex hull region. This means that e.g. a triangle and a circle are all convex hull regions whereas "E" or "T" shaped regions are not. Cluster area is here defined as the area of the 2D convex hull polygon divided by the total sum of cluster areas. A distance between two clusters is defined to be the distance between the centre points of the corresponding convex hull polygons. An overlap between cluster i and j is defined as"

Pij =

:if:(Ci) f"l :~: (Cj) :~(Ci U Cj) 100%

(51)

where the operator 4/: returns the number of elements in a set C. Ci and Cj designate the set of objects for cluster i and j, respectively.

Data collection. Two data sets are used to demonstrate the multiscale cluster analysis method and have been kindly provided by Dr. Roy Goodacre at Institute of Biological Sciences, University of Wales, Aberystwyth [64,65]. Ten microlitre aliquots of bacterial suspensions were evenly applied onto a sand-blasted aluminium plate. Prior to analysis the samples were oven-dried at 50~ for 30 min. Samples were run in triplicate. The F T - I R instrument used was the Bruker IFS28 F T - I R spectrometer (Bruker Spectro-spin, Banner Lane, Coventry, U K ) e q u i p p e d with an MCT (mercury-cadmiumtelluride) detector cooled with liquid N2. The aluminium plate was then loaded onto the motorised stage of a reflectance TLC accessory. The wave-

394 number range is in the mid-IR regions: 4000-600cm -1. Spectra were acquired at a rate of 20 s -1. The spectral resolution used was 4 cm -l. 256 spectra were co-added and averaged. The digital sampling level was set to produce 882 data points.

U T I data set description. The UTI data set contains in total five different bacterial species (classes), but for the multiscale cluster analysis, only three of them are actually used in the analysis. The three different bacterial species are: E. coli, P. mirabilis and P. aeruginosa which are referred to as cluster 1, 2 and 3 respectively. Twenty-two E. coli (Ea-Eq), 15 P. mirabilis (Pa-Pj) and 15 P. aeruginosa (Aa-Aj) were isolated from the urine of patients with urinary tract infection (UTI) and prepared as described previously [66]. In total there are 148 (4 x 37) infrared spectra in this data set. Eubacterium data set description. The Eubacterium data set contains the reflectance infrared spectra of four different bacteria: E. thnidum, E. infirmum, E. exiguum and E. tardum which are referred to cluster 1, 2, 3, and 4. Four replicates for each sample is used. Four E. timidum (Ta-Te), four E.infirmum ( l a - l d ) , four E. exiguum (2a-2e), five E. tardum (Na-Ne), and five eubacterial hospital isolates (Ha-He) were prepared as described previously [64]. In total there are 88 infrared spectra (4 x 22 samples) in this data set. In order to perform FWT on the data sets it was necessary to have a data length as a power of two. Just adding zeros may introduce ringing effects due to sharp edges. In order to avoid this, a smoothing operation was performed to ensure a smooth truncation to zero at the low wavenumber end. The two data sets are shown in Fig. 20.

UTI results. The taxonomic information has here been used directly to determine the D F A directions. Since the UTI data set contains three clusters the number of significant DFA dimensions is two. Using the method described earlier it was found that the optimal wavelet for this data set was Coiflet 5. The results from the multiscale cluster analysis of this data set are summarised in Fig. 21. The interpretation of this multiscale cluster analysis is straightforward: All the clusters are overlapped until scale no. 3 is added when they all separate.

395 Eubact data set

0.8 i

0.6 0.4

-0.2 4000

i 3500

i 3000

i 2500

i 2000

Wavenu~,

i , 1500

t 1000

t 500

0

cm-~

UTI dam set I

i

i

I 3500

3000

i

i

2500

2000

i

0.8 0.6 -I~ 0.4 0.2 0 -0.2--0.4 4000

I

/

I

Wavenu~,

i

1500

I

I(XX)

I

500

cm-~

Fig. 20 The Eubact amt UTI ~&ta sets.

A scale dendrogram for the same analysis is shown in the upper part of Fig. 22. Where in the spectral domain is the feature located is responsible for the separation of cluster 1, 2 and 3 after adding scale 3? By employing the method of systematic variation of mask vectors to scale 3, it is found that of 223 = 256 possible masking vectors there is only non-overlap between the clusters in 48 of the combinations. For these combinations it was observed that wavelet variable 5 in scale 3 (covers region 2024-1534 cm -l in the wavenumber domain) was always selected.

E u b a c t results. For this data set DFA was used in an unsupervised mode. The optimal wavelet was found to be Symmlet 9. Fig. 23 shows the results from the multiscale cluster analysis.

After adding the first scale it is easy to see that cluster 1 is different from the others. Cluster 1 is close to the other cluster, but does not overlap with any of

396

4

5

|

|

cSb

t~

03

c3 Fig. 21 The multiscale cluster analysis result on the UTI data set. For each reconstruction a DFA is performed and the 2D scores are plotted.

Wavelet scale 0

1

2

3

4

5

6

7

8

9 3 2

4U 3

i

t

-1

2 ,,

,

|

Fig. 22 This figure shows a scale dendrogram for each of the two data sets analysed ( U T I and Eubacterium). Scale dendrograms as used here efficientO' summarise the qualitative change of the overlap structure of the clusters involved after adding the different wavelet scales. Note that other measures than cluster overlap could have been used in the scale dendrogram. Another possible measure would be the cluster area or shape.

397

5

,3

7

8

9

,%

J

Fig. 23

them. This suggests that there must be very broad scale features that are making the cluster 1 spectra different from the others. After adding scale 3, cluster 2 separates out from the others. Clusters 3 and 4 remain overlapped until scale 5 is added. This suggests that relatively narrow features are making these two sets of spectra different. Adding more scales does not change the overlap structure between the clusters. This means that there is an optimal number of scales needed to achieve the separation of all the clusters. Overlap between the clusters is not the only property of interest. Another possible cluster property is the relative area and how it changes with the addition of scales. The relative area of a cluster is related to the correlation between the objects in the cluster. Table 2 confirms that the relative cluster areas are becoming smaller as more scales are added. However, some scales have a larger impact than others on the change of the relative area for a cluster. The table shows that cluster 1 is more dispersed than the others and needs more scales to become compact. After adding scale 5 there is a significant decrease in the relative area of cluster 1. Below we investigate which regions that appear to be associated with this large change of relative area.

398 Table 2. Areas of clusters in 2D D F A score space for each addition of a wavelet scale. The Eubacterium data set.

Sca~ added

C&ster 1

Cluster2

Cluster 3

Cluster 4

1 2 3 4 5 6 7 8 9

29 24 30 31 9 6 3 3 4

22 21 9 6 3 2 4 4 4

27 42 17 8 6 5 5 5 4

9 10 4 2 3 5 3 3 2

Finding important regions. As shown above, cluster 1 is already separated from the other clusters after scale 1. Where are these very broad regions approximately located that are associated with this? For scale 1 there are two wavelet coefficients that represents wavelet functions covering half of the spectral region each. By using the masking method described in Section 3.3, there are four possible masking vectors for these two regions: {0 0}, {1 0}, {0 1}, {1 1}. For each of these masking combinations, a multiscale cluster analysis is performed and the overlap between cluster 1 and the others is recorded. There are only two cases where cluster 1 separates from the other clusters: {0 1}, {1 1}. This means that presence of the right region is necessary to produce a zero overlap between all other clusters. This result suggests that there is a feature in the right half of the spectrum that makes cluster 1 different from the others. The wavenumbers for this region are: 202452 cm -1 (actually the region is over 2024-600 cm -1 since 600 cm -l is the lower detection limit for the IR instrument used). A thick line in Fig. 24A shows where this region is localised. The standard deviation spectrum of the data set is also plotted in this figure. After adding scale 3 it was observed above that cluster 2 separates out from all the others. The next problem is to localise the region(s) that is(are) responsible. Since scale 3 has 23 = 8 wavelet coefficients, there are in total 28 = 256 different masking vectors possible to test out. For each of these a D F A is performed and the overlap of cluster 2 with the other clusters is recorded. In 74 combinations a non-overlap situation was observed. In all ot these combinations, wavelet coefficient no. 3 in scale 3 was always present.

399

0.35

0.3

C

0.25

ro r162 =_ o

.8 ,

"0 f04

2

1

0

1 11 1

1

1

3

3

3

3

22

1

3

11111

1

1 1 1 1

1

32 1

3 3

3

3

3

3

3

$ 3

13

3

3

3

1 3

-2

0

2

4

1st Variate

Fig. 6 Discriminant plots for the paraxylene data produced by suppl)'ing the coel~'cients resulting from the A WA to Fisher's linear discriminant analysis.

Butanol

11

1 1

:~JII'B1 l m I

2"~'~

2

2

22

2

22

2

2

l

I

I

I

-2

0

2

4

....

1st Variate

Fig. 7 Discriminant plots for the butanol data produced b)' supplying the coefficients resulting from the A WA to Fisher's linear discriminant analysis.

450

Table 3. Classification results for wavelet and scaling coefficients produced using filter coefficients from the Daubechies family with Nf = 16. Data Seagrass

Train Test Train Test Train Test

Paraxylene Butanol

X[3](0)

X[3]( 1)

X[4] (0)

X[4]( 1)

98.79 100 62.67 50.67 85.42 82.98

99.39 98.04 68.00 58.67 87.50 82.98

100 100 81.33 56.00 93.75 76.60

100 99.02 80.00 61.33 87.50 87.23

y = xT[~ + t; with I I - (131,132,..., [~p)T, I~- (gl, g2, ..., t~n)T. In practice, the vector of regression coefficients II, is usually unknown and is typically estimated by the least squares method. The least squares method calculates regression coefficients so that the residual sum of squares aT~ is minimized. The least squares solution is b-

(xxT)-lXy

where b - ( b l , . . . , bp) T is the estimate of the true regression coefficients II. The estimated response is then ~, -- XTb. The M L R model assumes the residuals are independent and gi ~ N(0, 0"2). 3.2 Regression assessment criteria

In this section, we will describe three regression criteria relevant to Section 3.5. These criteria can be used to assess how well a model is performing. The three criteria are - the residual sum of squares (RSS), the R-squared (R 2) measure and the predictive residual sum of squares (PRESS). The residual sum of squares and R-squared criteria both measure how well the model fits the data. These criteria are respectively defined n

RSS - Z ( Y i - yi)2 i=l

451

R2

~-~in_-_l (Yi - 5/i) 2

1 --

----

~-~in=l (Yi- ~)2

1

RSS TSS'

where , ~ - ~ i n l yi/n is the mean response and the total sum of squares T S S - Z i n l ( Y i - .Yi)2The RSS measures the sum of squared deviations between the actual and predicted values of the response. A lower measure of the RSS is preferred. The R 2 criterion ranges from zero to one, with values closer to one being preferred, provided that a high R 2 is not a consequence of overfitting. We will test the performance of the adaptive wavelet algorithm for regression purposes using an independent test set. For this reason we have decided to ~ formulate an R 2 measure for the test set which is denoted by Rtest RSStest Rt2est -- 1 -- TSStes~ The residual and total sum of squares for the testing data are defined, respectively to be nt RSStest- Z ( Y l - 5'i)2 i-1 nI -t)2 TSStest- Z (Yl- Yi i=l where y ' - ( Y ' I , - . . , Y'n)T is the response values of the independent test set, Y'= (Yl, ,!)~) T are the predicted test response values, n' the number of t objects in the test data set and ' 2 ' - ~ i =n'l Yi/n' is the mean of the test responses. Define the PRESS statistic to be PRESS --- ~in=l (Yi- ~/-i) 2. Here, Y-i is the predicted value for Yi, but object xi was "left out' when estimating the parameters in the regression model. Another way of calculating the PRESS statistic is simply by using ( Y i - Y-i) -- Y i - .Yi

1-hii

where hii is the ith element along the diagonal of the hat matrix H - x T ( x x T ) - I X . This avoids the need to leave out observations in turn.

452

3.3 Regression criterion functions for the adaptive wavelet algorithm A suitable criterion function for regression analysis should reflect how well the response values are predicted. In the adaptive wavelet algorithm, the criterion function considered for regression is based on the PRESS statistic and is then converted to a leave-one-out cross-validated R-squared measure as follows CVRSQ = 1 -

PRESS TSS

(6)

The formulation of Eq. (6) using the hat matrix makes the leave-out-one method of cross-validation quite a useful and relatively inexpensive procedure to employ. The cross-validated R-squared criterion function is defined as PRESS ~CVRSQ(X

[j](T)) - - 1 --

TSS

The actual regression model used for predicting the response is - (x[J] (~))Tb.

3.4 Explanation of the data sets Two data sets and three responses were used for evaluating the performance of the various regression procedures. These data sets will be referred to as the sugar and wheat data. A summary of each of these data sets is presented in Table 4. Here the number of spectra in each training and test set is displayed, as well as the response(s) which are to be modelled by each spectral data set. The dimensionality of both data sets is p = 512.

Sugar data The sugar data were supplied by Dr Nils Burding at the Bureau of Sugar Experiment Station in Gordonvale. The training sugar data contain 100

Table 4. Description of the spectral data sets used for regression. Data Set

Train

Test

Responses

Sugar Wheat

100 60

89 40

brix, fibre protein

453 digitized spectra for which log 1/reflectance was measured at the 512 wavelengths 916, 9 1 8 , . . . , 1938 nm. The test set contains 89 spectra. Fig. 8 shows five sample spectra from the sugar training data which were used to model the responses, brix and fibre. At 1100 nm there is a distortion which arises from a change in instrumentation. One detector is used to measure the radiation reflected for wavelengths less than 1100 nm and another detector is used to measure the radiation reflected for wavelengths greater than 1100 nm (inclusively). The change in receptors gives rise to the jump. Wheat data

The wheat data set was accessed from Professor Philip K. Hopke and has previously been discussed in the literature, see for example [6]. The training wheat data contain 60 spectra for which log 1/reflectance was measured at the 512 wavelengths 1100, 1102..... 2122 nm. The test set contains 40 spectra. Fig. 8 shows five sample spectra from the wheat training data. The wheat training data were used to model protein content. 3.5 Results

The adaptive wavelet algorithm (AWA) is applied to the regression spectral data sets described in Section 3.4. The AWA is applied with similar settings as those used for classification. The (m, q, J0) settings for which the AWA is applied, are again (4,3,2), (4,2,2), (8,1,1), (2,5,3), (2,5,4), (2,7,3), and (2,7,4). The most conceivable difference between the AWA when applied for regression (as opposed to classification) is the criterion function which is implemented. Here, the cross-validated R-squared criterion which is based on the PRESS statistic, is the regression criterion function which is implemented

r (3 ttO (3 Q

|

L

2

!

!

i

!

!

!

!

0

-- -2

I 1000

~00 11

,.,I 1200

1300

1400

wavelength

1500 (nm)

1600

1700

Fig. 8 Five sample spectra./i'om the sugar data.

1800

1900

454 by the AWA. A similar banded selection strategy used for classification is used for regression. Here, the band ~ at some level J0 in the DWT which produces the largest regression criterion m e a s u r e (~CVRSQ(X~](T)) forms the basis of the optimization routine. The same coefficients are later supplied to MLR. If the algorithm chose to optimize over a scaling band (i.e. ~ = 0), then for the same (m, q, J0) settings the experiment was repeated, where optimization was over the wavelet band producing the largest CVRSQ measure at initialization. The optimization routine halted if 2000 iterations of the optimization routine had been performed or sooner if an optimal value was obtained. The (m, q, J0) settings which produced the highest test R-squared measures are displayed in Table 5 for each of the data sets. Also shown is the number of filter coefficients (No), used in computing the DWT and the number of coefficients (Ncoef) in each of the bands for the respective (m, q, J0) settings. It seems that brix achieved the highest test R-squared measure, followed by protein and then fibre. For the brix response the (2,5,5) setting produced the best results. When the fibre response was modelled using the AWA, the best setting in terms of the Rt2estmeasure was (8,1,1). The best results for the wheat data were also obtained with the (2,5,5) setting where optimization was over a wavelet band.

.

o~ 2 tI

.

.

.

.

.

.,

~ -2

,

,

,

1900

2000

2100

,

1100

1200

1300

1400

1500 1600 1700 wavelength (nm)

,I

1800

Fig. 9 Five sample spectra from the wheat data.

Table

Brix Fibre Protein

5. R - s q u a r e d

m 2 8 2

q 5 1 5

values resulting from the AWA.

jo 5 1 5

Nf 12 16 12

Ncoef 16 8 16

"c 1 6 2

Train 0.975 0.872 0.975

Test 0.971 0.801 0.825

455

Table 6. Regression results for wavelet and scaling coefficients produced using filter coefficients from the Daubechies family with Nf = 16. Data Brix

Fibre Protein

Train Test Train Test Train Test

X [4](0)

X [4]( I )

X 131 (0)

X [31 (I)

0.975 0.973 0.781 0.692 -

0.961 0.949 0.797 0.723 0.952 0.704

0.740 0.753 0.647 0.533 0.763 0.263

0.525 0.530 0.707 0.569 0.795 0.108

3.5.1 Regression using Daubechies' wavelets This section is similar to Section 1.5.1 in that we perform the 2-band DWT on each data set using filter coefficients from the Daubechies family with N f - 16. The coefficients X[4](0), X[4](1), X[3](0), X[3](1) are supplied to MLR. The DWT was performed on the original (uncentred data), but the coefficients and response variables were centred, prior to them entering 2 the M L R model. The Rtrai n and Rtest for each response are displayed in Table 6.

Due to numerical instabilities it was not possible to obtain regression results for the protein model when the scaling coefficients from band(4,0) were supplied to MLR. This problem arises from the condition number of the matrix (X[4](0)Tx[4](0)) being quite large (3.133e+ 17). Care should also be taken when interpreting the results for the scaling coefficients from the wheat data in band(3,0) for the same reason. A higher Rt2est is obtained for the brix response using Daubechies wavelets, whilst, the AWA produces a higher R~est value for the fibre and protein responses.

References 1. G. McLachlan, Discriminant Analysis and Statistical Pattern Recognition, Wiley, New York (1992). 2. D. Hirst, Error-rate Estimation in Multiple-Group Linear Discriminant Analysis, Technometrics, 38 (1996), 389-399. 3. S. Brier, Monthly Weather Review, 78 (1950), 1-31.

456

4. D. Coomans and I. Broeckaert, Potential Pattern Recognition in Chemical and Medical Decision Making, Research Studies Press, Wiley, Schichester, (1986). 5. Y. Mallet, D. Coomans, J. Kautsky and O. de Vel, Classification Using Adaptive Wavelets for Feature Extraction, IEEE-PAMI, 10 (1997), 1058-1066. 6. J. Kalivas, Two Reference Data Sets of Near Infrared Spectra, Chemometrics and Intelligent Laboratory Systems, 37 (1997), 255-259.


457

Chapter 19 Wavelet-Based Image Compression O. de Vel, D. Coomans and Y. Mallett Statistics and Intelligent Data Anal~'sis Group, School of Computer Science, Mathematics and Ph3"sics, James Cook University, To~'nsville, Australia

I Introduction Many applications generate an exponentially increasing amount of information or data which needs to be stored, processed and transmitted in an efficient way. Typical information-intensive applications include spectral and highresolution image analysis. For example, a computerised axial tomography (CAT) image slice of size 512 • 512 and pixel depth (i.e. number of possible colours or grey-levels) of 8 bits occupies 0.25 MB of storage memory. For 60 such slices in a patient scan used in 3-D reconstruction, the total storage requirements is of the order of 15 MB. As a result of the possibly many stages involved in image analysis, each image in itself may generate other images, thereby further increasing the storage requirements for the image analysis procedure. For example, the raw CAT image slices can be processed to create a set of segmentated images used for interpretation such as volumetric analysis. Unfortunately, current storage hardware is inadequate for storing large amounts of such data as might be found in a patient database. Furthermore, if these data were to be transmitted over a network, the effective transmission times can be large. A solution is to employ compression techniques which may be capable of achieving a reduction in storage and transmission demands by a factor of more than 20 without significant loss in perceived image quality. Much of the information in a smooth image is highly correlated by virtue of the fact that, for example, pixel values are not spatially random and that the value of one pixel indicates the likelihood of its neighbours" values. Several types of correlation exist in an image"

1. Spatial correlation: Pixel values in a neighbourhood of a given pixel are generally similar. Exceptions include pixels in the neighbourhood of a pixel which forms the edge of an object in the image.

458

2. Sequential correlation: This occurs when two or more images are taken at different times (e.g. as a set of video frames) or different spatial positions (e.g. CAT image slices). The same pixel in adjacent image frames or slices is generally strongly correlated. 3. Spectral correlation: The spectral decomposition (Fourier transform) of an image is often smooth. Rapid fluctuations in the energy content of adjacent frequencies are uncommon. That is, spectral frequencies in a neighbourhood of frequencies are correlated. The presence of one or more of spatial, spectral and temporal correlations (and, therefore, the existence of an inherently high degree of redundancy) indicates that there exists a description of the image that has a significantly lower rank (for the definition of rank, see Chapter 4). That is, there exists in the image a set of features that captures most of the independent features. This suggests that an image is a good candidate for compression. Compression schemes can be broadly classified as loss-less or lossy compression. Loss-less compression schemes assume no loss of information during a compression-decompression cycle. This is most suited to data that need to be reconstructed exactly. Lossy compression schemes allow a certain error during a compression-decompression cycle, as long as the information loss is tolerable (i.e. the quality of the data is acceptable). The degree of tolerance to information loss is dictated by the particular application and some distortion metric appropriate to the application at hand is employed to measure the quality of the compression (see Section 2.1). For example, images which are used for simple visual display purposes can tolerate some loss as long as the images are psycho-visually acceptable. However, images that are used for segmentation or classification (e.g. medical or micro-fractographic industrial X-ray images) may not tolerate much information loss, particularly in the region of interest in the image. Lossy compression schemes have the advantage that a higher compression can be achieved compared with loss-less compression schemes. Most compression algorithms generally use a combination of both lossy and loss-less compression schemes with some facility made available to select the degree of loss of quality. In Section 2 we introduce the fundamentals of image compression and overview the various compression algorithms. We review the transformation techniques used in image compression in Section 3. Section 4 describes image compression using optimal task-based and best-basis image compression algorithms.

459

2

Fundamentals of image compression

The standard procedure used in image compression algorithms comprises of three stages namely, an invertible transformation, quantisation and redundancy removal (see Fig. 1). The decompression phase usually involves the reverse procedure followed in the compression phase. In the case of multidimensional (spatial or temporal) imagery, the compression algorithm usually includes additional encoding/decoding algorithms to exploit the inherent spatio-temporal correlation in the set of images. In such cases the overall compression/decompression is generally asymmetric: that is, the space-time complexity of the compression and decompression phases is different to allow for fast visualisation. The invertible transformation stage uses a different mathematical basis of features in an attempt to decorrelate the data. The resulting data will have a set of features that capture most of the independent features in the original data set. Typical features used include frequency and spatial location. The transformation is nearly loss-less as it is implemented using real arithmetic and is subject to (small) truncation errors. Examples of invertible transforms include the discrete cosine transform (DCT), the discrete wavelet transform (DWT) and the wavelet packet transform (WPT). We will investigate these transforms later.

Fig. 1 Stages of image compression.

460 If the transformation stage is effective in decorrelating the data, then the transformed image pixel data will have a large number of features with small real number values. The quantisation stage performs the essential rank reduction by replacing the transformed data stream of real numbers by a stream of reduced length with lower-precision coefficients or symbols that can be coded using a finite number of digits. The higher the compression required, the smaller the number of coefficients generated. Two kinds of quantisation can be performed namely, scalar and vector quantisation. Scalar or regular quantisation partitions the real axis into non-overlapping intervals and associates each real number with a coefficient or symbol associated with the interval to which it belongs. Prior to scalar quantisation, the decorrelated features are mapped onto the real axis (e.g. the DCT maps its 2D block structure onto the real line by "zig-zagging" through the block from low to high frequencies to exploit the fact that much of the relevant information contained in most image typologies is described by the set of lower frequency features). A quantisation table is used to store the pairs of intervals and symbols. Vector quantisation replaces a group of features (the real numbers) with a symbol. For example, wavelet compression can use groups of wavelet coefficients that are associated with the same spatial location. The fewer the number of groups, the higher the compression. The stream of coefficients emanating from the quantisation stage may still be redundant. The redundancy removal stage replaces the coefficients by a more efficient alphabet of variable-length characters. For example, some coefficients may be more frequent than others and these are allocated shorterlength codes compared with infrequent coefficients that are allocated longer codes. Variable-length coding algorithms are also called entropy coding algorithms and examples include the efficient Huffman and arithmetic coding [1]. Unfortunately, entropy codes require the statistics (probabilities) of the coefficients to be known a priori. Universal coding (UC) algorithms attempt to measure the statistics during the actual coding operation and adapt themselves in order to maximise the compression. Example (UC) algorithms include substitutional (or dictionary) methods [2]. The resulting compression from the redundancy removal stage is loss-less. Compression algorithms that do not conform to the above three-stage scenario also exist. One popular set of algorithms is based on the observation that natural images exhibit self-similarity at different scales. That is, information of an object at one level of magnification or resolution is repeated at a different level of magnification. A portion of an image at one

461 scale may be approximated by another portion of the image at different scale. For example, the land/sea coastline has a similar appearance at different map scales. Compression is then achieved by only storing the nonsimilar parts of the image. Algorithms which exploit the self-similarity property of images include fractal algorithms [3], weighted finite state automata [4] and generalised stochastic automata [5]. Fractal image compression algorithms generally perform better at higher compression ratios compared with the more traditional DCT-based algorithms (e.g. JPEG). NB: The compression ratio is defined as the number of bits required to store the original image divided by the number of bits required to store the compressed image. 2.1 Performance measures for image compression

When comparing different lossy image compression algorithms one usually desires the compressed image to be of the same visual quality as the original image. The most common measure of quality is the mean square error (MSE) or distortion defined as: 1

N-1

-

MSE - ~ y ~ Ixi - x i li=0 ..-..

where xi and x i are the input and reconstructed image pixel values. Alternatively, the peak signal-to-noise ratio (PSNR), measured in decibels (dB), is defined as: M2 PSNR - l0 logl0 D~where M is the maximum peak-to-peak value in the signal and D is the noise level. For an 8-bit image M = 256. In general, however, distortion measures based on squared error are not satisfactory when assessing the quality of an image, particularly at high compression ratios. An important consideration in determining the required image quality is the task or application for which the image is to be used. Each image application may require a different quality measure. For example, an image broadcast would be more concerned with tonal reproduction whereas a task involving the interpretation of X-rays would be more interested in image sharpness, etc. In many cases a perceptually weighted MSE may be more appropriate. It is known that, based on studies in visual psy-

462 chometrics, the human visual system has a reduced sensitivity at low frequencies and a very marked insensitivity at high frequencies. So, for example, errors are less visible in bright and "busy" (in terms of edges and discontinuities) areas of the image.

3

Image decorrelation using transform coding

As mentioned previously, smoothly varying images are generally characterised by a high degree of redundancy due to the presence of one or more spatial, sequential and spectral correlation. We can apply a change of mathematical basis in an attempt to decorrelate the image data, resulting in data that will have features that capture most of the independent features in the original image. We consider three transformations that have shown to decorrelate smooth images: the Karhunen-Loeve Transform (KLT), the Discrete Cosine Transform (DCT) and the Discrete Wavelet Transform (DWT). We first briefly describe the KLT and DCT.

3.1 The Karhunen-Loeve transform (KL T) The basic idea of the Karhunen-Loeve Transform (KLT) is that, if the correlation in the image is known, then it is possible to calculate the optimal mathematical basis by using an eigen-decomposition. The optimal basis is defined here as the one that minimises the overall root-mean-square distortion. Consider an image I = I(X) where X = (x1, x2, x 3 , . . . , XN)T is the vector of N image pixels. The intensity (pixel value) of the jth pixel, xj, is assumed to be a wide-sense stationary random variable with a non-negative value. We calculate the positive definite autocovariance matrix AI = E[XX T] and we can find the orthogonal matrix U that diagonalises A~. That is, UAIU T is diagonal with the diagonal values, referred to as the eigenvalues, being the uncorrelated coefficients or features in the transformed space. The optimal basis is given by the associated set of eigenvectors. The decorrelated image corresponds to the KLT basis transform Y = UX and, since the image is completely decorrelated (there are no off-diagonal values), it is considered to be an optimal transform. The autocovariance matrix of Y is given as E[YY T] = E [ u x x T u T] = E[UAIU T] which, as stated above, is diagonal. Besides decorrelating the image, the KLT has another useful property: the

463 KLT coefficients (eigenvalues) are ordered according to decreasing variance and compact the energy of the image into a few large coefficients. This allows the compression ratio to be set a priori by simply selecting the appropriate number the coefficients. Unfortunately, this approach has some significant disadvantages: 9 The time-complexity is generally O(N 3) for the diagonalisation algorithm and O(N 2) for the basis transformation. 9 The basis is a function of the image data since it depends on the autocovariance matrix for the given image. That is, each image will have its own basis transform. 9 The autocovariance matrix varies considerably from image to image, though the KLT assumes statistical stationarity. For these reasons the KLT is seldom used in practice. To circumvent these problems, sub-optimal basis transforms are employed which effectively decorrelate the image but are image-independent and have a reduced (linear or linear-log) time-complexity.

3.2 The discrete cosine transform (DCT) Very often image statistics are assumed to be isotropic (though this is not quite correct since there are vertical correlations between pixels in successive image scan lines) and the autocovariance matrix is modeled as some decreasing function of the geometric distance between any two pixels in the image. The autocovariance matrix is of the following Toeplitz form (assuming unit variance and zero mean):

A I z

1 p p2

p

p2

p3

...

1 p

p 1

... ...

p3

p2

p

p2 p 1

..-

where P is the correlation coefficient with the condition that P is large (P ~ 1). This model gives reasonably good agreement with experimental data for natural images (generally, P is found to be P > 0.9).

464 The DCT is defined as an inner product of cosine basis functions: YO -=- ~

YJ -

1 N-1 Zi=0 Xi

~N-1 Z i=0

((2i+l)j) xi cos 2p 4N

k_1

2,.. '

~

N_ 1 "'

The DCT was developed as an approximation to the KLT where, for large values of p, the DCT approximately diagonalises the above matrix A,. In fact, the DCT is asymptotically equivalent to the KLT of a stationary process when the image block size tends to infinity (N ~ ~ ) . Even for small values of N (say, N = 64), the basis functions for the KLT and DCT for many natural images look remarkably similar. Also, the DCT can be computed very efficiently with a time-complexity O(N log N) as opposed to O(N 3) for the case of the KLT. The computation can be made even more very efficient by blocking the image into K square blocks, each block with ~ x ~ pixels where N = K 2 N' (i.e. create disjoint blocks of sub-images of, say, 8 x 8 or 16 x 16 pixels), thereby reducing the DCT computations to each block. This makes the DCT the preferred algorithm in many standard commercial compression algorithms such as JPEG. Unfortunately, the DCT has some shortcomings. The blocking effect can create annoying high-frequency effects at the block boundaries due to the inherent discontinuities at image block boundaries and it also effectively reduces the compression of the entire image since the correlation across image block boundaries is not removed. The DFT can be used in lieu of the DCT. However, the D F T has rather severe blocking effects which is more noticeable than with the DCT, thus making the DCT the preferred option. To attenuate the blocking effect, smoothly overlapping blocks are used rather than disjoint blocks. The orthogonality of the overlapping blocks can still be achieved by using the lapped orthogonal transform (LOT) which is an elegant extension of the DCT. While the blocking effects are attenuated with the LOT, some other artifacts such as ringing appear around the block edges and its increased level of complexity makes LOTs less attractive. An alternative approach is to use wavelet transform coding schemes.

465

3.3 Wavelet transform coding An easy way to construct a multi-dimensional (e.g. 2-D) wavelet transform is, for example, to implement the tensor products of the 1-D counterparts. That is, we apply the 1-D wavelet transform separately along one dimension at a time. This, as we shall see shortly, results in one scaling function and three different "mother" wavelet functions. Although simple, such a separable decomposition has some drawbacks. For example, the number of free parameters used in the design of a separable 2-D wavelet transform is very much reduced (though this could also be seen as an advantage!). Also, only a rectangular partitioning is possible with a separable decomposition, and the same limitations appearing in one dimension will appear in two dimensions. To overcome such drawbacks, non-separable decompositions are necessary. One non-separable technique is to sub-sample separately as before, but use non-separable filters (four different filters for two dimensions). True multi-dimensional treatment of wavelets leading to single scaling and wavelet functions is possible but with a significant increase in implementation time-complexity e.g. quincunx lattice (along the diagonal) and hexagonal lattice sub-sampling [6]. For this reason we only discuss the separable decomposition technique, where both the filtering and sub-sampling are separable. We recall from Chapter 8 that the separable wavelet decomposition involves applying the 1-D wavelet decomposition to the rows and columns of the image matrix, I(x,y) where the image I is a finite sequence indexed by the two Cartesian coordinates x and y with N• column-pixels and Ny row-pixels, respectively. This decomposition results in a partially ordered set of subspaces Is,t(x,y), called sub-bands, where t >~ 0 indicates the level of the decomposition and 0 < s < (mm) t is the sub-band index at the given decomposition level. At a given level t, the orthogonal rectangular sub-bands correspond to disjoint covers of the wavenumber space, where the dimensionality of each sub-band Is. t is (Nx m -t) x (Ny m -t) owing to the m-way decomposition Normally, m -- 2 (the dyadic decomposition scheme) and we have four image sub-bands resulting from each octave sub-band in successive decomposition steps as shown in Fig. 2. At the image boundaries, periodic extension is normally used (see Chapter 4). The sub-bands in each successive dyadic decomposition step are generally labelled simply as IL,L, IL,H, IH,L, and IH,H ("L" for "low" frequency, and

466

l Ioo /

Level t=O

1,

l Io,,1 I,, / /,2,~ I 13,~/ Io,]l,,dl.,dl~,d /1I8.dl 12. 2~/1dl6~ 3.1~2tl1~. / ,3.t2/

Level t=l

Level t=2

11,o.ti,,.t1,,t1.,5.t

Fig. 2 A 2-D recursive separable wavelet decomposition (shown for m = 2).

"H" for "high" frequency). The ILL sub-band corresponds to an "average" or "smoothed" image, whereas the sub-bands IL,H, IH.L, and IH,H are the "detailed" images. The IL,H, and IH,L, sub-bands capture the horizontal and vertical edges, respectively, whereas the IH.H sub-band captures the diagonal details. The image sub-band decomposition scheme for a single decomposition level is shown in Fig. 3. We observe that the lowest band (ILL), being a low-pass, down-sampled version of the original input image, has many characteristics of the original image. Most of the image correlation, except along some of the edges, remains in the IL.H, IH.L, and IH,H sub-bands. This is due to the two-stage (L and H) directional filtering, where edges in the image are confined to certain directions in a given sub-band. Wavelet-based software packages generate the sub-bands in the form of a grouped display of the smoothed and detailed sub-bands. For example, the grouped display of the sub-bands in a single decomposition step as generated by S + WAVELETS TM is shown in Fig. 4. Consistent with the notation used in this book, the "cl" and " d l " labels represent the smoothed and detailed coefficients, respectively, so that " c l - c l " corresponds to the smoothed (ILL) wavelet coefficients, whereas " c l - d l " , " d l - c l " and " d l - d l " correspond to the detailed wavelet coefficients (IL,H, IH,L, and IH,H sub-bands, respectively). NB: In some wavelet software packages (e.g. S + WAVELETSTM), the

467

Fig. 3 Image wavelet decomposition for a single (dyadic) decomposition level.

convention used for the index of the origin in a grouped display is that the index is located in the lower left-hand corner of the grouped display whereas in other packages (such as MATLABTM), it is located in the upper left-hand corner. With a larger number of decomposition levels, the 2-D DWT is displayed as shown (for the case of 3 levels) in Fig. 5. The labels " L L H H " ,

468

Fig. 4 Image wavelet coefficient matrices (shown for a single decomposition level).

LH

d2

LLLH

HH

LLHH HL

_ 03

LLLLLI-.LLLHI-

C3

LLLLLL.LLLHL c3

d3

LLHL d2

dl

Fig. 5 A 2-D D WT image wavelet coefficients for multiple decomposition levels (shown for three levels).

etc. indicate the corresponding wavelet coefficients in each decomposition level sub-band. As mentioned before, when dealing with values of m greater than 2, the decomposition scheme is slightly more complicated by the fact that we have more than three sub-bands of wavelet coefficients at each level of the decomposition scheme. In fact, there are m 2 sub-bands at each decomposition level of the 2D DWT. For the case of h levels, we have a total of (m 2-1)h + 1 D W T sub-bands or, for the dyadic case, a total of 3h + 1 sub-

469 bands. Fig. 6 shows the decomposition scheme for two levels and for the case of m = 4. Note that we have used the same labelling format for the smooth and detailed sub-bands as Chapter 8. That is, for level j, the smoothed subband coefficients are labelled as "cj'" and the m - 1 detailed sub-band coefficients as "d~ 1) d~2) d~m-l)'' 9

9

"

"

"

~

Whilst the 2 D - D W T provides an efficient space-frequency characterisation of a given image, it only uses a fixed decomposition of the pixel space. As in the case of the 1-D wavelet packet transform, we can extend the wavelet packets to two dimensions. That is, the 2D wavelet packet transform (2D-WPT) generates a more general, full m2-ary tree representation with a total of m 2 + m 4 + -.- m 2h sub-bands for h levels. Each sub-band in a given level of the tree splits into a smoothed sub-band and m2-1 detailed sub-bands, resulting in a tree that resembles an m-way pyramidal "stack" of sub-bands. For the case of a dyadic decomposition scheme, this corresponds to a pyramidal sub-band structure where each sub-band is decomposed into 22 = 4 sub-bands at each successive (higher) level (see Fig. 2). Fig. 7 shows results of the third level of the 2D W P T for the dyadic case - a total of 2 2 ( 4 - 1 ) - - 64 sub-bands at the third level, where each sub-band block consists of 16 x 16 pixels for an image size equal to 256 x 256. We note in passing that the maximum number of independent mZ-ary orthonormal tree representations is (mm) h, a potentially very large number of trees. The set of such trees is often referred to as a bases dictionary.

Fig. 6 Image wavelet coefficients jbr a multi-band decomposition scheme (shown for m = 4 and for the case of two levels).

470

Fig. 7 2-D W P T image wavelet coefficients for level 3 (for the dyadic case, m = 2).

Looking at either the 2D D W T or WPT sub-band images it is clear that the sub-bands are related; that is, they are not independent. Of particular interesl in image compression are horizontal or vertical edges, which will appear in the smoothed sub-band image as well as in every detailed sub-band that was generated by horizontal filtering (for horizontal edges) or vertical filtering (for vertical edges). It may be more appropriate to only select a useful subset of sub-bands. A variety of algorithmic approaches may be used. An algorithm that is quite useful is the e m b e d d e d zero-tree wavelet encoding (EZW) algorithm which exploits the self-similarity between the different wavelet bands [7]. It uses a simple heuristic - if a wavelet coefficient on one decomposition level is set to zero, it is likely that the wavelet coefficients corresponding to the same locations on the finer decomposition levels are set to zero as well. That is, for natural images with rapidly decaying spectra, it is

471 unlikely to find significant high-frequency energy if there is little low-frequency energy in the same spatial location. The EZW algorithm produces good compression results without requiring a priori knowledge of the image statistic. An alternative approach is to choose the best full or partial (i.e. pruned) mZ-ary tree representation. In this case, we have to search for the "optimal" full or partial tree from the bases dictionary subject to minimising some information theory-based (or other)cost f u n c t i o n - called best-basis search algorithm due to Coifman and Wickerhauser [8]. The search algorithm used is generally based on the dynamic programming paradigm and the cost function used is determined by the application at hand. This "adaptive" bestbasis search algorithm is computationally efficient and has also been used in a variety of applications including, classification/discriminant analysis, regression, dimensionality reduction etc. In the case of image compression, a simple cost function often used is thresholding- that is, sub-band coefficients with values less than some predetermined threshold are ignored. Other bestbasis approaches have also been developed, including tree pruning using a Lagrangian cost function [9] and tree building based on the local transform gain metric (the ratio of the sum-to-product of the wavelet sub-band variances) [10]. Fig. 8 shows the set of best-basis sub-band blocks obtained for the case of a threshold cost function (using the source image in Fig. 3) and

-H [ l I

I|1 IIIII |Jill

II1

]

1

[ 1

Fig. 8 2-D W P T best-basis of image #1 Fig. 3 (.for the dyadic case, !11 = 2, using a symmlet of width equal to 8, a "threshold" cost function, and for a maximum of six levels).

472

with an eight-point symmlet. The best-basis chooses smaller sub-band blocks to capture the detailed features in the image and larger blocks to represent lower-frequency information. Many of the sub-band blocks in Fig. 8 are grouped smoothed sub-bands (lower left of display) as one would expect with a normal 2D DWT. However, some sub-bands are also present in other parts of the WPT, indicating that some of the information in the image is best captured by the detailed coefficients. By overlaying the WPT (for each level) with the best-basis, we can easily identify the wavelet coefficients that best capture the different regions of the image. More details on the best-basis search algorithm are given in Chapter 6). One inherent problem with the best-basis technique is the choice of the wavelet type to use. Generally, a standard ("off-the-shelf") wavelet is chosen prior to the best-basis search; for example, a coiflet or symmlet. This choice is made independently of the best-basis search and of the application at hand. A more flexible approach involves designing the wavelet in conjunction with the best-basis search. That is, the wavelet is customised to the task at hand and integrated with the best-basis search. The reader is directed to Chapter 8 for more information on custom, task-specific wavelets. Different wavelet customisation/best-basis search integration methodologies are possible, namely the wavelet customisation is made (i) independently of (see Fig. 9), or (ii) integrated with the best-basis search (see Fig. 10). The former case is simpler to implement and has reduced computational requirements, whereas the latter is better adapted to the task at hand (i.e. should give optimal image compression performance) but requires larger computational resources. We present results for the latter (integrated) case in the next section (Section 4). From Chapter 8 we note that the construction of a task-specific wavelet proceeds by generating a normalised vector v of dimensionality m-1 and a further N d m - 1 normalised vectors ui, each of length m (Nf is the number of wavelet filter coefficients). The total number of free parameters required to construct the wavelet is therefore Npar =

( N f - m)(m - 1) m

nt-

m- 2

For the dyadic case (m = 2), N p a r = N f / 2 - 1. For short compact wavelets (i.e. small Nf ), the number of parameters is not a large number and, consequently, the search space has a low dimensionality. Furthermore we shall show that, experimentally, the hyper-surface in the search space is smooth.

473

I Construct TaskSpecific Wavelet I

........................~

1

J~

Image data

[ Calcuiaie WLe,et I Update Wavelet Parameters

!Wavelet Construction

Parameters

l ......................................] ......)))))))/)/)//))I//)///-//)/)))))))//_ ._._.;_

[

..

Evaluate Compression Ratio

............_.Yest.............. ~ ~

............

Best-basis Search

Fig. 9 Independent wavelet construction and best-basis search.

Therefore we can easily determine the optimal values using a simple hillclimbing procedure.

Integrated task-specific wavelets and best-basis search for image compression As stated in the previous section, optimal wavelet image compression can be achieved by integrating the process of wavelet construction with best-basis search. The best-basis search using a standard "off-the-shelf" Coifman wavelet ( N f - 12) for four levels is shown in Fig. 11. In this case the threshold cost function used was simply the constant value 0.2. The resulting compression ratio obtained was 9.50. Fig. 12 shows the result when the task-specific wavelet construction is integrated with the best-basis search (with the same threshold cost function). Here, the compression ratio is 9.71, an improvement of 2.2% compared with

474

i ............................... :__~::-_'~_"_:::-:-'Z:',_:;_5"_S??'L::?;I

i ! ............ i ............................

i ~,

i

517; ...........................

Construct TaskSpecific Wavelet

~

i

....

[

,ma e

.......

.................

Data

l

........ i i:iiiiiii!iiiii:=i iii

i i

Evaluate 1 _ Compression Radio ]

Update Wavelet Parameters

Best-basis Search

~

.

' :Wavelet ~Construction

1

Yes

Fig. 10 Integrated wavelet construction and best-basis search.

the standard wavelet. We note that the wavelet basis sub-band block distribution is similar except for the lowest level (the wavelet coefficients in each sub-band block will also be different). The complexity of the search space for wavelet construction is dependent on the choice of the number of parameters, Npar, as this variable determines the dimensionality of the search space. The search complexity is exponential in N p a r , s o the value of N p a r should be kept as small as possible. So, for the case of Nf = 12 and m = 2 that was used in Figs. 11 and 12, we have a dimensionality equal to Npar = 5. Fig. 13 shows a simple search space for Nf = 6 and m - 2 (and, therefore, N p a r = 2) where we have chosen 50 points along each parameter dimension, that is a total of 2500 points on the search hyper-surface. Observe the highly regular and near-symmetric distribution of the search space, enabling the use of simple hill-climbing search algorithms (the parameter values corresponding to the white parts of the surface represent the optimal wavelet parameters). Such regularity and symmetry properties have been observed for a wide range of image typologies

475

Fig. 11 Results of the best-basis search .for a standard C o f m a n wavelet (.(our levels, m = 2, filter length is NI = 12).

476

Fig. 12 Results of the integrated task-specific wavelet construction and best-basis search(four levels, m = 2, filter length Nf = 12).

477

Fig. 13 Search space for task-specific wavelet construction (m = 2 and Nf = 6).

[11]. Furthermore, it is conjectured that these properties of the search space scale up well to higher dimensions (i.e. Npar > 2), thereby significantly reducing the computational time of search.

5

Acknowledgements

The authors would like to thank Dr S. Aeberhard for generating the results for the integrated task-specific wavelet construction and best-basis search.

References 1. I. Witten, R. Neal and J. Cleary, Arithmetic Coding for Data Compression, Computer Practices, 30 (1987), 520-540. 2. J. Ziv and A. Lempel, A universal algorithm for sequential data compression, IEEE Transactions on Information Theoo', 23 (1977), 337-343. 3. Y. Fisher (Ed), Fractal Image Compression." Theory and Application, SpringerVerlag, Berlin, (1994). 4. K. Culik and J. Kari, Image compression using weighted finite automata. Comput. and Graphics, 17 (1993), 305-313. 5. B. Litow and O. de Vel, On Digital Images which cannot be Generated by Small Generalised Stochastic Automata, In Mathematical Foundations of Computer Sci-

478

6. 7. 8. 9. 10.

11.

ence Workshop on Randomised Algorithms (R. Freivalds Ed), RWTH, Aachen, (1998). J. Shapiro, Embedded image coding using zerotress of wavelet coefficients, IEEE Transactions on Signal Processing, 41 (1993), 3445-3462. M. Vetterli and J. Kovacevic, Wavelets and Subband Coding, Prentice-Hall (1995). R. Coifman and M. Wickerhauser, Entropy-based algorithms for best-basis selection, IEEE Transactions on Information Theory, 38 (1992), 713-718. K. Ramchandran and M. Vetterli, Best wavelet packet in a rate-distortion sense, IEEE Transactions on Image Processing, 2 (1993), 160-175. M. Mandal, S. Panchanathan and T. Aboulnasr, Choice of wavelets for image compression, In Information Theory and Applications II, Lecture Notes in Computer Science LNCS 1133, (P. Fortier, J.Y. Chouinard and T. Gulliver (Eds) Springer-Verlag, (1996), 239-249. O. de Vel and S. Aeberhard, Image-specific adaptive wavelet compression, submitted to lEE Journal of Vision, Image and Signal Processing (1998).


479

CHAPTER 20 Wavelet Analysis and Processing of 2-D and 3-D Analytical Images S.G. Nikolov I *, M. Wolkenstein 2 and H. Hutter 2 l Image Communications Group, Centre for Communications Research, University of Bristol, Merchant Venturers Building, Woodland Road, Bristol BS8 1UB, UK; e-mail." stavri.nikolov(~j bristol.ac.uk 2Research Group on Physical Anal)'sis and Computer Based Analytical Chemistry, Institute of Analytical Chemistry, Vienna UniversiO' of Technolog)', Getreidemarkt 9/151, Vienna 1060, Austria," e-mail." mwolken~ mail.zserv.tu~'ien.ac.at, h.hutter(a tuwien.ac.at

I

Introduction

The rapid progress in high technology poses new challenges to analytical chemistry. Besides the development of new or improved techniques, the general trend in the development of analytical methods and instrumentation is to increase the information content extracted from analytical signals and analytical images. By analytical images, here in this chapter, we mean all images acquired by any of the analytical chemistry techniques described below. Many of these new techniques often reach out to the very limits of physics: when individual atoms are observed, or when single ions are detected, or monolayers on the surface of materials are selectively analysed. A wide variety of scientific instruments directly produce images in a form suitable for computer acquisition and computer analysis. The majority oJ these images are two-dimensional (2-D) images. Imaging has played a majoI role for a very long time in biology, chemistry, and physics, if one considers, the widespread use of microscopic techniques like light microscopy or electron microscopy. The most common type of images obtained in microscop5 show the intensity of light, or any other radiation that has come through th~ sample. These images are called transmission images and they are generatec

* Member of the Research Group on Physical Analysis and Computer Based Analytica Chemistry, IAC, Vienna University of Technology, from 1993 until 1996.

480 by techniques such as light microscopy or transmission electron microscopy. In transmission images, the absorption of the radiation at each point is a measure of the density of the specimen along the radiation path. Some radiation energies may be selectively absorbed by the sample, according to its composition. Some other techniques used in analytical chemistry are based on a completely different principle of operation. Images are acquired by scanning devices, where an analysing beam, either radiation or particles, is scanned in a raster pattern over the specimen, and the interaction with the sample is measured by a detector. Examples of such techniques are Electron Probe Microanalysis (EPMA), Secondary Ion Mass Spectrometry (SIMS), Auger Electron Spectrometry (AES) or Confocal Scanning Light Microscopy (CSLM). The instruments used in these techniques provide a time-varying signal, which can be related to the spatial locations on the sample by knowing the scanning speed and parameters. The interaction may differ selectively according to the composition of the analysed spot. A completely different class of analytical techniques, e.g. Atomic Force Microscopy (AFM) or Scanning Tunnelling Microscopy (STM), generate images, where the pixel brightness is used to record distances. Many instruments used in analytical chemistry capture more than one single image. Multiple images may constitute a series of views of the same area using different radiation wavelengths, or they may as well be images of different elemental distributions on the specimen surface. Such images are often called multispectral images. Another scenario is when several different techniques provide complementary information, often in the form of images, about one and the same specimen. The collection of such images is called a multimodality image. A different multiple image case is a time sequence, where the one and the same specimen is imaged in consecutive moments of time. Some of the instruments described above can also produce three-dimensional (3-D) images of the specimen. These images usually comprise series of parallel slices through the specimen. Common methods capable of generating 3-D images are: various serial sectioning methods used in microscopy; Computing Tomography (CT) or Magnetic Resonance Imaging (MRI) used in medicine; or methods such as SIMS, where a collection of images is produced by physically eroding the specimen and capturing 2-D images at different depths.

481 Automated processing and computerised measurement of analytical images can be employed to extract specific information very accurately and reproducibly. The processing of analytical images itself is mainly used for two main purposes: (a) to improve the visual appearance of images for enhanced interpretation by a human observer; (b) to prepare the images for quantitative measurement of the inherent features and structures. As stated before, the main goal of the analytical chemist is the extraction of interesting information from the measured data. The achievement of this goal is often complicated by the presence of noise in the data. Processing digital images to tackle the noise reduction problem is one of the main applications of image processing. Since this problem is common in various fields of science and technology a large number of noise reduction techniques have been proposed. Yet, a severe problem of many of the classical smoothing operations is the loss of resolution. A possible solution to this problem is the use of wavelet de-noising. Wavelet de-noising of analytical images results in increased noise suppression compared to other state-of-theart filtering algorithms, while the most important features in the input image are well preserved. Several publications show that wavelet de-noising produces better reconstruction results when compared to most traditional linear smoothing methods, especially in cases of high spatial variability of the original data. Besides improving the visual perception of the image, the main purpose of any de-noising technique is always to enhance further image processing. Many automated image processing and evaluation methods may lead to very poor results, or may even not be at all applicable, if the noise variance in the image is too high. Therefore, some pre-processing steps have to be performed to reduce the noise and enhance the image quality, and most of all, to enable further processing. In many cases, wavelet pre-processing of the input image results in superior performance of the following processing steps. Such an example is the classification of analytical images, which is significantly improved when wavelet de-noising is applied prior to classification. Another application inherently associated with image de-noising is image compression. Image compression, similar to de-noising, removes unimportant or undesired details from an image, and thus compresses it. There exist a variety of data compression techniques in different application areas, many of which have been well standardised. Data compression techniques can be

482 divided into two groups: lossless and lossy data compression. Examples of |ossy data compression are the Joint Photographic Experts Group (JPEG) and the Motion Picture Experts Group (MPEG) standards for still images and movies, respectively. Data compression is another very successful application of wavelets. Many studies show the superiority of wavelet compression algorithms to other compression methods. Other interesting applications of wavelet analysis are the extraction of features from analytical images. Wavelets can be used for edge detection and texture analysis. Some of the extracted features can be used to align (register) multimodality analytical images. Image registration is the first step in the process of combining the information from the various modalities, i.e. image fusion. With the availability of several different instruments, which are used in everyday analysis of different specimens, in one and the same chemical laboratory, image fusion is becoming a very important and active field of research. Wavelet transform .fusion, or the fusion of images in the wavelet domain, provides us with additional tools to combine analytical images. This chapter describes several applications of wavelet methods to analyse and process analytical images. A short list of online resources on wavelets and wavelet analysis with focus on their application to analytical images is included at the end of the chapter, together with an extensive bibliography on the subject and the names of the software programs used to process the images displayed in this study.

2

The 2-D and 3-D wavelet transform

The one-dimensional (I-D) discrete wavelet transform (DWT) defined in the first part of the book can be generalised to higher dimensions. The most general case has been studied by Lawton and Resnikoff [1]. An N-dimensional (N-D) DWT is described also in [2]. The separable extension of the wavelet transform (WT) to three dimensions, for example, is explained in [2, 3,4]. In this chapter, for simplicity and because of the problems studied, only the theory of the 2-D and 3-D DWT will be outlined, and only separable 2-D and 3-D wavelets will be considered. These wavelets are constructed from one-dimensional wavelets. Separable wavelets are most frequently used in practice, since they lead to significant reduction in the computational complexity.

483 In this chapter a 2-D image refers to any 2-D intensity function I(x,y), where x and y denote spatial coordinates and the value of I at any point (x,y) is proportional to the brightness (or grey level) of the image at that point. Similarly, a 3-D image refers to any three-dimensional intensity function I(x,y,z), where x, y, and z denote spatial coordinates. A digital image is an image I(x,y) or I(x,y,z) that has been discretised in both spatial coordinates and intensity. The theory of the 2-D and 3-D presented in this chapter closely follows [4]. To extend the 1-D wavelet transform to 2-D and 3-D, we have to find the multiresolution approximations of L2(R 2) and L2(R3), where I(x,y) E LZ(R 2) and I(x,y,z)E L2(R3), respectively. Separable versions of the approximations can be defined, where each vector space V2, is decomposed as a tensor product of two identical subspaces of L2(R2), or as a tensor product of three identical subspaces of L2(R3). Let us first study the 2-D case. We can define a 2-D scaling function (I)(x,y) = ~(x)~(y),

(1)

where ~(x) is the 1-D scaling function of V~,. The three wavelets 9

(x,y) -

q~2(x,y) -- ,(x)qb(y),

(2)

tlJ3(x,y) - ~(x)q/(y), can be used to build an orthonormal basis of L2(R2). This orthonormal basis is 2-Jw~j ( x - 2-Jn, y - 2-Jm), 2-Jv~j ( x - 2-Jn, y - 2-Jm),

(3)

2-Jv~j ( x - 2-Jn, y - 2-Jm), where (n,m,j) E Z 3. Here Z denotes the set of integer numbers. Now we will explain how to compute the DWT of a 2-D image, using a pyramidal algorithm, i.e. a filter pyramid with quadrature mirror filters (QMF) L and H. This method is usually used in signal and image processing to compute a DWT. Let the image I(x,y) be a square matrix of dimensions N x N where N is a power of two. The low-pass filter L and the high-pass filter H are applied first to the matrix rows and the output is downsampled by two. This results in two new matrices LrI and HrI (where Fr means that the

484 filter is applied to the matrix rows, while Fc - to the matrix columns) both having dimensions N x (N/2). Next, H and L are applied to the columns of the matrices LrI and HrI resulting in matrices LcL~I, HcLrI, LcHrI and HcHrI, all of dimensions (N/2) x (N/2). The input matrix I is divided into four matrices or channels. These channels are LLI, HLI, LHI, and HHI, when we omit the row and column indexes. The matrix LLI is a smoother copy of the image I, while the matrices HLI, LHI and HHI contain the vertical, horizontal and vertical-horizontal high frequencies. Thus, one band of the DWT is computed. The same procedure continues with the matrix LLI producing another band of the wavelet decomposition, and so on, until a single number, which is the average of the whole original matrix I, is obtained. Fig. 1 illustrates the two-dimensional pyramidal algorithm. Generally, each smoother approximation S2J+,I of I at scale 2j+l is decomposed into a low-pass subimage SzjI (or the LL channel) and into three detail subimages D~jI (HL channel), D~jI (LH channel) and D~jI (HH channel). The corresponding inner products are S2JI - { (I(x,y)* qb2j(-x)*2j (-y))(2-in, 2-Jm) } (n,m)cZ 2 DljI -- { (I(x,Y)**2J (-x)*2J (-Y))(2-Jn, 2-Jm) }(n,m)cZ2 (4) D2I - { (I(x,Y)**2J (-x)*2J (-Y))( 2-jn, 2-Jm) }(n,m)EZ 2 D3.I - {(I(x,y)**2J(-x),2j(_y))(2-Jn ~ 2-Jm)}(n,m)eZ_~" 2J

HLI

HL(LLI)

HHI

HH(LLI)

LHI LH(LLI)

Fig. 1 Two-dimensional pyramidal wavelet decomposition." Each of the four channels & one band of the 2-D WT can be named using the follow&g notation." LLI, LHI, HLI, HHI, where I & the 2-D image, L stands for a low-pass filter, H stands for a high-pass filter, and the filters are applied first along the y direction (right position), and then along the x direction (left position). Three bands of the wavelet decomposition are displayed.

485 To compute the inverse DWT, at each scale 2j+~, we can reconstruct the approximation SzJ+,I of I(x,y) by S2J+'I - Z

(S2JI)m.n(I)(2Jx - n, ZJY - m)

m.n + Z [(D~jI)m.n ~ l (2Jx - n, 2Jy - m) m,n --t-(D~jI)m.n tI/2 (2Jx - n, -+-(D~jI)m.n 1"I~3(2Jx -

2Jy -

m)

n.2Jy - m)].

(5)

Thus, the whole image I(x,y) can be reconstructed from the pyramid by using shifted and dilated versions of the four functions (I), ~ 1 ~ 2 and tI/3 (see Eqs. (1) and (2)). Now we will briefly discuss the 3-D case. Here, we will define a separable version of V2J as a multiresolution approximation of L2(R3). In this case the scaling function can be defined as 9 (x,y,z) = ~(x)~(y)~(z) and the corresponding wavelets as ~pl(x,Y,Z)- r W2(x,y,z)- +(x)qt(y)~(z), ~p3(x,y,z)- qb(x)~(y)~(z), 9 4 ( x , y , z ) - qt(x)~(y)~)(z), ~ 5 ( x , y , z ) - ,(x)(~(y),(z), W6(x,y,z)- qt(x)~/(y)~)(z), ~IJT(x,y,z)- q/(x)/l/(y)/l/(z). Fig. 2 shows one band of the 3-D pyramidal decomposition. Here, a 3-D image (volume) I(x,y,z) is decomposed into eight 3-D subvolumes (channels). Each channel of the decomposition is labelled by a three-letter label, where each letter denotes the filter type (L or H) in the x, y and z direction. The volume is decomposed into a low-pass subvolume S2JI (or the LLL channel) and seven detail subvolumes {DC~I}c__l.....7 (the high frequency channels). The whole process can be repeated with the low frequency subvolume, in order to compute another band of the 3-D wavelet decomposition. Similar to the 2-D

486

Fig. 2 Three-dimensional pyramidal wavelet decomposition. Each of the eight channels in one band of the 3-D W T can be named using the following notation." LLLI, LLHI, LHLI, LHHI, HLLI, HLHI, HHLI, HHHI, where I is the 3-D image, L stands for a low-pass filter, H stands for a high-pass filter, and the filters are applied first along the z direction (right-most position), then along the y direction (middle position), and finally along the x direction (left-most position).

case, we can reconstruct the approximation S2j+,I of I(x,y,z) at scale 2j+l by computing S2j+, I - ~ (S2JI)m,n,k (I)(2Jx - n, 2Jy - m, 2 J z - k) m,n,k + Z [(D~jI)m,n,k ~p1 (2ix _ n, 2Jy -- m, 2Jz -- k) n,m,k + (D~j I) m,n,k V2 2 J x - n, 2Jy - m, 2 J z - k) + (D~jI) m,n,k ~ 3 2Jx -- n, 2Jy -- m, 2 J z - k) D4 ~4 2ix -- n, 2Jy - m, 2 J z - k) + ( 2jI)m,n,k Ds ~ps 2Jx - n, 2Jy - m, 2 J z - k) -+-( 2jI) m,n,k + (O~J I)m,n, k ~Ij6 (2iX -- n 2Jy - m, 2 J z - k) 7 I) m,n,k ~p7 (2Jx_ n, 2 J y - m 2 J z - k)]. --b( D 2j More details about the theory of the 2-D and 3-D wavelet transform can be found in [4].

487

3

Mathematical measures

For assessing the de-noising and compression performance of the WT a quantitative evaluation of the reconstructed or decompressed images was carried out. Two mathematical measures were used to evaluate the output results after applying different filtering algorithms. The mean square error (MSE) is an estimator showing how close the reconstructed image I (e.g. the de-noised image or the decompressed image) is to the original input image I. The MSE is defined as l

M

MSE - M - N ~

N

Z(I(x'Y)-

[(x'Y))2

(6)

x=l y=l or

1

M

MSE - M . N - K Z

N

Z

K

~(I(x,y,z)-

"~

I(x,y,z))-

(7)

x=l y=l z=l

in the 2-D or 3-D case, respectively. The peak signal-tonoise ratio (PSNR) is another figure, which can be derived from the mean square error. The PSNR is defined as follows: PSNR - 10 log

I2max

MSE1/2

(8)

where Imax is the maximum grey level of the image. In this chapter, whenever we refer to the signal-to-noise ratio (SNR) we actually mean the PSNR.

4

Image acquisition

4.1 S I M S images The instrument we used to acquire the SIMS image included in this chapter is a double focusing Secondary Ion Microscope C A M E C A IMS3f, with a typical lateral resolution of 1-3 ~m and a typical depth resolution of 5 nm. An intensive primary beam (primary ion 0 2 , primary beam intensity 2 gA, primary beam energy 5.5 keV) homogeneously illuminates the sample by scanning rapidly over an area of up to 500 x 500 ~tm2. The ion optical system of the mass spectrometer produces a mass-filtered secondary ion image S(x,y) of the surface, which is registered using a CCD camera system (Pulnix TM 760) in combination with a double micro-channel-plate fluorescent screen

488 assembly (Galileo HOT). The camera signal is digitised by an ITI 151 image processor and is stored on the controlling computer [5]. Under the bombardment with the primary ions the surface of the sample is etched. The typical erosion rate is approximately three atomic layers per second. The measurement of the lateral distributions over time allows the determination of the 3-D elemental distributions S(x,y,z), yielding a signal with N chemical dimensions (number of masses or elements measured) and three spatial dimensions.

4.2 EPMA images A JEOL JSM 6400 scanning electron microscope (SEM) and a LINK eXL EDX energy disperse spectrometer were used for this work. A fine electron beam (acceleration voltage 20 kV, working distance 39 mm) is scanned in a raster pattern (512 x 512 scanning steps) over the surface of the sample, producing secondary or backscattered electrons and X-rays. The X-ray images are formed by selecting an energy (energy resolution 20 eV/channel) corresponding to a particular element, and then registering all detected Xrays in an image E(x,y), in which the brightness of each pixel is proportional to the X-ray intensity of the element.

5

Wavelet de-noising of 2-D and 3-D SIMS images

Images captured by analytical techniques are usually noisy. Noisy images may occur because of various reasons, such as counting statistics in the image detector due to a small number of incident particles (photons, electrons, ions) in techniques such as SEM or SIMS, or instability of the light source, or the detector, etc. The noise pattern depends on the phenomena under consideration and the instruments used, with common noise models like Gaussian noise and Poisson noise. De-noising is the process of reconstruction of the underlying original signal from the noisy one, with the objective of removing as much of the noise as possible, while preserving the major signal features.

5.1 De-noising via thresholding Unlike the sine and cosine functions in Fourier analysis, which are localised in frequency but not in time, i.e. a small frequency change in the Fourier transform (FT) produces changes everywhere in the time domain, wavelets are localised both in frequency/scale (via dilations of the mother wavelet),

489 and in time (via translations of the mother wavelet). This leads to a very compact representation of large classes of functions and operators in the wavelet domain. Images with sharp spikes and edges, for instance, are well approximated by substantially fewer wavelet basis functions than sine and cosine functions. In the wavelet decomposition of signals and images, as it was described before, the filter L is an averaging or smoothing filter (low-pass filter), while its mirror counterpart H produces details (high-pass filter). With the exclusion of the last remaining smooth components all wavelet coefficients in the final decomposition correspond to details. If the absolute value of a detail is small and if we omit it (set it to zero), the general picture would not change much. Therefore, thresholding of the wavelet coefficients is a good way of removing unimportant or undesired details from a signal (see Fig. 3). Thresholding techniques are successfully used in numerous data processing domains, since in most cases a small number of wavelet coefficients with large amplitudes preserves most of the information about the original data set. Different thresholding methods like

9 hard thresholding (HT) i~2JI _ { 0 ( D2J I

if DzjI < z if D2JI > ~;

(9)

9 soft thresholding (ST) I~2jI - sign(D2jI)(]D2jI ] - ~)+,

(10)

where (x)+-

x 0

when x _> 0 whenx J and apply the inverse DWT, producing the image estimate I(x,y). This filter in the wavelet domain shrinks the wavelet coefficients to zero. Because the few large wavelet coefficients preserve almost the whole energy of the signal, the shrinkage reduces the noise without distorting much the image features (see Fig. 4). In the reconstructions of images resulting from this algorithm the noise is significantly suppressed, while sharp features in the original are still sharp in the reconstruction [8,10].

5.2 Gaussian and Poisson distributions All measurement data produced by counting single events are characterised by Poisson statistics. Let us have a 2-D SIMS image I(x,y). In this case, we can apply the Anscombe [11] variance-stabilising transformation to the image, i.e. P(x,y) = 2v/I(x,y) + 3/8, and then use the de-noising algorithm proposed above, as if the whole image has Gaussian white noise with cy = 1. As investigations made by Starck et al. [12] show, the variance of the stabilised Poisson image P(x,y) is, from a practical point of view, equal to 1 irrespective of the mean value of I(x,y). However, in cases when the mean value of the Poisson parameter is under 10, a generalisation of the Anscombe formula should be preferred [12].

5.3 Wavelet de-noising of 2-D SIMS images To demonstrate the result of the wavelet shrinkage algorithm, several examples with different measurement time, i.e. SNR, were measured. In Fig. 5 an A1 distribution is displayed. More examples can be found in [8]. Miscellaneous wavelets, including the Haar wavelet, Daubechies and Coiflet wavelets, were tested. The SIMS image shown in Fig. 5 was de-noised using a Coiflet with 3 vanishing moments. The top-right and the bottom-right images in Fig. 5 present close-ups of regions of interest and the corresponding denoised close-ups. Soft thresholding was applied to the wavelet coefficients.

492

Fig. 4 De-noising via wavelet shrinkage." Lena image, 512 x 512 pixels, 256 grey levels (top left); Lena image, close-up (top right); Lena image with additive Gaussian noise, standard deviation ~ -- 10, close-up (bottom left); de-noised Lena image, universal soft thresholding (Coiflet wavelet with 3 vanishing moments), close-up (bottom right).

For assessing the performance of the above-described wavelet de-noising algorithm a quantitative evaluation of the reconstruction was carried out. As figures of merit, the MSE (Eq. (6)) and the SNR (Eq. (8)) were used. Wavelet de-noising was compared with the optimal MSE Wiener filter [2]. Wiener filter reconstructions were calculated using the wiener2 function from the

493

Fig. 5 De-noising of a S I M S image." (a) original #nage, 512 x 512 pixels, 256 grey levels; (b) original image, close-up; (c) de-noised image, universal soft thresholding," (d) denoised image, universal soft thresholding, close-up. Measurement parameters." primary ions." Cs +; primary intensity." 1 nA; primao" beam diameter." 0.3 l~m,"primary ion energy." 6.5 keV; scanning steps." 512 x 512; step width: 0.1 ltm; analytical area." 51.2 x 51.2 l~m; measurement time per pixel." 1 ms; measurement time per #nage." 256 s," detected secondary ions." 27Al-.

MATLAB Image Processing Toolbox [13]. The block size of the Wiener filter was tuned to find the least MSE reconstruction. Since quantification requires true images for comparative reasons, the evaluation was carried out on the basis of a simulated image (Fig. 6), which has simple features, such as rectangular bumps with increasing widths, resembling structures in some real

494

495 SIMS images. A simulated image with Poisson statistics and an SNR = 3.8 was created. The Anscombe transform was applied prior to filtering. The simulated image was processed by both wavelet shrinkage and Wiener filtering. The results obtained from 100 noisy replicates are presented in [8]. We calculated not only the MSE of the whole image but also the MSE of one cross-section (along column 80) and the MSE of the individual bumps. Thus, a better quantification of how features with different widths are reconstructed was achieved. Generally, it can be concluded that wavelets give comparable MSE to the Wiener filter, while in the same time they gain a much better SNR improvement, though mainly due to the fact that they smooth more the background and the top plateaus of the rectangular bumps. In order to prove this last statement we calculated the gradients and measured additionally the reconstruction quality of the bump edges alone. The Wiener filter gives a 5-10% better reconstruction of the bump edges, which confirms the previous statement. Another easily observed trend is that the wider the bumps are, the smaller the wavelet MSE becomes. By comparing the standard deviations of the MSE, it can be seen that the wavelet MSE tends to be more stable than the corresponding Wiener MSE. Another comparison between wavelets and the results of two state-of-the-art adaptive filters (one based on fitting splines with adaptively chosen tension and the other using adaptive truncation of the empirical Fourier series) applied to various artificially generated signals, may be found in [6]. An extensive comparison of wavelet filtering with various other widely used filtering techniques can be found in [16]. Simulated images of point sources and an elliptical galaxy were processed by a wavelet image restoration technique with a multiresolution support in [12]. All investigations clearly show that wavelet shrinkage algorithms produce better reconstructions than most traditional linear smoothing methods, especially in cases of high spatial variability of the original data. In all wavelet reconstructions the noise is efficiently suppressed and most of the image features well preserved after processing.

Fig. 6 Simulated Bumps image, Poisson statistics." (a) original image, 256 x 256 pixels, 256 grey levels, background grey level = 10, bump plateau at grey level 20; (b) noisy image; (c) original image, close-up; (d) noisy #nage, close-up; (e) wavelet de-noised image (Coiflet 3), close-up," (f) Wiener de-noised #nage, 3 x 3 window, close-up.

496 Since a wavelet basis is not unique, finding the optimal wavelet for a specific problem is often a difficult task. Some wavelet properties, such as the smoothness of the wavelet, the number of vanishing moments, etc., may point the right direction to the optimal wavelet. Usually, one has either the option of using a wavelet from a library of wavelets, the way this has been done in this chapter, or he may construct his own wavelets, which have some desired characteristics. Determining the optimal threshold in the de-noising process is usually a result of careful exploration of the data. Multiresolution Analysis (MRA) plots (Fig. 3) reveal the structure of the data at different scales, and thus help the observer in acquiring the best threshold for a certain data set. Some thresholding methods, like universal thresholding, derive the nearly optimal (or optimal in some sense - in the case of universal thresholding the nearly optimal in minimax sense) threshold from the data, provided some initial normalisation conditions are met.

5.4 Wavelet de-noising of 3-D SIMS images The multiscale wavelet transform of a signal contains all frequency information in the different scales of the transform. High frequency information resides in the fine levels and low frequencies in the coarse levels. By analogy with the Fourier transform, the narrower a peak, the higher the frequencies, which are required to describe it. Thus, optimal de-noising depends on the amount of noise and the size and shape of the features of interest. This explains previous publications on de-noising of Gaussian-shaped peaks [14,15,16] reporting the optimal filter width to be between one and two times the full width at half maximum of the data features. Although wavelet denoising via thresholding does not have a parameter such as the filter width, it does have a parameter with similar characteristics, i.e. the level of decomposition of the wavelet transform. The values of this parameter correspond to the filter width parameter of other de-noising filters. The wavelet de-noising algorithm described above for 2-D images, can be extended to process 3-D images [9] using a 3-D DWT. Keeping in mind that SIMS images have a resolution between slices (z axis), which is different from the resolution within one slice (xy plane), optimal de-noising therefore should be accomplished using different coarse levels (levels of decomposition) of the WT for the three spatial dimensions (strictly speaking a different coarse level for the z axis). This assumption is proven by the quantitative evaluation of the reconstruction we carried out. As in the 2-D case, since the quantification

497 of the reconstruction performance of a filter requires true data for comparative reasons, the evaluation was first carried out using a simulated 3-D image comprising several features often found in real 3-D SIMS images such as: low p i x e l intensities, peak areas of vao'ing sizes, edges showing a Gaussian-like shape (as a result of the cross-section of the electron beam). To simulate Poisson noise, each pixel was replaced by a random number, chosen from the Poisson distribution, with the pixel intensity as parameter. After applying the wavelet de-noising, a quantitative evaluation of the filtered images was made using the MSE (see Eq. (7)) as figure of merit. The simulated volume (Fig. 7) used in the quantitative evaluation is a set of 128 images, each of size 256 x 256 pixels, comprising spherical features with different diameters (8 to 38 pixels) and different feature intensities (SNR ~ 0.9 to SNR ~ 5). Three-dimensional SIMS image sets normally have different resolutions within a slice and between slices. The pixel distances within a slice are normally between 1 and 3 Jam, the distance between slices can vary from 10 to 100 nm. These different resolutions were taken into account when sampling the simulated volume and generating the simulated images. Round features were chosen because they show no preferred edge orientation and bear a great resemblance to structures in captured SIMS images. These features were smoothed using a Gaussian weighted filter with a standard deviation of one pixel, which simulates the cross-section of the electron beam. Fig. 7 shows a 'true' 3-D volume and one of the 128 slices. The Poisson noise was created using a MATLAB Statistics Toolbox routine [13].

Fig. 7 Simulated S I M S volume." original 3-D image (left); one representative z slice (right). The whole image set comprises 128 slices, each of size 256 x 256 pixels, 256 grey levels, background grey level = 12, feature ,~rev levels 15 to 30.

498 The optimal level of decomposition within one slice was found to be identical with the one found in a previous publication [16]. The optimal level of decomposition for the z axis is higher, due to the higher resolution between the slices. The results of the evaluation for different combinations of the coarse level (levels of decomposition) of the WT are plotted in Fig. 8. Now the question arises, whether these different resolutions not only demand different coarse levels for optimal de-noising, but also call for the use of different filter banks? To answer this question we de-noised our simulated SIMS volume using all possible combinations of wavelets. The results of this investigation are summarised in Table 1. They show that the assumption above cannot be confirmed. This seems to be in correspondence with the results reported by Wang and Huang [17], which were obtained for the compression of medical images using a separable 3-D wavelet transform.

Fig. 8 Wavelet filter reconstructions using different combinations of the level of decomposition of the WT. M S E is the mean square error between the original and the reconstructed volume.

Table 1. Evaluation of all possible combinations of wavelets.

7

.Y

$ t I

L '4

2

2

s

7

.2

5

t I

L Z

2

2

s

MSE Haar Coiflet 2 Coiflet 3 Daubechies 4 Daubechies 6 Symmlet 6 Symmlet 8 Villasenor 2 Villasenor 6 Antonini

Wavelet (xy-plane) Coij7et 2 Haar 1.341 1 1.0259 1.2199 0.9241 1.2209 0.9201 1.2642 0.9603 1.2324 0.9385 1.2269 0.9258 1.2202 0.9 162 1.2206 0.9 192 1.1850 0.9 182 1.2155 0.9200

MSE Haar Coiflet 2 Coiflet 3 Daubechies 4 Daubechies 6 Symmlet 6 Symmlet 8 Villasenor 2 Villasenor 6 Antonini

Wrrv~lc't(.~j'-plr/nr) Svmmlet 6 Splnilet 8 1.0179 1.01 12 0.9 197 0.9 152 0.9 148 0.9121 0.9557 0.9506 0.9334 0.9296 0.9202 0.9 178 0.9093 0.904 1 0.9 125 0.9066 0.9 133 0.9092 0.9141 0.9084

Daubechies 4 1.1362 1.0275 1.0247 1.0630 1.0399 1.0299 1.0207 1.0224 1.0144 1.0222

Dauhechies 6 1.0512 0.9484 0.9450 0.9825 0.9589 0.9504 0.9360 0.9380 0.9364 0.9387 Antonini 1.025 1 0.9236 0.9200 0.9607 0.9374 0.9253 0.91 19 0.9 142 0.9 173 0.9 163

500 The effect of wavelet de-noising is demonstrated in Figs. 9 and 10. The first of the two figures shows our simulated SIMS volume contaminated with Poisson noise. The noise severely degrades the rendering of the 3-D image. After wavelet de-noising (Fig. 10), the noise in the reconstruction is suppressed to a large extent and most of the image features are only slightly distorted. A real 3-D SIMS image is presented in Fig. 11. The measured specimen is a high-speed steel S 6-5-2 (W 6%, Mo 5%, V 2%). The SIMS volume displayed in Fig. 11, was then de-noised using the filter which gave the best results in

Fig. 9 Simulated S I M S volume." the 3-D image in Fig. 7 has been degraded with Poisson noise: noisy 3-D image (left); one representative z slice (right).

Fig. 10 Simulated S I M S volume: the 3-D image in Fig. 9 has been processed using wavelet de-noising." 3-D reconstruction (left) - c o m p a r e with Fig. 7; one representative z slice (right).

501

Fig. 11 Real S I M S volume." 3-D image (left); one representative z slice (right).

our evaluation of the de-noising performance of the simulated volume, i.e. a Symmlet with eight vanishing moments. The left image in Fig. 12 shows the 3-D reconstruction of the whole de-noised volume, the right image is one representative slice. Again, as with the simulated image, in the reconstruction of the wavelet processed volume the noise is significantly reduced. Additionally, 3-D wavelet de-noising was compared to traditional 2-D wavelet de-noising of separate volume slices. Three-dimensional de-noising

Fig. 12 Real S I M S volume. the 3-D image in Fig. 11 has been processed using wavelet denoising: 3-D reconstruction (left); one representative z slice (right).

502 easily outperforms its 2-D counterpart (Fig. 13) at low additional computational costs, i.e. the de-noising time for 3-D wavelet de-noising is only 50 percent longer than for the 2-D wavelet de-noising.

6

Improvement of image classification by means of de-noising

6.1 Classification 6.1.1 B a s i c s

Classification is a procedure utilised in aerial and medical imaging, as well as in microscopy, in order to extract features of interest from multispectral images. Numerous classifiers such as the k-nearest neighbour classifier [18], neural networks [19] and fuzzy c-mean clustering [20], just to mention a few, have been described and used by the scientific community. Analytical tools like multidimensional histograms [21], scatter diagrams, and concentration histogram imaging [22], have been applied to Scanning Auger Microscopy (SAM) [23,24] and SIMS images [25,26,27].

Fig. 13 Wavelet filter reconstruction using two- and three-dimensional wavelet de-noising. M S E is the mean square error between the original and the reconstructed volume.

503 The inherent presence of noise in analytical images often leads to false clusters in the classified images or to misclassification of some features. This section investigates the extent to which de-noising algorithms improve the subsequent classification of images. Geometric features in digital images, such as texture and shape, lead to pixel populations in coherent clusters and can therefore be treated further by multivariate statistical means to extract information, i.e. image segmentation for correlation of positional data can be performed [24]. A following step is the classification of the image features. In order to identify different objects in feature space, it is necessary to establish their frequency distribution. The resulting clusters ideally represent the relationship of the constituents in the original image. Each picture element can be assigned to an object employing certain classification strategies. Some classification algorithms, which have been used for the classification of analytical images, are neural networks and fuzzy c-means clustering [28]. 6.1.2 Scatter diagrams A scatter diagram is used to represent the frequency distribution of grey levels, which point out the position of the objects in two-dimensional space [21,23,25]. Scatter diagrams for higher dimensional space can also be computed [22]. Fig. 14 demonstrates the construction principle for a two-dimensional scatter diagram. Many pixels in these diagrams tend to pile up at the same spots as they possess the same relative frequency distribution of grey levels in both input images. Therefore, the scatter plot allows the determination of pixel clusters, outliers and gradients in terms of their density. Classification of images of analytical samples assigns each picture element of the image to a chemical phase, which is of great importance for several major analytical techniques such as EPMA and SIMS. Fig. 15 displays two elemental distributions of a soldered industrial metal sample acquired with SIMS and a classified image showing the different chemical phases of the sample. Fig. 16 shows two noiseless images and the same images with added Gaussian noise. The corresponding scatter diagrams and the classified images are shown as well. 6.2 Results The scatter plot allows the determination of pixel clusters (representing single sample phases), outliers and gradients in terms of their density. These clusters can be separated using different classification strategies [28,20]. Then, the separated picture elements are projected back onto the original images to

504

Fig. 14 Schematic construction of a scatter diagram." image A (top left); image B (bottom left); scatter diagram (right). The pixels at location (205,37) have intensities 41 and 21, respectively, and therefore, histogram bin (41,21) is incremented.

display a new classified image. Fig. 17 shows the application of wavelet denoising and subsequent classification of two SIMS images. The specimen is a soldering alloy used to join steel and chromium. The solder material is a nickel base alloy (Cr 7.0%, Fe 3.0%, B 3.0%, Ni 82.5%, Si 4.5%, B 3%) in the form of a foil. The scatter plot of the noisy images in Fig. 17 (middle left) shows four distinct peaks, where the clusters are overlapping due to the noise, and therefore class assignment is uncertain. The classified image shows many misclassified pixels and the phase boundaries are blurred. De-noising decreases the noise variance. Thus, it both reduces the extension of the clusters in the scatter plot and increases their separability, which consequently improves the classification performance (Fig. 17, bottom). Generally, de-noising the images prior to classification substantially improves the classification results [29,30]. The number o f misclassified pixels ( M C P ) rapidly decreases when Fourier or wavelet de-noising filters are ap-

505

Fig. 15 Classification of two elemental distributions. ion micrograph of B (top l e f t ) ion micrograph of Cr (top right)" scatter diagram (bottom left)" class(fi'ed image (bottom right). Measurement parameters." primary ions. O+" primar)" beam intensity." 2 laA" pri mary beam energy 5.5 ke V," scanned area." 500 x 500 Ira1,"anah'sed area diameter." 400 lira.

plied prior to classification. In [30] it was concluded, that the self organising map ( S O M ) classifier and the f u z z ) ' c-means clustering classifier show the same trends when investigating the effect of de-noising on the following classification. The better the reconstruction is with respect to the MSE, the smaller the number of misclassified pixels. Although this is generally true, our study of sub-optimal Fourier filtering (FF) showed, that some commonly used methods for determination of the FF cut-off frequency, such as the Kirmse algorithm, do not necessarily lead to optimal classification. A further investigation in the general framework of multiscale methods [12] and scale-space theory [31,32] may be classification at different scales. If for some types of images classification gives the same or very similar results at several scales, then it may be performed on smaller, smoothed copies of the

506

Fig. 16 Simulated noiseless and noisy images, together with the corresponding scatter plots and classified images: the two noiseless images having two intensity value areas (60 and 70) (top). The scatter plot shows four clearly separated clusters. The same images as on the top, but with additive Gaussian noise (var = 20) (bottom). The four clusters in the scatter diagram are not discernible, and therefore automatic classification fails.

original images. This will considerably diminish the computation time for classification, which is of great importance, especially in the case of nonlinear classifiers such as neural networks.

7

Compression of 2-D and 3-D analytical images

7.1 Basics

In chemical analysis, many types of instruments now provide far more information than the integrated properties of a homogeneous sample. The correlation of local spatial and chemical information produces pictures revealing the composition and structure of non-homogeneous samples. In material science new imaging techniques lead to 3-D structural representations of material objects and their inner structure. Some of these advanced analytical methods, e.g. SIMS, are capable of producing series of 2-D sections, resulting in 3-D spatially resolved information about element distributions of signifi-

507

Fig. 17 Classification of two SIMS images." noisy Si image (top left); noisy Ni image (top right); scatter diagram of the original noisy images (middle left); classified image from the original noisy images (middle right); scatter diagram of the reconstructed images (bottom left); classified image from the reconstructed (wavelet de-noised) images (bottom right). De-noising prior to classification leads to significantly better classification results.

cantly large and representative volumes (106 lain3) in a relatively short time (about 1 h). However, very often, large amounts of data are obtained. SIMS images are typically digitised at a minimum resolution of 256 • 256 pixels with 16 bits per pixel. A single 2-D image, therefore, occupies at least 0.125 MB of storage space. A typical 3-D SIMS image set consists of 64 slices, thus requiring a minimum of 8 MB of storage. If we take into account that SIMS instruments capture images of four to eight 3-D distributions simultaneously, this results in 32 to 64 MB of data for only one analysis of one specimen.

508 Obviously data compression is important to bring these numbers down and make 3-D SIMS image analysis and processing a more manageable task. Several methods have been proposed to reduce the storage space for image data. The discrete cosine transform (DCT), which is the basis for the JPEG standard, has been widely used for still image compression. Although it can be efficiently implemented and it performs well for high bit-rate compression, serious blocking artefacts are a well-known disadvantage of DCT-based coding. As an alternative transform, the discrete wavelet transform not only can overcome the blocking artefacts, but also can achieve better overall performance in most cases. An effective approach to data compression using wavelets was introduced by Wickerhauser [33]. Data coding is one of the most visible applications of wavelets. Compression ratios of about 10:1 can be achieved without significant loss of visual detail. The FBI has adopted a standard for digital fingerprint image compression, based on wavelet compression algorithms (see Fig. 18). This standard is described in the work of Bradley [34] and Brislawn and Hopper [3 5,36]. Compressing an image set with multiple slices is different from compressing only a single 2-D image. Multiple slices are normally correlated to each other. In other words, there are some structural similarities between adjacent slices. Although it is possible to compress an image set slice by slice, more efficient compression can be achieved by exploring the correlation between slices. In

Fig. 18 FBI-digitised left thumb fingerprint." original image (left)" compressed image with compression ratio of 26:1 (right). Figure courtesy of Chris Brislawn, Los Alamos National Laboratory.

509 this section, a separable 3-D DWT with varying wavelet banks is applied to compress a 3-D SIMS image.

7.2 Quantisation After the DWT of the image has been computed, the second step in the image compression process is quantisation. The purpose of quantisation is to reduce data entropy by compromising the precision of the data. The quantisation step maps a large number of input values into a smaller set of output values. This step is not invertible, thus it introduces the so-called quantisation noise. Therefore, the original data cannot be recovered exactly after quantisation. Hence, it is very important to design a quantisation strategy which selectively quantises the wavelet coefficients and preserves the image quality. In the wavelet decomposition of signals and images, as it was described in the previous sections, the filter L is a low-pass filter while its mirror counterpart H is a high-pass. Wavelet transformed data thus consists of two types of channels: a single low-resolution channel, which contains most of the energy, and multiple high-resolution channels, which contain the edge information. All transformed data is represented by floating point values. A quantiser is operating on these channels to produce a sequence of symbols. From an MSE perspective, the minimum entropy is approximately achieved for a given distortion by uniform quantisation [37]. However, if the quantiser is applied to the high frequency channels, where the sample values are often small, the coding efficiency can be improved using a larger quantisation interval around zero. In this study, we have used the embedded family of quantizers described in [38].

7.3 Entropy coding After quantisation the channels with discrete levels are represented by integers. In the third step this data is further entropy coded to reduce the bit rate. Entropy coding assigns fewer bits to integers with higher frequency of occurrence and more bits to integers with lesser frequency of occurrence. This fully invertible step allows us to represent the data in even less space than the original data after quantisation. For a detailed description see [39].

7.4 Results The measured specimen is a high-speed steel S 6-5-2 (W 6%, Mo 5%, V 2%). Two different test volumes, i.e. 3-D SIMS images of the sample, were re-

510 corded. One of them is displayed in Fig. 19. The definition of a cutting function produces a better view of the internal structures and the intensity distribution inside the overall volume [40]. Fig. 20 shows the PSNR versus the compression ratio for both test volumes. Even at very low bit rates, i.e. high compression ratios, relatively high PSNR

Fig. 19 3-D S I M S image." the whole image set consists of 64 2-D images (256 • 256 pixels), each having 256 grey levels.

Fig. 20 3-D S I M S image compression at different compression ratios, where PSNR is the peak signal-to-noise ratio.

511 can be achieved. To compare 3-D wavelet compression with 2-D image compression methods, both tested volumes were compressed using: (a) 3-D wavelet compression; (b) 2-D wavelet compression method, and (c) standard 2-D JPEG compression. More details about the results reported in this section can be found in [41]. The 2-D wavelet compression algorithm used was similar to the 3-D compression algorithm except that the 2-D WT of each slice was computed. Multiple slices were compressed slice by slice with the 2-D method. The same applies for the compression using the JPEG algorithm. Fig. 21 shows the PSNR (Eq. (8)) versus the compression ratio of the 3-D and 2-D methods. We only show the results for one of the test volumes here. The results for the second volume are very similar. The compression ratio of the 3-D wavelet method is much higher than that of the 2-D wavelet method at comparable PSNR. For a very low compression ratio the JPEG algorithm yields a slightly higher PSNR, but for higher compression ratios 3-D wavelet compression easily outperforms both 2-D methods. Fig. 22 presents the original and decompressed images of the first image set at a ratio of 1:32, using the 2-D and 3-D compression methods. At the same compression ratio Fig. 22 shows very little difference between the 3-D wavelet decompressed image and the original, whereas both 2-D methods

Fig. 21 3-D S I M S image compression using two- and three-dimensional wavelet compression and JPEG compression.

512

Fig. 22 3-D SIMS image compression using two- and three-dimensional wavelet compression and JPEG compression." original image (top left)," JPEG compression (top right); 2-D wavelet compression (bottom left); 3-D wavelet compression (bottom right).

reveal some clearly visible artefacts. Tables 2 and 3 in [41] summarise the results of applying the different wavelet filters at a compression ratio of 1:32. Three-dimensional SIMS images normally have different resolutions within a slice and between slices. As already mentioned in the de-noising section, Wang and Huang [17] used a separable 3-D WT for compression of medical images and proposed to apply a second wavelet filter bank in the slice direction to take into account the different correlation between the slices and within one slice. They concluded that in general this gives better results only if the distance between slices is much greater than the pixel distance within a slice. Since in the case of SIMS volumes the distance between slices is much lower than the lateral resolution within a slice the application of a different wavelet filter in the z direction should not yield a better performance. Although this assumption is not proven by the quantitative evaluation of the reconstruction we carried out, the results for a combination of different filters do not significantly differ from the results when using the same filter in all directions.

513

Table 2. Advantages and disadvantages of EPMA and SIMS.

Advantages

Disadvantages

8

EPMA Good quantification and high resolution, particularly for electron signals (SE, BSE).

SIMS High local detection power (typically 3.5}5- the thresholded wavelet gradient maxima of the EPMA image at scale 23 (top left); {A23S(x,y) > 2} - the thresholded wavelet gradient maxima of the S I M S image at scale 23 (top right); the EPMA image E(x,y) (middle left)," the S I M S image S(x,y) (middle right); the corrected (transformed) EPMA image E(x,y) (bottom right). The dark crosses mark the positions of the control points {Pk}k-i.....Q. In this case Q = 15. The selection of the control points was made manually using the top row images.

less distorted than the SIMS image. Again, the same method as above can be used to compute the corrected SIMS image S(x,y) and to compare and combine it with E(x,y). 9.2 I m a g e f u s i o n 9.2.1 Basics

The successful fusion of images acquired from different modalities or instruments is of great importance in many applications, such as medical imaging, microscopic imaging, remote sensing, computer vision, and robotics.

536

Image fusion can be defined as the process by which several images, or some of their features, are combined together to form a single image. Let us consider the case where we have only two original images I1 and I2, which have the same size and which are already aligned using some image registration algorithm. Our aim is to combine the two input images into a single fused image I. Image fusion can be performed at different levels of the information representation. Four different levels can be distinguished according to [73], i.e. signal, pixel, feature and symbolic levels. To date, the results of pixel level image fusion in areas such as remote sensing and medical imaging are primarily intended for presentation to a human observer for easier and enhanced interpretation. Therefore, the visual perception of the fused image is of paramount importance when evaluating different fusion schemes. In the case of pixel level fusion, some generic requirements can be imposed on the fusion result: (a) the fused image should preserve, as closely as possible, all relevant information contained in the input images; (b) the fusion process should not introduce any artefacts or inconsistencies, which can distract or mislead the human observer, or any subsequent image processing steps [74]; (c) in the fused image, irrelevant features and noise should be suppressed to a maximum extent. Pixel level fusion algorithms vary from very simple, e.g. image averaging, to more complex, e.g. Principal Component Analysis (PCA), pyramid based image fusion and wavelet transform fusion. Several approaches to pixel level fusion can be distinguished, depending on whether the images are fused in the spatial domain (spatial domain image fusion), or they are transformed into another domain, and their transforms are fused (frequency domain image fusion or image transform fusion). After the fused image is generated, it may be further processed and some features of interest may be extracted.

9.2.2 Wavelet transform image fusion The general idea of all wavelet based image fusion schemes is that the wavelet transforms of the two registered input images I1 and I2 are computed and these transforms are combined utilising some kind of fusion rule ~ (see Fig. 37). Then, the inverse wavelet transform W~ 1 is computed, and the fused image I is reconstructed by I -- W~ 1(z[W2JI1, W2JI2]) ,

(33)

where W2JI 1 and W2jI2 are the wavelet transforms of the two input images. The fusion rule Z is actually a set of fusion rules ; ( - ~j which defines the fusion of each pair of corresponding channels c for each band j of the wavelet transforms (Fig. 38).

537

IDwT. l ...... I

':l

I

IDWT~U

Iow l [

registered inputimages

II

I

~'~

cfUoSect~V~eelet ant

fused image

wavelet coefficients

Fig. 37 Fusion of the wavelet transforms of two 2-D images.

HLI,

HHI,

HLI2

HHI:

+ HL(LLI,) HH(LLI,]

HL(LLI:) HH(LLI~]

LHI, LH(LLI,)

LHI2 LH(LLI:)

Fig. 38 Fusion of the different bands and channels of the WT of two 2-D images.

The wavelet transform offers several advantages over similar pyramid-based techniques, when applied to image fusion: (a) the wavelet transform is a more compact representation than the image pyramid. This becomes of very great importance when it comes to fusion of 3-D and 4-D images. The size of the WT is the same as the size of the image. On the other hand, the size of a Laplacian pyramid, for instance, is 4/3 of the size of the image; (b) the wavelet transform provides directional information, while the pyramid representation does not introduce any spatial orientation in the decomposition process [75]; (c) in pyramid based image fusion, the fused images often contain blocking effects in the regions where the input images are significantly different. No such artefacts are observed in similar wavelet based fusion results [75]; (d) images generated by wavelet image fusion have better SNR than images generated by pyramid image fusion, when the same fusion rules are used [76]. When subject to human analysis, wavelet fused images are also better perceived according to [75,76]. A number of wavelet based techniques for fusion of 2-D images [77,78,75,71,76,74] and 3-D images [79] have been proposed in the literature.

538 Some of the fusion rules, which can be used to combine the wavelet coefficients of two images, are presented below:

1. fusion by averaging [71] - for each band of the decomposition and for each channel the wavelet coefficients of the two images are averaged;

2. fusion by maximum [75,71,74] - for each band of the decomposition and for each channel, the maximum of the absolute values of the respective wavelet coefficients is taken; 3. high~lowfusion [71] - the high frequency information is kept from one of the images while the low frequency information is taken from the other; composite fusion [79] - various combinations of the different channels and bands of W~I~ and W2jI2 are composed; 5. de-noising and fusion [79]- the wavelet coefficients of the high frequency channels are thresholded by either hard or soft thresholding, and then are combined by using some of the other fusion rules; 6. fusion of the WT maxima [79] - the WT maxima (the multiscale edges of the image) can be linked to construct graphs. These graphs can be combined instead of combining all the wavelet coefficients. This fusion technique is based on the multiscale edge detection results reported in [44]. ,

Several other 2-D WT image fusion algorithms have been published, which are based on some of the principles of visual perception, e.g. fusion using an area-based selection rule with a consistency verification [75] or contrast sensitivity fusion [76]. Since these methods have been designed specifically to enhance the perception and interpretation of fused 2-D images, their threedimensional analogues are difficult to construct. An extensive study of the application of wavelet transform fusion to 2-D SIMS images can be found in [80]. Here, only two examples of wavelet transform image fusion are included. In each case three 2-D SIMS images, i.e. a multispectral SIMS image, are combined to produce a single image (see Figs. 39 and 40). In both figures the input images display a steel alloy containing about 1% A1. The alloy was produced by hot isostatic pressing and is intended to show better high-temperature oxidation resistance, as well as a more homogeneous distribution of all alloy compounds, when compared to conventionally treated materials. In Fig. 40, weighted averaging in the wavelet domain results in an improved combination of the input images than simple averaging in the spatial domain. Some of the fusion rules proposed

539

Fig. 39 Wavelet transform fusion of 2-D SIMS images. Input images at location A." mass 12 (C) (top left); mass 43 (AlO) (top middle); mass 68 (CrO) (top right). The fused image, produced by fusion by averaging in the wavelet domain (Daubechies 4 wavelet), is shown below. The negatives of the input images and the fused image are displayed. All three input images are of a steel alloy produced by hot isostatic pressing and containing about 1% Al. Measurement parameters for all 3 input images." a Cs + primary ion beam (primary energy: 5.5 k V, primary ion current." 2 pA) was applied to sputter the sample; diameter of the imaged surface area: 150 tlm. The images shown in this figure and in Fig. 40 were kindly provided by Thomas Stubbings, Vienna University of Technology.

above have been compared visually in [80] to find the best fusion results for a large class of multispectral SIMS images. Fig. 41 shows the fusion of two 3-D phantom images, i.e. a solid cube and a 3-D texture, which consists of a grid of lines parallel to the three axes. Crosssections of the input images and the fused image are also displayed in Fig. 41. Some multimodality or multispectral images are made up of both smooth and textured regions. Such images can be segmented in terms of smooth and textured regions by analysing their wavelet transforms as already described in this chapter, and depending on each pair of regions to be combined (i.e. smooth with smooth region, smooth with textured region, textured with textured region) different fusion rules can be used. A very important advantage of 3-D WT image fusion over alternative image fusion algorithms is that it may be combined with other 3-D image processing algorithms working in the wavelet domain, such as: 'smooth versus

540

Fig. 40 Wavelet transform fusion of 2-D S I M S images. Input images at location B." mass 26 (CN) (top left); mass 43 (AIO) (top middle)," mass 68 (CrO) (top right). Two fused images, produced by averaging in the spatial domain (bottom left) and weighted averaging in the wavelet domain (Daubechies 4 wavelet) (bottom right) are also shown. The negatives of the input images and the fused images are displayed. All three input images are of the same steel alloy as in Fig. 39, only a different location is being imaged. The measurement parameters are also the same as for the images in Fig. 39.

textured' region segmentation [52]; 3-D image registration based on features extracted in the wavelet domain; volume compression [3,4], where only a small part of all wavelet coefficients are preserved; and volume rendering [3,4], where the volume rendering integral is approximated using multiresolution spaces. The integration of 3-D WT image fusion in the broader framework of 3-D WT image processing and visualisation will bring new benefits to the research community and will show the full potential of wavelet based methods in volumetric image analysis.

10

Computation and wavelets

All computations were performed on Pentium II PCs, running Windows 95/ 98/NT, or on a Silicon Graphics (SGI) Power Challenge L computer and an SGI 02 computer, running IRIX 6.2. A number of standard MATLAB [13] toolboxes, e.g. the Statistics Toolbox, the Image Processing Toolbox, and the Fuzzy Logic Toolbox, were used to obtain some of the results in this chapter. Several wavelet toolboxes for MATLAB were also used, including WaveLab

541

Fig. 41 Wavelet tran,~florm (Daubechies 4 wavelet)fusion o[two 3-D phantom #nages: first input image (top le[?) and second input image (top right). Cross-sections at v = 32: first input image (second le['t), second input h~zage fsecond right]; fused image using averaging (third left)," fused image using weighted avera~hlg (z~ = 0.75) (third right); fused image using maximum (bottom l~:ft),'./itsed hnage usitzg the high frequency wavelet coefficients of the second #nage and the lowJi'equencv wavelet coe~cients of the first image (bottom right). In all cases the .[itsion was done in the ~ravelet doma#l. Vohmle size." 64 x 64 x 64 voxels.

[81] and WaveBox [82]. In addition, the Wavelet Workbench [83] toolbox for IDL, which is actually WaveLab ported to IDL, and the Wavelet Fusion Toolbox for IDL [84], were used to do some of the calculations on SGI computers. The SOM package [85], originally written by Teuvo Kohonen

542 and co-workers, was ported to Windows by the authors and used to classify some of the images. A library of functions for calculation of scatter diagrams and statistical measures was written by the authors. Some of the figures in this chapter were generated by two image processing programs, e.g. MIE [86] and X-image [87], which were developed by the authors. Some of the filtering algorithms were implemented in C and C ++ as stand-alone programs. The quantisation and entropy coding algorithms, used for the wavelet compression of 2-D and 3-D SIMS images, were adapted from the Wavelet Image Compression Construction Kit written by Geoff Davis. Finally, the wave2 program [88], developed by Mallat et al., was used on SGI machines to compute the wavelet transform maxima (multiscale edges) of the SIMS and EPMA images presented in this study. Various mother wavelets, e.g. Haar, Daubechies 4, 6, 8, 12 and 20, Symmlet 6 and 8, Coiflet 2 and 3, Villasenor 1 to 5 and Antonini, spline wavelets, and different levels of decomposition, were employed in the computations.

II

Conclusions

In this chapter we have summarised some recent results considering the application of wavelet analysis and processing to 2-D and 3-D analytical images. We have introduced the reader to the theory of the 2-D and 3-D separable discrete wavelet transform. Several applications of algorithms based on the wavelet transform have been described: (a) wavelet de-noising of 2-D and 3-D SIMS images; (b) improvement of analytical image classification by means of wavelet de-noising; (c) compression of 2-D and 3-D analytical images; (d) feature extraction from analytical images, i.e. edge detection and texture analysis in the wavelet domain; (e) registration of images obtained by complementary techniques (EPMA and SIMS), based on their wavelet transform maxima (multiscale edges); (f) wavelet fusion of multispectral SIMS images. Throughout the chapter, parallels to applications of wavelet techniques in microscopy and medical imaging have been included and some ideas for future research have been outlined. All these successful new applications clearly show that wavelet methods have already found their place in analytical chemistry in general, and in analytical image processing in particular. In the near future, the integration of some of the above-described techniques, within the general framework of wavelet analysis, will certainly lead to improved interpretation and quantification of analytical images.

543 12

Acknowledgements

The research presented in this chapter was supported by the Austrian Scientific Research Council (projects $5902, $6205), the Jubilee fund of the Austrian National Bank (Project 6176), and the Austrian Society for Microelectronics. The authors would like to thank: Wen Liang Hwang, Stephane Mallat, and Sifen Zhong, from New York University, for designing, writing and providing the wave2 program; David Donoho and his team from Stanford University, for writing and providing the WaveLab toolbox for Matlab; Amara Graps and Research Systems Inc. for porting WaveLab to IDL; and Geoff Davis for providing the Wavelet Image Compression Construction Kit. Special thanks also go to Hans du Buf, Nishan Canagarajah, and Thomas Stubbings, for some of the images used in the chapter.

13

Online resources

Following is a list of online resources on wavelets, wavelets and image processing, and wavelet analysis of analytical images. This list is by no means a complete, or even a comprehensive collection, of all the online resources available on these topics. The aim of compiling such a list is to provide the reader of this chapter with a good starting point for his further investigations in the field of wavelet analysis and processing of analytical images. This collection of online resources was made at the time of publication of the book. A more up-to-date version of the list can be found at: www.iac.tuwien.ac.at/webfac/WWW/waveletbook.html

13.1 Wavelets and wavelet analysis www.wavelet.org (The Wavelet Digest- a free monthly newsletter edited by Wim Sweldens) www.amara.com/current/wavelet.html (Amara's wavelet page) www.mat.sbg.ac.at/~uhl/wav.html (Andreas Uhl's wavelet page) www.mame.syr.edu/faculty/lewalle/tutor/tutor.html (Jacques Lewalle's tutorial on wavelet analysis of experimental data)

544 paos.colorado.edu/research/wavelets (Christopher Torrence's practical guide to wavelet analysis) www.mathsoft.com/wavelets.html (The MathSoft wavelet resources page with links to many reprints available as PostScript files on the Internet)

13.2 Wavelets and image processing www.multiresolution.com (Image and data analysis: The multiscale approach (book) + MR/1 software) www.cs.nyu.edu/cs/faculty/mallat (St6phane Mallat group at New York University) www.summus.com/publish/wavelets/wavelet.htm (Summus' image compression technology using wavelets) research.microsoft.com/~geoffd (The Wavelet Image Compression Construction Kit) zeus.ruca.ua.ac.be/VisionLab/wta/wta.html (wavelets for texture analysis) vivaldi.ece.ucsb.edu/projects/registration/registration.html (multisensor image fusion using the wavelet transform) www.eecs.lehigh.edu/~zhz3/zhz3.htm (Zhong Zhang's web page about registration and fusion of multisensor images using the wavelet transform) www. fen.bris.ac.uk/elec/research/ccr/imgcomm/fusion.html (image fusion using a 3-D WT)

13.3 Wavelet analysis and processing of analytical images www.iac.tuwien.ac.at/webfac/WWW/cobac.html (wavelet processing of 2-D and 3-D analytical images) www.iac.tuwien.ac.at/~cmitter/wavebib.html (Christian Mittermayr's bibliography on wavelets in analytical chemistry)

545 About the authors Stavri Nikolov was born in Sofia, Bulgaria, in 1967. At present he is a

researcher in medical imaging at the Image Communications Group, Centre for Communications Research, University of Bristol, UK. He obtained an MSc in computer science from ~St. Kliment Ohridski' Sofia University and a PhD (Microscopy Image Processing) from Vienna University of Technology in 1992 and 1996, respectively. His research interests include wavelet image analysis, medical image analysis, image fusion, and volumetric data processing and visualisation. He is currently developing new algorithms for fusion of medical images and for automatic navigation in volumetric images. He is member of the British Machine Vision Association and associate member of IEEE. Martin Wolkenstein was born in Vienna, Austria, in 1971. He received his

diploma and PhD degrees in chemistry from Vienna University of Technology, in 1996 and 1998, respectively. In 1995 he joined the Physical Analysis Group at the Institute of Analytical Chemistry, Vienna University of Technology, where he worked on the application of various image processing techniques, i.e. neural networks, fuzzy logic and wavelets, to analytical data. His main research interests are 3-D microscopy, imaging, image processing, and data visualisation. He is currently serving his military service at the NBC defence school in Vienna, Austria. Herbert Hutter was born in Bregenz, Austria, in 1962. He received his

diploma in physics and his PhD degree in analytical chemistry from Vienna University of Technology, in 1985 and 1990, respectively. Since 1992 he is the head of the Physical Analysis Group at the Institute of Analytical Chemistry, Vienna University of Technology. His main field of research is the characterisation of trace element distributions in materials with Secondary Ion Mass Spectrometry (SIMS). In addition, he and his group have made contributions to various methods for data analysis of multispectral three-dimensional distributions. His main research interests are material science, 3-D microscopy, and image processing. Current projects in close cooperation with several industrial partners investigate the influence of trace elements on material characteristics.

546

References 1. W.M. Lawton and H.L. Resnikoff, Multidimensional Wavelet Bases, Aware Report AD910130, Aware Inc., Cambridge, MA, 1991. 2. W.H. Press, S.A. Teukolsky, W.T. Vetterling and B.P. Flannery, Numerical Recipes in C: The Art of Scientific Computing: Second Edition, Cambridge University Press, Cambridge, UK, 1992. 3. S. Muraki, Approximation and Rendering of Volume Data Using Wavelet Transforms, Proc. Visualization '92, 1992, pp. 21-28. 4. M. Gross, Visual Computing, Springer-Verlag, Berlin, 1994. 5. H. Hutter and M. Grasserbauer, Three-Dimensional Stereometric Analysis of Materials with SIMS, Mikrochimica Acta, 107 (1992), 137-148. 6. D.L. Donoho, Nonlinear Wavelet Methods for Recovery of Signals, Densities and Spectra from Indirect and Noisy Data, in: Different Perspectives on Wavelets, Proceedings of Symposia in Applied Mathematics, American Mathematical Society, 47 (1993), 173-205. 7. D.L. Donoho and I. Johnstone, Ideal Spatial Adaptation Via Wavelet Shrinkage, Technical Report, Department of Statistics, Stanford University, Stanford, 1992. 8. S.G. Nikolov, H. Hutter and M. Grasserbauer, De-Noising of SIMS Images via Wavelet Shrinkage, Chem. and Intell. Lab. Systems, 34 (1996), 263-273. 9. M.G. Wolkenstein and H. Hutter, De-noising Secondary Ion Mass Spectrometry Image Sets Using A Three-Dimensional Wavelet Transformation, submitted to Analytical Chemistry (May 1998). 10. M. Wolkenstein, Optimization of the Visualization Process for Three-Dimensional Analytical Data, PhD Thesis, Vienna University of Technology, Vienna, Austria, 1998. 11. F.J. Anscombe, The Transformation of Poisson, Binomial and Negative-Binomial Data, Biometrika, 15 (1948), 246-254. 12. J-L. Starck, F. Murtagh and A. Bijaoui, Image Processing and Data Analysis: The Multiscale Approach, Cambridge University Press, Cambridge, UK, 1998. 13. MATLAB 5.0, The Mathworks Inc., Natick, MA. 14. C.G. Enke and T.A. Nieman, Signal-to-Noise Ratio Enhancement by LeastSquares Polynomial Smoothing, Analytical Chemistry, 48 (1976), 705a-712a. 15. C.R. Mittermayr, S.G. Nikolov, H. Hutter and M. Grasserbauer, Wavelet Denoising of Gaussian Peaks: A Comparative Study, Chem. and Intell. Lab. Systems, 34 (1996), 187-202. 16. M. Wolkenstein, H. Hutter, S.G. Nikolov, I. Schmitz and M. Grasserbauer, Comparison of Wavelet Filtering with Established Techniques for EPMA Image De-Noising, J. Trace and Microprobe Techniques, 15 (1) (1997), 33-49. 17. J. Wang and H.K. Huang, Medical Image Compression by Using Three-Dimensional Wavelet Transformation, IEEE Trans. Med. lmag., 15 (4) (1996). 18. T. Cover and P. Hart, Nearest Neighbor Pattern Classification, IEEE Transactions on Information Theory, IT- 13 (1967), 21-27. 19. T. Kohonen, Self-Organization and Associative Memory: Third Edition, SpringerVerlag, Berlin, 1989.

547

20. J.C. Bezdek and P.F. Castelaz, Prototype Classification and Feature Selection with Fuzzy Sets, IEEE Transactions on Systems, Man, and Cybernetics, SMC-7 (1977), 87-92. 21. D.S. Bright, D.E. Newbury and R.B. Marinenko, Concentration-Concentration Histograms: Scatter Diagrams Applied to Quantitative Compositional Maps, in: Microbeam Analysis, (D.E. Newbury, Ed), San Francisco Press Inc., 1988. 22. D.S. Bright and D.E. Newbury, Concentration Histogram Imaging, Analytical Chemistry, 63 (4) (1991), 243-250. 23. M.M. E1 Gomati, D.C. Peackock, M. Prutton and C.G. Walker, Scatter Diagrams in Energy Analysed Digital Imaging: Application to Scanning Auger Microscopy, Journal of Microscopy, 147 (1987), 149-158. 24. S.D. Boehmig and B.M. Reichl, Segmentation and Scatter Diagram Analysis of Scanning Auger Images- A Critical Comparison of Results, Fresenius Journal of Analytical Chemistry, 346 (1993), 223-226. 25. D.E. Newbury and D.S. Bright, Concentration Histogram Images: A Digital Imaging Method for Analysis of SIMS Compositional Maps, in: Secondary Ion Mass Spectrometry (SIMS VII), (A. Benninghoven et al., Eds), Wiley, NY, 1990, pp. 929-933. 26. H. Hutter and M. Grasserbauer, Chemometrics for Surface Analysis, Chemometrics and Intelligent Laborato O, Systems, 24 (1994), 99-116. 27. C. Latkoczy, H. Hutter and M. Grasserbauer, Classification of SIMS Images, Mikrochim. Acta, 352 (1995), 537-543. 28. M. Wolkenstein, H. Hutter, C. Mittermayr, W. Schiesser and M. Grasserbauer, Classification of SIMS Images Using a Kohonen Network, Analytical Chemistry, 69 (1997), 777ff. 29. M.G. Wolkenstein, H. Hutter, S.G. Nikolov and M. Grasserbauer, Improvement of SIMS Image Classification by Means of Wavelet De-Noising, Fresenius J. Analytical Chemistry, 357 (1997), 783-788. 30 S.G. Nikolov, M.G. Wolkenstein, H. Hutter and M. Grasserbauer, Improving Image Classification by De-Noising, submitted to Mikrochim. Acta. 31. A.P. Witkin, Scale-Space Filtering, Proceedings of 8th International Joint Conference on Artificial Intelligence, 1983, pp. 1019-1022. 32. T. Lindeberg, Scale-Space: A Basis for Early Vision, Technical Report CVAP 120, Royal Institute of Technology, Stockholm, Sweden, 1989. 33. M. Wickerhauser, Adapted Wavelet Analysis from Theory to Software, IEEE Press, 1994. 34. J.N. Bradley, C.M. Brislawn and T. Hopper, The FBI Wavelet/Scalar Quantization Standard for Gray-Scale Fingerprint Image Compression, in: SPIE Proc. Visual Info. Processing II, Orlando, FL, 1992, pp. 293-304. 35. T. Hopper, C.M. Brislawn and J.N. Bradley, WSQ Gray-Scale Fingerprint Image Compression Specification, Technical Report IAFIS-IC-0110-v2, FBI, 1993. 36. T. Hopper, Compression of Gray-Scale Fingerprint Images, in: SPIE Proc. Wavelet Application, Orlando, FL, Vol. 2242, 1994, pp. 180-187. 37. N. Jayant and P. Noll, Digital Coding of Waveforms, Prentice Hall, Englewood Cliffs, NJ, 1984.

548

38. D. Taubman and A. Zakhor, Multirate 3-D Subband Coding of Video, IEEE Transactions on Image Processing, 3 (5) (1994). 39. T. Bell, J. Cleary and I. Witten, Text Compression, Prentice Hall, 1990. 40. M. Wolkenstein, H. Hutter and M. Grasserbauer, Visualization of N-Dimensional Analytical Data on Personal Computers, Trends in Analytical Chemistry, 17 (3) (1998), 120-128. 41. M.G. Wolkenstein and H. Hutter, Compression of Secondary Ion Microscopy Image Sets Using a Three-Dimensional Wavelet Transformation, submitted to Microscopy and Microanalysis (March 1999). 42. J.C. Russ, Computer-Assisted Microscopy: The Measurement and Analysis of Images, Ist Edition, Plenum Press, New York, NY, 1990. 43. B. Lin, Wavelet Phase Filter for Denoising in Tomographic Image Reconstruction, PhD Thesis, Illinois Institute of Technology, 1994. 44. S. Mallat and S. Zhong, Characterisation of Signals from Mulitscale Edges, IEEE Transactions on PAMI, 14 (7) (1992), 710-732. 45. D. Marr, Vision, W. H. Freeman and Company, New York, 1982. 46. D. Marr and E. Hildreth, Theory of Edge Detection, in: Proc. R. Soc. Lond., Vol. 207, 1980, pp. 180-217. 47. Y. Meyer, Wavelets: Algorithms and Applications, Society for Industrial and Applied Mathematics, Philadelphia, 1993. 48. S. Mallat and S. Zhong, Complete Signal Representation with Multiscale Edges, Robotics Report No. 219 483, Courant Institute of Mathematical Sciences, New York University, 1989. 49. S. Mallat and S. Zhong, Wavelet Transform Maxima and Multiscale Edges (Coifman et al., Eds), Bartlett and Jones, 1990. 50. T.R. Reed and J.M.H. du Buf, A Review of Recent Texture Segmentation and Feature Extraction Techniques, Computer Vision, Graphics and Image Processing: Image Understanding, 57 (3) (1993), 359-372. 51. S. Livens, P. Scheunders, G. van de Wouwer and D. van Dyck, Wavelets for Texture Analysis, Technical Report, VisieLab, Department of Physics, University of Antwerp, 1997. 52. R. Porter and N. Canagarajah, A Robust Automatic Clustering Scheme for Image Segmentation Using Wavelets, IEEE Transactions on Image Processing, 5 (4) (1996), 662-665. 53. A. Laine and J. Fan, Texture Classification by Wavelet Packet Signatures, IEEE Trans. on PAMI, 15 (11) (1993), 1186-1190. 54. W.Y. Ma and B.S. Manjunath, Texture Features and Learning Similarity, In Proc. IEEE Computer Vision and Pattern Recognition Conference, 1996, pp. 425-430. 55. P. Brodatz, Textures: A Photographic Album for Artists and Designers, New York, Dover, 1966. 56. R. Porter and N. Canagarajah, Robust Rotation-Invariant Texture Classification: Wavelet, Gabor Filter and GMRF Based Schemes, lEE Proc.-Vis. Image Signal Process., 144 (3) (1997), 180-188. 57. D.A. Clausi, Texture Segmentation of SAR Sea Ice Imagery, PhD Thesis, University of Waterloo, 1996.

549

58. S. Livens, P. Scheunders, G. van de Wouwer, D. van Dyck, H. Smets, J. Winkelmans and W. Bogaerts, A Texture Analysis Approach to Corrosion Image Classification, Microsc., Microanal., Microstruct., 7 (2) (1996), 143-152. 59. F. Lumbreras and J. Serrat, Wavelet Filtering for Segmentation of Marble Images, Technical Report No. 5, Univ. Autonoma de Barcelona, 1996. 60. L.G. Brown, A Survey of Image Registration Techniques, ACM Computing Surveys, 24 (4) (1992), 325-376. 61. P.A. van den Elsen, E. Pol. and M. Viergever, Medical Image Matching- A Review with Classification, Eng. Med. Biol., 12 (1) (1993), 26-39. 62. D.I. Barnea and H.F. Silverman, A Class of Algorithms for Fast Digital Registration, IEEE Trans. Comput., C-21 (1972), 179-186. 63. W.K. Pratt, Correlation Techniques for Image Registration, IEEE Trans. on Aerospace and Electronic Systems, AES-10 (1974)~ 353-358. 64. B.S. Reddy and B.N. Chatterji, An FFT-Based Technique for Translation, Rotation, and Scale-Invariant Image Registration, IEEE Trans. hnag. Proc., 5 (8) (1996) 1266-1271. 65. R. Watt, Understanding Vision, Academic Press Limited, London, UK, 1991. 66. S.D. Boehmig, B.M. Reichl, H. Stoeri and H. Hutter, Automatic Matching of SAM, SIMS and EPMA Images, Fresenius J. Analytical Chemistry, 349, (1993). 67. S.D. Boehmig, B.M. Reichl, M.M. Eisl and H. Stoeri, A Template Matching Technique Based on Segmented Images Using Pyramids, in: Proceedings of RECPAD 94, 1994. 68. S.D. Boehmig, Bild- und Signalrestauration in der Oberflaechenanalytik, PhD Thesis, Vienna University of Technology, Vienna, Austria, 1995. 69. S.G. Nikolov, Wavelet Transform Algorithms for Analytical Data Processing, PhD Thesis, Vienna University of Technology, Vienna, Austria, 1996. 70. J.P. Djamdji, A. Bijaoui and R. Maniere, Geometrical Registration of Images" The Multiresolution Approach, Photogrammetry and Remote Sensing Journal, 59 (5) (1993), 645-653. 71. J. Le Moigne and R.F. Cromp, The Use of Wavelets for Remote Sensing Image Registration and Fusion, TR-96-171, NASA Goddard Space Flight Center, 1996. 72. G. Wolberg, Digital Image Warping, IEEE Computer Society Press, Los Alamitos, California, 1990. 73. M.A. Abidi and R.C. Gonzalez (Eds), Data Fusion in Robotics and Machine Intelligence, Academic Press, 1992. 74. O. Rockinger, Pixel-Level Fusion of Image Sequences Using Wavelet Frames, in: Proceedings in Image Fusion and Shape Variability Techniques, Leeds, UK, (K. V. Mardia and C. A. Gill and I. L. Dryden, Eds.), Leeds University Press, 1996, pp. 149-154. 75. H. Li, B.S. Manjunath and S.K. Mitra, Multisensor Image Fusion Using the Wavelet Transform, Graphical Models and Image Processing, 57 (3) (1995), 235245. 76. T.A. Wilson, S.K. Rogers and L.R. Myers, Perceptual Based Hyperspectral Image Fusion Using Multiresolution Analysis, Optical Engineering, 34 (11) (1995)~ 31543164.

550 77. I. Koren, A. Laine and F. Taylor, Image Fusion Using Steerable Dyadic Wavelet Transforms, in: Proceedings 1995 IEEE International Conference on Image Processing, IEEE, Washington, DC, 1995, pp. 232-235. 78. L.J. Chipman and T.M. Orr, Wavelets and Image Fusion, in: Proceedings 1995 IEEE International Conference on Image Processing, IEEE, Washington, DC, 1995, pp. 248-251. 79. S.G. Nikolov, D.R. Bull, C.N. Canagarajah, M. Halliwell and P.N.T. Wells, Image Fusion Using a 3-D Wavelet Transform, in: Proceedings of the Seventh International Conference on Image Processing and its Applications, Manchester, UK, IEE, 1999, pp. 235-239. 80. T. Stubbings, S.G. Nikolov and H. Hutter, Fusion of 2-D SIMS Images Using the Wavelet Transform Microchimica Acta, 624 (2000), pp 1-6. 81. WaveLab, D. Donoho et al., Stanford University. 82. WaveBox, C. Taswell, Stanford University. 83. Wavelet Workbench for IDL, A. Graps and Research Systems Inc.. 84. Wavelet Fusion Toolbox for IDL, S.G. Nikolov et al., Bristol University, UK, 1999. 85. SOM (Self-Organinizing Maps), T. Kohonen et al., Helsinki University of Technology, Helsinki, Finland. 86. MIE (Multispectral Image Enhancement), S.G. Nikolov, Vienna University of Technology, Vienna, Austria, 1992. 87. X-image, S.G. Nikolov, Vienna University of Technology, Vienna, Austria, 1995. 88. Wave2, W.L. Hwang, S. Mallat and S. Zhong, New York University.

551

Index abstraction level 352 adaptive wavelet algorithm (AWA) 177, 189, 194, 199, 200, 440, 442 adaptive wavelets 437, 448 analytical images 479 apodisation 28-31 approximate derivative calculation 211 autocorrelation function 124 banded matrix 87 band-pass filter 23 base frequency 16 baseline drift 207 bases dictionary 469 basis 12, 13 basis transformation 9, 12, 13 best-basis 155, 473 biorthogonal wavelets 79, 252 boundary handling 99, 111, 113, 114, 116, 117 B-spline wavelets 226 calibration 323 chemical dynamics 274 chemical kinetics 279 Cheybychev polynomial 12, 13 chromatography 205 circulant matrix 87 classification 437, 502 cluster analysis 378 coefficient position retaining method (CPR) 244 compact support 76 compression 126, 128 compression ratio 294 constant-padding extension 113 continuous wavelet transform (CWT) 59, 62

control charts 415 control points 528 convolution 5, 7 convolution integral 7 correct classification rate (CCR) 440 cost function 160, 162 cut-off frequency 24

Daubechies family 63 wavelets 76 denoising 126, 488 derivative technique 211 determinant 89 diagonal matrix 86 difference of Gaussians (DOG) 233, 234 dilation equation 70 discriminant analysis 437, 439 discriminant function analysis (DFA) 391 discriminant partial least squares (DPLS) 373 discrete cosine transform (DCT) 463 discrete wavelet transform (DWT) 65, 91, 97 dynamic nuclear magnetic resonance spectroscopy 255

early transition detection (ETD) 311 edge detection 513 electrochemistry 225 electron probe microanalysis (EPMA) 488 embedded zero-tree wavelet encoding (EZW) 470 end effects 132 entropy 160, 171, 174, 192 entropy coding 509

552

factor analysis (FA) 213 fast Fourier transform (FFT) 14, 15 fast wavelet transform (WT) 74 feature extraction 513 selection 324, 326, 331 filter coefficients 185 filter coefficients conditions 185 filter matrix 96 finite impulse response (FIR) filter 101, 126 Fourier basis 13 domain 14 integral 4 polynomial series 13 transform (FT) 3, 9, 14, 18, 60 fractal structures 282 frequency analysis 124 domain 14 localisation 38 resolution 16 functional data analysis 352 fusion rules 538 Gabor transform 39 generating optimal linear PLS estimations (GOLPE) 370 genetic algorithm 325, 369 geometry matrix 356 Haar transform 52 wavelet 77 hard thresholding 132 higher multiplicity wavelets 179 high-pass filter 23, 92 high-pass filter coefficients 73 hyphenated instruments 213 identity matrix 86 image classification 502

compression 459 decorrelation 462 fusion 536 transform fusion 536 immune neural network (INN) 220 impulse response 6 infrared spectroscopy (IR) 243 inner product 354 inverse wavelet transform 63 joint basis 167 joint best-basis 171,294 Karhunen-Loeve transform (KLT) 462 Laplacian method 515 lapped orthogonal transform (LOT) 464 Lawton matrix 185 library compression 293 linear filtering 126 linear regression 10 low-pass filter 23, 92 low-pass filter coefficients 73 masking method 367 mass spectrometry 254 matrix 85 addition 88 multiplication 88 operations 88 polynomial product 89 product 88 properties 89 rank 90 theory 85 transpose 86 m-band discrete wavelet transform 180 mean square error (MSE) 461,487 median filter 129, 138 minimum description length (MDL) 293 minor 89 misclassification rate (MCR) 440 molecular structure 264

553

'mother' wavelet 59 moving average 25 multiple linear regression (MLR) 448 multiscale denoising 422 edge detection 517, 518 edge point 521 edge representation 521 filtering 130 median filtering 138 representation 121 statistical process control (MSSPC) 415 multiresolution 68, 69, 362 multiresolution analysis (MRA) 65, 91 multi-tree wavelet packet transform 94 mutual information 372 near-infrared spectroscopy (NIR) 333 neural networks (NNs) 166, 219 noise characterization 123 suppression 208 non-linear basis 357 filtering 128 non-singular matrix 90 nuclear magnetic resonance spectroscopy (NMR) 255 on-line multiscale (OLMS) filtering 139 optimal bit allocation (OBA) algorithm 252 optimal scale combination method (OSC) 366 oscillographic chronopotentiometry 235 partial least squares (PLS) 323, 373 parsimonious models 361 patterned matrix 86 pattern recognition 219, 251 peak detection 210 periodic extension 111 periodisation 103

periodised wavelets 99 permutation matrix 87 phase 15 photoacoustic spectroscopy (PA) 256 polynomial approximation 9 extension 114 product 89. 186 power spectrum 17, 124 principal component analysis (PCA) 166, 297 principal component regression (PCR) 323 pyramid algorithm 43, 75 quantization 509 quantum mechanics 264 rectification 425 regression 448 relevant component extraction-partial least squares (RCE-PLS) 341 resolution enhancement 210 sampling point representation 352 scale dendrogram 377 scale-error complexity (SEC) 365 plot 364 scale filter 92 scaling coefficients 70, 72. 75, 92 scaling function 69 scalogram 359 scanning electron microscope (SEM) 488 scatter diagram 503 secondary ion mass spectrometry (SIMS) 254, 487, 488, 491. 496, 507 segmentation 513 semiorthogonal wavelets 79 separable wavelets 482 shift matrix 87 short time Fourier transform 35, 60 signal enhancement 208

554

signal-to-noise ratio (SNR) 487 simulated annealing 326 singular matrix 90 smoothing 209 smoothness 77 soft thresholding 132, 489 spline basis 355 spline wavelet 226 standardization 250 statistical process control (SPC) 415 symmetric extension 111 symmetry 76 task-specific wavelets 473 texture analysis 521 threshold selection 131 three-dimensional (3-D) images 480, 483 wavelet transform 482 time domain 14 series 274 time-frequency analysis 35 domain 38 Toeplitz matrix 88 translation-invariant (TI) filtering 133 translation-rotation transformation (TRT) 246 two-dimensional (2-D) images 483 scaling function 483 wavelet packet transform 469 wavelet transform 482

universal threshold 131 ultraviolet-visible (UV-VIS) spectroscopy 250 vanishing moments 77 variable length coding (VLC) algorithm 252, 460 variables selection 383, 386, 390 variance tree 172 vector 86 voltammetry 225, 233 wavelet basis functions 59, 72 coefficients 73, 75, 92 coefficient method 213 decomposition 72 families 76 filter 92 matrix 95 neural network (WNN) 248, 251 on an interval 116 packet coefficients 155 packet decomposition 156 packet functions 154 packet transform 53, 94, 151 properties 76, 80 series 72 spectrum 126 window factor analysis (WFA) 216, 217 windowed Fourier transform 60 zero frequency 16 zero padding 113, 244

Wavelets in Chemistry (Data Handling in Science and Technology)

Practical Data Analysis in Chemistry (Data Handling in Science and Technology 26)

Chemometrics: A Textbook (Data Handling in Science and Technology, 2)

PC'S for Chemists (Data Handling in Science and Technology, 5)

Chemometrics: A Textbook (Data Handling in Science and Technology, 2)

PC'S for Chemists (Data Handling in Science and Technology, 5)

Signal Treatment and Signal Analysis in NMR (Data Handling in Science and Technology)

Data Analysis for Hyphenated Techniques (Data Handling in Science and Technology, V. 17)

Wavelets in Chemistry, Volume 22

Multivariate Pattern Recognition in Chemometrics: Illustrated by Case Studies (Data Handling in Science and Technology)

Handbook of Chemometrics and Qualimetrics, Volume Part A (Data Handling in Science and Technology, V. 20a)

Wavelets: Tools for Science and Technology

Microsystem Technology in Chemistry and

Adaption of Simulated Annealing to Chemical Optimization Problems (Data Handling in Science and Technology)

Open Source Approaches in Spatial Data Handling

Open Source Approaches in Spatial Data Handling

Large Scale Data Handling in Biology

Design and Optimization in Organic Synthesis, Volume 24: Second Revised and Enlarged Edition (Data Handling in Science and Technology)

Mathematics in science and technology

Handling experimental data

Wavelets: Tools for Science & Technology

Nature-inspired methods in chemometrics: genetic algorithms and artificial neural networks, Volume 23 (Data Handling in Science and Technology)

Wavelets: Tools for Science & Technology

Practical Data Analysis in Chemistry

Vacumm Technology: Calculations in Chemistry

Developments in Spatial Data Handling: 11th International Symposium on Spatial Data Handling

Progress in Spatial Data Handling: 12th International Symposium on Spatial Data Handling

Wavelets in geodesy and geodynamics

Advances in Potato Chemistry and Technology

Advances in Potato Chemistry and Technology

Cotton Fiber Chemistry and Technology (International Fiber Science and Technology)

Wavelets in Chemistry (Data Handling in Science and Technology)

Practical Data Analysis in Chemistry (Data Handling in Science and Technology 26)

Chemometrics: A Textbook (Data Handling in Science and Technology, 2)

PC'S for Chemists (Data Handling in Science and Technology, 5)

Chemometrics: A Textbook (Data Handling in Science and Technology, 2)

PC'S for Chemists (Data Handling in Science and Technology, 5)

Signal Treatment and Signal Analysis in NMR (Data Handling in Science and Technology)

Data Analysis for Hyphenated Techniques (Data Handling in Science and Technology, V. 17)

Wavelets in Chemistry, Volume 22

Multivariate Pattern Recognition in Chemometrics: Illustrated by Case Studies (Data Handling in Science and Technology)

Handbook of Chemometrics and Qualimetrics, Volume Part A (Data Handling in Science and Technology, V. 20a)

Wavelets: Tools for Science and Technology

Microsystem Technology in Chemistry and

Adaption of Simulated Annealing to Chemical Optimization Problems (Data Handling in Science and Technology)

Open Source Approaches in Spatial Data Handling

Open Source Approaches in Spatial Data Handling

Large Scale Data Handling in Biology

Design and Optimization in Organic Synthesis, Volume 24: Second Revised and Enlarged Edition (Data Handling in Science and Technology)

Mathematics in science and technology

Handling experimental data

Wavelets: Tools for Science & Technology

Nature-inspired methods in chemometrics: genetic algorithms and artificial neural networks, Volume 23 (Data Handling in Science and Technology)

Wavelets: Tools for Science & Technology

Practical Data Analysis in Chemistry

Vacumm Technology: Calculations in Chemistry

Developments in Spatial Data Handling: 11th International Symposium on Spatial Data Handling

Progress in Spatial Data Handling: 12th International Symposium on Spatial Data Handling

Wavelets in geodesy and geodynamics

Advances in Potato Chemistry and Technology

Advances in Potato Chemistry and Technology

Cotton Fiber Chemistry and Technology (International Fiber Science and Technology)

Recommend Documents