Noise Reduction in Speech Processing

Springer Topics in Signal Processing Volume 2 Series Editors J. Benesty, Montreal, QC, Canada W. Kellermann, Erlangen, Germany

Springer Topics in Signal Processing Edited by J. Benesty and W. Kellermann

Vol. 1: Benesty, J.; Chen, J.; Huang, Y. Microphone Array Signal Processing 250 p. 2008 [978-3-540-78611-5]

Vol. 2: Benesty, J.; Chen, J.; Huang, Y.; Cohen, I. Noise Reduction in Speech Processing 240 p. 2009 [978-3-642-00295-3]

Jacob Benesty · Jingdong Chen · Yiteng Huang · Israel Cohen

Noise Reduction in Speech Processing

123

Prof. Dr. Jacob Benesty Universit´e de Quebec Inst. National de la Recherche Scientifique (INRS) 800 de la Gauchetiere Ouest Montreal QC H5A 1K6 Canada [email protected]

Yiteng Huang 9 Sylvan Dr. Bridgewater NY 08807 USA [email protected]

Jingdong Chen Lucent Technologies Bell Laboratories 600-700 Mountain Avenue Murray Hill NJ 07974-0636 USA [email protected]

Prof. Israel Cohen Technion - Israel Institute of Technology Dept. Electrical Engineering 32000 Haifa Technion City Israel [email protected]

ISBN 978-3-642-00295-3 e-ISBN 978-3-642-00296-0 DOI 10.1007/978-3-642-00296-0 Springer Dordrecht Heidelberg London New York Springer Topics in Signal Processing ISSN 1866-2609 Library of Congress Control Number: 2009921819 c Springer-Verlag Berlin Heidelberg 2009 ° This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Cover design: WMXDesign GmbH, Heidelberg Printed on acid-free paper Springer is part of Springer Science+Business Media (springer.com)

Preface

Noise is everywhere and in most applications that are related to audio and speech, such as human-machine interfaces, hands-free communications, voice over IP (VoIP), hearing aids, teleconferencing/telepresence/telecollaboration systems, and so many others, the signal of interest (usually speech) that is picked up by a microphone is generally contaminated by noise. As a result, the microphone signal has to be cleaned up with digital signal processing tools before it is stored, analyzed, transmitted, or played out. This cleaning process is often called noise reduction and this topic has attracted a considerable amount of research and engineering attention for several decades. One of the objectives of this book is to present in a common framework an overview of the state of the art of noise reduction algorithms in the single-channel (one microphone) case. The focus is on the most useful approaches, i.e., filtering techniques (in different domains) and spectral enhancement methods. Readers who are interested in algorithms based on multiple microphones (or microphone arrays) are invited to consult the recent book by J. Benesty, J. Chen, and Y. Huang, Microphone Array Signal Processing, Berlin, Germany: Springer-Verlag, 2008, and the references therein. Several books on speech enhancement can be found in the literature. But they either do not cover state-of-the-art noise reduction techniques or are not rigorous in the explanations on how and why the noise reduction algorithms really work. This is how the idea of writing a book on Noise Reduction in Speech Processing came up. Therefore, the other objective of this text is to derive the most important and well-known techniques in this area of research and engineering in a rigorous way yet clear way, and prove many fundamental and intuitive results often taken for granted. This book is especially written for graduate students and research engineers who work on noise reduction for speech and audio applications and want to understand the subtle mechanisms behind each approach. We hope the readers will find many new and interesting concepts that are presented in this book useful and inspiring.

VI

Preface

We would like to take this opportunity to thank again Christoph Baumann, Carmen Wolf, and Petra Jantzen from Springer (Germany) for their wonderful help in the preparation and publication of this manuscript. Working with them is always a pleasure and a wonderful experience. We also thank Dr. Emanu¨el Habets for reading a first draft of this book and giving us many useful comments. Finally, we deeply thank Huawei Technologies, Shenzhen, China, and especially the Media & Communication Lab of Huawei Research, for sharing with us some of their visions of the telecommunications of the future.

Montr´eal, QC/ Murray Hill, NJ/ Bridgewater, NJ/ Haifa

Jacob Benesty Jingdong Chen Yiteng Huang Israel Cohen

Contents

1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 Noise Reduction in Speech Processing . . . . . . . . . . . . . . . . . . . . . . 1 1.2 The Paradigm for Noise Reduction . . . . . . . . . . . . . . . . . . . . . . . . 6 1.3 A Brief History of Noise Reduction Research . . . . . . . . . . . . . . . . 7 1.4 Organization of the Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1.5 Some Notes to the Reader . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2

Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 In the Time Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 In the Frequency Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 In the Karhunen-Lo`eve Expansion (KLE) Domain . . . . . . . . . . . 2.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

15 15 16 18 20

3

Performance Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Signal-to-Noise Ratio . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Noise-Reduction Factor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Speech-Distortion Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Speech-Reduction Factor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

21 21 24 25 27 28

4

Mean-Squared Error Criterion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 In the Time Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 In the Frequency Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 In the KLE Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

31 31 32 34 36

5

Pearson Correlation Coefficient . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1 Correlation Coefficient Between Two Random Variables . . . . . . 5.2 Correlation Coefficient Between Two Random Vectors . . . . . . . . 5.3 Frequency-Domain Versions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4 KLE-Domain Versions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

37 37 38 39 39

VIII

Contents

5.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 6

Fundamental Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1 In the Time Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 In the Frequency Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3 In the KLE Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

41 41 46 50 57

7

Optimal Filters in the Time Domain . . . . . . . . . . . . . . . . . . . . . . . 7.1 Wiener Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2 Tradeoff Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3 Subspace Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4.1 Experimental Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4.2 Effect of Forgetting Factor on Performance . . . . . . . . . . . 7.4.3 Effect of Filter Length on Performance . . . . . . . . . . . . . . . 7.4.4 Performance in Different Noise Conditions . . . . . . . . . . . . 7.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

59 59 64 67 68 68 70 72 74 75

8

Optimal Filters in the Frequency Domain . . . . . . . . . . . . . . . . . . 8.1 Wiener Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2 Parametric Wiener Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3 Tradeoff Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.4 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.4.1 Impact of Input SNR on Filter Gain and Speech Distortion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.4.2 Noise Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.4.3 Performance Comparison in NYSE Noise . . . . . . . . . . . . . 8.4.4 Performance Comparison in Car Noise . . . . . . . . . . . . . . . 8.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

77 77 81 82 86

9

86 86 89 93 94

Optimal Filters in the KLE Domain . . . . . . . . . . . . . . . . . . . . . . . 95 9.1 Class I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 9.1.1 Wiener Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 9.1.2 Parametric Wiener Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 9.1.3 Tradeoff Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 9.2 Class II . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 9.2.1 Wiener Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 9.2.2 Tradeoff Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 9.3 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 9.3.1 Impact of Forgetting Factor on Performance of Class-I Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 9.3.2 Effect of Filter Length on Performance of Class-I Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 9.3.3 Estimation of Clean Speech Correlation Matrix . . . . . . . 114

Contents

IX

9.3.4 Performance of Class-I Filters in Different Noise Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 9.3.5 Impact of Forgetting Factor on Performance of Class-II Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 9.3.6 Effect of Filter Length on Performance of Class-II Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 9.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 10 Optimal Filters in the Transform Domain . . . . . . . . . . . . . . . . . . 123 10.1 Generalization of the KLE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 10.2 Performance Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 10.2.1 SNR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 10.2.2 Noise-Reduction Factor . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 10.2.3 Speech-Distortion Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 10.2.4 Speech-Reduction Factor . . . . . . . . . . . . . . . . . . . . . . . . . . . 130 10.3 MSE Criterion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 10.4 PCC and Fundamental Properties . . . . . . . . . . . . . . . . . . . . . . . . . 132 10.5 Examples of Filter Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 10.5.1 Wiener Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 10.5.2 Parametric Wiener Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 10.5.3 Tradeoff Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 10.5.4 Examples of Unitary Matrices . . . . . . . . . . . . . . . . . . . . . . . 145 10.6 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 10.6.1 Performance of Wiener Filter in White Gaussian Noise . . 146 10.6.2 Effect of Filter Length on Performance . . . . . . . . . . . . . . . 148 10.6.3 Performance of Tradeoff Filter in White Gaussian Noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148 10.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 11 Spectral Enhancement Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 11.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 11.2 Performance Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 11.2.1 SNR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 11.2.2 Noise-Reduction Factor . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 11.2.3 Speech-Distortion Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 11.2.4 Speech-Reduction Factor . . . . . . . . . . . . . . . . . . . . . . . . . . . 158 11.3 MSE Criterion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 11.4 Signal Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 11.5 Signal Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 11.5.1 MMSE Spectral Estimation . . . . . . . . . . . . . . . . . . . . . . . . . 162 11.5.2 MMSE Spectral Amplitude Estimation . . . . . . . . . . . . . . . 163 11.5.3 MMSE Log-Spectral Amplitude Estimation . . . . . . . . . . . 164 11.6 Spectral Variance Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 11.6.1 GARCH Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 11.6.2 Modeling Speech Spectral Variance . . . . . . . . . . . . . . . . . . 168 11.6.3 Model Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

X

Contents

11.7 Spectral Variance Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170 11.7.1 Relation to Decision-Directed Estimation . . . . . . . . . . . . . 172 11.8 Summary of Spectral Enhancement Algorithm . . . . . . . . . . . . . . 174 11.9 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176 11.10 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 12 A Practical Example: Multichannel Noise Reduction for Voice Communication in Spacesuits . . . . . . . . . . . . . . . . . . . . . . . . 183 12.1 Problem Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 12.2 Problem Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186 12.2.1 Sources of Noise in Spacesuits . . . . . . . . . . . . . . . . . . . . . . . 186 12.2.2 Noise Cancelling Microphones . . . . . . . . . . . . . . . . . . . . . . . 188 12.3 Suggested Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192 12.3.1 Nearfield, Wideband Microphone Array Beamforming for Speech Acquisition in Spacesuits . . . . . . . . . . . . . . . . . 193 12.3.2 Multichannel Noise Reduction: a More Practical Microphone Array Signal Processing Technique . . . . . . . 200 12.3.3 Single-Channel Noise Reduction . . . . . . . . . . . . . . . . . . . . . 201 12.3.4 Adaptive Noise Cancellation . . . . . . . . . . . . . . . . . . . . . . . . 202 12.4 Algorithm Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203 12.4.1 In-Helmet Multichannel Acoustic Data Collection . . . . . 203 12.4.2 Performance Evaluation of Beamforming Algorithms . . . 208 12.4.3 Validation of Multichannel Noise Reduction Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214 12.4.4 Validation of Single-Channel Noise Reduction Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214 12.4.5 Feasibility Assessment of Using Adaptive Noise Cancellation in Spacesuits . . . . . . . . . . . . . . . . . . . . . . . . . . 214 12.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227

1 Introduction

1.1 Noise Reduction in Speech Processing The world we live in is full of different kinds of sounds; it is obviously hard to imagine a world without any sound sources. Some sounds may originate from far-away sources but they die away before reaching our ears so we do not hear them. Some others, however, reach our ears, but their energies are so weak that we do not perceive them clearly. There are sounds that we can clearly hear. Among those, some may convey very important information while others may not carry anything really useful, but purely interfere with the sounds that do contain useful information. The sound that carries the information that we want or need, is normally labelled as “desired.” The sound that does not contain any useful information but interferes with the desired sound, is usually referred to as “noise.” The term “noise” was coined in 1905 by Einstein when he saw that it could be the instrument to establish the existence of atoms [34]. Ever since, noise has permeated every field of science and technology and has served as a driving force for countless great discoveries and inventions, such as the existence of atoms, the Brownian motion, the Big Bang theory, the information theory, the optimal filtering theory, to name a few. There are many exciting stories behind noise [34]. One example of great interest is the Big Bang story. In the 1930s, Bell Laboratories built a large rotating antenna in Crawford Hill, New Jersey, to identify and understand various sources of noise that impaired the then-recently introduced transatlantic radio-telephone service. Having eliminated all known sources, the team found that there was a persistent residual “hiss” no matter where the antenna was pointed. At that time there were two competing theories of the universe. In one corner was the Big Bang model, which proposed a unique moment of creation followed by a rapid expansion. Hubble had observed that galaxies were moving away from each other and that the further they are, the faster they were moving away. Also, Gamow and Alpher had shown that the Big Bang could explain the abundance of hydrogen and helium. They and others predicted that the primordial radia-

J. Benesty et al., Noise Reduction in Speech Processing, Springer Topics in Signal Processing 2, DOI 10.1007/978-3-642-00296-0_1, © Springer-Verlag Berlin Heidelberg 2009

2

1 Introduction

tion, according to the Big Bang model, should be detectable as an isotropic field with a black body temperature of about 10 K. In the other corner was the Steady-State model, invented by Hoyle, Gold, and Bondi, which harked back to the conservative view of an eternal universe, except that it included an element of continuous creation and expansion. In the earlier 1960s, Wilson and Penzias, who were working at Bell Laboratories, detected the isotropic residual radiation using the 6-meter aperture antenna at Crawford Hill. When Penzias called Dicke (who was a prominent physicist at Princeton University and was working on how to find and measure the leftover radiation at that time) about what was measured and to seek advice on a possible explanation, he immediately realized what Penzias and his colleague Wilson had discovered. This example shows the importance of noise in the confirmation of the Big Bang theory. However, as noise permeates every research and engineering field, its definition also evolves. Now, many fields have their own definition and history of noise, and in most cases, the definition has well departed from its original meaning as basically an idea, a subject, a field, or an instrument. Such an evolution, although accompanied with distortion, does make the description and understanding of the problems in a specific field easier. For the problems covered in this book, we follow the tradition built in the signal and speech processing fields and the term noise is used to signify any unwanted signal that interferes with measurement, processing, and communication of the desired information-bearing speech signal. With this definition of noise, it is then of great interest to know the impact of noise on speech communications. As a matter of fact, this has been one of the most studied problems in human-to-human and human-to-machine communications over the last four decades. This problem, as perhaps we all know, is very much complicated because noise can cause profound changes to the characteristics of the speech signal. These changes can be classified into two categories, depending on in what stage they are introduced during a communication process. • Changes in the talking/transmitting side. The talker, when hears the background noise, tends to alter his/her speaking style in an effort to increase communication efficiency over the noisy medium. This phenomenon is called the Lombard effect. This effect leads to changes in both speech intensity and characteristics. The intensity change has been experimentally quantified by, among others, Pearsons, Bennett, and Fidell [99]. Briefly, in many noisy environments, the speaker would increase his/her vocal intensity by 0.6 decibel (dB) for each dB increase in the background noise up to a ceiling level. However, the changes in speech characteristics are very difficult to model. • Changes in the listening/receiving end. In the Lombard effect, noise causes changes in speech statistics through an indirect way. But in most situations, noise may directly affect the speech signal. This has two forms. One

1.1 Noise Reduction in Speech Processing

3

is when the noise and speech signals are statistically dependent. As a result, the additive noise directly modifies the spectrum and characteristics of the speech signal. The other is when the noise is independent of the speech signal, so it does not cause any changes to the speech characteristics. However, the characteristics of the observed signal is very different from those of the desired speech since the observed signal is a mixture of the desired speech and noise. Regardless how the speech characteristics are modified by the presence of noise, the noise effect will profoundly affect the listener’s perception and machine’s processing of the observed speech. On the positive side, if the intensity of the noise is not too high, voice communication is still possible, but it would be less natural and less comfortable. On the negative side, if the noise is strong and the speech signal is completely immersed into it, voice communication would become difficult and even impossible. Therefore, in order to make voice communication feasible, natural, and comfortable in the presence of noise regardless of the noise level, it is desirable to develop digital signal processing techniques to “clean” the recorded signal before it is stored, transmitted, or played out. This problem is generally referred to as either noise reduction or speech enhancement. The fact that noise can be any unwanted signal makes noise reduction an extremely difficult problem to tackle. As of today, it is still not clear how this problem can be described into a general mathematical model, not even mention the solutions. So, instead of searching for a generic solution, researchers and engineers in the speech field are adopting a pragmatic approach: dividing noise into several categories based on the mechanism how it is generated and conquering each category using a different approach. Commonly, noise is divided into four basic categories as explained below. • Additive noise. Additive noise can come from various sources. Some are from natural sound sources, while others may be artificially introduced, for example, the comfort noise in speech codecs. • Echo. Acoustic echo occurs due to the coupling between loudspeakers and microphones. The existence of echo will make conversations very difficult or even impossible. • Reverberation. Reverberation is the result of multipath propagation and is introduced by an enclosure. It can cause spectral distortion, which impairs speech intelligibility. • Interference. Interference comes from concurrent sound sources. In teleconferencing and telecollaboration, it is possible that each communication site has multiple participants and loudspeakers. So, there can be multiple competing sound sources. Combating these four categories of noise has led to the developments of diverse acoustic signal processing techniques. They include noise reduction (or

4

1 Introduction

speech enhancement), echo cancellation and suppression, speech dereverberation, and source separation, each of which is a rich subject of research. A broad coverage of these research areas can be found in [15], [72]. This monograph will mainly focus on the (additive) noise reduction, which aims at eliminating or mitigating the effects of additive noise. So, from now on, we shall narrow the definition of noise down to additive noise. Under this narrowed definition, the observed microphone signal can be modeled as a superposition of the clean speech and noise. The objective of noise reduction, then, becomes to restore the original clean speech from the mixed signal. This can be described as a parameter estimation problem and the optimal estimate of the clean speech can be achieved by optimizing some criterion, such as the mean-squared error (MSE) between the clean speech and its estimate, the signal-to-noise ratio (SNR), the a posteriori probability of the clean speech given its noisy observations, etc. Unfortunately, an optimal estimate formed from a signal processing perspective does not necessarily correspond to the best quality when perceived by the human ear. This inconsistency between objective measures and subjective quality judgement has forced researchers to rethink performance criteria for noise reduction. The objective of the problem has subsequently been broadened, which can be summarized as to achieve one or more of the following three primary goals: • to improve objective performance criteria such as intelligibility, SNR, etc., • to improve the perceptual quality of a degraded speech, and • as a preprocessor, to increase robustness of other speech processing applications (such as speech coding, echo cancellation, automatic speech recognition, etc.) to noise. The different goals may lead to distinct speech estimates. It is very hard to satisfy all three goals at the same time. With a specified goal (performance criterion), the difficulty and complexity of the noise reduction problem can vary tremendously, depending on many factors such as the number of microphone channels. In general, the larger the number of microphones, the easier the noise reduction problem. For example, when an array of microphones can be used, a beam can be formed and steered to a desired direction. As a result, signal propagating from the desired direction will be passed through without degradation, while signals originating from all other directions will either suffer a certain amount of attenuation or be completely rejected [15]. In the two-microphone case, with one microphone picking up the noisy signal and the other measuring the noise field, we can use the second microphone signal as a noise reference and eliminate the noise in the first microphone by means of adaptive cancellation [6]. However, most of today’s communication terminals are equipped with only one microphone. In this case, the noisy speech is the only resource that is accessible to us, so noise reduction becomes a very challenging problem for many sophisticated reasons. First, a reference of the noise is not accessible and the clean speech cannot be preprocessed prior to being corrupted by noise. Second, the nature and characteristics of the noise

1.1 Noise Reduction in Speech Processing

5

Periodogram spectrum (dB)

(a)

1

0

1

2 Frequency (kHz)

3

4

3

4

Periodogram spectrum (dB)

0

(b) 2 Frequency (kHz)

Fig. 1.1. Spectra of two different noise signals: (a) periodogram spectrum of a car noise signal and (b) periodogram spectra of noise in a conference room measured at two time instants (two seconds apart).

can change dramatically from application to application and even vary in time in one application. To illustrate this, Fig. 1.1 shows the spectra of a car noise signal and a noise signal recorded in a conference room [spectra computed at two different time instants (two seconds apart)]. It is seen that the spectrum of the car noise is very different from that of the conference room noise. Even for the same conference room, the characteristics of the noise are different when measured at different points in time. In addition, the speech signal is highly nonstationary. It is very difficult to estimate the fast changing statistics of speech in the presence of noise. Despite these challenges, a tremendous amount of attention has been dedicated to this problem since there are so many applications that require a solution to it.

6

1 Introduction

Fig. 1.2. General block diagram of a noise reduction system.

1.2 The Paradigm for Noise Reduction This book addresses the challenging problem of how to achieve noise reduction by analyzing and processing the noisy speech measured by only a single microphone without requiring additional information. The general model used throughout this book is shown in Fig. 1.2. The model begins with a clean speech signal (from a desired speaker), x(k), being corrupted with an unwanted additive noise, v(k). The noisy signal, y(k), which is a superposition of x(k) and v(k) is first processed to determine whether the desired speech is currently present or absent. The noise and speech statistics such as the covariance matrices and power spectral densities are estimated based on the detection results and the input noisy speech. These statistics will be used to estimate a noise reduction filter. This filter can be optimal in the sense that it optimizes some error criterion (e.g., MSE). It may be suboptimal, where parameters are introduced to better control the quality of the output speech. The estimated filter is applied to the noisy speech to filter out the noise signal, thereby producing an output signal, z(k), which is supposed to be an estimate of the clean speech, x(k). The problem addressed in this book is of great importance from both the theoretical and practical viewpoints. To illustrate this, we give the very good example of multiparty conferencing explained by Diethorn in [38]. In multiparty conferencing, the background noise picked up by the microphone of each point of the conference combines additively at the network bridge with the noise signals from all other points. The loudspeaker at each location of the conference therefore reproduces the combined sum of the noise sequences from all other locations. Consider a three-point conference in which the room noise at all locations is stationary and independent with power σv2 . Each loudspeaker

1.3 A Brief History of Noise Reduction Research

7

receives noise from the other two locations, resulting in a total noise power that is 2σv2 , which is 3 dB greater than that of a two-point conference. Now if we consider a P -point conference, each side receives a total noise power of (P − 1)σv2 , which is 10 log10 (P − 1) dB greater than σv2 . This is still the ideal condition. In practice, there may be many processing operations in the voice communication network, such as speech coding, transcoding, automaticgain-control processing, etc. Each operation may boost the noise level. So, the noise problem is extremely serious, particularly when the number of conferees is large, and without noise reduction, communication in this context is almost impossible. Besides multiparty conferencing, there are many other applications that require a noise reduction technology. The following is a short list: • • • • • • • •

hands-free communications, hearing aids, audio bridging, teleconferencing and telepresence, hands-free human-machine interfaces, car and mobile phones, cockpits and noisy manufacturing, high-quality speech coding.

To summarize this section, we can definitely claim that there are many speech related applications that require a technique to reduce noise. Greater efforts from both the research and engineering communities are indispensable to develop practical and reliable solutions before the noise reduction technology can be widely deployed.

1.3 A Brief History of Noise Reduction Research Research on noise reduction has been going on for more than four decades. A significant amount of progress has been achieved over this period. To gain an appreciation of the basic concepts and fundamental techniques that were developed, it is worthwhile to briefly review some research highlights and milestones. The reader, however, is cautioned that such a review is narrowly focused and might not be comprehensive. The earliest attempts to devise noise reduction algorithms for enhancing speech signals were made in the 1960s. In 1960, at Bell Laboratories, Schroeder proposed a system for reducing noise in telecommunication environments. The schematic diagram of his system is shown in Fig. 1.3. Note that this diagram is slightly modified from its original form in [108] for ease of exposition. As seen, the input noisy signal, y(t) (where t is the continuous time), which is a superposition of the clean speech, x(t), and noise, v(t), i.e., y(t) = x(t)+v(t), is divided into M subbands. For each subband, a rectifier and a lowpass filter are applied in tandem to estimate the noisy speech envelope. The noise level in the

8

1 Introduction

Fig. 1.3. Schematic diagram of the Schroeder’s noise reduction system where BFP, RECTF, LFP, and NLE denote, respectively, bandpass filter, rectifier, lowpass filter, and noise level estimator.

corresponding subband is then estimated using an analog circuit with resistors, capacitors, and diodes, and the noise estimate is subsequently subtracted from the noisy speech envelope, resulting in an estimate of the clean speech envelope for the subband. A second rectification process is applied to force the negative results to zero due to the subtraction. The rectified clean speech envelope estimate, which is served as a gain filter, is then multiplied with the unmodified subband signal. Finally, the fullband signal, z(t), is constructed from all the subband outputs, where z(t) is basically an estimate of x(t). As seen, the Schroeder’s system is actually a spectral subtraction technique, but with all analog implementation. However, this work was largely unknown in the noise reduction research community until more formal methods were developed in the late 1970s. The 1970s was a decade in which great impetus was given to the spectral modification based techniques. This wave of interest in noise reduction was due largely to the confluence of digital signal processing (DSP) algorithms and DSP hardwares, both of which had come to prominence at that time. In 1974, Weiss, Aschkenasy, and Parsons developed a “spectrum shaping” method that used amplitude clipping in filter banks to remove low-level excitation, presumably noise [122]. A few years later, Boll, in his informative paper [18], reinvented the spectral subtraction method but in the digital domain. Boll was perhaps the first to formulate the magnitude subtraction in the framework of digital short-time Fourier analysis, which had earlier been under development by, among others, Allen [1] and Portnoff [101]. Shortly after, McAulay and Malpass cast the spectral subtraction approach in a framework of statistical spectral estimation, and presented a broad class of estimators including the magnitude and power subtraction, the Wiener filtering, the maximum likelihood envelope estimator, etc. [92]. They were also the first to make connections between the spectral subtraction and the Wiener filter. Almost at the same time, Lim and Oppenheim, in their landmark work [85], presented

1.3 A Brief History of Noise Reduction Research

9

one of the first comprehensive treatments of methods of noise reduction and speech enhancement. The spectrum subtraction methods were discussed, also within an estimation framework, and were compared to other techniques of speech enhancement. Also, Sondhi, Schmidt, and Rabiner published results from a series of implementation studies that roots in Schroeder’s work in the 1960s [112]. In the 1980s, many fundamental ideas in noise reduction surfaced and were published. By and large, these ideas can be broadly classified into two categories, namely, spectral restoration based on estimation theory and speech model based methods. The spectral restoration technique treats noise reduction as a robust spectral estimation problem, i.e., estimating the spectrum of the clean speech from that of the noisy speech. In 1984, Ephraim and Malah developed an optimal spectral amplitude and some spectral phase estimators using the statistical estimation theory [44]. Their paper was widely cited in the noise reduction research and (together with McAulay and Malpass’ work) was most notable as an early attempt to apply statistical theory to noise reduction. It is also the first showing, by theory, that the optimal estimate of the clean speech spectral phase is the spectral phase of the noisy speech. Therefore, the basic problem of noise reduction becomes one of estimating only the clean speech spectral amplitude (this result was used previously but mainly based on experimental observations). Following this work, many statistical spectral estimators were developed, including the minimum mean-squared error (MMSE) log-spectral amplitude estimator, the maximum-likelihood (ML) spectral amplitude estimator, the ML spectral power estimator, the maximum a posteriori (MAP) spectral amplitude estimator, etc. Today, there are still tremendous efforts to search for better spectral amplitude estimators, which are inspired by McAulay and Malpass’ and Ephraim and Malah’s work. The model-based approaches also formulate noise reduction as an estimation problem. Comparatively, in this category of methods, a mathematical model is used to represent human speech production and parameter estimation is carried out in the model space, which normally has a much lower dimensionality than the original signal space. Lim and Oppenheim studied the harmonic (or sinusoidal) model for speech and developed a method to reduce noise using comb filtering [86]. In 1987, Paliwal and Basu published a paper that used the linear predictive coding (LPC) model and the Kalman filter to reduce noise [97]. The basic idea underlying the LPC model is that a given speech sample at time k can be approximated as a linear combination of the past few speech samples. The LPC model was well studied and widely used in speech analysis, speech coding, and speech recognition in the 1980s, but Paliwal and Basu were the first to combine the LPC model with the Kalman filter for noise reduction. Since then, many efforts have been made to make this technique more practical. A number of significant milestones were achieved in the 1990s, from both the theoretical and application points of view. From the theoretical side, the hidden Markov models (HMMs) were borrowed from speech recognition and

10

1 Introduction

applied to noise reduction [46], [47], [48]. The HMM based method is basically similar to the statistical spectral estimator developed in the 1980s. The difference is that the statistical spectral estimators assume the explicit knowledge of the joint probability distribution of the clean speech and noise signals so that the conditional expected value of the clean speech (or its sample spectrum), given the noisy speech, can be evaluated, while in the HMM based approach, the unknown probability distributions of the speech and noise are learned from the training sequence. Also, subspace methods were developed by Dendrinos, Bakamidis, and Carayannis [37] and by Ephraim and Van Trees [49]. In essence, the subspace approach projects the noisy signal vector into a different domain either via the Karhunen-Lo`eve (KL) transform through eigenvalue decomposition of an estimate of the correlation matrix of the noisy signal [49] or by using the singular value decomposition of a Toeplitz-structured data matrix specially arranged from the noisy signal vector [78]. Once transformed, the speech signal only spans a portion of the entire space, and as a result, the entire vector space can be divided into two subspaces: the signal-plus-noise and the noise only. The noise statistics can then be estimated from the noise only subspace. From the application side, the noise reduction technology has been increasingly used in telecommunications. Many parametric codecs, such as the enhanced variable rate codec (EVRC), the adaptive multi-rate (AMR) codec, etc., integrated noise reduction into speech compression.

1.4 Organization of the Book The material in this book is organized into twelve chapters, including this one. We attempted to cover the most basic concepts and fundamental techniques used in noise reduction from a signal processing perspective in the next eleven chapters. The material discussed in these chapters is as follows. Chapter 2 provides an overview of the noise reduction problem. The most straightforward and classical approach to tackling this problem is through linear filtering in the time (original signal) domain. However, it is often very advantageous to deal with this problem in a transform space. This is due to the fact that if the transform is properly selected, the speech and noise signals can be better separated in that space, making it easier to estimate the noise statistics, and thereby optimizing the noise reduction filter. So, we discuss how to model the problem not only in the time domain, but also in the two well-known transform spaces, i.e., the frequency and Karhunen-Lo`eve expansion (KLE) domains. A key issue in the design and implementation of noise reduction filters is how to assess their performance. Chapter 3 focuses on the performance metrics. We present several useful measures based on signal processing techniques. These measures will not only help properly design filters in the different domains, but also help us understand how noise reduction works in real-world applications and what price we need to pay for achieving noise reduction.

1.4 Organization of the Book

11

Most noise reduction filters are derived by optimizing some error criteria. The most used error criterion is, by far, the MSE. In Chapter 4, we present the MSE and its normalized form in the time, frequency, and KLE domains. We will demonstrate that the normalized form, called normalized MSE (NMSE, which can be in either subband or fullband) in all domains depends explicitly on the input SNR, and some other performance measures presented in Chapter 3, which makes the NMSE not only useful in deriving different optimal filters, but also powerful in analyzing the noise reduction performance. In the context of noise reduction, it is very important to directly compare the output SNR with the input SNR. This is essential in order to tell whether the filter is indeed able to reduce noise relative to the speech level. However, the MSE and NMSE, which are overwhelmingly used to derive noise reduction filters, are not explicitly related to the output SNR even though intuitively they should depend on it. In Chapter 5, we present an alternative criterion to the MSE, called the squared Pearson correlation coefficient (SPCC), in which the output SNR appears naturally. We develop several forms of this coefficient in the different domains. In Chapter 6, we discuss many interesting properties of the SPCC in the different domains. These fundamental properties establish relationships between the SPCC, the input SNR, the output SNR, and several other performance measures. Armed with these properties it is possible in many cases to derive and analyze optimal noise reduction filters by a simple inspection of a certain form of the SPCC. In Chapter 7, we derive various forms of noise reduction filters in the time domain. The major focus is on the Wiener filter, which is an optimal filter in the minimum MSE (MMSE) sense, and many known algorithms are related to this filter in one way or another. We discuss many fundamental and interesting properties of this filter and demonstrate that the Wiener filter achieves noise reduction at the cost of adding speech distortion. To manage the compromise between noise reduction and speech distortion, we present a tradeoff filter, which is derived from a constrained MSE criterion where a parameter is used to adjust the compromise between the amount of noise reduction and the amount of speech distortion. We also discuss the well-known subspace method, which can be viewed as a special way of implementing the tradeoff filter. While the time domain is the most straightforward domain to work with, the derived noise reduction filters are often less flexible in terms of performance tuning. For example, in practice, noise is not necessarily white and in many cases its energy may concentrate only in some frequencies or frequency bands. In such situations, it is advantageous to design noise reduction filters on a subband basis. In Chapter 8, we investigate the design of noise reduction filters in the frequency domain, which is an alternative to the time domain. The frequency-domain filters are, so far, the most popular approaches to noise reduction because: 1) the filters at different frequencies (or frequency bands) are designed and used independently with each other, this design offers sig-

12

1 Introduction

nificant flexibility in dealing with colored and band-limited noise; 2) most of our knowledge and understanding of speech production and perception is related to frequencies; and 3) thanks to the fast Fourier transform (FFT), the implementation of frequency-domain filters are generally very efficient. We develop some widely used classical frequency-domain filters and discuss their properties. Although it is widely studied, the frequency domain is not the only domain we can work with. There are other domains that may offer more advantages for the design of noise reduction algorithms. In Chapter 9, we consider the KLE domain. We derive two broad classes (the classification is basically based on the subband filter length) of optimal filters in this domain. The first class, similar to the frequency-domain filters, estimates a frame of the clean speech by filtering the corresponding frame of the noisy speech while the second class does noise reduction by filtering not only the current frame but also a number of previous consecutive frames of the noisy speech. We demonstrate that better noise reduction performance can be achieved with the Class II filters when the parameters associated with this class are properly chosen. Chapter 10 is basically an extension of Chapter 9. In this chapter, we introduce a new transform domain, where any unitary (or orthogonal) matrix can be used to construct the forward (for analysis) and inverse (for synthesis) transforms. This new domain can be viewed as a generalization of the KLE domain. The advantages of working in this generalized domain are multiple, such as different transforms can be used to replace each other without any requirement to change the algorithm formulation (optimal filter) and it is easier to fairly compare different transforms for their noise reduction performance. We address the design of different optimal and suboptimal filters in such a generalized transform domain, including the Wiener filter, the parametric Wiener filter, some tradeoff filters, etc. We also compare different transforms, including the KL, Fourier, cosine, and Hadamard transforms, for their noise reduction performance. Chapter 11 can be seen as an extension of Chapter 8. Chapter 8 considers the optimal noise reduction filters in the continuous frequency domain where the frequency-domain representation of a discrete speech signal is obtained through the discrete-time Fourier transform (DTFT). In Chapter 11, we shift our focus from the DTFT domain to the discrete Fourier transform (DFT) domain, where the frequency-domain representation of the input speech signal is achieved through the short-time discrete Fourier transform (STDFT), or simply the short-time Fourier transform (STFT). Using the STFT as opposed to DTFT, we have to take many effects such as circular convolution (which may cause the aliasing problem) and finite frame length (which may change the statistical distribution of the spectrum) into consideration. We formulate the noise reduction problem in the STFT domain as both a filtering and estimation problem. We present statistical models for the speech and noise signals in this domain and derive estimators for the speech signal using various distortion measures.

1.5 Some Notes to the Reader

13

Noise reduction can find numerous applications in various fields of speech processing. It is, however, a very difficult problem since its technical objectives and the acoustic environments can vary significantly from one application to another. As a result, it is very difficult, if not impossible, to find a versatile algorithm that can really work for all the applications in different practical environments. To illustrate the difficulty of this problem, we provide an example in Chapter 12, where we consider the noise reduction problem in the helmet of an astronaut’s spacesuit. Of course, the major focus of this chapter is not the finding of the best noise reduction algorithm for the specific problem; instead, we attempt to provide some deep analysis of the acoustic challenges and the validation procedure of the suggested solutions that can be more useful and can enrich the reader’s fund of knowledge. Although we restrict our attention in this book to noise reduction using a single microphone, we will also briefly examine the possibility of using multiple microphones to achieve noise reduction in this chapter, while most of the studied multichannel techniques can be found in the book Microphone Array Signal Processing [15].

1.5 Some Notes to the Reader Successful noise reduction systems require the knowledge and expertise from a wide range of disciplines such as signal processing, statistics, estimation and detection theory, pattern recognition, speech processing, communications, and speech perception, to name a few. It is very difficult for any single person to master all these disciplines. The philosophy and purpose of this book is to provide in-depth discussions of a number of fundamental topics in noise reduction research from a signal processing viewpoint so that the reader can have a good understanding of the fundamentals of this problem without necessarily having to be an expert in all disciplines enumerated above. Our basic goal is to obtain optimal filters for noise reduction and we resort to suboptimal filters if the compromise between the amount of noise reduction and the amount of speech distortion has to be made. We have tried to give deep insights into the noise reduction problem by including results that we deem to be most useful and important in practice. The mathematical notation for all symbols is defined within the context of the discussion and when it first appears in the text. In general, (column) vectors are bold lowercase and matrices are bold uppercase.

2 Problem Formulation

In this chapter, we explain the traditional formulation of the (additive) noise problem and its reduction by linear filtering in the time, frequency, and Karhunen-Lo`eve expansion domains.

2.1 In the Time Domain The noise reduction problem consists of recovering the signal of interest (clean speech or desired signal) x(k) of zero mean from the noisy observation (microphone signal) y(k) = x(k) + v(k),

(2.1)

where k is the discrete time index, and v(k) is the unwanted additive noise, which is assumed to be a zero-mean random process (white or colored) and uncorrelated with x(k). The signal model given in (2.1) can be written in a vector form if we process the data by blocks of L samples: y(k) = x(k) + v(k),

(2.2)

where
T y(k) = y(k) y(k − 1) · · · y(k − L + 1) ,
T x(k) = x(k) x(k − 1) · · · x(k − L + 1) ,
T v(k) = v(k) v(k − 1) · · · v(k − L + 1) ,

(2.3a) (2.3b) (2.3c)

and superscript T denotes transpose of a vector or a matrix. Since x(k) and v(k) are uncorrelated, the correlation matrix of the noisy signal is equal to the sum of the correlation matrices of the desired and noise signals, i.e., Ry = Rx + Rv , J. Benesty et al., Noise Reduction in Speech Processing, Springer Topics in Signal Processing 2, DOI 10.1007/978-3-642-00296-0_2, © Springer-Verlag Berlin Heidelberg 2009

(2.4)

16

2 Problem Formulation

where
 Ry = E y(k)yT (k) ,
 Rx = E x(k)xT (k) ,
 Rv = E v(k)vT (k) ,

(2.5a) (2.5b) (2.5c)

are the correlation matrices1 of the signals y(k), x(k), and v(k), respectively, and E[·] denotes mathematical expectation. The objective of noise reduction is to estimate x(k) from the observation vector y(k). Usually, we estimate the noise-free speech, x(k), by applying a linear transformation to the microphone signal [10], [11], [25], [72], [88], [119], i.e., z(k) = Hy(k) = H [x(k) + v(k)]

(2.6)

= xF (k) + vF (k), where H is a filtering matrix of size L × L, xF (k) = Hx(k) is the filtered clean speech (or filtered desired signal), and vF (k) = Hv(k) is the filtered noise, which is often called the residual noise. The correlation matrix of the estimated signal is
 Rz = E z(k)zT (k) = HRx HT + HRv HT .

(2.7)

Therefore, with this time-domain formulation, the noise reduction problem becomes one of finding an optimal filter that would attenuate the noise as much as possible while keeping the clean speech from being dramatically distorted.

2.2 In the Frequency Domain In the frequency domain, (2.1) can be rewritten as Y (jω) = X(jω) + V (jω),

(2.8)

where j is the imaginary unit (j 2 = −1), and Y (jω), X(jω), and V (jω) are respectively the discrete-time Fourier transforms (DTFTs) of y(k), x(k), and v(k), at angular frequency ω (−π < ω ≤ π). Another possible form for (2.8) is Y (ω)ejϕy (ω) = X(ω)ejϕx (ω) + V (ω)ejϕv (ω) , 1

(2.9)

Here, we implicitly assume that all involved signals are stationary or quasistationary.

2.2 In the Frequency Domain

17

where for any random signal A(jω) = A(ω)ejϕa (ω) , A(ω) and ϕa (ω) are its amplitude and phase at frequency ω, A ∈ {Y, X, V }, a ∈ {y, x, v}. We recall that the DTFT and the inverse transform [96] are2 A(jω) =

∞ 

a(k)e−jωk ,

k=−∞

a(k) =

1 2π



π

A(jω)ejωk dω.

(2.10) (2.11)

−π

Using the power spectral density (PSD) of the noisy signal and the fact that x(k) and v(k) are uncorrelated, we get φy (ω) = φx (ω) + φv (ω), where

(2.12)

  2 φa (ω) = E |A(jω)|
 = E A2 (ω)

is the PSD of the signal a(k) [which is the inverse DTFT of A(jω)] at frequency ω. An estimate of X(jω) can be obtained by multiplying Y (jω) with a complex gain, i.e., Z(jω) = H(jω)Y (jω) = H(jω) [X(jω) + V (jω)]

(2.13)

= XF (jω) + VF (jω), where Z(jω) is the frequency-domain representation of the signal z(k), and XF (jω) = H(jω)X(jω) and VF (jω) = H(jω)V (jω) are, respectively, the filtered clean speech and noise in the frequency domain. The PSD of z(k) can then be written as 2

φz (ω) = |H(jω)| φy (ω) 2

= |H(jω)| [φx (ω) + φv (ω)] .

(2.14)

We can go back to the time domain using (2.11) to obtain the estimate  π 1 z(k) = Z(jω)ejωk dω. (2.15) 2π −π 2

For the DTFT to exist, the time series a(k) must be absolutely summable, i.e., +∞ k=−∞ |a(k)| < ∞. This condition does not hold for stationary signals. Nevertheless, we will still use this spectral representation for stationary signals in the rest of this book for convenience of presentation and notation, knowing that the discrete time Fourier (DFT) will lead to similar theoretical results.

18

2 Problem Formulation

The objective of noise reduction in the frequency domain is then to find an optimal filter3 H(jω) at each frequency ω that would attenuate the noise greatly with as little distortion as possible to the desired signal (speech).

2.3 In the Karhunen-Lo` eve Expansion (KLE) Domain We first briefly recall the basic principle of the so-called Karhunen-Lo`eve expansion (KLE) and then show how we can work in the KLE domain. Let the L × 1 vector x(k) denote a data sequence drawn from a zeromean stationary process with the correlation matrix Rx . This matrix can be diagonalized as follows [59] QT Rx Q = Λ,

(2.16)

where
 Q = q1 q2 · · · qL and
 Λ = diag λ1 λ2 · · · λL are, respectively, orthogonal and diagonal matrices. The orthonormal vectors q1 , q2 , . . . , qL are the eigenvectors corresponding, respectively, to the eigenvalues λ1 , λ2 , . . . , λL of the matrix Rx . The vector x(k) can be written as a combination (expansion) of the eigenvectors of the correlation matrix Rx as follows x(k) =

L 

cx,l (k)ql ,

(2.17)

l=1

where cx,l (k) = qTl x(k), l = 1, 2, . . . , L

(2.18)

are the coefficients of the expansion and l is the subband index. The representation of the random vector x(k) described by (2.17) and (2.18) is the Karhunen-Lo`eve expansion (KLE) [63]. Equations (2.17) and (2.18) are, respectively, the synthesis and analysis parts of this expansion. From (2.18), we can easily verify that E [cx,l (k)] = 0, l = 1, 2, . . . , L 3

As will become clearer in Chapter 8, such an positive, and its value is usually in the range to the noisy speech, this filter only modifies phase of Y (jω). As a result, it is often called

(2.19)

optimal filter is in general real and between 0 and 1. So, when applied the gain, but does not change the a gain filter.

2.3 In the Karhunen-Lo`eve Expansion (KLE) Domain

and

 E [cx,i (k)cx,j (k)] =

λi , i = j . 0, i = j

19

(2.20)

It can also be checked from (2.18) that L  l=1

2

c2x,l (k) = x(k)2 ,

(2.21)

where x(k)2 is the Euclidean norm of x(k). The previous expression shows the energy conservation through the KLE process. One of the most important aspects of the KLE is its potential to reduce the dimensionality of the vector x(k) for low-rank signals. This idea has been extensively exploited in different manners for noise reduction where the signal of interest (speech) is assumed to be a low-rank signal. In the following, we show how to work directly in the KLE domain. Left-multiplying (2.2) by qTl , we get cy,l (k) = qTl y(k)

= qTl x(k) + qTl v(k) = cx,l (k) + cv,l (k), l = 1, 2, . . . , L.

(2.22)

Expression (2.22) is equivalent to (2.2) but in the KLE domain. Again, we see that L  l=1 L  l=1

2

(2.23)

2

(2.24)

c2y,l (k) = y(k)2 , c2v,l (k) = v(k)2 .

We also have

 E [cy,i (k)cy,j (k)] =

λi + qTi Rv qi , i = j . qTi Rv qj , i = j

(2.25)

In the rest of this work, we assume that |qTi Rv qj |  λi + qTi Rv qi or |qTi Rv qj | ≈ 0, for i = j 4 , so that we can estimate each one of the coefficients cx,l , l = 1, 2, . . . , L, without being related to the others. Clearly, our problem this time is to find an estimate of cx,l (k) by passing cy,l (k) through a linear filter, i.e., cz,l (k) = hTl cy,l (k) = hTl [cx,l (k) + cv,l (k)] , l = 1, 2, . . . , L, 4

If the noise is white, then qTi Rv qj = 0, ∀i = j.

(2.26)

20

2 Problem Formulation

where
T hl = hl,0 hl,1 · · · hl,Ll −1

(2.27)

is a finite-impulse-response (FIR) filter of length Ll , and
T cy,l (k) = cy,l (k) cy,l (k − 1) · · · cy,l (k − Ll + 1) ,
T cx,l (k) = cx,l (k) cx,l (k − 1) · · · cx,l (k − Ll + 1) ,
T cv,l (k) = cv,l (k) cv,l (k − 1) · · · cv,l (k − Ll + 1) .

(2.28a) (2.28b) (2.28c)

We see that the filters hl , l = 1, 2, . . . , L, can take different lengths in the different subbands. The variance of the signal cz,l (k) is
 E c2z,l (k) = hTl Rcx ,l hl + hTl Rcv ,l hl , l = 1, 2, . . . , L,

(2.29)

where
 Rcx ,l = E cx,l (k)cTx,l (k) ,
 Rcv ,l = E cv,l (k)cTv,l (k) , are the correlation matrices of the signals cx,l (k) and cv,l (k), respectively. Apparently, reducing the noise in the KLE domain comes to the design of the FIR filters hl , l = 1, 2, . . . , L. Finally, an estimate of the vector x(k) would be z(k) =

L 

cz,l (k)ql .

(2.30)

l=1

2.4 Summary In this chapter, we have described the fundamental formulation of the noise reduction problem in the time, frequency, and KLE domains. This formulation will be frequently used throughout the rest of this book to derive the most well-known and useful noise reduction filters.

3 Performance Measures

In this chapter, we present some very useful measures that are necessary to properly design filters in the different domains. These definitions will also help us better understand how noise reduction works in real-world applications and what price we need to pay for this. As a matter of fact, all known approaches in the single-channel (one microphone) case add distortion to the desired signal. Therefore, in most applications, it is essential to find a good compromise between the amount of noise reduction and the degree of speech distortion, and the measures presented in this chapter can guide us to achieve such a compromise.

3.1 Signal-to-Noise Ratio One of the most important measures in noise reduction is the signal-to-noise ratio (SNR). From the signal model given in (2.1), we define the input SNR as the ratio of the intensity of the signal of interest (speech) over the intensity of the background noise, i.e., iSNR =

σx2 , σv2

(3.1)

where
 σx2 = E x2 (k) and
 σv2 = E v 2 (k) are the variances of the signals x(k) and v(k), respectively. This definition of the input SNR can also be written in different forms. With the signal model shown in (2.2), it is easy to check that

J. Benesty et al., Noise Reduction in Speech Processing, Springer Topics in Signal Processing 2, DOI 10.1007/978-3-642-00296-0_3, © Springer-Verlag Berlin Heidelberg 2009

22

3 Performance Measures

σx2 = tr (Rx ) /L and σv2 = tr (Rv ) /L, where tr(·) denotes the trace of a matrix. Therefore, the input SNR can be written as iSNR =

tr (Rx ) . tr (Rv )

According to the Parseval’s relation, we have  π 1 σx2 = φx (ω)dω 2π −π and σv2 =

1 2π



(3.2)

(3.3)

π −π

φv (ω)dω.

(3.4)

Substituting (3.3) and (3.4) into (3.1), we get another form of the input SNR: π φx (ω)dω iSNR = −π . (3.5) π φ (ω)dω −π v In the KLE domain, if we apply the matrix diagonalization of (2.16) to (3.2), we can readily deduce that L

l=1 λl . T l=1 ql Rv ql

iSNR = L

(3.6)

In the frequency and KLE domains, it is also important to examine the SNR in each subband. So, we define the subband input SNRs in these two domains as iSNR(ω) =

φx (ω) , ω ∈ (−π, π] φv (ω)

(3.7)

and iSNRl =

λl , l = 1, 2, . . . , L. qTl Rv ql

(3.8)

After noise reduction with the time-domain model given in (2.6), the output SNR can be written as

3.1 Signal-to-Noise Ratio


 E xTF (k)xF (k)  oSNR(H) =
T E vF (k)vF (k)

 tr HRx HT . =

tr HRv HT

23

(3.9)

One of the most important goals of noise reduction is to improve the SNR after filtering [11], [24]. Therefore, we must design a filter, H, in such a way that oSNR(H) ≥ iSNR. In the frequency and KLE domains, the subband output SNRs are 2

oSNR [H(jω)] =

|H(jω)| φx (ω) 2

|H(jω)| φv (ω) = iSNR(ω), ω ∈ (−π, π]

(3.10)

and oSNR(hl ) =

hTl Rcx ,l hl , l = 1, 2, . . . , L. hTl Rcv ,l hl

(3.11)

In general, oSNR(hl ) = iSNRl except when hl is a scalar. It is interesting to observe that the frequency-domain subband output SNR is not influenced by H(jω). We now define the fullband output SNRs for the frequency- and KLEdomain filters: π 2 |H(jω)| φx (ω)dω oSNR(H) = −π (3.12) π 2 |H(jω)| φv (ω)dω −π and L hTl Rcx ,l hl oSNR (h1:L ) = l=1 . L T l=1 hl Rcv ,l hl

(3.13)

It is of great importance to find the complex gains H(jω), ω ∈ (−π, π] and the FIR filters hl , l = 1, 2, . . . , L, in such a way that oSNR(H) ≥ iSNR and oSNR (h1:L ) ≥ iSNR. Property 3.1. We always have L 

iSNRl ≥ iSNR,

(3.14)

l=1 L  l=1

oSNR(hl ) ≥ oSNR (h1:L ) .

(3.15)

24

3 Performance Measures

This means that the aggregation of the subband SNRs is greater than or equal to the fullband SNR. Proof. The two previous inequalities can be shown by using the following inequality: 

  al a bl al l l = · ≤ , (3.16) b b b bl l l i l i l

l

where al and bl are positive reals.

3.2 Noise-Reduction Factor Another important measure in noise reduction is the noise-reduction factor, which quantifies the amount of noise being attenuated by the filter. With the time-domain formulation, this factor is defined as [11], [24] ξnr (H) =

tr (Rv )

. tr HRv HT

(3.17)

By analogy to the above time-domain definition, we define the subband noisereduction factors in the frequency and KLE domains as ξnr [H(jω)] =

φv (ω) 2

|H(jω)| φv (ω) 1 = 2 , ω ∈ (−π, π] |H(jω)|

(3.18)

and ξnr (hl ) =

qTl Rv ql , l = 1, 2, . . . , L. hTl Rcv ,l hl

(3.19)

The larger the value of ξnr [H(jω)] [or ξnr (hl )], the more the noise is reduced at frequency ω (or subband l). After the filtering operation, the residual noise level at frequency ω (or subband l) is expected to be lower than that of the original noise level, therefore this factor should have a lower bound of 1. The fullband noise-reduction factors for the frequency- and KLE-domain filters are π φv (ω)dω ξnr (H) = π −π (3.20) 2 |H(jω)| φv (ω)dω −π π φv (ω)dω = π −1−π ξ [H(jω)] φv (ω)dω −π nr

3.3 Speech-Distortion Index

25

and L

qT Rv ql ξnr (h1:L ) = Ll=1 Tl l=1 hl Rcv ,l hl L qTl Rv ql = L l=1 . −1 T l=1 ξnr (hl ) ql Rv ql

(3.21)

The fullband noise-reduction factor for the frequency-domain (or KLEdomain) approach is the ratio of the energy of the noise over the weighted −1 −1 energy of the noise with the weighting ξnr [H(jω)] [or ξnr (hl )]. Same as in (3.18) [or (3.19)], ξnr (H) [or ξnr (h1:L )] is expected to be lower bounded by 1. Indeed, if ξnr [H(jω)] ≥ 1, ∀ω [or ξnr (hl ) ≥ 1, ∀l], we deduce from (3.20) [or (3.21)] that ξnr (H) ≥ 1 [or ξnr (h1:L ) ≥ 1]. Property 3.2. We always have L 

ξnr (hl ) ≥ ξnr (h1:L ) .

(3.22)

l=1

This means that the aggregation of the subband noise-reduction factors in the KLE domain is greater than or equal to the fullband noise-reduction factor. Proof. This inequality can be shown by using (3.16).

3.3 Speech-Distortion Index The filtering operation adds distortion to the speech signal. In order to evaluate the amount of speech distortion, the concept of speech-distortion index has been introduced in [11], [24]. With the time-domain model, the speechdistortion index is defined as   T E [xF (k) − x(k)] [xF (k) − x(k)] υsd (H) = E [xT (k)x(k)]   T E [Hx(k) − x(k)] [Hx(k) − x(k)] = tr (Rx )
 tr (H − I)Rx (H − I)T = , (3.23) tr (Rx ) where I is the identity matrix. We can extend this definition to other domains. Indeed, we define the subband speech-distortion indices in the frequency and KLE domains as

26

3 Performance Measures

υsd [H(jω)] =

  2 E |H(jω)X(jω) − X(jω)| φx (ω) 2

= |1 − H(jω)| , ω ∈ (−π, π] and E

(3.24)

 2  hTl cx,l (k) − cx,l (k)

υsd (hl ) =

, l = 1, 2, . . . , L.

λl

(3.25)

The subband speech-distortion index has a lower bound of 0 and an upper bound of 1 for optimal filters to be derived later in this book. The higher the value of υsd [H(jω)] [or υsd (hl )], the more the speech is distorted at frequency ω (or subband l). The fullband speech-distortion indices for the frequency- and KLE-domain filters are   π 2 E |H(jω)X(jω) − X(jω)| dω −π π υsd (H) = φ (ω)dω −π x π 2 φx (ω) |1 − H(jω)| dω = −π π φ (ω)dω −π x π υsd [H(jω)] φx (ω)dω = −π π (3.26) φ (ω)dω −π x and L

l=1 E

υsd (h1:L ) = L =

 2  hTl cx,l (k) − cx,l (k) L l=1 λl

υsd (hl ) λl . L l=1 λl

l=1

(3.27)

Equation (3.26) [or (3.27)] is the ratio of the weighted energy of the speech with the weighting υsd [H(jω)] [or υsd (hl )] over the energy of the speech. If υsd [H(jω)] ≤ 1, ∀ω [or υsd (hl ) ≤ 1, ∀l], we see from (3.26) [or (3.27)] that υsd (H) ≤ 1 [or υsd (h1:L ) ≤ 1]. Property 3.3. We always have L 

υsd (hl ) ≥ υsd (h1:L ) .

(3.28)

l=1

This means that the aggregation of the subband speech-distortion indices in the KLE domain is greater than or equal to the fullband speech-distortion index. Proof. Easy to show by using the inequality (3.16).

3.4 Speech-Reduction Factor

27

3.4 Speech-Reduction Factor This measure is somewhat similar to the noise-reduction factor. Since the noise is reduced by the filtering operation, so is the speech. This speech reduction implies, in general, speech distortion. With the time-domain formulation, the speech-reduction factor is defined as ξsr (H) =

tr (Rx )

. tr HRx HT

(3.29)

We define the subband speech-reduction factors in the frequency and KLE domains as ξsr [H(jω)] =

φx (ω) 2

|H(jω)| φx (ω) 1 = 2 , ω ∈ (−π, π] |H(jω)|

(3.30)

and ξsr (hl ) =

λl , T hl Rcx ,l hl

l = 1, 2, . . . , L.

(3.31)

The larger the value of ξsr [H(jω)] [or ξsr (hl )], the more the speech is reduced at frequency ω (or subband l). After the filtering operation, the speech level at frequency ω (or subband l) is typically lower than that of the original speech level, therefore this factor should have a lower bound of 1. The fullband speech-reduction factors for the frequency- and KLE-domain filters are π φx (ω)dω ξsr (H) = π −π (3.32) 2 |H(jω)| φx (ω)dω −π π φx (ω)dω = π −1−π ξ [H(jω)] φx (ω)dω −π sr and L ξsr (h1:L ) = L

l=1

λl

T l=1 hl Rcx ,l hl L λl = L l=1 . −1 l=1 ξsr (hl ) λl

(3.33)

The fullband speech-reduction factor for the frequency domain (or KLE domain) approach is the ratio of the energy of the speech over the weighted

28

3 Performance Measures

−1 −1 energy of the speech with the weighting ξsr [H(jω)] [or ξsr (hl )]. Same as in (3.30) [or (3.31)], ξsr (H) [or ξsr (h1:L )] is expected to be lower bounded by 1. Indeed, if ξsr [H(jω)] ≥ 1, ∀ω [or ξsr (hl ) ≥ 1, ∀l], we deduce from (3.32) [or (3.33)] that ξsr (H) ≥ 1 [or ξsr (h1:L ) ≥ 1].

Property 3.4. We always have L 

ξsr (hl ) ≥ ξsr (h1:L ) .

(3.34)

l=1

This means that the aggregation of the subband speech-reduction factors in the KLE domain is greater than or equal to the fullband speech-reduction factor. Proof. This inequality can be easily shown by using (3.16).

3.5 Discussion We have presented several measures in this chapter. It is fair to ask whether all these measures are relevant and could give us a good idea on how a specific linear filter will behave in terms of reducing the noise and distorting the desired signal. The input SNR (independent of the filtering operation) is certainly a measure of great importance from both signal processing and perception points of view. Indeed, the human ear can do a pretty good job in judging the SNR values when we listen to some audio signals that are corrupted by additive noise with different SNRs. The subband input SNR, which is a narrowband definition, is less obvious to understand in our context since we usually do not listen to narrowband signals. But this measure is relevant since it is closely related to the input SNR. The output SNR (computed after the processing is done in the time domain) and the fullband output SNR (computed after the processing is done in the transform domain) are also very reliable. They tell us indeed whether the SNR is improved or not and by how much. The noise-reduction factor is a relative measure because it does depend on the distortion (or reduction) of the desired signal. It is possible in some situations to have this factor much larger than 1 and yet the SNR has not been improved. So this measure has to be handled with care. The speech-distortion index and speech-reduction factor are very rough measures of distortion. Refined measures able to detect different kinds of distortions are more complicated to derive and much more research is needed in this topic. Nevertheless, four of the (fullband) measures are simply related as shown in the following property: Property 3.5. We always have oSNR(H) ξnr (H) = . iSNR ξsr (H)

(3.35)

3.5 Discussion

29

Proof. This is easy to see by combining expressions (3.1), (3.9), (3.17), and (3.29). From (3.35) we observe that oSNR(H) > iSNR if and only if ξnr (H) > ξsr (H). So is it possible that with a judicious choice of the filtering matrix, H, we can have ξnr (H) > ξsr (H)? The answer is yes. A generally rough and intuitive justification to this answer is quite simple: improvement of the SNR is due to the fact that speech signals are partly predictable. In this situation, H is a kind of a complex predictor or interpolation matrix and as a result, ξsr (H) can be close to 1 while ξnr (H) can be much larger than 1. This observation is essential in order to understand how noise reduction happens when it happens. We have similar relations to (3.35) for the filters in the frequency and KLE domains: oSNR(H) ξnr (H) = , iSNR ξsr (H) oSNR(h1:L ) ξnr (h1:L ) = . iSNR ξsr (h1:L )

(3.36) (3.37)

Even though the complex gains H(jω), ω ∈ (−π, π] and the FIR filters hl , l = 1, 2, . . . , L, may be interpreted differently than the time-domain filtering matrix H (since the formers act mostly like gain functions), the objective stays the same.

4 Mean-Squared Error Criterion

Error criteria play a critical role in deriving optimal noise reduction filters. Although many different criteria can be defined, the mean-squared error (MSE) is, by far, the most used one because of its simplicity in terms of deriving useful filters. This chapter presents relevant error signals from which meaningful MSE criteria are built in the different domains.

4.1 In the Time Domain We define the error signal vector between the estimated and desired signals as e(k) = z(k) − x(k)

(4.1)

= Hy(k) − x(k), which can also be written as the sum of two error signal vectors: e(k) = ex (k) + ev (k),

(4.2)

ex (k) = (H − I) x(k)

(4.3)

where

is the speech distortion due to the linear transformation, and ev (k) = Hv(k)

(4.4)

represents the residual noise [49]. Having defined the error signal, we can now write the MSE criterion: 
 J (H) = tr E e(k)eT (k) (4.5)

 = tr (Rx ) + tr HRy HT − 2tr (HRyx )

 = tr (Rx ) + tr HRy HT − 2tr (HRx ) ,

J. Benesty et al., Noise Reduction in Speech Processing, Springer Topics in Signal Processing 2, DOI 10.1007/978-3-642-00296-0_4, © Springer-Verlag Berlin Heidelberg 2009

32

4 Mean-Squared Error Criterion

where
 Ryx = E y(k)xT (k) is the cross-correlation matrix between the observation and desired signals, which can also be expressed as Ryx = Rx
 since Rvx = E v(k)xT (k) = 0 [x(k) and v(k) are assumed to be uncorrelated]. Similarly, using the uncorrelation assumption, expression (4.5) can be structured in terms of two MSEs, i.e., 
 
 J (H) = tr E ex (k)eTx (k) + tr E ev (k)eTv (k) (4.6) = Jx (H) + Jv (H) . For the particular transformation H = I (the identity matrix), we get J (I) = tr (Rv ) ,

(4.7)

so there will be neither noise reduction nor speech distortion. Using this particular case of the MSE, we define the normalized MSE (NMSE) as J (H) J˜ (H) = J (I) = iSNR · υsd (H) +

1 , ξnr (H)

(4.8)

where Jx (H) , tr (Rx ) tr (Rv ) ξnr (H) = . Jv (H)

υsd (H) =

(4.9) (4.10)

This shows the connection between the NMSE and the speech-distortion index and the noise-reduction factor defined in Chapter 3.

4.2 In the Frequency Domain We define the error signal between the estimated and desired signals at frequency ω as E(jω) = Z(jω) − X(jω) = H(jω)Y (jω) − X(jω).

(4.11)

4.2 In the Frequency Domain

33

This error can also be put into the form: E(jω) = Ex (jω) + Ev (jω),

(4.12)

Ex (jω) = [H(jω) − 1] X(jω)

(4.13)

where

is the speech distortion due to the complex filter, and Ev (jω) = H(jω)V (jω)

(4.14)

represents the residual noise. The frequency-domain (or subband) MSE is then   2 J [H(jω)] = E |E(jω)|

(4.15)

2

= φx (ω) + |H(jω)| φy (ω) − 2R [H(jω)φyx (jω)] 2

= φx (ω) + |H(jω)| φy (ω) − 2R [H(jω)φx (ω)] , where R(·) is the real part of a complex number and φyx (jω) = E [Y (jω)X ∗ (jω)] = φx (ω) is the cross-spectrum between the observation and speech signals. The subband MSE is also     2 2 J [H(jω)] = E |Ex (jω)| + E |Ev (jω)| (4.16) = Jx [H(jω)] + Jv [H(jω)] . For the particular gain H(jω) = 1, ∀ω, we get J (1) = φv (ω),

(4.17)

so there will be neither noise reduction nor speech distortion. With this particular case of the MSE, we define the frequency-domain (or subband) NMSE as J [H(jω)] J˜ [H(jω)] = J (1) = iSNR(ω) · υsd [H(jω)] +

1 , ξnr [H(jω)]

(4.18)

where Jx [H(jω)] , φx (ω) φv (ω) ξnr [H(jω)] = . Jv [H(jω)]

υsd [H(jω)] =

(4.19) (4.20)

34

4 Mean-Squared Error Criterion

The frequency-domain NMSE depends explicitly on the subband speechdistortion index and the subband noise-reduction factor. Sometimes, it is also important to examine the MSE from the fullband point of view. We define the fullband MSE and fullband NMSE as  π 1 J (H) = J [H(jω)] dω (4.21) 2π −π  π  π 1 1 2 2 = φx (ω) |1 − H(jω)| dω + φv (ω) |H(jω)| dω 2π −π 2π −π = Jx (H) + Jv (H) and J (H) J˜ (H) = 2π π φ (ω)dω −π v π π 2 2 φ (ω) |1 − H(jω)| dω φv (ω) |H(jω)| dω −π x π = + −π π φ (ω)dω φ (ω)dω −π v −π v 1 = iSNR · υsd (H) + , ξnr (H)

(4.22)

where Jx (H) , φ (ω)dω −π x π φv (ω)dω ξnr (H) = −π . Jv (H)

υsd (H) = π

(4.23) (4.24)

Again, we see clearly that the fullband NMSE depends explicitly on the fullband speech-distortion index and the fullband noise-reduction factor.

4.3 In the KLE Domain In the KLE domain, the error signal between the estimated and desired signals in the subband l is el (k) = cz,l (k) − cx,l (k) =

hTl cy,l (k)

(4.25)

− cx,l (k),

which can also be written as the sum of two error signals: el (k) = ex,l (k) + ev,l (k), where

(4.26)

4.3 In the KLE Domain

ex,l (k) = hTl cx,l (k) − cx,l (k)

35

(4.27)

is the speech distortion due to the FIR filter, and ev,l (k) = hTl cv,l (k)

(4.28)

represents the residual noise. From the error signal (4.25), we give the corresponding KLE-domain (or subband) MSE criterion:
 J (hl ) = E e2l (k) (4.29) = λl + hTl Rcy ,l hl − 2hTl rcy cx ,l , where
 Rcy ,l = E cy,l (k)cTy,l (k) , rcy cx ,l = E [cy,l (k)cx,l (k)] , are the correlation matrix of the signal cy,l (k) and cross-correlation vector between the signals cy,l (k) and cx,l (k), respectively. Expression (4.29) can be structured in a different way:

 J (hl ) = E e2x,l (k) + E e2v,l (k) (4.30) = Jx (hl ) + Jv (hl ) . For the particular FIR filter
T il = 1 0 · · · 0 of length Ll , we get


 J (il ) = E c2v,l (k) = qTl Rv ql ,

(4.31)

so there will be neither noise reduction nor speech distortion. Using this particular case of the MSE, we define the KLE-domain (or subband) NMSE as J (hl ) J˜ (hl ) = J (il ) = iSNRl · υsd (hl ) +

1 , ξnr (hl )

(4.32)

where Jx (hl ) , λl qT Rv ql ξnr (hl ) = l . Jv (hl )

υsd (hl ) =

(4.33) (4.34)

36

4 Mean-Squared Error Criterion

The KLE-domain NMSE depends explicitly on the subband speech-distortion index and the subband noise-reduction factor. We define the fullband MSE and fullband NMSE as L

J (h1:L ) =

1 J (hl ) L

(4.35)

l=1

=

L

L

l=1

l=1

1 1 T (hl − il )T Rcx ,l (hl − il ) + hl Rcv ,l hl L L

= Jx (h1:L ) + Jv (h1:L ) and J (h1:L ) J˜ (h1:L ) = L L T l=1 ql Rv ql L L (hl − il )T Rcx ,l (hl − il ) hTl Rcv ,l hl = l=1 L + l=1 L T T l=1 ql Rv ql l=1 ql Rv ql 1 = iSNR · υsd (h1:L ) + , ξnr (h1:L )

(4.36)

where Jx (h1:L ) υsd (h1:L ) = L , l=1 λl L qT Rv ql ξnr (h1:L ) = l=1 l . Jv (h1:L )

(4.37) (4.38)

Same as for the frequency-domain approach, the fullband NMSE with the KLE depends explicitly on the fullband speech-distortion index and the fullband noise-reduction factor.

4.4 Summary In this chapter, we have presented the MSE and NMSE criteria in the time, frequency, and KLE domains. It has been demonstrated that the (subband or fullband) NMSE in all domains depends explicitly on the input SNR, speechdistortion index, and noise-reduction factor, which makes the NMSE not only useful in deriving different optimal filters, but also powerful in analyzing noise reduction performance. In noise reduction, it is also very important to directly compare the output SNR with the input SNR. This is essential in order to tell whether the filter is really able to achieve noise reduction. However, the MSE and NMSE do not show explicitly the output SNR even though intuitively (and implicitly) they should depend on it. In the next chapter, we will present another criterion, called the Pearson correlation coefficient, in which the output SNR appears naturally.

5 Pearson Correlation Coefficient

This chapter develops several forms of the Pearson correlation coefficient in the different domains. This coefficient can be used as an optimization criterion to derive different optimal noise reduction filters [14], but is even more useful for analyzing these optimal filters for their noise reduction performance.

5.1 Correlation Coefficient Between Two Random Variables Let a and b be two zero-mean real-valued random variables. The Pearson correlation coefficient (PCC) is defined as1 [41], [98], [105] ρ (a, b) =

E (ab) , σa σb

(5.1)

E 2 (ab) . σa2 σb2

(5.2)

  where E (ab) is the cross-correlation between a and b, and σa2 = E a2 and σb2 = E b2 are the variances of the signals a and b, respectively. In the rest, it will be more convenient to work with the squared Pearson correlation coefficient (SPCC): ρ2 (a, b) =

One of the most important properties of the SPCC is that 0 ≤ ρ2 (a, b) ≤ 1.

(5.3)

The SPCC gives an indication on the strength of the linear relationship between the two random variables a and b. If ρ2 (a, b) = 0, then a and b are said 1

This correlation coefficient is named after Karl Pearson who described many of its properties. Actually to be precise, the true definition of the PCC is when the expectation operator in (5.1) is replaced by a sum over the samples.

J. Benesty et al., Noise Reduction in Speech Processing, Springer Topics in Signal Processing 2, DOI 10.1007/978-3-642-00296-0_5, © Springer-Verlag Berlin Heidelberg 2009

38

5 Pearson Correlation Coefficient

to be uncorrelated. The closer the value of ρ2 (a, b) is to 1, the stronger the correlation between the two variables. If the two variables are independent, then ρ2 (a, b) = 0. But the converse is not true because the SPCC detects only linear dependencies between the two variables a and b. For a non-linear dependency, the SPCC may be equal to zero. However, in the special case when a and b are jointly normal, “independent” is equivalent to “uncorrelated.”

5.2 Correlation Coefficient Between Two Random Vectors We can generalize the idea of the SPCC between two random variables to two random vectors. Indeed, let
T a = a1 a2 · · · aL ,
T b = b1 b2 · · · bL , be two zero-mean real-valued random vectors of length L. We define the SPCC between a and b as   E 2 aT b 2

. ρ (a, b) = (5.4) E (aT a) E bT b Let Πa and Πb be two permutation matrices. If Πa = Πb then ρ2 (Πa a, Πa b) = ρ2 (a, b). In general, if Πa = Πb , we have ρ2 (Πa a, Πb b) = ρ2 (a, b). Property 5.1. We always have 0 ≤ ρ2 (a, b) ≤ 1.

(5.5)

Proof. From the definition (5.4), it is clear that ρ2 (a, b) ≥ 0. To show that ρ2 (a, b) ≤ 1, let us define the positive quantity:
 E (a − cb)T (a − cb) ≥ 0, (5.6) where c is a real number. The development of the previous expression gives:


     E (a − cb)T (a − cb) = E aT a − 2cE aT b + c2 E bT b . (5.7) In particular, for   E aT a c= E (aT b) we get

(5.8)

5.4 KLE-Domain Versions



  E 2 aT a E bT b    E aT a − 2E aT a + ≥ 0, E 2 (aT b) 

which implies that

  

E aT a E bT b E 2 (aT b)

39



≥ 1.

(5.9)

(5.10)

Therefore ρ2 (a, b) ≤ 1.

5.3 Frequency-Domain Versions Let A(jω) and B(jω) be the DTFTs of the two zero-mean real-valued random variables a and b. We define the subband SPCC, which is also known as the magnitude squared coherence function (MSCF), between A(jω) and B(jω) at frequency ω as 2

|ρ [A(jω), B(jω)]| =

|E [A(jω)B ∗ (jω)]|     2 2 E |A(jω)| E |B(jω)| 2

(5.11)

2

=

|φab (jω)| , φa (ω)φb (ω)

where superscript ∗ denotes complex conjugation. It is clear that the subband SPCC always takes its values between 0 and 1. We can generalize this idea to infinite vectors (containing all frequencies). We will refer to this definition as the fullband SPCC, which also takes its values between 0 and 1. In this case, we have   2 π   ∗ A(jω)B (jω)dω E  −π 2    |ρ (A, B)| =  π (5.12) π 2 2 E −π |A(jω)| dω E −π |B(jω)| dω  2  π   −π φab (jω)dω     =  π π φ (ω)dω φ (ω)dω −π a −π b =

E 2 (ab) σa2 σb2

= ρ2 (a, b) .

5.4 KLE-Domain Versions Let a and b be two zero-mean real-valued random variables and ca,l and cb,l their respective representations in the KLE domain and in the subband l. We

40

5 Pearson Correlation Coefficient

define the subband SPCC (or MSCF) between ca,l and cb,l in the subband l as ρ2 (ca,l , cb,l ) =

E 2 (c cb,l )

a,l

. E c2a,l E c2b,l

(5.13)

The vector form or fullband SPCC is

  E 2 cTa cb  , ρ (ca , cb ) = E (cTa ca ) E cTb cb 2

(5.14)

where ca and cb are two vectors of length L containing all the elements ca,l and cb,l , l = 1, 2, . . . , L, respectively. It is clear that 0 ≤ ρ2 (ca , cb ) ≤ 1,

(5.15)

0 ≤ ρ (ca , cb ) ≤ 1.

(5.16)

2

5.5 Summary This chapter developed different forms of the so-called Pearson correlation coefficient in the time, frequency, and KLE domains. Each of these forms has many interesting properties, which are very useful not only for deriving, but also for analyzing optimal filters in the context of noise reduction. We will elaborate on these properties in the next chapter.

6 Fundamental Properties

Having introduced in the previous chapter several forms of the SPCC in different domains, we now study their fundamental properties. Armed with these properties it is possible in many cases to derive and analyze optimal noise reduction filters by a simple inspection of a certain form of the SPCC.

6.1 In the Time Domain The SPCC between the desired and observation vectors x(k) and y(k) [as defined in (2.2)] is   E 2 xT y 2 ρ (x, y) = E (xT x) E (yT y) tr(Rx ) = tr(Ry ) iSNR = . (6.1) 1 + iSNR So this SPCC says how much the observation signal is noisy. Indeed, a value of ρ2 (x, y) close to 1 implies that the speech is largely dominant while a value of ρ2 (x, y) close to 0 implies that the noise is largely dominant. The SPCC between the two vectors x(k) and z(k) [as defined in (2.6)] is ρ2 (x, z) =

=

tr2 (HRx )

 tr(Rx )tr HRy HT tr2 (HRx ) oSNR(H)

· . T 1 + oSNR(H) tr(Rx )tr HRx H

Property 6.1. We have

J. Benesty et al., Noise Reduction in Speech Processing, Springer Topics in Signal Processing 2, DOI 10.1007/978-3-642-00296-0_6, © Springer-Verlag Berlin Heidelberg 2009

(6.2)

42

6 Fundamental Properties

ρ2 (x, z) = ρ2 (x, xF ) · ρ2 (xF , z) ,

(6.3)

tr2 (HRx )

 tr(Rx )tr HRx HT

(6.4)

oSNR(H) . 1 + oSNR(H)

(6.5)

where ρ2 (x, xF ) = and ρ2 (xF , z) =

The SPCC ρ2 (x, xF ) can be viewed as another definition of the speechdistortion index. If H = I (no speech distortion) then ρ2 (x, xF ) = 1. The closer the value of ρ2 (x, xF ) is to 0, the more the speech signal is distorted. The SPCC ρ2 (xF , z) shows the SNR improvement and reaches its maximum when oSNR(H) is maximized. Intuitively, we would like to find a matrix H in such a way that the signal vector z(k) is as close as possible to the vector x(k); mathematically, this can be translated by maximizing the SPCC ρ2 (x, z) or by minimizing ρ−2 (x, z). Clearly, from this property [eq. (6.3)], the SNR improvement is made at the expense of speech distortion. Maximizing the output SNR is equivalent to maximizing the SPCC ρ2 (xF , z). The filter derived from this optimization will introduce a large distortion to the desired signal since we try to make the signal vector Hy(k) = z(k) as close as possible to the signal vector Hx(k) = xF (k) [instead of x(k)], which is a filtered version of x(k). Now, if we take the natural logarithm of ρ−2 (x, z), we find that 



 ln ρ−2 (x, z) = ln [tr (Rx )] + ln tr HRy HT − ln tr2 (HRx ) . (6.6) It is instructive to compare this criterion to the MSE J (H) [eq. (4.5)]. Property 6.2. We have ρ2 (x, z) ≤

oSNR(H) , 1 + oSNR(H)

(6.7)

with equality when H = I. Proof. This property follows immediately from (6.3) since ρ2 (x, xF ) ≤ 1. Property 6.3. We have ρ2 (xF , y) = ρ2 (x, xF ) · ρ2 (x, y) . Proof. Indeed

(6.8)

6.1 In the Time Domain

ρ2 (xF , y) = =

43

tr2 (HRx )  tr HRx HT tr(Ry )

tr2 (HRx ) iSNR  · T tr HRx H tr(Rx ) 1 + iSNR

= ρ2 (x, xF ) · ρ2 (x, y) . Property 6.4. We have ρ2 (xF , y) ≤

iSNR , 1 + iSNR

(6.9)

with equality when H = I. Proof. This property follows immediately from (6.8) since ρ2 (x, xF ) ≤ 1. The SPCC between the two vectors v(k) and y(k) is another way to see how much the microphone signal is affected by the noise. This SPCC is tr(Rv ) tr(Ry ) 1 = . 1 + iSNR

ρ2 (v, y) =

(6.10)

So a value of ρ2 (v, y) close to 1 implies that the noise is largely dominant while a value of ρ2 (v, y) close to 0 implies that the speech is largely dominant. Property 6.5. We have ρ2 (x, y) + ρ2 (v, y) = 1

(6.11)

and iSNR =

ρ2 (x, y) . ρ2 (v, y)

(6.12)

Proof. Easy to see from (6.1) and (6.10). The input SNR is the ratio between two SPCCs and the sum of these two SPCCs is equal to 1. We can give another simple interpretation of Property 6.5. Let us define the complex number

(x, v) = ρ (x, y) + jρ (v, y) = cos θ + j sin θ,

(6.13)

where θ is the angle of (x, v) for which the modulus is equal to 1. On the complex plane, this complex number is on the unit circle. Since 0 ≤ ρ (x, y) ≤ 1 and 0 ≤ ρ (v, y) ≤ 1, therefore 0 ≤ θ ≤ π2 . Hence

44

6 Fundamental Properties

lim iSNR = ∞,

(6.14)

lim iSNR = 0.

(6.15)

θ→0 θ→ π 2

In other words, (x, v) is purely real (resp. imaginary) if and only if the input SNR is equal to infinity (resp. zero). In this interpretation, we represent the input SNR by the angle θ for which it takes its values between 0 and π/2. The SPCC between the two vectors v(k) and z(k) is ρ2 (v, z) =

=

tr2 (HRv )

 tr(Rv )tr HRy HT tr2 (HRv ) 1

· . T 1 + oSNR(H) tr(Rv )tr HRv H

(6.16)

Property 6.6. We have ρ2 (v, z) = ρ2 (v, vF ) · ρ2 (vF , z) ,

(6.17)

tr2 (HRv )

 tr(Rv )tr HRv HT

(6.18)

ρ2 (vF , z) =

1 . 1 + oSNR(H)

(6.19)

ρ2 (v, z) ≤

1 , 1 + oSNR(H)

(6.20)

where ρ2 (v, vF ) = and

Property 6.7. We have

with equality when H = I. Proof. This property follows immediately from (6.17) since ρ2 (v, vF ) ≤ 1. Property 6.8. We have ρ2 (vF , y) = ρ2 (v, vF ) · ρ2 (v, y) . Proof. Indeed ρ2 (vF , y) = =

tr2 (HRv )  tr HRv HT tr(Ry )

tr2 (HRv ) 1  · T tr HRv H tr(Rv ) 1 + iSNR

= ρ2 (v, vF ) · ρ2 (v, y) .

(6.21)

6.1 In the Time Domain

45

Property 6.9. We have ρ2 (vF , y) ≤

1 , 1 + iSNR

(6.22)

with equality when H = I. Proof. This property follows immediately from (6.21) since ρ2 (v, vF ) ≤ 1. Property 6.10. We have ρ2 (xF , z) + ρ2 (vF , z) = 1

(6.23)

and oSNR(H) =

ρ2 (xF , z) . ρ2 (vF , z)

(6.24)

Proof. Easy to see from (6.5) and (6.19). The output SNR is the ratio between two SPCCs depending on H and the sum of these two SPCCs is equal to 1. Now, let us define the complex number after the filtering processing:

(xF , vF ) = ρ (xF , z) + jρ (vF , z) = cos θF + j sin θF ,

(6.25)

where θF is the angle of (xF , vF ) for which the modulus is equal to 1. On the complex plane, this complex number is on the unit circle. Since 0 ≤ ρ (xF , z) ≤ 1 and 0 ≤ ρ (vF , z) ≤ 1, therefore 0 ≤ θF ≤ π2 . Hence lim oSNR(H) = ∞,

(6.26)

lim oSNR(H) = 0.

(6.27)

θF →0 θF → π 2

In other words, (xF , vF ) is purely real (resp. imaginary) if and only if the output SNR is equal to infinity (resp. zero). In this interpretation, we represent the output SNR by the angle θF for which it takes its values between 0 and π/2. It is easy to check that oSNR(H) > iSNR is equivalent to ρ2 (xF , z) > 2 ρ (x, y). In this case, ρ2 (vF , z) < ρ2 (v, y). With the angle interpretation, oSNR(H) > iSNR is equivalent to cos2 θF > cos2 θ which implies that θF < θ. To finish this part, we link the noise-reduction and speech-reduction factors with the SPCCs: ρ2 (xF , z) ρ2 (vF , z) · ξ (H) = · ξnr (H) , sr ρ2 (x, y) ρ2 (v, y)

(6.28)

and with the angles: tan2 θ · ξsr (H) = tan2 θF · ξnr (H) .

(6.29)

46

6 Fundamental Properties

6.2 In the Frequency Domain The subband SPCC (or MSCF) between the two signals X(jω) and Y (jω) [as defined in (2.8)] at frequency ω is φx (ω) φy (ω) iSNR(ω) = . 1 + iSNR(ω)

|ρ [X(jω), Y (jω)] |2 =

(6.30)

This subband SPCC tells us how much the observation signal is noisy at frequency ω. A value of |ρ [X(jω), Y (jω)] |2 close to 1 implies that the speech is largely dominant while a value of |ρ [X(jω), Y (jω)] |2 close to 0 implies that the noise is largely dominant at frequency ω. The fullband SPCC between the two signals X(jω) and Y (jω) is π φx (ω)dω 2 |ρ (X, Y ) | = −π π φ (ω)dω −π y iSNR 1 + iSNR = ρ2 (x, y) . =

(6.31)

As expected, the SPCC between the two vectors x(k) and y(k) is identical to the fullband SPCC between X(jω) and Y (jω). The subband SPCC between the two signals X(jω) and Z(jω) [as defined in (2.13)] at frequency ω is |ρ [X(jω), Z(jω)] |2 = |ρ [X(jω), Y (jω)] |2 .

(6.32)

The subband SPCC between X(jω) and Z(jω) is equal to the subband SPCC between X(jω) and Y (jω) and does not depend on H(jω); the same way the subband input SNR is equal to the subband output SNR and does not depend on H(jω). The fullband SPCC between the two signals X(jω) and Z(jω) is  2  π   −π H(jω)φx (ω)dω  2    |ρ (X, Z) | =  π (6.33) π 2 φ (ω)dω |H(jω)| φy (ω)dω −π x −π  2  π   −π H(jω)φx (ω)dω  oSNR(H)   · =  π . π 2 φ (ω)dω |H(jω)| φx (ω)dω 1 + oSNR(H) −π x −π Property 6.11. We have |ρ (X, Z) |2 = |ρ (X, XF ) |2 · |ρ (XF , Z) |2 ,

(6.34)

6.2 In the Frequency Domain

47

where  2  π   −π H(jω)φx (ω)dω  2    |ρ (X, XF ) | =  π π 2 φ (ω)dω |H(jω)| φ (ω)dω x x −π −π

(6.35)

and |ρ (XF , Z) |2 =

oSNR(H) . 1 + oSNR(H)

(6.36)

The fullband SPCC |ρ (X, XF ) |2 is a speech-distortion index. If H(jω) = 1, ∀ω (no speech distortion) then |ρ (X, XF ) |2 = 1. The closer the value of |ρ (X, XF ) |2 is to 0, the more the speech signal is distorted. The fullband SPCC |ρ (XF , Z) |2 shows the SNR improvement and reaches its maximum when oSNR(H) is maximized. The maximization of |ρ (X, Z) |2 leads to an optimal filter. Property 6.12. We have |ρ (X, Z) |2 ≤

oSNR(H) , 1 + oSNR(H)

(6.37)

with equality when H(jω) = 1, ∀ω. Proof. This property follows immediately from (6.34) since |ρ (X, XF ) |2 ≤ 1. Property 6.13. We have |ρ (XF , Y ) |2 = |ρ (X, XF ) |2 · |ρ (X, Y ) |2 .

(6.38)

Proof. Indeed  2  π   −π H(jω)φx (ω)dω  2    |ρ (XF , Y ) | =  π π 2 |H(jω)| φ (ω)dω φ (ω)dω x y −π −π  2  π   −π H(jω)φx (ω)dω  iSNR   · =  π π 2 1 + iSNR |H(jω)| φx (ω)dω φ (ω)dω −π −π x = |ρ (X, XF ) |2 · |ρ (X, Y ) |2 . Property 6.14. We have |ρ (XF , Y ) |2 ≤ with equality when H(jω) = 1, ∀ω.

iSNR , 1 + iSNR

(6.39)

48

6 Fundamental Properties

Proof. This property follows immediately from (6.38) since |ρ (X, XF ) |2 ≤ 1. The MSCF between V (jω) and Y (jω) is another way to see how much the observation signal is affected by the noise at frequency ω. This MSCF is φv (ω) φy (ω) 1 = . 1 + iSNR(ω)

|ρ [V (jω), Y (jω)] |2 =

(6.40)

Property 6.15. We have |ρ [X(jω), Y (jω)] |2 + |ρ [V (jω), Y (jω)] |2 = 1

(6.41)

and iSNR(ω) =

|ρ [X(jω), Y (jω)] |2 . |ρ [V (jω), Y (jω)] |2

(6.42)

Proof. Easy to see from (6.30) and (6.40). The subband input SNR at frequency ω is the ratio of two MSCFs and the sum of these two MSCFs is equal to 1. The fullband SPCC between V (jω) and Y (jω) is 1 1 + iSNR = ρ2 (v, y) .

|ρ (V, Y ) |2 =

(6.43)

The SPCC between the two vectors v(k) and y(k) is equal to the fullband SPCC between V (jω) and Y (jω). The subband SPCC between the two signals V (jω) and Z(jω) at frequency ω is |ρ [V (jω), Z(jω)] |2 = |ρ [V (jω), Y (jω)] |2 .

(6.44)

The fullband SPCC between the same signals is  2  π   −π H(jω)φv (ω)dω  2    |ρ (V, Z) | =  π (6.45) π 2 φ (ω)dω |H(jω)| φ (ω)dω v y −π −π  2  π   −π H(jω)φv (ω)dω  1   · =  π . π 2 1 + oSNR(H) φ (ω)dω |H(jω)| φ (ω)dω v v −π −π

6.2 In the Frequency Domain

49

Property 6.16. We have |ρ (V, Z) |2 = |ρ (V, VF ) |2 · |ρ (VF , Z) |2 ,

(6.46)

where  2  π   −π H(jω)φv (ω)dω  2    |ρ (V, VF ) | =  π π 2 φ (ω)dω |H(jω)| φ (ω)dω v v −π −π

(6.47)

and |ρ (VF , Z) |2 =

1 . 1 + oSNR(H)

(6.48)

1 , 1 + oSNR(H)

(6.49)

Property 6.17. We have |ρ (V, Z) |2 ≤ with equality when H(jω) = 1, ∀ω. Proof. This property follows immediately from (6.46) since |ρ (V, VF ) |2 ≤ 1. Property 6.18. We have |ρ (VF , Y ) |2 = |ρ (V, VF ) |2 · |ρ (V, Y ) |2 .

(6.50)

Proof. Indeed  2  π   −π H(jω)φv (ω)dω  2    |ρ (VF , Y ) | =  π π 2 |H(jω)| φv (ω)dω φ (ω)dω −π −π y  2  π   −π H(jω)φv (ω)dω  1   · =  π π 2 |H(jω)| φv (ω)dω φ (ω)dω 1 + iSNR −π −π v = |ρ (V, VF ) |2 · |ρ (V, Y ) |2 . Property 6.19. We have |ρ (VF , Y ) |2 ≤

1 , 1 + iSNR

(6.51)

with equality when H(jω) = 1, ∀ω. Proof. This property follows immediately from (6.50) since |ρ (V, VF ) |2 ≤ 1.

50

6 Fundamental Properties

Property 6.20. We have |ρ (XF , Z) |2 + |ρ (VF , Z) |2 = 1

(6.52)

and oSNR(H) =

|ρ (XF , Z) |2 . |ρ (VF , Z) |2

(6.53)

Proof. Easy to see from (6.36) and (6.48). The fullband output SNR for the frequency-domain approach is the ratio of two fullband SPCCs and the sum of these two SPCCs is equal to 1. It can be checked that oSNR(H) > iSNR is equivalent to |ρ (XF , Z) |2 > 2 ρ (x, y). In this situation, |ρ (VF , Z) |2 < ρ2 (v, y).

6.3 In the KLE Domain The subband SPCC (or MSCF) between the two signals cx,l (k) and cy,l (k) [as defined in (2.22)] in the subband l is λl λl + qTl Rv ql iSNRl = . 1 + iSNRl

ρ2 (cx,l , cy,l ) =

(6.54)

This SPCC tells us how much the observation signal is noisy in the subband l. A value of ρ2 (cx,l , cy,l ) close to 1 implies that the speech is largely dominant while a value of ρ2 (cx,l , cy,l ) close to 0 implies that the noise is largely dominant in the subband l. The subband SPCC between the two signals cx,l (k) and cz,l (k) [as defined in (2.26)] in the subband l is

ρ2 (cx,l , cz,l ) =

=

2 iTl Rcx ,l hl λl · hTl Rcy ,l hl

2 iTl Rcx ,l hl λl ·

hTl Rcx ,l hl

·

oSNR(hl ) . 1 + oSNR(hl )

(6.55)

Property 6.21. We have



 ρ2 (cx,l , cz,l ) = ρ2 cx,l , hTl cx,l · ρ2 hTl cx,l , cz,l , where

(6.56)

6.3 In the KLE Domain

 ρ2 cx,l , hTl cx,l =



iTl Rcx ,l hl

51

2

λl · hTl Rcx ,l hl

(6.57)

and

 ρ2 hTl cx,l , cz,l =

oSNR(hl ) . 1 + oSNR(hl )

(6.58)

Property 6.22. We have ρ2 (cx,l , cz,l ) ≤

oSNR(hl ) , 1 + oSNR(hl )

(6.59)

with equality when hl = il .

 Proof. This property follows immediately from (6.56) since ρ2 cx,l , hTl cx,l . Property 6.23. We have



 ρ2 hTl cx,l , cy,l = ρ2 cx,l , hTl cx,l · ρ2 (cx,l , cy,l ) .

(6.60)

Proof. Indeed

2 iTl Rcx ,l hl   hTl Rcx ,l hl · λl + qTl Rv ql 2

iTl Rcx ,l hl iSNRl = T · 1 + iSNRl hl Rcx ,l hl · λl

 = ρ2 cx,l , hTl cx,l · ρ2 (cx,l , cy,l ) .

 ρ2 hTl cx,l , cy,l =

Property 6.24. We have

 ρ2 hTl cx,l , cy,l ≤

iSNRl , 1 + iSNRl

(6.61)

with equality when hl = il .

 Proof. This property follows immediately from (6.60) since ρ2 cx,l , hTl cx,l ≤ 1. The SPCC between the two signals cv,l (k) and cy,l (k) is another way to see how much the microphone signal is affected by the noise in the subband l. This SPCC is qTl Rv ql qTl Ry ql 1 = . 1 + iSNRl

ρ2 (cv,l , cy,l ) =

(6.62)

52

6 Fundamental Properties

So a value of ρ2 (cv,l , cy,l ) close to 1 implies that the noise is largely dominant while a value of ρ2 (cv,l , cy,l ) close to 0 implies that the speech is largely dominant in the subband l. Property 6.25. We have ρ2 (cx,l , cy,l ) + ρ2 (cv,l , cy,l ) = 1

(6.63)

and iSNRl =

ρ2 (cx,l , cy,l ) . ρ2 (cv,l , cy,l )

(6.64)

Proof. Easy to see from (6.54) and (6.62). The subband input SNR is the ratio between two SPCCs and the sum of these two SPCCs is equal to 1. The SPCC between the two signals cv,l (k) and cz,l (k) in the subband l is ρ2 (cv,l , cz,l ) =

=

2 iTl Rcv ,l hl qTl Rv ql · hTl Rcy ,l hl

2 iTl Rcv ,l hl qTl Rv ql

·

hTl Rcv ,l hl

·

1 . 1 + oSNR(hl )

(6.65)

Property 6.26. We have



 ρ2 (cv,l , cz,l ) = ρ2 cv,l , hTl cv,l · ρ2 hTl cv,l , cz,l ,

(6.66)

where

 ρ2 cv,l , hTl cv,l =



iTl Rcv ,l hl

2

qTl Rv ql · hTl Rcv ,l hl

(6.67)

and

 ρ2 hTl cv,l , cz,l =

1 . 1 + oSNR(hl )

(6.68)

Property 6.27. We have ρ2 (cv,l , cz,l ) ≤ with equality when hl = il .

1 , 1 + oSNR(hl )

(6.69)

 Proof. This property follows immediately from (6.66) since ρ2 cv,l , hTl cv,l ≤ 1.

6.3 In the KLE Domain

Property 6.28. We have



 ρ2 hTl cv,l , cy,l = ρ2 cv,l , hTl cv,l · ρ2 (cv,l , cy,l ) .

53

(6.70)

Proof. Indeed

2 iTl Rcv ,l hl   hTl Rcv ,l hl · λl + qTl Rv ql

2 iTl Rcv ,l hl 1 = T · T 1 + iSNR hl Rcv ,l hl · ql Rv ql l

 T 2 2 = ρ cv,l , hl cv,l · ρ (cv,l , cy,l ) .

 ρ2 hTl cv,l , cy,l =

Property 6.29. We have

 ρ2 hTl cv,l , cy,l ≤

1 , 1 + iSNRl

(6.71)

with equality when hl = il .

 Proof. This property follows immediately from (6.70) since ρ2 cv,l , hTl cv,l ≤ 1. Property 6.30. We have



 ρ2 hTl cx,l , cz,l + ρ2 hTl cv,l , cz,l = 1 and

 ρ2 hTl cx,l , cz,l

. oSNR(hl ) = ρ2 hTl cv,l , cz,l

(6.72)

(6.73)

Proof. Easy to see from (6.58) and (6.68). The subband output SNR for the KLE-domain approach is the ratio of two SPCCs depending on hl and the sum of these two SPCCs is equal to 1.

For Ll > 1,  it is easy to see that oSNR(h l ) > iSNR is equivalent to T 2 ρ hl cx,l , cz,l > ρ2 (cx,l , cy,l ). In this case, ρ2 hTl cv,l , cz,l < ρ2 (cv,l , cy,l ). Now, let us define the four vectors of length L:
T cv (k) = cv,1 (k) cv,2 (k) · · · cv,L (k) , (6.74a)
T cx (k) = cx,1 (k) cx,2 (k) · · · cx,L (k) , (6.74b)
T cy (k) = cy,1 (k) cy,2 (k) · · · cy,L (k) , (6.74c)
T cz (k) = cz,1 (k) cz,2 (k) · · · cz,L (k) , (6.74d)

54

6 Fundamental Properties

and the two filtered vectors of length L:
T cv,F (k) = hT1 cv,1 (k) hT2 cv,2 (k) · · · hTL cv,L (k) ,
T cx,F (k) = hT1 cx,1 (k) hT2 cx,2 (k) · · · hTL cx,L (k) .

(6.75a) (6.75b)

The SPCC between the two vectors cx (k) and cy (k) is L ρ2 (cx , cy ) = L

λl L T l ql Rv ql

l=1

l=1 λl + iSNR = 1 + iSNR = ρ2 (x, y) .

(6.76)

As expected, the SPCC between the two vectors x(k) and y(k) is identical to the SPCC between the two vectors cx (k) and cy (k). The SPCC between the two vectors cx (k) and cz (k) is

 L ρ2 (cx , cz ) = L

T l=1 il Rcx ,l hl

2

L

hTl Rcy ,l hl 2 T l=1 il Rcx ,l hl oSNR(h1:L ) = L · . L T 1 + oSNR(h1:L ) l=1 λl · l=1 hl Rcx ,l hl λl ·

 L

l=1

l=1

(6.77)

Property 6.31. We have ρ2 (cx , cz ) = ρ2 (cx , cx,F ) · ρ2 (cx,F , cz ) ,

(6.78)

where

 L

2 T i R h c ,l l x l=1 l ρ2 (cx , cx,F ) = L L T l=1 λl · l=1 hl Rcx ,l hl

(6.79)

and ρ2 (cx,F , cz ) =

oSNR(h1:L ) . 1 + oSNR(h1:L )

(6.80)

Property 6.32. We have ρ2 (cx , cz ) ≤

oSNR(h1:L ) , 1 + oSNR(h1:L )

(6.81)

with equality when hl = il , ∀l. Proof. This property follows immediately from (6.78) since ρ2 (cx , cx,F ) ≤ 1.

6.3 In the KLE Domain

55

Property 6.33. We have ρ2 (cx,F , cy ) = ρ2 (cx , cx,F ) · ρ2 (cx , cy ) .

(6.82)

Proof. Indeed

 L

2 T i R h c ,l l l x l=1 ρ2 (cx,F , cy ) = L  L  T T l=1 hl Rcx ,l hl · l=1 λl + ql Rv ql

 2 L T i R h c ,l l l x l=1 iSNR = L · L T 1 + iSNR l=1 hl Rcx ,l hl · l=1 λl = ρ2 (cx , cx,F ) · ρ2 (cx , cy ) . Property 6.34. We have ρ2 (cx,F , cy ) ≤

iSNR , 1 + iSNR

(6.83)

with equality when hl = il , ∀l. Proof. This property follows immediately from (6.82) since ρ2 (cx , cx,F ) ≤ 1. The SPCC between the two vectors cv and cy is L T l=1 ql Rv ql ρ2 (cv , cy ) = L T l=1 ql Ry ql 1 = 1 + iSNR = ρ2 (v, y) .

(6.84)

As expected, the SPCC between the two vectors v(k) and y(k) is identical to the SPCC between the two vectors cv (k) and cy (k). The SPCC between the two vectors cv (k) and cz (k) is

 ρ2 (cv , cz ) = L

L T l=1 il Rcv ,l hl

2

L qTl Rv ql · l=1 hTl Rcy ,l hl

 2 L T l=1 il Rcv ,l hl 1 = L · . (6.85) L T T 1 + oSNR(h1:L ) l=1 ql Rv ql · l=1 hl Rcv ,l hl l=1

Property 6.35. We have ρ2 (cv , cz ) = ρ2 (cv , cv,F ) · ρ2 (cv,F , cz ) , where

(6.86)

56

6 Fundamental Properties

 L

T l=1 il Rcv ,l hl

ρ2 (cv , cv,F ) = L

l=1

qTl Rv ql ·

L

l=1

2

hTl Rcv ,l hl

(6.87)

and ρ2 (cv,F , cz ) =

1 . 1 + oSNR(h1:L )

(6.88)

Property 6.36. We have ρ2 (cv , cz ) ≤

1 , 1 + oSNR(h1:L )

(6.89)

with equality when hl = il , ∀l. Proof. This property follows immediately from (6.86) since ρ2 (cv , cv,F ) ≤ 1. Property 6.37. We have ρ2 (cv,F , cy ) = ρ2 (cv , cv,F ) · ρ2 (cv , cy ) . Proof. Indeed

 L

ρ2 (cv,F , cy ) = L

T l=1 il Rcv ,l hl

(6.90)

2

 L  hTl Rcv ,l hl · l=1 λl + qTl Rv ql

 2 L T l=1 il Rcv ,l hl 1 = L · L T T 1 + iSNR h R h · q R q cv ,l l v l l=1 l l=1 l l=1

= ρ2 (cv , cv,F ) · ρ2 (cv , cy ) . Property 6.38. We have ρ2 (cv,F , cy ) ≤

1 , 1 + iSNR

(6.91)

with equality when hl = il , ∀l. Proof. This property follows immediately from (6.90) since ρ2 (cv , cv,F ) ≤ 1. Property 6.39. We have ρ2 (cx,F , cz ) + ρ2 (cv,F , cz ) = 1

(6.92)

and oSNR(h1:L ) =

ρ2 (cx,F , cz ) . ρ2 (cv,F , cz )

(6.93)

Proof. Easy to see from (6.80) and (6.88). The fullband output SNR for the KLE-domain approach is the ratio of two SPCCs depending on hl , l = 1, 2, . . . , L and the sum of these two SPCCs is equal to 1. It can be checked that oSNR(h1:L ) > iSNR is equivalent to ρ2 (cx,F , cz ) > 2 ρ (cx , cy ). In this case, ρ2 (cv,F , cz ) < ρ2 (cv , cy ).

6.4 Summary

57

6.4 Summary In this chapter, we discussed many interesting properties of the SPCC in different domains. These fundamental properties establish relationships between the SPCC and the input and output SNRs. Armed with these properties, it is possible in many cases to derive and analyze optimal noise reduction filters by a simple inspection of a certain form of the SPCC, which will be exploited in the next several chapters.

7 Optimal Filters in the Time Domain

Having discussed different error criteria and performance measures, we now begin our search for reliable and practical noise reduction filters. In this chapter, we restrict our attention to filters in the time domain. Particularly, much emphasis is on the Wiener filter [125] as it is directly derived from the MSE criterion and most well-known algorithms are somehow related to it.

7.1 Wiener Filter If we differentiate the MSE criterion, J (H) [eq. (4.5)], with respect to H and equate the result to zero, we easily find the Wiener filter matrix: HTW = R−1 y Rx = I − R−1 y Rv .

(7.1)

This optimal filter depends on the correlation matrices Ry and Rv : the first one can be estimated during speech-and-noise periods while the second one can be estimated during noise-only intervals assuming that the statistics of the noise do not change much with time. Now, if we substitute (2.16) into (7.1) we get another useful form of the time-domain Wiener filter:  −1 HTW = Q Λ + QT Rv Q ΛQT . (7.2) Let us define the following normalized correlation matrices: ˜ v = Rv , R σv2 ˜ x = Rx , R σx2 ˜ y = Ry . R σy2 J. Benesty et al., Noise Reduction in Speech Processing, Springer Topics in Signal Processing 2, DOI 10.1007/978-3-642-00296-0_7, © Springer-Verlag Berlin Heidelberg 2009

60

7 Optimal Filters in the Time Domain

A third way to write the Wiener filter is 

I ˜ −1 R ˜x +R v iSNR ˜ −1 R ˜x = ρ2 (x, y)R

HTW =

−1

˜ −1 R ˜x R v

(7.3)

y

˜ −1 R ˜ v, = I − ρ (v, y)R y 2

We can see from (7.3) that lim

HW = I,

(7.4)

lim HW = 0L×L ,

(7.5)

iSNR→∞ iSNR→0

where 0L×L is an L × L matrix with all its elements being zeros. Clearly, the Wiener filter may have a disastrous effect for low input SNRs since it may remove everything (noise and speech). Property 7.1. With the optimal Wiener filter given in (7.1), the output SNR is always greater than or equal to the input SNR, i.e., oSNR(HW ) ≥ iSNR. Proof. Let us evaluate the SPCC between the two vectors y(k) and zW (k) = HW y(k): ρ2 (y, zW ) =

tr2 (HW Ry )

 tr (Ry ) · tr HW Ry HTW

tr (Rx ) tr (Rx ) · tr (Ry ) tr (HW Rx ) ρ2 (x, y) = 2 . ρ (x, zW ) =

Therefore ρ2 (x, y) = ρ2 (y, zW ) · ρ2 (x, zW ) ≤ ρ2 (x, zW ). Using (6.1) and Property 6.2 in the previous expression, we get iSNR oSNR(HW ) ≤ , 1 + iSNR 1 + oSNR(HW ) that we can slightly rearranged to 1 1+ which implies that

1 iSNR

≤

1 1+

1 oSNR(HW )

,

7.1 Wiener Filter

61

1 1 ≥ . iSNR oSNR(HW ) As a result oSNR(HW ) ≥ iSNR. The minimum MSE (MMSE) and minimum NMSE (MNMSE) are obtained by replacing HW in (4.5) and (4.8):   J (HW ) = tr (Rx ) − tr Rx R−1 y Rx   = tr (Rv ) − tr Rv R−1 (7.6) y Rv ,   −1 tr Rv Ry Rv J˜ (HW ) = 1 − ≤ 1. (7.7) tr (Rv ) We can compute the speech-distortion index by substituting (7.1) into (3.23): υsd (HW ) = 1 −

oSNR(HW ) + 2 ≤ 1. iSNR · ξnr (HW )

(7.8)

Using (4.8) and (7.8), we get the noise-reduction factor: oSNR(HW ) + 1 ≥ 1. iSNR − J˜ (HW )

(7.9)


 J˜ (HW ) = iSNR 1 − ρ2 (x, zW ) .

(7.10)

ξnr (HW ) = Property 7.2. We have

Proof. Indeed   tr Rx R−1 Rx tr (R ) x y J˜ (HW ) = − tr (Rv ) tr (Rv )    tr Rx R−1 y Rx = iSNR 1 − tr (Rx )
 2 = iSNR 1 − ρ (x, zW ) . Therefore, the NMSE is minimized when the SPCC between the two vectors x(k) and z(k) is maximized. This SPCC can be rewritten as follows: ρ2 (x, zW ) =

1 1 + oSNR(HW ) · . ξsr (HW ) oSNR(HW )

(7.11)

We observe that the Wiener filter is compromising between speech reduction (i.e., speech distortion) and output SNR improvement.

62

7 Optimal Filters in the Time Domain

Property 7.3. We have iSNR iSNR ≤ J˜ (HW ) ≤ . 1 + oSNR(HW ) 1 + iSNR

(7.12)

Proof. Since ρ2 (x, zW ) ≥

iSNR 1 + iSNR

and with the help of (7.10), we easily get J˜ (HW ) ≤

iSNR . 1 + iSNR

Since ρ2 (x, zW ) ≤

oSNR(HW ) 1 + oSNR(HW )

and, again, with the help of (7.10), we obtain iSNR ≤ J˜ (HW ) . 1 + oSNR(HW ) Hence, we get better bounds for the MNMSE than the usual ones [0 ≤ J˜ (HW ) ≤ 1]. Property 7.4. We have 2

[1 + oSNR(HW )] (1 + iSNR) [1 + oSNR(HW )] ≤ ξnr (HW ) ≤ . (7.13) iSNR · oSNR(HW ) iSNR2 Proof. Easy to show by using (7.9) and the bounds of J˜ (HW ) [eq. (7.12)]. Property 7.5. We have 1 [1 + oSNR(HW )]

2

≤ υsd (HW ) ≤

1 + oSNR(HW ) − iSNR . (7.14) (1 + iSNR) [1 + oSNR(HW )]

Proof. Easy to show by using (7.8) and the bounds of ξnr (HW ) [eq. (7.13)]. Particular case: white noise. We assume here that the noise picked up by the microphone is white (i.e., Rv = σv2 I). In this situation, the Wiener filter matrix becomes HW = I − σv2 R−1 y , where

(7.15)

7.1 Wiener Filter

63

Ry = Rx + σv2 I. It is well known that the inverse of the Toeplitz matrix Ry as follows [8], [82]:   1 −c21 · · · −cL1 1/E1 0 · · ·  −c12 1 · · · −cL2   0 1/E2 · · ·   R−1 . .. . . ..   .. .. . . y =   .. . .  . . . . −c1L −c2L · · ·

1

0

0

can be factorized 0 0 .. .

   , 

(7.16)

· · · 1/EL

where the columns of the first matrix in the right-hand side of (7.16) are the linear interpolators of the signal y(k) and the elements El in the diagonal matrix are the respective interpolation-error powers. Using the factorization of R−1 y in (7.6) and (7.7), the MMSE and MNMSE can be rewritten, respectively, as L  2  1 J(HW ) = Lσv2 − σv2 , El

(7.17)

L σv2  1 ˜ J(HW ) = 1 − . L El

(7.18)

l=1

l=1

Assume that the noise-free speech signal, x(k), is very well predictable. In this scenario, El ≈ σv2 , ∀ l, and replacing this value in (7.18) we find ˜ W ) ≈ 0. From (4.8), we then deduce that υsd (HW ) ≈ 0 (no speech that J(H distortion) and ξnr (HW ) ≈ ∞ (infinite noise reduction). Notice that, from a theoretical point of view (and with white noise), this result is independent of the SNR. Also   0 c12 · · · c1L  c21 0 · · · c2L    HW ≈  . . . (7.19) .   .. .. . . ..  cL1 cL2 · · · 0 and since HW x(k) ≈ x(k), this means that ξsr (HW ) ≈ 1; as a result oSNR(HW ) ≈ ∞ and we can almost perfectly recover the signal x(k). At the other extreme case, let us see now what happens when the signal of interest x(k) is not predictable at all. In this situation, El ≈ σy2 , ∀ l and cij ≈ 0, ∀ i, j, i = j. Using these values, we get iSNR I, 1 + iSNR ˜ W ) ≈ iSNR . J(H 1 + iSNR HW ≈

(7.20) (7.21)

With the help of the two previous equations, it’s straightforward to obtain

64

7 Optimal Filters in the Time Domain

ξnr (HW ) ≈

1+

υsd (HW ) ≈

1 iSNR 1

2 , 2,

(1 + iSNR) SNR(HW ) ≈ iSNR.

(7.22) (7.23) (7.24)

While some noise reduction is achieved (at the price of speech distortion), there is no improvement in the output SNR, meaning that the Wiener filter has no positive effect on the microphone signal y(k). This analysis, even though simple, is quite insightful. It shows that the Wiener filter may not be that bad after all, as long as the desired signal is somewhat predictable. However, in practice some discontinuities could be heard from a voiced signal to an unvoiced one, since for the former the noise will be mostly removed while it will not for the latter.

7.2 Tradeoff Filters The time-domain NMSE as shown in (4.8) is the sum of two terms. One depends on the speech distortion while the other one depends on the noise reduction. Instead of minimizing the NMSE with respect to H as we already did to find the Wiener filter, we can minimize the speech-distortion index with the constraint that the noise-reduction factor is equal to a value that is greater than one. Mathematically, this is equivalent to min Jx (H) subject to Jv (H) = β · tr (Rv ) , H

(7.25)

where 0 < β < 1 in order to have some noise reduction. If we use a Lagrange multiplier, µ, to adjoin the constraint to the cost function, (7.25) can be rewritten as HT = arg min L(H, µ),

(7.26)

L(H, µ) = Jx (H) + µ [Jv (H) − β · tr (Rv )]

(7.27)

H

with

and µ ≥ 0. From (7.26), we can easily derive the optimal filter: HTT = (Rx + µRv )

−1

Rx −1

= [Ry + (µ − 1)Rv ] [Ry − Rv ] 

−1 −1 = (1 − µ)I + µ HTW ,

(7.28)

where the Lagrange multiplier, µ, satisfies Jv (HT ) = β ·tr (Rv ), which implies that

7.2 Tradeoff Filters

65

1 > 1. β

(7.29)

 1 ˜ J(HT ) − β , iSNR

(7.30)

ξnr (HT ) = From (4.8), we get υsd (HT ) =

˜ T ) ≥ β. and because υsd (HT ) ≥ 0, J(H ˜ ˜ Since J(HW ) ≤ J(HT ), ∀µ, we also have   1 1 1 υsd (HT ) ≥ υsd (HW ) + − . iSNR ξnr (HW ) ξnr (HT )

(7.31)

Therefore, ξnr (HT ) ≥ ξnr (HW ) implies that υsd (HT ) ≥ υsd (HW ). In practice it’s not easy to determine the optimal µ. Therefore, when this parameter is chosen in an ad-hoc way, we can see that for • • • •

µ = 1, HT = HW : Wiener filter; µ = 0, HT = I: distortionless filter; µ > 1, results in low residual noise at the expense of high speech distortion; µ < 1, we get little speech distortion but not so much noise reduction.

Property 7.6. With the tradeoff filter given in (7.28), the output SNR is always greater than or equal to the input SNR, i.e., oSNR(HT ) ≥ iSNR. √ Proof. The SPCC between the two vectors x(k) and x(k) + µv(k) is √

tr2 (Rx ) tr (Rx ) [tr (Rx ) + µtr (Rv )] iSNR = . µ + iSNR √ The SPCC between the two vectors x(k) and HT x(k) + µHT v(k) is ρ2 (x, x +

ρ2 (x, HT x +

√

µv) =

µHT v) = =

tr2 (HT Rx )   tr (Rx ) tr HT (Rx + µRv ) HTT tr (HT Rx ) . tr (Rx )

Another way to write the same SPCC is the following: ρ2 (x, HT x +

√

tr2 (HT Rx ) oSNR (HT )

· µ + oSNR (HT ) tr (Rx ) tr HT Rx HTT √ = ρ2 (x, HT x) · ρ2 (HT x, HT x + µHT v) oSNR (HT ) ≤ . µ + oSNR (HT )

µHT v) =

66

7 Optimal Filters in the Time Domain

Now, let us evaluate the SPCC between the two vectors x(k) + √ HT x(k) + µHT v(k): ρ2 (x +

√

µv, HT x +

√

√

µv(k) and

µHT v) =

2

tr [HT (Rx + µRv )]   [tr (Rx ) + µtr (Rv )] · tr HT (Rx + µRv ) HTT tr (Rx ) tr (Rx ) · tr (Rx ) + µtr (Rv ) tr (HT Rx )  √  ρ2 x, x + µv . = 2 √ ρ x, HT x + µHT v =

Therefore ρ2 (x, x +

√

iSNR µ + iSNR √ √ √ = ρ2 (x + µv, HT x + µHT v) · ρ2 (x, HT x + µHT v) √ ≤ ρ2 (x, HT x + µHT v) oSNR (HT ) ≤ . µ + oSNR (HT )

µv) =

As a result oSNR (HT ) ≥ iSNR. We can find another tradeoff filter by minimizing the residual noise with the constraint that we allow some level of speech distortion. Mathematically, this is equivalent to min Jv (H) subject to Jx (H) = β2 · tr (Rx ) , H

(7.32)

where β2 > 0 in order to have some noise reduction. If we use a Lagrange multiplier, µ2 , to adjoin the constraint to the cost function, (7.32) can be rewritten as HT,2 = arg min L(H, µ2 ),

(7.33)

L(H, µ2 ) = Jv (H) + µ2 [Jx (H) − β2 · tr (Rx )]

(7.34)

H

with

and µ2 ≥ 0. The optimal solution to this optimization problem is HTT,2 =



−1 Rv Rx + Rx , µ2

(7.35)

7.3 Subspace Approach

67

where the Lagrange multiplier, µ2 , satisfies Jx (HT,2 ) = β2 · tr (Rx ), which implies that υsd (HT,2 ) = β2 > 0.

(7.36)

From (4.8), we get ξnr (HT,2 ) =

1 , ˜ T,2 ) − β2 · iSNR J(H

(7.37)

˜ T,2 ) > β2 · iSNR. and because ξnr (HT,2 ) > 0, J(H ˜ W ) ≤ J(H ˜ T,2 ), ∀µ, we also have Since J(H 1 1 ≥ + iSNR [υsd (HW ) − υsd (HT,2 )] . ξnr (HT,2 ) ξnr (HW )

(7.38)

Therefore, υsd (HT,2 ) ≤ υsd (HW ) implies that ξnr (HT,2 ) ≤ ξnr (HW ). From a practical point of view, the two tradeoff filters derived here are fundamentally the same since by taking µ = 1/µ2 , we see that HT = HT,2 .

7.3 Subspace Approach In [54], it is shown that two symmetric matrices Rx and Rv can be jointly diagonalized if Rv is positive definite. This joint diagonalization was first introduced by Jensen et al. [78] and then by Hu and Loizou [66], [67], [68] in the single-channel noise reduction problem. For our time-domain model we get Rx = BT Λjd B,

(7.39a)

T

Rv = B B, Ry = BT [I + Λjd ] B,

(7.39b) (7.39c)

where B is a full rank square matrix but not necessarily orthogonal, and the diagonal matrix
 Λjd = diag λjd,1 λjd,2 · · · λjd,L (7.40) are the eigenvalues of the matrix R−1 v Rx with λjd,1 ≥ λjd,2 ≥ · · · ≥ λjd,L ≥ 0. Applying the decompositions (7.39a)–(7.39c) in (7.28), the tradeoff filter becomes HT = BT Λjd (Λjd + µI)

−1

B−T .

(7.41)

Therefore, the estimation of the speech signal, x(k), is done in three steps: first we apply the transform B−T to the noisy signal; second the transformed signal

68

7 Optimal Filters in the Time Domain −1

is modified by the gain function Λjd (Λjd + µI) ; and finally we transform back the signal to its original domain by applying the transform BT . Usually, a speech signal can be modelled as a linear combination of a number of some (linearly independent) basis vectors smaller than the dimension of these vectors [49], [64], [76]. As a result, the vector space of the noisy signal can be decomposed in two subspaces: the signal-plus-noise subspace of length Ls and the noise subspace of length Ln , with L = Ls + Ln . This implies that the last Ln eigenvalues of the matrix R−1 v Rx are equal to zero. Therefore, we can rewrite (7.41) to obtain the subspace filter:   Σ 0Ls ×Ln T HS = B B−T , (7.42) 0Ln ×Ls 0Ln ×Ln where

 Σ = diag

λjd,1 λjd,2 λjd,1 +µ λjd,2 +µ

···

λjd,Ls λjd,Ls +µ

 (7.43)

is an Ls × Ls diagonal matrix. We now clearly see that noise reduction with the subspace method is achieved by nulling the noise subspace and cleaning the speech-plus-noise subspace via a reweighted reconstruction. Some noise reduction can be achieved by only nulling the noise subspace and leaving intact the speech-plus-noise subspace (µ = 0). This algorithm was developed in [37].

7.4 Experiments Having formulated different noise reduction filters in the time domain, we are now ready to evaluate their performance. A number of experiments have been carried out in various noise and operation conditions. In this section, we will present some results, which illustrate the impact of some important parameters on the performance and highlight the merits and limitations inherent in these noise reduction algorithms. 7.4.1 Experimental Setup The clean speech signal used in our experiments was recorded from a female speaker in a quiet office environment. It was sampled at 8 kHz and quantized with 16 bits (2 bytes). The overall length of the signal is 30 s. The first 5 s of this signal and its spectrogram is visualized in Fig. 7.1. The noisy speech is obtained by adding noise to the clean speech where the noise signal is properly scaled to control the input SNR. We considered three types of noise: computer generated white Gaussian noise, car noise, and babbling noise. The car noise is recorded in a car compartment with the car running at 55 miles per hour on a highway. The resulting signal is also digitized with

7.4 Experiments

69

Amplitude

0.5

0

Frequency (kHz)

−0.5 0 4

1

2

3

4

5 1.0 0.8 0.6

2 0.4 0.2 0 0

1

2

Time (s)

3

4

5

0

Fig. 7.1. Clean speech (first 5 s) and its spectrogram.

Amplitude

0.2

0

Frequency (kHz)

−0.2 0 4

1

2

3

4

5 1.0 0.8 0.6

2 0.4 0.2 0 0

1

2

3

4

5

0

Time (s)

Fig. 7.2. Waveform (first 5 s) and spectrogram of car noise.

a sampling rate of 8 kHz. Figure 7.2 plots the first 5 s of this noise and its spectrogram. Compared with the white Gaussian noise which has a constant spectral energy at all frequencies, the car noise is colored in nature, and most of its energy is concentrated in low frequencies (below 2 kHz). In general, it is less stationary than the white Gaussian noise and its characteristics may change according to the road condition, running speed, and many other factors.

70

7 Optimal Filters in the Time Domain

Amplitude

0.2

0

−0.2 0

1

2

3

4

5

Frequency (kHz)

4

1.0 0.8 0.6

2 0.4 0.2 0 0

1

2

3 Time (s)

4

5

0

Fig. 7.3. Waveform (first 5 s) and spectrogram of NYSE babbling noise.

The babbling noise is recorded in a New York Stock Exchange (NYSE) room. This noise consists of sounds from various sources such as electric fans, telephone rings, and even speakers. As illustrated in Fig. 7.3, this noise is neither stationary nor white and its statistics change significantly from time to time. 7.4.2 Effect of Forgetting Factor on Performance To compute the noise reduction time-domain filters, we need to know the covariance matrices Ry and Rv . Since the noisy signal is accessible, the covariance matrix Ry can be estimated from its definition given in (2.5a) by approximating the mathematical expectation with sample average. However, due to the fact that speech is nonstationary, the sample average has to be performed on a short-term basis so that the estimated covariance matrix can follow the short-term variations of the speech signal. In speech processing, one of the most popularly used approaches to estimating Ry that takes into account the speech nonstationarity is the recursive method, where an estimate of Ry at time instant k is obtained as Ry (k) = αy Ry (k − 1) + (1 − αy )y(k)yT (k),

(7.44)

where αy is a forgetting factor that controls the influence of the previous data samples on the current estimate of the noisy signal covariance matrix. This method will be adopted in this experiment.

7.4 Experiments

71

20 HT , µ = 4

oSNR (dB)

18

HW

16

14

12

HT , µ = 0.6 (a)

10 0.9

0.92

0.94

0.96

0.98

1

−10

10 log(υsd )

−14

HT , µ = 4

−18 HW

HT , µ = 0.6

−22

(b)

−26 0.9

0.92

0.94

0.96

0.98

1

αy

Fig. 7.4. Performance versus the forgetting factor αy for different filters in white Gaussian noise with an iSNR = 10 dB and L = 20. (a) Output SNR and (b) speechdistortion index.

The computation of the noise signal covariance matrix Rv requires a noise estimator, which can be achieved in different ways and will be addressed in the next chapter. This experiment aims at studying the effect of αy on noise reduction performance. We consider the stationary white Gaussian noise and compute the matrix Rv directly from the noise signal using a long-term sample average.

72

7 Optimal Filters in the Time Domain

The results of this experiment1 is plotted in Fig. 7.4 where the output SNR and speech-distortion index are computed according to (3.9) and (3.23) respectively [at each time instant, the noise reduction filter is estimated and then directly applied to the clean speech and noise to obtain xF (k) and vF (k), and the output SNR and speech-distortion index are subsequently computed]. It is seen from Fig. 7.4 that both the output SNR and speech-distortion index bear a nonmonotonic relationship with the forgetting factor αy . Specifically, the output SNR first increases as αy increases and then decreases. In comparison, the speech-distortion index first decreases with αy and then increases. Apparently, the value of αy plays a critical role on the noise reduction performance of the studied filters. On the one hand, the forgetting factor αy cannot be small; otherwise, the estimation variance of Ry (k) will be large, which can lead to performance degradation in noise reduction. Furthermore, Ry (k) may tend to be rank deficient, causing numerical stability problems. On the other hand, this forgetting factor αy cannot be too large. If it is too large (close to 1), the recursive estimate will essentially be a long-term average and will not be able to follow the short-term variations of the speech signal. As a result, the nature of the speech is not fully taken advantage of, which limits the noise reduction performance. From Fig. 7.4, we can see that the optimal noise reduction performance (highest output SNR and lowest speech distortion) occurs when αy is between 0.985 and 0.995. So, in the subsequent experiments, we will set αy to 0.985. 7.4.3 Effect of Filter Length on Performance Another important parameter for all the time-domain noise reduction filters is the filter length (also the frame size) L. So, in this second experiment, we study the impact of the filter length L on the performance of noise reduction. Again, the noise is white Gaussian with an iSNR = 10 dB. Based on the previous experiment, we set αy = 0.985 and the noise covariance matrix Rv is computed using a long-term average. The results are depicted in Fig. 7.5. It is clear that the length L should be reasonably large enough to achieve good noise reduction performance. When L increases from 1 to 20, the output SNR improves while speech distortion decreases. But if we continue to increase L, there is either marginal additional SNR improvement (for the tradeoff filter with µ = 4), or the output SNR even slightly degrades (for the Wiener filter and the tradeoff filter with µ = 0.6), and there is also some increase in speech distortion. In general, good performance for all the studied algorithms is achieved when the filter length L is between 10 and 20. This result coincides with what was observed in [24]. The reason behind this is that speech is somewhat predictable. It is this predictability that helps us achieve noise reduction without noticeably distorting the desired speech signal. In order to 1

Note that we do not present the results of the subspace method here because it does not perform better than the tradeoff filter.

7.4 Experiments

73

20

oSNR (dB)

18

HT , µ = 4

16

14

12

HT , µ = 0.6 HW (a)

10 0

10

20

30

40

50

60

70

80

70

80

−10

HT , µ = 4

−14 10 log(υsd )

HW HT , µ = 0.6

−18

−22

(b)

−26 0

10

20

30

40 L

50

60

Fig. 7.5. Performance versus the filter length L for different filters in white Gaussian noise with an iSNR = 10 dB and αy = 0.985. (a) Output SNR and (b) speechdistortion index.

fully take advantage of the speech predictability, the filter length needs to be larger than the order of speech prediction, which is in the range between 10 and 20 for an 8-kHz sampling rate. But if we continue to increase L, the additional performance improvement will be limited. In theory, there shouldn’t be performance degradation for large L. However, in practice, the estimation variance of the covariance matrix Ry increases with L, which generally leads to performance degradation.

74

7 Optimal Filters in the Time Domain

7.4.4 Performance in Different Noise Conditions In this experiment, we test the performance of the different filters in various noise conditions with different input SNRs. We consider three types of noise: white Gaussian, car, and NYSE. Based on the previous experimental results, we set L = 20 and αy = 0.985. Again, we do not use any noise estimator, but compute the noise covariance directly from the noise signal. However, since we have now both stationary and nonstationary noise, we choose to compute the noise signal covariance matrix using the recursive method through Rv (k) = αv Rv (k − 1) + (1 − αv )v(k)vT (k),

(7.45)

where Rv (k) is an estimate of the covariance matrix Rv at time k and αv , similar to αy , is a forgetting factor. Theoretically, the optimal value of αv depends on the characteristics of the noise. A large αv should be used if the noise is stationary. But if the noise is nonstationary, the value of αv should be set smaller. However, in this experiment, we do not assume to know the nonstationarity of each type of noise and set αv = 0.995 for all the three noise signals. (Note that this value of αv is determined through experiments. We fixed αy to 0.985 and L = 20, but change αv from 0 to 1. In average, we found that the best noise reduction performance is achieved when αv = 0.995.) The results for this experiment are shown in Fig. 7.6, where we only plotted the results of the Wiener filter and the tradeoff filter with µ = 4 to simplify the presentation. It is seen from Fig. 7.6 that, in the same type of noise, the lower the input SNR, the more the noise is reduced (higher SNR improvement). But the speech-distortion index increases rapidly as the input SNR decreases. When the input SNR is very low (e.g., below 0 dB), the speech-distortion index becomes very large, which may cause significant negative impact to the speech signal (instead of improving the speech quality, it may degrade it due to the large speech distortion). To circumvent this problem in practical noise reduction systems, we suggest to use grace degradation, i.e., when the input SNR is above a certain threshold (around 10 dB), the noise reduction filters can be directly applied to the noisy speech; but when the input SNR is below some lower threshold (around or below 0 dB), we should leave the noisy speech unchanged; if the input SNR is between the two thresholds (we call it the grace degradation range), we can use some suboptimal filter so that there is a smooth transition in speech quality from low input SNR to high input SNR environments. If we use the Gaussian noise as the baseline, it is seen that both the Wiener and tradeoff filters perform consistently worse in the NYSE noise. This is due to the fact that the NYSE noise is nonstationary, therefore it is more difficult to deal with. In car noise, however, both filters yielded better performance in low input SNRs but poorer performance in high input SNRs. As what caused this behavior and how to take advantage of the low-pass nature of the car

7.5 Summary 30

HT , µ = 4, WGN

25 20

75

HW , WGN HT , µ = 4, Car

oSNR (dB)

HW , Car

15 10 5 0

HT , µ = 4, NYSE

−5

HW , NYSE (a)

−10 −10

−5

0

0

5

10

20

25

HT , µ = 4, NYSE HW , NYSE

−5

HT , µ = 4, Car

−10 10 log(υsd )

15

HW , Car

−15 −20 −25

HT , µ = 4, WGN HW , WGN

−30 −35

(b)

−40 −10

−5

0

5 10 iSNR (dB)

15

20

25

Fig. 7.6. Performance for different input SNRs and noise conditions. The parameters used in this experiment are L = 20, αy = 0.985, and αv = 0.995.

noise to achieve better performance will be discussed in the following two chapters.

7.5 Summary In this chapter, we derived several forms of time-domain filters for noise reduction. The Wiener filter is optimal in the MSE sense and many known algorithms are somehow related to it. We discussed many important and in-

76

7 Optimal Filters in the Time Domain

teresting properties of the Wiener filter and showed that the it achieves SNR improvement at the cost of speech distortion. We also derived two tradeoff filters, which are also optimal from an MMSE point of view. The only difference between the Wiener and tradeoff filters is that the latter one is deduced from a constrained MSE criterion where a parameter is used to control the compromise between the amount of noise reduction and the amount of speech distortion. The subspace method can be viewed as a special way of implementing the tradeoff filter. Using experiments, we have also illustrated the impact of some important parameters on the performance of noise reduction time-domain filters. While the time domain is the most straightforward and natural domain to work with, the derived noise reduction filters are often not very flexible in terms of performance tuning. For example, in practice, noise is not necessarily white and in many cases its energy may be concentrated only in some frequency bands (such as the car noise). In such situations, it is advantageous to design a noise reduction filter in the frequency domain, which will be discussed in the next chapter.

8 Optimal Filters in the Frequency Domain

We now investigate the design of noise reduction filters in the frequency domain, which is an alternative to the time-domain approach studied in the previous chapter. In the real world, frequency-domain filters are, by far, the most popular methods. This is mainly because: 1) the filters at different frequencies (or frequency bands) are designed and handled independently with each other, therefore there is a significant flexibility in dealing with colored noise; 2) most of our knowledge and understanding of speech production and perception is related to frequencies; and 3) thanks to the fast Fourier transform (FFT), the implementation of frequency-domain filters are generally very efficient. So, in this chapter, we develop some widely used classical frequency-domain filters and discuss their properties.

8.1 Wiener Filter Taking the gradient of J [H(jω)] [eq. (4.15)] with respect to H ∗ (jω) and equating the result to 0 lead to −E {Y ∗ (jω) [X(jω) − HW (jω)Y (jω)]} = 0.

(8.1)

φy (ω)HW (jω) = φxy (jω).

(8.2)

Hence

But φxy (jω) = E [X(jω)Y ∗ (jω)] = φx (ω). Therefore the optimal filter can be put into the following forms:

J. Benesty et al., Noise Reduction in Speech Processing, Springer Topics in Signal Processing 2, DOI 10.1007/978-3-642-00296-0_8, © Springer-Verlag Berlin Heidelberg 2009

78

8 Optimal Filters in the Frequency Domain

φx (ω) φy (ω) φv (ω) = 1− φy (ω) iSNR(ω) = . 1 + iSNR(ω)

HW (jω) =

(8.3)

We see that the noncausal Wiener filter is always real and positive. Therefore, from now on we will drop the imaginary unit from HW (jω), i.e., HW (ω), to accentuate the fact that this filter is a real number. The same frequencydomain filter can also be easily deduced from the framework of Bayesian risk theory in the particular case where the model is Gaussian [39], [58], [126]. Another way to define the Wiener filter is with the MSCFs. Indeed, it is easy to see that HW (ω) = |ρ [X(jω), Y (jω)] |2 = 1 − |ρ [V (jω), Y (jω)] |2 .

(8.4)

These fundamental forms of the Wiener filter, although obvious, do not seem to be known in the literature. They show that they are simply related to two MSCFs. Since 0 ≤ |ρ [X(jω), Y (jω)] |2 ≤ 1, therefore 0 ≤ HW (ω) ≤ 1. The Wiener filter acts like a gain function. When the level of noise is high at frequency ω, |ρ [V (jω), Y (jω)] |2 ≈ 1, then HW (ω) is close to 0 since there is a large amount of noise that has to be removed. When the level of noise is low at frequency ω, |ρ [V (jω), Y (jω)] |2 ≈ 0, then HW (ω) is close to 1 and is not going to affect much the signals since there is little noise that needs to be removed. Now, let us define the complex number1

[X(jω), V (jω)] = ρ [X(jω), Y (jω)] + jρ [V (jω), Y (jω)] = cos θ(ω) + j sin θ(ω),

(8.5)

where θ(ω) is the angle of [X(jω), V (jω)] for which the modulus is equal to 1. On the complex plane, [X(jω), V (jω)] is on the unit circle. Since 0 ≤ ρ [X(jω), Y (jω)] ≤ 1 and 0 ≤ ρ [V (jω), Y (jω)] ≤ 1, therefore 0 ≤ θ(ω) ≤ π2 . We can then rewrite the Wiener filter as a function of the angle θ(ω): HW (ω) = cos2 θ(ω) = 1 − sin2 θ(ω).

(8.6)

Hence lim HW (ω) = 1,

(8.7)

lim HW (ω) = 0.

(8.8)

θ(ω)→0 θ(ω)→ π 2 1

Notice that both ρ [X(jω), Y (jω)] and ρ [V (jω), Y (jω)] are real numbers.

8.1 Wiener Filter

79

We deduce the subband noise-reduction factor and subband speechdistortion index ξnr [HW (ω)] =

1 ≥ 1, cos4 θ(ω)

υsd [HW (ω)] = sin4 θ(ω) ≤ 1,

(8.9) (8.10)

and the subband MNMSE J˜ [HW (ω)] = HW (ω),

(8.11)

which is exactly the Wiener filter. We see clearly how noise reduction and speech distortion depend on the angle θ(ω) in the noncausal Wiener filter. When θ(ω) increases so does ξnr [HW (ω)]; at the same time υsd [HW (ω)] increases. Property 8.1. With the optimal noncausal Wiener filter given in (8.3), the fullband output SNR is always greater than or equal to the input SNR, i.e., oSNR(HW ) ≥ iSNR. Proof. Let us evaluate the fullband SPCC between Y (jω) and ZW (jω) = HW (ω)Y (jω):  2 π HW (ω)φy (ω)dω −π    |ρ (Y, ZW ) |2 =  π π 2 (ω)φ (ω)dω φ (ω)dω HW y −π y −π π π φ (ω)dω φx (ω)dω −π x = π · π −π φ (ω)dω −π HW (ω)φx (ω)dω −π y =

|ρ (X, Y ) |2 . |ρ (X, ZW ) |2

Therefore |ρ (X, Y ) |2 = |ρ (Y, ZW ) |2 · |ρ (X, ZW ) |2 ≤ |ρ (X, ZW ) |2 . Using (6.31) and Property 6.12 in the previous expression, we get iSNR oSNR(HW ) ≤ , 1 + iSNR 1 + oSNR(HW ) as a result oSNR(HW ) ≥ iSNR. Substituting (8.3) into (4.22), we find the fullband MNMSE: π 2 φv (ω)φ−1 y (ω)dω ˜ J (HW ) = 1 − −π π ≤ 1. φ (ω)dω −π v

(8.12)

80

8 Optimal Filters in the Frequency Domain

We can compute the fullband speech-distortion index by substituting (8.3) into (3.26): υsd (HW ) = 1 −

oSNR(HW ) + 2 ≤ 1. iSNR · ξnr (HW )

(8.13)

Using (4.22) and (8.13), we get the fullband noise-reduction factor: ξnr (HW ) =

oSNR(HW ) + 1 ≥ 1. iSNR − J˜ (HW )

(8.14)

Property 8.2. We have
 J˜ (HW ) = iSNR 1 − |ρ(X, ZW )|2 .

(8.15)

Proof. Indeed π π 2 φ (ω)dω φx (ω)φ−1 y (ω)dω −π x ˜ J (HW ) = π − −π π φ (ω)dω φ (ω)dω −π v −π v   π 2 −1 φ (ω)φy (ω)dω −π x π = iSNR 1 − φ (ω)dω −π x
 = iSNR 1 − |ρ(X, ZW )|2 . Therefore, the fullband NMSE is minimized when the fullband SPCC between the signals X(jω) and Z(jω) is maximized. This fullband SPCC can be rewritten as follows: |ρ(X, ZW )|2 =

1 1 + oSNR(HW ) · . ξsr (HW ) oSNR(HW )

(8.16)

We observe that the noncausal Wiener filter is compromising between speech reduction (i.e., speech distortion) and fullband output SNR improvement. Property 8.3. We have iSNR iSNR ≤ J˜ (HW ) ≤ . 1 + oSNR(HW ) 1 + iSNR Proof. Since |ρ(X, ZW )|2 ≥

iSNR 1 + iSNR

and with the help of (8.15), we easily get J˜ (HW ) ≤

iSNR . 1 + iSNR

(8.17)

8.2 Parametric Wiener Filter

81

Since |ρ(X, ZW )|2 ≤

oSNR(HW ) 1 + oSNR(HW )

and, again, with the help of (8.15), we obtain iSNR ≤ J˜ (HW ) . 1 + oSNR(HW ) Hence, we get better bounds for the fullband MNMSE than the usual ones [0 ≤ J˜ (HW ) ≤ 1]. Property 8.4. We have 2

[1 + oSNR(HW )] (1 + iSNR) [1 + oSNR(HW )] ≤ ξnr (HW ) ≤ . (8.18) iSNR · oSNR(HW ) iSNR2 Proof. Easy to show by using (8.14) and the bounds of J˜ (HW ) [eq. (8.17)]. Property 8.5. We have 1 2

[1 + oSNR(HW )]

≤ υsd (HW ) ≤

1 + oSNR(HW ) − iSNR . (8.19) (1 + iSNR) [1 + oSNR(HW )]

Proof. Easy to show by using (8.13) and the bounds of ξnr (HW ) [eq. (8.18)]. Very often in practice, the ensemble averages are unknown, so it is convenient to approximate the PSDs used in the Wiener filter by sample estimates [38], [119]: 2 ˆ W (ω) = 1 − V (ω) H Y 2 (ω)

(8.20)

= |ˆ ρ [V (jω), Y (jω)] |2 . This form of the Wiener filter is the starting point of so many spectrum-based noise reduction techniques [33], [44], [86], [88], [119].

8.2 Parametric Wiener Filter Some applications may need aggressive noise reduction. Other applications on the contrary may require little speech distortion (so less aggressive noise reduction). An easy way to control the compromise between noise reduction and speech distortion is via the parametric Wiener filter2 [50], [85]: 2

There is nothing optimal about the parametric Wiener filter but for convenience of presentation we included it in this chapter.

82

8 Optimal Filters in the Frequency Domain


β2 HG (ω) = 1 − sinβ1 θ(ω) ,

(8.21)

where β1 and β2 are two positive parameters that allow the control of this compromise. For (β1 , β2 ) = (2, 1), we get the noncausal Wiener filter developed previously. Taking (β1 , β2 ) = (2, 1/2), leads to HP (ω) =

1 − sin2 θ(ω)

(8.22)

= cos θ(ω), which is the power subtraction method studied in [44], [50], [85], [92], [112]. The pair (β1 , β2 ) = (1, 1) gives the magnitude subtraction method [17], [18], [108], [109], [122]: HM (ω) = 1 − sin θ(ω) ! = 1 − 1 − cos2 θ(ω).

(8.23)

We can verify that the subband noise-reduction factors for the power subtraction and magnitude subtraction methods are ξnr [HP (ω)] = ξnr [HM (ω)] =

1 , θ(ω) 1

(8.24)

cos2

2,

[1 − sin θ(ω)]

(8.25)

and the corresponding subband speech-distortion indices are 2

υsd [HP (ω)] = [1 − cos θ(ω)] , υsd [HM (ω)] = sin2 θ(ω).

(8.26) (8.27)

We can also easily check that ξnr [HM (ω)] ≥ ξnr [HW (ω)] ≥ ξnr [HP (ω)] ,

(8.28)

υsd [HP (ω)] ≤ υsd [HW (ω)] ≤ υsd [HM (ω)] .

(8.29)

The two previous inequalities are very important from a practical point of view. They show that, among the three methods, the magnitude subtraction is the most aggressive one as far as noise reduction is concerned, a very wellknown fact in the literature [38], but at the same time it’s the one that will likely distorts most the speech signal. The smoothest approach is the power subtraction while the Wiener filter is between the two others in terms of speech distortion and noise reduction. Several other variants of these algorithms can be found in [62], [87], [110].

8.3 Tradeoff Filter An important filter can be designed by minimizing the speech distortion with the constraint that the residual noise level is equal to a positive number smaller

8.3 Tradeoff Filter

83

than the level of the original noise. This optimization problem can be translated mathematically as min Jx [H(jω)]

H(jω)

subject to Jv [H(jω)] = β · φv (ω),

(8.30)

2

(8.31)

where Jx [H(jω)] = |1 − H(jω)| φx (ω), 2

Jv [H(jω)] = |H(jω)| φv (ω),

(8.32)

and 0 < β < 1 in order to have some noise reduction at frequency ω. If we use a Lagrange multiplier, µ ≥ 0, to adjoin the constraint to the cost function, we easily find the tradeoff filter: φx (ω) φx (ω) + µφv (ω) φy (ω) − φv (ω) = φy (ω) + (µ − 1)φv (ω) iSNR(ω) = . µ + iSNR(ω)

HT (ω) =

(8.33)

This filter can be seen as a noncausal Wiener filter with adjustable input noise level µφv (ω). √ The MSCF between the two signals X(jω) and X(jω) + µV (jω) at frequency ω is √

iSNR(ω) . (8.34) µ + iSNR(ω) √ The MSCF between the two signals V (jω) and X(jω)+ µV (jω) at frequency ω is √ µ 2 |ρ [V (jω), X(jω) + µV (jω)]| = . (8.35) µ + iSNR(ω) |ρ [X(jω), X(jω) +

2

µV (jω)]| =

Therefore, we can write the tradeoff filter as a function of these two MSCFs: √ 2 HT (ω) = |ρ [X(jω), X(jω) + µV (jω)]| √ 2 = 1 − |ρ [V (jω), X(jω) + µV (jω)]| . (8.36) Now, let us define the complex number3 √

µ [X(jω), V (jω)] = ρ [X(jω), X(jω) + µV (jω)] + √ jρ [V (jω), X(jω) + µV (jω)] = cos θ(ω, µ) + j sin θ(ω, µ), 3




Notice that both ρ X(jω), X(jω) + are real numbers.

√






µV (jω) and ρ V (jω), X(jω) +

(8.37) √

µV (jω)



84

8 Optimal Filters in the Frequency Domain

where θ(ω, µ) is the angle of µ [X(jω), V (jω)] for which the modulus is equal to
1. On the complex plane, µ [X(jω), V (jω)]
is on the unit circle. Since 0 ≤ √ √ ρ X(jω), X(jω) + µV (jω) ≤ 1 and 0 ≤ ρ V (jω), X(jω) + µV (jω) ≤ 1, therefore 0 ≤ θ(ω, µ) ≤ π2 . We can then rewrite the tradeoff filter as a function of the angle θ(ω, µ): HT (ω) = cos2 θ(ω, µ) = 1 − sin2 θ(ω, µ).

(8.38)

We deduce the subband noise-reduction factor and subband speech-distortion index ξnr [HT (ω)] =

1 ≥ 1, cos4 θ(ω, µ)

(8.39)

υsd [HT (ω)] = sin4 θ(ω, µ) ≤ 1.

(8.40)

Property 8.6. With the tradeoff filter given in (8.33), the fullband output SNR is always greater than or equal to the input SNR, i.e., oSNR(HT ) ≥ iSNR. Proof. The fullband SPCC between the two variables X(jω) and X(jω) + √ µV (jω) is

|ρ (X, X +

√

 π 2 µV )| =  π

φ (ω)dω −π x

=

2

−π

φx (ω)dω

π −π

φx (ω)dω + µ

 



π

φ (ω)dω −π v

iSNR . µ + iSNR

The fullband SPCC between the two variables X(jω) and HT (ω)X(jω) + √ µHT (ω)V (jω) is |ρ (X, HT X +  π π =

√

2

µHT V )| =  π

φ (ω)dω −π x

 

π −π

−π

HT (ω)φx (ω)dω

2

HT2 (ω)φx (ω)dω + µ

π

HT2 (ω)φv (ω)dω −π

HT (ω)φx (ω)dω π . φ (ω)dω −π x

−π

Another way to write the same fullband SPCC is the following:



8.3 Tradeoff Filter

85



2 H (ω)φ (ω)dω T x √ 2   × |ρ (X, HT X + µHT V )| =  π π 2 (ω)φ (ω)dω φ (ω)dω H x x T −π −π π −π

oSNR (HT ) µ + oSNR (HT ) 2

= |ρ (X, HT X)| · |ρ (HT X, HT X + oSNR (HT ) ≤ . µ + oSNR (HT )

√

µHT V )|

2

Now, let us evaluate the fullband SPCC between the two variables X(jω) + √ √ µV (jω) and HT (ω)X(jω) + µHT (ω)V (jω): π φ (ω)dω √ √ 2 −π x π |ρ (X + µV, HT X + µHT V )| = π × φ (ω)dω + µ −π φv (ω)dω −π x π φx (ω)dω π −π HT (ω)φx (ω)dω −π    ρ X, X + √µV 2 =    . ρ X, HT X + √µHT V 2 Therefore |ρ (X, X +

√

2

iSNR µ + iSNR √ √ 2 = |ρ (X + µV, HT X + µHT V )| × √ 2 |ρ (X, HT X + µHT V )| √ 2 ≤ |ρ (X, HT X + µHT V )| oSNR (HT ) ≤ . µ + oSNR (HT )

µV )| =

As a result oSNR (HT ) ≥ iSNR. The tradeoff filter can be more general if we make the factor β dependent on the frequency, i.e., β(ω). By doing so, the control between noise reduction and speech distortion can be more effective since each frequency ω can be controlled independently of the others. With this consideration, we can easily see that the optimal filter derived from the criterion (8.30) is now HT (ω) =

iSNR(ω) , µ(ω) + iSNR(ω)

(8.41)

where µ(ω) is the frequency-dependent Lagrange multiplier. This approach can now provide some noise spectral shaping for masking by the speech signal [49], [68], [69], [75], [120], [121].

86

8 Optimal Filters in the Frequency Domain

8.4 Experiments We have discussed in the previous sections several noise reduction filters in the frequency domain. While some filters (such as the Wiener filter) are optimal in the MMSE sense, others are constructed empirically (such as the parametric Wiener filter) and have no optimality properties associated with them, but they make good sense for managing the compromise between noise reduction and speech distortion. In this section, we study the performance of these main frequency-domain filters through experiments. The experimental setup is the same as used in the previous chapter. 8.4.1 Impact of Input SNR on Filter Gain and Speech Distortion As seen from the previous sections, all the main frequency-domain filters, including the Wiener, parametric Wiener, and tradeoff filters, are essentially a function of the subband input SNR, i.e., iSNR(ω). Figure 8.1 plots the theoretical gain of each filter versus the input SNR at frequency ω where the gain is shown in both linear and dB scales. With no exception, we see that the gain of each filter is always in the range between 0 and 1, and its value is proportional to the subband input SNR. Since a lower gain corresponds to more noise attenuation, we can expect significant amount of noise suppression when iSNR(ω) is small while less noise reduction when iSNR(ω) is large. This is, of course, reasonable. When iSNR(ω) is small, the signal is very noisy, so there will be much noise to be suppressed. But if iSNR(ω) is large, the signal is relatively clean, so, there won’t be much noise to be cleaned. Comparatively, the speech-distortion index decreases exponentially with the increase of the input SNR as shown in Fig. 8.2. Since a higher speechdistortion index indicates more speech distortion, we can expect significant amount of speech distortion when the subband input SNR is low. Putting Figs. 8.1 and 8.2 together, one can clearly see that noise reduction is always associated with speech distortion. The more the noise is reduced, the more the speech is distorted. As for how these filters behave in practical environments, this will be studied through experiments. But first let us discuss how to estimate the noise signal so that we can implement the frequency-domain filters. 8.4.2 Noise Estimation An important issue, in the implementation of an algorithm for noise reduction, is the estimation of the statistics of the noise signal. The accuracy of this estimator greatly affects the noise reduction performance. There has been a tremendous effort in tackling this problem. Representative approaches include estimating noise in the absence of speech, minimum statistics method, quantile-based method, and sequential estimation using single-pole recursion.

8.4 Experiments

87

1 HT (ω), µ = 0.1

0.8 HP (ω)

Gain

0.6

HW (ω)

HM (ω)

0.4

HT (ω), µ = 10

0.2 (a)

0 −30

−20

−10

0

10

20

30

Gain (dB)

0 −10

HT (ω), µ = 10

−20

HM (ω)

−30

HW (ω)

−40

HP (ω)

−50

HT (ω), µ = 0.1

−60 −70 (b)

−80 −30

−20

−10

0 iSNR(ω) (dB)

10

20

30

Fig. 8.1. Theoretical gain of different filters as a function of the subband input SNR, i.e., iSNR(ω). (a) Gain in linear scale and (b) Gain in dB.

Usually, a noisy speech signal is not occupied by speech all the time. In a large percentage of the time it is occupied by noise only. Therefore, a voice activity detector (VAD) can be designed to distinguish speech and non-speech segments for a given noisy signal and the noise can then be estimated from regions where the speech signal is absent. This basic noise estimation relies on a VAD with high detection accuracy. When noise is strong so that the input SNR becomes rather low, the distinction between speech and noise segments could be difficult. Moreover, noise is estimated intermittently and obtained

88

8 Optimal Filters in the Frequency Domain 0

10 log[υsd H(jω)] (dB)

−10

HT (ω), µ = 0.1

−20

HP (ω)

−30

HW (ω)

−40

HM (ω)

−50

HT (ω), µ = 10

−60 −70 −80 −30

−20

−10

0 iSNR(ω) (dB)

10

20

30

Fig. 8.2. Theoretical speech-distortion index as a function of the subband input SNR, i.e., iSNR(ω).

only during speech silent periods. This may cause problems if the noise is non-stationary, which is the case in many applications. To avoid explicit speech/non-speech detection, the so-called minimum statistics method was developed [89]. This technique is based on the assumption that during a speech pause, or within brief periods between words and even syllables, the speech energy is close to zero. As a result, a short-term power spectrum estimate of the noisy signal, even during speech activity, decays frequently due to the noise power. Thus, by tracking the temporal spectral minimum without distinguishing between speech presence and speech absence, the noise power in a specific frequency band can be estimated. Although a VAD is not necessary in this approach, the noise estimate is often too small to provide sufficient noise reduction due to the significant variability inherent to the spectral estimation. Instead of using minimum statistics, Hirsch et al. proposed a histogram based method which achieves a noise estimate from the subband energy histograms [65]. A threshold is set over which peaks in the histogram profile are attributed to speech. The highest peak in the profile below this threshold is treated as noise energy. This idea was extended in [114] to a quantile-based noise estimation approach, which works on the assumption that even in active speech sections of the input signal not all frequency bands are permanently occupied with speech, and for a large percentage of the time the energy is at the noise level. Thus, this method computes short-term power spectra and sorts them. The noise estimate is obtained by taking a value near to the median of the resulting profiles.

8.4 Experiments

89

Recently, sequential estimation methods [23], [38] were developed, where noise is estimated sequentially using a single-pole recursive average with an implicit speech/non-speech decision embedded. Briefly, the noisy signal y(k) is segmented into blocks of L samples. Each block is then transformed via a DFT (discrete Fourier transform) into a block of L spectral samples. Successive blocks of spectral samples form a two-dimensional time-frequency matrix denoted by Yn (jω), where the subscript n is the frame index and denotes the time dimension. Then an estimate of the magnitude of the noise spectrum is formulated as  αa |Vˆn−1 (jω)| + (1 − αa )|Yn (jω)|, if |Yn (jω)| ≥ |Vˆn−1 (jω)| |Vˆn (jω)| = , αd |Vˆn−1 (jω)| + (1 − αd )|Yn (jω)|, if |Yn (jω)| < |Vˆn−1 (jω)| (8.42) where αa is the “attack” coefficient and αd is the “decay” coefficient. Meanwhile, to reduce its temporal fluctuation, the magnitude of the noisy speech spectrum is smoothed according to the following recursion:  βa |Y¯n−1 (jω)| + (1 − βa )|Yn (jω)|, if |Yn (jω)| ≥ |Y¯n−1 (jω)| ¯ |Yn (jω)| = , βd |Y¯n−1 (jω)| + (1 − βd )|Yn (jω)|, if |Yn (jω)| < |Y¯n−1 (jω)| (8.43) where again βa is the “attack” coefficient and βd the “decay” coefficient. To further reduce the spectral fluctuation, both |Vˆn (jω)| and |Y¯n (jω)| are averaged across the neighboring frequency bins around ω. Finally, an estimate of the noise spectrum is obtained by multiplying |Vˆn (jω)|/|Y¯n (jω)| with Yn (jω), and the time-domain noise signal is obtained through the inverse DFT and the overlap-add technique. To illustrate the accuracy of the sequential noise estimator, we provide an example where a clean speech signal (same as used in Chapter 7) is corrupted by the car noise with an iSNR = 10 dB. The estimated noise and its spectrogram are plotted in Fig. 8.3. Comparing Fig. 8.3 with Fig. 7.2, one can see that during the absence of speech, the noise estimate is a good approximation of the noise signal. During the presence of speech, the noise estimate consists of some minor speech components. However, such minor amount of speech remained in the noise estimate is almost inaudible. An apparent advantage of this sequential noise estimation technique is that it does not require an explicit VAD. In addition, such a method is in general able to capture the noise characteristics in both the presence and absence of speech, therefore it works even in the environments where the noise is not very stationary. 8.4.3 Performance Comparison in NYSE Noise Based upon the previous sequential noise estimation method, we implemented a frequency-domain noise reduction system using the overlap-add technique. Briefly, the input noisy speech is partitioned into overlapped frames with a

90

8 Optimal Filters in the Frequency Domain

Amplitude

0.2

0

Frequency (kHz)

−0.2 0 4

1

2

3

4

5 1.0 0.8 0.6

2 0.4 0.2 0 0

1

2

3

4

5

0

Time (s)

Fig. 8.3. Waveform and spectrogram (first 5 s) of the estimated car noise with an iSNR = 10 dB. 0.5 0

Amplitude

−0.5 0 0.5

1

2

3

4

5

1

2

3

4

5

1

2

3

4

5

0

−0.5 0 0.5 0 −0.5 0

Time (s)

Fig. 8.4. Waveforms of the clean, noisy, and enhanced speech (with the frequencydomain Wiener filter) in NYSE noise with an iSNR = 10 dB.

frame length of 8 ms (corresponding to 64 points at an 8-kHz sampling rate) and an overlap factor of 75%. Each frame is multiplied with a Kaiser window and then transformed into the frequency domain using the FFT. The noise spectrum is then estimated using the sequential noise estimator, and a gain

8.4 Experiments 4

91

1.0

2

Frequency (kHz)

0.8 0 4 0.6 2 0.4

0 4

0.2

2 0 0

1

2

3

4

5

0

Time (s)

Fig. 8.5. Spectrograms of the clean, noisy, and enhanced speech (with the frequencydomain Wiener filter) in NYSE noise with an iSNR = 10 dB.

filter is then computed according to (8.3), (8.22), (8.23), and (8.33) respectively. The estimated gain filter is subsequently applied to the noisy speech spectrum. The gain-modified noise speech spectrum is then transformed back to the time domain using the inverse FFT (IFFT). A synthesis window, which is the same as the analysis window, is applied, and the result is overlapped with, and added to the previous estimated signal to obtain the final enhanced speech. Figure 8.4 illustrates the first 5 seconds of the waveforms of the clean speech, noisy speech corrupted by the NYSE noise at an iSNR = 10 dB, and the enhanced speech using the frequency-domain Wiener filter. It shows, at least visibly, that the background noise is significantly reduced using the Wiener filter. To visualize the clean speech estimate in the frequency domain, the spectrograms of the clean, noisy, and enhanced speech signals are displayed in Fig. 8.5. It can be seen that the background noise is dramatically removed, and the spectrogram of the estimated speech resembles that of the original speech. The detailed evaluation of the output SNR and speech-distortion index as a function of the input SNR in the NYSE noise is sketched in Fig 8.6. [Note that, similar to the previous chapter, we first computed the filtered speech and noise signals XF (jω) and VF (jω), and the output SNR and speech-distortion index are subsequently estimated. This way, the results from different chapters can be compared]. It is noticed that the output SNR is always larger than the input SNR, but the amount of SNR improvement depends on the input SNR. Roughly, the lower the input SNR, the larger the SNR improvement. However, the speech-distortion index tends to increase as more SNR improvement is

92

8 Optimal Filters in the Frequency Domain 35 HT (ω), µ = 10

30 HM (ω)

oSNR (dB)

25 20 HW (ω)

15 HP (ω)

10 5 (a)

0 0

5

10

15

20

25

30

0 −5

H M(ω) HT (ω), µ = 10

−10 −15 −20

HW (ω) HP (ω)

−25 −30 −35 (b)

−40 0

5

10

15 iSNR (dB)

20

25

30

Fig. 8.6. Performance of the different methods in different input SNRs with NYSE noise: (a) output SNR versus input SNR and (b) speech-distortion index.

gained. It is clear that noise reduction is achieved at the price of speech distortion, as we have pointed out before. Comparatively, the magnitude subtraction method achieves a higher SNR gain as compared to the Wiener filter, but it also has a larger speech-distortion index. The power subtraction is less effective than the Wiener filter in terms of SNR improvement, but it causes less speech distortion. Therefore, these two approaches can be chosen to replace the Wiener filter if a compromise between noise reduction and speech distortion is needed. If more noise reduction is required, we can use the magnitude subtraction. If, on the other hand, we

8.4 Experiments 35 30

93

HT (ω), µ = 10 HM (ω)

oSNR (dB)

25 20

HW (ω)

15

HP (ω)

10 5 (a)

0 0

5

10

15

20

25

30

10 log(υsd )

0 −5

HM (ω)

−10

HT (ω), µ = 10

−15 −20

HW (ω) HP (ω)

−25 −30 −35 (b)

−40 0

5

10

15 iSNR (dB)

20

25

30

Fig. 8.7. Performance of the different methods in different input SNRs with car noise: (a) output SNR versus input SNR and (b) speech-distortion index.

need to keep the speech distortion low, we can use the power subtraction. Alternatively, the compromise can be achieved through the tradeoff filter by selecting an appropriate value of the parameter µ. 8.4.4 Performance Comparison in Car Noise Another set of experiments were performed in the car noise condition. The results are plotted in Fig. 8.7. In general, we see that each method has gained more SNR improvement in the car noise than that in the NYSE noise. But

94

8 Optimal Filters in the Frequency Domain

the trend of the SNR improvement and speech-distortion index with respect to the input SNR in the car noise is similar to that in the NYSE noise.

8.5 Summary In this chapter, we discussed many noise reduction filters in the frequency domain. Both the Wiener and tradeoff filters are optimal in the MMSE sense. The difference between the two is that the former is derived from an unconstrained MSE criterion, while the latter is obtained from a constrained MSE criterion. The magnitude subtraction and power subtraction filters (or more generally the parametric Wiener filter) have no optimality properties associated with them, but they make good sense in practical situations for managing the compromise between noise reduction and speech distortion. We have also discussed many interesting properties of these filters and some of these properties were verified through experiments.

9 Optimal Filters in the KLE Domain

We have seen the advantages of transforming the noisy speech into the frequency domain and then performing noise reduction. Although it is the most studied one, the frequency domain is not the only domain in which we can design interesting filters for noise reduction. There are other domains that may offer more advantages in the filter design. In this chapter, we consider the KLE domain. we derive two broad classes of optimal filters in this domain depending basically on the subband filter length. The first class, similar to the frequency-domain filters, estimates a frame of the clean speech by filtering the corresponding frame of the noisy speech while the second class does noise reduction by filtering not only the current frame but also a number of previous consecutive frames of the noisy speech.

9.1 Class I In this first category of filters, we consider the particular case where L1 = L2 = · · · = LL = 1. Hence hl = hl,0 , l = 1, 2, . . . , L, are simply scalars. For this class of filters, we always have oSNR(hl,0 ) = iSNRl , ∀l.

(9.1)

Therefore, the subband output SNR cannot be improved with respect to the subband input SNR. But the fullband output SNR can be improved with respect to the input SNR and from Chapter 3 we know that it is upper bounded (for all filters) as follows: oSNR (h1:L ) ≤

L  l=1

iSNRl =

L 

oSNR(hl,0 ).

(9.2)

l=1

9.1.1 Wiener Filter By minimizing J (hl,0 ) [eq. (4.29)] with respect to hl,0 , we easily find the Wiener filter:

J. Benesty et al., Noise Reduction in Speech Processing, Springer Topics in Signal Processing 2, DOI 10.1007/978-3-642-00296-0_9, © Springer-Verlag Berlin Heidelberg 2009

96

9 Optimal Filters in the KLE Domain

hW,l,0

  E c2x,l (k)  =  E c2y,l (k)   E c2v,l (k)  = 1−  E c2y,l (k) λl λl + qTl Rvv ql iSNRl = . 1 + iSNRl =

(9.3)

This filter is the equivalent form of the frequency-domain Wiener filter (see Chapter 8). Another way to define the Wiener filter is with the subband SPCCs. Indeed, it is easy to see that hW,l,0 = ρ2 (cx,l , cy,l ) = 1 − ρ2 (cv,l , cy,l ).

(9.4)

Since 0 ≤ ρ2 (cx,l , cy,l ) ≤ 1, therefore 0 ≤ hW,l,0 ≤ 1. The KLE-domain Wiener filter acts like a gain function. When the level of noise is high in the subband l, ρ2 (cv,l , cy,l ) ≈ 1, then hW,l,0 is close to 0 since there is a large amount of noise that has to be removed. When the level of noise is low in the subband l, ρ2 (cv,l , cy,l ) ≈ 0, then hW,l,0 is close to 1 and is not going to affect much the signals since there is little noise that needs to be removed. Now, let us define the complex number

(cx,l , cv,l ) = ρ(cx,l , cy,l ) + jρ(cv,l , cy,l ) = cos θl + j sin θl ,

(9.5)

where θl is the angle of (cx,l , cv,l ) for which the modulus is equal to 1. On the complex plane, (cx,l , cv,l ) is on the unit circle. Since 0 ≤ ρ(cx,l , cy,l ) ≤ 1 and 0 ≤ ρ(cv,l , cy,l ) ≤ 1, therefore 0 ≤ θl ≤ π2 . We can then rewrite the Wiener filter as a function of the angle θl : hW,l,0 = cos2 θl = 1 − sin2 θl .

(9.6)

Hence lim hW,l,0 = 1,

(9.7)

lim hW,l,0 = 0.

(9.8)

θl →0 θl → π 2

We deduce the subband noise-reduction factor and subband speechdistortion index

9.1 Class I

1 ≥ 1, cos4 θl υsd (hW,l,0 ) = sin4 θl ≤ 1, ξnr (hW,l,0 ) =

97

(9.9) (9.10)

and the subband MNMSE J˜ (hW,l,0 ) = hW,l,0 ,

(9.11)

which is exactly the Wiener filter. We see clearly how noise reduction and speech distortion depend on the angle θl in the KLE-domain Wiener filter. When θl increases so does ξnr (hW,l,0 ); at the same time υsd (hW,l,0 ) increases. It is also easy to check the formula: 2 1 υsd (hW,l,0 ) = 1 − ! + . ξ (h ξnr (hW,l,0 ) nr W,l,0 )

(9.12)

Property 9.1. With the optimal KLE-domain Wiener filter given in (9.3), the fullband output SNR is always greater than or equal to the input SNR, i.e., oSNR(hW,1:L ) ≥ iSNR. Proof. Let us evaluate the fullband SPCC between the two vectors cy (k) and T cz,W (k) = [hW,1,0 cy,1 (k) hW,2,0 cy,2 (k) · · · hW,L,0 cy,L (k)] : 

 2 hW,l,0 E c2y,l (k)      ρ2 (cy , cz,W ) =  L L 2 2 2 E c (k) · h E c (k) l=1 y,l l=1 W,l,0 y,l     L L 2 2 E c (k) E c (k) x,l x,l l=1 l=1  ·   =  L L 2 2 l=1 E cy,l (k) l=1 hW,l,0 E cx,l (k) L l=1

=

ρ2 (cx , cy ) . ρ2 (cx , cz,W )

Therefore ρ2 (cx , cy ) = ρ2 (cy , cz,W ) · ρ2 (cx , cz,W ) ≤ ρ2 (cx , cz,W ) . Using (6.76) and Property 6.32 in the previous expression, we get iSNR oSNR(hW,1:L ) ≤ , 1 + iSNR 1 + oSNR(hW,1:L ) as a result oSNR(hW,1:L ) ≥ iSNR.

98

9 Optimal Filters in the KLE Domain

Substituting (9.3) into (4.36), we find the fullband MNMSE: 2  −1 L  T T q R q λ + q R q v l v l l l l l=1 J˜ (hW,1:L ) = 1 − ≤ 1. L T l=1 ql Rv ql

(9.13)

We can compute the fullband speech-distortion index by substituting (9.3) into (3.27): υsd (hW,1:L ) = 1 −

oSNR(hW,l:L ) + 2 ≤ 1. iSNR · ξnr (hW,l:L )

(9.14)

Using (4.36) and (9.14), we get the fullband noise-reduction factor: oSNR(hW,1:L ) + 1 ≥ 1. iSNR − J˜ (hW,1:L )

(9.15)


 J˜ (hW,1:L ) = iSNR 1 − ρ2 (cx , cz,W ) .

(9.16)

ξnr (hW,1:L ) = Property 9.2. We have

Proof. Indeed  −1 L L λ2l λl + qTl Rv ql λl J˜ (hW,1:L ) = L l=1 − l=1  L T T l=1 ql Rv ql l=1 ql Rv ql   −1  L 2 T l=1 λl λl + ql Rv ql = iSNR 1 − L l=1 λl
 2 = iSNR 1 − ρ (cx , cz,W ) . Therefore, the fullband NMSE is minimized when the fullband SPCC between the two vectors cx (k) and cz,W (k) is maximized. This fullband SPCC can be rewritten as follows: ρ2 (cx , cz,W ) =

1 1 + oSNR(hW,1:L ) · . ξsr (hW,1:L ) oSNR(hW,1:L )

(9.17)

We observe that the KLE-domain Wiener filter is compromising between speech reduction (i.e., speech distortion) and fullband output SNR improvement. Property 9.3. We have iSNR iSNR ≤ J˜ (hW,1:L ) ≤ . 1 + oSNR(hW,1:L ) 1 + iSNR Proof. Since ρ2 (cx , cz,W ) ≥

iSNR 1 + iSNR

(9.18)

9.1 Class I

99

and with the help of (9.16), we easily get J˜ (hW,1:L ) ≤

iSNR . 1 + iSNR

Since ρ2 (cx , cz,W ) ≤

oSNR(hW,1:L ) 1 + oSNR(hW,1:L )

and, again, with the help of (9.16), we obtain iSNR ≤ J˜ (hW,1:L ) . 1 + oSNR(hW,1:L ) Hence, we get better bounds for the fullband MNMSE than the usual ones [0 ≤ J˜ (hW,1:L ) ≤ 1]. Property 9.4. We have 2

[1 + oSNR(hW,1:L )] (1 + iSNR) [1 + oSNR(hW,1:L )] ≤ ξnr (hW,1:L ) ≤ . iSNR · oSNR(hW,1:L ) iSNR2 (9.19) Proof. Easy to show by using (9.15) and the bounds of J˜ (hW,1:L ) [eq. (9.18)]. Property 9.5. We have 1 [1 + oSNR(hW,1:L )]

2

≤ υsd (hW,1:L ) ≤

1 + oSNR(hW,1:L ) − iSNR . (1 + iSNR) [1 + oSNR(hW,1:L )] (9.20)

Proof. Easy to show by using (9.14) and the bounds of ξnr (hW,1:L ) [eq. (9.19)]. It is of great interest to understand how the time-domain Wiener filter filter (see Chapter 7)  −1 HW = QΛ Λ + QT Rv Q QT (9.21) is related to the KLE-domain Wiener filter given in (9.3). Substituting the KLE-domain Wiener filter into (2.30), we see that the estimator of the vector x(k) can be written as zKLE,W (k) =

L 

hW,l,0 cy,l ql

l=1

" =

L 

# hW,l,0 ql qTl

y(k)

l=1

= HKLE,W y(k).

(9.22)

100

9 Optimal Filters in the KLE Domain

Therefore, the time-domain filter HKLE,W =

L 

hW,l,0 ql qTl

(9.23)

l=1

is strictly equivalent to the KLE-domain filter hW,l,0 , l = 1, 2, . . . , L. Substituting (9.3) into (9.23), we easily find that 

−1 HKLE,W = QΛ Λ + diag QT Rv Q QT . (9.24) Clearly, the two filters HW and HKLE,W are very close. For example if the noise is white, then HW = HKLE,W . Also the orthogonal matrix

Q tends to  diagonalize the Toeplitz matrix Rv . In this case, QT Rv Q ≈ diag QT Rv Q and as a result, HW ≈ HKLE,W . 9.1.2 Parametric Wiener Filter One practical way to control the compromise between noise reduction and speech distortion is via the parametric Wiener filter [50]:
β2 hG,l,0 = 1 − sinβ1 θl ,

(9.25)

where β1 and β2 are two positive parameters that allow the control of this compromise. For (β1 , β2 ) = (2, 1), we get the KLE-domain Wiener filter developed in the previous subsection. Taking (β1 , β2 ) = (2, 1/2), leads to hP,l,0 = 1 − sin2 θl = cos θl ,

(9.26)

which is the equivalent form of the power subtraction method. The pair (β1 , β2 ) = (1, 1) gives the equivalent form of the magnitude subtraction method: hM,l,0 = 1 − sin θl ! = 1 − 1 − cos2 θl .

(9.27)

We can verify that the subband noise-reduction factors for the power subtraction and magnitude subtraction methods are ξnr (hP,l,0 ) = ξnr (hM,l,0 ) =

1 , cos2 θl 1 (1 − sin θl )

(9.28) 2,

and the corresponding subband speech-distortion indices are

(9.29)

9.1 Class I 2

υsd (hP,l,0 ) = (1 − cos θl ) , 2

υsd (hM,l,0 ) = sin θl .

101

(9.30) (9.31)

We can also easily check that ξnr (hM,l,0 ) ≥ ξnr (hW,l,0 ) ≥ ξnr (hP,l,0 ) ,

(9.32)

υsd (hP,l,0 ) ≤ υsd (hW,l,0 ) ≤ υsd (hM,l,0 ) .

(9.33)

The two previous inequalities show that, among the three methods, the magnitude subtraction is the most aggressive one as far as noise reduction is concerned, a very well-known fact in the literature [38], but at the same time it’s the one that will likely distorts most the speech signal. The smoothest approach is the power subtraction while the Wiener filter is between the two others in terms of speech distortion and noise reduction. In the previous subsection, we have shown that it was possible to rewrite the KLE-domain Wiener filter into the time domain. This way, it is more convenient to compare the two Wiener filters from a theoretical point of view (in the time and KLE domains). Following the same line of thoughts, it is easy to show that the KLE-domain power subtraction and magnitude subtraction algorithms are equivalent to 

−1/2 HKLE,P = QΛ1/2 Λ + diag QT Rv Q QT

(9.34)



−1/2 HKLE,M = I − Q I + Λdiag−1 QT Rv Q QT .

(9.35)

and

9.1.3 Tradeoff Filter The tradeoff filter is obtained by minimizing the speech distortion with the constraint that the residual noise level is equal to a value smaller than the level of the original noise. This is equivalent to solving the problem min Jx (hl,0 ) hl,0

subject to Jv (hl,0 ) = β · qTl Rv ql ,

(9.36)

where 2

Jx (hl,0 ) = (1 − hl,0 ) λl ,

(9.37)

qTl Rv ql ,

(9.38)

Jv (hl,0 ) =

h2l,0

·

and 0 < β < 1 in order to have some noise reduction in the subband l. If we use a Lagrange multiplier, µ ≥ 0, to adjoin the constraint to the cost function, we get the tradeoff filter:

102

9 Optimal Filters in the KLE Domain

λl λl + µ · qTl Rv ql   E c2y,l (k) − qTl Rv ql  =  E c2y,l (k) + (µ − 1)qTl Rv ql

hT,l,0 =

=

iSNRl . µ + iSNRl

(9.39)

This filter can be seen as a KLE-domain Wiener filter with adjustable input noise level µ · qTl Rv ql . √ The subband SPCC between the two signals cx,l (k) and cx,l (k)+ µcv,l (k) in the subband l is ρ2 (cx,l , cx,l +

√

µcv,l ) =

iSNRl . µ + iSNRl

(9.40)

√ The subband SPCC between the two signals cv,l (k) and cx,l (k) + µcv,l (k) in the subband l is √ µ ρ2 (cv,l , cx,l + µcv,l ) = . (9.41) µ + iSNRl Therefore, we can write the tradeoff filter as a function of these two subband SPCCs: √ hT,l,0 = ρ2 (cx,l , cx,l + µcv,l ) √ = 1 − ρ2 (cv,l , cx,l + µcv,l ) . (9.42) Now, let us define the complex number √ √

µ (cx,l , cv,l ) = ρ (cx,l , cx,l + µcv,l ) + jρ (cv,l , cx,l + µcv,l ) = cos θl (µ) + j sin θl (µ),

(9.43)

where θl (µ) is the angle of µ (cx,l , cv,l ) for which the modulus is equal to 1. On the complex  plane, µ (cx,l , cv,l) is on the√ unit circle. Since 0 ≤ √ ρ cx,l , cx,l + µcv,l ≤ 1 and 0 ≤ ρ cv,l , cx,l + µcv,l ≤ 1, therefore 0 ≤ θl (µ) ≤ π2 . We can then rewrite the tradeoff filter as a function of the angle θl (µ): hT,l,0 = cos2 θl (µ) = 1 − sin2 θl (µ).

(9.44)

We deduce the subband noise-reduction factor and subband speech-distortion index ξnr (hT,l,0 ) =

1 ≥ 1, cos4 θl (µ)

υsd (hT,l,0 ) = sin4 θl (µ) ≤ 1.

(9.45) (9.46)

9.1 Class I

103

Using Properties 3.2 and 3.3, we see how the fullband noise-reduction factor and fullband speech-distortion index are bounded: L 

1 , cos4 θl (µ) l=1  L   4 0 ≤ υsd (hT,1:L ) ≤ min sin θl (µ), 1 . 1 ≤ ξnr (hT,1:L ) ≤

(9.47)

(9.48)

l=1

Property 9.6. With the tradeoff filter given in (9.39), the fullband output SNR is always greater than or equal to the input SNR, i.e., oSNR(hT,1:L ) ≥ iSNR. √ Proof. The fullband SPCC between the two vectors cx (k) and cx (k)+ µcv (k) is

 2 L λl l=1 √    ρ2 (cx , cx + µcv ) =  L L L T l=1 λl l=1 λl + µ l=1 ql Rv ql =

iSNR . µ + iSNR

The fullband SPCC between the two vectors cx (k) and cx,F (k) + is ρ2 (cx , cx,F +

√

√

µcv,F (k)

µcv,F ) =

 L

2 h λ T,l,0 l l=1

    L L L 2 2 TR q λ h λ + µ h · q v l l=1 l l=1 T,l,0 l l=1 T,l,0 l L hT,l,0 λl = l=1 . L l=1 λl Another way to write the same fullband SPCC is the following:

 L

2 h λ T,l,0 l l=1 √ oSNR (hT,1:L )   · ρ2 (cx , cx,F + µcv,F ) =  L L µ + oSNR (hT,1:L ) 2 l=1 λl l=1 hT,l,0 λl √ = ρ2 (cx , cx,F ) · ρ2 (cx,F , cx,F + µcv,F ) oSNR (hT,1:L ) ≤ . µ + oSNR (hT,1:L ) Now, let us evaluate the fullband SPCC between the two vectors cx (k) + √ √ µcv (k) and cx,F (k) + µcv,F (k):

104

9 Optimal Filters in the KLE Domain

ρ2 (cx +

√

µcv , cx,F +

√

L µcv,F ) = L

λl L

l=1

l=1

λl + µ L

l=1

qTl Rv ql

×

l=1 λl L hT,l,0 λl  l=1  √ 2 ρ cx , cx + µcv . = 2 √ ρ cx , cx,F + µcv,F

Therefore ρ2 (cx , cx +

√

iSNR µ + iSNR √ √ = ρ2 (cx + µcv , cx,F + µcv,F ) × √ ρ2 (cx , cx,F + µcv,F ) √ ≤ ρ2 (cx , cx,F + µcv,F ) oSNR (hT,1:L ) ≤ . µ + oSNR (hT,1:L )

µcv ) =

As a result oSNR (hT,1:L ) ≥ iSNR. Furthermore, by using Property 3.1, we see how the fullband output SNR for the tradeoff filter is bounded: iSNR ≤ oSNR (hT,1:L ) ≤

L 

iSNRl .

(9.49)

l=1

As we already did for other filters in this class, we can write the KLEdomain tradeoff filter into the time domain. Indeed, substituting (9.39) into (2.30), we find that 

−1 HKLE,T = QΛ Λ + µ · diag QT Rv Q QT ,

(9.50)

which is identical to the filter proposed in [103]. This filter can be compared to the time-domain tradeoff filter (see Chapter 7)  −1 HT = QΛ Λ + µ · QT Rv Q QT .

(9.51)

We see that if the noise is white, the two filters are the same. We can make the tradeoff filter more general by making the factor β dependent on the subband index l, i.e., βl . By doing so, the control between noise reduction and speech distortion can be more effective since each subband l can be controlled independently of the others. With this consideration,

9.2 Class II

105

we can easily see that the optimal filter derived from the criterion (9.36) is now iSNRl hT,l,0 = , (9.52) µl + iSNRl where µl is the subband-dependent Lagrange multiplier. This approach can now provide some noise spectral shaping for masking by the speech signal [49], [68], [69], [75], [120], [121].

9.2 Class II Although they can improve the fullband SNR, the optimal filters derived in Class I have no impact on the subband SNR. In this section, we consider another category of filters hl , l = 1, 2, . . . , L, with length Ll where at least one of them has a length Ll > 1. In this scenario it is possible to improve both the subband (when Ll > 1) and fullband SNRs at the same time. 9.2.1 Wiener Filter From the KLE-domain MSE [eq. (4.29)], we easily deduce the KLE-domain filter: hW,l = R−1 cy ,l Rcx ,l il

 = Il − R−1 R il , c ,l v cy ,l

(9.53)

where Il is the identity matrix of size Ll × Ll . Another way to write this filter is

−1 Il ˜ −1 R ˜ c ,l ˜ −1 R ˜ hW,l = +R R (9.54) x cv ,l cv ,l cx ,l il iSNRl ˜ −1 R ˜ = ρ2 (cx,l , cy,l ) R cy ,l cx ,l   ˜ −1 R ˜ c ,l il , = Il − ρ2 (cv,l , cy,l ) R v cy ,l where ˜ c ,l = Rcx ,l , R x λl R cv ,l ˜ c ,l = R . v qTl Rv ql We can see from (9.54) that lim

hW,l = il ,

(9.55)

lim

hW,l = 0Ll ×1 ,

(9.56)

iSNRl →∞ iSNRl →0

where 0Ll ×1 is a vector of length Ll with all its elements being zeros.

106

9 Optimal Filters in the KLE Domain

Property 9.7. With the optimal KLE-domain Wiener filter given in (9.53), the subband output SNR is always greater than or equal to the subband input SNR, i.e., oSNR (hW,l ) ≥ iSNRl , ∀l. Proof. Let us evaluate the SPCC between cy,l (k) and hTW,l cy,l (k)

2

 iTl Rcy ,l hW,l  ρ2 cy,l , hTW,l cy,l =  

λl + qTl Rv ql hTW,l Rcy ,l hW,l =

λl λl · λl + qTl Rv ql iTl Rcx ,l hW,l

=

ρ2 (c , c )

x,l y,l  . ρ2 cx,l , hTW,l cy,l

Therefore





 ρ2 (cx,l , cy,l ) = ρ2 cy,l , hTW,l cy,l · ρ2 cx,l , hTW,l cy,l ≤ ρ2 cx,l , hTW,l cy,l .

But iSNRl , 1 + iSNRl

 oSNR (hW,l ) ρ2 hTW,l cx,l , hTW,l cy,l = , 1 + oSNR (hW,l ) ρ2 (cx,l , cy,l ) =

and





 ρ2 cx,l , hTW,l cy,l = ρ2 cx,l , hTW,l cx,l · ρ2 hTW,l cx,l , hTW,l cy,l

 ≤ ρ2 hTW,l cx,l , hTW,l cy,l .

Hence iSNRl oSNR (hW,l ) ≤ . 1 + iSNRl 1 + oSNR (hW,l ) As a result oSNR (hW,l ) ≥ iSNRl , ∀l. The KLE-domain minimum MSE (MMSE) and KLE-domain minimum NMSE (MNMSE) are obtained by replacing hW,l in (4.29) and (4.32): J (hW,l ) = λl − iTl Rcx ,l R−1 cy ,l Rcx ,l il = qTl Rv ql − iTl Rcv ,l R−1 cy ,l Rcv ,l il ,

(9.57)

iTl Rcv ,l R−1 cy ,l Rcv ,l il T ql Rv ql

(9.58)

J˜ (hW,l ) = 1 −

≤ 1.

9.2 Class II

107

We can compute the subband speech-distortion index by substituting (9.53) into (3.25): υsd (hW,l ) = 1 −

oSNR(hW,l ) + 2 ≤ 1. iSNRl · ξnr (hW,l )

(9.59)

Using (4.32) and (9.59), we get the subband noise-reduction factor: ξnr (hW,l ) =

oSNR(hW,l ) + 1 ≥ 1. iSNRl − J˜ (hW,l )

(9.60)

Property 9.8. We have 

 J˜ (hW,l ) = iSNRl 1 − ρ2 cx,l , hTW,l cy,l .

(9.61)

Proof. Indeed iTl Rcx ,l R−1 λl cy ,l Rcx ,l il − qTl Rv ql qTl Rv ql " # iTl Rcx ,l hW,l = iSNRl 1 − λl 

 = iSNRl 1 − ρ2 cx,l , hTW,l cy,l .

J˜ (hW,l ) =

Therefore, the subband NMSE is minimized when the subband SPCC between the two variables cx,l (k) and hTW,l cy,l (k) is maximized. This subband SPCC can be rewritten as follows:

 1 1 + oSNR(hW,l ) ρ2 cx,l , hTW,l cy,l = · . (9.62) ξsr (hW,l ) oSNR(hW,l ) We observe that the KLE-domain Wiener filter is compromising between subband speech reduction (i.e., subband speech distortion) and subband output SNR improvement. Property 9.9. We have iSNRl iSNRl ≤ J˜ (hW,l ) ≤ . 1 + oSNR(hW,l ) 1 + iSNRl Proof. Since

 ρ2 cx,l , hTW,l cy,l ≥

iSNRl 1 + iSNRl

and with the help of (9.61), we easily get J˜ (hW,l ) ≤

iSNRl . 1 + iSNRl

(9.63)

108

9 Optimal Filters in the KLE Domain

Since

 ρ2 cx,l , hTW,l cy,l ≤

oSNR(hW,l ) 1 + oSNR(hW,l )

and, again, with the help of (9.61), we obtain iSNRl ≤ J˜ (hW,l ) . 1 + oSNR(hW,l ) Hence, we get better bounds for the KLE-domain MNMSE than the usual ones [0 ≤ J˜ (hW,l ) ≤ 1]. Property 9.10. We have 2

[1 + oSNR(hW,l )] (1 + iSNRl ) [1 + oSNR(hW,l )] ≤ ξnr (hW,l ) ≤ .(9.64) iSNRl · oSNR(hW,l ) iSNR2l Proof. Easy to show by using (9.60) and the bounds of J˜ (hW,l ) [eq. (9.63)]. Property 9.11. We have 1 2

[1 + oSNR(hW,l )]

≤ υsd (hW,l ) ≤

1 + oSNR(hW,l ) − iSNRl . (9.65) (1 + iSNRl ) [1 + oSNR(hW,l )]

Proof. Easy to show by using (9.59) and the bounds of ξnr (hW,l ) [eq. (9.64)]. Property 9.12. With the optimal KLE-domain Wiener filter given in (9.53), the fullband output SNR is always greater than or equal to the input SNR, i.e., oSNR(hW,1:L ) ≥ iSNR. Proof. The proof is identical to the one given after Property 9.1. It is remarkable that the KLE-domain filter for Ll > 1 can improve both the subband and fullband SNRs. As a consequence, we should expect more noise reduction with this filter than the Wiener filters in the time and frequency domains. Properties 9.2, 9.3, 9.4, and 9.5 apply here as well. 9.2.2 Tradeoff Filter The tradeoff filter is obtained by solving the following optimization problem: min Jx (hl ) hl

where

subject to Jv (hl ) = β · qTl Rv ql ,

(9.66)

9.2 Class II


2  Jx (hl ) = E (il − hl )T cx,l , 

 2 Jv (hl ) = E hTl cv,l ,

109

(9.67) (9.68)

and 0 < β < 1 in order to have some noise reduction in the subband l. If we use a Lagrange multiplier, µ(≥ 0), to adjoin the constraint to the cost function, we easily find the optimal filter: −1

hT,l = (Rcx ,l + µRcv ,l ) Rcx ,l il
−1   = Rcy ,l + (µ − 1)Rcv ,l Rcy ,l − Rcv ,l il  −1 = Il + (µ − 1)R−1 R hW,l , cy ,l cv ,l

(9.69)

where the Lagrange multiplier satisfies Jv (hl ) = β·qTl Rv ql . In practice it’s not easy to determine µ. Therefore, when this parameter is chosen in an ad-hoc way, we can see that for • µ = 1, hT,l = hW,l ; so the tradeoff and Wiener filters are identical; • µ = 0, hT,l = il ; therefore, with this filter there is neither noise reduction nor speech distortion; • µ > 1, corresponds to more aggressive noise reduction compared to Wiener, so the residual noise level would be lower, but it is achieved at the expense of higher speech distortion; • µ < 1, corresponds to less aggressive noise compared to Wiener, in this situation we get less speech distortion but less noise reduction as well. Property 9.13. With the KLE-domain tradeoff filter given in (9.69), the subband output SNR is always greater than or equal to the subband input SNR, i.e., oSNR (hT,l ) ≥ iSNRl , ∀l. Proof. The subband SPCC between the two signals cx,l (k) and hTT,l cx,l (k) + √ T µhT,l cv,l (k) in the subband l is

ρ2 cx,l , hTT,l cx,l +

√



µhTT,l cv,l =

=

2 iTl Rcx ,l hT,l

 λl hTT,l Rcx ,l hT,l + µhTT,l Rcv ,l hT,l iTl Rcx ,l hT,l . λl

Another way to write the same subband SPCC is the following:

110

9 Optimal Filters in the KLE Domain



ρ2 cx,l , hTT,l cx,l +

√



2 iTl Rcx ,l hT,l

oSNR (hT,l ) · λl · hTT,l Rcx ,l hT,l µ + oSNR (hT,l )

 = ρ2 cx,l , hTT,l cx,l ×

 √ ρ2 hTT,l cx,l , hTT,l cx,l + µhTT,l cv,l

µhTT,l cv,l =

≤

oSNR (hT,l ) . µ + oSNR (hT,l )

Now let us evaluate the subband SPCC between the two signals cx,l (k) + √ √ µcv,l (k) and hTT,l cx,l (k) + µhTT,l cv,l (k) in the subband l:

 √ √ ρ2 cx,l + µcv,l , hTT,l cx,l + µhTT,l cv,l =  2 iTl (Rcx ,l + µRcv ,l ) hT,l    λl + µ · qTl Rv ql hTT,l (Rcx ,l + µRcv ,l ) hT,l λl λl · T T λl + µ · ql Rv ql il Rcx ,l hT,l   √ ρ2 cx,l , cx,l + µcv,l

. = √ ρ2 cx,l , hTT,l cx,l + µhTT,l cv,l =

Therefore ρ2 (cx,l , cx,l +

√

iSNRl µ + iSNRl

 √ √ = ρ2 cx,l + µcv,l , hTT,l cx,l + µhTT,l cv,l ×

 √ ρ2 cx,l , hTT,l cx,l + µhTT,l cv,l

 √ ≤ ρ2 cx,l , hTT,l cx,l + µhTT,l cv,l

µcv,l ) =

≤

oSNR (hT,l ) . µ + oSNR (hT,l )

As a result oSNR (hT,l ) ≥ iSNRl , ∀l. Property 9.14. With the KLE-domain tradeoff filter given in (9.69), the fullband output SNR is always greater than or equal to the input SNR, i.e., oSNR(hT,1:L ) ≥ iSNR. Proof. The proof is identical to the one given after Property 9.6.

9.3 Experiments

111

From the two previous properties, we see that the tradeoff filter in the KLE domain has the potential to improve both the subband and fullband SNRs. It is straightforward to get a more general form of the tradeoff filter by just making the factor β dependent on the subband index l. We then obtain hT,l = (Rcx ,l + µl Rcv ,l )

−1

Rcx ,l il ,

(9.70)

where now the Lagrange multiplier depends also on the subband index l. This filter could be useful in practice if we want to give a particular shape to the residual noise.

9.3 Experiments In this section, we study the performance of the two classes of noise reduction filters in the KLE domain through experiments. The experimental setup is the same as used in Chapter 7. 9.3.1 Impact of Forgetting Factor on Performance of Class-I Filters To implement the Class-I filters developed in Section 9.1, we need to know the correlation matrices Ry and Rv . The matrix Ry can be directly estimated from the noisy signal using the recursive approach given in (7.44). Therefore, the forgetting factor αy plays a critical role in the estimation accuracy of the correlation matrix Ry , which in turn may significantly affect the noise reduction performance. As explained in Chapter 7, the forgetting factor αy cannot be too large. If it is too large (close to 1), the recursive estimate will essentially be a long-term average and will not be able to follow the short-term variations of the speech signal. As a result, the nature of the speech signal cannot be fully taken advantage of, which limits the noise reduction performance. Conversely, if αy is too small, the estimation variance of Ry (k) will be large, which, again, may lead to performance degradation in noise reduction. Furthermore, Ry (k) may tend to be rank deficient, causing numerical stability problems. Therefore, a proper value of the forgetting factor is very important. In this experiment, we attempt to find the optimal forgetting factor by directly examining the noise reduction performance. White noise is used in this experiment with an iSNR = 10 dB. Similar to that in Section 7.4, we tend not to use any noise estimator, but compute the correlation matrix Rv directly from the noise signal using a long-term sample average. Figure 9.1 plots both the output SNR and speech-distortion index as a function of αy . It is seen that, for all the investigated algorithms, both the output SNR and speech-distortion index bear a nonmonotonic relationship with αy . Specifically, the output SNR first increases as αy increases and then

112

9 Optimal Filters in the KLE Domain 20 HKLE,T , µ = 4

oSNR (dB)

18

HKLE,W

16

14 HKLE,P

12

HKLE,T , µ = 0.6 (a)

10 0.9

0.92

0.94

0.96

0.98

1

−10

10 log(υsd )

−14

HKLE,T , µ = 4

−18 HKLE,W

HKLE,T , µ = 0.6 HKLE,P

−22

(b)

−26 0.9

0.92

0.94

0.96

0.98

1

αy

Fig. 9.1. Performance versus the forgetting factor αy in white Gaussian noise with an iSNR = 10 dB and L = 20.

decreases, but the speech-distortion index first decreases with αy and then increases. The optimal noise reduction performance (highest output SNR and lowest speech distortion) appears when αy is in the range between 0.985 and 0.995. This result is similar to what was observed with the time-domain filters in Chapter 7. It is also seen from Fig. 9.1 that the power subtraction method yielded the least SNR gain, but it also has the lowest speech distortion as compared to the Wiener filter and the tradeoff filter with µ = 4 and µ = 0.6. The performance of the tradeoff filter depends on the value of µ. When µ = 4, this method

9.3 Experiments

113

20

oSNR (dB)

18

HKLE,T , µ = 4

16

14 HKLE,P

12

HKLE,T , µ = 0.6 HKLE,W (a)

10 0

10

20

30

40

50

60

70

80

70

80

−10

HKLE,T , µ = 4

−14 10 log(υsd )

HKLE,W HKLE,T , µ = 0.6

−18 HKLE,P

−22

(b)

−26 0

10

20

30

40 L

50

60

Fig. 9.2. Performance versus filter length L in white Gaussian noise with an iSNR = 10 dB and αy = 0.985.

achieved higher output SNRs than the Wiener filter, but at the cost of higher speech distortion as shown in Fig. 9.1(b). When µ = 0.6, the tradeoff filter yielded less SNR improvement as compared to the Wiener. All these agreed very well with the theoretical analysis given in the previous sections. 9.3.2 Effect of Filter Length on Performance of Class-I Filters Another important parameter that affects the performance of all the Class-I filters is the filter length L. So, in this experiment, we study the impact of

114

9 Optimal Filters in the KLE Domain

the filter length (also the frame size) L on the performance of noise reduction. Again, white noise is used with an iSNR = 10 dB and the noise correlation matrix is directly computed from the noise signal using a long-term average. Based on the previous experiment, we set αy = 0.985. Figure 9.2 depicts the results. It is safe to say that the length L should be reasonably large enough to achieve good noise reduction performance. When L increases from 1 to 20, the output SNR improves while speech distortion decreases. But if we continue to increase L, there is either marginal additional SNR improvement (for the tradeoff method with µ = 4), or even SNR degradation (for the Wiener filter, the power subtraction, and the tradeoff filter with µ = 0.6), and there is also some increase in speech distortion. In general, good performance for all the studied algorithms is achieved when the filter length L is around 20. This result coincides with what was observed with the time-domain filters in Chapter 7. 9.3.3 Estimation of Clean Speech Correlation Matrix Previously, we have used the recursive approach to compute the noisy signal correlation matrix Ry . When noise is not stationary, we also used the recursive approach to estimate the correlation matrix Rv . We then assumed that at time k, the clean speech correlation matrix Rx can be calculated as Rx (k) = Ry (k) − Rv (k).

(9.71)

Substituting (7.44) and (7.45) into (9.71), one can readily check that Rx (k) is not in a recursive form (if αy = αv ) even though both Ry (k) and Rv (k) are computed recursively. Now we consider to use the recursive method to compute Rx , i.e., Rx (k) = αx Rx (k − 1) + (1 − αx )x(k)xT (k),

(9.72)

where αx is a forgetting factor. Putting (9.72) and (7.45) together and assuming that at any time instant k the relationship (9.71) should be satisfied, we have Ry (k) = αx Ry (k − 1) + (1 − αx )y(k)yT (k) + (αv − αx )Rv (k − 1) − (αv − αx )v(k)vT (k).

(9.73)

To examine the impact of αx on the Class-I filters, we repeated the experiment of Section 9.3.1. The only difference is that now the correlation matrix Ry (k) is computed according to (9.73) and the matrix Rv is also computed recursively using (7.45) with αv = 0.995. The results are plotted in Fig. 9.3. It is seen from Fig. 9.3 that the output SNR decreases monotonically as the forgetting factor αx increases; but it decreases slowly at first and then more rapidly. This result is different from what was observed from Fig. 9.1. In comparison, the speech-distortion index bears a nonmonotonic relationship with

9.3 Experiments

115

22 HKLE,T , µ = 4

20

oSNR (dB)

18

16

HKLE,P HKLE,T , µ = 0.6

14

HKLE,W

12 (a)

10 0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1

0.95

1

−10

HKLE,T , µ = 4

−14 10 log(υsd )

HKLE,W

−18

HKLE,T , µ = 0.6

−22

HKLE,P (b)

−26 0.6

0.65

0.7

0.75

0.8 αx

0.85

0.9

Fig. 9.3. Performance versus the forgetting factor αx in white Gaussian noise with an iSNR = 10 dB, αv = 0.995, and L = 20.

αx . It first decreases with αx and then increases. Again, it can be seen that the value of αx plays a very important role on the noise reduction performance of the studied filters. Comparing Figs. 9.3 and 9.1, one can see that the estimator in (9.73) produces a more consistent noise reduction performance (with respect to the value of the forgetting factor) than the recursive estimator given in (7.44). This is more true in the subband case where, if we need to estimate the correlation matrices, the forgetting factors are much smaller. We will come back to this point when we will discuss the Class-II filters.

116

9 Optimal Filters in the KLE Domain

9.3.4 Performance of Class-I Filters in Different Noise Conditions In this experiment, we test the performance of the Class-I filters in different input SNRs and noise conditions. We consider three types of noise (see Chapter 7): white Gaussian, car, and NYSE. Based on the previous experiment, we set L = 20 and compute the matrix Ry (k) according to (9.73) with αx = 0.985. Again, we do not use any noise estimator, but compute the noise correlation matrix directly from the noise signal using the recursive method with αv = 0.995 for all the three noise signals. The results of this experiment are shown in Fig. 9.4, where we only plotted the results of the Wiener filter and the tradeoff filter with µ = 4 to simplify the presentation. It is noticed from Fig. 9.4 that the output SNR is always larger than the input SNR, but the amount of SNR improvement depends on the input SNR and noise characteristics. Roughly, the lower the input SNR, the larger the SNR improvement. However, the speech-distortion index tends to increase as more SNR improvement is gained. This result coincides with what was observed with the time- and frequency-domain filters. If the input SNR is very low (e.g., below 0 dB), the noise reduction filters may even lead to speech quality degradation (instead of improving the speech quality) because of the significant speech distortion. Again, we suggest to use the grace-degradation technique in practical systems to better control the performance. It is also seen from Fig. 9.4 that both the Wiener and tradeoff filters are less effective in suppressing the NYSE noise as compared to the white noise and the car noise. The main reason for this is due to the fact that the NYSE noise is highly nonstationary. Another way to look at this problem is by examining the eigenvalue distribution of the correlation matrices of the different noise signals. We computed the correlation matrices (with L = 20) of the clean speech and all the three noise signals using the long-term average. The diagonalization process given in (2.16) was then applied to find the eigenvalues of all these matrices. As shown in Fig. 9.5, the white noise has constant eigenvalues. The car noise has only a couple of large eigenvalues and all the others tend to be close to 0. Comparatively, the NYSE noise has a similar eigenvalue distribution to the clean speech. These eigenvalue distribution patterns indicate that the NYSE noise is more difficult to detect making the noise suppression more challenging. 9.3.5 Impact of Forgetting Factor on Performance of Class-II Filters Unlike the Class-I filters where each frame may have a different transformation Q, the Class-II algorithms assume that all the frames share the same transformation Q (otherwise, filtering the KLE coefficients across different frames would not make much sense). In this situation, the estimation of Q is relatively easier than that for the Class-I filters. We can simply use a long-term sample average to compute the correlation matrices Ry and Rv , thereby obtaining

9.3 Experiments 30

HKLE,T , µ = 4, WGN

25

oSNR (dB)

117

HKLE,W , WGN

20

HKLE,T , µ = 4, Car

15

HKLE,W , Car

10 5 HKLE,W , NYSE

0

HKLE,T , µ = 4, NYSE

−5

(a)

−10 −10

−5

0

0

5

10

15

20

25

HKLE,T , µ = 4, NYSE HKLE,W , NYSE

−5

HKLE,T , µ = 4, Car

10 log(υsd )

−10

HKLE,W , Car

−15 −20 −25

HKLE,T , µ = 4, WGN HKLE,W , WGN

−30 −35

(b)

−40 −10

−5

0

5 10 iSNR (dB)

15

20

25

Fig. 9.4. Performance as a function of the input SNR with white Gaussian noise (WGN), car noise, and NYSE babbling noise.

an estimate of Rx . The KLT matrix Q can then be computed using the eigenvalue decomposition. In our study, we found that the estimation accuracy of the matrix Q plays a less important role in noise reduction performance of the Class-II methods than it does in performance of the Class-I filters. We can even replace the matrix Q by the Fourier matrix without degrading the noise reduction performance, which indicates that the idea of the Class-II filters can also be used in the frequency-domain approaches. However, following the theoretical development in Section 9.2, we still use the transformation matrix Q in our experiments, with the correlation matrices Ry and Rv being estimated

118

9 Optimal Filters in the KLE Domain 0.05 Car

0.04 NYSE Value

0.03 Speech

0.02 WGN

0.01

0 0

5

10 l

15

20

Fig. 9.5. Eigenvalues of the clean speech and different noise signals with L = 20.

using a long-term average and the matrix Rx being computed as Ry − Rv . The matrix Q is then applied to each frame of the signals to compute the KLE coefficients cy,l and cv,l . In order to estimate Rcv ,l , we need to have an estimate of the noise signal v(k). Although we have developed a noise detector in Chapter 8, we compute the noise statistics directly from the noise signal in this experiment to avoid the influence of the noise estimation error on the parameter optimization. Specifically, the KLT Q is applied to the noise signal v(k) to obtain the KLE coefficients cv,l (k). The matrix Rcv ,l is then computed using the recursive method Rcv ,l (k) = αcv ,l Rcv ,l (k − 1) + (1 − αcv ,l )cv,l (k)cTv,l (k),

(9.74)

where αcv ,l , same as αv and αy , is a forgetting factor. In the previous analysis, we have pointed out that when the forgetting factors are small, which is the case in the Class-II filters, it is better to assume that the speech correlation matrix is in a recursive form. So, we assume to have Rcx ,l (k) = αcx ,l Rcx ,l (k − 1) + (1 − αcx ,l )cx,l (k)cTx,l (k).

(9.75)

In this case, at time k, Rcy ,l (k) should be computed according to Rcy ,l (k) = αcx ,l Rcy ,l (k − 1) + (1 − αcx ,l )cy,l (k)cTy,l (k) + (αcv ,l − αcx ,l )Rcv ,l (k − 1) − (αcv ,l − αcx ,l )cv,l (k)cTv,l (k).

(9.76)

9.3 Experiments

119

22

20

oSNR (dB)

18

16

hT,l , µ = 0.6

14

hW,l

12

hT,l , µ = 4 (a)

10 0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0.7

0.8

0.9

1

−10 hT,l , µ = 4

10 log(υsd )

−15

−20 hW,l

−25

hT,l , µ = 0.6

(b)

−30 0.2

0.3

0.4

0.5

0.6 αcx,l

Fig. 9.6. Performance as a function of the forgetting factor αcx ,l in white Gaussian noise (WGN).

The forgetting factors αcx ,l and αcv ,l (l = 1, . . . , L) play an important role in noise reduction performance of the Class-II filters. In principle, each subband l may take a different forgetting factor. But for simplicity, in this study, we assume the same forgetting factor for all the subbands, i.e., αcx ,1 = αcx ,2 = · · · = αcx ,L = αcx , and αcv ,1 = αcv ,2 = · · · = αcv ,L = αcv . Again, white Gaussian noise is used. We set Ll to 3 and αcv to 0.91 (this value of αcv was obtained trough experiments) and examine the noise reduction performance for different values of αcx . The result of this experiment is plotted in Fig. 9.6. It is seen that, for all the three configured algorithms, the output

120

9 Optimal Filters in the KLE Domain 30 hT,l , µ = 4

oSNR (dB)

25

20

hW,l

15

hT,l , µ = 2 (a)

10 0

5

10

15

20

25

30

35

40

30

35

40

−10

hT,l , µ = 4

−12

10 log(υsd )

hT,l , µ = 2

−14

hW,l

−16

−18 (b)

−20 0

5

10

15

20 Ll

25

Fig. 9.7. Performance as a function of the filter length Ll in white Gaussian noise (WGN) with an iSNR = 10 dB, αay = 0.8, αav = 0.91, and L = 20.

SNR monotonically decreases (slowly when αcx < 0.9 and then rapidly) as αcx increases. In comparison, the speech-distortion index first decreases as αcx increases and then increases. Taking into account both SNR improvement and speech distortion, one can see that good noise reduction performance is obtained with αcx being in the range between 0.7 and 0.9. 9.3.6 Effect of Filter Length on Performance of Class-II Filters In the next experiment, we study the impact of the filter length Ll on the noise reduction performance. Again, the background noise is white and no

9.4 Summary

121

noise estimator is used. The parameters used in this experiment are αcx = 0.8, αcv = 0.91, iSNR = 10 dB, and L = 20. The results are depicted in Fig. 9.7. It is seen from Fig. 9.7(a) that the output SNRs for the tradeoff filters with µ = 4 and µ = 2 increase with Ll . For the Wiener filter, however, the output SNR increases first to its maximum and then decreases. In comparison, the speech-distortion indices with both methods decrease first and then increase with Ll . Taking into account both SNR improvement and speech distortion, we would suggest to use Ll between 5 and 10. Comparing Figs. 9.7 and 9.2, one can see that, with the same L, the optimal filters in Class II can achieve much higher SNR gain than the filters of Class I. The Class-II filters also have slightly more speech distortion. But the additional amount of distortion compared to that of the Class-I filters is not significant. This indicates that the Class-II filters may have a great potential in practice.

9.4 Summary In this chapter, we discussed two broad classes of optimal noise reduction filters in the KLE domain. While the filters of Class I achieve a frame of speech estimate by filtering only the corresponding frame of the noisy speech, the filters of Class II are inter-frame techniques, which obtain noise reduction by filtering not only the current frame, but also a number of previous consecutive frames of the noisy speech. We have also discussed some properties and implementation issues of these filters. Through experiments, we have investigated the optimal values of the forgetting factors and the length of the optimal filters. We also demonstrated that better noise reduction performance can be achieved with the filters of Class II when the parameters associated with this class are properly chosen, which demonstrated the great potential of the filters in this category for noise reduction. It should be pointed out that the underlying idea of the filters of Class II can also be used in the frequency domain, which will be left to the reader’s investigation.

10 Optimal Filters in the Transform Domain

In this chapter, we reformulate the noise reduction problem into a more generalized transform domain. There are at least two advantages doing this: first, different transforms can be used to replace each other without any requirement to change the algorithm formulation (optimal or suboptimal filter) and second, it is easier to fairly compare different transforms for their noise reduction performance. To do so, we need to generalize the KLE concept. Therefore, we recommend the reader to be familiar with the KLE (explained in Chapter 2) before reading this part.

10.1 Generalization of the KLE In this section, we are going to generalize the principle of the KLE to any given unitary transform U. For that, we need to use some of the concepts presented in [12], [106], [107], [129]. The basic idea behind this generalization is to find other ways to exactly diagonalize the correlation matrix Ry 1 . The Fourier matrix, for example, diagonalizes approximately Ry (since this matrix is Toeplitz and its elements are usually absolutely summable [60]). However, this approximation may cause more distortion to the clean speech when noise reduction is performed in the frequency domain. We define the square root of the positive definite matrix Ry as T R1/2 = Qy Λ1/2 y y Qy ,

(10.1)

where the matrices Qy and Λy are, respectively, orthogonal and diagonal matrices. Expression (10.1) is very useful in the derivation of a generalized form of the KLE. 1

In Chapters 2 and 9, we diagonalized the correlation matrix Rx . In this part, it is far better to diagonalize Ry instead of Rx for numerical reasons (Ry is much better conditioned than Rx ) since all optimal filters depend on the inverse of the correlation matrix that we choose to diagonalize.

J. Benesty et al., Noise Reduction in Speech Processing, Springer Topics in Signal Processing 2, DOI 10.1007/978-3-642-00296-0_10, © Springer-Verlag Berlin Heidelberg 2009

124

10 Optimal Filters in the Transform Domain

Consider the L × L unitary matrix
 U = u1 u2 · · · uL , where UH U = UUH = I and superscript H denotes transpose conjugate of a vector or a matrix. We would like to minimize the positive quantity gH (ul )R1/2 y g(ul ) subject to the constraint gH (ul )ul = uH l g(ul ) = 1.

(10.2)

Under this constraint, the process y(k) is passed through the filter
T g(ul ) = g0 (ul ) g1 (ul ) · · · gL−1 (ul ) with no distortion along ul and signals along vectors other than ul tend to be attenuated. Mathematically, this is equivalent to minimizing the following cost function:
 H JS [g(ul )] = gH (ul )R1/2 (10.3) y g(ul ) + µS 1 − g (ul )ul , where µS is a Lagrange multiplier. The minimization of (10.3) leads to the following solution: g(ul ) =

R−1/2 ul y −1/2 uH ul l Ry

.

(10.4)

We define the spectrum of y(k) along ul as φy (ul ) = gH (ul )Ry g(ul ).

(10.5)

Substituting (10.4) into (10.5) gives φy (ul ) =

1 −1/2 uH ul l Ry

2 .

(10.6)

Expression (10.6) is a general definition of the spectrum of the signal y(k), which depends on the unitary matrix U. Using (10.4) and (10.6), we get 1/2 R1/2 y g(ul ) = φy (ul )ul .

(10.7)

By taking into account all vectors ul , l = 1, 2, . . . , L, (10.7) can be written into the following general form 1/2 R1/2 y G(U) = UΦy (U),

where

and


 G(U) = g(u1 ) g(u2 ) · · · g(uL )
 Φy (U) = diag φy (u1 ) φy (u2 ) · · · φy (uL )

is a diagonal matrix.

(10.8)

10.1 Generalization of the KLE

125

Property 10.1. The correlation matrix Ry can be diagonalized as follows: GH (U)Ry G(U) = Φy (U).

(10.9)

Proof. This form follows immediately from (10.8). Property 10.1 shows that there are an infinite number of ways to diagonalize the matrix Ry , depending on how we choose the unitary matrix U. Each one of these diagonalizations gives a representation of the spectrum of the signal y(k) in the subspace U. Expression (10.9) is a generalization of the KLT; the only major difference is that G(U) is not a unitary matrix except for the case where U = Qy . For this special case, it’s easy to verify that G(Qy ) = Qy and Φy (Q) = Λy , which is the KLT formulation. Property 10.2. The vector y(k) can be written as a combination (expansion) −1/2 of the vectors of the matrix U = R1/2 (U) as follows y UΦy y(k) =

L 

cy (k, ul )

l=1

=

L 

R1/2 y 1/2

φy (ul )

ul

cy (k, ul ) ul ,

(10.10)

l=1

where cy (k, ul ) = gH (ul )y(k), l = 1, 2, . . . , L

(10.11)

are the coefficients of the expansion and ul =

R1/2 y 1/2

φy (ul )

ul , l = 1, 2, . . . , L.

(10.12)

Expressions (10.10) and (10.11) are the time- and transform-domain representations of the vector signal y(k). Proof. Expressions (10.10) and (10.11) can be shown by substituting one into the other. Property 10.3. We always have E [cy (k, ul )] = 0, l = 1, 2, . . . , L and
 E cy (k, ui ) c∗y (k, uj ) =



φy (ui ), i = j . 0, i = j

Proof. These properties can be verified from (10.11).

(10.13)

(10.14)

126

10 Optimal Filters in the Transform Domain

It can be checked that the Parseval’s theorem does not hold anymore if U = Qy . This is due to the fact that the matrix G(U) is not unitary. Indeed L 

L
  2 E |cy (k, ul ) | = φy (ul )

l=1

l=1

  = tr Ry G(U)GH (U) = tr (Ry ) .

(10.15)

This is the main difference between the KLT and the generalization proposed here for U = Q. This difference, however, should have no impact on most applications and Properties 10.1, 10.2, and 10.3 are certainly the most important ones. We define the spectra of the clean speech, x(k), and noise, v(k), in the subspace U as φx (ul ) = gH (ul )Rx g(ul ), l = 1, 2, . . . , L, H

φv (ul ) = g (ul )Rv g(ul ), l = 1, 2, . . . , L.

(10.16) (10.17)

Of course, φx (ul ) and φv (ul ) are always positive real numbers. We can now apply the three previous properties to our noise reduction problem. Indeed, with the help of Property 10.2 and substituting (2.2) into (10.11), we get cy (k, ul ) = gH (ul )y(k) = gH (ul )x(k) + gH (ul )v(k) = cx (k, ul ) + cv (k, ul ) , l = 1, 2, . . . , L. We also have from Property 10.3, E




cy (k, ui ) c∗y

 (k, uj ) =



φx (ui ) + φv (ui ), i = j . 0, i = j

(10.18)

(10.19)

Expression (10.18) is equivalent to (2.2) but in the transform domain. Similar to the KLE case, our problem becomes one of finding an estimate of cx (k, ul ) by multiplying cy (k, ul ) with a (complex) scalar hl , i.e., cz (k, ul ) = hl cy (k, ul ) = hl [cx (k, ul ) + cv (k, ul )] = cx,F (k, ul ) + cv,F (k, ul ) , l = 1, 2, . . . , L. The variance of the signal cz (k, ul ) is
 E |cz (k, ul ) |2 = |hl |2 φy (ul ), l = 1, 2, . . . , L.

(10.20)

(10.21)

Finally by using Property 10.2, we can go back into the time domain and see that an estimate of the vector x(k) would be

10.2 Performance Measures

z(k) =

L 

127

cz (k, ul ) ul

l=1

= R1/2 y

"

L 

# hl ul uH l

R−1/2 y(k) y

l=1

= H(U) [x(k) + v(k)] ,

(10.22)

where
 H −1/2 H(U) = R1/2 y U diag h1 h2 · · · hL U Ry

(10.23)

is an L × L (time-domain) filtering matrix, which depends on the unitary ma
T trix U and is equivalent to the transform-domain filter h = h1 h2 · · · hL . Moreover, it can be checked, with
 the help of Property 10.1, that the correlation matrix Rz = E z(k)zH (k) can be diagonalized as follows:
 GH (U)Rz G(U) = diag |h1 |2 φy (u1 ) |h2 |2 φy (u2 ) · · · |hL |2 φy (uL ) . (10.24) We see from the previous expression how the coefficients hl , l = 1, 2, . . . , L, affect the spectrum of the estimated signal z(k) in the subspace U, depending on how they are optimized.

10.2 Performance Measures Like we did it in the time, frequency, and KLE domains (see Chapter 3), we are going to define the most important measures for noise reduction but in this generalized transform-domain context. 10.2.1 SNR With the proposed transform-domain representation, we define the subband and fullband input SNRs, respectively, as
 E |cx (k, ul ) |2 iSNR(ul ) = E [|cv (k, ul ) |2 ] φx (ul ) = , l = 1, 2, . . . , L, (10.25) φv (ul ) L φx (ul ) iSNR(U) = l=1 . (10.26) L l=1 φv (ul ) In general, iSNR(U) = iSNR. But for U = Qy , iSNR(Qy ) = iSNR. After noise reduction with the model given in (10.22), the output SNR is

128

10 Optimal Filters in the Transform Domain

  tr H(U)Rx HH (U) . oSNR [H(U)] =  tr H(U)Rv HH (U)

(10.27)

With the transform-domain model shown in (10.20) and after noise reduction, the subband output SNR is |hl |2 φx (ul ) |hl |2 φv (ul ) = iSNR(ul ), l = 1, 2, . . . , L

oSNR(ul ) =

(10.28)

and the fullband output SNR is L

oSNR(h, U) = l=1 L

l=1

|hl |2 φx (ul ) |hl |2 φv (ul )

.

(10.29)

In general, oSNR(h, U) = oSNR [H(U)]. But
 in the special case where U = Qy , we have oSNR(h, Qy ) = oSNR H(Qy ) . It is important to observe that the subband output SNR is equal to the subband input SNR; therefore these SNRs are not influenced by the coefficients hl , l = 1, 2, . . . , L. Property 10.4. We always have L 

iSNR(ul ) ≥ iSNR(U),

(10.30)

oSNR(ul ) ≥ oSNR(h, U).

(10.31)

l=1 L  l=1

This means that the aggregation of the subband (input or output) SNRs is greater than or equal to the fullband (input or output) SNR. Proof. The two previous inequalities can be easily shown by using the inequality (3.16). 10.2.2 Noise-Reduction Factor We define the noise-reduction factor for the model in (10.22) as ξnr [H(U)] =

tr (Rv )  . tr H(U)Rv HH (U)

(10.32)

The larger the value of ξnr [H(U)], the more the noise is reduced. After the filtering operation, the residual noise level is expected to be lower than that of the original noise level, therefore this factor should be lower bounded by 1. The subband noise-reduction factor is

10.2 Performance Measures

ξnr (hl ) =

1 , l = 1, 2, . . . , L |hl |2

129

(10.33)

and the corresponding fullband noise-reduction factor is L ξnr (h, U) = L

l=1

l=1

φv (ul )

|hl |2 φv (ul )

.

(10.34)

In general, ξnr (h, U) = ξnr [H(U)]. But for U = Qy , ξnr (h, Qy ) =
 ξnr H(Qy ) . Property 10.5. We always have L 

ξnr (hl ) ≥ ξnr (h, U).

(10.35)

l=1

This means that the aggregation of the subband noise-reduction factors is greater than or equal to the fullband noise-reduction factor. Proof. Easy to show by using the inequality (3.16). 10.2.3 Speech-Distortion Index The filtering operation adds distortion to the speech signal; so a measure is needed to quantify the amount of speech distortion. With the model given in (10.22), we define the speech-distortion index as   H E [x(k) − H(U)x(k)] [x(k) − H(U)x(k)] υsd [H(U)] = . (10.36) tr (Rx ) This index is lower bounded by 0 and expected to be upper bounded by 1 for optimal filters. The higher the value of υsd [H(U)], the more the speech is distorted. In the transform domain with the formulation given in (10.20), we define the subband and fullband speech-distortion indices, respectively, as υsd (hl ) = |1 − hl |2 , l = 1, 2, . . . , L

(10.37)

and L υsd (h, U) =

|1 − hl |2 φx (ul ) . L l=1 φx (ul )

l=1

(10.38)

In general, υsd (h, U) = υsd [H(U)]. But for the special case of U = Qy ,
 υsd (h, Qy ) = υsd H(Qy ) .

130

10 Optimal Filters in the Transform Domain

Property 10.6. We always have L 

υsd (hl ) ≥ υsd (h, U) .

(10.39)

l=1

This means that the aggregation of the subband speech-distortion indices is greater than or equal to the fullband speech-distortion index. Proof. Easy to show by using the inequality (3.16). 10.2.4 Speech-Reduction Factor This factor measures the amount of speech reduced by the filtering operation. We define the speech-reduction factor for the model in (10.22) as ξsr [H(U)] =

tr (Rx )  . tr H(U)Rx HH (U)

(10.40)

The larger the value of ξsr [H(U)], the more the desired speech is reduced. After the filtering operation, the residual speech level is expected to be lower than that of the original speech level, therefore this factor should be lower bounded by 1. The subband speech-reduction factor is ξsr (hl ) =

1 , l = 1, 2, . . . , L |hl |2

and the corresponding fullband speech-reduction factor is L φx (ul ) ξsr (h, U) = L l=1 . 2 l=1 |hl | φx (ul )

(10.41)

(10.42)


 In general, ξsr (h, U) = ξsr [H(U)]. But for U = Qy , ξsr (h, Qy ) = ξsr H(Qy ) . Property 10.7. We always have L 

ξsr (hl ) ≥ ξsr (h, U).

(10.43)

l=1

This means that the aggregation of the subband speech-reduction factors is greater than or equal to the fullband speech-reduction factor. Proof. Easy to show by using the inequality (3.16). Property 10.8. We always have oSNR [H(U)] ξnr [H(U)] = , iSNR ξsr [H(U)] oSNR (h, U) ξnr (h, U) = . iSNR(U) ξsr (h, U)

(10.44) (10.45)

Proof. Easy to see by using the different definitions of the involved measures.

10.3 MSE Criterion

131

10.3 MSE Criterion Let us define the transform-domain error signal between the estimated and desired signals in the subband l: e (k, ul ) = cz (k, ul ) − cx (k, ul ) = hl cy (k, ul ) − cx (k, ul ) , l = 1, 2, . . . , L.

(10.46)

This error can also be written as the sum of two error signals: e (k, ul ) = ex (k, ul ) + ev (k, ul ) ,

(10.47)

ex (k, ul ) = (hl − 1)cx (k, ul )

(10.48)

where

is the speech distortion due to the complex gain, and ev (k, ul ) = hl cv (k, ul )

(10.49)

represents the residual noise. From the error signal (10.46), we give the corresponding transform-domain MSE criterion in the subband l:
 J (hl , ul ) = E |e (k, ul ) |2 (10.50) 2

= φx (ul ) + |hl | φy (ul ) − 2R [hl φyx (ul )] 2

= φx (ul ) + |hl | φy (ul ) − 2R [hl φx (ul )] , where φyx (ul ) = E [cy (k, ul ) c∗x (k, ul )] = φx (ul ) is the cross-spectrum between the observation and speech signals. Expression (10.50) can be structured in a different way:

 J (hl , ul ) = E |ex (k, ul ) |2 + E |ev (k, ul ) |2 (10.51) = Jx (hl , ul ) + Jv (hl , ul ) . For the particular gain hl = 1, ∀l, we get J (1, ul ) = φv (ul ),

(10.52)

so there will be neither noise reduction nor speech distortion. Therefore, we define the transform-domain NMSE in the subband l as

132

10 Optimal Filters in the Transform Domain

J (hl , ul ) J˜ (hl , ul ) = J (1, ul ) = iSNR(ul ) · υsd (hl ) +

1 , ξnr (hl )

(10.53)

where Jx (hl , ul ) , φx (ul ) φv (ul ) ξnr (hl ) = . Jv (hl , ul )

υsd (hl ) =

(10.54) (10.55)

The transform-domain NMSE depends explicitly on the subband input SNR, the subband speech-distortion index, and the subband noise-reduction factor. We define the fullband MSE and fullband NMSE as L

J (h, U) =

1 J (hl , ul ) L

(10.56)

l=1

=

L

L

l=1

l=1

1 1 |hl − 1|2 φx (ul ) + |hl |2 φv (ul ) L L

= Jx (h, U) + Jv (h, U) and J (h, U) J˜ (h, U) = L L l=1 φv (ul ) L L 2 2 l=1 |hl − 1| φx (ul ) l=1 |hl | φv (ul ) = +  L L l=1 φv (ul ) l=1 φv (ul ) 1 = iSNR(U) · υsd (h, U) + , ξnr (h, U)

(10.57)

where Jx (h, U) υsd (h, U) = L , l=1 φx (ul ) L φv (ul ) ξnr (h, U) = l=1 . Jv (h, U)

(10.58) (10.59)

The fullband NMSE depends explicitly on the fullband input SNR, the fullband speech-distortion index, and the fullband noise-reduction factor.

10.4 PCC and Fundamental Properties Let ca (k, ul ) and cb (k, ul ) be two signals in the transform domain. We define the subband SPCC or the MSCF between ca (k, ul ) and cb (k, ul ) in the subband l as

10.4 PCC and Fundamental Properties

|E [ca (k, ul ) c∗b (k, ul )]| E [|ca (k, ul ) |2 ] E [|cb (k, ul ) |2 ] |φab (ul )|2 = . φa (ul )φb (ul )

133

2

|ρ [ca (k, ul ) , cb (k, ul )] |2 =

(10.60)

Now, let us define the two vectors of length L:
T ca (k, U) = ca (k, u1 ) ca (k, u2 ) · · · ca (k, uL ) ,
T cb (k, U) = cb (k, u1 ) cb (k, u2 ) · · · cb (k, uL ) . We define the fullband SPCC as 
T  E c (k, U) c∗ (k, U) 2 a b |ρ [ca (k, U) , cb (k, U)] | = E [cTa (k, U) c∗a (k, U)] E [cTa (k, U) c∗a (k, U)] L | l=1 φab (ul )|2 = L . (10.61) L l=1 φa (ul ) · l=1 φb (ul ) 2

It is clear that 0 ≤ |ρ [ca (k, ul ) , cb (k, ul )] |2 ≤ 1, 0 ≤ |ρ [ca (k, U) , cb (k, U)] |2 ≤ 1.

(10.62) (10.63)

The MSCF between the two signals cx (k, ul ) and cy (k, ul ) in the subband l is φx (ul ) φx (ul ) + φv (ul ) iSNR(ul ) = . 1 + iSNR(ul )

|ρ [cx (k, ul ) , cy (k, ul )] |2 =

(10.64)

This MSCF tells us how much the observation signal is noisy in the subband l. A value of |ρ [cx (k, ul ) , cy (k, ul )] |2 close to 1 implies that the speech is largely dominant while a value of |ρ [cx (k, ul ) , cy (k, ul )] |2 close to 0 implies that the noise is largely dominant in the subband l. The fullband SPCC between the two signals cx (k, U) and cy (k, U) is L

φx (ul ) L l=1 φx (ul ) + l=1 φv (ul ) iSNR(U) = . 1 + iSNR(U)

|ρ [cx (k, U) , cy (k, U)] |2 = L

l=1

(10.65)

The subband SPCC between the two signals cx (k, ul ) and cz (k, ul ) in the subband l is |ρ [cx (k, ul ) , cz (k, ul )] |2 = |ρ [cx (k, ul ) , cy (k, ul )] |2 .

(10.66)

134

10 Optimal Filters in the Transform Domain

The subband SPCC between cx (k, ul ) and cz (k, ul ) is equal to the subband SPCC between cx (k, ul ) and cy (k, ul ) and does not depend on hl ; the same way the subband input SNR is equal to the subband output SNR and does not depend on hl . The fullband SPCC between the two signals cx (k, U) and cz (k, U) is  2  L   l=1 hl φx (ul ) 2    (10.67) |ρ [cx (k, U) , cz (k, U)] | =  L L 2 l=1 φx (ul ) l=1 |hl | φy (ul )  2  L   l=1 hl φx (ul )   × =  L L 2 l=1 φx (ul ) l=1 |hl | φx (ul ) oSNR(h, U) . 1 + oSNR(h, U) Property 10.9. We have |ρ [cx (k, U) , cz (k, U)] |2 = |ρ [cx (k, U) , cx,F (k, U)] |2 × |ρ [cx,F (k, U) , cz (k, U)] |2 , (10.68) where  2  L   l=1 hl φx (ul ) 2    (10.69) |ρ [cx (k, U) , cx,F (k, U)] | =  L L 2 l=1 φx (ul ) l=1 |hl | φx (ul ) and |ρ [cx,F (k, U) , cz (k, U)] |2 =

oSNR(h, U) . 1 + oSNR(h, U)

(10.70)

The fullband SPCC |ρ [cx (k, U) , cx,F (k, U)] |2 is a speech-distortion index. If hl = 1, ∀l (no speech distortion) then |ρ [cx (k, U) , cx,F (k, U)] |2 = 1. The closer the value of |ρ [cx (k, U) , cx,F (k, U)] |2 is to 0, the more the speech signal is distorted. The fullband SPCC |ρ [cx,F (k, U) , cz (k, U)] |2 shows the SNR improvement and reaches its maximum when oSNR(h, U) is maximized. The minimization of |ρ [cx (k, U) , cz (k, U)] |−2 leads to an optimal filter. Property 10.10. We have |ρ [cx (k, U) , cz (k, U)] |2 ≤

oSNR(h, U) , 1 + oSNR(h, U)

(10.71)

with equality when hl = 1, ∀l. Proof. This property follows |ρ [cx (k, U) , cx,F (k, U)] |2 ≤ 1.

immediately

from

(10.68)

since

10.4 PCC and Fundamental Properties

135

Property 10.11. We have |ρ [cx,F (k, U) , cy (k, U)] |2 = |ρ [cx (k, U) , cx,F (k, U)] |2 × |ρ [cx (k, U) , cy (k, U)] |2 . (10.72) Proof. Indeed  2  L   l=1 hl φx (ul ) 2     |ρ [cx,F (k, U) , cy (k, U)] | =  L L 2 |h | φ (u ) φ (u ) l x l l l=1 l=1 y  2  L   l=1 hl φx (ul )   × =  L 2 L l=1 |hl | φx (ul ) l=1 φx (ul ) iSNR(U) 1 + iSNR(U) = |ρ [cx (k, U) , cx,F (k, U)] |2 × |ρ [cx (k, U) , cy (k, U)] |2 . Property 10.12. We have |ρ [cx,F (k, U) , cy (k, U)] |2 ≤

iSNR(U) , 1 + iSNR(U)

(10.73)

with equality when hl = 1, ∀l. Proof. This property follows |ρ [cx (k, U) , cx,F (k, U)] |2 ≤ 1.

immediately

from

(10.72)

since

The MSCF between cv (k, ul ) and cy (k, ul ) is another way to see how much the observation signal is affected by the noise in the subband l. This MSCF is φv (ul ) φy (ul ) 1 = . 1 + iSNR(ul )

|ρ [cv (k, ul ) , cy (k, ul )] |2 =

(10.74)

Property 10.13. We have |ρ [cx (k, ul ) , cy (k, ul )] |2 + |ρ [cv (k, ul ) , cy (k, ul )] |2 = 1

(10.75)

and iSNR(ul ) =

|ρ [cx (k, ul ) , cy (k, ul )] |2 . |ρ [cv (k, ul ) , cy (k, ul )] |2

(10.76)

136

10 Optimal Filters in the Transform Domain

Proof. Easy to see from (10.64) and (10.74). The subband input SNR in the subband l is the ratio of two MSCFs and the sum of these two MSCFs never exceeds 1. The fullband SPCC between cv (k, U) and cy (k, U) is |ρ [cv (k, U) , cy (k, U)] |2 =

1 . 1 + iSNR(U)

(10.77)

The subband SPCC between the two signals cv (k, ul ) and cz (k, ul ) in the subband l is |ρ [cv (k, ul ) , cz (k, ul )] |2 = |ρ [cv (k, ul ) , cy (k, ul )] |2 .

(10.78)

The fullband SPCC between the same signals is  2  L   l=1 hl φv (ul ) 2    (10.79) |ρ [cv (k, U) , cz (k, U)] | =  L L 2 φ (u ) |h | φ (u ) v l l y l l=1 l=1  2  L   l=1 hl φv (ul )   × =  L L 2 φ (u ) |h | φ (u ) v l l v l l=1 l=1 1 . 1 + oSNR(h, U) Property 10.14. We have |ρ [cv (k, U) , cz (k, U)] |2 = |ρ [cv (k, U) , cv,F (k, U)] |2 × |ρ [cv,F (k, U) , cz (k, U)] |2 , (10.80) where  2  L   l=1 hl φv (ul ) 2    (10.81) |ρ [cv (k, U) , cv,F (k, U)] | =  L L 2 l=1 φv (ul ) l=1 |hl | φv (ul ) and |ρ [cv,F (k, U) , cz (k, U)] |2 =

1 . 1 + oSNR(h, U)

(10.82)

Property 10.15. We have |ρ [cv (k, U) , cz (k, U)] |2 ≤ with equality when hl = 1, ∀l.

1 , 1 + oSNR(h, U)

(10.83)

10.4 PCC and Fundamental Properties

Proof. This property follows |ρ [cv (k, U) , cv,F (k, U)] |2 ≤ 1.

immediately

from

137

(10.80)

since

Property 10.16. We have |ρ [cv,F (k, U) , cy (k, U)] |2 = |ρ [cv (k, U) , cv,F (k, U)] |2 × |ρ [cv (k, U) , cy (k, U)] |2 . (10.84) Proof. Indeed  2  L   l=1 hl φv (ul ) 2    |ρ [cv,F (k, U) , cy (k, U)] | =  L 2 L |h | φ (u ) φ (u ) l v l y l l=1 l=1  2  L   l=1 hl φv (ul )    × =  L L 2 |h | φ (u ) φ (u ) l v l v l l=1 l=1 1 1 + iSNR(U) = |ρ [cv (k, U) , cv,F (k, U)] |2 × |ρ [cv (k, U) , cy (k, U)] |2 . Property 10.17. We have |ρ [cv,F (k, U) , cy (k, U)] |2 ≤

1 , 1 + iSNR(U)

(10.85)

with equality when hl = 1, ∀l. Proof. This property follows |ρ [cv (k, U) , cv,F (k, U)] |2 ≤ 1.

immediately

from

(10.84)

since

Property 10.18. We have |ρ [cx (k, U) , cy (k, U)] |2 + |ρ [cv (k, U) , cy (k, U)] |2 = 1

(10.86)

and iSNR(U) =

|ρ [cx (k, U) , cy (k, U)] |2 . |ρ [cv (k, U) , cy (k, U)] |2

(10.87)

Proof. Easy to see from (10.65) and (10.77). Property 10.19. We have |ρ [cx,F (k, U) , cz (k, U)] |2 + |ρ [cv,F (k, U) , cz (k, U)] |2 = 1

(10.88)

and oSNR(h, U) =

|ρ [cx,F (k, U) , cz (k, U)] |2 . |ρ [cv,F (k, U) , cz (k, U)] |2

(10.89)

138

10 Optimal Filters in the Transform Domain

Proof. Easy to see from (10.70) and (10.82). The fullband input or output SNR for the transform-domain approach is the ratio of two fullband SPCCs and the sum of these two SPCCs never exceeds 1. It can be checked that oSNR(h, U) > iSNR(U) is equivalent to |ρ [cx,F (k, U) , cz (k, U)] |2 > ρ2 [cx (k, U) , cy (k, U)]. In this situation, |ρ [cv,F (k, U) , cz (k, U)] |2 < ρ2 [cv (k, U) , cy (k, U)].

10.5 Examples of Filter Design In this section, we develop some important optimal and suboptimal filters in the transform domain. 10.5.1 Wiener Filter Taking the gradient of J (hl , ul ) with respect to h∗l and equating the result to 0 lead to   −E c∗y (k, ul ) [cx (k, ul ) − hW,l cy (k, ul )] = 0. (10.90) Hence φy (ul )hW,l = φxy (ul ). But

(10.91)


 φxy (ul ) = E cx (k, ul )c∗y (k, ul ) = φx (ul ).

Therefore the optimal filter can be put into the following forms: φx (ul ) φy (ul ) φv (ul ) = 1− φy (ul ) iSNR(ul ) = . 1 + iSNR(ul )

hW,l =

(10.92)

We see that the transform-domain Wiener filter is always real and positive and its form is similar to that of the frequency-domain Wiener filter (see Chapter 8). Another way to define the Wiener filter is with the MSCFs. Indeed, it is easy to verify that hW,l = |ρ [cx (k, ul ), cy (k, ul )] |2 = 1 − |ρ [cv (k, ul ), cy (k, ul )] |2 .

(10.93)

Now, let us define the complex number2 2

Notice that both ρ [cx (k, ul ), cy (k, ul )] and ρ [cv (k, ul ), cy (k, ul )] are real numbers.

10.5 Examples of Filter Design

139

[cx (k, ul ), cv (k, ul )] = ρ [cx (k, ul ), cy (k, ul )] + jρ [cv (k, ul ), cy (k, ul )] = cos θ(ul ) + j sin θ(ul ),

(10.94)

where θ(ul ) is the angle of [cx (k, ul ), cv (k, ul )] for which the modulus is equal to 1. On the complex plane, [cx (k, ul ), cv (k, ul )] is on the unit circle. Since 0 ≤ ρ [cx (k, ul ), cy (k, ul )] ≤ 1 and 0 ≤ ρ [cv (k, ul ), cy (k, ul )] ≤ 1, therefore 0 ≤ θ(ul ) ≤ π2 . We can then rewrite the Wiener filter as a function of the angle θ(ul ): hW,l = cos2 θ(ul ) = 1 − sin2 θ(ul ).

(10.95)

Hence lim hW,l = 1,

(10.96)

lim

(10.97)

θ(ul )→0 θ(ul )→ π 2

hW,l = 0.

We deduce the subband noise-reduction factor and subband speechdistortion index ξnr (hW,l ) =

1 ≥ 1, cos4 θ(ul )

υsd (hW,l ) = sin4 θ(ul ) ≤ 1,

(10.98) (10.99)

and the subband MNMSE J˜ (hW,l , ul ) = hW,l ,

(10.100)

which is exactly the transform-domain Wiener filter. We see clearly how noise reduction and speech distortion depend on the angle θ(ul ) in the Wiener filter. When θ(ul ) increases so does ξnr (hW,l ); at the same time υsd (hW,l ) increases. Property 10.20. With the optimal transform-domain Wiener filter given in (10.92), the fullband output SNR is always greater than or equal to the fullband input SNR, i.e., oSNR(hW , U) ≥ iSNR(U). Proof. Let us evaluate the fullband SPCC between cy (k, U) and cz,W (k, U) = T [hW,1 cy (k, u1 ) hW,2 cy (k, u2 ) · · · hW,L cy (k, uL )] :  2 L h φ (u ) W,l y l l=1    |ρ [cy (k, U), cz,W (k, U)] |2 =  L L 2 φ (u ) φ (u ) h l l l=1 y l=1 W,l y L L φx (ul ) φx (ul ) = l=1 · L l=1 L l=1 φy (ul ) l=1 hW,l φx (ul ) =

|ρ [cx (k, U), cy (k, U)] |2 . |ρ [cy (k, U), cz,W (k, U)] |2

140

10 Optimal Filters in the Transform Domain

Therefore |ρ [cx (k, U), cy (k, U)] |2 = |ρ [cy (k, U), cz,W (k, U)] |2 × |ρ [cx (k, U), cz,W (k, U)] |2 ≤ |ρ [cx (k, U), cz,W (k, U)] |2 . Using (10.65) and Property 10.10 in the previous expression, we get iSNR(U) oSNR(hW , U) ≤ , 1 + iSNR(U) 1 + oSNR(hW , U) as a result oSNR(hW , U) ≥ iSNR(U). Substituting (10.92) into (10.57), we find the fullband MNMSE: L J˜ (hW , U) = 1 −

φ2v (ul )φ−1 y (ul ) ≤ 1. L l=1 φv (ul )

l=1

(10.101)

We can compute the fullband speech-distortion index by substituting (10.92) into (10.38): υsd (hW , U) = 1 −

oSNR(hW , U) + 2 ≤ 1. iSNR(U) · ξnr (hW , U)

(10.102)

Using (10.57) and (10.102), we get the fullband noise-reduction factor: ξnr (hW , U) =

oSNR(hW , U) + 1 ≥ 1. iSNR(U) − J˜ (hW , U)

(10.103)

Property 10.21. We have   J˜ (hW , U) = iSNR(U) 1 − |ρ[cx (k, U), cz,W (k, U)]|2 .

(10.104)

Proof. Indeed L L φ2x (ul )φ−1 y (ul ) l=1 φx (ul ) ˜ J (hW , U) = L − l=1 L l=1 φv (ul ) l=1 φv (ul )   L 2 −1 l=1 φx (ul )φy (ul ) = iSNR(U) 1 − L l=1 φx (ul )   = iSNR(U) 1 − |ρ[cx (k, U), cz,W (k, U)]|2 . Therefore, the fullband NMSE is minimized when the fullband SPCC between the signals cx (k, U) and cz (k, U) is maximized. This fullband SPCC can be rewritten as follows:

10.5 Examples of Filter Design

|ρ[cx (k, U), cz,W (k, U)]|2 =

141

1 1 + oSNR(hW , U) · . (10.105) ξsr (hW , U) oSNR(hW , U)

We observe that the transform-domain Wiener filter is compromising between speech reduction (i.e., speech distortion) and fullband output SNR improvement. Property 10.22. We have iSNR(U) iSNR(U) ≤ J˜ (hW , U) ≤ . 1 + oSNR(hW , U) 1 + iSNR(U)

(10.106)

Proof. Since |ρ[cx (k, U), cz,W (k, U)]|2 ≥

iSNR(U) 1 + iSNR(U)

and with the help of (10.104), we easily get J˜ (hW , U) ≤

iSNR(U) . 1 + iSNR(U)

Since |ρ[cx (k, U), cz,W (k, U)]|2 ≤

oSNR(hW , U) 1 + oSNR(hW , U)

and, again, with the help of (10.104), we obtain iSNR(U) ≤ J˜ (hW , U) . 1 + oSNR(hW , U) Property 10.23. We have 2

[1 + oSNR(hW , U)] ≤ ξnr (hW , U) ≤ iSNR(U) · oSNR(hW , U) [1 + iSNR(U)] [1 + oSNR(hW , U)] . iSNR2 (U)

(10.107)

Proof. Easy to show by using (10.103) and the bounds of J˜ (hW , U) [eq. (10.106)]. Property 10.24. We have 1 2

[1 + oSNR(hW , U)]

≤ υsd (hW , U) ≤

1 + oSNR(hW , U) − iSNR(U) . [1 + iSNR(U)] [1 + oSNR(hW , U)] (10.108)

142

10 Optimal Filters in the Transform Domain

Proof. Easy to show by using (10.102) and the bounds of ξnr (hW , U) [eq. (10.107)]. To finish this study on the Wiener filter, let us convert the transformdomain Wiener filter, hW , into the time-domain filtering matrix, HW (U). Indeed, with (10.23) and (10.92) we can rewrite, equivalently, the transformdomain Wiener filter into the time domain:
 H −1/2 −1 HW (U) = R1/2 , (10.109) y U I − Φy (U)Φv (U) U Ry where

  Φv (U) = diag GH (U)Rv G(U)

(10.110)

is a diagonal matrix whose nonzero elements are the elements of the diagonal of the matrix GH (U)Rv G(U). Now if we substitute (10.9) into (7.1) (see Chapter 7), the time-domain Wiener filter [given in (7.1)] can be written as  
 −1/2 −1/2 H HW = R1/2 U I − Φ (U) G(U) R G(U) Φ (U) UH R−1/2 . v y y y y (10.111) It is clearly seen that if the matrix GH (U)Rv G(U) is diagonal, the two filters HW and HW (U) are identical. In this scenario, it would not matter which unitary matrix we choose. 10.5.2 Parametric Wiener Filter One convenient way to control the compromise between noise reduction and speech distortion is via the so-called parametric Wiener filter:
β2 hG,l = 1 − sinβ1 θ(ul ) ,

(10.112)

where β1 and β2 are two positive parameters that allow the control of this compromise. For (β1 , β2 ) = (2, 1), we get the transform-domain Wiener filter developed previously. Taking (β1 , β2 ) = (2, 1/2), leads to hP,l =

1 − sin2 θ(ul )

(10.113)

= cos θ(ul ), which is the equivalent form of the power subtraction method. The pair (β1 , β2 ) = (1, 1) gives the equivalent form of the magnitude subtraction method: hM,l = 1 − sin θ(ul ) ! = 1 − 1 − cos2 θ(ul ).

(10.114)

10.5 Examples of Filter Design

143

We can verify that the subband noise-reduction factors for the transformdomain power subtraction and magnitude subtraction methods are ξnr (hP,l ) = ξnr (hM,l ) =

1 , cos2 θ(ul ) 1

(10.115)

[1 − sin θ(ul )]

2,

(10.116)

and the corresponding subband speech-distortion indices are 2

υsd (hP,l ) = [1 − cos θ(ul )] , υsd (hM,l ) = sin2 θ(ul ).

(10.117) (10.118)

We can also easily check that ξnr (hM,l ) ≥ ξnr (hW,l ) ≥ ξnr (hP,l ) , υsd (hP,l ) ≤ υsd (hW,l ) ≤ υsd (hM,l ) .

(10.119) (10.120)

The two previous inequalities are very important from a practical point of view. They show that, among the three methods, the magnitude subtraction is the most aggressive one as far as noise reduction is concerned but at the same time it’s the one that will likely distorts most the speech signal. The smoother approach is the power subtraction while the Wiener filter is between the two others in terms of speech distortion and noise reduction. Since 0 ≤ hG,l ≤ 1, then oSNR (hG , U) ≥ iSNR(U). Therefore, all three methods improve the (fullband) output SNR. The two particular transform-domain filters derived above can be rewritten, equivalently, into the time domain. • Power subtraction:
1/2 H −1/2 −1 HP (U) = R1/2 U Ry . y U I − Φy (U)Φv (U)

(10.121)

• Magnitude subtraction:   −1/2 1/2 HM (U) = R1/2 U I − Φ (U)Φ (U) UH R−1/2 . y y v y

(10.122)

These two filters are, of course, not optimal in any sense but they can be very practical. 10.5.3 Tradeoff Filter Another useful filter can be designed by minimizing the speech distortion with the constraint that the residual noise is equal to a positive value smaller than the level of the original noise. This optimization problem can be translated mathematically as

144

10 Optimal Filters in the Transform Domain

min Jx (hl , ul ) hl

subject to

Jv (hl , ul ) = βl φv (ul ), l = 1, 2, . . . , L, (10.123)

where 0 < βl < 1 in order to have some noise reduction. Using a Lagrange multiplier, µl (≥ 0), to adjoin the constraint to the cost function, we can derive the optimal filter: φx (ul ) φx (ul ) + µl φv (ul ) φy (ul ) − φv (ul ) = φy (ul ) + (µl − 1)φv (ul )

hT,l =

=

(10.124)

1 − |ρ [cv (k, ul ) , cy (k, ul )] |2 , l = 1, 2, . . . , L. 1 + (µl − 1)|ρ [cv (k, ul ) , cy (k, ul )] |2

Hence, hT,l is a transform-domain Wiener filter with adjustable input noise level µl φvv (ul ). Since 0 ≤ hT,l ≤ 1, ∀µl ≥ 0, then oSNR (hT , U) ≥ iSNR(U). Therefore, this method improves the (fullband) output SNR. The Lagrange multiplier must satisfy Jv (hT,l , ul ) = βl φv (ul ) = h2T,l φv (ul ), l = 1, 2, . . . , L. Substituting (10.124) into (10.125), we can find

1 µl = iSNR(ul ) √ − 1 , l = 1, 2, . . . , L βl and from (10.125), we also have ! hT,l = βl , l = 1, 2, . . . , L.

(10.125)

(10.126)

(10.127)

The Lagrange multiplier µl can always be chosen in an ad-hoc way if we prefer. Then, we can see from (10.124) that there are four cases: • µl = 1; in this case, the tradeoff and Wiener filters are the same, i.e., hT,l = hW,l ; • µl = 0; in this circumstance, we have hT,l = 1 and there is no noise reduction and no speech distortion; • µl > 1; this situation corresponds to a more aggressive noise reduction at the expense of higher speech distortion as compared to the Wiener filter; • µl < 1; this case corresponds to a more conservative noise reduction with less speech distortion as compared to the Wiener filter. With (10.23) and (10.127) we can rewrite, equivalently, the transformdomain tradeoff filter into the time domain:
 1/2 β1 β2 · · · βL UH R−1/2 HT (U) = R1/2 . (10.128) y U diag y

10.5 Examples of Filter Design

145

10.5.4 Examples of Unitary Matrices There are perhaps a very large number of unitary (or orthogonal) matrices that can be used in tandem with the different noise reduction filters presented in this section. But does a transformation exist in such a way that an optimal filter maximizes noise reduction while it minimizes speech distortion at the same time? The answer to this question is not straightforward. However, intuitively we believe that some unitary matrices will be more effective than others for a given noise reduction filter. The first obvious choice is the KLT developed in Chapter 2. In this case, U = Qy where Qy contains the eigenvectors of the correlation matrix, Ry , of the noisy signal, y(k), for which the spectral representation are the eigenvalues of Ry . This choice seems to be the most natural one since the Parseval’s theorem is verified. Another choice for U is the Fourier matrix:
 F = f1 f2 · · · fL , (10.129) where T 1
1 exp(jωl ) · · · exp[jωl (L − 1)] fl = √ L

(10.130)

and ωl = 2π(l−1)/L, l = 1, . . . , L. Even though F is unitary, the matrix G(F) constructed from F is not; as a result, the Parseval’s theorem does not hold but the transform signals at the different frequencies are uncorrelated. Filters in this new Fourier domain will probably perform differently as compared to the classical frequency-domain filters. In our application, all signals are real and it may be more convenient to select an orthogonal matrix instead of a unitary one. So another choice close to the previous one is the discrete cosine transform:
 C = c1 c2 · · · cL , (10.131) where  T cl = c(1) c(2) cos π(2l−1) · · · c(L) cos π(2l−1)(L−1) 2L 2L

(10.132)

! ! with c(1) = 1/L and c(l) = 2/L for l = 1. We can verify that CT C = CCT = I. Another possible choice for U is the Hadamard transform. We are now going to give an example of a Hadamard matrix. Let us define first the two matrices: H1 = 1,   1 1 1 1 H2 = √ = √ H2 . 1 −1 2 2

(10.133) (10.134)

146

10 Optimal Filters in the Transform Domain

A Hadamard matrix of order 2p can be constructed from the two previous matrices as follows:    1 H2p−1 H2p−1 H2p = √   2p H2p−1 −H2p−1 1 = √ H2 ⊗ H2p−1 (10.135) 2p for p ≥ 2, where ⊗ is the Kronecker product. It can be verified that H2p HT2p = HT2p H2p = I.

10.6 Experiments We have formulated the noise reduction problem in a generalized transform domain and discussed the design of different noise reduction filters in that domain. In this section, we study several filters through experiments and compare different transforms for their impact on noise reduction performance. The experimental setup is the same as used in Chapter 7. 10.6.1 Performance of Wiener Filter in White Gaussian Noise In this experiment, we examine the performance of the Wiener filter in a white Gaussian noise with an iSNR = 10 dB. Similar to the time- and KLE-domain filters, the implementation of the generalized transform-domain Wiener filter requires to know the correlation matrices Ry and Rv . Although different approaches can be used to compute Ry from the noisy signal y(k), we will adopt the recursive method given in (7.44). Same as in Section 7.4, we do not to use any noise estimator, but compute the noise correlation matrix Rv directly from the noise signal. Since the background noise is stationary, we use a long-term sample average to compute Rv . With the above ways of computing Ry and Rv , the performance of the Wiener filter given in (10.92) is mainly affected by three major elements: the forgetting factor αy , the frame length L, and the transformation matrix U. In the first experiment, we fix the frame length to L = 32 and study the effect of the forgetting factor αy with different transforms on the performance. For the matrix U, we choose to compare four well-known transforms: KL, Fourier, cosine, and Hadamard. Figure 10.1 plots the output SNR and the speech-distortion index for different transforms as a function of the forgetting factor αy . It is seen that the output SNR for all the studied transforms first increases as αy increases and then decreases. The highest output SNR is obtained when αy is between 0.985 and 0.995. This agrees with what has been observed with the time- and KLE-domain filters.

10.6 Experiments

147

20 KL Fourier

18

Cosine oSNR (dB)

16

14

Hadamard

12

10 (a)

8 0.9

0.92

0.94

0.96

0.98

1

−5 Hadamard

Cosine

10 log(υsd )

−10

Fourier KL

−15

(b)

−20 0.9

0.92

0.94

0.96

0.98

1

αy

Fig. 10.1. Noise reduction performance of the Wiener filter versus αy in white Gaussian noise: iSNR = 10 dB and L = 32.

Generally, the speech-distortion index υsd decreases as the forgetting factor αy increases. So, the larger the value of αy , the smaller the speech distortion index. The only exception is with the KL transform, where the speech-distortion index first decreases slightly and then increases as αy increases from 0.9 to its upper bound 1. We also see from Fig. 10.1 that the Fourier and cosine transforms yielded similar performance. When αy is reasonably large (e.g., ≥ 0.98), the KL, Fourier, and cosine transforms produced similar output SNRs. Comparatively, however, the KL transform has much lower speech distortion. When the value

148

10 Optimal Filters in the Transform Domain

of αy is in a reasonable range (e.g., ∈ [0.96, 1]), the Hadamard transform can also improve the SNR. But its performance is much inferior to that of the other three transforms. The transform-domain Wiener filter with the KL transform is not identical to the KLE-domain Wiener filter (Class I) derived in Chapter 9. Indeed, it is noticed from Fig. 10.1 that the speech-distortion index with the KL transform is somehow slightly different from what was observed with the KLE-domain Wiener filter in Fig. 9.1. There are three major reasons for this difference: 1) in Chapter 9, the matrix U is obtained by eigenvalue decomposition of the correlation matrix Rx , while the matrix U in this chapter is derived from the correlation matrix Ry ; 2) the frame length L in Fig. 9.1 is 20 while it is 32 in this experiment (because we also used the Hadamard transform for which the value of L has to be a power of 2); 3) in this section, a small positive constant has been applied to regularize the correlation matrix Ry so that the inverse of its square root, i.e., R−1/2 , can be computed reliably, while we do y not regularize any matrix in Chapter 9. 10.6.2 Effect of Filter Length on Performance In this experiment, we study the effect of the frame length L on the noise reduction performance. Same as in the previous experiment, white Gaussian noise is used with an iSNR = 10 dB. Again, the noise correlation matrix is computed using a long-term average. Based on the previous results, we set αy = 0.99. Figure 10.2 depicts the output SNR and speech-distortion index, both as a function of L. It is seen that, as L increases, the output SNR of the Wiener filter using the KL transform first increases and then decreases. Good performance with this transform is obtained when L is between 20 and 80. The Fourier and cosine transforms yielded similar performance. For the Hadamard transform, a larger L corresponds to a less SNR gain and a larger speech-distortion index, which indicates that a small frame length L should be preferred if the Hadamard transform is used. Generally, however, the Hadamard transform is much inferior to the KL, Fourier, and cosine transforms in performance. 10.6.3 Performance of Tradeoff Filter in White Gaussian Noise Now we evaluate the performance of the transform-domain tradeoff filter given in (10.124) in different conditions. From the theoretical analysis, we already know that if µ = 1, the tradeoff filter is the Wiener filter. Increasing the value of µ will give more noise reduction, but will also lead to more speech distortion. In this experiment, we set µ = 4. Again, the noise used is a white Gaussian random process with an iSNR = 10 dB. The noise correlation matrix is computed using a long-term average. We first fix the frame length L to 32 and investigate the effect of αy and different transforms on the performance.

10.6 Experiments

149

20 KL

Fourier

18

oSNR (dB)

16 Cosine

14 Hadamard

12

10 (a)

8 0

20

40

60

80

100

120

140

160

140

160

0 Hadamard Fourier

10 log(υsd )

−4

−8 Cosine

−12 KL

−16

(b)

−20 0

20

40

60

80 L

100

120

Fig. 10.2. Noise reduction performance of the Wiener filter versus L in white Gaussian noise: iSNR = 10 dB and αy = 0.99.

Figure 10.3 portrays the output SNR and speech-distortion index as a function of αy . Similar to the Wiener filter case, the output SNR (for all the studied transforms) first increases and then drops as αy increases. The largest SNR gain for each transform is obtained when αy is between 0.985 and 0.995. The KL transform yielded the best performance (with the highest output SNR and lowest speech-distortion index). The Fourier and cosine transforms behave similarly. In general, the performance of the Hadamard transform is

150

10 Optimal Filters in the Transform Domain 20 KL

18

oSNR (dB)

16 Cosine

14

Hadamard

Fourier

12

10 (a)

8 0.9

0.92

0.94

0.96

0.98

1

0 Hadamard

−2 −4 Cosine

10 log(υsd )

−6 −8 −10

Fourier

−12

KL

−14 −16 −18 (b)

−20 0.9

0.92

0.94

0.96

0.98

1

αy

Fig. 10.3. Noise reduction performance of the tradeoff filter versus αy in white Gaussian noise: iSNR = 10 dB, L = 32, and µ = 4.

relatively poor as compared to the other three transforms, again, indicating that this transform is less effective for the purpose of noise reduction. Comparing Figs. 10.3 and 10.1, one can see that the output SNR of the tradeoff filter is boosted with the use of a larger value of µ, but this is achieved at the price of adding more speech distortion, which corroborates the analysis presented in Section 10.5.3. To investigate the effect of the frame length L on the performance, we set αy = 0.99 and change L from 4 to 160 (note that, for the Hadamard transform, the length L can only take values that are powers of 2). All other

10.6 Experiments

151

22 KL

20

Fourier

oSNR (dB)

18 Cosine

16

14 Hadamard

12 (a)

10 0

20

40

60

80

100

120

140

160

140

160

0 Fourier Hadamard

10 log(υsd )

−4

−8 Cosine

−12 KL

−16

(b)

−20 0

20

40

60

80 L

100

120

Fig. 10.4. Noise reduction performance of the tradeoff filter versus L in white Gaussian noise: iSNR = 10 dB, αy = 0.99, and µ = 4.

conditions are the same as used in the previous experiment. The results are shown in Fig. 10.4. Similar to the Wiener filter case, we observe that the output SNR for the KL transform first increases to its maximum and then drops as L increases. However, there are two major differences as compared to the Wiener filter case: 1) the largest SNR gain with the tradeoff filter appears when L is in the range of 40−120, while such performance occurs when L in the range of 40−80 for the Wiener filter; 2) although the performance with the KL transform decreases if we keep increasing L after the optimal performance is achieved, the performance degradation with L is almost negligible. The reason

152

10 Optimal Filters in the Transform Domain

for these two differences can be explained as follows. In this experiment, we set µ = 4 and all the βl in the diagonal matrix diag1/2 [β1 β2 · · · βL ] that are less than 0 are forced to 0. After a certain threshold, if we further increase L, the signal subspace that consists of all the positive βl value do not increase much. In other words, even though we increases L, which results in a larger size for Ry , we are still dealing with a signal subspace of similar order. As a result, the performance does not change much. Again, the Fourier and cosine transforms have similar performance. Comparatively, the effect of L on the Fourier, cosine, and Hadamard transforms in the subspace case is almost the same as that in the Wiener filter situation. The only difference is that now we have achieved a higher SNR gain. But the speech distortion is also higher.

10.7 Summary In this chapter, we formulated the noise reduction problem in a generalized transform domain, where any unitary (or orthogonal) matrix can be used to construct the forward (for analysis) and inverse (for synthesis) transforms. The advantages of working in this generalized domain are multiple, such as different transforms can be used to replace each other without any requirement to change the algorithm formulation (optimal filter) and it is easier to fairly compare different transforms for their noise reduction performance. We have addressed the design of different optimal and suboptimal filters in such a generalized transform domain, including the Wiener filter, the parametric Wiener filter, the tradeoff filter, etc. We have also compared, through experiments, four different transforms (KL, Fourier, cosine, and Hadamard) for their noise reduction performance. In general, the KL transform yielded the best performance. The Fourier and cosine transforms have quite similar performance, which is slightly poorer than that of the KL transform. While the Hadamard transform can improve the SNR, it is much less effective than the other three transforms for noise reduction.

11 Spectral Enhancement Methods

In this chapter, we focus on spectral enhancement methods in the shorttime Fourier transform (STFT) domain. We present statistical models for the speech and noise signals, and derive estimators for the speech signal using various distortion measures. We also describe an example of a speech enhancement algorithm and demonstrate its performance on speech signals degraded by various additive noise types.

11.1 Problem Formulation In the STFT domain, (2.1) can be rewritten as Y (n, m) = X(n, m) + V (n, m),

(11.1)

where Y (n, m), X(n, m), and V (n, m) are respectively the STFTs of y(k), x(k), and v(k), at time-frame n and frequency-bin m. We recall that the STFT and the inverse transform are A(n, m) =

M −1 

a(k + nL) ψ(k) e−j M

2π

km

,

(11.2)

˜ − nL) ej 2π N m(k−nL) , A(n, m) ψ(k

(11.3)

k=0

a(k) =

−1 M  n

m=0

where A ∈ {Y, X, V }, a ∈ {y, x, v}, ψ(k) is an analysis window of size M (e.g., Hamming window), L is the framing step (number of samples separating two ˜ successive time frames), and ψ(k) is a synthesis window that is biorthogonal to the analysis window ψ(k) [124]. Another possible form for (11.1) is Ay (n, m)ejϕy (n,m) = Ax (n, m)ejϕx (n,m) + Av (n, m)ejϕv (n,m) ,

J. Benesty et al., Noise Reduction in Speech Processing, Springer Topics in Signal Processing 2, DOI 10.1007/978-3-642-00296-0_11, © Springer-Verlag Berlin Heidelberg 2009

(11.4)

154

11 Spectral Enhancement Methods

where for any random signal A(n, m) = Aa (n, m)ejϕa (n,m) , Aa (n, m) and ϕa (n, m) are its amplitude and phase at time-frame n and frequency-bin m = 0, . . . , M − 1, a ∈ {y, x, v}. Using the variances of the noisy spectral coefficients and the fact that x(k) and v(k) are uncorrelated, we get φy (n, m) = φx (n, m) + φv (n, m), where

(11.5)

  2 φa (n, m) = E |A(n, m)|
 = E A2a (n, m)

is the variance of a spectral coefficient of the signal a(k) at time-frame n and frequency-bin m. An estimate of X(n, m) can be obtained by multiplying Y (n, m) with a complex gain, i.e., Z(n, m) = H(n, m)Y (n, m) = H(n, m) [X(n, m) + V (n, m)]

(11.6)

= XF (n, m) + VF (n, m), where Z(n, m) is the STFT of the signal z(k). The variance of a spectral coefficient of z(k) is then 2

φz (n, m) = |H(n, m)| φy (n, m) 2

= |H(n, m)| [φx (n, m) + φv (n, m)] .

(11.7)

We can go back to the time domain to obtain the estimate z(k) =

−1 M  n

˜ − nL) ej 2π N m(k−nL) . Z(n, m) ψ(k

(11.8)

m=0

The spectral enhancement problem is generally formulated as deriving an estimator Z(n, m) for the speech spectral coefficients, such that the expected value of a certain distortion measure is minimized. Let d [X(n, m), Z(n, m)] denote a distortion measure between X(n, m) and its estimate Z(n, m), let φˆx (n, m) denote an estimate for the variance of a speech spectral coefficient X(n, m), and let φˆv (n, m) denote an estimate for the variance of a noise spectral coefficient V (n, m). Then an estimator for X(n, m) which minimizes the expected distortion given φˆx (n, m), φˆv (n, m), and the noisy spectral coefficient Y (n, m) is obtained by solving the minimization problem     min E d [X(n, m), Z(n, m)]  φˆx (n, m) , φˆv (n, m) , Y (n, m) . (11.9) Z(n,m)

11.2 Performance Measures

155

In particular, restricting ourselves to a squared error distortion measure of the form 2

d [X(n, m), Z(n, m)] = |g[Z(n, m)] − g[X(n, m)]| ,

(11.10)

where g(X) is a specific function of X (e.g., X, |X|, log |X|, ej∠X ), the estimator Z(n, m) can be calculated from     g[Z(n, m)] = E g[X(n, m)]  φˆx (n, m) , φˆv (n, m) , Y (n, m) . (11.11) The design of a particular spectral enhancement algorithm requires to specify the function g(X) that determines the fidelity criterion of the estimator, and estimators φˆx (n, m) and φˆv (n, m) for the speech and noise spectral variances.

11.2 Performance Measures 11.2.1 SNR The input SNR in the STFT domain is defined as the ratio of the intensity of the signal of interest (speech) over the intensity of the background noise, i.e.,  M −1 φx (n, m) iSNR = n m=0 . M −1 n m=0 φv (n, m)

(11.12)

We also define the subband input SNR and segmental input SNR in the STFT domain as φx (n, m) , m = 0, 1, . . . , M − 1, φv (n, m) M −1 φx (n, m) iSNR(n) = m=0 . M −1 m=0 φv (n, m)

iSNR(n, m) =

(11.13) (11.14)

After noise reduction with the STFT domain model given in (11.6), the subband, segmental, and fullband output SNRs are 2

oSNR [H(n, m)] =

|H(n, m)| φx (n, m) 2

|H(n, m)| φv (n, m) = iSNR(n, m), m = 0, 1, . . . , M − 1,

M −1 2 |H(n, m)| φx (n, m) oSNR [H(n)] = m=0 , M −1 2 m=0 |H(n, m)| φv (n, m)

(11.15)

(11.16)

156

11 Spectral Enhancement Methods

 M −1 2 |H(n, m)| φx (n, m) oSNR(H) = n m=0 . M −1 2 n m=0 |H(n, m)| φv (n, m)

(11.17)

It is important to find a complex gain H(n, m), n ∈ ZZ, m = 0, 1, . . . , M − 1, in such a way that oSNR(H) ≥ iSNR. Property 11.1. We always have M −1 

iSNR(n, m) ≥ iSNR(n) ,

(11.18)

m=0 −1 M  n

iSNR(n, m) ≥

iSNR(n) ≥ iSNR ,

(11.19)

n

m=0

M −1 



oSNR [H(n, m)] ≥ oSNR [H(n)] ,

(11.20)

m=0 −1 M  n

oSNR [H(n, m)] ≥

m=0



oSNR [H(n)] ≥ oSNR(H).

(11.21)

n

This means that the aggregation of the subband SNRs is greater than or equal to the segmental SNR and the aggregation of the segmental SNRs is greater than or equal to the fullband SNR. Proof. Easy to show by using inequality (3.16). 11.2.2 Noise-Reduction Factor By analogy to the time-domain and frequency-domain definitions, we define the subband, segmental, and fullband noise-reduction factors in the STFT domain as ξnr [H(n, m)] =

φv (n, m) 2

|H(n, m)| φv (n, m) 1 = 2 , m = 0, 1, . . . , M − 1, |H(n, m)| M −1 φv (n, m) ξnr [H(n)] = M −1 m=0 , 2 m=0 |H(n, m)| φv (n, m)  M −1 φv (n, m) ξnr (H) =  Mn−1 m=0 . 2 n m=0 |H(n, m)| φv (n, m)

(11.22) (11.23) (11.24)

The segmental and fullband noise-reduction factors are ratios of the noise −1 energy over a weighted energy of the noise with the weighting ξnr [H(n, m)]. After the filtering operation, the residual noise level at time-frame n and frequency-bin m is expected to be lower than that of the original noise level, therefore ξnr [H(n, m)], ξnr [H(n)], and ξnr (H) are expected to larger than 1.

11.2 Performance Measures

157

Property 11.2. We always have M −1 

ξnr [H(n, m)] ≥ ξnr [H(n)] ,

(11.25)

m=0 −1 M  n

ξnr [H(n, m)] ≥



ξnr [H(n)] ≥ ξnr (H).

(11.26)

n

m=0

This means that the aggregation of the subband noise-reduction factors is greater than or equal to the segmental noise-reduction factor and the aggregation of the segmental noise-reduction factors is greater than or equal to the fullband noise-reduction factor. Proof. Easy to show by using inequality (3.16). 11.2.3 Speech-Distortion Index The definition of the speech-distortion index in the frequency domain is extended to the STFT domain. We define the subband speech-distortion index in the STFT domain as   2 E |H(n, m)X(n, m) − X(n, m)| υsd [H(n, m)] = φx (n, m) 2

= |1 − H(n, m)| , m = 0, 1, . . . , M − 1.

(11.27)

The subband speech-distortion index is lower bounded by 0 and expected to be lower than 1 for optimal filters. The higher the value of υsd [H(n, m)], the more the speech is distorted at time-frame n and frequency-bin m. The segmental and fullband speech-distortion indices in the STFT domain are  M −1  2 E |H(n, m)X(n, m) − X(n, m)| m=0 υsd [H(n)] = M −1 m=0 φx (n, m) M −1 2 |1 − H(n, m)| φx (n, m) = m=0 M −1 m=0 φx (n, m) M −1 υsd [H(n, m)] φx (n, m) = m=0 M −1 (11.28) m=0 φx (n, m) and  M −1 υsd (H) =

n

m=0

 M −1 =

n

  2 E |H(n, m)X(n, m) − X(n, m)|  M −1 n m=0 φx (n, m)

υsd [H(n, m)] φx (n, m) .  M −1 n m=0 φx (n, m)

m=0

(11.29)

158

11 Spectral Enhancement Methods

Equations (11.28) and (11.29) are ratios of the weighted energy of the speech with the weighting υsd [H(n, m)] over the energy of the speech. If υsd [H(n, m)] ≤ 1, ∀(n, m), we see from (11.28) and (11.29) that υsd [H(n)] ≤ 1 and υsd (H) ≤ 1. Property 11.3. We always have M −1 

υsd [H(n, m)] ≥ υsd [H(n)] ,

(11.30)

m=0 −1 M  n

υsd [H(n, m)] ≥



υsd [H(n)] ≥ υsd (H).

(11.31)

n

m=0

This means that the aggregation of the subband speech-distortion indices is greater than or equal to the segmental speech-distortion index and the aggregation of the segmental speech-distortion indices is greater than or equal to the fullband speech-distortion index. Proof. Easy to show by using inequality (3.16). 11.2.4 Speech-Reduction Factor We define the subband speech-reduction factor in the STFT domain as ξsr [H(n, m)] =

φx (n, m) 2

|H(n, m)| φx (n, m) 1 = 2 , m = 0, 1, . . . , M − 1. |H(n, m)|

(11.32)

The larger the value of ξsr [H(n, m)], the more the speech is reduced at timeframe n and frequency-bin m. The segmental and fullband speech-reduction factors in the STFT domain are M −1 φx (n, m) ξsr [H(n)] = M −1 m=0 (11.33) 2 m=0 |H(n, m)| φx (n, m) M −1 m=0 φx (n, m) = M −1 −1 , ξ m=0 sr [H(n, m)] φx (n, m)  M −1

ξsr (H) =  Mn−1 n

φx (n, m)

m=0

(11.34)

2

|H(n, m)| φx (n, m)  M −1 n m=0 φx (n, m)

m=0

=  M −1 n

m=0

−1 ξsr [H(n, m)] φx (n, m)

.

The segmental and fullband speech-reduction factors in the STFT domain are ratios of the energy of the speech over the weighted energy of the speech with −1 the weighting ξsr [H(n, m)].

11.3 MSE Criterion

159

Property 11.4. We always have M −1 

ξsr [H(n, m)] ≥ ξsr [H(n)] ,

(11.35)

m=0 −1 M  n

ξsr [H(n, m)] ≥



ξsr [H(n)] ≥ ξsr (H).

(11.36)

n

m=0

This means that the aggregation of the subband speech-reduction factors is greater than or equal to the segmental speech-reduction factor and the aggregation of the segmental speech-reduction factors is greater than or equal to the fullband speech-reduction factor. Proof. Easy to show by using inequality (3.16). Property 11.5. The fullband and segmental measures mentioned above are related in the STFT domain by oSNR(H) ξnr (H) = , iSNR ξsr (H) oSNR [H(n)] ξnr [H(n)] = . iSNR(n) ξsr [H(n)]

(11.37) (11.38)

Proof. This is easy to see by combining expressions (11.12), (11.17), (11.24), and (11.34), and combining expressions (11.14), (11.16), (11.23), and (11.33). Hence, a segmental or fullband increase in the SNR, i.e., oSNR [H(n)] > iSNR(n) or oSNR(H) > iSNR, can be obtained only when the segmental, or respectively the fullband, noise-reduction factor is larger than the corresponding speech-reduction factor, i.e., ξnr [H(n)] > ξsr [H(n)] or ξnr (H) > ξsr (H).

11.3 MSE Criterion We define the error signal between the estimated and desired signals at timeframe n and frequency-bin m as E(n, m) = Z(n, m) − X(n, m) = H(n, m)Y (n, m) − X(n, m).

(11.39)

This error can also be put into the form: E(n, m) = Ex (n, m) + Ev (n, m),

(11.40)

Ex (n, m) = [H(n, m) − 1] X(n, m)

(11.41)

where

160

11 Spectral Enhancement Methods

is the speech distortion due to the complex gain, and Ev (n, m) = H(n, m)V (n, m)

(11.42)

represents the residual noise. The subband MSE in the STFT domain is then   2 J [H(n, m)] = E |E(n, m)|     2 2 = E |Ex (n, m)| + E |Ev (n, m)| 2

(11.43)

2

= |1 − H(n, m)| φx (n, m) + |H(n, m)| φv (n, m) = Jx [H(n, m)] + Jv [H(n, m)] . For the particular gain H(n, m) = 1, ∀(n, m), we get J (1) = φv (n, m),

(11.44)

so there will be neither noise reduction nor speech distortion. Therefore, we define the subband NMSE in the STFT domain as J [H(n, m)] J˜ [H(n, m)] = J (1) = iSNR(n, m) · υsd [H(n, m)] +

1 , ξnr [H(n, m)]

(11.45)

where Jx [H(n, m)] , φx (n, m) φv (n, m) ξnr [H(n, m)] = . Jv [H(n, m)]

υsd [H(n, m)] =

(11.46) (11.47)

The STFT-domain NMSE depends explicitly on the subband speechdistortion index and the subband noise-reduction factor. We define the segmental and fullband MSEs as J [H(n)] =

=

M −1 1  J [H(n, m)] M m=0

(11.48)

M −1 M −1 1  1  2 2 φx (n, m) |1 − H(n, m)| + φv (n, m) |H(n, m)| M m=0 M m=0

= Jx [H(n)] + Jv [H(n)] ,

11.4 Signal Model

J(H) =

N −1 M −1 1   J [H(n, m)] N M n=0 m=0

161

(11.49)

N −1 M −1 1   2 = φx (n, m) |1 − H(n, m)| + N M n=0 m=0 N −1 M −1 1   2 φv (n, m) |H(n, m)| N M n=0 m=0

= Jx (H) + Jv (H). The segmental and fullband normalized MSEs are J [H(n)] J˜ [H(n)] = M M −1 (11.50) m=0 φv (n, m) M −1 M −1 2 2 φx (n, m) |1 − H(n, m)| φv (n, m) |H(n, m)| = m=0 M −1 + m=0M −1 m=0 φv (n, m) m=0 φv (n, m) 1 = iSNR(n) · υsd [H(n)] + , ξnr [H(n)] J(H) ˜ J(H) = N M N −1 N −1 M −1 =

n=0 n=0 m=0 φv (n, m) N−1 M −1 2 n=0 m=0 φx (n, m) |1 − H(n, m)| + M −1 m=0 φv (n, m) N −1 M −1 2 n=0 m=0 φv (n, m) |H(n, m)| M −1 m=0 φv (n, m)

= iSNR · υsd (H) +

(11.51)

1 . ξnr (H)

Same as in the time, frequency, and KLE domains, in the STFT domain the NMSE (subband, segmental, or fullband) depends explicitly on the input SNR, speech-distortion index, and noise-reduction factor.

11.4 Signal Model In this section, we present a Gaussian statistical model that takes into account the time-correlation between successive spectral components of the speech signal. It is commonly assumed that expansion coefficients in different frequencybins are statistically independent [31], [32], [44], [90]. This allows to formulate independent estimation problems for each frequency-bin m, which greatly simplifies the resulting algorithms. A Gaussian statistical model in the STFT domain relies on the following assumptions [30]:

162

11 Spectral Enhancement Methods

1. The noise spectral coefficients {V (n, m)} are zero-mean statistically independent Gaussian random variables. The real and imaginary parts of V (n, m) are
independent  and identically distributed (iid) random variables ∼ N 0, 12 φv (n, m) . 2. Given {φx (n, m)}, the speech spectral coefficients {X(n, m)} are generated by ! X(n, m) = φx (n, m) W (n, m), (11.52) where {W (n, m)} is a white Gaussian
process  with zero mean (E [W (n, m)] = 0) and unit variance (E |W (n, m)|2 = 1). The probability density function (pdf) of W (n, m) is given by p [W (n, m)] =


 1 exp −|W (n, m)|2 . π

(11.53)

The first assumption requires that the overlap between successive time frames would be small (less than 50%) [31]. The second assumption implies that the speech spectral coefficients {X(n, m)} are conditionally zero-mean statistically independent random variables given their variances {φx (n, m)}, satisfying   1 |X(n, m)|2 p [ X(n, m) | φx (n, m)] = exp − . (11.54) π φx (n, m) φx (n, m) Note that successive spectral coefficients are generally correlated, since the random processes {X(n, m) | n = 0, 1, . . .} and {φx (n, m) | n = 0, 1, . . .} are not independent. However, given φx (n, m), X(n, m) is statistically independent of X(n , m ) for all n = n and m = m . Hence, the time-correlation between successive spectral coefficients of the speech signal is taken into account, while still considering the scalar estimation problem formulated in (11.9).

11.5 Signal Estimation In this section, we derive estimators for X(n, m) using various fidelity criteria, assuming that φˆx (n, m) and φˆv (n, m) are given. Fidelity criteria that are of particular interest for speech enhancement applications are minimum mean-squared error (MMSE) [85], MMSE of the spectral amplitude (MMSE-SA) [44], and MMSE of the log-spectral amplitude (MMSE-LSA) [27], [45]. The MMSE estimator is derived by substituting into (11.11) the function g[X(n, m)] = X(n, m). The MMSE-SA estimator is obtained by using g[X(n, m)] = |X(n, m)|, and the MMSE-LSA estimator is obtained by using g[X(n, m)] = log |X(n, m)|. 11.5.1 MMSE Spectral Estimation Let HMSE (n, m) denote a gain function that satisfies

11.5 Signal Estimation

163

E [X(n, m) | φx (n, m) , φv (n, m) , Y (n, m) ] = HMSE (n, m)Y (n, m). (11.55) Then, by substituting g[X(n, m)] = X(n, m) into (11.11) we have Z(n, m) = HMSE (n, m) Y (n, m).

(11.56)

The specific expression for HMSE (n, m) depends on the statistical model:  1 HMSE (n, m) = X(n, m) p [X(n, m) | φx (n, m) , φv (n, m) ] dX, Y (n, m) where p [X(n, m) | φx (n, m) , φv (n, m) ] = p [Y (n, m) | X(n, m) , φv (n, m) ] p [X(n, m) | φx (n, m) ] . p [Y (n, m) | φx (n, m) , φv (n, m) ] For a Gaussian model, the gain function depends only on the input SNR [85]: HMSE (n, m) =

iSNR(n, m) , 1 + iSNR(n, m)

(11.57)

and the MMSE estimator for X(n, m) reduces to Z(n, m) =


iSNR(n, m) Y (n, m),
1 + iSNR(n, m)

(11.58)


where iSNR(n, m) = φˆx (n, m)/ φˆv (n, m) is an estimate of iSNR(n, m). 11.5.2 MMSE Spectral Amplitude Estimation Estimators which minimize the MSE of the spectral amplitude, or log-spectral amplitude, have been found advantageous to MMSE spectral estimators in speech enhancement applications [44], [45], [100]. An MMSE-SA estimator is obtained by substituting g[X(n, m)] = |X(n, m)| = Ax (n, m) into (11.11), and combining the resulting amplitude estimate with the phase of the noisy spectral coefficient Y (n, m). Let |Y (n, m)|2 γ(n, m) = (11.59) φv (n, m) denote an instantaneous input SNR. Note that E [γ(n, m)] =

φy (n, m) φx (n, m) + φv (n, m) = = iSNR(n, m) + 1. φv (n, m) φv (n, m)

The spectral gain that satisfies

164

11 Spectral Enhancement Methods

E [Ax (n, m) | φx (n, m) , φv (n, m) , Y (n, m) ] = HSA (n, m)Ay (n, m)

(11.60)

is given by [44]  1 Ax (n, m) p [X(n, m) | φx (n, m) , φv (n, m) ] dX Ay (n, m)   √ ! π ϑ(n, m) ϑ(n, m) = exp − 2 γ(n, m) 2      ϑ(n, m) ϑ(n, m) × [1 + ϑ(n, m)]I0 + ϑ(n, m) I1 , 2 2 (11.61)

HSA (n, m) =

where ϑ(n, m) =

γ(n, m) iSNR(n, m) , 1 + iSNR(n, m)

(11.62)

and I0 (·) and I1 (·) denote the modified Bessel functions of zero and first order, respectively. Combining the resulting amplitude estimate with the phase of the noisy spectral coefficient yields   ˆ m) ˆ m) π ϑ(n, ϑ(n, Z(n, m) = exp − × 2 γˆ (n, m) 2 $    % ˆ m) ˆ m) ϑ(n, ϑ(n, ˆ ˆ [1 + ϑ(n, m)]I0 + ϑ(n, m) I1 Y (n, m), 2 2 √

(11.63) where γˆ (n, m) =

|Y (n, m)|2 , φˆv (n, m)


ˆ m) = γˆ (n, m) iSNR(n, m) . ϑ(n,
1 + iSNR(n, m)

Figure 11.1 displays parametric gain curves describing HSA (n, m) for several values of γ(n, m). For a fixed value of the instantaneous input SNR, the LSA gain is a monotonically increasing function of iSNR(n, m). However, for a fixed value of iSNR(n, m), the LSA gain is a monotonically decreasing function of γ(n, m). 11.5.3 MMSE Log-Spectral Amplitude Estimation An MMSE-LSA estimator is obtained by substituting g[X(n, m)] = log[Ax (n, m)] into (11.11), and combining the resulting amplitude estimate with the phase of the noisy spectral coefficient. The spectral gain that satisfies

11.5 Signal Estimation

165

20

HSA (n, m) (dB)

10 0

−10 γ = −15 dB γ = −7 dB γ = 0 dB γ = 7 dB γ = 15 dB

−20 −30 −40 −20

−15

−10

−5

0

5

10

15

20

iSNR(n, m) (dB) Fig. 11.1. Parametric gain curves describing the MMSE spectral amplitude gain function HSA (n, m) for a Gaussian model, obtained by (11.61).

E {log[Ax (n, m)] | φx (n, m) , φv (n, m) , Y (n, m) } = HLSA (n, m)Ay (n, m) (11.64) is given by [45]  1 HLSA (n, m) = log[Ax (n, m)] p [X(n, m) | φx (n, m) , φv (n, m) ] dX Ay (n, m) "  # iSNR(n, m) 1 ∞ e−x = exp dx . (11.65) 1 + iSNR(n, m) 2 ϑ(n,m) x The integral in (11.65) is the well known exponential integral of ϑ(n, m) and it can be numerically evaluated, e.g., using the expint function in MATLAB. Alternatively, it may be evaluated by using the following computationally efficient approximation, which was developed by Martin et al. [91]   ∞ −x for ϑ < 0.1  −2.31 log10 (ϑ) − 0.6 , e expint(ϑ) = dx ≈ −1.544 log10 (ϑ) + 0.166 , for 0.1 ≤ ϑ ≤ 1 .  −0.52 ϑ−0.26 x ϑ 10 , for ϑ > 1 (11.66) Combining the resulting amplitude estimate with the phase of the noisy spectral coefficient yields "  #
iSNR(n, m) e−x 1 ∞ Z(n, m) = exp dx Y (n, m). (11.67) 2 ϑ(n,m) x
ˆ 1 + iSNR(n, m) Figure 11.2 displays parametric gain curves describing HLSA (n, m) for several values of γ(n, m). For a fixed value of the instantaneous input SNR, the LSA gain is a monotonically increasing function of iSNR(n, m). Similar to HSA (n, m), for a fixed value of iSNR(n, m), the LSA gain is a monotonically

166

11 Spectral Enhancement Methods 20

HLSA (n, m) (dB)

10 0

−10 γ = −15 dB γ = −7 dB γ = 0 dB γ = 7 dB γ = 15 dB

−20 −30 −40 −20

−15

−10

−5

0

5

10

15

20

iSNR(n, m) (dB) Fig. 11.2. Parametric gain curves describing the MMSE log-spectral amplitude gain function HLSA (n, m) for a Gaussian model, obtained by (11.65).

decreasing function of γ(n, m). The behaviors of HSA (n, m) and HLSA (n, m) are related to the useful mechanism that counters the musical noise phenomenon [22]. Local bursts of the instantaneous input SNR, during noiseonly frames, are “pulled down” to the average noise level, thus avoiding local buildup of noise whenever it exceeds its average characteristics. As a result, the MMSE-SA and MMSE-LSA estimators generally produce lower levels of residual musical noise, when compared with the MMSE spectral estimators.

11.6 Spectral Variance Model In this section, we present a novel modeling approach for the speech spectral
 component variances φx (n, m) = E |X(n, m)|2 . This approach is based on generalized autoregressive conditional heteroscedasticity (GARCH) modeling, which is widely-used for modeling the volatility of financial time-series such as exchange rates and stock returns [20], [43]. Similar to financial time-series, speech signals in the STFT domain are characterized by heavy tailed distributions and volatility clustering [30]. GARCH models enable parametrization of the time variation of speech spectral variances, and derivation of mathematically tractable estimates, while taking into consideration the heavy-tailed distribution and variation clustering. 11.6.1 GARCH Model Let {d(k)} denote a real-valued discrete-time stochastic process and let ψ(k) denote the information set available at time k [e.g., {d(k)} may represent a sequence of observations, and ψ(k) may include the observed data through time k]. Then, the innovation (prediction error) ε(k) at time k in the MMSE

11.6 Spectral Variance Model

167

sense is obtained by subtracting from d(k) its conditional expectation given the information through time k − 1, ε(k) = d(k) − E [d(k) | ψ(k − 1) ] .

(11.68)

The conditional variance (volatility) of d(k) given the information through time k − 1 is by definition the conditional expectation of ε2 (k),
 σ 2 (k) = var [d(k) | ψ(k − 1) ] = E ε2 (k) | ψ(k − 1) . (11.69) Autoregressive conditional heteroscedasticity (ARCH) models, introduced by Engle [42] and generalized by Bollerslev [19], are widely-used in various financial applications such as risk management, option pricing, foreign exchange, and the term structure of interest rates [20]. They explicitly parameterize the time-varying volatility in terms of past conditional variances and past squared innovations (prediction errors), while taking into account excess kurtosis (i.e., heavy tail behavior) and volatility clustering, two important characteristics of financial time-series. ARCH and GARCH models explicitly recognize  the difference between the unconditional variance 2 E |d(k) − E[d(k)]| and the conditional variance σ 2 (k), allowing the latter to change over time. The fundamental characteristic of these models is that magnitudes of recent innovations provide information about future volatility. Let {w(k)} be a zero-mean unit-variance white noise process with some specified probability distribution. Then a GARCH model of order (p, q), denoted by ε(k) ∼ GARCH(p, q), has the following general form ε(k) = σ(k) w(k), (11.70)
2  2 2 2 2 σ (k) = f σ (k − 1), . . . , σ (k − p), ε (k − 1) . . . , ε (k − q) ,(11.71) where σ(k) is the conditional standard deviation given by the square root of (11.71). That is, the conditional variance σ 2 (k) is determined by the values of p past conditional variances and q past squared innovations, and the innovation ε(k) is generated by scaling a white noise sample with the conditional standard deviation. The ARCH(q) model is a special case of the GARCH(p, q) model with p = 0. The most widely-used GARCH model specifies a linear function f in (11.71) as follows, σ 2 (k) = κ +

q 

αi ε2 (k − i) +

i=1

p 

βj σ 2 (k − j),

j=1

where the values of the parameters are constrained by κ > 0 , αi ≥ 0 , βj ≥ 0 , q p   αi + βj < 1. i=1

j=1

i = 1, . . . , q , j = 1, . . . , p,

(11.72)

168

11 Spectral Enhancement Methods

The first three constraints are sufficient to ensure that the conditional variances {σ 2 (k)} are strictly positive. The forth constraint is a covariance stationarity constraint, which is necessary and sufficient for the existence of a finite unconditional variance of the innovations process [19]. Many financial time-series such as exchange rates and stock returns exhibit volatility clustering phenomenon, i.e., large changes tend to follow large changes of either sign and small changes tend to follow small changes. Equation (11.72) captures the volatility clustering phenomenon, since large innovations of either sign increase the variance forecasts for several samples. This in return increases the likelihood of large innovations in the succeeding samples, which allows the large innovations to persist. The degree of persistence is determined by the lag lengths p and q, as well as the magnitudes of the coefficients {αi } and {βj }. Furthermore, the innovations of financial timeseries are typically distributed with heavier tails than a Gaussian distribution. Bollerslev [19] showed that GARCH models are appropriate for heavy-tailed distributions. 11.6.2 Modeling Speech Spectral Variance The variances of the speech coefficients are hidden from direct observation, in the sense that even under perfect conditions of zero noise [V (n, m) = 0 ∀(n, m)], the values of {φx (n, m)} are not directly observable. Therefore, our approach is to assume that {φx (n, m)} themselves are random variables, and to introduce conditional variances which are estimated from the available information (e.g., the clean spectral coefficients through time-frame n − 1, or the noisy spectral coefficients through time-frame n) [30]. Let X0r = {X(n, m) | n = 0, . . . , r, m = 0, . . . , M − 1} represent the set of clean speech  spectral coefficients up to frame r, and let φx|r (n, m)
E A2x (n, m) | X0r denote the conditional variance of X(n, m) given the clean spectral coefficients up to frame r. Then, our statistical model for the speech spectral variance relies on the following assumption [30]: the conditional variance φx|n−1 (n, m), referred to as the one-frame-ahead conditional variance, is a random process which evolves as a GARCH(1, 1) process:
 φx|n−1 (n, m) = φmin + µ A2x (n − 1, m) + δ φx|n−2 (n − 1, m) − φmin , (11.73) where φmin > 0 ,

µ ≥ 0,

δ ≥ 0,

µ + δ < 1,

(11.74)

are the standard constraints imposed on the parameters of the GARCH model [20]. The parameters µ and δ are, respectively, the moving average and autoregressive parameters of the GARCH(1,1) model, and φmin is a lower bound on the variance of X(n, m).

11.6 Spectral Variance Model

169

11.6.3 Model Estimation The maximum-likelihood (ML) estimation approach is commonly used for estimating the parameters of a GARCH model [43]. In this section we derive the ML function of the model parameters, by using the spectral coefficients of the clean speech signal on some interval n ∈ [0, N − 1]. For simplicity, we assume that the parameters are constant during the above interval and are independent of the frequency-bin index m. In practice, the speech signal can be divided into short time segments and split in frequency into narrow subbands, such that the parameters can be assumed to be constant in each time-frequency region. Furthermore, we generally do not have a direct access to the clean spectral coefficients. However, the expectation-maximization (EM) algorithm [36], [93] can be utilized for solving this problem by iteratively estimating the clean spectral coefficients and the model parameters from the noisy measurements. Let X0N−1 denote the set of clean speech spectral coefficients employed for the model estimation, let H1 denote the set of time-frequency bins where
T the signal is present, and let λ = µ δ φmin denote the vector of unknown parameters. Then, the conditional variance φx|n−1 (n, m) can recursively be calculated from past spectral coefficients X0n−1 by using (11.73) and the parameter vector λ. Hence, for a Gaussian model, the logarithm of the conditional density of X(n, m) given the clean spectral coefficients up to frame n−1 can be expressed as 
 A2x (n, m) log p X(n, m)  X0n−1 ; λ = − − log φx|n−1 (n, m) − log π, φx|n−1 (n, m) (11.75) where (n, m) ∈ H1 . It is convenient to regard the speech spectral coefficients in the first frame (n = 0) as deterministic, with the values of φx|−1 (0, m) in the first frame initialized to their minimal value φmin , and maximize the loglikelihood when conditioned on the first frame (for sufficiently large sample size, the spectral coefficients of the first frame make a negligible contribution to the total likelihood). The log-likelihood conditional on the spectral coefficients of the first frame is given by  
 L(λ) = log p X(n, m)  X0n−1 ; λ . (11.76) (n,m)∈H1 ∩n∈[1,N −1]

Substituting (11.75) into (11.76) and imposing the constraints in (11.74) on the estimated parameters, the ML estimates of the model parameters can be obtained by solving a constrained minimization problem:    A2x (n, m) minimize + log φx|n−1 (n, m) (11.77) ˆmin , µ φx|n−1 (n, m) φ ˆ , δˆ (n,m)∈H1 ∩n∈[1,N −1]

subject to the constraints

170

11 Spectral Enhancement Methods

φˆmin > 0 , µ ˆ ≥ 0 , δˆ ≥ 0 , µ ˆ + δˆ < 1.

(11.78)

For given of the parameters, the sequences of conditional  numerical values  variances φx|n−1 (n, m) can be calculated from (11.73) and used to evaluate the series in (11.77). The result can then be minimized numerically by using the Berndt, Hall, Hall, and Hausman [16] algorithm as in Bollerslev [19]. Alternatively, the function fmincon of the Optimization Toolbox in MATLAB can be used to find the minimum of the constrained nonlinear function of the model parameters, similar to its use within the function garchfit of the GARCH Toolbox.

11.7 Spectral Variance Estimation In this section, we assume that the model parameters µ, δ, and φmin are known, and derive a recursive estimator for the speech spectral variance. The speech variance estimation approach is closely related to the variance estimation approach introduced in [30], [31]. We start with an estimate φˆx|n−1 (n, m) that relies on the noisy observations up to time-frame n − 1, and “update” the variance by using the additional information Y (n, m). Then, the variance is “propagated” ahead in time, following the rational of Kalman filtering, to obtain a conditional variance estimate at time-frame n + 1 from the information available at time-frame n. Assuming an estimate φˆx|n−1 (n, m) for the one-frame-ahead conditional variance of X(n, m) is available, an estimate for φx|n (n, m) can be obtained by calculating its conditional mean given Y (n, m) and φˆx|n−1 (n, m). By definition, φx|n (n, m) = A2x (n, m). Hence,     φˆx|n (n, m) = E A2x (n, m)  φˆx|n−1 (n, m) , Y (n, m) . (11.79) We can write     E A2x (n, m)  φˆx|n−1 (n, m) , Y (n, m) = HSP (n, m)A2y (n, m),

(11.80)

where HSP (n, m) represents the MMSE gain function in the spectral power domain [32]. The specific expression for HSP (n, m) depends on the particular statistical model. For a Gaussian model, the spectral power gain function is given by  1 HSP (n, m) = 2 A2x (n, m) p [X(n, m) | φx (n, m) , φv (n, m) ] dX Ay (n, m)   iSNR(n, m) 1 iSNR(n, m) = + . (11.81) 1 + iSNR(n, m) γ(n, m) 1 + iSNR(n, m) Substituting (11.80) into (11.79), we obtain an estimate for φx|n (n, m) given by

11.7 Spectral Variance Estimation



171




n−1 (n, m)
n−1 (n, m) iSNR 1 iSNR + A2y (n, m)
n−1 (n, m) γ(n, m) 1 + iSNR
n−1 (n, m) 1 + iSNR  2 φˆx|n−1 (n, m)φv (n, m) φˆx|n−1 (n, m)Ay (n, m) = + . φv (n, m) + φˆx|n−1 (n, m) φv (n, m) + φˆx|n−1 (n, m)

φˆx|n (n, m) =

(11.82) Equation (11.82) is the update step of the recursive estimation, since we start with an estimate φˆx|n−1 (n, m) that relies on the noisy observations up to timeframe n−1, and then update the estimate by using the additional information Y (n, m). To formulate the propagation step, we assume that we are given at timeframe n −1 an estimate φˆx|n−2 (n− 1, m) for the conditional variance of X(n− 1, m), which has been obtained from the noisy measurements up to frame n − 2. Then a recursive MMSE estimate for φx|n−1 (n, m) can be obtained by calculating its conditional mean given φˆx|n−2 (n − 1, m) and Y (n − 1, m):     φˆx|n−1 (n, m) = E φx|n−1 (n, m)  φˆx|n−2 (n − 1, m) , Y (n − 1, m) . (11.83) Substituting (11.73) into (11.83), we have     φˆx|n−1 (n, m) = φmin + µ E A2x (n − 1, m)  φˆx|n−2 (n − 1, m) , Y (n − 1, m)   + δ φˆx|n−2 (n − 1, m) − φmin . (11.84) Equation (11.79) implies that     E A2x (n − 1, m)  φˆx|n−2 (n − 1, m) , Y (n − 1, m) = φˆx|n−1 (n − 1, m). Substituting this into (11.84), we obtain   φˆx|n−1 (n, m) = φmin + µ φˆx|n−1 (n − 1, m) + δ φˆx|n−2 (n − 1, m) − φmin . (11.85) Equation (11.85) is called the propagation step, since the conditional variance estimates are propagated ahead in time to obtain a conditional variance estimate at time-frame n from the information available at time-frame n − 1. The propagation and update steps are iterated as new data arrive, following the rational of Kalman filtering. The algorithm is initialized at the first time-frame, say n = 0, with φˆx|−1 (0, m) = φmin for all the frequency bins, m = 0, . . . , M − 1. Then, for n = 0, 1, . . ., the estimate φˆx|n (n, m) is calculated by using the update step (11.82) and φˆx|n (n + 1, m) is subsequently calculated by using the propagation step (11.85).

172

11 Spectral Enhancement Methods

11.7.1 Relation to Decision-Directed Estimation The recursive spectral variance estimator φˆx|n (n, m) is closely related to the decision-directed estimator of Ephraim and Malah [44]. The decision-directed estimator is given by  2  ˆ2 φˆDD x (n, m) = α Ax (n−1, m)+(1−α) max Ay (n, m) − φv (n, m), βφv (n, m) , (11.86) where α (0 ≤ α ≤ 1) is a weighting factor that controls the tradeoff between noise reduction and transient distortion introduced into the signal. The parameter β is a lower bound on the estimated input SNR to further reduce the level of residual musical noise [22]. The decision-directed estimator is particularly useful when combined with the MMSE spectral, or log-spectral, magnitude estimators [22], [44], [45]. It results in perceptually colorless residual noise, but is heuristically motivated and its theoretical performance is unknown due to its highly nonlinear nature. Furthermore, the parameters of the decision-directed estimator have to be determined by simulations and subjective listening tests for each particular setup of time-frequency transformation and speech enhancement algorithm. Since the decision-directed approach is not supported by a statistical model, the parameters are not adapted to the speech components, but are set to specific values in advance. The update step (11.82) of the recursive estimator can be written as
 φˆx|n (n, m) = α(n, m) φˆx|n−1 (n, m) + [1 − α(n, m)] A2y (n, m) − φv (n, m) , (11.87) where α(n, m) is given by φˆx|n−1 (n, m) α(n, m) = 1 −  2 . φv (n, m) + φˆx|n−1 (n, m)

(11.88)

Substituting (11.84) into (11.87) with µ ≡ 1 we have     φˆx|n (n, m) = α(n, m) E A2x (n − 1, m)  φˆx|n−2 (n − 1, m) , Y (n − 1, m)
 + [1 − α(n, m)] A2y (n, m) − φv (n, m) + α(n, m)φmin .(11.89) The expression (11.89) is an alternative practical form of the decision2 directed  estimator, with α replaced by α(n, m), Ax (n − 1, m) replaced  by E A2 (n − 1, m)  φˆx|n−2 (n − 1, m) , Y (n − 1, m) , and instead of the pax

rameter β, which represents a lower bound on the estimated input SNR φˆDD x (n, m)/ φv (n, m), we have a parameter φmin , which is a lower bound on φˆx|n−1 (n, m). Accordingly, a special case of the recursive estimator with µ ≡ 1 degenerates to a “decision-directed” estimator with a time-varying frequencydependent weighting factor α(n, m). It is interesting to note that the weighting factor α(n, m), given by (11.88), is monotonically decreasing as a function of the one-frame-ahead conditional

11.7 Spectral Variance Estimation

173

SNR (dB)

20 10 0 −10 0

10

20

30

40

50

60

time-frame n Fig. 11.3. SNR in successive time-frames: instantaneous input SNR, γ(n, m) (dotDD
ted line); decision-directed estimate of the input SNR, iSNR (n, m) (dashed line);
n (n, m) (solid line). and recursive estimate of the input SNR, iSNR


n−1 (n, m) = φˆx|n−1 (n, m)/ φv (n, m). A decision-directed esinput SNR, iSNR timator with a larger weighting factor is indeed preferable during speech absence (to reduce musical noise phenomena), while a smaller weighting factor is more advantageous during speech presence (to reduce signal distortion) [22]. The above special case of the recursive estimator conforms to such a desirable behavior. Moreover, the general form of the recursive estimator provides an additional degree of freedom for adjusting the value of µ in (11.85) to the degree of spectral nonstationarity. This may produce even further improvement in the performance. The different behaviors of the recursive estimator φˆx|n (n, m) (11.82) and the decision-directed estimator φˆDD x (n, m) (11.86) are illustrated in the example of Fig. 11.3. The analyzed signal contains only white Gaussian noise during the first and last 20 frames, and in between it contains an additional sinusoidal component at the displayed frequency with 0 dB SNR. The signal is transformed into the STFT domain using half overlapping Ham
n (n, m) = φˆx|n (n, m)/ φv (n, m) and ming windows. The SNR estimates, iSNR DD
(n, m) = φˆDD iSNR x (n, m)/ φv (n, m), are obtained by using the parameters β = −25 dB, α = 0.98, µ = 0.9. The signal estimate Z(n, m) is recursively obtained by applying HLSA (n, m) to the noisy spectral measurements [see (11.67)]. Figure 11.3 shows that when the instantaneous SNR γ(n, m) is sufficiently low, the recursive input SNR estimate is smoother than the decision-directed estimate, which helps reducing the level of musical noise. When γ(n, m) inDD
n (n, m) is initially slower than iSNR
(n, m), creases, the response of iSNR but it then builds up faster to the instantaneous SNR. When γ(n, m) is sufDD
(n, m) follows the instantaneous SNR with a delay of ficiently high, iSNR
n (n, m) follows the instantaneous SNR without delay. 1 frame, whereas iSNR

174

11 Spectral Enhancement Methods


n (n, m) is immediate, while When γ(n, m) decreases, the response of iSNR DD
(n, m) is delayed by 1 frame. As a consequence, when comthat of iSNR pared with the decision-directed estimator, the recursive estimator produces a lower level of musical noise while not increasing the audible distortion in the enhanced signal [31].

11.8 Summary of Spectral Enhancement Algorithm In this section, we present an example of a speech enhancement algorithm, which is based on an MMSE log-spectral amplitude estimation under a Gaussian model, improved minima-controlled recursive averaging (IMCRA) noise estimation [28], and recursive estimation of the input SNR. The performance of the algorithm is demonstrated on speech signals degraded by various additive noise types. The implementation of the speech enhancement algorithm is summarized in Table 11.1. For each time-frame n we recursively estimate the STFT coefficients of the clean speech {X(n, m) | m = 0, . . . , M − 1} from the noisy STFT coefficients {Y (n, m) | k = 0, . . . , M − 1}, where M is the length of the analysis window. We typically use a Hamming window of 32 ms length and a framing step of 8 ms (i.e., M = 512 and L = 128 for a sampling rate of 16 kHz). In the first frame (n = 0) we compute {Y (0, m) | m = 0, . . . , M − 1} by applying the discrete Fourier transform to a short-time section of the noisy data yψ (0) = [y(0)ψ(0) y(1)ψ(1) · · · y(M − 1)ψ(M − 1)]T , where ψ(n) is the analysis window. In the following frames (n > 0), the section of noisy data is updated with L additional samples yψ (n) = [y(nL)ψ(0)

y(1+nL)ψ(1)

···

y(M −1+nL)ψ(M −1)]T (11.90)

and subsequently {Y (n, m) | m = 0, . . . , M − 1} is computed by applying the discrete Fourier transform to yψ (n). Since the speech signal x(k) is assumed real, once we have spectral coefficients {Z(n, m) | m = 1, . . . , (M − 1)/2}, the spectral coefficients for (M − 1)/2 < m ≤ M − 1 are obtained by Z(n, m) = Z ∗ (n, M − m). The DC component Z(n, 0) is set to zero, and a sequence {zn (k) | k = 0, . . . , M − 1} is obtained by applying the inverse discrete Fourier transform to {Z(n, m) | m = 0, . . . , M − 1}: zn (k) =

M −1 1  2π Z(n, m) ej M km . M m=0

(11.91)

Employing the weighted overlap-add method [35], we compute the following sequence

11.8 Summary of Spectral Enhancement Algorithm

175

Table 11.1. Summary of a spectral enhancement algorithm. Initialization at the first time-frame n = 0: φˆv (0, m) = A2y (0, m); φˆx|−1 (−1, m) = φmin , for all m; Read M − L samples of the noisy signal y(k). For all time-frames n = 0, 1, . . . Read L new samples of the noisy signal, and compute yψ (n) using (11.90). Compute {Y (n, m) | m = 0, . . . , M − 1} by applying the discrete Fourier transform to yψ (n). For frequency-bins m = 1, . . . , (M − 1)/2 Update the variance of the noise spectral coefficient φˆv (n, m) using [28]. Compute φˆx|n−1 (n, m) using the propagation step of the recursive variance estimation (11.85). Compute φˆx|n (n, m) using the update step of the recursive variance estimation (11.82). ˆ m) using
Compute iSNR(n, m) = φˆx|n (n, m)/ φˆv (n, m), and compute ϑ(n, (11.62). Compute the speech spectral estimate Z(n, m) using (11.67). Let Z(n, 0) = 0, and let Z(n, m) = Z ∗ (n, M − m) for m = (M + 1)/2, . . . , M − 1. Apply the inverse discrete Fourier transform to {Z(n, m) | m = 0, . . . , M − 1}, and compute L new samples of the enhanced speech signal using (11.91)–(11.93).

$ on (k) =

˜ zn (k) , for 0 ≤ k ≤ M − L − 1 on−1 (k + L) + M ψ(k) ˜ zn (k) , M ψ(k)

for M − L ≤ k ≤ M − 1

, (11.92)

˜ where ψ(k) is the synthesis window. Then, according to (11.3), for each timeframe n, we obtain L additional samples of the enhanced speech signal: z(k + nL) = on (k) ,

k = 0, . . . , L − 1.

(11.93)

˜ The synthesis window ψ(k) should satisfy the completeness condition [124]  n

˜ − nL) ψ(k − nL) = 1 ψ(k M

for all k.

(11.94)

Given analysis and synthesis windows that satisfy (11.94), any signal x(k) ∈ 2 (Z) can be perfectly reconstructed from its STFT coefficients X(n, m). However, for L < M (over-redundant STFT representation) and for a given analysis window ψ(k), there might be an infinite number of solutions to (11.94).

176

11 Spectral Enhancement Methods 40

oSNR (dB)

30 20 10 White Gaussian noise Car interior noise F16 cockpit noise Babble noise

0

−10 −5 30

(a)

0

5

10

15

20

White Gaussian noise Car interior noise F16 cockpit noise Babble noise

25

10 log (ξnr )

20 15 10 5 0 −5 −2

(b)

0

5

10

20

White Gaussian noise Car interior noise F16 cockpit noise Babble noise

−4

10 log (υsd )

15

−6 −8

−10 −12 −14 −5

(c)

0

5

10

15

20

iSNR (dB) Fig. 11.4. Performance of the spectral enhancement algorithm in Table 11.1 in different input SNR conditions.

A reasonable choice of a synthesis window is the one with minimum energy [102], [124], given by ψ(k) ˜  2 ψ(k) = . (11.95) M  ψ (k − L)

11.9 Experimental Results The speech signals used in our evaluation are taken from the TIMIT database [57]. They include 20 different utterances from 20 different speakers, half male and half female. The speech signals are sampled at 16 kHz and degraded by

11.9 Experimental Results

177

Amplitude

0.5

0

−0.5 0

0.5

1

1.5

2

2.5

3

Frequency (kHz)

8

1 0.8

6

0.6 4 0.4 2

0.2

0

0 0

0.5

1

1.5

2

2.5

3

Time (s)

Fig. 11.5. Waveform and spectrogram of a clean speech signal: “Higher toll rates also are helping boost revenues.”

various noise types from the Noisex92 database [118], which include white Gaussian noise, car interior noise, F16 cockpit noise, and babble noise. The input SNR is in the range [−5, 20] dB. The average fullband output SNR, noise-reduction factor, and speech-distortion index obtained by the applying the spectral enhancement algorithm to the noisy signals are displayed in Fig. 11.4. The results show that the fullband output SNR, oSNR(H), is larger than the fullband input SNR in all the tested conditions. However, the increase in SNR depends on the input SNR level and on the type of background noise. The increase in SNR is larger for lower input SNR levels and for noise characteristics which are different than the speech signal. The faster the noise spectrum varies in time, the less reliable is the noise spectrum estimator, and consequently the lower is the increase in SNR that can be achieved by the spectral enhancement algorithm. For car interior noise, most of the noise energy is concentrated in the lower frequencies, and its characteristics change slowly in time compared to the speech signal. Therefore, the output SNR and noise-reduction factor can be large, while keeping the speech-distortion index small. On the other hand, the characteristics of babble noise are similar to speech signals and the variations in time are faster, when compared to the other noise types. Therefore, the noise spectrum estimator is least reliable for babble noise and the speech enhancement performance is inferior to that achievable in the other noise environments. In particular, the noise-reduction factor is smallest for babble noise and largest for car interior noise, while the

178

11 Spectral Enhancement Methods 0.5

Amplitude

0

−0.5 0.5

0

−0.5 0

0.5

1

1.5

2

2.5

3

8

1 0.9

6

0.8 4 Frequency (kHz)

0.7 2 0.6 0 8

0.5 0.4

6 0.3 4 0.2 2

0.1

0

0 0

0.5

1

1.5

2

2.5

3

Time (s)

Fig. 11.6. Waveforms and spectrograms of the noisy and enhanced speech in white Gaussian noise environment with fullband iSNR = 5 dB (oSNR = 16.5 dB, ξnr = 12.1 dB, υsd = −12.4 dB).

speech-distortion index is smallest for car interior noise and largest for babble noise. A clean utterance from a female speaker is shown in Fig. 11.5. Figures 11.6– 11.9 show its noisy versions in different environments (with input SNR of 5 dB) and the enhanced speech signals. The enhanced speech signals are obtained by using the spectral enhancement algorithm in Table 11.1. The fullband input SNR is the same for all the signals shown in Figs. 11.6–11.9, but the subband input SNR in the STFT domain, iSNR(n, m), varies significantly depending on the type of background noise. The noise spectrum is uniform in time and frequency for the white Gaussian noise. Hence, in time-frequency bins where

11.9 Experimental Results

179

0.5

Amplitude

0

−0.5 0.5

0

−0.5 0

0.5

1

1.5

2

2.5

3

8

1 0.9

6

0.8 4 Frequency (kHz)

0.7 2 0.6 0 8

0.5 0.4

6 0.3 4 0.2 2

0.1

0

0 0

0.5

1

1.5

2

2.5

3

Time (s)

Fig. 11.7. Waveforms and spectrograms of the noisy and enhanced speech in car interior noise environment with fullband iSNR = 5 dB (oSNR = 16.3 dB, ξnr = 11.6 dB, υsd = −17.5 dB).

the speech spectral variance φx (n, m) is high, the spectral gain H(n, m) is closer to 1, in order to reduce the speech-distortion index. In time-frequency bins where the speech spectral variance φx (n, m) is low, the spectral gain H(n, m) is closer to 0, in order to increase the noise-reduction factor. For car interior noise, one can apply a significant noise reduction in the lower frequencies and subtle noise reduction in the higher frequencies, since most of the noise energy is concentrated in the lower frequencies. Hence, the subband noise-reduction factors are large in the lower frequencies and small in the higher frequencies, while the subband speech-distortion indices are small in the higher frequencies and large in the lower frequencies. The fullband noise-

180

11 Spectral Enhancement Methods 0.5

Amplitude

0

−0.5 0.5

0

−0.5 0

0.5

1

1.5

2

2.5

3

8

1 0.9

6

0.8 4 Frequency (kHz)

0.7 2 0.6 0 8

0.5 0.4

6 0.3 4 0.2 2

0.1

0

0 0

0.5

1

1.5

2

2.5

3

Time (s)

Fig. 11.8. Waveforms and spectrograms of the noisy and enhanced speech in F16 cockpit noise environment with fullband iSNR = 5 dB (oSNR = 14.7 dB, ξnr = 10.7 dB, υsd = −12.7 dB).

reduction factor would generally be large, while keeping the fullband speechdistortion index low. For babble noise, the noise spectrum varies significantly in time and frequency. Hence, to restrict the fullband speech-distortion index to be smaller than a certain threshold, one has to restrict the spectral gain H(n, m) to larger values, and thus restrict the fullband noise-reduction factor to smaller values. This generally yields higher residual noise levels than those achieved in other noise environments.

11.10 Summary

181

0.5

Amplitude

0

−0.5 0.5

0

−0.5 0

0.5

1

1.5

2

2.5

3

8

1 0.9

6

0.8 4 Frequency (kHz)

0.7 2 0.6 0 8

0.5 0.4

6 0.3 4 0.2 2

0.1

0

0 0

0.5

1

1.5

2

2.5

3

Time (s)

Fig. 11.9. Waveforms and spectrograms of the noisy and enhanced speech in babble noise environment with fullband iSNR = 5 dB (oSNR = 8.8 dB, ξnr = 4.4 dB, υsd = −16.0 dB).

11.10 Summary In this chapter, we formulated the noise reduction problem in the STFT domain and the derived MMSE, MMSE-SA and MMSE-LSA estimators for the speech spectral coefficients. The estimators rely on Gaussian statistical models for the speech and noise signals. The Gaussian model for the speech signal is integrated with a GARCH model for the spectral variances, thus enabling to take into account time-correlation between successive spectral components of the speech signal. We showed that the resulting recursive spectral variance estimator is closely related to the decision-directed estimator of Ephraim and

182

11 Spectral Enhancement Methods

Malah. We described the beneficial behaviors of the MMSE-SA and MMSELSA estimators, which enable the mechanism that counters the musical noise phenomenon. Finally, we have demonstrated the design of a spectral enhancement algorithm and discussed its performance in different noisy environments.

12 A Practical Example: Multichannel Noise Reduction for Voice Communication in Spacesuits

Noise reduction can find numerous applications in various fields of speech processing. Since acoustic environments and technical objectives vary from application to application, it is impossible (at least impossible at the present time) to find a universally feasible approach. In the design of a microphone array system and the choice of a well-suited noise reduction algorithm, there is no simple rule to follow and experiences are still crucial. In this chapter, we use a practical example to explain the primary considerations that need to be kept in mind. We think that what can be interesting to the readers is not the finding of the best noise reduction algorithm for the specific problem under investigation but instead, it is the analysis of the acoustic challenges and the validation procedure of the suggested solutions that can be more useful and that can enrich the reader’s fund of knowledge.

12.1 Problem Description Collaboration and cooperation between the crewmembers in space and the mission control center on the earth are the lifeline of astronauts and space shuttles. Clear and reliable voice communications are essential to astronaut safety and the success of every NASA flight mission. But widely varying working conditions of a space shuttle and the special design of an astronaut’s spacesuit form an extreme acoustic environment that imposes unique challenges for capturing and transmitting speech communications to and from a crewmember, as summarized in, but not limited to, the followings [2]. • Noise heard in the spacesuit has complicated characteristics in either the temporal, spectral, or spatial domain: generally non-stationary, inherently wideband, and possibly either directional or dispersive. In addition, during launch, entry descent, and landing, ambient noise penetrating from outside the spacesuit is at a very high level, while during on-orbit and Extravehicular Activity (EVA) operations suit-borne noise makes it difficult to achieve an adequate SNR for satisfactory voice communication.

J. Benesty et al., Noise Reduction in Speech Processing, Springer Topics in Signal Processing 2, DOI 10.1007/978-3-642-00296-0_12, © Springer-Verlag Berlin Heidelberg 2009

184

12 Practical Example

• The helmet of a spacesuit is made of highly reflective materials, which create a very reverberant in-suit acoustic environment. Strong reverberation will severely distort the speech uttered by the suit subject and hence remarkably reduce its intelligibility. Moreover, strong reverberation leads to a more dispersive noise field, which makes beamforming less effective. • Spacesuits can operate at several non-standard static pressures during EVA: specific static pressure levels include 30, 50, 55, 100, 101 kPa. These changes in static pressure level will alter the density of the medium and hence the speed of propagating acoustic waves. Varying and indeterminate acoustic transduction adds another layer of difficulty to our efforts to model the acoustic environment for combating noise and reverberation. The current solution is a communication cap-based audio (CCA) system. As shown in Fig. 12.1, astronauts wear such a cap under their helmets. It has redundant, built-in microphones and earphones. The use of close-talking, noise cancelling microphones of a CCA can dramatically help improve speech intelligibility, but only when the microphones are very close to the crewmember’s mouth. Moving them away from the mouth, even though only for a very short distance, can cause great performance degradation. Such sensitivity gives arise to a number of recognized logistical issues and inconveniences that cannot be resolved with incremental improvements to the basic design of the CCA systems: • the communication cap and the microphone booms cannot be adjusted during EVA operations (which last from four to eight hours) or during launch and entry, • the microphones that are right next to the mouth can interfere in the suit subject from eating and drinking and, on the other hand, can easily be contaminated by food and drinks, • since the communication cap needs to be individually tailored and the microphones need to be properly mounted according to the head size of each astronaut, caps in a number of different sizes have to be built and maintained (e.g., there are five sizes for Space Shuttle and International Space Station Extravehicular Mobility Unit CCA), and • wire fatigue and blind mating of the connectors are also problems with the CCA. Therefore a great effort is under way in NASA to develop a multiplemicrophone audio system that is integrated with spacesuits and would be able to possess similar performance to a CCA. Innovations in beamforming and multichannel acoustic signal processing technologies are then solicited to improve in-helmet voice communication with better experiences. We (the research team of WeVoice, Inc.) were selected for a NASA Phase I SBIR (small business innovation research) award and were funded to carry out a 6-month pilot feasibility study in 2008.

12.1 Problem Description Skullcap Earphone

185

Perspiration Absorption Area

Acoustical Tube Tube Outlet to Ear Helmet Helmet Ring

Chin Cup

Cable/Connector carrying input and output audio data

Noise Canceling Microphone Microphone Boom

(a)

(b) Fig. 12.1. Illustration of the current communication cap-based audio (CCA) system in a spacesuit helmet: (a) schematic of the CCA system and (b) an example of the CCA system worn by a female astronaut.

186

12 Practical Example

interconnect wiring nylon/spondex top teflon sidepiece and pocket

ear cup

microphone ear seal

interface cable and connector

microphone (a)

(b)

(c)

Fig. 12.2. Three currently used NASA communication caps: (a) shuttle/international space station extravehicular mobility unit (EMU) cap, (b) advanced crew escape suit (ACES) cap, and (c) developmental cap.

12.2 Problem Analysis 12.2.1 Sources of Noise in Spacesuits There are two forms of noise in spacesuits: ambient noise that penetrates from the outside into the inside of the spacesuits, and suit structure-borne noise. The level of ambient noise is subject to a large range of variations, depending on the operation that the space shuttle/vehicle and the suit subject are undertaking. During launch, entry descent, and landing, the sound pressure level (SPL) of impulse ambient noise is < 140 dB. But during on-orbit and EVA operations, the surrounding environment of a crewmember is quiet. The

12.2 Problem Analysis

187

Table 12.1. Upper limits on continuous on-orbit noise level by frequency. Band Center Frequency (Hz) 63 125 250 500 1k 2k 4k 8k 16k Sound Pressure Level (dB) 72 65 60 56 53 51 50 49 48 Table 12.2. Sound pressure level (SPL) chart of typical noise fields. SPL (dB)

Perception

85 – 95

Very High Noise:

Construction Site

Speech Almost

Loud Machine Shop

Impossible to Hear

Noisy Manufacturing

High Noise:

Assembly Line

75 – 85

Typical Environments

Speech is Difficult Crowded Bus/Transit Waiting Area

65 – 75

55 – 65

to Hear

Very Noisy Restaurant/Bar

Medium Noise:

Department Store

Must Raise Voice

Bank/Public Area

to be Heard

Supermarket

Low Noise:

Doctor’s Office

Speech is Easy

Hospital

to Hear

Hotel Lobby

upper limits on continuous on-orbit noise level by frequency are shown in Table 12.1. By comparing these SPL values to those of typical noise fields given in Table 12.2, we see that the crewmembers talk and listen as if they were at best in a doctor’s office and at worst in a supermarket. Therefore, during EVA operations, ambient noise is at most a minor problem, and structure-borne noise is more imperative to be solved for in-helmet voice communication. Four sources have been identified for the structure-borne noise in spacesuits [2]: • airflow and air inlet hissing noise, as well as fan/pump noise due to required air supply and circulation, • arm, leg, and hip bearing noise, • suit-impact noise (e.g., footfall), and • swishing-like noise due to air movement caused by walking (since the suits are closed pressure environments). For head-mounted CCA systems, since the suit subject’s body does not transmit bearing and impact noise, only airflow-related noise needs to be properly controlled. However, for an integrated audio system, structure-borne vibration

188

12 Practical Example

easily adds on the microphone outputs since the microphones are mounted directly on the suit structure. 12.2.2 Noise Cancelling Microphones Noise cancelling microphones are nothing mysterious but close-talking differential microphone arrays. A differential microphone consists of an array of pressure sensors which are exposed to the incident sound. The microphone is therefore responsive to the spatial derivatives (gradients) of the scalar acoustic pressure field. Its output is a finite-difference approximation to the sum of those spatial derivatives. The response of a first-order differential microphone array (FODMA) is the combination of the zeroth-order signal and the first-order spatial derivative, while an ith-order (i ≥ 1) array has a response proportional to a linear combination of signal derived from spatial derivatives up to, and including order i. An FODMA, as depicted in Fig. 12.3(a), consists of two omnidirectional sensor elements with inter-element spacing d. The (zero-order) responses of the two omnidirectional microphones due to a sound source at a distance of r as a function of frequency f are Hn (rn ; f ) =

e−j2πf rn /c , n = 1, 2, rn

(12.1)

where r1 = r2 =

! !

r2 − rd cos θ + d2 /4, r2 + rd cos θ + d2 /4,

θ is the incident angle of the sound source with respect to the sensor axis, and c is the speed of sound whose value (in m/s) in air (similarly in pure oxygen, which is diatomic gas) can be calculated from the air temperature tair (in degrees Celsius) using the following formula ) tair c ≈ 331.3 × 1 + . (12.2) 273.15 Then the FODMA response can be written as HFODMA (r, θ; f ) = H1 (r1 ; f ) − H2 (r2 ; f ) =

e−j2πf r1 /c e−j2πf r2 /c − . r1 r2

(12.3)

Similarly the response of a second-order differential microphone array (SODMA), as shown in Fig. 12.3(b), is deduced as HSODMA (r, θ; f ) =

e−j2πf r1 /c e−j2πf r2 /c e−j2πf r3 /c −2 + , r1 r2 r3

(12.4)

12.2 Problem Analysis

Zero-order outputs Zero-order outputs

1

+ −

First-order outputs

189

1

+ −

+ −

2

+ −

2 FODMA Output SODMA Output

(a)

3

(b)

Fig. 12.3. Schematics of (a) a first-order differential microphone array (FODMA) and (b) a second-order differential microphone array (SODMA).

where

! r1 = r2 − 2rd cos θ + d2 , r2 = r, ! r3 = r2 + 2rd cos θ + d2 .

Figure 12.4(a) plots the FODMA and SODMA responses to an on-axis sound source (θ = 0◦ ) evaluated at r = 50 mm and 1 m for d = 10 mm. The farfield responses indicate a highpass filter with a slope of 6i dB/octave for an ithorder (i = 1, 2) differential microphone array. Consider two on-axis sound sources: one is the sound source of interest in the near field at a distance of rs , and the other is a noise source in the far field at a distance of rn . Then the gain in signal-to-noise ratio (GSNR) by using a differential microphone array (DMA) over using an omnidirectional microphone is found as   |HDMA (rs , θs ; f )| rn GSNRDMA = 20 log10 − 20 log10 , (12.5) |HDMA (rn , θn ; f )| rs where DMA can be either FODMA or SODMA. Figure 12.4(b) presents the GSNRs for rs = 15 mm, θs = θn = 0◦ , and d = 10 mm. It is clearly demonstrated that differential microphones inherently suppress farfield noise. The higher the order of a DMA and the larger the distance of a noise source, the more attenuation is gained against the noise. Moreover, the farfield noise signal is more suppressed in the low frequencies than in the high frequencies. While differential microphones are found very useful in situations where the background noise level is very high (like what exist in spacesuits), a well-

190

12 Practical Example

Differential Microphone Response (dB)

−20

−40

FODMA:

mm

SODMA:

mm

−60

−80

−100

FODMA:

mm

SODMA:

mm

−120

mm

−140 2 10

3

10 Frequency (Hz)

10

4

(a) 50 45 SODMA:

mm

mm

mm

40

Gain in SNR (dB)

35 30 25 20

SODMA: FODMA:

mm mm

15

FODMA:

mm

10 5 0 2 10

3

10 Frequency (Hz)

4

10

(b) Fig. 12.4. Performance comparison of an FODMA and an SODMA for near-field speech acquisition. (a) Differential microphone array response and (b) gain in SNR over using an omnidirectional microphone.

12.2 Problem Analysis

191

0 −10

SODMA Response (dB)

−20 −30 −40 −50 −60 −70 −80 2 10

3

10 Frequency (Hz)

10

4

(a)

(b) Fig. 12.5. Illustration of SODMA’s sensitivity to (a) the range and (b) the incident angle of the near-field sound source of interest.

192

12 Practical Example

known, prominent drawback of differential microphone arrays is that their frequency response and level are extremely sensitive to the position and orientation of the arrays relative to the speaker’s mouth. Figure 12.5 visualizes the sensitivity of an SODMA’s response to r and θ. The inter-element spacing is d = 10 mm. Suppose that the microphone is initially placed 15 mm away from the mouth. When we move it only 45 mm (which is significantly less than an average distance between a man’s mouth and one of his ears) further away, as seen from Fig. 12.5(a), the low-frequency speech components (between 100 Hz and 1 kHz) are attenuated more than 40 dB. At such a distance with such a level of attenuation, the captured speech signal becomes very weak and submerged in the background noise. Very poor speech intelligibility can then be expected and noise cancelling microphones no longer work as their name implies. In addition, we can see that the farther we move the microphone away from the mouth, the more attenuation is applied to the high-frequency components of the speech signal. This will not only reduce its intelligibility but also lower its quality. If the distance between the microphone and the mouth is fixed but the mouth deviates from the axis of the differential microphone, then from Fig. 12.5(b) we see that the response also changes remarkably. Since differential microphones are supposedly in close proximity to a user’s mouth, a small displacement of the mouth can lead to a large change in the incident angle with respect to the array axis. The sensitivity of a differential microphone to dislocation and disorientation makes it necessary to perform frequency and level equalization to its response according to the range and incident angle of its user’s mouth [116]. However, this equalization will not affect the SNR and the directional response pattern. In the axis direction (θ = 0◦ ), the response is always the largest. In other words, the main lobe cannot be electrically steered to the mouth. Therefore, it can be concluded that noise cancelling microphones will not be used in the next-generation voice communication systems integrated with spacesuits.

12.3 Suggested Algorithms We have learned that noise inside the helmet of a spacesuit has complicated characteristics. In addition, because of the presence of strong reverberation, the suit subject’s speech and noise signals are mixed in a complex way in microphone outputs. To extract clean speech with high intelligibility and quality, noise reduction and speech enhancement will be explored in three domains: the time, frequency, and space domains. Hopefully a line can be drawn between noise and the interested speech signal. As illustrated by Fig. 12.6, we thought that there could be four techniques that are potentially helpful and deserve to be carefully studied: namely beamforming, multichannel noise reduction, adaptive noise cancellation, and single-channel noise reduction. As will be explained below, these algorithms (except beamforming and multichannel noise reduction) are not mutually exclusive, but in fact complementary and should

12.3 Suggested Algorithms

Mouth Range and Incident Angle with respect to the Microphone Array

Head Position Calibration Acoustic Source Localization

Head Motion Tracker 4 3 2

5 1

+ −

193

+ −

+ −

+ −

Beamforming

Multichannel Noise Reduction

Single−Channel Noise Reduction

Speech out to the Far End Adaptive Noise Cancellation

Noise Reference

Fig. 12.6. Block diagram of the suggested algorithms to be used in the integrated spacesuit audio (ISA) system for in-helmet voice communication.

be used in concert. In general spatial cues are more vulnerable to improper processing than temporal/spectral cues. Consequently spatial processing will be carried out prior to time-frequency analysis. 12.3.1 Nearfield, Wideband Microphone Array Beamforming for Speech Acquisition in Spacesuits Beamforming is a means of spatial filtering, which lets the signal coming from the pre-specified look direction pass through while suppressing signals propagating from other directions. The origin and the first implementation of the idea can be traced a long way back to a century ago. Nowadays one can find various beamformers in the literature. But their structures are not significantly different. The general structure of a beamformer is shown in Fig. 12.7. When the speech source of interest is in the far field of the microphone array, its corresponding time difference of arrival (TDOA) between the first and the nth (n = 2, 3, . . . , N , where N is the number of microphones) microphone outputs τn is dependent only on the incident angle θ of the speech source. But when the speech source is in the near field, the TDOA τn depends on the position of the speech source rs .

194

12 Practical Example

(b) Fig. 12.7. General structure of a beamformer when the speech source of interest is in (a) the far field and (b) the near field of the microphone array.

12.3 Suggested Algorithms

195

There are two parameters to tune for a beamformer: the TDOAs τn (rs ) and the filter coefficients hn . The TDOAs specify the look direction (or more precisely the look position in the nearfield case) and the filter coefficients control the beam pattern of the array: the beamwidth of the mainlobe, the characteristics of the sidelobes, and the position of the nulls. For a delay-andsum beamformer (known as the classical beamforming algorithm), hn = 1/N (∀n = 1, 2, . . . , N ). As a result, speech signal originating from the desired direction are summed in phase, while other signals undergo destructive interference. If noise is incoherent, the improvement in SNR would be 10 log10 N dB. When N is small, only a very limited gain in SNR can be obtained, which is insufficient and practically not so useful. For coherent noise, the performance of the delay-and-sum beamformer is strongly dependent on the direction of arrival of the noise signal. If the direction of noise coincides with that of speech signal, no SNR gain can be possibly produced. Since the microphone weights of the classical delay-and-sum beamformer are constants and frequency independent, its spatial response is a function of frequency, as shown in Figs. 12.8 and 12.9. As the frequency increases, the beam becomes commensurately narrower. This implies that when the speech source deviates from the look direction, speech distortion will be observed in the beamformer output. The high-frequency components are more attenuated than the low-frequency components, i.e., the speech signal will be low-pass filtered. In order to have a constant-beamwidth spatial response over a wide frequency range of speech spectrum, harmonically-nested subarrays [51], [52], [83] and the more generalized filter-and-sum beamformer [77], [115] were proposed. Harmonically-nested subarrays are large in size and require a great number of microphones. For integrated spacesuit audio (ISA) systems, a filter-and-sum beamformer is practically more useful. In the design of a fixed filter-and-sum beamformer, the microphone weights can be computed using a least-squares (LS) filter design method [94], [95], which can yield satisfactory results [15] as exemplified by the far-field response of a filter-and-sum beamformer visualized in Fig. 12.10 in comparison with that of a delay-and-sum beamformer previously shown in Fig. 12.8. But when a nearfield speech source is concerned, the LS algorithm was found not as stable as an LCMV (linearly constrained minimum variance) algorithm. Spatial responses of fixed beamformers are static since the microphone weights, once designed, are fixed. Fixed beamformers are data-independent spatial filters. During the design of a fixed beamformer, noise field is not known and the isotropic model that is a reasonable first-order approximation of most of the real noise fields is commonly used. If the assumption matches the acoustic condition in which the array is used, this approach leads to a good beamformer design. However, such a simple model can never be real in practice and therefore a fixed beamformer is at best suboptimal even though it can be steered to accurately follow the movement of the speech source. For example, if noise is coming from a point sound source, ideally the beamformer

196

12 Practical Example

Beamformer Response (dB)

0 −10 −20 −30 −40 0 1000 Fr eq 2000 ue nc 3000 y( Hz 40000 )

30

60

120

90

150

180

dence (deg.)

Angle of Inci (a) N = 4

Beamformer Response (dB)

0 −10 −20 −30 −40 0 1000 Fr eq 2000 ue nc 3000 y( Hz 40000 )

30

60

120

90

150

180

dence (deg.)

Angle of Inci (b) N = 8

Fig. 12.8. Response of a delay-and-sum beamformer for two equally spaced linear arrays containing respectively (a) N = 4 and (b) N = 8 microphones spaced by d = 20 mm and with the look direction θ = 90◦ . The sound sources are assumed in the far field.

12.3 Suggested Algorithms

197

Fig. 12.9. Directional patterns of a delay-and-sum beamformer for two equally spaced linear arrays containing respectively N = 4 (red solid line) and N = 8 (blue dashed line) microphones spaced by d = 20 mm and with the look direction θ = 90◦ . The sound sources are assumed in the far field.

should be able to put a null in that direction. However, only by chance a fixed beamformer can do so. In this line of thoughts, adaptive beamformers were proposed. Adaptive beamformers try to track the variation of the surrounding noise field and adaptively search for the optimum location of the nulls that can most significantly reduce noise under the constraints that the interested speech signal is not distorted at the beamformer output. This is then formulated as the widely-known linearly constrained minimum variance (LCMV) problem. The

198

12 Practical Example

Beamformer Response (dB)

0 −10 −20 −30 −40 0 1000 Fr eq 2000 ue nc 3000 y( Hz 40000 )

30

60

120

90

150

180

dence (deg.)

Angle of Inci

Fig. 12.10. Far-field response of a filter-and-sum beamformer for an equally spaced linear array containing N = 4 microphones spaced by d = 20 mm and with the look direction θ = 90◦ . In the design of such a beamformer with the least-squares method, the width of the mainlobe was set as 40◦ .

difficulty of implementing an adaptive beamformer is how to properly define the constraint. In an anechoic environment, only the direction of the speech source needs to be known and the LCMV solution leads to the minimum variance distortionless response (MVDR) beamformer due to Capon [21]. But in a real reverberant environment, the linear constraint of the LCMV algorithm requires the knowledge of the impulse responses from the speech source to each microphone [13]. The LCMV beamformer was first developed by Frost [53] and is also called the Frost beamformer. One variant of the Frost beamformer is the so-called generalized sidelobe canceller (GSC) proposed by Griffiths and Jim [61]. The GSC transforms the LCMV algorithm from a constrained optimization problem into an unconstrained form. Therefore, the GSC and LCMV beamformers are essentially the same while the GSC can lower the computational cost by forcing the constraint into the front-end of the array processing [79], [123]. The LCMV beamformer is theoretically appealing since it seems to be an optimal spatial filter. But in practice, when an MVDR beamformer is used in a reverberant environment or when the estimates of the acoustic impulse responses in the Frost/GSC beamformer have errors (which is certainly inevitable), the so-called signal cancellation problem would occur and only a suboptimal performance can be achieved.

12.3 Suggested Algorithms

199

Our perception of speech remarkably benefits from spatial hearing since it can be easily experienced by observing the difference in understanding between using both ears and with either ear covered when listing in an enclosed space where there are multiple (both speech and noise) sound sources at the same time. This is a striking capability since we accomplish it with only two ears and without much efforts. Consequently some people may mistakenly believe that a small array of only two microphones would be able to give us quite some gain in SNR via beamforming. One needs to understand that spatial filtering is only a part, rather than the entirety, of the cocktail-party effect of human auditory systems. With a man-made microphone array system, we summarize that • for incoherent noise sources, the SNR gain is low if a small number of microphones are used; • for coherent noise sources whose directions are different from that of the speech source, a theoretically optimal gain in SNR can be high but is difficult to obtain due to practical limitations (e.g., unavailability of a precise a priori knowledge of the acoustic impulse responses from the speech sources to the microphones, inconsistent responses of the microphones across the array, etc.); and • for coherent noise sources that are in the same direction as the speech source, beamforming (as a spatial filter) is ineffective. One way to achieve a higher noise reduction gain at a price of some speech distortion is to replace the hard constraint in the LCMV beamformer with a soft constraint [81], [113], which becomes nonlinear and allows for some deviation in magnitude and phase for signals propagating from the look direction. The other way is to have a post-filter at the beamformer output [80], [127], [128]. The idea of sequential beamforming and post-filtering is based on the discovery that the multichannel Wiener filter, which minimizes the MSE in the context of speech extraction with a microphone array, can be factorized as a product of the LCMV beamformer and a single-channel Wiener filter for noise reduction [111]. The former approach involves a nonlinear optimization problem and a gradient search has to be employed. In addition, the amount of speech distortion is hard to adjust. It is the post-filtering technique which has attracted an increased interest and has achieved some successes. In Section 12.3.3 that follows, we will discuss various single-channel noise reduction algorithms for post-filtering. An essential requirement of microphone array beamformers, either fixed or adaptive, is the knowledge about the position of the suit subject’s mouth. This can be met by using acoustic source localization and tracking method or infrared marker-based 3D motion tracking systems. While acoustic methods can make the whole system more compact with no need to add additional hardware, their level of precision is not in the same order of magnitude as that of marker-based 3D motion tracking systems, not even to mention the reliability in the face of strong noise and reverberation. A small marker will be

200

12 Practical Example

placed on the suit subject’s head and would presumably cause little distraction or logistical inconvenience. The marker-based motion tracking system provides the marker’s position relative to the system’s origin. This origin needs to be aligned to the origin of the microphone array. Otherwise the data of the mouth position cannot be used. The alignment can be manually carried out, but an automatic way with the help of acoustic source localization techniques is by all means more preferable. The calibration is performed only once when the spacesuit is put on or when the audio system is rebooted. During this operation mode, the acoustic condition can be well controlled1 and a large amount of acoustic data can be collected. Therefore a high degree of accuracy of acoustic source localization algorithms can be expected. For the state of the art of time delay estimation and acoustic source localization, the interested reader can refer to a recent survey [73] and the references therein. 12.3.2 Multichannel Noise Reduction: a More Practical Microphone Array Signal Processing Technique Performance degradation of an adaptive beamformer in reverberant environments is a well-known problem due to the unavailability of an accurate a priori knowledge of the acoustic channel impulse responses from the speech source to each microphone. As clearly explained in the previous chapters, a beamformer has two effects: dereverberation and denoising. The performance degradation of an adaptive beamformer in reverberant environments is mainly caused by poor dereverberation results [10], [15]. If we cannot do a good job with a beamformer in dereverberation, the better strategy may be not to do it at all. Therefore the reverberant speech signal at one arbitrary microphone across the array, instead of the original speech source signal, is what to be estimated from the multiple microphone outputs [26], [74]. This suggests that multichannel noise reduction should be used. In the so-called transfer function generalized sidelobe canceller (TF-GSC) algorithm proposed by Gannot et al. [55], [56], transfer function ratios with respect to various microphone pairs (instead of transfer functions themselves in traditional adaptive beamforming algorithms) are blindly estimated by exploiting the non-stationarity of speech. But this only partially takes advantage of the idea of multichannel noise reduction. In [15], we showed a trick which theoretically eliminates the need to estimate the transfer function ratios, which leads to a more practical approach to reducing noise with multiple microphones. Another convenience of using a multichannel noise reduction algorithm compared to a traditional adaptive beamformer is that the array geometry and the sensor consistency in gain and frequency response are not critical. This makes the design and fabrication of the microphone array in spacesuits much easier and more flexible. 1

For example, air supply hasn’t been turned on yet, the suit subject stands still, and even the helmet hasn’t been put on the suit.

12.3 Suggested Algorithms

201

In the traditional microphone array processing, the original speech signal is to be recovered and the Wiener filter (which is the optimal solution in the MMSE sense) can be factorized into the product of an LCMV beamformer and a post-filter [111]. For multichannel noise reduction, the reverberant speech signal received at one microphone across the array is the goal and the Wienerfilter solution again can be factorized into the product of an LCMV filter and a post-filter [15]. While the LCMV filters are different in the two approaches, the post-filters are identical. Therefore the single-channel noise reduction methods to be discussed below are shared by both approaches. 12.3.3 Single-Channel Noise Reduction While microphone array beamforming or the multichannel noise reduction technique can effectively attenuate noise theoretically without introducing much distortion to speech, the achievable gain in SNR is limited by the number of microphones to be employed. If that number is small due to the size of the helmet, the residual noise can still be quite strong. Both speech distortion and additive noise impair speech intelligibility. So it is not an optimal solution, in terms of speech intelligibility, to keep speech distortion very low while allowing strong residual noise. Instead, attaining a balance between speech distortion and noise level is what we desire for. Single-channel noise reduction methods boost the SNR but inevitably incur speech distortion. They essentially trade off speech distortion for noise reduction, and therefore can be ideally used as a post-filtering step for microphone arrays. So far a large number of single-channel noise reduction algorithms have been invented and they can be broadly classified into three classes: the class of filtering techniques, the class of spectral restoration methods, and the class of speech model-based approaches. The first two classes of methods have been extensively studied in this book while typical algorithms in the third class include harmonic-model-based methods [85], linear-prediction-model-based Kalman filtering approaches [97], and hidden-Markov-model-based statistical methods [46]. While the filtering and spectral restoration methods tackle the noise reduction problem from different perspectives, their implementations are all based on the estimates of second-order statistics (SOS) of the noisy speech and noise signals. Accurate and timely estimation of these statistics is crucial to the success of these algorithms. A common way to estimate the SOS of noise is to implement a voice activity detector (VAD) and then perform the estimation when speech is absent [28], [40], [89], [104]. In order to ensure that the noise SOS estimated during silent periods are useful when speech is present, a stationary or slowly varying noise signal has to be assumed. If noise unfortunately varies fast in time, none of the filtering and spectral restoration algorithms can work well. Another way is to directly estimate the SOS of the clean speech signal from multiple microphone outputs. Apparently this method can only be used when the developed noise reduction algorithm serves as a post-filter

202

12 Practical Example

of a microphone array system. With this speech spectral estimation method, noise has to be incoherent at the multiple microphones [127], which limits its use in practice. 12.3.4 Adaptive Noise Cancellation As clearly explained above, a nonstationary noise signal is in general difficult to reduce due to the lack of the ability to instantaneously track its time-varying statistics. Good examples inside spacesuits are spacesuit bearing noise and footfall. They are short bursting and randomly occur. If they have no overlap in time with the suit subject’s speech, we can develop a sophisticated detector to detect noise transients and actively eliminated them. The detection criterion can be based on a single-channel microphone output or multichannel microphone outputs [29]. However, bearing/footfall noise and speech can be simultaneously observed. In these cases, only adaptive noise cancellation (ANC) can possibly produce some good results. Among the four noise sources summarized previously in Section 12.2.1, spacesuit bearing noise and suit-impact noise signals are transmitted by the spacesuit from the lower parts of the suit to the microphones around the helmet. If we have a microphone woven in the inner fabric of the suit at a place in the upper torso where the suit subject’s speech is inaudible, the sensor will only pick up vibrations that propagate through and its output can be nicely used as a noise reference signal. Then we can use ANC and remove the components that are linearly correlated to the noise reference signal from the multichannel microphone array outputs. If the beamformer is fixed, it makes no difference in performance (without taking into account the complexity) whether ANC or beamforming is first performed. But if an adaptive beamformer is employed, the order is definitely critical and the favor goes to ANC being first performed, as depicted in Fig. 12.6. In ANC, a reference noise signal is assumed available, as illustrated by Fig. 12.11. This reference signal v2 (k) is used to estimate the noise v1 (k) in the primary microphone output y(k) by a linear filter, and this estimate vˆ1 (k) may then be subtracted from y(k) to form an estimate of the speech signal x(k). If x(k), v1 (k), and v2 (k) are jointly wide-sense stationary processes, then a Wiener filter may be designed to find the MMSE estimate of v1 (n). In practice, however, a stationarity assumption is not generally appropriate. Therefore, as an alternative to the Wiener filter, an adaptive filter is used. Needless to emphasize, the key to the successful operation of the adaptive noise canceller is the availability of a good reference signal v2 (k) which has a strong correlation with v1 (k) but ideally has no correlation with the speech signal x(k). The performance of the employed adaptive algorithm is also critical. If the adaptive algorithm cannot converge fast enough to track time variation of the noise signals, the estimate of the noise signal cannot be reliable. The problem that the adaptive filter faces in ANC is similar to that in acoustic echo cancellation (AEC). Thereafter a number of recently

12.4 Algorithm Validation

1

2

203

−

Adaptive Filter

Fig. 12.11. Illustration of the concept of adaptive noise cancellation when a reference noise signal v2 (k) is available.

proposed fast adaptive algorithms that were found useful for AEC can also be helpful for ANC. They include the frequency-domain adaptive filters [4], the improved proportionate NLMS (IPNLMS) algorithm [5], and the exponentiated gradient (EG) algorithm [7].

12.4 Algorithm Validation 12.4.1 In-Helmet Multichannel Acoustic Data Collection For algorithm analysis and validation, two sets of multichannel acoustic data were collected from inside of several spacesuits at the beginning of this research. One set of the data was measured by us (i.e., WeVoice, Inc.) and NASA engineers from the Glenn Research Center (GRC) and Johnson Space Center (JSC) in March 2008. The other was measured by Begault and Hieronymus from the NASA Ames Research Center (ARC) in early 2007 [2]. WeVoice-GRC Experimental Setup In the WeVoice-GRC data collection effort, two NASA’s current prototype planetary exploration suits were used. They are the Mark III (a joint effort between Air-Lock, Inc. and ILC Dover, Inc.) and the REI (Rear Entry I-Suit developed and fabricated by ILC Dover, Inc.) suits, as shown in Fig. 12.12. Both are rear-entry suits, unlike the EMU currently in use, which is a waistentry suit. The Mark III incorporates a hybrid mixture of hard and soft suit components, including hard upper torso (HUT), hard brief and hip/thigh elements made of graphite/epoxy composite, bearings at the shoulder, upper arm, hip, waist, and ankle, and soft fabric joints at the elbow, knee, and ankle, and a pair of military flight boots. The I-Suit is primarily a soft suit, yet

204

12 Practical Example

(a)

(b) Fig. 12.12. Comparison of two NASA’s prototype planetary exploration spacesuits: (a) the Mark III with a round helmet neck ring and (b) the REI (Rear Entry I-Suit) with an oval ring.

incorporates a limited number of bearings at the wrist, shoulder, upper arm, upper hip, upper leg joints (the other joints on this suit are fabric joints). In addition, it has a pair of mountaineering boots. The I-Suit represents a compromise between an HUT suit and an all soft suit, such as the Apollo A7LB suit. As illustrated in Fig. 12.12, the Mark III has a round helmet neck

12.4 Algorithm Validation

(a)

205

(b)

Fig. 12.13. Illustration of two equispaced, linear microphone arrays and their mounting apparatuses developed in the multichannel acoustic data collection experiments. (a) The SMS array mounted in a Mark III suit and (b) the Pro-Audio microphone array mounted in a REI suit.

ring while the REI has an oval ring. The Mark III is much (about 50 pounds) heavier but has better mobility than the I-Suit [117]. Two types of microphone array and acoustic data acquisition systems were used. One is an SPL (Sound Pressure Level) Measurement System (SMS) and the other is a digital Pro-Audio system. Each system has its own 4-element linear microphone array with 4 cm inter-element spacing located in front of the suit subject. Specialized microphone array mounting apparatuses were developed at NASA-JSC, as illustrated in Fig. 12.13. In addition to the 4element array, a fifth microphone (of the same type as that used in the array) was positioned on the rear panel (for the study on adaptive noise cancellation) and a close-talking microphone was also used. The rear panel on the Mark III suit had an acoustic foam panel covering the rear where the microphone was located. The SMS provided by the NASA GSC EVA Audio Team is a NISTcalibrated acoustic data collection system. Its array consists of 4 G.R.A.S. Type 40BE pre-polarized, free-field instrumentation microphones with type 26CB preamp modules. An NI-4722 24-bit DAQ performs analog-to-digital conversion (ADC) and provides the IEPE current supply for the microphone preamps. The Pro-Audio system was designed by WeVoice, Inc. It employs an array of 4 MWM Acoustics EM6022 omni-directional electret microphone, a Grace M802 microphone preamp with digital audio outputs, a Rosetta 800 Apogee audio interface, and a PC Laptop (Windows XP) with Adobe Audition software. The block diagrams of these two multichannel acoustic data acquisition systems are presented in Fig. 12.14. In the experiments, the Mark III and REI suits were fitted with both the SMS and the Pro-Audio microphone arrays. An extensive set of tests were run under three measurement conditions: 1) subject standing, 2) subject walking slow, and 3) subject walking fast. The experiments were conducted

206

12 Practical Example

Behind Panel Com−Cap w/ Andrea Noise Cancelling Mic

Suit Bulkhead w/ Bendix Connectors

SPADEL PC Data Acquisition System

Andrea APS 100

40BE 26CB

40BE

26CB

40BE

26CB

40BE

26CB

40BE

26CB

30 Foot Bundle of RG−174 Coaxial Cable (Male BNC Ends) External Pigtail Connector

G.R.A.S Type 40BE Free−Field Mic Element

G.R.A.S. Type 26CB Mic PreAmp

Panel

8

Andrea APS 100

7

3.5 mm Plug

6 5

5

4

4

3

3

Single Fiber Optic Cable

Firewire

Laptop Equipped w/ Adobe Audition

2

2

ADAT 1

1 MWM Acoustics EM6022 Omni−directional Electret Microphones

Monitor Headphone

Long cable for movement when out of donning stand

XLR Connectors

Apogee Rosetta 800 Audio Interface

Com−Cap w/ Andrea Noise Cancelling Mic

Hatch

(a)

13−pin Bendix Connectors

Grace m802 Mic Preamp and ADC

(b) Fig. 12.14. Block diagrams of (a) the SPL measurement and (b) the digital ProAudio multichannel acoustic data acquisition systems.

using either a Liquid Air Backpack (LAB) or a K-Bottle based air supply. The K-Bottle supply simulates an umbilical configuration while the LAB simulates a PLSS (Primary, or Portable, Life Support System/Subsystem) based configuration. Over 11 gigabytes of data were collected.

12.4 Algorithm Validation

207

Andrea ANC-700 active noise cancelling microphone on communication cap

YOGA EM060 omni-directional electret microphones

Fig. 12.15. Mark III suit with helmet bubble removed showing a head-mounted Andrea ANC-700 active noise cancelling microphone on the communication cap and two Yoga EM060 electret microphones as placed along the helmet ring. (Courtesy Durand R. Begault and James L. Hieronymus, NASA ARC.)

ARC Experimental Setup In the experiments that Begault and Hieronymus conducted [2], data were collected by a digital Pro-Audio system in a Mark III suit with LAB. While different microphone array, preamp, and ADC were used, their system architecture is similar to that of the WeVoice Pro-Audio system as illustrated in Fig. 12.14(b). Attached to the communication cap was also an Andrea ANC700 active noise cancelling microphone used as a baseline reference. But the array consisted of only two YOGA EM 060 omni-directional electret microphones, which were positioned around the helmet-mounting ring of the suit on foam rubber mounts at 7 and 4 o’clock, respectively (the top of the ring is 12 o’clock and the bottom 6 o’clock), as shown in Fig. 12.15. The subject read a set of spoken dialogue system commands with the suit pressurized in the following 5 conditions: 1) subject inside suit on the “donning stand,” a device that holds the suit in place, 2) subject standing, 3) subject walking (footfall impacts and hip bearing noise), 4) subject walking and moving arms (shoulder bearing noise plus walking noise), and 5) subject seated in a rover seat (volume change and head lower in helmet area).

208

12 Practical Example

In-Helmet Multichannel Acoustic Database and Data Preprocessing Among the huge amount of multichannel acoustic data, not all the recordings are immaculate and can be used for validation of beamforming and multichannel noise reduction algorithms. Only typical, relevant recordings of good quality are selected to form a database for future use. A description of the database is presented in Table 12.3. All these multichannel sound clips were sampled at 48 kHz and quantized with 16 bits. Shown in Fig. 12.16(a) is a period of waveform extracted from grc stand.wav for which the suit subject stood with no body movement and hence only air flow related noise was present. Clearly seen from Fig. 12.16(c), the recording contained a loud low-frequency sound, which is similar to rumble noise (rumble noise is a loud low-frequency sound and is usually caused by ball bearings). A high-pass filter (HPF) therefore needs to be used to preprocess the microphone outputs. Such a high-pass filtering is critical to the beamforming and multichannel noise reduction algorithms that follow. Multichannel algorithms that exploit spatial sampling generally can yield just a theoretically marginal gain in speech enhancement for low-frequency signals. If a loud low-frequency noise signal dominates in the microphone output like what is shown in Fig. 12.16(a), then the full-band improvement of those multichannel algorithms will not manifest even though they can do a pretty good job in high-frequency bands. The processed signal using an HPF with the cutoff frequency of 100 Hz is shown in Fig. 12.16(b) and a comparison of spectrum of the original and processed noise signals is presented in Fig. 12.16(c). Summarized in Table 12.4 are the SNRs of the recorded multichannel signals and the signals after being HP filtered. These SNRs will be used as the baseline references to assess the beamforming and multichannel noise reduction algorithms. Since speech is non-stationary in nature, estimating the SNR of a noise-corrupted speech signal is not a theoretically trivial problem. Sophisticated speech-model-based methods (e.g., hidden Markov model-based approaches) are computationally intensive, while simple algorithms may not be always reliable. In our research, a frame-based, histogram analysis algorithm is used for SNR estimation of speech signals. This algorithm divides a sufficiently long sequence of speech samples into frames and computes the energy of each frame. The average of the top 20% of the frame energies is regarded as the speech energy and the average of the bottom 20% as the noise energy. Their ratio leads to the SNR estimate of the signal. 12.4.2 Performance Evaluation of Beamforming Algorithms In this study, fixed beamformers targeted for a nearfield speech sound source were first investigated. But unfortunately these fixed beamformers produced only marginal, if not nonexistent, gains in SNR. For these results, two possibilities can be surmised: 1) the speech source and the dominant noise sources

(Pro-Audio)

LAB

standing & looking right standing & looking left walking walking & moving arms

arc stand lookright.wav arc stand lookleft.wav arc walk.wav arc walk movearm1.wav arc walk movearm2.wav

standing & looking forward

arc stand lookfwd.wav

25.8

26.5

30

23.5

24

22.5

21

donning stand & looking left

arc donning lookleft.wav

23 22

donning stand & looking forward

82

60

2

5

6

Duration (s)

arc donning lookright.wav donning stand & looking right

arc donning lookfwd.wav

Mark III

walking fast

standing

walking fast

grc stand.wav

walking slow

wv walkslow

standing

Measurement Condition

wv walkfast

wv stand.wav

ARC

LAB

LAB

grc walkfast

Mark III

Mark III

File Name

(SMS)

GRC

(Pro-Audio)

WeVoice

DAQ System Spacesuit Air Supply

12.4 Algorithm Validation 209

Table 12.3. Description of the multichannel acoustic database collected from inside spacesuits.

12 Practical Example

Amplitude

210

0.5 0 −0.5 0

0.5

1

1.5

2

2.5

1.5

2

2.5

Time (s)

(a)

Amplitude

0.2 0.1 0 −0.1 −0.2 0

0.5

1 Time (s)

(b) −20 Original Recording Processed Signal with an HPF

Magnitude (dB)

−40 −60 −80 −100 −120 −140 0

5

10 15 Frequency (kHz)

20

(c) Fig. 12.16. Illustration of low-frequency noise recorded inside spacesuits and removing it with a high-pass filter (HPF). (a) A short period of waveform extracted from grc stand.wav, (b) the processed signal using a high-pass filter with the cut-off frequency of 100 Hz, and (c) the comparison of spectrum between the original and the processed noise signals (corresponding to the circled parts).

12.4 Algorithm Validation

211

Table 12.4. SNRs of the recorded multichannel signals and the signals after being high-pass filtered. SNR (dB) Microphone Outputs File Name

#1

#2

#3

HP-Filtered Signals

#4

#1

#2

#3

#4

wv stand.wav

21.309 19.683 19.567 21.508

22.891 22.393 22.012 22.335

wv walkslow.wav

19.062 18.606 17.942 19.717

20.496 20.784 20.997 20.553

wv walkfast.wav

19.431 19.028 19.061 19.251

20.265 20.750 20.686 19.755

grc stand.wav

15.417 15.417 16.145 15.614

21.912 21.912 20.385 21.143

grc walkfast.wav

17.067 15.541 15.590 16.216

17.766 18.027 17.207 18.094

SNR (dB) Microphone Outputs #2

HP-Filtered Signals

File Name

#1

#1

#2

arc donning lookfwd.wav

16.874

17.007

28.902

27.959

arc donning lookright.wav

18.560

18.152

29.507

27.300

arc donning lookleft.wav

18.066

18.328

28.618

28.744

arc stand lookfwd.wav

14.950

14.558

26.601

26.012

arc stand lookright.wav

17.593

16.829

26.881

25.933

arc stand lookleft.wav

16.853

16.292

27.646

27.383

arc walk.wav

17.247

17.800

20.888

20.869

arc walk movearm1.wav

13.539

12.743

20.427

20.111

arc walk movearm2.wav

12.380

12.953

19.146

18.554

are in the same direction with respect to the array, 2) the assumption that the suit subject’s mouth can be treated as a point sound source is problematic inside the helmet. Recall that in the WeVoice and GRC measurements, the two arrays were mounted under the suit subject’s jaw (see Fig. 12.13). Consequently, both the subject’s mouth and the airflow inlet are normal to the array axis and the first surmise above is very likely true. Presented in Fig. 12.17 is a comparison of the magnitude of coherence between the signals in two periods of recording extracted from wv stand.wav: one when speech was present and the other when there was only noise. It is evident that the noise signals at the four microphones are fairly coherent. We studied the second surmise via a number of time delay estimation (TDE) algorithms for acoustic source localization. They included the full-band GCC-PHAT (generalized cross correlation-phase transform) method [84], the

212

12 Practical Example

Magnitude of Coherence

1 0.8 0.6 0.4 0.2 0 0

Mic. 1−2 Mic. 1−3 Mic. 1−4 500

1000

1500 2000 2500 Frequency (Hz)

3000

3500

4000

3000

3500

4000

(a)

Magnitude of Coherence

1 0.8 0.6 0.4 0.2 0 0

Mic. 1−2 Mic. 1−3 Mic. 1−4 500

1000

1500 2000 2500 Frequency (Hz) (b)

Fig. 12.17. Comparison of the magnitude of coherence between the signals in two periods of recording extracted from wv stand.wav: (a) when speech was present and (b) when there was only noise.

subband GCC-PHAT algorithm, the adaptive eigenvalue decomposition decomposition algorithm [3], [70], the blind SIMO system identification based approach [71], and the spatial multichannel correlation technique [9], [23]. A voice activity detector was used and flawed detections around ambiguous speech-noise boundaries were manually corrected. But still none of the TDE algorithms can produce satisfyingly consistent and accurate results. While these can be mainly attributed to strong reverberation and loud noise inside

12.4 Algorithm Validation

213

Table 12.5. Summary of the simulation results of an adaptive subband MVDR beamformer (MVDR-BF) followed by a Wiener filter for single-channel noise reduction (WF-SCNR) with the multichannel acoustic data gathered in spacesuits. Input Microphone SNR (dB)

Output SNR (dB) Average MVDR-

WF-

Gain

BF

SCNR

(dB)

22.89 22.39 22.01 22.33

22.97

34.62

12.21

wv walkslow.wav 20.50 20.78 21.00 20.55

21.25

31.29

10.58

wv walkfast.wav 20.27 20.75 20.69 19.76

21.47

31.45

11.08

grc stand.wav

21.91 21.91 20.38 21.14

22.04

32.11

10.77

grc walkfast.wav 17.76 18.02 17.21 18.09

17.97

26.64

8.87

File Name wv stand.wav

#1

#2

#3

#4

Input Mic. Output SNR (dB) Average SNR (dB) File Name

#1

#2

MVDR-

WF-

Gain

BF

SCNR

(dB)

arc donning lookfwd.wav 28.91 27.96

29.55

42.74

14.31

arc donning lookright.wav 29.49 27.30

28.50

41.80

13.41

arc donning lookleft.wav

29.02 28.74

29.44

42.02

13.14

arc stand lookfwd.wav

26.59 26.02

26.92

39.38

13.07

arc stand lookright.wav

26.90 25.93

26.37

38.95

12.54

arc stand lookleft.wav

27.65 27.39

27.49

40.60

13.08

arc walk.wav

20.90 20.87

21.97

32.08

11.20

arc walkarm1.wav

20.42 20.11

21.22

32.34

12.07

arc walkarm2.wav

19.14 18.56

18.65

29.43

10.58

the helmet, a negative impact caused by the problematic assumption that the mouth can be treated as a point sound source (by observing its size with respect to the distance from the array) is certainly unable to be eliminated. In addition to fixed beamformers, two adaptive beamforming algorithms were also tested: the full-band and subband MVDR beamformers. The speech source was believed to be in the broadside. The subband MVDR beamformer performed better than the full-band implementation. The subband MVDR results are presented in Table 12.5. This study suggests that we need to move the microphone array to the side of the suit subject (such that the mouth and the airflow inlet could be in different directions to the array) in the followingup efforts for the development of ISA systems.

214

12 Practical Example

12.4.3 Validation of Multichannel Noise Reduction Algorithms The simulation results of the subband MVDR algorithm for multichannel noise reduction (MCNR) are summarized in Table 12.6. By comparing these results to that presented in Table 12.5, it is very clear that the MVDR for MCNR performs better than the MVDR beamformer. In each set of the three measurements, the performance of both the beamformer and MCNR degrades as the subject increases movement and the noise becomes more non-stationary. Figure 12.18 visualizes the non-stationarity of the noise signals recorded under different conditions. When the suit was on the donning stand, the noise signal was at a low level and reasonably stationary. But as the subject’s movement increased, the captured noise became more and more non-stationary. When the subject walked and moved his arms, the noise sounds like sanding wood. In this case, noise reduction would be very challenging. 12.4.4 Validation of Single-Channel Noise Reduction Algorithms Tables 12.5 and 12.6 also include the performance of a Wiener-filter singlechannel noise reduction algorithm as a post filter following the subband MVDR beamformer and the subband MVDR multichannel noise reduction, respectively. One can see that the Wiener filter for single-channel noise reduction we developed in this research is relatively robust and can reliably produce about 12 dB gain in SNR in all studied measurement conditions. In our experiments, the algorithm was tuned to yield such a level of gain with the amount of speech distortion that caused little intelligibility loss according to our informal listening tests. If more speech distortion is allowed, more gain in SNR can be achieved. Figure 12.19 visualizes the noise reduction procedure and plots the waveforms of the first microphone, the MCNR, and the SCNR outputs for the recording wv stand.wav. The background airflow noise is clearly minimized. 12.4.5 Feasibility Assessment of Using Adaptive Noise Cancellation in Spacesuits As previously explained, the key to the success of applying an adaptive noise cancellation algorithm inside spacesuits is that the reference noise signal is highly correlated with the noise components, but is little correlated with the speech components of the primary microphone outputs. These two conditions are equally important. By examining the reference microphone (Microphone 5 in the WeVoice and GRC measurements) signals, we found that they contain very evident speech signals from the suit subject, as seen from a sample of Microphone 5 outputs in comparison with the first microphone output shown in Fig. 12.20. Therefore, adaptive noise cancellation cannot be employed with the current installation of Microphone 5. In the future, Microphone 5 will be moved further away from the spacesuit helmet, ideally closer to the spacesuit bearings.

12.4 Algorithm Validation

215

Amplitude

0.1 0.05 0 −0.05 −0.1 0

1

2

3

4

5

3

4

5

3

4

5

3

4

5

Time (s)

(a)

Amplitude

0.1 0.05 0 −0.05 −0.1 0

1

2 Time (s)

(b)

Amplitude

0.2 0.1 0 −0.1 −0.2 0

1

2 Time (s)

(c)

Amplitude

0.1 0.05 0 −0.05 −0.1 0

1

2 Time (s)

(d) Fig. 12.18. Comparison of stationarity among the noise signals in the ARC measurements under different conditions: (a) subject inside suit on the “donning stand” (looking forward), (b) subject standing (looking forward), (c) subject walking, and (d) subject walking and moving arms.

216

12 Practical Example 1

Amplitude

0.5 0 −0.5 −1 0

1

2

3 Time (s)

4

5

6

4

5

6

4

5

6

(a) 1

Amplitude

0.5 0 −0.5 −1 0

1

2

3 Time (s)

(b) 1

Amplitude

0.5 0 −0.5 −1 0

1

2

3 Time (s)

(c) Fig. 12.19. Waveforms of (a) the first microphone, (b) the output of the subband MVDR algorithm for multichannel noise reduction, and (c) the output of the Wiener filter for single-channel noise reduction in the processing of the recording wv stand.wav.

12.5 Summary

217

Table 12.6. Summary of the simulation results of a subband MVDR algorithm for multichannel noise reduction (MVDR-MCNR) followed by a Wiener filter for singlechannel noise reduction (WF-SCNR) with the multichannel acoustic data gathered in spacesuits. Input Microphone SNR (dB)

Output SNR (dB) Average MVDR-

WF-

Gain

MCNR

SCNR

(dB)

22.89 22.39 22.01 22.33

30.69

41.82

19.42

wv walkslow.wav 20.50 20.78 21.00 20.55

28.14

40.19

19.49

wv walkfast.wav 20.27 20.75 20.69 19.76

29.09

40.98

20.62

grc stand.wav

21.91 21.91 20.38 21.14

25.36

37.73

16.39

grc walkfast.wav 17.76 18.02 17.21 18.09

20.54

29.82

12.05

File Name wv stand.wav

#1

#2

#3

#4

Input Mic. Output SNR (dB) Average SNR (dB)

MVDR-

WF-

Gain

#1

MCNR

SCNR

(dB)

arc donning lookfwd.wav 28.91 27.96

31.91

44.85

16.42

arc donning lookright.wav 29.49 27.30

32.20

45.66

17.27

arc donning lookleft.wav

29.02 28.74

31.89

44.30

15.43

arc stand lookfwd.wav

26.59 26.02

28.73

40.88

14.58

arc stand lookright.wav

26.90 25.93

29.15

42.05

15.63

arc stand lookleft.wav

27.65 27.39

30.30

42.74

15.22

arc walk.wav

20.90 20.87

23.25

32.68

11.80

arc walkarm1.wav

20.42 20.11

21.28

33.30

13.03

arc walkarm2.wav

19.14 18.56

19.87

31.27

12.42

File Name

#2

12.5 Summary This chapter described one of our ongoing efforts to help NASA develop an integrated spacesuit audio system for in-helmet voice communication during extravehicular activities. The motivation underlying such a project was first explained by the necessities for the enhanced intelligibility and quality of captured speech, comfort and ease of use, and logistical convenience. Then the unique challenges imposed by the extreme acoustic environment due to the special design of spacesuits were analyzed. Four noise reduction techniques were proposed and the theoretical considerations as to why these techniques can potentially be helpful were comprehensively discussed. Finally the experimental setup and procedure as well as the preliminary validation results were

218

12 Practical Example 1

Amplitude

0.5 0 −0.5 −1 0

1

2

3 Time (s)

4

5

6

4

5

6

(a) 1

Amplitude

0.5 0 −0.5 −1 0

1

2

3 Time (s)

(b) Fig. 12.20. Waveforms of (a) the 1st and (b) the 5th microphone outputs in the recording wv stand.wav.

reported and the established directions for future work were sketched. We use such a real-life example to show how the problem of noise reduction is tackled in practice and believe that the experience can be beneficial to the readers who are probably working on distant speech acquisition systems in different acoustic environments but with similar noise reduction tools.

Acknowledgments This work was financially supported by the NASA SBIR program. We are grateful to the NASA engineers, in particular Dr. O. Scott Sands, Dr. James L. Hieronymus, and Craig Bernard, for their strong support and constructive advice and guidance, without which this project could never have made so much progress.

References

1. J. B. Allen, “Short-time spectral analysis, synthesis and modification by discrete Fourier transform,” IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-25, pp. 235–238, June 1977. 2. D. R. Begault and J. L. Hieronymus, “Acoustical issues and proposed improvements for NASA spacesuits,” in Proc. AES 122nd Conv., 2007. 3. J. Benesty, “Adaptive eigenvalue decomposition algorithm for passive acoustic source localization,” J. Acoust. Soc. Am., vol. 107, pp. 384–391, Jan. 2000. 4. J. Benesty and D. Morgan, “Frequency-domain adaptive filtering revisited, generalization to the multi-channel case, and application to acoustic echo cancellation,” in Proc. IEEE ICASSP, 2000, vol. 2, pp. 789–792. 5. J. Benesty and S. L. Gay, “An improved PNLMS algorithm,” in Proc. IEEE ICASSP, vol. 2, 2002, pp. 1881–1884. 6. J. Benesty and Y. Huang, eds., Adaptive Signal Processing: Applications to Real-World Problems. Berlin, Germany: Springer-Verlag, 2003. 7. J. Benesty, Y. Huang, and D. R. Morgan, “On a class of exponentiated adaptive algorithms for identification of sparse impulse responses,” in Adaptive Signal Processing: Application to Real-World Problems, J. Benesty and Y. Huang, eds., Berlin, Germany: Springer, 2003, pp. 1–22. 8. J. Benesty and T. Gaensler, “New insights into the RLS algorithm,” EURASIP J. Applied Signal Process., vol. 2004, pp. 331–339, Mar. 2004. 9. J. Benesty, J. Chen, and Y. Huang, “Time-delay estimation via linear interpolation and cross correlation,” IEEE Trans. Speech Audio Process., vol. 12, pp. 509–519, Sept. 2004. 10. J. Benesty, S. Makino, and J. Chen, eds., Speech Enhancement. Berlin, Germany: Springer-Verlag, 2005. 11. J. Benesty, J. Chen, Y. Huang, and S. Doclo, “Study of the Wiener filter for noise reduction,” in Speech Enhancement, J. Benesty, S. Makino, and J. Chen, eds., Berlin, Germany: Springer-Verlag, 2005, Chapter 2, pp. 9–41. 12. J. Benesty, J. Chen, and Y. Huang, “A generalized MVDR spectrum,” IEEE Signal Process. Lett., vol. 12, pp. 827–830, Dec. 2005. 13. J. Benesty, J. Chen, Y. Huang, and J. Dmochowski, “On microhone-array beamforming from a MIMO acoustic signal processing perspective,” IEEE Trans. Audio, Speech, Language Process., vol. 15, pp. 1053–1065, Mar. 2007.

220

References

14. J. Benesty, J. Chen, and Y. Huang, “On the importance of the Pearson correlation coefficient in noise reduction,” IEEE Trans. Audio, Speech, Language Process., vol. 16, pp. 757–765, May 2008. 15. J. Benesty, J. Chen, and Y. Huang, Microphone Array Signal Processing. Berlin, Germany: Springer-Verlag, 2008. 16. E. K. Berndt, B. H. Hall, R. E. Hall, and J. A. Hausman, “Estimation and inference in nonlinear structural models,” Annals of Economic and Social Measurement, vol. 4, pp. 653–665, 1974. 17. M. Berouti, R. Schwartz, and J. Makhoul, “Enhancement of speech corrupted by acoustic noise,” in Proc. IEEE ICASSP, 1979, pp. 208–211. 18. S. F. Boll, “Suppression of acoustic noise in speech using spectral subtraction,” IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-27, pp. 113–120, Apr. 1979. 19. T. Bollerslev, “Generalized autoregressive conditional heteroskedasticity,” J. Econometrics, vol. 31, pp. 307–327, Apr. 1986. 20. T. Bollerslev, R. Y. ChouKenneth, and F. Kroner, “ARCH modeling in finance: a review of the theory and empirical evidence,” J. Econometrics, vol. 52, pp. 5–59, Apr.-May 1992. 21. J. Capon, “High resolution frequency-wavenumber spectrum analysis,” Proc. IEEE, vol. 57, pp. 1408–1418, Aug. 1969. 22. O. Capp´e, “Elimination of the musical noise phenomenon with the Ephraim and Malah noise suppressor,” IEEE Trans. Speech, Audio Process., vol. 2, pp. 345–349, Apr. 1994. 23. J. Chen, J. Benesty, and Y. Huang, “Robust time delay estimation exploiting redundancy among multiple microphoens,” IEEE Trans. Speech Audio Process., vol. 11, pp. 549–557, Nov. 2003. 24. J. Chen, J. Benesty, Y. Huang, and S. Doclo, “New insights into the noise reduction Wiener filter,” IEEE Trans. Audio, Speech, Language Process., vol. 14, pp. 1218–1234, July 2006. 25. J. Chen, J. Benesty, Y. Huang, and E. J. Diethorn, “Fundamentals of noise reduction,” in Springer Handbook of Speech Processing, J. Benesty, M. M. Sondhi, and Y. Huang, eds., Berlin, Germany: Springer-Verlag, 2007, Chapter 43, Part H, pp. 843–872. 26. J. Chen, J. Benesty, and Y. Huang, “A minimum distortion noise reduction algorithm with multiple microphones,” IEEE Trans. Audio, Speech, Language Process., vol. 16, pp. 481–493, Mar. 2008. 27. I. Cohen and B. Berdugo, “Speech enhancement for non-stationary noise environments,” Elsevier Signal Process., vol. 81, pp. 2403–2418, Nov. 2001. 28. I. Cohen, “Noise spectrum estimation in adverse environments: improved minima controlled recursive averaging,” IEEE Trans. Speech, Audio Process., vol. 11, pp. 466–475, Sept. 2003. 29. I. Cohen, “Multichannel post-filtering in nonstationary noise environments,” IEEE Trans. Signal Process., vol. 52, pp. 1149–1160, May 2004. 30. I. Cohen, “Modeling speech signals in the time-frequency domain using GARCH,” Elsevier Signal Process., vol. 84, pp. 2453–2459, Dec. 2004. 31. I. Cohen, “Relaxed statistical model for speech enhancement and a priori SNR estimation,” IEEE Trans. Speech, Audio Process., vol. 13, pp. 870–881, Sept. 2005.

References

221

32. I. Cohen, “Speech spectral modeling and enhancement based on autoregressive conditional heteroscedasticity models,” Elsevier Signal Process., vol. 86, pp. 698–709, Apr. 2006. 33. I. Cohen and S. Gannot, “Spectral enhancement methods,” in Springer Handbook of Speech Processing, J. Benesty, M. M. Sondhi, and Y. Huang, eds., Berlin, Germany: Springer-Verlag, 2007, Chapter 44, Part H, pp. 873–901. 34. L. Cohen, “The history of noise,” IEEE Signal Process. Magazine, vol. 22, pp. 20–45, Nov. 2005. 35. R. E. Crochiere and L. R. Rabiner, Multirate Digital Signal Processing. Englewood Cliffs, New Jersey: Prentice-Hall, 1983. 36. A. P. Dempster, N. M. Laird, and D. B. Rubin, “Maximum likelihood from incomplete data via the EM algorithm,” J. Royal Statistical Society (B), vol. 39, no. 1, pp. 1–38, 1977. 37. M. Dendrinos, S. Bakamidis, and G. Carayannis, “Speech enhancement from noise: a regenerative approach,” Speech Commun., vol. 10, pp. 45–57, Feb. 1991. 38. E. J. Diethorn, “Subband noise reduction methods for speech enhancement,” in Audio Signal Processing for Next-Generation Multimedia Communication Systems, Y. Huang and J. Benesty, eds., Boston, MA, USA: Kluwer, 2004, Chapter 4, pp. 91–115. 39. E. J. Diethorn, “Foundations of spectral-gain formulae for speech noise reduction,” in Proc. IWAENC, 2005, pp. 181–184. 40. G. Doblinger, “Computationally efficient speech enhancement by spectral minima tracking in subbands,” in Proc. 4th Eurospeech, 1995, pp. 1513–1516. 41. O. J. Dunn and V. A. Clark, Applied Statistics: Analysis of Variance and Regression. New York: Wiley, 1974. 42. R. F. Engle, “Autoregressive conditional heteroskedasticity with estimates of the variance of united kingdom inflation,” Econometrica, vol. 50, pp. 987–1007, July 1982. 43. R. F. Engle, ed., ARCH Selected Readings. New York: Oxford University Press Inc., 1995. 44. Y. Ephraim and D. Malah, “Speech enhancement using a minimum meansquare error short-time spectral amplitude estimator,” IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-32, pp. 1109–1121, Dec. 1984. 45. Y. Ephraim and D. Malah, “Speech enhancement using a minimum meansquare error log-spectral amplitude estimator,” IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-33, pp. 443–445, Apr. 1985. 46. Y. Ephraim, D. Malah, and B.-H. Juang, “On the application of hidden Markov models for enhancing noisy speech,” IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-37, pp. 1846–1856, Dec. 1989. 47. Y. Ephraim, “A Bayesian estimation approach for speech enhancement using hidden Markov models,” IEEE Trans. Signal Process., vol. 40, pp. 725–735, Apr. 1992. 48. Y. Ephraim, “Statistical-model-based speech enhancement systems,” Proc. IEEE, vol. 80, pp. 1526–1555, Oct. 1992. 49. Y. Ephraim and H. L. Van Trees, “A signal subspace approach for speech enhancement,” IEEE Trans. Speech Audio Process., vol. 3, pp. 251–266, July 1995. 50. W. Etter and G. S. Moschytz, “Noise reduction by noise-adaptive spectral magnitude expansion,” J. Audio Eng. Soc., vol. 42, pp. 341–349, May 1994.

222

References

51. J. L. Flanagan, J. D. Johnston, R. Zahn, and G. W. Elko, “Computer-steered microphone arrays for sound transduction in large rooms,” J. Acoust. Soc. Am., vol. 78, pp. 1508–1518, Nov. 1985. 52. J. L. Flanagan, D. A. Berkley, G. W. Elko, J. E. West, and M. M. Sondhi, “Autodirective microphone systems,” Acustica, vol. 73, pp. 58–71, Feb. 1991. 53. O. L. Frost, III, “An algorithm for linearly constrained adaptive array processing,” Proc. IEEE, vol. 60, pp. 926–935, Aug. 1972. 54. K. Fukunaga, Introduction to Statistical Pattern Recognition. San Diego, CA: Academic Press, 1990. 55. S. Gannot, D. Burshtein, and E. Weinstein, “Signal enhancement using beamforming and nonstationarity with applications to speech,” IEEE Trans. Signal Process., vol. 49, pp. 1614–1626, Aug. 2001. 56. S. Gannot, D. Burshtein, and E. Weinstein, “Analysis of the power spectral deviation of the general transfer function GSC,” IEEE Trans. Signal Process., vol. 52, pp. 1115–1121, Apr. 2004. 57. J. S. Garofolo, “Getting started with the DARPA TIMIT CD-ROM: an acoustic phonetic continuous speech database,” National Institute of Standards and Technology (NIST), Gaithersburg, Maryland, Tech. Rep., (prototype as of Dec. 1988). 58. S. J. Godsill and P. J. W. Rayner, Digital Audio Restoration: a Statistical Model Based Approach. Berlin, Germany: Springer-Verlag, 1998. 59. G. H. Golub and C. F. Van Loan, Matrix Computations. Baltimore, MD: The Johns Hopkins University Press, 1996. 60. R. M. Gray, “Toeplitz and circulant matrices: a review,” Foundations and Trends in Communications and Information Theory, vol. 2, pp 155–239, 2006. 61. L. J. Griffiths and C. W. Jim, “An alternative approach to linearly constrained adaptive beamforming,” IEEE Trans. Antennas Propagat., vol. AP-30, pp. 27– 34, Jan. 1982. 62. J. H. L. Hansen, “Speech enhancement employing adaptive boundary detection and morphological based spectral constraints,” in Proc. IEEE ICASSP, 1991, pp. 901–904. 63. S. Haykin, Adaptive Filter Theory. Fourth Edition, Upper Saddle River, NJ: Prentice-Hall, 2002. 64. K. Hermus, P. Wambacq, and H. Van hamme, “A review of signal subspace speech enhancement and its application to noise robust speech recognition,” EURASIP J. Advances Signal Process., vol. 2007, Article ID 45821, 15 pages, 2007. 65. H. G. Hirsch and C. Ehrlicher, “Noise estimation techniques for robust speech recognition,” in Proc. IEEE ICASSP, 1995, vol. 1, pp. 153–156. 66. Y. Hu and P. C. Loizou, “A subspace approach for enhancing speech corrupted by colored noise,” IEEE Signal Process. Lett., vol. 9, pp. 204–206, July 2002. 67. Y. Hu and P. C. Loizou, “A subspace approach for enhancing speech corrupted by colored noise,” in Proc. IEEE ICASSP, 2002, pp. I-573–I-576. 68. Y. Hu and P. C. Loizou, “A generalized subspace approach for enhancing speech corrupted by colored noise,” IEEE Trans. Speech Audio Process., vol. 11, pp. 334–341, July 2003. 69. Y. Hu and P. C. Loizou, “A perceptually motivated approach for speech enhancement,” IEEE Trans. Speech Audio Process., vol. 11, pp. 457–465, Sept. 2003.

References

223

70. Y. Huang, J. Benesty, and G. W. Elko, “Adaptive eigenvalue decomposition algorithm for realime acoustic source localization,” in Proc. IEEE ICASSP, 1999, pp. 937–940. 71. Y. Huang and J. Benesty, “A class of frequency-domain adaptive approaches to blind multichannel identification,” IEEE Trans. Signal Process., vol. 51, pp. 11–24, Jan. 2003. 72. Y. Huang, J. Benesty, and J. Chen, Acoustic MIMO Signal Processing. Berlin, Germany: Springer-Verlag, 2006. 73. Y. Huang, J. Benesty, and J. Chen, “Time delay estimation and source localization,” in Springer Handbook of Speech Processing and Speech Communication, J. Benesty, M. M. Sondhi, and Y. Huang, eds., Berlin, Germany: SpringerVerlag, 2007, Chapter 51, Part H, pp. 1043–1063. 74. Y. Huang, J. Benesty, and J. Chen, “Analysis and comparison of multichannel noise reduction methods in a common framework,” IEEE Trans. Audio, Speech, Language Process., vol. 16, pp. 957–968, July 2008. 75. F. Jabloun and B. Champagne, “Incorporating the human hearing properties in the signal subspace approach for speech enhancement,” IEEE Trans. Speech Audio Process., vol. 11, pp. 700–708, Nov. 2003. 76. F. Jabloun and B. Champagne, “Signal subspace techniques for speech enhancement,” in Speech Enhancement, J. Benesty, S. Makino, and J. Chen, eds., Berlin, Germany: Springer-Verlag, 2005, Chapter 7, pp. 135–159. 77. E. Jan and J. L. Flanagan, “Microphone arrays for speech processing,” in Prof. Int. Symp. Signals, Systems, and Electronics, 1995, pp. 373–376. 78. S. H. Jensen, P. C. Hansen, S. D. Hansen, and J. A. Sorensen, “Reduction of broad-band noise in speech by truncated QSVD,” IEEE Trans. Speech Audio Process., vol. 3, pp. 439–448, Nov. 1995. 79. C. W. Jim, “A comparison of two LMS constrained optimal array structures,” Proc. IEEE, vol. 65, pp. 1730–1731, Dec. 1977. 80. Y. Kaneda and M. Tohyama, “Noise suppression signal processing using 2-point received signals,” Electron. Commun. Japan, vol. 67-A, no. 12, pp. 19–28, Apr. 1984. 81. Y. Kaneda and J. Ohga, “Adaptive microphone-array system for noise reduction,” IEEE Trans. Acoust. Speech Signal Process., vol. ASSP-34, pp. 1391– 1400, June 1986. 82. S. Kay, “Some results in linear interpolation theory,” IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-31, pp. 746–749, June 1983. 83. W. Kellermann, “A self steering digital microphone array,” in Proc. IEEE ICASSP, 1991, vol. VI, pp. 3581–3584. 84. C. H. Knapp and G. C. Carter, “The generalized correlation method for estimation of time delay,” IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-24, pp. 320–327, Aug. 1976. 85. J. S. Lim and A. V. Oppenheim, “Enhancement and bandwidth compression of noisy speech,” Proc. IEEE, vol. 67, pp. 1586–1604, Dec. 1979. 86. J. S. Lim, ed., Speech Enhancement. Englewood Cliffs, NJ: Prentice-Hall, 1983. 87. Y. Lu and P. C. Loizou, “A geometric approach to spectral subtraction,” Speech Communication, vol. 50, pp. 453–466, 2008. 88. P. Loizou, Speech Enhancement: Theory and Practice. Boca Raton, FL: CRC Press, 2007.

224

References

89. R. Martin, “Noise power spectral density estimation based on optimal smoothing and minimum statistics,” IEEE Trans. Speech Audio Process., vol. 9, pp. 504–512, May 2001. 90. R. Martin, “Speech enhancement using MMSE short time spectral estimation with Gamma distributed speech priors,” in Proc. IEEE ICASSP, 2002, pp. I–253–I–256. 91. R. Martin, D. Malah, R. V. Cox, and A. J. Accardi, “A noise reduction preprocessor for mobile voice communication,” EURASIP J. Applied Signal Process., vol. 2004, pp. 1046-1058, July 2004. 92. R. J. McAulay and M. L. Malpass, “Speech enhancement using a soft-decision noise suppression filter,” IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-28, pp. 137–145, Apr. 1980. 93. G. J. McLachlan and T. Krishnan, The EM Algorithm and Extensions. New York: Wiley, 1997. 94. M. Okuda, M. Ikehara, and S. Takahashi, “Fast and stable least-squares approach for the design of linear phase FIR filters,” IEEE Trans. Signal Process., vol. 46, pp. 1485–1493, June 1998. 95. A. V. Oppenheim and R. W. Schafer, Discrete-Time Signal Processing. Englewood Cliffs, NJ: Prentice-Hall, 1989. 96. A. Oppenheim, A. Willsky, and H. Nawab, Signals and Systems. Upper Saddle River, NJ: Prentice Hall, 1996. 97. K. K. Paliwal and A. Basu, “A speech enhancement method based on Kalman filtering,” in Proc. IEEE ICASSP, 1987, pp. 177–180. 98. K. Pearson, “Mathematical contributions to the theory of evolution.–III. Regression, heredity and panmixia,” Philos. Trans. Royal Soc. London, Ser. A, vol. 187, pp. 253–318, 1896. 99. K. S. Pearsons, R. L. Bennett, and S. Fidell, “Speech levels in various noise environments,” EPA report 600/1-77-025, Environmental Protection Agency, Washington, DC, USA, 1977. 100. J. Porter and S. Boll, “Optimal estimators for spectral restoration of noisy speech,” in Proc. IEEE ICASSP, 1984, pp. 18A.2.1–18A.2.4. 101. M. R. Portnoff, “Time-frequency representation of digital signals and systems based on short-time Fourier analysis,” IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-28, pp. 55–69, Feb. 1980. 102. S. Qian and D. Chen, “Discrete Gabor transform,” IEEE Trans. Signal Process., vol. 41, pp. 2429–2438, July 1993. 103. A. Rezayee and S. Gazor, “An adpative KLT approach for speech enhancement,” IEEE Trans. Speech Audio Process., vol. 9, pp. 87–95, Feb. 2001. 104. C. Ris and S. Dupont, “Assessing local noise level estimation methods: application to noise robust ASR,” Speech Commun., vol. 34, pp. 141–158, 2001. 105. J. L. Rodgers and W. A. Nicewander, “Thirteen ways to look at the correlation coefficient,” The Amer. Statistician, vol. 42, pp. 59–66, Feb. 1988. 106. I. Santamaria and J. V´ıa, “Estimation of the magnitude squared coherence spectrum based on reduced-rank canonical coordinates,” in Proc. IEEE ICASSP, 2007, pp. III-985–988. 107. L. L. Scharf and J. T. Thomas, “Wiener filters in canonical coordinates for transform coding, filtering, and quantizing,” IEEE Trans. Signal Process., vol. 46, pp. 647–654, Mar. 1998.

References

225

108. M. R. Schroeder, “Apparatus for suppressing noise and distortion in communication signals,” U.S. Patent No. 3,180,936, filed Dec. 1, 1960, issued Apr. 27, 1965. 109. M. R. Schroeder, “Processing of communication signals to reduce effects of noise,” U.S. Patent No. 3,403,224, filed May 28, 1965, issued Sept. 24, 1968. 110. B. L. Sim, Y. C. Tong, J. S. Chang, and C. T. Tan, “A parametric formulation of the generalized spectral subtraction method,” IEEE Trans. Speech, Audio Process., vol. 6, pp. 328–337, July 1998. 111. K. U. Simmer, J. Bitzer, and C. Marro, “Post-filtering techniques,” in Microphone Arrays: Signal Processing Techniques and Applications, M. S. Brandstein and D. B. Ward, eds., Berlin: Springer, 2001, pp. 39–60. 112. M. M. Sondhi, C. E. Schmidt, and L. R. Rabiner, “Improving the quality of a noisy speech signal,” Bell Syst. Techn. J., vol. 60, pp. 1847–1859, Oct. 1981. 113. M. M. Sondhi and G. W. Elko, “Adaptive optimization of microphone arrays under a nonlinear constraint,” in Proc. IEEE ICASSP, 2002, pp. 981–984. 114. V. Stahl, A. Fischer, and R. Bippus, “Quantile based noise estimation for spectral subtraction and Wiener filtering,” in Proc. IEEE ICASSP, 2000, vol. 3, pp. 1875–1878. 115. C. Sydow, “Broadband beamforming for a microphone array,” J. Acoust. Soc. Am., vol. 96, pp. 845–849, Aug. 1994. 116. H. Teutsch and G. W. Elko, “An adaptive close-talking microphone array,” in Proc. IEEE WASPAA, 2001, pp. 163–166. 117. K. S. Tomas and H. J. MaMann, US Spacesuits. (Springer-Praxis Books in Space Exploration), Berlin, Germany: Springer-Verlag/Chichester, UK: Praxis, 2006. 118. A. Varga and H. J. M. Steeneken, “Assessment for automatic speech recognition: II. NOISEX-92: a database and an experiment to study the effect of additive noise on speech recognition systems,” Speech Commun., vol. 12, pp. 247–251, July 1993. 119. P. Vary and R. Martin, Digital Speech Transmission: Enhancement, Coding and Error Concealment. Chichester, England: John Wiley & Sons Ltd, 2006. 120. R. Vetter, “Single channel speech enhancement using MDL-based subspace approach in Bark domain,” in Proc. IEEE ICASSP, 2001, pp. 641–644. 121. N. Virag, “Single channel speech enhancement based on masking properties of the human auditory system,” IEEE Trans. Speech Audio Process., vol. 7, pp. 126–137, Mar. 1999. 122. M. R. Weiss, E. Aschkenasy, and T. W. Parsons, “Processing speech signals to attenuate interference,” in Proc. IEEE Symposium on Speech Recognition, 1974, pp. 292–295. 123. S. Werner, J. A. Apolin´ ario, Jr., and M. L. R. de Campos, “On the equivalence of RLS implementations of LCMV and GSC processors,” IEEE Signal Process. Lett., vol. 10, pp. 356–359, Dec. 2003. 124. J. Wexler and S. Raz, “Discrete Gabor expansions,” Elsevier Signal Process., vol. 21, pp. 207–220, Nov. 1990. 125. N. Wiener, Extrapolation, Interpolation and Smoothing of Stationary Time Series. New York: John Wiley & Sons, 1949. 126. P. J. Wolfe and S. J. Godsill, “Efficient alternatives to the Ephraim and Malah suppression rule for audio signal enhancement,” EURASIP J. Applied Signal Process., vol. 2003, pp. 1044–1051, Oct. 2003.

226

References

127. R. Zelinski, “A microphone array with adaptive post-filtering for noise reduction in reverberant rooms,” in Proc. IEEE ICASSP, 1988, pp. 2578–2581. 128. R. Zelinski, “Noise reduction based on microphone array with LMS adaptive post-filtering,” Electron. Lett., vol. 26, pp. 2036–2037, Nov. 1990. 129. C. Zheng, M. Zhou, and X. Li, “On the relationship of non-parametric methods for coherence function estimation,” Elsevier Signal Process., vol. 11, pp. 2863– 2867, Nov. 2008.

Index

adaptive noise cancellation, 202 additive noise, 3, 15 ambient noise, 186 analysis window, 153, 174 ARCH model, 167 babble noise, 177 beamforming, 193 car interior noise, 177 cocktail-party effect, 199 communication cap-based audio system, 184 complex gain, 17, 154 complex number, 43, 45, 138 complex plane, 43, 45, 139 conditional variance, 167, 168 constrained minimization, 169 correlation matrix, 15 cosine matrix, 145 cross-correlation matrix, 32 cross-spectrum, 33, 131 decision-directed SNR estimation, 172 delay-and-sum beamformer, 195 desired signal, 1, 15 diagonalization, 18, 123 differential microphone array, 189 discrete-time Fourier transform (DTFT), 16 distortion measure, 155 domain frequency, 16 generalized KLE, 123

Karhunen-Lo`eve expansion (KLE), 18 STFT, 153 time, 15 transform, 126 echo, 3 eigenvalue decomposition, 18 error signal frequency domain, 32 KLE domain, 34 STFT domain, 159 time domain, 31 transform domain, 131 expectation-maximization, 169 F16 cockpit noise, 177 filter-and-sum beamformer, 195 finite-impulse-response (FIR) filter, 20 first-order differential microphone array, 188 Fourier matrix, 145 frequency-domain filter parametric Wiener, 81 tradeoff, 82 Wiener, 77 frequency-domain optimal filters, 77 fullband input SNR, 127 fullband MSE, 34, 36, 132, 160 fullband noise-reduction factor, 24, 129, 156 fullband normalized MSE, 34, 36, 132, 161 fullband output SNR, 23, 128, 155

228

Index

fullband SPCC, 39, 40 fullband speech-distortion index, 26, 129, 157 fullband speech-reduction factor, 27, 130, 158 gain in signal-to-noise ratio (GSNR), 189 GARCH, 168, 169 GARCH model, 166 Gaussian model, 161, 163 generalized KLE, 123 generalized sidelobe canceller, 198 Hadamard matrix, 146 harmonically-nested subarray, 195 hidden Markov model (HMM), 9 IMCRA, 174 input SNR, 21, 155 intelligibility, 4 interference, 3 interpolation-error power, 63 inverse DTFT, 17 inverse STFT, 153 joint diagonalization, 67 Karhunen-Lo`eve expansion (KLE), 18 analysis, 18 synthesis, 18 KLE-domain filter magnitude subtraction method, 100 parametric Wiener, 100 power subtraction method, 100 tradeoff, 101, 108 Wiener, 95, 105 KLE-domain optimal filters, 95 Lagrange multiplier, 64, 66, 83, 85, 101, 105, 109 linear interpolator, 63 linear transformation, 16 linearly constrained minimum variance, 197 log-spectral amplitude estimation, 164 Lombard effect, 2 LPC model, 9

magnitude squared coherence function (MSCF), 39, 132 magnitude subtraction method, 82, 142 maximum-likelihood (ML), 169 mean-squared error (MSE) criterion, 31, 131, 159 minimum statistics, 88 minimum variance distortionless response, 198 MMSE, 162 MMSE spectral estimation, 162 MMSE-LSA, 162, 164 MMSE-SA, 162, 163 model estimation, 169 MSE frequency domain, 32 KLE domain, 34 STFT domain, 160 time domain, 31 transform domain, 131 multichannel noise reduction, 200 multiparty conferencing, 6 multiple-microphone audio system, 184 musical noise, 166 noise, 1 babbling, 70, 177 car, 68, 177 F16 cockpit, 177 white, 62, 177 noise cancelling microphone, 188 noise estimation, 86 noise problem formulation, 15, 153 noise reduction, 1, 16 noise spectral shaping, 85, 105, 111 noise subspace, 68 noise-reduction factor, 24, 128, 156 noncausal Wiener filter, 78 normalized MSE frequency domain, 33 KLE domain, 35 STFT domain, 160 time domain, 32 transform domain, 131 one-frame-ahead conditional variance, 168 orthogonal matrix, 18 orthonormal vector, 18

Index output SNR, 22, 127 overlap-add, 174 Parseval’s theorem, 126 Pearson correlation coefficient (PCC), 37, 132 performance measure, 21, 155 post-filtering, 199 power spectral density, 17 power subtraction method, 82, 142 propagation step, 171 residual noise frequency domain, 33 KLE domain, 35 STFT domain, 160 time domain, 31 transform domain, 131 reverberation, 3 Schroeder’s noise reduction system, 8 second-order differential microphone array, 188 segmental input SNR, 155 segmental MSE, 160 segmental noise-reduction factor, 156 segmental normalized MSE, 161 segmental output SNR, 155 segmental speech-distortion index, 157 segmental speech-reduction factor, 158 short-time Fourier transform (STFT), 153, 174 signal model, 161 signal-plus-noise subspace, 68 signal-to-noise ratio (SNR), 21, 127, 155 single-channel noise reduction, 201 single-pole recursive average, 89 sound pressure level, 186 spacesuit, 183 spatial filtering, 193 SPCC, 37 between two variables, 37, 132 between two vectors, 38, 133 spectral amplitude estimation, 163 spectral enhancement, 153 spectral enhancement algorithm, 174 spectral enhancement problem, 154 spectral variance model, 166 spectrum, 124

229

speech distortion frequency domain, 33 KLE domain, 35 STFT domain, 160 time domain, 31 transform domain, 131 speech enhancement, 1 speech processing, 1 speech-distortion index, 25, 129, 157 speech-reduction factor, 27, 130, 158 STFT, 153 subband input SNR, 22, 127, 155 subband MSE, 33, 35, 131, 160 subband noise-reduction factor, 24, 128, 156 subband output SNR, 23, 128, 155 subband SPCC, 39, 40 subband speech-distortion index, 25, 129, 157 subband speech-reduction factor, 27, 130, 158 synthesis window, 153, 175 time delay estimation, 211 time difference of arrival, 193 time-domain filter subspace, 67 tradeoff, 64 Wiener, 59 time-domain optimal filters, 59 Toeplitz matrix, 63 transform-domain filter parametric Wiener, 142 tradeoff, 143 Wiener, 138 transform-domain optimal filters, 123 transform-domain representation, 125 transform-domain Wiener filter, 138 unit circle, 43, 45, 139 unitary matrix, 124, 145 update step, 171 voice activity detector (VAD), 87 volatility, 167 volatility clustering, 168 white Gaussian noise, 177