Advanced Mapping of Environmental Data (Geographical Information Systems)

This page intentionally left blank Advanced Mapping of Environmental Data This page intentionally left blank Adva...

Author: Mikhail Kanevski

128 downloads 2047 Views 7MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

This page intentionally left blank

Advanced Mapping of Environmental Data


Advanced Mapping of Environmental Data Geostatistics, Machine Learning and Bayesian Maximum Entropy

Edited by Mikhail Kanevski Series Editor Pierre Dumolard

First published in Great Britain and the United States in 2008 by ISTE Ltd and John Wiley & Sons, Inc. Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permitted under the Copyright, Designs and Patents Act 1988, this publication may only be reproduced, stored or transmitted, in any form or by any means, with the prior permission in writing of the publishers, or in the case of reprographic reproduction in accordance with the terms and licenses issued by the CLA. Enquiries concerning reproduction outside these terms should be sent to the publishers at the undermentioned address: ISTE Ltd 6 Fitzroy Square London W1T 5DX UK

John Wiley & Sons, Inc. 111 River Street Hoboken, NJ 07030 USA

www.iste.co.uk

www.wiley.com

© ISTE Ltd, 2008 The rights of Mikhail Kanevski to be identified as the author of this work have been asserted by him in accordance with the Copyright, Designs and Patents Act 1988. Library of Congress Cataloging-in-Publication Data Advanced mapping of environmental data : geostatistics, machine learning, and Bayesian maximum entropy / edited by Mikhail Kanevski. p. cm. Includes bibliographical references and index. ISBN 978-1-84821-060-8 1. Geology--Statistical methods. 2. Machine learning. 3. Bayesian statistical decision theory. I. Kanevski, Mikhail. QE33.2.S82A35 2008 550.1'519542--dc22 2008016237 British Library Cataloguing-in-Publication Data A CIP record for this book is available from the British Library ISBN: 978-1-84821-060-8 Printed and bound in Great Britain by Antony Rowe Ltd, Chippenham, Wiltshire.

Table of Contents

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi Chapter 1. Advanced Mapping of Environmental Data: Introduction . . . . . 1 M. KANEVSKI 1.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2. Environmental data analysis: problems and methodology . 1.2.1. Spatial data analysis: typical problems. . . . . . . . . . 1.2.2. Spatial data analysis: methodology . . . . . . . . . . . . 1.2.3. Model assessment and model selection . . . . . . . . . 1.3. Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3.1. Books, tutorials . . . . . . . . . . . . . . . . . . . . . . . 1.3.2. Software. . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.4. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.5. References . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . .

1 3 3 5 8 12 12 12 14 15

Chapter 2. Environmental Monitoring Network Characterization and Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 D. TUIA and M. KANEVSKI 2.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2. Spatial clustering and its consequences . . . . . . . . . . . . . 2.2.1. Global parameters . . . . . . . . . . . . . . . . . . . . . . . 2.2.2. Spatial predictions . . . . . . . . . . . . . . . . . . . . . . . 2.3. Monitoring network quantification. . . . . . . . . . . . . . . . 2.3.1. Topological quantification . . . . . . . . . . . . . . . . . . 2.3.2. Global measures of clustering . . . . . . . . . . . . . . . . 2.3.2.1. Topological indices . . . . . . . . . . . . . . . . . . . 2.3.2.2. Statistical indices . . . . . . . . . . . . . . . . . . . . . 2.3.3. Dimensional resolution: fractal measures of clustering . 2.3.3.1. Sandbox method . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

. . . . . . . . . . .

19 20 21 22 23 23 23 23 24 26 27

vi


2.3.3.2. Box-counting method . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.3.3. Lacunarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4. Validity domains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5. Indoor radon in Switzerland: an example of a real monitoring network . 2.5.1. Validity domains. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5.2. Topological index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5.3. Statistical indices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5.3.1. Morisita index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5.3.2. K-function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5.4. Fractal dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5.4.1. Sandbox and box-counting fractal dimension . . . . . . . . . . . 2.5.4.2. Lacunarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.7. References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . .

30 33 34 36 37 37 38 38 39 40 40 42 43 44

Chapter 3. Geostatistics: Spatial Predictions and Simulations . . . . . . . . . 47 E. SAVELIEVA, V. DEMYANOV and M. MAIGNAN 3.1. Assumptions of geostatistics . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2. Family of kriging models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.1. Simple kriging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.2. Ordinary kriging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.3. Basic features of kriging estimation . . . . . . . . . . . . . . . . . . . . 3.2.4. Universal kriging (kriging with trend) . . . . . . . . . . . . . . . . . . . 3.2.5. Lognormal kriging. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3. Family of co-kriging models . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.1. Kriging with linear regression . . . . . . . . . . . . . . . . . . . . . . . . 3.3.2. Kriging with external drift . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.3. Co-kriging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.4. Collocated co-kriging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.5. Co-kriging application example . . . . . . . . . . . . . . . . . . . . . . . 3.4. Probability mapping with indicator kriging. . . . . . . . . . . . . . . . . . . 3.4.1. Indicator coding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.2. Indicator kriging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.3. Indicator kriging applications . . . . . . . . . . . . . . . . . . . . . . . . 3.4.3.1. Indicator kriging for 241Am analysis . . . . . . . . . . . . . . . . . 3.4.3.2. Indicator kriging for aquifer layer zonation . . . . . . . . . . . . . 3.4.3.3. Indicator kriging for localization of crab crowds . . . . . . . . . . 3.5. Description of spatial uncertainty with conditional stochastic simulations 3.5.1. Simulation vs. estimation. . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.2. Stochastic simulation algorithms . . . . . . . . . . . . . . . . . . . . . . 3.5.3. Sequential Gaussian simulation . . . . . . . . . . . . . . . . . . . . . . . 3.5.4. Sequential indicator simulations . . . . . . . . . . . . . . . . . . . . . .

47 49 50 50 51 56 56 58 58 58 59 60 61 64 64 66 69 69 71 74 76 76 77 81 84

Table of Contents

vii

3.5.5. Co-simulations of correlated variables . . . . . . . . . . . . . . . . . . . 88 3.6. References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 Chapter 4. Spatial Data Analysis and Mapping Using Machine Learning Algorithms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 F. RATLE, A. POZDNOUKHOV, V. DEMYANOV, V. TIMONIN and E. SAVELIEVA 4.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2. Machine learning: an overview . . . . . . . . . . . . . . . . . . . . . . 4.2.1. The three learning problems . . . . . . . . . . . . . . . . . . . . . 4.2.2. Approaches to learning from data. . . . . . . . . . . . . . . . . . 4.2.3. Feature selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.4. Model selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.5. Dealing with uncertainties . . . . . . . . . . . . . . . . . . . . . . 4.3. Nearest neighbor methods . . . . . . . . . . . . . . . . . . . . . . . . . 4.4. Artificial neural network algorithms . . . . . . . . . . . . . . . . . . . 4.4.1. Multi-layer perceptron neural network. . . . . . . . . . . . . . . 4.4.2. General Regression Neural Networks . . . . . . . . . . . . . . . 4.4.3. Probabilistic Neural Networks. . . . . . . . . . . . . . . . . . . . 4.4.4. Self-organizing (Kohonen) maps . . . . . . . . . . . . . . . . . . 4.5. Statistical learning theory for spatial data: concepts and examples . 4.5.1. VC dimension and structural risk minimization . . . . . . . . . 4.5.2. Kernels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5.3. Support vector machines . . . . . . . . . . . . . . . . . . . . . . . 4.5.4. Support vector regression . . . . . . . . . . . . . . . . . . . . . . 4.5.5. Unsupervised techniques . . . . . . . . . . . . . . . . . . . . . . . 4.5.5.1. Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5.5.2. Nonlinear dimensionality reduction . . . . . . . . . . . . . . 4.6. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.7. References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . .

95 96 96 100 101 103 107 108 109 109 119 122 124 131 131 132 133 137 141 142 144 146 146

Chapter 5. Advanced Mapping of Environmental Spatial Data: Case Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 L. FORESTI, A. POZDNOUKHOV, M. KANEVSKI, V. TIMONIN, E. SAVELIEVA, C. KAISER, R. TAPIA and R. PURVES 5.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2. Air temperature modeling with machine learning algorithms and geostatistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.1. Mean monthly temperature . . . . . . . . . . . . . . . . . . 5.2.1.1. Data description . . . . . . . . . . . . . . . . . . . . . . 5.2.1.2. Variography . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.1.3. Step-by-step modeling using a neural network . . . .

. . . . . . . 149 . . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

150 151 151 152 153

viii


5.2.1.4. Overfitting and undertraining . . . . . . . . . . . . . . . . . . . 5.2.1.5. Mean monthly air temperature prediction mapping . . . . . . 5.2.2. Instant temperatures with regionalized linear dependencies . . . . 5.2.2.1. The Föhn phenomenon . . . . . . . . . . . . . . . . . . . . . . . 5.2.2.2. Modeling of instant air temperature influenced by Föhn . . . 5.2.3. Instant temperatures with nonlinear dependencies . . . . . . . . . . 5.2.3.1. Temperature inversion phenomenon . . . . . . . . . . . . . . . 5.2.3.2. Terrain feature extraction using Support Vector Machines . . 5.2.3.3. Temperature inversion modeling with MLP . . . . . . . . . . . 5.3. Modeling of precipitation with machine learning and geostatistics. . . 5.3.1. Mean monthly precipitation . . . . . . . . . . . . . . . . . . . . . . . 5.3.1.1. Data description . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.1.2. Precipitation modeling with MLP . . . . . . . . . . . . . . . . . 5.3.2. Modeling daily precipitation with MLP . . . . . . . . . . . . . . . . 5.3.2.1. Data description . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.2.2. Practical issues of MLP modeling . . . . . . . . . . . . . . . . . 5.3.2.3. The use of elevation and analysis of the results . . . . . . . . . 5.3.3. Hybrid models: NNRK and NNRS . . . . . . . . . . . . . . . . . . . 5.3.3.1. Neural network residual kriging . . . . . . . . . . . . . . . . . . 5.3.3.2. Neural network residual simulations . . . . . . . . . . . . . . . 5.3.4. Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4. Automatic mapping and classification of spatial data using machine learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4.1. k-nearest neighbor algorithm . . . . . . . . . . . . . . . . . . . . . . 5.4.1.1. Number of neighbors with cross-validation . . . . . . . . . . . 5.4.2. Automatic mapping of spatial data . . . . . . . . . . . . . . . . . . . 5.4.2.1. KNN modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4.2.2. GRNN modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4.3. Automatic classification of spatial data . . . . . . . . . . . . . . . . 5.4.3.1. KNN classification . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4.3.2. PNN classification . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4.3.3. Indicator kriging classification . . . . . . . . . . . . . . . . . . . 5.4.4. Automatic mapping – conclusions . . . . . . . . . . . . . . . . . . . 5.5. Self-organizing maps for spatial data – case studies . . . . . . . . . . . 5.5.1. SOM analysis of sediment contamination . . . . . . . . . . . . . . . 5.5.2. Mapping of socio-economic data with SOM . . . . . . . . . . . . . 5.6. Indicator kriging and sequential Gaussian simulations for probability mapping. Indoor radon case study . . . . . . . . . . . . . . . . . . . . . . . . . 5.6.1. Indoor radon measurements . . . . . . . . . . . . . . . . . . . . . . . 5.6.2. Probability mapping. . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.6.3. Exploratory data analysis. . . . . . . . . . . . . . . . . . . . . . . . . 5.6.4. Radon data variography . . . . . . . . . . . . . . . . . . . . . . . . . 5.6.4.1. Variogram for indicators . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . .

154 156 159 159 160 163 163 164 165 168 169 169 171 173 173 174 177 179 179 182 184

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

185 185 187 187 188 190 192 193 194 197 199 200 200 204

. . . . . .

. . . . . .

209 209 211 212 216 216

Table of Contents

5.6.4.2. Variogram for Nscores . . . . . . . . . . . . . . . . . . . . . . . . 5.6.5. Neighborhood parameters . . . . . . . . . . . . . . . . . . . . . . . . . 5.6.6. Prediction and probability maps. . . . . . . . . . . . . . . . . . . . . . 5.6.6.1. Probability maps with IK . . . . . . . . . . . . . . . . . . . . . . . 5.6.6.2. Probability maps with SGS . . . . . . . . . . . . . . . . . . . . . . 5.6.7. Analysis and validation of results. . . . . . . . . . . . . . . . . . . . . 5.6.7.1. Influence of the simulation net and the number of neighbors . . 5.6.7.2. Decision maps and validation of results . . . . . . . . . . . . . . 5.6.8. Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.7. Natural hazards forecasting with support vector machines – case study: snow avalanches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.7.1. Decision support systems for natural hazards. . . . . . . . . . . . . . 5.7.2. Reminder on support vector machines . . . . . . . . . . . . . . . . . . 5.7.2.1. Probabilistic interpretation of SVM . . . . . . . . . . . . . . . . . 5.7.3. Implementing an SVM for avalanche forecasting . . . . . . . . . . . 5.7.4. Temporal forecasts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.7.4.1. Feature selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.7.4.2. Training the SVM classifier . . . . . . . . . . . . . . . . . . . . . 5.7.4.3. Adapting SVM forecasts for decision support. . . . . . . . . . . 5.7.5. Extending the SVM to spatial avalanche predictions . . . . . . . . . 5.7.5.1. Data preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.7.5.2. Spatial avalanche forecasting. . . . . . . . . . . . . . . . . . . . . 5.7.6. Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.8. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.9. References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

ix

. . . . . . . . .

217 218 219 219 220 221 221 222 225

. . . . . . . . . . . . . . .

225 227 228 229 230 230 231 232 233 237 237 239 241 241 242

Chapter 6. Bayesian Maximum Entropy – BME . . . . . . . . . . . . . . . . . . 247 G. CHRISTAKOS 6.1. Conceptual framework . . . . . . . . . . . . . . . . . . . . . . . . . 6.2. Technical review of BME . . . . . . . . . . . . . . . . . . . . . . . 6.2.1. The spatiotemporal continuum . . . . . . . . . . . . . . . . . 6.2.2. Separable metric structures . . . . . . . . . . . . . . . . . . . 6.2.3. Composite metric structures . . . . . . . . . . . . . . . . . . . 6.2.4. Fractal metric structures . . . . . . . . . . . . . . . . . . . . . 6.3. Spatiotemporal random field theory . . . . . . . . . . . . . . . . . 6.3.1. Pragmatic S/TRF tools . . . . . . . . . . . . . . . . . . . . . . 6.3.2. Space-time lag dependence: ordinary S/TRF . . . . . . . . . 6.3.3. Fractal S/TRF . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3.4. Space-time heterogenous dependence: generalized S/TRF . 6.4. About BME . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4.1. The fundamental equations . . . . . . . . . . . . . . . . . . . 6.4.2. A methodological outline . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

. . . . . . . . . . . . . .

247 251 251 253 255 256 257 258 260 262 264 267 267 273

x


6.4.3. Implementation of BME: the SEKS-GUI . . 6.5. A brief review of applications . . . . . . . . . . . 6.5.1. Earth and atmospheric sciences . . . . . . . . 6.5.2. Health, human exposure and epidemiology . 6.6. References . . . . . . . . . . . . . . . . . . . . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

275 281 282 291 299

List of Authors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309

Preface

This volume is a collection of lectures and seminars given at two workshops organized by the Institute of Geomatics and Analysis of Risk (IGAR) at the Faculty of Geosciences and Environment of the University of Lausanne (www.unil.ch/igar): – Workshop I, October 2005: “Data analysis and modeling in environmental sciences towards risk assessment and impact on society”; – Workshop II, October 2006 (S4 network modeling tour): “Machine Learning Algorithms for Spatial Data”. During the first workshop many topics related to natural hazards were considered. One of the lectures was given by Professor G. Christakos on the theory and applications of Bayesian Maximum Entropy (BME). The second workshop was organized within the framework of the S4 (Spatial Simulation for Social Sciences, http://s4.parisgeo.cnrs.fr/index.htm) network modeling tour of young researchers. The main topics considered were related to machine learning algorithms (neural networks of different architectures and statistical learning theory) and their applications in geosciences. Therefore, the book is actually a composition of three topics concerning the analysis, modeling and presentation of spatiotemporal data: geostatistical methods and models, machine learning algorithms and the Bayesian maximum entropy approach. All these three topics have quite different theoretical hypotheses and background assumptions. Usually, they are published in different volumes. Of course, it was not possible to cover both introductory and advanced topics taking into account the limits of the book. Authors were free to select their topics and to present some theoretical concepts along with simulated/illustrative and real case studies. There are some traditional examples of environmental data mapping using different techniques but also

xii


advanced topics, which cover recent research activities. Obviously, this volume is not a textbook on geostatistics, machine learning and BME. Moreover, it does not cover all currently available techniques for environmental data analysis. Nevertheless, it tries to explain the main theoretical concepts and to give an overview of applications for the selected methods and models. We hope that the book will be useful both for professionals and experts interested in environmental data analysis and mapping. The book can expand the knowledge of tools currently available for the analysis of spatiotemporal data. Let us remember that in general the selection of an appropriate method should depend on the quality and quantity of data and on the objectives of the study. The book consists of six chapters. Chapter 1 is an introduction to the topics of environmental data mapping. Chapter 2 deals with a characterization of monitoring networks and studies monitoring network clustering and its effect on spatial predictions. The main focus is given to global cluster detection methods such as fractal dimension. Integration of the characteristics of the prediction space is also discussed via the concept of validity domain. Chapter 3 is devoted to traditional and recently developed models in geostatistics. Geostatistics is still a dynamically developing discipline. It has contributed to different topics of data analysis during the last 50 years. Chapter 4 gives an introduction to machine learning algorithms and explains some particular models widely used for environmental data: multilayer perceptron, general regression neural networks, probabilistic neural networks, self-organizing maps, support vector machines and support vector regression. Chapter 5 describes real case studies with the application of geostatistical models and machine learning algorithms. The presented case studies cover different topics: topo-climatic modeling, pollution mapping, analysis of socio-economic spatial data, indoor radon risk and natural hazard risk assessment. An interesting section deals with a so-called “automatic mapping” (spatial prediction and spatial classification) using general regression and probabilistic neural networks. Such applications can be important in on-line data analysis and environmental decision support systems. Chapter 6 is completely devoted to the Bayesian maximum entropy approach to spatiotemporal data analysis. It is a separate part of the book presenting BME from conceptual introduction to recent case studies dealing with environmental and epidemiological applications.

Preface

xiii

We would like to acknowledge the Faculty of Geosciences and Environment of the University of Lausanne for the financial support of both workshops. The S4 network (Professor Denise Pumain) played an important role in organizing the second workshop. The scientific work resulting in a collection of papers presented in this volume is the result of several projects, financed by the Swiss National Science Foundation: 105211-107862; 100012-113506, 200021-113944, Scope project IB7310-110915, and the Russian Foundation for Fundamental Research: 07-0800257. The support for the preparation of Chapter 6 was provided by a grant from the California Air Resources Board, USA (Grant No. 55245A). We acknowledge the following institutions and offices that have kindly provided us with data: Swiss Federal Office for Public Health, MeteoSwiss, Swisstopo, Swiss office of statistics, CIPEL (Lausanne), and sportScotland Avalanche Information Service (SAIS) for the avalanche recordings and meteorological data in the Lochaber region of Scotland, UK. I would like to acknowledge the authors who have contributed directly to this volume for their interesting works and fruitful collaboration. Finally, all the authors acknowledge Professor P. Dumolard (who initiated this project) and ISTE Ltd. for the collaboration and opportunity to publish this book. M. Kanevski Lausanne, April 2008


Chapter 1

Advanced Mapping of Environmental Data: Introduction

1.1. Introduction In this introductory chapter we describe general problems of spatial environmental data analysis, modeling, validation and visualization. Many of these problems are considered in detail in the following chapters using geostatistical models, machine learning algorithms (MLA) of neural networks and Support Vector Machines, and the Bayesian Maximum Entropy (BME) approach. The term “mapping” in the book is considered not only as an interpolation in two- or threedimensional geographical space, but in a more general sense of estimating the desired dependencies from empirical data. The references presented at the end of this chapter cover the range of books and papers important both for beginners and advanced researchers. The list contains both classical textbooks and studies on contemporary cutting-edge research topics in data analysis. In general, mapping can be considered as: a) a spatiotemporal classification problem such as digital soil mapping and geological unit classification, b) a regression problem such as mapping of pollution and topo-climatic modeling, and c) a problem of probability density modeling, which is not a mapping of values but “mapping” of probability density functions, i.e., the local or joint spatial distributions conditioned on data and available expert knowledge.

Chapter written by M. KANEVSKI.

2


As well as some necessary theoretical introductions to the methods, an important part of the book deals with the presentation of case studies. These are both simulated problems used to illustrate the essential concepts and real life applications. These case studies are important complementary parts of the current volume. They cover a wide range of applications: environmental data analysis, pollution mapping, epidemiologic spatiotemporal data analysis, socio-economic data classification and clustering. Several case studies consider multivariate data sets, where variables can be dependent (linearly or nonlinearly correlated) or independent. Common to all case studies is that data are geo-referenced, i.e. they are located at least in a geographical space. In a more general sense the geographical space can be enriched with additional information, giving rise to a high dimensional geo-feature space. Geospatial data can be categorical (classes), continuous (fields) or distributions (probability density functions). Let us remember that one of the simplest problems – the task of spatial interpolation from discrete measurements to continuous fields – has no single solution. Even with a very simple interpolation method just by changing one or two tuning parameters many different “maps” can be produced. Here we are faced with an extremely important question of model assessment and model selection. The selection of the method for data analysis, modeling and predictions depends on the quantity and quality of data, the expert knowledge available and the objectives of the study. In general, two fundamental approaches when working with data are possible: deterministic models, including the analysis of data using physical models and deterministic interpolations, or statistical models which interpret the data as a realization of a random/stochastic process. In both cases models and methods depend on some hypotheses and have some parameters that should be tuned in order to apply the model correctly. In many cases these two groups merge, and deterministic models might have their “statistical” side and vice versa. Statistical interpretation of spatial environmental data is not trivial because usually only one realization (measurements) of the phenomena under study exists. These cases are, for example, geological data, pollution after an accident, etc. Therefore, some fundamental hypotheses are very important in order to make statistical inferences when only one realization is available: ergodicity, second-order stationarity, intrinsic hypotheses (see Chapter 3 for more detail). While some empirical rules exist, these hypotheses are very difficult to verify rigorously in most cases. An important aspect of spatial and spatiotemporal data is the anisotropy. This is the dependence of the spatial variability on the direction. This phenomenon can be

Introduction

3

detected and characterized with structural analysis such as the variography presented below. Almost all of the models and algorithms considered in this book (geostatistics, MLA, BME) are based on the statistical interpretation of data. Another general view on environmental data modeling approaches is to consider two major classes: model-dependent approaches (geostatistical models – Chapter 3 and BME – Chapter 6) and data-driven adaptive models (machine learning algorithms – Chapter 4). Being applied without the proper understanding and lacking interpretability, the data-driven models were often considered as black or gray box models. Obviously, each data modeling approach has its own advantages and drawbacks. In fact, both approaches can be used as complementary tools, resulting in hybrid models that can overcome some of the problems. From a machine learning point of view the problem of spatiotemporal data analysis can be considered as a problem of pattern recognition, pattern modeling and pattern prediction or pattern completion. There are several major classes of learning approaches: – supervised learning. For example, these are the problems of classification and regression in the space of geographical coordinates (inputs) based on the set of available measurements (outputs); – unsupervised learning. These are the problems with no outputs available, where the task is to find structures and dependencies in the input space: probability density modeling, spatiotemporal clustering, dimensionality reduction, ranking, outlier/novelty detection, etc. When the use of these structures can improve the prediction for a small amount of available measurements, this setting is called semisupervised learning. Other directions such as reinforcement learning exist but are rarely used in environmental spatial data analysis and modeling. 1.2. Environmental data analysis: problems and methodology 1.2.1. Spatial data analysis: typical problems First let us consider some typical problems arising when working with spatial data.

4


Figure 1.1. Illustration of the problem of environmental data mapping

Given measurements of several variables (see Figure 1.1 for the illustration) and a region of the study, typical problems related to environmental data mapping (and beyond, such as risk mapping, decision-oriented mapping, simulations, etc.) can be listed as follows: – predicting a value at a given point (marked by “?” in Figure 1.1, for example). If it is the only point of interest, perhaps the best way is simply to take a measurement there. If not, a model should be developed. Both deterministic and statistical models can be used; – building a map using given measurements. In this case a dense grid is usually developed over the region of study taking into account the validity domain (see Chapter 2) and at each grid node predictions are performed finally giving rise to the raster model of spatial predictions. After post-processing of this raster model different presentations are possible – isolines, 3D surfaces, etc. Both deterministic and statistical models can be used; – taking into account measurement errors. Errors can be either independent or spatially correlated. Statistical treatment of data is necessary; – estimating the prediction error, i.e. predicting both unknown value and its uncertainty. This is a much more difficult question. Statistical treatment of data is necessary;

Introduction

5

– risk mapping, which is concerned with uncertainty quantification for the unknown value. The best approach is to estimate a local probability density function, i.e. mapping densities using data measurements and expert knowledge; – joint predictions of several variables or prediction of a primary variable using auxiliary data and information. Very often in addition to the main variable there are other data (secondary variables, remote sensing images, digital elevation models, etc.) which can contribute to the analysis of the primary variable. Additional information can be “cheaper” and more comprehensive. There are several geostatistical models of co-predictions (co-kriging, kriging with external drift) and co-simulations (e.g. sequential Gaussian co-simulations). As well as being more complete, secondary information usually has better spatial and dimensional resolutions which can improve the quality of final analysis and recover missing information in the principal monitoring network. This is an interesting topic of future research; – optimization of the monitoring network (design/redesign). A fundamental question is always where to go and what to measure? How can we optimize the monitoring network in order to improve predictions and reduce uncertainties? At present there are several possible approaches: uncertainty/variance-based, Bayesian approach, space filling, optimization based on support vectors (see references); – spatial stochastic conditional simulations or modeling of spatial uncertainty and variability. The main idea here is to develop a spatial Monte Carlo model which can produce (generate) many realizations of the phenomena under study (random fields) using available measurements, expert knowledge and well defined criteria. In geostatistics there are several parametric and non-parametric models widely used in real applications (Chapter 3 and references therein). Post-processing of these realizations gives rise to different decision-oriented maps. This is the most comprehensive and the most useful information for an intelligent decision making process; – integration of data/measurements with physical models. In some cases, in addition to data science-based models – meteorological models, geophysical models, hydrological models, geological models, models of pollution dispersion, etc. are available. How can we integrate/assimilate models and data if we do not want to use data only for the calibration purposes? How can we compare patterns generated from data and models? Are they compatible? How can we improve predictions and models? These fundamental topics can be studied using BME. 1.2.2. Spatial data analysis: methodology The generic methodology of spatial data analysis and modeling consists of several phases. Let us recall some of the most important.

6


– Exploratory spatial data analysis (ESDA). Visualization of spatial data using different methods of presentation, even with simple deterministic models helps to detect data errors and to understand if there are patterns, their anisotropic structures, etc. An example of sample data visualization using Voronoï polygons and Delaunay triangulation is given in Figure 1.2. The presence of spatial structure and the WestEast major axis of anisotropy are evident. Geographical Information Systems (GIS) can also be used as tools both for ESDA and for the presentation of the results. ESDA can also be performed within moving/sliding windows. This regionalized ESDA is a helpful tool for the analysis of complex non-stationary data.

Figure 1.2. Visualization of raw data (left) using Voronoï polygons and Delaunay triangulation (right)

– Monitoring network analysis and descriptions. The measuring stations of an environmental monitoring network are usually spatially distributed in an inhomogenous manner. The problem of network homogenity (clustering and preferential sampling) is closely connected to global estimations, to the theoretical possibility of detecting phenomena with a monitoring network of the given design. Different topological, statistical and fractal measures are used to quantify spatial and dimensional resolutions of the networks (see details in Chapter 2). – Structural analysis (variography). Variography is an extremely important part of the study. Variograms and other functions describing spatial continuity (rodogram, madogram, generalized relative variograms, etc.) can be used in order to characterize the existence of spatial patterns (from a two-point statistical point of view) and to quantify the quality of machine learning modeling using variography of the residuals. The theoretical formula for the variogram calculation of the random variable Z(x) under the intrinsic hypotheses is given by:

J ( x, h )

1 2

Var ^Z (x) Z (x h)`

^

1 2 E Z ( x) Z ( x h ) 2

`

J (h)

where h is a vector separating two points in space. The corresponding empirical estimate of the variogram is given by the following formula

Introduction

J ij (h)

7

1 N (h ) ¦ Z i ( x ) Z i ( x h ) 2 2 N (h) i 1

where N(h) is a number of pairs separated by vector h. The variogram has the same importance for spatial data analysis and modeling as the auto-covariance function for time series. Variography should be an integral part of any spatial data analysis independent of the modeling approach applied (geostatistics or machine learning). In Figure 1.3 the experimental variogram rose for the data shown in Figure 1.2 is presented. A variogram rose is a variogram calculated in several directions and at many lag distances. A variogram rose is a very useful tool for detecting spatial patterns and their correlation structures. The anisotropy can be clearly seen in Figure 1.3. – Spatiotemporal predictions/simulations, modeling of spatial variability and uncertainty, risk mapping. The following methods are considered in this book: - Geostatistics (Chapter 3). Geostatistics is a well known approach developed for spatial and spatiotemporal data. It was established in the middle of the 20th century and has a long successful history of theoretical developments and applications in different fields. Geostatistics treats data as realizations of random functions. The geostatistical family of kriging models provides linear and nonlinear modeling tools for spatial data mapping. Special models (e.g. indicator kriging) were developed to “map” local probability density functions, i.e. modeling of uncertainties around unknown values. Geostatistical conditional stochastic simulations are a type of spatial Monte Carlo generator which can produce many equally probable realizations of the phenomena under study based on well defined criteria. - Machine Learning Algorithms (Chapter 4). Machine Learning Algorithms (MLA) offer several useful information processing capabilities such as nonlinearity, universal input-output mapping and adaptivity to data. MLA are nonlinear universal tools for obtaining and modeling data. They are excellent exploratory tools. Correct application of MLA demands profound expert knowledge and experience. In this book several architectures widely used for different applications are presented: neural networks: multilayer perceptron (MLP), probabilistic neural network (PNN), general regression neural network (GRNN), self-organizing (Kohonen) maps (SOM), and from statistical learning theory: Support Vector Machines (SVM), Support Vector Regression (SVR), and other kernel-based methods. At present, the conditional stochastic simulations using machine learning is an open question.

8


Figure 1.3. Experimental variogram rose for the data from Figure 1.2

- Bayesian Maximum Entropy (Chapter 6). Bayesian Maximum Entropy (BME) is based on recent developments in spatiotemporal data modeling. BME is extremely efficient in the integration of general expert knowledge and specific information (e.g. measurements) for the spatiotemporal data analysis, modeling and mapping. Under some conditions BME models are reduced to geostatistical models. – Model assessments/model validation. This is a final phase of the study. The “best” models are selected and justified. Their generalization capabilities are estimated using a validation data set – a completely independent data set never used to develop and to select a model. – Decision-oriented mapping. Geomatics tools such as Geographical Information Systems (GIS) can be used to efficiently visualize the prediction results. The resulting maps may include not only the results of data modeling but other thematic layers important for the decision making process. – Conclusions, recommendations, reports, communication of the results. 1.2.3. Model assessment and model selection Now let us return to the question of data modeling. As has already been mentioned, in general, there is no single solution to this problem. Therefore, an extremely important question deals with model selection and model assessment procedures. First we have to choose the “best” model and then estimate its generalization abilities, i.e. its predictions on a validation data set which has never been used for model development.

Introduction

9

Model selection and model assessment have two distinct goals [HAS 01]: – Model selection: estimating the performance of different models in order to choose the best one: the most appropriate, the most adapted to data, best matching some prior knowledge, etc. – Model assessment: having chosen a model, model assessment deals with estimating its prediction error on new independent data (generalization error). In practice these problems are solved either using different statistical techniques or empirically by splitting the data into three subsets (Figure 1.4): training data, testing data and validation data. Let us note that in this book the traditional definition used in environmental modeling is used. The machine learning community splits data in the following order: training/validation/testing. The training data subset is used to train the selected model (not necessarily the optimal or best model); the testing data subset is used to tune hyper-parameters and/or for the model selection, and the validation data subset is used to assess the ability of the selected model to predict new data. The validation data subset is not used during the training and model selection procedure. It can be considered as a completely independent data set or as additional measurements. The distribution of percentages between data subsets is quite free. What is important is that all subsets characterize the phenomena under study in a similar way. For environmental spatial data it can be the clustering structure, the global distributions and variograms which should be similar for all subsets. Model selection and model assessment procedures are extremely important especially for data-driven machine learning algorithms, which mainly depend on data quality and quantity and less on expert knowledge and modeling assumptions.

Figure 1.4. Splitting of raw data

10


A scheme of the generic methodology of using machine learning algorithms for spatial environmental data modeling is given in Figure 1.5. The methodology is similar to any other statistical analysis of data. The first step is to extract useful information (which should be quantified, e.g. as information described by spatial correlations) from noisy data. Then, the quality of modeling has to be controlled by analyzing the residuals. The residuals of training, testing and validation data should be uncorrelated white noise. Unfortunately in many applied publications this important step of the residual analysis is neglected. Another important aspect of environmental decisions both during environmental modeling or environmental data analysis and forecasting deals with uncertainties of the corresponding modeling results. Uncertainties have great importance in intelligent decisions; sometimes they can be even more important than the particular prediction values. In statistical models (geostatistics, BME) this procedure is inherent and under some hypotheses confidence intervals can be derived. With MLA this is a slightly more difficult problem, but many theoretical and operational solutions have already been proposed.

Figure 1.5. Methodology of MLA application for spatial data analysis

Introduction

11

Concerning mapping and visualization of the results one possibility to summarize both predictions and uncertainties is to use “thick isolines” which characterize the uncertainty of spatial predictions (see Figure 1.6). For example, under some hypotheses which depend on the applied model, the interpretation is that with a probability of 95% an isoline of the predefined decision level can be found in the thick zone. Correct visualization is important in communicating the results to decision makers. It can be used also for monitoring network optimization procedures by demonstrating regions with high or unacceptable uncertainties. Let us note that such a visualization of predictions and uncertainties is quite common in time series analysis.

Figure 1.6. Combining predictions with uncertainties: “thick isolines”

In this section some basic problems of spatial data analysis, modeling and visualization were presented. Model-based (geostatistics, BME) methods and datadriven algorithms (MLA) were mentioned as possible modeling approaches to these tasks. Correct application of both of them demands profound expert knowledge of data, models, algorithms and their applicability. Taking into account the complexity of spatiotemporal data analysis, the availability of good literature (books, tutorials, papers) and software modules/programs with user-friendly interfaces are important for learning and applications.

12


In the following section some of the available resources such as books and software tools are given. The list is very short and far from being complete for this very dynamic research discipline, sometimes called environmental data mining. 1.3. Resources Some general information, including references to the conferences, tutorials and software for the methods considered in this book can be found on the Internet, in particular on the following sites: – web resources on geostatistics and spatial statistics can be found at http://www.ai-geostats.org; – on machine learning: http://www kernel-machines.org/, http://www.supportvector.net/; http://mloss.org/about/ – machine learning open source software; http://www.cs.iastate.edu/~honavar/Courses/cs673/machine-learning-courses.html – index of ML courses, http://www.patternrecognition.co.za/tutorials html – machine learning tutorials; very good tutorials on statistical data mining can be found on-line at http://www.autonlab.org/tutorials/list html; – Bayesian maximum entropy: some resources related to Bayesian maximum entropy (BME) methods. For a more complete list or references see Chapter 6; see also the BMELab site at http://www.unc.edu/depts/case/BMElab. 1.3.1. Books, tutorials The list of books, given in the reference section below, is not complete but gives good references on introductory and advanced topics presented in the book. Some of these are more theoretical, while some concentrate more on the applications and case studies. In any case, most of them can be used as text books for the educational purposes as well as references for research. 1.3.2. Software All contemporary data analysis and modeling approaches are not feasible without powerful computers and good software tools. This book does not include a CD with software modules (unfortunately). Therefore, below we would like to recommend some cheap and “easy to find” software with short descriptions. – GSLIB: a geostatistical library with Fortran routines [DEU 1997]. The GSLIB library, which first appeared in 1992, was an important step in geostatistics applications and stimulated new developments. It gave many researchers and

Introduction

13

students the possibility of starting with geostatistical models and learning corresponding algorithms having access to the codes. Description: the GSLIB modeling library covers both geostatistical predictions (family of kriging models) and conditional geostatistical simulations. There is a version of GSLIB with user interfaces which can be found at http://www.statios.com/WinGslib. – S-GeMS is a piece of software for 3D geostatistical modeling. Description: it implements many of the classical geostatistics algorithms, as well as new developments made at the SCRF lab, Stanford University. It includes a selection of traditional and the most recent geostatistical models: kriging, co-kriging, sequential Gaussian simulation, sequential indicator simulation, multi-variate sequential Gaussian and indicator simulation, multiple-point statistics simulation, as well as standard data analysis tools (histogram, QQ-plots, variograms) and interactive 3D visualization. Open source code is available at http://sgems.sourceforge net. – Geostat Office (GSO). An educational version of GSO comes with a book [KAN 04]. The GSO package includes geostatistical tools and models (variography, spatial predictions and simulations) and neural networks (multilayer perceptron, general regression neural networks and probabilistic neural networks). – Machine Learning Office (MLO) is a collection of machine learning software modules: multilayer perceptron, radial basis functions, general regression and probabilistic neural networks, support vector machines, self-organizing maps. MLO is a set of software tools accompanying the book [KAN 08]. – R (http://www.r-project.org). R is a free software environment for statistical computing and graphics. It is a GNU project which is similar to the S language and environment. There are several contributed modules dedicated to geostatistical models and to machine learning algorithms. – Netlab [NAB 01]. This consists of a toolbox of Matlab® functions and scripts based on the approach and techniques described in “Neural Networks for Pattern Recognition” by Christopher M. Bishop, (Oxford University Press, 1995), but also including more recent developments in the field. http://www.ncrg.aston.ac.uk/netlab. – LibSVM: http://www.csie ntu.edu.tw/~cjlin/libsvm is quite a popular library for Support Vector Machines. – TORCH machine learning library (http://www.torch.ch). The tutorial on the library, http://www.torch.ch/matos/tutorial.pdf, presents TORCH as a machine learning library, written in C++, and distributed under a BSD license. The ultimate objective of the library is to include all of the state-of-the-art machine learning algorithms, for both static and dynamic problems. Currently, it contains all sorts of artificial neural networks (including convolutional networks and time-delay neural networks), support vector machines for regression and classification, Gaussian mixture models, hidden Markov models, k-means, k-nearest neighbors and Parzen

14


windows. It can also be used to train a connected word speech recognizer. And last but not least, bagging and adaboost are ready to use. – Weka: http://www.cs.waikato.ac.nz/~ml/weka. Weka is a collection of machine learning algorithms for data-mining tasks. The algorithms can either be applied directly to a dataset or taken from your own Java code. Weka contains tools for data pre-processing, classification, regression, clustering, association rules and visualization. It is also well-suited for developing new machine learning schemes. – Machine Learning Open Source Software (MLOSS): http://mloss.org/about. The objective of this new interesting project is to support a community creating a comprehensive open source machine learning environment. – SEKS-GUI (Spatiotemporal Epistematics Knowledge Synthesis software library and Graphic User Interface). Description: advanced techniques for modeling and mapping spatiotemporal systems and their attributes based on theoretical modes, concepts and methods of evolutionary epistemology and modern cognition technology. The interactive software library of SEKS-GUI explores heterogenous space-time patterns of natural systems (physical, biological, health, social, financial, etc.); accounts for multi-sourced system uncertainties; expresses the system structure using space-time dependence models (ordinary and generalized); synthesizes core knowledge bases, site-specific information, empirical evidence and uncertain data; and generates meaningful problem solutions that allow an informative representation of the real-world system using space-time varying probability functions and the associated maps (predicted attribute distributions, heterogenity patterns, accuracy indexes, system risk assessment, etc.). http://geography.sdsu.edu/Research/Projects/SEKS-GUI/SEKS-GUI.html. Manual: Kolovos, A., H-L Yu, and Christakos, G., 2006. SEKS-GUI v.0.6 User Manual. Dept. of Geography, San Diego State University, San Diego, CA. – BMELib Matlab library (Matlab®) and its applications can be found on http://www.unc.edu/depts/case/BMElab/. 1.4. Conclusion The problem of spatial and spatiotemporal data analysis is becoming more and more important: many monitoring stations around the world are collecting high frequency data on-line, satellites produce a huge amount of information about Earth on a daily basis, an immense amount of data is available within GIS. Environmental data are multivariate and noisy; highly variable at many geographical scales – from local variability in hot spots to regional trends; many of them are unique (only one realization of the phenomena under study); usually environmental data are spatially non-stationary.

Introduction

15

The problem of the reconstruction of random fields using discrete data measurements has no single solution. Several important, and difficult to verify, hypotheses have to be accepted and tuning of the model-dependent parameters has to be carried out before arriving at a “unique and in some sense the best” solution. In general, different data analysis approaches – both model-based and datadriven can be considered as complementary. For example, MLA can be efficiently used already at the phase of exploratory data analysis or for de-trending in a hybrid scheme. Moreover, there are links between these two groups of methods, such that, under some conditions, kriging (as a Gaussian process) can be considered as a particular neural network and vice versa. Therefore, in this book, three currently different approaches are presented as possible solutions to the same problem of analysis and mapping of spatial data. Each of them has its own advantages and drawbacks in comparison with the others. In some cases they have quite unique properties to solve more specific tasks: geostatistical simulations and BME to model joint probability density functions (random fields), MLA when working with high dimensional and multivariate data. Hybrid models based on both approaches can overcome some difficulties and produce better results. We propose to apply different methods and tools in order to produce alternative and complementary results which can improve decision-making processes. 1.5. References [ABE 05] ABE S., Support Vector Machines for Pattern Classification, Springer, 2005. [BIS 07] BISHOP C.M., Pattern Recognition and Machine Learning, Springer, 2007. [CHE 98] CHERKASSKY V. and MULIER F., Learning from Data, John Wiley & Sons, 1998. [CHI 99] CHILES J.-P., DELFINER P., Geostatistics: Modelling Spatial Uncertainty, Wiley series in probability and statistics, John Wiley and Sons, 1999. [CHR 92] CHRISTAKOS G., Random Field Models in Earth Sciences, Academic Press, San Diego, CA, 1992. [CHR 98] CHRISTAKOS G. and HRISTOPULOS D.T., Spatiotemporal Environmental Health Modelling, Kluwer Academic Publ., Boston, MA, 1998. [CHR 00a] CHRISTAKOS G., Modern Spatiotemporal Geostatistics, Oxford University Press, New York, 2000. [CHR 02c] CHRISTAKOS G., BOGAERT P. and SERRE M.L., Temporal GIS, SpringerVerlag, New York, NY, with CD-ROM, 2002.

16


[CHR 05] CHRISTAKOS G., OLEA R.A., SERRE M.L., YU H.L. and WANG L-L., Interdisciplinary Public Health Reasoning and Epidemic Modelling: The Case of Black Death, Springer-Verlag, New York, NY, 2005. [CRE 93] CRESSIE, N., Statistics for Spatial Data, John Wiley and Sons, NY, 1993. [CRI 00] CRISTIANINI N. and SHAWE-TAYLOR J., Support Vector Machines, Cambridge University Press, 2000. [DAV 88] DAVID M., Handbook of Applied Advanced Geostatistical Ore Reserve Estimation, Elsevier Science Publishers, Amsterdam B.V., 216 p., 1988. [DEU 97] DEUTSCH C.V. and JOURNEL A.G., GSLIB: Geostatistical Software Library and User’s Guide, Oxford University Press, 1997. [DOB 07] DOBESCH H., DUMOLARD P., and DYRAS I (eds.), Spatial Interpolation for Climate Data: The Use of GIS in Climatology and Meteorology, Geographical Information Systems series, ISTE, 2007. [DUB 03] DUBOIS G., MALCZEWSKI J., and DE CORT M. (eds.), Mapping Radioactivity in the Environment, Spatial Interpolation Comparison 97, European Commission, JRC Ispra, EUR 20667, 2003. [DUB 05] DUBOIS G. (ed.), Automatic Mapping Algorithms for Routine and Emergency Data, European Commission, JRC Ispra, EUR 21595, 2005. [DUD 01] DUDA R., HART P. and STORK D., Pattern Classification, 2nd edition, John Wiley & Sons, 2001. [GAN 63] GANDIN L.S., Objective Analysis of Meteorological Fields, Israel program for scientific translations, 1963, Jerusalem. [GOO 97] GOOVAERST P., Geostatistics for Natural Resources Evaluation, Oxford University Press, 1997. [GRU 06] DE GRUIJTER J., BRUS D., BIERKENS M.F.P. and KNOTTERS M., Sampling for Natural Resource Monitoring, Springer, 2006. [GUY 06] GUYON I., GUNN S., NIKRAVESH M., and ZADEH L. (eds.), Feature Extraction: Foundations and Applications, Springer, 2006. [HAS 01] HASTIE T., TIBSHIRANI R., and FRIEDMAN J., The Elements of Statistical Learning, Springer, 2001. [HAY 98] HAYKIN S., Neural Networks: a Comprehensive Foundation, Pearson Higher Education, 2nd edition, 842 p., 1999. [HIG 03] HIGGINS N. A. and JONES J. A., Methods for Interpreting Monitoring Data Following an Accident in Wet Conditions, National Radiological Protection Board, Chilton, Didcot, 2003. [HYV 01] HYVARINEN A., KARHUNEN J., OJA E., Independent Component Analysis, Wiley Interscience, 2001. [ISA 89] ISAAKS E., SHRIVASTAVA M., Applied Geostatistics, Oxford University Press, 1989.

Introduction

17

[ISA 90] ISAAKS E. H. and SRIVASTAVA R. M., An Introduction to Applied Geostatistics, Oxford University Press, 1990. [JEB 04] JEBARA T., Machine Learning: Discriminative and Generative, Kluwer Academic Publ., 2004. [JOU 78] JOURNEL A.G. and HUIJBREGTS C.J., Mining Geostatistics, Academic Press, 600 p., London, 1978. [KAN 04] KANEVSKI and MAIGNAN, M., Analysis and Modelling of Spatial Environmental Data, EPFL Press, 2004. [KAN 08] KANEVSKI M., POZDNOUKHOV A. and TIMONIN V., Machine Learning Algorithms for Environmental Spatial Data. Theory, Applications and Software, EPFL Press, Lausanne, 2008. [KOH 00] KOHONEN T., Self-Organising Maps, Springer, NY, 2000. [LEE 07] LEE J and VERLEYSEN M., Nonlinear Dimensionality Reduction, Springer, NY, 2007. [LEN 06] LE N.D. and ZIDEK J.V., Statistical Analysis of Environmental Space-Time Processes, Springer, NY, 2006. [LLO 06] LLOYD C.D., Local Models for Spatial Analysis, CRC Press, 2006. [MAT 63] MATHERON G., Principles of Geostatistics Economic Geology, vol. 58, December 1963, p. 1246-1266. [MUL 07] MULLER W.G., Collecting Spatial Data. Optimum Design of Experiments for Random Fields, 3rd edition, Springer, NY, 2007. [NAB 01] NABNEY I., Netlab: Algorithms for Pattern Recognition, Springer, 2001. [RAS 06] RASMUSSEN C.E. and WILLIAMS C.K.I., Gaussian Processes for Machine Learning, MIT Press, 2006. [SCH 05] SCHABENBERGER O. and GOTWAY C., Statistical Methods for Spatial Data Analysis, Chapman and Hall/CRC, 2005. [SCH 06] SCHÖLKOPF B. et al. (eds.), Semi-Supervised Learning, Springer, 2006. [SCH 98] SCHÖLKOPF B., SMOLA A., and MÜLLER K., “Nonlinear Component Analysis as a Kernel Eigenvalue Problem”, Neural Computation, vol. 10, 1998, p. 1299-1319. [SHA 04] SHAWE-TAYLOR J. and CRISTIANINI N., Kernel Methods for Pattern Analysis, Cambridge University Press, 2004. [VAP 06] VAPNIK V., Estimation of Dependences Based on Empirical Data (2nd Edition), Springer, 2006. [VAP 95] VAPNIK V., The Nature of Statistical Learning Theory, Springer, 1995. [VAP 98] VAPNIK V., Statistical Learning Theory, Wiley, 1998. [WAC 95] WACKERNAGEL H., Multivariate Geostatistics, 3rd edition, Springer-Verlag, 387 p., Berlin, 2003.


Chapter 2

Environmental Monitoring Network Characterization and Clustering

2.1. Introduction The quality of environmental data analysis and propagation of errors are heavily affected by the representativity of the initial sampling design [CRE 93, DEU 97, KAN 04a, LEN 06, MUL 07]. Geostatistical methods such as kriging are related to field samples, whose spatial distribution is crucial for the correct detection of the phenomena. Literature about the design of environmental monitoring networks (MN) is widespread and several interesting books have recently been published [GRU 06, LEN 06, MUL 07] in order to clarify the basic principles of spatial sampling design. In [POZ 06] a new approach for spatial sampling design (monitoring networks optimization) based on Support Vector Machines was proposed. Nonetheless, modelers often monitoring networks that suffer Clustering can be related to the reaching certain regions. Figure networks.

receive real data coming from environmental from problems of non-homogenity (clustering). preferential sampling or to the impossibility of 2.1 shows three examples of real monitoring

Chapter written by D. TUIA and M. KANEVSKI.

20


Figure 2.1. Examples of clustered MN: (top-left) Cs137 survey in Briansk region (Russia); (top-right) heavy metals survey in Japan; (bottom-right) indoor radon survey in Switzerland

In order to deal with this problem, declustering methods have been developed, to estimate the non-biased global parameters by weighting the distribution function according to the degree of spatial clustering [DEU 97]. Several specific declustering techniques have been proposed, going from simple random and cell methods to Maximum Likelihood-based [ALL 00], two-point declustering [RIC 02] and more complex approaches based on Bayesian Maximum Entropy formalism [KOV 04]. Declustering of clustered preferential sampling for histogram and semivariogram inference was proposed in [OLE 07]. Declustering methods are delicate and are linked to an unavoidable loss of the initial information. In that sense, a rigorous characterization of the MN is necessary in order to understand whether or not these operations are necessary. This chapter deals with exploratory spatial data analysis, paying particular attention to the quantitative characterization of MN, in order to give to the analyst the tools necessary to evaluate the adequacy of a network to detect an environmental phenomenon. 2.2. Spatial clustering and its consequences Spatial clustering of an MN can influence global estimations and spatial predictions and leads to erroneous conclusions about environmental phenomena such as pollution. In this chapter, the term clustering is used in a purely spatial context: only the spatial repartition of samples is considered and clusters in


21

functional/variable space (such as pollutant concentrations) are not considered (the functional approach, like functional box-counting, can generalize most of the measures considered below [LOV 87]). In this sense, clustering can be defined as the spatial non-homogenity of measurement points. Figure 2.2 shows two monitoring networks: the first is characterized by a random repartition of samples, while the second is clustered.

Figure 2.2. Example of MN: (left) random distribution of samples; (right) clustered distribution of samples

The measures described in this chapter imply the spatial stationarity of the phenomenon under study. Non-stationary measures will not be discussed here. 2.2.1. Global parameters Clustered monitoring networks often do not represent the true spatial pattern of phenomena and modeling processes based on raw data produce biased results. This non-representativity leads to a risk of over- and under-estimation of global parameters (e.g., mean, variance) and therefore to an erroneous reconstruction of the probability distribution governing the phenomenon. Figure 2.3 shows an example of a random variable (this is a simulated example).

Figure 2.3. Simulation of an environmental phenomenon and sampling schemes used (left: random; right: clustered)

22


If this phenomenon is sampled with the networks shown in Figure 2.2, the differences in observed mean and variance of the phenomenon are evident (Table 2.1); the clustering of samples in areas characterized by small concentrations decreases the value of the observed mean, implying an incorrect observation of the phenomenon. The histogram that will be used for modeling is therefore biased and does not represent the true phenomenon. Such errors can lead to under- or overestimation of environmental risk and must be avoided. Mean

Variance

Real

0.26

0.77

Random MN

0.26

0.79

Clustered MN

0.09

0.77

Table 2.1. Observed mean and variance of the artificial phenomenon sampled with both MN shown in Figure 2.2. The first line shows parameters estimated using all data

2.2.2. Spatial predictions The use of a clustered MN for spatial prediction can lead to incorrect spatial conclusions about the extent of a polluted area. Following the example used in the previous section, random (left in Figure 2.2) and clustered (right in Figure 2.2) networks were used to produce a pollution map using a kriging model (see Chapter 3). Figure 2.4 shows that the oversampling in small concentration areas leads to a regional under-estimation of risk and that small contaminated areas (hot spots) are not detected.

Figure 2.4. Spatial interpolation of both networks (left: random; right: clustered) using kriging


23

2.3. Monitoring network quantification In this section, several clustering measures will be discussed. Particular attention will be paid to the fractal clustering measures. In principle, quantitative clustering measures can be aggregated into topological, statistical and fractal measures [KAN 04a]. 2.3.1. Topological quantification The topological structure of space can be quantified by Euclidean geometry expressed by topological dimension: an object that can be disconnected by another of dimension n has a dimension n+1 (Figure 2.5). The usual representation of space is therefore bounded to integer dimensions. For example, a surface environmental process should be analyzed with a MN covering the entire two-dimensional space (topological dimension of 2).

Figure 2.5. Examples of topological dimensions

2.3.2. Global measures of clustering Several methods exist in order to highlight clustering [CRE 93, KAN 04a]. Below is a non-exhaustive list of well-known methods useful to quantify departures from the homogenous repartition of samples. Both the simulated and real data considered in this chapter deal with two-dimensional geographical space (longitudelongitude coordinates or corresponding projections). 2.3.2.1. Topological indices Topological indices evaluate the level of MN clustering by estimating the homogenity of the two-dimensional space covering provided by the MN. In that

24


sense, a quasi-quantitative index is the area of Voronoï polygons [THI 11, PRE 85, STO 95, OKA 00]. If the samples are homogenously distributed, the areas of the Voronoï polygons are constant for every polygon associated with every sample (except for the samples located close to the boundaries of the region). If there is some clustering, the surface distribution varies from small areas (clustered areas) to large (regions where only a few samples are available). Therefore, the area/frequency distribution of the polygons can be interpreted as an index of spatial clustering [NIC 00, KAN 04a, PRO 07]. An example of the analysis based on Voronoï polygons is given in Figure 2.6.

Clustered Random

30 00

25 00

Number of polygons

20 00

15 00

10 00

5.00

0.00 5. 0

5.50

6.00

6.50

7.00

7 50

8.00

-5.00

Area (sqkm)

Figure 2.6. Voronoï polygon area for the clustered (left, above) and homogenous (left, below) areas. Frequency/area for the networks (right)

2.3.2.2. Statistical indices Several statistical indices have been developed to highlight the presence of spatial clustering, the most common probably being Moran’s index [MOR 50], a weighted correlation coefficient used to analyze departures from spatial randomness. Other indices can be used to discover the presence of clusters: – the Morisita index [MOR 59]: the region is divided into Q identical cells and the number of samples ni within every cell i is counted. Then, the size of the cells is increased and the process is iterated, returning the size-dependent Morisita index I'. I'

¦ Q

Q i 1

n i ( n i 1)

N ( N 1)

[2.1]


25

where N is the total number of samples. A homogenous process will show a Morisita index fluctuating around the value of 1 for all scales considered, because of the homogenous distribution of the samples within the boxes at every scale. For the clustered MN, the number of empty cells for small scales increases the value of the index. The index has been used in a wide range of environmental applications, from ecological studies [SHA 04, BON 07] to risk analysis [OUC 86, TUI 07a]. Examples of Morisita diagrams for two simulated networks (homogenous and clustered) are given in Figure 2.7. Clustered samples

Morisita index

Random samples

Cell's size

Figure 2.7. Morisita index for random (dashed) and clustered (solid) MN

The K-function (or Ripley’s K) [RIP 77, MOE 04] can be used to calculate the degree of spatial randomness in the spatial distribution of samples. The K-function is

K'

O1 E >I(dij R)@

[2.2]

where O is the density (number per unit area) of samples and I is an indicator function giving 1 if the considered samples are within a circle of radius R and 0 otherwise.

Clustered samples Random Poisson

0.0 e+00

0.0 e+00

4.0 e+07

8.0 e+07

K

8.0 e+07

Homogenous samples Random Poisson

4.0 e+07

K

1.2 e+08


1.2 e+08

26

0

1000

2000

3000

4000

5000

6000

0

1000

2000

delta

3000

4000

5000

6000

delta

Figure 2.8. (Solid) K-functions for the homogenous (left) and clustered (right) datasets. Comparison with spatial randomness configuration (dashed)

The iteration for different delta values allows us to describe the properties at different scales (as with the Morisita index). The K-function can then be compared to the K-function associated with a function representing spatial randomness, usually a Poisson process. Here, Krnd ~ R2. Plotting both functions shows a departure from spatial randomness (dashed line, Figure 2.8), i.e., clustering. It should be noted that the K-function method is comparable to the sandbox counting measure for fractal sets considered below. 2.3.3. Dimensional resolution: fractal measures of clustering Fractal resolution [MAN 82, MAN 94] can be used as a measure of clustering of a monitoring network [LOV 86, KAN 04a, TUI 07a]. Ideal fractals are self-similar objects that reproduce their structure throughout the scales. The word “fractus” comes from the Latin “irregular”. Fractal objects are often characterized by non-integer dimensions: they have fragmented shapes and reproduce their structures using many scales. The purpose of a monitoring network is to detect, understand and model spatiotemporal phenomena (natural or artificial) via the observations at a finite number of locations in space. As well as spatial resolution, monitoring networks are characterized by dimensional resolution, i.e. their ability to detect D-dimensional phenomena in a D-dimensional Euclidean space. Dimensional resolutions are characterized by fractal dimensions and for clustered monitoring networks they are


27

less than D (in our case D=2). By their fractal nature, clustered monitoring networks have a dimensional resolution lower than 2, and thus they can detect only phenomena of (2-df) dimension [LOV 1986, TUI 2007b]. The determination of the dimensional resolution allows us to analyze the appearance of self-similar structures in the monitoring network, i.e. the repetition of the configuration of points throughout the scale and the non-homogenity of the network. Let us note that clustered MN are not mathematical fractal objects, therefore, they could be self-similar only over a limited number of scales. Several methods exist to evaluate the fractal dimension of networks [MAN 94, SMI 96]. In the following sections, two of these methods (the most widely used in applications), the sandbox and the box-counting algorithm, are presented. The remainder of the section focuses on another interesting measure of fractal objects, the lacunarity. 2.3.3.1. Sandbox method With the sandbox method (also called the radial method) [FED 88], the number of neighbors within a circle of radius R centered on the current point is averaged on the whole dataset (Figure 2.9, left). This average number of neighbors follows a power law:

P(R) v R df SAND

[2.3]

where dfSAND is the fractal dimension of the network measured with the sandbox method. Using a log-transform of [2.3] it is possible to plot log[P(R)] as a function of log[R] and to derive dfSAND as the slope of the linear regression fitting the data of the plot. The sandbox method is based on local neighborhood measures between samples and can be interpreted as a measure of the density of samples at different scales. Therefore, using the sandbox method allows us to detect the appearance of clustering as a departure from a homogenous situation, for which the fractal dimension is equal to 2 (the number of points for a homogenous set increases with R2).

28


Log(mean Number of points) 1.00 0.60

Slope = dfSAND

Log(S(L)) 1.00 0.60 Slope = 0.20 dfBOX

0.20 0.20 0.60 1.00

0.20 0.60 1.00

Log(Rad s)

Log(mean Number of points)

Log(L) L = 1, N = 256, S(L) = 13

Log(S(L))

1.00

1.00

0.60

0.60 0.20

0.20 0.20 0.60 1.00

0.20 0.60 1.00

Log(Radius)


Log(L) L = 4, N = 16, S(L) = 10

Log(S(L))

1.00

1.00

0.60

0.60 0.20

0.20 0.20 0.60 1.00

0.20 0.60 1.00

Log(Radius)


Log(L) L = 8, N = 4, S(L) = 4

Log(S(L))

1.00

1.00

0.60

0.60 0.20

0.20 0.20 0.60 1.00

Log(Radius)

0.20 0.60 1.00

Log(L) L = 16, N = 1, S(L) = 1

Figure 2.9. Calculation of the fractal dimension with the sandbox (left) and box-counting (right) methods

Figure 2.10 shows the results of the sandbox method to both networks considered in this section. The slope of the regression function fitting the curves gives the fractal dimension. The homogenous samples are associated with a fractal dimension of 1.832, while the clustered samples have a dimension of 1.197. The value of 2 is not reached by the homogenous network because the distribution of samples is not regular: the value of two can be reached by a regular grid of samples, while a real network is characterized by slight over-densities at small scales resulting in a small level of clustering and a decrease of the fractal dimension. The second effect deals with the finite number of points and ergodic fluctuations around a homogenous measure. Moreover, the points on the boundary of the region produce a boundary effect that results in reducing the mean number of neighbors and thus the true dimension (Figure 2.11). One way to avoid the boundary effect is to introduce correlation factors, as is usually done for K-function calculations (see e.g. [DIG 03]).


29

1.6

Clustered samples Homogenous samples

1.4

log (mean number of samples)

y = 1.1975x - 3.2784 1.2

1

0.8 y = 1.832x - 5 8202 0.6

0.4

0.2

0 3.2

33

3.4

35

3.6

3.7

3.8

3.9

log (radius)

Figure 2.10. Fractal dimension measured with the sandbox method for the homogenous (dashed) and clustered (solid) samples

Figure 2.11. Boundary effects for peripheral samples (right): the number of neighbors for these samples is less than expected (left)

4

30


2.3.3.2. Box-counting method The box-counting method (also called the grid-method) [SMI 89] covers the region under study with a regular grid of N boxes (as in the case of Morisita index calculation) and counts the number of boxes necessary to cover the whole network S(L). The size of the boxes, L, is then gradually decreased, and the number of boxes necessary to cover the samples is counted (Figure 2.9 – second method, right). The scales and the number of boxes follow a power law

S(L) v Ldf BOX

[2.4]

where dfBOX is the fractal dimension of the network measured with the box-counting method. Using a log-transform of [2.4] it is possible to plot log[S(L)] as a function of log[L] and to derive dfBOX as the slope of a linear regression fitting the data. The least squares technique is used to fit the regression Contrary to the previously discussed sandbox method, the box-counting method is useful for the calculation of the degree of spatial coverage of space by the network. 2

Clustered samples 1.8

Homogenous samples 1.6 y = -1.8266 x + 8.4447

log(number of boxes)

1.4

1.2 y = -1.5917 x + 7.396 1

0.8

0.6

0.4

0.2

0 3.4

3.6

3.8

4

4.2

4.4

4.6

4.8

log(size of boxes)

Figure 2.12. Fractal dimension measured with the box-counting method for the homogenous (dashed) and clustered (solid) samples


31

Figure 2.12 shows the results for the clustered and homogenous networks. Clustering of both networks is particularly distinguishable for small box sizes, where the effect of local clustering can be detected. As both networks are covering the entire space, the box-counting method cannot detect the difference between them at large scales (the curves are very similar for log (box sizes) greater than 4). To sum up, the fractal measures considered in this section show two different approaches to quantify the dimensionality of a monitoring network, i.e. if the network is appropriate to detect a D-dimensional phenomenon in a D dimensional Euclidean space: the first, the sandbox method, calculates a measure of the local densities of samples at different scales, while the second, the box-counting method, is based on an estimate of the spatial covering of the region under study by the network. Both indices are complementary, because they describe different clustering properties. Complementarities of the indices are shown in the two following “toy” examples. Figure 2.13 shows two similar monitoring networks associated with the same numbers of samples: the first presents a cluster of samples in the middle, while the second is homogenously distributed.

Figure 2.13. Artificial monitoring networks: left with a cluster; right – homogenous distribution of measurement points

Table 2.2 shows the values of fractal dimensions calculated with both methods on both MN considered; only the sandbox method is able to detect a spatial clustering, by the strong change in local densities, while the box-counting measure remains almost unchanged, because the 2-dimensional space is covered by both networks in the same way.

32

Advanced Mapping of Environmental Data Clustered

Homogenous

Sandbox fractal dimension

1.67

1.98

Box-counting fractal dimension

1.88

1.91

Table 2.2. Fractal dimensions calculated on the networks of Figure 2.13

Figure 2.14 shows two other monitoring networks associated with the same numbers of samples: the first present a regular distribution of samples in two regions of the 2-dimensional space, while in the second the samples are distributed regularly in the whole region.

Figure 2.14. Artificial monitoring networks: (left) regular samples distributed in two regions; (right) regular samples distributed in the whole region

2 regions

Regular

Sandbox fractal dimension

1.86

1.91

Box-counting fractal dimension

1.79

2

Table 2.13. Fractal dimensions calculated on the networks of Figure 2.14

In this case, the result is unlike the previous example: the box-counting method detects the change in spatial distribution of samples due to the appearance of empty regions throughout the scales. The sandbox method, which analyzes the local densities, does not detect this change, because the distribution remains regular for both networks. Changes in the sandbox results are related to the bigger extent of the boundary of the first network: in this case, the boundary effects discussed above are stronger and tend to decrease the value of the fractal dimension.


33

2.3.3.3. Lacunarity Although the fractal dimension can highlight the presence of clustering, it is still difficult to interpret: fractal dimension shows a departure from spatial homogenity, but it is not a structural index. Two MN can share the same dimension, and still be very different [TUI 07b]. Both networks in Figure 2.15 share the same fractal dimension (1.093), even if a single cluster characterizes the first, while the second shows two of them.

Figure 2.15. Monitoring networks characterized by the same fractal dimension; (left) one single cluster in the center; (right) two clusters

In order to include additional information in the analysis of the dimensionality of a network, the second moment of the distribution, the variance, can be included (the fractal measures discussed above are based on the first moment only). An example of an index showing this property is the lacunarity [MAN 82, MAN 94, ALL 91]. At a descriptive level, lacunarity can be interpreted as a lack of rotational or translational invariance in an object. This property of fractals measures the degree of non-uniformity (heterogenity) of structure of an object: i.e., it measures the structural variance within the object [SMI 96]. An object is said to be lacunar if its structure is highly variable. Several methods have been proposed to calculate the lacunarity of a set of points. No general agreement exists about the best one to use. In this section, we will present the gliding box method proposed by Allain and Cloitre [ALL 91, PLO 96]. A box of size l is placed at the origin of the set and the mass s (number of events) within the box is counted. Then, the box is moved one space along the set and the process is iterated. Since the gliding box has counted the masses for every possible position, a mass distribution n(s, l) is defined. The frequency distribution is then

34


converted to a probability distribution by dividing n(s, l) by the total number of boxes of size l. Having defined this probability distribution, it is possible to derive the first and second moments of the distribution:

E(1)

¦ sQ(s,l)

[2.5]

s 1

E(2)

¦ s Q(s,l) 2

[2.6]

s 1

The lacunarity /(l) can then be defined, as shown in equation [2.7]. The calculation is repeated for different sizes l, returning a size-dependent index:

/(l)

E(2) E 2 (1)

[2.7]

2.4. Validity domains In the previous section some methods to evaluate the presence of clustering in a MN have been discussed. Even if the methods are simple and computationally light, the interpretation of the results in terms of predictive power is often difficult; for example, a network related to a fractal dimension of 1.4 is clustered, but which type of phenomena can be detected and predicted by a network related to such a level of clustering? Moreover, the correct estimate of real clustering measures deals with a finite number of measurement points and complex geographical regions where phenomena are monitored and studied; surveys are limited by constraining factors that can be geographical (areas that cannot be reached with instruments), political (administrative limits between countries), geomorphologic or socio-economic (for example, phenomena for which sampling out of inhabited areas is senseless). These factors define a Validity Domain (VD, or space of interest) that constrains the prediction space, decreasing the dimensionality of the phenomenon to be detected. In fact, the fractal dimension of the phenomenon studied in the mapping space will no longer be 2. Figure 2.16 shows two sampling design schemes for a hypothetical survey of forests: the one on the left side is homogenously distributed in two-dimensional space, but is not realistic (samples occur outside the forest areas). The one on the


35

right side is clustered, but in a forest-related space, i.e. by taking into account the VD of interest, it can be considered much more homogenous.

Figure 2.16. Sampling in forest areas. Homogenous 2D sampling (left column); homogenous sampling within the VD (right column)

Although the existence of VD seems obvious, their integration into cluster analysis is not trivial. One way to include the VD in cluster analysis is to proceed with a comparative analysis: in order to define the indices related to a homogenous sampling within the VD, some artificial MN with known properties in terms of clustering are generated and are considered as reference measures. For each of them, homogenously distributed (uniformly distributed in a Cartesian (x,y) coordinate system) random points have been generated within the VD. Departures from the homogenous distribution (clustering) are then calculated as differences (in terms of indices) in comparison with the simulated networks. The reference measures calculated on a simulated network are subjected to a certain degree of fluctuation related to the finite number of points and the randomness of the simulation procedure. Two points are crucial: – in order to deal with the finite number of points, each generated network has the same number of points (N) as the real monitoring network: the number of points

36


being the same, differences in the results are only related to the differences in the level of clustering and ergodic fluctuations; – in order to take into account ergodic and other types of fluctuations, the networks have been generated using a type of bootstrapping technique: a large number of points M (with M >> N) have been produced within the validity domain and then several random samplings (bootstrapping) with N number of points have been extracted. Such a procedure can quantify uncertainty and sensitivity of the results due to the finite number of points and VD definition. The number of points M is usually defined by the following relation:

M|

Lx Ly

G2

[2.8]

where Lx and Ly are the limits of the region and G is the required spatial resolution of the phenomena. The analysis of clustering within the VD can lead to the conclusion that there is no clustering within the validity domain. By limiting the following analysis to the VD of interest, declustering procedures can be avoided and the network can be considered as spatially optimized. The analyst should not forget that, if this option is chosen, the mapping of environmental variables outside the VD will be biased. In summary, the calculation of clustering indices within the relevant VD can be used to interpret indices such as fractal dimensions or Morisita diagrams. The VD are the regions of interest related to the phenomenon under study and important for the predictions; out of these regions, the location of points is not relevant or even senseless (e.g. forest fires in waterplans or water quality measures in desert areas). Such a procedure prevents us from erroneous conclusions that would come from the comparison with standard values from homogenous distribution (2 for fractals or 1 for Morisita index). The real clustering is thus the difference between the values obtained on the true MN and the artificial generated on the VD, representing the situation of spatial homogenity within the VD of interest. In summary, “only relative measures matter”. 2.5. Indoor radon in Switzerland: an example of a real monitoring network Let us consider a real case study based on the Swiss indoor radon monitoring network (SIRMN). This dataset represents about 29,000 measurements in dwellings,


37

and is used for indoor radon modeling and spatial predictions [KAN 04b]. Data are highly clustered, variable and anisotropic at different scales. It is very difficult to find spatial structures on raw data using traditional variography. The application of regularized and non-regular variography on transformed data can reveal spatial structures but still with a high nugget effect [KAN 04b]. 2.5.1. Validity domains In this study, 10 MN extracted by bootstrap for each of the VD presented below have been used for comparison with the real monitoring network (Figure 2.17): – random samples within the rectangular region covering the data under study. This is a theoretical homogenous network and it does not take into account any boundaries; – random samples within the political boundaries of the region under study. This network configuration allows us to take into account complex boundary effects; – random samples within the populated regions of the area under study: this kind of VD can be justified by the phenomena where priorities in prediction are given to the populated regions.

Figure 2.17. Artificial MN used for the study of the Swiss indoor radon survey; (left) random points in 2D bounding box; (center) random points within political boundaries; (right) random points within populated regions

2.5.2. Topological index The Voronoï polygon area analysis (see Figure 2.18) shows a different distribution of areas-of-influence depending on the level of clustering. On one hand, the behavior of the curves is quite different for the real data and for the random points on the squared grid. On the other hand, the more constraining the VD is, the more the curves begin to be similar. Analysis of the Kullbach-Leibler divergence for these curves (Table 2.3) confirms these observations.

38

Advanced Mapping of Environmental Data Min K

Mean K

Max K

Homogenous square VD

3.401

3.625

3.722

Political boundaries VD

2.511

2.634

2.809

Populated areas VD

0.340

0.366

0.384

Table 2.3. Kullbach-Leibler divergence K between the real SIRMN and the artificial ones reflecting known clustering properties

0

Figure 2.18. Voronoï polygon area analysis for the SIRMN

2.5.3. Statistical indices 2.5.3.1. Morisita index The Morisita index (Figure 2.19) confirms the previous results: for real data, the index is higher than 1, which is the value of the index for a homogenous network, at all the scales. This shows a clustered distribution of points. The value of 1 is only reached asymptotically at larger scales, where one box covers the whole space, giving a value of I' of 1. Comparison with the simulated networks shows that the real data are distributed similarly to the random points over the populated regions VD, but also that the level


39

of clustering is significantly higher for every scale. The simulated networks can be interpreted as homogenous for large scales (for scales larger than 200 km I' ) and the random distribution on populated regions can be distinguished from the distribution within the political boundaries only at scales smaller than 100 km.

Figure 2.19. Morisita index for the SIRMN

2.5.3.2. K-function The application of the K-functions confirms the previous observations. It is interesting to note that the network generated in the squared grid corresponds to a random Poisson point process.

40


Figure 2.20. K-functions for the SIRMN

2.5.4. Fractal dimension 2.5.4.1. Sandbox and box-counting fractal dimension The analysis of the fractal dimension on the SIRMN showed a general loss of dimensionality related to the degree of clustering (Table 2.4 and Figure 2.21). dfSANDa

dfBOX Min Homogenous square VD

2

Mean 2

Max

Min

2

1.954

Mean 1.956

Max 1.958

Political boundaries VD

1.828

1.83

1.832

1.925

1.928

1.930

Populated areas VD

1.751

1.753

1.755

1.7

1.703

1.706

Na

1.71

Na

Na

1.52

Na

SIRMN

Table 2.4. Fractal dimension of the considered networks (ten for every type). a Radius considered from 500 m to 50 km; Na: the real dataset consists of one unique realization, min and max statistics are not informative


Figure 2.21. Fractal dimension index for the SIRMN; above: sandbox method; below: box-counting method

41

42


The squared random configuration of points is associated with a fractal dimension close to 2 for the sandbox method and of 2 for the box-counting method, i.e. a regularly distributed monitoring network for both methods. The slight difference of the sandbox method (dfSAND = 1.96) is related to the random generation of points, which allows the presence of small clusters in the local distribution. The use of increasingly constraining validity domains is reflected in the dimensional index by a consistent decrease of the fractal dimension: for the boxcounting method, the generation of random samples on the Swiss surface is related to a dfBOX of 1.83, while the use of the population validity domain decreases the fractal dimension to 1.75. The fractal dimension of real MN is equal to 1.71, and is very close to a randomly distributed network on populated areas. Differences appear only at very small cell sizes, when the effects of very local clustering appear. For larger scales (log(L) greater than 5 in the figure), the four networks are equivalent because they homogenously cover this region approximately in the same way. Regarding the sandbox method, even if the tendency of the decrease of fractality is the same, the difference between the real samples and the network generated on the populated regions VD is stronger and about 0.2. This difference can be explained by the computational method of dfSAND: the sandbox method is based on local neighborhood measures between samples and is more sensitive to high local clustering than the box-counting method, which is more a method for calculating the degree of coverage of space by the network. Raw SIRMN and populated area networks covering the space in a similar way, the difference of clustering between them is less visible with the box-counting method. Real SIRMN case study has shown dimensions between 1.52 and 1.71 depending on the method used. The dimensionality of a regular two-dimensional network fluctuates around 2, while the values of the populated regions VD are closer: between 1.70 and 1.75. Since the prediction of a phenomenon as indoor radon is strongly linked to inhabited areas, we can consider the following: if, on the one hand, the real network appears to be heavily clustered for predictions in twodimensional space, on the other hand it appears to be homogenous for predictions in populated regions. Therefore, the modeling should be constrained to populated regions, thus avoiding the need for declustering methods. 2.5.4.2. Lacunarity The lacunarity index calculated for the four networks is shown in Figure 2.22. The global tendency to clustering is confirmed by the curves and differences in the point patterns can be guessed. Specifically, it can be observed between MN of real data and MN of data on populated region networks and the two others.


43

This was expected, since the real samples have been taken in populated regions (indoor radon is measured in dwellings). The non-sampled area distribution is therefore very similar for both networks and for the curves calculated.

Figure 2.22. Lacunarity index calculated for the four networks

2.6. Conclusion The first questions regarding environmental spatial data deal with the spatial and dimensional resolution of a given monitoring network, i.e. which phenomena can be detected by the monitoring network and at what resolutions. Clustering (nonhomogenity) and preferential sampling already give rise to biased estimates of global statistics such as mean and variance values. Therefore, correct quantification of monitoring network quality and following selection of an appropriate declustering technique are extremely important both for exploratory data analysis and spatial predictions.

44


In this chapter topological, statistical and fractal measures were introduced to quantify clustering of simulated and real monitoring networks. The important concept of a validity domain – important regions for the analysis and predictions – was introduced. It was demonstrated that in the case of complex regions under study relative values between indices are more important than absolute values. In fact, such studies have close relationships with traditional – representativity of raw data, splitting of data into training/validation/testing data subsets, and recent trends in machine learning – transductive and semi-supervised learning. 2.7. References [ALL 91] ALLAIN C. and CLOITRE M., “Characterizing the lacunarity of random and deterministic fractal sets”, Physical Review A, 44, 1991, p. 3552-3558. [ALL 00] ALLARD D. and GUILLOT G., “Clustering geostatistical data”, Proceedings of the 6th International Geostatistics Congress, Cape Town, South Africa, 2000, 15 p. [BON 07] BONJORNE de ALMEIDA L., GALETTI M., “Seed dispersal and spatial distribution of Attalea geranensis (Arecaceae) in two remnants of Cerrado”, Acta Oecologica, in press. [CRE 93] CRESSIE N., Statistics for Spatial Data, John Wiley and Sons, NY, 1993. [DEU 97] DEUTSCH C.V. and JOURNEL A.G., GSLIB, Geostatistical Software Library and User Guide, Oxford University Press, NY, 1997. [DIH 03] DIGGLE P.J., Statistical Analysis of Spatial Point Processes, second edition, Oxford University Press, London, 2003. [FED 88] FEDER J., Fractals, Plenum Press, NY, 1988. [GRU 06] DE GRUIJTER J., BRUS D., BIERKENS M.F.P. and KNOTTERS M., Sampling for Natural Resource Monitoring, Springer, NY, 2006. [KAN 04a] KANEVSKI M., MAIGNAN M., Analysis and Modelling of Spatial Environmental Data, EPFL Press, Lausanne, 2004. [KAN 04b] KANEVSKI M., MAIGNAN M., PILLER G., “Advanced analysis and modelling tools for spatial environmental data. Case study: indoor radon data in Switzerland”, Proceedings of the XVIII International Conference Enviroinfo 2004, Geneva. [KOV 04] KOVITZ J.L., CHRISTAKOS G., “Spatial statistics for clustered data”, Stochastic Environment Research and Risk Assessment, 18(3), 2004, p. 147-166. [LEN 06] LE N.D., ZIDEK J.V., Statistical Analysis of Environmental Space-Time Processes, Springer, NY, 2006. [LOV 86] LOVEJOY S., SCHERTZER D. and LADOY P., “Fractal characterization of inhomogeneous geophysical measuring networks”, Nature, 319, 1986, p. 43-44 [LOV 87] LOVEJOY S. SCHERTZER D. and TSONIS A., “Functional Box-Counting and Multiple Elliptical Dimensions in Rain” Science, 1987, p. 1036-1038.


45

[MAN 94] MANDELBROT B.B., “Fractals, lacunarity, and how it can be tuned and measured”, NONNENMACHER, T.F., LOSA, G.A., WEIBEL, E.R. (eds.), Fractals in Biology and Medecine, Birkhäuser Verlag, Boston, 1994, p. 21-28. [MOE 04] MOLLER J., WAAGEPETERSEN R.P., Statistical Inference and Simulation for Spatial Point Processes, Chapman & Hall, Boca Raton, 2004. [MOR 50] MORAN P.A.P, “Notes on continuous stochastic phenomena“, Biometrika, 37, 1950, p. 17-23. [MOR 59] MORISITA M., “Measuring of the dispersion of individuals and analysis of the distribution patterns”, Mem. Fac. Sci. Kyushu Univ., Ser E., 2, 1959, p. 214-235. [MUL 07] MULLER W.G., Collecting Spatial Data. Optimum Design of Experiments for Random Fields, Third edition, Springer, NY, 2007. [NIC 00] NICHOLSON T., SAMBRIDGE M., GUDMUNDSSON O., “On entropy and clustering in earthquake hypocenter distributions”, International Journal of Geophysics, 142, 2000, p. 37-51. [OKA 00] OKABE A., BOOTS B., SUGIHARA K., Spatial Tessellations: Concepts and Applications of Voronoï Diagrams, John Wiley & Sons, 1992. [OLE 07] OLEA R., “Declustering of Clustered Preferential Sampling for Histogram and Semivariogram Inference”, Mathematical Geology, vol. 39, 2007, p. 453-467. [OUC 86] OUCHI, T., UEKAWA, T., “Statistical analysis of the spatial distribution of earthquakes – variation of the spatial distribution of earthquakes before and after large earthquakes”, Physics of the Earth and Planetary Interiors, 44(3), 1986, p. 211-225. [PLO 96] PLOTNICK R.E., GARDNER R.H., HARGROVE W.W., PRESTEGAARD K. and PERLMUTTER M., “Lacunarity analysis: a general technique for the analysis of spatial patterns”, Physical Review E, 53, 1996, p. 5461-5468. [POZ 06] POZDNOUKHOV A. and KANEVSKI M., “Monitoring network optimization for spatial data classification using support vector machines”, Int. Journal of Environment and Pollution, vol. 28, 2006, p. 465-484. [PRE 85] PREPARATA F.P. and SHAMOS F.I., Computational Geometry, Springer, NY, 1985. [PRO 07] PRODANOV D., NAGELKERKE N., MARANI E., “Spatial clustering analysis in neuroanatomy: applications of different approaches to motor nerve fiber distribution”, Journal of Neuroscience Methods, 160, 2007, p. 93-108. [RIC 02] RICHMOND A., “Two-points declustering for weighting data pairs in experimental variogram calculations”, Computers and Geosciences, 2(2), 2002, p. 231-241. [RIP 77] RIPLEY B.D., “Modelling spatial patterns”, Journal of the Royal Statistical Society, B39, 1977, 172-212. [SHA 04] SHAHID SHAUKAT S., ALI SIDDIQUI I., “Spatial pattern analysis of seed bank and its relationship with above-ground vegetation in an arid region”, Journal of Arid Environments, 57, 2004, p. 311-327.

46


[SMI 89] SMITH T.G., MARKS W.B., LANGE G.D., SHERIFF W.H. and NEALE E.A., “A fractal analysis of cell images”, Journal of Neuroscience Methods, 27, 1989, p. 173180. [SMI 96] SMITH T.G., LANGE G. and MARKS W.B., “Fractal methods and results in cellular morphology – dimensions, lacunarity and multifractals”, Journal of Neuroscience Methods, 69, 1996, p. 123-136. [STO 95] SOTYAN D., KENDALL W.S., MECKE J., Stochastic Geometry and its Applications, 2nd Ed., J. Wiley and Sons, Chichester, 1995. [TES 94] TESSIER Y., LOVEJOY S., and SCHERTZER D., “Analysis and Simulation of the Global Meteorological Network”, Journal of Applied Meteorology, vol. 33, 1994, p. 1572-1586. [THI 11] THIESSEN A.H., “Precipitation average for large areas”, Monthly Weather Review, 39, 1911, p. 1082-1084. [TUI 07a] TUIA D., LASAPONARA R., TELESCA L., and KANEVSKI M., “Identifying spatial clustering phenomena in forest-fire sequences”, Physica A, 376, 2007, p. 596-600. [TUI 07b] TUIA D., KAISER C. and KANEVSKI M., “Clustering in environmental monitoring networks: dimensional resolutions and pattern detection”, in GEOENV VI: Proceedings of the Sixth European Conference on Geostatistics and Environmental Applications, Sprinter, 2007.

Chapter 3

Geostatistics: Spatial Predictions and Simulations

3.1. Assumptions of geostatistics Geostatistics dates back to the first introduction of kriging in 1954 [MAT 1954]. The principles of geostatistics were developed by Matheron [MAT 1963] and extended in later works [JOU 1978; CRE 1993; CHI 1999]. An independent contribution to spatial data modeling and interpolations (objective analysis of meteorological fields) was made by L. Gandin [GAN 63]. Geostatistics considers a spatial phenomenon as a random process Z(x), where the argument x denotes location in space (x=(x,y) in a two-dimensional space). Available measurement data (Z(x1), …,Z(xN)) from N locations (x1,…,xN) are treated as realizations of the random process Z(x). Measurements in spatial statistics are unique for every sampling location and represent a single existing realization unlike in classical statistics, where measurements are considered as multiple samples of a random variable. For example, we can measure porosity of a core sample from a well core plug, but cannot make a repeated measurement from the exactly identical location. The repeated measurement in this case would come from a very close but not identical location. This limitation is overcome by an assumption of Z(x) spatial continuity – a similar behavior in the vicinity of a measurement. Mathematically, the measure of continuity is described by the spatial correlation structure, which reflects how similar the values are in respect to their mutual location in space. In classical geostatistics, spatial correlation is described by a covariance function or a variogram. These characteristics are related to stationarity assumptions. Spatial Chapter written by E. SAVELIEVA, V. DEMYANOV and M. MAIGNAN.

48


stationarity in a strict sense states that a distribution of Z(x) is invariant to its location; i.e. the distribution function is the same in any two parts of the considered region. Thus, if we take 10 samples from one region they would feature exactly the same distribution as 10 samples from another region. This is a very heavy assumption, which is impractical in the majority of real problems. Therefore, several weaker assumptions about stationarity are used in practice: – second-order stationarity – the mean m of Z(x) is constant over the whole study area and the covariance function C(x+h, x) for any two locations x+h and x separated by a vector h depends only on h: x E[Z(x)]=m=const & x E[(Z(x+h)-m)(Z(x)-m)]=C(x+h, x)=C(h)

[3.1]

– intrinsic hypothesis – the mathematical expectation of the difference between function Z(x) values at any two locations x+h and x separated by a vector h is zero and the variance for the differences between these values (called a variogram) depends only on vector h: x E[Z(x+h)-Z(x)]=0 & x Var[Z(x+h)-Z(x)]=2J(h)

[3.2]

Intrinsic hypothesis is a weaker assumption than the second-order stationarity as it does not imply knowledge of the mean m. Analysis and modeling of the spatial correlation structure is the key part of any geostatistical analysis as it is directly inserted into kriging estimation procedure [JOU 78; DAV 88]. It means that geostatistical analysis usually starts with calculation of raw spatial correlation functions based on the measured data. Assuming intrinsic hypothesis, the standard formula for statistically unbiased estimation of the variance is given by [JOU 78, CLA 84; CHI 99]: G J(h)

¦ ¦ Z ( x i

j

) Z ( x j ) , G 2N( h ) 2

i

[3.3]

where h is the separation vector and N(h) is the number of pairs of samples separated by h. The summation is performed for all pairs (Z(xi),Z(xj)), such as ||xi, xj||[h-'h, h+'h], where 'h is the tolerance inserted in order to find a statistically sufficient number of pairs separated exactly by the separation vector h in arbitrary distributed raw data. Tolerance 'h is also a vector composed of lag tolerance h and direction tolerance. To prevent the same pair being used twice the sum is divided by 2 (semi-variogram). Under assumption of the second-order stationarity the statistical estimate of the covariance is used instead of the variogram.


49

Under assumption of the second-order stationarity both the covariance and the variogram exist and are related: J(h)=C(0)-C(h), where C(0) is a prior variance of the random process. There are some problems with estimation of a raw variogram associated with poor selection of tolerance, clustered samples, deterministic trends, outliers, etc. [DAV 88, PAR 91]. To improve the situation and to simplify the spatial correlation structure analysis, preliminary statistical analysis [KAN 04], a declustering procedure [JOU 83, DEU 89, CRO 83, SCH 93, BOU 97] and treating outliers [PAR 91] were proposed. The behavior of a variogram can depend on the orientation of the vector h separating the pairs. Such a situation is known as anisotropy [JOU 78]. Details of different types of variogram anisotropy are described in [ZIM 93]. Now, let us briefly consider key geostatistic models used for spatial predictions and spatial simulations. 3.2. Family of kriging models There are several kriging models (kriging family) with the same basic principles differing by some assumptions (or knowledge) on the data (process). Any kriging belongs to the BLUE (best linear unbiased estimator) class. Consequently, the basic principles of a kriging model are: – it is a linear estimator – a kriging estimate Z*(x0) at location x0 is obtained as a linear combination of known values Z(xi):

Z * ( x0 )

N ( x0 )

¦w ( x i

0

)Z ( x i )

[3.4]

i 1

where N(x0) is the number of samples from the neighborhood of x0 taken into account for the estimation; – it is an unbiased estimator – the mean value is reproduced by the kriging estimate (E{Z*(x0)}=E{Z(x0)}); – it is the best estimator among all estimators of the linear class, which minimizes the estimation error variance (Var{Z*(x0)-Z(x0)}omin). The value of the variance can be estimated together with the corresponding kriging estimate. This is called the kriging variance and is referred to as the kriging error.

50


Conditions of unbiasedness and estimation variance minimization are used to find the weights wi for the linear estimator. They lead to a system of linear equations, the form of which depends on the type of kriging, some of which are described below. 3.2.1. Simple kriging Simple kriging works under the assumption of second-order stationarity and with known mean values (E[Z(x)]=m) [JOU 78; CRE 93; CHI 99]. Knowledge of the mean automatically provides the unbiasedness of the estimator:

Z * ( x)

m ¦i

n( x) 1

wi ( x)( Z ( xi ) m)

[3.4a]

The estimation error minimization leads to the system of simple kriging equations: N ( x0 )

¦w ( x j

0

)C ( x i x j ) C ( x i x 0 ) i 1,! , N ( x 0 ) ,

[3.5]

j 1

where wj(x0) are the weight coefficients for the linear combination and C(xi-xj) is the covariance for the vector separating the locations xi and xj. The covariance matrix is easy to estimate in the case with the known mean. This system of equations has a single solution if the covariance matrix is non-singular, the covariance model is positively defined and there are no collocated samples in the data. The variance of a simple kriging estimate is given by the so-called simple kriging variance VSK:

V SK ( x 0 )

C (0)

N ( x0 )

¦ w C (x i

i

x0 ) .

[3.6]

i 1

3.2.2. Ordinary kriging Ordinary kriging differs from simple kriging by the unknown mean. The mean is assumed to be constant over the field, but it is unknown. Such a situation is more realistic, as the mean value is usually unknown for the real field and is not necessarily adequately represented by the sample mean.


51

An ordinary kriging estimator [JOU 78; CRE 93; CHI 99] works under the assumption of intrinsic hypothesis. The lack of knowledge leads to additional assumptions. To fulfil the unbiasedness, an additional constraint is imposed over the weights: N ( x0 )

¦w ( x i

) 1.

0

i 1

[3.7]

Minimization of the estimation error provides the system of equations called an ordinary kriging system (N(x0)+1 linear equations with N(x0)+1 unknowns). In a more general way it can be written in terms of the variogram: N ( x0 )

¦ w (x j

0

)J ij P

J i 0 i 1,!, N ( x0 )

j 1

[3.8]

N ( x0 )

¦ w (x j

0

) 1

j 1

where P is a Lagrangian multiplier, introduced because of the variance minimization with a constraint and J is a semi-variogram. Note that Jij can be calculated for each data pair Z(xi) and Z(xj) (i,j=1,…,Nx0 – the number of data in the neighborhood of (x0)), whereas Ji0 is approximated by the fitted theoretical variogram model J(h), with the separating vector h=xi-x0 as an argument. An ordinary kriging variance appears as follows:

V ok ( x0 ) P

N ( x0 )

¦ w ( x )J i

0

i0

.

[3.9]

i 1

3.2.3. Basic features of kriging estimation All models of the kriging family have a set of common features according to the basic principles of kriging. First, kriging weights do not depend on the variable values, as they are defined by a spatial correlation structure described by the covariance or the variogram model. Second, kriging is an exact estimator – the estimate honors the conditioning data exactly. Third, kriging features a smoothing effect on the estimates – the kriging estimate cannot exceed the data maximum or go below the data minimum. Smoothing is characterized by the variability of the kriging estimates. Fourth, the kriging variance is not higher than the variance of the initial data.

52


Figures 3.1 and 3.2 illustrate kriging estimates calculated in the 1D case. The estimates exactly honor the conditioning data and smoothly interpolate in between. Simple kriging (SK) and ordinary kriging (OK) estimates are quite similar except in the extrapolation regions at the edges. Kriging with the variogram model (correlation range 50) with a nugget effect (40% of the a priori variance) provides quite different estimates: they also exactly honor the observation data but feature much higher variance around the data locations, which results in a spiky pattern as if the data are less representative of the general function pattern. The kriging variance plotted in Figure 3.2 shows just slightly higher values for OK. The kriging model with high nugget clearly demonstrates high estimation variance between the data. The variance at the data locations is zero as the data values are reproduced exactly.

Figure 3.1. Kriging estimates in 1D case (spherical variogram, correlation range – 50, nugget – 2, sill – 3)

Figure 3.2. Kriging estimation error in 1D case


53

Smoothness of the kriging estimate depends on the variogram model used and in particular its correlation range. Ordinary kriging estimates with long (10) and short (1) variogram model ranges are presented in Figure 3.3. Both estimates honor the data points exactly and demonstrate different smoothness in between the data.

Figure 3.3. Kriging estimates with long and short variogram model ranges in 1D case

A 2D example of ordinary kriging estimates is presented in Figure 3.4 calculated with different values of variogram parameters. Higher nugget results in a smoother estimated pattern because the estimates are allowed to deviate more from the conditioning data towards the mean (compare Figures 3.4a and b). Different orientation of the anisotropy and the ratio between the ranges in the perpendicular directions may result in huge differences between the estimate patterns (see Figure 3.4c–f). As the estimations are based on only 6 points, the uncertainty about the variogram parameters is very large (as it is impossible to build a consistent variogram model based on just 6 items of data). The basic variogram model was chosen to be of the spherical type whereas the nugget, the correlation range and its orientation may vary.

54


Figure 3.4. Ordinary kriging estimates in 2D with different variogram model parameter values: a) nugget=0, angle=0, range along=5, range across=5; b) nugget=0.5, angle=0, range along=5, range across=5; c) nugget=0, angle=90, range along=40, range across=10; d) nugget=0, angle=90, range along=40, range across=20; e) nugget=0, angle=90, range along=40, range across=30; f) nugget=0, angle=30, range along=40, range across=20


55

One of the benefits of all kriging models is an estimate of the corresponding kriging variance, which describes the distribution of the estimation error. Kriging variance under some assumptions can be considered as a measure of the estimate’s uncertainty. It is important to remember that kriging variance is not conditional to the data. It is defined by the initial sample locations (sample density) and the spatial correlation structure (this feature has already been illustrated in Figures 3.4). Kriging variance is higher in zones with lower density of the initial samples. This property is illustrated in Figure 3.5, where kriging estimates are plotted along with the kriging estimation variance. The data used for kriging come from Spatial Interpolation Comparison contest SIC 2004 (see [SAV 05]). Comparison of the two plots shows that the kriging variance depends on the sample density (plotted with + marks) and does not depend on the actual value of the samples.

a)

b) Figure 3.5. Kriging estimate (a) and kriging variance (b); crosses indicate initial data locations [SAV 05]

56


3.2.4. Universal kriging (kriging with trend) Assumption of a constant mean considered for simple or ordinary kriging is sometimes difficult to accept. However, in some cases it is possible to account for local variations by assuming a smooth trend function m(x) as a mean estimate. Universal kriging models local mean (trend of the function) as a linear combination of basic functions fk(x) with k=1,…,K:

m( x c )

K

¦ a ( xc ) f k

k

( xc ) ak ( xc ) ak

x c W ( x ) .

[3.10]

k 0

The requirement of unbiasedness in this case leads to a set of constraints: N ( x0 )

¦w ( x i

0

) f k ( xi )

f k ( x0 ) .

[3.11]

i 1

The system of universal kriging equations appears as follows: K N ( x0 ) ° ¦ w j ( x0 )C R ( x i x j ) ¦ P k ( x0 ) f k ( x i ) C R ( x i x0 ) i 1,! , N ( x0 ) k 0 ° j1 N ( x0 ) ° w j ( x0 ) 1 ® ¦ j 1 ° N ( x0 ) ° w j ( x0 ) f k ( x j ) f k ( x0 ) k 1,! , K ¦ ° j 1 ¯

[3.12] where CR(h) is covariance function of residuals (R(x)=Z(x)-m(x)). A universal kriging variance can also be introduced:

V 2 UK ( x 0 )

N ( x0 )

¦ i 1

K

w i ( x )C R ( x i x 0 ) ¦ P k ( x 0 ) f k ( x 0 ) . [3.13] k 0

3.2.5. Lognormal kriging Lognormal kriging is the early attempt to perform nonlinear kriging [REN 79, DOW 82, HAA 90]. Usually, it is applied to data with lognormal statistical distribution.


57

Lognormal kriging is an ordinary kriging performed on a transformed variable (Y(x)=ln(Z(x))). The semi-variogram is calculated for the transformed models and modeled. Then, ordinary kriging system [3.8] is solved for the transformed values. The estimates of the transformed variable are obtained. The main problem with lognormal kriging concerns back transform to the original values so as to preserve the unbiasedness for the final estimate. In the case of lognormal distribution, the back transform using kriging variance (Vok2(x0)) and Lagrangian multiplier (P) is obtained in logarithmic scale:

1 2 ½ ( x0 ) P ¾ . Z ( x 0 ) exp®Y ( x 0 ) V OK 2 ¿ ¯

[3.14]

Figure 3.6. Experimental variogram for original data (a) and lognormal data (b)

Variogram plots in Figure 3.6 show how sensitive the variogram can be to the lognormal transformation of the data. It is difficult to detect any correlation in the original data due to the influence of very large values (see Figure 3.3a). However, after lognormal transformation, which mitigates the influence of the high value tail, a stationary correlation structure becomes clear (see Figure 3.6b). Thus, lognormal transformation allows application of kriging to the lognormal values, which is called a lognormal kriging.

58


3.3. Family of co-kriging models Sometimes in spatial prediction problems there is more than one variable to consider. Observations of the spatial variables can often be correlated due to their origin or natural phenomena (e.g. rainfall and cloud cover, temperature and elevations, porosity and permeability, contamination from radioactive nuclides). Joint consideration of correlated data improves accuracy of the spatial predictions and allows us to use a large number of cheaper measurements to estimate a variable characterized by only a few items of data, which may be difficult to obtain. Geostatistics offers a selection of methods for spatial interpolation, which accounts for secondary correlated data. In simple kriging models, secondary data can be used in the form of an external drift, which forms a trend surface to be used in estimates. A family of co-kriging models allows us to account for the secondary variable including its own spatial correlation structure more accurately. 3.3.1. Kriging with linear regression A simple linear regression relation between the two variables can be used to derive the value of the correlated variable from the one already estimated, e.g. using ordinary kriging. This method is very simple, computationally inexpensive and does not require any additional variogram modeling. It only requires the estimated value of the first variable in every location where the second variable is estimated (presumably a grid) and the linear regression coefficient. The latter has to be defined based on the prior knowledge of the relationship between the two variables and the correlation between their data distributions. However, linear regression suffers from a few drawbacks due to its simplicity. Using linear regression assumes that both variables have the same spatial correlation structure, which may not be the case. Furthermore, estimated values of the second variable mimic the spatial distribution of the first variable and may not exactly reproduce the conditioning data. This means that both variables will have exactly the same spatial variability and distribution but on the different scales according to the variable ranges. This can lead, for example, to an under-estimation of the peak value and an over-estimation of the low values of the second variable, if its distribution is more variable than the distribution of the first variable. 3.3.2. Kriging with external drift Kriging with external drift models the trend with the help of another function y(x) defined in the same field (secondary variable):

m( x )

a 0 ( x ) a1 ( x ) y( x ) .

[3.15]


59

Kriging with external drift can be considered as a modification of kriging with trend (section 3.2.3); the difference is in trend modeling. Kriging with trend [3.10] becomes kriging with external drift if K=1 and f1(x)=y(x). N ( x0 ) ° ¦ w i ( x 0 )C R ( x j x i ) P 0 ( x 0 ) P1 ( x 0 ) y( x j ) C R ( x j x ) ° i1 N ( x0 ) ° wi ( x0 ) 1 ® ¦ i 1 ° N ( x0 ) ° w i ( x 0 ) y( x i ) y( x 0 ) ¦ ° i 1 ¯

j 1,! , N ( x0 )

[3.16] where CR(h) is the covariance function of residuals (R(x)=Z(x)-m(x)). It is important to remember that to apply kriging with external drift, the secondary variable needs to be known at every location to be estimated. 3.3.3. Co-kriging Co-kriging is a generalization of kriging to the multivariate case. An estimate is carried out as a linear combination of the variable under estimation (primary variable ZD1) and other variables (K secondary variables ZD): *

Z D1 ( x 0 )

K

nD

wD Z D ( x ) . ¦¦ D i

i

[3.17]

1 i 1

To perform co-kriging we need to calculate cross-variograms (cross-covariances) for pairs of variables to describe the spatial cross-correlation structure of every pair of variables, in addition to auto-variograms (covariances) for every variable:

J DE ( h ) E [( ZD ( x h ) ZD ( x ))( Z E ( x h ) Z E ( x ))] ,

C DE ( h )

1 N(h) ¦ Z D ( x )Z E ( x h ) m Z D m Z E N( h ) i 1

60


Co-kriging can be simple or ordinary. Simple co-kriging covers the requirement on unbiasedness automatically. In the case of ordinary co-kriging, unbiasedness is achieved by fulfilment of the constraint nD

¦ wD i

i 1

GDD

0

1, D D 0 . ® ¯0, otherwise

[3.18]

An ordinary co-kriging equation system has the form:

K ni j ° ¦ ¦ w E J ij ( hD E ) P i °j 1E 1 ® ni ° w Ei G ii 0 ¦ °¯ E 1

J ii ( hD 0 ) 0

i

1,..., K ; D i

1,..., n i

1,..., K [3.19]

Co-kriging also allows us to estimate the variance of an error (co-kriging variance): 2 V CK

¦¦ wD J i

ii0

(hD 0 ) P i0 J i0i0 (0) .

[3.20]

3.3.4. Collocated co-kriging Collocated co-kriging is a modification of the full co-kriging for the case with a linear correlation between the variables. It simplifies the full co-kriging equations by deriving the cross correlation term C12 and the auto-variogram model of the secondary variable C22(xĮ-x) leaving just the a priori variance of the secondary variable C22(0):

U

C12 (0) , C12 (h) C11 (0)

UC11 (h)

[3.21]

where ȡ is a linear regression correlation coefficient and h=xĮi-x is the distance vector from the simulation location x to the observation data xai. Collocated co-kriging requires estimated values of the first variable used as the secondary data at each estimated location. For collocated co-kriging we can consider


61

both simple and ordinary cases, analogous to kriging and full co-kriging models. The collocated simple co-kriging estimate in the case of two variables Z1 and Z2 at a location x is given by:

Z*SCK( x )

n1( x )

¦w i1 1

SCK i1

( x ) Z1( xi1 ) m1 wsSCK( x )Z2 ( x ) m2 m1 [3.22]

where m1 and m2 are the known means of corresponding variables. Weights wi1 and w2 are determined from the following system of equations for k1 observations of the primary variable Y1: n1 ( x) SCK SCK ° ¦ wi1 ( x) C11 ( xD1 xi1 ) w2 ( x) U C11 ( xD1 x) C11 ( xD1 x) , D 1 1,..., k1 ° i1 1 ® n1 ( u ) ° wiSCK ( x) U C11 ( xD 2 xi1 ) w2SCK ( x) C 22 (0) U C11 (0) ¦ 1 °¯ i1 1

[3.23] where C11 and C22 are auto-correlations of the correlated variables Y1 and Y2. Collocated co-kriging is faster than full co-kriging, avoids the instability caused by highly redundant secondary data and does not require modeling of the crosscovariance function C12(h) and secondary covariance C22(h) at lags |h|>0. The tradeoff for this is in providing secondary variable value at every estimated location and ignoring the secondary data at locations other than those being estimated. It is also better adapted for avoiding screen effect because of the limited use of secondary data. 3.3.5. Co-kriging application example This case study deals with the prediction of air temperature in Kazakh Priaralie. The selected region covers 1,400,000 km2 with 400 monitoring stations. The primary variable is average long-term air temperatures (qC) in June. Additional information is the elevation of the locations above sea level. This information is available on a dense grid from a Digital Elevation Model.

62


The correlation between the air temperature and the altitude is linear and equals 0.9 (see Figure 3.7). The correlation linearity allowed us to use any of the geostatistical models (co-kriging, collocated co-kriging and kriging with external drift) for modeling. Comparison of the methods is performed on a specially selected validation data set – not used during the estimation. The similarity between the training and the validation data sets was controlled by comparing summary statistics, histograms and spatial correlation structures (variograms). Similarity of spatial structures of the obtained datasets with the initial data was considered as even more important than statistical factors.

Figure 3.7. The scatter plot between altitude and air temperature in June

Results of geostatistical modeling are presented in Table 3.1 (errors on the validation dataset) and in Figure 3.8 (the estimation on the dense grid with known altitude values). It can be seen that the best results among geostatistical methods on a dataset test are obtained using kriging with external drift. Co-kriging results are worse than kriging results because of the screening effect [WAC 95]. Kriging and collocated kriging demonstrate similar patterns while kriging with external drift represents not only the large-scale structure but also small-scale variability effects ignored by kriging and co-kriging models.


Figure 3.8. Geostatistical estimates of air temperature in June on a grid: (a) kriging, (b) co-kriging, (c) collocated co-kriging and (d) kriging with external drift

Model

Correlation

RMSE

MAE

MRE

Kriging

0.874

3.13

2.04

-0.06

Co-kriging

0.796

3.97

2.45

-0.11

Collocated co-kriging

0.881

3.05

1.95

-0.07

Kriging with external drift

0.984

1.19

0.91

-0.03

Table 3.1. The air temperature test results for geostatistical models

63

64


3.4. Probability mapping with indicator kriging The analysis of environmental spatial data is always associated with uncertainty. Major uncertainties arise from the limited amount of initial information (there is an infinite number of possible distributions honoring any initial data), sampling uncertainty, unpredictable stochastic source terms and measurement errors. Thus, the presence of uncertainties is impossible to ignore, and special attention should be paid to its characterization. Classical geostatistics (kriging family of regression methods) accompanies an estimate by a variance, which can be (under some assumptions) treated as a description of the uncertainty. Another way of dealing with uncertainties is a probabilistic approach. It replaces estimates of values by the estimates of the local probability density functions (pdf). Post-processing of local pdf gives rise to reaching a mapping of: the probability of exceeding a value (for example, a critical level of contamination), the probability of belonging to an interval of values (for example, of finding an important amount of biomass), etc. Probabilistic description and corresponding mappings link with a risk analysis through the estimation of a probability of a dangerous event. Thus, geostatistical probability mapping is often treated as risk mapping. Indicator kriging is a non-parametric approach to estimate the local probability distribution function and to perform all types of probability and risk mapping. The non-parametric approach makes no assumptions about a model of the probability distribution [JOU 83]. The only assumption of indicator kriging concerns spatial continuity of the process; this means that information on what is going on at a spatial location can be obtained from the values at the neighboring locations. This approach is useful for the reconstruction of a local probability distribution function and for the estimation of the probability of a specific event. Indicator kriging is the kriging of a binary indicator variable. Indicator kriging can be applied to both categorical and continuous variables, which are nonlinearly transformed into binary values. 3.4.1. Indicator coding Indicator coding is a non-parametric estimation based on probability distribution discretization as a step function of a series of K threshold (cut-off) values zk for continuous variable Z(x):

F ( x; z k | (n))

Pr ob^Z ( x) d z k | (n)` k

1,", K .

[3.24]


65

Each sample is coded as a vector of K values I(x,zk), where I(x,zk) is the indicator transform carried out as follows:

I ( x, zk )

1, Z ( x ) d zk ®0, otherwise . ¯

[3.25]

Figure 3.9 illustrates indicator transformation of the well-log measurements. A threshold value zk is used to divide continuous log data into two categories, assuming two types of geological facies, which correspond to the well-log ranges above and below the threshold value. Two binary indicator variables I1 and I2 are constructed for the chosen two categories. In case of two indicator variables a one step pdf is estimated by indicator kriging at each local point. More detailed pdf is obtained by increasing the number of thresholds and the corresponding indicator variables.

Figure 3.9. Indicator transformation of continuous and categorical data into binary variables

The basic point of indicator approach is to select a set of thresholds. The number of thresholds has to be sufficient to represent the probability distribution, and the computational effort has to be reasonable. It appears that the reasonable number of thresholds depends on the number of samples and is the interval [5, 15]. The values for K thresholds are selected so as to split the data into K+1 classes of a nearly equal number of samples (use K deciles). Use of full indicator kriging requires a set of variogram models corresponding to each indicator variable.

66


Conditional probability can be interpreted in terms of indicators, i.e., as an expectation of an indicator conditioned to neighboring information (n):

F ( x; z k | (n))

E^I ( x; z k ) | (n)`.

[3.26]

An indicator transform is also applicable to the categorical type of data (classification problem). The categorical information is defined as a set of possible states sk, k=1,…,K. Each location of an area under study belongs to one of the states (S(x)). The uncertainty is modeled analogously to the continuous case by a conditional probability distribution function:

p ( x; s k | (n))

Pr ob^S ( x)

s k | (n)` k

1, ! , K .

[3.27]

Here again (n) indicates the neighboring information used for the estimation of the probability distribution. It can be explained by the same reasons as for a continuous case. The indicator transform for categorical data is also similar to the continuous case. It can be seen in Figure 3.9, where the categorical type (geologic face) is transformed into an indicator variable. In the multi-class case, the transform is performed for all classes. Possible states (classes) replace threshold values. The indicator transform appears as follows: I ( x; s k )

^

1, S ( x ) sk k 0, otherwise

1, ! , K .

[3.28]

3.4.2. Indicator kriging

Most often, members of the kriging family are used for an estimation of unknown indicators. They can be considered as a simple and ordinary kriging performed on indicators. A spatial correlation structure of indicators is used during the estimation. This is why careful selection of the thresholds is very important. Too many (or too few) zero indicator values significantly complicate variogram modeling. As for kriging there exist different types of indicator kriging: simple indicator kriging (where the indicator mean is known [SOL 86]); ordinary indicator kriging (where the indicator mean is constant through all the area but unknown). A linear form provides the estimate of the indicator variable: n( x)

i * ( x; z k )

¦ w ( x; z i

i 1

k

) I ( xi ; z k )

[3.29]


67

with coefficients wi(x;zk) obtained from solving the (simple or ordinary) kriging equation system. To construct ccdf using indicator kriging we need to calculate and model K semi-variograms and to solve K kriging systems for each location under estimation. Averaged over the whole zone, the indicator gives a global probability distribution function:

E I ( x; z k )

F (zk ) ,

[3.30]

This allows us to treat proportions of the samples with the values below zk as the means for simple indicator kriging. In the case of a categorical variable, means are replaced by a priori class probabilities, expected proportions of a class. This works in the case of stationarity. If there are local concentrations of high values (hot spots), the case of local stationarity a priori probability distribution functions used as means for simple kriging do not correctly reflect local situations. Ordinary kriging provides more reliable local estimates. Indicator kriging uses information on only one threshold at a time. Co-kriging formalism allows us to use the information on all K thresholds. In theory, co-kriging is a better estimator since it uses all information on all thresholds. However, in practice the computational complications of co-kriging overcome the improvements of obtained estimates. For indicator co-kriging with K thresholds, calculations and modeling of K(K-1) auto- and cross-semi-variograms of indicators are required. In addition co-kriging matrices are larger then kriging matrices. The probabilistic description of indicator estimates imposes some constraints on them. For each location these constraints are the following [GOO 97]:

>F ( x; z k | (n))@* >F ( x; z k | (n))@*

i * ( x, z k ) [0,1]

[3.31]

i * ( x, z k ) d >F ( x; z k ' | (n))@* i * ( x, z k ' ) z k ' ! z k [3.32]

Sometimes these constraints can be distorted at some locations. Such situations need to be found out and corrected. Usually, this is carried out by simple averaging of upward and downward corrections. Upward correction checks the non-decreasing feature of estimates while zk is increasing, while downward correction checks the non-increasing feature in the opposite direction.

68


In order to use probability distribution values for arbitrary thresholds, the obtained discrete values have to be interpolated through the intervals and extrapolated at the ends. Linear interpolation is a simple and rather good method for internal intervals:

>F ( z )@Lin

ª z z k 1 º F * ( z k 1 ) « » [ F * ( z k ) F * ( z k 1 )] z ( z k 1 , z k ] ¬ z k z k 1 ¼ [3.33]

The tails of the estimated distribution function can be interpolated in the same way, only the expected local minimum or maximum is required. For the upper tail value, a hyperbolic model can be used. This allows us to extrapolate the positively skewed upper tail of a distribution toward an infinite upper bound. The hyperbolic model is the following:

>F ( z )@Hyp

1

O zZ

z ! z K ,

[3.34]

where parameter Z t1 controls the speed of reaching the cdf’s limiting value. The smaller Z is, the longer the tail of the distribution is. Parameter O identifies the sample cumulative frequency F*(zK):

O

z KZ >1 F * ( z K )@ .

[3.35]

In the categorical case the probability notation also imposes constraints on the estimated indicators:

> p( x; s k | (n))@* K

¦ > p( x; s k 1

k

i * ( x; s k ) [0,1] k

| (n))@*

1,!, K ,

[3.36]

K

¦ i * ( x; s

k

) 1

[3.37]

k 1

The following two corrections help if constraints [3.36] and [3.37] are violated. All estimates with the conditional probability outside the interval [0, 1], are set to the closest border. All K estimates are standardized using the sum to agree with constraint [3.37].


69

3.4.3. Indicator kriging applications

3.4.3.1. Indicator kriging for 241Am analysis The indicator approach was applied to 241Am soil contamination data [KAN 03]. The aim was to reconstruct local distribution functions at the validation locations and, based on them, work out the probability estimates of exceeding the given thresholds: 17, 27 and 38 pCi/g. The analysis was performed on 163 samples. A more detailed description of data can be found in [KAN 06].

Northing (km )

< 1.83 > 1.83

1800

1900

2000

2100

2200

2550 2600 2650 2700 2750 2800 2850 2900

Northing (km )

2550 2600 2650 2700 2750 2800 2850 2900

The most appropriate levels to reconstruct a distribution function are quintiles of the initial data. In this case, 7 quintiles (with corresponding values 1.83, 4.79, 7.295, 8.67, 11.111, 14.874, 21.71 pCi/g) were selected. Two additional cut-offs at the critical levels of 27 and 38 pCi/g were considered for better accountancy of the high-valued tail of the distribution. Figure 3.10 presents results of indicator transform for four cuts (1.83, 8.67, 21.71 and 38 pCi/g). Black filled circles present locations with an indicator equal to one; white circles indicate zero indicators. Indicator variograms for the selected cut-offs were calculated and fitted with the theoretical models. Ordinary indicator kriging was performed. Local cumulative probability distribution functions were estimated. Figure 3.11 presents 15 examples of local cumulative pdfs.

2300

< 8.67 > 8.67

1800

1900

Northing (km )

< 21.71 > 21.71

1800

1900

2000

2100

2200

Easting (km )

2000

2100

2200

2300

Easting (km )

2300

2550 2600 2650 2700 2750 2800 2850 2900

Northing (km )

2550 2600 2650 2700 2750 2800 2850 2900

Easting (km )

< 38 > 38

1800

1900

2000

2100

2200

2300

Easting (km )

Figure 3.10. Indicator transform for four cut-off levels (1.83, 8.67, 21.71 and 38 pCi/g). Black filled circles represent locations with indicator equal to 1; white circles indicate zero indicators


40

80

c c df

0

40

0.0 0.4 0.8

c c df

0

0.0 0.4 0.8

c c df

80

41

80

0

40

62

72

89

97

80

0

40

80

0

40

80

c c df

c c df

c c df

c c df

40

0

40

80

0

40

A m241

A m241

A m241

103

104

107

109

112

40

80

40 A m241

80

0

40 A m241

80

c c df

c c df

c c df

0

0

40 A m241

80

80

0.0 0.4 0.8

A m241

0.0 0.4 0.8

A m241

A m241

80

0.0 0.4 0.8

50

0.0 0.4 0.8

A m241

0.0 0.4 0.8

A m241

0.0 0.4 0.8

A m241

c c df

0

40

0.0 0.4 0.8

c c df

0

36

A m241

0.0 0.4 0.8

0

0.0 0.4 0.8

80

0.0 0.4 0.8

c c df

40

32

A m241

0.0 0.4 0.8

0

c c df

31

0.0 0.4 0.8

c c df

11

0.0 0.4 0.8

70

0

40

80

A m241

Figure 3.11. Indicator kriging: local cumulative probability density functions at 15 testing locations (Ɣ) – sample value plotted at 0.5 p-value. The plots are labels according to sample numbers

Based on the indicator kriging with 9 thresholds, probability maps of exceeding given critical levels (17, 27 and 38 pCi/g) were constructed and presented in Figure 3.12. In these figures white marks indicate measurements actually above the indicated level. Most of the plots show that the measured value matches the modeled P50 value closer.


71

Figure 3.12. Probability maps of exceeding given critical levels (17, 27 and 38 pCi/g). White marks indicate measurements really being above the indicated level

3.4.3.2. Indicator kriging for aquifer layer zonation This example illustrates the application of indicator kriging to a classification task: zonation of a geological aquifer layer. A detailed description of the data and the tasks can be found in [SAV 02 and SAV 03]. This parameter zonation approach is an alternative to the analysis of the spatial variation of the continuous hydraulic parameters. The parameter zonation approach is primarily motivated by the lack of measurements that would be needed for direct spatial modeling of the hydraulic properties. In the current case a hydrogeological layer is presented by 5 zones with different hydraulic properties: class 1 – gravel type 1; class 2 – gravel type 2; class 3 – gravel type 3; class 4 – sand; class 5 – silt. The initial data is a set of 225 samples. Figure 3.15 presents the initial data by using Voronoï polygons. Here Voronoï tessellation is used only for clearer visualization, as a simple post plot presents overlapping of close samples. The data are transformed into indicators for 5 possible values of a categorical function. The corresponding variogram is estimated and modeled for each indicator.

72


Figure 3.13. Initial data (classes)

Indicator kriging provides estimates of probabilities of belonging to any class. Figure 3.14 presents examples of such probability maps for class 1 (gravel 1) and class 4 (sand). The final classification solution depends on the class with the largest estimated probability. Figure 3.15 presents the classification result and the probability of the winning class. The solution rule can be changed if we only want highly probable classes to compete. This means that if no class overpasses a level of probability (for example, 0.7) at a location, this location is treated as unclassified. Figure 3.16 presents the classification result using such a classification rule with the level of probability set at 0.7. White zones (unclassified zones – zones with low probability of all classes) are zones of uncertainty for the current classification problem.


Figure 3.14. Probabilities of belonging to class 1 (gravel 1) and class 4 (sand)

Figure 3.15. Zonation by indicator kriging: classification (left) and probability of the class winner (right)

73

74


Figure 3.16. Classification by indicator kriging with strong classifying rule – the winning class needs to have a probability higher than 0.7

3.4.3.3. Indicator kriging for localization of crab crowds This example deals with the spatial distribution of opilio crabs in the Bering Sea. Measurements were taken by trawl survey and are presented in catch numbers. The range of values varies from zero (nothing was found) up to 821,422 crabs. Such strong variability significantly complicates any interpolation. However, in reality, the interpolated value is of no special interest; the actual goal is to determine where to find a crowd of crabs. The probability of finding a crowd seems to be a sufficient type of estimate. A number over 5,000 crabs is considered to be a crowd. Figure 3.15 presents the spatial distribution of indicator transformed measured values with the level zk=5,000. Indicator kriging allows us to estimate the probability of finding a crowd equal to or higher than 5,000 individuals. The indicator kriging estimate gives the probability of being below or above the given value. Figure 3.16 presents the probability of finding a crowd of opilio crabs (more than 5,000 crabs in a catch). Light marks indicate the distribution of the initial data from this class of values. The correspondence seems to be quite good.


75

Figure 3.15. Spatial distribution of indicator transformed number of opilio crabs (level 5,000 numbers)

Figure 3.16. Probability of finding a crowd of opilio crabs (more than 5,000 individuals). White marks indicate locations of measurements of such crowds

76


3.5. Description of spatial uncertainty with conditional stochastic simulations 3.5.1. Simulation vs. estimation

Spatial estimation models (e.g. kriging), which were described above, provide a single regression point estimate of a variable value for a chosen set of parameter values. Such point estimates are important in prediction mapping problems. However, a single estimate does not describe the range of uncertainty of the estimated variable. The reality is naturally more complex than all the sophisticated prediction models we can possibly think of. Even the very best prediction model is not able to reflect all the peculiarities of the real spatial pattern. Thus, our models can represent an average property value with a resolution relevant to a chosen grid. Prediction grid resolution is usually lower than the resolution of the measurement data (measurement support). For example, in large scale radioactive pollution mapping the common grid resolution is 102-103 m, while the observations are collected from the scale of 10-2 m (soil sample), 1 m (repeated samples) to 102 m (aerial gamma survey). Detailed samples collected with a high spatial resolution can describe local variability, which remains an essential feature of spatial patterns in environmental problems. Spatial regression models are not able to reproduce this variability as far as they impose a smooth surface honoring the available data. Stochastic simulation approach, as opposed to regression, provides multiple estimates of variable values at every considered location, which are calculated as stochastic realizations of the unknown random function. Conditional stochastic simulations are able to honor the observation data by reproducing the data exactly, like in kriging. Multiple realizations of a spatial process bring several important benefits for solving a prediction problem: – stochastic realizations do not smooth out the estimated pattern between the data locations, which is essential in preserving the realistic variability of the spatial pattern; – realizations represent a range of uncertainty with a local distribution of possible values. Capturing the local variability allows us to describe small scale variations, which may be reflected by observations, but are not reproduced by smoothed regression estimates away from data locations; – multiple realizations have equal probability by the construction algorithm. This allows us to assess the probability of the true unknown value being over a certain level and derive confidence bounds, which would encompass the real value with a certainty probability. Principal differences between stochastic simulation and estimation are illustrated in Figure 3.17, where five stochastic realizations of a 1D pattern are plotted against two regression estimates (simple and ordinary kriging) conditioned by six items of


77

data. Kriging estimates are smooth between the data, which are honored exactly. Conditional stochastic simulations provide multiple variable realizations between the data, which are also honored exactly. It is clearly shown that stochastic realizations can be larger than the maximum observation value and smaller than the minimum value, which bound the kriging estimates. We should also note that the variability of the realizations is smaller in the areas with more data (e.g. x belongs to [5,20] in Figure 3.17), which naturally restrict the simulations from being too far from the surrounding data.

Figure 3.17. Multiple stochastic simulation realizations against kriging estimates (simple kriging and ordinary kriging using a spherical variogram, correlation range – 50, nugget – 2, sill – 3)

Stochastic simulations can still be carried out in the absence of data measurements based only on the prior knowledge of a global distribution and its structure. Such unconditional simulations still preserve global properties of the pattern: statistics (mean, variance, etc.), shape of the distribution density function (histogram) and spatial correlation (variogram). 3.5.2. Stochastic simulation algorithms

There exists a great variety of stochastic simulations algorithms used in spatial modeling based on a Monte Carlo technique in one way or another. Most of the methods fall into one of the two categories: cell-based (or pixel-based) models and object-based models [CHI 99; DEU 02].

78


Object-based algorithms model the variable value in local vicinities according to the predefined geometrical shapes (objects), which altogether form a pattern. These shapes are placed over the modeling region following one or another optimization technique, which can minimize the objective function based on data, the distribution statistics or spatial structure. In this case, spatial correlation is determined solidly by the geometric shapes as an alternative to a variogram. Object-based algorithms benefit from clear interpretability, based on the choice of realistic object shapes, which reflect the nature of the modeled phenomenon. This also makes up one of the major weaknesses of the object approach: the choice of the shapes assumes good knowledge of the pattern structure and is subject to vast uncertainties. Another drawback of the object-based approach is its possible computational cost, once an optimization technique requires multiple refitting of the spatial patterns to data. Data conditioning can be poor and numerically complex to achieve by iterative optimization. Also, conditioning the object-based realization to data from different scales (e.g. soft probabilities) is not straightforward as it would require a complex objective function and lead to a further increase of computations. However, objectbased models are widely used in such fields where prior information about the pattern structure is available (geology, hydrology, etc.). Some examples of possible object-based simulation realizations are shown in Figure 3.18.

a)

b)

Figure 3.18. Examples of object-based model stochastic realizations: a) fluvial channels deposited in a river system; b) aeolian dunes occurring in a desert landscape

In the cell-based approach, unlike the object-based method, we model the pattern value in every grid cell sequentially – cell by cell. This approach does not entail costly optimization algorithm once data conditioning becomes straightforward in every particular cell where the data is located. There are cell-based algorithms to model both continuous and categorical variables. Here we will describe just two of the most widely used in spatial modeling: sequential indicator simulations and sequential Gaussian simulations.


79

The most recently developed cell-based models are associated with multiplepoint statistics [STR 02]. Multi-point statistics simulation is based on a spatial correlation model represented by a multiple point statistical moment rather than on a second-order two-point statistical moment (variogram) as in the most conventional geostatistical simulations. The multi-point statistic is described by a training image, which represents the global correlation structure more accurately than the conventional variogram. The use of the global structural information from a training image brings more realism and interpretability as in the case of object-based models. Thus, global structural dependencies obtained from a training image are conditioned to the local data using the sequential simulation principle (described below). Therefore, unlike in the object-based approach, data conditioning on all scales is straightforward in multi-point simulation and no iterative optimization algorithm is involved [CAE 05]. Simulated annealing is used as another cell-based algorithm, which is related to the object-based approach in some sense. It can be treated as a cell-based algorithm in the sense that it models realizations in each consecutive grid sell. However, as in the object-based approach, simulated annealing employs an optimization technique to minimize the objective function (variogram/histogram-based) by means of moving the simulated grid values around [DEU 98]. In a way it can be seen as a variant of the object-based approach, where all objects have a unique shape – a basic grid cell. Although simulated annealing shares the similar optimization techniques with object-based modeling, it does not imply any realistic prior shapes and therefore uses a variogram to represent spatial correlation. Actually, simulated annealing is a much more general algorithm which is an analogy to metal cooling in thermodynamics and is based on the Boltzmann relation between temperature and energy [MET 53]. Most geostatistical cell-based algorithms are based on the sequential simulation principle, which represents the joint probability distribution function of the entire pattern as a product of N local probability distribution functions conditioned by n observed data: F(x1,…,xN; z1,…,zN |(n))= F(xN; zN |(n+N-1)) F(xN-1; zN-1 |(n+N-2)) … F(x1; z1 |(n)) In practice, the sequential simulation principle is produced in a way where each of the sequentially simulated cells are then used to simulate further cells along with the conditioning data. The sequence in which the cells are simulated is determined by a random path regenerated for each stochastic realization. Sequential simulation algorithm steps are illustrated in Figure 3.19 showing how a simulated value for a single realization is randomly drawn from the modeled local cdf and then added to the data set simulation at further locations.

80


Random selection of the simulation cell

24 14.9

?

50.2 Modeling of the local pdf (parametric or non-parametric)

24 30.1±9

14.9 50.2

Random draw of the simulation value from the local pdf

21.1

Simulated value is added to the data pool to be used in further simulations

24

35.2

30.1

39.1

? 14.9 50.2

Figure 3.19. Sequential simulation algorithm


81

The stochastic nature of simulation is embedded in the algorithm by random sampling from a local probability density function, which is constructed at each simulated cell location. Each random draw from the distribution corresponds to a single stochastic realization. The question is how to obtain this probability distribution function. There are several ways of doing this: parametric and nonparametric. The parametric approach entails assumption about the form and shape of the distribution, defined analytically (e.g. Gaussian). The non-parametric approach implies the definition of the local distribution function directly using a set of p-quantiles and the relevant interpolation between them. Parametric and nonparametric approaches provide different types of sequential simulations: Gaussian and truncated Gaussian algorithms, indicator simulations, direct simulations. Multipoint statistics simulations are non-parametric algorithms as they do not assume any analytical form of the local pdf but obtain it from the probability of the multi-point statistics pattern (data event) in the training image. It is worth noting that the sequential simulation algorithm is subject to the screen effect [ISA 89]. This occurs when the kriging weights of the data points that fell between one of the sampling points and the simulated point are decreased. Thus, some of the previously simulated values accounted for in the simulation act as a screen for the original sample values. This may lead to the appearance of negative weights. In practice it is not necessary to use all the available sample data for building the cdf at the simulated location. To overcome the screen effect, the number of conditional samples from the neighborhood of the simulated point can be restricted to the closest points with respect to the octant search. 3.5.3. Sequential Gaussian simulation

Sequential Gaussian simulation is widely used to model spatially distributed continuous variables (e.g. porosity, concentration, intensity, amount of rainfall) [DEU 02, GOO 97]. The key assumption behind the algorithm is a joint normal distribution of the spatial variable. This means that all the variable components at all evaluated locations are jointly normally distributed. This property – called multinormality – makes all local distributions Gaussian. The Gaussian distribution is determined by just two parameters – the mean and the variance, which makes it parametrically attractive to calculate. However, multi-normality is quite a heavy assumption, which cannot be checked in practice. There exist several tests for binormality, a weaker assumption, which states that every pair of values is jointly normally distributed [EME 04]. A sequential Gaussian simulation algorithm consists of the following steps [DEU 98]:

82


1. Normal transformation of the original data to standard Gaussian distribution N(0,1) is performed first if the data are not normally distributed initially. Usually, it is used, unless the data are lognormally distributed. The transformation is carried out using an approximate analytical function ĳ. A tabular inverse function ĳ-1 is constructed simultaneously to be used in the back transformation. 2. Choice of the simulated location along a random path, which visits all the points to be simulated. 3. Calculation of a simple kriging (SK) estimate and variance at the chosen location using the normal score variogram model (see section 3.2) and the conditioning data in the local neighborhood. Simple kriging estimation is calculated with a constant known mean equal to zero for the normalized data. 4. Construction of a local normal distribution with the mean equal to the SK estimate and variance equal to the SK variance at the simulated point. 5. Random draw of a stochastic realization value from the constructed local normal distribution at random according to the Gaussian probability density function. 6. Addition of the simulated value to the pool of conditioning data and choice of the next location (step 2) to be simulated. The previously simulated data are used in the simulation at further locations along with the observations. 7. Back transformation of the simulated realization to the original distribution values. Interpolation/extrapolation function has to be used between the values of the tabular function ĳ-1 and for the tails of the distribution. Multiple stochastic realizations are obtained by repeating steps, starting with step 2 through to step 7. The random path is regenerated for each stochastic realization in step 2. Gaussian models are theoretically consistent models and have several benefits. They are well known, and easy to calculate and integrate. Sampling from the local Gaussian distribution ensures that the set of spatial realizations will keep the form and shape of the local variability. The use of any other distribution would result in a variety of shapes of the local distributions. One of the drawbacks of the Gaussian approach is the maximum entropy – it imposes maximum “disorder” in the data. Maximum entropy results in poor connectivity of the extreme values, which is not always the case in nature. This is perhaps not the best choice when spatial correlation between high extreme values is of special interest. One possibility is to take a non-parametric model such as indicator-based simulations. A normal score variogram is also more stable and robust than a raw variogram, which eases the variogram modeling. A variogram of the normal score data must have a total sill equal to 1, according to the standard Gaussian variance.


Figure 3.20. Stochastic realizations of sequential Gaussian simulations with different variogram parameters: a) nugget c0=0, angle Į=60, main range along direction Į R=40, minor range r=8; b) c0=0.4, Į=60, R=40, r=8; c) c0=0, Į=60, R=8, r=40; d) c0=0, Į=60, R=40, r=40; e) c0=0, Į=60, R=80, r=4; f) c0=0, Į=60, R=8, r=8; g) c0=0, Į=0, R=40, r=8; h) c0=0, Į=90, R=8, r=40

83

84


To simulate a categorical non-continuous variable, another type of sequential Gaussian simulation can be used, truncated Gaussian simulation, which is based on the same Gaussian assumption [DEU 02]. A set of equally probable realizations of the spatial variable distribution is a result of sequential Gaussian simulation. The simulated realizations share the same global distribution characteristics (mean, variance, histogram, etc.) and spatial correlation, reproduce exactly the same conditioning data, but differ in local peculiarities. The difference between the realizations characterizes the uncertainty of the model. In order to obtain statistical inference for the spatial distribution mean, variance, p-quantiles, etc., further postprocessing of the the realizations can be carried out. Post-processing of stochastic realizations offers a wide range of probabilistic results. Thus, an averaging over the realizations provides a smooth E-type estimate that can be compared to a regression type (kriging) estimate as a single solution. The difference between the realizations allows us to evaluate the range of variability of the spatial estimates. Local probability distribution functions (pdf) can be calculated based on the multiple realizations at each locality assuming that all the realizations have equal probability. A probability of the true value of exceeding the chosen threshold can be calculated from the local pdfs, which are based on the kriging mean and variance. Similarly, an estimate corresponding to a certain pquantile can be obtained based on the same probability density functions. Stochastic realizations heavily depend on the normal score variogram model parameters used. Figure 3.20 illustrates a realization calculated with different variogram nugget, ranges and anisotropy direction angle. Overlayed arrows show the pattern structures corresponding to the variogram ranges along and across the anisotropy direction. 3.5.4. Sequential indicator simulations

Sequential indicator simulation is a non-parametric method, which does not assume any analytical form of the local pdf, unlike in Gaussian simulation. Indicator simulation is based on the indicator approach which allows us to estimate local pdf using indicator kriging (see section 3.3). Indicator simulation is a cell-based algorithm which models values in each cell sequentially along a chosen random path. It follows the sequential principle, when previously simulated data are used in the next evaluations. A sequential indicator simulation algorithm consists of the following steps [DEU 98]: 1. Indicator transformation of data according to the set of thresholds (cut-offs). A global proportion (conditional density function value) and a variogram model are


85

built for each indicator variable. Note that global proportions should be estimated taking into account clustering of the observation data, so an appropriate declustering algorithm can be applied, e.g. [DEU 89]. 2. Definition of a random path through all the simulation points. 3. Estimation of probabilities for each indicator variable using indicator kriging and normalization of their sum to 1. 4. Construction of a local probability density function based on the estimated probabilities at the point. 5. Random draw of a simulated realization value from the probability density function and calculation of the corresponding values for indicator variables. 6. Addition of the simulated indicator value to the set of conditioning data to be used in simulation at further locations. 7. Move to the next simulation point along the chosen random path and repeat steps 3-6. 8. Multiple stochastic realizations are obtained by repeating steps 2-7. Indicator simulation ensures the approximate representation of the average proportion for each category (indicator variable) given the global distribution and the indicator variogram for each category. Conditional indicator simulation realizations honor the observations and reproduce the spatial correlation structure (variogram). In the case of modeling a continuous variable simulated indicator realization provides values of the actual variable sampled from the local pdfs, evaluated by indicator kriging. If the number of realizations is large, the realizations reproduce the estimated pdf fairly well. Thus, averaged realizations (E-type) and maps of probability quantiles are approximately the same as those obtained from indicator kriging. The benefit of simulation is in the realizations themselves, which can be used as an input for further risk modeling resulting in the evaluation of uncertainty for decision-making. In the case of modeling a categorical variable, spatial realizations of indicator variables for each category can be combined into a joint realization of a categorical staochastic pattern. A set of multiple realizations would characterize uncertainty in the occurance of one or another category in every location. The probability of any category occuring in a particular location is determined by the corresponding pdf value, which is based on the family of realizations. Figures 3.21 and 3.22 illustrate the influence of the anisotropic variogram range on the simulation results.

86


a)

b)

c)

d) Figure 3.21. Sequential indicator simulation realizations with different horizontal correlation ranges: a) r=160, b) r=80, c) r=40, d) r=20 and vertical range R=8


87

a)

b)

c)

d) Figure 3.22. Sequential indicator simulation realizations with different vertical correlation ranges: a) r=20, b) r=10, c) r=5, d) r=2 and horizontal range R=80

88


3.5.5. Co-simulations of correlated variables

The geostatistical stochastic simulation of correlated variables is called cosimulation and is based on a corresponding co-kriging model. Any co-kriging model discussed in section 3.3 can be used for co-simulations. In the case of a simple linear regression relationship between the two variables the value of the correlated variable can be obtained from one already simulated, e.g. using Gaussian simulations. Analogous to estimation each realization of correlated variable simply mimics the realization of the first one. Collocated co-kriging (see section 3.3.4) provides the mean and variance to calculate local normal probability density functions (pdf) of the primary correlated variable in case of Gaussian simulations. The local pdf for a secondary variable was similarly obtained based on a simple co-kriging estimate (as the mean) and variance. Therefore, the correlated variables are sampled independently of two separate local pdfs. Thus, the simulated distribution patterns are not the same as in the case of simulation with linear regression. Collocated co-simulation implies its own spatial correlation (variogram) model for the simulated correlated variable, which accounts for peculiarities of its own spatial structure. Thus, an additional modeling effort is needed to fit the variogram model for the second correlated variable. In collocated co-simulations the conditioning data for both variables are honored as in sequential Gaussian simulation. In comparison to the simple simulation with linear regression the increased computational costs for an additional stochastic simulation round is traded for more accurate modeling of spatial correlation of the second simulated variable. However, collocated co-kriging also assumes linear correlation between the variable and, thus, only uses a single piece of secondary data at the estimated point, while the full co-kriging used all the secondary data from the local neighborhood. Such simplification may not affect the kriging estimate if the neighboring secondary data do not differ much but the variance can be over-estimated, which would lead to larger variability of stochastic realizations [DEU 02]. An example of collocated cosimulation is presented below in this section. The third way of simulating correlated variables is based on the full co-kriging method (see section 3.3.3). Sequential Gaussian co-simulation with full co-kriging simultaneously calculates individual local pdfs for all correlated variables the simulated location based on the co-kriging estimates and estimation errors as Gaussian means and variances. In full co-simulation the sampling of each simulated variable is carried out simultaneously independently of the relevant distributions. This provides spatial variability of individual distributions based on the different spatial correlation structure and taking into account their joint correlation not assuming just a linear regression.


89

An example of modeling porosity and permeability in a sub-service reservoir illustrates stochastic Gaussian collocated co-simulation. Porosity and permeability is usually highly correlated in permeable porous media (e.g. sands). A linear correlation between porosity and log-permeability can often be assumed. This assumption is used in a synthetic PUNQ-S3 case study, which is a benchmark case in the oil industry [FLO 01]. In the case study horizontal permeability was assumed correlated with porosity with a correlation over 80%. Auto-variograms for porosity and permeability are shown in Figure 3.23 as anisotropic rose contour diagrams. Actually, they feature quite similar strong anisotropic structures, which is also reflected by the cross-variogram (see Figure 3.23). The sequential Gaussian simulation realizations of porosity are presented in Figure 3.24. They look quite different due to the lack of conditioning data (from only 6 well locations). The corresponding permeability realizations calculated by sequential Gaussian cosimulations are presented in Figure 3.25.

Figure 3.23. Auto- and cross-variogram contours for porosity and permeability

Figure 3.24. Stochastic porosity realizations from sequential Gaussian simulations

90


Figure 3.25. Stochastic permeability realizations from sequential Gaussian co-simulations

Note that it is possible to calculate multiple stochastic realizations of permeability based on each porosity realization. This will result in additional variability in permeability results imposed by porosity realizations. Alternatively, using a single porosity realization as a secondary correlated variable input in permeability co-simulation will feature a reduced level of uncertainty associated with permeability only. Multiple stochastic realizations of porosity and permeability reproduce spatial correlation characterized by variograms (see Figures 3.26 and 3.27). The generated sets of porosity/permeability realizations are used as an input into the flow simulation model to simulate uncertainty of the oil production forecast (see [DEM 04]). Cumulative oil production profiles for 50 stochastic realizations are presented in Figure 3.28a in comparison with the known “TRUTH” case solution in the synthetic case. The histogram in Figure 3.28b shows the uncertainty distribution of the total cumulative oil production by the end of the forecasting period of 16.5 years against that for the “TRUTH” case.


91

Figure 3.26. Reproduction of omnidirectional variogram by 50 stochastic realizations of porosity (left) and permeability (right)

Figure 3.27. Reproduction of directional variograms by 50 stochastic realizations of porosity (left) and permeability (right)

a)

b)

Figure 3.28. Flow simulation: a) oil production forecasts with 50 realizations of poro/perm fields against the “TRUTH” case (solid line); b) histogram of the distribution of FOPT production predictions after 16.5 years against the “TRUTH” case (solid line at 3,870,000)

92


3.6. References [BOU 97] BOURGAULT G., “Statistical declustering weights”, Mathematical Geology, vol. 29, p. 277-290, 1997. [CAE 05] CAERS J., Petroleum Geostatistics, Society of Petroleum Engineers, 2005. [CHI 99] CHILES J-P. and DELFINER P., Geostatistics: Modeling Spatial Uncertainty, John Wiley & Sons 1999. [CLA 84] CLARK I., Practical Geostatistics, Elsevier Applied Science Publishers, London and NY, 1984. [CRE 93] CRESSIE N., Statistics for Spatial Data, John Wiley & Sons, 1993. [CRO 83] CROZEL D. and DAVID M., “The combination of sampling and kriging in regional estimation of coal resources”, Mathematical Geology, vol. 15, p. 571-574, 1983. [DAV 88] DAVID M., Handbook of Applied Advanced Geostatistical Ore Reserve Estimation, Elsevier Science Publishers, Amsterdam B.V., 216 p., 1988. [DEM 04] DEMYANOV V., CHRISTIE M. and SUBBEY S., “Neighbourhood Algorithm with Geostatistical Simulations for Uncertainty Quantification Reservoir Modeling: PUNQ-S3 Case study”, 9th European Conference on Mathematics in Oil Recovery ECMOR IX 2004, Cannes, France, September 2004. [DEU 82] DEUTSCH C. and JOURNEL A., GSLIB: Geostatistical Software Library, Oxford University Press, 1998. [DEU 89] DEUTSCH C. DECLUS A., “Fortran 77 program for determining optimal spatial declustering weights”, Computer & Geosciences, 15(3), p. 325-332, 1989. [DEU 02] DEUTSCH C., Geostatistical Reservoir Modeling, Oxford University Press, 2002. [DOW 82] DOWD P.A., “Lognormal kriging – the general case”, Mathematical Geology, vol. 14, p. 475-499, 1982. [EME 05] EMERLY X., “Variogram of order Ȧ: A tool to validate a bivariate distribution model”, Mathematical Geology, 37(2), p. 163-181, 2005. [FLO 01] FLORIS F.J.T., BUSH M.D., CUYPERS M., ROGGERO F. and SYVERSEEN A-R., “Methods for quantifying the uncertainty of the production forecasts: a comparative study”, Petroleum Geosciences, vol. 7, p. S87-S96, 2001. [GAN 63] GANDIN L.S., Objective Analysis of Meteorological Fields, Israel program for scientific translations, 1963, Jerusalem. [GOO 97] GOOVAERTS P., Geostatistics for Natural Resources Evaluation, Oxford University Press, 1997. [HAA 90] HAAS T.C., “Lognormal and moving window methods of estimating acid depositions”, Journal of American Statistical Association, vol. 14, p. 950-963, 1990. [JOU 78] JOURNEL A.G. and HUIJBREGTS C.J., Mining Geostatistics, Academic Press, 600 p., London, 1978.


93

[JOU 83] JOURNEL A.G., “Non-parametric estimation of spatial distributions”, Mathematical Geology, vol. 15, p. 445-468, 1983. [ISA 89] ISAAKS E. and SHRIVASTAVA M., Applied Geostatistics, Oxford University Press, 1989. [KAN 03] KANEVSKI M., BOLSHOV L., SAVELIEVA E., DEMYANOV V., PARKIN R., TIMONIN V., CHERNOV S., MCKENNA S., “Spatio-temporal analysis of ground water contamination”, Proceedings of IAMG 2003, Portsmouth, UK. [KAN 04] KANEVSKI M. and MAIGNAN M., Analysis and Modelling of Spatial Environmental Data, EFPL Press, 2004. [KAN 06] KANEVSKI M., DEMYANOV V., SAVELIEVA E., PARKIN R., POZDNOUKHOV A., TIMONIM V., BOLSHOV L. and MCKENNA S., “Validation of geostatistical and machine learning models for spatial decision-oriented mapping”, Proceedings of StatGIS 99, J. Heyn, Klagenfurt, 2006. [MET 85] METROPOLIS N., ROSENBLUTH A., TELLER A. and TELLER E., “Equations of state calculations by fast computing machines”, J. of Chem. Physics, 21(6), p. 10871092, 1985. [MAT 54] MATHERON G., “Démonstration approchée de la convergence vers la loi lognormale du schéma doublement binomial”, Note Statistique, no. 5, CG, Ecole des Mines de Paris. [MAT 63] MATHERON G., “Principles of geostatistics”, Economic Geology, vol. 58, p. 1246-1266, December 1963. [MYE 91] MYERS D., “Pseudo-cross variograms, positive-definiteness, and cokriging”, Mathematical Geology, 23(6), p. 805-816, 1991. [PAR 91] PARKER H.M., “Statistical treatment of Outlier data in epithermal gold deposit reservoir estimation”, Mathematical Geology, vol. 23, no. 2, 1991. [PAR 03] PARKIN R., DEMYANOV V., KANEVSKI M., TIMONIN V. and MCKENNA S., “Improved uncertainty assessment for environmental decision making with hybrid models: ANN and stochastic simulations”, Proceedings for StatGIS 2003, Springer. [PAR 04] PARKIN R., KANEVSKI M., SAVELIEVA E., DEMYANOV V. and MCKENNA S., “Geostatistical Analysis if radioecological spatio-temporal data”, Izvestia of Russian Academy of Sciences, Applied Energy, 3, p. 59-73, 2004 (in Russian). [REN 79] RENDU J.M., “Normal and lognormal estimation”, Mathematical Geology, vol. 11, p. 407-422, 1979. [SAV 02] SAVELIEVA E., KANEVSKI M., TIMONIN V., POZDNUKHOV A., MURRAY C., SCHEIBE T., XIE Y., THORNE P. and COLE C., “Uncertainty in the hydrogeologic structure modelling”, in Proceedings of IAMG2002 Conference, September 2002, Berlin, Germany, p. 481-486. [SAV 03] SAVELIEVA E., KANEVSKI M., TIMONIN V., POZDNUKHOV A., MURRAY C., SCHEIBE T., XIE Y., THORNE P. and COLE C., “Aquifer hydrogeologic layer zonation at the Hanford Site”, Proceedings of IAMG2003 Conference, September 2003, Portsmouth, UK.

94


[SAV 05] SAVELIEVA E., “Using Ordinary Kriging to Model Radioactive Contamination Data”, Applied GIS, vol. 1, no. 2, p. 10.1-10.10, 2005. [SCH 93] SCHOFIELS N., “Using entropy statistic to infer population parameter from spatially clustered sampling”, in Geostatistics Troia’92, Volume 1, Quantative Geology and Geostatistics (ed. Soares A.), Kluwer Academic Publishers, Dordrecht, p. 109-119, 1993. [SOL 86] SOLOW A.R., “Mapping by simple indicator kriging”, Mathematical Geology, vol. 18, no. 3, 1986. [STR 02] STREBELLE S., “Conditional simulation of complex geological structure using multiple-point statistics”, Math. Geology, vol. 34, p. 1-22, 2002. [WAC 95] WACKERNAGEL H., Multivariate Geostatistics, Springer-Verlag, Berlin, 256 p., 1995. [ZIM 93] ZIMMERMAN D.L., “Another look at anisotropy in geostatistics”, Mathematical Geology, vol. 25, no. 4, 1993.

Chapter 4

Spatial Data Analysis and Mapping Using Machine Learning Algorithms

4.1. Introduction This chapter presents a brief introduction to the broad field of machine learning (ML), an exciting research subject in which statistics, computer science and artificial intelligence overlap. As data becomes easier to collect, techniques that can handle massive datasets involving a large number of variables in an efficient manner become essential. ML real-world applications have flourished over the last two decades, especially in the fields of data mining, bioinformatics, speech and character recognition, web and text mining and, more recently, environmental science and remote sensing data analysis. Machine learning can be broadly defined as a set of mathematical, computational and statistical methods that aim to automatically learn rules and dependencies from examples. These rules can be a curve providing the best fit for a set of points from a predictive point of view, a discriminative function that guides a classification task, such as in soil type recognition from remote sensing data, or even hypotheses about the spatial and temporal distribution of a certain phenomenon. The examples usually take the form of a database that registers numerical, categorical or qualitative attributes of samples representing the phenomenon under study.

Chapter written by F. RATLE, A. POZDNOUKHOV, V. DEMYANOV, V. TIMONIN and E. SAVELIEVA.

96


At present there is excellent literature on machine learning algorithms and their applications. Here we can mention references in which the reader can find detailed explanations of theories, applications and algorithms [ABE 05, BIS 06, CRI 00, GUY 06, HAS 01, HAY 98, JEB 04, KOH 00, RAS 06, SCH 06, SHA 04, VAP 06, VAP 95, VAP 98]. 4.2. Machine learning: an overview 4.2.1. The three learning problems When confronted with a data modeling problem in environmental science, we want to perform tasks such as building a map, predicting to which category a certain soil belongs, evaluate a risk related to a specific pollutant, etc. These tasks, however numerous, usually fall into one or a combination of these categories: – regression – classification – density estimation Building a map is a classical task of regression or function approximation. Given a finite set of points, we want to build a function predicting values at any location in space. The goal is to learn a functional relationship from a training set ^x i , y i ` , where xi is a p-dimensional vector, often called an input, and yi is a vector of continuous values. The latter is usually called the target or output vector. Input vectors xi are assumed to be drawn independently from the same (unknown) probability distribution. Figure 4.1 illustrates a typical function approximation problem in one dimension. Given a finite set of points – sampled here from the function sin x x with added Gaussian noise – the goal is to build a function that will represent the data well while being useful for prediction, without knowledge about the original functional form sin x x .


97

Figure 4.1. An example of regression problem

In Figure 4.1 the solid line represents a 2nd order polynomial fitted by least squares approximation, while the dashed line is a spline interpolation. Both models represent a valid hypothesis regarding the function that generated the data points, although none of them is likely to be the best, i.e., the one that will provide the lowest generalization error on new data. In this specific case, one of the two models is too simple (the solid line), while the other is too complex. The issue of selecting a good compromise between these cases will be the focus of section 4.2.4. The methods most often encountered in practice are multivariate linear regression, kriging, splines, generalized linear models [HAS 01], generalized additive models [HAS 90], neural networks such as multi-layer perceptron (MLP) and general regression neural network (GRNN), support vector regression (SVR) [SCH 98] and Gaussian processes [RAS 06]. MLP, GRNN and SVR will be explained in further detail in the following sections. A classification problem can be formulated the same way, but yi is a onedimensional vector of discrete values, e.g., yi = {-1,1,}. For example, from a satellite image, we could want to output, based on the reflectance of several spectral channels, if the region on the image is a forest, a desert, water or an urban area. This is a typical multiclass classification task. From a practical point of view, the objects

98


to be classified are represented using variables that are able to separate the objects correctly. For instance, even though they share the same shape and color, only one variable is necessary to discriminate between pumpkins and oranges: their diameter. The variables are often called features in this context. Figure 4.2 illustrates a twoclass and two-variable classification problem. Classification methods include linear discriminant analysis, probabilistic neural networks, decision trees and support vector machines (SVM).

Figure 4.2. Classification problem with two classes

In Figure 4.2 the solid line is a linear Bayes classifier, while the dashed line is a SVM with a Gaussian kernel. Even though the SVM achieves a better class separation, the linear classifier is sufficient for this problem and is a better choice for prediction in that case. Density estimation is a task that arises when we want to model the probability distribution underlying a certain phenomenon for visualization or clustering purposes, for instance. In this case, only the values ^x i ` are available. The most common way of dealing with such a problem is with so-called generative approaches [BIS 06]. These methods assume a functional form for the probability distribution, e.g., Gaussian, and then try to estimate the corresponding parameters. It is important to note that this functional form can be and usually is a combination of distributions. The most commonly used generative methods are Gaussian mixture


99

models, Bayesian networks and Markov random fields. The interested reader will find a good introduction to these methods in [BIS 04]. We have just mentioned that density estimation has often been achieved using generative methods. In fact, all machine learning methods are either generative or discriminative. The reader may find a thorough description of generative and discriminative models in [JEB 04]. Generative methods try to build a model of the whole joint probability distribution of the input and output sequence. Using Bayes’ rule, we know that

p x, y

p x y p y

[4.1]

As the above formula illustrates, this implies estimating both the conditional distribution of the inputs and the distribution of the outputs. Discriminative models, on the other hand, only try to model the conditional probability distribution of the outputs. In a two-class classification problem, we usually want to find whether or not this inequality is respected for every new point:

p y x ! 0.5

[4.2]

Unlike generative methods, it is impossible to generate new data from this type of model, since we do not have knowledge of the full joint distribution. However, predicting the output is usually the ultimate goal in regression or classification, and estimating the full distribution, which is a very difficult problem, is not useful to perform as an intermediate task. Note that p y x , the posterior (or predictive) probability of the output represents the probability of a data point belonging to class 1 (or -1, depending on the formulation of the problem). Most methods do not provide such a distribution, but instead only the most likely class label (1 or -1). Estimating the posterior distribution explicitly is slightly more difficult. We will come back to this point in section 4.2.6.

Most of the tasks we have mentioned rely on an inductive principle. From a finite set of locations, we build a general model able to predict the value of the function at any location. As underlined by Vapnik [VAP 95], we should not solve a problem that is more general than what is required. In many cases, we only need to know the values of that general model at particular locations. Traditionally, we solve a function estimation problem by first performing induction (constructing a general model from a finite dataset) and, subsequently, deduction (evaluation of the function at particular locations of interest). Transductive inference allows us to perform these tasks in only one step, from specific points to specific points. Figure 4.3 illustrates the different types of inference schemes. Transductive methods include nearest neighbor methods and GRNN. These methods have in common that they only use a

100


combination of data points in their vicinity in order to build a prediction. The output of a test point is usually a distance-based average of neighboring outputs. These methods are often referred to as lazy learners, as no training phase is necessary.

Figure 4.3. Illustration of inductive, deductive and transductive inference

4.2.2. Approaches to learning from data Apart from the task itself, the specificities of the dataset may influence the type of learning approach that will be used. We should first check what the nature of the data is, and more particularly: – whether the set of training points is labeled or not (with continuous or discrete values); – whether there are many missing labels or data. The answer to these questions will make the learning problem fall into one of these main categories: supervised learning, unsupervised learning and semisupervised learning. Supervised learning takes place when we are given a dataset ^x i , y i ` , where xi is a p-dimensional vector and yi is a one-dimensional vector of discrete (classification) or continuous (function approximation) values. We want to find a mapping from the input xi to the output yi. Thus, it is the type of learning most often encountered when dealing with classification or regression problems. The availability of the target vector yi suggests the use of a criterion measuring the discrepancy between yi and the output values predicted by the model that is being built in order to select the appropriate mapping.


101

Unsupervised learning deals with the case where the labels yi are not available. Only the data points xi are given, and the goal is thus to extract information regarding the distribution of the data and potential clusters of points. Unsupervised learning encompasses the general problems of clustering and density estimation, which are intimately related. Unsupervised learning usually requires a greater a priori knowledge about the problem. In fact, no output data is available to measure the adequacy of a given model. We must therefore rely on assumptions about the structure of the data. Semi-supervised learning is an intermediate situation between the two we have encountered so far. In this case, a small number of training samples is labeled, and most of the data is unlabeled. This is the case that occurs most often in real-life situations, as collecting data is relatively easy, but labeling it requires the manual annotation of the data, which can be prohibitively long, when feasible at all. In semisupervised learning, unlabeled data, which represents the unconditional distribution of the inputs, px , is used to bring information to the models obtained using the labeled data, which is p y x . Popular semi-supervised learning algorithms include expectation-maximization (EM) with mixtures of Gaussians and transductive support vector machines. Even though semi-supervised learning is traditionally associated with classification problems, recent techniques have emerged dealing with semi-supervised clustering and regression. 4.2.3. Feature selection It is often said that a well-posed problem is already half-solved. This is especially true in machine learning, where we can face a problem involving a large number of variables, very often redundant. More variables can mean more information, but also more noise in the data. A first example can be given by the coordinate system: the most obvious variables to consider in a spatial environmental context are the coordinates x, y or x, y, z of a data point or, equivalently, its latitude and longitude. Even if only spatial variables are considered, the origin of the coordinate system is arbitrary; a rotation of the system may decrease the correlation between x and y. A second example can be a situation where several environmental or meteorological variables are available and would bring very useful information: temperature, time, wind speed, etc. Selecting the most informative variables is essential in this case. These two cases illustrate the two major problems of feature extraction and feature selection. A good introduction to this important subject can be found in [GUY 03], and an exhaustive review of the field is presented in [GUY 06].

102


Feature selection can be defined as the task of selecting, among the available variables, the ones that are the most correlated with the output and uncorrelated between them. This is a classical problem in statistics and many methods have been popularized in order to deal with it. Forward and backward stepwise selection is a standard method implemented in most statistical packages, but it is suitable for rather small datasets as the procedure is computationally greedy. Mallows’ Cp [MAL 73], The Akaike Information Criterion (AIC) [AKA 74] and the Minimum Description Length (MDL) [RIS 78] are other popular statistical methods. Feature extraction aims at building features that are, by construction, uncorrelated or independent by combining existing features. The most well-known feature extraction method is Principal Component Analysis (PCA). PCA finds statistically uncorrelated features by building a linear combination of the initial features. We will come back to this technique in section 4.6. Figure 4.4 illustrates a case where performing feature extraction could be useful in a classification context.

Figure 4.4. Two-class classification problem. Two features are used, although only one is necessary to perfectly separate the data if we rotate the coordinate system

Using these features, both of them are necessary to separate the data. However, if we rotate the coordinate system by 45°, only one feature is necessary. This is easily achieved with PCA.


103

4.2.4. Model selection Selecting the right model is a difficult task which involves the determination of an appropriate functional form of a model and its parameters. Firstly, it is useful to distinguish between several classes of models which differ by their underlying assumptions about the data: parametric, non-parametric and semi-parametric models. Parametric models (sometimes called “global” approaches or “model-driven” approaches) make strong assumptions about the underlying phenomenon. In fact, a functional form is first assumed (linear, polynomial, exponential, etc.), and the learning task consists of estimating the parameters of the chosen model. The errors are usually assumed to be Gaussian-distributed. Classical multivariate regression and linear discriminant analysis are good examples of this type of approach. The very good interpretability of these models compensates for their strong (and often false) assumptions. Parametric models often require a very large number of data points, especially if the number of parameters is large. Non-parametric models (“local” approaches, “data-driven” approaches) make very few assumptions about the data. However, this advantage has the counterpart that the obtained model is not interpretable and must be used as a “black box”. This fact is the main reason why domain experts are usually very reluctant to use such approaches. Nonetheless, very often the context does not allow the use of parametric or physical models and this type of approach is the best choice. Classical examples of local approaches include nearest neighbor methods, probabilistic neural networks (PNN) and general regression neural networks (GRNN). Semi-parametric models have both parametric and non-parametric components. Kriging with external drift can be seen as a semiparametric model, as one component of the kriging estimator is model-driven (drift modeling) and another is data-driven (variography). Choosing between these types of models is an important task that will influence the reconstruction of the process. When a lot of prior knowledge is available, parametric models are often more interesting due to their interpretability. With very little prior knowledge, we should favor semi-parametric or non-parametric models, which rely on fewer assumptions. If possible, any available prior knowledge about the problem must be taken into account in order to make the choice of a general model (linear, nonlinear, polynomial, etc.) Once a type of model has been selected, a number of parameters must be adjusted. To this end, numerous methods are available. One of the most popular methods for model selection is the comparison of test errors. Data are randomly

104


divided into three parts: the training set, the test set and the validation set. The training set is used to build the model, by minimizing a given criterion. If we want to compare models, we then calculate the error obtained on the test set. The validation set is used in order to obtain an estimation of the generalization error, i.e., the error that can be expected on new data. It is important to never use data that has been used to train the models in order to estimate the generalization error, as the models would be biased favorably with respect to this data. Note that in the machine learning literature, the terms “test” and “validation” are used conversely. We have adopted the other convention to follow geostatistical terminology. Very often, the empirical error is minimized on the training set. However, if we reach a global minimum on the empirical error, it is likely that the model will be too adjusted to the training data, which means that predictions on unknown data are likely to be erroneous. This situation is called overfitting. If, on the contrary, the model is not optimized enough, or the class of functions chosen is too simple, it is likely that the data will not be well represented by the model: this situation is called underfitting. In Figure 4.1, these two situations are well represented. The function plotted with the solid line is not complex enough to capture the variation in the dataset. The opposite situation occurs with the model represented by the dashed line: the function passes through every data point. In this case, the empirical error on the training set is zero, but the generalization error is likely to be high, as the noise in the training set has also been fitted. Figure 4.5 illustrates this principle.

Figure 4.5. Evolution of the training and generalization error as the model complexity increases


105

This problem is also known as the bias-variance dilemma. The total variance of a predictor can always be decomposed between these two terms. A model that is too simple exhibits low variance: training the model with another random sample coming from the same distribution would not modify the model greatly. However, it has a high bias, i.e., the model is likely to be far from the “true” hypothesis. Conversely, a model that is too complex has a low bias, i.e., it represents the data well. However, it has a very high variance in the sense that a different random sample would provide a radically different model.

Figure 4.6. Evolution of the bias, variance and generalization error

Very often, we are confronted by a problem where the dataset is not large enough to be able to split it into training, test and validation sets. In fact, if the training dataset is too small, we may not capture all the information contained in the data. To get round this problem, techniques based on statistical resampling – more specifically jackknife methods – are commonly used. Jackknife estimates the standard error on a statistic by systematically recalculating the statistical estimate leaving out one observation at a time from the sample. The most popular jackknife-based method is cross-validation (CV). CV is extremely useful when we do not want to “waste” the information brought by a large part of the dataset in order to estimate the test error by re-using the training data to do so. Cross-validation is usually performed using either of the following two schemes: – leave-one-out cross-validation (LOOCV); – k-fold cross-validation (KFCV).

106


LOOCV can be summarized with the following procedure: 1) remove one point x i , y i from the dataset; 2) train the model yˆ with the remaining points; 3) calculate the error on the removed point, e.g., ei

y yˆ x i

i

2

;

4) repeat 1 to 3 for every point in the dataset; 5) the average error

1 n ¦ ei is an estimation of the test error. n i1

LOOCV is a very simple scheme, but has the drawback of being very computationally greedy – as many models as there are data points have to be trained. KFCV is a more efficient method to estimate the test error. Rather than withdrawing one point at a time from the dataset, it works by randomly splitting the dataset into equal-sized partitions, which are sequentially removed. It can be summarized as follows: 1) split the dataset randomly into k partitions of size n; 2) remove the first partition from the dataset; 3) train the model yˆ i on the remaining data; 4) calculate the error on the removed points, i.e.,

ei

1 n ¦ yi yˆi ; ni1

5) repeat 2 to 4 for all the k partitions; 6) the test error is 1 k

k

¦

e j.

j 1

One parameter has to be tuned: k, the number of partitions. If the number of partitions is equal to the size of the dataset, we come back to LOOCV. When dealing with time series, a similar method to cross-validation is very often used: sequential validation. The model is trained using the data available at time t, and the data coming in at time t+1 is used to estimate the error on the current model. At each step, the whole dataset is used for training and the error is estimated on “fresh” data. This is a method of continuously improving the models.


107

4.2.5. Dealing with uncertainties As mentioned in section 4.2.1, most machine learning algorithms only output a class label or a prediction. However, in many applications, it is also very important to estimate the posterior probability distribution p y x , which gives a measure of how confident we are in a prediction.

In classical pattern recognition problems such as handwritten digit classification, predictive uncertainty can be seen as side-information. However, in environmental, medical or safety applications, the importance of predictive uncertainty is dramatically increased. A meteorologist who predicts that “there is a 99% probability of having a tsunami tomorrow” makes a completely different prediction from another one who states that “there is a 60% probability of having a tsunami tomorrow”. However, a binary classifier would in both cases state that “there will be a tsunami tomorrow”. Many methods exist to deal with this problem, and it is still a hot topic of research (see for example [QUI 05]). For classification, the most straightforward (and somewhat heuristic) method is to map the results of a binary classifier on a sigmoid function. The closer the point is to the decision boundary, the more uncertain it is. Figure 4.7 illustrates that principle.

Figure 4.7. Mapping of a binary classifier output (left) on a sigmoid (right) in order to obtain probabilistic outputs. The flatness of the sigmoid has to be tuned

In the case of support vector machines (section 4.5.3), this methodology has been applied in [PLA 99]. Other methods exist based on data resampling. One of the most popular of these is bagging (bootstrap aggregation) [BRE 94]. For a given prediction that we want to output, we train N models, each of which is trained using a bootstrap sample (draw

108


with replacement) of the data. The average prediction of the N bagged predictors is the most confident prediction, and its variance reflects the uncertainty at that location. 4.3. Nearest neighbor methods The simplest classification and regression algorithms are perhaps the k-nearest neighbors (KNN) methods. KNN methods predict the value of a new point by using simple combinations of training points that lie in its vicinity. Let x be a test point and y* its output value we want to predict. We define N k x as the neighborhood

of size k of x. This neighborhood is defined as the k points xi closest to x, which have outputs yi. Finding this neighborhood implies calculating the distance from x to all points in the dataset, and selecting the k closest points. This distance is usually Euclidean, but other distance measures can be used, or the Euclidean distance can be weighted by a decreasing function. The output can simply be expressed as

y*

1 ¦ yi k xi N k x

[4.3]

The application to regression is direct. In the case of classification, the yi are discrete. The average is thus equivalent to a majority vote. Figure 4.8 plots the decision boundaries generated with a 1-nearest neighbor algorithm and a 20-nearest neighbor.

Figure 4.8. Application of k-nearest neighbors to a two-class classification problem. The decision boundaries for k=1 and k=20 are shown


109

It is clear that increasing the value of k provides a smoother decision boundary. The limit case of k=1 generally overfits the data. The other limit case, k=N, where N is the size of the dataset, outputs the same decision for any test point. As mentioned earlier, the distance can be weighted by a decreasing function of distance. This is called kernel smoothing, which includes general regression neural networks. We will come back to this topic in section 4.4. It is worth mentioning that local methods such as KNN very often perform surprisingly well in low dimensions. However, when dealing with problems involving a large number of variables, these methods may not be reliable. In fact, the more variables are included in a model, the more the data become sparse in the variable space, and the neighbors of a point become far apart. This problem is known as the curse of dimensionality in the machine learning community. K-nearest neighbor methods are considered in detail along with real case studies in Chapter 5. 4.4. Artificial neural network algorithms In this section we will describe several artificial neural network (ANN) models and illustrate them with simple examples. Among the most frequently used ANNs are multi-layer perceptron (MLP) and kernel based neural networks [HAY 98]. General Regression Neural Networks (GRNN) and Probabilistic Neural Networks (PNN) belong to the family of kernel neural networks. Common features of these ANN are that they are feed forward neural networks and learn with supervision (with known expected outputs). Feed forward neural networks propagate information from the input to the output without recurrence and the error flow is propagated backwards from output to the input modifying the ANN parameters according to the captured dependencies from data. More comprehensive real case studies using these models will be presented in Chapter 5. An ANN of a totally different type that is also described in this section – Self-Organizing Map (SOM) – is used in classification problems and is based on unsupervised learning. 4.4.1. Multi-layer perceptron neural network MLP was developed in the 1960s as an algorithm that mimics the signal propagation process in human neurons. MLP consists of basic generic elements – neurons – which are mathematical analogs of brain neurons. An artificial neuron is capable of propagating the data and modifying itself accordingly whilst training.

110


MLP is a later development of a simpler single layer perceptron, invented in 1957 [ROS 57]. A single layer perceptron solves a simple regression problem evaluating the output y from the input vector X=(x1, …,xi,...,xn):

y*

¦w x i

i

b

[4.4]

i

or

y*

1

¦ w 1 exp( x ) b i

i

[4.5]

i

where wi are the weights of the connections coming to the neuron and b is the bias. These hyper parameters are determined through the training procedure by minimizing the error between the target variable data y and the perceptron prediction output y* using conventional optimization algorithms (e.g. gradient descent, etc.).

Figure 4.9. Sigmoid function used as a nonlinear element in a perceptron

A single layer perceptron (see Figure 4.10) is capable of solving linearly separable binary classification (with equation [4.4]) or regression (with equation [4.5]) using the sigmoid function, see Figure 4.9) problems [MIN 69]. The latter case is identical to logistic regression. The perceptron weights wi are updated through the learning procedure when the input data x are presented to input and the corresponding output y* is compared with the expected output y: wc = w+Įw(y-y*)wx, where Į corresponds to the learning rate

[4.6]


111

Figure 4.10. Single layer perceptron

MLP is an extension of the single layer perceptron based on the addition of a hidden layer of neurons between the input and the output neurons (see Figure 4.11). MLP with a nonlinear element (sigmoid or of another type) in each hidden neuron is a universal approximator capable of estimating every continuous function with just one hidden layer. MLP estimation with a single hidden layer with m neurons is calculated as a weighted sum:

f m ( x, w, v)

m

¦w

i

s ( X vi ) wo

[4.7]

i 1

where wi are the weights corresponding to each neuron connection and w0 is an additive bias corresponding to the entire hidden layer; s(·) is a sigmoid activation function, which represents a nonlinear element in MLP; Ȟi is a gain – activation function steepness parameter.

Figure 4.11. MLP structure [3-6-1]

112


The MLP weights wi are obtained through the training procedure [HAY 98]. MLP uses a back propagation learning algorithm where the weights are consequently modified according to the backward error propagation from the output to the input. Thus, the error is propagated in the opposite direction to the data flow from the input to the output. At the beginning, the weights are initialized at random. Then, the data are consequently presented to the inputs. The discrepancy between the MLP output and the expected output available from the data produces the error, which has to be minimized using conventional iterative optimization techniques. Error minimization between the expected output and the MLP output can be performed using a wide selection of known optimization algorithms. Optimization algorithms can be divided into two groups: gradient and stochastic. Gradient optimization algorithms are based on calculating the gradient of the minimized function and are very good in finding local minima. Gradient optimization algorithms vary in performance efficiency and speed. Among first order slower methods there are: conjugate gradients, steepest descent, Newton’s methods. Faster algorithms such as The Levenberg-Marquardt algorithm and quasi-Newton methods use 2nd order derivatives or their approximations [NOC, 1999]. However, gradient methods do not achieve the global minimum. Very often gradient optimization gets stuck in local minimum. Stochastic optimization methods have the ability to jump out of local minima due to their stochastic nature. Thus, they have a better chance of seeking the global minimum, though they tend to converge slower than the gradient methods. Among stochastic optimization methods there are: simulated annealing, genetic algorithms, bootstrap, Hamiltonian Monte-Carlo, particle swarm. In practice, a combination of stochastic and fast gradient methods provide a good result in MLP training. First, stochastic methods seek globally an appropriate starting point for the following gradient search. This allows us to avoid entrapment in multiple local minima closest to the initialization point. Then, gradient optimization is used to improve the minima found by the stochastic search. The efficiency of MLP training largely depends on the careful selection of data used to tune the model parameters (the MLP weights). Usually, all the data are split into three parts of different size: training set, test set and validation set. The training data is used directly: the data are sequentially presented to the MLP input, propagated through the network and the output obtained is finally compared with the expected output from the training data set. The training error calculated as the mean squared difference between the MLP output and the expected output is then minimized as described above through repeated propagation of the training data to the network. The test data are also propagated to the network in the same way as the training data and the test error is calculated accordingly. The principle difference of the test set is that the test error is not minimized and thus not propagated backwards from the MLP outputs to the input. The test error is calculated and compared for every training iteration for quality control purposes. Thus, both the training and the


113

test data are used to choose and train the estimation model. The validation data are usually the ones initially hidden – not used at all or retained by the customer. The validation data are used as an independent data set to validate predictions of the chosen MLP configuration. Once the choice of the validation set is beyond the modeler’s control, careful selection of the data for training and testing becomes crucial. Both training and test data sets should represent the original distribution. Thus, a simple random sample may not be adequate in the case where the data are clustered in high dimensions. More advanced sampling techniques from high dimensional space with declustering are used. It is important that both training and test data include the outliers and extreme values. In practice, the test set is smaller than the training set especially when the total amount of data is limited. The size of the test set can range between 10-25% of the total amount of data available. In case of large amounts of available data this ratio may be higher. During training it is crucial to identify the optimal number of iterations of the optimization algorithms – the number of weight updates. The decision on when the training should be terminated is made based on the profiles of the training and testing errors calculated during iterations (see Figure 4.12). The training error can be minimized until it steadily approaches 0 with the increasing number of iterations. Zero training error means exact reproduction of training data by MLP predictions, which is called over-fitting. An over-fitted MLP is able to reproduce only the data selected for training and lacks the generalization ability to predict data other than the training data. An increase of the test error is evidence of MLP over-fitting. Thus, the optimal MLP is chosen according to the minimum test error.

Figure 4.12. Error minimization whist training: training and testing errors

114


Let us illustrate how MLP works with a “toy” regression problem of predicting a 1D function. An analytical function was chosen for the synthetic example:

y

sin(0.4 x) 0.01x sin(2 x) x

[4.8]

The chosen function features a semi-periodic structure with a nonlinear trend and multiple local minima in argument range [0; 20] (see Figure 4.13). Random Gaussian noise (with a zero mean and variance ı2=0.07) was added to the function to generate the data for modeling. An MLP interpolation model is used to reconstruct the target function pattern based on a limited number of data. 20 randomly sampled items of data were generated to be used for the MLP training.

Figure 4.13. Theoretical function and 20 randomly sampled items of data corrupted with noise

Different MLP structures can be used for interpolation. They vary by the number of hidden layers and neurons and also by the type of optimization algorithm used for training. A single hidden layer is sufficient to model quite complex patterns in a simple 1D problem. In general, MLP with a single hidden layer models nonlinearity using the activation function in its neurons. Thus, MLP is able to model nonlinear patterns even in high dimensions.


115

The number of neurons in the hidden layer characterizes the MLP’s capability of modeling multiple local features. Generally speaking, the more neurons there are in the hidden layer, the more degrees of freedom MLP possesses for modeling complexity. The number of hidden neurons would also depend on the amount of data available for training. On the one hand, MLP with many neurons would not be able to train with too few data. On the other hand, MLP with too few hidden neurons would not be able to capture complex dependencies represented by a large amount of data. For instance, a linear pattern is determined by just two degrees of freedom, thus it can be modeled with a single neuron perceptron described above with no hidden layers, which has just two parameters – the weight and the bias. A quadratic polynomial function with 3 or more degrees of freedom requires a nonlinear element – at least a single hidden neuron – to model it. In practice, we do not know the shape of the considered pattern beforehand, although we may assume its level of complexity: multiple scale, presence of local peaks and dips, etc. In the case of such a complex data pattern, the MLP has to include a fairly large number of neurons, and, thus, a large a number of data points for training is required to represent the pattern correctly. In practice, the ratio between the amount of data and the number of MLP connections (which represent the degrees of freedom) should be much larger than 1. In case the ratio is 1 the amount of data equals the number of MLP weights (excluding the bias for each layer). In such a situation MLP would not be able to learn the pattern and each weight would just represent one item of the data. If the ratio between the amount of data and the number of MLP weights is close to 2, then the MLP has a good chance of learning a pattern from data. A single hidden layer MLP with fewer degrees of freedom may be incapable of learning too complex patterns. The MLP structure with two hidden layers is usually used for interpolation of high dimensional problems with multiple inputs and outputs. It can be interpreted that the neurons in one hidden layer reflect the valleys of the manifold where the solution lies, while the neurons in the other hidden layer characterize smaller scale local variations within the valleys. No more than two hidden layers is necessary to analyze even very sophisticated data. In addition, too many neurons in a single layer leads to significant computation times for training that can be reduced by rearranging the neurons into two hidden layers. Figure 4.14 shows the performance of MLP with a different number of hidden neurons in a single layer – 3, 5, 10 and 20. MLP with a small number of neurons are able to capture some of the pattern trends given just 20 items of data for training. The MLP with just 3 hidden neurons provides the smallest amount of detail, while adding 2 hidden neurons allows us to model additional small scale peaks. MLP with a larger number of hidden layers (10 and 20) are able to reproduce most of the training data exactly, which leads to over-fitting. As stated above, over-fitting means that the model predicts very well on training data and looses its ability to generalize on testing and validation datasets. The evidence of over-fitting is also shown in Table 4.1, where the training and validation errors are presented. Validation was

116


performed using 800 values of the true function within the interpolation region. It is clearly seen that the validation error increases with the number of hidden neurons if no test data is used to control the training (see Figure 4.15 (left)). MLP prediction in the extrapolation region on the edge of the data region can vary significantly leading to high validation errors.

Figure 4.14. MLP trained without a test data set, predictions obtained with a different number of neurons in the hidden layer: 3, 5, 10 and 20

The problem of over-fitting can be overcome by using a separate set of test data to control the learning process and stop the training before the MLP loses its ability to generalize. The test data are presented to the MLP to calculate the corresponding MLP output, but the mismatch between the output and the target data is not propagated back though the MLP and, thus, does not influence the weight optimization. The mismatch between the test data and the corresponding MLP output is called the test error. The minimum of the test error corresponds to the optimally trained MLP, which is able to accurately predict the data that is different from the training. We have selected 5 out of the 20 items of initial training data for testing purposes, leaving the remaining 15 items of data for training with the test set


117

control. The MLP prediction result for a different number of hidden neurons is presented in Figure 4.16. Figure 4.15 (right) shows dependence of the training and the validation error from the number of hidden neurons for the MLP trained with the test data. Use of the test data does not impact much on MLP with 3 and 5 hidden neurons; they do not suffer from over-fitting. However, the performance of the MLP with 10 and 20 hidden layers is significantly improved with the use of the test data. Errors summarized in Table 4.1 show that the validation error is similar for all MLP configurations when the test data are used. The training error and the minimum of the validation error suggests that the MLP with 5 hidden neurons provides the best prediction for this particular problem. Hidden neurons

Use of test data

Training error

Test error

Validation error

3

No

0.063

-

0.091

3

Yes

0.072

0.031

0.093

5

No

0.030

-

0.166

5

Yes

0.006

0.022

0.085

10

No

0.016

-

0.168

10

Yes

0.075

0.033

0.094

20

No

0.015

-

0.204

20

Yes

0.073

0.037

0.088

Table 4.1. Comparison of MLP interpolation with different number of hidden neurons without using the test data in training and with the test data to control the training and avoid over-fitting

The selection of training algorithm has a significant impact on MLP prediction. Careful choice of the training algorithms allows us to improve the MLP prediction quality. Usually, a combination of stochastic and 2nd order gradient (e.g. LevenbergMarquardt) algorithms provides the best results. However, the training results may be sensitive to the initialization of the weights and the starting point of the gradient optimization. Therefore, multiple results of the gradient optimization can be considered in order to reach the global minimum (or the lower local minimum) rather than one of multiple local minima.

118


Figure 4.15. Training and validation error for MLP with varied number of hidden neurons: trained using only 20 data for training (left), trained using 15 training data and 5 data for testing (right)

Figure 4.16. MLP trained with a test data set, predictions obtained with a different number of neurons in the hidden layer: 3, 5, 10 and 20


119

4.4.2. General Regression Neural Networks Another representative of the kernel-based method for the regression task is a General Regression Neural Network (GRNN). A GRNN is another name for a wellknown statistical non-parametric method called the Nadaraya-Watson Kernel Regression Estimator. It was proposed independently by Nadaraya and Watson in 1964 [NAD 64; WAT 64]. In 1991 it was interpreted by Specht in terms of neural networks [SPE 91]. This method is based on kernel density estimation using the Parzen method [FAN 97; HAR 89]. Omitting the details of mathematical background, let us present the final formula for the regression estimation of using available measurements Zi:

§ x xi · V ¸¹ i 1 N § x xi · K¨ ¦ ¸ © V ¹ i 1

Z(x)

N

¦ Z K ¨© i

Z (x)

i 1, 2! , N

[4.9]

where N is a number of training points and Zi is a function value of the i-th training point with coordinate xi. To simplify the understanding of the GRNN estimations, the normalized weighting function as a function of x can be defined as

Wi ( x )

§ x xi · K¨ ¸ © V ¹ N § x xj · K¨ ¦ ¸ j 1 © V ¹

i 1, 2! , N

[4.10]

The denominator of [4.10] gives us the normalization property N

¦W ( x) i

1 x

[4.11]

i 1

Now we can rewrite equation [4.9] in a simplified form as N

Z ( x)

¦W ( x)Z i

i 1

i

[4.12]

120


In this form, equation [4.12] describes the prediction at point average of the Zi observations for all N training points.

x as a weighted

The core of this method is a kernel K(·). It depends on two parameters: the distance to the predicted point and parameter ı. ı is a positive number called the bandwidth or simply the width of the kernel. Note that xi, in fact, is the center of the i-th kernel. Generally different types of kernels can be used, but the Gaussian kernel is usually chosen.

§ x xi · K¨ ¸ © V ¹

1 (2SV )

2 p/2

§ x xi exp ¨ ¨ 2V 2 ©

2

· ¸ ¸ ¹

i 1, 2! , N

[4.13]

where p is the number of dimensions of input vector x. Finally, the GRNN’s estimation formula with a Gaussian kernel and without a normalization term is § x xi 2 · Z exp ¨ ¸ ¦ i ¨ 2V 2 ¸¹ i 1 © N § x xi 2 · exp ¨ ¸ ¦ ¨ 2V 2 ¸¹ i 1 © N

Z ( x)

[4.14]

Note that in fact GRNN, according to [4.14], is a linear estimator (prediction depends linearly on weights), but also that the weights are estimated nonlinearly according to nonlinear kernel [4.13]. The model described above is the simplest GRNN algorithm. One of the useful improvements is to use multidimensional kernels instead of one-dimensional kernels as in [4.13]. When the ı² parameter is a scalar, we are dealing with an isotropic model. In a more general case, parameter ı² may be presented as a covariance matrix Ȉ. A covariance matrix is a squared symmetric matrix with dimension p by p and with the number of parameters equal to p(p+1)/2. In the general anisotropic case, [4.13] can be rewritten as

§ x xi · K¨ ¸ © V ¹

(2S )

p/2

1 § 1 · exp ¨ ( x xi )T 6 j 1 ( x xi ) ¸ 1/ 2 (det 6 j ) © 2 ¹

where det means determinant and Ȉ-1 is the inverse of the Ȉ matrix.

[4.15]


121

Model [4.15] is anisotropic and is much more flexible for modeling data. Such models can be very useful in the case of complex multidimensional data. For example, for 2D spatial mapping we can use 2D parameter V=(Vx,Vy,Vxy). Usually, only the diagonal of the Ȉ matrix is used. In this case the number of adaptive (free) V values equals the number of dimensions p: V = (V1,…,Vp). The only adaptive (free) parameter in the GRNN model with a Gaussian kernel is

V (iso- or anisotropic), the width of the kernel.

In order to demonstrate how GRNN works, let us consider a simple “toy” one-dimensional problem (Figure 4.17). A simple sine function represents a true function of an underlying structure of a collection of sample data. In order to generate a training set, this sine function is sampled at a wide variety of points, and random noise is added. This true function and training samples are shown in Figure 4.17a. Now let us examine the effect produced by the variety of V values presented, the smoothing parameter. Figure 4.17b shows what happens when a very small value of V is used. The GRNN follows the training data closely, almost moving from point to point. If the data is known to be clear (without noise), the GRNN makes an excellent interpolation algorithm, analogous to the nearest neighbor method. However, this result will be acceptable only if the density of the training data is high enough. In other cases, an “overfitting” effect, which is well-known in neural networks, may appear and such solutions will not be optimal. Thus, since in most cases the data are distorted by noise, straightforward interpolation is not an acceptable option.

a)

b)

c)

d)

Figure 4.17. A simple “toy” problem to illustrate the influence of the parameter V on the GRNN result: a) true function and noised training samples; b) too small V; c) a perfect V; d) too large V

122


A larger smoothing parameter V gives the result shown in Figure 4.17c, which is almost ideal. Figure 4.17d illustrates the effect of a smoothing parameter that is too large. The global structure of the training set has been completely missed by the algorithm, leading to an oversmoothing situation. Thus, we can come to the conclusion that choosing the value of smoothing parameter V is a vital problem for the GRNN model and that such a choice is data dependent. For the estimation of V, the cross-validation procedures discussed in section 4.2.5 may be implemented. Usually, in order to find the optimal value of bandwidth a grid search is used. It is therefore necessary to define an interval of V values [Vlow, Vhigh] and is the number of steps M. Then, the validation is repeated for all the M V-values

V i V low (i 1)

V high V low M

i 1,..., M

[4.16]

The final result (optimal V value) corresponds to the model with the smallest cross-validation error. The interval and the number of steps have to be consistent in order to catch the expected optimal (with minimum of the error) value. Reliable limits are the minimum distance between points and size of area under study. In fact, the real interval is much smaller and can be defined in accordance with the monitoring dataset features and/or prior expert knowledge about studied phenomenon. 4.4.3. Probabilistic Neural Networks

The GRNN model described above is a typical kernel-based model used for regression tasks. A similar model for classification problems is the Probabilistic Neural Network (PNN). Like GRNN, it was developed by Specht in terms of neural networks in 1990 [SPE 90]. It uses the same mathematical background for the density estimation. However, in the case of classification, the estimated conditional densities for all classes are used to label a point to one of the classes. The whole data set is divided into subsets according to the class membership. Thus, the probability density functions for each class are estimated independently.


123

Finally, a Bayesian optimal or maximum a posteriori (MAP) decision rule is used to make a decision

C ( x) {c1 , c2 ,..., cK } argmax P (ci ) p ( x | ci )

i 1, 2,..., K

[4.17]

ci

where K is the number of classes (or generators of random variables), ci (i=1,2,..,K), P(ci) is a prior probability and p(x|ci) is a distribution (for all input space x). Prior probability can be interpreted as the initial (guess) class conditional distribution p(x|ci) for all x. Generally, the prior class distribution is highly dependent on the specific task and should be determined by an additional (physical, expert etc.) knowledge of the problem. In fact, a PNN model can make a prediction using these prior probabilities even without measurement! However, really in most cases none of this additional information is available. In these cases, all P(ci) are assumed to be equal (P(c1)=P(c2)=…=P(ck)). The conditional distribution is defined by the following formula:

p( x | ci )

2S V 2

§ x x(n) i exp ¨ ¦ 2 ¨ 2V Ni n 1 © Ni

1 p

2

2

· ¸ ¸ ¹

[4.18]

(n)

where Ni is the number of samples (or class size) that belong to class ci. xi represents the n-th sample of class ci.

The difference between [4.13] and [4.18], as mentioned above, is only in the data used for the estimation. In the case of regression, all N points are used, and here only points belonging to specified class ci are used. In order to make prediction [4.17], PNN just compares the values obtained with [4.18] for different classes (taking into account prior probability P(ci)) and attributes the class membership related to the maximum value. An important and very useful characteristic of any model using a Bayesian framework is the possibility of obtaining confidence measure of the prediction. This means that PNN can not only label a point with some of the classes, but also produces a probability of it belonging to all of them. These probabilities are called posterior – final, after measurements and calculations (in comparison to prior – initial, before measurements and calculations).

124


The Bayesian confidence (a posteriori probability of x belonging to class ci) is defined by

P ( x | ci )

P(ci ) p( x | ci ) K

¦ P(ci ) p( x | ck )

[4.19]

k 1

The above discussion about cross validation showed that training of GRNN is very simple, because only parameter V is to be optimized. The same cross-validation procedure can be applied for PNN training/tuning as well. It is only necessary to modify the target/error function. In general, continuous error function can be used for the minimization despite the fact that the classification error is generally a discrete value (number of misclassified points). This can be performed due to Bayes posterior probabilities [4.19] which are continuous functions within the limits [0; 1]. Therefore, a continuous error function for the V optimization procedure can be defined as follows

e( x | ci )

>1 P ( x | ci )@

2

[4.20]

GRNN and PNN are two models which are quite efficient for automatic mapping for regression and classification tasks (see Chapter 5 for case studies). Now, let us consider self-organizing (Kohonen) maps, that are neural networks based on unsupervised learning algorithms. 4.4.4. Self-organizing (Kohonen) maps

A self-organizing map (SOM) is a type of artificial neural network. SOM is a powerful tool when dealing with highly multivariable problems. SOM was developed by T. Kohonen. Since then SOM has been successfully applied in many areas of interest, such as finance, medicine, robotics, (speech, image, signal) pattern recognition, classifications for physical and chemical systems and many others. A huge list of publications (containing more than 3,000 works) on SOM theory and applications can be found online at http://neuron-ai.tuke.sk/NCS/VOL1/P4_html/node35 html.


125

The characteristic features of SOM are: one hidden layer of neurons organized in a lattice structure and unsupervised competitive learning. As mentioned above, in unsupervised learning there are no examples with known answers to be learnt by the network, not even for a data subset. The goal of such a learning procedure is to organize training data into clusters according to their similarity and correlation criteria. That is why unsupervised learning can be referred to as self-organizing learning. Successful performance of unsupervised learning requires redundancy in training data. Once the network has been tuned to the statistical regularities of the training data, it develops the ability to specify encoding features of the data and thereby create new classes automatically. The learning procedure (as for any other neural approaches) is carried out by the modification of weights (wij) assigned to links between neurons (i and j):

w ij ( n 1 )

w ij ( n ) 'w ij ( n ) ,

[4.21]

where n is the number of iterations of the learning procedure. Modification of the weights follows some rules, such as, for example, Hebbian or competitive rules. The Hebbian learning is a two-phase rule [KOH 00, HAY 98]: – if two neurons on both sides of a synapse (connection) are activated simultaneously (synchronously), then the strength of that synapse is selectively increased; – if two neurons on both sides of a synapse are activated asynchronously, then that synapse is selectively weakened or eliminated. A direct Hebbian rule for modification at iteration step n of a link connecting neuron producing output xi and neuron obtaining input xi and simultaneously producing output yj is

'w ij ( n ) Kx i ( n ) y j ( n ) ,

[4.22]

where K is the learning rate. The problem with a direct Hebbian learning rule is the risk of uncontrollable increase of the value of the weights. There are different mathematical tricks to overcome this problem; one of them is the so-called Oja learning rule [OJA 82]

126

Advanced Mapping of Environmental Data p · § 'w ij ( n ) Ky j ( n )¨¨ x i ( n ) ¦ yk ( n )w ik ( n )¸¸ , k 1 ¹ ©

[4.23]

where p is the number of neurons in the layer. A one-layer ANN with linear neurons learned by the Oja learning rule performs principal component analysis. Oja’s network without linear constraint on neurons produces nonlinear component analysis and is useful for data compression problems. It is also called a “bottleneck network”. The self-organization rule comes from ideas based on Shannon’s information theory [KOH 00]. Its basis can be a Linsker principle of maximum mutual information (or Informax), which states that the synaptic links of the multilayered neural network should be organized so as to maximize the amount of the information that is preserved when signals are transformed passing layers of the network. In other words, mutual signal input and signal output information at each network’s layer should be maximized. Development in this direction has led to independent component analysis [HYV 01]. Such networks are useful for the separation of a signal from noise for all types of signal recognition systems. The simplest and the easiest rule is competitive learning. The output neurons of the network compete to be activated. The competitive comparison between neurons is based on some measure of similarity to a given input. Only one neuron is the winner for each input (winner-take-all competition). The learning process consists of the modification of the winner’s weight (wi) in order to make it more similar to the input pattern:

'w i ( n ) K x( n ) w i ( n )

[4.24]


127

Figure 4.18. One step of competitive learning procedure

Illustration of one step of the competitive learning is presented in Figure 4.18. An input sample is presented to a neural network. The node most similar to the input is selected according to a certain metric. Once the winner has been selected, it is modified to become more similar to the input. The new set of nodes contains the modified winner and all the other nodes. The learning procedure is applied to every input. This algorithm is useful for clustering the data according to internal features. Nonetheless it suffers from non-converging oscillations of the algorithm and the optimal number of neurons is unknown. Too few neurons decrease the quality of separation within clusters, while too many neurons can produce dead (not learned) neurons with unpredictable consequences. Competition learning of an SOM differs from the standard algorithm by a modification step – not only is the winner’s neuron modified, but also its neighbors. The procedure looks like the stretching of an elastic net over the data. It prevents dead neurons, and winning neighbors are likely to be winners for data that are not in the training dataset. The vectors assigned to the neighbors in SOM are neighbors in the data space as well. Therefore, we obtain not only the input data set quantization but regulating the input data set into the map (structure of neurons).

128


Now, let us give a theoretical presentation of SOM. The hidden layer of SOM is organized as an array (M) in the lattice structure. The type of the array can be different, but rectangular or hexagonal structures are the most common. Usually, the array is 1 or 2-dimensional. Each neuron (riM) possesses a vector of weights (mi = [Pi1, Pi2,..., Pin]T, mi Rn) also called a reference vector. Its dimension (n) is equal to the dimension of the input data space. There are no links issuing signals between neurons, but they have some knowledge on the vicinity through the neighborhood. The reply of the net to the presentation of an input vector x (x Rn) is given by the winner node cM, the closest one to the input according to the accepted metric (|x-mc|) - c=argmini{|x-mi|}. More often, the Euclidean distance is used. The training starts with the initialization of reference vectors. Usually, initial weights are defined as random values in the range of corresponding coordinates of the input dataset. During the training, the vector mi referencing to the node ri changes its values according to the input vectors. The modification on input x(t) which is presented to the net in iteration t of the learning process is defined by the following formula:

m i ( t 1 ) m i ( t ) K( t ) hci ( t )>x( t ) m i ( t )@

[4.25]

where K(t) is the learning rate (0

Soft data also obtained from model predictions. Illustration shows mean estimate along with a confidence interval for a point prediction. General knowledge reconciled with site-specific data. In special case of homogenous/stationary mean and covariance with hard (monitoring) data, the BME reduces to ordinary kriging.

Time ---->

Integration of model results (soft data) with hard data. Uncertainty is reduced and greater resolution is obtained.

Water Quality Parameter

F

Time ---->

G Integration of uncertain monitoring data with hard data and model predictions. Uncertainty is reduced near soft data space-time locations.


Integration Stage (3 steps are shown independently. In fact, BME performs integration simultaneously)

Soft data identified for space/time locations where hard data is absent. Illustration shows three measurements with uncertainty expressed as probability distribution.


C


Time ---->

Time ---->

Figure 6.28. An illustration of the BME stages implemented in the water quality study of the Catawba River reservoir system in western North Carolina [LOB 07]

294


– The study of multi-scale data features and their effects on the estimation of space-time mortality distributions in California, USA [CHO 03]. Artificial effects were filtered out using soft information at the data points themselves. The generated BME map displayed more variability at the local scale, and the contours lines of mortality rate followed more the outline of county boundaries (for which the information was collected), rather than the centroid locations (which were arbitrary choices). Accuracy measures demonstrated that the multiscale approach offered more accurate mortality predictions at the local scale than existing approaches that did not account for scale effects. – The characterization of the geographical dependence contaminant exposure in south-west Russia due to the Chernobyl fallout (Ukraine). The extent and magnitude of radioactive soil contamination by 137Cs was estimated which allowed incorporation of a variety of knowledge bases leading to improved prediction accuracy and informative soil contamination maps; see [SAV 05, PAR 05]. – The determination of the space-time extent of lead contamination at the Cherry Point Air Force site (North Carolina) and the corresponding health impact to nearby communities; see [AUG 02]. A composite lead dataset spread out over 14 years of sampling was used. The study analyzed the neurological impairment (depression in arithmetic ability in children) and lung cancer effects due to lead. It aimed at developing a general exposure and health effect assessment framework that is flexible enough to consider other contaminants of concern at Superfund sites. The study framework included demographic information and generated estimates of the population impact due to contamination exposure. – The study of causal associations between environmental exposure and health effects in the state of North Carolina, by synthesizing sources of physical exposure and population health knowledge [CHR 00c]. The strength and consistency of the exposure-effect association were evaluated on the basis of health effect predictions that the combined physical-health analysis generated in space-time. Potential confounders were accounted for in the quantitative analysis, which resulted only in a slightly different strength in the reported mortality-temperature association. – The analysis and mapping of syphilis distributions in Baltimore (USA) with the purpose of optimizing intervention and prevention strategies; see [LAW 06]. Covariance plots indicated that the distribution of the density of syphilis cases exhibited both spatial and temporal dependence. Disease maps suggested that syphilis increased within two geographic core areas of infection and spread outwards; see Figure 6.29. A new core area of infection was established to the northwest. As the outbreak waned, the density diminished and receded in all core areas. Morbidity remained elevated in the two original central and new northwestern core areas after the outbreak.

Bayesian Maximum Entropy – BME

295

Figure 6.29. Yearly changes (1994–2002) in the spatial distribution of syphilis infection density in Baltimore (Maryland, USA). The composite space-time BME analysis was used to produce spatially and temporally dependent maps. All maps share the same scale ranging from a minimum of 0 cases per km2 to a maximum of 60 cases per km2 [LAW 06]

– The health effects of ozone exposure in eastern USA [CHR 99b]. Spatiotemporal exposure distributions were generated and provided the input to toxicokinetic laws linked to population impact models that, in turn, were integrated with relationships describing how health effects are distributed across populations. The analysis helped health scientists and administrators derive valuable conclusions about the expected health impact on specific population cohorts within a geographical area and time period.

296


– To study lifetime population damage due to exposure to arsenic in drinking water across Bangladesh [SER 03b]. BME provided the means to assimilate a variety of KB (physical, epidemiologic, carcinogenetic and demographic) and uncertainty sources (soft data, measurement errors and secondary information). Maps of naturally occurring arsenic distribution in Bangladesh drinking water were generated. Global indicators of the adverse health effects on the population were derived (e.g., Figure 6.30), and valuable insight was gained by blending information from different scientific disciplines.

Figure 6.30. Bladder cancer maps (number of cases per km2) of Bangladesh using the empirical exposure-response (linear) model and the multistage carcinogenetic (nonlinear) model. In both cases, the results indicated an increased lifetime bladder cancer probability for the population due to arsenic [SER 03b]


297

– To estimate residential level ambient particulate matter (PM2.5 and PM10) and ozone exposures at multiple time-scales in North Carolina, USA (Figure 6.31) and to study health effects of air pollution on lupus [YU 07c]. Since the spatiotemporal estimation of long-term exposure in residential areas on the basis of air quality system observations may suffer from missing data due to scarce monitoring across space and inconsistent monitoring periods at different geographical locations, the study developed two upscaling methods (data aggregation followed by exposure estimation; and exposure estimation followed by aggregation). The methods were applied at multiple temporal scales of particulate matter and ozone exposure estimation in the residential areas considered in the health study.

Figure 6.31. Spatiotemporal map of BMEmean estimates of PM2.5/ PM10 on (a) August 25, (b) August 31 and (c) September 6 (1996) [YU 07c]

298


Figure 6.32. Total geographical area in Europe infected by Black Death at different times –denoted in black (from [CHR 07]

– The comparative study of space-time patterns and geographical propagation dynamics of major epidemics, such as the Black Death epidemic in 14th century Europe and the bubonic plague in late 19th-early 20th century India; see [CHR 05c, CHR 07, WAN 05, YU 06]. For the first time, a series of detailed space-time maps of important characteristics of the two epidemics (mortality, infected area propagation, centroid evolution, etc.) were obtained (e.g., Figures 6.32 and 6.33). The maps integrated a variety of interdisciplinary knowledge bases generating a comparative epidemic modeling that led to a number of interesting findings. Epidemic indicators confirmed that Black Death mortality was two orders of magnitude higher than that of bubonic plague. Modern bubonic plague is a rural disease typically devastating small villages in the countryside, whereas the Black Death indiscriminately attacked both large urban centers and the countryside. The epidemics had reverse areal extension features in response to annual seasonal variations. During the Indian epidemic, the disease disappeared and reappeared several times at certain locations; in Europe, once the disease entered a place, it lasted for a time proportional to the population and then disappeared for several years. On average, the Black Death was much faster than bubonic plague to reach virgin territories, despite the fact that India is slightly larger in area than Western Europe and had a railroad network almost instantly moving infected rats, fleas and people from one end of the subcontinent to the other. These findings throw new light


299

on the epidemics and need to be taken into consideration in the discussion concerning the two devastating diseases and the lessons learned from them. In this section, an attempt was made to communicate how BME analysis and modeling, with its full conceptual and technical beauty, can be applied across disciplines. For this purpose, a series of case studies were discussed that transcend a variety of disciplines.

Figure 6.33. Space-time mortality rate maps (per thousands) of bubonic plague in India during 1902-1903 [YU 06]

For a more thorough and detailed discussion of the concepts, techniques and real-world case studies presented above, the reader is encouraged to consult the original sources. 6.6. References [AKI 07] AKITA Y., CARTER G. and SERRE M.L., “Spatiotemporal non-attainment assessment of surface water Tetrachloroethene in New Jersey”, J. of Environmental Quality vol. 36(2), p. 508-520, 2007.

300


[AUG 02] AUGUSTINRAJ A., A Study of Spatiotemporal Health Effects due to Water Lead Contamination, MS Thesis, Dept. of Environ. Sci. and Engin., Univ. of North Carolina, Chapel Hill, NC. [BOG 96] BOGAERT P., “Comparison of kriging techniques in a space-time context”, Mathematical Geology, 28, p. 73-86, 1996. [BOG 02a] BOGAERT P., “Spatial prediction of categorical variables: the BME approach”, Stochastic Environ. Research and Risk Assessment, vol. 16, p. 425-448, 2002. [BOG 02b] BOGAERT P. and D’OR D., “Estimating soil properties from thematic soil maps – The BME approach”, Soil Science Soc. of America Journal, 66, p. 1492-1500, 2002. [BOG 04a] BOGAERT P., “Predicting and simulating categorical random fields: the BME approach”, Proceed. of the 1st Intern. Conf. for Advances in Mineral Resources Management & Environ. Geotechnology, (AMIREG 2004), p. 119-126, 2004. [BOG 04b] BOGAERT P. and WIBRIN M.A., “Combining categorical and continuous information within the BME paradigm”, in Proceed. GeoEnv V-Geostatistics for Environmental Applications, Neuchatel, Switzerland, October 13-15, 2004. [BOG 07] BOGAERT P. and FASBENDER D., “Bayesian data fusion in a spatial prediction context: a general formulation”, Stochastic Environmental Research and Risk Assessment, vol. 21, p. 695-709, 2007. [CHO 03] CHOI K-M, SERRE M.L. and CHRISTAKOS G., “Efficient mapping of California mortality fields at different spatial scales”, J. of Exposure Analysis & Environmental Epidemiology, vol. 13, p. 120-133, 2003. [CHO 06] CHOI K-M, CHRISTAKOS G. and WILSON M.L., “El Niño effects on influenza mortality risks in the state of California”, J. Public Health, vol. 120, p. 505-516, 2006. [CHO 08] CHOI K-M, YU H-L and WILSON M.L., “Spatiotemporal analysis of influenza mortality risks in the state of California during the period 1997-2001”, Stochastic Environmental Research and Risk Assessment, 2008, available online, DOI 10.1007/s00477-007-0168-4. [CHR 84] CHRISTAKOS G., “On the problem of permissible covariance and variogram models”, Water Resources Research, vol. 20, p. 251-265, 1984. [CHR 90a] CHRISTAKOS G., “Random Field Modelling and its Applications in Stochastic Data Processing”, Applied Sciences, PhD Thesis, 1990, Harvard University, Cambridge, MA. [CHR 90b] CHRISTAKOS G., “A Bayesian/maximum-entropy view to the spatial estimation problem”, Mathematical Geology, vol. 22, p. 763-776, 1990. [CHR 91a] CHRISTAKOS G., “On certain classes of spatiotemporal random fields with application to space-time data processing”, IEEE Trans Systems, Man, and Cybernetics vol. 21(4), p. 861-875, 1991. [CHR 91b] CHRISTAKOS G., “Some applications of the BME concept in Geostatistics”, in Fundamental Theories of Physics, Kluwer Acad. Publ., Amsterdam, The Netherlands, p. 215-229, 1991.


301

[CHR 92] CHRISTAKOS G., Random Field Models in Earth Sciences. Academic Press, San Diego, CA, 1992. [CHR 96] CHRISTAKOS G. and BOGAERT P., “Spatiotemporal analysis of springwater ion processes derived from measurements at the Dyle Basin in Belgium”, IEEE Trans. Geosciences and Remote Sensing, vol. 34, p. 626-642, 1996. [CHR 98a] CHRISTAKOS G. and LI X., “Bayesian maximum entropy analysis and mapping: A farewell to kriging estimators?”, Mathematical Geology, vol. 30(4), p. 435-462, 1998. [CHR 98b] CHRISTAKOS G. and HRISTOPULOS D.T., Spatiotemporal Environmental Health Modelling, Kluwer Academic Publ., Boston, MA, 1998. [CHR 98c] CHRISTAKOS G., “Spatiotemporal information systems in soil and environmental sciences”, Geoderma, vol. 85(2-3), p. 141-179, 1998. [CHR 99a] CHRISTAKOS G., HRISTOPOULOS D.T. and SERRE M.L., “BME studies of stochastic differential equations representing physical laws-Part I”, 5th Annual Conference, Intern. Assoc. for Mathematical Geology, Trodheim, Norway, p.63-68, 1999. [CHR 99b] CHRISTAKOS G. and KOLOVOS A., “A study of the spatiotemporal health impacts of ozone exposure”, J. of Exposure Analysis & Environmental Epidemiology, vol. 9, p. 322-335, 1999. [CHR 00a] CHRISTAKOS G., Modern Spatiotemporal Geostatistics, Oxford Univ. Press, New York, 2000. [CHR 00b] CHRISTAKOS G. and PAPANICOLAOU V., “Norm-dependent covariance permissibility of weakly homogeneous spatial random fields”, Stochastic Environmental Research and Risk Assessment, vol. 14, p. 1-8, 2000. [CHR 00c] CHRISTAKOS G. and SERRE M.L., “A spatiotemporal study of exposure-health effect associations”, J. of Exposure Analysis & Environmental Epidemiology, vol. 10, p. 168-187, 2000. [CHR 00d] CHRISTAKOS G., HRISTOPOULOS D.T. and BOGAERT P., “On the physical geometry concept at the basis of space/time geostatistical hydrology”, Advances in Water Resources, vol. 23, p. 99-810, 2000. [CHR 01] CHRISTAKOS G., SERRE M.L. and KOVITZ J., “BME representation of particulate matter distributions in the state of California on the basis of uncertain measurements”. J. of Geophysical Research, vol. 106(D9), p. 9717-9731, 2001. [CHR 02a] CHRISTAKOS G., “On the assimilation of uncertain physical knowledge bases: Bayesian and non-Bayesian techniques”, Advances in Water Resources, vol. 25, p. 12571274, 2002. [CHR 02b] CHRISTAKOS G., “On a deductive logic-based spatiotemporal random field theory”, Probability Theory & Mathematical Statistics (Teoriya Imovirnostey ta Matematychna Statystyka), vol. 66, p. 54-65, 2002. [CHR 02c] CHRISTAKOS G., BOGAERT P. and SERRE M.L., Temporal GIS, SpringerVerlag, New York, NY, with CD-ROM, 2002.

302


[CHR 04] CHRISTAKOS G., KOLOVOS A., SERRE M.L. and VUKOVICH F., “Total ozone mapping by integrating data bases from remote sensing instruments and empirical models”, IEEE Trans. Geosciences and Remote Sensing, vol. 42(5), p. 991-1008, 2004. [CHR 05a] CHRISTAKOS G., “Recent methodological developments in geophysical assimilation modelling”, Reviews of Geophysics, vol. 43, p. 1-10, 2005. [CHR 05b] CHRISTAKOS G., Random Field Models in Earth Sciences, Dover Publ. Inc., Mineola, NY, 2005. [CHR 05c] CHRISTAKOS G., OLEA R.A., SERRE M.L., YU H.L. and WANG L.-L., Interdisciplinary Public Health Reasoning and Epidemic Modelling: The Case of Black Death, Springer-Verlag, New York, NY, 2005. [CHR 06] CHRISTAKOS G., “Modelling with Spatial and Temporal Uncertainty”, in Encyclopedia of Geographical Information Science (GIS), Springer, NY, 2006. [CHR 07] CHRISTAKOS G., OLEA R.A. and YU H.-L., “Recent results on the spatiotemporal modelling and comparative analysis of Black Death and bubonic plague epidemics”, J. Public Health, vol. 121, p. 700-720, 2007. [CHR 08] CHRISTAKOS G., Treatise on Epistematics, Springer, New York, NY, 2008. [CRE 93] CRESSIE N., Statistics for Spatial Data, J. Wiley, NY, 1993. [DOR 01] D’OR D., BOGAERT P. and CHRISTAKOS G., “Application of the BME approach to soil texture mapping”, Stochastic Environmental Research and Risk Assessment, vol. 15, p. 87-100, 2001. [DOR 03] D’OR D. and BOGAERT P., “Continuous-valued map reconstruction with the Bayesian Maximum Entropy”, Geoderma, vol. 112, p. 169-178, 2003. [DOU 04] DOUAIK A., VAN MEIRVENNE M., TOTH T. and SERRE M.L., “Space-time mapping of soil salinity using probabilistic BME”, Stochastic Environmental Research and Risk Assessment, vol. 18, p. 219-227, 2004. [DOU 05] DOUAIK A., VAN MEIRVENNE M. and TOTH T., “Soil salinity mapping using spatio-temporal kriging and Bayesian maximum entropy with interval soft data”, Geoderma, vol. 128, p. 234-248, 2005. [ELO 08] ELOGNE S., HRISTOPULOS D.T. and VAROUCHAKIS M., “An application of Spartan spatial random fields in environmental mapping: focus on automatic mapping capabilities”, Stochastic Environmental Research and Risk Assessment, vol. 22(5), 2008, forthcoming. [FAS 07a] FASBENDER D., RADOUX J., and BOGAERT P., “Adaptable Bayesian data fusion for image pansharpening”, IEEE Trans on Geosciences and Remote Sensing, 2007, forthcoming. [FAS 07b] FASBENDER D., TUIA D., BOGAERT P. and KANEVSKI M., “Support-based implementation of Bayesian Data Fusion for spatial enhancement: Applications to ASTER thermal images”, submitted to Geoscience and Remote Sensing Letters, 2007. [FED 88] FEDER J., Fractals, Plenum Press, NY, 1988.


303

[GOO 94] GOODALL C. and MARDIA K.V., “Challenges in multivariate spatio-temporal modelling”, in Proceed. of the XVIIth Intern. Biometric Confer, 1-17, Hamilton, Ontario, Canada, 8-12 August 1994. [GOO 97] GOOVAERTS P., Geostatistics for Natural Resources Evaluation, Oxford Uni. Press, New York, NY. [HAA 95] HAAS T.C., “Local prediction of spatio-temporal process with an application to wet sulfate deposition”, J. of the Amer. Statistical Assoc., vol. 90, p. 1189-1199, 1995. [HRI 01] HRISTOPULOS D.T. and CHRISTAKOS G., “Practical calculation of nonGaussian multivariate moments in BME analysis”, Mathematical Geology, vol. 33(5), p. 543-568, 2001. [HRI 03] HRISTOPULOS D.T., “Spartan Gibbs random field models for geostatistical applications”, SIAM Journal of Scientific Computing, vol. 24(6), p. 2125-2162, 2008. [KOL 02] KOLOVOS A., CHRISTAKOS G., SERRE M.L. and MILLER C.T., “Computational BME solution of a stochastic advection-reaction equation in the light of site-specific information”, Water Resources Research, vol. 38, p. 1318-1334, 2002. [KOL 04] KOLOVOS A., CHRISTAKOS G., HRISTOPULOS D.T. and SERRE M.L., “Methods for generating non-separable spatiotemporal covariance models with potential environmental applications”, Advances in Water Resources, vol. 27, p. 815-830, 2004. [KOL 06] KOLOVOS A., YU H.-L, and CHRISTAKOS G., SEKS-GUI v.0.6 User Manual. Dept. of Geography, San Diego State University, San Diego, CA, 2006. [KOV 04a] KOVITZ J. and CHRISTAKOS G., “Spatial statistics of clustered data”, Stochastic Environmental Research and Risk Assessment, vol. 18(3), p. 147-166, 2004. [KOV 04b] KOVITZ J. and CHRISTAKOS G., “Assimilation of fuzzy data by the BME method”, Stochastic Environmental Research and Risk Assessment, vol. 18(2), p. 79-90, 2004. [KYR 99] KYRIAKIDIS P.C. and JOURNEL A.G., “Geostatistical space-time models: a review”, Mathematical Geology, vol. 31(6), p. 651–684, 1999. [LAW 06] LAW D.C., BERNSTEIN K., SERRE M.L., SCHUMACHER C.M., LEONE P.A., ZENILMAN J.M., MILLER W.C., and ROMPALO A.M., “Modelling an Early Syphilis Outbreak through Space and Time Using the Bayesian Maximum Entropy Approach”, Annals of Epidemiology 16(11): 797-804, 2006. [LEE 07a] LEE S.-J., BALLING R. and GOBER P., “Bayesian Maximum Entropy mapping and the soft data problem in urban climate research”, Annals of the Association of American Geographers, 2007, forthcoming. [LEE 07b] LEE S.-J. and WENTZ E.A., “Applying BME to extrapolating local-scale water consumption in Maricopa county, Arizona”, Water Resources Research, 2007, forthcoming. [LOB 07] LOBUGLIO J.N., CHARACKLIS G.W. and SERRE M.L., “Cost-effective water quality assessment through the integration of monitoring data and modelling results”, Water Resources Research, vol. 43, 2007, doi:10.1029/2006WR005020.

304


[MAC 03] MA C., “Spatio-temporal stationary covariance models”, J. of Multivariate Analysis, vol. 86, p. 97-107, 2003. [MYE 89] MYERS D.E., “To be or not to be…stationary: That is the question”, Mathematical Geology, vol. 21, p. 347-362, 1989. [MYE 02] MYERS D.E., “Space-time correlation models and contaminant plumes”, Environmetrics, vol. 13, p. 535-554, 2002. [ORT 07a] ORTON T.G. and LARK R.M., “Accounting for the uncertainty in the local mean in spatial prediction by BME”, Stochastic Environmental Research and Risk Assessment, vol. 21(6), p. 773-784, 2007. [ORT 07b] ORTON T.G. and LARK R.M., “Estimating the local mean for Bayesian maximum entropy by generalized least squares and maximum likelihood, and an application to the spatial analysis of a censored soil variable”, J. of Soil Science, vol. 58, p. 60-73, 2007. [PAP 06] PAPANTONOPOULOS G. and MODIS K., “A BME solution of the stochastic three-dimensional Laplace equation representing a geothermal field subject to sitespecific information”, Stochastic Environmental Research and Risk Assessment, vol. 20(1-2), p. 23-32, 2006. [PAR 05] PARKIN R., SAVELIEVA E. and SERRE M.L., “Soft geostatistical analysis of radioactive soil contamination”, in Ph. Renard (ed.) GeoENV V-Geostatistics for Environ. Applications, Kluwer Acad. Publishers, Dordrecht, 2005. [POR 06] PORCU E., GREGORI P. and MATEU J., “Nonseparable stationary anisotropic space-time covariance functions”, Stochastic Environmental Research and Risk Assessment, vol. 21(2), p. 113-122, 2006. [PUA 07] PUANGTHONGTHUB S., WANGWONGWATANA S., KAMENS R.M. and SERRE M.L., “Modelling the space/time distribution of Particulate Matter in Thailand and optimizing its monitoring network”, Atmospheric Environment, 2007, available online: doi:10.1016/j.atmosenv.2007.06.051. [QUE 07] QUERIDO A., YOST R., TRAORE S., DOUMBIA M.D., KABLAN R., KONARE H., and BALLO A., “Spatiotemporal mapping of total Carbon stock in agroforestry systems of Sub-Saharan Africa”, in Proceed. of ASA-CSSA-SSSA Intern. Annual Meetings, November 4-8, New Orleans, Louisiana, 2007. [QUI 04] QUILFEN Y., CHAPRON B., COLLARD F. and SERRE M.L., “Calibration/validation of an altimeter wave period model and application to TOPEX/Poseidon and Jason-1 Altimeters”, Marine Geodesy, vol. 27, p. 535-550, 2004. [SAV 05] SAVELIEVA E., DEMYANOV V., KANEVSKI M., SERRE M.L. and CHRISTAKOS G., “BME-based uncertainty assessment of the Chernobyl fallout”, Geoderma, vol. 128, p. 312-324, 2005. [SER 99a] SERRE M.L. and CHRISTAKOS G., “Modern Geostatistics: Computational BME in the light of uncertain physical knowledge – The Equus Beds study”, Stochastic Environmental Research and Risk Assessment, vol. 13(1), p. 1-26, 1999.


305

[SER 99b] SERRE M.L. and CHRISTAKOS G., “BME studies of stochastic differential equations representing physical laws-Part II”, 5th Annual Confer, Intern Assoc for Mathematical Geology, Trondheim, Norway, p. 93-98, 1999. [SER 01] SERRE M.L., CHRISTAKOS G., HOWES J. and ABDEL-REHIEM A.G., “Powering an Egyptian air quality information system with the BME space/time analysis toolbox: Results from the Cairo baseline year study”, Geostatistics for Environ. Applications, P. Monestiez, D. Allard and R. Froidevaux (eds.), Kluwer Acad. Publ., Dordrecht, The Netherlands, p. 91-100, 2001. [SER 03a] SERRE M.L. and CHRISTAKOS G., “Efficient BME estimation of subsurface hydraulic properties using measurements of water table elevation in unidirectional flow”, in Calibration and Reliability in Groundwater Modelling: A Few Steps Closer to Reality. K. Kovar and Z. Hrkal (eds.), IAHS Publ. no. 277, Oxfordshire, UK, p. 321-327, 2003. [SER 03b] SERRE M.L., KOLOVOS A., CHRISTAKOS G. and MODIS K., “An application of the holistochastic human exposure methodology to naturally occurring Arsenic in Bangladesh drinking water”, Risk Analysis, vol. 23(3), p. 515-528, 2003. [SER 03c] SERRE M.L., CHRISTAKOS G., LI H. and MILLER C.T., “A BME solution to the inverse problem for saturated groundwater flow”, Stochastic Environmental Research and Risk Assessment, vol. 17(6), p. 354-369, 2003. [VYA 04] VYAS V.M., TONG S.N., UCHRIN C., GEORGOPOULOS P.G. and CARTER G.P., “Geostatistical estimation of horizontal hydraulic conductivity for the KirkwoodCohansey aquifer”, Jour. of the American Water Resources Assoc., vol. 40(1), p. 187-195, 2004. [WAN 05] WANG L.-L., Spatiotemporal Analysis of Black Death in France. M.S. Thesis, Dept. of Environ. Sci. and Engin., Univ. of North Carolina, Chapel Hill, NC, 2005. [WIB 06] WIBRIN M.-A., BOGAERT P. and FASBENDER D., “Combining categorical and continuous spatial information within the Bayesian Maximum Entropy paradigm”, Stochastic Environmental Research and Risk Assessment, vol. 20, p. 423-434, 2006. [YU 05] YU H.-L. and CHRISTAKOS G., “Porous media upscaling in terms of mathematical epistemic cognition”, SIAM J. on Appl. Math., vol. 66(2), 2005, p. 433-446. [YU 06] YU H.-L. and CHRISTAKOS G., “Spatiotemporal modelling and mapping of the bubonic plague epidemic in India”, Intern. Jour. of Health Geographics, vol. 5(12), 2006, Internet online journal [http://www.ij-healthgeographics.com/content/5/1/12]. [YU 07a] YU H.-L., KOLOVOS A., CHRISTAKOS G., CHEN J.-C., WARMERDAM S. and DEV B., “Interactive spatiotemporal modelling of health systems: The SEKS-GUI framework”, Stochastic Environmental Research and Risk Assessment – Special Issue on “Medical Geography as a Science of Interdisciplinary Knowledge Synthesis under Conditions of Uncertainty”, D.A. Griffith and G. Christakos (eds.), vol. 21(5), p. 555-572, 2007.

306


[YU 07b] YU H.-L., CHRISTAKOS G., MODIS K. and PAPANTONOPOULOS G., “A composite solution method for physical equations and its application in the Nea Kessani geothermal field (Greece)”, J. of Geophysical Research-Solid Earth., vol. 112, 2007, B06104, doi:10.1029/2006JB004900. [YU 07c] YU H.-L., CHEN J.-C., CHRISTAKOS G. and JERRETT M., “Estimating residential level ambient PM10 and Ozone exposures at multiple time-scales in the Carolinas with the BME method”, Epidemiology, 2007, submitted.

List of Authors

George CHRISTAKOS Department of Geography, Storm Hall 314, San Diego State University, 5500 Campanile Drive, San Diego, CA 92182-4493, USA Vasily DEMYANOV Institute of Petroleum Engineering, Heriot-Watt University, Edinburgh, EH14 4AS, UK Loris FORESTI Institute of Geomatics and Analysis of Risk (IGAR), University of Lausanne, Amphipôle, 1015 Lausanne, Switzerland Christian KAISER Institute of Geography (IGUL), University of Lausanne, Antropôle 1015 Lausanne, Switzerland Mikhail KANEVSKI Institute of Geomatics and Analysis of Risk (IGAR), University of Lausanne, Amphipôle, 1015 Lausanne, Switzerland Michel MAIGNAN Institute of Mineralogy and Geochemistry, University of Lausanne, Antropôle, 1015 Lausanne, Switzerland Alexei POZDNOUKHOV Institute of Geomatics and Analysis of Risk (IGAR), University of Lausanne, Amphipôle, 1015 Lausanne, Switzerland

308


Ross PURVES Department of Geography, University of Zürich – Irchel, Zurich, Switzerland Frédéric RATLE Institute of Geomatics and Analysis of Risk (IGAR), University of Lausanne, Amphipôle, 1015 Lausanne, Switzerland Elena SAVELIEVA Environmental Modelling and System Analysis Lab. Nuclear Safety Institute (IBRAE), Russian Academy of Sciences, 52 B. Tulskaya, Moscow, 113191, Russia Rafael TAPIA Institute of Geomatics and Analysis of Risk (IGAR), University of Lausanne, Amphipôle, 1015 Lausanne, Switzerland Vadim TIMONIN Institute of Geomatics and Analysis of Risk (IGAR), University of Lausanne, Amphipôle, 1015 Lausanne, Switzerland Devis TUIA Institute of Geomatics and Analysis of Risk (IGAR), University of Lausanne, Amphipôle, 1015 Lausanne, Switzerland

Index

A a posteriori probability 124 accuracy 14, 58, 221, 234, 235, 236, 248, 250, 268, 277, 282, 289, 291, 294 ANNEX model 188, 190, 192–194, 199, 200 anisotropy 2, 6, 7, 49, 53, 84 atmospheric 210, 227, 247, 281, 282 automatic mapping 124, 150, 185, 187, 190, 193, 199 avalanches 149, 225, 226, 234, 241

B Bayesian 1, 5, 8, 12, 20, 99, 123, 124, 150, 194, 247, 250, 271, 276 behavior 47, 49, 141, 146, 167, 218, 225, 256, 262, 291 biological 14, 247, 248 BME 1, 8, 247, 248 box-counting 21, 27, 28, 30–32, 40–42 brain 109, 248, 250, 270

C classification 1–3, 13–15, 66, 71–74, 95– 102, 107–110, 122, 124, 131, 136, 137, 149, 165, 185, 186, 192–194, 200, 204– 209, 223–233, 239, 251, 275 cluster 31, 33, 35, 142–144, 256 clustering 2, 3, 6, 9, 14, 19, 20–44, 85, 98, 101, 127, 141, 142, 144, 201, 202, 206, 214, 215, 218, 225, 260

cognition 14 complex relationships 169 conditional distribution 99, 123 confidence measure 123 conjugate gradient 112, 154, 175 continuum 251–253, 255 co-simulations 5, 88–90 covariance 7, 47–51, 56, 59, 61, 120, 144, 145, 211, 248, 253, 254, 257, 259–263, 265, 266, 269, 270, 273, 276, 278, 283, 285, 292–294 cross-validation 105, 106, 122, 135, 137, 158, 187, 188, 190, 193–195, 229, 232, 233, 285

D decision support systems 149, 185, 226, 227, 228, 241 decision-oriented mapping 4, 8, 209 density estimation 96, 98, 99, 101, 119, 122, 141 de-trending 15 digital elevation model (DEM) 5, 61, 151, 154, 163, 164, 168, 169, 199, 237–239 directional variogram 91, 152, 153, 181 discriminative models 99 distribution of weights 176 drift 5, 58, 59, 62, 63, 103, 152, 157, 158, 198, 230, 231

310


E Earth 14, 254, 282 elevation 5, 58, 61, 150–156, 159–161, 163, 164, 166–174, 177–179, 189, 190, 230, 238, 289 empirical 1, 2, 6, 14, 104, 131, 132, 229, 235, 250, 255, 256, 267, 271, 277, 282, 285, 287, 296, entropy 82, 250 epidemiology 281, 291 epistematics 14, 248, 250, 273, 275 epistemology 14 Euclidean 23, 26, 31, 108, 143, 186, 188, 250–257, 262, 263, 270 Euclidean distance 108, 128, 142, 143, 186, 201, 253, 254, 256, 257 evolutionary 14, 141, 250, 270 experimental variogram 7, 8, 57, 197, 216– 218, 221 exploratory spatial data analysis (ESDA) 6, 20 exposure 210, 247, 281, 291, 294–297 extreme precipitation 168, 173, 174

F feature selection 101, 231 Föhn 159–162 forecast 90, 225, 226, 227, 229, 231, 234– 237, 239, 241 fractals 26, 33, 36, 256

G Gaussian kernel 98, 120, 121, 136, 137, 140 General Regression Neural Network (GRNN) 7, 13, 97, 99, 103, 109, 119– 122, 124, 146, 149, 185, 188, 190–192, 194, 199, 200, 241 generalization 8, 9, 59, 97, 104, 105, 113, 131, 132, 133, 141, 153, 154, 185, 198, 204, 205, 209, 228, 255, 275, 277 generative methods 98, 99 geo-features 151, 164, 242 Geostat Office 13 geostatistics 3, 5, 7, 10–13, 47, 58, 64, 149, 150, 168, 179, 197, 209, 247

GSLIB 12, 13 GUI 14, 275–278, 281

H health 14, 250, 251, 281, 291, 292, 294–297 Hebbian learning 125 heterogenity 257, 258, 265, 280 hybrid models 3, 15, 168, 179

I indoor radon 20, 36, 37, 42, 43, 149, 209– 213, 217, 218, 224, 225 infinity-norm distance 186 intrinsic hypothesis 2, 6, 48, 48, 51 inverse distance weighting 157, 158, 168 isotropy 254

J, K joint distribution 99, 220, 234 kernel-based methods 7 kernel width 120, 188, 190, 232, 233 kernels 120, 132, 136, 145, 229 K-function 25, 26, 28, 39, 40 k-nearest neighbors 13, 108, 186 kriging co-kriging 5, 13, 58–63, 67, 88, 282 collocated 62 indicator 7, 64–67, 69–74, 84, 85, 197–200, 209, 211, 215, 217, 241 lognormal 56, 57 ordinary 50 simple 50, 52, 58, 67, 77, 82, 211, 286, 290 universal 56 with external drift 5, 58, 59, 62, 63, 103, 152, 157, 158, 198

L lacunarity 27, 33, 34, 42, 43 laws 248, 250, 253, 255, 256, 258, 271, 272, 276, 277, 288, 293, 295 lazy learning 186 learning rate 110, 125, 128, 130, 205 leave-one-out cross-validation (LOOCV) 105, 106, 187, 188, 190, 193, 195

Index Levenberg-Marquardt 112, 117, 175, 176, 178, 179 LibSVM 13 local correlations 171, 173, 177 logic 273, 275, 276

M machine learning 1, 3, 6, 7, 9–14, 44, 95, 96, 99, 101, 104, 107, 109, 131–133, 146, 149, 150, 151, 157, 163, 168, 179, 184, 185, 194, 197, 199, 226, 227, 241, 242 Manhattan distance 186 maximum a posteriori (MAP) decision rule 123 meteorological stations 151, 165 meteorology 181 methodology 3, 5, 10, 107, 137, 150, 153, 160, 168, 181, 197, 248, 270, 271, 274, 281, 288 Minkowski distance 186 model assessment 2, 8, 9 model selection 2, 8, 9, 103, 131, 154, 233 Morisita index 24, 25, 26, 30, 36, 38, 39 moving windows statistics 173 multi-layer perceptron (MLP) 7, 97, 109– 118, 150, 153–162, 165, 166, 168, 171– 183, 187, 188, 190, 192

N Nadaraya-Watson Kernel Regression Estimator 119 natural hazard 149, 150, 225–227, 242 nearest neighbor methods 99, 103, 108, 109, 226, 227, 241 Netlab 13 neural networks 1, 7, 13, 97, 98, 103, 109, 119, 121, 122, 124, 133, 149, 168, 178, 190, 241 neural network residual kriging (NNRK) 173, 176, 179, 181–184 neural network residual simulations 173, 179, 182 neurons 109, 111, 114–118, 125–130, 153– 156, 162, 163, 166, 167, 171, 175, 176, 180, 203 n-fold cross-validation 187

311

non-Euclidean 250, 252–255, 262, 263, 270 nonlinear 7, 56, 103, 110, 111, 114, 115, 120, 126, 133, 135, 139, 144, 145, 150, 159, 160, 163, 164, 167–169, 179, 184, 185, 201, 204, 209, 226, 228, 229, 237, 250, 271, 277, 296 nonlinear dimensionality reduction 144 nonstationarity 152 Nscore transform 211

O optimization algorithms 110, 112, 113, 174, 179, 185 orographic precipitation 159 overfitting 104, 121, 131, 137, 154–156, 179, 181, 182, 188, 228, 233

P Parzen window 13–14 physical 2, 5, 14, 103, 123, 124, 159, 190, 209, 210, 217, 226, 227, 237, 247–249, 251–253, 255, 256, 260, 266, 267, 270– 272, 275, 281, 288, 289, 293, 294, 296 Poisson point process 39 precipitation 159, 160, 168–174, 177–179, 181, 182, 184, 192, 193, 196–199 principal component analysis (PCA) 102, 126, 144, 145 prior probability 123 Probabilistic Neural Network (PNN) 7, 13, 98, 103, 109, 122–124, 146, 149, 192, 194–196, 198–200, 241 probability density function (pdf) 1, 2, 5, 7, 15, 64, 65, 69, 70, 80–82, 84, 85, 88, 122, 184, 209, 211, 213, 220, 250, 257, 258, 260, 267, 268, 270, 272, 277, 280, 289, 293 probability mapping 64, 209, 211

Q, R quantile 81, 84, 85, 184, 213, 223 R 13 random field 5, 15, 99, 257, 258, 259, 266, 273, 285

312


realizations 5, 7, 47, 76–79, 82–91, 183, 209, 220, 257, 258 recursive feature elimination 165, 231 regression 1, 3, 7, 13, 14, 27, 28, 30, 58, 60, 64, 76, 84, 88, 96, 97, 99–101, 103, 108–110, 114, 119, 122–124, 137, 146, 149, 157, 158, 185, 186, 190–193, 199, 241, 272 regression kriging 97, 157, 158 residual kriging 173, 179, 286 residual simulations 173, 179, 182 risk 21, 22, 85, 96, 125, 131, 132, 136, 137, 150, 209, 250, 292 analysis 25, 64 assessment 14, 227, 268, 281 mapping 4, 5, 7, 64, 235 RMSE 63, 154, 155, 158, 159, 162, 166, 171, 172, 175, 178, 188, 191

S sandbox counting 26 science 5, 95, 96, 150, 248, 250, 257, 268, 269, 272, 275, 281, 282, 291 SEKS 14, 275–278, 281 self-organizing (Kohonen) maps 7, 124 sequential Gaussian simulations (SGS) 78, 83, 89, 176, 183, 184, 209, 211, 215, 217–221, 223–225 S-GeMS 13 simulated annealing 79, 112, 141 simulation cell-based 77–79, 84 Gaussian 13, 78, 81, 83, 84, 88, 89, 176, 183, 209, 211 indicator 13, 78, 81, 84–87 multiple-point 13 object-based 78 smoothing parameter 121, 122 software 11–14, 251, 275, 281 space-time dependence 14, 248, 250, 253, 254, 257, 259, 260, 261, 263, 264, 273, 276, 277, 292 spatiotemporal 1, 2, 3, 7, 8, 11, 14, 149, 150, 225–227, 240

stationarity 21, 47, 48, 67, 152, 215, 217, 218, 264, 265 second-order 2, 48–50 strict 48 statistical learning theory 7, 131, 226, 228 statistics 12, 13, 40, 43, 47, 62, 77– 79, 81, 95, 102, 131, 151–153, 169–171, 173, 177, 183, 187, 188, 191–193, 214, 223, 247, 249, 250, 271, 291 stochastic 2, 5, 64, 76–79, 81–85, 88– 91, 112, 248–250, 257, 259, 263, 265, 269, 272, 273, 275, 276, 285 simulations 7, 76, 77, 88, 182, 241 structural analysis 3, 6 supervised learning 3, 44, 100, 101 support vector 5, 133, 135, 139, 228, 229, 232, 233, 236 machines (SVM) 1, 7, 13, 19, 98, 101, 107, 132–134, 136, 137, 144, 146, 164, 165, 225, 226, 228–241 regression (SVR) 7, 97, 137–141, 157, 158, 168 synapse 125, 176 synthesis 14, 248, 273, 275, 281 system attributes 247

T temperature 58, 61, 62, 63, 79, 101, 150– 168, 179, 187–189, 191, 192, 199, 230, 231, 238, 271, 285, 287, 289, 290, 294 gradients 159–161, 231, 238 inversion 151, 159, 163, 165–167 temporal scale 150, 168, 169, 297 terrain features 163 testing 9, 10, 44, 70, 113, 115, 116, 118, 141, 153–156, 168, 202, 219, 229 theory 7, 67, 124, 126, 131–133, 135, 150, 153, 226, 228, 248, 249, 251, 257, 258, 265, 266, 273, 274, 276 topo-climatic 1, 185 topographical 163, 164, 167, 173, 282 TORCH 13 training and validation curves 155, 163

Index transductive 44, 99, 100, 101 trend 14, 44, 49, 56, 58, 59, 114, 115, 152, 168, 179, 181–183, 192, 215, 217, 260, 265, 271

U uncertainty 4, 5, 7, 11, 36, 53, 55, 64, 66, 72, 76, 84, 85, 90, 107, 108, 182, 196, 211, 224, 226, 229, 248–250, 257, 268, 273, 275, 276, 282, 285, 289, 291–293, 296 unsupervised learning 3, 100, 101, 109, 124, 125

313

V, W validity domains 34, 37, 42 variability 2, 5, 7, 14, 51, 58, 62, 74, 76, 77, 82, 84, 88, 90, 144, 150, 168, 169, 179, 182, 210, 211, 225, 237, 240, 264, 270, 282, 289, 291, 294 variogram model 51–54, 58, 60, 65, 66, 82, 84, 88, 153, 197, 198, 209, 216–218, 221, 222, 225 variography of residuals 6, 155, 190 VC dimension 131 Voronoï polygons 6, 24, 71, 215 Weka 14

Advanced Mapping of Environmental Data (Geographical Information Systems)

Groundwater Modeling Using Geographical Information Systems

Community Participation and Geographical Information Systems

Web and Wireless Geographical Information Systems

Operational Research and Networks (Geographical Information Systems)

Information systems and data compression

Uncertainty in Geographical Information (Research Monographs in Geographic Information Systems,)

Textbook of Remote Sensing and Geographical Information Systems

Advanced Agent-Based Environmental Management Systems

Advanced Information Systems Engineering Workshops (LNBIP 83)

Web and Wireless Geographical Information Systems, 7 conf., W2GIS 2007

An Introduction to Geographical Information Systems (3rd Edition)

An Introduction to Geographical Information Systems (3rd Edition)

Modelling Urban Development with Geographical Information Systems and Cellular Automata

The Modeling Process in Geography (Geographical Information Systems)

Advanced Information Systems Engineering - CAiSE 2011

Agent-Based Models of Geographical Systems

Encyclopedia of information systems,

Encyclopedia of information systems,

Elevation Data for Floodplain Mapping

Encyclopedia of information systems,

Data Mining: Special Issue in Annals of Information Systems

Fundamentals of Spatial Information Systems (Apic Studies in Data Processing)

Analyzing Environmental Data

Advanced Environmental Monitoring

Advanced Environmental Monitoring

GEO-DATA. The World Geographical Encyclopedia

Geo-Data - The World Geographical Encyclopedia

Classics from IJGIS: Twenty years of the International Journal of Geographical Information Science and Systems

Advanced Data Mining Techniques

Advanced Mapping of Environmental Data (Geographical Information Systems)

Groundwater Modeling Using Geographical Information Systems

Community Participation and Geographical Information Systems

Web and Wireless Geographical Information Systems

Operational Research and Networks (Geographical Information Systems)

Information systems and data compression

Uncertainty in Geographical Information (Research Monographs in Geographic Information Systems,)

Textbook of Remote Sensing and Geographical Information Systems

Advanced Agent-Based Environmental Management Systems

Advanced Information Systems Engineering Workshops (LNBIP 83)

Web and Wireless Geographical Information Systems, 7 conf., W2GIS 2007

An Introduction to Geographical Information Systems (3rd Edition)

An Introduction to Geographical Information Systems (3rd Edition)

Modelling Urban Development with Geographical Information Systems and Cellular Automata

The Modeling Process in Geography (Geographical Information Systems)

Advanced Information Systems Engineering - CAiSE 2011

Agent-Based Models of Geographical Systems

Encyclopedia of information systems,

Encyclopedia of information systems,

Elevation Data for Floodplain Mapping

Encyclopedia of information systems,

Data Mining: Special Issue in Annals of Information Systems

Fundamentals of Spatial Information Systems (Apic Studies in Data Processing)

Analyzing Environmental Data

Advanced Environmental Monitoring

Advanced Environmental Monitoring

GEO-DATA. The World Geographical Encyclopedia

Geo-Data - The World Geographical Encyclopedia

Classics from IJGIS: Twenty years of the International Journal of Geographical Information Science and Systems

Advanced Data Mining Techniques

Recommend Documents