Self-organising Data Mining

Johann-Adolf Mueller Frank Lemke Self-organising Data Mining An Intelligent Approach To Extract Knowledge From Data Th...

Author: Frank Lemke

156 downloads 1229 Views 5MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

Johann-Adolf Mueller Frank Lemke

Self-organising Data Mining An Intelligent Approach To Extract Knowledge From Data

Theory Real World Applications Software Future Technologies

30.37 8.45 15.23 33.76 56.17 14.41 52.05 87.72 57.89 67.26 66.75 56.35 34.18 23.52 44.26 87.87

30.4 8.52 25.38 78.62 63.69 79.05 90.25 23.16 55.26 78.62 15.79 1.523 13.18 77.42 16.05 57.81

0.37 2.42 7.18 11.22 2.76 0.251 21.45 9.57 2.69 1.48 0.294 0.998 3.32 4.75 11.07 23.44

305.8 207.2 231.7 243.5 278.9 211.2 427.1 401.1 389.3 343.5 323.4 312.9 269.6 289.3 279.4 312.7

132.0 99.7 108.4 368.2 551.6 790.4 143.9 59.47 77.4 187.4 188.3 234.6 276.1 477.8 498.2 402.1

0.56 0.089 0.98 1.54 4.201 2.02 1.162 0.034 0.73 2.311 3.46 1.85 2.47 1.036 0.92 1.763

29.38 31.44 42.65 45.12 56.09 78.78 36.32 27.14 55.67 78.06 29.23 11.87 19.43 21.37 16.86 25.38

cluster patterns networks rules

Mueller, Johann-Adolf; Lemke, Frank: Self-Organising Data Mining. An Intelligent Approach To Extract Knowledge From Data. Berlin , Dresden 1999 1. Edition

Copyright © 1999, Johann-Adolf Mueller, Frank Lemke Dresden, Berlin All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without the prior written permission of the copyright holder.

2

Contents Preface

1

Knowledge Discovery from Data

9

1.1 Models and their application in decision making 1.2 Relevance and value of forecasts 1.3 Theory driven approach 1.4 Data driven approach 1.5 Data mining References

2

Self-organising Data Mining

31

2.1 Involvement of users in the data mining process 2.2 Automatic model generation • regression based models • rule based modelling • symbolic modelling • nonparametric models

2.3 Self-organising data mining References

3

Self-organising Modelling Technologies

57

3.1 Statistical Learning Networks 3.2 Inductive approach - The GMDH algorithm • induction • principles • model of optimal complexity

References

4

Parametric GMDH Algorithms

77

4.1 Elementary models (neurons) 4.2 Generation of alternate model variants 4.3 Nets of active neurons 4.4 Criteria of model selection 4.5 Validation References

5

Nonparametric Algorithms

103

5.1 Objective Cluster Analysis 5.2 Analog Complexing 5.3 Self-organising Fuzzy Rule Induction 5.4 Logic based rules References

Contents

3

J.A. Mueller and F. Lemke

6

Application of Self-organising Data Mining

125

6.1 Spectrum of self-organising data mining methods 6.2 Choice of appropriate modelling methods 6.3 Application fields 6.4 Synthesis 6.5 Software tools References

7

KnowledgeMiner

147

7.1 General features 7.2 GMDH implementation • elementary models and active neurons • generation of alternate model variants • criteria of model selection • systems of equations

7.3 Analog Complexing implementation • features • example

7.4 Fuzzy Rule Induction implementation • fuzzification • rule induction • defuzzification • example

7.5 Using models • the model base • finance module

8

Sample Applications

177

8.1 ... From Economics • national economy • stock prediction • balance sheet • sales prediction • solvency checking • energy consumption

8.2 ... From Ecology • water pollution • water quality

8.3 ... From other Fields • heart disease • U.S. congressional voting behavior

References

Index

4

Contents

226

Preface

This book is dedicated to Prof. A.G. Ivakhnenko, the father of GMDH, to his eighty fifth’ birthday

Preface The rapid development of information technology, continuing computerization in almost every field of human activity and distributed computing has led to a flood of data stored in data bases and data warehouses. In the 1960s, Management Information Systems (MIS) and then, in the 1970s, Decision Support Systems (DSS) were praised for their potential to supply executives with mountains of data needed to carry out their jobs. While these systems have supplied some useful information for executives, they have not lived up to their proponents’ expectations. They simply supplied too much data and not enough information to be generaly useful. Today, there is an increased need for information - contextual data - non obvious and valuable for decision making from a large collection of data. This is an interactive and iterative process of various subtasks and decisions and is called Knowledge Discovery from Data. The engine of Knowledge Discovery - where data is transformed into knowledge for decision making - is Data Mining. There are very different data mining tools available and many papers are published describing data mining techniques. We think that it is most important for a more sophisticated data mining technique to limit the user involvement in the entire data mining process to the inclusion of well-known a priori knowledge. This makes the process more automated and more objective. Most users’ primary interest is in generating useful and valid model results without having to have extensive knowledge of mathematical, cybernetic and statistical techniques or sufficient time for complex dialog driven modelling tools. Soft computing, i.e., Fuzzy Modelling, Neural Networks, Genetic Algorithms and other methods of automatic model generation, is a way to mine data by generating mathematical models from empirical data more or less automatically. In the past years there has been much publicity about the ability of Artificial Neural Networks to learn and to generalize despite important problems with design, development and application of Neural Networks: • Neural Networks have no explanatory power by default to describe why results are as they are. This means that the knowledge (models) extracted by Neural Networks is still hidden and distributed over the network. • There is no systematical approach for designing and developing Neural Networks. It is a trial-and-error process. • Training of Neural Networks is a kind of statistical estimation often using algorithms that are slower and less effective than algorithms used in statistical software. • If noise is considerable in a data sample, the generated models systematically tend to being overfitted. In contrast to Neural Networks that use • Genetic Algorithms as an external procedure to optimize the network architecture and • several pruning techniques to counteract overtraining,

Preface

5


this book introduces principles of evolution - inheritance, mutation and selection - for generating a network structure systematically enabling automatic model structure synthesis and model validation. Models are generated from the data in the form of networks of active neurons in an evolutionary fashion of repetitive generation of populations of competing models of growing complexity and their validation and selection until an optimal complex model - not too simple and not too complex has been created. That is, growing a tree-like network out of seed information (input and output variables' data) in an evolutionary fashion of pairwise combination and survival-of-the-fittest selection from a simple single individual (neuron) to a desired final, not overspecialized behavior (model). Neither, the number of neurons and the number of layers in the network, nor the actual behavior of each created neuron is predefined. All this is adjusted during the process of self-organisation, and therefore, is called self-organising data mining. A self-organising data mining creates optimal complex models systematically and autonomously by employing both parameter and structure identification. An optimal complex model is a model that optimally balances model quality on a given learning data set ("closeness of fit") and its generalisation power on new, not previously seen data with respect to the data's noise level and the task of modelling (prediction, classification, modelling, etc.). It thus solves the basic problem of experimental systems analysis of systematically avoiding "overfitted" models based on the data's information only. This makes self-organising data mining a most automated, fast and very efficient supplement and alternative to other data mining methods. The differences between Neural Networks and this new approach focus on Statistical Learning Networks and induction. The first Statistical Learning Network algorithm of this new type, the Group Method of Data Handling (GMDH), was developed by A.G. Ivakhnenko in 1967. Considerable improvements were introduced in the 1970s and 1980s by versions of the Polynomial Network Training algorithm (PNETTR) by Barron and the Algorithm for Synthesis of Polynomial Networks (ASPN) by Elder when Adaptive Learning Networks and GMDH were flowing together. Further enhancements of the GMDH algorithm have been realized in the “KnowledgeMiner” software described and enclosed in this book. KnowledgeMiner is a powerful and easy-to-use modelling and prediction tool designed to support the knowledge extraction process on a highly automated level and has implemented three advanced self-organising modelling technologies: GMDH, Analog Complexing and self-organising Fuzzy Rule Induction. There are three different GMDH modelling algorithms implemented - active neurons, enhanced network synthesis and creation of systems of equations - to make knowledge extraction systematically, fast and easy-to-use even for large and complex systems. The Analog Complexing algorithm is suitable for prediction of the most fuzzy processes like financial or other markets. It is a multidimensional search engine to select most similar past system states compared with a chosen (actual) reference state from a given data set. All selected patterns will be synthesized to a most likely, most optimistic and most pessimistic prediction. KnowledgeMiner does this in an objective way using GMDH finding out the optimal number of synthesized patterns and their composition. Fuzzy modelling is an approach to form a system model using a description language based on fuzzy logic with fuzzy predicates. Such a language can describe a dynamic multiinput/multi-output system qualitatively by means of a system of fuzzy rules. Therefore, the generated models can be • • • •

6

linear/nonlinear time series models, static/dynamic linear/nonlinear multi-input/single-output models, systems of linear/nonlinear difference equations (multi-input/multi-output models), systems of static/dynamic multi-input/multi-output fuzzy rules described analytically in all four cases, as well as Preface

Preface

• nonparametric models obtained by Analog Complexing. This book provides a thorough introduction to self-organising data mining technologies for business executives, decision makers and specialists involved in developing Executive Information Systems (EIS) or in modelling, data mining or knowledge discovery projects. It is a book for working professionals in many fields of decision making: Economics (banking, financing, marketing), business oriented computer science, ecology, medicine and biology, sociology, engineering sciences and all other fields of modelling of ill-defined systems. Each chapter includes some practical examples and a reference list for further reading. The accompanying diskette/internet download contains the KnowledgeMiner Demo version and several executable examples. This book offers a comprehensive view to all major issues related to selforganising data mining and its practical application for solving real-world problems. It gives not only an introduction to self-organising data mining, but provides answers to questions like: • what is self-organising data mining compared with other known data mining techniques, • what are the pros, cons and difficulties of the main data mining approaches, • what problems can be solved by self-organising data mining, specifically by using the KnowledgeMiner modelling and prediction tool, • what is the basic methodology for self-organising data mining and application development using a set of real-world business problems exemplarily, • how to use KnowledgeMiner and how to prepare a problem for solution. The book spans eight chapters. Chapter 1 discusses several aspects of knowledge discovery from data as an introductory overview and understanding, such as why it is worth building models for decision support and how we think forecasting can be applied today to get valuable predictive control solutions. Also considered are the pros, cons and difficulties of the two main approaches of modelling: Theory-driven and data-driven modelling. Chapter 2 explains the idea of a self-organising data mining and put it in context to several automated data-driven modelling approaches. The algorithm of a self-organising data mining is introduced and we describe how self-organisation works generally, what conditions it requires, and how existing theoretical knowledge can be embedded into the process. Chapter 3 introduces and describes some important terms in self-organising modelling: Statistical Learning Networks, inductive approach, GMDH, nonphysical models, and model of optimal complexity. Chapter 4 focuses on parametric regression based GMDH algorithms. Several algorithms on the principles of self-organisation are considered, and also the important problem of selection criteria choice and some model validation aspects are discussed. In chapter 5, three nonparametric algorithms are discussed. First, there is the Objective Cluster Analysis algorithm that operates on pairs of closely spaced sample points. For the most fuzzy objects, the Analog Complexing algorithm is recommended selecting the most similar patterns from a given data set. Thirdly, a self-organising fuzzy-rule induction can help to describe and predict complex objects qualitatively. In chapter 6 we want to point to some application opportunities of self-organising data mining from our own experience. Selected application fields and ideas on how a self-organising modelling approach can contribute to improve results of other modelling methods - simulation, Neural Networks and econometric modelling (statistics) - are suggested. Also included in this chapter is a discussion on a synthesis of model results, its goals and its options while the last part gives a short overview of existing self-organising data mining software. Preface

7


In chapter 7 the KnowledgeMiner software is described in more detail to give the reader an understanding of its self-organising modelling implementations and to help examining the examples included in the accompanied diskette or Internet download. Chapter 8 explains based on several sample applications from economics, ecology, medicine and sociology how it is possible to solve complex modelling, prediction, classification or diagnosis tasks systematically and fast using the knowledge extraction capabilities of a self-organising data mining approach. Since self-organising data mining will evolve quickly, especially the KnowledgeMiner software, the following Internet addresses can be referenced to for news, updates and new versions, but also for new research results and other discussions and comments reflecting this book: http://www.knowledgeminer.net http://www.informatik.htw-dresden.de/~muellerj .

We would like to extend our thanks to our wives and our entire families for their encouragement and understanding during the writing of this book. A special thanks to Julian Miller from Script Software International for his supporting work and for his active promotion of the KnowledgeMiner software from the beginning. Thank you also to Russell Gum for proof reading the manuscript and to all persons who helped with their comments, suggestions and critics. This electronic edition is a preprint of the book, and it serves all KnowledgeMiner users as a documentation and guide about theory and application of self-organising data mining. It may also form the basis for discussing these items in the KnowledgeMiner discussion forum: http://network54.com/Hide/Forum/goto?forumid=13476 .

Comments and remarks are appreciated.

September 17, 1999 Johann-Adolf Mueller

8

Preface

Frank Lemke

1


Recently, knowledge discovery from data and data mining tools and methods have received an increased attention for many decision tasks in business, ecology, sociology, biology and medicine, engineering sciences a/o. There are very different interpretations and expectations of what knowledge discovery can do and of which methods or approaches are preferable. This chapter discusses several aspects of this problem as an introductory overview and understanding. Section 1.1 explains why it is worth building models for decision support and lists the problems of modelling of complex systems. In section 1.2, we want to answer, how we think forecasting can be applied today to get valuable predictive control solutions while section 1.3 and 1.4 draw attention to the pros, cons and difficulties of the two main approaches of modelling: theory-driven and data-driven modelling. Finally, section 1.5 will focus on the subtasks of knowledge discovery and especially on data mining as its central part.

1.1 Models and their application in decision making Decision making in every field of human activity needs problem detection in addition to a decision maker's impression that a problem exists or that something is wrong. To know what is, what will happen or what might happen under certain circumstances is often very difficult even for simple systems. Therefore, humans are seeking proxies for real processes. A substitute that can generate reliable information about a real system and its behaviour is called a model. Sometimes, these models are traditional mathematical descriptions that can help one to understand the system. Other times, we are satisfied by information about the expected behaviour without specific information about the real system. Anyway, models form the basis for any decision. It is worth building models to aid decision making for the following reasons: Models make it possible • to recognize the structure and function of objects (subject of identification). This leads to a deeper understanding of the problem. Usually, models can be analyzed more readily than the original problem; • to find appropriate means for exercising an active influence on the objects (subject of control); • to predict what the respective objects have to expect in the future (subject of prediction), but also to experiment with models, and thus answer "what-if“ type questions. Example: National economy One important task given to economic sciences is to improve the quality of planning and management on all national economic levels. In economic research and elaboration of plans and strategies, the analysis of economic systems, that is, making studies on the level of developments achieved so far and on existing deviations from a plan, and the prediction of economic indices, that is, the determination of potential development possibilities of the economy under study, are gaining importance. The purpose of such studies is to create preconditions appropriated for expected future developments, and to find the laws and factors of influence causing these developments. Recent history has shown that an increased application of modelling is becoming more important. The quality of prediction, the determination of the basic proportions of prospective economic planning, balancing and scientific substantiation of planning the development of the economy can be improved essentially by modelling.

Models and their application in decision making

9

J.A. Mueller and F. Lemke Additionally, mathematical models make it possible to select factors and instruments for control (see section 8.1.1). Example: Financial analysis The task of financial analysis is to analyse the value of financial assets and products such as currencies, international interests, bonds, stocks and commodities that play an increasing role for many investors. Within technical analysis, there have been developed many trading indicators using a wide spectrum of mathematical methods. The main disadvantage of all indicators that are computed on historical data is this: since only historical data are considered, a trading signal may be generated a few days after it might have been advisable to trade. This can result in large losses. Using the principles of predictive control, it is possible to reduce the time delay in generating trading signals. Generally, the predictive control process is split into two parts, the prediction and the control process. The first is responsible for modelling and prediction of the development of the focused portfolio as a very noisy dynamic process. The second part has to generate a trading signal based on the information extracted in the modelling module (predictions and their interval of uncertainty). Example: Ecological systems One most important problem of our time is the effective use of natural resources, such as air, water, energy a/o. Within the scope of physical-geographical process research, it is necessary to develop effective tools for analysis, prediction and control of NO3-, Cl- discharges, of seeping water, and outflow of drainage for several landscape.

This book will focus on mathematical modelling for analysis and prediction of complex objects. Obviously, mathematical modelling forms the core of almost all decision support systems. However, in economy, ecology, biology, sociology etc. many objects are complex ill-defined systems that can be characterized by: • inadequate a priori information about the system, • great number of unmeasurable variables, • noisy and short data samples and • fuzzy objects with linguistic variables. Related to modelling this means • to realize a systematical holistic approach, • to take in consideration the inadequate a priori information about the real systems and • to describe the vagueness and fuzziness of linguistic variables (qualitative description). For such ill-defined systems the hard systems thinking based on the assumption that the world can be understood objectively and that knowledge about the world can be validated by empirical means need to be substituted by a soft systems paradigm [Keys, 91]. This approach is based on the reflection that humans only can understand the nature of the world incomplete and fuzzy. In this sense, models not reflect the world, but they capture the logic of the world. They are useful constructions that may help to discuss and to consider the world. However, they are not direct representations of the world. Problems of complex objects modelling are: • system identification, • pattern recognition, • function's approximation and extrapolation and • forecasting of random processes and events. They can be solved by deductive logical-mathematical or by inductive sorting-out methods. In the first case, models can be derived from existing theory (theory-driven approach or theoretical 10

Models and their application in decision making

1


systems analysis) considering the fact, however, that any theory itself is a model of the world, already, and was seen from a special viewpoint. Deductive methods have advantages in cases of rather simple modelling problems. Here, the theory of the object being modelled is well known and valid, and it thus is possible to develop a model from physically-based principles employing the users knowledge of the problem (see section 1.3). Using the opposite approach - inductive sortingout methods -, models can be derived from data (data-driven approach or experimental systems analysis; see section 1.4). In many cases, it is impossible to create models by theoretical systems analysis. Here, sophisticated applications of inductive sorting-out methods may be able to reveal carefully guarded secrets from nature. However, they usually cannot solve an analysis of causes of events for such fuzzy objects. A pragmatical solution to this modelling problem may be a unification of both methodologies.

1.2 Relevance and value of forecasts In the previous section we pointed out why it is worth building models. To use them for analysis, identification and control is quite reasonable because humans tend seeking to understand how things work in detail. However, what the value of forecasts can be? More, is it possible to predict complex systems at all? Are all scientific efforts in that direction a kind of fortune-telling? How predictions, if they will have any value, can contribute to better understand and handle our daily problems? All these questions are still discussed in science. One opinion concerns the fundamental limits on formal methods when applied to complex structures like economic or ecologic forecasts. One consequence is that one cannot know which of a set of competing forecasting methods and models will realize the best forecast since experience has shown that different models forecast better different times. So, they conclude, it is reasonable to ask whether it is ever possible to state in advance the conditions in which a theory will be rejected based on predictive failure and whether those conditions will hold in future. Consequently, the theories the forecasting models are build on have to pass a predictive test also. If these tests fail, are theories rejected? If not, why not? So, why should these forecasts be taken anymore seriously than those of the fairground fortune-teller? The contrary opinion formulates that their forecasts are more scientific than fortune-telling because their predictions are based on explicit theory-driven models. The explicit nature of the modelling process and its replicability are key elements that distinguish scientific forecasting from fortunetelling. These elements allow learning to take place and permit theories to gain or lose credibility. Forecast models are a method for processing information in a consistent way that is replicable; the evidence from their predictions sometimes offers support to various theories. Models based on very wide ranging theories will outperform alternatives more regularly than those that are data based approximations. We think that modern information technology along with steadily increasing computational power of today's computer hardware provide a new, third view to the prediction problem “what variables, which method, which model.” This idea is an important part of what we call self-organising data mining. It can be outlined as follows and is described detailed in the following chapters of the book. 1. What variables Choice of appropriate variables is one part of pre-processing and is strongly connected to the initial problem one is seeking models for. Most important here is to find out the most relevant variables for a specific modelling task from a large set of available, pre-selected variables, which in turn form

Relevance and value of forecasts

11


the information source for modelling. Along with some solid theoretical knowledge on what variables are useful for modelling (if available), we suggest to apply self-organising technologies at this stage of modelling, already, to select significant variables or classes of variables more objectively. We call this set of relevant variables a “nucleus.”(see chapter 1.5). 2. Which method The answer to this question is task dependent. However, it is object dependent as well. This means, that a same modelling task - classification or prediction for example - may require different modelling methods for different objects. Successful modelling requires adequateness between object and model concerning their fuzziness and descriptive power. In other words, a modelling method need to reflect an object's uncertainty appropriately. For example, regression is a very good tool to predict almost certain processes like many technical systems. However, one can hardly expect that regression models will does well on fuzzy processes that are contradictory themselves sometimes. Financial markets are an example here (see chapter 6.2). Several automatically working modelling methods have been developed in the past years based on cybernetic principles of self-organisation. They allow to find one or more algorithms appropriated for different modelling tasks and different level of objects’ uncertainty as well. 3. Which model The key fact here is that any model created by any method (scientific or not) can only reflect a method-specific and therefore also freedom-of-choice dependent subset of the true behaviour of the object. This is, because any model can be created on a section of the real-world’s infinity only, and it is the main reason why different models predict better at different times. One solution to increase robustness and reliability of prediction is creating several different models, first, using different modelling methods. Since it is impossible, however, to know in advance which model will best forecast for actual conditions, we suggest a synthesis of all models or predictions to a hybrid solution. It can be expected that this collective solution reflects reality more thorough than any single model can do over a longer time (see chapter 6.4). Self-organisation of models, fast computer hardware, and advanced software technologies make this approach realistic today. Based on multiple application of self-organising modelling, it is possible, and we think also reasonable, to extract reliable knowledge from data more readily, more universal, and more robust that makes forecasting something different from fortune-telling. Systems theory and practice as well have been showing that predictive controlled systems have a less disturbance-sensitive, more stable and adaptive behaviour than systems that work on historical and actual information, only. Extreme, often very costly situations can be avoided in this way.

1.3 Theory-driven approach The theory-driven approach or the approach of theoretical systems analysis is based on known theory of the object being modelled and therefore, the model is derived from physically-based principles. This approach leads to the hard systems method of modelling. The elements of a model are represented by different variables expressed by numerical values. Then, the cause-effect relationships need to be formulated as mathematical equations or their equivalents, which are manipulateable by algebra, computer-based techniques or similar methods. A key assumption of this approach is that it is possible to construct and manipulate a model of the problem under study. The 12

Theory-driven approach

1


analytic process involves down-breaking parts of the world into constituent parts that will be simpler than their aggregate. These smaller parts are then more manageable, and an understanding of the original focus of interest is gained by bringing these separate pieces of knowledge together. The model gives an explanation of the behaviour of the actual processes of the real system. In theory-driven modelling however, the user has to know a priori much about the system itself: She has to describe the essential structure of the original system at least so far as it is necessary for the model purpose. Additionally, the influence structure of the original system must be known and understood and the environmental influences as well. Example: Marketing A company wants to model and predict its product demand. It has recorded several characteristics for past periods (month or weeks) which it expects or knows due to market theory to have an impact on product demand. These variables might be: dollars spent for advertising, number of products in the pipeline, number of products in stock, number of new consumers, consumer confidence, consumer income, and inflation as input variables or environmental influences and number of products sold, profit margin, and gross profit as output variables of the system. The theory-driven modelling approach now forces the company’s research staff to formulate qualitatively 1. the interdependence structure between all three output and all seven input variables and 2. to define the relevant dynamics (time lags) for each variable. This means, e.g., they have to decide, whether the number of products sold is influenced by the dollars spent on advertising or by the number of products in pipeline or by both or by any other possible constellation. Then, the staff has to agree also, for instance, whether consumer confidence of one month, two month or six month ago has an influence on next month products sells. Most important is, however, that the problem is composed of several aspects: global, macro and micro economical, psychological and political. So, the question lastly is: What theory they have to focus on at a given stage, and how the theories can be joined? What are the rules for connecting different theories? Here, marketing theory may be somewhat helpful already. Usually, however, these rules are not known completely. So, since theory-driven modelling relies on them anyway, apparently the research staff members have to make wild guesses several times in their work. Each time they change their assumptions, they may get totally different results. This, of course, is reasonable. However, what or who will decide what the true assumptions are? This is the problem.

This means, for complex, ill-defined systems the modeller a priori has only insufficient knowledge about the relevant theory of the system under study. Theory-driven modelling is affected considerably by the fact that the modeller is a priori uncertain regarding selection of the model structure due to insufficient knowledge about interference factors (actual interference factors and factors of influence that cannot be measured). This insufficient a priori information concerns necessary knowledge about the object such as those relating to: • the main factors of influence (endogenous variables) and classification of variables into endogenous and exogenous, • the functional form of the relation between the variables including the dynamic specification of the model, and • the description of errors such as their correlation structure. Therefore, the comprehensive application of theory-driven approach in practice and theoretical systems research is hampered by several essential problems linked to modelling. These are, for example: a. The formation of models requires much qualified scientific work. The elaboration of the model of the economic development in the FRG was made by two to three scientists and some staff


13


members working permanently from 1962 up to 1968 [Krelle, 69]. b. The different scope of knowledge about the subject of study forces the modeller to an arbitrary approach to include specialised scientific facts into a mathematical description (selection of influence, necessary complexity of the model etc.) and therefore it influences the uncertainty of the results. The tacit hope that an actual process will continue to go as presupposed in the assumptions is, seriously taken, not at all justified. The effect of simplification of models is seen in basic problems of classical econometrics and simulation, and it implies to make various assumptions, which are often resulting in a considerable shortage regarding the models' validity in reality. c. The parameter identification of systems of equations is linked to considerable difficulties and causes the modeller to study the conditions for that identification. d. The model-adequate estimate of the unknown model coefficients of parametric models is based on traditional statistical statements and methods within the framework of mathematical statistics, and it is linked to various assumptions that cannot be controlled in practice (or only with great difficulties). Here, typical examples are the assumption of normally distributed noise or the independence of observed values. Additionally, these preconditions have to be considered when interpreting the results of estimation. e. For any required model quality, a specific object can be described commonly by various mathematical models with a sufficient same degree of preciseness so that it is up to the modeller to select the one or the other model. However, what will be the criterion for that choice? Random, sensitivity? f. The differential equation or its difference analogy is a very good model for the description of deterministic behaviour as found within a large class of systems. However, only a subset of variables of real-world systems show deterministic behaviour and can be described by this model. Therefore, from a more general perspective, it is necessary to describe fuzzy objects. g. Traditional mathematical modelling is characterized by the stigma that modelling means simplification, isomorphic relation and one-sided reflection and that, on the other hand, mathematical modelling needs exact description. h. Finally, there is the Zadehs principle of incompatibility: „As the complexity of a system increases, our ability to make precise and significant statements about its behaviour diminishes until a threshold is reached beyond which precision and significance (or relevance) become almost mutually exclusive characteristics . . . A corollary principle may be stated succinctly as: the closer one look at a real-world problem the fuzzier becomes its solution.“ [Kosko, 94] To solve the problems mentioned above, a considerable amount of specialised-scientific systems analysis, modelling, mathematical-statistical and computational work is necessary. This causes the need to extent the arsenal of well-proven principles of modelling by new and appropriate procedures. Further development of modelling has also to contribute to improve the epistemological function of the models. This function is limited due to its links to the subjective ideas of the modeller, the uncertainty of a priori information and the two-stage model formation (a. model structure identification, b. parameter estimation). Additionally, it is necessary to reduce the high share of time modelling has at the solution of tasks (according to [Shannon, 75] it is 75-90 per cent) by means of a computerised draft of mathematical models. Computer aided design of mathematical models may soon prove highly valuable in bridging that gap. A variety of computer based ideas has 14


1


already been developed in the past for improving the computing instruments in this application domain supporting modelling and simulation in this way. A possible trend of further development is based on theoretical systems analysis and leads to simulation. The methodology of simulation (algorithmic description) is developed successfully on the level of mathematical modelling and on the level of computerised programming. As a result, special simulation languages have been developed to support the application of models on a computer as executable programs [Banks, 98]. The simulation languages provide simulation specific or even application specific language constructs for this purpose. The system-dynamic-access for modelling the dynamics of existing economic processes as applied by Forrester, for example, has proven its value ([Ruth, 97]). In this context, endeavours are being made to elaborate models that are as closed as possible. It results in applying hypothesises on the functional dependence of the system values that are partly proved only to a small extent. Problems linked to this conception in case of insufficient a priori information can be solved developing open models with scenarios as input values. One most important task in further development of modelling tools is to develop the basis of program technology. The purpose is to establish highly qualified program systems with a maximum degree of automation at the stages of construction, solution and analysis of models. The design of comfortable, problem-oriented user interfaces is primarily aimed at an improved usability of the computing tools ([Ruth, 97]). The extent of knowledge required for a successful operation of a simulation study is a central issue. Next to a sound knowledge in the application domain itself, fundamental methodological knowledge in modelling and simulation is essential in particular. Experience gained from using such theory-driven tools has shown: A dialog-assisted model selection helps users who seek access to the process of model selection, who have time for setting up several dialog windows and want to have some information about the inherent logic of movement of the process investigated, besides good results concerning selected items. However, there are several problems generated mainly by the reduced expression capability of models, which can be characterized as problems of intelligibility, modifiability, credibility and efficiency [Klahr, 80]. The literature shows several ways to limit the involvement of users in the model building process. Klußmann, e.g., describes a software prototype that will make the generation of simulation models more easy and faster by extracting as much input information as possible from existing data sources of a company (CAD systems, MRP systems) [Klußmann, 95]. The data will be transferred through a neutral data model that gives the possibility for mainly automatic generation of simulation models such as material flow systems. In [Heeg, 98] is described an application of this approach in bid preparation characterized by a shortage of information. An automatic generation of simulation models enables a wide use of simulation in the critical time dependent bid preparation process. Practical applicability of modelling techniques and tools is significant for their extensive and various supply at user support. The user is normally interested in the solution of the initial problem and may have no expert knowledge about deductive mathematical modelling. Knowledge based activities in modelling like qualitative and quantitative formulation of tasks, systems analysis, selection of model structures and estimation procedures, validation, planning of experiments and interpretation of results, must be supported by methods of artificial intelligence. The following enumeration includes potential problem areas about simulation studies that can be avoided by some additional knowledge adapted from artificial intelligence [Häuslein, 91]: • utilisation of the computing instrument, • selection of appropriate simulation methods and languages, • selection of existing models and model components from a model base, • instructions for modelling,


15


• design of simulation experiments, • running simulation experiments, • analysis of simulation outputs. Nevertheless, efforts of using tools of artificial intelligence in this way were not successful in many cases in the past years [Müller, 90]. That is, because methods of artificial intelligence are based on knowledge extraction of human skills on a subjective and creative domain - modelling (which is called also knowledge engineering bottleneck). A support by conventional or knowledge-based methods cannot compensate a lack in competence in the simulation domain. Also, there is a lack between the state of knowledge in the related scientific fields and the methodological capabilities in modelling and simulation. Many scientific perceptions cannot be considered in the models because they cannot be represented adequately in a notation of simulation language. Additionally, in this way it is not possible to solve the significant problems of modelling for complex systems as mentioned before: • inadequate a priori information, • great number of unmeasurable variables, • noisy and short data samples, • ill-defined objects with fuzzy characteristics.

1.4 Data-driven approach As outlined in the previous section, problems of complex object modelling like identification, pattern recognition, function's approximation and extrapolation and forecasting of random processes and events cannot be solved by deductive logical-mathematical methods. Here, knowledge extraction from data, i.e., to derive a model from experimental measurements, has advantages when a priori only little knowledge or no definite theory is on hand. This is particularly true for objects with fuzzy characteristics. The data-driven approach generates a description of the system behaviour from observations of real systems evaluating how it behaves (output) under different conditions (input). This is similar to statistical modelling and its goal is to infer general laws from specific cases. The mathematical relationship that assigns an input to an output and that imitates the behaviour of a real-world system using these relationships usually has nothing to do with the real processes running in the system, however. The system is not described in all of its details and functions. It is treated as a black box. The task of experimental systems analysis is to select mathematical models from data of N observations or cases and of M system variables xit , i=1, 2, ..., M, t=1, 2, ..., N, to select the structure of the mathematical model (structure identification) and to estimate the unknown parameters (parameter identification). Commonly, statistically based principles of model formation are used that require the modeller to have a priori information about the structure of the mathematical model available. There exist many papers about identification of complex systems using this approach [Isermann, 88; Cherkassky, 98]. A good deal of work goes into identifying, gathering, cleansing and labeling data, into specifying the questions to be asked for it, and into finding the right way to view it to discover useful patterns. Unfortunately, this traditional processing can take up a big part of the whole project effort. Obviously, methods of experimental systems analysis cannot solve an analysis of causes of events for such fuzzy objects. There are severall important facts that have to be underlined. First of all, the goal of data-driven modelling is to estimate the unknown relation between output (y) and input (x) from a set of past observations. Very important is the fact that models obtained in this way are only able to represent a relation between input and output for which the observed samples

16

Data-driven approach

1


of input and output have its values. Secondly, many other factors that are not observed or controlled may influence the system's output. Therefore, knowledge of observed input values does not uniquely specify the output. This uncertainty of the output is caused by the lack of knowledge of the not observed factors. It results in statistical dependency between the observed inputs and outputs [Cherkassky, 98]. Thirdly, there is a difference between statistical dependency and causality. Cherkassky [Cherkassky, 98] has underlined that the task of learning/estimation of statistical dependency between (observed) inputs and outputs can occur in the following situations or any of their combintions: • output causally depend on the (observed) inputs; • inputs causally depend on the output(s); • input-output dependency is caused by other (unobserved) factors; • input-output correlation is noncausal. It follows that causality cannot be inferred from data analysis alone. Instead each of the 4 possibilities or their combinations is specified, and, therefore, causality must be assumed or demonstrated by arguments outside the data [Cherkassky, 98]. With insufficient a priori information about the system, there are several methodological problems one has to focus on before applying data-driven methodologies. Besides those mentioned in section 1.3, the incomplete - since finite - data basis we always use leads to an indeterminacy of the model and the computational data derived from it. Also, the effectiveness of different conclusions drawn from this data by means of mathematical statistics is limited. This incompleteness of the theoretical knowledge and the insufficience of the data basis causes the following problems in detail: a. The model-adequate estimation of the unknown coefficients of parametric models is commonly based on traditional statistical approaches and assumptions. Statistical analysis includes some useful hypothesis tests. However, they can, if they can, only be verified in real life practice with large efforts, and they also have different importance. Typical assumptions about stochastic behaviour of the data are less important than those about the structural behaviour. b. According to the set principle of modelling, there can exist many models with a sufficient same degree of adequateness for a given sample of input and output observations. Therefore, the task of selecting a model from an existing data sample of endogenous and exogenous variables is, according to Tichonov [Tichonov, 74], an ill-posed task. It is not possible, generally, to select an optimal model from the quantity of possible models without some additional, socalled external information. For example, the regression learning problem is ill-posed in case of absence of any assumption about the nature of the continuous valued approximating functions. For limited training data, the solution that minimises the empirical risk is not unique. There is an infinite number of functions from the class of continuous functions that can interpolate that data points yielding the minimum solution with respect to a given loss function. For noise-free data, one of these solutions is the target function, but for noisy data this may not be the case [Cherkassky, 98].


17


Simple numerical example Consider the following data set : y 9 9 9 9 9 9 9 6

a 1 2 3 4 5 6 7 99

b 8 7 6 5 4 3 2 1

c 1 2 3 4 5 6 7 5

The variable y was generated by the equation y = b + c . The testing question here is whether model selection from data (here: self-organising modelling described in chapter 4 and 5) can identify this relation explicitely when using y as output variable and a, b, c as input variables. In result of self-organising modelling this model with a zero error variance was selected as the „best“ one: y = 9.3 - 0.033a - 0.033b. From this result one may conclude that the algorithm fails identifying the „true“ relationship. When evaluating the created model, however, the model will prove as a valid solution. Looking at the second and third best self-organized models y = 0.00001 + b + c y = 9 - 0.0319a + 0.0319c which have a zero error variance too, apparently there are more than only one „true“ model for this data set. Here, it is due to collinearities between the variables, but this effect is true for all ill-defined problems. Only inclusion of external information can generate regular results.

c. Considering time series, most of them contain a growing trend component. With modelling based on these instationary time series, the danger emerges that relations between different variables with a similar growth trend will be established, although these relations do not exist in reality. Collinearities between the predictor variables confuse the interpretation of the associated parameters, but can also be harmful to predictions. d. Generally, dynamic systems are characterized by various growth processes. Therefore, differential equations obtained from observed growth processes are, as a rule, not stable. Utilisation of unstable models or models that include unstable partial models is very dangerous, because a small deviation in the initial conditions will cause much larger deviations in the model results. e. Modelling is based on the assumption that functional relations are of relative constancy over the evaluated period. To satisfy this requirement, short time series have to be applied. This means, the modeller has to meet contradictory requirements when establishing mathematical models. Due to the limited quantity of data the uncertainty of estimation increases with many model parameters. On the other hand, however, reality is reflected more convincingly with growing model complexity since reality is complex. Example: Analysis and prediction of drainage outflow Analysis and prediction of a drainage outflow of selected parameters of the meteorological regime, the upper soil horizon, soil properties a.o. [Müller, 96] have shown that models generated on observations of all four seasons of the year are not able to analyse and predict the daily, weekly or monthly processes. Such models

18


1


describe general laws of the seasons, and therefore, generalize differently. Only specific models generated separately on observations of spring or summer are able to reflect the distinct behavior of each period.

f. To judge the quality of models merely by formal criteria like the closeness of fit of model and true system is doubtful. Instead, it is necessary to have a purposeful judgement of the quality of model adaptation based on the suitability of the model to solve a predefined task. Transforming the requirements on a model to an adequately formalised criterion will, however, produce considerable difficulties. g. A rule for parametric models is that the number of unknown model parameters must be smaller than the number of observations. However, complex systems require to measure many systems variables since the necessary dimension of the state space which the system trajectory will be completely described in without redundancy is commonly not known. On the other hand, the number of observations cannot be extended infinitely, because many economical and ecological systems, for example, are characterized by a strongly restricted set of available observations. If the number of predictors is large, one problem is that traditional modelling methods quickly become ill-behaved and computational unmanageable due to the large number of parameters that have to be estimated simultaneously. If the number of systems variables in a data sample is larger than the number of observations, the modelling task is called an under-determined task. Such under-determined tasks can be solved by means of inductive selection procedures that will be described in this book. h. Nalimov [Nalimov, 83] has suggested that it is not possible for complex systems to distinguish between dependent stochastic variables and independent deterministic variables. These variables, as a whole, represent a fixed pattern in which each variable is connected to all the others. They are interwoven and it is not possible to tell cause (reason) from effect (action). To solve this collection of possible problems, it is necessary to develop appropriate data-driven tools for automatic modelling. Most users' primary interest is in results in their field and they may not have time for learning advanced mathematical, cybernetic and statistical techniques and/or for using dialog-driven modelling tools. Knowledge discovery from data and in particular data mining techniques and tools can help humans in analysing the mountains of data and in turning information located in the data into successful decision making.

1.5 Data Mining The idea of extracting valuable information from data is not new. In section 1.4 was already underlined that „the aim of statistical inference and inductive modelling is to infer general laws from specific cases - to summarise observations about a phenomenon into a coherent model of the underlying data-generating mechanism.“ [Elder, 96]. However, there are two circumstances that give a new importance to this methodology. 1. Modern information technology, wholesale computerisation of business transactions, and distributed computing delivers a flood of data to decision makers, and there is a question how to leverage them. This has changed the application of information technology in business and other fields. Business data is seen as a valuable commodity in its own right, it represents the current state of business. In combination with historical data, they contain information about Data Mining

19


the whole business situation, the potential developments, changes and possibilities. Today, this data are stored in data warehouses, which are “simply a single, complete, and consistent store of data obtained from a variety of sources and made available to end users in a way they can understand and use in a business context” [Devlin, 97]. Only those who can extract the nuggets of business information under conditions of competition will gain an advantage in their business and decision making. 2. Traditional reporting tools and statistically based query tools help the manager only if she knows, already, what she is looking for. Commonly, queries are very complex, and the user already has to have an imagination about relationships in the data when he wants to accumulate all of the information together to check or validate a hypothesis by several statistically based tools. In the past ten years there has been developed many of powerful, specialised software for viewing, exploring and analysing large volumes of data (e.g., the SAS system). If data are not suitable for visual exploration, there is an alternative to summarize data using established statistical methods such as factor, cluster or correspondence analysis. Modern statistical modelling tools make it possible for an analyst to think about the problem at a higher level. Data Warehouses manage multidimensional data bases and make them available for online retrieval. Online analytical processing (OLAP) tools are enhancing data analysis capabilities by providing • consolidation upward and drill down in many dimensions, • dynamic viewing and analysing data from many perspectives and through complex formulaes on large data volumes [Devlin, 97]. They help organisations to answer specific questions. However, they can only deliver a partial view of the object and do not inform about general tendencies in the data. Today, there is a significant need to discover information non-obvious and valuable for decision making from a large collection of data. This information includes, for example, new key facts, general tendencies and relationships, significant and useful patterns of information. Extracting this information requires a new generation of techniques and tools that are able to help humans intelligently and automatically analysing both large and small data sets. These techniques and tools are subject of data mining, and whithin knowledge discovery they have to turn information located in the data into valuable information for successful decision making. Knowledge discovery from data, and, specifically, data mining as its heart, is an interactive and iterative process of solving several subtasks and decisions like data selection and preprocessing, choice and application of data mining algorithms, and analysis of the extracted knowledge. SAS Institute, Inc. , e.g., has formulated the data mining process as a five-step process, called SEMMA: sampling, exploration, manipulation, modelling and assessment. IBM Corp. has another interpretation of the data mining process, and other companies may have their own terms as well. An overview of possible steps comprising knowledge discovery from data, according to Fayyad et al. [Fayyad, 96], is shown in figure 1.1. Data mining itself includes not only a straightforward utilisation of a single analytical technique. It consists of processes which many methods and techniques are appropriate for depending on the nature of the inquiry. This set of methods contains data visualisation, tree-based models, neural networks, methods of mathematical statistics (clustering and discriminant analysis, e.g.), and methods of artificial intelligence. Also included in the spectrum of modelling methods are methods of knowledge extraction from data using self-organising modelling technology that will be described in the following chapters of this book.

20

Data Mining

1


Fig.1.1: Knowledge discovery from data [Fayyad, 96]

Principle tasks of knowledge discovery and data mining are (see also [Fayyad, 96]): 1. Data selection and preprocessing, 2. Choice and application of data mining algorithms and 3. Analysis of extracted knowledge. 1. Data selection and preprocessing 1.1 Task definition In this first step, it is necessary to decide the goal of the modelling process, the objectives of the end user, the actual conditions that limit the possibilities of modelling, and to formulate a priori relevant information about the application domain. Any decision task is not a formal task but a task of a special field of science. Therefore, it is advisable to include the information about the system into modelling that is well known a priori, i.e., the knowledge about input /output relations, structural information and causal relations, and knowledge accumulated in systems research by modelling large scale systems (e.g., analysis of structure, stability, sensitivity and diagnosis of multicollinearity). A hardest task has been the normalisation of common knowledge which, in contrast, is nearly transparent in human reasoning. 1.2 Data selection Often, limitations of the data itself, which are rarely collected for knowledge extraction, are the major barrier for obtaining high quality knowledge. It is necessary to decide, which data are important for the task we are trying to realise. Not all available information is usable for data mining. A detailed understanding of the structure, coverage, and quality of the information is required. Therefore, it is necessary to preselect a set of variables from the available information that might have an impact on the users’ decision making process and that can be observed and measured or transformed in a measurable numerical value. In practical applications, it is impossible to obtain complete sets of variables. Therefore, the modelled systems are open systems, and all important variables that are not included in the data set (for what reason ever) are summarised as noise. Data Mining

21


1.3 Data reduction A next step of high priority is the objective choice of essential variables from all preselected ones. For under-determined tasks (insufficient a priori information), it is apparently necessary to identify a subset of linear independent variables and, by their linear combinations, to express all other factors of interest. This task is equivalent to the analysis of observability and controllability of a given dynamic system, i.e., to find out the necessary dimension of the state space in which it is possible to describe the system trajectory without redundancy and to transform the given system from a given initial state to the target state in the state space. To solve this task, it is necessary to find out the number and the concrete assortment of independent variables. One way to realise this structure identification is creating a model nucleus [Ivachnenko, 92]. A nucleus represents a subset of significant variables that specifies the essential dynamic behaviour of the entire system. All remaining variables are a static or dynamic derivation of the nucleus, only. A nucleus can be obtained in several ways. Example: Macroeconomic modelling Figure 1.2 and 1.3 show exemplary results of cluster analysis and principal components analysis considering 12 macroeconomical variables of the GDR economy [Ivakhnenko, 92]. Both figures display several classes of variables from which a nucleus can be selected by picking a representative variable from each class. Once defined, the nucleus can be dynamically modelled by means of input-output structures like systems of simultaneous equations.

Fig. 1.2: Cluster analysis considering 12 macroeconomical variables of the GDR economy

22

Data Mining

1


Fig 1.3: Results of principal components analysis considering 12 macroeconomical variables of the GDR economy Example: Financial forecasting Based on price data (August 30, 1996 - January 9, 1998) of 53 stocks (the 30 stocks composing the DAX stock index and 23 other national and foreign equities), the following nucleus of 10 variables was selected by means of cluster analysis : nucleus: BMW, Commerzbank, SAP, VEBA, Nestle, IBM, DIT GB, DIT Geldmarkt, DAX, Dow Jones. Fig. 1.4, for example, shows a clustering tree from which the BMW stock was selected as a representative of all other equities of this tree. In figure 1.5, the corresponding equity curves of the sub-clusters C1, ..., C5 and the entire cluster C are displayed. It is obvious that they show a very similar behaviour for the evaluated period allowing to pick a single asset as a proxy for all assets. Table 1.1 lists the mean errors of long-term predictions of a forecast horizon of 5, 10 and 30 days for two portfolios. Portfolio P1 consists exclusively of assets of the nucleus, while portfolio P2 has included also other assets: Portfolio P1 : BMW, SAP, IBM, Nestle, Dow Jones Portfolio P2 : Bayer, SAP, Thyssen, VW, IBM. The predictions were obtained by means of models generated a. on all 53 variables (m a) b. on 10 variables of nucleus (m b ). In portfolio P2 , the models of those variables which are not included in the nucleus were obtained by means of dynamic models of variables of the nucleus (dynamic derivation of the nucleus). c. on variables of the corresponding portfolio (mc), exclusively.

Data Mining

23

J.A. Mueller and F. Lemke The time lag was T=1. The results of table 1.1 show that the best predictions are gained using the nucleus. P1

P1

P1

P2

P2

P2

prediction period

5

10

30

5

10

30

ma mb mc

2.2 1.8 2

2.8 2.4 2.7

5.8 7 6.2

2.7 2.4 3

3.2 2.6 3.5

4.6 6.6 6.8

Table 1.1: Mean long-term prediction error (MAD [%]) for two portfolios

BMW

Thyssen

Siemens

Lufthansa

C5

Linde

Karstadt

Hoechst

C4

Escada

Henkel

C3

Daimler

Dt. Bank

BMW

C2

Bayer

BASF

Allianz

C1

Fig. 1.4: Clustering tree from which was selected BMW as a representative Table 1.2 compares the mean prediction errors of portfolio P2 obtained for different models of those variables that are not included in the nucleus. They were modelled as linear/nonlinear static and linear dynamic derivations of the nucleus. The maximum time lag for the nucleus and the dynamic derivations were identical (T=1, 5, 10). Here, results show that dynamic models did best, while an increased time lag did not end up in smaller prediction errors, because the applied self-organising modelling algorithm avoided overfitting (chapt.3 and 4).

prediction period linear nonlinear dynamic

T=1 5 3.8 3.8 2.4

10 4.5 4.5 2.6

30 7.8 7.8 6.6

T=5 5 5.3 3.5 2.4

10 6.3 4.5 3.3

30 7.2 7.4 5.2

T=10 5 5.2 2.8 2.8

10 6.1 3.7 3.1

30 7.4 8.4 6

Table 1.2: Mean long-term prediction error (MAD [%]) of portfolio P2

24

Data Mining

1


Fig. 1.5: Subclusters C1, ..., C5 and cluster C of normalized variables (x-µ)/σ

The GMDH algorithm, which is described later in this book, also provides an efficient way for selecting essential variables by linear model construction. Here, the selected models suggest an ensemble of variables (leading variables), which we can expect the most consistent pattern analysis for. To obtain a mathematical description of the complete system, it is necessary to describe all remaining variables as a function of the leading variables. This sampling into representative classes of variables will significantly reduce redundance, and thus, will reduce the time required to get crucial information from data mining. Example: Financial forecasting Using price data of the 53 equities above and a maximum time lag of one, a dynamic system of equations was generated by GMDH. The obtained structure is shown in figure 1.6. Utilizing the known algorithms of structure analysis, the sequence of evaluation of all variables can be created (fig. 1.7). A1 (x51 , x41 , x25 , x52 ) and A2 (x47 , x 44 , x 40 , x 50 , x 28 , x 27 , x 37 ) are interdependent equations.

Data Mining

25


Fig. 1.6: Structure of the system of equations (

- portfolio P1/P2)

Fig. 1.7: Structure analysis: sequence of evaluation of variables

1.4 Data cleaning Any practical data samples are raw data, and therefore, they must go through a process known as data cleaning. In business transactions, for example, not all operational transactions are correct. They might contain inaccurate values. Data cleaning realises basic steps of statistics to remove noise or outlier, handle missing data fields, or other inconsistencies in the data. Visualisation can be used to identify outliers or out-of-range data easily. Special application programs can help to check the data and to detect worse data, and missing or incorrect values can be set to neutral values using statistical information. Example: Financial analysis As an example for data cleaning, let’s have a look at financial price data again. Figure 1.8 shows the equity curves of Microsoft and the DIT-Geldmarktfonds portfolio, where sudden changes are recognizable.

26

Data Mining

1


stock

before

Microsoft DIT-Geldmarktfonds

February 20 February 13

after 284 51.61

February 23 February 16

144 50.31

Without any background information, such sudden changes in the equity curves may be interpreted as significant price drops. The truth is, however, that these anomalies are caused by exceptions requiring data cleaning: Microsoft: Microsoft has introduced a stock split, and its share holders got two new shares for an old share. Therefore, all prices until February 20 have to be divided by two to reflect the new price level. DIT Geldmarktfonds: At February 16, 1998 all share holders got a dividend of 1.30 DM/share added to the share volume immediately by 0.02519 shares. Therefore, the prices before February 16 must be divided by 1.02519.

Fig. 1.8: DIT-Geldmarktfonds and Microsoft equity curves

1.5 Data transformation The enhancement of clean and selected data sometimes involves generating new data items for one or more variables by combining two or more variables. This method is realised by some extension of an input data sample by adding new columns of generalised variables. But an increase in the regressor number can perfect regression only. This can be done by calculating the sum, the product or difference of two variables, their ratio, or other simple nonlinear functions of the initial variables. For some given variables x1, x 2, ...,x M , this extension could be:

Data Mining

27


v 1 = x 1 x 2 , v 2 = x 1 x 3 , v 3 = x 2 x 3 , ..., v g =

1 1 1 , vg +1 = , vg +2 = , ..., x1 x2 x3

Another kind of transformation could be turning a date into a day of the week or day of the year. Example: Analysis and prediction of drainage outflow [Müller, 96] Models created for periods in which the drainage outflow was not always essentially different from zero were especially influenced by threshold values. They were considered for air temperature and for soil temperature in 5 cm and 20 cm depth in the following manner: z = x sign(x-s), where sign a =

{

1 x≥0 with s assuming the threshold value. 0 x 1, it is possible to use the mean value

110

Analog Complexing

5

¯l i =


1 k 3 k + 1 j=0

M

3 (x r , i + j − x r , N − k + j ) 2 .

r=1

If time-series are nonstationary and if there is no transformation used, Motnikar et al. [Motnikar, 96] propose a two-stage comparison algorithm. They define and distinguish between two types of pattern characteristics: eliminating and ordering. Each characteristic should have an intuitive logical meaning and a simple analytical expression. Based on experience, they define three eliminating characteristics: local extremes, trend and season and four ordering characteristics: mean, standard deviation, rank and season. For each ordering characteristic, a distance measure was defined separately [Motnikar, 96]. According to this, in a first stage all patterns are ranked by an eliminating factor De according to their difference in the eliminating characteristics. In a second stage, the ordering factor is calculated, where the distances δi between the ordering characteristics are weighted automatically or by the user’s judgement about their importance for explaining the characteristics of a given timeseries: Σ = gi δi . As a similarity measure, dissimilarity D is used

D=

3T , De

where T is a time factor that favours patterns that are chronically close to the reference pattern. Years

61-64 62-65 63-66 64-67 65-68 66-69 67-70 68-71 69-72 70-73 71-74 72-75 73-76 74-77 75-78 76-79 77-80

number of main components

selected

1

2

3

4

4.979 1.955 1.627 5.657 0.475 2.573 5.023 5.282 4.486 1.028 5.211 5.259 4.128 5.148 4.48 3.315 3.882

15.667 13.443 9.345 10.462 3.114 12.253 16.583 16.026 12.962 12.155 9.494 7.294 14.455 15.803 10.178 3.532 8.548

23.256 18.826 9.956 14.591 10.888 19.045 20.376 16.242 19.825 18.466 10.955 14.408 17.488 17.469 15.692 8.081 16.257

27.594 23.236 13.605 15.962 15.606 23.015 22.546 17.363 24.528 23.21 15.566 15.335 21.316 20.238 16.295 9.604 16.548

analogs

* * *

* *

*

Table 5.1: Mean squared error, reference pattern: 1981-1984 (4 years)

Analog Complexing

111


Finally, a pattern is inserted into the set of similar patterns when the dissimilarity D is smaller than a threshold value Dmax chosen a priori by the modeller. In our procedure, where a transformation is used, the total sum of the squares obtained by the least squares estimation of the unknown transformation parameters (step 2) provides the information on the similarity of two patterns. Table 5.1 lists the sum of the squares σ for a pattern length of four obtained on a data set of a national economy (12 variables and 25 observations). Up to four main components were used. The results have shown that a reduction of the dimension of the state space (number of variables) by an analysis of main components has only a relative effect on the selected best analogues. With growing number of main components, the same six analogous patterns were selected (*).

b. Selection For the given reference pattern, it is necessary to select several, most similar patterns Pk (i), i ∈ J and their combination to a forecast. The selection task is a four-dimensional problem with the following dimensions: I. set of variables used, II. length k of the pattern (number of observations forming the pattern), III. number of analogues selected and IV. values of the weight coefficients the patterns are complexed with Here, the first two dimensions of selection are considered. The other tasks are covered in step 4. I. set of variables used The first task of finding essential variables objectively is a common problem for all data mining tasks and was considered in more detail in section 1.5. Experience has shown that the effect of data reduction or the generation of a nucleus depends from data. Not necessarily, a generation of a nucleus or inclusion of other variables will give better results. Example: Financial forecasting The 53 equities introduced in chapt. 1.5 were used to generate predictions by means of Analog Complexing for two portfolios: Portfolio P1: BMW, SAP, IBM, Nestle, Dow Jones Portfolio P2: Bayer, SAP, Thyssen, VW, IBM. The predictions were obtained from models generated a. on all 53 variables (ma), b. only on the 10 variables of the nucleus (mb ) (portfolio P1) , c. on the variables of the portfolio (mc), d. only on the variable itself (md).

112

Analog Complexing

5

prediction period pattern width: 5-20 ma mb mc md pattern width 5-10 ma mb mc md


P1

P1

P1

P2

P2

P2

5

10

30

5

10

30

2,1 2,2 2,1 1,7

2,5 2,8 2,9 2,5

7,6 7,8 7,4 4,8

3,2

3,9

6,2

3,3 3,3

4,0 3,8

7,2 6,8

2,3 2,5 2,5 1,8

2,3 2,8 2,9 2,5

7,0 7,0 7,2 7,2

3,0

3,6

6,4

3,8 3,3

4,1 4,0

6,8 6.8

Table 5.2: Mean long-term prediction error (MAD [%]) for two portfolios and different sets of considered variables Table 5.2 contains the mean long-term prediction errors (MAD [%]) for forecast horizons of 5, 10, and 30 days. Here, but also for other portfolios for other prediction periods, the best results were obtained if the model was created only on the variable itself.

With Analog Complexing, Ivakhnenko proposes the following procedure [Ivakhnenko, 91]. Using a „sliding window“, the F best patterns with a length of one are selected. Then, one step forecasts can be calculated, and the corresponding error on all data is: ∆ x ij = xˆ ij (B ) − xˆ ij (A 1 , A 2 , ..., A F ) , where i position of sliding window for the j-th variable, B - reference pattern, and A1 ,A2 , ..., AF analogous patterns. For the complete data set, the error of the j-th variable is 1 N−1 CVj = 3 ∆ x ij . N − 1 j=1 This can be used as a measure of efficiency of the j-th variable on the mean value 1 M CV = 3 CVj . M j=1 Now, the most efficient variables can be selected when ranking all variables according to their CVj .

Analog Complexing

113


II. length k of the pattern (number of observations forming the pattern) An important, but unsolved task is choosing an optimal pattern length. The residual variance s ∗k 2 =

k σ2 , k−F k

with k - length of pattern, F - number of patterns used for prediction and σ2k - mean squared error, explains the relation between the minimum of s*2k and the minimal prediction error MAD pretty good. Therefore, the minimum of s*2k can be used to detect the optimal pattern length. Fig. 5.6 shows this dependence of MAD and s*2k (MSE*) from pattern length k for the DAX and DOW JONES indexes. Motnikar [Motnikar, 96] suggests three alternatives for finding out an appropriate pattern length k. They enable the analyst to choose the pattern length so that • it minimises the chosen criteria’s function (such as mean absolute error (MAE), mean absolute percentage error (MAPE) or mean squared error (MSE)) of a one-step forecast on a chosen validation data set, • it is equal to the maximum significant lag in the time-series’ partial auto-correlation function, • it is equal to the mean distance between the neighbouring turning points.

Fig. 5.6: Optimal pattern length

4. Combining forecasts Because of the selection procedure, there are F most similar patterns Pk (j), j ∈ J with a length of k observations. Each selected analogue has its own continuation giving a forecast when transformed to the reference pattern. Therefore, F forecasts are obtained needed to be combined to one forecast in some way. III. number of analogues selected First, it must be decided which of the F analogues in which combination are most effective to form a forecast. The two special cases, F=1 pattern and F=N-k patterns, set the limits for finding 114

Analog Complexing

5


the optimal number. One way to get a solution is experience. Based on heuristic tests, Balzer [Balzer, 76] suggests for meteorological problems, for example, to select 4..10 patterns. Later he found out that the best results were obtained when using 35 patterns for summer periods and 20 patterns for winter periods [Balzer, 84]. IV. values of the weight coefficients the patterns are complexed with A second problem is combining several patterns to one pattern as a forecast. Here again, different solutions are known. Balzer [Balzer,84] proposes for meteorological tasks, e.g., to use the arithmetic mean for several variables like the minimum and maximum temperature, while for other variables like precipitation (rain) the geometric mean and for the relative duration of sunshine the median would be well. Generally, linear combinations of two analogues can be used Y(B) = λ 1 Y(A1 ) + λ 2 Y(A2 ) with λ 1 + λ 2 =1 , or Y(B) = (1-λ)Y(A1 ) + λY(A2 ), where 0 < λ < 1, A 1 , A 2 - first and second analogue of the reference pattern B, λ -parameter, and λ = 0.1 , 0.2, ... , 1.0. According to this, for 3 analogues we obtain: Y(B) = λ 1 Y(A1 ) + λ 2 Y(A2 ) + λ 3 Y(A3 ) with λ 1 + λ 2 + λ 3 =1 , or Y(B) = (1- λ 2 − λ 3 )Y(A1 ) + λ 2 Y(A2 ) + λ 3 Y(A3 ) with 0 < λ 2 + λ 3 < 1 and λ 2 : 0.1, 0.2, ..., 0.9, 1.0 , λ 3 : 0.9, 0.8, ... , 0.1. To reduce the possible values of λ, the initial values for the search procedure can be calculated by

λ2 =

l −2 2 l −1 2 + l −2 2 + l −3 2

, λ3 =

l −3 2 l −1 2 + l −2 2 + l −3 2

,

where li - mean distance between B and Ai . A very effective approach is embracing the task of detecting the unknown number of analogues and the task of estimating the unknown weight coefficients to form the best prediction Analog Complexing

115


xN+i={x 1N+i , x2N+i , ..., xmN+i}, i=1, 2, ..., τ, where τ -forecast horizon, of the m systems variables. This can be resolved by a linear combination of the continuations of the selected analogous pattern, i.e., x N + i = g 0 + 3 g j x Tj + k + i , i = 1, 2, ..., τ . j 0J

When using a parametric selection procedure like GMDH, the most relevant patterns and the unknown parameter g0 , gj , j ∈ J, will be detected automatically. The only problem here is the small number of observations (pattern length k). Therefore, for very small pattern lengths (k 120 mg/dl) (1 = true; 0 = false) x7 - restecg: 19 restecg: resting electrocardiographic results -- Value 0: normal -- Value 1: having ST-T wave abnormality (T wave inversions and/or ST elevation or depression of > 0.05 mV) -- Value 2: showing probable or definite left ventricular hypertrophy by Estes' criteria

... From other Fields

217


x8 - thalach: thalach: maximum heart rate achieved x9 - exang: exang: exercise induced angina (1 = yes; 0 = no) x10 - oldpeak: oldpeak = ST depression induced by exercise relative to rest x11 - slope: slope: the slope of the peak exercise ST segment

x12

-- Value 1: upsloping -- Value 2: flat -- Value 3: downsloping - ca: ca: number of major vessels (0-3) colored by flourosopy

x13 - thal: thal: 3 = normal; 6 = fixed defect; 7 = reversable defect x14 - num: diagnosis of heart disease (angiographic disease status) -- Value 0: < 50% diameter narrowing -- Value 1: > 50% diameter narrowing (the predicted attribute) For the Cleveland data set also information related to costs were available. The cost is in Canadian dollars and the cost information is from the Ontario Health Insurance Program's fee schedule. The costs here are for individual tests, considered in isolation. When tests are performed in groups, there may be discounts, due to shared common costs. The data sets contain many missing values, which are marked by a -9. attribute costs [Can$] age: 1 sex: 1 cp: 1 trestbps: 1 chol: 7,27 fbs: 5,2 restecg: 15,5 thalach: 102,9 exang: 87,3 oldpeak: 87,3 slope: 87,3 ca: 100,9 thal: 102,9 Solution We started with the reduced Long Beach data set of 14 attributes using both GMDH and self-organising Fuzzy Rule Induction. Then we created a GMDH model using the complete Long Beach data set (76 attributes) to figure out how the other attributes would contribute to the model. To confirm the results on another data set, a second GMDH model was generated on the reduced Cleveland data set. All GMDH models we consider here are static nonlinear models and for FRI each attribute was fuzzified into 5 linguistic variables. To get a sharper difference between presence and absence of disease, we transformed the target variable into -1 for absence, and 1-4 for presence correspondingly.

218


8

Sample Applications

Results Long Beach We divided the data set into 180 cases for learning and 20 cases for evaluating how the models do on new data. The complex nonlinear GMDH model was composed of this subset of attributes: x14 = f(x1 , x 2 , x 3 , x 6 , x 8 , x 9 , x 10 , x 11 , x 13 ).

Figure 8.25 plots this model vs. x14 (the target) and table 8.24 shows the classification results on both learning and testing data. 4 3 2 1 0 -1 -2 -3 x14

MODEL LB14

PREDICTION

Fig. 8.25: Nonlinear GMDH model vs. target variable x14 Long Beach 14

Long Beach 76

GMDH

GMDH

Fuzzy Rule Induction

Has/Has Not Class 0-4 Has/Has Not Class 0-4 Has/Has Not Class 0-4 train false classified Accuracy [%] test false classified Accuracy [%]

37

116

2

24

3

-

79.44

35.56

98.89

86.67

98.33

-

7

16

2

5

2

-

65.00

20.00

90.00

75.00

90.00

-

Table 8.24: Classification results for the Long Beach data set

The complete data set of 76 attributes contains some factors that are stated as ‘unused’ so that 64 variables were used effectively. The generated model is shown in fig. 8.26 and is a function of these eight attributes: x76 = f(x10 , x 11 , x 56 , x 59 , x 60 , x 62 , x 64 , x 66 ),

with x10 - oldpeak, x11 - slope, x56 - year of cardiac cath, x59 - ladprox, x60 - laddist, x62 cxmain, x64 - om1, and x66 - rcaprox. The data source information indicates that attributes 59 to 66 are vessels. ... From other Fields

219


4 3 2 1 0 -1

x76

MODEL LB76

PREDICTION

Fig. 8.26: Nonlinear GMDH model vs. target variable x14

This model shows an increased classification capability (table 8.24) and is mainly composed of attributes that are not included in the reduced data set. Since we are no medical experts, we cannot comment this fact here. Compared with other published results of about 80% classification accuracy for the binary classification task, these GMDH models seem to have a significantly higher accuracy (98% in table 8.24). From Fuzzy Rule Induction that employs the eight extracted attributes from the GMDH model above as inputs, this model was generated for the binary classification task (table 8.24): IF PB_om1 OR PB_ladd OR PB_cxmain OR PB_ladp OR PB_rcap THEN Presence

When modelling all five levels separately, this model was obtained for absence of disease IF NB_om1 & NB_cxmain & NB_rcap & NB_ladp & NB_cxmain & NB_ladd & NB_rcap THEN Absence

and the following rules are created for the different levels of presence: IF NS_slop OR PB_slop & PS_cyr OR NB_ladp & ZO_slop & PB_cyr OR NB_cyr OR NS_cyr & NB_cxmain OR NB_ladp & ZO_slop & PB_cyr THEN NS_class (class 1) IF PM_old & PB_om1 OR ZO_slop & PM_cyr OR PM_old & PB_om1 OR NS_cyr & PB_om1 THEN ZO_class (class 2) IF NB_old & PB_om1 OR PB_ladd & PB_rcap OR PB_ladp & PB_cxmain & PB_cxmain & PB_rcap OR ZO_cyr & PB_om1 OR PB_ladd & PB_rcap OR PB_cxmain & PB_ladp & PB_rcap THEN PS_class (class 3) IF PM_old & NB_slope OR PB_old & NB_ladp THEN PB_class (class 4)

220


8

Sample Applications

Cleveland The reduced Cleveland data set was devided into 270 cases for learning and 33 cases for prediction, and GMDH has created a nonlinear model x14 = f(x2 , x 3 , x 7 , x 9 , x 10 , x 11 , x 12 ) which is composed of a similar set of relevant input variables as the reduced Long Beach model. The classification power compared with the latter model, however, is much higher as shown in table 8.25. Unfortunately, we couldn’t create a model for the complete Cleveland data set, because there was a warning that these data seems to be corrupted. So we couldn’t confirm the results with this data. Cleveland 14 GMDH Has/Has Not Class 0-4 train false classified Accuracy [%] test false classified Accuracy [%]

47

150

82,59

44,44

5

24

84,85

27,27

Table 8.25: Classification results for the Cleveland data set

Besides classification accuracy, the efforts (time, costs) for applying a classification model are quite important also. Since GMDH selects a subset of attributes necessary to obtain a corresponding classification quality, some positive effects may appear from this perspective also. Table 8.26 examplarily expresses this advantage compared with a model that would use all 13 provided variables (like many NNs do, e.g.). Cleveland 14 all attributes

GMDH

Long Beach 14 all attributes

GMDH

costs per patient [CAN$]

600,57

483,20

600,57

475,90

advantage [%]

0,00

19,54

0,00

20,76

Table 8.26: Cost reduction from using GMDH models

Summary Using two different data sources, GMDH has shown that it can extract valuable information for classification purposes. Based on its strong advantages, speed, self-selection of relevant input variables and generation of an analytical model for interpretation purposes, the complete data set of 76 attributes was applicable and has proven to significantly increase classification accuracy using a small subset of attributes only. With this knowledge, detection of disease may also get more efficient by reducing both time and costs for the corresponding procedure.


221


8.3.2 U.S. Congressional Voting Behavior Data Source Source: Congressional Quarterly Almanac, 98th Congress, 2nd session 1984, Volume XL: Congressional Quarterly Inc. Washington, D.C., 1985. Donor: Jeff Schlimmer ([email protected]) Date: 27 April 1987 Problem This data set includes votes for each of the U.S. House of Representatives Congressmen on the 16 key votes identified by the Congressional Quarterly Almanac (CQA). The CQA lists nine different types of votes: voted for, paired for, and announced for (these three simplified to yea), voted against, paired against, and announced against (these three simplified to nay), voted present, voted present to avoid conflict of interest, and did not vote or otherwise make a position known (these three simplified to an unknown disposition). The goal is to find a model that reflects a Congressman's party affiliation (Democrat, Republican) based on her or his votes of a fixed set of attributes. Information Used This data set was taken from the UCI Machine Learning Repository (http://www.ics.uci.edu/~mlearn/MLSummary.html), and it contains 435 instances (267 democrats, 168 republicans) and 17 Boolean valued attributes. x1 =y - Class Name: the target variable (democrat, republican) x2 - handicapped-infants x3 - water-project-cost-sharing x4 - adoption-of-the-budget-resolution x5 - physician-fee-freeze x6 - el-salvador-aid x7 - religious-groups-in-schools x8 - anti-satellite-test-ban x9 - aid-to-nicaraguan-contras x10 - mx-missile x11 - immigration x12 - synfuels-corporation-cutback x13 - education-spending x14 - superfund-right-to-sue x15 - crime x16 - duty-free-exports x17 - export-administration-act-south-africa This data set of N=435 instances was differently divided into learning (Nl ) and validation data subsets (N v ): a) Nl =200, Nv =235 and b) Nl =400, Nv =35 222


8

Sample Applications

The first Nl instances of the data set we used to create a model, and then this model was applied on the remaining Nv votes to predict corresponding party affiliation (out-of-sample prediction). Solution Since all attributes are Boolean valued, Fuzzy Rule Induction is a good choice here. The target variable y is encoded into two values: A zero stands for republician and a one for democrat. The attributes x 2 - x17 have three values (nay , yea, unknown), so they are transformed into three linguistic variables using the span based fuzzification method (section 7.4.1): N_ is equivalent to "nay", ZO_ to "unknown" , and P_ substitutes "yea". The resulting data set of 48 linguistic variables was used to create a logic model. Results A) KnowledgeMiner has created this simple rule that best describes party affiliation: IF NOT_ZO_adoption of-the-budget-resolution & N_physician-fee-freeze OR NOT_P_physician-fee-freeze & NOT_ZO_mx-missile THEN Party Affiliation = Democrat

Using only 3 attributes (adoption of-the-budget-resolution , physician-fee-freeze, mx-missile) from potentially 16 attributes, this rule shows only 5 false classified cases, which is an accuracy of 97.5% (table 8.27). This model quality is almost confirmed when applying this model on the remaining 235 instances. Here, 12 cases cannot be described correctly by the model, so that the model's overall classification accuracy is 96.1%.

Nl false classified classification accuracy [%]

5 97.5

Model A Nv Nl + N v 12 94.8

17 96.1

Nl 16 96.0

Model B Nv Nl + N v 1 97.1

17 96.1

Table 8.27: Classification accuracy of the generated rules

B) We run the second test to check if classification accuracy can be increased using a larger learning data set for rule induction. Here, KnowledgeMiner has created this rule for party affiliation: IF N_physician-fee-freeze OR NOT_P_physician-fee-freeze & NOT_ZO_mx-missile THEN Party Affiliation = Democrat

The rule's classification error is 16 on learning and 1 on validation data (table 8.27). This result has two interesting aspects. First, the generated rules of both tests are almost identical showing that the votes are very consistent so that a small data set is well suited already for Fuzzy Rule Induction. The second aspect is that not only the overall classification error of both rules is same, but also that


223


the same instances are false classified by the rules A and B. This underlines the similarity of both rules. Based on these conclusions, we created another rule using all 17 false classified instances only. Important is that from these 17 instances 14 are democrats. So this data set isn't well balenced. Interestingly, the following rule with a classification accuracy of 100% was obtained: IF P_physician-fee-freeze THEN Party Affiliation = Democrat

Obviously, this rule describes a minority vote of the 14 democrats and the 3 republicians demonstrating an opposite opinion in the question of physician-fee-freeze than their party's majority. It also suggests that the 1984 U.S. House of Representatives voting was esentially driven by the problem of physician-fee-freeze. Summary In this example FRI was applied to analyse voting behavior. The rules were generated using different data subsamples within a minute. They show a consistent dependency structure that suggests that the 1984 Congressional voting was completely decided by just two attributes, physician-feefreeze and mx-missile. Also, a minority vote model was generated so that the entire data set is 100% described by two rules.

Additional examples can be downloaded with the most recent version of KnowledgeMiner at http://www.knowledgeminer.net

224


.

8

Sample Applications

References Bischoff, R., Bleile, C., Graalfs, J.: Der Einsatz Neuronaler Netze zur betriebswirtschaftlichen Kennzahlenanalyse . Wirtschaftsinformatik 33 (1991) No. 5 pp. 375-385 Farlow, S.J (ed.).: Self-organizing Methods in Modelling , Marcel Dekker. New York, 1984 Fujita, S., Koi, H.: Application of GMDH to environmental system modelling and management. In Self-organizing Methods in Modelling, S.J.Farlow (ed.). Marcel Dekker. New York, 1984, pp. 257-276 Guangzhong Liu, Xiaofeng Li, J.-A. Mueller: The improvement of BP Algorithm and Selfadjustment of Architectural Parameters . (expected in 2000) Kingdon, J.: Intelligent Systems and Financial Forecasting . Springer. London, Berlin, ... 1997

References

225

Self-organising Data Mining

Data Mining

Data Mining on Multimedia Data

Data Mining on Multimedia Data

Visual Data Mining - Theory

Mining Text Data

Advanced Data Mining Techniques

Data Mining Cookbook

Data mining and warehousing

Advanced Data Mining Techniques

Principles of data mining

Visual Data Mining

Data mining in bioinformatics

Principles of Data Mining

Mining graph data

Data Mining Withputational Intelligence

Mining imperfect data

High Performance Data Mining

Advanced Data Mining Techniques

Data Mining in Bioinformatics

Privacy Preserving Data Mining

Domain Driven Data Mining

Urban Data Mining

Oxford Clinical Data Mining

Principles of data mining

Privacy Preserving Data Mining

Data Mining Cookbook

Data mining your website

Advanced Data Mining Techniques

Principles of Data Mining

Data Mining the Web

Self-organising Data Mining

Data Mining

Data Mining on Multimedia Data

Data Mining on Multimedia Data

Visual Data Mining - Theory

Mining Text Data

Advanced Data Mining Techniques

Data Mining Cookbook

Data mining and warehousing

Advanced Data Mining Techniques

Principles of data mining

Visual Data Mining

Data mining in bioinformatics

Principles of Data Mining

Mining graph data

Data Mining Withputational Intelligence

Mining imperfect data

High Performance Data Mining

Advanced Data Mining Techniques

Data Mining in Bioinformatics

Privacy Preserving Data Mining

Domain Driven Data Mining

Urban Data Mining

Oxford Clinical Data Mining

Principles of data mining

Privacy Preserving Data Mining

Data Mining Cookbook

Data mining your website

Advanced Data Mining Techniques

Principles of Data Mining

Data Mining the Web

Recommend Documents