Library of Congress Cataloging-in-Publication Data Lin, C. T. (Chin Teng). Neural fuzzy systems : a neuro-fuzzy synergism to intelligent systems / Chin-Teng Lin, C. S. George Lee. p. cm. Includes bibliographical references and índex. ISBN 0-13-235169-2 1. Intelligent control systems. 2. Fuzzy systems. 3. Neural networks (Computer science) I. Lee, C. S. G. (C. S. George) II. Title. TJ217.25.L57 1995 629.8'9—dc20 96-170 CIP Editorial/production supervisión: bookworks / Karen Fortgcmg Cover design director: Jerry Votta Cover design: Amane Altiparmakian Manufacturing manager: Alexis R. Heydt Acquisitions editor: Bernard Goodwin © 1996 by Prentice Hall P T R Prentice-Hall, Inc. A Simón & Schuster Company Upper Saddle River, NJ 07458 MATLAB is a registered trademark of the Math Works, Inc. The publisher offers discounts on this book when ordered in bulk quantities. For more information, contact Corporate Sales Department Prentice Hall P T R One Lake Street Upper Saddle River, N e w Jersey 07458 Phone: 800-383-3419 FAX: 201-236-7141 E-mail:
[email protected] All rights reserved. N o part of this book may be reproduced, in any form or by any means without permission in writíng from the publisher. Printed in the United States of America 10
9
8
7
6
5
4
3 2
1
ISBN 0-13-235169-2 Prentice-Hall International (UK) Limited, London Prentice-Hall of Australia Pty. Limited, Sydney Prentice-Hall Cañada Inc., Toronto Prentice-Hall Hispanoamericana S.A., México Prentice-Hall of India Prívate Limited, New Delhi Prentice-Hall of Japan, Inc., Tokyo Simón & Schuster Asia Pte. Ltd., Singapore Editora Prentice-Hall do Brasil, Ltda., Rio de Janeiro
This book is dedicated to the Chiao-Tung University Centennial (1896-1996)
Contents
PREFACE xiii l
INTRODUCTION 1 1.1 1.2 1.3 1.4 1.5 1.6
PART I 2
Motivation 1 Fuzzy Systems 3 Neural Networks 4 Fuzzy Neural Integrated Systems Organization of the Book 8 References 9
7
FUZZY SYSTEMS BASICS OF FUZZY SETS 10 2.1 2.2
Fuzzy Sets and Operation on Fuzzy Sets Extensions of Fuzzy SetConcepts 22 2.2.1 2.2.2
2.3 2.4 2.5
Other Kinds of Fuzzy Sets, 22 Further Operations on Fuzzy Sets, 23
Extensión Principie and Its Applications 2.3.1 2.3.2
10
29
Operations ofType-2 Fuzzy Sets, 31 Consistency Degree ofTwo Fuzzy Sets, 32
Concluding Remarks Problems 34
33
3
FUZZY RELATIONS 37 3.1 3.2 3.3
Basics of Fuzzy Relations 37 Operations on Fuzzy Relations 41 Various Types of Binary Fuzzy Relations 3.3.1 3.3.2 3.3.3
3.4 3.5 3.6 4
47
Similarity Relations, 49 Resemblance Relations, 51 Fuzzy Partial Ordering, 52
Fuzzy Relation Equations 54 Concluding Remarks 59 Problems 60
FUZZY MEASURES 63 4.1
Fuzzy Measures 4.1.1 4.1.2 4.1.3
4.2 4.3 4.4 4.5
64
Belief and Plausibility Measures, 65 Probability Measures, 72 Possibility and Necessity Measures, 74
Fuzzy Integráis 80 Measures of Fuzziness 83 Concluding Remarks 85 Problems 86
POSSIBILITY THEORY AND FUZZY ARITHMETIC 89 5.1 5.2
Basics of Possibility Theory Fuzzy Arithmetic 93 5.2.1 5.2.2 5.2. 3
5.3 5.4
89
Interval Representarían ofUncertain Valúes, 94 Operations and Properties of Fuzzy Numbers, 97 Ordering of Fuzzy Numbers, 108
Concluding Remarks Problems 111
111
FUZZY LOGIC AND APPROXIMATE REASONING 114 6.1 6.2
Linguistic Variables Fuzzy Logic 118 6.2.1 6.2.2
6.3
123
Categorical Reasoning, 123 Qualitative Reasoning, 126 Syllogistic Reasoning, 127 Dispositional Reasoning, 129
Fuzzy Expert Systems 6.4.1 6.4.2
6.5 6.6
Truth Valúes and Truth Tables in Fuzzy Logic, 119 Fuzzy Propositions, 121
Approximate Reasoning 6.3.1 6.3.2 6.3.3 6.3.4
6.4
114
131
MILORD, 132 Z-II, 136
Concluding Remarks Problems 140
139
7
FUZZY LOGIC CONTROL SYSTEMS 142 7.1
Basic Structure and Operation of Fuzzy Logic Control Systems 7.1.1 7.1.2 7.1.3 7.1.4 7.1.5
7.2 7.3 7.4 7.5 7.6 8
Design Methodology of Fuzzy Control Systems 159 Stability Analysis of Fuzzy Control Systems 166 Applications of Fuzzy Controllers 172 Concluding Remarks 175 Problems 177
APPLICATIONS OF FUZZY THEORY 180 8.1
Fuzzy Pattern Recognition 8.1.1 8.1.2
8.2 8.3 8.4 8.5 8.6
180
Classification Methods Based on Fuzzy Relations, 182 Fuzzy Clustering, 186
Fuzzy Mathematical Programming Fuzzy Databases 193 8.3.1 8.3.2
190
Fuzzy Relational Databases, 194 Fuzzy Object-Oriented Databases,
Human-Machine Interactions Concluding Remarks 202 Problems 203
196
199
II
ARTIFICIAL NEURAL NETWORKS
9
INTRODUCTION TO ARTIFICIAL NEURAL NETWORKS 205 9.1 9.2
Fundamental Concepts of Artificial Neural Networks Basic Models and Learning Rules ofANNs 207 9.2.1 9.2.2 9.2.3
9.3 9.4 9.5' 10
142
Input-Output Spaces, 143 Fuzzifier, 145 Fuzzy Rule Base, 145 Inference Engine, 145 Defuzzjfier, 156
205
Processing Elements, 207 Connections, 211 Learning Rules, 212
Distributed Representations Concluding Remarks 221 Problems 221
217
FEEDFORWARD NETWORKS AND SUPERVISED LEARNING 224 10.1
Single-Layer Perceptron Networks 10.1.1 10.1.2
10.2
Multilayer Feedforward Networks 10.2.1 10.2.2 10.2.3
Contents
224
Perceptron Learning Rule, 225 Adaline, 231
235
Back Propagation, 236 Learning Factors ofBack Propagation, 244 Time-Delay Neural Networks, 250
10.3
Other Feedforward Networks 10.3.1 10.3.2 10.3.3
10.4 10.5 11
Concluding Remarks Problems 257
Hopfield Networks 11.1.1 11.1.2
11.2
11.3
284
Hopfield Networks for Optimization Problems, 284 Boltzmann Machines, 291
Concluding Remarks Problems 298
296
UNSUPERVISED LEARN1NG NETWORKS 301 12.1
Unsupervised Learning Rules 12.1.1 12.1.2 12.1.3 12.1.4
12.2 12.3 12.4 12.5 12.6 12.7 12.8 12.9 12.10
301
Signal Hebbian Learning Rule, 302 Competitive Learning Rule, 304 Differential Hebbian Learning Rule, 308 Differential Competitive Learning Rule, 309
Hamming Networks 309 Self-Organizing Feature Maps 311 Adaptive Resonance Theory 314 12.4.1 12.4.2
The Instar-Outstar Model—Shunting Activation Equations, 315 Adaptive Resonance Theory, 321
Counterpropagation Networks 326 Radial Basis Function Networks 328 Adaptive Bidirectional Associative Memories Hierárchical Networks—Neocognitron 332 Concluding Remarks 336 Problems 337
331
RECURRENT NEURAL NETWORKS 340 13.1
Feedback Backpropagation Networks 13.1.1 13.1.2
13.2
341
Recurrent Backpropagation Networks, 341 Partially Recurrent Networks, 345
Fully Recurrent Networks 13.2.1
viii
272
Recurrent Autoassociative Memory—Hopfield Memory, 273 Bidirectional Associative Memory, 277 Temporal Associative Memory, 282
Optimization Problems 11.3.1 11.3.2
11.4 11.5
263
Discrete Hopfield Networks, 263 Continuous Hopfield Networks, 267
Associative Memories 11.2.1 11.2.2 11.2.3
13
256
SINGLE-LAYER FEEDBACK NETWORKS AND ASSOCIATIVE MEMORIES 263 11.1
12
253
Functional-Link Networks, 253 Tree Neural Networks, 254 Wavelet Neural Networks, 255
349
Real-Time Recurrent Learning, 350 Contents
13.2.2 13.2.3 13.2.4
13.3
Reinforcement Learning 13.3.1 13.3.2 13.3.3
14
Concluding Remarks
13.5
Problems
367
379
380
GENETIC ALGORITHMS 382
14.3 14.4
14.5 14.6 14.7
Basics of Genetic Algorithms 382 FurtherEvolutionof Genetic Algorithms
393
14.2.1 14.2.2
Improved Selection Schemes, 393 Advanced Operators, 394
14.4.1 14.4.2 14.4.3
Genetic Algorithms for Neural Network Parameter Learning, 399 Genetic Algorithms for Path Planning, 404 Genetic Algorithms for System Identification and Controls, 405
Hybrid Genetic Algorithms 398 Applications of Genetic Algorithms
399
Genetic Programming 406 Concluding Remarks 411 Problems 411
STRUCTURE-ADAPTIVE NEURAL NETWORKS 414 15.1
Simulated Evolution for Neural Network Structure Learning 15.1.1 15.1.2
15.2 15.3 15.4
424
Weight Decay, 424 Connection and Node Pruning, 425
Growing Neural Networks 15.3.1 15.3.2
427
Input Space Partitioning, 427 Prototype Selection, 433
Growing and Pruning Neural Networks 15.4.1 15.4.2
414
Genetic Algorithms for Designing Neural Networks, 414 Evolutionary Programming for Designing Neural Networks, 420
Pruning Neural Networks 15.2.1 15.2.2
16
353
Associative Reward-Penalty, 368 REINFORCE Algorithms, 371 Temporal Difference Methods, 373
13.4
14.1 14.2
15
Time-Dependent Recurrent Backpropagation, Second-Order Recurrent Networks, 357 The Extended Kalman Filter, 363
Activity-Based Structural Adaptation, Function Networks, 439
15.5
Concluding Remarks
15.6
Problems
437 437
442
443
APPLICATIONS ÓF NEURAL NETWORKS 445 16.1 16.2 16.3
Contents
Neural Networks in Control Systems 445 Dynamic Backpropagation for System Identification and Control, 448
16.1.1 16.1.2
Cerebellar Model Articulation Controller, 457
Neural Networks in Sensor Processing 464 Neural Networks in Communications 468 ix
16.4 16.5 16.6
PART III 17
Neural Knowledge-Based Systems Concluding Remarks 474 Problems 475 .
FUZZY NEURAL INTEGRATED SYSTEMS INTEGRATING FUZZY SYSTEMS AND NEURAL NETWORKS 478 17.1
Basic Concept of Integrating Fuzzy Systems and Neural Networks 17.1.1 17.1.2 17.1.3
17.2
17.2.3
17.3 17.4
482
Fuzzy Inference Systems as Universal Approximators, 483 Equivalence ofSimplified Fuzzy Inference Systems and Radial Basis Function Networks, 487 Stability Analysis of Neural Networks Using Stability Conditions of Fuzzy Systems, 489
Concluding Remarks Problems 494
494
NEURAL-NETWORK-BASED FUZZY SYSTEMS 496 18.1 18.2
Neural Realization of Basic Fuzzy Logic Operations 496 Neural Network-Based Fuzzy Logic Inference 498 18.2.1 18.2.2 18.2.3
18.3
18.4 18.5
Fuzzy Inference Networks, 498 Fuzzy Aggregation Networks, 504 Neural Network-Driven Fuzzy Reasoning, 507
Neural Network-Based Fuzzy Modeling 18.3.1 18.3.2 18.3.3
19
478
General Comparisons of Fuzzy Systems and Neural Networks, 478 Choice of Fuzzy Systems or Neural Networks, 480 Reasons for Integrating Fuzzy Systems and Neural Networks, 481
The Equivalence of Fuzzy Inference Systems and Neural Networks 17.2.1 17.2.2
18
470
511
Rule-Based Neural Fuzzy Modeling, 511 Neural Fuzzy Regression Models, 517 Neural Fuzzy Relational Systems, 523
Concluding Remarks Problems 531
530
NEURAL FUZZY CONTROLLERS 533 19.1 19.2
Types of Neural Fuzzy Controllers 534 Neural Fuzzy Controllers with Hybrid Structure-Parameter Learning 19.2.1 19.2.2
19.3
Parameter Learning for Neural Fuzzy Controllers 19.3.1 19.3.2 19.3.3
535
Fuzzy Adaptive Learning Control Network, 535 Fuzzy Basis Function Network with Orthogonal Least Squares Learning, 545
551
Neural Fuzzy Controllers with Fuzzy Singleton Rules, 551 Neural Fuzzy Controllers with TSK Fuzzy Rules, 556 Fuzzy Operator Tuning, 559
Contents
19.4
Structure Learning for Neural Fuzzy Controllers 19.4.1 19.4.2
19.5
On-Line Structure Adaptive Neural Fuzzy Controllers 19.5.1 19.5.2 '
19.6
20
592
FALCON with Reinforcement Learning, 592 Generalized Approximate Reasoning-Based Intelligent Controller, 600
Concluding Remarks Problems 605
604
FUZZY LOGIC-BASED NEURAL NETWORK MODELS 609 20.1
Fuzzy Neurons 20.1.1 20.1.2 20.1.3
20.2
20.3
614
Fuzzy Perceptron, 614 Fuzzy Classification with the Back-propagation Network, 618 Fuzzy Associative Memories, 620 Fuzzy ART Models, 626 Fuzzy Kohonen Clustering Network, 635 Fuzzy RCE Neural Network, 639 Fuzzy Cerebellar Model Articulation Controller, 641
Neural Networks with Fuzzy Training 20.3.1 20.3.2 20.3.3
20.4 20.5
609
Fuzzy Neuron of Type I, 610 Fuzzy Neuron of Type II, 612 Fuzzy Neuron of Type IIL 613
Fuzzification of Neural Network Models 20.2.1 20.2.2 20.2.3 20.2.4 20.2.5 20.2.6 20.2.7
21
573
FALCON with On-Line Supervised Structure and Parameter Learning, 573 FALCON with ARTNeural Learning, 579
Neural Fuzzy Controllers with Reinforcement Learning 19.6.1 19.6.2
19.7 19.8
561
Fuzzy Logic Rule Extraction from Numerical Training Data, 562 Genetic Algorithms for Fuzzy Partition oflnput Space, 567
643
Neural Networks with Fuzzy Teaching Input, 643 Neural Networks with Fuzzy Parameters, 648 Fuzzy Control for Learning Parameter Adaptation, 654
Concluding Remarks Problems 658
657
FUZZY NEURAL SYSTEMS FOR PATTERN RECOGNITION 661 21.1
Fuzzy Neural Classification 21.1.1 21.1.2 21.1.3
21.2
Fuzzy Neural Clustering '21.2.1 21.2.2
21.3
Contents
678
Fuzzy Competitive Learning for Fuzzy Clustering, 678 Adaptive Fuzzy Leader Clustering, 680
Fuzzy Neural Models for Image Processing 21.3.1 21.3.2
21.4
661
Uncertainties with Two-Class Fuzzy Neural Classification Boundaries, 661 Multilayer Fuzzy Neural Classification Networks, 667 Genetic Algorithms for Fuzzy Classification Using Fuzzy Rules, 674
684
Fuzzy Self-Supervised Multilayer Network for Object Extraction, 684 Genetic Algorithms with Fuzzy Fitness Function for Image Enhancement, 690
Fuzzy Neural Networks for Speech Recognition
692
xi
21.5 21.6 21.7 A
Fuzzy-Neural Hybrid Systems for System Diagnosis Concluding Remarks 700 Problems 702
MATLAB FUZZY LOGIC TOOLBOX 704 A.l
Demonstration Cases 706 A. 1.1 A 1.2 A. 1.3
A.2
A Simple Fuzzy Logic Controller, 706 A Neural Fuzzy System—ANFIS, 707 Fuzzy c-means Clustering, 708
Demonstration of Fuzzy Logic Applications A.2.1 A.2.2
B
696
709
A Fuzzy Controller for Water Bath Temperature Control, 709 A Neural Fuzzy Controller for Water Bath Temperature Control, 712
MATLAB NEURAL NETWORK TOOLBOX 715 B. 1
Demonstration of Various Neural Network Models B.l.l B.1.2 B.1.3 B.1.4 B.1.5 B.1.6 B.1.7 B.1.8 B.1.9 B.l.lO B.l.l 1
B.2
Demonstration of Neural Network Applications B.2.1 5.2.2
716
Hebbian Learning Rule, 716 Perceptron Learning Rule, 717 Adaline, 718 BackPropagation, 719 Hopfield Network, 722 Instar Learning Rule, 723 Competitive Learning Rule, 724 LVQ Learning Rule (Supervised Competitive Learning Rule), 726 Self-Organizing Feature Map, 726 Radial Basis Function Network, 727 Elman Network, 729
730
Adaptive Noise Cancelation Using the Adaline Network, 730 Nonlinear System Identification, 731
BIBLIOGRAPHY 734 INDEX 783
xii
Contents
Preface
Fuzzy systems and neural networks have attracted the growing interest of researchers, scientists, engineers, practitioners, and students in various scientific and engineering areas. Fuzzy sets and fuzzy logic are based on the way the brain deals with inexact information, while neural networks (or artificial neural networks) are modeled after the physical architecture of the brain. Although the fundamental inspirations for these two fields are quite different, there are a number of parallels that point out their similarities, The intriguing differences and similarities between these two fields have prompted the writing of this book to examine the basic concepts of fuzzy set theory, fuzzy logic, fuzzy logic control systems, and neural networks; to explore their applications separately and in combination; and to explore the synergism of integrating these two techniques for the realization of intelligent systems for various applications. Since the publication of Lotfi Zadeh's seminal work, "Fuzzy Sets," in 1965, the number and variety of applications of fuzzy logic have been growing. The performance of fuzzy logic control and decisión systems critically depends On the input and output membership functions, the fuzzy logic control rules, and the fuzzy inference mechanism. Although a great amount of literature has been published dealing with the applications and the theoret¿cal issues of fuzzy logic control and decisión systems, a unified and systematic design methodology has yet to be developed. The use of neural networks to automate and synthesize the design of a general fuzzy logic control or decisión system presents a novel and innovative approach and a viable design solution to this difficult design and realization problem. The publication of J. J. Hopfield's seminal work on a single-layer feedback neural network with symmetric weights in 1982 revived neural network research activity from its doldrums in the 1960s and 1970s because of the ability of neural networks to classify, store, xiii
recall, and associate information or patterns. The performance of neural networks depends on the computational function of the neurons in the network, the structure and topology of the network, and the learning rule or the update rule of the connecting weights. Publication of the article "Learning Representations by Backpropagation Errors," by Rumelhart, Hinton, and Williams in 1986 has further extended and improved the learning ability of neural networks. This concept of trainable neural networks further strengthens the idea of utilizing the learning ability of neural networks to learn the fuzzy control rules, the membership functions, and other parameters of a fuzzy logic control or decisión system. Fuzzy sets and fuzzy logic were developed as a means for representing, manipulating, and utilizing uncertain information and to provide a framework for handling uncertainties and imprecisión in real-world applications, while neural networks were developed to provide computational power, fault tolerance, and learning capability to the systems. This happy marriage of techniques from fuzzy logic systems and neural networks reaps the benefits of both neural networks and fuzzy logic systems. That is, the neural networks provide the connectionist structure (fault tolerance and distributed representation properties) and learning abilities to the fuzzy logic systems, and the fuzzy logic systems provide a structural framework with high-level fuzzy IF-THEN rule thinking and reasoning to the neural networks. It is this synergistic integration of neural networks and fuzzy logic systems into functional systems with low-level learning capability and high-level thinking and reasoning that separates our book from other books on fuzzy set theory, neural networks, neural fuzzy systems, and fuzzy neural networks. This textbook was written to provide engineers, scientists, researchers, and students involved in fuzzy systems, neural networks, and fuzzy neural integrated systems with a comprehensive, well-organized, and up-to-date account of basic principies underlying the design, analysis, and synthesis of fuzzy neural integrated systems. This book is the outgrowth of lecture notes for courses taught by the authors at Purdue University and the National Chiao-Tung University. The material has been tested extensively in the classroom as well as through numerous tutorial short courses and several intensive summer courses taught by both authors for the past several years. The suggestions and criticisms of students in these courses have had a significant influence on the way the material is presented in this book. The mathematical level in all chapters is well within the grasp of first-year gradúate students in a technical discipline such as engineering, computer science, or technology requiring introductory preparation in classical set theory, discrete mathematics, matrix computations, probability, and computer programming. In presenting the material, emphasis is placed on the utilization of developed results from basic concepts. In addition to the MATLAB® examples in the appendices, numerous examples are worked out in the text to illustrate the discussion, and exercises of various types and complexity are included at the end of each chapter. Some of these problems allow the reader to gain further insight into the points discussed in the text through practice in problem solution. Others serve as supplements and extensions of the material in the book. A complete solutions manual is also available to the instructor from the publisher. This textbook consists of three major parts and two appendices: Part I (Chapters 2 to 8) covers fundamental concepts and operations of fuzzy sets, fuzzy relations, fuzzy measures, possibility theory, fuzzy logic and approximate reasoning,
xiv
Preface
and their application to fuzzy logic control systems and other systems such as pattern recognition systems and fuzzy expert systems. Part II (Chapters 9 to 16) covers important concepts and topics in neural networks, including single-layer and multilayer feedforward networks, recurrent networks, and unsupervised learning, supervised learning, and reinforcement learning for training neural networks. Genetic algorithms and their use for the structure and parameter learning of neural networks are also explored. Part ID (Chapters 17 to 21) covers three major integrated systems and their rationale for integration. Neural fuzzy (control) systems, fuzzy neural networks (for pattern recognition), and fuzzy neural hybrid systems are discussed and explored. - Finally, Appendices A and B illustrate the computer simulations of fuzzy logic systems and neural networks using the MATLAB Fuzzy Logic Toolbox and the Neural Network Toolbox. These toolboxes are useful for student projects and independent studies. The authors have used the materials from this book to teach three different gradúate courses (15-week semester courses): • Fuzzy Control Systems: Chapters 1-8 and Appendix A • Introduction to Neural Networks: Chapter 1, Chapters 9-16, and Appendix B • Neuro-Fuzzy Control Systems: Chapters 1 and 2, Sections 3.1, 3.2,3.4, and 6.1-6.3, Chapter 7, Chapters 9-10, Sections 12.1-12.4 and 12.6, Chapter 17, and Sections 19.1-19.3, and 19.5-19.6 Other combinations of the material in this book for teaching are also possible. For instance, for a neural network gradúate course with an emphasis on (fuzzy) pattern recognition, Chapters 2-3 and 9-13 plus selected sections in Chapters 20 and 21 can be covered in one semester; Chapters 17-21 can be taught in a second-year gradúate course on neural fuzzy systems with a prerequisite of courses in fuzzy logic and neural networks.
ACKNOWLEDGMENTS We are indebted to a number of individuáis who, directly or indirectly, assisted in the preparation of the text. In particular, we wish to extend our appreciation to Professors L. A. Zadeh, G. N. Saridis, T. T. Lee, J. Yen, W. H. Tsai, C. C. Teng, M. J. Syu, W. P. Yang, C. C. Jou, M. J. Chung, C. L. Chen, S. F. Su, and Dr. Harold Su. As is true with most projects carried out in a university environment, our students over the past few years have influenced not only our thinking but also the topics covered in this book. The following individuáis have worked with us in the course of their gradúate programs: C. J. Lin, C. F. Juang, I. F. Chung, Y. C. Lú, Dr. W. Hsu, Dr. S. Park, G. H. Kim, T. K. Yin, Y. J. Wang, L. K. Sheu, M. C. Kan, W. F. Lin, S. C. Hsiao, W. C. Lin, C. P. Lee, and C. Y. Hu. Thanks is also due to I. F. Chung, Linda Stovall, Kitty Cooper, C. F. Juang, W. F. Lin, S. C. Hsiao, and Dee Dee Dexter for typing numerous versions of the manuscript. We would also like to express our appreciation to Purdue University's National Science Foundation Engineering Research Center for Intelligent Manufacturing Systems, the
Preface
xv
National Science Council of Taiwan, R.O.C., the Ford Foundation, and the Spring Foundation of National Chiao-Tung University for their sponsorship of our research activities in neuro-fuzzy systems, robotic assembly and manufacturing, and related areas. Finally, we thank Bemard M. Goodwin, editor and publisher at Prentice Hall, for his strong support throughout this project; Karen Fortgang, production editor at bookworks, for her skillful coordination of the production of the book; and Cristina Palumbo of the MathWorks, Inc. for her continued support of our use of MATLAB and the Fuzzy Logic Toolbox and the Neural Network Toolbox. Last but not the least, we would like to thank our parents, M. T. Lin and Y. L. Kuo, and M. K. Lee and K. H. Un, for their constant encouragement. Without their constató encouragement, this book would not have been possible. C. T. Lin would also like to thank his brother C. I Lin for providing an excellent environment for him to finish the manuscript; C. S. G. Lee would like to thank his wife, Pei-Ling, and his children, Lawrence, Steven, and Anthony, for understanding why their daddy could not spend more time with them at their ballgames. C. T. Lin Hsinchu,
Taiwan
C. S. G. Lee West Lafayette,
xvi
Indiana
Contents
7 Introduction
1.1 MOTIVATION Fuzzy systems (FSs) and neural networks (NNs) have attracted the growing interest of researchers in various scientific and engineering areas. The number and variety of applications of fuzzy logic and neural networks have been increasing, ranging from consumer products and industrial process control to medical instrumentation, information systems, and decision analysis. Fuzzy logic (FL) is based on the way the brain deals with inexact information, while neural networks are modeled after the physical architecture of the brain. Although the fundamental inspirations for these two fields are quite different, there are a number of parallels that point out their similarities. Fuzzy systems and neural networks are both numerical model-free estimators and dynamical systems. They share the common ability to improve the intelligence of systems working in an uncertain, imprecise, and noisy environment To a certain extent, both systems and their techniques have been successfully applied to a variety of control systems and devices to improve their intelligence. Both fuzzy systems and neural networks have been shown to have the capability of modeling complex nonlinear processes to arbitrary degrees of accuracy. Although fuzzy systems and neural networks are formally similar, there are also significant differences between them. Fuzzy systems are structured numerical estimators. They start from highly formalized insights about the structure of categories found in the real world and then articulate fuzzy IF-THEN rules as a kind of expert knowledge. Fuzzy systems combine fuzzy sets with fuzzy rules to produce overall complex nonlinear behavior. Neural networks, on the other hand, are trainable dynamical systems whose learning, noise-
1
tolerance, and generalization abilities grow out of their connectionist structures, their dynamics, and their distributed data representation. Neural networks have a large number of highly interconnected processing elements (nodes) which demonstrate the ability to learn and generalize from training patterns or data; these simple processing elements also collectively produce complex nonlinear behavior. In light of their similarities and differences, fuzzy systems and neural networks are suitable for solving many of the same problems and achieving some degree of machine intelligence. Their differences have prompted a recent surge of interest in merging or combining them into a functional system to overcome their individual weaknesses. This innovative idea of integration reaps the benefits of both fuzzy systems and neural networks. That is, neural networks provide fuzzy systems with learning abilities, and fuzzy systems provide neural networks with a structural framework with high-level fuzzy IF-THEN rule thinking and reasoning. Consequently, the two technologies can complement each other, with neural networks supplying the brute force necessary to accommodate and interpret large amounts of sensory data, and fuzzy logic providing a structural framework that utilizes and exploits these low-level results. The incorporation of fuzzy logic and neural network techniques has been used in the conception and design of complex systems in which analytical and expert system techniques are used in combination. Moreover, the synergism of fuzzy logic and neural network techniques has been employed in a wide variety of consumer products, endowing these products with the capability to adapt and learn from experience. Most neuro-fuzzy products are fuzzy rule-based systems in which neural network techniques are used for learning and/or adaptation. Viewed from a much broader perspective, fuzzy logic and neural networks are constituents of an emerging research area, called soft computing, a term coined by Lotfi Zadeh (the father of fuzzy logic). It is believed that the most important factor that underlies the marked increase in machine intelligence nowadays is the use of soft computing to mimic the ability of the human mind to effectively employ modes of reasoning that are approximate rather than exact. Unlike traditional hard computing whose prime desiderata are precision, certainty, and rigor, soft computing is tolerant of imprecision, uncertainty, and partial truth. The primary aim of soft computing is to exploit such tolerance to achieve tractability, robustness, a high level of machine intelligence, and a low cost in practical applications. In addition to fuzzy logic and neural networks, another principal constituent of soft computing is probabilistic reasoning, which subsumes genetic algorithms, evolutionary programming, belief networks, chaotic systems, and parts of learning theory. Among these, genetic algorithms and evolutionary programming are similar to neural networks in that they are based on low-level microscopic biological models. They evolve toward finding better solutions to problems, just as species evolve toward better adaptation to their environments. Coincidentally, fuzzy logic, neural networks, genetic algorithms, and evolutionary programming are also considered the building blocks of computational intelligence as conceived by James Bezdek. Computational intelligence is low-level cognition in the style of the human mind and is in contrast to conventional (symbolic) artificial intelligence. In the partnership of fuzzy logic, neural networks, and probabilistic reasoning, fuzzy logic is concerned in the main with imprecision and approximate reasoning, neural networks with learning, and probabilistic reasoning with uncertainty. Since fuzzy logic, neural networks, and probabilistic reasoning are complementary rather than competitive, it is frequently advantageous to employ them in combination rather than 2
Introduction
Chap. 1
exclusively. Various important topics, both in theories and applications, of fuzzy systems and neural networks (with genetic algorithms) and their synergism are covered in this book. 1.2 FUZZY SYSTEMS In the past decade, fuzzy systems have supplanted conventional technologies in many scientific applications and engineering systems, especially in control systems and pattern recognition. We have also witnessed a rapid growth in the use of fuzzy logic in a wide variety of consumer products and industrial systems. Prominent examples include washing machines, camcorders, autofocus cameras, air conditioners, palmtop computers, vacuum cleaners, automobile transmissions, ship navigators, subway trains, combustion control regulators, and cement kilns. The same fuzzy technology, in the form of approximate reasoning, is also resurfacing in information technology, where it provides decision-support and expert systems with powerful reasoning capabilities bound by a minimum of rules. It is this wealth of deployed, successful applications of fuzzy technology that is, in the main, responsible for current interest in fuzzy systems. Fuzzy sets, introduced by Zadeh in 1965 as a mathematical way to represent vagueness in linguistics, can be considered a generalization of classical set theory. The basic idea of fuzzy sets is quite easy to grasp. In a classical (nonfuzzy) set, an element of the universe either belongs to or does not belong to the set. That is, the membership of an element is crisp—it is either yes (in the set) or no (not in the set). A fuzzy set is a generalization of an ordinary set in that it allows the degree of membership for each element to range over the unit interval [0,1]. Thus, the membership function of a fuzzy set maps each element of the universe of discourse to its range space which, in most cases, is set to the unit interval. One of the biggest differences between crisp and fuzzy sets is that the former always have unique membership functions, whereas every fuzzy set has an infinite number of membership functions that may represent i t This enables fuzzy systems to be adjusted for maximum utility in a given situation. In a broad sense, as pointed out by Lotfi Zadeh, any field can be fuzzified and hence generalized by replacing the concept of a crisp set in the target field by the concept of a fuzzy set For example, we can fuzzify some basic fields such as arithmetic, graph theory, and probability theory to develop fuzzy arithmetic, fuzzy graph theory, and fuzzy probability theory, respectively; we can also fuzzify some applied fields such as neural networks, genetic algorithms, stability theory, pattern recognition, and mathematical programming to obtain fuzzy neural networks, fuzzy genetic algorithms, fuzzy stability theory, fuzzy pattern recognition, and fuzzy mathematical programming, respectively. The benefits of such fuzzification include greater generality, higher expressive power, an enhanced ability to model real-world problems, and a methodology for exploiting the tolerance for imprecision. Hence, fuzzy logic can help to achieve tractability, robustness, and lower solution cost. Fuzziness is often confused with probability. The fundamental difference between them is that fuzziness deals with deterministic plausibility, while probability concerns the likelihood of nondeterministic, stochastic events. Fuzziness is one aspect of uncertainty. It is the ambiguity (vagueness) found in the definition of a concept or the meaning of a term such as "young person" or "large room." However, the uncertainty of probability generally relates to the occurrence of phenomena, as symbolized by the concept of randomness. In Sec. 1.2
Fuzzy Systems
3
other words, a statement is, probabilistic if it expresses some kind of likelihood or degree of certainty or if it is the outcome of clearly defined but randomly occurring events. For example, the statements "There is a 50-50 chance that he will be there," "It will rain tomorrow," "Roll the dice and get a four" have the uncertainty of randomness. Hence, fuzziness and randomness differ in nature; that is, they are different aspects of uncertainty. The former conveys "subjective" human thinking, feelings, or language, and the latter indicates an "objective" statistic in the natural sciences. From the modeling point of view, fuzzy models and statistical models also possess philosophically different kinds of information: fuzzy memberships represent similarities of objects to imprecisely defined properties, while probabilities convey information about relative frequencies. One major feature of fuzzy logic is its ability to express the amount of ambiguity in human thinking and subjectivity (including natural language) in a comparatively undistorted manner. Thus, when is it appropriate to use fuzzy logic? When the process is concerned with continuous phenomena (e.g., one or more of the control variables are continuous) that are not easily broken down into discrete segments; when a mathematical model of the process does not exist, or exists but is too difficult to encode, or is too complex to be evaluated fast enough for real-time operation, or involves too much memory on the designated chip architecture; when high ambient noise levels must be dealt with or it is important to use inexpensive sensors and/or low-precision microcontrollers; when the process involves human interaction (e.g., human descriptive or intuitive thinking); and when an expert is available who can specify the rules underlying the system behavior and the fuzzy sets that represent the characteristics of each variable. With these properties, fuzzy logic techniques find their applications in such areas as [Munakata and Jani, 1994] (1) control (the most widely applied area), (2) pattern recognition (e.g., image, audio, signal processing), (3) quantitative analysis (e.g., operations research, management), (4) inference (e.g., expert systems for diagnosis, planning, and prediction; natural language processing; intelligent interface; intelligent robots; software engineering), and (5) information retrieval (e.g., databases). 1.3 NEURAL NETWORKS Fundamentally, there are two major different approaches in the field of artificial intelligence (AI) for realizing human intelligence in machines [Munakata, 1994]. One is symbolic AI, which is characterized by a high level of abstraction and a macroscopic view. Classical psychology operates at a similar level, and knowledge engineering systems and logic programming fall in this category. The second approach is based on low-level microscopic biological models. It is similar to the emphasis of physiology or genetics. Artificial neural networks and genetic algorithms are the prime examples of this latter approach. They originated from modeling of the brain and evolution. However, these biological models do not necessarily resemble their original biological counterparts. Neural networks are a new generation of information processing systems that are deliberately constructed to make use of some of the organizational principles that characterize the human brain. The main theme of neural network research focuses on modeling of the brain as a parallel computational device for various computational tasks that were performed poorly by traditional serial computers. Neural networks have a large number of highly interconnected processing 4
Introduction
Chap. 1
elements (nodes) that usually operate in parallel and are configured in regular architectures. The collective behavior of an NN, like a human brain, demonstrates the ability to learn, recall, and generalize from training patterns or data. Since the first application of NNs to consumer products appeared at the end of 1990, scores of industrial and commercial applications have come into use. Like fuzzy systems, the applications where neural networks have the most promise are also those with a real-world flavor, such as speech recognition, speech-to-text conversion, image processing and visual perception, medical applications, loan applications and counterfeit checks, and investing and trading. Models of neural networks are specified by three basic entities: models of the processing element themselves, models of interconnections and structures (network topology), and the learning rules (the ways information is stored in the network). Each node in a neural network collects the values from all its input connections, performs a predefined mathematical operation, and produces a single output value. The information processing of a node can be viewed as consisting of two parts: input and output. Associated with the input of a node is an integration function (typically a dot product) which serves to combine information or activation from an external source or other nodes into a net input to the node. A second action of each node is to output an activation value as a function of its net input through an activation function which is usually nonlinear. Each connection has an associated weight that determines the effect of the incoming input on the activation level of the node. The weights may be positive (excitatory) or negative (inhibitory). The connection weights store the information, and the value of the connection weights is often determined by a neural network learning procedure; It is through adjustment of the connection weights that the neural network is able to learn. In a neural network, each node output is connected, through weights, to other nodes or to itself. Hence, the structure that organizes these nodes and the connection geometry among them should be specified for a neural network. We can first take a node and combine it with other nodes to make a layer of nodes. Inputs can be connected to many nodes with various weights, resulting in a series of outputs, one per node. This results in a single-layer feedforward network We can further interconnect several layers to form a multilayer feedforward network. The layer that receives inputs is called the input layer and typically performs no computation other than buffering of the input signal. The outputs of the network are generated from the output layer. Any layer between the input and the output layers is called a hidden layer because it is internal to the network and has no direct contact with die external environment. There may be from zero to several hidden layers in a neural network. The two types of networks mentioned above are feedforward networks since no node output is an input to a node in the same, layer or preceding layer. When outputs are directed back as inputs to same- or preceding-layer nodes, the network is a feedback network. Feedback networks that have closed loops are called recurrent networks. The third important element in specifying a neural network is the learning scheme. Broadly speaking, there are two kinds of learning in neural networks: parameter learning, which concerns the updating of the connection weights in a neural network, and structure learning, which focuses on the change in the network structure, including the number of nodes and their connection types. These two kinds of learning can be performed simultaneously or separately. Each kind of learning can be further classified into three categories: supervised learning, reinforcement learning, and unsupervised learning. These three categories of learning will be covered in detail in Part II of this book. Sec. 1.3
Neural Networks
5
Among the existing neural network models, two of the important ones that started the modern era of neural networks are the Hopfield network and the back-propagation network. The Hopfield network is a single-layer feedback network with symmetric weights proposed by John Hopfield in 1982. The network has a point attractor dynamics, making its behavior relatively simple to understand and analyze. This result provides the mathematical foundation for understanding the dynamics of an important class of networks. The back-propagation network is a multilayer feedforward network combined with a gradient-descent-type learning algorithm called the back-propagation learning rule by Rumelhart, Hinton, and Williams [1986b], In this learning scheme, the error at the output layer is propagated backward to adjust the connection weights of preceding layers and to minimize output errors. The back-propagation learning rule is an extension of the work of Rosenblatt, Widrow, and Hoff to deal with learning in complex multilayer networks and thereby provide an answer to one of the most severe criticisms of the neural network field. With this learning algorithm, multilayer networks can be reliably trained. The back-propagation network has had a major impact on the field of neural networks and is the primary method employed in most of the applications. Another important tool for the structure and parameter learning of neural networks is genetic algorithms (GAs). Genetic algorithms are search algorithms based on the mechanics of natural selection and natural genetics. The mathematical framework of GAs was developed in the 1960s and is presented in Holland's pioneering book [Holland, 1975]. Genetic algorithms have been used primarily in optimization and machine learning problems. A simple GA processes a finite population of fixed-length binary strings called genes. Genetic algorithms have three basic operators: reproduction of solutions based on their fitness, crossover of genes, and mutation for random change of genes. Another operator associated with each of these three operators is the selection operator, which produces survival of the fittest in the GA. Reproduction directs the search toward the best existing strings but does not create any new strings, the crossover operator explores different structures by exchanging genes between two strings at a crossover position, and mutation introduces diversity into the population by altering a bit position of the selected string. The mutation operation is used to escape the local minima in the search space. The combined action of reproduction and crossover is responsible for much of the effectiveness of a GA's search, while reproduction and mutation combine to form a parallel, noise-tolerant hill-climbing algorithm. The structure and parameter learning problems of neural networks are coded as genes (or chromosomes), and GAs are used to search for better solutions (optimal structure and parameters) for NNs. Furthermore, GAs can be used to find the membership functions and fuzzy rules of a fuzzy logic system. Neural networks offer the following salient characteristics and properties: 1. Nonlinear input-output mapping: Neural networks are able to learn arbitrary nonlinear input-output mapping directly from training data. 2. Generalization: Neural networks can sensibly interpolate input patterns that are new to the network. From a statistical point of view, neural networks can fit the desired function in such a way that they have the ability to generalize to situations that are different from the collected training data. 3. Adaptivity: Neural networks can automatically adjust their connection weights, or even network structures (number of nodes or connection types), to optimize their behavior as controllers, predictors, pattern recognizers, decision makers, and so on. 6
Introduction
Chap. 1
4. Fault tolerance: The performance of a neural network is degraded gracefully under faulty conditions such as damaged nodes or connections. The inherent fault-tolerance capability of neural networks stems from the fact that the large number of connections provides much redundancy, each node acts independently of all the others, and each node relies only on local information. With these properties, when is it appropriate to use neural networks? When nonlinear mappings must be automatically acquired (e.g., robot control and noise removal); when only a few decisions are required from a massive amount of data (e.g., speech recognition and fault prediction); when a near-optimal solution to a combinatorial optimization problem is required in a short time (e.g., airline scheduling and network routing); and when there are more input variables than can be feasibly utilized by other approaches [Simpson, 1992a]. Moreover, despite the possibility of equally comparable solutions to a given problem, several additional aspects of a neural network solution are appealing, including (VLSI) parallel implementations that allow fast processing; less hardware which allows faster response time, lower cost, and quicker design cycles; and on-line adaptation that allows the networks to change constantly according to the needs of the environment
1.4 FUZZY NEURAL INTEGRATED SYSTEMS Fuzzy logic and neural networks (with genetic algorithms) are complementary technologies in the design of intelligent systems. Each method has merits and demerits. A comparison of these techniques with symbolic AI and conventional control theory is presented in Table 1.1 [Fukuda and Shibata, 1994]. To combine their merits and overcome their demerits, some integration and synthesis techniques have been proposed. In this book, we shall mainly discuss the synergism of fusing fuzzy logic techniques and neural networks techniques into an integrated system. Neural networks are essentially low-level computational structures and algorithms that offer good performance in dealing with sensory data, while fuzzy logic techniques often deal with issues such as reasoning on a higher level than neural networks. However, since fuzzy systems do not have much learning capability, it is difficult for a human opera-
TABLE l . l Comparisons of Fuzzy Systems (FS), Neural Networks (NN), Genetic Algorithms (GA), Conventional Control Theory, and Symbolic AI*
Mathematical model Learning ability Knowledge representation Expert knowledge Nonlinearity Optimization ability Fault tolerance Uncertainty tolerance Real-time operation
FS
NN
GA
Control Theory
Symbolic AI
SG B G G G B G G G
B G B B G SG G G SG
B SG SB B G G G G SB
G B SB SB B SB B B G
SB B G G SB B B B B
The fuzzy terms used for grading are good (G), slightly good (SG), slightly bad (SB), and bad (B). Sec. 1.4
Fuzzy Neural Integrated Systems
7
tor to tune the fuzzy rules and membership functions from the training data set. Also, because the internal layers of neural networks are always opaque to the user, the mapping rules in the network are not visible and are difficult to understand; furthermore, the convergence of learning is usually very slow and not guaranteed. Thus, a promising approach for reaping the benefits of both fuzzy systems and neural networks (and solving their respective problems) is to merge or fuse them into an integrated system. This fusion of two different technologies can be realized in three directions, resulting in systems with different characteristics: 1. Neural fuzzy systems: use of neural networks as tools in fuzzy models 2. Fuzzy neural networks: fuzzification of conventional neural network models 3. Fuzzy-neural hybrid systems: incorporation of fuzzy logic technology and neural networks into hybrid systems. The first two systems represent supportive combinations, where one technology assists the other, and the third system exhibits collaborative combination, where two technologies are incorporated intimately to perform a common task. Neural fuzzy systems, or neuro-fuzzy systems, aim at providing fuzzy systems with automatic tuning abilities. With this approach, we witness the use of neural networks in learning or tuning membership functions and fuzzy rules of fuzzy systems. Neural network learning techniques can thus substantially reduce development time and cost while improving the performance of fuzzy systems. After learning, the user can understand the acquired rules in the network. With respect to learning speed, neural fuzzy systems are usually faster than conventional neural networks. Fuzzy neural networks retain the basic properties and functions of neural networks with some of their elements being fuzzified. In this approach, a network's domain knowledge becomes formalized in terms of fuzzy sets, later being applied to enhance the learning of the network and augment its interpretation capabilities. For instance, a neural network can be fuzzified in such a way that it leams the mapping between input-output fuzzy sets. Furthermore, fuzzy logic can be used to determine the learning step of neural networks according to the state of convergence. By incorporating fuzzy principles into a neural network, more user flexibility is attained and the resultant network or system becomes more robust. In a fuzzy-neural hybrid system, both fuzzy logic techniques and neural networks are utilized separately to establish two decoupled subsystems which perform their own tasks in serving different functions in the combined system The architecture of fuzzy-neural hybrid systems is usually application-oriented. Making use of their individual strengths, fuzzy logic and neural network subsystems complement each other efficiently and effectively to achieve a common goal. 1.5 ORGANIZATION OF THE BOOK This book covers basic concepts and applications of fuzzy sets, fuzzy logic, and neural networks and their integration synergism. Thus, it is logically divided into three major parts. Part I (Chaps. 2-8) covers fuzzy set theory and its applications. In these seven chapters, we cover fundamental concepts and operations of fuzzy sets, fuzzy relations, fuzzy measures, possibility theory, fuzzy logic and approximate reasoning, and their application 8
Introduction
Chap. 1
to fuzzy logic control systems and other systems such as pattern recognition systems and fuzzy expert systems. Part II (Chaps. 9-16) covers important concepts and topics involving neural networks. Single-layer and multilayer feedforward networks as well as recurrent networks are covered. Unsupervised learning, supervised learning, and reinforcement learning for training neural networks are also discussed. Genetic algorithms along with their applications (Chap. 14) are briefly introduced, and the use of GAs and other techniques for the structure learning of neural networks are explored in Chap. 15. Various applications of neural networks are then discussed in Chap. 16. Part III (Chaps. 17-21) covers the basic concepts of integrating fuzzy logic and neural networks into a working functional system. Three major integrated systems are discussed and explored: neural fuzzy (control) systems in Chaps. 18 and 19, fuzzy neural networks (for pattern recognition) in Chaps. 20 and 21, and neural fuzzy hybrid systems in Chap. 21. The rationale for their integration is discussed, and methods for realizing their integration are also explored with examples. Various applications of such integration are also considered. Finally, Apps. A and B are included to illustrate computer simulations of fuzzy logic systems and neural networks using MATLAB neural network and fuzzy logic toolboxes. Some typical and interesting examples are worked out to illustrate the characteristics of these toolboxes which will be useful for student projects and independent study.
1.6 REFERENCES A complete bibliography consisting of more than 1100 references is included at the end of the book for the reader to further pursue the subject area of interest to him or her. The bibliography is organized in alphabetical order by author and contains all the pertinent information for each reference cited in the text. In addition, the concluding remarks at the end of each chapter discuss references that are keyed to specific topics discussed in the chapter. In addition to those, the general references cited below are representative of publications dealing with topics of interest regarding fuzzy neural (integrated) systems (including fuzzy systems, neural networks, genetic algorithms, and their synergism) and related fields. They include major journals and conference proceedings. IEEE Transactions on Fuzzy Systems', Fuzzy Sets and Systems; IEEE Transactions on Neural Networks; International Neural Network Society Journal, Neural Networks', Evolutionary Computation', IEEE Transactions on Systems, Man, and Cybernetics', International Journal ofApproximate Reasoning', Adaptive Behavior, Evolutionary Computation', Complex Systems', Proceedings of IEEE International Conference on Fuzzy Systems; Proceedings of IEEE International Conference on Neural Networks; Proceedings of IEEE Conference on Evolutionary Computation; Proceedings of International Conference on Genetic Algorithms; Proceedings of International Fuzzy Systems Association (IFSA) World Congress; Proceedings of International Conference on Fuzzy Logic, Neural Nets, and Soft Computing; Proceedings of the North American Fuzzy Information Processing Society (NAFIPS) Biannual Conference; Proceedings of the NASA Joint Technology Workshop on Neural Networks and Fuzzy Logic; Proceedings of the Conference of Parallel Problem Solving from Nature; Proceedings of the Workshop on the Foundations of Genetic Algorithms. Sec. 1.6
References
9
2 Basics of Fuzzy Sets
In this chapter, we introduce the principal concepts and mathematical notions of fuzzy set theory—a theory of classes of objects with unsharp boundaries. We first view fuzzy sets as a generalization of classical crisp sets by generalizing the range of the membership function (or characteristic function) from {0, l} to a real number in the unit interval [0,1]. Various basic concepts of fuzzy sets such as representation, support, a-cuts, convexity, and fuzzy numbers are then introduced. The resolution principle, which can be used to expand a fuzzy set in terms of its a-cuts, is discussed and proved. Various set-theoretic operations and properties involving crisp sets and fuzzy sets are discussed; further fuzzy set operations such as r-norms, f-conorms, and other aggregation operations are also considered. Finally, the extension principle, which allows the generalization of crisp mathematical concepts to the fuzzy set framework, is presented with several examples. The material covered in this chapter will be used extensively in later chapters.
2.1 FUZZY SETS AND OPERATION ON FUZZY SETS A classical (crisp) set is a collection of distinct objects. It is defined in such a way as to dichotomize the elements of a given universe of discourse into two groups: members and nonmembers. Finally, a crisp set can be defined by the so-called characteristic function. Let U be a universe of discourse. The characteristic function (X) of a crisp set A in U takes its values in {0,1} and is defined such that \LA (X) = 1 if x is a member of A (i.e., x £ A ) and 0 otherwise. That is, 1 Pm (*) = " Q 10
if and only if x SEA if and only if x £ A.
(2.1)
Note that (i) the boundary of set A is rigid and sharp and performs a two-class dichotomization (i.e., x £ A or x £ A), and (ii) the universe of discourse U is a crisp set. A fuzzy set, on the other hand, introduces vagueness by eliminating the sharp boundary that divides members from nonmembers in the group. Thus, the transition between full membership and nonmembership is gradual rather than abrupt. Hence, fuzzy sets may be viewed as an extension and generalization of the basic concepts of crisp sets; however, some theories are unique to the fuzzy set framework. A fuzzy set A in the universe of discourse U can be defined as a set of ordered pairs, (2.2)
A={0t,M*))Uec/},
where is called the membership function (or characteristic function) of A and jjl^W is the grade (or degree) of membership of x in A, which indicates the degree that x belongs to A. The membership function p^(-) maps U to the membership space M, that is, U—+M. When M = { 0 , l}, set A is nonfuzzy and p,A(-) is the characteristic function of the crisp set A. For fuzzy sets, the range of the membership function (i.e., M) is a subset of the nonnegative real numbers whose supremum is finite. In most general cases, M is set to the unit interval [0,1]. Example 2.1 Let U be the real line R and let crisp set A represent "real numbers greater than and equal to 5"; then we have A={(x,ixA(.x))\xeU}, where the characteristic function is x5, J which is shown in Fig. 2.1(a). Now let fuzzy set Ar.represent "real numbers close to 5." Then we have - C
A={(X,M(.X))\X(=U},
(*) A
1
1
r
1.5
0
^
4 5 6
I
-c
xeu .
(a)
3
4
5
1
6
\ *e I
7
(b)
Figure 2.1 Characteristic functions of crisp set A and fuzzy set A in Example 2.1. 1 If there is no confusion between a fuzzy set A and a crisp set A, the tilde above the fuzzy set A will be eliminated to simplify the notation.
Sec. 2.1
Fuzzy Sets and Operation on Fuzzy Sets
11
where the membership function is 1 + 10 (x — 5)2 ' which is shown in Fig. 2.1(b). The fuzzy set A in the above example can also be represented as A = { ( x , ( i / i W ) | ^ ( x ) = [l + 1 0 ( x - 5 ) 2 ] - 1 } .
(2.3)
It is noted that we can use another membership function for fuzzy set A in Example 2.1; for example, (x r (x) ™
5-. 1 + (x - 5) 2
(2.4)
These two different membership functions show that assignment of the membership function of a fuzzy set is subjective in nature; however, it cannot be assigned arbitrarily. A qualitative estimation reflecting a given ordering of the elements in A may be sufficient. Furthermore, estimating membership functions is complicated, and a better approach is to utilize the learning power of neural networks to approximate them. The above example shows that fuzziness is a type of imprecision that stems from a grouping of elements into classes that do not have sharply defined boundaries; it is not a lack of knowledge about the elements in the classes (e.g., a particular parametric value). It is worth pointing out that |xA (x) G [0, 1] indicates the membership grade of an element x G U in fuzzy set A and that it is not a probability because X ^ W ^ ^ The grades of membership basically reflect an ordering of the objects in fuzzy set A. Another way of representing a fuzzy set is through use of the support of a fuzzy set. The support of a fuzzy set A is the crisp set of all x £ U such that (X) > 0. That is, Supp(A) = {x G U\ |xA (x) > 0}.
(2.5)
Example 2.2 Assume that in an examination, all the possible scores are U ={ 10,20,..., 100}. Consider three fuzzy sets, A = "High Score," B = "Medium Score," and C = "Low Score," whose membership functions are defined in Table 2.1.
TABLE 2.1
12
Fuzzy Sets in Example 2.2
Numerical Score
High Score (A)
10 20 30 40 50 60 70 80 90 100
0 0 0 0 0.1 0.3 0.5 0.8 1 1
Medium Score (B) 0 0 0.1 0.5 0.8 1 0.8 0.5 0 0
Low Score (C) 1 1 0.9 0.7 0.5 0.3 0.1 0 0 0
Basics of Fuzzy Sets
Chap. 2
Then we have Supp(A) = Supp ("High Score") = {50, 60,70,80,90,100}, Supp(B) = Supp ("Medium Score") = {30,40, 50,60,70, 80}, Supp(Q = Supp ("Low Score") = {10,20,30,40,50,60,70}. Instead, if U— [0,100] and
(•) is defined by the following continuous membership function,
( 0
for0<x
(2-27>
where V indicates the max operation. It is clear that ACAUB
and
BCAUB.
(2.28)
V x E U.
(2.29)
4. Equality: A and B are equal if and only if M * ) = M * )
Hence, if \i.A (x) # |xB (x) for somex G U, then A ¥= B. This definition of "equality" is crisp. To check the degree of equality of two fuzzy sets, we can use the similarity measure [Lin and Lee, 1992]: E (A, B) = degree (A = B) a
(2.30)
where fl and U denote intersection and union of A and B, respectively. When A = B, E (A, B) = 1; when |A n = 0 (i.e., A and B do not overlap at all), E (A, B) = 0. In most general cases, 0 ^ E (A, B) ^ 1. 5. Subset: A is a subset of B; that is, A C B if and only if Vx G U.
(2.31)
If A C B and A ¥= B, then A is a proper subset of B; that is, ACB. Again, the definition of subset is crisp. To check the degree that A is a subset of B, we can use the subsethood measure [Kosko, 1992a]: S (A, B) = degree (ACB)
Sec. 2.1
4
Fuzzy Sets and Operation on Fuzzy Sets
^ ^
.
(2.32)
19
Example 2.6 Consider the two membership functions of the fuzzy sets A and B in Table 2.1. Obviously, A + B, A is not a subset of B, and B is not a subset of A either. Moreover, we have A = 1/10 + 1/20 + 1/30 + 1/40 + 0.9/50 + 0.7/60 + 0.5/70 + 0.2/80, A n B = 0.1/50 + 0.3/60 + 0.5/70 + 0.5/80, AUB = 0.1/30 + 0.5/40 + 0.8/50 + l/60 + 0.8/70 + 0.8/80 + l/90 + l/lOO, F(A m V ;
'
r/A
m
lAnBl |AU5|
0.1 4-0.3+ 0.5+ 0.5 . 0.1+0.5 + 0.8 + 1+0.8 + 0.8 + 1 + 1
lAnfil \A\
0.1+0.3+ 0.5+ 0.5 0.1+0.3+0.5 + 0.8+1 + 1
1.4 3.7
1-4 6 a38
' '
"
A justification of the choice of negation, min, and max as complement, intersection, and union operators, respectively, was given in Bellman and Giertz [1973] where these operators were shown to be the only operators that meet a set of requirements (axioms). With the above definitions, the reader can verify the following facts [refer to Eqs. (2.29) and (2.31) for definitions of "equality" and "subset"]. (A D B)a =Aaf)Ba
and
(A U B)a =Aa U Ba,
but (A) a * A a .
(2.33)
As in the case of crisp sets, we have the double-negation law (involution) and DeMorgan's laws for fuzzy sets. 6. Double-negation law (involution): A=A.
(2.34)
ADB=AUB.
(2.36)
7. DeMorgan's laws:
HoweveMhe law of the excluded middle (i.e., EUE = U) and the law of contradiction (i.e., E fl E = 0 ) of the crisp set E are no longer true and valid irj fuzzy sets. That is, for fuzzy set A, AUA^U
and
ADA
¥= 0,
(2.37)
which means that because of the lack of precise boundaries, complementary sets are overlapping and cannot cover the universal set U perfectly. On the contrary, these two laws are the necessary characteristics of crisp sets, which bring together a crisp set E and its complement E to provide the whole set; that is, nothing exists between E and E. The other properties of fuzzy set operations that are common to crisp set operations are listed in Table 2.2. However, it is noted that the above discussion about the properties of fuzzy set operations is based on the definitions of complement, intersection, and union operations in Eqs. (2.24), (2.25), and (2.27), respectively. As we will see in the next section, there are other definitions of these operations and hence some properties mentioned above may fail to hold while others may become true for various definitions. As a matter of fact, it has been shown that if the excluded-middle laws hold for fuzzy sets, then union and intersection cannot be idempotent and are no longer mutually distributive [Kandel, 1986; Klir 20
Basics of Fuzzy Sets
Chap. 2
TABLE 2.2
Properties of Fuzzy Set Operations
Idempotence Distributivity Commutativity Associativity Absorption Law of zero Law of identity Double-negation law De Morgan's laws
AUA = A,AflA = A An(Buc> = ( A n s ) u ( A n o ALi(finc) = (/iuB)n(Auc) A\JB = B\JA, ADB=BnA (AUfl)UC = AU(BUC) (An5)nc=An(sno AU(ADS) = A, An(AUfl) = A A U1/ = {/, A P10 = 0 AU0=A, AC\U = A A=A AUB = AnB, ADS = AUS
andFolger, 1988]. Hence, when we choose union and intersection to combine fuzzy sets, we have to give up either excluded-middle laws or distributivity and idempotency. Next, we shall introduce some popular algebraic operations on fuzzy sets. Further fuzzy set operations will be discussed in the next section. 8. Cartesian product: Let A v A2,...,An be fuzzy sets in Ux, U2,..., Un, respectively. The Cartesian product of A p A 2 ,..., An is a fuzzy set in the product space f / j X U2 X • • • X Un with the membership function as Mv^X^X-xa.(xi>x2>-=
™
xlE.Ul,x2G.
to) U2,...,xn
> V-A2 to) • - M-A to) ]. €E Un.
(2.38)
9. Algebraic sum: The algebraic sum of two fuzzy sets, A + B, is defined by P-A+fi to £ Mvt to + M-B to - ihA to • V-B to(2.39) 10. Algebraic product: The algebraic product of two fuzzy sets, A • B, is defined by P-A B to £ M-A to • V-B
to.
(2.40)
11. Bounded sum: The bounded sum of two fuzzy sets, A ffi B, is defined by M-a«b to ^ min{ 1, (x^ (x) +
(x)}.
(2.41)
< 12. Bounded difference: The bounded difference of two fuzzy sets, A © B, is defined by V^AeB to = max{ 0, \s,A (x) - ^ (x)}.
(2.42)
Example 2.7 Let A ={(3,0.5), (5,1), (7,0.6)} and B ={(3,1), (5,0.6)}. Then we can obtain A XB ={[(3,3),0.5], [(5,3), 1], [(7,3),0.6], [(3,5),0.5], [(5,5),0.6], [(7,5), 0.6]}, A + B ={(3,1), (5,1), (7,0.6)}, A • B ={(3,0.5), (5,0i6), (7,0)}, A ® B ={(3,1), (5,1), (7,0.6)}, AeB ={(3,0), (5,0.4), (7,0.6)}. Sec. 2.1
Fuzzy Sets and Operation on Fuzzy Sets
21
2.2 EXTENSIONS OF FUZZY SET CONCEPTS In the previous section, we introduced basic definitions and operations on fuzzy sets. The membership functions were assumed to be crisp, and the membership space M was restricted to the space of real numbers. Specific operations of set complement, intersection, and union were given in Eqs. (2.24), (2.25), and (2.27), where negation, min, and max operators were adopted in these operations, respectively. These specific operations are called the standard operations of fuzzy sets that are always used in possibility theory [Zadeh, 1978a]. More on possibility theory will be presented in Chaps. 4 and 5. The basic concept covered in Sec. 2.1 can be extended in two possible directions. The first one concerns other kinds of fuzzy sets including different structures of the membership space and different assumptions about the membership functions. This extension is treated in Sec. 2.2.1. The second extension concerns the operations of fuzzy sets. It is understood that the standard operations of fuzzy sets are not the only possible generalization of crisp set operations. Several different classes of functions, which possess proper properties, have been proposed. These will be discussed in Sec. 2.2.2. 2.2.1 Other Kinds of Fuzzy Sets An extension of ordinary fuzzy sets is to allow the membership values to be a fuzzy set instead of a crisply defined degree. A fuzzy set whose membership function is itself a fuzzy set is called a type-2 fuzzy set. A type-1 fuzzy set is an ordinary fuzzy set. Hence, a type-2 fuzzy set is a fuzzy set whose membership values are type-1 fuzzy sets on [0,1]. A type-2 fuzzy set in a universe of discourse U is characterized by & fuzzy membership function p.A as [Mizumoto andTanaka, 1976] ^ C / — [ 0 , 1][)(1-7) [1 — (1 — a) (1 — b) J*
t
rG (-oo, oo) 7e(0,
\-(\-y)ab
,og|l
Sec. 2.2
,-J_ - l} r
a + b-(2-y)ab
Frank [1979]
+
Range
+
C--IK*"1) j
oo)
wG(0,oo) a£(0
"
I)
XG(0, oo) +
2
|3 maxfo fr}+0 ~ P ) (" 2
+
p e [ 0 , l ]
y parameter indicates compensation between intersection and union
Extensions of Fuzzy Set Concepts
7 f= [0,1]
25
3. Bounded sum: a © b = min (I, a + b). b — 0, a,
(2.65)
a = 0,
(2.66)
4. Drastic sum: a V b = • b, 1, S. Disjoint sum: a Ab
a,b>
0.
= max{ min (a, 1 — b), min (1 - a, b)}
(2.67)
One typical parametric f-conorm is the Yager union which is defined by the function (a, b) = min [1, (a w + bw)l/w ] ,
(2.68)
where w E (0, oo). For w = 1, the Yager union becomes the bounded sum of Eq. (2.65). We can show that when w—*co,sw(a, b) ~ max (a, b), and when w —• 0, sw (a, b) becomes the drastic sum [Klir and Folger, 1988]. It is observed that the membership grade decreases as w increases. Some other classes of parametric f-conorms are also shown in Table 2.3. The relations among various f-norms (fuzzy intersections) and f-conorms (fuzzy unions) are characterized by the following theorem. Theorem 2.2 Let A and B be fuzzy sets in the universe of discourse U. The f-norms [Eqs. (2.56)-(2.59)] are bounded by the inequalities fdp (a, b) = frai„ (a,b)^t (a, b) £ fmax (a, b) = min (a, b), (2.69) i where tip (a, b) is the drastic product in Eq. (2.59). Similarly, the r-conorms [Eqs. (2.63)(2.67)] are bounded by the inequalities max (a, b) =
(a,b)Ss
(a, fo)< jmax (a, b) = sds (a, b),
(2.70)
where .s^ (a, b) is the drastic sum in Eq. (2.66). Proof: Since the proof of Eq. (2.70) is similar to that of Eq. (2.69), we shall prove only Eq. (2.69). Using the boundary condition of the f-norm, we have t(a, 1) = a and t(1, b)= b. Then by the monotonicity condition of the f-norm, we obtain t(a,b)st(a,
l) = a
and
t(a,b)^t(l,b)
= b.
Hence, we conclude that f (a, b) £ min (a, b), which is the second inequality in Eq. (2.69). For the first inequality in Eq. (2.69), when b = 1, f (a, b) = a, and when a = 1, t (a, b) = b (boundary conditions). Hence, the first inequality holds when a = 1 or b = 1. Since t (a, b) E [0,1], it follows from the second inequality in Eq. (2.69) that f (a, 0) = f (0, b) = 0. By the monotonicity condition, we have f (a, b) 2 f (0, V) = f (a, 0) = 0, which completes the proof of the first inequality in Eq. (2.69). Hence, the standard min and max operations are, respectively, the upper bound of f-norms (the weakest intersection) and the lower bound of f-conorms (the strongest union). As mentioned before, the Yager intersection [Eq. (2.60)] and the Yager union [Eq. (2.68)] become the standard min and max operations, respectively, as w —* oo, and become the f m i n and J m a x operations, respectively, as w—• 0. Hence, the Yager class of fuzzy intersections and unions 26
Basics of Fuzzy Sets
Chap. 2
covers the entire range of the operations as given by the inequalities in Eqs. (2.69) and (2.70). As discussed in Sec. 2.1, the standard operations of fuzzy sets in Eqs. (2.24), (2.25), and (2.27) do not satisfy the law of the excluded middle and the law of contradiction, although they satisfy the properties in Table 2.2. If it is desired, we may preserve these two laws in our choice of fuzzy union and intersection operations by sacrificing idempotency and distributivity. We can easily verify that the standard complements f min and s m a x satisfy the law of the excluded middle and the law of contradiction. Another combination of these properties involves standard complement, bounded product, and bounded sum. The f-norms and f-conorms are a kind of aggregation operations on fuzzy sets, which are operations by which several fuzzy sets are combined to produce a single set. Such operations play an important role in the context of decision making in a fuzzy environment. In general, an aggregation operation is defined by [Klir and Folger, 1988] h: [0, l] n —»[0,1],
2,
(2.71)
such that HA (X) = h (fxA| (x), |xA2 (x)
m ^ (x))
V x E U.
(2.72)
As indicated by Eqs. (2.69) and (2.70), all f-norms are smaller than the min operator intersection function and all f-conorms are greater than the max operator union function. Naturally, there is a class of aggregation operators lying between the min operator and the max operator. These operators are called averaging operators since they realize the idea of trade-offs between the minimum and the maximum degree of membership of the aggregated sets. Hence, averaging operators are aggregation operators for which min (a l5 a2„.an)
< h
a2,..„ an) < max (av a^,..., an),
(2.73)
where at, = |xA (x), i = 1,..., n. One typical parametric averaging operator is the generalized means which is defined as ha{ava2,...,an)
/ £ + t£ + • • • + a"\Va 1 —J ,
(2.74)
where a E R but a # 0. It can be verified that when a approaches -oo, ha (av a2,..., an) becomes m i n ^ , a 2 , . . . , a n ) , and when a approaches oo, ha(al,a2,...,ar) becomes max ( a p a2,..., an). Hence, the generalized means covers the entire interval between the min and the max operators. The full scope of fuzzy aggregation operators is shown in Fig. 2.6. An important extension of the generalized means is the weighted generalized means defined by ha{ax, a2,..„ a„; w,, w2,..„ wj a
+ w2aa2 + • • • +
(2.75)
where > 0 and , wt = 1. The wi are weights expressing the relative importance of the aggregated sets. The weighted generalized means are useful in decision-making problems where different criteria differ in importance (weights). More averaging operators can be found in [Zimmerman, 1991], Finally, based on f-norms and f-conorms, we shall further introduce some operations of fuzzy sets that are central to fuzzy logic and fuzzy reasoning. They will be useful in Chaps. 6 and 7. In the following, let A and B be fuzzy sets in universal sets U and ^ respectively. Sec. 2.2
Extensions of Fuzzy Set Concepts
27
0
Yager class 5 *w
Yager class
S
w
0
Generalized means St o
—o'.
max
min f-norms (Intersection operations)
f-conorms (Union operations)
Averaging operations
Aggregation operations Figure 2.6 The full scope of fuzzy aggregation operators. 1. Fuzzy conjunction: The fuzzy conjunction of A and B is denoted as A A B and defined by HAAB
to
Y)
= ' (^A
to,
(y)),
where t is a f-norm.
(2.76)
2. Fuzzy disjunction: The fuzzy disjunction of A and B is denoted as A V B and defined by MVWB
to y) = s (JJLA ( X ) , |xB (y)),
where-s is a f-conorm.
(2.77)
3. Fuzzy implication: The fuzzy implication of A and B is denoted asA—*B and has five different definitions [Lee, 1990]. In the following, f is a f-norm, s is a f-conorm, and the complement is the standard complement. Material implication: A —• B ~ s (A, B).
(2.78)
Prepositional calculus: A
(2.79)
B = s (A, f (A, B)).
Extended prepositional calculus: A—>B = s(AXB,B).
(2.80)
Generalization of modus ponens: A—>B = sup{ k E [0,1], f (A, k) < B}.
(2,81)
Generalization of modus tollens: A ->• B = inf{ k G [0,1 ] , 5 (fl, k) < A}.
(2.82)
Example 2.8 Let us see some special cases of the above operations. If f = min operator, s = max operator, and the material implication is adopted, then we have HAAB to Y) = m i n ( M * ( X ) , | x a M-AVSTOY) =
M-A-S to y)
ma* (HA to.
(Y)),
HB
to) -
= max (1 - |xA (x),
(2.83)
(y)).
If f = algebraic product, s = algebraic sum, and the material implication is adopted, then we have M-AAB to y ) ~
M-A TO H B ( Y ) >
HAVB to Y ) ~ HA to + HB(Y) P-A-B
28
to y) ~
1 -
M-A TO +
-
HA to HB TO >
HA TO HB
(2.84)
TO-
Basics of Fuzzy Sets
Chap. 2
If t = bounded product, s — bounded sum, and the material implication is adopted, then we obtain Haabtoy) = max [0, (xA (x) + |x s (y) - 1], M*vfito;y) =
min
[LM7tto
+
Hi>Cy)]>
Ha-b to y) = min [1,1 - \x.A (x) +
(2.85) (y) ].
2.3 EXTENSION PRINCIPLE AND ITS APPLICATIONS The extension principle, introduced by Zadeh [1978a], is one of the most important tools of fuzzy set theory. This principle allows the generalization of crisp mathematical concepts to the fuzzy set framework and extends point-to-point mappings to mappings for fuzzy sets. It provides a means for any function/that maps an n-tuple (JC1? x2,..., xn) in the crisp set U to a point in the crisp set V to be generalized to mapping n fuzzy subsets in U to a fuzzy subset in V. Hence, any mathematical relationship between nonfuzzy elements can be extended to deal with fuzzy entities. Furthermore, the extension principle is very useful for dealing with set-theoretic operations for higher-order fuzzy sets. We shall first state the extension principle and then show some of its applications. Given a f u n c t i o n / : U—>- V and a fuzzy set A in U, where A = JJL 1/JC1 + | x 2 / x 2 + • • • + |x n /x n , the extension principle states that / (A) =/(m-iAI + \hJX2 + • • • + l^nAn) =
M-I //
(2.86)
to) + v j f to) + • • • + v-Jf to)-
If more than one element of U is mapped to the same element y in V by/(i.e., a many-toone mapping), then the maximum among their membership grades is taken. That is, Hf{A) 0 0 = max [M-a to) ]'
(2.87)
m=y where xi are the elements that are mapped to the same y. Quite often, the function/that is of interest maps n-tuples in U to a point in V. Let U be a Cartesian product of universes U = Ul X U2 X • • • X Un andAp A 2 ,..., An be n fuzzy sets in t/,, U2,..., Un, respectively. The function/maps an n-tuple (JCJ, x2,...,xn) in the crisp set Uto a point y in the crisp set V; that is, y =/ (xv x2,..., xn). The extension principle allows the function/ (xv x2,..., xn) to be extended to act on the n fuzzy subsets of U, A,, A 2 ,..., An, such that B=f(A),
(2.88)
where B is the fuzzy image (fuzzy set) of A,, A 2 ,..., An through / (•). The fuzzy set B is defined by B={(y,\i.B(y))\y=fXxvx2,...,xn),
(xvx2,...,xn)GU},
(2.89)
where M-BO) =
sup min [|xA] ( x j , ^ (xl>x2 xJeU
y=f(xl,x2
Sec. 2.3
xn)
Extension Principle and Its Applications
( x
2
) ^ (xn) ], "
(2.90)
29
with an additional condition that fi g (y) = 0 if there exists no (x,, x 2 ,..., x n ) G J7 such that y = / ( x 1 , x 2 , . . . , x n ) . Let us consider several numerical examples to illustrate this important extension principle. Example 2.9 Let U ={ 1,2,3,4,5,6,7,8,9,10}. A fuzzy set A = "Large" is given as A = "Large" = 0.5/6 + 0.7/7 + 0.8/8 + 0.9/9 + l/lO. If the function/ is a squaring operation and is indicated by y = =f(x) = x2, then by the extension principle, the fuzzy set B = "Large"2 can be easily calculated as B = "Large"2 = 0.5/36 + 0.7/49 + 0.8/64 + 0.9/81 + l/lOO. Example 2.10 We are given U ={-2, -1,0,1,2} and a fuzzy set A ={ (-1,0.5), (0,0.8), (1,1), (2,0.4)}. The function/ is a squaring operation and is indicated by y=f(x) = x2. It is easier to see if one constructs a table listing all the relevant information and calculations: Y=/(x) = x2
* 0.5 0.8 1.0 0.4
-1 0 1 2
M-BOO
max{0.5,1.0} = 1.0 max{0.8} = 0.8 max{0.5,1.0} = 1.0 max{0.4} = 0.4
1 0 1 4
From the above calculations, it is obvious that we have two points (-1 and 1) in U mapped into a single point y = 1. The membership grade of y = 1 is taken as the maximum membership grade of x = -1 and x = 1, which gives us 1. Hence, the fuzzy image B is B = l / l + 0.8/0 + 0.4/4. Example 2.11 Suppose / is a function mapping ordered pairs from Ul ={-1,0, l} and U2 ={ - 2,2} to V ={-2, -1,2, 3}, and/ (x,, x2) = xf + x2. Let A, and A2 be fuzzy sets defined on Ux and U2, respectively, such that A, = 0.5/-1 + 0.1/0 + 0.9/1 and A2 = 0.4/-2 + 1.0/2. Use the extension principle to derive/(Aj, A2). The calculation process can be illustrated in the following table.
-1 -1 0 0 1 1
M-A,
*2
MA2
0.5 0.5 0.1 0.1 0.9 0.9
-2 2 -2 2 -2 2
0.4 1.0 0.4 1.0 0.4 1.0
y=f{xl,x2) = x[ + x2 min{0.5,0.4} min{0.5, 1.0} min{0.1,0.4} min{0.1, 1.0} min{0.9,0.4} min{0.9,1.0}
-1 3 -2 2 -1 3
From these table calculations, it is clear that we have two ordered pairs (-1,-2) and (1, -2) that map to the same point y = -1, and (-1,2) and (1,2) are mapped to the same point y = 3. Thus, their respective maximum membership grades must be taken. MaO^-l) = max [min (p,Al (x, = -1),
(x2 = -2)), min (|xA] (x, = 1),
(x2 = -2)) ]
= max [min (0.5,0.4), min (0.9,0.4) ] = 0.4,
30
Basics of Fuzzy Sets
Chap. 2
HB
(-2) = max [min (p,A[ (0) , \XAL (-2)) ] = max [min (0.1,0.4) ] = 0.1, (2) = max [min (n.A[ (0),
(2)) ] = max [min (0.1,1) ] = 0.1,
HB (3) = max [min (p,A) (-1), \LAi (2)), min (n,A) (1), ji,Aj (2)) ] = max [min (0.5,1), min (0.9,1) ] - 0.9. The fuzzy set B obtained from the extension principle is B = 0.1/-2 + 0.4/-1 + 0.1/2 + 0.9/3. Justification or proof of the extension principle based on Zadeh's rule of fuzzy compositional inference can be found in [Yager, 1986]. We shall next show some applications of the extension principle.
2.3.1 Operations of Type-2 Fuzzy Sets The extension principle can he used to define the operations of intersections (e.g., an algebraic product), union (e.g., an algebraic sum), and complement of type-2 fuzzy sets. Let p.A(x) and |xB(x) be fuzzy grades for type-2 fuzzy sets A and B, respectively, and they are defined by w e [0,1],
u G [0, 1],
(2.91)
where/and g depend on x as well as on u or w. Using the extension principle, we haye the following: 1. Min operator [A fl B; see Eq. (2.56)]: HA 10 - t 2.2 Using the resolution principle, express the following fuzzy sets in the form of Eq. (2.16): (a) A = 0.1/2 + 0.4/3 + 0.5/4 + 0.6/5 + l/7 -I-1/8 + 0.5/9, (b) B = 0.1/2 + 0.4/-1 + 0.1/2 + 0.9/3, (c) C = 0.1/0.04 + 0.3/0.08 + 0.5/0.16 + 0.7/0.32. 2.3 Determine the convexity of the fuzzy sets A, B, and C in Prob. 2.1. 2.4 Show that a fuzzy set A is convex if and only if JM (Xx, + [ 1 - X ] x 2 ) > min [|xA (x,), (JLATO)1. wherex,, x2 6 U, X S [0,1], 2.5 Given the following fuzzy sets, verify whether they are also fuzzy numbers: (a) 0.1/1 + 0.8/2 + 0.9/3 + 0.4/4 + 0.5/5, (b) 0.1/7 + 0.1/8 + 1/9 + 0.2/10 + 0.2/11, (c) fvH to/x, where (a (x) = x, x E U, U = [0,1], (d) Jyn, (x)/x, where n (x) = x2, x G U, U = [0,1], 2.6 Show that the S function and the u function, defined in Eqs. (2.18) and (2.19), respectively, are fuzzy numbers. 2.7 Given fuzzy sets A and B as A = 0.2/1 +0.9/2 + 0.7/3 + 0.6/4 + 0.1/5 and B = 0.3/1 + 1.0/2 + 0.5/3 + 0.4/4 + 0.1/5, compute: (a) The subsethood value S (A, B) and S (B, A), (b) Their fuzzy similarity measure E (A, B), (c) Their difference, Difference (A, fl) £ 1— E (A, B). 2.8 Repeat the calculations in Prob. 2.7 for the following fuzzy sets A and B: 2-x
if 1 S x < 2
x
ifOsxCl
0
otherwise,
'3 — 2x Ms to =
34
1
ifl<x) HAns W ' ( c ) Ha$b (•*) - (d) MJ (x).
36
Basics of Fuzzy Sets
Chap. 2
3 Fuzzy Relations
A crisp binary relation represents the presence or absence of association; interaction, or interconnectedness between the elements of two sets. Fuzzy binary relations are a generalization of crisp binary relations to allow for various degrees of association between elements. Degrees of association can be represented by membership grades in a fuzzy binary relation in the same way that degrees of set membership are represented in the fuzzy set. Thus, fuzzy binary relations are in fact fuzzy sets. This chapter introduces basic concepts and operations on fuzzy relations, and both the relation-relation composition and the setrelation composition are studied via the max-min and min-max compositions. This eventually leads us to fuzzy relation equations which play an important role in areas such as fuzzy system analysis, design of fuzzy controllers, and fuzzy pattern recognition. Various notions of crisp relations such as equivalence and partial ordering are generalized to fuzzy similarity and fuzzy partial ordering.
3.1 BASICS OF FUZZY RELATIONS The notion of relations is basic in science and engineering, which is essentially the discovery of relations between observations and variables. The traditional crisp relation is based on the concept that everything is either related or unrelated. Hence, a crisp relation represents the presence or absence of interactions between the elements of two or more sets. By generalizing this concept to allow for various degrees of interactions between elements, we obtain the fuzzy relation. Hence, a fuzzy relation is based on the philosophy that everything is related to some extent or unrelated. More specifically, crisp relations and fuzzy relations can be defined in terms of subsets. 37
A crisp relation among crisp sets X},X2,...,Xn is a crisp subset on the Cartesian product X j X X2 X • • • X Xn. This relation is denoted"by R (Xv X2,..Xn). Hence R(XvX2,...,Xn)
C X , XX2 X • • • X X ,
(3.1)
where X , X X 2 X - • • XXn = {(xvxr...,xn)\x.<E
Xt
for all i €=. {l,..., n}}. _
(3.2)
It is interpreted that the relation R exists among {XpX2,..., X j if the tuple (xl,x2,...,xn) is in the set R (Xj,X 2 > ...,X n ); otherwise the relation R does not exist among {Xt, X 2 , . . . , X J . A fuzzy relation is a fuzzy set defined on the Cartesian product of crisp sets {Xt, X 2 ,..., X J , where tuples (xl,x2,...,xn) may have varying degrees of membership (xrx2,...,xn) within the relation. That is,
/?(X 1 ,X 2 ,...,X n )=J ;f ^ x
^^(x,,^,...,^,^,...,^),
x.GX,
(3.3)
In the simplest case, consider two crisp sets Xu X 2 . Then R(XU X 2 ) = {((*„ x2), ^ ( x , , x2)) | (x„ x2) E X , X X 2 }
(3.4)
is a fuzzy relation on Xy X X 2 . It is clear that a fuzzy relation is basically a fuzzy set Example 3.1 Let X = {xhx2} = {New York City (NYC), Tokyo (TKO)} and F = {y„y2.>3} = {Taipei (TPE), Hong Kong (HKG), Bejing (BJI)}. Let R represent the relation, "very close." If R is a crisp relation, it might be defined by the following characteristic function (xiT yj): y\
yi
>3
TPE HKG BJI x,
NYC
0
0
0
x2
TKO
1
1
1
However, if R is considered a fuzzy relation, it may be defined as having the following membership function (x;, yj): y\ >2 ?3 TPE HKG BJI X,
NYC
x 2 TKO
0.3
0.1
0.1
1
0.7
0.8
That is, R (X, Y) = 0.3/(NYC, TPE) + 0.l/(NYC, HKG) + 0.l/(NYC, BJI) + l/(TKO, TPE) + 0.7/(TKO, HKG) + 0.8/(1X0, BJI). Example 3.2 Let X = Y = R1, the real line. For x G X, y S Y, the fuzzy relation R = "y is much larger than x" on XX Y can be defined by the following membership function: 38
Fuzzy Relations
Chap. 3
for x a y (*» y) —
for x < y .
l + [ 5/ ( y - x f ]
A special fuzzy relation called a binary (fuzzy) relation plays an important role in fuzzy set theory. A binary fuzzy relation is a fuzzy relation between two sets X and Y and is denoted by R (X, Y). When X ¥=Y, binary relations R (X, Y) are referred to as bipartite graphs. When X = Y, R is a binary fuzzy relation on a single set X, and the relations are referred to as directed graphs, or digraphs, and are denoted by R (X, X) or R (X2). The relations defined in Examples 3.1 and 3.2 are both binary fuzzy relations. There are convenient forms of representation of binary fuzzy relations R (X, Y) in addition to the membership function. L e t X ={xvx2,...,xl} and Y = { y p y2,---,y„}- First, the fuzzy relation R(X, Y) can be expressed by an n X m matrix as M-R TO' M * 2 >
y2)
M'*(*i, y j "
Hr to' y2)
HR(x2,ym)
Hr to> y2)
p-r to- yJ-
R(X, Y) = -M*„,:Vi)
(3.5)
Such a matrix is called & fuzzy matrix and is used in Examples 3.1 and 3.2. A binary fuzzy relation can also be expressed by a graph called a fuzzy graph. Each element in X and Y (denoted by JC, and yJt respectively) corresponds to a vertex (node) in the fuzzy graph. Elements of X X Y with nonzero membership grades in R (X, Y) are represented in the graph by links (arcs) connecting the respective vertices. These links are labeled with the value of the membership grade (xh yj). When X each of the sets X and Y can be represented by a set of vertices such that vertices corresponding to one set are clearly distinguished from nodes representing the other set. In this case, the link connecting two vertices is an undirected link and the fuzzy graph is an undirected binary graph, which is called a bipartite graph in graph theory. On the other hand, when X = Y, the fuzzy graph can be reduced to a simpler graph in which only one set of vertices corresponding to set X is used. In this case, directed links are used and a vertex is possibly connected to itself. Such graphs are called directed graphs. Example 3.3 Let X
x2, x}, x4}. Consider the following binary fuzzy relation on X:
(
X
l
*2
*3
x4
0.3
0
0.7
0
0
1
0.9
0.6
0.2
0
0.5
0
0.3
0
1
R(X,X) = x4
Then the bipartite graph and simple fuzzy graph of R (X, X) are shown in Fig. 3.1(a) and (b), respectively. Sec. 3.1
Basics of Fuzzy Relations
39
Figure 3.1 Graph representations of a binary fuzzy relation in Example 3.3. (a) Bipartite graph, (b) Simple fuzzy graph.
The domain of a binary fuzzy relation R (X, Y) is the fuzzy set "dom R (X, Y)" with the membership function M-domR to = m a x
yer
M-* to y)
for e a c h x G X.
(3.6)
The range of a binary fuzzy relation R (X, 10 is the fuzzy set "ran R (X, F)" with the membership function ax I W 0 ' ) = mxSX M*.:>')
for each y 6 F.
(3.7)
Analogous to the height of a fuzzy set, the height of a fuzzy relation R is a number H(R) defined by H (R) = sup sup (x, y). (3.8) yBYxGX
If H (R) = 1, then R is a normal fuzzy relation; otherwise it is a subnormal fuzzy relation. Similar to the resolution principle for fuzzy sets, every binary fuzzy relation R (X, Y) can be represented in its resolution form
U
(3.9)
where AR is the level set of R and <xRa is defined as in Eq. (2.15). Hence, a fuzzy relation can be represented as a series of crisp relations comprising its a-cuts, each scaled by the value a . Example 3.4 Consider the fuzzy relation
40
0.4
0.5
0
0.9
0.5
0
0
0
0.3
0.3
0.9
0.4
Fuzzy Relations
Chap. 3
Using the resolution principle, R can be expressed as R = 0.3Rq3 + OAR04 + 0.5RQ5 + 0.9^9 "1
1
0"
1
1
0
0
0
1
.1
1
1.
"1
1
0"
1
1
0
0
0
0
.0
1
1.
+ 0.4
"0
1
0"
1
1
0
0
0
0
.0
1
0.
+ 0.5
+ 0.9
"0
0
0"
1
0
0
0
0
0
_0
1
0.
3.2 OPERATIONS ON FUZZY RELATIONS Since the fuzzy relation from X to Y is a fuzzy set in X X T , the operations for fuzzy sets discussed in Sec. 2.1 can be extended to fuzzy relations. In this section, we shall introduce some operations that are specific for fuzzy relations. Given a fuzzy relation R (X, Y), let [R | Y] denote the projection of R onto Y. Then [/? \ Y] is a fuzzy set (relation) in Y whose membership function is defined by =
^[/yri (
V-r
to
(3-10)
Since the standard max operation is a f-conorm, the above projection can be generalized by replacing the max operator with any f-conorm [see Eqs. (2.63)-(2.67)]. This projection concept can be extended to an n-ary relation R(Xly X2,• ••, X J . Example 3.5 Consider the fuzzy relation >2
yi i / 0.2
x
0.1 *3 ^0.9
y3
y4
ys
0.5
I
0
0.6\
0
0.7
0.4
0
0.2
0
0.2
1 /
Then = 1/jcj + 0.1+ .
l/x3,
Y] = 0.9/y, + 0.5/y 2 + l/y3 + 0.4/y A + l/y5.
Consider a 3-ary (ternary) fuzzy relation. Let X1 ={ a, b}, X2 ={ c, d}, X3 = { f , g), and R (X,, X2, X3) = 0.3/acf + OJ/adf + 0.1 /beg + 0.8/bdf + l/bdg. Then ^ 1.2 =
= 0.3 lac + 0.1/ad + 0.1 /be + l/bd,
J?I>3 - [/?i{X„ X 3 }] = 0.7/af + O.S/bf + 1/bg, «2,3 = [i?i{X 2 ,X 3 }] = 0.3/cf + 0.1 /eg + 0.8/df + l/dg,
Sec. 3.2
Operations on Fuzzy Relations
41
Rl = [R{Xx\ = 0.l/a+l
/b,
R2 — [i?lX2] = 0.3/c + \/d, R3 - [/?IX3] = 0.8// + l/g. Given a fuzzy relation R (X) or a fuzzy set R on X, let F] denote the cylindric extension of R into Y. Then Y] is a fuzzy relation in X X Y with the membership function defined by ^[/fm
to
=
f^/f
to
for
every x G X, y G Y.
(3.11)
Equivalently, we can use the Cartesian product to define the cylindric extension. Assume X X Y is the whole Cartesian product space. Let R .be a fuzzy set on X, where X = Then the cylindric extension of R into Y ={jj,y 2 >---»y m } can be obtained by [/?TF]=/?X7,
(3.12)
where we consider that Y — l / y j + 1 fy2 + • • • + 1 /ym. Similarly, if R is a fuzzy set on Y, then [R]X]=XXR.
(3.13)
For example, assume X ={xv x j and Y ={yj, y2}. Let R = \lr (xx)/Xj + y = V^I + l/y 2 -'Then
( x 2 ) / x 2 and'
[Kt Y] = « X Y = ^ (x^Axj, y,) + ^ ( x , ) / ^ , y 2 ) +hr
to)/to-
+
to)/
to'
>2)'
which is exactly what Eq. (3.11) states. The concept of cylindric extension can be used to extend an r-ary relation R(Xh X2,...,Xr) to an n-ary relation 7?(X 1 ,X 2 ,...,X r ,...,X„), w h e r e n > r. Example 3.6 Continuing from Example 3.5, we have 1
1
1
1
1
[ [ * i x n r ] = 0.7
0.7
0.7
0.7
0.7
1
1
1
1.
. 1 , "0.9
0.5
1
0.4
[ [ / ? i n t x ] = 0.9 .0.9
0.5
1
0.4
1
0.5
1
0.4
1.
1"
Also, the membership functions of the cylindric extension of Rt 2, R13, R2}, and R{ in Example 3.5 are For example, ^["1.2
42
=
^ l . j U s l ^ ' d ' s ) = M*ii2(a, d) = 0.7.
Fuzzy Relations
Chap. 3
[R,T{X2, X3H
K I T * J
K A T X J
0.3
0.7
0.3
0.7
0.3
0
0.1
0.7
0.7
0.7
0.8
0.7
0.7
0
1
0.7
0.1
0.8
0.3
1
0.1
1
0.1
1
1
0.8
0.8
1
1
1
1
1
(a,c,f) (a, c, g) (a,d,f) (a,d,g) 0b,c,f) (b, c, g) (b,d,f) Cb,d,g)
FromEq. (3.12), it is clear that the cylindric extension produces the "largest" fuzzy relation (in the sense of membership grades of elements of the extended Cartesian product) that is compatible with the given projection. The cylindric extension thus derives its most nonspecific n-dimensional relation from one of its r-dimensional projections, where n> r. When, we compare the fuzzy relation R(X,Y) in Example 3.5 and [ [/? IX] t y] in Example 3.6, we find that they are quite different. In fact, fuzzy relations that can be reconstructed from one of their projections by cylindric extension are rather rare. In order to reconstruct a fuzzy relation that will be as close to its original relation as possible, it is desirable to reconstruct the fuzzy relation from several of its projections. The cylindric closure provides an approximate solution to this problem. Let Y = Fj X Y2 X • • • X YN be the whole Cartesian product space, where Yi may be more than one dimension. Given a set of projections of a fuzzy relation {/?,|/?£ is a projection on Y^ i £ [1, n]}, the cylindric closure of these projections, denoted as cyl{#,}, is defined by cyi{/?i}=[/?1T(y-y1)]n[/?2T(y-r2)]n---n[^nT(y-yn)],
(3.14)
where (Y - YJ) is the Cartesian product space without (i.e., y t X y 2 X • • • X y,_! X y I + I X • • • X y„) and the intersection FL can be any f-norm operator. However, the min operator is used if the max operator is used to determine the projections. Example 3.7 The cylindric closures of R{ 2 , R, CYIK,2,/?,
3
,/?2
3}
=
and R2 3 in Examples 3.5 and 3.6 are
[/?,,2TX3]
n
[/?, 3 T X 2 ]
= 0.3 lacf + Q.l/adf
n [tf 2>3 tx,]
+ 0.1 Jbcf + 0.1 /beg + 0.8 /bdf
+
l/bdg,
which is very close to the original relation R (X,, X2, X3) in Example 3.5. It is noted that when we are restricted to the binary fuzzy relation R (X, Y), the cylindric closure reduces to the Cartesian product in Eq. (2.38). That is, let RX = [i? IX] and Ry£ [J? 1 7 ] , and then cyl{Rx,
= [RX f y] fl = —•
x y] n [X x /? y ] X
Operations on Fuzzy Relations
(3.15)
Ry.
It is clear that we have the relation R (X, Y)CRXX
Sec. 3.2
f X]
RY.
43
Example 3.8 Consider the relation R (X, Y) used in Examples 3.5 and 3.6. We have
Axxnr=[[/nx]Tr]n[[*int*I=
0.9
0.5
1
0.4
1
0.7
0.5
0.7
0.4
0.7
0.9
0.5
1
0.4
1
Comparing Rx X Ry with R (X, Y) in Example 3.5, we find that R (X,
Y)CRxXRY.
For a fuzzy relation R (X, Y), the inverse fuzzy relation R~l (X, Y) is defined by \iR-1 (y,x) £ inR (x, y)
for all {x,y)6XX
Example 3.9 Let R (X, Y) be a fuzzy relation on X = {x,, x2, x^} and Y
Y.
(3.16)
>>2} such that
y\
X, /0.2 R(X,Y)
= x2
1 \
0
0.4
0.8
0
Then ,
R-1(X,Y)
_
= Rr(X,Y)=
/0.2
(
y2 V 1
0
0.8\
0.4
0 /
^^
J.
Another important operation of fuzzy relations is the composition of fuzzy relations. Basically, there are two types of composition operators: max-min and min-max compositions. These compositions can be applied to both relation-relation compositions and setrelation compositions. We shall first consider relation-relation compositions. Let P (X, Y) and Q (Y, Z) be two fuzzy relations on X X Y and Y X Z, respectively. The max-min composition of P (X, Y) and Q (Y, Z), denoted as P (X, Y) o Q(Y,Z), is defined by Hpoq <x, z) = max min [fi p (x, y),
(y, z) ]
f o r all x £ 1 , z E Z.
(3.17)
Dual to the max-min composition is the min-max composition. The min-max composition of P (X, Y) and Q (Y, Z), denoted as P (X, Y) • Q (Y, Z ) , is defined by v-pnQ to d =
Cm-p to y), M-e 0'. z) ]
for all x £ X, 2 £ Z.
(3.18)
From the above definition, it is clear that the following relation holds:
P(x, Y) O Q(Y, Z)-P
(X, Y) O Q(Y, Z).
(3.19)
The max-min composition is the most commonly used. Equation (3.17) indicates that the max-min composition for fuzzy relations can be interpreted as indicating the strength of the existence of a relational chain between the elements of X and Z. 44
Fuzzy Relations
Chap. 3
Example 3.10 Let X ={*,, x j , Y ={yi,y 2 ,y3}, and Z={z,,z 2 }. Given two fuzzy e(r,z)as Zl yi y3 y\ /0.5 x, /0.3 0 0.7> 0.7\ Q(Y,Z) = y2 0 P (X, Y) = jc2\0.8 1 0.4/ >3 \0.2 Then fo.: >.3 HP°Q(x,Z)
= I
0.7
0.8
0.4
]
0.5
1
0
0.9
0.2
0.6
.5) V (0 AO) V (0.7 A 0.2) (0.3 A 1 ) _ r (0.3 AO.;
V
relations P(X, Y) and z2 1 \ 0.9 0.6/
(0 A 0.9) V (0.L 7 A 0 . 6 ) ~ |
5) V (1 A 0) V (0.4 A 0.2) (0.8 A 1 ) v (1 A0.9) V (0, L4 L (0.8 AO.: Zl
_ x , /0.3 x 2 \0.5
A0.6)J
z2 0.6\ 0.9/'
The max-min composition can be generalized to other compositions by replacing the min operator with any f-norm operator. In particular, when the algebraic product of Eq. (2.57) is adopted, we have the max product composition denoted as P (X, Y)Q Q (Y, Z) and defined by Hpoq to z) =
Hp to y)' M-Q to z)]
for all x G X, z G Z.
(3.20)
Similarly, the min-max composition can be generalized to other compositions by replacing the max operator with any f-conorm operator. Since from Eqs. (2.69) and (2.70), the min operator is a maximal f-norm and the max operator is the smallest f-conorm, we obtain the following inequalities characterizing the range of all possible results of compositions: max f [|x p (x, y ) , |x 0 (y, z) ] < max min [p-p (x, y), \l0 (y, z) ] ,
(3.21)
m i n j [ | x p ( x , y ) , | x 0 ( y , z ) ] > m i n m a x [ ( j L F ( x , y ) , p .u0 ( y , z ) ] . (3.22) ySY * yBY Similar to the properties of the composition of crisp binary relations, the max-min, max product, and min-max compositions have the following properties: P ° Q ¥= Q o P, (P ° QYl = Q~l ° (P o 0
(3.23) P'\
o R = Po (Q o R).
(3.24) (3.25)
Example 3.11 Assume there are four courses offered: fuzzy theory (FT), fuzzy control (FC), neural networks (NNs), and expert systems (ESs). Three students, Peter, Mary, and John, are planning to take one of these courses based on their different preferences for theory, application, hardware, and programming. We shall show that the composition of fuzzy relations can help them in their decision making. Sec. 3.2
Operations on Fuzzy Relations
45
Let X={Peter, Mary, John}, Y ={yl,y2,y3,y3,y4} ={ theory, application, hardware, programming }, and Z = { FT, FC, NN, ES}. Assume the student's interest is represented by the fuzzy relationP(X,Y): yi yi >4 Peter (0.2 I 0.1\ 0.8 P (X, 7) = Mary
1
0.1
0
John \0.5
0.9
0.5
0.5 1>
The properties of the courses are indicated by the fuzzy relation Q (Y, Z):
Q(Y,Z) =
FT yi / 1 yi 0.2 0 >4 1,0.1
FC 0.5
NN 0.6
ES 0.1N
1
0.8
0.8
0.3
0.7
0
0.5
0.8
1 >
Then, the max-min composition of P (X, Y) and Q (Y, Z) can help these students to choose the proper courses: * FT FC NN ES Peter /0.2 1 0.8 0.8 P o Q = Mary
1
John ^0.5
0.5
0.6
0.5
0.9
0.8
1
Hence, Peter is encouraged to take the fuzzy control course, Mary should take the fuzzy theory course, and John should take the expert systems course. As in the case of relation-relation compositions, we have max-min and min-max compositions for set-relation compositions. Let A be a fuzzy set on X and R (X, Y) be a fuzzy relation on X X Y. The max-min composition of A and R {X, Y), denoted as A ° R (X, Y), is defined by |x Ao/J (y) £ max min xex
(x), p.* (x, y) ]
for all yGY.
(3.26)
The dual composition, the min-max composition of A and R (X, 7 ) , is denoted as A • R (X, Y) and is defined by M-a• r OO = ™n max
to,
^ (x, y) ]
for all yz)] foreachxGX,y e F , a n d z e Z . 46
Fuzzy Relations
(3.28) Chap. 3
Like the max-min compositions, the min operator can be generalized by replacing it with any f-norm operator. Example 3.12 Let fuzzy relations P and Q be defined as in Example 3.10. Then Pit Q is defined as in the accompanying table x
y
z
\i-P{rQ(x,y,z)
yi yi
Zl
*1 *1 *2 *2 " X2 *2 *2
y$ y\ y\ yi y-i y*
0.3 0.3 0.2 0.6 0.5 0.8 0.9 0.2 0.4
Zl 22 Zl Z2 Zl Z2
3.3 VARIOUS TYPES OF BINARY FUZZY RELATIONS In this section, we are concerned with binary fuzzy relations on a single set X. Various important types of binary fuzzy relations ace distinguished on the basis of three different characteristic properties: reflexivity, symmetry, and transitivity. A fuzzy relation R (X, X) is reflexive if and only if / ' p.,, (x, x) = 1
for all x G X.
(3.29)
If this is not the case for some x G X , then R (X, X) is irreflexive. If Eq. (3.29) is not satisfied for all x G X, then the relation is called antireflexive. A fuzzy relation R (X, X) is symmetric if and only if (x, y) —
(y, x)
for all x, y G X.
(3.30)
If-it is not satisfied for some x, y G X, then the relation is called asymmetric. If the equality is not satisfied for all members of the support of the relation, then the relation is called antisymmetric. If Eq. (3.30) is not satisfied for all x, y G X, then R (X, X) is called strictly antisymmetric. A fuzzy relation R (X, X) is transitive (or, more specifically, max-min transitive) if and only if M-* to y) - max min ySY
(x, y), y.R (y, z) ]
for all (x, z) G X 2 .
(3.31)
If this is not true for some members of X, R (X, X) is called nontransitive. If the inequality of Eq. (3.31) does not hold for all (x, z) G X 2 , then R (X, X) is called antitransitive. The max-min transitive can be generalized to alternative transitives by replacing the min operator with any f-norm. For example, using the algebraic product, we obtain the max product transitive which is defined by ^ to z) Sec. 3.3
max ySY
(x, y) \x,R (y, z) ]
Various Types of Binary Fuzzy Relations
for all (x, z) G X 2 .
(3.32) 47
Example 3.13 The following are examples of fuzzy relations that satisfy the above properties. R.=
1
0.8
0.3"
"0.3
1
0.9"
0.3
1
0.6
0
0.7
0.2
.0.4
0
1.
.0.5
0
1
0
0.6"
"1
0
0.3
0.8
0.1
0.3
0.8
0.7
0.5.
.0.5
0
0.5.
«„=l 0 .0.5
0.5
0.7"
0,
0.3
0.1
0.3.
_0.7
0.1
0.
0.6"
p.i
0.5
0.7"
1
0.2
0.3
0.2.
P
o .0
It is clear that Ra is reflexive, Rb is antireflexive, Rc is symmetric, Rd is antisymmetric, R„ if strictly antisymmetric, and Rf is transitive. The fuzzy relation in Rf is transitive because "0.1 *r-
0.5
0
1
.0
0.3
0.7"
"0.1
0.5
0
1
.0
0.3
0.2 2 0.2.
0.2" RfoRf.
0.2 = 0.2.
Example 3.14 The fuzzy relation "x and y are very near" is reflexive, symmetric, and nontransitive. The fuzzy relation "x andy do not look alike" is antireflexive, symmetric, and nontransitive. The fuzzy relation "x is greatly smaller than y" is antireflexive, strictly antisymmetric, and transitive. When a fuzzy relation R (X, X) is nontransitive or antitransitive, we may want to find a transitive fuzzy relation that contains R (X, X) and is minimum in the sense of membership grade. This is the transitive closure of a fuzzy relation. The transitive closure of a fuzzy relation R (X, X) is denoted by RT (X, X); it is a fuzzy relation that is transitive and contains R (X, X), and its elements have the smallest possible membership grades. When X has n elements, the transitive closure Rr (X, X) can be obtained by Rr(X,X) Example 3.15 Let X={xltx2,
= RUR2U
- • • UR",
where /?' =
o R, i > 2.
(3.33)
jtj} and let a fuzzyrelationR(X,X) be given as "0.1
0.5
0.2"
R = 0.4
0.7
0.8
0.1
0.4
0
Then the fuzzy closure of R can be obtained as follows: "0.4
0.5
0.5"
R* = R oR = 0.4
0.7
.0.1
0.1
"0.4
0.5
0.5"
0.7
0.4
0.7
0.7
0.4.
.0.1
0.1
o:4_
RT(X,X) = RU&UF?--
48
"0.4
0.5
0.5"
0.4
0.7
0.8
.0.1
0.1
0.4.
Fuzzy Relations
Chap. 3
From this example, it is noted that we do not need to calculate all the i = 2,3,..., n, in deriving the transitive closure of R. We can unify only the terms R,R2,...,Rk, when Rlc+l = tf*. We have discussed three variants of reflexivity, four variants of symmetry, and three variants of transitivity. By considering their combinations, a total of 3 X 4 X 3 = 36 different types of binary fuzzy relations on a single set can be identified. Some important ones with their properties are shown in Table 3.1 and will be discussed in the following sections. TABLE 3.1
Various Types of Binary Fuzzy Relations on a Single Set Property
Fuzzy Relation
Reflexive
Similarity Resemblance Partial ordering Preordering Strict ordering
X X X X
Antireflexive
Symmetric
Antisymmetric
X X
Transitive X
X X
X
X X X
3.3.1 Similarity Relations A binary fuzzy relation that is reflexive, symmetric, and transitive is known as a similarity relation. Similarity relations are the generalization of equivalence relations in binary crisp relations to binary fuzzy relations. In order to understand similarity relations, we need to understand equivalence relations in binary crisp relations and then extend them to similarity relations. A binary crisp relation C (X, X) is called an equivalence relation if it satisfies the following conditions: Reflexivity: (x, i ) 6 C (X, X) for every xEX.
(3.34)
Symmetry: If (x, y) G C (X, X), then (y, x)EC Transitivity: If (x, y)EC (X, X) and (y, z)EC then (JC, Z) G C (X, X),
(X, X), x,y EX.
(3.35)
(X, X),
y, Z, G X.
(3-36)
For each element x in X, one can define a crisp set Ax that contains all the elements of X that are related to x by the equivalence relation. That is, Ax={y I (x,y)GC(X,X)}.
(3.37)
Here, Ax is a subset of X. The element x is itself contained in Ax because of the reflexivity. Because C (X, X) is transitive and symmetric, each member of Ax is related to all the other members of Ax. This set Ax is called an equivalence class of C (X, X) with respect to x. Furthermore, the family of all such equivalence classes defined by the relation forms a partition on X and is denoted by it (C).
Sec. 3.3
Various Types of Binary Fuzzy Relations
49
Example 3.16 Let X ={-10,-9,...,-1,0,1 10,11,12,13,14}. Let C(X,X) ={(x,y) \ x = y(mod5)}; that is, (x — y) is divisible by 5. Then, clearly, C (X, X) is an equivalence relation. There are exactly five equivalence classes in X: A*, = A_10 = A.5 — A5 = A10 ={ -10, -5,0,5,10}, A, =A_9=A^=A6
= AU ={-9,-4,1,6,11},
A2 = A.j = A_3 = A7 = A12 ={-8, -3,2,7,12}, A3 = A_7 = A_2 = A„ = A13 ={-7, -2,3,8,13}, A4 = A^ = A., = A9 = A14 ={-6,-1,4,9,14}. The partition of X with respect to C (X, X) is TT (C) ={A0, A,, A2, A3, A4}. Hence, an equivalence relation C (X, X) partitions the set X by putting those elements that are related to each other in the same equivalence class. A similarity relation can be considered to effectively group elements into crisp sets whose members are "similar" to each other to some specified degree. Note that when this degree of similarity is 1, the grouping is then an equivalence class. This concept is clear when the similarity relation is represented in its resolution form [Eq. (3.9)], that is, R = U a e A ^ a i ? a . Then each a-cut Ra is an equivalence relation representing the presence of similarity between the elements to the degree a . For this equivalence relation Ra, there is a partition on X, orRa. Hence, each similarity relation is associated with the set TT(*)=M*a) |aGA«}
(3.38)
of partitions. Example 3.17 LetX ={
x
2< -*3> x4
(y)
forallyGy.
(3.49)
Proof: If Eq. (3.49) does not hold, that is, maxp,A(x)< |xfl(y) for some yEY, xex then it is clear that we cannotfinda (x, y) that will satisfy Eq. (3.48). Hence, a solution R does not exist On the other hand, assume that Eq. (3.49) holds. Let x* EX be the element at which (JLa (x*) is maximal, that is, \JLA (X*) = maxjSX |xA (x). If we define the membership function of R as p.* (x*, y) = (y) for all y E Y, and (x, y) = 0 for all x E X and x^x* and for all yEY, then this R clearly satisfies Eq. (3.48). Hence, R is a solution to problem PI and we can conclude that problem PI has solution(s) if the inequality of Eq. (3.49) holds. In order to solve problem P I (and problem P2 later), we need to introduce the a-operation. For any a, b £ [0,1], the a-operator is defined as ctab£
]
if a < b •r * if a>b.
1
(3-50)
For fuzzy sets A and BinX and Y, respectively, the a-composition of A and B forms a fuzzy relation A £ B in X X Y which is defined by fl ^ „ B ( x , y ) eM-AWotp.B(y)= i
if|iA(x)
HB
W •
(3.51)
Furthermore, the a-composition of a fuzzy relation R and a fuzzy set B is denoted by R 2> B and is defined by « B to = min [ji,, (x, y) a |xB (y) ] . (3.52) y£Y With the above a-operator and a-composition, the following properties will be useful for determining the solutions of problem PI. Theorem 3.2 Let R be a fuzzy relation on X X Y. For any fuzzy sets A and B in X and Y, respectively, we have R CA A o (A
56
(A ° R),
(3.53)
B) C B.
(3.54)
Fuzzy Relations
Chap. 3
Proof: Let us prove Eq. (3.53) first Rewriting the right-hand side of Eq. (3.53) in terms of membership functions, we have HA°(A./i)(x>y) ~ ^ W
W
= C'A W « [ ^ f e ) A f c y ) ) ]
= HA (•*)« jmax£ ma^(|x A (z) A ( z , y), |XA(x) A ( x , y)) j J a
I*a to 3 0.8
0.6) = x 2
(
yi l
l
ys l \
0.5
l
0.6
x3
0.6> 0.8 x 3 ^0.5 < 1 J With the solution R, it can be checked that A ° R = B and RCR, where R is given in Example 3.21. Next, consider problem P2; that is, given R and B, determine A such that A o R = B. Similar to Theorems 3.1-3.3 for problemPl, the next three theorems characterize the solutions of problem P2.
Sec. 3.4
Fuzzy Relation Equations
57
Theorem 3.4 Problem P2 has no solution if the following inequality holds: max (JLD (X, y) < (y) for some yEY. (3.56) x&C The proof of this theorem is obvious by observing Eq. (3.48). This theorem allows us, in certain cases, to determine quickly that problem P2 has no solution. However, the converse of Eq. (3.56) is only a necessary and not a sufficient condition for the existence of a solution to problem P2. Example 3.23 Consider the max-min composition of A - o R: y\ xx /o.8 A o R = [xl,x2,x3]
O X2
1
*3 \0.3
yi 0.5 \ 0.7
= [0.5,1] = B.
0.2/
Since max^^ix^^.yj) = max[0.5,0.7,0.2] = 0 . 7 < p.B(y2) = 1, this set-relation equation has no solution. Consider another fuzzy set-relation equation: yi *,/l A o R = [jcj, x2,x3]
\\
o xA 1
1
xXl
1/
= [0.5,0.7] = B.
Although max ^ |iR (x, y,) = 1 > yy.B (y,) = 0.5 and max ^ there is obviously no solution to the above set-relation equation.
(x, y2)=l>
I ( y 2 ) = 0.7,
Theorem 3.5 Let R be a fuzzy relation on XX Y. For any fuzzy sets A and B in X and Y, respectively, we have (R2.B)
(3.57)
RQB,
(3.58)
(A o R).
AQR
Proof:
°
Let us prove Eq. (3.57)first[refer Eq. (3.52)]:
M-(Ria)»R(x,y) =
max
x&C
= max X&C
< max{ x&C
min[ fij, (x, y) a^B (y)] A min L zsy, z*y
(x, z) ap. s (z)) A
(x, y) [
J
(x, y) a^B (y)) A
J
(x, y)
(x, y) otp,B (y) ] A [ ^ (x, y) ]} for ally
which completes the proof of Eq. (3.57). Again A is used in the above proof for the min operation to simplify the notation. For Eq. (3.58), we have
58
Fuzzy Relations
Chap. 3
y) = min-l H-K (x, y) a max [(xA (x) A p.* (x, ,y)]} ye Y [ = min]
(x, y) a max
y£Y I > min{ yEY S
JJLA
max (|xA (z) A jx^ (z, y)), p.A (X) A
(x, y) [
L zex, (x, y) a [|xA (x) A
(X)
JI (x, y) ]}
for every x E X,
which completes the proof of Eq. (3.58). We are ready to derive the following theorem for the solution of problem P2. Theorem 3.6 If a solution to problem P2 exists, then the largest fuzzy set A that satisfies A o R = B is A: A= R
(3.59)
whose membership function is given by Eq. (3.52). Proof: FromEq. (3.58), we have the relationship A C R2>B=A. Then because of the monotonicity of the max-min composition, we have B= A ° RQA o R. From Eq. (3.57), we obtain A o RGB. Hence, A o R = B. Moreover, since A C A , A is the largest solution to A ° R = B. Example 3.24 From Example 3.21, we are given R and B. Then, have y\ yi ys
R*B=X2 X3
/0.7
I
0.4\
0.5
0.9
0.6
^0.2
0.6
0.3,
^0.5 A0.8 A l \
1A 0.8 A1 \
1 A1A1 )
x, =
*2
A
Y\ Z
(0.5 \
y2 0.8
