Intelligent Decision Systems in Large-Scale Distributed Environments

Pascal Bouvry, Horacio González-Vélez, and Joanna Koł odziej (Eds.) Intelligent Decision Systems in Large-Scale Distrib...

Author: Pascal Bouvry | Horacio González-Vélez | Joanna Kolodziej

166 downloads 1229 Views 7MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

Pascal Bouvry, Horacio González-Vélez, and Joanna Koł odziej (Eds.) Intelligent Decision Systems in Large-Scale Distributed Environments

Studies in Computational Intelligence, Volume 362 Editor-in-Chief Prof. Janusz Kacprzyk Systems Research Institute Polish Academy of Sciences ul. Newelska 6 01-447 Warsaw Poland E-mail: [email protected] Further volumes of this series can be found on our homepage: springer.com Vol. 340. Heinrich Hussmann, Gerrit Meixner, and Detlef Zuehlke (Eds.) Model-Driven Development of Advanced User Interfaces, 2011 ISBN 978-3-642-14561-2 Vol. 341. Stéphane Doncieux, Nicolas Bredeche, and Jean-Baptiste Mouret(Eds.) New Horizons in Evolutionary Robotics, 2011 ISBN 978-3-642-18271-6 Vol. 342. Federico Montesino Pouzols, Diego R. Lopez, and Angel Barriga Barros Mining and Control of Network Traffic by Computational Intelligence, 2011 ISBN 978-3-642-18083-5 Vol. 343. Kurosh Madani, António Dourado Correia, Agostinho Rosa, and Joaquim Filipe (Eds.) Computational Intelligence, 2011 ISBN 978-3-642-20205-6 Vol. 344. Atilla El¸ci, Mamadou Tadiou Koné, and Mehmet A. Orgun (Eds.) Semantic Agent Systems, 2011 ISBN 978-3-642-18307-2

Vol. 351. Ngoc Thanh Nguyen, Bogdan Trawiński, and Jason J. Jung (Eds.) New Challenges for Intelligent Information and Database Systems, 2011 ISBN 978-3-642-19952-3 Vol. 352. Nik Bessis and Fatos Xhafa (Eds.) Next Generation Data Technologies for Collective Computational Intelligence, 2011 ISBN 978-3-642-20343-5 Vol. 353. Igor Aizenberg Complex-Valued Neural Networks with Multi-Valued Neurons, 2011 ISBN 978-3-642-20352-7 Vol. 354. Ljupco Kocarev and Shiguo Lian (Eds.) Chaos-Based Cryptography, 2011 ISBN 978-3-642-20541-5 Vol. 355. Yan Meng and Yaochu Jin (Eds.) Bio-Inspired Self-Organizing Robotic Systems, 2011 ISBN 978-3-642-20759-4 Vol. 356. Slawomir Koziel and Xin-She Yang (Eds.) Computational Optimization, Methods and Algorithms, 2011 ISBN 978-3-642-20858-4

Vol. 345. Shi Yu, Léon-Charles Tranchevent, Bart De Moor, and Yves Moreau Kernel-based Data Fusion for Machine Learning, 2011 ISBN 978-3-642-19405-4

Vol. 357. Nadia Nedjah, Leandro Santos Coelho, Viviana Cocco Mariani, and Luiza de Macedo Mourelle (Eds.) Innovative Computing Methods and Their Applications to Engineering Problems, 2011 ISBN 978-3-642-20957-4

Vol. 346. Weisi Lin, Dacheng Tao, Janusz Kacprzyk, Zhu Li, Ebroul Izquierdo, and Haohong Wang (Eds.) Multimedia Analysis, Processing and Communications, 2011 ISBN 978-3-642-19550-1

Vol. 358. Norbert Jankowski, Włodzisław Duch, and ¸ bczewski (Eds.) Krzysztof Gra Meta-Learning in Computational Intelligence, 2011 ISBN 978-3-642-20979-6

Vol. 347. Sven Helmer, Alexandra Poulovassilis, and Fatos Xhafa Reasoning in Event-Based Distributed Systems, 2011 ISBN 978-3-642-19723-9

Vol. 359. Xin-She Yang and Slawomir Koziel (Eds.) Computational Optimization and Applications in Engineering and Industry, 2011 ISBN 978-3-642-20985-7

Vol. 348. Beniamino Murgante, Giuseppe Borruso, and Alessandra Lapucci (Eds.) Geocomputation, Sustainability and Environmental Planning, 2011 ISBN 978-3-642-19732-1

Vol. 360. Mikhail Moshkov and Beata Zielosko Combinatorial Machine Learning, 2011 ISBN 978-3-642-20994-9

Vol. 349. Vitor R. Carvalho Modeling Intention in Email, 2011 ISBN 978-3-642-19955-4 Vol. 350. Thanasis Daradoumis, Santi Caballé, Angel A. Juan, and Fatos Xhafa (Eds.) Technology-Enhanced Systems and Tools for Collaborative Learning Scaffolding, 2011 ISBN 978-3-642-19813-7

Vol. 361. Vincenzo Pallotta, Alessandro Soro, and Eloisa Vargiu (Eds.) Advances in Distributed Agent-Based Retrieval Tools, 2011 ISBN 978-3-642-21383-0 Vol. 362. Pascal Bouvry, Horacio González-Vélez, and Joanna Koł odziej (Eds.) Intelligent Decision Systems in Large-Scale Distributed Environments, 2011 ISBN 978-3-642-21270-3

Pascal Bouvry, Horacio González-Vélez, and Joanna Kołodziej (Eds.)

Intelligent Decision Systems in Large-Scale Distributed Environments

123

Editors

Pascal Bouvry

Dr. Joanna Koł odziej

Computer Science and Communications Research Unit University of Luxembourg, 6 rue R. Coudenhove-Kalergi L-1359 Luxembourg E-mail: [email protected]

Department of Mathematics and Computer Science University of Bielsko-Biala ul. Willowa 2 43-309 Bielsko-Biala, Poland E-mail: [email protected]

Horacio González-Vélez School of Computing and IDEAS Research Institute Robert Gordon University AB25 1HG Aberdeen, Scotland United Kingdom E-mail: [email protected]

ISBN 978-3-642-21270-3

e-ISBN 978-3-642-21271-0

DOI 10.1007/978-3-642-21271-0 Studies in Computational Intelligence

ISSN 1860-949X

Library of Congress Control Number: 2011928070 c 2011 Springer-Verlag Berlin Heidelberg This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typeset & Cover Design: Scientific Publishing Services Pvt. Ltd., Chennai, India. Printed on acid-free paper 987654321 springer.com

To our Families and Friends with Love and Gratitude

Preface

One of the most challenging open problems in intelligent decision systems is to effectively handle the inherent complexity of large-scale distributed environments such as clusters, constellations, grids, ad-hoc and P2P networks, and clouds. Collectively owned and managed by distinct organisational bodies, such complex large-scale distributed systems typically encompass computational resources from different institutions, enterprises, and individuals and are governed by heterogenous administrative policies and regulations. They must dynamically adapt to organisational relationships and real-world uncertainties. System management techniques must therefore be able to group, predict, and classify different sets of rules, configuration directives, and environmental conditions to impose dissimilar usage policies on various users and resources. They must effectively deal with uncertainties in system information|that may be incomplete, imprecise, fragmentary, or overloading|to control specific constituents and objects within intricate configurations. Decision scenarios ought to be outlined assuming partial visibility of environmental conditions, user heterogeneity, and resource dynamism in order to determine and select adequate evaluation criteria and assignment scores to render a final integrated decision result. This compendium herewith presents novel concepts in the analysis, implementation, and evaluation of the next generation of intelligent techniques for the formulation and solution of complex decision problems in large-scale distributed systems. Its fifteen chapters are structured into three main parts: I. Foundations: Chapters 1 to 6 introduce the general concepts of decision making techniques and problems in heterogeneous domains, followed by a selection of innovative approaches to grid scheduling, artificial intelligence metaheuristics, power awareness, and supply-chain modelling. A number of evolutionary algorithms, constraints, and applications are discussed throughout. II. Agents: A growing number of complex distributed systems are composed of customisable knowledge-based building blocks (agents). Chapters 7 to 11 discuss novel agent-oriented concepts and techniques for ontology-driven

VIII

Preface

optimisation, multi-repository data management, game theory modelling, evolutionary communication algorithms, and sensor mobility. III. Infrastructure: Collectively known as computational resources or simply infrastructure, computing elements, storage, and services represent a crucial component in the formulation of intelligent decisions in large systems. Consequently, Chapters 12 to 15 showcase techniques and concepts for marketbased energy awareness, data compaction, complex system testing, and lexical representation of large-scale distributed systems. I. Foundations The complexity of the decision-making process in large-scale heterogeneous environments emanates from several and usually conflicting criteria, under which the decisions must be made. In distributed systems, such decisions are partitioned among the entities composing the system and coupled by communication protocols. Various types of the computational grid users, their relationships and access scenarios are presented by Kołodziej and Xhafa in Chapter 1. The authors identify and analyse some fundamental features arising in the user behaviour in grid scheduling, such as dynamics, selfishness, cooperativeness, trustfulness, and symmetric and asymmetric roles. They define the general concept of the asymmetric Stackelberg game as a case study of the decision process in the grid scheduling problem, where new scheduling criteria|such as security and resource reliability|are specified. Finally, some economy-based approaches are discussed. The task scheduling in computational grids remains challenging open problem as a result of the complex hierarchic environment structure. Consequently, multi-level optimisation for user strategies and decisions has long been explored. An interesting combination of a recursive convex clustering algorithm and the cellular genetic algorithm is proposed by Pecero et al. in Chapter 2. The authors present this hybrid scheduler as an effective global optimiser of both computation and communications costs for workflow applications. An experimental evaluation of the algorithm is demonstrated with various real-world workflow applications. Dorronsoro et al. present three novel co-evolutionary algorithms for multiobjective optimisation in Chapter 3. As co-evolution is driven by distributed optimisation within sub-populations, the authors demonstrate the efficacy of their parallel implementations of three nature-inspired algorithms|a non-dominated sorting genetic algorithm, a strength Pareto evolutionary algorithm, and a multi-objective cellular genetic algorithm|using well-known benchmarks and real-world scenarios. Power awareness and energy consumption have long dominated the research agenda in high-performance computing. In Chapter 4, Kessaci et al. survey the main techniques to cut down energy consumption in large computing and embedded systems and, then, present a case study based on a parallel bi-objective hybrid genetic algorithm, which optimises both completion time and energy consumption. In Chapter 5, Jarosz and Burczyński firstly define the various types of the multicriteria decision-making problem using the multi-objective optimisation paradigm. They survey the research on bio-inspired meta-heuristic approaches and then

Preface

IX

formulate and analyse a game-theoretical model for the system users, which is optimised by an artificial immune algorithm. As a result of their economic and human advantages, evolutionary algorithms for industrial optimisation have attracted significant attention, particularly for supplychain problems. Ibrahimov et al. discuss a pair of realistic large-scale scenarios for the wine and mining industries in Chapter 6, where the use of decision support systems with evolutionary components have resulted in tangible savings. II. Agents Peer-to-peer networks demand innovative schemas for data sharing as a result of the number of nodes, autonomy, and heterogeneity. In Chapter 7, Li et al. propose an ontology-driven approach to qualitatively enhance data sharing and querying in large-scale systems. By adding a semantic layer to standard data sharing schemata, they demonstrate topology-aware connectivity as well as an improvement in query routing. Inherent to large scale distributed environments, geographically dispersed data repositories pose a challenge to known techniques for data gathering and processing. This is particularly true for building and executing workflows that often require highly structured sources. In Chapter 8, Byrski et al. describe an agent-based framework to assemble workflows based on user constraints and processing history, and evaluate its suitability looking up a profile for a scientist. By modelling strategic choices and subsequent interaction, game theory provides a solid foundation to understand decision making of autonomic agents. Peleteiro, Burguillo, and Bazzan simulate the multi-generation spatial interaction of cooperators and defectors for the iterated prisoner’s dilemma in Chapter 9. Their main contribution lies in the explorative analysis of coalitions through reinforcement learning techniques. The autonomic nature of an agent poses an increasing level of complexity when the number of agents and the size of their working environment surpasses the scope of non-heuristic comprehensive algorithmic solutions. In Chapter 10, Otto and Niemann aptly illustrate the applicability of evolutionary heuristics to model agent communications in large distributed systems through three case studies in healthcare scheduling, supply chain management, and location of public facilities (p-median problem). Major modern networks encompass countless devices with diverse sensing and communication capabilities. Tziritas, Khan, and Loukopoulos introduce a mobile agent framework in Chapter 11, where interacting communication agents reside at all participating sensing nodes. Their work analyses the efficient placement of agents using a heuristic algorithm based on graph colouring.

X

Preface

III. Infrastructure As resources in large-scale systems are not owned by a single entity, market-oriented models|where individual entities can freely trade commodities|have long entertained researchers and practitioners. By using market-oriented game theory, Okaie and Nakano provide a rational model of a peer-to-peer network to trade resources using energy as the common currency in Chapter 12. Energy consumption is therefore optimised through mutual node interaction and negotiation. Of significant relevance to multiple domains such as telecommunications, bioinformatics, and circuit design, data compaction has long been of interest to numerous research groups. Logofatu, Gruber, and Dumitrescu propose the heuristic determination of test sets for data compaction in Chapter 13 using MapReduce. Having deployed their parallel solution in a large distributed system, they show its scalability using five benchmarks with a large amount of data. The non-linear behaviour of large-scale distributed environments has been consistently scrutinised using complex systems modelling. As testing of complex systems is typically intricate and emergent, Todinov introduces a stochastic algorithm and an associated tool for virtual accelerated testing of complex systems in Chapter 14. His approach shows significant improvements over existing path-finding algorithms, making it particularly suitable for large complex systems with a substantial number of components. The representation of large concurrent configuration requires high-level, flexible descriptive elements. In Chapter 15, Szpyrka, Matyasik, and Mrówka present a modelling language derived from Milner’s seminal work on the calculus of communicating systems as well as process algebras. It includes formal model verification, rule-based decision support, and a graphical representation of component interconnection.

Acknowledgements We are grateful to all the contributors of this book, for their willingness to work on this interdisciplinary book project. We thank the authors for their interesting proposals of the book chapters, their time, efforts and their research results, which makes this volume an interesting complete monograph of the latest research advances and technology development on the next generation distributed and complex emergent intelligent decision systems. We also would like to express our sincere thanks to the reviewers, who have helped us to ensure the quality of this volume, and the members of the “Interdisciplinary Centre for Security, Reliability, and Trust (SnT)" and “Computer Science and Communications (CSC) Research Unit " of University of Luxembourg. We gratefully acknowledge their time, assistance and valuable remarks and comments. Our special thanks go to Prof. Janusz Kacprzyk, editor-in-chief of the Springer’s Studies in Computational Intelligence Series, and to Dr. Thomas Ditzinger, editorial assistant of Springer Verlag in Heidelberg, for their editorial assistance and excellent cooperative collaboration in this book project.

Preface

XI

Finally, we would like to send our warmest gratitude message to our friends and families for their patience, love, and support in the preparation of this volume. We strongly believe that this book ought to serve as a reference for students, researchers, and industry practitioners interested or currently working in the evolving interdisciplinary area of intelligent decision systems using emergent distributed computing paradigms. Luxembourg, Aberdeen, and Bielsko-Biała, March 2011

Pascal Bouvry Horacio González-Vélez Joanna Kołodziej

Contents

1

2

Task Allocation Oriented Users Decisions in Computational Grid . . . Joanna Kołodziej, Fatos Xhafa 1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Users’ Layers in the Grid Architecture . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Grid Users Relations in Grid Scheduling . . . . . . . . . . . . . . . . . . . . . . 1.3.1 Hierarchic Grid Infrastructure . . . . . . . . . . . . . . . . . . . . . . . . . 1.3.2 Users’ Requirements, Relations and Strategies in Job Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.4 Game-Theoretic Models for Scheduling and Resource Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.5 Solving the Grid Users’ Games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.5.1 GA-Based Hybrid Approach . . . . . . . . . . . . . . . . . . . . . . . . . . 1.6 A Case Study: Non-cooperative Asymmetric Stackelberg Game of the Grid Users in Independent Batch Scheduling . . . . . . . . . . . . . 1.6.1 Players’ Cost Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.6.2 Experiments Setting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.6.3 Computational Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.7 Other Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.7.1 Computational Economy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.7.2 Neural Networks and Markov Decision Processes . . . . . . . . 1.8 Conclusions and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Efficient Hierarchical Task Scheduling on GRIDS Accounting for Computation and Communications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Johnatan E. Pecero, Frederic Pinel, Bernabé Dorronsoro, Grégoire Danoy, Pascal Bouvry, Albert Y. Zomaya 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.1 System Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.2 Application Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.3 Scheduling Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1 1 3 6 6 7 10 13 13 15 16 17 18 19 19 21 22 23 25

25 27 27 29 31

XIV

Contents

2.3 Resource Management System and Grid Scheduling . . . . . . . . . . . . 2.3.1 Resource Management System . . . . . . . . . . . . . . . . . . . . . . . . 2.3.2 Workflow Scheduling on the Grid: A Brief Taxonomy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Proposed Approach: The Hierarchical Scheduler with Cooperative Local Schedulers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.1 Recursive Convex Clustering Algorithm . . . . . . . . . . . . . . . . 2.4.2 DAG Partitioning Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.3 Local Scheduler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

4

Multi-objective Cooperative Coevolutionary Evolutionary Algorithms for Continuous and Combinatorial Optimization . . . . . . . Bernabé Dorronsoro, Grégoire Danoy, Pascal Bouvry, Antonio J. Nebro 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Base Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 The Proposed Cooperative Coevolutionary Evolutionary Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.1 Cooperative Coevolutionary Evolutionary Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.2 Multi-objective Cooperative Coevolutionary Evolutionary Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.3 Parallel Multi-objective Cooperative Coevolutionary Evolutionary Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.1 Continuous Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.2 Real-World Combinatorial Problem . . . . . . . . . . . . . . . . . . . . 3.6 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.1 Configuration of Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.2 Methodology for the Comparisons . . . . . . . . . . . . . . . . . . . . . 3.6.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7 Conclusion and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Parallel Evolutionary Algorithms for Energy Aware Scheduling . . . . Yacine Kessaci, Mohand Mezmaz, Nouredine Melab, El-Ghazali Talbi, Daniel Tuyttens 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Energy Aware Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

31 31 32 35 36 39 39 41 44 44 49

50 51 52 53 53 56 57 60 60 60 62 62 64 66 71 72 75

76 77

Contents

4.3 Optimization Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.1 Aggregation Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.2 Lexicographic Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.3 Pareto Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4 System-Level Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.1 Hardware-Level Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.2 Software-Level Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5 Approaches According to Targeted Execution System . . . . . . . . . . . 4.5.1 Embedded Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5.2 Computing Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.6 Problem Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.6.1 System Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.6.2 Application Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.6.3 Energy Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.6.4 Scheduling Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.7 A Case Study: A Parallel Evolutionary Algorithm . . . . . . . . . . . . . . 4.7.1 Hybrid Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.7.2 Insular Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.7.3 Multi-start Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.8 Experiments and Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.8.1 Experimental Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.8.2 Hybrid Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.8.3 Insular Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.8.4 Multi-start Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.9 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Biologically-Inspired Methods and Game Theory in Multi-criterion Decision Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Paweł Jarosz, Tadeusz Burczyński 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Multi-criteria Decision Making and Multiobjective Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.1 No-Preference Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.2 Posteriori Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.3 Priori Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.4 Interactive Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.5 Multiobjective Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3 Methods for Multiobjective Optimization . . . . . . . . . . . . . . . . . . . . . 5.3.1 Evolutionary Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.2 Artificial Immune Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.3 Game Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.4 Hybrid Evolutionary-Game Algorithm . . . . . . . . . . . . . . . . . 5.3.5 Immune Game Theory MultiObjective Algorithm IMGAMO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

XV

78 79 80 80 80 81 82 83 83 84 85 85 86 88 89 89 89 92 92 93 93 95 95 97 97 98 101 101 102 104 104 104 105 105 106 106 107 111 112 113

XVI

6

7

Contents

5.4 Numerical Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4.1 The ZDT2 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4.2 The ZDT3 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4.3 The ZDT6 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.5 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

116 118 118 120 122 123

Advanced Planning in Vertically Integrated Supply Chains . . . . . . . . Maksud Ibrahimov, Arvind Mohais, Sven Schellenberg, Zbigniew Michalewicz 6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 Literature Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2.1 Supply Chain Management . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2.2 Time-Varying Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2.3 Computational Intelligence . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3 Wine Supply Chain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3.1 Maturity Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3.2 Vintage Intake Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3.3 Crushing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3.4 Tank Farm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3.5 Bottling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3.6 Environmental Factors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4 Advanced Planning in Mining . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4.2 Constraints and Business Rules . . . . . . . . . . . . . . . . . . . . . . . 6.4.3 Functionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.5 Conclusion and Future Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

125

Efficient Data Sharing over Large-Scale Distributed Communities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Juan Li, Samee Ullah Khan, Qingrui Li, Nasir Ghani, Nasro Min-Allah, Pascal Bouvry, Weiyi Zhang 7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3 System Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3.1 Problem Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3.2 A Multilayered Semantic Sharing Scheme . . . . . . . . . . . . . . 7.3.3 From Schema to Ontology . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3.4 Semantic Similarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4 Semantics-Based Self-clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4.1 Joining the Right Semantic Cluster . . . . . . . . . . . . . . . . . . . . 7.4.2 Dynamic Self-adjusting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

126 127 128 130 131 132 133 133 134 135 136 137 139 141 141 143 144 144 146 149

150 151 152 152 152 153 155 156 156 157

Contents

7.5 Query Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.5.1 Problems of Query Evaluation . . . . . . . . . . . . . . . . . . . . . . . . 7.5.2 Semantics-Based Forwarding . . . . . . . . . . . . . . . . . . . . . . . . . 7.5.3 Containment-Based Caching . . . . . . . . . . . . . . . . . . . . . . . . . . 7.6 Experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

9

Hierarchical Multi-Agent System for Heterogeneous Data Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Aleksander Byrski, Marek Kisiel-Dorohinicki, Jacek Dajda, Grzegorz Dobrowolski, Edward Nawarecki 8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2 AgE – Agent-Based Computation Framework . . . . . . . . . . . . . . . . . . 8.3 Panorama of Systems for Integration of Heterogeneous Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.4 Basic Model of Data Transformation . . . . . . . . . . . . . . . . . . . . . . . . . 8.5 Hierarchical Data Integration and Processing . . . . . . . . . . . . . . . . . . 8.5.1 System Environment and User Interaction . . . . . . . . . . . . . . 8.5.2 Agent-Based Data Integration Workflow Model . . . . . . . . . . 8.5.3 Multi-Agent System Structure . . . . . . . . . . . . . . . . . . . . . . . . 8.5.4 Tasks, Objects and Data Types . . . . . . . . . . . . . . . . . . . . . . . . 8.5.5 Tree of Agents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.5.6 Roles of Agents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.5.7 Actions of Agents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.5.8 Resources of the System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.6 Searching for Personal Profile of a Scientist – An Example . . . . . . . 8.6.1 Construction of Scientist’s Profile . . . . . . . . . . . . . . . . . . . . . 8.6.2 Example Data Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.6.3 Set of Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.6.4 System Environment and Structure . . . . . . . . . . . . . . . . . . . . 8.6.5 Agents, Their Actions and Their Goals . . . . . . . . . . . . . . . . . 8.6.6 System Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Emerging Cooperation in the Spatial IPD with Reinforcement Learning and Coalitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ana Peleteiro, Juan C. Burguillo, Ana L. Bazzan 9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.3 Prisoner’s Dilemma . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.4 The Game . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.4.1 Spatial Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.4.2 Basic Game Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

XVII

157 157 158 158 159 162 163 165

165 167 170 172 173 173 175 176 177 178 178 179 179 180 180 181 181 183 183 183 184 185 187 187 188 189 190 190 191

XVIII

Contents

9.4.3 Agent Roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.4.4 Scenarios and Agent Actions . . . . . . . . . . . . . . . . . . . . . . . . . 9.5 Reinforcement Learning Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . 9.5.1 Q-Learning (QL) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.5.2 Learning Automata (LA) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.5.3 Action Selection and States . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.6 Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.7 Results Using the Coordination Game . . . . . . . . . . . . . . . . . . . . . . . . 9.7.1 Scenario without Coalitions . . . . . . . . . . . . . . . . . . . . . . . . . . 9.7.2 Scenario with Coalitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.8 Results Using a Prisoner’s Dilemma Approach . . . . . . . . . . . . . . . . . 9.8.1 Scenario without Coalitions . . . . . . . . . . . . . . . . . . . . . . . . . . 9.8.2 Scenario with Coalitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.9 Conclusions and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Evolutionary and Economic Agents in Complex Decision Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Stephan Otto, Christoph Niemann 10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.2 Environments and Complex Decision Systems . . . . . . . . . . . . . . . . . 10.2.1 Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.2.2 Decision Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.3 Complex Decision Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.3.1 Software Agents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.3.2 Economic and Market-Based Models . . . . . . . . . . . . . . . . . . 10.3.3 Evolutionary Computation and Agents . . . . . . . . . . . . . . . . . 10.4 Case Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.4.1 Hybrid Decision Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.4.2 Evolutionary Agents Optimize Supply Chain Structures . . . 10.4.3 Evolutionary Agents Optimize the p-median Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.5 Conclusion and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 On Reconfiguring Embedded Application Placement on Smart Sensing and Actuating Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . Nikos Tziritas, Samee Ullah Khan, Thanasis Loukopoulos 11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1.1 Application Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1.3 Related Work and Contributions . . . . . . . . . . . . . . . . . . . . . . . 11.2 Problem Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2.1 System Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2.2 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

192 192 193 193 193 194 194 194 195 196 197 198 199 204 204 207 207 208 208 209 209 210 211 214 217 217 221 223 225 226 231 232 232 233 233 235 235 235

Contents

11.3 Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3.1 The APR Problem with 2 Nodes . . . . . . . . . . . . . . . . . . . . . . . 11.3.2 The Agent Exchange Algorithm . . . . . . . . . . . . . . . . . . . . . . . 11.3.3 Extending to N Nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3.4 Greedy Algorithmic Approach . . . . . . . . . . . . . . . . . . . . . . . . 11.4 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4.1 Experimental Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4.2 Comparison against the Optimal . . . . . . . . . . . . . . . . . . . . . . . 11.4.3 Experiments with a Larger Network . . . . . . . . . . . . . . . . . . . . 11.4.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 A Game Theoretic Approach to Dynamic Network Formation in Market-Oriented Resource Providing Networks . . . . . . . . . . . . . . . . . . Yutaka Okaie, Tadashi Nakano 12.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2 Network Formation Game Example . . . . . . . . . . . . . . . . . . . . . . . . . . 12.3 The Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.3.1 Agents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.3.2 Platforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.4 Simulation Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.4.1 Simulation Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.4.2 Default Simulation Configurations . . . . . . . . . . . . . . . . . . . . . 12.4.3 Simulation Results: Simple Scenario . . . . . . . . . . . . . . . . . . . 12.4.4 Simulation Results: Realistic Scenario . . . . . . . . . . . . . . . . . . 12.5 Theoretical Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.5.1 Edgeless Topologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.5.2 Fully Connected Topologies . . . . . . . . . . . . . . . . . . . . . . . . . . 12.5.3 d-Regular Topologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.5.4 Hub Topologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.5.5 Summary of Theoretical Analysis . . . . . . . . . . . . . . . . . . . . . 12.6 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Distributed Evolutionary Algorithm Using the MapReduce Paradigm – A Case Study for Data Compaction Problem . . . . . . . . . . Doina Logof˘atu, Manfred Gruber, Dumitru (Dan) Dumitrescu 13.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2 Problem Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3 Recent Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.4 Parallel Evolutionary Algorithm Using MapReduce . . . . . . . . . . . . .

XIX

237 237 239 240 244 244 244 245 246 248 248 249 251 251 254 255 256 256 257 257 259 259 262 264 269 270 271 272 274 274 276 276 279 279 280 281 282

XX

Contents

13.5 Implementation Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.6 Experimental Results and Statistical Tests . . . . . . . . . . . . . . . . . . . . . 13.7 Conclusions and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

284 287 288 290

14 Virtual Accelerated Life Testing of Complex Systems . . . . . . . . . . . . . . Michael T. Todinov 14.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.1.1 Arrhenius Stress-Life Relationship and Arrhenius-Type Acceleration Life Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.1.2 Inverse Power Law Relationship (IPL) and IPL-Type Acceleration Life Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.1.3 Eyring Stress-Life Relationship and Eyring-Type Acceleration Life Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.1.4 A Motivation for the Proposed Method . . . . . . . . . . . . . . . . . 14.2 Limitations of Available Analytical Methods for Determining the Reliability of Large and Complex Systems . . . . . . . . . . . . . . . . . 14.3 Efficient Representation of Reliability Networks with Complex Topology and a Large Number of Components . . . . . . . . . . . . . . . . . 14.3.1 Representing the Topology of a Complex Reliability Network by an Array of Pointers to Dynamic Arrays . . . . . 14.3.2 Updating the Link Arrays after a Component Failure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.4 Existence of Paths to Each End Node in a Complex Reliability Network Represented by Adjacency Arrays and Link Arrays . . . . . 14.5 Accelerated Time to Failure of a Complex System . . . . . . . . . . . . . . 14.6 A Software Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.7 A Solved Test Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.8 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

293

15 Alvis – Modelling Language for Concurrent Systems . . . . . . . . . . . . . . Marcin Szpyrka, Piotr Matyasik, Rafał Mrówka 15.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.2 Related Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.3 Communication Diagrams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.4 Language Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.5 System Layers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.6 Rule-Based Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.7 Alvis Model Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.8 Agent and Model State . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.9 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

293 295 295 296 296 297 300 301 302 303 306 308 311 313 313 315 315 317 319 322 329 330 333 336 339 340

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343

List of Contributors

Ana L. Bazzan Instituto de Informatica, Universidade Federal do Rio Grande do Sul, 15064-Porto Alegre, RS, Brazil; e-mail: [email protected] Pascal Bouvry Computer Science and Communications Research Unit, University of Luxembourg, 6 rue R. Coudenhove-Kalergi, L-1359 Luxembourg; e-mail: [email protected] ´ Tadeusz Burczynski ul. Warszawska 24, 31-155 kraków, Poland; Silesian University of Technology, Department for Strength of Materials and Computational Mechanics, ul. Akademicka 2A, 44-100 Gliwice, Poland; e-mail: [email protected] Juan C. Burguillo Telematics Engineering Department, University of Vigo, 36310-Vigo, Spain; e-mail: [email protected] Aleksander Byrski Department of Computer Science, AGH University of Science and Technology, Al. Mickiewicza 30, 30-059 Kraków, Poland; e-mail: [email protected] Grégoire Danoy Computer Science and Communications Research Unit, University of Luxembourg, 6 rue R. Coudenhove-Kalergi, L-1359 Luxembourg; e-mail: [email protected]

XXII


Jacek Dajda Department of Computer Science, AGH University of Science and Technology, Al. Mickiewicza 30, 30-059 Kraków, Poland; e-mail: [email protected] Grzegorz Dobrowolski Department of Computer Science, AGH University of Science and Technology, Al. Mickiewicza 30, 30-059 Kraków, Poland; e-mail: [email protected] Bernabé Dorronsoro Interdisciplinary Centre for Security, Reliability, and Trust, University of Luxembourg, 6 rue R. Coudenhove-Kalergi, L-1359 Luxembourg; e-mail: [email protected] Dumitru (Dan) Dumitrescu Universitatea Babes-Bolyai, Facultatea de Matematica si Informatica, Str. Mihail Kogalniceanu, Nr. 1, Ro-400084, Cluj-Napoca, Romania; e-mail: [email protected] Nasir Ghani Department of Electrical and Computer Engineering, University of New Mexico, Albuquerque, USA; e-mail: [email protected] Manfred Gruber Hochschule Múnchen, Fakultät für Informatik und Mathematik, FK 07 Lothstrasse 64, Gebaeude R, D-80335 München Germany; e-mail: [email protected] Maksud Ibrahimov School of Computer Science, University of Adelaide, South Australia 5005, Australia; e-mail: [email protected] Paweł Jarosz Cracow University of Technology, Institute of Computer Science, ul. Warszawska 24, 31-155 Krakó, Poland; e-mail: [email protected] Yacine Kessaci INRIA Lille-Nord Europe, CNRS/LIFL Université Lille 1, Parc Scientifique de la Haute Borne 40, avenue Halley Bât.A, Park Plaza 59650, Villeneuve d’Ascq, France; e-mail: [email protected]


XXIII

Samee Ullah Khan Department of Electrical and Computer Engineering, North Dakota State University, Fargo, USA; e-mail: [email protected] Marek Kisiel-Dorohinicki Department of Computer Science, AGH University of Science and Technology, Al. Mickiewicza 30, 30-059 Kraków, Poland; e-mail: [email protected] Joanna Kołodziej University of Bielsko-Biała, ul. Willowa 2, 43-309 Bielsko-Biała, Poland; e-mail: [email protected] Juan Li Department of Computer Science, North Dakota State University, Fargo, USA; e-mail: [email protected] Qingrui Li Department of Computer Science, North Dakota State University, Fargo, USA; e-mail: [email protected] Doina Logof˘atu Hochschule Múnchen, Fakultát für Informatik und Mathematik, FK 07 Lothstrasse 64, Gebaeude R D-80335 München Germany; e-mail: [email protected] Thanasis Loukopoulos Dept. of Informatics and Computer Technology, TEI of Lamia, Lamia, Greece, Center for Research and Technology Thessaly (CERETETH), Volos, Greece; e-mail: [email protected] Piotr Matyasik AGH University of Science and Technology, Department of Automatics, Al. Mickiewicza 30, 30-059 Kraków, Poland; e-mail: [email protected] Nouredine Melab INRIA Lille-Nord Europe, CNRS/LIFL Université Lille 1, Parc Scientifique de la Haute Borne 40, avenue Halley Bât.A, Park Plaza 59650, Villeneuve d’Ascq, France; e-mail: [email protected] Mohand Mezmaz UMons/FPMs, 9, rue de Houdain 7000 Mons, Belgium; e-mail: [email protected]

XXIV


Zbigniew Michalewicz School of Computer Science, University of Adelaide, South Australia 5005, Australia; Institute of Computer Science, Polish Academy of Sciences, ul. Ordona 21, 01-237 Warsaw, Poland; Polish-Japanese Institute of Information Technology, ul. Koszykowa 86, 02-008 Warsaw, Poland; e-mail: [email protected] Nasro Min-Allah Department of Computer Science, COMSATS Institute of Information Technology, Islamabad, Pakistan; e-mail: [email protected] Arvind Mohais SolveIT Software, Pty Ltd., 99 Frome Street, Adelaide, SA 5000 Australia; e-mail: [email protected] Rafał Mrówka AGH University of Science and Technology, Department of Automatics, Al. Mickiewicza 30, 30-059 Kraków, Poland; e-mail: [email protected] Tadashi Nakano Frontier Research Base for Global Young Researchers, Frontier Research Center, Graduate School of Engineering, Osaka University, 2-1 Yamadaoka, Suita, Osaka 565-0871, Japan; e-mail: [email protected] Edward Nawarecki Department of Computer Science, AGH University of Science and Technology, Al. Mickiewicza 30, 30-059 Kraków, Poland; e-mail: [email protected] Christoph Niemann Management Information Systems, University of Bayreuth, Germany; e-mail: [email protected] Antonio J. Nebro E.T.S.I. Informática, University of Málaga, Boulevard Louis Pasteur s/n. 29018, Málaga, Spain; e-mail: [email protected] Yutaka Okaie Frontier Research Center, Graduate School of Engineering, Osaka University, 2-1 Yamadaoka, Suita, Osaka 565-0871, Japan; e-mail: [email protected]


XXV

Stephan Otto Fraunhofer Center for Supply Chain Services SCS, Nuremberg, Germany; e-mail: [email protected] Johnatan E. Pecero Computer Science and Communications Research Unit, University of Luxembourg, 6 rue R. Coudenhove-Kalergi, L-1359 Luxembourg; e-mail: [email protected] Ana Peleteiro Telematics Engineering Department. University of Vigo, 36310-Vigo, Spain; e-mail: [email protected] Frederic Pinel Computer Science and Communications Research Unit, University of Luxembourg, 6 rue R. Coudenhove-Kalergi, L-1359 Luxembourg; e-mail: [email protected] Sven Schellenberg SolveIT Software, Pty Ltd., 99 Frome Street, Adelaide, SA 5000 Australia; e-mail: [email protected] Marcin Szpyrka AGH University of Science and Technology, Department of Automatics, Al. Mickiewicza 30, 30-059 Kraków, Poland; e-mail: [email protected] El-Ghazali Talbi INRIA Lille-Nord Europe, CNRS/LIFL Université Lille 1 Parc Scientifique de la Haute Borne 40, avenue Halley Bât.A, Park Plaza 59650, Villeneuve d’Ascq, France; e-mail: [email protected] Michael T.Todinov Department of Mechanical Engineering and Mathematical Sciences, Oxford Brookes University, Oxford OX33 1HX, UK; e-mail: [email protected] Daniel Tuyttens UMons/FPMs, 9, Rue de Houdain 7000 Mons, Belgium; e-mail: [email protected] Nikos Tziritas Dept. of Computer and Communication Engineering, Univ. of Thessaly, Volos, Greece Center for Research and Technology Thessaly (CERETETH), Volos, Greece; e-mail: [email protected]

XXVI


Fatos Xhafa Department of Languages and Informatic Systems, Technical University of Catalonia, Campus Nord, Ed. Omega C/Jordi Girona 1-3 08034 Barcelona, Spain; e-mail: [email protected] Weiyi Zhang Department of Computer Science, North Dakota State University, Fargo, USA; e-mail: [email protected] Albert Y. Zomaya Advanced Network Research Group, School of Information Technologies, University of Sydney, Sydney, NSW 2006, Australia; e-mail: [email protected]

Chapter 1

Task Allocation Oriented Users Decisions in Computational Grid Joanna Kołodziej and Fatos Xhafa

Abstract. Tasks scheduling and resource allocation are crucial issues in hierarchic Computational Grid (CG) systems. The Grid users usually belong to different administrative domains with a various communication protocols, the access to the resources, the resource usage policies. In such a complex large-scale environment the game theory has shown to be an effective tool for modelling the Grid users’ decision process, the multi-criteria users’ requirements and the task and resource allocation strategies. In this chapter, we firstly highlight the main Grid users types, their relationships, resource access and usage scenarios in CGs. Then, we identify and analyze some fundamental features arising in the users’ behavior in Grid scheduling. We present a concept of the non-cooperative asymmetric Stackelberg game as a case study of the Grid users behavior decision process, under new requirements arising in allocation problems, such as asymmetric users relations, security and reliability restrictions. We also discuss how computational economy-based approaches, such as market mechanisms, and computational paradigms, such as Neural Networks and Markov decision processes, can be used to model the users’ decisions.

1.1 Introduction Over the last decade we have witnessed the Grid Computing to become popular in solving the large-scale problems from science, engineering, and industry. Such Joanna Kołodziej Department of Mathematics and Computer Science, University of Bielsko-Biała, 43-309 Bielsko-Biała, Poland e-mail: [email protected] Fatos Xhafa Department of Languages and Informatic Systems, Technical University of Catalonia, Campus Nord, Ed. Omega C/Jordi Girona 1-3, 08034 Barcelona, Spain e-mail: [email protected] P. Bouvry et al. (Eds.): Intelligent Decision Systems, SCI 362, pp. 1–24. c Springer-Verlag Berlin Heidelberg 2011 springerlink.com

2

J. Kołodziej and F. Xhafa

systems enable the virtualization of a wide range of resources, thus, the various types of Grid systems, such as Computational Grids (CGs), Desktop, Enterprise, Data Grids, etc., can be design and studied. Among them, CGs were primarily concerned with the development of the efficient resolution methods for solving the complex decision problems from high performance computing. In traditional distributed computing systems, there are found usually a few user types. As the users belong to the same administrative domain, their profiles and requirements were rather “flat" and their access to resources a priori taken for granted. Differently from such systems, in CGs the different users should co-exist and perform their tasks under the complex nature of Grid environment. Grid users can have various requirements implying a variety of users’ relationships and behavior scenarios, including selfishness, cooperativeness, trustworthiness, leadership, etc. It is thus very important to analyze and model such users’ requirements and behavior to predict their needs and actions in order to optimize the Grid system performance at both local and global levels. The approaches proposed so far in the literature have shown some limitations in an effective translation of the complex nature of CGs and may not overcome the problem of various access rights of the Grid users, the conflicting objectives of the users and the administrative owners, the various local resource access policies or security requirements. Heuristic-based approaches can be very efficient in solving the decision problems in the traditional job-shop scheduling [14], but they usually just partially cover the requirements of the Grid-enabled applications. Recently, the researchers investigate the usefulness of game theory in supporting the Grid users’ decisions in the task and resource allocations. Using game-theoretic models enables including more requirements and features into the decision model, such as security and resource reliability. The meta-heuristics can be successfully applied to solving the users’ games and finding the equilibrium states in the users decision processes. The objective of this work is to present a comprehensive analysis of Grid users’ behavior and decisions to define a basis for the Grid architecture and application designers. We firstly specify a simple categorization of the Grid user types and define their relationships and access scenarios in CGs according to their requirements. Then, we analyze some fundamental features arising in the users’ behavior in Grid scheduling, such as dynamics, selfishness, cooperativeness, trustfulness, symmetric and asymmetric roles. We present the general concept of the non-cooperative game approach and focus on the asymmetric Stackelberg game as a case study of the Grid users behavior decision process, where new requirements arising in allocation problems, such as asymmetric users relations, security and reliability restrictions in CGs are specified. We use GA-based hybrid algorithms for finding the equilibrium points of the game. The proposed hybrid resolution methods are experimentally evaluated through the Grid simulator under the heterogeneity, and the large-scale and dynamics scenarios. The relative performance of the Grid schedulers is measured by the makespan and flowtime metrics. We also survey the most relevant research proposals in the literature for using the computational economy-based approaches, such as market mechanisms, and computational paradigms, such as neural networks

1

Task Allocation Oriented Users Decisions in Computational Grid

3

and Markov decision processes for modelling the real-life Grid users decisions scenarios. The rest of the chapter is organized as follows. In Sect. 1.2 we define the main types of Grid users and provide a list of their requirements arising in Grid scenarios of data and resource management. Then, the case of users’ relationship in scheduling processes in CGs is discussed and analyzed in Sect. 1.3. In Sect. 1.4 we define the general game models for Grid scheduling and present the concept of the asymmetric Stackelberg game of the Grid users. The resolution meta-heuristic methods for solving the game are defined in Sect. 1.5. The experimental evaluation of proposed hybrid meta-heuristics is presented in Sect. 1.6. The use of other approaches including computational economy and neural networks is discussed in Sect. 1.7. We end the chapter in Sect. 1.8 with some conclusions and indications for future work.

1.2 Users’ Layers in the Grid Architecture In the large-scale distributed Grid environment the Grid users can play various roles, from Grid administrator and community or Virtual Organization administrator , through the node administrator, service owner or user group administrator, to the service end-user . Based on users’ categorization defined in [16] we present in Fig. 1.1 a brief characteristics of six main types of the Grid users.

Fig. 1.1 Main users types in Computational Grid

! " " "

" # ! "

"$

"" %

" #

4


We can divide the users into two groups: the class of system users and the class of power users. In the first class, a system user (denoted Grid-sys) plays a role of the Infrastructure System Administrator. He is an expert in computing and his main duties are the system administration of the Grid nodes, possibly with infrastructure delivery and security expertise. He is also in charge of monitoring the users activities and system performance. The Service Provider, denoted by SP, has an expertise in computing, authorization and possibly identity management. This user usually needs to authenticate, authorize and account for the Grid Service-End Users (SEU). SEU users do not need to be the experts in computing, they also could not be familiar with the Grid architecture and management. Their main interests are in submitting tasks and applications to be solved; additionally, they can upload the necessary data, scripts or source codes, can run queries, executable code or scripts via SPs. In the second group of power users, we can identify three main types of the Grid users: • PUA: Power User Agnostic of grid resource node. The main activities of this user are the program development and data management. PUA usually is not concerned where processing takes place in the Grid system. • PUS: Power User requiring Specific Grid resource nodes. His duties and activities are similar to PUA’s; additionally he may have more platform dependent and system administrative expertise. Also, Grid node owners may wish to have a direct authentication, authorization and accounting relationship with the PUSs (differently from PUA users). • PUDS: Power User Developing a Service should have high degree of computing (like PUA and PUS) and developing expertise (like SP). If needed, he may interact with and accounting for SEUs in an experimental manner. Grid user types arise and are located at different administrative domains, while operating at various levels of a wide Grid infrastructure. There is a variety of ways, in which the access management requirements of each group of users and special polices defined by the resource owners, may be fulfilled. Each such scenario, as well as Grid technology in general, should evolve in response to the requirements of the users’ communities. It can be assumed for example that SEUs cannot interact directly with any Grid nodes and there is an added defence of a restrictive application layer between the user and the grid node, and PUAs can have an access to the resources from the outside of Grid middleware. In Table 1.1 we present the users’ general requirements with respect to the Grid administration, service, application and porting, usability and resource utilization (see also [7]). It should be noted that the requirements on standardized authentication and authorization mechanisms as well as a globally accepted trustworthy of the Grid user are crucial at all Grid levels. The trustworthiness of the Grid user can be defined as a user’s authentication trustworthiness parameter (UAT), which can be expressed as a degree of user’s trustworthiness who has passed system authentication and must be taken as a basic qualification in judging access requests. The system should make

1


5

Table 1.1 Grid Users’ general requirements Requirement type

User type Grid General Requirements Reliable grid middleware; Quality of Service All users Easy access to data and databases, fine grained access policies All users Advance reservation or the information when your job will be schedAll users uled: monitoring of jobs, estimate of queue delay Standardized authentication and authorization mechanisms All users A globally accepted, trustworthy Grid user identity All users Administration Requirements Secure communication and transfer (e.g. TLS) Grid-sys, PUDS, SEU Uniform configuration across all Grid components Gris-sys, PUDS Ability to deploy and operate different versions of same software PUA, PUS, Grid-sys Standard mechanisms to fault tolerance, error handling, service recovGrid-sys, ery, outages and maintenance scheduling for all Grid components Testing and monitoring of Grid components with automatic alerts Grid-sys Fast, easy site installation and verification procedures for new sites joinGrid-sys ing the grid Service Requirements Encryption and protection of data on grid storage elements SP, PUA, PUS Fast access and reliable transfer of massive amounts of data SP, SEU, Grid-sys Ad-hoc integration of arbitrary data sources that lie outside the Grid SP Application and Porting Requirements Consistent API for all middleware components Gris-sys Standardized error codes and error handling procedures All users Framework to easily use authentication, authorization and secure comSP, PUS, SEU munication Client APIs for most prominent programming languages PUA, PUS Utilization and Usability Requirements Visualization of the completed results SEU Application steering modification of parameters during computation SEU Safe and easy authentication procedures SEU, SP Reliable real-time and instantaneous job submission for high priority SEU jobs for e.g. risk and disaster management, recovery, etc.

an access control decision based on the user’s authentication trustworthiness. Although the user has passed system authentication, we can not make certain whether he is trusted or not. There are some uncertainties in authentication systems, such as the uncertainties of the authentication mechanisms, authentication rules and authentication conclusions. These uncertainties in the authentication process can be modelled using the Fuzzy Logic. It has widely been applied in Grid computing for intrusion detection systems as well as for the prediction of the users’ actions and decisions [2, 25]). In other approaches the concept of authentication trustworthiness is analyzed and applied. In [21] the authentication trustworthiness Role-Based Access Control (RBAC) model is proposed. The model associates authentication trustworthiness RBAC model, and the authentication trustworthiness of the authenticated user is defined to activate his roles and permissions. Another possible solution of the effective authentication/authorization could come directly from the Virtual Organizations (VOs). The users becoming members of a VO are authorized and have granted access to VO’s resources. The VO membership contributes to distributing user management overheads and removes the need for replication of administrative efforts across the Grid. The use of VOs to

6


administer users is an attractive feature of Grids. Thus the concept of VOs is also a key issue in the Grid architecture development and the designing a proper architectural model of Computational Grid (CG) is one of the most important issues in an efficient resources, tasks and users management according to the various users’ requirements. The hierarchical layered structure seems to be well suited to capture the realistic administrative features of a real-life, large-scale distributed CG environment. In the following sections we identity the Grid users relationships and decisions in solving the scheduling problems.

1.3 Grid Users Relations in Grid Scheduling 1.3.1 Hierarchic Grid Infrastructure Computational Grid, hierarchical from nature, is usually modelled as a multi-level system, which allows an efficient management of the geographically distributed resources and tasks scheduling under various criteria, including security and resource reliability requirements. The model can be defined as a hybridization of centralized and decentralized modules [22]. In the centralized module, there is a central authority (meta-scheduler or meta-broker), who has a complete knowledge of the system by monitoring the resources and interacts with local job dispatchers in order to define the optimal schedules. In the decentralized module, the local schedulers interact with each other to manage the tasks pool. They have the knowledge about the resource clusters, but they cannot monitor the whole system. The hierarchy in such a system can usually consists of two or three levels. The general concept of such systems is presented in Fig. 1.2. An example of one of the simplest hierarchical bi-level Grid model is the MetaBroker model (MB) [8]. In this system, Grid users submit their applications to the meta-broker. MB uses also the information supplied by the resource owners to match the users’ tasks needs to appropriate machines. The three-level Grid system is defined in Kwok et al. [13]. The model is made up of the global, the inter-site and the intra-site levels. At the intra-site level, there is a federation of autonomous machines. The resource owners send an information about the computational capacities of the machines to the local managers, who defines the “Grid sites reputation indexes" and send them to the global scheduler. The global scheduler, operating at the global level, performs the tasks scheduling adaptively according to available scheduling algorithms implemented as well as scheduling modes (such as immediate and batch) specified by users for their tasks and applications.

1


7

Fig. 1.2 Two- and three-level hierarchical architectures of Computational Grids ((a) and (b), respectively)

1.3.2 Users’ Requirements, Relations and Strategies in Job Scheduling The roles of Meta-broker, Local Managers and Global Authorities in the scheduling process depend, in particular, on the End-Users’ (SEUs’) requirements defined in the following way: Requirements for Specifying a Single Computational Task. At the basic level, the end-user will need to be able to specify and submit a single monolithic application (with well defined input and output data), a bag-of-tasks application with no dependencies among them. The user may as well be required by the system to provide information on types of tasks (e.g. data intensive vs. CPU intensive computing) and an estimation of task workload (e.g. in millions of instructions). In many cases users should be able to submit their tasks/applications as either executable or source code, in which case will need to be compiled and linked for further execution. The later, may require software deployment not available in the Grid node. Due to the complexity of the compilation and execution processes on heterogeneous architectures on the Grid, it is recommended to build and test the users’ application on a specific platform prior to submitting them to the Grid. Finally, in most cases data is assumed to be shipped with the task/application. In a more advanced setting, data should be accessed at some Grid nodes, thus, it is suggested to situate processing nodes “close" to data nodes. Requirements for Specifying a Job of Multiple Tasks. The user should be able to specify a complex job that involves the execution of multiple dependent tasks. The input and output data for the tasks must be defined for each task, providing the

8


ability to specify relational data. Some graphical interfaces and representation of the graph structures (i.e. Directed Acyclic Graph (DAG)) may be provided for the specification of tasks dependencies. Access to Remote Data. The input and output data specified by the user may be stored remotely, thus he will need to provide the location of the remote data. If a ubiquitous wide-area file system is in operation on the Grid, the user would only have to care about the location of files and data with respect to some root location under which they are stored. Resource Specification. The user should be able to specify resource requirements, such as the number of processors and the hardware attributes required for that task. He may wish to target the particular types of resources, but, should not have to concern with the type of resource management on the Grid, nor with resource management systems on individual resources on the Grid. Resource reliability. In some cases the machines in the grid system could be unavailable due to dynamics or special policies of the resource owners. The user should be able to get some information about the resource reliability to avoid or, at least, to reduce the cost of possible abortion of hist tasks. It is of course upon system’s criteria to activate re-scheduling, task migration, and pre-emption policies. Trustfulness of Resources - Secure Scheduling. The user could require to allocate his tasks in the most trustful resources. Thus, he should be able to verify the trust indexes of the resources and define the security demands for his tasks on resources to be allocated to. Standardized authentication and authorization mechanisms requirements. Users on the computational Grid will most likely make use of an standardized certificate authentication scheme. The certificates can be digitally signed by a certificate authority, and kept in the users repository, which is recognized by the resources and resource owners. It is desirable for a certificate to be automatically created by the user’s interface application when a user submits a task to the Grid. Job Monitoring and Control. Users should be able to monitor and control the progress status of their tasks. The various types of Grid users can act independently or form the short- and long-term coalitions to satisfy the requirements of the SEUs. In Fig. 1.3 we present three main users relationship scenarios that reflect the realistic access management models in Grid scheduling. In the Non-cooperative scenario the users act independently of each other. It can be observed in a realistic Grid that the cooperation during the tasks/application submission is rather difficult to happen at large scale. Grid users usually independently of each other submit their tasks/applications to the Grid system. Also the resource owners act selfishly in order to maximize the resource utilization. Their principal interest is to execute tasks from local users rather than contributing to the execution of the remote tasks.

1


9

! " "

% & '(

!# $ $

& $ % )!*$( +

Fig. 1.3 The users’ relations in Computational Grids

Cooperativeness means that the users can form a coalition to plan in advance their actions. This model is useful for the intra-site Grid negotiations, where the local job dispatchers can define the joint execution capabilities parameters for the clusters of the Grid sites and notify them to the global scheduler. In Semi-cooperative model each user can select a partner for the cooperation. This scenario is applied in the multi-round auction models to incorporate rescheduling mechanism. The selection of the partner for cooperation is a complex issue and has been addressed in many research work. For instance, one can select trusted partners using trustworthiness models. Being located at the different levels in the hierarchical Grid structure, the users can have different privileges to the resources, so two following scenarios can be additionally analyzed: • Symmetric scenario. In this case there are no special privileges in the resource usage for the Grid users. • Asymmetric scenario. In this case there is a privileged user (Leader), who can have a full access to resources as opposed to the rest of users who can be granted only limited access to resources. The Leader could also be the owner of a large portion of the task pool, as it is reasonable to allocate first his tasks at best resources in the system.

10


The multi-level Grid structure has a direct impact on the users behavior, strategies and decisions. The central control of the Grid system, which is provided by the Global Schedulers and Meta-Brokers, can be implemented by gathering the system at a single virtual site, executing the decisions at this site and propagating the decisions to Grid clusters. In the local clusters the local decision makers can act independently of the other Grid partners adjusting their individual jobs acceptance and migration policies with respect to the local user community rules, current load, and the current state of the whole system. For example, if a site or a local cluster is heavily loaded, the acceptance of further external jobs from the other Grid partners would lead to further congestions which worsens the whole Grid performance. Therefore, there is a need to search for dynamic scheduling approaches, which can be effective in solving the multi-level decision problems and are able to flexibly adapt the decisions to the dynamical changes of the Grid states. Recently the game theory has been found as a powerful support for modelling the complex users decisions in various relation scenarios. In fact, the Grid hierarchical systems can be easily transformed to the game-theoretical models in order to push the concept of CG into mainstream computing (market-based approaches) and to solve efficiently the scheduling problem under various criteria defined jointly as a scheduling objective.

1.4 Game-Theoretic Models for Scheduling and Resource Management Game theory is playing an important role in computer science, where it is being used as a support for modelling interactive computations or multi-agent systems. Recently, Internet computing is seen as a new domain of applications of game theory, which in combination with economic theory can develop algorithms for finding equilibria in computational markets, computational auctions, Grid and P2P systems as well as security and information markets. An important challenge in using gametheoretic models for Grid scheduling and resource management is the large size scale of the Grid system and the fact that resources cross different administrative domains. The basic assumptions that underlie the game-based models are that game players are rational and pursue well-defined objectives (cost or pay-off functions), and they take into account their knowledge or expectations of other player’s behavior. In many cases Grid users and resource owners are likely to behave in different manners and can specify additional security and resource reliability requirements. In such a case their behavior cannot be characterized using conventional techniques. Game-theoretical models are quite natural tools for addressing this problem. All scheduling criteria can be aggregated and defined as a joint users’ cost or pay-off functions, which make these kinds of models very useful in the analysis of the various users’ strategies in the resource allocation process. The main game variants, namely non-cooperative, semi-cooperative, cooperative, symmetric and asymmetric games cope directly with the relationship and privilege scenarios of the users presented in Sec. 1.3.2. The solution of each of those games

1


11

is an equilibrium state, in which each player holds correct expectations concerning the other players’ behavior (see [10] for the detailed analysis). Due to the large scale hierarchic nature of Grid systems, the non-cooperative asymmetric game is a potential model for the integrating various and usually conflicting scheduling criteria like security, resource utilization and resource reliability. In the non-cooperative model, the users act independently of each other. They are selfish and cannot form any coalition. This game scenario is very useful for defining the aggregate scheduling criteria. In [8] the authors propose the Grid users’ game, in which the task execution and resource utilization costs are defined as the bi-objective players’ cost functions. The resource owners selfish behavior in the hierarchical structure was modelled as a non-cooperative game in [12]. The authors considered the risky and insecure conditions in online scheduling in Grids caused by software vulnerability and distrusted security policy. Recently, in [12] the non-cooperative Grid End-users’ game model was introduced for addressing the requirements for security and resource reliability in Grid scheduling. The asymmetric End-users’ model is defined in [11] by using the Stackelberg game scenario. In this case one player (user) is acting as a Leader with a privileged access to resources. He assigns his tasks firstly, and the rest of the users (Followers) react rationally to the Leader’s action. The Followers do not cooperate with each other, but their decisions depend on the Leader’s action. This model illustrates very well the real-life situation, where the roles of the users are in fact asymmetric with regard to their access rights and usage of resources. It must also be noted that in many economical models the sellers and buyers stand in an asymmetric position. Having a control on a large resource pools and maintaining the large fraction of the task batch for scheduling can be the reasons of having some privileges in the resource access (especially in the case of the tasks with critical deadlines and some special reliability and security requirements) or in the setting of the reasonable resource utilization pricing policies. The Stackelberg games have been well-studied in the game theory literature (see, e.g. [4]). Roughgarden [18] defined a Stackelberg game model for scheduling tasks on a set of machines with load-dependent latencies in order to minimize the total latency of the system. Let us denote by N the number of users (players). The total number of tasks n in a given batch can be calculated as the sum of the tasks submitted by all users, i.e. n = ∑Nl=1 kl , where kl is the number of tasks of the user l = 1, . . . , N. A schedule x is encoded by the following vector: x = [x1 , . . . , xN ],

(1.1)

where: xl = [xk +1 , . . . , xk +k ] (l−1)

(l−1)

l

(1.2)

denotes the strategy vector of the user l (kl is the number of the l-user’s tasks, and l k (l−1) = k1 + . . . + k(l−1) ). The coordinates of vector x are the decision variables of the user l and indicate the numbers of the machines, to which his tasks are assigned.

12


We use this notation for the definition of the users’ strategies and actions under the game-theoretic framework.

Asymmetric Scenario – Stackelberg Game For illustrating the asymmetric scenario in non-cooperative Grid users’ game we define the Stackelberg game , in which one user acts as a Leader and the rest of players (users) are his Followers. Formally it can be defined as a two-level game in the following way: • Leader’s Level: Leader’s action I - Leader chooses his initial strategy x1 = [x1 , . . . , x k1 ], where k1 denotes the number of tasks submitted by the Leader. • Followers’ Level: Followers’ action - Followers minimize simultaneously their cost functions relative to the Leader’s choice: ⎧ 2 N 1 2 ⎪ ⎪ ⎨ xF = arg minx2 ∈J2 {Q2 (x , x , . . . , x )} .. (1.3) . ⎪ ⎪ ⎩ N N 1 xF = arg minxN ∈JN {QN (x , . . . , x )} where J1 is the set of the Leader’s strategies. Let us denote by xF = [x1 , x2F , . . . , xNF ] the vector, which is interpreted as the result of the Followers’ action. • Leader’s Level: Leader’s action II - Leader updates his strategy by minimizing his cost function Ql taking into account the result of Followers’ actions. The following vector xG = [xL , x2F , . . . , xNF ], where: xL = arg min Q(x1 , x2F , . . . , xNF ) x1 ∈J1

(1.4)

is a solution of the whole game. It is assumed that the Followers can play an ordinary non-cooperative symmetric game (i.e. non of them have any special privileges to the resources), but they must know the Leader’s action first. We denote by Q game multi-cost function . This function is interpreted as a total cost of playing the game by all users (including Leader) and can be defined in the following way: Q := Q1 + QF ;

(1.5)

where Q1 is the Leader’s cost function and N

QF :=

∑ Ql ;

(1.6)

l=2

is a Followers’ cost function. An optimal solution of the game is called Stackelberg equilibrium .

1


13

The problem of the minimization of the cost function Q can be defined as a hierarchical procedure composed of two cooperating modules: Leader’s Module, in which the values of the functions Q1 and Q are calculated and optimized, and the Followers’ Module - which solves the local level problem of the minimization of the Followers’ cost function QF .

1.5 Solving the Grid Users’ Games Due to multiple constraints and different optimization criteria in a dynamic environment, resource allocation in Grids still remains a challenging, complex and computationally hard global optimization problem. Thus, heuristic approaches are candidates for the effective Grid schedulers . Meta-heuristic-based scheduler could achieve robustness and is appropriate for tackling the various scheduling attributes, like immediate and batch scheduling , multi-objectivity, decentralized and hierarchical Grid architectures, etc. [22]. Another important feature of meta-heuristics, which is useful in Grid scheduling, is that they can be easily hybridized with other approaches. It makes the Grid schedulers better adapted to the various Grid types and specific types of applications. Abraham et al. [1] present a model for hybridization of GA, SA and TS heuristics; each GA-based hybrid, namely GA+SA and GA+TS improves the efficiency of the genetic scheduler. A GA+TS hybrid approach was by Xhafa et al. [24]. Ritchie and Levine [17] combine an ACO with a TS algorithm for the problem. In [23] a basic unstructured MA is combined with several local search algorithms in order to identify the best performance of the resulting Memetic algorithm. The hybridization ability and dealing with various conflicting optimization criteria at the same time are the crucial features, which makes the meta-heuristics the appropriate methods for solving the Grid users’ games. In such games various scheduling and resource management criteria, like security and resource utilization and reliability, are integrated into the users’ cost functions. It is different to the other approaches, where those criteria are addressed separately, what contradicts in fact with the complex nature of Grid systems. In the following Subsec. 1.5.1 we present the main concepts of four hybrid GA-based schedulers, previously defined in [12] and [11], which are well-suited for optimizing the game multi-cost function Q.

1.5.1 GA-Based Hybrid Approach A general concept of hybridization of four GA-based approaches for solving the asymmetric Grid users’ game is presented in Table 1.2. The GA-based meta-heuristics act as global and local optimizers in the Leader’s and Followers’ Modules. Each hybrid is defined as a combination of two methods - Risky Genetic Algorithm (RGA) and Secure Genetic Algorithm (SGA)- in the Leader’s Module- and two local level optimizers - Player’s Genetic Algorithm (PGA) and a modified MCT heuristic, namely Player’s Minimum Completion Time (PMCT), in the Followers’ Module. The main difference between RGA and SGA

14


Table 1.2 Hybrid meta-heuristics for risky and secure-assured task scheduling. Hybrid Leader’s Followers’ Meta-heuristic Module Module RGA-GA RGA PGA RGA-PMCT RGA PMCT SGA-GA SGA PGA SGA-PMCT SGA PMCT

algorithms is the method of the evaluation of the population by the users’ cost functions Ql . The users’ cost of playing the game can be calculated differently in the case of considering the security condition and in the case of ignoring all security requirements (we call these two scenarios the secure and risky modes and define the appropriate players’ cost functions formulas in Subsec. 1.6.1). The general template of the GA-based schedulers at the Leader’s level is given in Alg. 1. The process of initialization of the population in the hybrid GAs is defined as a two-steps procedure. In the first step we define P0 as a candidate initial population. It consists of the incomplete schedules computed by the Leader using one of the initialization methods for GA-based schedulers (see e.g. [23]). Each schedule from this set contains just the values of the Leader’s decision variables. All those “incomplete" chromosomes are sent to the Followers, who fill in the respective parts of each schedule by using one of the ad-hoc heuristics. The updated P0 is then evaluated under the game cost function Q = ∑l=1...N Ql defined as the fitness. The crossover and mutation operations are performed separately on Leader’s and Followers’ decision variables. Thus, in each generation the Followers can update their own decision (including the initial choices) due to possible changes on availability of resources introduced by the Leader. Local schedulers in Follower’s module We used two modifications of the well-known Grid schedulers for optimizing the users cost function in the Followers’ Module. The first, called Player’s Genetic Algorithm, is a simple extension of the classical GA-based scheduler [23] applied independently for each follower with the cost function Ql as a fitness. The GA operations are executed on sub-schedules of the length kl labeled just by the tasks submitted by user l. The GA procedure is run sequentially for the “queue" of users. The second method, called Player’s Minimum Completion Time - (PMCT), is a modification of Minimum Completion Time - MCT. In this method, a task is assigned to the machine yielding the earliest completion time (defined as the sum of ready_time for the machine and time of computing all tasks assigned there). The process is repeated until there remain tasks to be assigned. The template of the main mechanism of PMCT procedure is defined in Alg. 2.

1


15

Algorithm 1. A hybridized GA-based scheduler template - Leader’s level 1: Generate P0 containing μ “incomplete" schedules; t = 0; 2: Send P0 to the Followers to complete the respective parts of all schedules in P0 (using ad-hoc heuristics); P0 (F) is created; 3: Update the population P0 according to the Followers’ solutions;P0 := P0 (F); 4: Evaluate P0 ; 5: while not termination-condition do 6: Select the parental pool T t of size λ ; T t := Select(Pt ); 7: Perform crossover procedures separately on Leader’s and Followers’variables on pairs of individuals in T t (F) with probability pc ; Pct := Cross(T t ); 8: Perform mutation procedures separately to Leader’s and Followers’ variables on individuals in Pct with probability pm ; Pmt := Mutate(Pct ); 9: Evaluate Pmt ; 10: Create a new population Pt+1 of size μ from individuals in Pt and Pmt ; Pt+1 := Replace(Pt ; Pmt ) 11: t := t + 1; 12: end while 13: return Best found individual as solution;

Algorithm 2. PMCT algorithm template 1: Receive the population of schedules and ready_times of the machines from the Leader’s Module; 2: for all Schedule in the population do 3: Calculate the completion times of the machines in a given schedule; 4: for all Follower do 5: for all Follower’s Task do 6: Find the machine that gives minimum completion time; 7: Assign task to its best machine; 8: Update the machine completion time; 9: end for 10: Calculate the minQl value for a given schedule; 11: end for 12: Send the minQl values to the Leader’s Module; 13: end for

A simple case study on the four hybrid GA-based schedulers approaches in solving the users’ Stackelberg game is presented in the following section.

1.6 A Case Study: Non-cooperative Asymmetric Stackelberg Game of the Grid Users in Independent Batch Scheduling An Independent Job Batch Scheduling is often referred as a fundamental scheduling problem in Computational Grids. It usually arises in data intensive computing such as data mining and massive processing applications. For this class of Grid-enabled

16


applications, batch mode is appropriate task processing mode. In such a case, jobs or applications are grouped in batches and scheduled as a group. The independent scheduling problem can be formalized using the Expected Time To Compute matrix (ETC) model [3]. In this model the following input data has to be specified: • A number of independent tasks to be allocated to Grid resources in non-preemptive mode. • A number of machines candidates to participate in the allocation of tasks. • The workload (in millions of instructions) of each task. • The computing capacity of each machine (in Mips). • The ready times indicating when machines will have finished the previously assigned tasks. • The ETC matrix of size nb_tasks × nb_machines, where ETC[ j][m] is the value of the expected time to compute task j in machine m.

1.6.1 Players’ Cost Functions In our case study we consider the asymmetric Stackelberg game (see Sect. 1.4), for which the players cost functions are defined as follows: (s)

(ab)

Ql = Ql + Ql

,

(s)

(1.7) (ab)

where: Ql indicates the cost of security-assured allocation of the user tasks, Ql is the cost of possible abortion of the user’s task due the resource unavailability, and denotes a resource utilization cost. (s) The values of the function Ql depend on the scheduling strategy. We consider in this work two scheduling strategies: • risky mode - in which all risky and failing conditions are ignored by the users. (s) In this case Ql = 0, l = 1, . . . , N. (s)

• secure mode - in which Ql function is defined as follows: (s)

Ql =

(k1 +...+kl )

∑

j=(k1 +...+kl−1

Pf [ j][x j ] · ETC[ j][x j ] , (ETC)m(l) · kl +1)

(1.8)

where ETC[ j][x j ] is an element of the matrix ETC and (ETC)m(l) is the (expected) maximal computation time of the tasks of the user l in a given schedule. We denoted by Pf [ j][x j ] in Eq. 1.8 the probability of the failure of machine x j during the execution of task j. This probability is usually modelled by the negative exponential distribution: 0 , sd j ≤ tlx j Pf [ j][x j ] = (1.9) −λ (sd j −tlx j ) 1−e , sd j > tlx j

1


17

where λ is interpreted as a failure coefficient and is a global parameter of the model, and sd j and tlx j are the elements of security demand SD and trust level T L vectors ([13]). (ab) The cost of user’s tasks abortion Ql is defined using the following formulae: (ab) Ql

(k +...+k )

=

1 l Pab ( j) · ETC[ j][x j ] ∑ j=(k 1 +...+kl +1)

(ETC)m(l) · kl

,

(1.10)

where Pab ( j) is a task abortion probability, which is calculated as Pab ( j) = (1 − Px j ), where Px j denotes a reliability probability specified for a given machine x j .

1.6.2 Experiments Setting In the experimental evaluation of the proposed four hybrid meta-heuristics we integrated the schedulers with the discrete event-based Grid simulator HyperSim-G [24]. The experiments were conducted on two benchmarks composed by a set of static and dynamic instances. In both static and dynamic cases four Grid size scenarios are considered: small (32 hosts/512 tasks), medium (64 hosts/1024 tasks), large (128 hosts/2048 tasks), and very large (256 hosts/4096 tasks). The simulator setting is presented in Table 1.3.

Table 1.3 Setting of the grid simulator for static and dynamic cases Static setting Medium Large Very Large Nb. of hosts 64 128 256 Resource cap. (in MIPS) N(1000, 175) Total nb. of tasks 512 1024 2048 4096 Workload of tasks N(250000000, 43750000) Dynamic setting Init. hosts 32 64 128 256 Max. hosts 37 70 135 264 Min. hosts 27 58 121 248 Resource cap. (in MIPS) N(1000, 175) Add host N(625000, 93750) N(562500,84375) N(500000,75000) N(437500, 65625) Delete host N(625000, 93750) Total tasks 512 1024 2048 4096 Init. tasks 384 768 1536 3072 Workload N(250000000, 43750000) Interarrival E(7812.5) E(3906.25) E(1953.125) E(976.5625) Small 32

We have considered 15 Followers in the Stackelberg game and the number of the Leader’s tasks is a half of the whole task batch. The coefficients of SD and T L vectors, and the machines reliability probabilities Px j are defined as the uniformly generated fractions in the ranges [0.6;0.9], [0.3;1] and [0.85;1] respectively. The value of failure coefficient λ is 3.

18


In Table 1.4 we present the key parameters for the GA engines in Leader’s and Followers’ Modules. Table 1.4 Settings of GAs key parameters in Leader’s and Followers’ Modules. Parameter SGA and RGA PGA Population size 60 20 Intermediate pop. size 48 14 Crossover Prob. 0.8 0.8 Mutation Prob. 0.2 0.2 Stopping criterion 1000 iterations 500 iterations

A combination of the genetic operators for GAs was selected based on the results of the tuning process performed in [23]. We used linear ranking selection, cycle crossover (CX), rebalancing mutation and elitist generational replacement as the main evolutionary mechanism in Leader’s and Followers’ Modules. We also applied LJFR-SJFR (Longest Job to Fastest Resource – Shortest Job to Fastest Resource) as an initialization procedure to introduce more diversity to the initial population. As scheduling evaluation criteria we used Makespan and Flowtime metrics (see [24]) defined in the following way: • Flowtime: Let Fj denotes the time when task j finalizes. The flowtime can be calculated using the following formulae: Flowtime =

∑

Fj

(1.11)

j∈Task

Flowtime is usually considered as a QoS criterion as it expresses the response time to the submitted task execution requests of Grid users. • Makespan: Makespan is one of the basic metrics of a Grid systems performance: the smaller the value of makespan, the faster is the execution of tasks in the Grid system. Makespan can be calculated by the following formulae: Makespan = max Fj j∈Tasks

(1.12)

1.6.3 Computational Results Each experiment was repeated 30 times under the same configuration of parameters and operators. In Fig. 1.4 we present the averaged values of Makespan and Flowtime achieved by four hybrid meta-heuristics in solving the Stackelberg game in static and dynamic Grid scenarios. It can be observed that two PMCT hybrids outperform the RGA-GA and SGA-GA algorithms. For makespan values the differences in the results achieved by PMCT and GA hybrids are significant, while in the case of flowtime all values are at the same level, except those obtained for very large Grid size. The best results in all

1


19

4600000

1.00E+10 9.00E+09

4400000

8.00E+09 7.00E+09

4200000

6.00E+09 5.00E+09

4000000

4.00E+09 3800000

3.00E+09 2.00E+09

3600000

1.00E+09 3400000

0.00E+00 small

medium

large

very large

small

medium

large

very large

medium

large

very large

4900000

1.00E+10

4800000

9.00E+09

4700000

8.00E+09 7.00E+09

4600000

6.00E+09

4500000

5.00E+09 4400000

4.00E+09

4300000

3.00E+09

4200000

2.00E+09

4100000

1.00E+09

4000000

0.00E+00 small

medium

large

very large

small

Fig. 1.4 Experimental results for Stackelberg game: in static case - (a) average Makespan, (b) average Flowtime ; in dynamic case - (c) average Makespan, (d) average Flowtime

instances are achieved by SGA-PMCT algorithm. However, in the case of static scheduling scenario the efficiencies of RGA-PMCT and SGA-PMCT are very similar, while in the dynamic case, especially for makespan values, the differences in both schedulers performances are significant. Although the security requirements would imply some additional cost to the users of the Grid system, it is worth assuming this cost in order to allocate tasks to trustful resources. Our experimental analysis show that hybrid GA-based schedulers can be effective in solving the users games, however the main drawback of using such methods is their high computation complexity. The Stackelberg game scenario presented in Sec. 1.4 define a general case, which makes it useful in supporting the users decision process in various situations. In some real-life approaches the particular game scenarios can be applied for an efficient resource allocation. Most of them are based on the economical models. In the following section we briefly highlight the most popular game models used in Grid scheduling.

1.7 Other Approaches 1.7.1 Computational Economy Computational economy has been widely used for the resource management and architecture design of Grid systems. It allows the resource owners, acting as sellers, to earn money by letting others (mainly the Grid users, acting as buyers) use their (idle) computational resources. The pricing of the resources is driven by demand and supply.

20


A number of economic models for Grid resource management has been proposed in the literature so far. The most popular game-theoretical based approaches in Grid Computing are the commodity market, auctions and bargaining models. Commodity Market model This model is based on the Meta-broker architecture. It is assumed here that the service providers primarily charge the end user for the resources they consume and the pricing policies are based on the demand from the users and the supply of resources. The resource owners and service providers are selfish in this approach and the end-users may cooperate or not [6]. Auctions In this model there are two groups of participants: sellers (resource owners) and buyers (Grid end-users). The cooperation between users to form a coalition and win the auction is possible, but usually users behave selfishly. The auction mechanism can be defined in many ways (e.g. English, Dutch, First and Second Price auctions). All of them differ in terms of whether they are performed as open or closed auctions and the offer price for the highest bidder. The users’ strategies in various auctions scenarios are discussed e.g. in [9]. Bi-level synchronized auctions The First Price bidding auction mechanism has been extended by Kwok et al. [13] to define the resource management and global scheduling policy at the intra- and intersite levels in the 3-levels hierarchical Grid structure. In the intra-site bidding each machine owner in the site, who acts selfishly, declares the “execution capability" of the resource. The local manager monitors these amounts and sends a single value to the global scheduler. In the inter-site bidding the global scheduler should allocate tasks according to the values sent by the local dispatchers. The authors proved that the cooperation of the players at both levels are the optimal strategies for both level-auctions. However, for the successful execution of all those strategies some synchronization mechanism must be introduced, which can make the whole system inefficient in a large-scale dynamic environment. Bargaining models In this model the resource brokers bargain with resource providers for lower access price and longer usage duration. The negotiation process is guided by the end-users requirements (e.g., deadline) and can be provided directly between buyers (Endusers) and sellers (resource owners). The most recent study on the bargaining cooperative model application in optimizing the energy consumption in Grid is proposed in [20]. Each model can be easily modified to meet user or resource owners requirements. For example, the Posted Price Model [6] is an extension of commodity market by

1


21

considering posted prices (usually lower than regular ones) as a special offer to the Grid users.

1.7.2 Neural Networks and Markov Decision Processes As shown in previous sections, the game-based models can capture many realistic scenarios of Grid users’ relations and activities. However, some other proposals can be considered as promising tools for defining complex decision making processes of the Grid users. In [19] the authors introduced the users’ decision model based on the artificial neural network paradigm . Their model consists of three main components: • online module dedicated to the prediction of the users’ actions; • off-line model based on the analysis of statistical data acquired during user’s work; and • change detection module defined for the detection of trends and changes in users’ activities. The users’ decision mechanisms are based on the feed-forward neural networks trained by the back-propagation method. This method can be considered as an alternative to game-based ones for the characteristics of the dynamics of the Grid users’ decision process in the online scheduling. The authors additionally proposed the off-line model, where another neural network is applied, for the detection of normal/abnormal users activities, by analyzing the statistical data accumulated during the users’ actions. It seems that the proposed approach can be a good start for some more detailed analysis of the users decision processes, however the complexity of the model can be a main drawback for its successful application in real-life Grid scenarios. Another promising approach can be an application of Markov Decision Processes (MDP) in predicting the users’ decisions in Computational Grids. The theory of MDPs can be characterized as a mathematical tool for modeling sequential decision tasks. It has been successfully applied in large-scale intelligent systems mainly because of its ability to provide a realistic simulation of complex real-life problems and to define a mathematical background for the independently-developed methods of learning the users’ activities and behavior. The MDP model assumes the existence of a physical system that evolves in discrete time and that is controlled by an agent. The system dynamics are governed by probabilistic transition functions that maps and controls the system states. Formally the MDP can be characterized by the following four structures: • a finite state space (the system) S = {1, . . . , n} , • a finite set of control variables C(i) for each state i ∈ S, • transition probabilities p(i;c) ( j) for all c ∈ C(i), which express the probability of reaching the next state j after applying control c in state i, and • a cost g(i; c) associated to c ∈ C(i) and i ∈ S.

22


A strategy or policy λ can be defined as an infinite sequence of (π1 , . . . , πt , . . .) of mappings of the states S to control variables (the agent applies the control πt (i) in state i at time t, and πt (i) ∈ C(i) for all i ∈ S). The cost associated to strategy λ can be calculated as an (optimal) expected total cost of the agent’s control of the system, which is taken with respect to the probability distribution induced by the transition probabilities. Additionally some discount factor of the range [0, 1] can be specified (see[5]). To find an optimal (stationary) strategy which allow an effective minimization of the strategy cost is not an easy task in a large-scale networks. However, in the case of the significant cost reduction by the discount factor, the model can be optimized by the Dynamic Programming method, which makes the approach promising tool for solving the scheduling problems with the budget and time linear restrictions (see [15])). In [5] we can find the strong mathematical background and the successful applications of MDP in Grids in the case of limited agent’s knowledge of the system state. This approach can be start point for extending the idea for the more complex scheduling scenarios.

1.8 Conclusions and Future Work In this chapter we presented a study of the user types, their requirements and behaviors in Computational Grids (CGs). We have specified a categorization of the Grid users types and have defined their relationship and access scenarios in CGs corresponding to their requirements. We identified and analyzed some fundamental features arising in the users’ behavior in Grid scheduling process, such as dynamics, selfishness, cooperativeness, trustfulness, symmetric and asymmetric roles. We demonstrated the effectiveness of the game-theoretic models in supporting the Grid users’ decisions, where the various scheduling criteria, including security and resource reliability, must be considered in the same time. Due to large-scale of the Grid, the non-cooperative games seem to be the potential models for integrating various users’ requirements in Grid scheduling. As a case study we presented a concept of the non-cooperative asymmetric game scenario, namely the Stackelberg game and defined simple GA-based hybrid schedulers for approximating the game equilibrium states. A simple experimental analysis show the efficiency of the hybrid genetic-based schedulers as the resolution methods for game-based models, especially in the case of additional security and task abortion costs paid by the users. We discussed in the end of this chapter how computational economy-based approaches, such as market mechanisms, and computational paradigms, such as Neural Networks and Markov Decision Processes, can be used to predict the users’ actions. We believe that the presented game-based models concept can be successfully implemented also in Cloud Computing, where the secure scheduling and information management remain challenging problems. We believe that the modelling and the analysis presented in this chapter will serve as a basis for application designers in CGs to correctly embed user requirements and behaviors into Grid systems.

1


23

References 1. Abraham, A., Buyya, R., Nath, B.: Nature’s heuristics for scheduling jobs on computational grids. In: Proc. of the 8th IEEE International Conference on Advanced Computing and Communications, pp. 45–52 (2000) 2. Abraham, A., Jainb, R., Thomasc, J., Hana, S.Y.: D-SCIDS: Distributed soft computing intrusion detection system. Journal of Network and Computer Applications 30(1), 81–98 (2007) 3. Ali, S., Siegel, H.J., Maheswaran, M., Hensgen, D.: Task execution time modeling for heterogeneous computing systems. In: Proceedings of Heterogeneous Computing Workshop, pp. 185–199 (2000) 4. Baçsar, T., Olsder, G.J.: Dynamic Non-cooperative Game Theory, 2nd edn. Academic Press, London (1995) 5. Bonet, B.: An Epsilon-Optimal Grid-Based Algorithm for Partially Observable Markov Decision Processes. In: Proc. of International Conference on Machine Learning - ICML 2002, pp. 51–58 (2002) 6. Buyya, R., Bubendorfer, K.: Market Oriented Grid and Utility Computing. Wiley Press, Chichester (2009) 7. http://knowledge.eu-egi.eu/knowledge/index.php 8. Garg, S.K., Buyya, R., Segel, H.J.: Scheduling Parallel Aplications on Utility Grids: Time and Cost Trade-off Management. In: Mans, B. (ed.) Proc. of the 32nd ACSC, vol. 91, pp. 139–147. ACS (2009) 9. Ghosh, P., Roy, N., Basu, K., Das, S.K.: A Game Theory based Pricing Strategy for Job Allocation in Mobile Grids. In: Proc. of the 18th IEEE International Parallel and Distributed Processing Symposium (IPDPS 2004), Santa Fe, New Mexico (2004) 10. Khan, S.U., Ahmad, I.: Non-cooperative, semi-cooperative, and cooperative gamesbased grid resource allocation. In: Proceedings of International Parallel and Distributed Proceedings Symposium (IPDPS 2006), pp. 101–104 (2006) 11. Kołodziej, J., Xhafa, F.: A game-theoretic and hybrid genetic meta-heuristic model for security-assured scheduling of independent jobs in computational grids. In: Barolli, L., Xhafa, F., Venticinque, S. (eds.) Proc. of CISIS 2010, pp. 93–100. IEEE Press, Los Alamitos (2010) 12. Kołodziej, J., Xhafa, F., Kolanko, Ł.: Hierarchic genetic scheduler of independent jobs in computational grid environment. In: Otamendi, J., Bargieł, A. (eds.) Proc. of 23rd ECMS, Madrid, June 9-12, pp. 108–115. IEEE Press, Los Alamitos (2009) 13. Kwok, Y.-K., Hwang, K., Song, S.: Selfish Grids: Game-Theoretic Modeling and NAS/PSA Benchmark Evaluation. IEEE Transactions on Parallel and Distributing Systems 18(5), 1–16 (2007) 14. Mesghouni, K., Hammadi, S., Borne, P.: Evolutionary alogorithms for job-shop scheduling. Int. J. Appl. Math. Comput. Sci. 14(1), 91–103 (2004) 15. Montana, D., Zinky, J.: Optimizing Routing and Backlogs for Job Flows in a Distributed Computing Environment. In: Xhafa, F., Abraham, A. (eds.) Meta-heuristics for Scheduling in Distributed Computing Environments, Springer, Heidelberg (2008) 16. Norman, M.: Types of grid users and the Customer-Service Provider relationship: a future picture of grid use. In: Cox, J. (ed.) Proc. of the UK e-Science All Hands Meeting, Nottingham, September 18-21, pp. 37–44 (2006) 17. Ritchie, G., Levine, J.: A fast effective local search for scheduling independent jobs in heterogeneous computing environments, Technical Report, Centre for Intelligent Systems and their Applications, School of Informatics, University of Edinburgh (2003)

24


18. Roughgarden, T.: Stackelberg scheduling strategies. SIAM Journal on Computing 2, 332–350 (2004) 19. Shelestov, A., Skakun, S., Kussul, O.: Intelligent Model of user Behavior in Distributed Systems. Int. Journal on “Information Theories and Applications" 15, 70–75 (2008) 20. Subrata, R., Zomaya, A.Y., Landfeldt, B.: Cooperative power-aware scheduling in grid computing environments. J. Parallel Distrib. Comput. 70, 84–91 (2010) 21. Wang, L., Wei, L., Liao, X.-K., Wang, H.: AT-RBAC: An Authentication TrustworthinessBased RBAC Model. In: Jin, H., Pan, Y., Xiao, N., Sun, J. (eds.) GCC 2004. LNCS, vol. 3252, pp. 343–350. Springer, Heidelberg (2004) 22. Xhafa, F., Abraham, A.: Computational models and heuristic methods for grid scheduling problems. Future Generation Computer Systems 26, 608–621 (2010) 23. Xhafa, F., Carretero, J., Abraham, A.: Genetic Algorithm Based Schedulers for Grid Computing Systems. International Journal of Innovative Computing, Information and Control 3(5), 1–19 (2007) 24. Xhafa, F., Gonzalez, J.A., Dahal, K.P., Abraham, A.: A GA(TS) hybrid algorithm for scheduling in computational grids. In: Corchado, E., Wu, X., Oja, E., Herrero, Á., Baruque, B. (eds.) HAIS 2009. LNCS, vol. 5572, pp. 285–292. Springer, Heidelberg (2009) 25. Zhang, G., Sun, J.: Grid intrusion detection based on soft computing by modeling realuser’s normal behaviors. In: Proc. of IEEE International Conference on Granular Computing 2006, pp. 558–561 (2006)

Chapter 2

Efficient Hierarchical Task Scheduling on GRIDS Accounting for Computation and Communications Johnatan E. Pecero, Frederic Pinel, Bernabé Dorronsoro, Grégoire Danoy, Pascal Bouvry, and Albert Y. Zomaya

Abstract. This chapter proposes a novel Grid-based scheduling algorithm that optimizes both computation and communications costs of workflow applications. Based on a hierarchical two-steps optimization process, a super scheduler first applies a Recursive Convex Clustering Algorithm (RCCA) that efficiently clusters tasks while minimizing communication costs. In the second step, a resource-broker assigns the generated convex sets to resources clusters. Local schedulers then optimize the makespan for the group of tasks assigned to their cluster, using a graphic processing unit (GPU)-based parallel cellular Genetic Algorithm (cGA). The performance improvement brought by this novel two-step scheduling algorithm compared to a hierarchical list-scheduling approach is empirically demonstrated on different real-world workflow applications.

2.1 Introduction The problem of efficiently scheduling tasks on computational Grids has been deeply studied in the literature [12, 21, 23, 37, 30]. Nevertheless, it is still the focus of many research groups and the subject of a several international symposia. Basically, the Johnatan E. Pecero · Frederic Pinel · Grégoire Danoy · Pascal Bouvry Computer Science and Communications Research Unit, University of Luxembourg, 6 rue R. Coudenhove-Kalergi, L-1359 Luxembourg e-mail: [email protected] Bernabé Dorronsoro Interdisciplinary Centre for Security, Reliability, and Trust, University of Luxembourg, 6 rue R. Coudenhove-Kalergi, L-1359 Luxembourg e-mail: [email protected] Albert Y. Zomaya Advanced Network Research Group, School of Information Technologies, University of Sydney, Sydney, NSW 2006, Australia e-mail: [email protected]

P. Bouvry et al. (Eds.): Intelligent Decision Systems, SCI 362, pp. 25–47. c Springer-Verlag Berlin Heidelberg 2011 springerlink.com

26

J.E. Pecero et al.

problem consists in allocating a set of tasks onto a set of available resources to optimally execute them. The quality of a schedule is typically assessed by the time the latest task is finished, this indicator is known as the makespan. Many different variants have been defined in the literature to model this class of problem. In this chapter, a very realistic model is considered, in which the Grid is composed by a fixed number of clusters available to execute the jobs. When looking for the optimal schedule not only the cost of executing the jobs on the assigned resources is considered, but also the communication costs used by sending data both among clusters and resources in every cluster. We are particularly interested in finding schedules in such scenarios for very complex software programs that will be divided into a large number of jobs that have dependencies among them, defining some precedence constraints. Such scenario can be encountered in scientific simulation or rendering software, to name a few examples. The main contribution of this chapter is the design of a new algorithm that efficiently solves the problem. It is based on two-steps optimization process. In the first one, we use the convex-clustering algorithm [29] to assign efficiently sets of jobs to the different clusters, to minimize the communication costs. In the second step, a multi-agent parallel algorithm for GPU architectures was designed to optimize the schedule of the jobs assigned to every cluster in the first step. Every agent is in charge of efficiently scheduling the jobs assigned to a given cluster by executing an advanced cellular genetic algorithm (cGA), specially designed for this problem. Since there might be precedence constraints among the jobs assigned to different clusters, the agents synchronize among themselves to communicate the possible reductions obtained in the makespan of a given cluster. The proposed solution is highly scalable, because the problem is partitioned with the simple convexclustering heuristic, efficient, since the agents run in parallel into one single GPU, and accurate, thanks to the advanced cGAs that are used to solve every sub-problem (i.e., to schedule the tasks assigned to every cluster) [8, 15]. Figure 2.1 illustrates an example of our scenario. We suppose that the convexclustering heuristic divides the application into two sets of jobs, as it can be seen on the left-hand side of the figure. Then, in the second step, one agent per cluster optimizes the schedule of the jobs assigned to the corresponding cluster. In this example precedence constraints exist between jobs belonging to the sets of jobs 1 and 2 (jobs n3 and n10 must be executed after n1 and n7). Therefore, the agent optimizing the first set of jobs communicates to the other one the time when jobs n3 and n10 can be executed, according to its best local schedule at every time. Once the agents finish the optimization process, every set of jobs is assigned to a physical cluster with enough computing resources to execute it. The remainder of this chapter is organized as follow. The following Section describes the system, application and scheduling models used in this work. A description of resource management and the scheduling in the Grid is discussed in Section 2.3. Then, Section 2.4 describes in detail the proposed optimization approach. The results of our comparative evaluation study are presented in Section 2.5. Finally, Section 2.6 concludes the chapter.

2

Efficient Hierarchical Task Scheduling on GRIDS Accounting

n1 1 0

nn22 4 0

Grid Cluster i

6

2 n3

Set 1 of jobs

nn44 4

Application

27

0

Provider

0

n5 3

n6 3

0

Set 1 of jobs

cGA 0

0

nn77 3

n8 1

Completion time

0 8

Set 2 of jobs

2 n9

Set 2 of jobs

Cluster j

0

n10 3

Fig. 2.1 In our approach, the jobs of the application to execute are divided into groups of jobs, minimizing the communication costs among groups. Every group of jobs is assigned to a physical cluster, and our parallel algorithm optimizes the schedule of the jobs in every group

2.2 Models This section first illustrates the grid scheduling environment focusing on system and application models. Then the scheduling model used to represent the optimization problem is introduced, including an example of workflow scheduling . The latter problem is used to illustrate the proposed approach.

2.2.1 System Model The grid system R, considered in the current chapter consists of several sites or clusters, each of them including m computational hosts (Figure 2.2) [19]. That is, R = {s1 , s2 , ..., sr } and si = {hi,1 , hi,2 , ..., hi,m } ∪ di with 1 ≤ i ≤ r, where si is the i-th site participating in R, and hi , di , are respectively a set of host machines and data repository/storage at si . Let H = {h1 , h2 , ..., hr } denote a superset of all host sets in R, that is, a grid consists of a set of H of h nodes. Each site is an autonomous administrative domain. These sites are connected with each other through a wide area network . Hosts are composed of both spaceshared and time-shared machines and for the sake of simplicity and convenience, we assume that hosts are (almost) homogeneous in terms of their computing capability and capacity [16]. Since the targeted parallel system may be heterogeneous at many levels (computing power, interconnection network, performance, etc.), managing of such a platform towards a high performance computing one is a difficult task.

28

J.E. Pecero et al.

Pri m e r gy

Primer

y g

Pri merg y

Gigabit Switch

Data repository

Primer g y

Pri

mer g y

Pri

mergy

Prime

r g y

Prime rg y

Gigabit Switch

Pr ime rgy

Pri merg y

Data repository

Pr imergy

Prime rg y

y g

Prime

Data repository

Resource Information Services

Site

rgy

Pr imer

Gigabit Switch

rPimerg y

Grid Computing Infrastructure

Prime rgy

Pri m erg y

Pri mergy

Prime r gy

Gigabit Switch Prime rgy

Pri mergy

Pri merg y

Pri mer g y

Gigabit Switch

Data repository

Pr imer g y

Pri mergy

Data repository

Pr imerg y

Prime rg y

r im e P rgy

Gigabit Switch Prime r g y

Prime rg y

Data repository

Fig. 2.2 Grid model

One approach was proposed by Nasri, Steffenel, and Trystram in [24], in which the network computing system or Grid is subdivided in a set of homogeneous logical clusters with an accurate interconnection network. One main reason for constructing logical clusters is that machines may behave differently, therefore the easiest way to optimize communications is to group machines with similar performance. Additionally, virtualization technologies have facilitated the creation of such homogeneous systems. In this chapter we assume that the network topology of our Grid system is hierarchically divided in two levels with heterogeneous communications. The first (coarser-grained) level represents the inter-hosts connections, which uses a slower network than the second level (finer-grained) encompassing intra-host connections (see example in Figure 2.3). We assume that all network connections at the first level have the same data transfer speed Bw representing the transmission rate of a link connecting two sites without contention and measured in Mega bits per second (Mbps), and the same energy consumption rate ce measured in terms of Joule/Mb. The intra-processor or intra-host communications are assumed to perform with the same speed bw on all links without contentions. It is also considered that a message can be transmitted from one resource to another while a task is being executed on the recipient resource, which is possible in many systems. Hosts in the same site are able to access each other’s data repositories as if they were accessing their own, i.e., a set of data repositories in a site can be represented as a single data repository. This assumption is made since a site generally connects its hosts through a highbandwidth local area network. Furthermore, we consider topology heterogeneity, that is the clusters may differ in size, but we assume that the maximum number of hosts in each site is bounded by M. Finally, the availability and capacity of resources, e.g., hosts and network links, fluctuates. Therefore, the accurate completion time of a job on a particular host is difficult, if not impossible, to determine a priori. Moreover, the job may fail to complete due to a failure of the resource on which it is running. However, resource failures are not explicitly considered in this work.

2


29

Ethe Ethernet Switch 10/100 Mbps 10

Site A

Cluster 1

Site B

Cluster 2

Site C

Cluster 3

host

host

host

host

host

host

host

host

host

Switch t Gigabit Swit

Gigabit Switch Swit

Gigabit Switch Swit

Fig. 2.3 Hierarchical network topology

One example of our architectural model is an abstraction of a real-world Grid computing system, Grid’5000 [5, 14]. The goal of Grid’5000 is to build a highly reconfigurable and controllable experimental platform to allow experimental parallel and distributed computing research. The platform consists of nine geographically distributed sites, aggregating a total of 5, 000 CPUs. Each of the nine sites hosts at least one commodity cluster, and approximately among 100 and 1,000 possibly heterogeneous processors. Although the Grid’5000 platform is heterogeneous, it was established as a concerted effort with a goal of avoiding wide heterogeneity of processor performance. This may not be the case for other production grid platforms, in which finding a significant subset of the resources with low heterogeneity might be intractable. By contrast, an almost homogeneous subset of Grid’5000 can be easily identified. This subset comprises 545 processors distributed among six clusters. Although, 545 processors only represent slightly more than 10 percent of the overall platform, they still create a large homogeneous computing platform. For further details we refer the reader to [10].

2.2.2 Application Model With the increasing computing capabilities offered by Grid and application technologies, scientists and engineers are building more and more complex applications to manage and process large data sets, and execute scientific experiments on distributed resources. Such applications differ from typical parallel programs (running partitioned tasks) as they are generally constructed by interweaving interdependent tasks [19, 35]. Referred to as workflow applications, their interdependent tasks are structured on the basis of their control and data dependencies [36]. The workflow application may be represented as a precedence-constrained task graph typically modeled as a Directed Acyclic Graph (DAG) (Figure 2.4).

30

J.E. Pecero et al. n1

3

n2

2

n3

3

3

n7

3

n4

1

3

2

3

n8

1 3

3

n 11

2

3

n5

3

n6

1

3

2

3

n9

1 3

3

n 10

1 3

n 12

1

n 14 3

n 13

1

3

n 15

1

3

3

1

2 3

1

3

n 16

2

Fig. 2.4 A sample of a task graph application. The weight on nodes represents the computation time and the weight on arcs (in this example is constant used to illustrate the proposed approach) denotes the communication costs

The DAG, as a graph theoretic representation of a program, is a directed acyclic graph G = (V, E), where V is a finite set of n nodes (vertices) and E is a finite set of arcs. The node n j ∈ V is associated with one job n j of the modeled program or workflow application. Each arc (ni , n j ) ∈ E (with ni , n j ∈ V ) is a precedence constraint between tasks and is associated with the communication from job ni to job n j . Specifically, the output of job ni must be transmitted to job n j to start its execution. Every job n j , is associated with a value, denoted as p j , that represents its computation time. The computation time of a job ni on a host h j is pi j . However, in this chapter we consider homogeneous computing systems. The weight on an edge, represents the communication time between two jobs ni and n j . Let ≺ be the partial order of the jobs in G, ≺ its inverse relation and ∼ the equivalence relation defined as follows: for x and y ∈ V , x ∼ y if and only if x ≺ y and y ≺ x. Two jobs x and y such that x ∼ y are said independent. The computational model we adopt in this work is the hierarchical communication model [4]. In this model, it takes a time ckl to transfer data from job ni to job n j , ∀(ni , n j ) ∈ E among two hosts hk and hl on the same site si . However, the communication cost is only required when two tasks are assigned to different hosts. Therefore, the communication cost when two tasks are assigned to the same host can be ignored, i.e., it will be zero. The inter-communication cost among sites is Ci j and is more significant than the intra-communication cost. We consider the same model as for the intra-communication scheme. That is, if two jobs ni and n j that communicate are allocated on different sites si and s j , it takes Ci j units of time to transfer the data from si to s j , while Ci j = 0 if the two communicating jobs are executed on the same site si .

2


31

Applications can be classified based on the communication to computation ratio (CCR), which is a measure that indicates whether a workflow application is communication-intensive, computation-intensive or moderate. For a given application, it is computed by the average communication cost divided by the average computation cost on a target system. A very high CCR means that the workflow application is considered as a communication-intensive.

2.2.3 Scheduling Model In this chapter addressed the problem of scheduling a set of interdependent jobs, comprising a workflow application, onto a set of homogeneous hosts dispersed across multiple sites in a Grid. The primary goal of this scheduling is to make as many appropriate job-host matches as possible so that the , also called the schedule length, of a workflow application can be minimized. More formally, in terms of classical scheduling problems, the workflow application has to be scheduled on R, a set of hH sites (i.e., the different clusters of computing resources and available hosts). Those sites are organized in H groups of size h. The communication cost between two hosts in the same site is ckl , as previously mentioned, while it takes Ci j units of time if the two hosts belong to different sites. The i-th site is denoted as Hi• whereas the k-th host of the i-th site is denoted Hik ∈ Hi• . A solution to the problem of scheduling a workflow application on a Grid system is to determine the site and the host assignment and an execution date for each job of the application respecting scheduling constraints. This solution also has to take into account communications and the completion time or makespan (denote as Cmax ), i.e., the amount of time taken from the time the first job starts running to the time last job completes its execution. In the 3-fields notation this problem is denoted by RH (Hh )|prec, pi j ,C = (Ci j , ckl )|Cmax [3]. In terms of complexity, the problem RH (Hh )|prec, pi = 1,C = (Ci j , 0)|Cmax is NP-complete as it contains NP-complete problems as particular cases. For instance, with h = 1, Ci j = 1 and ckl = 0, RH |prec, p j = 1,Ci j = 1|Cmax is already NPcomplete [4].

2.3 Resource Management System and Grid Scheduling 2.3.1 Resource Management System As previously mentioned, a Grid is a hierarchical collection of resources which can themselves be clusters with their own local scheduler. The local resource schedulers, are responsible for scheduling and managing resources at a single site, or possibly a single cluster or resource. One of the main differences between a Grid scheduler and a local resource scheduler is that a Grid scheduler does not own the resources at a site and therefore does not have control over them. To communicate with these local schedulers, the grid scheduler can exploit resource management system services.

32

J.E. Pecero et al.

The architectural model of resource management system is influenced by the way the scheduler is structured. The structure of a scheduler depends on the number of resources on which jobs are scheduled, and the domain in which resources are located. In general, we can distinguish three different models for structuring schedulers: centralized, decentralized, or hierarchical [9, 18, 31, 35]. The centralized model can be used for managing single or multiple resources located either in a single or multiple domains. It can only support uniform policy and suits well for cluster management (or batch queuing) systems. On the contrary, its limited scalability makes it unsuitable for grid environments which might expand over time and must promote local policies imposed by resource owners. Unlike centralized model, both decentralized and hierarchical models allow tasks to be scheduled by multiple schedulers. Therefore, a scheduler only maintains information related to a sub-workflow. Compared to centralized scheduling, both models are more scalable since they limit the number of tasks managed by one scheduler. In a decentralized model, schedulers interact among themselves to decide which resource should be allocated to the jobs being executed. To take its scheduling decisions, each grid scheduler independently gathers resource information from a resource information service. In this model, there is no central leader responsible for scheduling, hence this model appears to be fault-tolerant and highly scalable. As resource owners can define the policy that schedulers can enforce, the decentralized scheme suits grid computing systems. However, as the status of remote jobs and resource is not available at single location, the generation of highly optimal schedule is questionable. Finally, the hierarchical scheduler is a hybrid model. At the highest level, this model can be seen as centralized. However, it also contains the essential properties of the decentralized model: fault-tolerance and scalability. This model also fits for the Grid environments as it allows remote resource owners to enforce their own local policy on external users, and removes single centralization points. The scheduler at the top of the hierarchy is called super-scheduler/resource broker that interacts with local schedulers to decide schedules.

2.3.2 Workflow Scheduling on the Grid: A Brief Taxonomy Different workflow management approaches exist in the context of Grid computing [35], which can be classified in five different elements: workflow design, information retrieval, workflow scheduling, fault tolerance and data movement. The following will only focus on workflow scheduling (interested readers may refer to [35] and [36] for a detailed description of other elements). As mentioned in the previous section, schedulers can be either local or global. Workflow scheduling belong to the global job scheduling class, as it focuses on mapping and managing the execution of inter-dependent tasks on shared resources that are not directly under its control [35, 36]. To date, two major types of workflow

2


33

Grid Workflow Scheduling

QoS-constraint based scheduling

Best-effort based scheduling

MetaHeuristics

Heuristics based

Listbased

Batch mode

Clusterbased

Dependency Dependency Critical Path batch based mode

Decomposition based

Duplicationbased

Evolutionarybased

Ant Simulated Tabu GRASP Annealing Search Colony

Deadlineconstrained

Heuristic Based

MetaHeuristics

Budgetconstrained

Heuristic Based

MetaHeuristics

Genetic Memetic Cellular Algorithm Algorithm Genetic

Fig. 2.5 A taxonomy of workflow scheduling algorithms in the Grid

scheduling exist [36], best-effort based and QoS constraint based scheduling (see Figure 2.5). Best-effort based scheduling attempts to minimize the execution time. On the other hand, QoS constraint based scheduling attempts to minimize performance under most important QoS constraints , for example cost minimization under deadline constraints or time minimization under budget constraints. In this chapter we concentrate on global scheduler and best-effort scheduling. Many best-effort approaches have been proposed, developed and implemented to schedule workflow applications in Grid systems. So far, the most widely used approaches are heuristic based and meta-heuristic based scheduling approaches. The main scheduling heuristics for workflow applications are list scheduling, clustering based, and duplication based. List scheduling heuristics are two phases algorithms, most commonly using task prioritization and processor selection. Different list scheduling heuristics use different attributes and strategies to decide the task priorities and the optimal resource for each task. Workflow-based list scheduling algorithms can be classified as either batch mode, dependency mode or dependency-batch mode. Batch mode scheduling groups workflow tasks into several independent ones and only considers tasks in the current group. The dependency mode ranks workflow tasks based on their weight value and the rank value of its inter-dependent tasks, while the dependencybatch mode further uses a batch mode algorithm to re-rank the independent tasks with similar rank values. List scheduling algorithms for workflow applications have been proposed for homogeneous (Earliest Task First (ETF) [17], Insertion Scheduling Heuristic (ISH) [11], Fast Load Balancing (FLB) [27], etc.) and heterogeneous (Heterogeneous Earliest Finish Time (HEFT) [32], Critical Path on a Processor (CPOP) [32], Fast Critical Path FCP [28], etc.) computing systems.

34

J.E. Pecero et al.

The ETF algorithm was developed for scheduling tasks on multiprocessor architectures with an arbitrary topology. The core of the ETF heuristic is the simple greedy strategy: schedule the earliest possible task first. The interesting feature of the algorithm is that it makes a scheduling decision based on the delay associated with message transfer between any pair of nodes in the network and the amount of data to be transferred between any pair of jobs ni and n j such that ni is an immediate predecessor of n j . Unlike other list scheduling heuristics, the set of ready or available tasks is not prioritized in a queue and scheduled on the basis of this ordering. Instead, at each scheduling step (triggered by a processor becoming available), the ETF algorithm aims at finding the job ni in the set of ready jobs which has the earliest start time (min (EST )), and assigns it to the available processor yielding this start time. For ties, where two tasks have the same earliest start time, some form of task priority is used to resolve the conflict. The authors in [4] adapted the ETF algorithm to consider hierarchical computing systems. These computing systems are a collection of clusters with hierarchical communications. The set of applications to be scheduled were workflow-based applications. One of the main assumptions is that intra-communications or communications in a local cluster are neglected. In this chapter, we further extend this algorithm to consider local communications and we use this algorithm for the experimental validations. We called the algorithm Hierarchical Earliest Task First (HETF). We now turn to the clustering algorithms. The scheduling under the model of precedence constraint is a trade-off between minimizing communication costs and maximizing the concurrency of the task execution. A natural idea is therefore to first determine which nodes should always be executed on the same processor before the actual scheduling. Cluster algorithms exploit this concept by grouping heavily communicating jobs to the same job clusters and by assigning jobs in a cluster to the same resource. This method can be used in the Grid scenarios to reduce the volume of high latency communication. The hierarchical scheduler proposed in this chapter follows the principle of clustering algorithms. In general, clustering algorithms have two phases: merging the nodes into clusters (the clustering step) and mapping of the clusters to the computing resources. The clustering step partitions the original application into group of jobs. The mapping or post-clustering phase can refines the clusters produced in the previous phase and get the final job-to-resource map. There exist mainly two classes of clustering algorithms: the algorithms based on the critical path analysis and the algorithms based on the structured properties of the precedence task graph. The algorithms based on the critical path analysis aim at shortening the longest execution path (taking into account both execution and communication costs) in a precedence task graph. The principle is to group tasks if they belong to the longest path and the relative parallel time is not increased. Another class of clustering algorithm which has been less studied is based on the decomposition of the precedence task graph. The principle is to cluster tasks based on structured properties using recursive decomposition of the application. The third scheduling heuristic class is duplication based. Task duplication is a special scheduling method that duplicates selected tasks to run on more than one

2


35

processor to reduce the inter-processor communication. The approach is conceived based on the fact that some child nodes must wait for the output from the parent nodes running on other processors. If the processors remain idle at different time slots, the spare time can be used to run the duplicated tasks of these parent nodes. Therefore, the parent and child nodes can be executed on the same processor and the output can be locally fed from parent to child without inter-processor communication. Some critical tasks can even have multiple instances running on more processors to further reduce the communication cost. Node duplication introduces more computational load into the computing system to decrease the cost of crucial communication. The challenge in scheduling with duplication based algorithms is the decision as to which nodes should be duplicated and where. Finally, metaheuristics are general algorithmic framework for finding solutions to optimization problems. Within this framework underlying local heuristics are guided and adapted to explore effectively the solution space. Metaheuristics are designed to be robust. They are generally applied to large-size and hard problem, and provide an efficient way of moving quickly toward a very good solution. Different metaheuristics have been applied for solving workflow scheduling problems; these metaheuristics include Genetic Algorithms, Simulated Annealing, Ant Colony Optimization, Greedy Randomized Adaptive Search Procedure (GRASP), Tabu Search [36].

2.4 Proposed Approach: The Hierarchical Scheduler with Cooperative Local Schedulers In this section, a description of the proposed workflow scheduling algorithm is given. Let us mention that different objectives should be considered at same time when scheduling on the Grid. However, in this chapter we only concentrate on the maximization of performance. Nevertheless, we consider that the proposed approach could be adapted to consider different objectives, as for example makespan and resource usage. The resource management and scheduler approach developed in this chapter can be classified as a hierarchical model. It includes a central manager (the superscheduler) and multiple lower-level sub-workflow schedulers . This central manager is responsible for controlling the workflow execution and assigning the subworkflows to the low-level schedulers. The major advantage of using the hierarchical architecture is that different scheduling policies can be deployed in the central manager and in lower-level schedulers. Although Grids have heterogeneous and dynamic resources, these features are not uniformly distributed across its topological structure, but are rather distributed hierarchically and locally. This is induced by the cluster-based composition of the Grid, as resources inside clusters have relatively homogeneous features and low communication costs. These distribution properties might bring out another possible way for new algorithms to deal with the Grid challenges. Our approach takes advantage of these Grid characteristics, with at the higher level, a Grid scheduler

36

J.E. Pecero et al.

which first finds a coarse scheduling in the global Grid and at a lower level a finer local scheduling in the different local clusters is achieved. This strategy has the following advantages: 1) at the higher level, where fine resources information is harder to obtain, global scheduling can use coarse information (such as communication delay of WAN links) to provide load balancing mechanisms. 2) At the lower level, it is "easy" for local scheduling to use more specific and accurate information (such as information from the local forecaster) to take better decisions [7]. Based on the above observations, a two-phase scheduling approach for workflows Grids is proposed. In the first phase, on the global Grid level, a Grid scheduler runs an algorithm called RCCA (Recursive Convex Clustering Algorithm) . In the second phase, the generated subgraphs are dispatched to clusters of computing resources, each of them running a multi-agent parallel algorithm for GPU architectures which optimizes the schedule of the jobs assigned in its own cluster. The next two sections proposed a detailed description of both scheduling phases.

2.4.1 Recursive Convex Clustering Algorithm The RCCA algorithm is based on the decomposition of the workflow graph into convex groups of tasks [6, 20, 25, 26, 29], in our case convex clusters. Informally, a cluster is convex if and only if any directed path between the jobs in this cluster is fully included in this cluster. Figure 2.6 a) depicts a convex clustering and all the clusters are convex. On the contrary, the clustering shown in Figure 2.6 b) is nonconvex since the path from job n1 to n3 through job n4 is not contained within the cluster. We use an algorithm with low complexity to generate the convex clustering. The algorithm is based on a recursive decomposition of the application. The principle is to determine two independent convex groups of tasks as large as possible that can be executed simultaneously without communications. If such decomposition leads to a clustering which longest path length is smaller than the sequential time, then this decomposition is applied recursively on each set. One of the main characteristics of the recursive approach is that the final clustering and each generated subgraph after the recursive decomposition remain directed acyclic graphs. The following provides a formal description of the algorithm, to this end some notations for convex groups are given. Let T be a convex group of tasks: T > : set of successors of tasks in T that are not in T, formally {x ∈ (V \T )|∃y ∈ T, y ≺ x}. T < : set of ancestors that are not in T, formally {x ∈ (V \T )|∃y ∈ T, x ≺ y}. T˜ : set of tasks that are independent from T, formally V \(T ∪ T < ∪ T > ).

2


n1

2

n2

2

n4

2

a)

n3

2

n1

2

n2

2

n4

2

n3

37

2

b)

Fig. 2.6 a) The clustering of the graph is convex. b) The clustering is non-convex

The granularity of a task graph is an important parameter that should be taken into account when analyzing the performance of an algorithm. Informally, the grain measures the ratio between communications and processing times. A precedence task graph is said coarse grain if all the communication costs are smaller than the maximum processing times of the tasks. In the opposite case, the graph is fine grain. If the measured grain is large, then parallelization is limited; whereas if a grain is too small, communication delays reduce the performance. Small grains are obtained by maximizing parallelism and large grains are constructed by combining small grains together on a single cluster. First, RCCA looks for coarse grain parallelism. The algorithm determines two independent convex groups of tasks, namely A and A˜ such that the smallest group is as large as possible (both groups A and A˜ can be executed simultaneously without communications). If such a decomposition leads to a clustering which critical path length ωRcluster is smaller than the sequential execution time, then RCCA applies recursively the same scheme in order to find a smaller grain parallelism (fine grain). The problem of how to determine the two independent convex groups of tasks A and A˜ is central in the RCCA algorithm. This problem, called DAG Partitioning Problem (DAG-PP in short), will be presented later. The RCCA algorithm is outlined in Algorithm 3. In Alg. 3, the Gather(G = (V, E)) function returns the clustering we are looking for. The first step returns a solution of the DAG-PP problem. The Clustering Condition is the criterion that indicates when a decomposition to the groups of tasks can be applied. The first condition that could be considered is at each iteration of RCCA to compute the schedule length of the clustered graph, if the next iteration leads to increase the schedule length, then stop the recursive decomposition and return the clustering and the schedule computed in the previous iteration. However, we consider another criterion that is, we have decided to apply the recursive decomposition

38

J.E. Pecero et al.

Algorithm 3. Pseudo-code of the Recursive Convex Clustering Algorithm 1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11:

Input G ˜ A> and A< Partition G into A, A, if Clustering Condition then RA =Gather(Induced(G, A)) ˜ RA˜ =Gather(Induced(G, A)) RA> =Gather(Induced(G, A> )); RA< =Gather(Induced(G, A< )); return RA< RA RA˜ RA> else return {(V, ≺)} end if

if the number of jobs on each cluster is greater than 50. The Induced(G, A) function returns the induced graph Ginduced . The operator is a fusion operator of distinct R clusterings. ≺ is an arbitrary order compatible with ≺. An illustration of RCCA is given in Figure 2.7. Figure 2.7 left depicts the precedence task graph and the first stage of the clustering where the graph is decomposed into four groups of tasks A< , A, A˜ and A> , the total execution time is equal to 20. At the second stage (Figure 2.7 right) only the graph induced by A˜ is decomposed ˜ B> , A> } which execution time is 19. again. The resulting clustering is {A< , A, B, B, Each resulting group of tasks is then submitted by the global scheduler to the clusters of machines, then the multi-agent for GPU optimizes and schedules each group of tasks locally. Once, agents finish the local schedule optimization, they send back the information to the global scheduler to compute the makespan of the partitioned workflow.

< A

3

n2

~ A

2

n3

0

0

n7

3

n4

1

3

2

0

n8

1 0

0

n 11

n1

3

n5

< A

2

3

3

n6

1

0

1

n9

A

0

n2

2

0

n 10

1 0

n 12

1

n14 3

0 3

n16

1

n 13

n 15

> A

2

2 0

~ A

2

n3

0

0

n7

n4

1

2

0

B

n8

1 0

1

3

> B

0

n 11

1

2

3

n5

3

n6

1

0

2

0

1

n9

A

0

0

n 10

1 0

n 12

1

0

0

1

~ B

0

3

n1

n 14 3

1

n 13

0

n 15

> A

1

0

0

1

2 0

1

3

n 16

2

Fig. 2.7 This figure describes the proposed recursive decomposition, the core of the hierarchical scheduler

2


39

2.4.2 DAG Partitioning Problem As mentioned in 2.4.1, determining the two independent groups of tasks A and A˜ is a central problem in RCCA (it corresponds to the first step of Alg. 3). Formally, DAG-PP problem can be expressed as: Instance: a precedence task graph G = (V, E). Solution: two subsets A and A˜ such that there are no nodes x ∈ A and y ∈ A˜ where / x≤G y or y≤G x. In particular, it implies that A ∩ A˜ = 0. ˜ = f (μ (A), μ (A)). ˜ Optimization: maximize min ( μ (A), μ (A)) DAG-PP is quite different from classical graph partitioning problems where the objective is to split the graph into balanced groups to minimize the number of arcs between groups. The complexity of the DAG-PP problem has been shown to be NPhard by a reduction from the balanced complete bipartite subgraph problem [13]. This justifies the application of heuristic or meta-heuristic approaches to solve the problem. In this work, we use a heuristic based on the following lemma: Lemma 2.1. Given two independent tasks x and y, both groups of tasks {x> } \ {y>} and {y> }\{x>} are independent. Proof. The proof is by contradiction. Consider two tasks z1 ∈ {x> } \ {y> } and z2 ∈ {y> } \ {x>}. There exists a partial order z1 ≺ z2 in the precedence task graph G. By definition of z1 , there exists a path between x and z1 , that is, x ≺ z1 , thus x ≺ z2 . This is impossible by definition of z2 because z2 ∈ / {x> }. By symmetry we can deduce the result. We can choose the tasks x and y randomly. Intuitively, this idea may be justified by the observation that if only a few tasks are independent, then the graph is close to a chain. Once x has been chosen, if x˜ is not empty, then we select one task y in this set. This process is repeated K times and the best result is kept. K is a constant that will be fixed experimentally. The complexity is in O(K.e), where e denotes the number of edges. If we assume that this algorithm leads to almost balanced partitions, then the complexity of a derived clustering algorithm is in O(K.e.log(n)).

2.4.3 Local Scheduler Section 2.4 presented the hierarchical structure of the algorithm. In this structure, the local scheduler is responsible for scheduling the group of tasks in their allocated clusters. For this purpose, a cellular genetic algorithm (cGA) is proposed. This algorithm is presented in the next paragraph. The local schedules can be searched for independently of each other, in parallel. Given the large number of groups of tasks, this parallelism should be exploited. Moreover, the cGA can also be parallelized. The performance of the cGA and its inherent parallelism makes this algorithm a good choice for the local scheduler. These sources of parallelism motivate an

40

J.E. Pecero et al. 6HOHFWLRQ

5HFRPELQDWLRQ

0XWDWLRQ 5HSODFHPHQW

Fig. 2.8 In cellular GAs, individuals are only allowed to interact with their neighbors.

implementation of the local scheduler on a massively parallel machine, such as a graphical processing unit (GPU). Cellular GAs [1, 22, 34] are structured population algorithms with a high explorative capacity. The individuals composing their population are (usually) arranged in a two dimensional toroidal mesh. This mesh is also called grid. Only neighboring individuals (i.e., the closest ones measured in Manhattan distance ) are allowed to interact during the breeding loop (see Figure 2.8). This way, we introduce a form of isolation in the population that depends on the distance between individuals. Hence, the genetic information of a given individual spreads slowly through the population (since neighborhoods overlap). The genetic information of an individual will need a high number of generations to reach distant individuals, thus avoiding premature convergence of the population. By structuring the population in this way, we achieve a good exploration/exploitation trade-off on the search space. This improves the capacity for the algorithm to solve complex problems [1, 2]. Algorithm 4. Pseudo-code for a synchronous cGA 1: while ! StopCondition() do 2: for all ind in population do 3: neigh ← get_neighborhood(ind); 4: parents ← select(neigh); 5: o f f spring ← recombine(p_comb, parents); 6: mutate(p_mut, o f f spring); 7: evaluate(o f f spring); 8: replace(ind, o f f spring); 9: end for 10: end while

A canonical cGA follows the pseudo-code of Alg. 4. In this cGA, each individual in the grid is evolved (line 2). A generation is the evolution of all individuals of the population. Individuals may only interact with individuals belonging to their neighborhood (line 3), so parents are chosen among the neighbors (line 4) with a given criterion. Recombination and mutation operators are applied to the individuals in lines 5 and 6, with probabilities p_comb and p_mut, respectively. Afterwards, the algorithm computes the fitness value of the new offspring individual (or individuals) (line 7), and replaces the current individual (or individuals) in the population

2


41

Table 2.1 Parameters for the cGAs Parameter Population Neighborhood Selection Recombination Mutation Replacement Stop condition

Value 6×6 NEWS binary tournament two-point crossover flip 1 not worse time 360 sec

(line 8), according to a given replacement policy. This loop is repeated until a termination condition is met (line 1), for example: the total elapsed processing time or a number of generations. For local scheduling within each cluster, each cGA is run with the parameters summarized in table 2.1. The population is a two-dimension grid. The neighborhood of an individual is the classic NEWS (after North, East, West y South), which is the one displayed on figure 2.8. One parent is always the individual evolved, the other is the best of 2 individuals randomly selected in the neighborhood. The recombination is the two point crossover. The recombination probability p_comb = 1. Mutation consists of selecting a task, and changing its assignment to another random machine. The mutation probability p_mut = 1. The offspring replaces the individual evolved if its fitness is less or equal to the individual’s fitness. The fitness function is the time necessary to execute the complete DAG, the lower the better. The group of tasks is being scheduled on a cluster, therefore the communication times initially given are significantly reduced when computing the makespan in the cluster (divided by 10). One cGA schedules a group of task onto a cluster. The cGAs can work in parallel, each on their respective group/cluster. Figure 2.9 presents an overview of the cGAs on the GPU. The host launches the execution of kernel threads to run the cGA. Threads are grouped into blocks, each block evolves one group of task. Within each block, a thread performs an evolutionary step on each individual of the population. There is 6 × 6 threads per block (our population size). The evolutionary steps (the kernels) are: fitness computation, selection, recombination, mutation, and replacement. The host code simply examines the stop condition and launches the kernels. Therefore, each evolutionary step is conducted on all the individuals of all the group of tasks in parallel. At the end of the evolution, the fitness scores are copied to host memory, then the host determines the best solution in each cGA.

2.5 Results We study in this section the behavior of the new proposed algorithm and compare it to HETF for a set of 5 different DAGs representing real-world problems. They are shown in Table 2.2, together with their number of tasks and the time required to execute them in a single machine. We have considered three different instances of

42

J.E. Pecero et al. Host

Device

while not stopcondition() each GPU block schedules a group of tasks on a cluster

select parents

recombine kernel launch mutate

fitness each thread evolves an individual schedule

replace

Fig. 2.9 GPU implementation of the local schedulers (cGAs) Table 2.2 Problem instances studied Problem Number of tasks LU 2079 Laplace 2116 MVA 2116 Cholesky 2144 Gauss 2208

Sequential time 11544.232447 11648.751073 11810.570820 11697.793477 12074.542733

every problem with distinct computation to communication cost ratios (CCR). More precisely, we used CCR = 1, 3, and 5. The value of CCR means the number of times that the computation times are faster than the communication times. Therefore, low CCR are suitable to define the scenario of a local cluster of computers, for instance. On the contrary, high CCR values model the cases of Grid and Cloud computing, in which communication links are slow. We are interested in the second case in this work. We present in tables 2.3 and 2.4 the results found by our proposed algorithm for every problem and the three different communication to computation ratios (CCR). The results of our algorithm are compared versus HETF, taken from the literature, in terms of the makespan and energy consumption of the solutions. The figures in these tables are the average value and its standard deviation after 30 independent runs. The best result for every problem is emphasized in bold font. We also include in the two columns on the right the results of applying the Wilcoxon test [29] to the results of the two algorithms for every problem and CCR. As it can be seen, we obtained significant differences with 95% confidence in all the comparisons. We can see in Table 2.3 that our proposed algorithm clearly outperforms HETF for all the problem instances with CCR > 1. These are the most common cases in Grid and Cloud computing. For the low communication costs, the algorithm from

2


43

Table 2.3 Mean and standard deviation for the makespan found by the two studied algorithms CCR 1 LU 3 5 1 Laplace 3 5 1 MVA 3 5 1 Cholesky 3 5 1 Gauss 3 5

Ours 1.9923 · 103 61.0125 2.3972 · 103 111.4048 3.3110 · 103 11.6485 2.9325 · 103326.6704 2.4294 · 103 259.3163 3.1345 · 103 252.4839 2.3682 · 103 194.4466 2.820 · 103 282.2871 3.0752 · 103 283.6937 2.3025 · 103 72.3232 2.4659 · 103 207.7333 3.5136 · 103 354.5924 3.1312 · 103 263.8590 2.9843 · 103 262.6271 3.2971 · 103 184.5191

Ref 1.4928 · 103 2.3126·10−13 2.8877 · 103 0.0721 4.3867 · 103 8.8943 1.6968 · 103 20.9849 3.3876 · 103 33.8556 5.1992 · 103 5.1992 1.7601 · 103 22.8767 3.5875 · 103 34.848 5.196 · 103 50.7468 1.4467 · 103 2.3126e−13 3.0602 · 103 0.0000 4.5143 · 103 0.0000 1.7143 · 103 54.3271 3.2374 · 103 83.1202 4.8512 · 103 168.9411

(R+ , R− ) (465, 0) (0, 465) (0, 465) (465, 0) (0, 465) (0, 465) (465, 0) (0, 465) (0, 465) (465, 0) (3, 462) (0, 465) (465, 0) (12, 453) (0, 465)

p-value 1.828 · 10−6 1.828 · 10−6 1.828 · 10−6 1.828 · 10−6 1.828 · 10−6 1.828 · 10−6 1.828 · 10−6 1.828 · 10−6 1.828 · 10−6 1.828 · 10−6 2.478 · 10−6 1.828 · 10−6 1.828 · 10−6 6.044 · 10−6 1.828 · 10−6

Table 2.4 Mean and standard deviation for the communications energy consumption by the solutions of the two studied algorithms CCR 1 LU 3 5 1 Laplace 3 5 1 MVA 3 5 1 Cholesky 3 5 1 Gauss 3 5

Ours 2.1129 · 104 1.1515·103 5.3491 · 104 3.9364·103 1.0111 · 105 6.5564·103 6.4347 · 103 367.3544 1.9237 · 104 1.228·103 2.8726 · 104 2.7379·103 6.2752 · 103 465.8009 1.7657 · 104 1.0873·103 3.0362 · 104 1.8756·103 2.7033 · 104 1.9435·103 6.5516 · 104 3.1223·103 1.0527 · 105 9.1922·103 2.4053 · 104 1.1721·103 6.6953 · 104 4.2302·103 1.0429 · 105 1.0564·104

Ref 1.1165 · 105 85.9902 3.0990 · 105 210.5061 5.1551 · 105 794.6926 1.1464 · 105 139.354 3.2312 · 105 130.594 5.3946 · 105 751.5158 1.1687 · 105 53.8055 3.2602 · 105 401.2074 5.3205 · 105 629.5397 1.1437 · 105 104.2723 3.2648 · 105 224.0742 5.4437 · 105 481.3494 1.1873 · 105 163.1744 3.3639 · 105 499.4106 5.6142 · 105 1.0035·103

(R+ , R− ) (0, 465) (0, 465) (0, 465) (0, 465) (0, 465) (0, 465) (0, 465) (0, 465) (0, 465) (0, 465) (0, 465) (0, 465) (0, 465) (0, 465) (0, 465)

p-value 1.828 · 10−6 1.828 · 10−6 1.828 · 10−6 1.828 · 10−6 1.828 · 10−6 1.828 · 10−6 1.828 · 10−6 1.828 · 10−6 1.828 · 10−6 1.828 · 10−6 1.828 · 10−6 1.828 · 10−6 1.828 · 10−6 1.828 · 10−6 1.828 · 10−6

the literature is better for the five problem instances. We can see in the table that when increasing the CCR value the results are more favorable to the proposed algorithm with respect to HETF. We compare in Table 2.4 the algorithms in terms of the energy used in the communications by the different solutions they find. As it can be seen, the new proposed

44

J.E. Pecero et al.

technique is clearly outperforming HETF for all the problems studied and every CCR. In fact, the energy requirements of the solutions provided by our proposed algorithm is one or two orders of magnitude lower for all the problems, with the exception of the Gauss problem with CCR = 5, for which our algorithm is 5 times faster.

2.6 Conclusion In this chapter, we have proposed a novel hierarchical two-step Grid-based scheduling algorithm for accounting communication and computation costs for workflow applications. In the first step, at the highest level of the hierarchy, the superscheduler applies a recursive decomposition algorithm to group jobs which communicate significantly with each other, and assign them to computing resources with a fast network, so as to minimize communication costs. In the second step of the optimization process, the resource-broker dispatches the generated subgraphs to clusters of computing resources. In this phase, the local schedulers are responsible for scheduling the group of jobs on their allocated clusters. Each of the local schedulers executes a multi-agent parallel algorithm for GPU architectures which optimizes the schedule of the jobs assigned on its own cluster accounting for computation costs. We have assessed the performance of the proposed approach on a set of different workflow applications that represent real-world problems. To validate the results we compared the performance of the provided approach against a hierarchical scheduler based on the well-known list scheduling approaches. The experimental results emphasize the interest of the developed algorithm. One important prospect is to extend the proposed approach to the case of heterogeneous computing resources. Moreover, the criterion to stop the recursive decomposition is based on a maximum number of jobs. It would be relevant to explore different stopping conditions of the recursive decomposition based on the granularity of the application. Furthermore, the local schedulers need not use the same algorithm. They could opt for algorithms more suitable to their resources or to the group of jobs to schedule. Finally, the approach seems to be well adapted to the new cloud computing systems where communications are specially costly. It would be interesting to investigate the provided scheduler in this context. Acknowledgements. The authors would like to acknowledge the support of Luxembourg FNR in the framework of GreenIT project (C09/IS/05) and the Luxembourg AFR grant scheme for providing the research grant PDR-08-010.

References 1. Alba, E., Dorronsoro, B.: Cellular Genetic Algorithms. In: Operations Research/Compuer Science Interfaces. Springer, Heidelberg (2008) 2. Alba, E., Tomassini, M.: Parallelism and Evolutionary Algorithms. IEEE Transactions on Evolutionary Computation 6, 443–462 (2002)

2


45

3. Bampis, E., Giroudeau, R., König, J.C.: An approximation algorithm for the precedence constrained scheduling problem with hierarchical communications. Theor. Comput. Sci. 290(3), 1883–1895 (2003) 4. Blachot, F., Huard, G., Pecero, J.E., Saule, E., Trystram, D.: Scheduling instructions on hierarchical machines. In: IEEE IPDPS-PDSEC 2010, USA (2010), doi:10.1109/IPDPSW.2010.5470711 5. Bolze, R., Cappello, F., Caron, E., Daydé, M., Desprez, F., Jeannot, E., Jégou, Y., Lanteri, S., Leduc, J., Melab, N., Mornet, G., Namyst, R., Primet, P., Quetier, B., Richard, O., Talbi, E.-G., Irena, T.: Scheduling on large scale distributed platforms: from models to implementations. Int. J. Found. Comput. Sci. 16(2), 217–237 (2005) 6. Deelman, E., Singh, G., Su, M.-H., Blythe, J., Gil, Y., Kesselman, C., Mehta, G., Vahi, K., Berriman, G.B., Good, J., Laity, A., Jacob, J.C., Katz, D.S.: Pegasus: a Framework for Mapping Complex Scientific Workflows onto Distributed Systems. Scientific Programming Journal 13(3), 219–237 (2005) 7. Dong, F., Akl, S.: An Adaptive Double-layer Workflow Scheduling Approach for Grid Computing. In: Proc. of the High Performance Computing Symposium (HPCS) 2007, Canada (2007) 8. Dorronsoro, B., Bouvry, P., Cañero, J.A., Maciejewski, A.A., Siegel, H.J.: Multiobjective robust static mapping of independent tasks on grids. In: International Conference on Evolutionary Computation (CEC), part of the IEEE World Congress on Computational Intelligence (WCCI), pp. 3389–3396 (2010) 9. Dutot, P.-F., Eyraud, L., Mounié, G., Trystram, D.: Scheduling on large scale distributed platforms: from models to implementations. Int. J. Found. Comput. Sci. 16(2), 217–237 (2005) 10. Dutot, P.-F., N’Takpé, T., Suter, F., Casanova, H.: Scheduling Parallel Task Graphs on (Almost) Homogeneous Multi-cluster Platforms. IEEE Trans on Parallel and Distributed Systems 20(7), 940–952 (2009) 11. El-Rewini, H., Ali, H., Lewis, T.: Task Scheduling in Parallel and Distributed Systems. PTR Prentice Hall, Englewood Cliffs (1994) 12. Garg, S., Buyya, R., Siegel, H.J.: Time and cost trade-off management for scheduling parallel applications on utility grids. Future Generation Computer Systems 26(8), 1344– 1355 (2010) 13. Gauja, B., Huard, G., Pecero, J., Thierry, E., Trystram, D.: Convex Scheduling for Grid Computing. In: WASC 2004 - 1st Workshop on Algorithms for Scheduling and Communication, Bertinoro, Italy (2004) 14. Grid5000 (2009), http://www.grid5000.org 15. Guzek, M., Pecero, J., Dorronsoro, B., Bouvry, P.: A cellular genetic algorithm for scheduling applications and energy-aware communication optimization. In: Workshop on Optimization Issues in Energy Efficient Distributed Systems (OPTIM), part of the International Conference on High Performance Computing & Simulation (HPCS), Caen, France, pp. 241–248 (2010) 16. He, L., Jarvis, S.A., Spooner, D.P., Bacigalupo, D., Tan, G., Nudd, G.R.: Mapping DAG-based applications to multiclusters with background workload. In: IEEE International Symposium on Cluster Computing and the Grid, vol. 2, pp. 855–862 (2005), doi:10.1109/CCGRID.2005.1558651 17. Hwang, J.J., Chow, Y.C., Angers, F.D., Lee, C.Y.: Scheduling precedence graphs in systems with interprocessor communication times. SIAM Journal on Computing 18(2), 244–257 (1989)

46

J.E. Pecero et al.

18. Krauter, K., Buyya, R., Maheswaran, M.: A taxonomy and survey of Grid resource management systems for distributed computing. Int. J. of Software: Practice and Experience 32(2), 135–164 (2002) 19. Lee, Y.C., Subrata, R., Zomaya, A.Y.: On the performance of a dual-objective optimization model for workflow applications on Grid platforms. IEEE Trans on Parallel and Distributed Systems 20(9), 1273–1284 (2009) 20. Lepére, R., Trystram, D.: A new clustering algorithm for scheduling with large communication delays. In: 16th IEEE-ACM annual International Symposium on Parallel and Distributed Processing (IPDPS 2002), USA (2002) 21. Mahjoub, A., Pecero, J.E., Trystram, D.: Scheduling with uncertainties on new computing platforms. Journal Comput Optim Appl. (2010) 22. Manderick, B., Spiessens, P.R.: Fine-grained parallel genetic algorithm. In: Schaffer, J. (ed.) Third International Conference on Genetic Algorithms (ICGA), pp. 428–433. Morgan Kaufmann, San Francisco (1989) 23. Martino, B.D., Dongarra, J., Hoisie, A., Yang, L.T., Zima, H.: Engineering the Grid: Status and Perspective. American Scientific Publishers (2006) 24. Nasri, W., Steffenel, L.A., Trystram, D.: Adaptive approaches for efficient parallel algorithms on cluster-based systems. International Journal of Grid and Utility Computing (IJGUC) 1(2), 98–108 (2009) 25. Pecero, J.E., Bouvry, P.: An improved genetic algorithm for efficient scheduling on distributed memory parallel systems. In: IEEE/ACS International Conference on Computer Systems and Applications, AICCSA 2010 (2010), doi:10.1109/AICCSA.2010.5587030 26. Pecero, J.E., Trystram, D., Zomaya, A.Y.: A new genetic algorithm for scheduling for large communication delays. In: Sips, H., Epema, D., Lin, H.-X. (eds.) Euro-Par 2009. LNCS, vol. 5704, pp. 241–252. Springer, Heidelberg (2009) 27. Radulescu, A., van Gemund, A.J.C.: Flb: Fast load balancing for distributed-memory machines. In: Proc. Int. Conf. on Parallel Processing (1999) 28. Radulescu, A., van Gemund, A.J.C.: Fast and effective task scheduling in heterogeneous systems. In: Proc. 9th Heterogeneous Computing Workshop, HCW (2000) 29. Sánchez, J.E.P., Trystram, D.: A new genetic convex clustering algorithm for parallel time minimization with large communication delays. In: Joubert, G.R., Nagel, W.E., Peters, F.J., Plata, O., Tirado, P., Zapata, E. (eds.) Parallel Computing: Current & Future Issues of High-End Computing, vol. 33, pp. 709–716. John von Newmann (2006) 30. Shivle, S., Siegel, H.J., Maciejewski, A.A., Sugavanam, P., Banka, T., Castain, R., Chindam, K., Dussinger, S., Pichumani, P., Satyasekaran, P., Saylor, W., Sendek, D., Sousa, J., Sridharan, J., Velazco, J.: Static allocation of resources to communicating subtasks in a heterogeneous ad hoc grid environment. Journal of Parallel and Distributed Computing, Special Issue on Algorithms for Wireless and Ad-hoc Networks 66(4), 600–611 (2006) 31. Tchernykh, A., Schwiegelson, U., Yahyapour, R., Kuzjurin, N.: On-line hierarchical job scheduling on grids with admisible allocation. J. Sched. 13(5), 545–552 (2010) 32. Topcuoglu, H., Hariri, S., Wu, M.: Performance-Effective and Low- Complexity Task Scheduling for Heterogeneous Computing. IEEE Trans. Parallel and Distributed Systems 13(3), 260–274 (2002) 33. Wilcoxon, F.: Individual comparisons by ranking methods. Biometrics Bulletin 1(6), 80– 83 (1945)

2


47

34. Whitley, D.: Cellular genetic algorithms. In: Forrest, S. (ed.) Fifth International Conference on Genetic Algorithms (ICGA), p. 658. Morgan Kaufmann, California (1993) 35. Yu, J., Buyya, R.: A taxonomy of workflow management systems for grid computing. Journal of Grid Computing 3(3-4), 171–200 (2006), doi:10.1007/s10723-005-9010-8 36. Yu, J., Buyya, R., Ramamohanarao, K.: Workflow scheduling algorithms for Grid computing. In: Xhafa, F., Abraham, A. (eds.) Metaheuristics for Scheduling in Distributed Computing Environments. Studies in Computational Intelligence, vol. 146, pp. 173–214. Springer, Heidelberg (2008) 37. Zomaya, A.Y., Chan, G.: Efficient clustering for parallel tasks execution in distributed systems. In: Proc. of Workshop NIDISC 2004, New Mexico, USA, pp. 167–177 (2004)

Chapter 3

Multi-objective Cooperative Coevolutionary Evolutionary Algorithms for Continuous and Combinatorial Optimization Bernabé Dorronsoro, Grégoire Danoy, Pascal Bouvry, and Antonio J. Nebro

Abstract. This chapter introduces three new multi-objective cooperative coevolutionary variants of three state-of-the-art multi-objective evolutionary algorithms, namely, Nondominated Sorting Genetic Algorithm II (NSGA-II), Strength Pareto Evolutionary Algorithm 2 (SPEA2) and Multi-objective Cellular Genetic Algorithm (MOCell). In such a coevolutionary architecture, the population is split into several subpopulations, each of them being in charge of optimizing a subset of the global solution by using the original multi-objective algorithm. Evaluation of complete solutions is achieved through cooperation, i.e., all subpopulations share a subset of their current partial solutions. Our purpose is to study the performance of the coevolutionary multi-objective approaches in two different contexts: continuous and combinatorial optimization. In the first case we deal with a set of well-known benchmark problems, while in the the second one we conduct a study about a real-world problem related to grid computing, the bi-objective robust scheduling problem of independent tasks. The goal in this problem is to minimize makespan (i.e., the time when the latest machine finishes its assigned tasks) and to maximize the robustness of the schedule (i.e., its tolerance to unexpected changes on the estimated time to complete the tasks). We have made a parallel, multithreaded implementation of the coevolutionary algorithms and we have analyzed the results obtained in terms of both the quality of the Pareto front approximations yielded by the techniques as well as the resulting speedups when running them on a multicore machine. Bernabé Dorronsoro Interdisciplinary Centre for Security, Reliability, and Trust, University of Luxembourg e-mail: [email protected] Grégoire Danoy · Pascal Bouvry Faculty of Sciences, Technology and Communication, University of Luxembourg e-mail: {gregoire.danoy,pascal.bouvry}@uni.lu Antonio J. Nebro E.T.S.I. Informática, University of Málaga, Spain e-mail: [email protected]


50

B. Dorronsoro et al.

3.1 Introduction Multi-objective optimization is a discipline that deals with the simultaneous optimization of a number of functions or objectives which usually are in conflict. This means that improving one of the functions leads to decreasing the quality of (some of) the others. Therefore, solving a multi-objective problem involves the search for a set of optimal solutions, known as Pareto optimal set , instead of a unique solution as in single-objective optimization. The correspondence of this set in the objective space is known as the Pareto front . All the solutions in the Pareto front are non-dominated, meaning that none is better than the others for all the objectives (a solution dominates another one if the former is better than the latter for all the considered objectives). In practice, a multi-objective algorithm is expected to find a limited number of solutions, belonging to the optimal Pareto front (or as close as possible to it), and being uniformly spread throughout it. The books by Coello et al. [6] and Deb [10] are excellent references for multi-objective optimization using evolutionary algorithms (EAs), the most widely used techniques in this field. In order to tackle problems of increasing size where “classical” EAs tend to perform poorly or are difficult to apply, researchers referred again to a nature inspired process so as to extend EAs, i.e., coevolution. In such an approach, instead of evolving one homogeneous population of individuals that represents a global solution, several representing specific parts of the global solution either compete or cooperate during the evolution. In this chapter, we focused on the Cooperative Coevolutionary model as introduced by Potter and De Jong [27], where each subpopulation evolves a part of the decision variables using a standard EA and all subpopulations evaluate complete solutions through a cooperative exchange of individuals. Performance improvements brought by such a division of the search space was already demonstrated on various single-objective problems such as [8, 9]. Although Cooperative Coevolutionary Evolutionary Algorithms (CCEAs) were introduced for single-objective optimization , the cooperative coevolutionary paradigm can also be applied to the multi-objective domain. The main contribution of this work is the design of three new CCEAs for multiobjective optimization (CCMOEA) based on three multi-objective EAs which are representative of the state-of-the-art, namely NSGA-II [12], MOCell [23], and SPEA2 [32]. The new CCMOEA will be compared versus the original techniques on a set of well known continuous optimization benchmarks as well as on a real world combinatorial problem. Additionally, our proposed CCMOEAs are parallelized, and their performance is analyzed also in terms of the speedup. The chapter structure is detailed next. The following section contains a brief survey on the main existing CCMOEAs. After that, Sect. 3.3 and Sect. 3.4 present the reference state-of-the-art multi-objective algorithms considered in this paper and their cooperative coevolutionary design, respectively. The continuous and combinatorial optimization problems selected for comparing the performance of all the algorithms are described in Sect. 3.5. The results of all the considered algorithms, as well as their configurations, are summarized and discussed in Sect. 3.6. Finally, we conclude our work in Sect. 3.7.

3

CCMOEAs for Continuous and Combinatorial Optimization

51

3.2 Related Work The main difference between CCEAs [25] and EAs comes from the adaptive nature of fitness evaluation in coevolutionary systems: the fitness of an individual is based on its interactions with other individuals from other so-called subpopulations. These interactions can be either negative or positive, leading respectively to competitive or cooperative coevolution, this work being focused on the latter. Cooperative coevolution, as defined in the framework proposed by Potter et al. in [27], consists in splitting the solution vector into different subcomponents that are evolved in different subpopulations which periodically exchange their partial solutions. CCEA models have been initially developed for single-objective problems optimization but more recently several multi-objective versions proved to be efficient [6], the most prominent cooperatives ones being listed hereafter. Parmee and Watson [26] proposed a CCMOEA that evolves simultaneously one population per objective function to optimize. Contrary to other MOEAs, this approach aims at finding a unique trade-off solution, which could be used for preliminary design. In [22] Mao et al. proposed an extension of the Genetic Symbiosis Algorithm (GSA) for multi-objective optimization problems. The multi-objective GSA (MOGSA) differs from the standard GSA with a second symbiotic parameters, which represent the interactions of the objective functions. One main drawback of this algorithm is that it requires the knowledge of the search space, which highly reduces its application possibilities. In [4] Barbosa et al. proposed the interactive genetic algorithm with coevolving weighting factors (IGACW). It is based on two populations, one population of solutions and the other of weights, which evolve in a round-robin process and exchange information through a shared fitness function. IGACW is highly dependent on the user’s solution ranking inputs it requires during the search, which makes it hardly comparable to any other CCMOEA. In [17], Keerativuttitumrong et al. proposed the Multiobjective Co-operative Co-evolutionary Genetic Algorithm (MOCCGA), which combines the multiobjective GA (MOGA) and Potter’s Co-operative Co-evolutionary Genetic Algorithm (CCGA). As defined in the CCGA framework, one subpopulation is created per decision variable and requires the best individual from each other subpopulations to complete a solution. Each subpopulation evolves using Fonseca and Fleming’s MOGA and assigns a fitness to its individuals based on their rank in the subpopulation local Pareto front. This local Pareto optimality perception however leads to a limited performance of the MOCCGA. A parallel implementation of MOCCGA was empirically validated using 1, 2, 4 and 8 cores. Maneeratana et al. proposed an extension of the MOCCGA in [20, 21] which used other MOEAs and added a fixed size archive that stores the non-dominated solutions. Another variant was introduced by Tan et al. [28] with their cooperativecoevolution evolutionary algorithm which also adds an archive plus a novel adaptive niching mechanism. Similarly to Keerativuttitumrong, a parallel version of CCEA was experimented. Finally, [16] presented a non-dominated sorting cooperative

52


coevolutionary algorithm (NSCCGA), which is essentially the coevolutionary extension of NSGAII. Lohn et al. [19] proposed the Lohn’s CGA, a model using two populations, one evolving solutions and the other evolving fitness functions (named objective vectors) in a cooperative way. Applied to an amplifier and antenna problem, this coevolutionary model did not provide any advantage as its performance was similar to a standard GA. Coello Coello and Reyes Sierra [7] proposed a Coevolutionary MOEA (COMOEA). It is a competitive CCEA with dynamic populations management. COMOEA provided results similar to state-of-the-art MOEAs on benchmark problems; however, its elitism strategy induces a high selection pressure which could degrade its performance on deceptive problems. REcently Goh et al. proposed a new competitive-cooperative co-evolution model (COEA) [14, 15] in which individuals from different species collaborate to solve the problem and compete at regular intervals. The model performance was assessed on various static benchmark functions and a dynamic variant, dCOEA, using a diversity scheme based on a competitive mechanism, was evaluated on dynamic problems. The CCMOEA framework proposed in this chapter extends the Potter and De Jong’s CCGA approach for multi-objective optimization, as already proposed by Keerativuttitumrong’s MOCCGA and its extensions by Maneeratana and Tan. The main differences of our approach come from: (1) the genericity of the framework which allows building different CCMOEAs, e.g. using NSGAII, SPEA2, or MOCell; (2) its distributed archives management (one per subpopulation) and the final process to generate the global Pareto front from their solutions; (3) its efficient parallelization and scalability possibilities; (4) the solutions evaluations achieved using a set of randomly chosen non-dominated solutions from the other subpopulations; (5) the validation of the algorithms on continuous and combinatorial domains. A description of the different MOEAs used in this work is provided in the next section while the CCMOEA framework is detailed in section 3.4.

3.3 Base Algorithms This section briefly describes the algorithms we have considered for this work (NSGA-II, SPEA2, and MOCell). The Non-dominated Sorting Genetic Algorithm , NSGA-II [12], is undoubtedly the most well-known and referenced algorithm in the multi-objective literature. It is a GA with a panmictic (i.e., non-structured) population, so it is based on obtaining a new population from the original one by applying the typical genetic operators (selection, crossover, and mutation); then, the individuals in the two populations are sorted according to their rank, and the best solutions are chosen to create a new population. In the case of having to select some individuals with the same rank, a density estimation based on measuring the crowding distance to the surrounding individuals belonging to the same rank is used to get the most promising solutions.

3


53

The Strength Pareto Evolutionary Algorithm , SPEA2 [32], works also on a panmictic population as NSGA-II does. In this algorithm, each individual has a fitness value assigned which is the sum of its strength raw fitness and a density estimation. The algorithm applies the selection, crossover, and mutation operators to fill an archive of individuals; then, the non-dominated individuals of both the original population and the archive are copied into a new population. If the number of non-dominated individuals is greater than the population size, a truncation operator based on calculating the distances to the k-th nearest neighbor is used. This way, the individuals having the minimum distance to any other individual are chosen. The Multi-objective Cellular Genetic Algorithm , MOCell [23], is characterized by the use of a decentralized population and an archive to store the non-dominated solutions found during the search. In MOCell individuals are arranged into a two dimensional toroidal grid, and only those individuals that are close to each other in the mesh are allowed to interact. The main advantage of using this population structure, called cellular, is that individuals are isolated by distance, and therefore good solutions will spread slowly through the population mesh, consequently keeping the diversity of solutions for longer [1]. From all the studied versions of MOCell in [23], we use here the so-called aMOCell4, which takes advantage of the non-dominated solutions stored in the archive by choosing one of them as one of the parents during the breeding loop.

3.4 The Proposed Cooperative Coevolutionary Evolutionary Algorithms In this chapter, a new design for cooperative coevolutionary multi-objective evolutionary algorithms is proposed. In order to first describe the cooperative coevolutionary paradigm used, Sect. 3.4.1 briefly introduces a well known cooperative coevolutionary genetic algorithm (CCGA) for single-objective optimization. Then, we explain in Sect. 3.4.2 our new generic design for cooperative coevolutionary multi-objective evolutionary algorithms that will be applied, as an example, in this work to NSGA-II, SPEA2, and MOCell. Later, Sect. 3.4.3 explains the parallel implementation for our CCMOEAs.

3.4.1 Cooperative Coevolutionary Evolutionary Algorithms The first cooperative coevolutionary genetic algorithm was developed for singleobjective optimization by Potter and De Jong in [27]. In this algorithm, the population is split into subpopulations of species. Differently to other types of EAs, each of these species represent a component of the complete solution to the optimization problem (e.g. a single decision variable) and are evolved using independent EAs. Complete solutions are then obtained by assembling representatives from each of the species (populations). Subpopulations therefore have to cooperate as the fitness of each individual depends on the quality of the solutions the individual participated in.

54


Fig. 3.1 In cooperative coevolutionary genetic algorithms, every subpopulation is focussed on the resolution of a subproblem. In this work, every subproblem is a subset of the variables of the whole problem to optimize

One simple implementation of this paradigm is to split the chromosomes of individuals, and then every subpopulation will be in charge of optimizing a subset of the problem variables, as it is shown in Fig. 3.1. At the end of every generation, the subpopulations are publishing their best local solution. For evaluating partial solutions in every subpopulation, we build a joint solution by using the best solutions for every other species and then we apply the regular fitness function. This is the approach we follow for our CCMOEAs implementation. We give a pseudocode of a classical CCEA in Algorithm 5. It starts by generating and evaluating the initial subpopulations. In the initial generation (t=0) individuals from a given subpopulation are matched with randomly chosen individuals from all other subpopulations. The best individual in each subpopulation is then retrieved

Algorithm 5. Pseudocode of a CCEA 1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: 14: 15: 16: 17: 18:

t ← 0 // Generations counter for each i ∈ [1, I] do Pi0 ← initialPopulation() // Initialize the I subpopulations evaluatePopulation( Pi0 ) // Evaluate every initial subpopulation end for // Migrate the best partial solution in every subpopulation to the others ∀ i ∈ [1, I] :: migrateBestSols( Pi0 ) while not StoppingCriterion( ) do t ← t +1 for each i ∈ [1, I] do Qti ← selectParents( Pit−1 ) // Evolve every subpopulation Qti ← variationOperators( Qti ) evaluatePopulation( Qti ) // Evaluate the new individuals Pit ← replacement( Pit−1 , Qti ) migrateBestSols( Pit ) end for end while solution ← assembleBestSols( Pit )

3


55

Solution Construction



Fig. 3.2 In a classical CCEA, populations share their best partial solution bdvX (bdv stands for best decision variables, and X is the population number) with the other coevolving ones. Every population is optimizing its assigned variables (in black), which are evaluated by applying the problem fitness function to the complete solution that is built with the partial solution to evaluate plus the best partial solutions of the other subpopulations, i.e., we use bdv2 and bdv3 to evaluate every partial solution of P1

based on its fitness value. The process of cooperative coevolution starts in the next generation (t=1). For this purpose, in each generation the EA operations (selection, crossover, mutation) follow a round-robin schedule. Only a single subpopulation is “active” in one cycle, while the other subpopulations are “frozen”. All individuals from the active subpopulation are matched with the best values of the frozen subpopulations. When the evolutionary process is completed, the solution of the problem is the composition of the best individuals from each subpopulation. We show in Fig. 3.2 how the information is shared between subpopulations. Every subpopulation Pi shares with the others its best partial solution bdvi . To evaluate a given partial solution s, the subpopulation builds a full solution by using the variable values from s and the best partial solutions of the other subpopulations bdv j , with j ∈ {1, . . . , NP} /i = j, NP being the number of subpopulations.

56


3.4.2 Multi-objective Cooperative Coevolutionary Evolutionary Algorithms We present in this section the design of the new proposed multi-objective CCEAs. For the multi-objective case, we can directly adopt the description presented in Sect. 3.4.1, with very few modifications. This way, every subpopulation is focussed on the optimization of a subset of the problem variables, and a multi-objective EA is run in every subpopulation to evolve the population of solutions to the corresponding subproblem. Every subpopulation will locally look for its own approximation to the Pareto front, and at the end of the run, the obtained solution sets of every subpopulation are merged into a single one that will be the output of the CCMOEA. The merging process of the solution sets found by all the subpopulations is achieved by choosing one of them and by adding to it all the solutions from the others. In case the resulting approximation set is full, the crowding [12] policy is used to remove solutions, although it could be the same one used to locally build the sets of non-dominated solutions in every subpopulation. The choice of crowding has been motivated by the fact that it demonstrated good performance on two dimensional problems [12], as it is the case of the problems studied in this work. Besides the archives management in the subpopulations, another difference of the multi-objective design with respect to the single-objective one is the way in which complete solutions are built for evaluation. As previously mentioned, in the case of single-objective optimization, the evaluation of a partial solution in a subpopulation is achieved by composing a complete solution with the best partial solutions from all the other subpopulations. In the case of multi-objective optimization, in most cases there will be more than a single best solution, i.e., a set of non-dominated ones. One could think on solving the problem by randomly choosing one of the nondominated solutions in every subpopulation; however, this would highly restrict the search performed by the algorithm, resulting in solution sets with poor accuracy and diversity. In the proposed CCMOEA, every subpopulation shares a number Ns of solutions randomly chosen from the non-dominated ones found so far. An example of how one subpopulation, P1 , shares its best solutions with the others is shown in Fig. 3.3, where Ns = 4. If the local Pareto front contains less than Ns non-dominated solutions, the set of Ns solutions is completed by other individuals randomly taken from the rest of the population. Another consequence of sharing multiple partial solutions is the number of possible complete solutions combinations. Indeed, for a given subpopulation, Ns partial solutions are received from the other subpopulations, therefore, there are NsNP−1 possible combinations, where NP is the number of subpopulations. Building and evaluating all the possible solutions would be extremely costly. In the proposed CCMOEA the solution is built by using s and the partial solution obtained by considering combinations of bdvi j with the same j index (i is the identifier of the population the solution belongs to, and j is the index of the solution in the shared list with

3


57



R A N D

Solution Construction R A N D

Fig. 3.3 In our CCMOEA, every population (for example, P1) shares with the other coevolving populations (P2 and P3 ) its four best partial solutions (bdv11 to bdv14 ). The partial solutions are evaluated by building complete solutions with the partial solutions of the other two subpopulations (bdv2X and bdv3Y , where X = Y )

the Ns partial solutions). An example is provided in Fig. 3.3 on how population P1 builds its complete solutions with bdv2X and bdv3X , ∀X ∈ {1, 2, 3, 4}. With the few adaptations mentioned hereinbefore, we can define a canonical generic framework for cooperative coevolutionary multi-objective optimization that can be applied to any multi-objective EA, as it will be detailed next.

3.4.3 Parallel Multi-objective Cooperative Coevolutionary Evolutionary Algorithms Similarly to single-objective CCEAs, the CCMOEA framework we propose is easily parallelizable since it is composed of several subpopulations evolved by independent MOEAs. In order to improve the computational performances of our CCMOEA, we designed a parallel implementation for multicore architectures. One thread is created per subpopulation, and a few synchronization points are kept in order to reproduce exactly the same behavior as the sequential algorithm. Asynchronous communications could also be implemented, and this is a matter of future research.

58


Algorithm 6. Parallel CCMOEA framework 1: t ← 1 2: // means parallel run 3: i ∈ [1, I] :: setup( P0 , i ) // Initialize every subpopulation 4: // ∀ means sequential run 5: ∀ i ∈ [1, I] :: broadcast( P0 , i ) // Share random local partial solutions in every subpopulation 6: i ∈ [1, I] :: evaluate( P0 , i ) // Evaluate solutions in every subpopulation 7: while not stoppingCondition( ) do 8: i ∈ [1, I] :: generation( Pt , i ) // Perform one generation to evolve the population ∀ i ∈ [1, I] :: broadcast( Pt , i ) // Share best local partial solutions in every subpopulation 10: t ← t + 1 // Increase generations counter 11: end while 12: mergeParetoFronts( ) // Merge the Pareto fronts found in the subpopulation into a single one 9:

A pseudocode of the proposed parallel implementation is shown in Algorithm 6. The different subpopulations are initialized in line 3. This initialization process creates new subpopulations of random partial solutions. As in the single-objective CCEA presented in Algorithm 5, each subpopulation then shares Ns randomly selected partial solutions (line 5). Once this is done, every subpopulation can evaluate its initial population members in parallel, as it is shown in line 6. The algorithm enters now in its main loop (the cooperative loop), from which it will not exit until the termination criterion is met. In every iteration of the loop, the subpopulations perform one generation of the MOEA in parallel (line 8), and then they synchronize to publish their best local partial solutions (line 9). At the end, we build a single Pareto front from the solutions in the local Pareto fronts in every subpopulation (line 12). This final Pareto front will be the output of the algorithm. We show in Fig. 3.4 the blocks diagram of the designed parallel CCMOEA framework (NP is the number of subpopulations). White circles are the synchronization points, black ones are bifurcation states (where several parallel threads are created and run). Arrows indicate the execution sequence order, and the black square is the end state. As it can be seen, the processes run in parallel for every subpopulation are: initialization, evaluation of the new solutions, and performing a new generation to evolve the solutions. Two synchronization points are required, before publishing their best partial solution(s) after a generation (1 and 3 in Fig. 3.4), and when checking the stopping criterion after the evaluation phase (2). These two processes are sequentially executed.

3


A1.setup(1)

A2.setup(2)

59

ANP.setup(NP)

1

broadcast(A1) broadcast(A2) broadcast(ANP)

A1.evaluate()

A2.evaluate()

ANP.evaluate()

2 YES A.stopCondition() NO

A1.generation()

A2.generation()

ANP.generation()

3 broadcast(A1) broadcast(A2) broadcast(ANP)

MergeParetoFronts()

Fig. 3.4 Design of the proposed parallel cooperative coevolutionary multi-objective genetic algorithm. White circles are synchronization points, while black ones are the states in which the parallel threads are run

60


3.5 Problems This section provides a description of the problems we have considered to evaluate our new CCMOEAs. In Sect. 3.5.1 we present two benchmarks of continuous problems that are well known in the literature. Then, Sect. 3.5.2 describes a real-world combinatorial problem we have chosen to complete the study.

3.5.1 Continuous Problems To evaluate our algorithms, we selected the Deb, Thiele, Laumanns and Zitzler (DTLZ) [13], and the Zitzler, Deb and Thiele (ZDT) [31] problem families, commonly used in the literature (mathematical definitions of the problems can be found in the related references). They are composed of 12 difficult academic problems with many different features in terms of search space, i.e. multimodality, deceptiveness, or epistasis, and Pareto front, e.g., convexity, uniformity, or continuity. For each of these benchmark functions, the number of problem variables can be specified. The values used for the following experiments are provided in Table 3.1. The number of objectives for DTLZ is also parameterizable, we used in this work the bi-objective formulation. The optimal Pareto front is known for all these continuous functions which will permit a precise evaluation of the performance of the proposed CCMOEAs. Table 3.1 Size of the studied continuous optimization problems Problem Number of variables DTLZ1 8 DTLZ2 to DTLZ6 12 DTLZ7 24 ZDT1, to ZDT3 32 ZDT4 and ZDT6 12

3.5.2 Real-World Combinatorial Problem This section describes the real-world combinatorial problem we tackled using our CCMOEAs, namely the robust static mapping problem of independent tasks on Grids (which we call RSMP). In this scheduling problem, we consider a set of n independent tasks T = {t1 ,t2 , . . . ,tn } that must be scheduled onto the set of k heterogeneous machines M = {m1 , m2 , . . . , mk }. We assume that an estimate of the time to compute task ti in every machine m j , ETCi, j , is known. Then, the RSMP problem is to allocate all of the tasks on the different machines in a way that the makespan of the schedule is minimized, while at the same time its robustness is maximized. These two objectives are defined below.

3


61

• Makespan is defined as the maximum completion time of all the resources used in the schedule. The completion time of a machine m j in schedule S is defined as: Fj (C) = ready j + ∑ Ct, j , (3.1) t∈S( j)

where ready j is the time when machine m j will be available (we consider ready j = 0 for every machine in this work), S( j) is the set of tasks assigned to m j , and C is a matrix with the actual times to compute the tasks in every machine (C = ETC when we are not considering errors in the estimation of the duration of tasks). Our makespan function is defined as: fM (x) = {max{Fj (C)} ,

(3.2)

where x represents an allocation. • The robustness metric we are using in this work was originally presented in [2]. It is defined as the minimum of the robustness radii of every machine used in the schedule. The robustness radius for a given machine m j is the smallest collective increase, based on the euclidean distance, in the error that would cost the finishing time of that machine to be τ times than original, and we calculate it with Eq. (3.3) taken from [2] —where M orig is the makespan of the schedule with the estimated ETC values and Fj (Corig ) is the estimated time to compute the assigned tasks to machine m j . Our robustness metric is the worst machine in terms of that value, as defined in Eq. (3.4). Specifically, by using this robustness metric, we ensure that if the collective difference of the actual task execution times versus the estimated times is within a certain calculated range, then the given makespan requirement will be met. rx (Fj ,C) =

τ · Morig − Fj (Corig ) number of applications allocated to m j

fR (x) = {min{rx (Fj ,C)} .

(3.3) (3.4)

Finally, our multi-objective RSMP problem is defined as finding schedules such that the makespan fM is minimized and the robustness fR is maximized – see Eq. (3.5). Notice that the two objectives to optimize are in conflict, because improving the makespan of a solution may lead to a decrease in its robustness, and vice-versa. Therefore our problem must be solved with multi-objective optimization techniques.

minimize fM (x) RSMP(x) = . (3.5) maximize fR (x) We have considered a benchmark of 12 instances with different features. They all consist of 512 tasks and 16 machines. These instances represent different classes

62


of ETC matrices. The classification is based on three parameters: task heterogeneity, machine heterogeneity, and consistency [3]. Instances are labelled as u_x_yyzz where: u x

stands for uniform distribution (used in generating the matrix). stands for the type of consistency (c for consistent, i for inconsistent, and s for semi-consistent). An ETC matrix is considered consistent when the following is true: if a machine mi executes a task t faster than machine m j , then mi executes all tasks faster than m j . Inconsistency means that a machine is faster for some tasks and slower for some others. An ETC matrix is considered semi-consistent if it contains a consistent sub-matrix. yy indicates the heterogeneity of the tasks (hi means high, and lo means low). zz indicates the heterogeneity of the resources (hi means high, and lo means low). We report computational results for the following 12 instances, for which we provide their Blazewicz [5] notation: • • • • • • • • • • • •

u_c_hihi: Q16|26.48 ≤ p j ≤ 2892648.25|Cmax; u_c_hilo: Q16|10.01 ≤ p j ≤ 29316.04|Cmax; u_c_lohi: Q16|12.59 ≤ p j ≤ 99633.62|Cmax; u_c_lolo: Q16|1.44 ≤ p j ≤ 975.30|Cmax; u_i_hihi: R16|75.44 ≤ p j ≤ 2968769.25|Cmax; u_i_hilo: R16|16.00 ≤ p j ≤ 29914.19|Cmax; u_i_lohi: R16|13.21 ≤ p j ≤ 98323.66|Cmax; u_i_lolo: R16|1.03 ≤ p j ≤ 973.09|Cmax; u_s_hihi: R16|185.37 ≤ p j ≤ 2980246.00|Cmax; u_s_hilo: R16|5.63 ≤ p j ≤ 29346.51|Cmax; u_s_lohi: R16|4.02 ≤ p j ≤ 98586.44|Cmax; u_s_lolo: R16|1.69 ≤ p j ≤ 969.27|Cmax.

3.6 Experiments This section contains all the details about our experimentation settings and results. The configuration of the algorithms is detailed in Sect. 3.6.1. The methodology followed to compare them is described in Sect 3.6.2. Finally, our results are summarized in Sect. 3.6.3.

3.6.1 Configuration of Algorithms We present in this section the configuration we used for the different algorithms to solve the continuous and the real-world combinatorial problems. In the two cases, the configuration of the algorithms in the subpopulations of the CCMOEA is exactly the same as that of the corresponding MOEA, with the exception of the length of the individuals. For the MOEA, the individual size is equal to the number of decision variables, whereas for the MOCCEA it is equal to the number of decision variables divided by the number of subpopulations.

3


63

Table 3.2 Configuration of the algorithms for the continuous benchmarks

Number of subpopulations Cores used Number of threads Population size Archive size∗ Final archive size Number of solutions shared Max. evaluations Population initialization Selection Recombination Probability Distribution index Mutation Probability Distribution index Independent runs ∗

NSGA-II, SPEA2 & MOCell CCNSGA-II, CCSPEA2 & CCMOCell — 4 1 4 1 1 per subpopulation 100 100 per subpopulation 100 100 per subpopulation 100 100, chosen among all the subpopulations — 20 100000 Random Binary tournament SBX 0.9 20 Polynomial 1 number o f variables

20 100

Not applicable for NSGA-II and CCNSGA-II.

The parameters of the algorithms for the continuous problems are shown in Table 3.2. The termination condition is set to 100000 of fitness function evaluations, and the initial population is randomly generated. The selection operator used is binary tournament, which randomly selects two solutions from the population and takes the dominating one. In the case both parents are non-dominated, then the operator chooses a random one (for MOCell, SPEA2, and their cooperative coevolutionary versions) or the one with highest crowding distance (for NSGA-II and CCNSGA-II). The variation operators used are the simulated binary crossover (SBX) [11] (with probability 0.9 and distribution index 20) and polynomial mutation [11] (with probability one over the length of the chromosome and distribution index 20). In order to look for concluding results, we perform 100 independent runs of the algorithms. Due to the decentralized nature of the CCMOEAs, some of their parameters are different than those of the compared MOEAs. For example, we have tested our CCMOEAs with 4 subpopulations, which are parallelized using 4 different threads. The compared MOEAs are sequentially running in one single thread. Additionally, every subpopulation shares its 20 best partial solutions with the others. The other parameters of the subpopulations’ algorithm are kept identical to the original algorithm’s. Therefore, both the population and the archive of the MOEAs are composed by 100 solutions, while in the case of the CCMOEAs every subpopulation contains a population and an archive of 100 solutions each. Finally, since we expect the algorithms to provide a final archive of 100 solutions, it is built from the solutions in all the archives in every subpopulation. The parameters used for the combinatorial problems are shown in Table 3.3. The termination condition is set to 500000 of fitness function evaluations, and the

64


Table 3.3 Configuration of the algorithms for the combinatorial problem

Number of subpopulations Cores used Number of threads Population size Archive size∗ Final archive size Number of solutions shared Max. evaluations Population initialization Selection Recombination Probability Mutation Probability Independent runs ∗

NSGA-II, SPEA2 & MOCell CCNSGA-II, CCSPEA2 & CCMOCell — 4 1 4 1 1 per subpopulation 100 100 per subpopulation 100 100 per subpopulation 100 100, chosen among all the subpopulations — 20 500000 Random Binary tournament DPX 0.9 Rebalance 1 number o f variables

100

Not applicable for NSGA-II and CCNSGA-II.

population is randomly generated. The solution representation is a string of integers which length is equal to the number of tasks, and the value assigned to every task is the machine to which it is scheduled. The selection operator used is binary tournament, the crossover operator is two-point crossover (DPX) (with probability 0.9) and the mutation operator is the rebalance operator [30] (with probability pm = 1.0/ChromosomeLength). The rebalance mutation operates by randomly choosing one task from the 25% of the highest completion time machines, and assigning it to one of the 25% of the lowest completion time machines. As for the continuous problems, we perform 100 independent runs of the algorithms and the CCMOEAs with 4 subpopulations that are parallelized using 4 threads. Every subpopulation shares its 20 best partial solutions with the others. The MOEAs are sequentially run in a single thread and the other parameters of the subpopulations’ algorithms are kept identical to the original algorithms’. Finally, the population and the archive size is 100 for the MOEAs and for each subpopulation of the CCMOEAs. The size of the final archive obtained by merging all CCMOEAs subpopulations archives is 100.

3.6.2 Methodology for the Comparisons The following quality indicators, well-known in the multiobjective literature, have been used to compare the performance of the algorithms: • Hypervolume (HV) [33]: Calculates the m-dimensional volume (in the objective space) covered by the solutions in the evaluated Pareto front Q and a dominated reference point W . Mathematically, for each solution i ∈ Q, a hypercube vi is constructed with the reference point W (e.g. constructed with a vector of worst objective function values) and the solution i as the diagonal corners of the

3


65

hypercube. The reference point can simply be found by constructing a vector of worst objective function values. Thereafter, a union of all hypercubes is found and its hypervolume is calculated, as shown in Eq. (3.6). Algorithms with the highest HV value perform best as this metric takes its maximum value when all the solutions in the evaluated Pareto front belong to the optimal one. ⎛ ⎞ HV = volume ⎝

|Q|

vi ⎠

(3.6)

i=1

• SPREAD : Estimates the distribution of the solutions along the Pareto front. SPREAD was originally defined in [12] for bi-objective problems, and later generalized for problems with more than two objectives in [24]. This work uses the latter version – defined in Eq. (3.7) – which is more generic, even when the considered problem is a two-objectives one. Algorithms with a SPREAD value equals to zero have an ideal distribution of the solutions in the Pareto front. ∑m i=1 (d(ei , S)) + ∑X∈S d(X , S) − d Δ= . m ∑i=1 d(ei , S) + |S| · d

(3.7)

where S is a set of solutions, S∗ is the set of Pareto optimal solutions, (e1 , . . . , em ) are m extreme solutions in S∗ , m is the number of objectives, and d(X , S) = min || f (X ) − f (Y )||2 Y ∈S,Y =X

d=

1 ∑∗ d(X , S) |S∗ | X∈S

(3.8) (3.9)

• Unary Additive Epsilon (Iε1+ ) indicator [18]: Provides a measure of the convergence, i.e. of the distance to the optimal Pareto front. Given an approximation set of a problem, S, the Iε1+ indicator is a measure of the smallest distance needed to translate every point in S so that it dominates the true Pareto front of the problem S∗ . More formally, given e1 = (e11 , ..., e1m ) and e2 = (e21 , ..., e2m ), where m is the number of objectives, Iε1+ (S) = inf {∀e2 ∈ S∗ ∃e1 ∈ S : e1 ≺ε e2 } ε ∈R

(3.10)

where e1 ≺ε e2 if and only if ∀1 ≤ i ≤ m : e1i < ε + e2i . Therefore, in this chapter different characteristics of the Pareto fronts reported by the algorithms are analyzed: SPREAD measures the distribution of the solutions in the Pareto front, Iε1+ measures the convergence (i.e., the distance to the optimal Pareto front), and HV is evaluating both convergence and distribution. A normalization is also applied on the objective function values with respect to the optimal Pareto prior to applying the metrics, so as to prevent any bias. Since the optimal

66


Table 3.4 Median and inter quartile range for HV metric NSGA-II ZDT1 6.60 · 10−1 3.0·10−4 ZDT2 3.27 · 10−1 2.5·10−4 ZDT3 5.15 · 10−1 1.0·10−4 ZDT4 6.60 · 10−1 3.6·10−4 ZDT6 3.98 · 10−1 4.7·10−4 DTLZ1 4.93 · 10−1 7.5·10−4 DTLZ2 2.11 · 10−1 2.2·10−4 DTLZ3 2.09 · 10−1 1.8·10−3 DTLZ4 1.72 · 10−1 8.0·10−2 DTLZ5 2.11 · 10−1 2.6·10−4 DTLZ6 1.86 · 10−1 2.3·10−2 DTLZ7 3.34 · 10−1 1.4·10−4

SPEA2 6.62 · 10−1 1.1·10−4 3.29 · 10−18.50·10−5 5.16 · 10−1 6.4·10−5 6.62 · 10−1 3.1·10−4 4.01 · 10−1 2.8·10−4 4.94 · 10−1 4.4·10−4 2.12 · 10−1 1.1·10−4 2.09 · 10−1 1.6·10−3 1.58 · 10−1 9.1·10−2 2.12 · 10−1 1.1·10−4 1.94 · 10−1 2.1·10−2 3.34 · 10−1 7.0·10−5

MOCell 6.62 · 10−1 2.0·10−5 3.29 · 10−1 1.8·10−5 5.16 · 10−1 1.4·10−5 6.62 · 10−1 2.9·10−4 4.01 · 10−1 2.3·10−4 4.94 · 10−1 8.8·10−4 2.12 · 10−1 2.5·10−5 2.09 · 10−1 2.3·10−3 1.26 · 10−1 1.0·10−1 2.12 · 10−1 2.3·10−5 1.84 · 10−1 2.3·10−2 3.33 · 10−1 1.2·10−2

CCNSGA-II 6.62 · 10−1 9.6·10−5 3.29 · 10−1 6.9·10−5 5.16 · 10−1 4.1·10−5 4.91 · 10−1 1.6·10−1 4.01 · 10−1 1.3·10−4 4.94 · 10−1 8.4·10−4 2.12 · 10−1 7.5·10−5 2.07 · 10−1 2.3·10−3 1.94 · 10−1 5.7·10−2 2.12 · 10−1 7.6·10−5 2.11 · 10−1 6.5·10−3 3.34 · 10−1 3.8·10−5

CCSPEA2 6.62 · 10−1 6.7·10−5 3.29 · 10−1 6.1·10−5 5.16 · 10−1 3.6·10−5 5.00 · 10−1 1.7·10−1 4.00 · 10−1 2.2·10−4 4.93 · 10−1 8.6·10−4 2.12 · 10−1 7.2·10−5 2.00 · 10−1 6.8·10−3 1.73 · 10−1 8.1·10−2 2.12 · 10−1 7.9·10−5 2.11 · 10−1 6.2·10−3 3.34 · 10−1 3.5·10−5

CCMOCell 6.62 · 10−1 2.0·10−5 3.29 · 10−1 1.9·10−5 5.16 · 10−1 1.3·10−5 1.87 · 10−1 1.7·10−1 4.01 · 10−1 2.17·10−4 5.00 · 10−1 3.2·10−4 2.12 · 10−1 2.0·10−5 2.09 · 10−1 1.8·10−3 1.37 · 10−1 1.0·10−1 2.12 · 10−1 2.2·10−5 1.80 · 10−1 2.6·10−2 3.24 · 10−1 3.3·10−2

Pareto front is not known for the combinatorial optimization problems, a reference Pareto front is created by merging all the Pareto fronts found by the four algorithms in the 100 runs into a single one. Such a front is generated for each problem instance studied.

3.6.3 Results This section presents the main results obtained during our experiments, run on a 12 cores Intel Xeon 2.26 GHz Processors with 24Gb of RAM under Linux Debian 2.6.26-2 and Java 1.5. It is structured as follows. We first focus on the behavior of the algorithms for the academic continuous benchmarks in Sect. 3.6.3.1, and we then concentrate on the combinatorial problem in Sect. 3.6.3.2. Finally, we end with some discussion summarizing all the results in Sect. 3.6.3.3. 3.6.3.1

Continuous Problems

We compare in this section the selected MOEAs versus their corresponding CCMOEAs. The results are shown in tables 3.4 to 3.6. Statistical confidence in our comparisons is assessed by performing the Wilcoxon test [29]. The grey background emphasizes significantly better results of the algorithm with respect to the corresponding one. Additionally, the overall best result per benchmark function found among the reference algorithms and their new cooperative coevolutionary version is shown in bold font. Table 3.4 shows the results obtained by the different algorithms in terms of the hypervolume metric. We can see that, in general, the CCMOEAs show a better overall behavior than the reference algorithms, since they are statistically better for 30 problems, and worse only for 6. Furthermore, the CCMOEAs always outperform the MOEAs, with the exceptions of problems ZDT4, DTLZ1, and DTLZ3; and only in the case of ZDT4 the MOEAs outperform the CCMOEAs in all cases.

3


67

Table 3.5 Median and inter quartile range for Δ metric NSGA-II ZDT1 3.71 · 10−14.2·10−2 ZDT2 3.88 · 10−13.5·10−2 ZDT3 7.43 · 10−11.9·10−2 ZDT4 4.07 · 10−14.5·10−2 ZDT6 6.43 · 10−15.5·10−2 DTLZ1 4.34 · 10−15.3·10−2 DTLZ2 3.85 · 10−13.9·10−2 DTLZ3 4.21 · 10−14.8·10−2 DTLZ4 3.96 · 10−17.9·10−2 DTLZ5 3.83 · 10−15.0·10−2 DTLZ6 5.96 · 10−15.3·10−2 DTLZ7 6.27 · 10−13.1·10−2

SPEA2 1.47 · 10−11.7·10−2 1.38 · 10−11.5·10−2 7.04 · 10−15.4·10−3 1.17 · 10−11.5·10−2 1.43 · 10−11.8·10−2 1.22 · 10−11.5·10−2 1.52 · 10−11.4·10−2 1.33 · 10−11.9·10−2 1.45 · 10−14.5·10−1 1.54 · 10−11.6·10−2 1.55 · 10−13.1·10−2 5.42 · 10−11.1·10−2

MOCell 5.50 · 10−2 1.4·10−2 6.18 · 10−2 1.4·10−2 7.01 · 10−1 6.7·10−4 5.9 · 10−21.6·10−2 4.77 · 10−2 1.1·10−2 6.08 · 10−2 1.2·10−2 1.02 · 10−1 1.9·10−2 1.32 · 10−1 2.8·10−2 1.09 · 10−1 9.1·10−1 1.04 · 10−1 2.1·10−2 1.15 · 10−1 2.6·10−2 5.19 · 10−1 9.4·10−5

CCNSGA-II 1.58 · 10−11.7·10−2 1.57 · 10−11.5·10−2 7.01 · 10−11.4·10−3 3.21 · 10−12.6·10−1 1.46 · 10−11.4·10−2 1.43 · 10−1 1.27·10−2 1.70 · 10−11.6·10−2 1.83 · 10−13.0·10−2 1.66 · 10−1 2.2·10−2 1.72 · 10−11.7·10−2 1.72 · 10−11.8·10−2 5.23 · 10−15.6·10−3

CCSPEA2 1.15 · 10−12.0·10−2 1.13 · 10−11.7·10−2 7.02 · 10−12.2·10−3 2.79 · 10−12.8·10−1 1.35 · 10−16.0·10−2 1.17 · 10−1 1.45·10−2 1.31 · 10−11.8·10−2 2.66 · 10−11.2·10−1 1.22 · 10−12.1·10−2 1.29 · 10−11.6·10−2 1.21 · 10−11.7·10−2 5.27 · 10−14.7·10−3

CCMOCell 5.88 · 10−2 1.5·10−2 6.93 · 10−2 1.9·10−2 7.01 · 10−1 9.3·10−4 4.71 · 10−1 2.0·10−1 5.61 · 10−2 1.1·10−2 6.47 · 10−21.30·10−2 9.87 · 10−2 1.9·10−2 1.46 · 10−1 2.7·10−2 1.05 · 10−1 9.1·10−1 1.00 · 10−1 1.9·10−2 1.21 · 10−1 4.3·10−2 5.19 · 10−1 3.0·10−4

CCSPEA2 6.77 · 10−3 6.5·10−4 6.66 · 10−3 6.4·10−4 7.22 · 10−3 9.5·10−4 1.32 · 10−1 2.0·10−1 5.83 · 10−3 5.5·10−4 3.64 · 10−3 4.7·10−4 6.27 · 10−3 6.4·10−4 1.27 · 10−2 6.3·10−3 6.51 · 10−3 1.5·10−3 6.30 · 10−3 7.2·10−4 6.02 · 10−3 5.5·10−4 6.79 · 10−3 7.1·10−4

CCMOCell 5.41 · 10−3 2.6·10−4 5.37 · 10−3 2.4·10−4 4.84 · 10−3 2.5·10−4 4.27 · 10−1 4.8·10−1 4.33 · 10−3 2.3·10−4 2.83 · 10−3 1.3·10−4 5.00 · 10−3 2.0·10−4 6.82 · 10−3 1.5·10−3 5.27 · 10−3 1.0 5.00 · 10−3 2.3·10−4 2.17 · 10−2 3.7·10−2 4.91 · 10−3 3.3·10−4

Table 3.6 Median and inter quartile range for Iε + metric NSGA-II ZDT1 1.29 · 10−2 3.3·10−3 ZDT2 1.22 · 10−2 2.3·10−3 ZDT3 7.25 · 10−3 2.0·10−3 ZDT4 1.24 · 10−2 3.2·10−3 ZDT6 1.42 · 10−2 3.3·10−3 DTLZ1 1.26 · 10−2 3.0·10−3 DTLZ2 1.10 · 10−2 2.8·10−3 DTLZ3 1.20 · 10−2 2.7·10−3 DTLZ4 1.16 · 10−2 4.4·10−3 DTLZ5 1.05 · 10−2 2.6·10−3 DTLZ6 2.74 · 10−2 2.2·10−2 DTLZ7 8.62 · 10−3 2.0·10−3

SPEA2 8.10 · 10−31.2·10−3 7.48 · 10−37.9·10−4 6.98 · 10−31.5·10−3 7.36 · 10−39.8·10−4 6.91 · 10−37.5·10−4 6.71 · 10−31.0·10−3 7.92 · 10−35.8·10−4 7.92 · 10−37.1·10−4 7.64 · 10−35.0·10−1 7.92 · 10−35.3·10−4 1.69 · 10−21.5·10−2 6.94 · 10−31.1·10−3

MOCell 5.42 · 10−3 2.4·10−4 5.32 · 10−3 2.2·10−4 4.14 · 10−3 2.0·10−4 5.67 · 10−3 3.3·10−4 5.60 · 10−3 2.8·10−4 5.87 · 10−3 6.3·10−4 7.92 · 10−3 0.0 7.92 · 10−3 3.7·10−4 5.24 · 10−3 1.0 7.92 · 10−3 0.0 2.06 · 10−2 3.1·10−2 4.18 · 10−3 1.8·10−4

CCNSGA-II 7.27 · 10−3 9.0·10−4 7.09 · 10−3 8.5·10−4 5.58 · 10−3 8.8·10−4 1.32 · 10−1 2.7·10−1 5.57 · 10−3 3.6·10−4 3.48 · 10−3 3.5·10−4 6.51 · 10−3 7.2·10−4 8.57 · 10−3 1.8·10−3 6.43 · 10−3 8.7·10−4 6.50 · 10−3 8.4·10−4 6.13 · 10−3 4.7·10−4 6.27 · 10−3 5.1·10−4

Regarding the Δ metric, we can see in Table 3.5 that the CCMOEAs generally provide better results than the corresponding MOEAs, as it happened before for the HV metric. The CCNSGA-II algorithm is outperforming NSGA-II in every problem. Contrary, MOCell finds solutions that are usually better spread than CCMOCell. We observe that MOCell is statistically better than CCMOCell for 6 problems and worse for 1 (there are no statistical differences for the other 5 problems). Finally, CCSPEA2 is outperforming SPEA2 for 9 functions, being worse for only 2. Table 3.6 contains the comparison of the algorithms in terms of the Iε + metric. Like in the case of the other two metrics, the CCMOEAs are generally performing better than their corresponding MOEAs for the studied benchmarks. They are outperforming the MOEAs for all the problems except for ZDT3, ZDT4, DTLZ3, and DTLZ7. From them, only in the case of ZDT4 the three MOEAs were statistically better than the CCMOEAs. Figure 3.5 provides a comparison of the computational speedup of the CCMOEAs with respect to the corresponding sequential MOEAs. The speedup

68

B. Dorronsoro et al. Super-linear speedup ZDT1 ZDT2 ZDT3

NSGAII SPEA2 MOCell

ZDT4 ZDT6 DTLZ1 DTLZ2 DTLZ3 DTLZ4 DTLZ5 DTLZ6 DTLZ7

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

Speedup

Fig. 3.5 Speedup of the CCMOEAs with respect to their corresponding MOEAs for the continuous optimization problems

is defined as the time the sequential algorithm takes over the time of the corresponding CCMOEA. We can see that it is possible to achieve super-linear speedups in some cases. The reason is that the total number of evaluations performed by the CCMOEAs is the same as for the MOEAs: 500000. Therefore, in the CCMOEAs with 4 subpopulations tested in this work, every subpopulation is doing 500000/4 = 125000 evaluations, and the subpopulations are evolving in parallel. Additionally, the chromosome length of individuals in every subpopulation is divided by 4 too. We can see that, in general, the lower speedups are obtained for the NSGAII algorithm (ranging from 1.2578 to 2.2969), while the speedups for the other two algorithms are comparable (between 2.7522 and 4.3708 for SPEA2 and with limits 2.4920 and 4.2071 for MOCell). We would like to emphasize at this point that the algorithms compared in terms of speedup in this study are not the same: the reference sequential algorithms are the state-of-the-art MOEAs, while the compared ones are their new proposed cooperative coevolutionary versions. Therefore, the CCMOEAs are finding better results, in general, than the compared MOEAs at higher speeds, and even sometimes at superlinear speedups, as it is the case of CCSPEA2 and CCMOCell for some problems. 3.6.3.2

Combinatorial Problems

We proceed to compare in this section the six studied algorithms on the RSMP problem. All the obtained results are summarized in tables 3.7 to 3.9. As in the case of the tables in Sect. 3.6.3.1, the best result for every problem is emphasized

3


69

Table 3.7 Median and inter quartile range for HV metric

u_c_hihi u_i_hihi u_s_hihi u_c_hilo u_i_hilo u_s_hilo u_c_lohi u_i_lohi u_s_lohi u_c_lolo u_i_lolo u_s_lolo

NSGA-II 4.20 · 10−1 2.1e−02 3.16 · 10−1 3.1e−02 4.34 · 10−1 3.0e−02 3.98 · 10−1 2.9e−02 4.31 · 10−1 2.3e−02 3.61 · 10−1 3.6e−02 4.15 · 10−1 2.3e−02 3.17 · 10−1 2.7e−02 4.07 · 10−1 2.3e−02 3.67 · 10−1 2.7e−02 3.60 · 10−1 2.8e−02 3.86 · 10−1 3.3e−02

SPEA2 5.68 · 10−1 1.7e−02 4.56 · 10−1 4.3e−02 5.58 · 10−1 2.2e−02 5.71 · 10−1 2.4e−02 5.15 · 10−1 4.2e−02 5.37 · 10−1 1.9e−02 5.64 · 10−1 2.0e−02 4.54 · 10−1 3.9e−02 5.52 · 10−1 1.9e−02 5.71 · 10−1 2.8e−02 4.84 · 10−1 4.4e−02 5.48 · 10−1 2.5e−02

MOCell 4.07 · 10−1 2.0e−02 3.03 · 10−1 2.6e−02 4.27 · 10−1 2.6e−02 3.76 · 10−1 2.3e−02 4.31 · 10−1 3.1e−02 3.55 · 10−1 2.8e−02 4.05 · 10−1 2.0e−02 3.10 · 10−1 2.7e−02 3.95 · 10−1 3.0e−02 3.48 · 10−1 2.4e−02 3.56 · 10−1 3.2e−02 3.67 · 10−1 4.1e−02

CCNSGA-II 3.04 · 10−14.0e−02 2.39 · 10−16.1e−02 4.00 · 10−13.7e−02 3.52 · 10−13.6e−02 4.03 · 10−15.8e−02 3.51 · 10−13.4e−02 2.87 · 10−14.2e−02 2.44 · 10−14.9e−02 3.47 · 10−14.0e−02 2.99 · 10−14.8e−02 3.06 · 10−16.2e−02 3.70 · 10−13.3e−02

CCSPEA2 5.93 · 10−1 9.2e−03 5.27 · 10−1 1.9e−02 5.87 · 10−1 8.1e−03 6.04 · 10−1 1.1e−02 5.77 · 10−1 1.3e−02 5.70 · 10−1 1.1e−02 5.89 · 10−1 1.1e−02 5.32 · 10−1 2.2e−02 5.87 · 10−1 9.8e−03 6.05 · 10−1 1.0e−02 5.58 · 10−1 1.7e−02 5.73 · 10−1 8.6e−03

CCMOCell 5.70 · 10−1 2.0e−02 4.88 · 10−1 2.6e−02 5.66 · 10−1 1.3e−02 5.81 · 10−1 1.6e−02 5.37 · 10−1 2.4e−02 5.45 · 10−1 2.1e−02 5.74 · 10−1 1.3e−02 4.88 · 10−1 3.3e−02 5.61 · 10−1 1.8e−02 5.76 · 10−1 2.5e−02 5.19 · 10−1 2.7e−02 5.51 · 10−1 1.6e−02



Table 3.8 Median and inter quartile range for Δ metric u_c_hihi u_i_hihi u_s_hihi u_c_hilo u_i_hilo u_s_hilo u_c_lohi u_i_lohi u_s_lohi u_c_lolo u_i_lolo u_s_lolo





with bold font, and the MOEAs are compared versus the corresponding CCMOEAs by means of the Wilcoxon test to asses statistical confidence to our study. Those algorithms statistically better are emphasized with grey background in the tables. We can see in Table 3.7 that CCSPEA2 and CCMOCell clearly outperform their corresponding MOEAs with statistical significance for all the problems by means of the hypervolume metric. On the contrary, NSGAII outperforms CCNSGAII for all the problems too. It can be seen in Table 3.8 that the three CCMOEAs are obtaining more diverse Pareto fronts, in general, than the standard MOEAs. Specifically, CCSPEA2 and CCMOCell are significantly better than SPEA2 and MOCell in all the problems, as it happened for the hypervolume metric. Regarding the other two algorithms, we found that, in terms of this indicator, CCNSGA-II is better than NSGA-II for 4 problems and worse for 5. For the other three problems, we did not find statistical significance on the comparison of the results. If we now pay attention to the accuracy of the solutions reported by the different algorithms, we can see in Table 3.9 that, once again, CCSPEA2 and CCMOCell are clearly outperforming SPEA2 and MOCell, respectively. However, in this case

70


Table 3.9 Median and inter quartile range for Iε + metric

u_c_hihi u_i_hihi u_s_hihi u_c_hilo u_i_hilo u_s_hilo u_c_lohi u_i_lohi u_s_lohi u_c_lolo u_i_lolo u_s_lolo







we did not find significant differences between SPEA2 and CCSPEA2 for all the problems; in three of them the two algorithms perform similarly. Like for the hypervolume metric, NSGA-II outperforms CCNSGA-II for all the problems in terms of Iε + metric too. Super-linear speedup CCNSGAII CCSPEA2 CCMOCell

u_c_hihi u_i_hihi u_s_hihi u_c_hilo u_i_hilo u_s_hilo u_c_lohi u_i_lohi u_s_lohi u_c_lolo u_i_lolo u_s_lolo 0

20

40

60

80

100

120

Improvement

Fig. 3.6 Speedup of the CCMOEAs with respect to their corresponding MOEAs for the combinatorial problems

We present in Fig. 3.6 an analysis of the speedup (calculated in the same way as in Sect. 3.6.3.1) of the CCMOEAs with respect to the equivalent state-of-the-art algorithms. As we can see, in the case of RSMP problem, all the CCMOEAs studied

3


71

perform super-linear speedups with respect to the original MOEAs. In fact, CCMOCell is more than 100 times faster than MOCell for 4 problems. The reasons for these extremely fast speedups are the same ones already mentioned in Sect. 3.6.3.1: every subpopulation is performing less evaluations (i.e., the total number of evaluations divided by the number of subpopulations), and the chromosome length of individuals is divided by the number of islands too. This second issue is particularly interesting in this problem, since the time of the recombination and mutation operators used is rapidly increasing with the chromosome length. Therefore, when reducing it, these operators are performed faster, allowing the CCMOEAs to greatly increase the speedup with respect to the standard MOEAs. 3.6.3.3

Discussion on the Results

We summarize and compare in this section the results of the studies made in sections 3.6.3.1 and 3.6.3.2, for the 24 considered problems. This benchmark of problems is composed by 12 continuous academic functions and 12 instances with different characteristics of a combinatorial real-world problem. We can conclude that the proposed CCMOEAs behave, in general, better than their corresponding MOEAs at considerably higher computation speeds. For both the continuous and combinatorial optimization problems studied, the CCMOEAs showed to be statistically better than the MOEAs in 149 cases, and worse in 49. If we pay attention to the continuous problems, we can see that the CCMOEAs outperform the equivalent MOEAs 76 times, while they are outperformed in 20 cases. Additionally, in the comparisons made on the combinatorial problems, the results clearly favor the CCMOEAs too, since they significantly outperform the MOEAs in 73 out of of the 108 comparisons made (the MOEAs were better only in 29 cases, and we did not find significant differences for the other 6 comparisons left). In a more detailed analysis, we can see that the CCMOEAs behave differently with respect to their corresponding MOEAs for the distinct kinds of problems studied. We observe that CCNSGA-II clearly outperforms NSGA-II for the continuous problems, but we obtained the opposite result on the combinatorial problems studied. In the case of the other two proposed CCMOEAs, they clearly outperform the MOEAs. In addition to the better quality of solutions, the new CCMOEAs are able to perform super-linear speedups with respect to the MOEAs. We found super-linear speedups for some of the continuous problems, and for all the combinatorial ones. For the latter ones, for which the time of executing the variation operators depends on the individual chromosome length, the CCMOEAs can be more than 100 times faster than the MOEAs.

3.7 Conclusion and Future Work In this chapter, three new cooperative coevolutionary variants of state-of-the-art multi-objective algorithms, i.e. Non-dominated Sorting Genetic Algorithm II (NSGA-II), Strength Pareto Evolutionary Algorithm 2 (SPEA2) and Multi-objective

72


Cellular Genetic Algorithm (MOCell) have been proposed. The improvement brought by those CCMOEAs both in terms of quality of solutions and computational performance, up to super-linear speedup with the proposed parallel version, has been assessed on continuous optimization benchmarks and one real-world combinatorial optimization problem (i.e., the RSMP problem). As future works, we plan to further benefit from the speedup provided by the parallel CCMOEAs by using asynchronous communications between the subpopulations. Another targeted objective is to benefit from the decomposition induced by the cooperative coevolutionary framework to tackle bigger instances of the RSMP problem. Acknowledgements. This work was completed with the support of Luxembourg FNR GreenIT project (C09/IS/05). Antonio J. Nebro is supported by the “Consejería de Innovación, Ciencia y Empresa”, Junta de Andalucía, under contract P07-TIC-03044 (the DIRICOM project), http://diricom.lcc.uma.es, and the Spanish Ministry of Science and Innovation, under contract TIN2008-06491-C04-01 (the M* project), http://mstar.lcc.uma.es.

References 1. Alba, E., Dorronsoro, B.: Cellular Genetic Algorithms. In: Operations Research/Compuer Science Interfaces. Springer, Heidelberg (2008) 2. Ali, S., Maciejewski, A.A., Siegel, H.J., Kim, J.K.: Measuring the robustness of a resource allocation. IEEE Transactions on Parallel and Distributed Systems 51(7), 630– 641 (2004) 3. Ali, S., Siegel, H.J., Maheswaran, M., Hensgen, D.: Representing task and machine heterogeneities for heterogeneous. Journal of Science and Engineering, Special 50 th Anniversary Issue 3, 195–207 (2000) 4. Barbosa, H.J., Barreto, A.M.: An interactive genetic algorithm with co-evolution of weights for multiobjective problems. In: Spector, L., Goodman, E.D., Wu, A., Langdon, W., Voigt, H.M., Gen, M., Sen, S., Dorigo, M., Pezeshk, S., Garzon, M.H., Burke, E. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2001), pp. 203–210. Morgan Kaufmann, San Francisco (2001) 5. Blazewicz, J., Lenstra, J.K., Rinnooy Kan, A.H.G.: Scheduling subject to resource constraints: classification and complexity. Discrete Applied Mathematics 5, 11–24 (1983) 6. Coello, C.A.C., Lamont, G.B., Veldhuizen, D.A.: Evolutionary Algorithms for Solving Multi-Objective Problems, 2nd edn. Springer, Heidelberg (2007) 7. Coello, C.A.C., Sierra, M.R.: A coevolutionary multi-objective evolutionary algorithm. In: Proc. of IEEE Congress on Evolutionary Computation, vol. (1), pp. 482–489 (2003) 8. Danoy, G., Bouvry, P., Martins, T.: hlcga: A hybrid competitive coevolutionary genetic algorithm. In: Proc. of International Conference on Hybrid Intelligent Systems, vol. 48 (2006), doi:http://doi.ieeecomputersociety.org/10.1109/HIS.2006.32 9. Danoy, G., Dorronsoro, B., Bouvry, P.: Overcoming partitioning in large ad hoc networks using genetic algorithms. In: GECCO, pp. 1347–1354 (2009) 10. Deb, K.: Multi-Objective Optimization using Evolutionary Algorithms. Wiley, Chichester (2001) 11. Deb, K., Agrawal, R.: Simulated binary crossover for continuous search space. Complex Systems 9, 115–148 (1995)

3


73

12. Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation 6(2), 182–197 (2002) 13. Deb, K., Thiele, L., Laumanns, M., Zitzler, E.: Scalable Test Problems for Evolutionary Multiobjective Optimization. In: Evolutionary Multiobjective Optimization. Theoretical Advances and Applications, pp. 105–145. Springer, Heidelberg (2005) 14. Goh, C.K., Tan, K.C.: A coevolutionary paradigm for dynamic multiobjective optimization. In: Evolutionary Multi-objective Optimization in Uncertain Environments. Studies in Computational Intelligence (SCI), vol. 186, pp. 153–185 (2009) 15. Goh, C.K., Tan, K.C.: A competitive-cooperative coevolutionary paradigm for dynamic multiobjective optimization. IEEE Transactions on Evolutionary Computation 13(1), 103–127 (2009) 16. Iorio, A.W., Li, X.: A cooperative coevolutionary multiobjective algorithm using nondominated sorting. In: Deb, K., et al. (eds.) GECCO 2004. LNCS, vol. 3102, pp. 537– 548. Springer, Heidelberg (2004) 17. Keerativuttitumrong, N., Chaiyaratana, N., Varavithya, V.: Multi-objective co-operative co-evolutionary genetic algorithm. In: Guervós, J.J.M., Adamidis, P.A., Beyer, H.-G., Fernández-Villacañas, J.-L., Schwefel, H.-P. (eds.) PPSN 2002. LNCS, vol. 2439, pp. 288–297. Springer, Heidelberg (2002) 18. Knowles, J., Thiele, L., Zitzler, E.: A Tutorial on the Performance Assessment of Stochastic Multiobjective Optimizers. TIK Report 214, Computer Engineering and Networks Laboratory (TIK), ETH Zurich (2006) 19. Lohn, J.D., Kraus, W.F., Haith, G.L.: Comparing a coevolutionary genetic algorithm for multiobjective optimization. In: CEC 2002: Proc. of the 2002 Congress on Evolutionary Computation, pp. 1157–1162. IEEE Computer Society, Washington (2002) 20. Maneeratana, K., Boonlong, K., Chaiyaratana, N.: Multi-objective optimisation by cooperative co-evolution. In: Yao, X., Burke, E.K., Lozano, J.A., Smith, J., MereloGuervós, J.J., Bullinaria, J.A., Rowe, J.E., Tiˇno, P., Kabán, A., Schwefel, H.-P. (eds.) PPSN 2004. LNCS, vol. 3242, pp. 772–781. Springer, Heidelberg (2004) 21. Maneeratana, K., Boonlong, K., Chaiyaratana, N.: Co-operative co-evolutionary genetic algorithms for multi-objective topology design. Computer-Aided Design & Applications 2, 487–496 (2005) 22. Mao, J., Hirasawa, K., Hu, J., Murata, J.: Genetic symbiosis algorithm for multiobjective optimization problem. In: Proc. of the 2001 Genetic and Evolutionary Computation Conference, San Francisco, California, pp. 267–274 (2001) 23. Nebro, A.J., Durillo, J.J., Luna, F., Dorronsoro, B., Alba, E.: MOCell: A cellular genetic algorithm for multiobjective optimization. International Journal of Intelligent Systems 24, 726–746 (2009) 24. Nebro, A.J., Luna, F., Alba, E., Dorronsoro, B., Durillo, J.J., Beham, A.: AbYSS: Adapting scatter search to multiobjective optimization. IEEE Transactions on Evolutionary Computation 12(4), 439–457 (2008) 25. Paredis, J.: Coevolutionary life-time learning. In: Ebeling, W., Rechenberg, I., Voigt, H.M., Schwefel, H.-P. (eds.) PPSN 1996. LNCS, vol. 1141, pp. 72–80. Springer, Heidelberg (1996) 26. Parmee, I.C., Watson, A.H.: Preliminary airframe design using co-evolutionary multiobjective genetic algorithms. In: Proceedings of the Genetic and Evolutionary Computation Conference, vol. 2, pp. 1657–1665. Morgan Kaufmann, Orlando (1999) 27. Potter, M., De Jong, K.: A cooperative coevolutionary approach to function optimization. In: Davidor, Y., Männer, R., Schwefel, H.-P. (eds.) PPSN 1994. LNCS, vol. 866, pp. 249– 257. Springer, Heidelberg (1994)

74


28. Tan, K.C., Yang, Y.J., Goh, C.K.: A distributed cooperative coevolutionary algorithm for multiobjective optimization. IEEE Transactions on Evolutionary Computation 10(5), 527–549 (2006) 29. Wilcoxon, F.: Individual comparisons by ranking methods. Biometrics Bulletin 1(6), 80– 83 (1945) 30. Xhafa, F., Alba, E., Díaz, B.D.: Efficient batch job scheduling in grids using cellular memetic algorithms. In: IPDPS, pp. 1–8. IEEE, Los Alamitos (2007) 31. Zitzler, E., Deb, K., Thiele, L.: Comparison of multiobjective evolutionary algorithms: Empirical results. Evolutionary Computation 8(2), 173–195 (2000) 32. Zitzler, E., Laumanns, M., Thiele, L.: SPEA2: Improving the strength Pareto evolutionary algorithm. Tech. Rep. 103, Computer Engineering and Networks Laboratory (TIK), ETH Zurich (2001) 33. Zitzler, E., Thiele, L.: Multiobjective evolutionary algorithms: A comparative case study and the strength pareto approach. IEEE Transactions on Evolutionary Computation 3(4), 257–271 (1999)

Chapter 4

Parallel Evolutionary Algorithms for Energy Aware Scheduling Yacine Kessaci, Mohand Mezmaz, Nouredine Melab, El-Ghazali Talbi, and Daniel Tuyttens

Abstract. Reducing energy consumption is an increasingly important issue in computing and embedded systems. In computing systems, minimizing energy consumption can significantly reduce the amount of energy bills. The demand for computing systems steadily increases and the cost of energy continues to rise. In embedded systems, reducing the use of energy allows to extend the autonomy of these systems. In addition, the reduction of energy decreases greenhouse gas emissions. Therefore, many researches are carried out to develop new methods in order to consume less energy. This chapter gives an overview of the main methods used to reduce the energy consumption in computing and embedded systems. As a use case and to give an example of a method, the chapter describes our new parallel bi-objective hybrid genetic algorithm that takes into account the completion time and the energy consumption. In terms of energy consumption, the obtained results show that Yacine Kessaci INRIA Lille-Nord Europe, CNRS/LIFL Université Lille 1. Parc Scientifique de la Haute Borne 40, avenue Halley Bât.A, Park Plaza 59650, Villeneuve d’Ascq, France e-mail: [email protected] Mohand Mezmaz UMons/FPMs, 9, rue de Houdain 7000 Mons, Belgium e-mail: [email protected] Nouredine Melab INRIA Lille-Nord Europe, CNRS/LIFL Université Lille 1. Parc Scientifique de la Haute Borne 40, avenue Halley Bât.A, Park Plaza 59650, Villeneuve d’Ascq, France e-mail: [email protected] El-Ghazali Talbi INRIA Lille-Nord Europe, CNRS/LIFL Université Lille 1. Parc Scientifique de la Haute Borne 40, avenue Halley Bât.A, Park Plaza 59650, Villeneuve d’Ascq, France e-mail: [email protected] Daniel Tuyttens UMons/FPMs, 9, rue de Houdain 7000 Mons, Belgium e-mail: [email protected] P. Bouvry et al. (Eds.): Intelligent Decision Systems, SCI 362, pp. 75–100. c Springer-Verlag Berlin Heidelberg 2011 springerlink.com

76

Y. Kessaci et al.

our approach outperforms previous scheduling methods by a significant margin. In terms of completion time, the obtained schedules are also shorter than those of other algorithms.

4.1 Introduction Computers use a significant and growing portion of the energy in the world. Therefore, energy-aware computing is crucial for large-scale systems that consume considerable amount of energy and embedded systems that utilize battery for their power. A recent study on power consumption by servers [14] shows that, in 2005, the power used by servers represented about 0.6% of total U.S. electricity consumption. That number grows to 1.2% when cooling and auxiliary infrastructures are included. In the same year, the aggregate electricity bill for operating those servers and associated infrastructure was about $2.7 billions and $7.2 billions for the U.S. and the world, respectively. The total electricity use for servers doubled over the period 2000 to 2005 in worldwide. The number of transistors integrated into today’s Intel Itanium 2 processor reaches nearly 1 billion. If this rate continues, the heat (per square centimeter) produced by future Intel processors would exceed that of the surface of the sun [13]. In this chapter, we present some important works of the literature about reducing energy consumption. These works are classified according to three criteria. The first criterion concerns the optimization method used to minimize the consumed energy. This method can be mono-objective or multi-objective approach. The second criterion concerns the level of the system on which an approach is based. Indeed, a method can be based on the hardware or the software part of a system. The third and last criterion concerns the type of system to which the approach is intended to be used. An approach can be developed for computing systems (C.S) or embedded systems (E.S). In order to give an example of a method and as a use case, we describe our new parallel bi-objective hybrid genetic algorithm that takes into account, not only makespan, but also energy consumption. Our new approach is a hybrid between a multi-objective parallel genetic algorithm and energy-conscious scheduling heuristic (ECS) [17]. The results clearly demonstrate the superior performance of ECS over the other algorithms like DBUS [2] and HEFT [27]. Genetic algorithms make it possible to explore a great range of potential solutions to a problem. The exploration capability of the genetic algorithm and the intensification power of ECS are complementary. A skillful combination of a metaheuristic with concepts originating from other types of algorithms lead to more efficient behavior. Our algorithm is effective as it profits from the exploration power of the genetic algorithm, the intensification capability of ECS, the cooperative approach of the island model, and the parallelism of the multi-start model. The island model and the hybridization improve the quality of the obtained results. The multi-start model reduces the running time of a resolution.

4

Parallel Evolutionary Algorithms for Energy Aware Scheduling

77

The remainder of the chapter is organized as follows. Section 4.2 gives an overview of our classification. Section 4.3 explains the classification according to the optimization method used to minimize the consumed energy. Section 4.4 describes the classification according to the level of the system on which an approach is based. Section 4.5 explains the classification according to the type of system to which the approach is intended to be used. Section 4.6 presents the application, system, energy and scheduling models used in our use case. Our algorithm is presented in Section 4.7. The results of our comparative experimental study are discussed in Section 4.8. The conclusion is drawn in Section 4.9.

4.2 Energy Aware Approaches This section presents an overview of the main methods published in the literature to reduce energy consumption. Our state of the art does not address the techniques used by computer system manufacturers. Indeed, manufacturers use some techniques in the design of electronic circuits and components to reduce their energy consumption. Our classification only focuses on methods intended to be exploited by the users of computing and embedded systems. As shown in Fig. 4.1, these methods can be classified according to three criteria.

Fig. 4.1 The classification criteria used

The first criterion concerns the type of the used optimization. This optimization can be mono-objective or multi-objective. In multi-objective optimization , the main approaches found in the literature are based on an aggregation, a lexicographic or a Pareto fitness function. The system-level is the second criterion on which methods can be classified. To reduce energy consumption , the methods are based either on software or hardware part of a system. In the hardware-level, the techniques used are generally virtualization and consolidation based approaches. While in the software-level, two techniques can be identified to address the task scheduling problem: Static methods where scheduling is done before the execution of a program, and dynamic methods, where the appropriate scheduling is calculated during the execution of a program. The third and last criterion used in our classification is the type of system to which the approach is intended to be used. In this criterion, a system can be either a computing or an embedded system. Reducing energy consumption in embedded

78

Y. Kessaci et al.

systems aims to increase the autonomy of devices. While the goal of reducing energy consumption in computing systems is to decrease the cost of the energy and the greenhouse gas emissions. Table 4.1 gives some examples of approaches with their classification using our taxonomy. Table 4.1 Examples of approaches with their classification Method

Level

System

Optimization

DPM&DVS[25] DynamicAssgn[10]

Hardware(Dyn.) Hardware(Dyn.)

Mono-objective Multi-objective(Lex.)

ECS[17]

Hardware(Dyn.)

E.S. C.S.(distributed computing) C.S.(high-performance computing) C.S.(cloud computing) C.S.(high-performance computing) C.S.(distributed computing) C.S.(cloud computing) E.S. C.S.(cluster computing)

ECTC and MaxUtil[16] Software(Cons.) Jitter[6] Hardware(Dyn.) MMF-DVFS[23]

Hardware(Stat.)

MO[1] PSAGA/PGA[18] TDVAS[24]

Software(Virt.) Hardware(Dyn.) Hardware(Stat.)

Multi-objective(Agg.) Mono-objective Multi-objective(Lex.) Multi-objective(Lex.) Multi-objective(Lex.) Mono-objective Multi-objective(Lex.)

Where (Dyn.) and (Stat.), meaning respectively (Dynamic) and (Static) are the two categories of the energy-aware task scheduling in the hardware-level. Concerning (Cons.) and (Virt.), they both describe two of the most widely used techniques in the software-level approach (Consolidation and Virtualization). (Agg.) and (Lex.), standing respectively for aggregation and lexicographic, are techniques used in multi-objective optimization.

4.3 Optimization Approaches An approach used to optimize the energy consumption can be mono-objective or multi-objective. The mono-objective approaches, such as [8, 16, 18, 19, 22, 25, 28], only aim to optimize the energy consumption with assuming some constraints, especially on the completion time. In addition to the consumed energy, multi-objective methods are also trying to optimize other objectives such as quality of service (QoS). QoS is usually expressed in terms of makespan. The energy consumption depends heavily on the QoS offered. Indeed, the minimization of energy consumption and maximizing the QoS are conflicting objectives. The goal of a multi-objective optimization method is to find a good compromise between these two objectives. In addition, the cost of a service can be another objective to be considered. More the QoS and energy consumption increase, more their cost increases. Table 4.2 gives some examples of the optimized objectives in different methods found in literature.

4


79

Table 4.2 Examples of the optimized objectives in different methods Method

Optimization

Criteria

DPM&DVS[25] DynamicAssgn[10] ECS[17] ECTC and MaxUtil[16] Jitter[6] MMF-DVFS[23] MO[1] PSAGA/PGA[18] TDVAS[24]

Mono-objective Multi-objective(Lex.) Multi-objective(Agg.) Mono-objective Multi-objective(Lex.) Multi-objective(Lex.) Multi-objective(Lex.) Mono-objective Multi-objective(Lex.)

Energy Energy and completion time Energy and completion time Energy Energy and completion time Energy and completion time Energy and QoS Energy Energy and completion time

As shown in Fig. 4.2, it is possible to identify three main categories according to the multi-objective approach chosen: Aggregation, lexicographic and Pareto approaches.

Fig. 4.2 Classification according to the optimization approaches

4.3.1 Aggregation Approach The aggregation (or weighted) method is one of the first and most used methods for generation of Pareto optimal solutions . It consists in using an aggregation function to transform a multi-objective problem into a mono-objective problem by combining the various objective functions into a single objective function generally in a linear way. The obtained results in the resolution of the problem depend strongly on the parameters chosen for the weight vector. In [17], the authors address the task scheduling problem on heterogeneous computing systems (HCSs) and propose an energy-conscious scheduling heuristic (ECS) that takes into account the completion time and energy consumption. The heuristic proposed in [17] tries to balance these two performance goals using a novel objective function called relative superiority (RS).

80

Y. Kessaci et al.

4.3.2 Lexicographic Approach In this traditional approach, the search is carried out according to a given preference order of the objectives. This order defines the significance level of the objectives. Then, a set of mono-objective problems are solved in a sequential manner. If the problem associated with the most significant objective function has a unique solution, the search provides the optimal solution and stops. Otherwise, the problem associated with the second most significant objective function is solved. The same stopping criteria and process are iterated until the treatment of the last function. In [23], the authors investigate the task scheduling problem, and propose a heuristic called maximum minimum frequency DVFS (MMF-DVFS). The proposed method operates in two stages. The goal of the first stage is to find a schedule of tasks that minimizes the makespan. The second stage tries to find the right setting of the processor to minimize energy consumption without changing the makespan of the schedule provided by the first stage.

4.3.3 Pareto Approach The Pareto approaches use the concept of dominance in the fitness assignment, contrary to the other approaches that use a scaling function or treat the various objectives separately. The main advantage of the Pareto approaches is that they do not need the transformation of the multi-objective problem into a mono-objective problem. In a single run, they are able to generate a diverse set of Pareto solutions in the concave portions of the convex hull of feasible objective space. For example, [20] and [26] are examples of methods that use a Pareto approach . In [20], the authors propose a multi-objective fuzzy genetic algorithm to optimize the energy saving scheduling tasks on heterogeneous chip multi-processor (CMP) system. In this algorithm, the Pareto set includes chromosomes with the shortest task execution time and the lowest system energy. In [26], the authors study the inter-relationships between energy consumption, resource utilization, and processor utilization. In [11], a cooperative game model based on the concept of Nash Bargaining Solution (NBS) is proposed. NBS allows to guarantee the Pareto optimality of the approach. The aim of this work is to simultaneously minimize the energy consumption and the subject to the constraints of deadlines and tasks’ architectural requirements.

4.4 System-Level Approaches To reduce energy consumption, various issues such as resource management in both software and hardware must be addressed. The system-level (i.e. hardware or software) used by a method is another criterion on which methods can be classified.

4


81

Software approaches are mainly based on virtualization or task consolidation. Virtualization consists to run on a single computer multiple operating systems as if they are running on separate computers. Hardware approaches use the opportunity offered by manufacturers in modern processors to adjust the voltage and frequency. This adjustment can vary the processor performance and thus its energy cost.

Fig. 4.3 Classification according to the system-levels

4.4.1 Hardware-Level Approaches Dynamic voltage frequency scaling (DVS) is a powerful management technique in computer architecture. This technique enables processors to dynamically increase or decrease voltage supply levels (VSLs). DVS to decrease voltage is known as undervolting , and DVS to increase voltage is known as over-volting. Under-volting is done in order to conserve power, and over-volting is done in order to increase computer performance. Dynamic frequency scaling (DFS) is another powerful conservation technique that works on the same principles as DVS. This technique reduces the number of instructions a processor can issue in a given amount of time, thus reducing performance. Voltage and frequency scaling are often used together to save power. When used in this way it is commonly known as dynamic voltage and frequency scaling (DVFS). Many methods, such as [5, 6, 10, 12, 17, 18, 20, 23, 24, 25, 28, 11], try to exploit this technique to reduce energy consumption. In [6], the authors present a system called Jitter, which reduces the frequency on nodes that are assigned less computation and therefore have slack time. This saves energy on these nodes, and the goal of Jitter is to attempt to ensure that they arrive just in time so that they avoid increasing overall execution time. [24] proposes an energy-efficient scheduling algorithm (TDVAS) using the dynamic voltage scaling technique to provide significant energy savings for clusters. The TDVAS algorithm aims at judiciously leveraging processor idle times to lower processor voltages (i.e., the dynamic voltage scaling technique or DVS), thereby reducing energy consumption experienced by parallel applications running on clusters. Reducing processor voltages, however, can inevitably lead to increased execution times of parallel task. The salient feature of the TDVAS algorithm is to tackle this problem by exploiting tasks precedence constraints.

82

Y. Kessaci et al.

In [12], the authors propose a new algorithm that reduces energy consumption in a parallel program executed on a power-scalable cluster using DVFS. Whenever the computational load is not balanced, parallel programs encounter slack time, that is, they must wait for synchronization of the tasks. This algorithm reclaims slack time by changing the voltage and frequency, which allows a reduction in energy consumption without impacting on the performance of the program. [10] also presents a novel dynamic algorithm for remapping tasks for energy efficient scheduling of DAG based applications for DVS enabled systems. There are a lot of work on task scheduling problem using the dynamic voltage and frequency scaling technique in heterogeneous computing and real-time embedded systems. As previously mentioned, the main idea of the methods proposed in the literature is to change the voltage level to reduce energy consumption. These voltage level changes are made by taking into account timing information of processors, such as idle or slack time, and timing information of tasks, such as task deadline, task release time and task execution time. According to the availability of timing information of tasks, it is possible to distinguish two categories of methods to solve an energy-aware task scheduling problem: and . 1. Static scheduling In static scheduling [5, 20, 23, 24, 28], timing information of tasks and timing information of processors are available in compile time before the deployment of tasks. This static scheduling involves most of the large-scale computational problems such as object recognition in machine vision applications, chemistry and bioinformatics. Having this information allows static schedulers to be developed by maximizing processor utilization with meeting all timing information of tasks. 2. Dynamic scheduling In dynamic scheduling [6, 10, 12, 17, 18, 25], also called online or real-time scheduling, the deadlines of tasks are available in compile time but their release and execution times should be estimated during run time. This dynamic scheduling not only involves dynamic large-scale approximation and optimization such as in weather forecasting and search algorithms but also is used in most of poweraware devices like laptops, wireless sensors and cell phones.

4.4.2 Software-Level Approaches At software-level, two main techniques are used to reduce energy consumption: Virtualization and . Operating system-level virtualization is a server method where the operating system allows for multiple isolated user-space instances. Virtualization can be seen as splitting an underlying hardware entity into smaller identical virtual entities running isolated from each other. Virtualization allows to reduce energy consumption

4


83

1. Virtualization In [8], an architecture for virtualizing and sharing hardware resources in future home environments is presented. The architecture aims at utilizing existing home resources in such a way that the consumed energy is minimized and the energy is efficiently used. A fully decentralized management system is proposed, interconnecting possibly thousands of homes in a peer-to-peer like manner. Energy optimization is done in a decentralized way by converging to a global energy optimum based on energy and performance metrics that have been defined. [1] investigates the energy consumption in office environments and discusses the potential of energy savings. An energy-efficient office management approach is suggested, based on resource virtualization , power management , and resource sharing . 2. Task consolidation Like virtualization, task consolidation is an effective method to increase resource utilization and in turn reduces energy consumption. The task consolidation problem is the process of assigning a set N of n tasks (service requests or simply services) to a set R of r resources without violating time constraints. This technique aims to maximize resource utilization by minimizing energy consumption. Indeed, a resource allocation strategy that takes into account resource utilization would lead to better energy efficiency. [26] exposes some of the complexities in performing consolidation for power optimization, and proposes some research directions to address the challenges involved. In [16], the authors present two energy-conscious task consolidation heuristics ECTC and MaxUtil. These two heuristics aim to maximize resource utilization and explicitly take into account both active and idle energy consumption. The proposed heuristics assign each task to the resource on which the energy consumption for executing the task is explicitly or implicitly minimized without the performance degradation of that task.

4.5 Approaches According to Targeted Execution System As shown in Fig. 4.4, two types of systems can be identified: Embedded systems and computing systems. The issue of reducing energy consumption interests these two types of systems for different reasons. In embedded systems, the main goal of reducing energy consumption is to increase the autonomy of devices, while in computing systems, the main goal is to reduce the energy cost.

4.5.1 Embedded Systems An embedded system can be defined as an electronic and autonomous system which is dedicated to a specific task. The resources of an embedded system are limited.

84

Y. Kessaci et al.

Fig. 4.4 Classification according to the system types

This limitation is usually spatial (limited size) and energy (consumption restricted). Many of today’s embedded systems, such as wireless and portable devices rely heavily on the limited power supply. Therefore, energy efficiency becomes one of the major design concerns for embedded systems. Many methods, such as [5, 18, 25, 28], are developed to reduce the energy consumption of embedded systems. In [5], some scheduling heuristics are presented that determine the best trade-off between three techniques: DVS, processor shutdown, and finding the optimal number of processors. Indeed, when peak performance is unnecessary, DVS can be used to reduce the dynamic power consumption of embedded multiprocessors. However, static power consumption is expected to increase significantly. Then it will be more effective to limit the number of employed processors, and use a combination of DVS and processor shutdown. [25] presents an optimal approach based on the time-indexed semi-Markov decision processes (TISMDP) for optimizing power management policies of embedded systems. In [18], the authors developed a genetic algorithm for the DVS scheduling problem. They also describe a Parallel genetic algorithm PGA for finding better schedules with less time by parallelizing the genetic algorithm. A hybrid parallel algorithm is also developed to further improve the search ability of parallel genetic algorithm by combining the parallel genetic algorithm with the technique of simulated annealing PSAGA.

4.5.2 Computing Systems Demand for cloud computing, high performance computing, cluster computing, etc. is growing. The industry is responding to the demand by more powerful systems, and consequently, creating highly energy-intensive products. The energy consumption of these systems is associated with various environmental, system performance and monetary issues. Scheduling parallel applications on computing systems, especially on large-scale computing systems, is challenging due to high energy consumption and significant communication latencies. Therefore, conserving energy consumption and reducing

4


85

schedule lengths are two important concerns in the design of economical and environmentally friendly systems. It is only recently that much attention has been paid to energy consumption in scheduling. [10] and [23] address the task scheduling problem on distributed computing systems. [17] and [20] investigate the same problem on respectively heterogeneous computing systems and chip multi-processor systems. With the chip multi-processor being more and more widespread used in the laptop, desktop and data center area, the power-performance scheduling issues are becoming challenges. In [24], the authors address the scheduling parallel applications on large scale clusters. In the past decade cluster computing platforms have been widely applied to support a variety of scientific applications. [19] proposes an energy efficient algorithm to schedule real-time tasks with data access requirements on grids. Taking into account both data locations and application properties, the authors design a distributed energy-efficient scheduler that aims to seamlessly integrate the process of scheduling tasks with data placement strategies to provide energy savings. An other approach in [11] based on a game theoretic is proposed. It deals with large scale distributed computing systems like grids. Consolidation of applications in cloud computing environments presents an important approach to streamline resource usage and improve energy efficiency. Based on the fact that resource utilization directly relates to energy consumption, the authors in [16] have modeled their relationship and developed two energy-conscious task consolidation heuristics. [26] outlines the challenges in finding effective solutions to the consolidation problem on cloud computing systems. In [8], an architecture for sharing computing resources among home environments in a peer-to-peer manner is proposed in order to improve the energy efficiency. [22] explores how to integrate power management mechanisms and policies with the virtualization technologies being actively deployed in large-scale datacenters to address costs and limitations in cooling or power delivery.

4.6 Problem Modeling In this section, we describe the system, application, energy and scheduling models used in our study.

4.6.1 System Model The target system used in this work consists of a set P of p heterogeneous processors/machines that are fully interconnected. Each processor p j ∈ P is DVS-enabled; in other words, it can operate with different VSLs (i.e., different clock frequencies). For each processor p j ∈ P, a set V j of v VSLs is random and uniformly distributed among three different sets of VSLs (Table 4.3). Since clock frequency transition overheads take a negligible amount of time (e.g., 10μ s- 150μ s [9], [21]), these

86

Y. Kessaci et al.

overheads are not considered in our study. The inter-processor communications are assumed to perform with the same speed on all links without contentions. It is also assumed that a message can be transmitted from one processor to another while a task is being executed on the recipient processor which is possible in many systems.

Table 4.3 Voltage-relative speed pairs Pair 1

Pair 2

Pair 3

Level

Voltage (vk )

Relative speed (%)

Voltage (vk )

Relative speed (%)

Voltage (vk )

Relative speed (%)

0 1 2 3 4 5 6

1.5 1.4 1.3 1.2 1.1 1.0 0.9

100 90 80 70 60 50 40

2.2 1.9 1.6 1.3 1.0

100 85 65 50 35

1.75 1.4 1.2 0.9

100 80 60 40

4.6.2 Application Model Parallel programs can be generally represented by a directed acyclic graph (DAG). A DAG, G = (N, E), consists of a set N of n nodes and a set E of e edges. A DAG is also called a task graph or macro-dataflow graph. In general, the nodes represent tasks partitioned from an application; the edges represent precedence constraints. An edge (i, j) ∈ E between task ni and task n j also represents inter-task communication. A task with no predecessors is called an entry task, nentry , whereas an exit task, nexit , is one that does not have any successors. Among the predecessors of a task ni , the predecessor which completes the communication at the latest time is called the most influential parent (MIP) of the task denoted as MIP(ni ). The longest path of a task graph is the critical path. The weight on a task ni denoted as wi represents the computation cost of the task. In addition, the computation cost of the task on a processor p j , is denoted as wi, j and its average computation cost is denoted as w¯ i . The weight on an edge, denoted as ci, j represents the communication cost between two tasks, ni and n j . However, a communication cost is only required when two tasks are assigned to different processors. In other words, the communication cost when tasks are assigned to the same processor is zero and thus can be ignored. The earliest start time of, and the earliest finish time of, a task ni on a processor p j is defined as

4


EST (ni , p j ) =

87

0 i f ni = nentry EFT (MIP(ni ), pk ) + cMIP(ni ),i otherwise

EFT (ni , p j ) = EST (ni , p j ) + wi, j Note that the actual start and finish times of a task ni on a processor p j , denoted as AST (ni , p j ) and AFT (ni , p j ) can be different from its earliest start and finish times, EST (ni , p j ) and EFT (ni , p j ), if the actual finish time of another task scheduled on the same processor is later than EST (ni , p j ).

Fig. 4.5 A simple task graph

In the case of adopting task insertion the task can be scheduled in the idle time slot between two consecutive tasks already assigned to the processor as long as no violation of precedence constraints is made. This insertion scheme would contribute in particular to increasing processor utilization for a communication intensive task graph with fine-grain tasks. A simple task graph is shown in Fig. 4.5 with its details in Table 4.4 and Table 4.5. The values presented in Table 4.5 are computed using two frequently used task prioritization methods, t-level and b-level. Note that, both computation and communication costs are averaged over all nodes and links. The t-level of a task is defined as the summation of the computation and communication costs along the longest path of the node from the entry task in the task graph. The task itself is excluded from the computation. In contrast, the b-level of a task is computed by adding the computation and communication costs along the longest path of the task from the exit task in the task graph (including the task). The b-level is used in this study. The communication to computation ratio (CCR) is a measure that indicates whether a task graph is communication intensive, computation intensive or moderate. For a given task graph, it is computed by the average communication cost divided by the average computation cost on a target system.

88

Y. Kessaci et al.

Table 4.4 Computation cost with VSL 0 Task

p0

p1

p2

0 1 2 3 4 5 6 7

11 10 9 11 15 12 10 11

13 15 12 16 11 9 14 15

9 11 14 10 19 5 13 10

Table 4.5 Task Priorities Task

b-level

t-level

0 1 2 3 4 5 6 7

101.33 66.67 63.33 73.00 79.33 41.67 37.33 12.00

0.00 22.00 28.00 25.00 22.00 56.33 64.00 89.33

4.6.3 Energy Model Our energy model is derived from the power consumption model in complementary metal-oxide semiconductor (CMOS) logic circuits. The power consumption of a CMOS-based microprocessor is defined to be the summation of capacitive, shortcircuit and leakage power. The capacitive power (dynamic power dissipation) is the most significant factor of the power consumption. The capacitive power (Pc ) is defined as Pc = ACV 2 f (4.1) where A is the number of switches per clock cycle, C is the total capacitance load, V is the supply voltage, and f is the frequency. Equation (4.1) clearly indicates that the supply voltage is the dominant factor; therefore, its reduction would be most influential to lower power consumption. The energy consumption of the execution of a precedence-constrained parallel application used in this study is defined as n

n

i=0

i=0

E = ∑ ACVi2 f .w∗i = ∑ α Vi2 w∗i

4


89

where Vi is the supply voltage of the processor on which task ni is executed, and w∗i is the computation cost of task ni (the amount of time taken for ni ’s execution) on the scheduled processor.

4.6.4 Scheduling Model The task scheduling problem in this study is the process of allocating a set N of n tasks to a set P of p processors (without violating precedence constraints) that minimizes makespan with energy consumption as low as possible. The makespan is defined as M = max{AFT (nexit )} after the scheduling of n tasks in a task graph G is completed. Although the minimization of makespan is crucial, tasks of a DAG in our study are not associated with deadlines as in real-time systems.

4.7 A Case Study: A Parallel Evolutionary Algorithm In this section, our new parallel bi-objective hybrid approach is presented.

4.7.1 Hybrid Approach In our approach illustrated in Fig. 4.6, a solution (chromosome) is composed of a sequence of N genes. The ith gene of a solution s is denoted s j . Each gene is defined by a task, a processor and a voltage. These three parts of s j are denoted respectively t(s j ), p(s j ) and v(s j ). This means that the task t(s j ) is assigned to the processor p(s j ) with the voltage v(s j ).

Fig. 4.6 Our hybrid GA (GA and ECS)

90

Y. Kessaci et al.

The new approach we propose is based on ECS which is not a population-based heuristic. ECS tries to construct in a greedy way one solution using three components. • A first component to build the task parts of each gene of the solution. • A second component to build the processor and voltage parts of these genes. • And a third component to calculate the fitness of a solution in terms of energy consumption and makespan. Unlike ECS, our approach provides a set of Pareto solutions. This approach is a hybrid between a multi-objective GA and the second component of ECS. The role of the GA is to provide good task scheduling. In other words, the GA builds task parts t(s1 ), t(s2 ), ..., t(sn ) of a solution s. Therefore, the mutation and crossover operators of the GA affect only the task part of the genes of each solution. The second component of ECS is called whenever a solution is modified by these two operators. The first role of this component is to correct the task order to take into account the precedence constraints in the task graph. Then the component completes the processor and voltage parts of the genes of the partial solutions provided by these operators. In other words, ECS builds the remaining parts p(s1 ), p(s2 ), ..., p(sn ) and v(s1 ), v(s2 ), ..., v(sn ) of the partial solutions provided by the mutation and crossover operators of the GA. The evaluation (fitness) operator of the GA is called once the task, processor and voltage parts of each gene of the solution are known. The role of this operator is to calculate the energy consumption and the makespan of each solution. The mutation operator is based on the first component of ECS. This first component returns all tasks scheduled according to their b-level values. The principle of our mutation operator is also based on the scheduling of tasks according to their b-level values. The b-level concept is explained in Section 4.6.2. It should be noted that one can choose the t-level values instead of those of b-level. First, the operator chooses randomly two integers i and j such that 1 ≤ i < j ≤ n and b − level(t(si )) < b − level(t(s j )). Then, the operator swaps the two tasks t(si ) and t(s j ) (see Fig. 4.7).

Fig. 4.7 The mutation operator

4


91

As illustrated in Fig. 4.8, the crossover operator uses two solutions s1 and s2 to generate two new solutions s 1 and s 2. To generate s 1, the operator: • considers s1 as the first parent and s2 as the second parent. • randomly selects two integers i and j such that 1 ≤ i < j ≤ N. • copies in s 1 all tasks of s1 located before i or after j. These tasks are copied according to their positions (s 1k = s1k if k < i or k > j). • copies in a solution s all tasks of s2 that are not yet in s 1. Thus, the new solution s contains ( j − i + 1) tasks. The first task is at position 1 and the last task at the position ( j − i + 1). • and finally, copies all the tasks of s to the positions of s 1 located between i and j (s 1k = sk−i+1 for all i ≤ k ≤ j). The solution s 2 is generated with the same method by considering s2 as the first parent and s1 as the second parent.

Fig. 4.8 The crossover operator

The other elements of the GA in the new approach are conventional. Indeed, our GA randomly generates the initial population. Its selection operator is based on a tournament strategy. The algorithm stops when no new best solution is found after a fixed number of generations.

92

Y. Kessaci et al.

4.7.2 Insular Approach The island model [3] is inspired by behaviors observed in the ecological niches. In this model, several evolutionary algorithms are deployed to evolve simultaneously various populations of solutions, often called islands. As shown in Fig. 4.9, the GAs of our hybrid approach asynchronously exchange solutions. This exchange aims at delaying the convergence of the evolutionary process and to explore more zones in the solution space. For each island, a migration operator intervenes at the end of each generation. Its role consists to decide the appropriateness of operating a migration, to select the population sender of immigrants or the receiver of emigrants, to choose the emigrating solutions, and to integrate the immigrant ones.

Fig. 4.9 The cooperative island approach

4.7.3 Multi-start Approach Compared to the GA, ECS is more costly in CPU time. The different evaluations of ECS are independent of each other. Therefore, their parallel execution can make the approach faster. The objective of the hybrid approach is to improve the quality of solutions. The island approach also aims to obtain solutions of better quality. The goal of the parallel multi-start approach is to reduce the execution time. As shown in Fig. 4.10, our parallelization is based on the deployment of the approach using the farmer-worker paradigm. The GA processes are farmers and ECS processes are workers.

4


93

Fig. 4.10 Illustration of the multi-start approach

4.8 Experiments and Results This section presents the results obtained from our comparative experimental study. The experiments aim to demonstrate and evaluate the contribution of the hybridization, the insular approach and the multi-start approach respectively compared to ECS, the hybrid approach and the insular approach.

4.8.1 Experimental Settings The performance of our approach was thoroughly evaluated with the Fast Fourier Transformation [4] task graph which is a real-world application. A large number of variations were made on this task graph for more comprehensive experiments. Various different characteristics of processors were also applied. Table 4.6 summarizes the parameters used in our experiments. Table 4.6 Experimental parameters Parameter

Value

The number of tasks The number of processors Processor heterogeneity CCR

∼20 ∼40 ∼60 ∼80 ∼120 02 04 08 16 32 64 100 200 random 0.1 0.2 1.0 5.0 10.0

The new approach is experimented on about 10,000 instances distributed equitably according to the number of tasks, the number of processors, the processor heterogeneity and the CCR (1/5 of instances have a number of tasks equal to ∼20, 1/5 of instances have a number of tasks equal to ∼40,..., 1/6 of instances have a number of processors equal to 2, etc.). The processor heterogeneity value (i.e. 100 and 200) is defined to be the percentage of the speed difference between the fastest processor and the slowest processor in a given system, when the random one is randomly generated without any constraint [15]. Experiments have been performed on a grid of three clusters. A total of 714 cores are used. The first two clusters are located at the University of Mons in Belgium, c de Lille1 in France. while the third cluster is at UniversitÃ

94

Y. Kessaci et al.

Energy(%) Makespan(%)

60

Improvement(%)

50 40 30 20 10 0 0

5

10 15 The number of processors

20

25

Fig. 4.11 Improvement according to the number of processors 55


50

Improvement(%)

45 40 35 30 25 20 15 10 5 20

40

60

80

The number of tasks

Fig. 4.12 Improvement according to the number of tasks

100

120

4

Parallel Evolutionary Algorithms for Energy Aware Scheduling 80

95


70

Improvement(%)

60 50 40 30 20 10 0 0

2

4

6

8

10

CCR

Fig. 4.13 Improvement according to the CCR

4.8.2 Hybrid Approach The hybrid approach is experimented on all instances of Table 4.6. Each instance is solved twice. The first resolution is done with ECS, and the second resolution with the new approach. These experiments are launched by a script on one of the cores of our grid according to their availability. Experiments show that our approach improves on average the results obtained by ECS. Indeed, the energy consumption is reduced by 47.49% and the makespan reduced by of 12.05%. In addition, our experiments show clearly that the more processors there are, the more the new approach improves the results of ECS Fig. 4.11. The other improvements according to the number of tasks and CCR are given by respectively Fig. 4.12 and Fig. 4.13.

4.8.3 Insular Approach The objective of the following experiments is to show that our island approach improves the quality of the solutions provided by the hybrid approach. This insular approach is useful when solving large instances. Therefore, the experiments, presented in this section, focus only on the large instances of Table 4.6. The instances used are those with the number of tasks is 120, the number of processors is 64, the value of CCR is 10, and the heterogeneity of processors is 200 (20 instances). Each instance is solved using 1, 5, 10, 30 or 50 islands. The parameters of the experiments are given by Table 4.7 in the following:

96

Y. Kessaci et al.

Table 4.7 Island model parameters Parameters

Value

Island topology Nb of generation Migration rate Nb of migrants Population size Nb of islands

Ring topology 200 20 5 20 1, 5, 10, 30, 50

An insular approach with 1 island is equivalent to the hybrid approach. Fig. 4.14 illustrates the S-metric average values obtained with different numbers of islands. These values are normalized with the average value obtained by the experiments using 1 island. The S-metric measures the hyper-volume defined by a reference point and a Pareto front. It allows to evaluate the quality of a Pareto front provided by an algorithm. Experiments show that whatever the number of used islands the insular approach improves the Pareto front obtained with the hybrid approach. As shown in Fig. 4.14, the use of 50 islands, instead of 1 island (i.e. the hybrid approach), improves the S-metric of the obtained Pareto front by 26%. In Fig. 4.14, the more the number of islands is used, the better the results will be.

1.3

S-metric

1.25

S-metric

1.2 1.15 1.1 1.05 1 0

5

10

15

20 25 30 No. of islands

Fig. 4.14 S-metric value according to the number of islands

35

40

45

50

4


97

4.8.4 Multi-start Approach This section presents the experiments done to assess the quality of our multi-start approach. The parameters of the instances used in our experiments are: The CCR is 0.1, 0.5, 1.0, 5.0 or 10.0, the number of processors is 8, 32, or 64, and the heterogeneity of processors is 100, 200 or random. The population of the GA contains 20 chromosomes. Therefore, 21 computing cores are used to solve each instance (20 cores to run the ECSs and 1 core to run the GA). In our case, an experiment can not have a speedup greater than 21. The average speedup obtained is 13.06. Our experiments show that the speedup increases proportionally to the number of processors on which the precedenceconstrained parallel application is run as shown in Fig. 4.15. The CCR and the heterogeneity of processors do not impact significantly the quality of the acceleration of our approach.

Real speedup Theoretical maximum speedup

25

Speedup

20

15

10

5

0 10

20

30 40 50 The number of processors

60

Fig. 4.15 Speed up according to the number of processors

4.9 Conclusions In this chapter, we presented a classification of different methods used in literature to reduce energy consumption. Our classification is made according to three criteria. These criteria are the optimization method used to minimize the consumed energy (i.e. mono-objective or multi-objective optimization), the level of the system on which an approach is based (i.e. hardware or software level), and the type of system to which the approach is intended to be used (i.e. computing or embedded systems). As a use case and to give an example of a method, we presented a new parallel biobjective hybrid genetic algorithm to minimize energy consumption and makespan.

98

Y. Kessaci et al.

The energy saving of our approach exploits the dynamic voltage scaling (DVS). According to our classification, the new method can be considered as an optimizing multi-objective method with a Pareto approach. It uses the hardware part of the system in a dynamic way. Our method is intended to be used in computing systems. Our new approach has been evaluated with the Fast Fourier Transformation task graph which is a real-world application. Experiments show that our bi-objective meta-heuristic improves on average the results obtained in the literature (see [27], [2] and [7]) particularly in energy saving. Indeed, the energy consumption is reduced by 47.5% and the completion time by 12%. The experiments of the insular approach also show that the more the number of islands is used, the better the results will be. The use of 50 islands, instead of 1 island (i.e. the hybrid approach), improves the S-metric of the obtained Pareto front by 26%. Furthermore, the multi-start approach is on average 13 times faster than the island approach using 21 cores. Therefore, one of the main perspectives of the work presented in this chapter is to determine the solving approach to choose among ECS, the hybrid approach, and the insular approach, according to the precedence-constrained parallel application at hand. If the insular approach is chosen, the major issue is to determine the best number of islands to be used. This future work aims to minimize the total amount of consumed energy by the chosen solving approach and by the precedence-constrained parallel application to be solved. It is clear, for example, that the insular approach is interesting for the large and resource consuming precedence-constrained parallel applications and the applications intended to be executed several times.

Acknowledgments Experiments presented in this chapter were carried out using the Grid’5000 experimental testbed, being developed under the INRIA ALADDIN development action with support from CNRS, RENATER and several Universities as well as other funding bodies (see https://www.grid5000.fr). Clusters of University of Mons were also used. We would like to thank the technical staffs of the Grid’5000 and the clusters of University of Mons for making their clusters accessible and fully operational.

References 1. Berl, A., de Meer, H.: A virtualized energy-efficient office environment. In: Proceedings of the 1st International Conference on Energy-Efficient Computing and Networking, eEnergy 2010, pp. 11–20. ACM, New York (2010) 2. Bozdag, D., Catalyurek, U., Ozguner, F.: A task duplication based bottom-up scheduling algorithm for heterogeneous environments. In: Proc. Int. Parallel and Distributed Processing Symp. (2005) 3. Cohoon, J.P., Hedge, S.U., Martin, W.N., Richards, D.: Punctuated equilibria: A parallel genetic algorithm. In: Grefenstette, J.J., Lawrence Erlbaum Associates (eds.) Proceedings of the Second International Conference on Genetic Algorithms, p. 148 (1987) 4. Cormen, T.H., Leiserson, C.E., Rivest, R.L.: Introduction to algorithms. MIT Press, Cambridge (1990)

4


99

5. de Langen, P., Juurlink, B.: Trade-offs between voltage scaling and processor shutdown for low-energy embedded multiprocessors. In: Vassiliadis, S., Bereković, M., Hämäläinen, T.D. (eds.) SAMOS 2007. LNCS, vol. 4599, pp. 75–85. Springer, Heidelberg (2007) 6. Freeh, V.W., Kappiah, N., Lowenthal, D.K., Bletsch, T.K.: Just-in-time dynamic voltage scaling: Exploiting inter-node slack to save energy in mpi programs. J. Parallel Distrib. Comput. 68, 1175–1185 (2008) 7. Garey, M.R., Johnson, D.S.: Computers and intractability: A guide to the theory of npcompleteness, pp. 238–239. W.H. Freeman and Co., New York (1979) 8. Hlavacs, H., Weidlich, R., Hummel, K., Houyou, A., Berl, A., de Meer, H.: Distributed energy efficiency in future home environments. Annals of Telecommunications 63, 473– 485 (2008), doi:10.1007/s12243-008-0045-2 9. Intel pentium m processor datasheet (2004) 10. Kang, J., Ranka, S.: Energy-efficient dynamic scheduling on parallel machines. In: Sadayappan, P., Parashar, M., Badrinath, R., Prasanna, V.K. (eds.) HiPC 2008. LNCS, vol. 5374, pp. 208–219. Springer, Heidelberg (2008) 11. Khan, S.U., Ahmad, I.: A cooperative game theoretical technique for joint optimization of energy consumption and response time in computational grids. IEEE Transactions on Parallel and Distributed Systems 20(3), 346–360 (2009) 12. Kimura, H., Sato, M., Hotta, Y., Boku, T., Takahashi, D.: Emprical study on reducing energy of parallel programs using slack reclamation by dvfs in a power-scalable high performance cluster. In: IEEE International Conference on Cluster Computing, pp. 1–10 (2006) 13. Koch, G.: Discovering multi-core: Extending the benefits of moore’s law. Technology Intel Magazine (2005) 14. Koomey, J.G.: Estimating total power consumption by servers in the u.s. and the world 15. Choon Lee, Y., Zomaya, A.Y.: “Minimizing energy consumption for precedenceconstrained applications using dynamic voltage scaling. In: CCGRID 2009: Proceedings of the 2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid, pp. 92–99 (2009) 16. Lee, Y., Zomaya, A.Y.: Energy efficient utilization of resources in cloud computing systems. The Journal of Supercomputing, 1–13 (2010), doi:10.1007/s11227-010-0421-3 17. Lee, Y.C., Zomaya, A.Y.: Minimizing energy consumption for precedence-constrained applications using dynamic voltage scaling. In: Proceedings of the 2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid, CCGRID 2009, pp. 92– 99. IEEE Computer Society, Washington, DC, USA (2009) 18. Lin, M., Ding, C.: Parallel genetic algorithms for DVS scheduling of distributed embedded systems. In: Perrott, R., Chapman, B.M., Subhlok, J., de Mello, R.F., Yang, L.T. (eds.) HPCC 2007. LNCS, vol. 4782, pp. 180–191. Springer, Heidelberg (2007) 19. Liu, C., Qin, X., Kulkarni, S., Wang, C., Li, S., Manzanares, A., Baskiyar, S.: Distributed energy-efficient scheduling for data-intensive applications with deadline constraints on data grids. In: IEEE International Performance, Computing and Communications Conference, IPCCC 2008, pp. 26–33 (2008) 20. Miao, L., Qi, Y., Hou, D., Wu, C.L., Dai, Y.H.: Energy saving task scheduling for heterogeneous cmp system based on multi-objective fuzzy genetic algorithm. In: IEEE International Conference on Systems, Man and Cybernetics, SMC 2009, pp. 3923–3927 (2009) 21. Min, R., Furrer, T., Chandrakasan, A.: Dynamic voltage scaling techniques for distributed microsensor networks. In: Proc. IEEE Workshop on VLSI, pp. 43–46 (2000) 22. Nathuji, R., Schwan, K.: Virtualpower: Coordinated power management in virtualized enterprise systems. In: Proceedings of Twenty-First ACM SIGOPS Symposium on Operating Systems Principles, SOSP 2007, pp. 265–278. ACM, New York (2007)

100

Y. Kessaci et al.

23. Rizvandi, N.B., Taheri, J., Zomaya, A.Y., Lee, Y.C.: Linear combinations of dvfs-enabled processor frequencies to modify the energy-aware scheduling algorithms. In: Proc. of IEEE International Symposium on Cluster Computing and the Grid, pp. 388–397 (2010) 24. Ruan, X., Qin, X., Zong, Z., Bellam, K., Nijim, M.: An energy-efficient scheduling algorithm using dynamic voltage scaling for parallel applications on clusters. In: Proceedings of 16th International Conference on Computer Communications and Networks, ICCCN 2007, pp. 735–740 (2007) 25. Simunic, T., Benini, L., Acquaviva, A., Glynn, P., De Micheli, G.: Dynamic voltage scaling and power management for portable systems. In: Proceedings of the 38th Annual Design Automation Conference, DAC 2001, pp. 524–529. ACM, New York (2001) 26. Srikantaiah, S., Kansal, A., Zhao, F.: Energy Aware Consolidation for Cloud Computing. In: Proceedings of HotPower 2008 Workshop on Power Aware Computing and Systems. USENIX (2008) 27. Topcuouglu, H., Hariri, S., Wu, M.Y.: Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans. Parallel Dist. Systems 13(3), 260– 274 (2002) 28. Zhuo, J., Chakrabarti, C.: Energy-efficient dynamic task scheduling algorithms for dvs systems. ACM Trans. Embed. Comput. Syst. 7(17), 1–25 (2008)

Chapter 5

Biologically-inspired Methods and Game Theory in Multi-criterion Decision Processes Paweł Jarosz and Tadeusz Burczyński

Abstract. In order to make decisions in multi-criteria environments there is a need to find solutions with compromises. All solutions which are compromises for all criteria create the set of solutions named the Pareto frontier. Based on these possibilities the decision maker can choose the best solution looking at the current preferences. It is not trivial to solve such a multiobjective problem, however there are several methods which try to do it, with differing degrees of success. Methods based on behaviors of different biological phenomena belong to the most successful set. Genetic and evolutionary algorithms and artificial immune systems are included that group of methods. On the other hand, the game theory is a branch which mathematically analyzes conflict situations. This chapter is dedicated to methods of finding solutions in multi-criteria environments using bio-inspired methods and the game theory, as well as coupling both approaches to create a new intelligent hybrid system of decision making.

5.1 Introduction Multi-criteria decision making is a sophisticated process of finding a solution in an environment with several objectives. Most of problems, which occur in the real life, consist of several objectives which should be taken into account. Solving such problems is a challenging task and has been deeply investigated. However, in real applications often simple methods are used. Market’s requirements lead to situations in Paweł Jarosz Cracow University of Technology, Institute of Computer Science e-mail: [email protected] Tadeusz Burczyński Cracow University of Technology, Institute of Computer Science, Silesian University of Technology, Department for Strength of Materials and Computational Mechanics e-mail: [email protected]


102

P. Jarosz and T. Burczyński

which solutions accepted by decision makers are not really optimal solutions. There are two reasons for such situation. First, the simple algorithms are easy to understand and implement. However, they have several disadvantages especially in terms of solutions feasibility and determining local optima. On the other hand, algorithms which do not have these disadvantages are very time consuming, providing the second reason for simple methods. The big challenge faced by researchers is how to create effective algorithms which will lead to optimal solutions. That is why there are several attempts to implement biologically inspired algorithms, which succeed solving single-objective problems. Because of different definitions of optimality and different kinds of expected results (not only one global optimum, but a set of compromise solutions) transferring these bio-inspired algorithms to multiobjective cases is not easy. Currently available methods are not fully acceptable by real life customers, so there is a need for new approaches, reducing the amount of computation time and creating better solutions. One area which has been explored very earnestly is evolutionary computations. Since the1980s new methods based on ideas of genetic algorithms have been appearing and have been presenting new features [16], [9]. Several topics remain open in this area, especially how individuals should be evaluated, how crossovers and mutations is implemented, what the structure of the population is and where the result set is stored. New ideas for addressing these problems and topics are included in methods presented by researchers today. Some of algorithms are quite competitive with other methods [18], [21]. Another area of interest has been Artificial Immune Systems . This is a more recent concept, with new methods and algorithms for multiobjective optimization appearing since about only ten years ago. Several interesting attempts have been described [1], [10]. The , on the other hand, is a branch which analyzes conflict situations and tries to solve such kinds of problems. There have been some attempts to couple the general ideas from the game theory with bio-inspired methods to improve the quality of result solutions [17], [12]. In this work authors try to define a multiobjective optimization problem , which is a very important step in the multi-criteria decision-making process. Then they briefly survey on evolutionary and immune methods for multiobjective problems. Next interesting attempts to couple the game theory with these biological methods are presented. Some numerical tests of chosen algorithms for some traditional testing functions are described at the end of this work.

5.2 Multi-criteria Decision Making and Multiobjective Optimization The multi-criteria decision making, which is analyzed in this work, is one of the most interesting kinds of optimization, while also being the most difficult one. The basic terms should be introduced in here, to start the further discussion.

5

Biologically-inspired Methods and Game Theory

103

The most important elements while defining the multiobjective problem are the goals, criteria or just objective functions. Objectives are written as f1 (x), f2 (x), ..., fk (x), where k is number of criteria in the problem. The space, where the values of objectives are analyzed is called the fitness space or the objective space. The vector of decision variables can be stated as x x = [x1 , x2 , x3 ..., xn ]

(5.1)

where n is the number of decision variables which are sought. A set of equality and non-equality constraints is imposed on decision variables: gi (x) f(ai) ci* replace ai

Find the worst individuals in Ab and replace them with new ones

Stop criteria?

YES STOP

Fig. 5.2 General flowchart of the Clonal Selection Algorithm

110


Artificial Immune Systems and especially the Clonal Selection algorithm have also been applied to multiobjective optimization. A few attempts are described below. Multiobjective Immune System Algorithm (MISA) Coello Coello and Cortez presented the MISA algorithm [1]. In this method, which is based on the clonal selection principles, solutions are represented by binary strings. The method uses the Pareto dominance and feasibility in order to evaluate solutions. An elitist mechanism is also implemented by introducing an external population to store non-dominated solutions. In order to ensure uniform distribution of solutions to create the Pareto front an adaptive grid is implemented. Another goal of such a grid is to limit secondary population size. A simple mechanism to deal with constraints is included in this algorithm. This solution looks quite valuable, as it addresses some very important multiobjective optimization issues. Authors performed several numerical tests with this algorithm. Multi-Objective Immune Algorithm (MOIA) Luh and Chueh [14] have presented a strongly biologically inspired algorithm for solving constrained multiobjective optimization problems. It is quite a complex method based on such terms as the clonal selection, DNA library, heavy and light protein chains and interleukin interactions. To the most important features of this algorithm belong: binary representation with heavy and light chain part (respectively most and least significant bits), transformation of constrained problems into non-constrained using the idea of penalization, and the avidity of solutions counts performance of the solution as well as similarity between them. Besides the traditional benchmarks problems, this method was used to design truss structures [14]. Vector Artificial Immune System (VAIS) Freschi [7] has presented another approach for solving multiobjective optimization problems based on Artificial Immune Systems. This algorithm makes parallel searching for optima and ensures diversity of results by applying suppression. The algorithm introduces an interesting way of calculating fitness values. Based on Pareto dominance each solution has its strength. The strengths sum of all solutions which dominate particular one is its fitness value. For non-dominated solutions, fitness is their strength. The author has compared this algorithm with results of NSGA-II.

5


111

Adaptive Clonal Selection Algorithm for Multiobjective Optimization (ACSAMO) In the algorithm presented by Wang [8] a standard weighted fitness function is defined. Weights are changed systematically during the algorithm execution. Then the cloning and the mutation are performed. The affinity of solutions is used in the process of the mutation and the non-dominated sorting and a crowding distance is used to perform selection. The authors present several benchmarks and comparisons with standard algorithms like NSGA-II and SPEA.

5.3.3 Game Theory The game theory deals with anlysis of conflict situations. This is a mathematical area, but the considerations also apply to the economics, psychology and recently also in engineering. The work, which is assumed to be the beginning of the game theory is accredited to John von Neumann and Oskar Morgensern [15]. In the game theory, the conflict situations are named games. They have several essential features: • number of players is known and finite • full game rules are set • payoffs are determined by payoff functions, which depend on all the players The possible players’ moves usually are named strategies. A strategy is a complete description of a player’s activity in all situations. Games can be described as follows: Γ = (Σ , π ) - N-player game in normal form I. ∑i - set of i player strategies 1 ≤ i ≤ N II. set of states in game Γ is defined as

∑ = ∑1 × ∑2 × ... × ∑i

(5.5)

πi : ∑1 × ∑2 × ... × ∑i → ℜ

(5.6)

(π1 (s), π2 (s), ..., πn (s)) ∈ ℜN s = (s = s1 , s2 , ..., sn )

(5.7)

III. i player payoff function

IV. payoff vector of state

V. ∑i - vector of values which a player can choose

∑1 = [1, 2, 3]∑2 = [4, 5, 6]

(5.8)

112


VI. ∑ - set of all states ∑1 × ∑2

∑ = ((1, 4), (1, 5), (1, 6)(2, 4), (2, 5), (2, 6)(3, 4), (3, 5), (3, 6))

(5.9)

VII. πi define the payoff for each state. Strategy sdi player i is named dominated strategy when inequality

πi (x1 , ..., xN ) ≤ πi (x1 , ..., xi−1 , sdi , xi+1 , ..., xN )

(5.10)

is true for all (x1 , ..., xN ) ∈ ∑ If in a set of strategies ∑i a dominating strategy exists, so it is non-dominated and each non-dominated strategy is equivalent to it. Strategy s is dominated state by s if the following inequality is strict at least for one i s ≤ s ⇒ πi (s) ≤ πi (s ) (5.11) States, which are non-dominated by any other state are named the equilibrium state or states optimal is Pareto sense. An element (s∗1 , s∗2 , ..., s∗N ) of set ∑ is named the equilibrium or Nash equilibrium state in the game Γ if inequality

πi (s∗1 , ..., s∗i−1 , si , s∗i+1 , ..., s∗N ) ≤ πi (s∗1 , ..., s∗N )

(5.12)

is true for all si ∈ ∑i , i ∈ (1, ..., N) . E(Γ ) will be named set of all equilibria of the game Γ s∗i is the best answer player i for strategies s∗1 , ..., s∗i−1 , s∗i+1 , ..., s∗N of other players. Such a state is acceptable for all players. It is important to note the similarities between the game theory definitions and the multiobjective optimization theory described earlier. Looking at these a multiobjective problem can be considered as a game.

5.3.4 Hybrid Evolutionary-Game Algorithm Periaux and his co-workers [5] created an algorithm based on the Nash equilibrum and evolutionary methods. Standard evolutionary algorithms for finding the Pareto optimal solutions generally require large numbers of fitness evaluations to find the set of non-dominated solutions. When the Nash- equilibrium is used as a seed for an evolutionary algorithm, it can increase the efficiency of the system. The hybridization of the Nash-Game and MOEA (Multi-Objective Evolutionary Algorithm) can speed up the search process. The authors present a detailed algorithm called the Hybrid Game, which introduces several players who have their own criteria. After finding the Nash solution, it is used as a seed in the HAPMOEA algorithm. The Hybrid-Game consists of one Pareto-player and several Nash-players providing dynamic elite information to the Pareto algorithm. Hence it can produce a Nash-equilibrium and Pareto non-dominated solutions simultaneously. It shows

5


113

Fig. 5.3 Schema of Hybrid Game algorithm

how a Nash-game acts as a pre-conditioner of the Pareto algorithm to speed up the capture of the Pareto frontier. The general schema of the Hybrid Game algorithm is presented in Fig. 5.3. Each Nash player has its own decision variables and uses its own optimization strategy. During their actions, the Nash players exchange their optimized design variables. The results are sent to the main algorithm - Pareto player. The system was tested successfully for aerodynamic design problems.

5.3.5 Immune Game Theory MultiObjective Algorithm IMGAMO The metaphor of the game theory and immunology is used to solve problems of multiobjective optimization [12]. Each player has its own objective (a payoff function in the Nash equilibrium). The Nash strategy for a particular player is the optimum solution for this player’s problem remembering that other players also play their best strategies. The solution of the optimized problem consists of several variables, each of which is assigned to one of the players. Each player optimizes only its decision variables (its strategy) taking the rest of them as unvarying. The rest of decision variables are set by taking the best solutions the from other players. Then all players use the immune algorithm to optimize their objectives. The general idea of the algorithm is presented in Fig. 5.4. The most important assumptions to the IMGAMO algorithm are as follows: • each player has its own fitness function (payoff function), • each player has assigned a part of decision variables of the solution (strategy of this player),

Criterium 1

CLONAL SELECTION ALGORITHM

ASSIGN PARAMETERS TO PLAYERS

Criterium N Criterium 2


STORE SOLUTIONS TO THE SECONDARY POPULATION

Player N


Player 2


Player 1

114

Fig. 5.4 General schema of Immune Game Theory Multi Objective algorithm

• the rest of decision variables are set as unvarying and taken from the best solutions from the other players, • all solutions are coded with real values, • the result of the algorithm (the determined Pareto frontier) is stored in the result_population, • each player uses the immune algorithm to optimize its objective. Parameters of the algorithm: • • • •

G - number of fitness functions in problem, pop_size - size of the population for each player, clon_number - maximum number of clones in clonal selection, exchange_factor - iteration number when an exchange decision variables between player is performed, • iterations - number of iterations of the algorithm.

5


115

Fig. 5.5 IMGAMO algorithm

The clonal selection is an element of the algorithm which is based on the Nash equilibrium. Each player searches for optimal values of his decision variables to receive maximum payoff, taking other players’ best variables values as unvarying. During the clonal selection the solutions which are best take into account all criteria searched and they approach the Pareto frontier. The goal of the suppression is to diversify of the solutions in populations. The Pareto frontier must fulfill two conditions. The first is to be as close as possible to the real frontier, for which clonal selection is used. Second it should be regularly distributed on this frontier. This is the task of the supression.

116


Fig. 5.6 Clonal selection algorithm

5.4 Numerical Tests In this work the results obtained by the IMGAMO algorithm in three classical difficult multiobjective test problems are presented. In order to evaluate it, comparisons to the MISA and NSGA 2 algorithms have been performed. MISA is representative of the Artificial Immune Systems, while NSGA II is the representative of evolutionary algorithms. For each problem the results are compared using two metrics: a spacing and a inverted generational distance. The parameters used in all experiments were adapted to evaluate fitness functions approximately 12000 times. The IMGAMO parameters are presented in Table 1.

5


117

Fig. 5.7 Pareto frontiers obtained using the tested algorithms for solving ZDT2 problem

Fig. 5.8 Pareto fronts obtained using the tested algorithms for solving ZDT2 problem - zoom of the fragment of the front

118


Table 5.1 The parameters for the IMGAMO algorithms IMGAMO iterations: 100 clon_number: 15 population_size: 100 exchange_factor: 3

5.4.1 The ZDT2 Problem A two-objective five-decision variable problem proposed by Ziztler, Deb and Thiele [20] was examined. Equation (5.13) defines the problem. minimize f1 (x) = x1 x1 2 minimize f2 (x) = g(x) 1 − g(x) n ∑i=2 xi g(x) = 1 + 9 n−1

(5.13)

where 0 ≤ xi ≤ 1, i = 1..n, n = 5 The true Pareto frontier has a non-convex Pareto optimal frontier and consists of the one curve (Fig.5.7 and Fig. 5.8). The values of metrics are presented in Table 2, Table 3 and Table 4. Results obtained by the IMGAMO and NSGA2 algorithms for the ZDT2 problem were very rewarding. The determined Pareto frontier was very close to the real frontier. There is a slight difference in metrics: GD was better for the NSGA2 and S was better for IMGAMO. The MISA algorithm did not lead to good results.

5.4.2 The ZDT3 Problem Another problem proposed by Zitzler et al. [20] was examined. Equation (5.14) defines that problem. minimize f1 (x) = x1 minimize f2 (x) = g(x) · x1 x1 · 1 − g(x) − sin(10π x1 ) g(x) n ∑i=2 xi g(x) = 1 + 9 n−1 where 0 ≤ xi ≤ 1, i = 1..n, n = 10

(5.14)

5


119

Fig. 5.9 Pareto fronts obtained using the tested algorithms for solving ZDT3

Fig. 5.10 Pareto fronts obtained using the tested algorithms for solving ZDT3 - zoomed fragment of the front

120


The optimal frontier consists of the several noncontinuous convex parts (Fig. 5.9, Fig.5.10). The values of metrics are presented in Table 2, Table 3 and Table 4. In the case of the ZDT3 problem the IMGAMO and NSGA2 algorithm again had performed quite well. The value of GD was better for the NSGA2 algorithm and S was better for the IMGAMO algorithm.

5.4.3 The ZDT6 Problem The problem proposed by Zitzler et al. [20] where the true Pareto frontier consists of one non-convex curve. Equation (5.15) defines the problem. minimize f1 (x) = 1 − e−4x1 sin6 4π x1 f1 (x) 2 minimize f2 (x) = g(x) 1 − g(x) n 0.25 ∑ xi g(x) = 1 + 9 i=2 n−1

(5.15)

where 0 ≤ xi ≤ 1, i = 1..n, n = 5 This is the most difficult problem among those analyzed. The Pareto-optimal solutions are not uniformly distributed along the global Pareto frontier and the density of the solutions is lower close to the global Pareto frontier (Fig. 5.11, Fig. 5.12). The values of metrics are presented in Table 2, Table 3 and Table 4.

Fig. 5.11 Pareto fronts obtained using the tested algorithms for solving ZDT6

5


121

Fig. 5.12 Pareto fronts obtained using the tested algorithms for solving ZDT6 - zoomed fragment of the front

In the case of the ZDT6 problem - the most difficult of the problems - the IMGAMO led to the best results. The NSGA2 algorithm yielded worse results, but the MISA algorithm still fared the worst. Table 5.2 The values of the metrics for ZDT2, ZDT3 and ZDT6 problems using MISA algorithm MISA GD

S

ZDT 2 Avg 1,06E-01 1,78E-01 Max 1,80E-01 3,58E-01 Min 6,99E-02 6,99E-02 ZDT 3 Avg 1,82E-01 1,64E-01 Max 2,55E-01 5,80E-01 Min 1,00E-01 9,40E-04 ZDT 6 Avg 6,12E-01 1,68E-01 Max 7,98E-01 3,20E-01 Min 5,10E-01 8,75E-02

122


Table 5.3 The values of the metrics for ZDT2, ZDT3 and ZDT6 problems using NSGA 2 algorithm NSGA2 GD

S

ZDT 2 Avg 1,20E-04 3,08E-02 Max 1,37E-04 1,08E-01 Min 1,00E-04 9,40E-03 ZDT 3 Avg 9,50E-04 1,07E-02 Max 1,00E-03 1,31E-02 Min 8,00E-04 4,20E-03 ZDT 6 Avg 6,52E-05 5,01E-03 Max 7,16E-05 5,60E-03 Min 6,12E-05 4,20E-03

Table 5.4 The values of the metrics for ZDT2, ZDT3 and ZDT6 problems using IMGAMO algorithm IMGAMO GD

S

ZDT 2 Avg 3,56E-03 1,57E-03 Max 7,50E-03 2,66E-03 Min 3,61E-06 2,68E-04 ZDT 3 Avg 0,0380 Max 0,1350 Min 0,0004

0,0847 0,2090 0,0075

ZDT 3 Avg 8,52E-03 1,18E-03 Max 1,14E-02 1,75E-03 Min 5,27E-03 7,70E-04

5.5 Concluding Remarks An overview of biologically inspired methods used for solving multiobjective optimization was presented. The most popular evolutionary algorithms as well as

5


123

Artificial Immune Systems were described. A very interesting approach of searching is the coupling these algorithms with the game theory paradigms, and two algorithms of such coupling were presented. The results of some tests which compared the performances of hybrid algorithm (IMGAMO), evolutionary algorithm (NSGA II) and immune algorithm (MISA) were also included. Looking at the results one can state that IMGAMO and NSGA2 algorithms are quite competitive. The another algorithm MISA - did not lead to good results in any of analyzed problems. Results of tests presented in this work as well as these not included here shows that new hybrid approaches are promising and rewarding.

Acknowledgments This work was supported by the Polish Ministry of Science and Higher Education under grant No. N N519 405437.

References 1. Coello, C.A., Cortés, N.C.: Solving multiobjective optimization problems using an artificial immune system. Genetic Programming and Evolvable Machines 6(2), 163–190 (2005) 2. Coello Coello, C.A., Lamont, G.B., Van Veldhuizen, D.A.: Evolutionary Algorithms for Solving Multi-Objective Problems (Genetic and Evolutionary Computation). SpringerVerlag New York, Inc, Secaucus (2006) 3. Dasgupta, D.: Advances in artificial immune systems. IEEE Computational Intelligence Magazine 1(4), 40–49 (2006) 4. Deb, K., Agrawal, S., Pratap, A., Meyarivan, T.: A fast elitist non-dominated sorting genetic algorithm for multi-objective optimisation: Nsga-ii. In: Deb, K., Rudolph, G., Lutton, E., Merelo, J.J., Schoenauer, M., Schwefel, H.-P., Yao, X. (eds.) PPSN 2000. LNCS, vol. 1917, pp. 849–858. Springer, Heidelberg (2000) 5. Gonzalez, L.F., Srinivas, K., Seop, D., Lee, C., Periaux, J.: Coupling hybrid-game strategies with evolutionary algorithms for multi-objective design problems in aerospace. In: Evolutionary and Deterministic Methods for Design, Optimization and Control with Applications to Industrial and Societal Problems, pp. 221–248 (2011) 6. Fonseca, C.M., Fleming, P.J.: Genetic algorithms for multiobjective optimization: Formulationdiscussion and generalization. In: Proceedings of the 5th International Conference on Genetic Algorithms, pp. 416–423. Morgan Kaufmann Publishers Inc., San Francisco (1993) 7. Freschi, F.: Multi-Objective Artificial Immune System for Optimization in Electrical Engineering. PhD thesis, Torino, Italy (2006) 8. Gao, J., Wang, J.: Wbmoais: A novel artificial immune system for multiobjective optimization. Comput. Oper. Res. 37(1), 50–61 (2010) 9. Goldberg, D.E.: Genetic Algorithms in Search, Optimization and Machine Learning, 1st edn. Addison-Wesley Longman Publishing Co., Inc., Boston (1989) 10. Gong, M., Jiao, L., Du, H., Bo, L.: Multiobjective immune algorithm with nondominated neighbor-based selection. Evol. Comput. 16(2), 225–255 (2008)

124


11. Jarosz, P., Burczyński, T.: Immune algorithm for multi-modal optimization - numerical tests in intelligent searching. In: Recent Developments in Artificial Intelligence Methods (2004) 12. Jarosz, P., Burczyński, T.: Coupling of immune algorithms and game theory in multiobjective optimization. In: Artifical Intelligence and Soft Computing, pp. 500–507 (2010) 13. Alam, S., Bui, L.T.: An introduction to multi-objective optimization. In: Multi-Objective Optimization in Computational Intelligence: Theory and Practice, pp. 1–19 (2008) 14. Luh, G.-C., Chueh, C.-H., Liu, W.-W.: MOIA: Multi-Objective Immune Algorithm. Engineering Optimization 35(2), 143–164 (2003) 15. Von Neumann, J., Morgenstern, O.: Theory of Games and Economic Behavior. Princeton University Press, Princeton (1944) 16. Schaffer, J.D.: Multiple objective optimization with vector evaluated genetic algorithms. In: Proceedings of the 1st International Conference on Genetic Algorithms, pp. 93–100. L. Erlbaum Associates Inc., Hillsdale (1985) 17. Sefrioui, M., Periaux, J.: Nash genetic algorithms: Examples and applications. In: Proceedings of the 2000 Congress on Evolutionary Computation CEC 2000, pp. 509–516. IEEE Press, California (2000) 18. Srinivas, N., Deb, K.: Muiltiobjective optimization using nondominated sorting in genetic algorithms. Evol. Comput. 2, 221–248 (1994) 19. Wierzchoń, S.T.: Function optimization by the immune metaphor. Task Quarterly 6 (2002) 20. Zitzler, E., Deb, K., Thiele, L.: Comparison of multiobjective evolutionary algorithms: Empirical results. Evolutionary Computation 8, 173–195 (2000) 21. Zitzler, E., Thiele, L.: Multiobjective evolutionary algorithms: A comparative case study and the strength Pareto approach (1999)

Chapter 6

Advanced Planning in Vertically Integrated Supply Chains Maksud Ibrahimov, Arvind Mohais, Sven Schellenberg, and Zbigniew Michalewicz

Abstract. During the last few years most production-based businesses have been under enormous pressure to improve their top-line growth and bottom-line savings. As a result, many companies are turning to systems and technologies that can help optimise their supply chain activities and improving short- and long-term demand forecasting. Given the inherent complexities of planning and scheduling in vertically integrated supply chains, many new methods (e.g., ant systems, evolutionary algorithms, fuzzy systems, genetic algorithms, neural networks, rough sets, swarm intelligence, simulated annealing, tabu search – collectively known as “Computational Intelligence” methods) have been introduced into software applications to help manage and and optimise this complexity. In this paper we discuss two realworld applications of advanced planning: one from wine industry and the other – from mining industry. Maksud Ibrahimov School of Computer Science, University of Adelaide, South Australia 5005, Australia e-mail: [email protected] Arvind Mohais SolveIT Software, Pty Ltd., 99 Frome Street, Adelaide, SA 5000 Australia e-mail: [email protected] Sven Schellenberg SolveIT Software, Pty Ltd., 99 Frome Street, Adelaide, SA 5000 Australia e-mail: [email protected] Zbigniew Michalewicz School of Computer Science, University of Adelaide, South Australia 5005, Australia. Institute of Computer Science, Polish Academy of Sciences, ul. Ordona 21, 01-237 Warsaw, Poland, Polish-Japanese Institute of Information Technology, ul. Koszykowa 86, 02-008 Warsaw, Poland e-mail: [email protected] P. Bouvry et al. (Eds.): Intelligent Decision Systems, SCI 362, pp. 125–148. c Springer-Verlag Berlin Heidelberg 2011 springerlink.com

126

M. Ibrahimov et al.

6.1 Introduction In managing supply chain thousands of individual decisions have to be made at regular intervals. These decisions are of different scope and significance (e.g., operational, tactical, strategic); there are also countless trade-offs between various decisions. Further, supply chain networks are extremely complex and they are set in constrained and dynamic environment, with many (usually conflicting) objectives. Any successful implementation of Advance Planning System for such complex network should be capable of addressing several key issues; these include dealing with huge search spaces, many objectives, variable constraints, and variability and uncertainty . Due to the high level of complexity, it becomes virtually impossible for deterministic systems or human domain experts to find an optimal solution – not to mention that the term "‘optimal solution"’ loses its meaning in multi-objective environment, as often we can talk only about trade-offs between different solutions. Moreover, the manual iteration and adjustment of scenarios (what-if scenarios and trade-off analysis), which is needed for strategic planning, becomes an expensive, if not unaffordable, exercise. Many texts on Advance Planning and Supply Chain Management (e.g., [28]) describe several commercial software applications (e.g., AspenTech – aspenONE, i2 Technologies – i2 Six.Two, Oracle – JDEdwards EnterpriseOne Supply Chain Planning, SAP – SCM, and many others), which emerged mainly in 1990s. However, it seems that the areas of supply chain management in general, and advanced planning in particular, are ready for a new genre of applications which are based on Computational Intelligence methods. Many supply chain related projects run at large corporations worldwide failed miserably (projects that span a few years and cost many millions). In [28] the authors wrote: “In recent years since the peak of the e-business hype Supply Chain Management and especially Advanced Planning Systems were viewed more and more critically by industry firms, as many SCM projects failed or did not realize the promised business value.”

The authors also identified three main reasons for such failures: • the perception that the more you spend on IT (e.g., APS) the more value you will get from it, • an inadequate alignment of the SCM concept with the supply chain strategy, and • the organizational and managerial culture of industry firms. While it is difficult to argue with the above points, it seems that the forth (and unlisted) reason is the most important: maturity of technology. Small improvements and upgrades of systems created in 1990s do not suffice any longer for solving companies’ problems of 21st Century. A new approach is necessary which would combine seamlessly the forecasting, simulation, and optimization components in a new architecture. Further, many existing applications are not flexible enough in the sense that they cannot cope with any exceptions, i.e., it is very difficult, if not impossible, to include some problem-specific features – and most businesses have some unique features which need tp be included in the underlying model, and are not

6

Advanced Planning in Vertically Integrated Supply Chains

127

adequately captured by off-the-shelf standard applications. Thus the results are often not realistic and the team of operators return to their spreadsheets and whiteboards rather than to rely on unrealistic recommendations of the software. Several studies have investigated optimisation techniques for various supply chain components, including the job-shop scheduling problem [7, 31, 10], planning and cutting problems [20, 19], routing problems [30], allocation and distribution problems [9, 36], to name a few. Many algorithms were developed to solve various supply chain components in a single silo environment [22, 32, 35]. However, optimisation of each silo without considering relationships with other silos usually does not lead to a globally optimal solution of the whole supply chain. Because of that, large businesses started to become more interested in optimisation of their whole system rather than the optimisation of single components of the system. This approach is commonly referred in the literature as coordinated supply chain management. L. Davis, one of the experts in supply chain optimization, describes this situation as follows [11]: “...companies should be able to use solutions to optimise across their sourcing and procurement, production and distribution processes all at the same time. Instead, supply chain solutions tend to break those functions out into separate modules, each of which runs separately from the others. The answer you get when you first do production and then do distribution or any of these functions independently is not as good as the answer you get when you do them together.”

And Ackoff [1] correctly identified this issue over 30 years ago: “Problems require holistic treatment. They cannot be treated effectively by decomposing them analytically into separate problems to which optimal solutions are sought.”

In this chapter we discuss two real-world applications of advanced planning systems (a core component of supply chain management) from two different industries: wine and mining. The rest of the chapter is organised as follows. The next section provides a brief literature review on coordinated supply chain, together with some discussion on time-varying constraints and the importance of Computational Intelligence methods. A few applications related to wine supply chain are presented in section 6.3; a brief discussion on complexity and challenges in decision-making processes is also provided. The next section discusses an advanced planning problem of the mining industry, which is loaded with a large number of constraints and dependencies. Section 6.5 concludes the chapter.

6.2 Literature Review In this section a brief literature review on coordinated supply chain management is provided, together with some discussion on time-varying constraints and the importance of Computational Intelligence methods.

128

M. Ibrahimov et al.

6.2.1 Supply Chain Management Supply chain management is a field that combines management of various business functions with a broad range of theoretical domains such as systems theory, inventory control, optimisation, logistics, mathematical and computer modelling and other. The Global Supply Chain Forum defines Supply Chain Management as the integration of key business processes across the supply chain for the purpose of creating value for customers and stakeholders as mentioned in [16]. The term supply chain management was first mentioned in 1982 by [24], however, the concept of supply chain management was born long before, in the early 20th century, with the creation of the assembly line. In 1960s appearance of IBM computers helped raising interest to modern supply chain management. In recent years, there has been an increased interest in solving supply chain management problems using Computational Intelligence methods. [23] looks at the problem of coordination of just-in-time production and transportation in a network of partially independent facilities of ready-mixed concrete. They optimise the network of independent and distributed production centres serving a number of customers distributed across a certain geographical area. This problem, which has high complexity and strict time delivery constraints, is approached with a meta-heuristic based on a hybrid genetic algorithm with combined constructive heuristics. [3] proposes algorithms using mixed-integer, non-linear programming model for multiobjective optimisation of a supply chain network based on the real world problem of a company that produces plastic products. They compare three approaches to find the set of Pareto-optimal solutions and discuss the pros and cons of each of them. [35] present a novel genetic algorithm to solve bi-criteria, multiple warehouse allocation problem. The proposed method finds the Pareto-front of wide range of nondominated solutions without the arbitrary determination of weighting coefficients. In [32], an evolutionary algorithm was developed for dealing with the coordination of supply chain flows between different members of the chain. The authors of that paper recognized the importance of an overarching algorithm that optimises the whole system. Their work looked at the flow of materials in terms of supply and demand, but did not consider the intricacies of production within each business silo. [17] applies genetic algorithms to solve a single machine scheduling problem with distinct dates and attempt to minimize all penalties. This method produces near optimal solutions, which they prove by comparison with an exact algorithm. [18] address the problem of inventory management of a refinery that imports several types of crude oil and proposes a mixed-integer linear programming model. [22] creates a linear programming model to optimise flat glass production. Large businesses typically split their business into operational components such as purchasing, production and distribution. In the past, organisations have concentrated their research and efforts on these single operational components of the supply chain. As optimisation of each individual silo in isolation may not lead to the global optimum, large businesses started to become more interested in optimisation of their whole system rather than optimisation of single components of the system. This approach is commonly referred in the literature as coordinated supply chain

6


129

management. Although the field of supply chain management emerged not a long time ago, the idea of coordinated planning was already proposed in 1960 by [8]. Their work investigates multi-echelon inventory problem based on sequential and tree-like models – and they approached the problem using recursive decomposition method. [33] provides a review of strategic production-distribution systems with global integration. The authors stress that very little research exists that addresses optimisation of the whole supply chain rather than its single components. They mainly concentrate on mixed integer programming models. [2] presents a problem of optimal location of warehouses starting with simple uncapacitated single-commodity case with zero echelons and ending with more complex capacitated multicommodity and multi-echelon facility location models. Review by [29] defines three categories of operational coordination: Buyer-Vendor coordination, Production-Distribution coordination and Inventory-Distribution coordination. Model described later in this paper belongs to the second category. Within production-distribution coordination category several researchers approached it with dynamic programming heuristics, mixed integer programming and Markov chains. Authors underline the complexity of problems of this category and discuss strategic planning supply chain models majority of the which are mixed integer programming based models. In one of the first papers on production-distribution coordination [12] the author considered several models with interacting silos and stochastic demands and discusses an analytical approach to find optimum inventory levels. Production orders are processed in weekly batches. [26] considers a single-product, three silo supply chain that include a factory, finished goods stockpile and a distribution centre. The whole model is demand driven and based on the Markov chain. The finished goods stockpile in this model is used more like a buffer between production and distribution silos. Authors present near-optimal algorithms to determine batch size, normal reorder point, and expedite reorder point. The companion paper [27] upgrades their system to support multi-product situations. [6] developed a two component demand driven model where products are first produced at the production plant and then distributed by the fleet of trucks to a number of retail outlets (also known as vehicle routing problem). Then authors compare two approaches, in the first one they solve the production and vehicle routing problems separately, and then together combined under single a optimisation algorithm. Production planning problem is solved optimally and the vehicle routing problem is solved using heuristics. Their results show an advantage of using the second approach. However, the authors used relatively small datasets for their problem. Multi-echelon inventory optimisation systems have been studied by various researchers. [4] uses a greedy algorithm for his multi-item, multi-echelon service parts distribution system, [5] solves his multiitem, two-echelon problem with Lagrangian decomposition, [34] approach the same problem with a greedy algorithm.

130

M. Ibrahimov et al.

6.2.2 Time-Varying Constraints The dynamic nature of real-world optimisation problems is well known, but a closer examination reveals a few different aspects of the problem that can be described as dynamic. In this section we introduce a classification of dynamic optimisation problems into three categories: 1. Time-varying objective functions. 2. Time-varying input variables. 3. Time-varying constraints. There is a large body of research literature that addresses dynamic property of such optimisation problems, and we will undertake a brief review of work in each of the areas. It will be noticed that while there is an abundance of work on problems fitting into the categories of time-varying objective functions and time-varying input variables, there are relatively few published reports dealing with dynamic optimisation problems of the “third kind”, that is one that deal with time-varying constraints. These are problems in which the constraints of the environment within which a solution must be found change from day to day. These varying constraints add an additional level of complexity to the problem because a good approach must be able to operate equally well regardless of how many of these constraints are in place, and in what particular combination they occur. An example of a varying constraint is the requirement that a new production schedule be created that meshes seamlessly with the existing schedule once a number of days of the current production schedule have been fixed. In general, we do not know what the current schedule might be at any given moment in time, yet we must create an algorithm that produces a new solution that matches up with the existing one, and still produces an optimal result for the long term. Good example of the types of problems that can be categorized as time-varying constraints can be found in [14], wherein the authors described circumstances that require the generation of regular new production schedules due to uncertainties (both expected and unexpected) in the production environment. They touch on typical examples such as machine breakdowns, increased order priority, rush orders arrival and cancellations. All of these issues are also considered in this article, from the perspective of an integrated evolutionary algorithms-based software solution. There are many other example of works recognizing this type of problem, for example [25, 15, 13], considering other Computational Intelligence methods. Based on our experience in solving real-world optimisation problems for commercial organisations, we have found that the type of problems commonly experienced in industry are those that belong to the second and third categories. Interestingly we have found that the vast majority of published research in evolutionary algorithms addresses the first category and to a lesser extent the second category. However, dynamic optimisation of the "‘third kind"’, i.e. where the problem involves time-varying constraints, although well-recognised in other domains, has been the subject of relatively few investigations in the literature. This observation

6


131

is especially true when we extend our search to fully-fledged application of a Computational Intelligence method to a dynamic real-world problem. In the same time, Computational Intelligence methods are very well suited for powering software applications for addressing these operational issues; we discuss them briefly in the following subsection.

6.2.3 Computational Intelligence Computational Intelligence is considered an alternative to classical artificial intelligence and it relies on heuristic algorithms (such as in fuzzy systems, neural networks and evolutionary computation). In addition, Computational Intelligence also embraces techniques such as swarm intelligence, fractals and chaos theory, artificial immune systems, and others. Computational Intelligence techniques often combine elements of learning, adaptation, evolution and fuzzy logic to create programs that are, in some sense, intelligent. An interesting question, which is being raised from time to time, asks for guidance on the types of problems for which Computational Intelligence methods are more appropriate than, say, standard Operation Research methods. From our perspective, the best answer to this question is given in a single phrase: complexity. Let us explain. Real-world problems are usually difficult to solve for several reasons, and include the following: • The number of possible solutions is so large as to forbid an exhaustive search for the best answer. • The evaluation function that describes the quality of any proposed solution is noisy or varies with time, thereby requiring not just a single solution but an entire series of solutions. • The possible solutions are so heavily constrained that constructing even one feasible answer is difficult, let alone searching for an optimum solution. Naturally, this list could be extended to include many other possible obstacles. For example, we could include noise associated with our observations and measurements, uncertainly about given information, and the difficulties posed by problems that have multiple and possibly conflicting objectives (which may require a set of solutions rather than a single solution). All these reasons are just various aspects of the complexity of the problem. Note that every time we solve a problem we must realise that we are in reality only finding the solution to a model of the problem. All models are a simplification of the real world - otherwise they would be as complex and unwieldy as the natural setting itself. Thus, the process of problem solving consists of two separate general steps: (i) creating a model of the problem, and (ii) using that model to generate a solution: Problem → Model → Solution.

132

M. Ibrahimov et al.

Note that the “solution” is only a solution in terms of the model. If our model has a high degree of fidelity, we can have more confidence that our solution will be meaningful. In contrast, if the model has too many unfulfilled assumptions and rough approximations, the solution may be meaningless, or worse. So, in solving real-world problems there are at least two ways to proceed: 1. we can try to simplify the model so that traditional methods might return better answers, or 2. we can keep the model with all its complexities, and use non-traditional approaches, to find a near-optimum solution. In other words, the more complex the problem (e.g., size of the search space, evaluation function, noise, constraints), the more appropriate it is to use a non-traditional method, e.g. Computational Intelligence method. Anyway, it is difficult to obtain a precise solution to a problem because we either have to approximate a model or approximate the solution. And a large volume of experimental evidence shows that this latter approach can often be used to practical advantage: many Computational Intelligence methods have already been incorporated into software applications that handle levels of supply chain complexity that is unapproachable by traditional methods.

6.3 Wine Supply Chain To address many of wine production challenges present in different parts of the wine supply chain, a suite of software applications that can optimise the end-to-end wine supply chain has been developed [21]. All these software applications were based on the paradigms of Computational Intelligence. These software applications include predictive modelling for grape maturity (using weather forecasts and readings on Baum, PH, and TA), vintage planning, crush scheduling, tank farm optimisation, bottling-line sequencing, and demand forecasting. When deployed together, these applications can optimise all planning and scheduling activities across a winery’s entire supply chain and can help wineries to optimise various business processes, predict and deal with unexpected events, and address key operational issues, such as: • • • •

Creating optimal harvest schedules that can accommodate last-minute changes. Maximising the utilisation of crushers, pressers, and fermenters. Optimising tank farm transfers and activities. Dealing with sudden changes (e.g., delayed transport, demand spikes, equipment failure, extreme weather conditions, etc.) • Improving resource allocation under risk and uncertainty. • Minimising transportation and other logistics costs. • Improving the accuracy of demand forecasts. We discuss briefly some of these applications in the following subsections.

6


133

6.3.1 Maturity Models Vintage intake plans are heavily dependent on the prediction of expected grape maturity dates. It is possible to export the prediction dates from some external system that functions as a black box providing only one date when it believes the harvesting should occur. However, limited visibility into the prediction process often prompts requests to revisit the prediction functionality of this process. So the maturity models deploy a new prediction module that provides improved prediction dates and visibility of the prediction-calculation provided. Grape maturity can be defined as the physiological age of the berry on the vine. It is important to define the optimal grape maturity for wine production and to develop clear chemical or biochemical traits that can be used to define the peak of ripeness. The definition of optimal maturity will vary depending upon the style of wine being made; the working definition of quality; varietal; rootstock; site; interaction of varietal, rootstock and site; seasonal specific factors; viticultural practices; and downstream processing events and goals. If a clear descriptive analysis of the quality target exists, then the time of harvest can be optimised to meet those goals. Several grape and cluster characteristics have been used to assess ripeness (e.g. sugar, pH, acidity, berry metabolites, berry proteins, taste). There are, of course, other non-compositional factors that influence the decision to harvest, including labour availability; seasonal changes such as rainfall; heat waves; tank space limitations; and other factors beyond the winemaker’s control. A “black box” prediction approach provides no audit capability for the user making it difficult to detect and promptly address issues related to accuracy of prediction. These factors can easily cause errors in forecasting maturity dates to go unnoticed and unrectified for prolonged periods of time. Decisions on when to book certain grapes for harvesting and crushing are relying heavily on the experience of the personnel involved in the process, and may result in a non-optimal allocation of harvesting and crushing resources. Each of these situations could result in higher costs for harvesting, transportation, and crushing, and reduction in grape quality.

6.3.2 Vintage Intake Planning Vintage intake planning manages the grape supply intake from the “vineyard to the weighbridge”. The functionality of this module supports the creation and maintenance of vintage intake plans that satisfy capacity constraints and facilitate the harvesting of grapes during periods of time when the quality is the highest. The Vintage Intake Planner has been designed around the following key components in the efficient planning of grape supply intake: • Will the block/sub block be available for harvesting during the optimum date range period, taking into account spray diary? Instead of assuming that each block is available for harvesting when required, the Vintage Intake Planner

134

M. Ibrahimov et al.

receives information that determines the availability to harvest. Example of the information required is the spray diary for each block. The grower currently supplies information on spray at a block / sub block level to satisfy the compliance requirement, but also to report on the periods when the block / sub block is under spray and not available for harvesting. • What is the best optimal / balanced grape supply intake plan taking into account the predicted optimum quality data range, block / sub block availability for picking / harvesting constraints, and winery constraints? If a Vintage Intake Planner provides a plan based just on the maturity dates provided, such plan would be unbalanced, as it does not take into account other constraints such as crushing / pressing capacity, logistics, etc. Thus it is important to include all the known constraints and information available at the time and provide a recommended optimal and balanced vintage intake plan. The recommended plan is available for the business users to make manual adjustments if required, save and submit the plan.

6.3.3 Crushing Crushers are used to process wine grapes and are often connected to different types of pressing machines, The optimal processing capacity of the crushing machines is about 40-45 tons per hour. However, but if necessary, it may be increased to 6080 tons per hour. The most important limiting factor is the capacity of the pressing machines and fermentation containers. The processing capacity for the pressing machines ranges from 4 to 12 tonnes per hour depending on the type of grapes. It is important to generate optimal schedules for all crushers over some time horizon. However, the generated weekly schedule may incur frequent changes due to contractual influences, weather conditions, seasonal influences, and daily production variances. When the changes occur, The Crush Scheduler re-optimises and generates alternative schedules to fill available capacity. Also, a variety of constraints are present in this part of the wine supply chain, including: • • • • • •

constraints in time (e.g., not processing fruit during the hottest part of the day). constraints in the throughput of presses. constraints in the throughput of fermentation vessels. the throughput of trucks via the crusher to be a continuous flow. scheduled repairs and maintenance of equipment. scheduled changeover and clean up (white to red, or lower grade grape-to-higher grade grape). • special demand-fulfilling variety shortages to address meeting capacity needs.

6


135

6.3.4 Tank Farm Wineries make daily decisions on the processing and storage of wines and juices in their tank farms, with major wineries having several hundred tanks that have differing capacities and attributes. These tanks may be insulated or refrigerated, for instance, and could have an agitator. Some of these tanks might be used to store juice after the grapes are crushed, fermented, and pressed, while others might be primarily designated for blending. Different types of juices may also require specific tank attributes for processing, such as refrigeration jackets or agitators. The process of planning and scheduling wine and juice transfers on the tank farm is far from trivial, as wine juices that need to be blended later should be kept together in nearby tanks. This ensures lower operational expenses and smoother process flow for the blending process. Second, the tanks should be close to their optimal filled capacity. This would eliminate “half-empty” tanks and maximise the number of empty tanks available for incoming juice. Third, the allocation of tanks should be flexible enough to accommodate last minute changes due to various unexpected events (e.g. changes in volume, demand, quality, etc.). Until recently, there has been a lack of software tools available that can manage and optimise tank farm activities, and as a result, most wineries have used a whiteboard approach which provides very little forward-looking visibility and does not allow for cost- or quality-optimised decision making. Substantial savings in spillage, labour and electricity use, throughput, and tank utilisation can be achieved through planning and scheduling optimisation. The Tank Farm Optimiser has been developed to replace the whiteboard approach for planning and scheduling tank farm activities. The system generates optimised tank farm plans for the entire vintage, given the physical constraints of the tank farm (e.g. tank attributes, transfer paths, etc.), incoming juice, bottling plan, and any user defined business rules. The planning horizon is in the order of 12 months, with daily or weekly “re-optimisation” as changes occur or new information becomes available. To decrease the overall cost of wine production on the tank farm, the Tank Farm Optimiser uses projected grape intake and bottling requirements to create a production schedule that minimises the number and distance of transfers between tanks, as well as the number of litres being transferred. The Tank Farm Optimiser used advanced non-linear optimisation algorithms (such as genetic algorithms) that consider the following key objectives when creating a production schedule: • • • • •

Minimisation of pack-up and pack-down transfers. Minimisation of changeovers from red to white in transfer paths and tanks. Minimisation of transfer distance. Minimisation of the number of litres transferred. Minimisation of ullage.

Additional objectives, such as environmental factors and quality issues, can also be added to the system to allow for trade-off analysis - these are discussed later in the paper.

136

M. Ibrahimov et al.

The user has full control over the optimisation process by being able to lock in manual decisions, set the business rules and constraints, and re-optimise after making changes. The Tank Farm Optimiser also provides elaborate reporting on a number of levels to suit different stakeholders; for example, the daily bottling programs for execution, a report on particular wine blends, or a report on expected production efficiency. The Tank Farm Optimiser takes into account the constraints and complexities of wine production to provide users with optimal scheduling of tank operations, environmental impact analysis, and reporting. Through the optimisation, advance planning, and visibility functionalities provided by the Tank Farm Optimiser, substantial business and environmental benefits can be realised by wineries in their tank farm operations and end-to-end supply chain.

6.3.5 Bottling The primary task of the Bottling-Line Scheduler is to generate optimal production schedules for the wineries’ bottling operations. The software uses advanced optimisation techniques for generating optimal production schedules. Opportunities for optimisation include manipulating the sequencing order, selecting which bottling lines to use, consolidating similar orders within the planning horizon, and suggesting changes to the requested dates that improve the overall schedule. Some of the key objectives are to maximise export and domestic service levels (i.e. DIFOT), maximising production efficiency, and minimising cost. The user can see exception reports where there are potential problems with work orders, or where there are items needing human attention. The user has full control over the optimisation process in that they are able to lock in manual decisions, set the business rules and constraints, re-optimise after making changes, and compare the current solution with an alternative plan. The software also provides a what-if module that can be used to analyse strategic business decisions and events such as capital investment in new equipment, or to look at operational decisions like adding or removing extra shifts, or even for crisis management (what is the impact of a bottling line going down or key staff being ill). Reporting is provided on a number of levels and to suit different stakeholders; for example the daily bottling programs for execution, a report on particular wine blends, or a report on expected production efficiency. The system generates robust schedules, such that small deviations from the planned schedule can be managed effectively (for instance, if excess volume is produced, or a machine breaks down for an hour, etc.). One aspect to robustness is to ensure sufficient lead time for supply of materials and for the release of wine from the tank farm. A significant part of the scheduling functionality is the ability for a user to adjust a schedule, rather than to completely re-schedule. This is important for two reasons: 1. Schedules are generated on a daily basis. The previous schedule needs to be taken into account when looking to integrate new work orders. One of the major

6


137

considerations is to minimise disruption to work orders close to being executed (i.e. schedule adherence), whilst allowing major re-ordering or the schedule for later orders to ensure the most optimal plan is produced. 2. The user will regularly need to change the suggested schedule, to take into account user expertise and knowledge outside the domain of the software. Where the user needs to change the schedule, a number of facilities is provided: • Locking an order in place. • Manually set the start date and production line for an order. • Setting Work Order specific rules (for example if a domestic Work Order must be produced by a certain date rather than being flexible). • Updating the configuration of the production environment, including business rules and constraints (for example, to let the system know about a machine breakdown making a bottling line unavailable). Once manual changes have been made the schedule can be re-optimised, ensuring that an optimal schedule is still produced whilst taking into account changes to the production environment and human expertise. Users are also able to generate an alternative to the current production schedule, with the system providing a comparison to help the user evaluate the impact of any changes. The comparison includes performance metrics (KPI’s) such as any difference in the number of late orders, and changes to production efficiency and cost (this could include measures such as cost per unit, total production throughput, production line utilisation, etc). This allows the user to experiment with different schedules before committing to making any changes; for example, trying to incorporate a last minute export order without disrupting existing orders.

6.3.6 Environmental Factors Further extensions to all of the presented modules (grape maturity, vintage planning, crush scheduling, tank farm optimisation, and bottling-line sequencing) include incorporation of water use and CO2 emissions to allow wineries to better understand and optimise the trade-offs between climate change related issues, such as CO2 emissions and water usage, and more traditional supply chain objectives like cost and volume. These applications support decision making processes on two levels: strategic and tactical. Strategic decision support includes: • Comparing different what-if scenarios and their KPIs; • Understanding the trade-offs between competing KPIs, such as average production cost vs. water use vs. CO2 output; • Simulate changes to plant equipment, warehouse capacities, product volumes and mix, etc.; and • Simulate changes to production and supply chain constraints and business rules. whereas tactical decision support includes:

138

M. Ibrahimov et al.

• Planning and scheduling of production and supply chain activities; • Optimising plans and schedules, e.g. for minimum CO2 emissions; • Managing unexpected events through dynamic re-planning and re-scheduling; and • Understanding why decisions are optimal, i.e. decision visibility and transparency. Thus the approved decisions address the effects of climate change through reduced energy usage resulting in reduced CO2 emissions, reduced water use, and increased ability to respond to adverse climate events, e.g. a heat wave or extreme rainfall. This suite of supply chain applications’ capability to handle complex, multiobjective supply chain problems allows wine companies to optimise their operational activities within the context of their business rules and constraints, e.g. quality standards, cost control, customer delivery requirements etc. For example, through the proper use of the Tank Farm Optimiser, a winery can realise four significant benefits: 1. Decrease in the number of wine transfers (note that this decrease equates to decreases in water consumption, carbon footprint, product loss, and maintenance costs together with increases in product quality ande safety). 2. Decrease in labour requirements. 3. Increase in tank utilisation by reducing “free working space” on the tank farm. Each of these three main benefits is covered in more details in the following paragraphs. Decrease in wine transfers. In most wineries a white board approach is used for making wine transfer decisions with no forward visibility or optimisation. Tanks are assigned to winemakers, then each winemaker uses his/her own group of tanks (which are not optimal). According to our estimations approximately one third of the transfers are non-core “pack up” and “pack down” operations. By introducing a centralised Tank Farm Optimiser, the number and distance of transfers can be significantly reduced. The Tank Farm Optimiser can simultaneously: • Reduce non-core transfers by 15% - 30% (for an overall reduction of 5% - 10% in tank farm transfers), and • Reduce the overall transfer distance. A reduction of 5% - 10% in tank farm transfers for a tank farm utilising 100 million litres of water and performing 5,000 transfers/year equates to: • A reduction of between 5 to 10 million litres in water consumption on the tank farm. This saving is achieved through fewer transfers, shorter transfers, and less “water intensive” transfers. • A reduction of between 5% - 10% in electricity consumption used for transfers, with corresponding reduction in carbon footprint. • Increase in product quality (each transfer erodes product quality).

6


139

• A reduction of between 5% - 10% in wine loss. On average, 300 litres of wine are lost during each transfer. If we assume an average cost of wine of $1.5/litre then the annual financial benefit is between $112,500 to $225,000 ($450 × 250 to $450 × 500). • A significant reduction in maintenance spend, as a reduction in transfers equates to a reduction in plant maintenance. • An increase in work safety as each transfer carries a risk of injury. Decrease in labour. Weekly payroll at a large winery may contain up to 10% of overtime wages. This may happen due to a lack of proactive work assignments. In such a situation employees oftentimes have little to do Monday through Tuesday, and then too much Wednesday through Friday. Also, due to a lack of advance planning, considerable time may be spent setting up transfers (e.g. if two transfers use the same transfer path, it is better to schedule them on different days. Otherwise, a lot of time is spent finding an alternative path, which is often more expensive in labour setup time and water use). Savings in this area can be achieved through “labour balancing” (i.e. advanced planning of transfers) and through the reduction in changeovers from red to white in both transfer paths and destination tanks (which requires more labour and water for cleaning). Through advance planning of transfers and transfer paths, overtime can be cut by at least 50%, and fixed labour by at least 10%. For a tank farm with a $2 million annual payroll this equates into an annual financial benefit of $280,000 ($200,000 overtime × 50% + $1,800,000 fixed × 10%). The realisation of this benefit would require process change in the handling of work orders at the winery. Specifically, the Tank Farm Optimiser would need to have the ability to delay or bring forward work orders within the week (i.e. move them plus/minus a few days) to achieve the “labour balancing” effect. Increase in tank utilisation. Tank farms keep a minimum of “free working space”, which may be anywhere between 10% and 13%. On a 100mln litres tank farm that equates to 10mln to 13mln litres of empty space. Through the advance planning of wine transfers (rather than using the white board approach), this “free working space” can be reduced by 1% to 1.5%., for a gain of 1.0 million to 1.5 million litres in capacity. This corresponds to an immediate one-off gain in new capacity (equivalent to several millions of dollars in capital investment) and represents one of the largest financial benefits of deploying the Tank Farm Optimiser. This increase in capacity will also provide the winery with a significant increase in throughput and decrease in average processing cost.

6.3.7 Summary Clearly, there are several significant benefits in optimising the whole wine supply chain – these include: • Better use of available production capacity.

140

M. Ibrahimov et al.

• reduced risk of late deliveries due to production capacity issues, supply issues, or scheduling errors, and better visibility of potential risk. • Schedulers can quantify the relative merits of different schedules, make informed decisions as to which scheduled to choose. • Higher confidence in production schedules may allow running at lower inventory levels. • Long-term planning from sales forecasts (e.g. assist with production capacity planning and production smoothing, supply planning for long lead-time items, and inventory planning, what-if scenarios for strategic and operational planning, testing the impact of changes on business rules, infrastructure investment, overtime or extra shifts). • Reporting on production (e.g. identification of capacity problems, identification of production or supply bottlenecks, high-level overview as well as information on specific orders). • Reduction in overall transfers, leading to less water consumed in the wine production process, a smaller carbon footprint, less spillage, less plant maintenance, and increased safety. • Reduction in labour requirements through labour balancing, a reduction in labour-intensive operations, and a reduction in overall transfers. • Reduction in “free working space” on the tank farm, leading to increased tank utilisation, capacity, and throughput. • Process improvement in the area of work order handling, by reducing paper handling and data duplication. • Provision of centralised applications that are maintained and supported. • Provision of a scalable platform for future extensions. • Straightforward integration with other applications for prediction and optimisation. • Provision of integrated views (carrier, winery, etc.) • Provision of integrated inputs (e.g. for Grower Liaison Officers and Logistics Coordinators). • Provision of optimised capacity planning (i.e. automated “smoothing”). • Automatic generation of robust, optimised production schedules that maximise service levels and utilisation, while minimising cost. • Faster feedback to production planners, management, sales, and other interested parties for placing orders or requesting changes. There are also a number of flow-on benefits, e.g. planners require less time to produce bottling plans, less chance of human error, identification of potential data problems, ability to handle dynamic changes to the schedule, whilst minimising the impact on existing Work Orders near their production date. To realise all these beefits, Computational Intelligence method must play the cenral role in the development of the software. It would be very difficult, for example, to build a linear model of such whole supply chain, representing all objectives, constraints, and dependencies – further, most standard software packages also failed in this complex environment over the last 15 years. As we have already indicated

6


141

in the Introduction, a new genre of applications (based on Computational Intelligence methods) is necessary to address complex issues of advanced planning in the supply chain management – and this section illustrated this point. The following section provides an overview of a complex planning problem from a very different industry’s point of view (mining) – to make a point, that the proposed approach is problem-independednt and can be applied across many different domains.

6.4 Advanced Planning in Mining In this section we present another real world example of application of Computational Intelligence methods in the mining industry. In the following sections we describe the problem and the objectives, present its complexities in terms of various constraints, and discuss briefly the main functionality of the application.

6.4.1 Problem Statement Since prehistoric times people were mining different ores and minerals out of the ground to produce various things for their lives. In the modern world mining is one of the most advanced fields of the industry. There are several main types of mining, in this case study the open pit type of mining problem will be described. In particular, a metal ore mining system will be discussed. A typical mine has a hierarchical structure. At the top level the mine is divided into several sub-mines called models. Typically each model is an island separated by non-ore material. Each model consists of one or several pits. Each pit in turn is sliced into horizontal layers called benches. Each bench contains multiple blocks which is the last level of hierarchy. Each block has specific information about its coordinates, tonnage, characteristics, percentage of different metals and non-metals (e.g. iron, aluminum, phosphorus etc) and waste. A block can fully consist of waste, or it can contain certain metals of high or low grade. There are two main types of plants on the mining site: crushers and washplants . Depending on the nature of the mine there can be other types of plants, however, in this chapter only these two are considered. Crusher is a plant designed to reduce large block chunks into smaller rocks for further processing. Different crushers have different crushing speeds measured in tonnes crushed per hour. Two main types of crushers considered in this chapter: high grade crushers and low grade crushers designed to operate with high and low grade materials respectively. In order to improve low grade ore it needs to be processed though the washplant. The process of washing removes contaminations, impurities, and other things that lower the quality of the ore. Several methods are known to achieve this task: magnetic separation, advanced gravity separation, jigging, washing and others.

142

M. Ibrahimov et al.

There are two types of mobile equipment available for mining: diggers and trucks. Trucks are used to transport material from one place to another. One or several truck fleets can be available. Each fleet has a number of trucks of the same type with same speed and load capacity. Diggers are used to excavate material and load it to trucks. Each digger has its own digging rate measured in tonnes of material per hour. They normally have very low speed. Mine models, pits, benches, crushers, washplants, stockpiles and wastedumps are connected by the road network. Materials are transported by the means of trucks through this network. Each pit has one or several pit exits through which trucks enter the pit. Each bench has one or several starting blocks with the road connected to them called toe blocks. The mining process is the following. First, a block is blasted by explosives, then a digger enters the blast area and excavates the blasted material. It loads the material onto trucks which drive it to the next destination. The destination can be different depending on the type and quality of material. All high grade materials from the excavated block are transported to high grade crusher and then crushed product is shipped to customers. The low grade material is transported first to the washplant to purify it, then to the low grade crusher and only after that shipped to customers. All excavated waste is taken to wastedumps which are basically piles of waste material. Along with the destinations mentioned above excavated material can also go to stockpiles which can be thought of as temporary place to put material that can be used in the future. The current problem is medium term planning and scheduling problem. Decisions have to be made in which order blocks of the mine should be excavated, how to utilise the equipment, crushers, washplants, stockpiles and truck fleets. In order to understand objective of the problem mining time period concept needs to be discussed. As a medium term scheduling problem, operation decisions should be made in time horizon of 4 to 6 years. The whole decision time horizon is divided into smaller time periods which are typically quarters but can vary depending on the specifics of the problem. Each time period has its own configuration diggers, crushers, washplants, wastedumps and stockpiles. Number of diggers and digging rates, crusher and washplant operating rates, capacities of wastedumps and stockpiles can be different across time periods. Along with that, each time period has its targets: • Tonnage target. As has been described earlier each block contains certain tonnage of desired metal. A block can also consist of total waste. A tonnage target is set for each time period and this much tonnes of ore is expected to produce by the end of the time period. • Quality target. All mining blocks have metal ore of a different quality. However, clients want ore of predefined quality. This means that during a time period such set of blocks should be excavated that when all ore mixed together the specified quality target should be met. It is very had to match the quality target exactly, that is why a quality tolerance is introduced measured in percentage from the target. All qualities within this tolerance level considered to be acceptable.

6


143

The objective of the problem is to dig blocks and utilise equipment and machines in such way to meet these two targets. However, this problem is even more complicated than is described so far, due to additional constraints. The addition of time periods to the problem adds an issue of time-varying constraints and time-varying objectives to the problem.

6.4.2 Constraints and Business Rules This subsection different types of problem specific dependencies and business rules. Most of them fall into one of two categories: dependency constraints and capacity constrains. The current formulation of the problem recognises two types of dependency constraints: block dependencies and area dependencies. Block dependency is the basic constraint of the problem. There are three types of block dependencies: • Clear Above Dependencies. Due to the physical nature of the mine, a block can not be excavated before the block on top of it is cleared. It can be thought of as a vertical dependency. • Clear Ahead Dependencies. This is a horizontal type of dependency. When digging the bench all side blocks should be excavated first before getting to the inner ones. • User Defined Dependencies. Sometimes due to certain business rules or specific circumstances there is a need to define custom dependencies between blocks. Another type of dependency constraint is area dependency. An area is basically a set of blocks defined by user. It can be arbitrary collection of blocks but usually it is a few adjacent pits that share certain characteristic. The concept of an area gives a flexibility to define a custom part of the mine. To define that one part of the mine should be excavated before another one these two parts should be marked as different areas and connected by the dependency. Capacity constraints, on the other hand, are the type of constraints that deal with physical workload of machines and equipment. All of them are hard constraints meaning violation of any of them will make a solution infeasible. The following list describes most of them: • Digger capacity constraint. Each digger can excavate only certain tonnage per time period. • Truck constraints. There are speed and capacity constraints for each type of truck. Furthermore, speed of the truck varies depending if it is loaded or not and depending on slope of the road. • Crushers constraints. As mentioned before crushers can operate with the limited speed and can not process more ore than they physically can handle. • Stockpile constraints. Each stockpile has a limited capacity. • Wastedump constraints. Each wastedump has a limited capacity. There are also some other constraints defined in this application, for example:

144

M. Ibrahimov et al.

• Pit tonnage limits. There is a limit of how much maximum tonnage can be excavated from each pit. Some pits may not have any limits. • Digger proximity constraint. Due to the safety reasons, two diggers can not operate too close to each other.

6.4.3 Functionality Taking into account all the dependency, capacity and other constraints described above the problem becomes highly complex combinatorial optimisation problem. The system that addresses the described mining problem has been implemented and the optimisation component was based on population based metaheuristic. This system was built for a large mining company, and it is currently in the process of being integrated into the existing IT structure. The key features of the system are the following: • Decision support system. The system is taking into account all constraints, configuration and targets and based on that produces a 5 years plan on how to utilise diggers, plants and other equipment and which block should each digger operate at given time. • Tradeoff analysis. There are some configurations where it is not possible to satisfy both tonnage and quality targets. Then the system produces to user a set of solutions which show different tradeoffs in tonnage and quality • Manual changes and what-if scenarios. High number of different business rules is the nature of any real-world system. It is not always possible to incorporate all business rules into the software (or it takes significant amount of time). That is why for any given solution produced by the system the operator can manually override decision made by system and see the impact of the change instantly. Apart from flexibility of manual changes this also opens another interesting feature of the application: the ability to analyse various what-if scenarios. In matter of minutes the user can see the result of adding or removing a digger or building another crusher on the mining site.

6.5 Conclusion and Future Works In this chapter we described challenges of real world applications which address key issues of supply chain management. The main goal of this research project is to integrate the disparate techniques, methods, and algorithms into an adaptive, dynamic and intelligent Advanced Planning System that deals with a whole vertically integrated supply chain and logistics management to provide decision support in complex and dynamic environments. The project investigates algorithms that are integrated with the actively changing data sets in order that solutions to problems and

6


145

can reflect amendments to models and data in real time. In essence, the intelligent Advanced Planning System would be able to ’learn’ and to adapt in order to provide superior support to decision makers, enormously increasing their predictive power, not only in a silo environment, but also in a large-scale supply chain operation. By developing new and improved methods for knowledge representation, combining prediction and optimisation algorithms, handling non-stationary constraints and “what-if” scenarios, the project aids the development of a new generation of Advanced Planning Systems for supply chain management in a wide variety of domains. Examples include distribution systems with assignment and location components, transportation components, routing components, marketing and customer relationship components, planning and scheduling components, amongst others. The project required simultaneous advances in several research directions. It was necessary to analyse a few complex business problems (as presented in this chapter), create a new type of test-case generator for non-stationary environments, and investigate several methodologies for knowledge representation (for example, diploid structures, memory-enhanced co-evolutionary algorithms) for optimisation in nonstationary environments. It was also necessary to investigate further the relationship between various prediction and optimisation algorithms, and to re-examine different methodologies for handling constrained, multi-objective optimisation problems in non-stationary environments (this includes methodologies for handling nonstationary constraints). We built on current world class practice and state of the art knowledge in commerce, artificial intelligence and operations research to produce innovation which is applicable to real-world business problems, promoting an innovation culture and economy. Finally, addressing the supply chain as a whole requires a quite significant amount of computing power. Hence the necessity to investigate computing mechanism to speed-up the process, i.e., parallel computing, without logically decomposing the process. Usually optimisation algorithms assume static data and a static objective function; they search for a snapshot of ’knowledge’ and for a near-optimum solution with respect to some fixed measure (or set of measures), whether it is maximisation of profit, minimisation of a completion time for some tasks, or minimisation of production costs. However, real-world applications operate in dynamic environments where it is often necessary to modify the current solution due to various changes in the setting of the problem, such as machine breakdowns or sickness of employees. Also, there are changes in the business environment, such as trends within the population of consumers, changes in weather patterns, or a change of economic indicators. Thus, it is important to investigate properties of adaptive algorithms that do not require a restart every time a change is recorded; in many commercial situations such restarts are impossible. It is therefore natural to approach non-stationary problems with Computational Intelligence techniques. Nature-inspired optimisation algorithms (evolutionary algorithms, simulated annealing, ant systems, swarm intelligence, neural networks) seem particularly appropriate, as nature is a continuously changing environment. It is important to investigate which extensions of these algorithms are useful in business/management scenarios. Unfortunately, most of the current research in

146

M. Ibrahimov et al.

modern heuristic methods ignores dynamic changes and assume that re-optimisation should be performed at regular intervals. Enormous benefits can be realised when dynamism is addressed explicitly in the context of Advanced Planning Systems, and this is precisely the main issue addressed in this proposal. Further, during the adaptation process, it is vital to keep the solution conforming to the primary objectives while minimising associated costs of adjusting the solution. This requires the adaptation issue to be treated under multiple criteria. This opens the door for the research direction of tackling the adaptation issue with multiobjective optimization. Interestingly, multi-objectivity can be used in a way that can effectively facilitate the adaptation process. Further, the use of multi-objectivity can result in a set of trade-off solutions (in contrast with single-objective cases where only one solution is found at the end). It is an information-rich data set, which can offer decision makers several alternatives to deal with changes, or can be used to approximate the search space later on. However, it poses a challenge in how to handle that data set since the set is usually large. This is still somewhat missing in the literature. The enormously complex nature of truly real-world problems poses a wide variety of challenges to any solution methodology. In order to harness the power and potential of modern heuristic methods and bring them to bear on these kinds of challenges, it is critical to acknowledge these complex realities and adapt, modify, and combine these methods so as to capture and grapple with as much of the intricacies as possible, making as few simiplifications as possible, and thereby retain as much value as practicable to the solutions thus generated by the resulting hybrid algorithms. It is in so doing that real benefit can be delivered to problems where concrete action in the real world is required to carry out day to day business.

References 1. Ackoff, R.L.: The future of operational research is past. JORS 30, 93–104 (1979) 2. Aikens, C.H.: Facility location models for distribution planning. European Journal of Operational Research 22(3), 263–279 (1985) 3. Altiparmak, F., Gen, M., Lin, L., Paksoy, T.: A genetic algorithm approach for multiobjective optimization of supply chain networks. Computers & Industrial Engineering; Special Issue on “Computational Intelligence and Information Technology: Applications to Industrial Engineering" 51(1), 196–215 (2006) 4. Caggiano, K.E., Jackson, P.L., Muckstadt, J.A., Rappold, J.A.: Optimizing Service Parts Inventory in a Multiechelon, Multi-Item Supply Chain with Time-Based Customer Service-Level Agreements. Operations Research 55(2), 303–318 (2007) 5. Caglar, D., Li, C.-L., Simchi-Levi, D.: Two-echelon spare parts inventory system subject to a service constraint. IIE Transactions 36(7), 655–666 (2004) 6. Chandra, P., Fisher, M.L.: Coordination of production and distribution planning. European Journal of Operational Research 72(3), 503–517 (1994) 7. Cheng, R., Gen, M., Tsujimura, Y.: A tutorial survey of job-shop scheduling problems using genetic algorithms–i: representation. Comput. Ind. Eng. 30(4), 983–997 (1996) 8. Clark, A.J., Scarf, H.: Optimal policies for a multi-echelon inventory problem. Manage. Sci 50(12 suppl.), 1782–1790 (2004)

6


147

9. Coit, D.W., Smith, A.E.: Solving the redundancy allocation problem using a combined neural network/genetic algorithm approach. Computers & Operations Research 23(6), 515–526 (1996) 10. Davis, L.: Job shop scheduling with genetic algorithms. In: Proceedings of the 1st International Conference on Genetic Algorithms, pp. 136–140. L. Erlbaum Associates Inc., Hillsdale (1985) 11. Davis, L.: Embracing complexity. toward a 21st century supply chain solution (2008), Web-resource http://sdcexec.com/online/printer.jsp?id=9012 12. Hanssmann, F.: Optimal inventory location and control in production and distribution networks. Operations Research 7(4), 483–498 (1959) 13. Holthaus, O.: Scheduling in job shops with machine breakdowns: an experimental study. Computers & industrial engineering 36(1) (1999) 14. Jain, A.K., Elmaraghy, H.A.: Production scheduling/rescheduling in flexible manufacturing. International Journal of Production Research 35(1), 281–309 (1997) 15. Kutanoglu, E., Sabuncuoglu, I.: Routing-based reactive scheduling policies for machine failures in dynamic job shops. International Journal of Production Research 39(14), 3141–3158 (2001) 16. Lambert, D.M.: Supply chain management: Implementation issues and research opportunities. The International Journal of Logistics Management 9(20), 1–20 (1998) 17. Lee, C.Y., Choi, J.Y.: A genetic algorithm for job sequencing problems with distinct due dates and general early-tardy penalty weights. Computers & Operations Research 22(8), 857–869 (1995) 18. Lee, H., Pinto, J.M., Grossmann, I.E., Park, S.: Mixed-integer linear programming model for refinery short-term scheduling of crude oil unloading with inventory management. Industrial & Engineering Chemistry Research 35(5), 1630–1641 (1996) 19. Levine, J., Ducatelle, F.: Ant colony optimization and local search for bin packing and cutting stock problems. The Journal of the Operational Research Society 55(7), 705–716 (2004) 20. Liang, K.-H., Yao, X., Newton, C., Hoffman, D.: A new evolutionary approach to cutting stock problems with and without contiguity. Computers & Operations Research 29(12), 1641–1659 (2002) 21. SolveIT Software Pty Ltd., http://www.solveitsoftware.com 22. Martin, C.H., Dent, D.C., Eckhart, J.C.: Integrated production, distribution, and inventory planning at libbey-owens-ford. Interfaces 23(3), 68–78 (1993) 23. Naso, D., Surico, M., Turchiano, B., Kaymak, U.: Genetic algorithms for supply-chain scheduling: A case study in the distribution of ready-mixed concrete. European Journal of Operational Research 177(3), 2069–2099 (2007) 24. Oliver, R.K., Webber, M.D.: Supply-chain management: Logistics catches up with strategy. In: Logistics, Chapman and Hall, Boca Raton (1982); reprint from Outlook (1982) 25. Petrovic, D., Alejandra, D.: A fuzzy logic based production scheduling/rescheduling in the presence of uncertain disruptions. Fuzzy Sets and Systems 157(16), 2273–2285 (2006) 26. Pyke, D.F., Cohen, M.A.: Performance characteristics of stochastic integrated production-distribution systems. European Journal of Operational Research 68(1), 23–48 (1993) 27. Pyke, D.F., Cohen, M.A.: Multiproduct integrated production–distribution systems. European Journal of Operational Research 74(1), 18–49 (1994) 28. Stadtler, H., Kilger, C.: Supply Chain Management and Advanced Planning. Springer, Heidelberg (2008)

148

M. Ibrahimov et al.

29. Thomas, D.J., Griffin, P.M.: Coordinated supply chain management. European Journal of Operational Research 94(1), 1–15 (1996) 30. Toth, P., Vigo, D.: The Vehicle routing problem. Society for Industrial and Applied Mathematics, Philadelphia (2001) 31. Van Laarhoven, P.J.M.: Job shop scheduling by simulated annealing. Operations Research 40, 113 (1992) 32. Vergara, F.E., Khouja, M., Michalewicz, Z.: An evolutionary algorithm for optimizing material flow in supply chains. Comput. Ind. Eng. 43(3), 407–421 (2002) 33. Vidal, C.J., Goetschalckx, M.: Strategic production-distribution models: A critical review with emphasis on global supply chain models. European Journal of Operational Research 98(1), 1–18 (1997) 34. Wong, H., Kranenburg, B., van Houtum, G.J., Cattrysse, D.: Efficient heuristics for twoechelon spare parts inventory systems with an aggregate mean waiting time constraint per local warehouse. OR Spectrum 29(4), 699–722 (2007) 35. Zhou, G., Min, H., Gen, M.: A genetic algorithm approach to the bi-criteria allocation of customers to warehouses. International Journal of Production Economics 86(1), 35–45 (2003) 36. Zielinski, K., Weitkemper, P., Laur, R., Kammeyer, K.-D.: Parameter study for differential evolution using a power allocation problem including interference cancellation. In: IEEE Congress on Evolutionary Computation, CEC 2006, pp. 1857–1864 (2006)

Chapter 7

Efficient Data Sharing over Large-Scale Distributed Communities Juan Li, Samee Ullah Khan, Qingrui Li, Nasir Ghani, Nasro Min-Allah, Pascal Bouvry, and Weiyi Zhang

Abstract. Data sharing in large-scale Peer Data Management Systems (PDMS) is challenging due to the excessive number of data sites, their autonomous nature, and the heterogeneity of their schema. Existing PDMS query applications have difficulty to simultaneously achieve high recall rate and scalability. In this chapter, we Juan Li Department of Computer Science, North Dakota State University, Fargo, USA e-mail: [email protected] Samee Ullah Khan Department of Electrical and Computer Engineering, North Dakota State University, Fargo, USA e-mail: [email protected] Qingrui Li Department of Computer Science, North Dakota State University, Fargo, USA e-mail: [email protected] Nasir Ghani Department of Electrical and Computer Engineering, University of New Mexico, Albuquerque, USA e-mail: [email protected] Nasro Min-Allah Department of Computer Science, COMSATS Institute of Information Technology, Islamabad, Pakistan e-mail: [email protected] Pascal Bouvry Faculty of Sciences, Technology and Communications, University of Luxembourg, Luxembourg e-mail: [email protected] Weiyi Zhang Department of Computer Science, North Dakota State University, Fargo, USA e-mail: [email protected]


150

J. Li et al.

propose an ontology-based sharing framework to improve the quality of data sharing and querying over large-scale distributed communities. In particular, we add a semantic layer to the PDMSs, which alleviates the semantic heterogeneity and assists the system to adjust its topology so that semantically related data sources can be connected. Moreover, we propose a comprehensive query routing and optimization scheme to enable scalable and efficient query evaluation. Simulation studies reveal that our proposed ontology-based data sharing framework effectively improves the query performance in terms of recall and scalability.

7.1 Introduction The continued growth of computational power and communication technologies is giving rise to a new information era. In this era, the broad availability of data coupled with increased capabilities and decreased costs of both storage and computing technologies enable users to share data and knowledge on an unprecedented scale. In a large distributed community, data are geographically distributed and owned by different organizations. The fact that users typically have little or no knowledge of data contributed by other participants in a large community poses a significant obstacle to their use. For this reason, distributed data sharing is a vital part of a distributed community, and an efficient data sharing infrastructure is crucial to make the distributed information available to users in a timely and reliable manner. However, data sharing in large-scale communities is very challenging due to the potential large amounts of data, diverseness, distributed arrangement, and dynamic nature. In addition, it is equally difficult to integrate the information sources with heterogeneous representation formats. This compels us to rethink how one must manage, i.e., store, retrieve, explore, and analyze this abundance of data. An effective data sharing solution for this environment must meet the following requirements: Scalability: Because of the participation of Personal Computers (PCs) and devices, such as sensors at the periphery of the Internet, the number of nodes and data involved in a community may grow from hundreds to tens of thousands. The aforementioned raises the problem of potential performance degradation. Consequently, applications must be scalable in handling the vast number of nodes and data involved in the community deployment. The system must be capable of spreading load among the participants themselves and serving large number of users at a very low cost. Interoperability: Different data may belong to different organizations and may have heterogeneous schema. The same data in different nodes can be defined differently. For example, one node may use the term processor to represent a computing resource, while another node may use CPU for description. On the other hand, the same term can also be used to represent different resources. For example, the term mouse might represent a computer mouse in one node, and a rodent in another. There are other relationships between concepts that also need to be considered. For instance, a query related to operating systems should be able to understand that Windows, UNIX, and Macintosh are different types of operating systems. It is

7

Efficient Data Sharing over Large-Scale Distributed Communities

151

also important for the sharing scheme to determine the relationships among concepts and locate all related resources; however they are represented. Therefore, the system must have a highly expressive logic with sophisticated reasoning capabilities. Appropriate data schema integration strategies must be used to reconcile the semantic difference. Robustness: PCs are not as stable as servers in classical clusters. With so many community data sources, the probability of some data failing is quite high. Therefore, node and data failures are the rule rather than the exception. In addition, data and their status change frequently. The system must adapt its behavior dynamically and should be able to provide reliable services in a highly dynamic environment, locating data when they are present and meeting the requirements of the application. Responsiveness: The response time includes network latency for propagating query over the network and the accumulated query processing time at each node. Response time is a key issue for data query applications. An effective data query system must respond to users quickly and efficiently with up-to-date information. In this chapter, we propose an ontology-based data sharing framework that aims to satisfy all of the aforementioned requirements. The proposed framework works in a fully decentralized and scalable way. It enables efficient sharing and collaborating over a large-scale community composed of geographically distributed data providers. The rest of the chapter is organized as follows. Section 7.2 introduces the related work. Section 7.3 gives an overview of the system design. Section 7.4 presents the scheme to cluster nodes based on their semantics. Section 7.5 describes the query forwarding strategy. In Section 7.6 we evaluate the proposed method with a comprehensive set of simulations. Concluding remarks are provided in Section 7.7.

7.2 Related Work Our proposed system is based on existing research on Peer Data Management Systems (PDMS) [6, 23]. PDMSs were proposed as the extension to distribute databases within the context of P2P systems. They are built of multiple peers, each of which provides a schema and accepts queries against the schema. Schema mappings are established between neighbor peers forming the basis for query answering. Queries are propagated and answered by all peers that can be reached though mappings over the neighbors. However, as mentioned, the schema mapping and neighbor forwarding strategy used by most PDMSs have serious problems such as inflexibility and low recall rate caused by strict schema mapping, information loss caused by query rewriting, long delay and large overhead caused by a large network population. Peer clustering [8, 14] has been proposed to reduce the query space and the semantic degradation caused by traversal of the semantic mappings. In [14], which is more similar to our approach, the authors propose a clustering process that extend classical clustering algorithm to distributed environment to group peers which are semantically related. Our proposed semantics-based topology adaptation, instead, incrementally cluster peers when peers join the system. The system also dynamically adjusts the topology to adapt to the evolving network. Unlike [14], our

152

J. Li et al.

clustering does not need to broadcast in clusters and the computation is much simpler. Therefore, it is easier to scale. Semantic web technologies are being used in PDMS to add intelligence and reconcile the semantic heterogeneity problem. In [17] the authors use the ontology mapping to facilitate schema mapping. However, how to locate semantically relevant peers to map is unspecified in the paper, and the scalability is still a problem. In [7] the is a distributed hash table (DHT) used [19, 20, 21, 24] approach to index peer’s ontology URI to expedite the ontology discovery. However, indexing URI in fact does not consider the semantic meanings, therefore, cannot address the semantic similarity problem.

7.3 System Overview 7.3.1 Problem Description A sharing community consists of autonomous nodes that store and manage structured relational data locally. Each data source has its own data schema. A node joins the community by connecting to one or more existing nodes (acquaintances) in the community. As pointed out by Tatarinov et al. [22], it is neither practical to maintain a global schema for all data sources to map their local schemas with, nor is it feasible for a data source to map its schema with all other sources. A realistic approach is to let each provider connect to and construct mapping with one or a few acquaintances, then queries can be translated between different schemas. Therefore, most of the current PDMSs [6, 23] use peer mappings and some suitable rewriting algorithms to propagate queries between acquainted peers. However, this strategy may cause some problems: • The nature of structured data stored in the overlay enforces strict methods of querying and query rewriting. Frequently, the user intends to obtain information that is semantically relevant to the posed query, rather than information that strictly complies with structural constraints. • Query rewriting based on multiple pair-wise schema mapping may cause information loss especially when the mapping pairs are not semantically similar enough. • For a large-scale network, query has to be forwarded through a large number of peers, which is obviously unscalable.

7.3.2 A Multilayered Semantic Sharing Scheme In this section, we address the aforementioned problems by exploiting the benefits provided by semantics through ontology. Ontology is defined as "an explicit specification of a conceptualization" [4]. Ontologies are (meta) data schemas, providing a controlled vocabulary of concepts, each with an explicitly defined and

7


153

machine-processable semantics . By defining shared and common domain theories, ontologies help both people and machines to communicate concisely, supporting the exchange of semantics and not only syntax. In general, an ontology of a domain consists of the four major components listed below: • Concepts: Concepts of a domain are abstractions of concrete entities de-rived from specific instances or occurrences. • Attributes: Attributes are characteristics of the concepts that may or may not be concepts by themselves. • Taxonomy: Taxonomy provides the hierarchical relationships between the concepts. • Non-taxonomic relationships: Non-taxonomic relationships specify nonhierarchical semantic relationships between the concepts. Along with the above four components, ontologies may also consist of instances of each of the concepts, and inference rules of the domain. Fig. 7.1 illustrates a portion of the ontology for a Computer Science Department. It indicates some of the major concepts and their hierarchical relationships only. In general, the ontology includes non-taxonomic relationships as well, for example: Student - take - Course, Professor - teach -Course, Worker - work - Department. By defining shared and common domain theories, ontologies help both people and machines to communicate concisely, supporting the exchange of semantics and not only syntax. We assume that there exist domain ontologies shared in the community. As shown in Fig. 7.2, we extend the traditional PDMS structure with a twolevel semantic dimension. The ontology layer utilizes the understanding of domain knowledge to model the schema of data sources, which enables more flexible peer mapping at the semantic-level. Semantic-level mapping overcomes the limitation of relational schema mapping and is able to map with more semantically related peers. The concept layer is a summarization of ontology. The concept layer simplifies the computation of semantic closeness between peers. This makes clustering semantically related peers much easier. Connecting semantically related nodes can reduce information loss caused by peer-wise query rewriting. Last but not least, query forwarding with semantic guidance may intelligently propagate queries to the right direction, thus reducing bandwidth used for forwarding queries and increasing recall rate. We present how semantics improves the system performance from these aspects in the following sections.

7.3.3 From Schema to Ontology As mentioned, we assume that there exist domain ontologies shared by members of the community. A data source can map its local schema to one or more ontologies. These mappings are peer-specific and may be constructed manually or semi-automatically. Our principles of mapping the rational schema to ontology are

154

J. Li et al.

%

&

!"

$

$ %

%#

#

Fig. 7.1 An example of an ontology snippet

Fig. 7.2 Layered architecture of data sharing

7


155

based on relational schema design. Specifically, we analyze Entity-Relationship (ER) models that support entities with attributes, relationships, and inheritance hierarchies between entities. We reverse the process of translating ER model to relational model. In this way, we convert entities, keys, foreign keys to entities and relationships (including hierarchical relationship) in ontology.

Fig. 7.3 Mapping from schema to ontology

Fig. 7.3 depicts a simplified example of this mapping, in which a relational schema presenting information about "professor" is mapped to an ontology that contains domain concepts about university, course and faculty. For simplicity, the ontology graph only shows part of the ontology that is related to the schema.

7.3.4 Semantic Similarity With the schema to ontology mapping defined, measuring the similarity between data sources can be converted to a problem of evaluating the similarity (or distance) between (sub-)graphs of the ontology graphs. We adopt the idea of distance-based approach [12, 16, 18, 11] to measuring the semantic similarity. The basic idea behind the distance-based approach is to identify the shortest path between two concepts in terms of the number of edges in the ontology graph and then translate that distance into semantic distance. Our approach improves the accuracy by integrating factors, such as the depth of a node in the ontology hierarchy and the type of links. The semantic distance between two concepts is defined as a "relative" distance between nodes and their common ancestors in an ontology graph, thus it integrates the edge weight with the depth and the length of the shortest path. Given two ontologies that are summarized as collections of concepts, the similarity between these two ontologies is the normalized similarity of all

156

J. Li et al.

related pairs of concepts. The similarity defined this way is not a symmetric relation. If the semantic similarity from A to B is larger than a user-defined similarity threshold t (0 < t ≤ 1), ontology A is said to be semantically related to ontology B. Note that the measurement defined above can be extended to evaluate the similarity of multiple ontologies. The idea is to place the concepts of the ontologies into a global ontology - WordNet [15], then the problem turns to a similarity measurement of same ontology again. Due to the lack of space, we are restricted to provide only surface-level information pertaining to semantic similarity measurement. We encourage the readers to find more details about our pro-posed solution by referring to [11].

7.4 Semantics-Based Self-clustering The establishment of schema mappings and query answering in PDMSs can be improved if semantically related peers are grouped together in the community network. The objective of semantic clustering is to make the system’s dynamic topology match the semantic clustering of peers. This would allow queries to be quickly propagated among relevant peers. In addition, this adaptation allows semantically related peers to establish ontology mapping relationships to reduce information loss. The rationale behind the topology adaptation is based on two theories of the existing research: (1) The most promising peers who are able to answer a particular query are those who are more similar with the querying peer [1]. (2) Connecting and constructing schema mapping with semantically similar nodes will reduce information loss caused by query reformulation [9]. In our system, semantic clustering is performed at different states of a nodes’ life time: (1) upon joining the network, a node chooses neighbors by their semantic similarity. (2) A node also gradually updates its links according to its query experiences, so that its topology always reflects its changing interest and data contents.

7.4.1 Joining the Right Semantic Cluster A node joins the network by connecting to one or more bootstrapping neighbors. Then the joining node issues a neighbor-discovery query and forwards the query to the network through its bootstrapping neighbors. The neighbor-discovery query routing is a process of greedily forwarding the query to neighbors with highest similarity. A neighbor-discovery query message includes several parts: (1) the querying node’s compressed concept vector, (2) a similarity threshold which is a criterion to determine if a node is semantically related to the query (optional), (3) a query Time To Live (TTL) to gauge how far the query should be propagated. When a node N receives a neighbor-discovery query Q which tries to find neighbors for a new joining node X, N computes the semantic similarity between X and itself. If N is semantically related to X, N will send a Neighbor Found reply to X. If

7


157

the query’s TTL has not expired, N computes the semantic similarity between X and each of N’s neighbors, and forwards the query to semantically related neighbors. If no semantically related neighbors are found, the query will be forwarded to a few "outsiders" of the cluster. The query will propagate with this strategy until enough matches have been located or TTL expires.

7.4.2 Dynamic Self-adjusting Because of the dynamic property of the large-scale network and the evolution of nodes’ ontology property, neighbor discovery for a node is not once and for all, but rather the first-step of the topology adaptation scheme. A node should keep updating its neighbor links according to its query experiences and other peer’s recommendations. A peer obtains experiences by accumulating queries received as a query forwarding router, and query results collected as a query requestor. Based on its experiences and peer recommendations, a node may add or delete neighbors according to the dynamic semantic environment. This way, the network topology is reconfigured with respect to peers’ semantic properties, and peers with similar ontologies are close to each other. Once a node joins the right cluster, i.e., it connects to the semantically related nodes, it can establish appropriate ontology mappings with its neighbors manually, semi-automatically or automatically. The ontology mapping is beyond the scope of this chapter. Readers can refer the state of the art in this field [3, 10] for more information.

7.5 Query Evaluation With the semantics-based topology constructed, query can be evaluated within the scope of querying node’s local cluster most of the time, because queries reflect the querying node’s ontology interest, and semantically related nodes are within the neighborhood of the querying node. The semantic clusters reduce the search time and decrease the network traffic by minimizing the number of messages circulating among clusters. However, how to minimize query cost inside clusters is still a challenging issue. In this section, we propose a comprehensive query routing and optimization scheme to address the challenges of semantic query evaluation.

7.5.1 Problems of Query Evaluation In a distributed and unstructured network, queries are evaluated by forwarding between neighbors. Because there are multiple paths between nodes, there are a large number of duplicated messages propagated in the network. There are many technologies [2, 13] to control duplicated messages by cutting flooding paths. However, in a schema-heterogeneous PDMS network, optimization cannot be done by simply cutting the flooding paths. In such environments, queries are forwarded and

158

J. Li et al.

translated according to the mappings between neighbors. Query rewrites based on different mappings can come to the same nodes by different paths. Arbitrarily cutting flooding paths may lose some query rewrites thus causing result loss. On the other hand, answering all variants of the query rewrites may not be necessary, because some rewrites may be the same, and some may be contained by others. Generating answers for redundant queries will produce redundant results, which will strain the network and waste the bandwidth. In order to solve this problem, we utilize the semantic similarity metric proposed in Section 7.3.4 to guide query forwarding. Moreover, we propose a containment-based caching strategy to reduce unnecessary query traffic.

7.5.2 Semantics-Based Forwarding A query q is uniquely identified by its query ID (q.id) issued by the originator. It also includes the ID of the query originator (q.originator) and the ID of the previous hop (q.lastHop) that passes the query to the current node, and the SQL query string (q.query). When a node receives a query from a neighbor, it first tries to answer the query with its local data. Then it will measure the semantic similarity between the query and the ontology summaries of its neighbors. Based on the similarity measure, the query will only be forwarded to semantically related neighbors (i.e., the similarity value is beyond the predefined threshold t). Before forwarding the query to a neighbor, the query will be reformulated to the neighbor’s ontology according to the ontology mapping. The translated query q’ is called a rewriting of the originate query. The query reformulation algorithms have been extensively studied in [6, 5]. Our system adopts the algorithm in [6] which produces maximally contained rewritings. We extend this algorithm to support semantic data defined with ontologies by adding the ontological containment, such as subsumption relationship of classes and properties, equivalent class and property, inverse and transitive property. Since the ontology mapping between two nodes rarely maps all concepts in one node to all concepts in the other, mappings typically lose some information and can be partial or incomplete; the reformulated query may deviate from the original query’s intention, and the query result should be evaluated at the querying node. Feedback on query results can be used to improve the quality of mappings. Moreover, nodes can learn from query results to update their neighbors. Therefore, when a node updates its semantic interests, the system is able to adjust that node’s links accordingly.

7.5.3 Containment-Based Caching To further reduce the query transmission cost, we propose an efficient caching mechanism based on the semantic query containment checking. In this approach, each node maintains two caches: (1) active_id_cache which stores the IDs of active queries (query’s Time to Live (TTL) is not expired yet), (2) query_cache that caches the best rewrites of all historically popular queries and their corresponding results.

7


159

Algorithm 7. Algorithm of optimized query evaluation 1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: 14: 15: 16: 17: 18: 19: 20: 21:

if q can be found in active_id_cache, i.e., it is an active query then if q is in the query_cache or q is contained by a query in the query_cache then ignore the query and return ; else if q contains or overlaps with queries in the query_cache then determine if q should be decomposed, and decompose q accordingly; end if else put q.id to active_id_cache; if q is in the query_cache then return the results cached; end if if q is contained by a query in the query_cache then get the result by using constraint on the cached result; return; else if q contains or overlaps with queries in the query_cache then determine if q should be decomposed, and decompose q accordingly; end if end if find results from local dataset for query q (or q’s sub-queries); forward q (or q’s sub-queries) to semantically related neighbors; put q into the query_cache, replacement policy may be used

The algorithm of query processing with containment-based caching is il-lustrated with the pseudocode in Alg. 7. The original query ID is passed along the query reformulation paths together with the rewrites. Each node remembers the current active queries (in active_id_cache) and a set of best rewrites for a distinct original query (in query_cache). The evaluation of the new incoming query is based on the status of the query: if it is a new query or an existing active query, if it can be found in the query_cache, or it is contained by queries in the query_cache, or it contains or overlaps with queries in the query_cache. The operations we take may vary from ignoring the query, returning cached results, processing and then returning the cached results, decomposing the query and evaluating the sub-queries, and evaluating the query from scratch. The detail of the evaluation is presented in Alg. 7. We can see that the query evaluation process is integrated with caching. Our later experiments demonstrate that this query optimization strategy dramatically reduces the query traffic.

7.6 Experiment In this section, we present the experimental results which demonstrate the performance of our searching scheme. We generated test data artificially. Our data does not claim to model real data, but shall rather provide reasonable approximation to evaluate the performance of the system. In our experiments, the total number of distinguished schema is 100. We

160

J. Li et al.

assume each data sites uses 1 to 3 schemas. The simulation is initialized by injecting nodes one by one into the network until a certain network size has been reached. After the initial topology is created, queries are injected into the network based on certain ratios. Inserted nodes are assumed to have one or more acquaintances. We expect that our self-clustering scheme will transform the topology into semantics-based communities . To verify this transformation, we examine an important network statistic, the clustering coefficient. The clustering coefficient (CC) is a measure of how well connected a node’s neighbors are with each other. According to one commonly used formula for computing the clustering coefficient of a graph (Eq. 7.1), the clustering coefficient of a node is the ratio of the number of existing edges and the maximum number of possible edges connecting its neighbors. The average over all |V | nodes gives the clustering coefficient of a graph (Eq. 7.2). CCv =

of edges between v s neighbors maximum of edges between v s neighbors

(7.1)

1 CCv |V | ∑ v

(7.2)

CC =

Fig. 7.4 shows plots of the clustering coefficient as a function of the number of nodes in the network. We observe that the clustering coefficient of our semantic clustering is much larger than that of the random power-law network. This indicates the emergence of clusters in the network. Semantic topology of the network is a crucial factor determining the efficiency of the query system. We measure the query evaluation system by recording the effectiveness of our semantic cluster-based topology. We compare the query performance of a semantic cluster topology with that of a semantics-free random topology. The performance measurement is based on the metric of recall rate which is defined as the number of results returned divided by the number of results actually available in the network. Fig. 7.5 depicts the findings. As it can be seen, query evaluation based on our adapted topology performs better as measured by recall rate. Semantic clusters effectively reduce the search space, and its ontology summary guides the query in the right direction. Therefore, the query system can locate results faster and more

Fig. 7.4 Comparison of clustering coefficient

7


161

accurately with this topology. This explains why our topology scales to large network size and why it achieves higher recall with smaller TTL. Besides all these reasons, another factor contributing the overall better recall rate of our topology is that it is able to locate semantically related results that cannot be located by the random topology. Because of the semantic heterogeneity of our experimental setup, relevant resources may be represented with different ontologies. The system using the semantic topology may use its ontology summary to find semantically related nodes and use the mapping defined to translate the query. Therefore, it can locate most of the relevant results. However, for unstructured random topology, they have no way to find semantically related resources. Therefore, they can only locate resources represented in the same ontology as the ontology of the querying node. Moreover, we show that our containment-based caching improves the query performance by reducing not only query traffic but also query latency. Fig. 7.6 and Fig. 7.7 demonstrate these two aspects respectively. In this experiment, we increase the skew degree of the query distribution from random to Zipf [25] with α =1.25. From Fig. 7.6 and Fig. 7.7 we can get two conclusions: • Caching, especially containment-based caching significantly reduces the query traffic and query latency. • Query distribution has a significant impact on the performance of caching. The more skewed the query distribution, the more effective the caching performs. According to [23], in an open and live distributed environment, query distribution is skewed and follows a Zipf distribution. Therefore, our caching scheme would be an effective strategy to improve the system performance.

(a) Comparison of query efficiency (recall vs. TTL)

(b) Comparison of system scalability (recall vs. network size) Fig. 7.5 Effect of topology adaptation on query performance

162

J. Li et al.

Fig. 7.6 Performance of caching (accumulated bandwidth vs. query distribution)

Fig. 7.7 Performance of caching (query latency vs. query distribution)

7.7 Conclusion In this chapter, we presented strategies for efficient sharing and querying over largescale distributed data sites. In particularly, we extended the traditional PDMS with a semantic layer. This layer allows peers to locate semantically related data sources that are not reachable by traditional schema mappings. Based on this semantic layer, we proposed an efficient clustering mechanism and an intelligent query routing scheme, which dramatically improves the query recall rate and the scalability of the system. As future work, we plan to further improve the aforementioned strategy by simplify the semantic similarity measurement for heterogeneous ontologies. Moreover, we plan to evaluate our system with real distributed data sets and deploy the system on large-scale distributed data sharing environments, such as a distributed medical community.

7


163

References 1. Bender, M., Crecelius, T., Kacimi, M., Miche, S., Xavier Parreira, J., Weikum, G.: Peerto-peer information search: Semantic, social, or spiritual? IEEE Bulletin of Computer Society Technical Committee on Data Engineering 30(2), 51–60 (2007) 2. Chawathe, Y., Ratnasam, S., Breslau, L., Lanhan, N., Shenker, S.: Making gnutella-like p2p systems scalable. In: Proceedings of ACM SIGCOMM 2003 (2003) 3. Choi, N., Song, I.Y., Han, H.: A survey on ontology mapping. ACM Sigmod Record 35(3), 34–41 (2006) 4. Gruber, T.R.: Principles for the design of ontologies used for knowledge sharing. International Journal Human-Computer Studies 43(3-4), 907–928 (1995) 5. Halevy, A.Y.: Answering queries using views: A survey. The VLDB Journal 10(4) (2001) 6. Halevy, A., Ives, Z., Madhavan, J., Mork, P., Suciu, D., Tatarinov, I.: The piazza peer data management system. IEEE Transactions on Knowledge & Data Engineering (TKDE) 16(7), 787–798 (2004) 7. Herschel, S., Heese, R.: Humboldt discoverer: A semantic p2p index for pdms. In: Proc. of the Workshop on Data Integration and the Semantic Web (2005) 8. Hose, K., Lemke, C., Sattler, K.U.: Processing relaxed skylines in pdms using distributed data summaries. In: Proc. of the 15th ACM International Conference on Information and Knowledge Management, pp. 425–434 (2006) 9. Hosea, K., Rothb, A., Zeitzc, A., Sattlera, K., Naumannb, F.: A research agenda for query processing in large-scale peer data management systems. Information Systems 33(7-8), 610–797 (2008) 10. Kalfoglou, Y., Schorlemmer, M.: Ontology mapping: the state of the art. The Knowledge Engineering Review 18, 1–31 (2003) 11. Li, J., Khan, S.U.: MobiSN: Semantics-based mobile ad hoc social network framework. In: Proc. of the IEEE Global Communications Conference (Globecom 2009), Honolulu, HI, USA, pp. 1–6 (2009) 12. Li, J., Vuong, S.T.: SOON: A scalable self-organized overlay network for distributed information retrieval. In: De Turck, F., Kellerer, W., Kormentzas, G. (eds.) DSOM 2008. LNCS, vol. 5273, pp. 1–13. Springer, Heidelberg (2008) 13. Li, J., Vuong, S.: Efa: an efficient content routing algorithm in large peer-to-peer overlay networks. In: Proc. of the Third IEEE International Conference on Peer-to-Peer Computing (2003) 14. Lodi, S., Penzo, W., Mandreoli, F., Martoglia, R., Sassatelli, S.: Semantic peer, here are the neighbors you want! In: Proc. of the 11th Extending Database Technology, pp. 26–37 (2008) 15. Miller, G.A., Beckwith, R., Fellbaum, C., Gross, D., Miller, K.J.: Introduction to wordnet: an on-line lexical database. In: Proc. of the International Journal of Lexicography (1990) 16. Pedersen, T., Patwardhan, S., Michelizzi, J.: WordNet: Similarity-measuring the relatedness of concepts. In: Proc. of the 19th National Conference on Artifical Intelligence, AAAI (2004) 17. Pires, C.E., Souza, D., Pachêco, T., Salgado, A.C.: A semantic-based ontology matching process for PDMS. In: Hameurlain, A., Tjoa, A.M. (eds.) Globe 2009. LNCS, vol. 5697, pp. 124–135. Springer, Heidelberg (2009) 18. Rada, R., Mili, H., Bicknell, E., Blettner, M.: Development and application of a metric on semantic nets. IEEE Transaction on Systems, Man, and Cybernetics 19(1), 17–30 (1989) 19. Ratnasamy, S., Francis, P., Handley, M., Karp, R., Shenker, S.: A scalable contentaddressable network. In: Proc. of the ACM SIGCOMM, pp. 161–172 (2001)

164

J. Li et al.

20. Rowstron, A., Druschel, P.: Pastry: Scalable, distributed object location and routing for large-scale peer-to-peer systems. In: Proc. of the IFIP/ACM International Conference on Distributed Systems Platforms (2001) 21. Stoica, I., Morris, R., Karger, D., Kaashoek, M.F., Balakrishnan, H.: Chord: A scalable peer-to-peer lookup service for internet applications. In: ACM SIGCOMM, pp. 149–160 (2001) 22. Tatarinov, I., Ives, Z., Madhavan, J., Halevy, A., Suciu, D.: The Piazza Peer Data Management System. In: ACM Sigmod Record (2003) 23. Valduriez, P., Pacitti, E.: Data management in large-scale p2p systems. In: Proc. of the International Conference on High Performance Computing for Computational Science, pp. 104–118 (2004) 24. Zhao, B.Y., Kubiatowicz, J.D., Joseph, A.D.: Tapestry: An infrastructure for faulttolerant wide-area location and routing. Tech. rep., UCB/CSD-01-1141 (2000) 25. Zipf, G.K.: Human Behaviour and the Principle of Least-Effort. Addison-Wesley, Cambridge (1949)

Chapter 8

Hierarchical Multi-Agent System for Heterogeneous Data Integration Aleksander Byrski, Marek Kisiel-Dorohinicki, Jacek Dajda, Grzegorz Dobrowolski, and Edward Nawarecki

Abstract. An agent-based framework dedicated to acquiring and processing distributed, heterogeneous data collected from the various Internet sources is proposed. Multi-agent based approach is applied especially in the aspects of: general architecture, organization and management of the framework. The sphere of data processing is structuralized by means of the workflow based approach. The concrete workflow is dynamically put together according to the user’s directives and information acquired so far, and after appropriate orchestration carried out by the agents. Possible application of the framework – the system devoted to searching for a personal profile of a scientist serves as an illustration of the presented ideas and solutions.

8.1 Introduction The growing importance of gathering and analyzing vast amount of information leads nowadays towards construction of complex systems that perform various caseoriented tasks with respect to data coming from numerous, often heterogeneous sources. The data must be found, merged and processed in order to have adequate, interesting results. All those activities form a process that has the specific „dynamics”. Its course strongly depends on what has been found or obtained from processing prior to a current moment of time. In order to support this process, complex – usually distributed – environments are constructed. In majority of them the process is a subject of full automation but some domains of application require also the user’s intervention in the process. Often the user’s decision is even necessary to carry on it. Aleksander Byrski · Marek Kisiel-Dorohinicki · Jacek Dajda · Grzegorz Dobrowolski · Edward Nawarecki Department of Computer Science, AGH University of Science and Technology, Al. Mickiewicza 30, 30-059 Kraków, Poland e-mail: {olekb,doroh,dajda,grzela,nawar}@agh.edu.pl P. Bouvry et al. (Eds.): Intelligent Decision Systems, SCI 362, pp. 165–186. c Springer-Verlag Berlin Heidelberg 2011 springerlink.com

166

A. Byrski et al.

In this contribution an agent-based framework dedicated to acquiring and processing distributed, heterogeneous data collected from the various Internet sources is proposed. The architecture of the framework – and consequently a particular system that can be built based on it – is modeled on AgE 1 platform that allow programming of the wide palette of soft-computing systems (see among others [7, 8, 9, 11]). The AgE platform can serve also as an experimental base of more general research like verification of functional integrity or [26] testing stochastic features of systems of that kind [10, 28]. In order to fulfill challenging requirements of distributed evolutionary computation like memetic optimization based on multi-deme algorithms [14] implemented around AgE or analysis of distributed data sources what is the subject of discussion here, multi-agent based approach [20, 21] is applied especially in the aspects of: general architecture, organization and management. Search strategies are implemented at an agent level, information sharing occurs through the local interaction between agents, the agents are providers of services, use of message exchange mechanism allows inferring the states of peer agents and thus their services. The concept of the framework is based on the so-called hierarchical data flow that structuralizes the applied processing. The operating agents are organized in trees; each tree deals with a coarse-grained, observed by the user task. If possible, parts of the tasks are delegated to the agents of lower levels of the trees to make a search enough wide or more subtle. The agents acquire the information and pass it to upper levels after appropriate filtering. The user can interact with the framework affecting the way of creating the mentioned tree-structures. Possibly even more complicated data processing is structuralized by means of the workflow based approach. The concrete workflow is dynamically put together according to the user’s directives and information acquired so far, and after appropriate orchestration carried out by the agents. The contribution begins with short presentation of AgE. Sec. 8.2 serves as introduction to agency in general as well as to the class of agent-based frameworks under consideration. Next comes a panorama of systems built for integration of heterogeneous information that gives a background for an original, presented in the course, framework. Sec. 8.4 contains yet another introductory material – a workflow model used in the presented framework. Sec. 8.5 gives detailed description of the hierarchical data integration and processing framework. The above is illustrated by the system devoted to searching for a personal profile of a scientist. The system is built around the proposed framework and integrates information taken from various sources in the Internet.

1

http://age.iisg.agh.edu.pl

8

Hierarchical Multi-Agent System for Heterogeneous Data Integration

167

8.2 AgE – Agent-Based Computation Framework During the last twenty years agent-based systems have been gaining more and more attention. Nowadays the idea of agency became common, however during these years many approaches utilizing biological, social and other inspiration allowed to enhance the agent-based systems yielding a distributed, autonomous systems aimed at solving different difficult problems. Agent-based systems represent a novel approach to analysis, design and implementation of complex (usually decentralized and distributed) computer systems. They offer plethora of concepts, techniques and tools allowing to improve the production process of software that may be applied to solve different problems [22, 5, 13, 12]. One distinguishable thing which these systems have in common is utilization of the abstraction of agent. Deliberations concerning agents and agent-based systems were first spotted in the research area of artificial intelligence. Gradually, they became more technological, concerning features of construction and typical applications of the agent-based systems. Nowadays, these systems may be perceived as an independent concept situated between artificial intelligence and software engineering. Agents are applied in many different situations [22] – starting from small applications as e.g. e-mail filters and ending on the complex critical systems such as medical support or flight control. The agent-based approach is also utilized by researchers using AgE platform and yielded promising results in heuristic soft-computing (see e.g. [7, 6, 29, 17]). AgE computational environment is developed as an open-source project at the Intelligent Information Systems Group of AGH-UST. AgE is a framework for development and run-time execution of distributed agent-based simulations and computations (applications). Fig. 8.1 presents the architecture of a system. User constructs the system by providing an input configuration in XML format which may be generated by a graphical designer tool. The configuration specifies the problem to solve, computation decomposition and problem-dependent parameters. Also, it defines a stop condition which must be fulfilled in order to finish the computation. After system start-up, environment and agents are created, configured and distributed among available nodes where their actions are executed. Additional services such as name, monitoring, communication and topology service are also run on these nodes to manage and monitor the computation. The output of the application is problem-dependent and is visualized by dedicated graphical tools and interpreted by the user. A computation is decomposed into agents which are responsible for performing a part of or the whole algorithm. Agents are structured into a tree composed of aggregates and agents, as shown in Fig. 8.2 according to algorithm decomposition (aggregates are also agents). It is assumed, that all agents at the same level are being executed in parallel. To increase performance, the top level agents (called workplaces) along with all their children can be distributed amongst many nodes. Agents, however, are not atomic assembly units, but they are further decomposed into functional units according to Strategy design pattern [18]. Strategies represent

168

A. Byrski et al.

System based on jAgE platform

Computing agents

Computing agents

User Computing agents

Fig. 8.1 AgE system architecture

Fig. 8.2 Agent tree structure

problem-dependent algorithm operators and may be exchanged without intruding the agents’ implementation. Their instances may be shared between agents as they provide various services to agents or others strategies. Both strategies and agents can use resource objects to represent problem-dependent data. Although, there may exist many different instances, all these objects are usually initialized in the same way with default or random values.

8


169

Each agent has an unique address which identifies it across the whole system. The address is registered in an external name service which tracks information about current agent locations. With the use of the environment, agents can communicate with their neighbor agents via messages or queries. They may also ask their neighbors to perform specific actions. The platform introduces two types of agents: thread-based and simple. The former are realized as separate threads so that the parallel processing is managed by Java Virtual Machine (similarly to Jade platform2). Such agents can communicate and interact with neighbors via asynchronous messages. However, a large number of such agents would significantly decrease the performance and efficiency of a computation because of frequent context switching, therefore the notion of simple agent was introduced. The concept of simple agent is based on event-driven simulation, that results in pseudo-parallel execution of agents’ tasks. The two phases are distinguished: • Execution of tasks related to computation semantics in the step() method. In case of an aggregate agent all it’s children perform their steps sequentially. While doing so, they can register various events, which may indicate actions to perform or communication messages, in the parent aggregate. • Processing of events registered in an event queue. Since events may be registered only in agents that possess children, this phase concerns only aggregate agents. The described idea of processing ensures that during execution of computational tasks of agents co-existing at the same level in the structure (agents with the same parent), the hierarchy remains unmodified, thus the tasks may be carried out in any order. From these agents perspective, they are processed in parallel. All changes to the agent structure are made by aggregates during processing of the events indicating actions such as addition of new agent, migration of an agent, killing an already existing agent, etc. They are visible for agents while performing the next step. Each agent exists in an environment, defined by the parent agent, which provides a context of agent’s processing. This means that the environment is responsible for communication between agents and for responding to queries from agents. What is more, according to the presented concept, the environment of simple agents determines the types of actions which may be ordered by child agents. It also provides concrete implementations of these actions and thereby supplies and influences agents’ execution. The actions can be executed via parent agent’s methods or external strategies specified in the input configuration. Actions realize the agent principle of goal level communication [4], because agent only lets the environment know what it expects to be done but it does not know how it will be done. Decision of how to execute the action is made by the parent agent who resolves proper action implementation according to Service Locator design pattern [16]. Both strategies and agents can have named properties, which are features of an object, which can be referenced during runtime by its name in order to access, modify or even monitor its value. Properties may be accessed via methods 2

http://jade.tilab.com/

170

A. Byrski et al.

or field. In the former case the accessor method has to be annotated with the use of PropertyGetter tag and, if the property may be modified, the mutator method has to be annotated with corresponding PropertySetter tag. In case of fieldbased properties, only the field has to be annotated with PropertyField tag this type of properties is always modifiable. For each class a list of its annotated properties may be retrieved and each named property of the class’s instance may be accessed in a uniform way. The properties may be of simple as well as reference types. In the presented agent platform, three types of units were distinguished: agents, strategies and resource objects. In our approach all of them should be loosely coupled [33], what means that the dependencies and associations between them are realized through well-defined interfaces and not by concrete implementations. This not only hides realization details but also allows for exchanging dependent classes without code modifications, thus without recompilation. This is essential in terms of flexible component assembly framework. Moreover, such units can have simple parameters with integer, double, boolean or string values which may be used to tune the algorithms for specific problems. These parameter values should be also set by the assembly framework based on the input configuration. Aggregate agents, which are also the agent environments, should have the ability to retrieve actions (implemented as strategies) by resolving them by name or type. The decision which implementation is provided should be made during run-time based on the system configuration, without any changes in the aggregate classes. What is more, environments can provide different implementations of the same action, depending on the level in the agent structure. For instance, migrating genotypes between two agents representing flocks has different implementation than migrating flocks between two islands, but still both are migration actions. As it was mentioned before, strategies provide services, thus they can be shared between many agents. Usually, there exists one instance of strategy of each type in a system. On the contrary, agents and resources are distinct objects and, in most cases, there are many instances of them in a system. This implies two different ways of providing assembling units: the first, which caches an instance and returns it in consecutive requests, and the second, which creates multiple instances of the same unit.

8.3 Panorama of Systems for Integration of Heterogeneous Information The problem of heterogeneous data integration affects various research domains from large database warehouses for business intelligence to medical systems supporting health center operation. It also includes several aspects such as resolving semantic heterogeneities of the available sources. Current state of research points out several approaches such as Data Warehousing, Federated Database Systems and Mediator-based approach.

8


171

Over the last few years, ontologies become one of the most promising techniques of database integration. For example, in [24], an infrastructure for seamless access to integrated clinical, genetic, and image databases is proposed. For integration purposes, a set of ontologies describing data in each database is used. The infrastructure provides 2 groups of services. The first one is aimed for resolving syntactic and semantic heterogeneities when accessing integrated databases. The second group acts as a core resource supporting the ontologies integration process. In [23] authors present a middle-ware to integrate information from heterogeneous enterprise systems through ontologies stored in XML standard. To realize the semantic integration authors utilize mediator/wrapper architecture including such functions as data descriptions, schema transformations and integrations based on the XML-Schema standard. The integration process usually requires complex architectures which constitute another important aspect of the problem. This complexity can be overcome by assembling system from independent specialized components. In [31], the ontologybased architectures are presented and compared. While they differ in details, the general concept relies on the same assumptions of the centralized and predefined flow of control. Beside providing the semantic data integrity, [1] proposes a way to integrate data sources also on the level of operations. This is achieved by describing data sources with a process description language SWPDL introduced in the paper. As result, the integration of sources is realized by composition of processes that refer to the identified use cases for the given information source. A similar idea is to integrate pure application systems which encapsulate databases rather than pure databases themselves is presented in [19]. As a result, the combination of generic a query with predefined function access is needed in order to integrate heterogeneous data sources. The proposed approach is to utilize the classic concept of federated database system (FDBS) but to equip it with a workflow management system (WfMS) that provides a middle access to the integrated applications and systems. Additionally, authors propose a lightweight description language based on XML to describe the global-to-local mapping of functions, which formulate the actual processing workflow. Integration of services rather than data is also proposed in [2]. The paper discusses integration of web services described by WSDL language using ontologybased mappings. In this way, the integration can include all web systems and repositories which provide a web service interface. To facilitate the creation of ontology-based mappings, the paper proposes OntoMat-Service tool, which trades off between having a reasonably easy to use interface for web services and the complexity of web service workflows. An interesting idea is to use annotations to describe the services and create mappings between different interfaces. Recently, the integration of data and services is regarded also in the context of agent-based systems. In [25] a Distributed Data Mining technique based on a multi-agent environment called SMAMDD (Multi-Agent System for Distributed Data Mining) has been introduced. This technique uses an integration model which assumes combining local models into the global, consistent one. This is done by

172

A. Byrski et al.

fetching the results of tasks performed by local agents and merging them into a global model. In order to achieve that, agents cooperate by exchanging messages. This improves the process of knowledge discovery and, in result, allows for obtaining more accurate results. Model integration applied to agent-based systems realizing data mining algorithms is also discussed in [15].

8.4 Basic Model of Data Transformation Data transformation in the described system may be modeled with help of workflows that may be visualized using a subset of UML standard [3]. Several symbols available in the Visual Paradigm basic diagram [32] and the feature of UML extensibility is utilized by assigning them new meaning, to be able to use them in data flow modeling. The basic element of a workflow is a task. The task is responsible for transforming the input data to a specific output. For example it can be responsible for filtering the specific data within all data set. Every task has an input and an output of specific predefined types that define the context of the task. Additionally, every task has a configuration used to tune it to current needs. Fig. 8.3 illustrates this concept.

Task

config

Fig. 8.3 Workflow task

Obviously, the number of possible inputs or outputs can be greater than one. The exact number can differ depending on a given task. However, assuming that the input or output types (call composite) may consist of other types (called basic), the operator illustrated in Fig. 8.3 also refers to these cases. Having defined the task as a core workflow element, it is time to describe how to build a workflow from tasks. To do this, it must be possible to combine two tasks. Tasks can be combined by considering the output of a task as an input to another task. In this way, a workflow of two tasks is built, which is illustrated in Fig. 8.4. Of course, the tasks cannot be combined freely, but must be compatible with each other. Their compatibility is defined at the level of their interfaces, which means that their output and input types must be compatible. Therefore, we can tell that two tasks are compatible if one of the following conditions is fulfilled: • Type1(Output) equals Type2(Input)

8


Task1

173

Task2

Task12

Fig. 8.4 The concept of tasks matching

• Type2(Input) is a generalization of Type1(Output) Obviously, a combination of tasks can be considered as a composite task with the same input type as for the first task and the output type of the second task. In Fig. 8.4, the combined Task12 has input type Type1(Input) and the output type Type2(Output). Having defined how to combine the tasks, whole workflows can be constructed. Apart from tasks, an important element of a workflow is a dataset. While tasks are responsible for transformations and the dynamics of the workflow, datasets are responsible for storing the information that the tasks operate on.

8.5 Hierarchical Data Integration and Processing In the current section an adaptation of the existing infrastructure offered by AgE platform to the problem of distributed data integration and processing is proposed. It seems, that the concept of agent fits well to the problem, as it might be entrusted an action of mining certain information, delegate refinement, merging, transformation to other agents and provide the intelligent interface for the user.

8.5.1 System Environment and User Interaction The system presented in our contribution follows several approaches already presented in some of the above mentioned works. E.g. autonomous agent-based approach that leads to decomposition of task or domain of interest, or ontology-like types to describe the processed data is used. However, as one of the main advantages of our system one may consider the possibility of introducing different levels of agent-user interactions, according to one of the following strategies:

174

A. Byrski et al.

• Autonomous work of the user: The main menu of the system proposes a list of data sources available to the analyst user in a given case, as well as the list of all core system functionalities corresponding to available data sources. After selecting a data source and corresponding functionality the analyst obtains results in a form defined by him or her. The analyst decides then whether to finish or continue the work with the system. • Mode of limited consultancy: The user, apart from the choice of the core functionality and importing corresponding data, enters a short description of a given case compliant with a chosen template-ontology. The system suggests the possible data transformations and reacts to the user expressing his/hers preferences by modifying the data flows. The action suggested by the system is undertaken only when it is approved by the user analyst. • Automated support of user activities: The system operates in an automated and autonomous way without the analyst participation. The analyst only receives the results of the analyzed scenarios. The analyst can only interrupt the sequence of system actions or introduce additional tasks, which can emerge from the already obtained results of the running sequence or from other kinds of data exploration. To provide the system with the capabilities required for implementing this class of strategies, advanced procedures of reasoning based on methods of artificial intelligence are required. The first strategy is indeed very simple and becomes a starting point for implementing the second one. This may be the most suitable for the user, who wants to influence the system’s work, not getting rid of all responsibilities for defining the data flows. The third strategy is a very high-level one and is to be considered in the final stage of system construction. Let us focus on the first possible strategy of the system interaction and consider the systems consisting of various components of data extraction, transformation and analysis that may be localized and taken advantage of by agents: • Data extraction components lead to establishing connection between the system and the external data sources localized in the Internet. • Data transformation components are used to process the acquired data to adapt them to specific needs of users or agents. • Data analysis components are used to perform actions such as inference or hypotheses testing on the already preprocessed data. Agents created in the system are assigned with a specific task (e.g. gathering the data, verifying some hypothesis etc., according to predefined ontology). It searches the data using the data extraction, transformation and analysis components, referring to them directly (or indirectly, delegating the work to other specialized agents, being the brokers for the desired functionality). The delegation might take form of an agent tree, where the root agent will delegate some task to subtree including blackboard, agents and connected to external resources.

8


175

As an example, a system for processing information that will be used to prepare a scientists profile will be presented. Starting with simple personal data such as name and surname, some information about the person may be gathered in different popular Internet portals (using data extraction and transformation components). The tree-like delegation may occur, when an agent finds the data about a specific scientist in a general system (e.g. OPI 3 ), discovers that the person works in a specific university, say AGH-UST and then delegates the action of refining of some of his/hers personal attributes by gathering additional facts in other systems, being e.g. part of some institutions connected with the person. Such a locally searching node of agent tree will take into considerations information located in the portals run by authorities connected directly with AGH-UST (e.g. SKOS 4 or BPP 5 ). In the end, some hypotheses can be verified, e.g. if a person under considerations cooperates with another one, and to what extent (using data analysis components).

8.5.2 Agent-Based Data Integration Workflow Model The process of data transformation of the system may be perceived as execution of parallel tasks that are assigned to agents. These tasks may be presented in the form of workflows (see Section 8.4). It is to note, that the workflows in the system do not show the actions that are executed in the time-dependent manner, but the system of cooperating processes that exchange the data among them. In Fig. 8.5 general system workflow is displayed. In the certain data context a group of agents will perform their tasks. Each task is an independent process, that can however exchange data with other ones. Each task takes some input data and depends on configuration. System-user interaction may lead to instantiate new task, just like in the presented example, execution of task 1 could lead (after receiving a signal from the user) to creating of the task 2. They seem independent, however they may exchange some data. The user’s decision make certain tasks start, in the end it may be said, that three processes run independently, communicating among themselves and sharing information. Note, that executing of one task, after receiving a signal from the user may lead to construction of tree of sub-tasks that are e.g. refining the general search performed in the root task (see subtask 1 and subtask 2 in Fig. 8.5). The above mentioned tree of tasks is shown in Fig. 8.6. Here “subtasks” and “subsubtasks” are constructed and executed. They are dependent on its root tasks and provide it with the results of their search.

3 4 5

http://www.opi.org.pl http://regent2.uci.agh.edu.pl/skos/ http://www.bpp.agh.edu.pl

176

A. Byrski et al.

Aggregate

input data 1

import strategy 1

input data 2

import strategy 2

config

input data 3

import strategy 3

config

config

config

subtask 1

subtask 2

config

merge

visualize 2

output data ds10

merge

visualize 2

output data ds11

analysis1

config

merge

analysis2

config

config

config

config

config

Fig. 8.5 General system workflow

task 1

subtask 1

subsubtask 1

subtask 2

subsubtask 2

Fig. 8.6 General workflow tree

This way of perceiving the system (as a set of autonomously executed, partially dependent tasks) shows that designing and implementation of the whole system with use of agent-based technologies seems appropriate.

8.5.3 Multi-Agent System Structure The system consist of a group of nodes distributed in the network (see Fig. 8.7). Each node may consist many agent trees. An agent tree is started by assigning a

8


177

certain task to an agent. The main goal of the agent is to interact with the given task, e.g. by filling out its missing attributes (e.g. if a task subject is a person, an agent may use external resources to find the subject’s email or other personal data).

Fig. 8.7 General structure of the system

8.5.4 Tasks, Objects and Data Types The data placed in the system is contained in objects, that are annotated with types (e.g. person) and set of properties (his/hers name, surname email, phone, place of work etc.). Some of the properties are simple, consisting of one real-value variable or a string. Other may be complex (e.g. realized as aggregation of other objects). The data is described by the set of types (annotating the actual structures of the programming language). The type structure may be described using ontologies, however it is not required for simple implementations of the system (see Section 15.8). These types are interrelated, creating a specific structure. Two types may be connected with association relation. The connection like this shows, that one type may be described in details by another one. For example, a person may have an attribute: address. The address may be then described in details by giving street name, apartment number and so on. Other possible connection is generalization/specialization among two similar types. It shows that one of the types is more general and other more specialized.

178

A. Byrski et al.

Objects are entrusted to agents. This creates a specific context for processing the data. Agents contain one or more objects and information about their relations. Agents may add or modify the objects and their relations. In order to modify certain object or relation, an agent must possess appropriate interface annotated with a type from the above mentioned type hierarchy. Agents contain objects and are given specific tasks by the user to fulfill them. For example agent may be given a tasks of finding personal data for a given object in a certain system.

8.5.5 Tree of Agents Agent s may be treated as containers of intelligent implementation of certain algorithms, aimed at attaining some predefined goals. The agents apply the contained algorithm to data (if possible). Their algorithms may incorporate parts of the globally available components (accessible via resources service). Agents are situated in nodes of agent tree, serving as a local environment. They access the objects contained in the task and interact with the resources to retrieve the data. An agent may delegate a task to an agent which can be a “subtree” in the system organizational structure, being a local environment in which the refinement of the search process will be performed.

8.5.6 Roles of Agents The agents may be assigned with roles that determine their goals. In a discussed system, that is aimed at processing various data gathered in the Internet, the roles will focus on different aspects of processing: • Updater: locates an interesting object being the part of the local task and updates it by filling out lacking information referring to certain data sources from external systems (e.g. filling out the e-mail of the person whose name and surname were placed there). • Verifier: performs a verification the correctness of the data based on different data source (e.g. correcting the phone number if different ones may be found in data sources). • Simplifier: removing data redundancy (e.g. merging information about the person’s employment found in different sources). • Delegate: delegates data processing to the subtree. The delegation is usually performed with help of user (by asking him/her to refine the search criteria or add new ones). • Filter: every root of subtree acts as a local environment for the agents but also as a filter for passing interesting part of the information to the upper agent (or to user via GUI).

8


179

8.5.7 Actions of Agents Generally agents assigned with all the above mentioned roles may perform one or more of the following actions: 1. Looking for suitable data on blackboard (based on annotations with the predefined types). 2. Retrieving the data from blackboard. 3. Applying the algorithm (performing appropriate calls to the services offered by the resources) to process the data. 4. Delegating the processing of the object to sub- (or upper) area. 5. Returning the data to the blackboard.

8.5.8 Resources of the System As system resources, various components that may be accessed in a fashion similar to Strategy design pattern [18] and applied to perform various transformation of the objects present in the systems may be understand. The available strategies are in fact reusable software components [30]. Their availability is assured by a platform implementing dependency injection/inversion of control pattern [27]. Several possible types of resources can be identified: • External systems adapters or façades. These are strategies that may be called in order to retrieve the data from the system. For instance, the portal containing the personal data about the employees of a certain company will be given an adapter that will translate the requests for getting the names, surnames, phone numbers etc. • Database adapters. These strategies will add possibilities of storing the results of the queries, retrieve them without the need to search for it in the original systems. These resources will have to verify the contained data in certain periods of time to check for updates. • Reusable code components. These are implementations of parts of certain algorithms. A general function for retrieving the data about a person (name, surname, email etc.) may be implemented in a different way depending on the system that is accessed. These resources are implemented based on the strategy design pattern [18].

180

A. Byrski et al.

8.6 Searching for Personal Profile of a Scientist – An Example The application of the proposed infrastructure to the problem of finding case-oriented information about scientists, starting with exploration of the most popular social networking system system (http://facebook.com/) is proposed. Later (or in parallel) the search can be refined based on other popular scientific data sources (e.g. DBLP http://www.informatik.uni-trier.de/~ley/db/ or Google Scholar http://scholar.google.com. Cases in the area of interest may include e.g. assisting in evaluation of a person that is applying for the scientific project.

8.6.1 Construction of Scientist’s Profile The manual process of gathering information about a particular scientist may look as follows: let us say that information Jan Kowalski (eng. John Doe) is found in Facebook, however then, additional information may be gathered after realizing, that he is Polish. In this case popular scientific data portal OPI (in Polish: Osrodek Przetwarzania Informacji, in English: Information Processing Center) may be referred to. Then after realization that he works at the AGH University of Science and Technology in Kraków, Poland, one could further refine the search by accessing the servers provided by the AGH University (e.g. AGH SKOS database of employees http://regent2.uci.agh.edu.pl/skos/ or bibliographic information BPP http://bpp.agh.edu.pl), thus adding new information to the profile. Following the publications presented by Jan Kowalski, other bibliography vortals such as DBLP http://www.informatik.uni-trier.de/~ley/db/ may be referred to. Of course the data found there should be cross-verified with the data gathered in BPP and the duplicates should be removed. The analyst may be interested only in the bibliographic data contained in BPP or in DBLP, and following his demands one or another source of information may be neglected. Looking for additional information to complete the profile of Jan Kowalski, other community services may be referred to, global ones, such as LinkedIn6, or more constrained, e.g.: GoldenLine7. The process of completing the profile for Jan Kowalski looks quite complex, it may be of course done manually, however the particular parts of this process may be performed automatically, and the way of dealing with the problem encourages application of agent technologies.

6 7

http://linkedin.com http://goldenline.pl

8


181

8.6.2 Example Data Flow Assuming, that the goal is to find all the important information about Jan Kowalski, the following flow of events may be considered: 1. Person data object is created by the user and assigned to a subtree. 2. An agent capable of filling out the data (fulfilling the role of updater) accesses the object. It refers to appropriate database (e.g. Facebook) and fills out such informations as email, ID, places of work, colleagues etc. 3. User interacts with the system and points out that one of the colleagues of Jan Kowalski, namely Andrzej Nowak is worth interest. The system delegates the search for the data concerning Andrzej Nowak to a instance of subtree capable of finding such information. 4. The user interacts with the system and points out that he wants to know, what Jan Kowalski and Andrzej Nowak have in common. In this way, after spotting the suitable data on the BB, an agent may activate and try to fill it out, refine, or delegate this action to other agents or subareas. There may be different additional actions (and agents) introduced. Here are some examples: • An agent to merge the information about publications retrieved from AGH BPP and DBLP. It should remove duplicates and construct one list of publications for the person. It may delegate the action of filling out the impact factor, computation of h-index and other parameters. It’s search may be refined after interaction with the user, and the publications of interest may be constrained e.g. to several years. • An agent to find the information about collaborators of a person and divide them based on e.g. country of provence, research interests, cooperation time period and others. • An agent to retrieve the information about the supervised projects and persons and fill out (by delegating) the information about their attributes and so on. The flow of data, as described above, may be presented with use of workflows defined in Section 8.4. In Fig. 8.8 the domain search decomposition may be observed. First the task 1 is created by the user in order to search for the information regarding Jan Kowalski. The user decides, that one of the collaborators of Jan Kowalski, namely Andrzej Nowak, may be interesting, so starts search for the feasible information in a capable subtree. Search for Jan Kowalski may create subtasks that will be performed in order to explore local systems provided e.g. by Jan Kowalski’s employer (see Fig. 8.9).

8.6.3 Set of Types In order to annotate the data used in the system, the following set of types is considered:

182

A. Byrski et al.

Aggregate config config

FACEBOOK

import strategy 1

Jan Kowalski JK @ AGH

JK @ DBLP JK AN friends

FACEBOOK

import strategy 2

config

visualize

output data ds6

Andrzej Nowak

config

config

config

Fig. 8.8 Scientists profile data search workflow

Jan Kowalski

JK@AGH

JK@AGH DEPT

JK@DBLP

JK@AGH BIB

Fig. 8.9 Scientists profile data search tree

• Personal data (properties such as name, email, phone number, place of work). • Institution (properties such as name, address, number of employees and so on). Notice, that the mentioned above personal data may be associated with Institution (place of work). • Bibliography position (properties such as name of the contribution, authors, journal name etc.). • Scientific projects (properties such as date of start/end, topic, abstract, collaborators, institution).

8


183

All the above mentioned types may have their specializations e.g. Personal data in general, personal data gathered from OPI or from AGH SKOS (properties may differ). The detailed description of the types and ontologies that may be used to describe the data is out-of-scope of this contribution and will be presented in another one.

8.6.4 System Environment and Structure The system consists of a tree-like hierarchy of agents described above (see Fig. 8.7). The agents may access external resources of data (starting from Facebook server, refining information by using other portals dedicated to serving information about scientists – e.g. employees database served at Universities, international citation services etc.). The agents trees are situated in AgE workplace that resides on the node (see Section 8.2).

8.6.5 Agents, Their Actions and Their Goals The acting of agents is based on the algorithms described above. Generally speaking agents access a suitable data contained by the task and, process it. The agents use the predefined resources to reach the goals. E.g. an agent capable of filling out the data of a person (assign all attributes) based on the OPI database may take responsibility for processing „person” data in its local task. Then, after discovering that the person of interest is a Pole, the further search may be delegated to an appropriate subtree and refine the search by accessing appropriate AGH servers. There may be many agents working in parallel in the area, e.g. if the area is responsible for filling out the data of bibliography for a certain person, one agent may be responsible for examining DBLP service, other for AGH BPP and so on. The agent tree is adapted right from the design and implementation of the AgE platform (see Section 8.2).

8.6.6 System Resources Resources are constructed with use of AgE strategies (see Section 8.2) and their aim is to provide homogeneous means to access the external data (based on the strategy design pattern). Therefore, the general interface for retrieving the data of a certain person may be defined and adjusted implementations may be provided for all accessible external resources. E.g. there will be different implementations for accessing the data of a person in Facebook, OPI, or AGH SKOS, though the interface of this component may be the same.

184

A. Byrski et al.

Thus resources such as personal data finder, publications finder, company profile identifier or scientific grants finder may be proposed, implemented in different ways for all potentially worthwhile data sources and incorporated into the system.

8.7 Conclusions In the course of this contribution an agent-based framework dedicated to acquiring and processing distributed, heterogeneous data, collected from the various sources also from Internet has been proposed. The framework assumes three modes of cooperation with a user: system is completely user-driven, system is semi-automatic, and system works fully autonomously. Because of this systems build over the framework can be of the decision support kind and search results can be reacher as an effect of human and artificial intelligence synergy. The framework provides the user with possibility of search domain decomposition using agent-based techniques. Implementing new task-oriented agents that may act in a different way with respect to a different information source creates sound base for flexible and effective organization of the system. The hierarchical (trees) structure of the agent organization allows delegation of the parts of the supported activities to specialized agents. The potential capabilities of the platform such as: distribution, decentralization, support for code reuse, and possibility of implementation of resilience support makes it an interesting system tool of relatively wide spectrum of application. The concept of the platform has been illustrated with a use-case – the system that searches for, gathers and processes data about scientists taken from wide-accessible Internet services. Last but not least, it is to mention two main advantages of the proposed system: • There may be any possible data source connected to the system. The only condition is that the user is able to define a scheme of accessing the data (e.g. by annotating certain elements of websites with available types) and implement it in the appropriate façade (strategy). Therefore the data may be imported from diverse sources such as databases, spreadsheet files, blog servers, news servers etc. • The approach is not aimed at mimicking the Google by creating the profiled version of this search engine. On the contrary, its goal is to define exactly what is the format of the data that is sought, resigning from searching everywhere. In authors’ opinion, this approach will lead to quick finishing of the system construction and will allow to quickly retrieve the most interesting data in a controlled manner. In the near future implementation and testing of the proposed framework are going to be finished. The focus will be put also on exploring different aspects of the framework that could not be described in details along this contribution (e.g. ontologies for object annotations).

8


185

Acknowledgment The research leading to these results has received funding from the European Community’s Seventh Framework Program – Project INDECT (FP7/2007-2013, grant agreement no. 218086).

References 1. Agarwal, S., Haase, P.: Process-based integration of heterogeneous information sources. In: Dadam, P., Reichert, M. (eds.) INFORMATIK 2004 - Informatik verbindet, Band 2, Beiträge der 34. Jahrestagung der Gesellschaft für Informatik e.V (GI), Ulm, September 20-24. LNI, vol. 51, pp. 164–169. GI (2004) 2. Agarwal, S., Handschuh, S., Staab, S.: Surfing the service web. In: International Semantic Web Conference 2003, pp. 211–226. Springer, Heidelberg (2003) 3. Ambler, S.W.: The Elements of UML 2.0 Style. Cambridge University Press, Cambridge (2005) 4. Bergenti, F., Gleizes, M.P., Zambonelli, F.: Methodologies and Software Engineering for Agent Systems. Kluwer Academic Publishers, Dordrecht (2004) 5. Bradshaw, J.M. (ed.): Software agents. MIT Press, Cambridge (1997) 6. Byrski, A., Kisiel-Dorohinicki, M.: Immunological selection mechanism in agent-based evolutionary computation. In: Klopotek, M., Wierzchoń, S., Trojanowski, K. (eds.) Proc. of the Intelligent Information Processing and Web Mining IIS IIPWM 2005, Gdansk, Poland. Advances in Soft Computing, Springer, Heidelberg (2005) 7. Byrski, A., Kisiel-Dorohinicki, M.: Agent-based evolutionary and immunological optimization. In: Proc. of the 7th International ConferenceComputational Science - ICCS 2007, Beijing, China, May 27-30, Springer, Heidelberg (2007) 8. Byrski, A., Kisiel-Dorohinicki, M.: Agent-based model and computing environment facilitating the development of distributed computational intelligence systems. In: Allen, G., Nabrzyski, J., Seidel, E., van Albada, G.D., Dongarra, J., Sloot, P.M.A. (eds.) ICCS 2009. LNCS, vol. 5545, pp. 865–874. Springer, Heidelberg (2009) 9. Byrski, A., Kisiel-Dorohinicki, M., Nawarecki, E.: Agent-based evolution of neural network architecture. In: Hamza, M. (ed.) Proc. of the IASTED Int. Symp.: Applied Informatics. IASTED/ACTA Press (2002) 10. Byrski, A., Schaefer, R.: Stochastic model of evolutionary and immunological multiagent systems: Mutually exclusive actions. Fundamenta Informaticae 95(2-3), 263–285 (2009) 11. Byrski, A., Dobrowolski, J., Tobola, K.: Agent-based optimization of neural classifiers. Prace Naukowe, Elektronika, Politechnika Warszawska (2008) 12. Byrski, A., Kisiel-Dorohinicki, M., Carvalho, M.: A crisis management approach to mission survivability in computational multi-agent systems. Computer Science 11 (2010) 13. Caglayan, A., Harrison, C.: Agent sourcebook. John Wiley & Sons, Inc., New York (1997) 14. Cantú-Paz, E.: A summary of research on parallel genetic algorithms. IlliGAL Report No. 95007. University of Illinois (1995) 15. Cao, L.: Data mining and multi-agent integration. Springer, Dordrecht (2009) 16. Malks, D., Alur, D., Crupi, J.: Core J2EE Patterns: Best Practices and Design Strategies. Prentice-Hall, Englewood Cliffs (2003)

186

A. Byrski et al.

17. Dre˙zewski, R., Siwik, L.: Agent-based co-operative co-evolutionary algorithm for multiobjective optimization. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2008. LNCS (LNAI), vol. 5097, pp. 388–397. Springer, Heidelberg (2008) 18. Gamma, E., Helm, R., Johnson, R., Vlissides, J.: Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley, Reading (1995) 19. Hergula, K., Härder, T.: A middleware approach for combining heterogeneous data sources - integration of generic query and predefined function access. In: Proceedings of the First International Conference on Web Information Systems Engineering (WISE 2000), vol. 1, IEEE Computer Society, Washington, DC, USA (2000) 20. Jennings, N.R., Sycara, K., Wooldridge, M.: A roadmap of agent research and development. Journal of Autonomous Agents and Multi-Agent Systems 1(1), 7–38 (1998) 21. Jennings, N.R., Wooldridge, M.J.: Software agents. IEE Review, 17–20 (1996) 22. Jennings, N.R., Wooldridge, M.J. (eds.): Agent technology: foundations, applications, and markets. Springer-Verlag New York, Inc., Secaucus (1998) 23. Li, S., Zhang, D.H., Zhou, J.T., Ma, G.H., Yang, H.: An xml-based middleware for information integration of enterprise heterogeneous systems. Materials Science Forum 532, 516–519 (2006) 24. Martín, L., Anguita, A., Maojo, V., Bonsma, E., Bucur, A.I.D., Vrijnsen, J., Brochhausen, M., Cocos, C., Stenzhorn, H., Tsiknakis, M., Doerr, M., Kondylakis, H.: Ontology based integration of distributed and heterogeneous data sources in acgt. In: Azevedo, L., Londral, A.R. (eds.) HEALTHINF (1), INSTICC - Institute for Systems and Technologies of Information, Control and Communication, pp. 301–306 (2008) 25. Pilatti de Paula, A.C.M., Avila, B.C., Scalabrin, E., Enembreck, F.: Multiagent-based model integration. In: Proceedings of the 2006 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology, WI-IATW 2006, pp. 11–14. IEEE Computer Society, Washington, DC, USA (2006) 26. Pietak, K., Wos, A., Byrski, A., Kisiel-Dorohinicki, M.: Functional integrity of multiagent computational system supported by component-based implementation. In: Proc. of Holomas 2009, Linz, Austria (2009) (accepted for printing) 27. Prasanna, D.R.: Dependency Injection. Manning Publications (2009) 28. Schaefer, R., Byrski, A., Smołka, M.: Stochastic model of evolutionary and immunological multi-agent systems: Parallel execution of local actions. Fundamenta Informaticae 95(2-3), 325–348 (2009) 29. Siwik, L., Dre˙zewski, R.: Agent-based multi-objective evolutionary algorithm with sexual selection. In: Proceedings of the IEEE World Congress on Computational Intelligence (WCCI 2008). IEEE, Los Alamitos (2008) 30. Szyperski, C.: Component Software: Beyond Object-Oriented Programming. AddisonWesley Longman Publishing Co., Inc., Boston (2002) 31. Tamma, V., Visser, P.R.S.: Integration of heterogeneous sources: Towards a framework for comparing techniques. In: Proceedings of the "6 Convegno dell’Associazione Italiana per l’Intelligenza Artificiale (AIIA), pp. 89–93 (1998) 32. Tsang, C.: Object-Oriented Technology from Diagram to Code with Visual Paradigm for UML. McGraw-Hill Science/Engineering/Math (2005) 33. Constantine, L., Stevens, W., Myers, G.: Structured Design. IBM Systems Journal

Chapter 9

Emerging Cooperation in the Spatial IPD with Reinforcement Learning and Coalitions Ana Peleteiro, Juan C. Burguillo, and Ana L. Bazzan

9.1 Introduction Game theory provides useful mathematical tools to understand the possible strategies that self-interested agents may follow when choosing an action. The context of evolution of cooperation has been extensively studied seeking general theoretical frameworks like the Prisoner’s Dilemma (PD) [2]. In his seminal work, Axelrod has shown that cooperation can emerge in a society of individuals with selfish motivations. Since that, Game Theory and the Prisoner’s Dilemma have been applied in biological, social, economical and ecological contexts. An interesting spatial version of the PD has been suggested and deeply analyzed in [22] trying to understand the role of local interactions in the emergence and maintenance of cooperation. This work focuses on the spatial interaction along generations of cooperators and defectors by means of computer simulations and learning in game theory. We agree with Axelrod [3] in that although the topic being investigated may be complicated, the assumptions underlying the agent-based model should be simple when our goal is to understand fundamental processes. This approach is followed along this chapter. Following the spatial configuration for the PD proposed by [22], we consider an agent population placed on a square lattice and simulate the dynamics of the population by means of agent cells (from now on we use agent and cell indistinctively). Ana Peleteiro Telematics Engineering Department, University of Vigo, 36310-Vigo, Spain e-mail: [email protected] Juan C. Burguillo Telematics Engineering Department, University of Vigo, 36310-Vigo, Spain e-mail: [email protected] Ana L. Bazzan Instituto de Informatica, Universidade Federal do Rio Grande do Sul, 15064-Porto Alegre, RS, Brazil e-mail: [email protected] P. Bouvry et al. (Eds.): Intelligent Decision Systems, SCI 362, pp. 187–206. c Springer-Verlag Berlin Heidelberg 2011 springerlink.com

188

A. Peleteiro, J.C. Burguillo, and A.L. Bazzan

The interaction between the agents is modeled as an n-person game, i.e., n agents interacting simultaneously. Every agent may behave as a defector or a cooperator when playing isolated, but they can also join coalitions (group of agents). The framework of this game is a spatial distribution where cells are agents that play the Iterated Prisoner’s Dilemma (IPD). The scenario is a spatial grid where every cell plays with its neighbors, using the standard 2x2 pay-off matrix. Depending on the action selected, and the action that its neighbor selects, every cell receives a pay-off. The main question analyzed in this chapter is the emergence of coalitions as a way to support cooperation in a defection environment. This question has been also addressed in [21]. However, the aim of our simulations is to investigate whether the use of learning techniques (we employ Learning Automata and Q-Learning) and coalition formation enhances the emergence and maintenance of cooperation, when compared to the behavior of individual agents. Therefore, the main contribution of this chapter is the exploration coalitions behavior in multi-agent spatial games, playing the IPD by means of reinforcement learning techniques. The remainder of the chapter is structured as follows: firstly, in Sect. 9.2 we present a state of art. In Sect. 9.3 we present the Prisoner’s Dilemma. After, in Sect. 9.4, we describe the approach we followed in our game. In Sect. 9.5 we explain the learning techniques the agents use, and in Sect. 9.6, Sect. 9.7 and Sect. 9.8 we describe the scenario and the results of our experiments. Finally, in Sect. 9.9 we present the conclusions and expose our future work.

9.2 Related Work Social and economic networks, as international alliances, trading agreements or cooperation among cooperations, play an important role in our society [14]. This is why there is a big interest in studying networks in a social science context from a sociological perspective [6] or from a formal game point of view [1]. In many situations, as for example in politics, in joint productions, or in trade alliances, two interacting individuals can obtain a benefit from coordinating their actions [13], even if the autonomous agents are selfish [30]. Thus forming coalitions can be a desired behavior to obtain shared goals. Coalition formation and collaboration in mobile agents is an open research area, where many proposals have been already presented, as for example forming coalitions of mobile agents using evolutionary algorithms [11]; in task allocating swarm coalitions [18]; in coalition formation in Unmanned Aerial Vehicles (UAV) [10] or in dynamic coalition formation under uncertainty in real world robotic systems, with temporal and spatial constraints [24]. However, in the present work, we focus on static agents and a spatial distribution to play and form coalitions. The methodology used to form a coalition allows autonomous agents to join to act as a group to increase their gains by collaborating with each other. In [17], a two-level coalition formation approach is defined, which means that agents try to maximize their own benefit, as well as the coalition benefit. In [16], the coalition

9

Emerging Cooperation in the Spatial IPD

189

formation is based on three driving factors, which are job demands, resource availability and profit objective. We must understand and quantify the tradeoff as the relation between the benefits vs. the costs of collaboration, in order to analyze, design and control/operate networks of agents. This last issue brings on many proposals to investigate this tradeoff, as for example in [31], where it is studied how the coalition formation depends on the payoff. The context of cooperative games and cooperation evolution has been extensively studied seeking general theoretical frameworks like the Prisoner’s Dilemma (PD) [2]. In his seminal work, Axelrod has shown that cooperation can emerge in a society of individuals with selfish motivations. Since that, Game Theory and the Generalized Prisoner’s Dilemma have been applied in biological, social, economical and ecological contexts [4, 12]. An interesting spatial version of the PD had been suggested and deeply analyzed by Nowak and other authors [26, 22, 15] trying to understand the role of local interactions in the maintenance of cooperation. Also in [8], a framework for memetic analysis of coalition formation in spatial prisoner’s dilemma is presented. Coalitional games capture opportunities for cooperation, since they model the ability of the agents to take joint actions as primitives [19]. In [25], Sandholm et al. establish three main research issues in the use of coalitional games in multi-agent systems: coalitional structure generation, teamwork and payoff division. The first games of coalition formation were introduced by Selten [27] and d’Aspremont et al. [9]. In [29], using the Iterated Prisoner Dilemma (IPD) game, the authors study the emergence of coalitions in a co-evolutionary learning environment, introducing the confidence concept for players in the coalition. Another view of the IPD game is proposed in [28], where the authors present a study of the local interactions on evolving coalitions in the NIPD game. In last examples, the game is not spatial. However, in [21], a spatial dilemma game with coalitions is proposed, even though the schema is not a general one and is only considered for the particular context evaluated in this chapter.

9.3 Prisoner’s Dilemma Game Theory [7] is a branch of applied mathematics that helps to understand the strategies that selfish individuals may follow when competing or collaborating in games and real scenarios [23]. The concept of cooperation evolution has been successfully studied using theoretical frameworks like the Prisoner’s Dilemma (PD) [2], which is one of the most well-known strategy games. It models a situation in which two entities have to decide whether cooperate or defect, without knowing what the other is going to do. Nowadays, the PD game has been applied to a huge variety of disciplines: economy, biology, artificial intelligence, social sciences, e-commerce, etc. Table 9.1 shows the general PD form matrix, which represents the rewards an entity obtains

190


depending on its action and the opponent’s one. In this matrix, T means the Temptation to defect, R is the Reward for mutual cooperation, P the Punishment for mutual defection and S the Sucker’s payoff. To be defined as a PD, the game must accomplish the next two conditions:

T >R>P>S 2R > T + S

(9.1)

Table 9.1 General Prisoner’s Dilemma Matrix

Player Ai Cooperates Player Ai Defects

Player A j Cooperates Player A j Defects R, R S, T T, S P, P

There is a variant of PD, which is the Iterated Prisoner’s Dilemma (IPD) [2], in which the game is played repeatedly. In it, the players can punish their opponents for previous non-cooperative behavior, remembering their opponent previous action and adapting their strategy. Game Theory shows that the optimal action if both players know that they are going to play exactly N times is to defect (it is the Nash equilibrium of the game) [7]. But when the players play an indefinite or random number of times, cooperation can emerge as a game equilibrium. The IPD game models transactions between two persons requiring trust and cooperative behavior, and that is why this game has fascinated researchers over the years.

9.4 The Game The approach we follow in this chapter is a spatial game where every agent may interact with several neighbors at a time. We consider a spatial structure of the population, i.e., if the interaction between agents is locally restricted to their neighbors, then, a stable spatial coexistence between cooperators and defectors becomes possible under certain conditions. The approach presented here is based and extends the works in [22].

9.4.1 Spatial Distribution For the spatial distribution of the cells we consider a two-dimensional square lattice consisting of N nodes, in which each cell is ruled by an agent (Fig. 9.1).

9


191

Fig. 9.1 Cell agent (A) and two neighborhoods: first with 4 cells A1,. . . ,A4, and second with 8 cells A1,. . . ,A8

If we let every node in the system to interact with the remaining (N-1) nodes, we have a panmictic population, i.e., a population where all the individuals are potential partners. But, in many real contexts like geography, biology, MANETs or social networks, each node interacts mainly with a set of local neighbors. In this chapter, we are mainly interested in the spatial effects of the PD game [22]. In our case, each agent Ai interacts only with the m closest agents (in evolutionary game theory this is called a m-person game). Each game is played by the n players (n = m + 1) simultaneously, and the payoff for each player depends on the number of cooperators and defectors in the neighborhood. Thus, we consider that each cell agent Ai interacts only with the agents in its neighborhood and can perform two actions against them: defect (D) or cooperate (C). Fig. 9.1 shows a cell agent A and two possible neighborhoods, which are defined depending on the distance that a cell is allowed to use to play with other cells. In this chapter we consider these two possible neighborhoods: 4-neighbors or 8-neighbors.

9.4.2 Basic Game Rules In the PD game (described in Sect. 9.3), each agent Ai has two options to act in a given situation: to cooperate (C) or to defect (D). The outcome of the interaction of Ai with A j depends on the action chosen by both agents (we can see the payoffs in Table 9.1). For example, suppose that agent Ai has chosen action C; in this case its payoff is R if A j has also chosen C and S if A j defects. The classical values for the Prisoner’s Dilemma are: T = 5, R = 3, P = 1, S = 0. In any one round (or "one-shot") game, choosing action D is a Nash equilibrium because it rewards the highest payoff for agent Ai no matter what the opponent chooses. However, the combined action DD (i.e. both play the Nash equilibrium) is

192


very poor socially speaking. This combined payoff is maximized if both cooperate. We want to see which is the behavior of the cells when we allow to form coalitions while playing the spatial IPD.

9.4.3 Agent Roles In [26, 22] every agent may behave as a defector or a cooperator playing isolated, but in the work presented here agents can join to create coalitions (groups of agents). Then, there are two possible roles in our MAS: 1. Independent cells: ruled by its own agent that may behave as a cooperator or a defector with its neighbors depending on its own decisions. They can join a group in their neighborhood or remain independent. 2. Coalition cells: these are the cells that belong to a coalition. The action they perform is decided among all the members of the coalition. They can become independent whenever their agents desire.

9.4.4 Scenarios and Agent Actions In this chapter we have two possible scenarios: allowing coalitions and not allowing coalitions. In the former case, the agents can only decide about which action they play, i.e., if they are cooperators or defectors. In the later case, the cells cannot only decide about the action to play, but they can also join or leave coalitions. The possible actions they may perform in this case are: join/remain in a coalition, go/remain independent playing C, go/remain independent playing D. Further, agents in coalitions have to decide whether it is better to cooperate or to defect against the agents not belonging to the group (henceforth called outsiders). Inside a coalition, the agent does not have to decide which is the behavior against its coalition mates, because we enforce the agent behavior to be cooperative. We could consider agents betraying their coalition members, but as a first approach, in this chapter we assume that the agents can not behave as defectors with their coalition members as all the actions are public. When an agent decides to join a group, it has to choose to which of the surrounding groups or cells aiming to form a group (in case there are more than one) it will join. This decision is taken depending on the gain of the group in the past. Finally, when a cell has joined a coalition, all the members decide how they behave against the outsiders by voting, i.e., each one had chosen before which is the best action to perform in their own opinion, but they vote to agree the coalition behavior, and the action performed is the one obtaining the highest number of votes.

9


193

As for the second and third actions (go/remain independent playing C and go/remain independent playing D), the agents decide that they want to remain independent, or, if they belong to a group, they want to become independent, choosing also which action they will perform: being a defector or a cooperator.

9.5 Reinforcement Learning Algorithms In this chapter we consider two popular reinforcement learning techniques that the agents use to decide their actions: Learning Automata (LA) and Q-learning (QL).

9.5.1 Q-Learning (QL) Q-Learning is a model-free technique popular in RL exactly because it does not require that the agent have access to information about how the environment works. QL works by estimating values for pairs state–action (the Q-values), which are numerical estimators of quality for a given pair of state and action. More precisely, a Q-value Q(s, a) represents the maximum discounted sum of future rewards an agent can expect to receive if it starts in state s, chooses action a and then continues to follow an optimal policy. The QL algorithm approximates Q(s, a) as the agent acts in a given environment. The update rule for each experience tuple s, a, s , r is as in Eq. 9.2, where α is the learning rate and γ is the discount for future rewards. If all pairs state-action are visited infinitely often during the learning process, then QL is guaranteed to converge to the correct Q-values with probability one [32]. (9.2) Q(s, a) ← Q(s, a) + α r + γ maxa Q(s , a ) − Q(s, a)

9.5.2 Learning Automata (LA) In the case of LA, the agent just considers its history and selects his next strategy depending on its experience and payoffs. We use the LR−I scheme [20] as defined in Eq. 9.3, where α ∈ [0, 1]. The LR−I scheme is defined as following: = pi,t + α (1 − pi,t ) pi,t+1 j,t+1 = p j,t (1 − α ) ∀ j=i : p

(9.3)

In these equations (α ) is a small learning factor. The first rule is used to reinforce the action chosen if it performed better than its alternatives in the considered state. At the same time, we apply the second rule to the other actions, decreasing its probability. In the next round, the agent will choose its new strategy using the updated probabilities.

194


9.5.3 Action Selection and States When using QL and LA, what we obtain is a vector with the probabilities of performing a concrete action in a state. We need to use an action selection mechanism to explore all the set of possible actions in every state. In this chapter we use the epsilon-greedy method: with probability ε an agent will explore the action space, i.e., an action is chosen depending on its probabilities. Besides, with probability 1 − ε the agent selects the action with the highest probability. Along the simulation, epsilon is decreased as described in the literature to reduce the exploration, allowing the agent to select the most successful actions. We want the agents to learn which action to take in each state, thus we define the state of an agent as the number of neighbors that cooperate and defect. For example, if we consider that an agent’s neighborhood is eight, one state could be four defectors and four cooperators. However, to compute the state in every round is not trivial, since the agents cannot know a priori what the other players are going to play (they do not know the actual state until they actually play). To solve this problem, we took the decision of inferring the present from the past, i.e., we take the previous state and from it we infer what is the best action to play in the present one.

9.6 Scenarios We consider two main scenarios to explore with the spatial games: the first is a scenario with a coordination matrix, i.e., the main objective is to synchronize among neighbor cells; while the second scenario uses the classical IPD matrices, and coordination is not an objective per se. In any of these two scenarios we shall use the QL and the LA algorithms, with 4 or 8 neighbors per cell, and allowing or not the creation of coalitions. In the case of a scenario without coalitions, an agent learns if with a concrete payoff matrix it is better to be a cooperator or a defector. In the case of allowing coalitions, the agent learns if it is better to be in a coalition or to be an independent cell (and in this case if it cooperates or defects). Besides, if a cell is in a coalition, it also has to learn which is the best action to play in the coalition against the outsiders.

9.7 Results Using the Coordination Game In this section we present the results obtained in the different scenarios considering a coordination game [5]. As we said before, we use LA and QL, and each agent plays against the closest eight neighbors (the eight surrounding cells). The coordination matrix we use is the following:

9


195

Table 9.2 Coordination Matrix Player’s Actions A j plays A A j plays B Ai plays A 1, 1 -1, -1 Ai plays B -1, -1 1, 1

In these type of coordination games, the players are rewarded if they coordinate their actions, meaning this that all players execute the same action. In this case, we consider that agents can play two actions: A and B, and depending on their action, and the opponents one, they obtain a reward.

9.7.1 Scenario without Coalitions First we present some results obtained when using the learning algorithms LA and QL without coalitions.

(a) Frequency of actions

(b) Groups formed.

Fig. 9.2 Scenario without coalitions and using the coordination matrix

In Fig. 9.2, we show the results when we use LA and 8-neighbors. In this case all the cells end up playing A or B but forming groups, i.e., there are spatial groups playing A and others playing B. They learn that by coordinating among neighbors they can obtain bigger gain. We can see in Fig. 9.2(a) that, in this case, there are more agents playing B (in black) than playing A (in green), but this can be the contrary in other simulation, since what this matrix rewards is coordination, not a concrete action. In Fig. 9.2(b) we see the groups that are formed in order to increase their gains. The results with 4-neighbors and with QL are practically the same.

196


9.7.2 Scenario with Coalitions In this case we allow the cells to form coalitions to decide which action to take. This may seem redundant, since with the coordination matrix what we want to achieve is coordination, and in last section we saw that even without explicitly allowing coalitions, the agents tend to group to maximize their gains. However, in this case we explicitly allow the cells to form coalitions to see what happens.

(a) Frequency of actions

(b) Behaviour against the outsiders

(c) Typical group distribution. Fig. 9.3 Coalitions and coordination matrix.

In Fig. 9.3 we show the results when we use LA and 8-neighbors. Fig. 9.3(a) shows the frequency of cells playing A (green), playing B (black) or belonging to a coalition (blue). We see that all the cells end up playing A or being in a coalition. Besides, in Fig. 9.3(b) we see that the best behavior is to cooperate. We can see in Fig. 9.3(c) a typical spatial distribution of the groups. However, there are also another possible behavior, which is that all the cells end up playing B or being in a coalition, and the cells in a coalition playing B against the outsiders. This remarks

9


197

that what is rewarded with this matrix is the coordination among the agents. As for the results with QL and 4-neighbors, they are very similar to the ones showed in Fig. 9.3.

9.8 Results Using a Prisoner’s Dilemma Approach In this section we present the results obtained in the different scenarios considering different PD matrices. As we said before, we use QL and LA, and each agent plays against the closest eight neighbors (the eight surrounding cells). In the experiments we have used the following matrices:

Table 9.3 Prisoner’s Dilemma Matrix biased to defection Player’s Actions A j Cooperates A j Defects Ai Cooperates 5, 5 0, 9 Ai Defects 9, 0 1, 1

Table 9.4 Non-Prisoner’s Dilemma Matrix biased to cooperation Player’s Actions A j Cooperates A j Defects Ai Cooperates 8, 8 3, 5 Ai Defects 5, 3 1, 1

Table 9.5 Classical Prisoner’s Dilemma Matrix Player’s Actions A j Cooperates A j Defects Ai Cooperates 3, 3 0, 5 Ai Defects 5, 0 1, 1

PD1 is a matrix biased to defection, since as we see, the payoff if a cell defects and the opponent cooperates is very high. In PD2, the matrix does not accomplish the conditions for the PD matrix values, but we use it to bias the matrix to cooperation. Finally, PD3 represents the standard PD matrix.

198


9.8.1 Scenario without Coalitions First we present some results obtained when using the learning algorithms without coalitions. The pictures only show the results of using the LA algorithm and with 8-neighbors, since for QL and 4-neighbors they are quite similar.

(a) PD1

(b) PD2

(c) PD3 Fig. 9.4 Frequency of Cells per Action in a scenario without coalitions

In Fig. 9.4 we show the different frequency of cells performing an action when using the three different matrix. The defectors are represented in black, and the cooperators in green. We can see that in the case of PD1 (Fig. 9.4(a)), we obtain that all the cells tend to defect, since in the beginning, if playing against a cooperator, they obtain the maximum gain, thus they learn that defecting is the best option. In the case of PD2 (Fig. 9.4(b)), all cells end up cooperating, since the matrix encourages to learn that being a cooperator provides good results. Finally, if playing with the standard PD3 matrix ( 9.4(c)), we see that all cells are defectors in the end.

9


199

9.8.2 Scenario with Coalitions Now we present the results when the cells can join to form coalitions, and they use LA and QL and play with 8-neighbors and 4-neighbors to decide which action to take. In Fig. 9.5 we show the results of the frequency of cells when using the LA algorithm to learn. The figures in the left represent the frequency of cells which are individual and cooperators (green line), individual and defectors (black line) or belonging to a coalition (blue line). The figures in the right represent the frequency of cells that belonging to a coalition decide to defect against the outsiders (black line) and the ones that decide to cooperate (green line). In Fig. 9.5(a) we see that, when using the PD1 matrix, the cells tend to join into a coalition and being defectors against the outsiders. This is a coherent result, since mutual cooperation has a big reward too with this matrix, and joining into cooperative coalitions results in big gains. Besides, if we defect against the outsiders, the gain obtained can be very high (up to 9), and this is the reason why cells defect to outsiders. In Fig. 9.5(b), with the PD2 matrix, we see that the number of cooperating cells exceeds the number of the ones belonging to a coalition. However, we also observe that internal behavior of the coalition cells is to cooperate against the outsiders. In this case cells learn that cooperating (being it within a group or individually) provides the highest benefit. Finally, in Fig. 9.5(c), we observe that in the standard PD case (PD3), the number of defectors is similar to the number of cells belonging to a coalition. This is caused because in this case the reward of mutual cooperation is not as high as in the PD1 matrix. Besides, we also observe that the internal behavior of the coalition cells against the outsiders is to defect. now we consider the results when the cells use QL. In Fig. 9.6 we can see the graphics, which follow the same schema as the ones in LA. The results obtained are similar to the ones obtained with LA. However, there are some differences. For example, in Fig. 9.6(a) (left) we see that the number of defectors is not as low as in LA. Besides, in Fig. 9.6(b) (left), the number of cooperators and cells belonging to a coalition is similar, when in the LA case, the number of cooperators exceeded the coalition ones. Finally, regarding the third situation, in this case the number of cells belonging to the coalition exceeds the number of defectors, while when using LA they appear in a similar percentage. As we have seen in the previous experiments, by using learning algorithms, coalitions emerge, and cells self-organize to obtain the highest benefit. Besides, by changing the payoff matrix, we observe that the behavior of the cells varies to adapt to the conditions that these biased matrix impose. We have also compared the results with two different learning algorithms, obtaining similar results with both of them. In Fig. 9.7 we show the distribution of coalitions and independent cells in the scenario at the end of the previous simulations. Each color square represents an agent: the isolated defectors are black, the cooperators green, and the cells belonging to a coalition have the same color (different to the previous mentioned). We see that

200


(a) LA and PD1

(b) LA and PD2

(c) LA and PD3 Fig. 9.5 LA with coalitions. The figures in the left represent the frequency of independent cells playing C, independent cells playing D and cells belonging to a coalition. The figures in the right represent the cells that belonging to a coalition play C or D against the outsiders.

9


201

(a) QL and PD1

(b) QL and PD2

(c) QL and PD3 Fig. 9.6 QL with coalitions. The figures in the left represent the frequency of independent cells playing C, independent cells playing D and cells belonging to a coalition. The figures in the right represent the cells that belonging to a coalition play C or D against the outsiders.

202


(a) QL and PD1

(b) LA and PD1

(c) QL and PD2

(d) LA and PD2

(e) QL and PD3

(f) LA and PD3

Fig. 9.7 Graphical display and comparison of coalitions for QL and LA

9


(a) LA 4-neighbors

(b) LA 8-neighbors

(c) QL 4-neighbors

(d) QL 8-neighbors Fig. 9.8 Comparison of results with 4-neighbors (left) and 8-neighbors (right)

203

204


even though there are some differences between the two learning algorithms, the results are similar. Finally, we present a comparison among the results with 4-neighbors and 8neighbors. In some cases the frequency of agents are similar to the ones with neighborhood eight, but in others we observe that by varying the neighborhood, the results are different. For example, in Fig. 9.8, we consider the matrix PD3 (i.e., the standard IPD matrix) playing with coalitions and we compare the results with LA and QL when using this two neighborhoods. In the former case, we observe that with 4-neighbors (Fig. 9.8(a)) all the cells end up being defectors, while with 8-neighbors ( 9.8(b)) coalitions appear. In the later case, we also observe that the results are different, since the number of agents belonging to a coalition is smaller in the case of 4-neighbors ( 9.8(c)) than in the case of 8-neighbors (Fig. 9.8(d)).

9.9 Conclusions and Future Work In this chapter we have presented the use of learning and computer simulations to study the interaction when playing the spatial IPD and allowing the formation of coalitions. The main contribution of the chapter is the exploration of the behavior of coalitions in multi-agent spatial games, playing collaborative and non collaborative spatial grid games by means of reinforcement learning techniques. We provide some experiments and results to show that by using learning algorithms we can model agents that learn which is the best behavior in a concrete situation. In our case, the different matrix and neighbors with which the agents have to play allow us to show different situations in which different agent behaviors may arise. Besides, by allowing the formation of coalitions, we explore the coalition behavior and how they emerge without need of any force, just because the benefits they obtain by joining are higher than the ones of remaining alone. As future work, we plan to use other learning algorithms, to include different kind of agents in the game and to apply the coalition formation to other more complex and realistic scenarios.

References 1. Aumman, Myerson: Endogenous formation of links between players and coalitions: An application of the Shapley value. Cambridge University Press, Cambridge (1988) 2. Axelrod, R.: The Evolution of Cooperation. Basic Books, New York (1984) 3. Axelrod, R.: The Complexity of Cooperation: Agent-based Models of Competition and Collaboration. Princeton University Press, Princeton (1997) 4. Axelrod, R.: On six advances in cooperation theory. In: Analyse und Kritik - Special Edition on the Evolution of Cooperation (2000) 5. Bazzan, A.L.C.: Joint learning in stochastic games: Playing coordination games within coalitions. In: Proc. of the Adaptive Learning Agents, ALA 2009 (2009) 6. Bellman, R., Berkovic, S.: Social Structure: A Network Approach. Cambridge University, Cambridge (1988)

9


205

7. Binmore, K.: Game theory and the social contract volume I: Playing fair. The MIT Press, Cambridge (1994) 8. Burguillo-Rial, J.C.: A memetic framework for describing and simulating spatial prisoner’s dilemma with coalition formation. In: AAMAS 2009: Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems, International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, pp. 441–448 (2009) 9. d’Aspremont, C., Jacquemin, A., Gabszewicz, J.J.: On the stability of collusive price leadership. Canadian Journal of Economics 16(1), 17–25 (1983) 10. George, J.M., Pinto, J., Sujit, P.B., Sousa, J.B.: Multiple UAV coalition formation strategies. In: AAMAS 2010: Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems, International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, pp. 1503–1504 (2010) 11. Gruszczyk, W., Kwa´snicka, H.: Coalition formation in multi-agent systems an evolutionary approach. In: International Multiconference on Computer Science and Information Technology, MCSIT 2008, pp. 125–130 (2008) 12. Hoffmann, R.: Twenty years on: The evolution of cooperation revisited. J. Artif. Societies and Social Simulation 3(2) (2000) 13. Jackson, M.O., Watts, A.: On the formation of interaction networks in social coordination games. Games and Economic Behavior 41, 265–291 (2000) 14. Jackson, M.O., Demange, G., Goyal, S., Van Den Nouwel, A.: A survey of models of network formation: stability and efficiency. In: Group Formation in Economics: Networks, Clubs and Coalitions. Cambridge University Press, Cambridge (2003) 15. Langer, P., Nowak, M.A., Hauert, C.: Spatial invasion of cooperation. Journal of Theoretical Biology 250(4), 634–641 (2008) 16. Lau, H.C., Zhang, L.: Task allocation via multi-agent coalition formation: Taxonomy, algorithms and complexity. In: IEEE International Conference on Tools with Artificial Intelligence, p. 346 (2003) 17. Lau, H.C., Zhang, L.: A two-level framework for coalition formation via optimization and agent negotiation. In: IAT 2004: Proceedings of the IEEE/WIC/ACM International Conference on Intelligent Agent Technology, pp. 441–445. IEEE Computer Society, Washington, DC, USA (2004) 18. Low, K.H., Leow, W.K., Ang, M.H.: Task allocation via self-organizing swarm coalitions in distributed mobile sensor network. In: AAAI 2004: Proceedings of the 19th national conference on Artifical intelligence, pp. 28–33. AAAI Press/The MIT Press (2004) 19. Michalak, T., Marciniak, D., Szamotulski, M., Rahwan, T., Wooldridge, M., McBurney, P., Jennings, N.: A logic-based representation for coalitional games with externalities. In: Autonomous Agents And MultiAgent Systems (AAMAS 2010), pp. 125–132 (2010) 20. Narendra, K.S., Thathachar, M.A.L.: Learning automata: an introduction. Prentice-Hall, Inc., Upper Saddle River (1989) 21. Nguyen, D., Ishida, Y.: Spatial dilemma strategies of intelligent agents: Coalition formation in environmental game. In: International Conference on Knowledge and Systems Engineering, pp. 126–129 (2009) 22. Nowak, M.A., May, R.M.: Evolutionary games and spatial chaos. Nature 359(6398), 826–829 (1992) 23. Osborne, M.J.: An Introduction to Game Theory. Oxford University Press, USA (2003) 24. Ramchurn, S.D., Polukarov, M., Farinelli, A., Jennings, N., Trong, C.: Coalition formation with spatial and temporal constraints. In: International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS 2010), pp. 1181–1188 (2010)

206


25. Sandholm, T., Larson, K., Andersson, K.M., Shehory, O., Tohme, F.: Coalition structure generation with worst case guarantees. Artificial Intelligence (1999) 26. Schweitzer, F., Behera, L., Muehlenbein, H.: Evolution of Cooperation in a Spatial Prisoner’s Dilemma. Advances in Complex systems 5(2-3), 269–299 (2002) 27. Selten, R.: A simple model of imperfect competition when 4 are few and 6 are many. International Journal of Game Theory 2, 141–201 (1973) 28. Seo, Y., Cho, S., Yao, X.: Emergence of cooperative coalition in NIPD game with localization of interaction and learning. In: Proc of Congress on Evolutionary Computation (CEC 1999), pp. 877–884. IEEE Press, Piscataway (1999) 29. Seo, Y., Cho, S., Yao, X.: Exploiting coalition in co-evolutionary learning. In: Proc. Congress Evol. Comput., pp. 1268–1275 (2000) 30. Shehory, O., Kraus, S.: Coalition formation among autonomous agents: Strategies and complexity. In: Castelfranchi, C., Muller, J.P. (eds.) From Reaction to Cognition, vol. 1, pp. 57–72 (1995) 31. Tanimoto, K.: Coalition formation interacted with transitional state of environment. In: IEEE International Conference on Systems, Man and Cybernetics, vol. 6, pp. 6–9 (2002) 32. Watkins, C.J.C.H., Dayan, P.: Q-learning. Machine Learning 8(3), 279–292 (1992)

Chapter 10

Evolutionary and Economic Agents in Complex Decision Systems Stephan Otto and Christoph Niemann

10.1 Introduction The complexity of decision making grows with the complexity of the environment. In particular, large and distributed environments demand systems having the ability to cope with the distributed nature of their environment. Further they must be able to adapt to changing circumstances. One major requirement for systems in large and distributed environments is flexiblity. Any system that is situated in an environment has to be able to adapt to the dynamics of the environment in a flexible way at runtime. This chapter introduces software agents and multi-agent system s as an approach to model systems for such environments. Software agents offer a flexible way to modify a system by changing the number of agents and their behaviour in the system. Building on multi-agent systems, the concept of market-based coordination provides a means to process large amounts of data and coordinate many agents. Evolutionary computation enables the agents to adapt to the environment by changing their behaviour in a dynamic way and by improving their performance continuously. This chapter introduces evolutionary, market-based systems as a promising approach to tackle the support of increasingly large environments. The combined approach can be used to create systems with the capability to adapt to a changing environment on system level by changing the number of agents in the system. On the micro-level, each agent is able to adapt its behaviour to the environment as well. The chapter is structured as follows. After a brief introduction to complex decision systems and environments in Section 10.2, the main concepts to be used Stephan Otto Fraunhofer Center for Supply Chain Services SCS, Nuremberg e-mail: [email protected] Christoph Niemann Chair of Management Information Systems, University of Bayreuth e-mail: [email protected] P. Bouvry et al. (Eds.): Intelligent Decision Systems, SCI 362, pp. 207–229. c Springer-Verlag Berlin Heidelberg 2011 springerlink.com

208

S. Otto and C. Niemann

in complex decision systems are introduced in Section 10.3. Three case studies (Section 10.4) describe the application of the approaches in different environments. The chapter concludes with an outlook and potential avenues for future work.

10.2 Environments and Complex Decision Systems In system theory a model comprises an environment and the system itself. A system is a set of elements and incorporates a relationship or structure between them. Anything outside of these elements is called the environment. In open systems there is an exchange of energy or information between the system and the environment; the system processes information from the environment and reacts. This chapter clarifies our understanding in terms of decision systems and their environments.

10.2.1 Environments This section describes our understanding of environments technical decision systems need to cope with. At first we will give an overview on current representative environments and extract common characteristics. This section also examines interactions between environments and systems. The following list is considered as a representative set of today’s distributed environments: • Service Oriented Architectures (SOA) are widely used in today’s business environments where so called services are combined to form applications. The idea here is to decompose, encapsulate and reuse functions as separate services. These services are accessible via a published interface and offer a defined set of functionality. Service discovery mechanisms allow restructuring in a flexible way during runtime. • Peer-to-Peer (P2P) Computing [33] refers to networked applications performed by equally privileged peers. In contrast to client/server architectures or SOA all peers have the same responsibilities and functionalities and act as client and server. • Grids provide the ability to dynamically link distributed and heterogeneous resources to solve problems. Originating from concepts for distributed infrastructures for science and engineering, the term Grid comprises infrastructure, tools and applications for reliable and secure resource sharing within dynamic and geographically distributed ’Virtual Organisations’. Grids tackle problems concerning coordinated and controlled resource sharing and problem solving. • Sensor Networks (SNW) are composed of sensor nodes, communicating wirelessly and organizing multi-hop networks. The principles are infrastructureless communication, distributed data (pre)processing, self configuration, scalability and self maintenance abilities during very long operation times.

10

Evolutionary and Economic Agents in Complex Decision Systems

209

Table 10.1 Properties of complex adaptive systems in selected environments property SOA P2P agent based x x heterogeneous x dynamic feedback x organization x emergence

Grid SNW CAS x x x x x x x x x x x x x x x

Since environments are electronic systems themselves, we propose a general trend of information systems towards complex adaptive systems (CAS) [16, 3]. Table 10.1 lists characteristics of the selected environments compared to the characteristics of complex adaptive systems [34]. As a consequence, decision systems need to cope with properties of complex adaptive systems.

10.2.2 Decision Systems Complex environments require a modelling approach that reflects their nature. To the environment, a system is solely defined by its interface [42]. However, the characteristics of the environment can give an advantage to modelling the system in a particular way. If the environment is of dynamic nature, a decision system that allows for dynamic adaptation of the decision making mechanism can prove to be more suitable than a system that depends on a fixed mechanism. Complex, large scale environments are often dynamic environments with a varying number of active entities and heterogeneous sources of information. A system to support such an environment has to accommodate to a varying number of actors that can have competing goals. Furthermore, the system must be able to react to feedback from the environment and incorporate the newly acquired information into the decision making. If at least some of the entities are humans, it is desirable to incorporate a notion of autonomy into the system. It can be allowed to act autonomously under certain circumstances, but it must relinquish control of the environment if the setting leaves a defined range of parameters. One design approach that lends itself to dynamic, complex environments are multi-agent systems (MAS) that model the environment with software agents.

10.3 Complex Decision Systems The complex nature of most environments demands complex decision systems. This section introduces some approaches to model complex systems while maintaining flexiblity. According to Kirn et al., the most important promise of (MAS) is their flexibility [21]. As the environment can behave in a dynamic way, a decision system

210


needs to cope for such changes during runtime. There is no possibility to specify the entire space of possible states during build time. Therefore, a system working in a open environment needs to adapt to changing circumstances at run time. In [34] an analogy between MAS and CAS is drawn by comparing their properties (see table 10.1). Therefore, we choose multi agent technology as a basic starting point to construct decision systems upon.

10.3.1 Software Agents Software agents are active software entities that are situated in an environment. They are loosely coupled and coordinate their actions by means of negotiation. The approach allows modelling the various entities of the environment as single agents. Each agent pursues its own goals; by means of coordination, they can cooperate. Software agents are computer programs, which can be described by their capabilities. One common set of capabilities has been suggested by Jennings and Wooldridge [50]: Autonomy: An agent acts without direct intervention of others (humans or artificial entities) and has at least partial control over its actions and internal state. Reactivity: Agents are able to sense their environment. The sensory input is used to determine actions that the agents use to change the state of the environment in order to satisfy their objectives. Proactiveness: Agents are not only able to react to changes in the environment, but they can try to change the state of the environment to meet their objectives without being triggered by an outside event. Proactive agents exhibit goal-driven behaviour. Social ability: Agents can act on their own to achieve individual goals. If they form a group of agents, they are able to communicate by exchanging messages. The interaction allows for cooperation among agents. MAS have advantages in modelling large-scale, dynamic systems, because they do not rely on a set of entities that is defined at design time. Instead, they are able to accommodate to dynamic environments by adding or removing agents from the system to adapt to a varying number of entities. Communication and coordination between the agents happens based on messages that the agents exchange according to a defined protocol. A very flexible way of reaching decisions in situations with competing goals is a market-based coordination that adapts economic principles to decisions in MAS. 10.3.1.1

Multi-Agent System

Definition 10.1 (Multi-Agent System (MAS)). A MAS = (A, E) consists of a finite set of agents A = {a1 , ..., a p } embedded in an environment E.

10


211

Since a MAS contains agents, the following definition for an agent is: Definition 10.2 (Agent). An agent a = (I, O, f , s, c) consists of a finite set of sensory inputs I = {i1 , . . . , im }, a finite set of effector outputs O = {o1 , . . . , oq }, a function f : I → O which maps sensory inputs to effector outputs, the strategy vector s = (s1 , s2 , . . . , sr ) determining or parameterizing f , and the agents current funds c. As a basic element of a decentralized system, an agent a is an autonomous sensoreffector entity with function f a . The strategy vector sa represents a set of parameters that are assigned to a. We consider the agent’s function fa to be determined by sa . Thus, the strategy sa of agent a determines the behavior. Access to a particular element in the strategy vector is given by the following convenience notation: sa (parameter) denotes parameter. Note that the dimension of s may vary between different agents. One class of problems that MAS can solve are distributed optimization problems. 10.3.1.2

Distributed Optimization Problem

The task of optimizing can be formulated as follows: Definition 10.3 (Distributed Optimization Problem (DOP)). A DOP is given by: minimize f (sA ), subject to γ (sA )

(10.1)

where sA = (sa11 , sa21 , . . . , sau1 , . . . , sa1m , sa2m , . . . , sav m ) ∈ SA∗ is the strategy vector of a MAS and SA∗ is the search space. The length of the strategy vector sa1 of agent a1 is denoted with u and the length of the strategy vector sam is denoted with v. The objective function is f and there are w constraints γi (sA ), i = 1, . . . , w imposed on sA . The strategy vector sA is distributed among the agents in A and the objective function f is unknown to A and therefore cannot be calculated by any single agent. The set γ (sA ) contains constraints of the DOP distributed over A. The combination of local strategy vectors of all agents sA forms the strategy vector and is also the distributed solution of the DOP. If constraints vary over time t, the DOP becomes even harder. The dimensions of sA are not fixed, rather they may vary over time, as agents enter or leave the system.

10.3.2 Economic and Market-Based Models Market-based approaches use the markets’ ability of having prices emerging from varying supply and demand. Adam Smith has called this property the “invisible hand” [43, p. 349] that can coordinate many actors on a market with individual and sometimes contradicting preferences. The metaphor explains how the market

212


allows for the fulfillment of individual preferences as well as the maximization of the economic welfare. Markets coordinate the parties by means of negotiations. Each actor negotiates with interested parties on the market to find the most suiting counterpart. If at least some of the actors are willing and able to compromise on at least some of their goals, even more contracts are possible. Software agents have been proposed as actors on electronic markets by various authors [6, 5, 7, 25], because MAS provide a convenient means to explore the actors’ individual behavior and its consequences on the system as a whole. To classify negotiations, a categorization in negotiation protocol negotiation protocol, negotiation object, and negotiation strategy [19] has proven to be useful. While the protocol and the object of the negotiation is public knowledge that is known and agreed upon by all parties, the strategy remains private knowledge and each party tries to capitalize on it. The parties try to conceal any information that could compromise their strategy or make it known to the other agents. If the strategy became public knowledge, an agent would suffer disadvantages in the negotiation process as others could exploit that knowledge. Among various negotiation mechanisms, this risk exists in most of them; the exception are stable negotiation mechanims [39] that can be found in cooperative environments. Markets, however, are usually competitive in nature. Therefore, the agents should keep their strategy secret and they should take precautions not to make it known. Because other agents could possibly engage in measures to infer their opponent’s strategy, each agent tries to put out to the public as few information as possible. One possibility to infer a strategy is by analyzing the actions (or offers in the negotiation) of the other agents. Since the offers result from the opponent’s reasoning that is governed by its strategy, the offers are the direct consequence of the strategy. If an agent can identify a pattern in the opponent’s offers, it can infer at least parts of the strategy. One example for such an inference is the use of Bayesian techniques (e.g. [53]). The negotiation object defines what the negotiation is about. Of particular interest is the differentiation in deal attributes and product attributes [36]. It defines which attributes of the object are part of the negotiation and which attributes are set before the negotiation actually starts. All parties in the negotiation have to agree on a set of attributes that is up for negotiation. If they cannot find an agreement on the deal attributes, a negotiation cannot take place. During the negotiation, this definition can change if all parties agree on the change. Furthermore, if a negotiation is set to occur on each deal attribute sequentially, the order of the subnegotiations is variable and can be subject for negotiation itself. A clever definition of the sequence can lead to strategic advantages [10]. Research in negotiations happens mainly on multi-issue negotiations [20, 23], since it provides a notedly higher complexity than single-issue negotiations. The third main criterion to classify negotiations is the negotiation protocol. It defines the steps that each party in the negotiation is allowed to take at the various stages during the negotiation. To reach an agreement in a negotiation, only defined actions are allowed for the parties. Each party can choose among the potential steps based on its strategy. Once an agreement has been reached, the exchange of goods

10


213

can happen during the post-negotiation phases of the transaction. Depending on the negotiation object, there are many potential protocols with varying degrees of freedom (for an overview, see [41]) that could be used. Auctions [22, 52] are a common choice for the negotiation protocol. As they are strictly defined, they mark the end of a spectrum of degrees of freedom that allows for the least choices within the protocol. The most liberal form of negotiation is constituted by bilateral negotiations. They allow for many different actions during the negotiation. The protocol is only weakly defined and sometimes even subject of the negotiation itself. Both parties agree on a series of offers and counter offers only. Beyond that, each party is free to come up with proposals as they see fit. There is no rule on how the offers have to be constructed. Various types of auctions such as the English auction , the Dutch auction or the Vickrey auction [48] provide a means to determine a price for a good based on how the market values the good. There is no need on the side of the seller to have an idea of the value of the good as an auction determines the prices solely based on the bids of the bidders (i.e. the demand side of the market). Once the objects of the negotiation grow more complex, single issue auctions cannot be used to negotiate on them. In such cases, there are more complex forms of auctions that could be used. If those types are not suitable either, the negotiation can turn to more liberal types such as bilateral negotiations. They allow for a freeform exchange of offers and counter offers between the participating parties. All participants have to agree on a protocol to use before the negotiation starts. One example for such a protocol is the alternating offers protocol [40] that has been studied extensively for automated negotiations (for example, see [49]). It has been used as the basic protocol for various modifications and adaptations, such as the monotonic concession protocol [39], which, in turn, is the foundation of many other protocols. While negotiations between more than two parties (m:n negotiations) constitute the most complex form of negotiations (for an overview, see [38]) and make use of sophisticated protocols (e.g. [26]), which require extensive testing [8], they do not form the main type of negotiation that is used in this chapter. We aim to coordinate various actors in an environment to provide a frictionless course of events. The main reason to utilize market based mechanisms is the intentional aggregation of all available data into just on one variable to decide upon. If the negotiation object is simplified into a single issue, we can build our protocols on the extensive body of knowledge in negotiation theory. Thus, we will consider single issue negotiations for the remainder of the chapter. In our model we assume a discrete time-line, where all actions take place at consecutive steps. Furthermore, agents have to pay a tax Ta (t) to the environment E at every time t for their actions and according to their strategy parameters sa . Given the tax, we can calculate the profit πa an agent a receives at time t by

πa (t) = Ra (t) − Pa (t) − Ta(t)

(10.2)

214


where Pa (t) denotes the payment a has to pay to other agents or the environment in order to follow its strategy sa . Finally a may have revenues Ra (t) from previously executed actions. Based on the profit πa (t), an agent accumulates funds ca over time expressed by ca (t + 1) = ca (t) + πa(t) (10.3) where ca (t + 1) denotes the funds of agent a at time (t + 1), ca (t) denotes a’s funds at time t and πa (t) is the profit of a at time t. Money cannot be ‘created’ by the agent. It is provided by the environment E representing the demand D of the market since it provides a specific amount of funds in return to services offered by the set of agents A.

10.3.3 Evolutionary Computation and Agents This section contributes to research on location problems by proposing an agent oriented decentralized evolutionary computation (EC) approach that exploits the flow of money or energy in order to realize distributed optimization. It uses the general market-based approach given in the previous section. The section presents one general outcome of our model: how adaptation occurs in the number and strategies of agents leading to an improvement at the system level. The novelty of this approach lies in the biology-inspired bottom-up adaptation method for inherent distributed problems. The decentralized evolutionary approach described in this section differs in two ways from traditional evolutionary agents. First, the fitness calculation is replaced by an economic model. Second, the entire agent population constructs one solution only. Our approach uses local operators for reproduction like mutation, recombination and selection regulated by market mechanisms (see section economic and market based agents). 10.3.3.1

Evolutionary Computation as Decentralized Adaptation Method

The assumptions on a system that applies this method are 1. Agents have the ability to communicate and to sense their environment at least locally. 2. There is no central manager, instead the DOP and its solution is distributed among A. 3. There is a limited amount of money (D) that will be provided by the environment E. The previous section has shown a market-based multi-agent system that consists of economic agents that have to solve a DOP collaboratively and fulfill some prerequisites. We re-use evolutionary algorithm theories of Holland [13, 16] concurrently with a decentralized economic agent perspective [44] as well as our economic market perspective as we believe that a decentralized market mechanism can be a

10


215

profound approach to replace the central fitness calculation in evolutionary algorithms. There is a variety of evolutionary computational models that have been proposed and studied that we will refer to as evolutionary algorithms [13, 16]. In our agent based approach we turn the typical evolutionary algorithm software design on its head by breaking up any central instance in the model. Instead, we move the genetic representation as well as the operators to the agents themselves as proposed in [45]. Local reproduction by an agent includes all steps and operations necessary to produce new offspring, such as local search, recombination and mutation. We introduce an agent specific variable θ that serves as a threshold and enables agents to reproduce. Whenever the funds of an agent exceed this threshold (ca ≥ θ ) it will reproduce. Advantages of local versus global reproduction are discussed in [28] in detail. As we focus on emerging global effects of local reproduction, based on [27], an agent’s algorithm is formulated in algorithm 8. It is motivated by modeling ecologies of organisms adapting in environments. Algorithm 8. Main loop, every agent executes Require: s and c(0) from parent 1: initialize f 2: loop 3: get sensory inputs I(t) 4: execute f 5: set effector outputs O(t) 6: calculate c(t) using equation (10.3) 7: if c(t) ≥ θ then 8: reproduce and split c(t) with child 9: else if c(t) < 0 then 10: die 11: end if 12: end loop

An agent a is set up with its strategy sa and funds c(0) by its parent at t = 0 (t denotes the lifetime of an agent). The initial population is created by assigning a random strategy and an initial reservoir of funds to every agent. Once started, an agent will continuously follow his strategy by executing f . Whenever c reaches θ the agent will reproduce (line 7-8) and create a new agent. In the opposite case, if an agent dies, its funds drop below zero (c < 0, line 9), indicating the bankruptcy of an inefficient agent. Therefore, the notion of ‘generation number’ does not exist and a perpetual reproduction and replacement of agents takes place instead. This mechanism makes survivor selection unnecessary and population size an observable. It can be regarded as a combination of biological and economic ideas [44], because no rational economic agent would create its own competition. However, in biology, this is the case. Given the MAS in the previous sections and algorithm 8 it is clear that the distribution of money within the system differs among the agents. Any agent acts selfishly

216


and tries to exploit the limited money based on its own strategy; however, we assume an agent has no particular intelligence about how this strategy relates to its own success. In the following sections a short description of the two basic properties of our model is given: adaptation in the number of agents and their strategy. Otto et al. more extensive investigation [35]. 10.3.3.2

Adaptation of the Number of Agents

Following the statement in [9], in natural environments the population size tends to stabilize around the carrying capacity of the ecosystem. We define the carrying capacity as demand D. As outlined in [13] the system can only be stable for bounded input when D does not grow in magnitude on its own accord. For further analysis, we approximate the creation of agents by a continuous adapπ (t) tation in population growth. For a particular agent a the ratio aθ is an expression of the estimated creation of new agents at time t. In fact, the number of agents increase/decrease stepwise and not continuously, depending on the distribution of Π (t) = ∑a∈A πa (t) and the funds in A. Dividing the overall profit Π (t) by θ , we get the average number of new/dying agents at time t + 1 (again, averaged for sufficient long runs) as |A(t + 1)| = |A(t)| +

Π (t) . θ

(10.4)

Thus |A(t + 1)| varies by factor Πθ compared to |A(t)|, where |A(t)| is the number of agents in A at time t. If Πθ is positive (negative), the number of agents will increase (decrease) depending on Π . This emergent behavior can be observed in real scenarios [14] where actors enter and leave the market. The rate at which agents enter or leave the market is directly correlated with the overall profit (see equation 10.4) and the market supply S and demand D in such a scenario. Even without any central control this effect can be observed. 10.3.3.3

Distributed Problem Optimization by Spread of Successful Strategies

Since there is no central instance in our model that can rank and compare the agents funds, no central selection based on fitness can be calculated and performed. Therefore, the system must evolve fitter strategies in an emergent self-organizing way. For the following, discussion we assume that strategies were reproduced in a pure way without disturbance of mutation or recombination. The proportion of strategy 1 sa in A(t) is pa (t) = |A(t)| , where |A(t)| denotes the number of agents. Based on evolutionary pressure, the agent population’s profit Π reaches zero after a sufficient amount of time. For long running simulation we can set limt→∞ Π (t) = 0 (proof is given in [35]). According to equation 10.4, the number of agents in the next time

10


217

step can be rewritten by |A(t + 1)| = |A(t)| and the expected proportion of sa at time step t + 1 is as follows: 1 + πaθ(t) pa (t + 1) = (10.5) |A(t)| with 1 + πaθ(t) being the original strategy plus the estimated additional quantity of sa that together form the number of samples of strategy Sa in A(t + 1). It follows that, based on the proportion pa (t + 1) > pa (t) for a positive profit, the part of sa in A grows. The proportion of a particular strategy sa grows as the ratio of the average profit πa of agent a. Strategies causing a positive/negative profit will receive an increasing/decreasing number of replications in the next time step. As a result, strategies inducing a positive profit may have to pay less tax compared to below zero profit strategies due to the usage of less (or more efficient) resource usage. Both are needed to streamline logistic networks while simultaneously improving service to the customer. Based on equation 10.2, one can see that tax T is the part of an agent’s profit determining variable which is not explicitly related to the flow of goods. With tax, the model’s basic conditions can be controlled and the agent population will adapt to it, if the tax is not too high. Therefore, an intrinsic property of systems using our model is the constant search for better resource utilization. We have proposed a decentralized evolutionary framework with self interested economic agents. The approach uses an economic model instead of a classic fitness function. This enables the complete distribution of evolutionary steps and their local execution in the agents’ scope. Another aspect is the distributed solution, which is a combination of partial solutions of all agents. In contrast to conventional evolutionary agents, where each individual contains the complete solution, we spread the solution over the whole population to address distributed problems.

10.4 Case Studies In this section, we will introduce three case studies from different domains to illustrate the broad usage of the approaches.

10.4.1 Hybrid Decision Systems Some domains must account for information that is unavailable to the computer systems in advance. It can only be used as an input in an online way, by acquiring the source for the information whenever it is needed. One potential domain that relies on such information is the healthcare sector. Much information is available as experiential knowledge only. Humans are able to decide in certain situations, but cannot provide the information in a way that would make it accessible before runtime.

218


Hybrid decision making is a way to introduce online, human decisions to MAS. An example is the treatment scheduling that can be improved by such a system. Software agents decide as much as they can. The final decision, however, is made by humans who contribute their knowledge in the process. A scheduler in a hospital has to consider the individual schedules of all current patients. One challenge is the dynamic nature of a hospital. New patients have to be accepted while others leave the hospital after they have been treated. Each new admission comes with a different priority. Emergencies must be treated immediately and cannot bear any waiting time. They cannot be appended at the end of the current schedule but need to be inserted at the immediate moment. Other treatments have to be postponed in that case, which results in waiting times for other patients. On the other hand, resources lie idle if patients do not show up on time. The scheduler (human or IS) can try to bring forward treatments but he is narrowly confined by medical constraints. Currently, modifications of a schedule are done manually by a qualified medical technical assistant [31]. This setup makes the quality of the schedule dependent on the experience of the employee. In addition, the setup does not scale well. With a growing number of treatments to monitor, humans suffer from “information overflow” [47], which decreases their decision making ability. Software agents have been proposed to decrease information overload by supporting humans [24]. Hybrid decision making takes this agent-based approach and combines it with market-based coordination. IS offer the possibility to support scheduling, since they are able to process great amounts of data very quickly. Because of the decentralized structure of the entire system, a decentralized modeling approach is the obvious choice for such a system. One possibility is to use an agent based system [18] that can represent the decentralized nature of a hospital, since (MAS) [51] work distributed by their nature. To provide the greatest possible flexibility, the approach represents passive resources and human actors in the hospital as “passive” and “active” software agents [11]. The real world is mapped into a logical image of the world that is used for all computations. An agent handles the entire communication within the logical world on behalf of its human principal. It communicates with its principal by using an additional, bi-directional communication channel (such as a mobile phone or another handheld device). Apart from active resources, passive resources also have their counterpart in the logical world. Software agents represent passive resources (like operating theaters or medical equipment) in the same way they represent humans. Passive resources are necessary for the scheduling, but do not add any own constraints (apart from the prohibition of an overbooking) to the scheduling problem. Necessary information to enable automated scheduling is the current activity of all humans in the hospital. However, in a clinical environment, doctors or nurses cannot be required to keep a log of their activities manually. Rather, the current activity has to be sensed by the software agents. The agents incorporate the activity into their scheduling mechanisms. This case study proposes an indirect way of monitoring the current activity of all actors by using an automated positioning system, based on RFID (radio frequency identification) in the broadest sense.

10


219

Software agents need to locate their respective principals by using their sensors. Various possible technologies to locate humans within a hospital exist, with WLAN positioning emerging as the most promising one. We subsume WLAN positioning under the heading RFID positioning, as it abstracts from the specific characteristics of the technologies. The concept relies on an accurate positioning system that provides location information about all humans in the hospital. However, it does not specify a particular technology to be used. By utilizing RFID technology with suitable sensors, the logical world is provided with location information. A software agent transforms the raw geographical location information into contextual location information by augmenting the raw data with semantics [37]. An additional step of abstraction allows to infer the current activity of an actor. This conclusion can be drawn by a rule-based system [30]; other approaches rely on more complex techniques such as pattern recognition [15] to infer the current activity. To support scheduling in hospitals, for an agent it is sufficient to know the binary status of its principal’s activity (i.e. active vs. idle), which it can infer by using a simple rule-based system. In the particular case of scheduling systems, the system defines some areas of activity (e.g. operation theaters) and some areas that correspond to idleness. The positions of actors in one of these zones correlate to their activity. If an agent knows that its principal is active, it cannot plan another appointment for its principal as the human cannot perform two activities at the same time. On the other hand, an idle principal is potentially available for an appointment. Since the agent does not make the final decision but generates suggestions for decisions only, the classification neither needs to be more precise nor entirely accurate. The accuracy of the approximation has to be traded against the requirement of a calm technology. Location information proves to be an integral part of the hybrid decision support, as it allows agents to sense their principals’ activity in an indirect way. In absence of another activity sensing technology, an agent would have to ask its principal about his/her current activity if no location information was available. Thus, location information proves to be a necessary feature of the MAS architecture that enables autonomous behavior [4]. To aggregate and process data from different sources, the chapter proposes a layered architecture as shown in Figure 10.1. Each layer processes different amounts of data according to its hierarchy. The quality of the information increases with every step of refinement, whereas the quantity of data decreases. The aggregation process consists of four steps. The data has to be identified, its context has to be detected, it has to be integrated with data from the clinical information system, and, finally, it can be optimized in a hybrid DSS. While the lower three layers deal with the aggregation of data, the actual decision making happens in the optimization layer. It utilizes the market-based approach to reduce the number of decisions that a human has to make. As soon as an agent receives information about a failure in the schedule, it can try to correct the schedule. The agent has to solve a co-allocation problem. Many different resources have to be available for a particular treatment at the same time.

220


decreasing quantity of data increasing quality of information

medical knowledge

data from clinical IS

semantics

decision support multi agent system + human decisions

optimization

integration

context detection

multi agent system

identification

RFID data

Fig. 10.1 Layered architecture for a scheduling system in hospitals

The agent tries to negotiate a schedule that corrects the failure while still meeting all other constraints. In the optimization layer, software agents are not able to decide autonomously anymore, because medical knowledge is necessary to assess a situation. Since only human actors posses medical knowledge, the software agents depend on their principals in their architecture [4]. The optimization layer is the first layer that relies on the concepts from DSS, which combine human and automated decision making power. Software agents are able to check the validity of the schedule more quickly in the logical world than humans ever could in the real world. If a post– or preponement has been detected, the agents react accordingly and try to correct the schedule. Throughout the three lower layers, the agents act fully autonomously, only the topmost layer requires communication between agent and principal. The agent generates suggestions for decisions, the human principal has to agree on or reject these suggestions. A human actor makes a decision based on the highly condensed data that is provided by the software agent. The search for information is conducted by the agent to the maximum extent possible. With the responsibility to correct the schedule a software agent has to consider various factors. On the one hand, it knows about the announced availability of other software agents, on the other hand, medical considerations have to be taken into account that are not specified at the time of the decision process. The software agents utilizes a market based concept to coordinate themselves with the know constraints. While medical staff provides services on an artificial market, the patients consume these services and need to be matched with the right medical staff. As

10


221

much information as possible is encoded in an artificial currency which serves as an aggregation of all operational data [32, 46]. As the currency does not include any medical information, the software agents generate suggestions that are proposed to their principals, if the current level of autonomy allows for such a delegation of decision making.

10.4.2 Evolutionary Agents Optimize Supply Chain Structures Logistics networks can be seen as complex networks that need to adapt their structure (e.g. transport routes, interactions) as reaction to environmental changes, e.g. the market demand, supplier unavailability or route changes. We model such distributed supply chains as a decentralized evolutionary MAS in order to draw an analogy to real world scenarios. The connections in supply chains can be seen as a complex network of coexisting but simple interdependent agent strategies producing the necessary transportation network. We describe how our decentralized approach can be used to solve inherently distributed problems where no central optimization algorithm exists. The simulation results show the applicability of evolutionary agents to transport network optimization. The setup consists of a multi-stage supply chain : retailer, wholesaler, distributor and producer. At every stage, a varying number of competing agents exists. The purpose of the supply chain is to meet customer demand, which is generated by orders at retailer agents. To demonstrate and verify the viability of the approach, we have conducted a large number of experiments using different settings and strategies such as mutation rate and searchrate (how often an agent searches for a new supplier). In defined intervals, every agent searches for a potential supplier in the previous step of the chain. In order to model decentrality, agents cannot ‘search’ for specific agents. Instead they will be connected to a random supplier agent. If the new supplier offers a price below the current supplier price the searching agent replaces its current supplier with the new one. Therefore the searchrate induces network dynamic and has direct influence on the network structure. This mechanism performs a search for the cheapest supplier with one supplier per agent at each time step t. The analysis applies to experiments with the following settings: map size is 100 × 100, customer agents A0 (|A0 | = 100) are normally distributed with N (p0 , 5), where p0 = {75, 75}, all other agents Ai , i = {1, . . . , k}, |Ai | = 10 are initial randomly distributed, initial funds of agents ca (0) = 140, initial inventory: 15, buying price for products at the producers site = 10, stock: sa (stockmin ) = 5 and sa (stockmax ) = 15, f it splitting threshold θa = 15 · sa(stockmax ) · pbuy · p pro (every agent is capable to buy +pro f it fill its inventory multiple times dependent to the position in the chain), pro f it = 10, searchCost = 10, price per distance unit pΔd = 0.01, penalty for one product backlog = 1, stockCost = 0.5. Every customer orders one product per round. Every setting was averaged over 50 independent simulation runs with 1500 rounds each. The evolutionary strategy consists of number values and it is recombined using the breeder evolutionary algorithm [29]. The search for a new supplier is a simple random connect to a supplier at the previous stage of the supply chain. The agent

222


performs a price request every searchrate rounds and replaces its current supplier if the new supplier is cheaper than the current one.

9000

7500

1% 5% 10% 8000 50%

10 15 20

7000 6500 6000

7000

Distance

Distance

5500 6000 5000

5000 4500 4000

4000

3500 3000

3000

2500 2000

2000 0

200

400

600

800

1000

Round

(a) mutation rate

1200

1400

1600

0

200

400

600

800

1000

1200

1400

1600

Round

(b) search rate

Fig. 10.2 Different mutation rates and search rates and their influence to the overall transport distances

10.4.2.1

Mutation Rate

Simulations were computed for mutation rates of 1, 5, 10 and 50 percent, the search rate is 10. Figure 10.2a) shows the results. The overall distance in the supply-chain in every round is plotted for the above mentioned mutation rates. At the beginning, the distances are almost equal for all mutation rates. Already after a few rounds, the first search process is carried out and some agents can reproduce, whereas others are removed from the agent system because they are running out of money. A high mutation rate causes extreme values for the agents strategy (e.g. position sa (x), safety stock sa (stockmin ) and maximum stock sa (stockmax )). Extreme mutation rates induce a random walk of the solution as shown in figure 10.2 (mutationrate = 50%). The best performance is shown with mutation rate of 1%. 10.4.2.2

Search Rate

Simulations were computed for search rates of 10, 15 and 20, the mutation rate is set to 1%. In figure 10.3b) the different results in the supply chain are plotted. In the simulations where the agents look for a more favorable supplier more often (e.g. decreased searchrate), agents that are further away from the market agents get removed from the system faster and agents closer to the customer agents are selected more often. The overall dynamic of the supply-chain increases with a decreasing search rate.

10


(a) round 1

223

(b) round 1500

Fig. 10.3 Snapshots of the supply chain at a) round 1 and b) round 1500

10.4.2.3

Location of Agents

Figure 10.3 shows the state of the network taken after round 1 and at the end of the simulation (round 1500). In Figure 10.3a), the initial random distribution of agents and the direction of the products can be seen. The customers area (mean) is shown in the right bottom corner as a thick black circle. After 1500 rounds the agents are concentrated around the customer agents. Due to the mutation, some agents are placed far away from the customer agents. These agents are unlikely to be selected as supplier and will be removed from the agent system immediately. Although the supplier search method is a simple random connect, the overall network of supplier to buyer connections gets minimized.

10.4.3 Evolutionary Agents Optimize the p-median Problem The facility location problem also known as p-median problem [1, 17] concerns the positioning of facilities such as bus-stops, broadcasting stations or supply stations in general, like cluster heads in sensor networks [2]. The objective is to minimize the weighted distance between demand points (or customers) and facilities. This section contributes to research on location problems by applying evolutionary agents on distributed optimization. Our study consists in positioning facilities on a discrete map by using two pmedian sample problem sets presented in [1, 12]. The set of locations L is given by the problem set and facilities are represented by agents. The agents strategy vector consists two values: the location sa (location) ∈ L and the profit factor sa (pro f it) ∈ R. An agent a offers service to customer at location j ∈ L according to

224


costa j = wsa (location) j ∗ sa (pro f it). Customers choose the agent with lowest cost and the selection is expressed as

1 customer j is served by agent a at time t σa j (t) = 0 otherwise Multiple agents can offer their service at the same location l and the ‘cheapest’ agent with lowest cost at location l is defined as a˙l . The set of cheapest agents is defined as A˙ ⊆ A with sa˙i (location) = sa˙ j (location), a˙ i, a˙ j ∈ A˙ and considered as the set of facilities P that form the solution to the p-median problem. For simplicity reasons the set of A˙ is denoted with A, if not stated otherwise. The p-median problem specific DOP is given as minimize f (sA ) =

l

A

∑ ∑ wsa (location) j

(10.6)

j=1 a=1

subject to Ra (t) =

πa (t) = Ra (t) − Pa (t) − Ta (t) l

A

∑ ∑ costa j (t) ∗ σa j (t)

(10.7)

j=1 a=1

Pa (t) = 0

(10.8) l

Ta (t) = 10 + ∑

A

∑ wa j ∗ σa j (t)

(10.9)

j=1 a=1

According to equations 10.7 and 10.9 the profit πa (t) is basically dependent on sa (pro f it). The fixed tax of 10 currency units is necessary to slowly remove agents ˙ Otherwise they would remain in the system forever and with no income from A. consume resources. As the fixed tax is negligible, a profit factor sa (pro f it) > 1 is important and induces an evolutionary pressure on agents to evolve a strategy with sa (pro f it) slightly over 1. Otherwise, an agent will be removed based on a negative profit as well as turning inactive. There is no direct ressource transfer among the agents (eq. 10.8) in this particular application but money is transfered between customers and agents. The splitting threshold θ is dynamically calculated as the average income over the last two time steps, since a fixed θ would need to be adjusted for every problem. The initial setting is a population of 20 agents, a mutation rate of 0.03, 10000 time steps and a profit factor of 2. It is important to set the initial profit factor high enough compared to the expected payments in order to get a running system. During the simulation, an asymptotic convergence of the profit factor to 1 is expected. The agents’ strategy is represented as a number vector that can be evolved using the common variable-length binary valued crossover and mutation operators [13]. We use uniform crossover and an implementation of the Breeder Genetic Algorithm [29] for mutation throughout all runs. Selection is disabled and results are obtained by averaging 50 simulation runs.

10


a)

cost

b)

Galvao100 problem set 6500 6000 5500 5000 4500 4000 3500 3000 2500 5

10

15

20 p

25

Alberta problem set 5.5e+007 5e+007 4.5e+007 4e+007 3.5e+007 3e+007 2.5e+007 2e+007 1.5e+007 1e+007 5e+006 0

distributed agents random optimal

30

35

225

distributed agents random optimal

30

40

50

60 p

70

80

Fig. 10.4 Distributed agents approach compared with random search and optimal values for multiple p on two problem sets (Galvao: 100 nodes, Alberta: 316 nodes)

Figure 10.4 shows the results of two different problem sets and compares costs different p-values to optimal values and random search. During the simulation, the agent population explores different values of p as the population size is changing. As mentioned in section 10.3.3, the population constructs one solution at every timestep. This solution varies over time and thus explores the search space. Therefore, multiple Pareto-optimal solutions can be found during a single simulation run by one population. Although the validation is performed on a mono-objective model we are thus faced with Pareto-optimal solutions. At the end of each independent run all Pareto-optimal values explored during the run are averaged (The averaged values are not necessarily Pareto optimal) for statistical relevance. The graphs show clear differences between distributed agents approach and random search. There are, however, significant differences in search space exploitation for different p-values in the distributed agents approach that need further investigations in different tax and starting conditions.

10.5 Conclusion and Future Work Most open systems are very complex in nature and require an equally complex decision making mechanism to provide reasonable decisions in the systems. This chapter has introduced two agent-based approaches that work in such environments. The market-based approach to decision making has been shown to provide a means to aggregate the behavior of various agents that have competing goals. Evolutionary agents as an extension of the former market-based approach are able to adapt to the environment by incorporating an evolutionary learning mechanism into the decision making. An economic model is used instead of a classical fitness function. This enables the complete distribution of evolutionary steps and their local execution in the agents’ scope. Another aspect of evolutionary agents is the distributed solution, which is a combination of partial solutions of all agents to form one solution in contrast to conventional evolutionary agents, where each individual contains the complete solution. Hence, evolutionary, market-based systems can arrive at decisions

226


among a number of agents in an efficient manner. In open systems, the agents are able to adapt to changing circumstances by applying an evolutionary approach. If a system relies on human knowledge for the decision making process, hybrid decision making provides a valuable approach to incorporate such knowledge. All three case studies have shown their merit in various cases. However, work has to be done to integrate the three in a system that is truly open, adaptive to changes, and hybrid. Furthermore, there is the need for research on each of the approaches on its own. A market-based approach provides a means to aggregate goals, but the process of doing so (i.e. the negotiations) determine the outcome to a large degree. Some of the interesting avenues for research include the question on how to arrive at an efficient outcome, if the process of getting there involves concessions that seem to be counter-productive in the short term but improve the outcome in the long term.

References 1. Alp, O., Drezner, Z., Erkut, E.: An efficient genetic algorithm for the p-median problem. Annals of Operations Research 122(1-4), 21–42 (2003) 2. Arthur, H., Daniel, J., Alejandro, B.: A3ME - an agent-based middleware approach for mixed mode environments. In: Proc. of The Second International Conference on Mobile Ubiquitous Computing, Systems, Services and Technologies (UBICOMM 2008), pp. 191–196. IEEE Computer Society Press, Los Alamitos (2008) 3. Bar-Yam, Y.: Dynamics of Complex Systems. Westview Press, Boulder (2003) 4. Castelfranchi, C., Falcone, R.: From automaticity to autonomy: The frontier of artifical agents. In: Hexmoor, H., Castelfranchi, C., Falcone, R. (eds.) Agent Autonomy, Multiagent Systems, Artificial Societies, and Simulated Organizations. Kluwer Academic Publishers, Boston, Dordrecht, London, pp. 103–136. Kluwer Academic Publishers, Dordrecht (2003) 5. Chavez, A., Dreilinger, D., Guttman, R., Maes, P.: A real-life experiment in creating an agent marketplace. In: Nwana, H.S., Azarmi, N. (eds.) Software Agents and Soft Computing: Towards Enhancing Machine Intelligence. LNCS, vol. 1198, pp. 160–179. Springer, Heidelberg (1997), doi:10.1007/3-540-62560-7_44 6. Chavez, A., Maes, P.: Kasbah: An agent marketplace for buying and selling goods. In: Proc. of the First Int. Conf. on the Practical Application of Intelligent Agents and MultiAgent Technology. Practical Application Company, London (1996) 7. Dignum, F.: Agents, markets, institutions, and protocols. In: Dignum and Sierra [8], pp. 98–114 (2001) 8. Sierra, C., Dignum, F.P.M. (eds.): Agent Mediated Electronic Commerce - The European AgentLink Perspective. LNCS (LNAI), vol. 1991. Springer, Heidelberg (2001) 9. Eiben, A., Schut, M.C.: New ways to calibrate evolutionary algorithms. In: Michalewicz, Z., Siarry, P. (eds.) Advanced in Metaheuristics and Optimization, pp. 153–178. Springer, Heidelberg (2007) 10. Fatima, S., Wooldridge, M., Jennings, N.R.: Optimal agendas for multi-issue negotiation. In: Proceedings of the 2nd International Joint Conference on Autononomous Agents and Multi-Agent Systems (AAMAS 2003), pp. 70–77 (2003) 11. Ferro, D.N., Valk, J.M., Salden, A.H.: A robust coalition formation framework for mobile surveillance incident management. In: de Walle, B.V., Burghardt, P., Nieuwenhuis, C. (eds.) Proceedings of the 4th International ISCRAM Conference, Delft, Netherlands (2007)

10


227

12. Galvão, R.D., ReVelle, C.: A lagrangean heuristic for the maximal covering location problem. European Journal of Operations Research 88, 114–123 (1996) 13. Goldberg, D.E.: Genetic Algorithms in search, optimization, and machine learning. Addison Wesley Longmann, Inc., Reading (1989) 14. Graudina, V., Grundspenkis, J.: Technologies and multi-agent system architectures for transportation and logistics support: An overview. In: Proc. of International Conference on Computer Systems and Technologies - CompSysTech, Varna, Bulgariapp, pp. IIIA.6– 1–IIIA.6–6 (2005) 15. Hodges, M.R., Pollack, M.E.: An ‘Object-use fingerprint’: The use of electronic sensors for human identification. In: Krumm, J., Abowd, G.D., Seneviratne, A., Strang, T. (eds.) UbiComp 2007. LNCS, vol. 4717, pp. 289–303. Springer, Heidelberg (2007) 16. Holland, J.H.: Hidden Order: How Adaptation Builds Complexity. Addison Wesley Publishing Company, Reading (1996) 17. Hosage, C.M., Goodchild, M.F.: Discrete space location-allocation solutions from genetic algorithms. Annals of Operations Research 6(2), 35–46 (1986) 18. Jennings, N.R.: On agent-based software engineering. Artificial Intelligence 117, 277– 296 (2000) 19. Jennings, N.R., Faratin, P., Lomuscio, A.R., Parson, S., Wooldridge, M., Sierra, C.: Automated negotiation: prospects, methods and challenges. Group Decision and Negotiation 10(2), 199–215 (2001), doi:10.1023/A:1008746126376 20. Jonker, C.M., Treur, J.: An agent architecture for multi-attribute negotiation. In: Proceedings of the 17th International Joint Conference on AI (IJCAI), vol. 2001/1, pp. 1195– 1201 (2001) 21. Kirn, S.: Flexibility of multiagent systems. In: Kirn, S., Herzog, O., Lockemann, P., Spaniol, O. (eds.) Multiagent Engineering, International Handbooks on Information Systems, pp. 53–70. Springer, Heidelberg (2006) 22. Kumar, M., Feldman, S.I.: Business negotiations on the internet. In: Proc of INET 1998 – Conference of the Internet Society (1998), http://www.isoc.org/inet98/proceedings/3b/3b_3.htm 23. Lang, F., Bodendorf, F.: Automated negotiation in agent based commercial grids. Journal of Decision Systems 15(1), 55–81 (2006), doi:10.3166/jds.15.55-81 24. Maes, P.: Agents that reduce work and information overload. Communications of the ACM 37(7), 30–40 (1994), doi:10.1145/176789.176792 25. Maes, P., Guttman, R.H., Moukas, A.G.: Agents that buy and sell. Communications of the ACM 42(3), 81–91 (1999), doi:10.1145/295685.295716 26. Matos, N., Sierra, C.: Evolutionary computing and negotiating agents. In: Noriega, P., Sierra, C. (eds.) AMET 1998 and AMEC 1998. LNCS (LNAI), vol. 1571, pp. 91–112. Springer, Heidelberg (1999) 27. Menczer, F., Belew, R.K.: Adaptive retrival agents: Internalizing local context and scaling up to the web. Mach. Learn 39(2-3), 203–242 (2000) 28. Menczer, F., Degeratu, M., Street, W.: Efficient and scalable pareto optimization by evolutionary local selection algorithms. Evolutionary Computation 8(3), 223–247 (2000) 29. Mühlenbein, H.: The breeder genetic algorithm - a provable optimal search algorithm and its application. In: Colloquium on Applications of Genetic Algorithms, vol. 67, pp. 5/1–5/3. IEEE, London (1994) 30. Müller, G., Eymann, T., Nopper, N., Seuken, S.: Emika system – architecture and prototypic realization. In: Proceedings of the IEEE Intl. Conference on Systems, Man and Cybernetics, Den Haag (2004)

228


31. Müller, G., Kreutzer, M., Strasser, M., Eymann, T., Hohl, A., Nopper, N., Sackmann, S., Coroam˘a, V.: Geduldige technologie für ungeduldige patienten – führt ubiquitous computing zu mehr selbstbestimmung? In: Mattern, F. (ed.) Total Vernetzt, pp. 159–186. Springer, Heidelberg (2003) 32. Niemann, C., Eymann, T.: Softwareagenten in der krankenhauslogistik – ein ansatz zur effizienten ressourcenallokation. HMD – Praxis der Wirtschaftsinformatik 251, 77–87 (2006) 33. Oram, A.: Peer-to-Peer: Harnessing the Power of Disruptive Technologies, 1st edn. O’Reilly Media, Inc., Sebastopol (2001) 34. Otto, S.: Ein agentenbasierter evolutionärer adaptions- und optimierungsansatz für verteilte systeme. Ph.D. thesis, Universität Erlangen - Nürnberg (2009) 35. Otto, S., Kirn, S.: Evolutionary adaptation in complex systems using the example of a logistics problem. International Transactions on Systems Science and Applications 2(2), 157–166 (2006) 36. de Paula, G.E., Ramos, F.S., Ramalho, G.L.: Bilateral negotiation model for agentmediated electronic commerce. In: Dignum, F.P.M., Cortés, U. (eds.) AMEC 2000. LNCS (LNAI), vol. 2003, pp. 1–14. Springer, Heidelberg (2001), doi:10.1007/3-54044723-7_1 37. Pradhan, S.: Semantic location. Personal and Ubiquitous Computing 4, 213–216 (2000), doi:10.1007/BF02391560 38. Raiffa, H.: The Art and Science of Negotiation. Harvard University Press, Cambridge (1982) 39. Rosenschein, J.S., Zlotkin, G.: Rules of Encounter: Designing Conventions for Automated Negotiation Among Computers, vol. 2. MIT Press, Cambridge (1998) 40. Rubinstein, A.: Perfect equilibrium in a bargaining model. Econometrica 50(1), 97–109 (1982) 41. Sandholm, T.W.: Distributed rational decision making. In: Weiss, G. (ed.) Multiagent Systems – a Modern Approach to Distributed Artificial Intelligence, pp. 201–258. MIT Press, Cambridge (2000) 42. Simon, H.A.: The Sciences of the Artificial. MIT Press, Cambridge (1996) 43. Smith, A.: An Inquiry into the Nature and Causes of the Wealth of Nations, a new edn, Hartford, vol. 1804 (2010), Goldsmiths’-Kress Library number 18816.20 44. Smith, R.E., Bonacina, C., Kearney, P., Eymann, T.: Integrating economics and genetics models in information ecosystems. In: Proceedings of the 2000 Congress on Evolutionary Computation CEC 2000, pp. 959–966. IEEE Press, La Jolla Marriott Hotel La Jolla (2000) 45. Smith, R.E., Taylor, N.: A framework for evolutionary computation in agent-based systems. In: Looney, C., Castaing, J. (eds.) Proceedings of the 1998 International Conference on Intelligent Systems, pp. 221–224. ISCA Press (1998) 46. Strasser, M.: Zur Selbstorganisation der Patientenlogistik mit allgegenwärtigen Rechnern. Eul, Lohmar (2008) 47. Toffler, A.: Future Shock. Random House (1970) 48. Vickrey, W.: Counterspeculation, auctions, and competitive sealed tenders. The Journal of Finance 16(1), 8–37 (1961) 49. Winoto, P., McCalla, G., Vassileva, J.: An extended alternating-offers bargaining protocol for automated negotiation in multi-agent systems. In: Chung, S., et al. (eds.) CoopIS 2002, DOA 2002, and ODBASE 2002. LNCS, vol. 2519, pp. 179–194. Springer, Heidelberg (2002), doi:10.1007/3-540-36124-3_11

10


229

50. Wooldridge, M., Jennings, N.R.: Intelligent agents: Theory and practice. The Knowledge Engineering Review 10(2), 115–152 (1995), doi:10.1017/S0269888900008122 51. Wooldridge, M.J.: An Introduction to MultiAgent Systems. John Wiley & Sons, Ltd., Chichester (2005) 52. Wurman, P.R., Wellman, M.P., Walsh, W.E.: The michigan internet auctionbot: A configurable auction server for human and software agents. In: Proceedings of the Second International Conference on Autonomous Agents, Agents 1998 (1998) 53. Zeng, D., Sycara, K.: Bayesian learning in negotiation. International Journal of Human– Computer Studies 48, 125–141 (1998)

Chapter 11

On Reconfiguring Embedded Application Placement on Smart Sensing and Actuating Environments Nikos Tziritas, Samee Ullah Khan, and Thanasis Loukopoulos

Abstract. In a smart home environment appliances and objects have sensing, actuating and wireless communication capabilities. Recent embedded middleware initiatives aim at providing an easy to use programming framework for developing applications that require the cooperation of sensing and actuating nodes. To achieve this, the mobile agent paradigm is adopted under which an application consists of a set of communication agents residing at nodes of the system with adequate resources. Of particular importance in such system is to decide which agent to place where so that network traffic is optimized. In this chapter we discuss the problem of reconfiguring an initial agent placement so that the resulting scheme is more energy efficient. We formulate the aforementioned agent placement problem (APR) which turns out to be particularly challenging due to the various constraints involved. A heuristic algorithm based on graph coloring is proposed and evaluated against a greedy approach under various scenarios. Nikos Tziritas Dept. of Computer and Communication Engineering, Univ. of Thessaly, Volos, Greece and Center for Research and Technology Thessaly (CERETETH), Volos, Greece e-mail: [email protected] Samee Ullah Khan Department of Electrical and Computer Engineering, North Dakota State University, Fargo, USA e-mail: [email protected] Thanasis Loukopoulos Dept. of Informatics and Computer Technology, TEI of Lamia, Lamia, Greece and Center for Research and Technology Thessaly (CERETETH), Volos, Greece e-mail: [email protected]


232

N. Tziritas, S.U. Khan, and T. Loukopoulos

11.1 Introduction In a smart environment, appliances, devices and apparatus are enhanced with sensing and/or actuating capabilities. Applications require the communication and cooperation of smart objects in order to achieve the required functionality. A number of middleware initiatives, e.g., Pobicos [13], Rovers [2], MagnetOS [9], aim at providing an easy way to code applications for smart environments by splitting the initial application into a number of cooperating mobile agents that can be freely installed and moved among the system nodes, given adequate resources. Such agent placement must be reconfigured from time to time due to changes in the environment. Here, we investigate the agent placement problem (APR) from the standpoint of optimizing the network traffic. The rest of the chapter is organized as follows. Sec. 11.1.1, Sec. 11.1.2 and Sec. 11.1.3 illustrate the application model, state our motivation and describe related work; Sec. 11.2 gives the problem definition; Sec. 11.3 illustrates the proposed algorithm which is experimentally evaluated in Sec. 11.4; finally, Sec. 11.5 concludes the chapter.

11.1.1 Application Model Consider the following application meant to identify a potential fire hazard: while (true) for each room R if avg("temp",R)>X & & avg("hum",R)X && avg(hum)X && avg(hum) 2 nodes. We also present a greedy method used for comparison reasons in the experiments.

11.3.1 The APR Problem with 2 Nodes Consider the APR problem for the case where the system consists of two nodes n1 and n2 and a monitoring node nm . All nodes are assumed to have 1-hop distance between each other. We illustrate the functionality of the algorithm through an example. Assume a total of 5 agents are already placed at the system’s nodes as follows: a1 , a2 and a3 are placed at n1 and a4 , a5 at n2 . Table 1 depicts the load generated due to agent communication, as well as the agents’ resource requirements.

Table 11.1 Agent communication load and resource requirements r(ak ) 2 1 2 3 2

a1 a2 a3 a4 a5

a1 0 1 0 2 0

a2 4 0 2 0 0

a3 0 0 0 4 5

a4 1 1 2 0 5

a5 0 0 3 0 0

Let the capacity of the two nodes (resource wise) be: r(n1 ) = 7 and r(n2 ) = 5. Assuming migrations incur no cost and that no functionality or mutual exclusion constraints exist, APR can be transformed to a graph coloring problem as follows. Let G(V, E) be the agent communication graph, whereby the vertices of the graph correspond one to one with the agents and an edge (ak , aw ) exists if ak and aw communicate with each other. Each edge has a weight w(ak , aw ) which equals the

238


communication cost between ak and aw across both directions, i.e., w(ak , aw ) = Ckw +Cwk . Furthermore, each vertex has a weight w(ak ) equaling the computational resource demands by ak . Fig. 11.2 shows the graph G of the example.

1 5 2

2

a2 1

a1

2 a3

6

8

3 a4 3

5

a5 2

Fig. 11.2 Agent communication graph

APR can then be restated as: having constructed the graph G, find a 2-color scheme (e.g., red, black) so that the total weight of the red vertices does not exceed r(n1 ), the total weight of the black vertices does not exceed r(n2 ) and the total weight of edges connecting one red with one black vertex is minimized. Notice, that the two problems are equivalent since we can rewrite (4) as (T old is a constant): minD = T new + α M. Thus, with the migration cost being zero only T new must be minimized, which is what the coloring problem does. Migration cost is included as follows. The communication graph is extended by adding two more vertices, representing the two system nodes (n1 and n2 in the example). These vertices have 0 weight and are colored, e.g., n1 is red and n2 black. Links of the form (n1 , ak ) and (n2 , ak ) connecting n1 respectively n2 , with all agent vertices are created. The respective weights of these links are defined as follows. If ak currently resides at n1 , then w(n1 , ak ) is set to equal the migration cost for transferring ak from n1 to n2 and w(n2 , ak ) is set to 0. The case where ak resides at n2 is symmetric. The reason for setting link weights as above, is that if ak is colored with the same color as the node it currently resides, then no migration cost should be charged while in the opposite case the migration cost must be charged. Fig. 11.3 illustrates the above extension for the agents a3 and a5 assuming that all agent sizes is 8 and that the constant α = 0.5. Red vertex (n1 ) is shown striped, while black vertex

11

On Reconfiguring Embedded Application

1 5 2

2 2

a2

a3

4

n1 0

0 1

a1

239

6

8 0

3 a5

a4

4

5 3

n2 0

2

Fig. 11.3 Extending the communication graph

(n2 ) is shown grayed. Since all migrations are assumed to be performed via the monitoring node (hop distance of 1 against n1 and n2 ), all edges whereby the migration cost must be charged have a weight of 4 (equals α *agent size*hop distance). Functionality constraints are included in the model by placing a fixed color at a vertex. For instance, if in the example L21 = 0, then a1 vertex will be painted in red, i.e., a1 will be forced to stay at n1 . Finally, mutual exclusion constraints are included by adding constraints on the allowable coloring. For example in modeling that Fkw = 1, it is equivalent to say that ak and aw vertices must have different colors.

11.3.2 The Agent Exchange Algorithm Here we present an algorithm namely, the agent exchange algorithm (AXA), to come with a solution for the 2-node version of APR. AXA uses the transformation of APR into the equivalent coloring problem presented in Sec. 11.3.1. The algorithm works in iterations. In each iteration, the edge with the highest weight is selected and the vertices it connects are merged. In case these vertices have a mutual exclusion constraint, the merging is not performed and the edge connecting them is colored grey (i.e., not to be considered further). Otherwise, the new vertex has the cumulative weight of the previous ones and their remaining edges. If any of the merged vertices is colored then the new vertex will also be colored (with the same color). In case the two vertices to be merged are colored with different color each, merging is not performed and the respective edge becomes grey. Fig. 11.4 shows the resulting graph by merging a3 with a5 . After, the merge is performed, AXA attempts to find a feasible coloring in the new graph. To this end it solves knapsack two times, once for n1 and once for n2 , with the candidate objects being the ak vertices (the benefit and size of each object

240


0 1 5

a2

2

1

2 a1

n1

4

a3,a5 4 11

3 a4 3

4 n2 0

Fig. 11.4 Resulting graph after merging

being the vertex’ weight). In the previous example (Fig. 11.4) by solving knapsack on n1 (the red node) we get the following objects to be placed: a1 , a2 , a3 , a5 , filling the resource capacity of n1 which is 7. Having obtained a knapsack solution for n1 , the algorithm checks if the remaining objects fit in n2 . In the example only a4 remains which fits in n2 since r(n2 ) was assumed 5. If so, the algorithm keeps the merged vertex without coloring it and proceeds with the next iteration. Otherwise, the algorithm attempts to find a valid placement by solving knapsack for n2 (the black node) and checking whether the remaining objects fit at n1 . If after trying both knapsack solutions AXA is unable to find a valid placement involving all the objects it backtracks to the graph state before merging, marking the edge under consideration as grey. The algorithm continues with the merging process by selecting the next uncolored edge of highest weight and so on so for until all remaining nodes and/or edges are colored. The final agent placement is derived by solving knapsack as above for the remaining (if any) uncolored nodes.

11.3.3 Extending to N Nodes Tackling the case of N > 2 nodes is done with the pair-wise reconfiguration algorithm (PRA), the pseudocode of which is shown in Alg. 9.

11


241

Algorithm 9. Pseudocode of PRA 1: found:=true; 2: while found do 3: found:=false; 4: for i=1 to N do 5: for j=1 to N do 6: apply AXA over (ni , n j ) pair; 7: if D > 0 then 8: found:=true; 9: keep AXA changes; 10: else 11: discard AXA changes; 12: end if 13: end for 14: end for 15: end while

PRA iterates through all node pairs applying AXA. If during an iteration AXA manages to define a better placement according to (4), the process reiterates, otherwise it ends producing the final agent placement. In order for AXA to successfully optimize locally, i.e., within a node pair, the agent placement, adaptations are required to the way agent communication load and migration costs are modeled. We illustrate them through an example. Assume the network of Fig. 11.5, with 7 nodes plus the monitoring node nm . Let the agents of Table 11.1 be already placed on n2 and n5 as follows: n2 has a1 , a2 , a3 and n5 has a4 and a5 . In other terms n2 and n5 in this example have the same role as n1 and n2 in the example of Sec. 11.3.1. Assuming only these agents exist in the network, the equivalent graph coloring problem is similar to the one in Fig. 11.3, with the exception being that the hop count must be taken into account both on edges representing agent communication (ak , aw ) and on edges representing migration cost (ni , ak ). So all w(ak , aw ) edge weights will be multiplied by a factor of 3 (the hop distance between n2 and n5 ), while all edge weights w(ni , ak ) will by multiplied by the hop distance between nm and the node of the opposite color with which ni was painted. For instance, w(n2 , a2 ) will remain 4 since the distance between nm and n5 (the black node) is 1, while w(n5 , a4 ) will now be 16 since hm2 = 4. Fig. 11.6 depicts the resulting problem graph. For clarity, the only edges connecting n2 and n5 with agent vertices are the ones related to a3 and a5 . In the general case, agents placed on nodes other than the pair in question (n2 , n5 ) might generate load towards some of the agents placed on the pair. Fig. 11.5 gives an example, whereby 3 more agents exist, namely: a6 which is placed at n1 , a7 at n3 and a8 at n7 . The figure also shows the load these agents generate towards the ones placed at n2 and n5 , specifically: C63 = 3, C65 = 2, C73 = 4, C75 = 3 and C85 = 4. Such external (to the node pair) load must be incorporated to the graph coloring model in order for it to map to APR correctly. This external load can be viewed as another form of node related cost in the problem graph, as was the case with

242


a8

:4 a5

a5: 2

a6

a3: 3

n1

a3: 4

n2

a5: 3

a7 n3

n4

n7

n6 n5 nm

Fig. 11.5 Example network

1 2

15

a2 3

a1

6

2 a3

18

4

n2

0

n5

0

0

24

0

9 a4 3

15

a5

16

2

Fig. 11.6 Including hop distance in the problem graph

migration. Consider for instance the migration of a5 from n5 to n2 . Aside from the migration cost of 16 to transfer a5 from nm to n2 there will also be a change on the cost of the external load directed to/from a5 . For instance, the load generated by (a5 , a8 ) communication will not incur a cost of 8, but rather a cost of 20 since the hop distance between the two agents will increase from 2 to 5. In order to incorporate the above case in the problem graph it suffices to augment w(n5 , a5 ) by the load incurred if a5 moved to n2 , i.e., 20 and w(n2 , a5 ) by the incurred load if a5 stayed in n5 , i.e., 8. Repeating the process for all the external loads of a5 results in w(n5 , a5 ) being augmented by a factor of: 20 (a8 ’s load) + 3 (a7 ’s load) + 2 (a6 ’s load) for a total of 25 and w(n2 , a5 ) being augmented by: 8 (a8 ’s load) + 6 (a7 ’s load) + 8 (a6 ’s load) for a total of 22. Fig. 11.7 illustrates the final graph coloring transformation

11


243

for the example of Fig. 11.5. Again, to avoid cluttering, only edges between n2 , n5 and a3 , a5 are shown.

1 2

15

a2 3

a1

6

2 a3

18

24

n2

0

22

24

7

9 a4 3

15

a5

31

n5

0

2

Fig. 11.7 Resulting problem graph

The adaptations done to include the external (to the node pair) communication load, correctly charge the relevant costs when the agent vertices will be colored. For instance, if a5 is colored with the same color as n5 (black), meaning a5 is placed at n5 , then the external cost of a5 directed towards n5 must be charged without any migration cost, which is what the (n2 , a5 ) link will do. Similarly, if a5 is placed at n2 , i.e., colored red, then both the migration cost and the external cost of the communication load towards a5 located at n2 must be charged, which is achieved through the (n5 , a5 ) link. However, a subtle change must be made to AXA in order for it to function properly. Recall, that AXA selects the link of highest weight and attempts to merge the nodes it involves. The rationale for the decision is to attempt to place together agents communicating heavily with each other. So, if a3 and a5 are placed together, then the communication load among them will be alleviated and a benefit of w(a3 , a5 ) = 24, will occur. However, the same is not true when considering edges involving a vertex representing a system node. For instance, if a5 , n5 are merged the actual benefit will not be 31, but rather the cost difference between placing a5 at n5 and at n2 . As discussed the actual cost will be charged correctly, through the link (n2 , a5 ), nevertheless, AXA would have begun the coloring/merging process from a less beneficial edge. For this reason, at the sorting step of AXA all edges of the form (ni , ak ) do not participate with their weights, but rather with the weight difference: w(ni , ak ) − w(n j , ak ), assuming ni and n j are the system nodes for which AXA runs.

244


11.3.4 Greedy Algorithmic Approach Thus far, we have shown how the APR problem when viewed locally from the standpoint of a node pair can be transformed into a graph coloring problem. We also discussed both an algorithm to derive a solution to the coloring problem (AXA) and how it can be invoked in order to tackle the global APR problem (PRA). For comparison reasons here, we discuss another algorithm to solve APR based on the greedy approach. Starting from the initial placement, Greedy iteratively selects an agent to migrate and performs the migration. Specifically, at each iteration all A ∗ N possible migrations are considered and the one that optimizes (4) the most, subject to the constraints (5)-(8) is selected. The process is repeated until no further beneficial migration can be defined.

11.4 Experiments This section describes the experimental evaluation of PRA. Section 11.4.1 presents the experimental setup. Section 11.4.2 gives a comparison of PRA and Greedy against exhaustive search for a small experiment, while in Section 11.4.3 we compare PRA against Greedy for a larger experimental setup. Finally, Sec. 11.4.4 summarizes the experimental findings.

11.4.1 Experimental Setup Due to the fact that the POBICOS middleware is currently under development we conducted the experimental evaluation using simulation experiments. The details of the simulation setup which roughly followed the ones in [14, 15] are briefly given below. Network generation. Two types of networks were constructed, one with 7 and one with 30 nodes. In both networks an extra node played the role of the monitor-ing node. Nodes were placed randomly in a 100 × 100 2D plain and assumed to be in range of each other if their Euclidean distance was less than 30 distance units. In the resulting network topology graph, a spanning tree was calculated and acted as the corresponding tree-based routing topology. Application generation. The application tree structure is generated randomly, based on the (given) number of non-generic agents. The initial non-generic agents are split in disjoint groups of 5, and for each group 2-5 agents are randomly chosen as children of a new generic agent. In next iterations, orphan (generic and non-generic) agents are (again) randomly split in groups of 5 and the process of parent creation is repeated, until a single agent remains which becomes the root of the application. With the above method which was used in [15] the resulting application is a tree,

11


245

its leaves consisting of non-generic agents. Since the scope of this work is broader tackling general application graphs as opposed to trees, we alter the resulting application tree as follows. For each generic agent two more non-generic agents were assumed to be its children, thus, these non-generic agents had two (or more) parents. Two different application structures were generated with this way app-10 and app-40, each with 10 and 40 non-generic agents respectively. Application traffic. We assumed that the communication load between a non-generic and a generic agent was between 10 to 50 data units per time unit. For the load between generic agents we considered three cases: (i) lavg: a generic agent sends the average of the load received from its children, corresponding to a data aggregation scenario; (ii) lsum: a generic agent sends to its parent the sum of the loads received from its children, corresponding to a forwarding scenario; and (iii) lmix: half of the generic agents (randomly chosen) generate load according to lavg and the other half according to lsum. Unless otherwise stated, the constant α (see (4)) governing the importance of migration cost versus communication load was set to 0.01. Other parameters. The size of agents varied uniformly between 100 and 1,000 data units. All the non-generic agents that have the same parent were assumed to share one common functionality resource requirement and had a mutual exclusion constraint among them. Non-generic agents with different parents were assumed to differ in at least one functionality requirement. In the experiments we begin with an initial placement and run the algorithms to define a better one. This initial placement is derived by placing the non generic agents first. Specifically for every group of non-generic agents with the same parent (let ng in cardinality), (1 + β )ng nodes (randomly selected) were assumed to have adequate functionality resources to hold the agents, i.e., for a node ni and an agent ak such as above, Lik = 1. Unless otherwise stated, constant β takes a value of 0.5. In the initial placement the non-generic agents were placed randomly to nodes having the required functionality in such a way so as to respect mutual exclusion constraints as well. Having placed the nongeneric agents, generic agents were placed afterwards, again in a random fashion. Last, in the experiments we assume that the computational resource of interest is memory and that all nodes start with an initial capacity equaling the size of the agents assigned to them by the initial placement.

11.4.2 Comparison against the Optimal In this set of experiments we compare both PRA and Greedy against the optimal solution derived through exhaustive search. For this reason we used the smaller 7node network type and app-10 application. Five different network topologies were generated and five different app-10 applications. Results depict the average of the combined runs (25 in total). First we recorded the performance of the algorithms regarding the quality of the placement scheme they reach, as a percentage of the optimal performance. Assuming that in the initial placement init communication load is incurred per time unit,

246


that in the optimal scheme opt communication load is incurred and that in the placement calculated by the algorithms alg communication load is incurred, the percentage of the optimal performance achieved by an algorithm is characterized by the ratio: ( init-alg )/( init-opt), i.e., how much load reduction an algorithm achieves compared to the optimal. Table 11.2 presents the results for PRA and Greedy for two different load types: lavg and lsum. We also varied the amount of extra free capacity available at the system nodes. So for instance lavg(2), means that each node had just enough capacity to hold the agents allocated there in the initial placement plus extra space equaling 2 times the average agent size. Table 11.2 Solution quality compared to the optimal lavg(1) lavg(2) lavg(3) lsum(1) lsum(2) lsum(3) Greedy 81.7% 88.4% 95.4% 86.8% 86.9% 86.9% PRA 85.5% 100% 100% 89.8% 100% 100%

We can observe from Table 11.2 that PRA constantly outperforms the simpler Greedy algorithm. In fact, the difference between PRA and the optimal scheme is not large when capacity is tight (plus one extra space for an agent), while with a less tight constraint, PRA achieves the optimal performance. It is also worth noting that the Greedy algorithm never achieves an optimal performance.

Table 11.3 Migrations performed

Greedy PRA

lavg(1) lavg(2) lavg(3) lsum(1) lsum(2) lsum(3) 9.2 9.4 10.4 8.7 10.1 10.1 8.8 10.0 10.0 9.2 9.5 9.5

We also recorded in Table 11.3 the number of migrations performed by each of the algorithms. Recall, that the application type used was app-10, involving 10 nongeneric agents and roughly 6 generic, for a total of 16 agents. Results here were mixed with PRA doing more or less migrations compared to Greedy depending on the scenario. However, the fact that in certain cases where PRA achieves the optimal, e.g., lavg(3), lsum(3), PRA also performs less migrations compared to Greedy, illustrates even more the merits of our approach.

11.4.3 Experiments with a Larger Network Here we conducted experiments using the larger network case (30 nodes + the monitor). Five different network topologies were generated and each experiment depicts the average. Eight applications of type app-40 were assumed to be initially placed, while the load model was lmix. We plot the percentage of load reduction achieved

11


247

Percentage of communication cost reduction

compared to the initial placement, i.e., (init-alg)/init. Since the exhaustive algorithm could not produce results within acceptable time, we only compared PRA against Greedy.

0,8 0,7 0,6 0,5 0,4

Greedy

0,3

PRA

0,2 0,1 0 1

2

4

8

Surplus capacity

Fig. 11.8 Performance of the algorithms against increased node capacity

Fig. 11.8 demonstrates the performance of the algorithms as more capacity is added at each node e.g., the value of 4 in the x-axis means that each node has capacity equaling the necessary one to hold the agents initially placed there, plus 4 times the average agent size. The first thing to notice, is that the achievable saves by both algorithms increase to the surplus capacity at the nodes which is expected since with tighter capacity agents that should have been placed together might not be able to do so. Notice that PRA manages to reduce the initial load by more than 60% in all cases and by roughly 10% more compared to Greedy, a fact that further reinforces the viability of our approach. Last, in Fig. 11.9 we measure the performance of the algorithms as the functionality constraints become less tight. Recall from Sec. 11.4.1 that each non-generic agent group having the same parent is assumed to require the same functionality resource. Assuming ng is the group size (5 in our case) then (1 + β )ng nodes are assumed to provide such functionality. In the x-axis of Fig. 11.9 we vary the constant β by 50%, 100%, 150% and 200% essentially increasing the number of possible hosts (functionality wise) from 5 to 7.5, 10, 12.5 and 15. As expected with more candidate locations available for each agent, there is an increased optimization potential compared to the random initial placement. Both PRA and Greedy exploit this potential resulting in a performance increase (PRA achieves roughly 80% savings by the end of the plot). Again PRA outperforms Greedy with their difference becoming small in the 150% and 200% case. In a sense, this result means that as the nodes of the

248


Percentage of communication cost reduction

0,9 0,8 0,7 0,6 0,5 0,4

Greedy

0,3

PRA

0,2 0,1 0 50%

100%

150%

200%

Functionality constraint

Fig. 11.9 Performance of the algorithms against looser functionality constraints

system become more homogeneous, Greedy might be a viable alternative, whereas for heterogeneous networks PRA is a clear winner.

11.4.4 Discussion Summarizing the experiments we can state the following: (i) judging from the optimization margin left by the initial placement, any random solution to APR will probably be particularly inefficient; (ii) PRA achieves performance close to optimal particularly if the computational capacity constraint is not very tight; and (iii) simpler algorithms based on a pure greedy paradigm cannot achieve equivalent performance compared to PRA particularly, in networks with a heterogeneity degree as is usually the case in a smart home environment. Before proceeding with the conclusions we would like to mention that the increased performance offered by PRA does not involve a prohibitive runtime cost. All the experiments were run in an ordinary laptop carrying an Intel Pentium Dual CPU T3200 processor at 2GHz with 3GB of memory. Even in the larger setup of Sec. 11.4.3 the running time of PRA never exceeded a couple of seconds.

11.5 Conclusions In this work we tackled the APR problem by iteratively solving it for node pairs. To do so we illustrated a graph coloring problem transformation and proposed an algorithm (AXA) to derive a solution for the equivalent problem. Through simulation experiments the final algorithmic scheme (PRA) was found to outperform a simpler greedy approach, while achieving the optimal solution in many cases. Due

11


249

to the targeted environment, i.e., smart homes, we considered the case of centralized execution. Nevertheless, our core contribution (AXA) is distributed in nature involving only a node pair. As part of our future work we plan to investigate adaptations to the centralized pairing mechanism (PRA) that will allow the algorithm to execute in a fully distributed manner.

Acknowledgment This work is funded by the 7th Framework Program of the European Community, project POBICOS - Platform for Opportunistic Behavior in Incompletely Specified, Heterogeneous Object Communities, contract nr. FP7-ICT-223984. Nikos Tziritas is partially supported by the Alexander S. Onassis Public Benefit Foundation in Greece.

References 1. Boulis, A., Han, C.-C., Shea, R., Srivastava, M.B.: Sensorware: Programming sensor networks beyond code update and querying. Pervasive and Mobile Computing Journal 3(4) (2007) 2. Domaszewicz, J., Roj, M., Pruszkowski, A., Golanski, M., Kacperski, K.: Rovers: Pervasive computing platform for heterogeneous sensor-actuator networks. In: Proceedings of WoWMoM (2006) 3. Fok, C.L., Roman, G.C., Lu, C.: Rapid development and flexible deployment of adaptive wireless sensor network applications. In: Proceedings of ICDCS 2005 (2005) 4. Kang, P., Borcea, C., Xu, G., Saxena, A., Kremer, U., Iftode, L.: Smart messages: A distributed computing platform for networks of embedded systems. The Computer Journal 47(4) (2004) 5. Kellerer, H., Pferschy, U., Pisinger, D.: Knapsack Problems. Springer, Heidelberg (2004) 6. Kothari, N., Gummadi, R., Millstein, T., Govindan, R.: Reliable and efficient programming abstractions for wireless sensor networks. In: Proceedings of PLDI 2007 (2007) 7. Laoutaris, N., Smaragdakis, G., Oikonomou, K., Stavrakakis, I., Bestavros, A.: Distributed placement of service facilities in large-scale networks. In: Proceedings of INFOCOM 2007 (2007) 8. Levis, P., Culler, D.: Mate: A tiny virtual machine for sensor networks. In. Proceedings of ASPLOS 2002 (2002) 9. Liu, H., Roeder, T., Walsh, K., Barr, R., Sirer, E.G.: Design and implementation of a single system image operating system for ad hoc networks. In: Proceedings of MOBISYS 2005 (2005) 10. Mottola, L., Picco, G.P.: Programming wireless sensor networks: Fundamental concepts and state of the art. To appear in ACM Computing Surveys 11. Ramachandran, U., Kumar, R., Wolenetz, M., Cooper, B., Agarwalla, B., Shin, J., Hutto, P., Paul, A.: Dynamic data fusion for future sensor networks. ACM Transactions on Sensor Networks 2(3) (2006) 12. Srivastava, U., Munagala, K., Widom, J.: Operator placement for in-network stream query processing. In: Proceedings of PODS 2005 (2005)

250


13. STREP/FP7-ICT: Platform for opportunistic behaviour in incompletely specified, heterogeneous object communities (pobicos), http://www.ict-pobicos.eu/index.htm 14. Tziritas, N., Loukopoulos, T., Lalis, S., Lampsas, P.: Agent placement in wireless embedded systems: Memory space and energy optimizations. In: Proceedings of IPDPS 2010, PMEO-UCNS Workshop (2010) 15. Tziritas, N., Loukopoulos, T., Lalis, S., Lampsas, P.: Gral: A grouping algorithm to optimize application placement in wireless embedded systems. In: Proceedings of IPDPS 2011 (2011) 16. Yang, X., Lim, H.B., Özsu, M.T., Tan, K.L.: In-network execution of monitoring queries in sensor networks. In: Proceedings of SIGMOD 2007 (2007) 17. Ying, L., Liu, Z., Towsley, D., Xia, C.H.: Distributed operator placement and data caching in large scale sensor networks. In: Proceedings of INFOCOM 2008 (2008)

Chapter 12

A Game Theoretic Approach to Dynamic Network Formation in Market-Oriented Resource Providing Networks Yutaka Okaie and Tadashi Nakano

Abstract. Game theory provides a set of tools for understanding complex real-world decision making situations. In this chapter, we apply game theory to develop a rational model of a market-oriented service overlay network in which peers trade resources and services through a common currency called energy. In our model, an overlay network is created by a set of non-cooperative resource providing peers, called platforms, that modify local connections to other platforms to maximize their own energy gains. Resource consuming peers, called agents, are simply designed to migrate platform-to-platform to find least expensive resources in the network. Computer simulations as well as theoretical analysis are conducted to study network structures that emerge from local interactions among a group of platforms and agents.

12.1 Introduction This chapter considers market-oriented service overlay networks in which participating peers provide computational resources (e.g., CPU cycles and storage) to other peers who use the resources to distribute services or applications over the Yutaka Okaie Frontier Research Center, Graduate School of Engineering, Osaka University, 2-1 Yamadaoka, Suita, Osaka 565-0871, Japan e-mail: [email protected] Tadashi Nakano Frontier Research Base for Global Young Researchers, Frontier Research Center, Graduate School of Engineering, Osaka University, 2-1 Yamadaoka, Suita, Osaka 565-0871, Japan e-mail: [email protected]


252

Y. Okaie and T. Nakano

Overlay (Logical) Network User Agent Pl tf Platform Service Request Logical Link Physical Ph i l Li Link k End Device

Underlay (Physical) Network Fig. 12.1 A conceptual framework of service overlay networks.

network. The use of resources in such networks is regulated by a market mechanism where resource consuming peers pay energy (e.g., lightweight currency) to resource providing peers. Such market oriented networks or equivalents are found in [4, 5, 6, 10, 16, 17, 22, 23, 24]. Fig. 1 illustrates a conceptual framework of market-oriented service overlay networks. As shown, we consider the three types of peers in overlay networks: service agents, resource platforms, and users. • Service agents (or simply called agents) implement specific network services such as web content delivery [24], multimedia streaming [4], value-added services [13] and some others [14]. Agents use resources provided by platforms in order to provide services to users. Agent also implement non-service related behaviors such as migration, replication, and death [4, 5, 17, 22, 24]. For example, agents may migrate toward users requesting the service that they provide. Agents may also replicate in response to increased service demands. Agents may also die to release resources to others. • Resource platforms (simply called platforms) provide execution environments and supporting facilities for agents [3, 22, 27]. Platforms manage the underlying physical layer including hardware resources such as CPU and memory. Platforms maintain the network connections by establishing logical links to other platforms, through which service agents migrate platform-to-platform. Platforms also provide discovery and routing services for users to find services available on the network.

12

A Game Theoretic Approach to Dynamic Network

Energy

User Agent

Energy gy

253

Services Resources (CPU Memory) (CPU,

Platform

Fig. 12.2 Energy exchange among users, agents, and platforms

• Users represent any entities that use services provided by agents. Users include human users, agents, and platforms. In the market-oriented service overlay networks, peers trade resources and services by exchanging energy (e.g., virtual money, lightweight currency [23]) (Fig. 12.2). For example, a platform provides resources to an agent and the agent pays some energy to the platform. The platform may later use the earned energy to buy services provided by some other agents. Similarly, an agent provides a service to a user and the user in return gives some energy to the agent. The agent may use the energy to use resources provided by platforms. Overlay networks are often constructed by a set of selfish (or rational) individuals [8, 25]; i.e., each individual seeks to maximize its own benefit. In this chapter, we develop a rational model of market oriented service overlay networks in which platforms attempt to maximize their own energy gains by adding or dropping logical links to other platforms, while agents simply migrate platform-to-platform through logical links to find the least expensive resources to minimize the energy consumption. Our model represents a class of non-cooperative network formation games, and the dynamics and structure of overlay networks to be formed are studied through computer simulation and theoretical analysis. The rest of the chapter is organized as follows. In Sec. 12.2, we give an example of the network formation game considered in this chapter. In Sec. 12.3, marketoriented service overlay networks are formulated as a non-cooperative network formation game. In Sec. 12.4, we present simulation studies conducted to identify steady state network topologies that emerge from the network formation game formulated in Sec. 12.2. In Sec. 12.5, exact conditions for steady-state network topologies identified in simulation are analyzed in terms of Nash equilibria . Sec. 12.6 provides a brief review on game theoretic studies of peer-to-peer overlay networks, and Sec. 12.7 discusses our planned future work to conclude this chapter.

254


Fig. 12.3 An example of network formation game

12.2 Network Formation Game Example The players of the network formation game considered in this chapter are the resource-providing platforms who compete for resource-consuming mobile agents. Each platform adds or drops links to other platforms so as to maximize their own benefits. We assume that each platform sells its own computing resources at a specific rate (e.g., some units of energy per day), called a resource price, and obtains energy based on the rate from agents who use the platform’s resources. We also assume that platforms who establish a link maintains the link and pays the link cost. Thus, the utility of a platform is given by (resource price of the platform) × (the number of agents on the platform) − (link cost per link) × (the number of links the platform maintains), where we assume that the link cost is the same for all the links to simplify the discussion in this section. Fig. 12.3 shows an example of network formation game. In this example, there are four platforms, P, Q, R, and S, which are represented by open circles in the figure. Resource prices of platforms are shown as the labels on them, e.g., P’s resource price is 3, Q’s is 6, and so on. At time t, some agents found in the network, each represented by a filled circle near the platform. For instance, 3 agents are staying on P, and 2 agents on S. The platforms at the arrowtail maintain the links, and all link costs are 2 for each link. Assume that at time t + 1, platform P adds a link to platform S(Fig. 12.3 (A)) and an agent on S migrates to P, then, P’s energy gain increases from +5 (3 × 3 − 2 × 2) at time t to +6 (3 × 4 − 2 × 3) at time t + 1. The impact of adding the link on other platforms’ utility is shown in Table 12.1. Assume instead that at time t + 1, P drops a link to Q and no agents migrate in the network, then, P’s energy gain at time increases from +5 (3 × 3 − 2 × 2) at time t to

12


255

Table 12.1 Energy gains in (A) Adding a Link in Fig. 12.3.

PP

PP Time t Platform PPP P +5 (3 × 3 − 2 × 2) Q −2 (6 × 0 − 2 × 1) R +10 (4 × 3 − 2 × 1) S +10 (5 × 2 − 2 × 0)

t +1 +6 (3 × 4 − 2 × 3) −2 (6 × 0 − 2 × 1) +10 (4 × 3 − 2 × 1) +5 (5 × 1 − 2 × 0)

Table 12.2 Energy gains in (B) Dropping a Link in Fig. 12.3.

PP

PP Time t Platform PPP P +5 (3 × 3 − 2 × 2) Q −2 (6 × 0 − 2 × 1) R +10 (4 × 3 − 2 × 1) S +10 (5 × 2 − 2 × 0)

t +1 +7 (3 × 3 − 2 × 1) −2 (6 × 0 − 2 × 1) +10 (4 × 3 − 2 × 1) +10 (5 × 2 − 2 × 0)

+7 (3 × 3 − 2 × 1) at time t + 1. The impact of dropping the link on other platforms’ utility is shown in Table 12.2. The network needs to achieve a certain level of connectivity in order to function as a resource-providing infrastructure. However, this may not be achieved when each platform behaves selfishly so as to maximize its own utility. To understand the formation of the network, the following sections perform modeling, simulation and analysis of the network formation game briefly described in this section.

12.3 The Model An overlay network is modeled as a time-dependent undirected graph G(t) = (N, L(t)) with a time-independent set of nodes N and a set of links L(t). A node i ∈ N represents a platform with its resource price denoted by Ri (t), L(t) represents a set of logical links between platforms, and the set of immediate neighbors on G(t) including i itself denoted by NBi (t). Logical links are used in two ways: (a) each of platforms and agents obtains information through one-hop logical links, (b) each of agents migrates between platforms through logical links. Ci (t) represents a set of agents on i. A collection of sets Ci (t) is pairwise disjoint, i.e., Ci (t) ∩C j (t) = 0/ for any i, j ∈ N with i = j. As described in the rest of this section, our model is time discrete in which at every time step t each of agents and platforms makes some decision. Note in our

256


model that any other aspects of market-oriented service overlay networks described in Section 1 such as users are not considered.

12.3.1 Agents At each time step t, each agent c ∈ Ci (t) (i ∈ N) migrates to j ∈ NBi (t) based on the following migration probability pi, j (t) =

R−j α (t)

(12.1)

α ∑k∈NBi (t) R− k (t)

where α represents the agent’s sensitivity to resource prices (α ≥ 0). Note that larger α migrates c in a more deterministic manner toward the least expensive platform in NBi (t) and smaller α in a more randomized manner, which prevents the agents from being trapped in a local optimum in a dynamic environment. It takes one unit time for c to migrate from one platform to another. Thus, if c migrates from i to j at time t, Ci (t + 1) = Ci (t) \ {c}, C j (t + 1) = Ci (t) ∪ {c}.

(12.2) (12.3)

With (12.1), the number of agents on i after a unit time can be written by |Ci (t + 1)| = ∑ p j,i (t) × C j (t) .

(12.4)

j∈NBi (t)

12.3.2 Platforms At each time step t, each platform i ∈ N earns the amount of energy based on the resource price, the number of agents, and the number of links; i.e., Ei (t) = Ei (t − 1) + Ri(t) × |Ci (t)| −

∑

βi, j ,

(12.5)

j∈ fi (NBi (t))

where fi (NBi (t)) is a set of i’s neighbors to which link i maintains and βi, j is the cost of maintaining a link (i, j). Note that the platform who establishes a link maintains the link and thus it pays for the link cost. Also, at each time step t, i updates the set of neighbors NBi by adding and/or dropping links, to maximize the estimated energy gains after a unit time. This is formulated as an optimization problem to find NBi that maximizes a utility function as follows. Maximize Ui (t, NBi ) = Ui (t, NBi , NB−i (t)) = Ri × Cî (t + 1, NBi ) − ∑ j∈ fi (NBi ) βi, j

(12.6)

12


257

whereNB−i (t) is a combination of immediate neighbors of all platforms other than i, and Cî (t + 1, NBi ) is the estimated number of agents on i at time t + 1. Here Eq. (12.6) can be further expanded with Cî (t + 1, NBi ) = ∑ pˆ j,i (t, NBi ) × C j (t) (12.7) j∈NBi

where pˆ j,i (t, NBi ) is the estimated migration probability that agents on j at time t migrate to i ( j ∈ NBi , i ∈ NB j (t)), and is given by pˆ j,i (t, NBi ) =

Ri −α . ∑k∈NB j (t) Rk −α

(12.8)

With Eqs. (12.7) and (12.8), the optimal solution NBi = NBopt for the problem Eq. (12.6) can be found, for instance, using a local search algorithm, and the i’s immediate neighbors at time t + 1 is set as follows. NBi (t + 1) = NBopt .

(12.9)

It is noticed that platform can precisely estimate the expected number of agents migrating to itself after a unit time.

12.4 Simulation Experiments The non-cooperative network formation game modeled in the previous section is expected to produce a complex behavior due to the dynamic natures of link addition and dropping among selfish platforms. It is generally an open research issue in peerto-peer overlay networks to identify global properties of a network that emerges from local interactions among individual nodes [11]. In this section, we perform computer simulation to understand the dynamics and structure of networks to be formed when platforms play the non-cooperative network formation game described in the previous section.

12.4.1 Simulation Algorithms Algorithm 10 shows the main body of the simulation algorithm. Peers are either platforms or agents. As shown in the algorithm, peers take an action sequentially; a peer modifies the set of neighbors via link addition and dropping if the peer is a platform; a peer may migrate to a neighboring platform if the peer is an agent. When all peers are done taking an action, the result of the simulation step is compiled, and the simulation step is incremented. Algorithm 11 shows the link addition algorithm for platforms. As shown, a peer adds a link to its neighbor’s neighbor if such a link optimally improves its utility value. Algorithm 12 shows the link dropping algorithm for platforms. Similar to Algorithm 11, a peer drops a link to its immediate neighbor

258


Algorithm 10. The Simulation Main Body 1: Initialize simulation step step = 0. 2: loop 3: for all peer in the network do 4: if peer is a platform then 5: Let peer collect energy from agents. 6: Let peer add a link. 7: Let peer drop a link. 8: else 9: Let peer migrate. 10: end if 11: end for 12: Compile the result of current simulation step. 13: Increment simulation step step. 14: end loop

Algorithm 11. Link Addition for i ∈ N 1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12:

Plat f ormToAdd = NULL UtilityMax = UtilityOld for all p such that p ∈ NBk (t) where k ∈ NBi (t), and p ∈ / NBi (t) do Compute UtilityNew with link (i, p). if UtilityNew −UtilityMax > 0 then Plat f ormToAdd = p UtilityMax = UtilityNew end if end for if Plat f ormToAdd! = NULL then Add link (i, Plat f ormToAdd) to G(t + 1). end if

Algorithm 12. Link Dropping for i ∈ N 1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12:

Plat f ormToDrop = NULL UtilityMax = UtilityOld for all p such that p ∈ NBi (t) \ {i} do Compute UtilityNew without link (i, p). if UtilityNew −UtilityMax > 0 then Plat f ormToDrop = p UtilityMax = UtilityNew end if end for if Plat f ormToDrop! = NULL then Drop link (i, Plat f ormToDrop) from G(t + 1). end if

12


259

Algorithm 13. Migration Behavior for c ∈ Ci (t) 1: for all j such that j ∈ NBi (t) do 2: Compute pi, j (t) using R j (t). 3: end for 4: Migrate to j ∈ NBi (t) based on pi, j (t).

to improve the utility value. Algorithm 12.4.1 shows the migration algorithm for agents. The simulation code is written in C#. Pajek [1] is used for graph visualization. The simulation program is executed on DELL Vostro 420 with Intel (R) Core(TM)2 Quad CPU Q8300 2.50 GHz and 2.49 GHz, and 4.0 GByte RAM on Windows 7 Enterprise.

12.4.2 Default Simulation Configurations By default, each simulation is run with either a 4-regular network topology or random network topology with the average node degree of 4, each consisting of 25 platforms (Fig. 12.4). All platforms have the same resource price Ri (t) = R = 50. 100 agents are initially placed per platform, which migrate platform-to-platform during simulation. α = 1.5, and β is varied as described in simulation results. Each simulation is terminated at 100 simulation steps during which any change in the link structure is recorded.

12.4.3 Simulation Results: Simple Scenario We first consider a simple scenario where all link costs are the same (βi, j = β ) and examine the impact of β on resultant network topologies. Notice from Eq. (12.6) that increasing β is equivalent to decreasing R, and thus, the impact of R is also examined explicitly. Figs. 12.5 and 12.6 show representative simulation results obtained from β ∈ {0, 250, 500, 700, 2000, 5000} in which 25 platforms initially form a random network in Fig. 12.5 and a regular network in Fig. 12.6. Tables 12.3 and 12.4 show changes in average node degrees of the networks shown in Figs. 12.5 and 12.6, respectively. Our detailed analysis on link structures reveals that in all cases at t = 100, platforms are either completely isolated or fully connected in a subset of platforms. β = 0 results in a complete graph of 25 platforms, indicating that all platforms tend to add links to be fully connected. β = 250 results in a complete graph consisting of a smaller number of platforms (14 in Fig. 12.5 and 17 in Fig. 12.6), while the remaining platforms are completely isolated. When β is further increased, the number of platforms constituting a complete graph decreases, while that of isolated

260


Fig. 12.4 Initial topologies used in simulation. (A) Regular and (B) random topologies. Table 12.3 Average node degrees of the topologies shown in Fig. 12.5.

@ β t @ @

0 250 500 700 2000 5000

10 18.88 7.72 5.60 6.16 4.16 0.00 50 24.00 2.80 3.60 2.24 0.48 0.00 100 24.00 7.28 3.60 2.24 0.48 0.00

platforms increases. When β = 5000, all platforms are completely isolated, resulting in an edgeless topology. In this manner, the average platform degree decreases as β increases, as shown in Tables 12.3 and 12.4. These simulation results indicate that fully connected topologies and edgeless topologies become stable depending on β . The simulation results also indicate that edgeless or fully connected topologies are more stable than the initial network topologies, i.e., random and regular topologies, as the initial topologies are transformed into either edgeless or fully connected topologies for all β values tested. It is also found that the steady-state topologies are achieved at relatively early stages of simulation (within 30 time steps) after which no link addition and dropping occur, except for one case with β = 250. When β = 250 in Fig. 12.5, a characteristic topology, called the hub topology (Fig. 12.7), is found at t = 17, which is maintained till t = 50. An extra set of simulations was run to identify that this hub topology often exhibits two layers of hierarchy; the upper layer platforms form a complete graph, the lower layer platforms maintain links to all the upper layer platforms, and no links are formed among lower layer platforms. These simulation results indicate that hub topologies may also achieve stable topologies depending on β .

12


E

261

0

E

250

E

500

E

700

E

2000

E

5000

t 10

t

50

t 100

t 10

t

50

t 100

Fig. 12.5 Evolution of network topologies started from a random topology at t = 0. Table 12.4 Average node degrees of the topologies shown in Fig. 12.6.

@ β t @ @

0

250 500 700 2000 5000

10 19.60 9.36 7.60 7.12 4.40 0.00 50 24.00 5.36 2.88 1.68 0.48 0.00 100 24.00 10.88 2.88 1.68 0.48 0.00

262


E

0

E

250

E

500

E

700

E

2000

E

5000

t 10

t

50

t 100

t 10

t

50

t 100

Fig. 12.6 Evolution of network topologies started from a regular topology at t = 0.

12.4.4 Simulation Results: Realistic Scenario Next we consider a more realistic scenario where link costs vary depending on the links. In this simulation, we use the following probability distributions to determine link costs and examine the impact of link costs on resultant network topologies. (Note that all distributions have the same average link cost of 2000.) (a) zero variance (as a comparison) (b) uniform distribution U (0, 4000) (c) exponential distribution with parameter λ = 1/2000 (d) normal distribution N(2000, 409600)

12


263

Fig. 12.7 An example hub topology found in simulation where each link is maintained by the platform that has the arrowtail. This topology consists of two groups of platforms: hub and child. There are 3 hubs and 8 children, and there exist links between each pair of child and hub, which are maintained by the child, links between each pair of hubs (i.e., links among hubs form an 3-clique), and no link among children.

Fig. 12.8 shows the sample distributions of link costs generated from the above four distributions, and Fig. 12.9 the degree distributions of resultant network topologies as representative simulation results, which can be summarized as follows. (a) Most platforms are completely isolated while remaining ones are fully connected as observed in the simple scenario. (b) Compared to (a), (b) generates less expensive links as well as more expensive links. Less expensive links can be maintained while more expensive links are not. As a result, platforms that were completely isolated in (a) are connected to other platforms in (b). It is also observed that a hub like topology emerges where a number of platforms establish a link to one common platform. This topology slightly differs from the hub topology observed in the simple scenario in that it has some links among child platforms. (c) This distribution generates a larger number of inexpensive links, and thus a larger number of links are maintained than (a) and (b). As observed in (b), platforms often form a hub-like topology. (d) Most platforms are completely isolated as in (a). This is because the variance of link costs is not large enough to impact the resultant network topology. We further consider a scenario where resource prices vary depending on the platforms. In this simulation, we use the following probability distributions to assign resource prices and examine the impact of resource prices on resultant network topologies. (Note that all distributions have the same average resource price of 50.) (e) zero variance (as a comparison) (f) uniform distribution U(0, 100) (g) exponential distribution with parameter λ = 1/50 (h) normal distribution N(50, 256) Fig. 12.10 shows the sample distributions of resource prices generated from the above four distributions, and Fig. 12.11 the degree distributions of resultant network topologies as representative simulation results. These simulation results are summarized as follows.

264


Fig. 12.8 Distributions of link costs (a) with zero variance, (b) generated from uniform distribution U(0, 4000), (c) exponential distribution (λ = 1/2000), and (d) normal distribution N(2000, 409600).

(e) All platforms are fully connected as observed in the simple scenario. (f) Compared to (e), (f) generates less expensive platforms as well as more expensive platforms. Less expensive platforms do not establish links to others, since the cost of maintaining a link is not paid by the agents that use the link to migrate to the platforms. Thus, some of the platforms that were fully connected in (e) are completely isolated in (f). Remaining platforms tend to establish links to a very expensive platform since agents are expected to migrate from the expensive platform to others. As a result, a hub topology emerges as a steady-state topology. (g) This distribution generates a larger number of inexpensive platforms than (e) and (f). These platforms do not establish links to others, resulting in an edgeless topology. (h) All platforms are fully connected as in (e). This is because the variance of resource prices is not large enough to impact the resultant network topology. To summarize, under the realistic scenario, fully connected or edgeless topologies are occasionally observed as in the simple scenario. In addition, hub topologies are identified as a stable topology, and/or a hub platform often emerges.

12.5 Theoretical Analysis The simulation experiments described in the previous section indicated that edgeless and fully connected topologies are more stable than regular or random topologies. The simulation experiments also identified that hub topologies may achieve stable

12


265

Fig. 12.9 Resultant degree distributions and network topologies obtained from link costs following (a), (b), (c) and (d) in Fig. 12.8, respectively. In (b), the highest degree platform (hub) is shown at the center, and the hub is connected to all other platforms except for the completely isolated platforms. In (c), the highest degree platform (hub) is also shown at the center, and the hub is connected to all other platforms other than the three platforms indicated by arrows.

topologies. In this chapter, we theoretically analyze and examine exact conditions for each of edgeless, fully connected, regular and hub topologies to achieve Nash equilibrium . Since it is difficult to characterize random topologies in general, such conditions for random topologies are not investigated in this chapter.

266


Fig. 12.10 Distributions of resource prices (e) with zero variance, (f) generated from uniform distribution U(0, 100), (g) exponential distribution (λ = 1/50), and (h) normal distribution N(50, 256).

A network topology achieves Nash equilibrium if no platform benefits from adding a link or dropping a link. The condition that no platform benefits from adding a link can be described as ∀(i, j) ∈ L : Δ Ui/add = Ui − Ui/add = Ri × Cieq − Cieq /add + βi, j > 0,

(12.10)

where Ui and Ui/add represent platform i’s utility in the topology and that estimated after i adds a link, and Ci and Ci /add the number of agents on i in the original eq

eq

topology and that estimated after i adds a link. The condition that no platform benefits from dropping a link can be described as ∀(i, j) ∈ L

| i maintains : Δ Ui/drop = Ui − Ui/drop = Ri × Cieq − Cieq /drop − βi, j > 0

(12.11)

Δ U j/drop = U j − U j/drop = R j × C jeq − C jeq /drop > 0

(12.12)

and

12


267

Fig. 12.11 Resultant degree distributions and network topologies obtained from resource prices following (e), (f), (g) and (h) in Fig. 12.10, respectively. In (f), the two highest degree platforms (hubs) are shown at the center, and hubs are connected to all other platforms except for the completely isolated platforms.

where L represents the set of links at the equilibrium, Ui and Ui/drop platform i’s utility in the topology and that estimated after i drops link (i, j), Ri and R j resource prices on i and j at the equilibrium, Ci and Ci /drop the number of agents on i eq

eq

in the topology and that estimated after i drops link (i, j), U j and U j/drop platform j’s utility in the topology and that estimated after j drops link (i, j), C jeq and

268


j

j

Add link

i

(i , j )

i

j

j

Drop link

i

A1

(i , j )

A2

i

A3

Fig. 12.12 The numbers of agentson i at steady-state ( Cieq and that after link (i, j) is added (Cieq /add ) or dropped (Cieq /drop). Labeled circles and undirected lines represent platforms corresponding to the labels and links between platforms. Arrows represent agent migrations, which are classified into three types, i.e., A1 , A2 , and A3 , according to the number of agents |Cieq | |Cieq | involved. In adding link (i, j), A1 represents the migration of di +1 agents, A2 that of di +2 |C jeq | agents, and A3 that of d j +2 agents. In dropping link (i, j), A1 represents the migration of |Cieq | |Cieq | |C jeq | di +1 agents, A2 that of di agents, and A3 that of d j agents.

C jeq /drop the number of agents on j in the topology and that estimated after j drops link (i, j). To simplify the analysis, we assume that resource prices are R (constant) for all platforms and link costs are β (constant) for all links. We also assume that the number of agents in Nash equilibrium is at steady state; for any link (i, j) ∈ L, the number of agents migrating from i to j and that from j to i are the same, i.e., Ci C j eq eq ∀i, j ∈ L : = . (12.13) di + 1 d j + 1 where di and d j are the degrees of platform i and j. For edgeless topologies (for all |C| i ∈ N, di = 0), we assume that ∀i ∈ N : Cieq = |N| .

12


269

With these assumptions, the differences in numbers of agents in Eqs. (12.10), (12.11), and (12.12) can be written as follows (see Fig. 12.12). C Ci C j eq eq Ci − Ci /add = ieq − + eq eq di + 1 di + 2 d j + 2 Ci C j eq eq = − (di + 1)(di + 2) d j + 2 2 Cieq Cieq Ci − Ci /drop = − eq eq di + 1 di di − 1 = Ci di (di + 1) eq 2 C jeq C jeq C j − C j /drop = − eq eq dj +1 dj dj −1 = C jeq d j (d j + 1)

(12.14)

(12.15)

(12.16)

Thus, Eq. (12.10), (12.11), and (12.12) can be described as ∀i, j ∈ N, (i, j) ∈ /L: Ci C j eq eq Δ Ui/add = − R+β > 0 (di + 1)(di + 2) d j + 2

(12.17)

and ∀(i, j) ∈ L | i maintains : di − 1 Δ Ui/drop = Ci R − β > 0 (12.18) di (di + 1) eq and d j − 1 Δ U j/drop = C jeq R > 0. (12.19) d j (d j + 1) In the special case where di = d j and Cieq = C jeq , Δ U j/drop ≥ Δ Ui/drop , meaning that when Eq. (12.18) is met, Eq. (12.19) is also met. Thus, in this case, we need to consider Eq. (12.18) only. This applies to fully connected and regular topologies.

12.5.1 Edgeless Topologies An edgeless topology consists of a set of platforms with no link among them. An edgeless topology achieves Nash equilibrium if no platform benefits from adding a

270


E

N |C| Fig. 12.13 With |N| R = 5000, the equilibrium region of |N| (positive integers) and β for edgeless topologies. The boundaries are not included except for |N| = 1.

link; i.e., Eq. (12.17) is met. Since di = d j = 0 and Cieq = C jeq , the condition for Nash equilibrium is given by

β > 0.

(12.20)

|C| With |N| R = 5000, the equilibrium region of |N| and β is depicted in Fig. 12.13.

12.5.2 Fully Connected Topologies A fully connected topology consists of a set of platforms where any pair of platforms is connected with a link. A fully connected topology achieves Nash equilibrium if no platform benefits from dropping a link; i.e., Eq. (12.18) is met. Since di = |N| − 1 |C| and Cieq = |N| , the condition for Nash equilibrium is given by |N| − 2 |C| R > β. |N| (|N| − 1) |N| With

|C| |N| R

(12.21)

= 5000, the equilibrium region of |N| and β is depicted in Fig 12.14.

12


271

E

N |C| Fig. 12.14 With |N| R = 5000, the equilibrium region of |N| (positive integers) and β for fully connected topologies. The boundaries are not included.

12.5.3 d-Regular Topologies A d-regular topology consists of platforms where each platform has the same degree d (0 < d < |N| − 1) and where platforms form a single connected component. A d-regular topology achieves Nash equilibrium if no platform benefits from either adding a link or dropping a link; i.e., Eqs. (12.17) and (12.18) are met. |C| Since di = d j = d and Cieq = C jeq = |N| , the condition for d-regular topologies to achieve Nash equilibrium is given by d |C| d − 1 |C| R 0. d(d + 1)(d + 2) |N|

(12.23)

Therefore, d > 2, which means that d = 2 (ring topologies) does not achieve Nash equilibrium, but weak Nash equilibrium. It also means that the degree must be more than 2 to achieve Nash equilibrium.

272


E

d |C|

Fig. 12.15 With |N| = 25 and |N| R = 5000, the equilibrium region of d (positive integers) and β for d-regular topologies. The boundaries are not included. |C| With |N| = 25 and |N| R = 5000, the equilibrium region of d and β is depicted in Fig 12.15. As shown, the conditions for regular topologies to achieve Nash equilibrium is more strict than those for edgeless and fully connected topologies (Fig. 12.13 and 12.14).

12.5.4 Hub Topologies A hub topology consists of two groups of platforms: hub and child (Fig. 12.7). There are h hubs and |N| − h children. A hub topology achieves Nash equilibrium if no child benefits from adding a link to a child or dropping the link to a hub, and if no hub benefits from dropping a link to a child or a hub; i.e., (A) Eq. (12.17) is met for children i and j, (B) Eqs. (12.18) and (12.19) are met for child i and hub j, and (C) Eqs. (12.18) and (12.19) are met for hubs i and j. The steady-state number of agents on each hub (|Chub |) and that on each child (|Chub |) satisfies the following equation from Eq. (12.13): |Chub | |Cchild | = . |N| h+1

(12.24)

The total number of agents is |C| = ∑i∈N |Ci | since a collection of sets Ci (i ∈ N) is pairwise disjoint. Thus, we have |C| =

∑ |Ci |

i∈N

= h × |Chub | + (|N| − h) × |Cchild | .

(12.25)

12


273

With Eqs. (12.24) and (12.25), the steady-state numbers of agents on hub and child are |C| |Chub | |N| = . (12.26) |Cchild | (2h + 1) |N| − h(h + 1) h + 1 (A) The condition for satisfying Eq. (12.17) for children i and j is, since di = d j = d and Cieq = C jeq = |Cchild |, given by

β>

|C| R h . (2h + 1) |N| − h(h + 1) h + 2

(12.27)

(B) The condition for satisfying Eqs. (12.18) and (12.19) for for child i and hub j is, since di = d, d j = |N| − 1, Cieq = |Cchild |, and C jeq = |Chub |, given by |C| R h−1 >β (2h + 1) |N| − h(h + 1) h

(12.28)

|C| R |N| − 2 > 0. (2h + 1) |N| − h(h + 1) |N| − 1

(12.29)

and

(C) The condition for satisfying Eqs. (12.18) and (12.19) for hubs i and j is, di = d j = |N| − 1 and Cieq = C jeq = |Chub |, given by |C| R |N| − 2 > β. (2h + 1) |N| − h(h + 1) |N| − 1

(12.30)

With Eqs. (12.27), (12.28), (12.29), and (12.30), the conditions for hub topologies to achieve Nash equilibrium is h (2h + 1) |N| − h(h + 1) h−1 < β< . h+2 |C| R h

(12.31)

For such β to exist, the right hand side needs to be greater than the left hand side in Eq. (12.31), i.e., h−1 h h−2 − = > 0. h h + 2 h(h + 2)

(12.32)

Therefore, h > 2, which means that h = 1 (star topologies) never achieves Nash equilibrium. It also means that the number of hubs must be more than 2 to achieve Nash equilibrium. |C| With |N| = 25 and |N| R = 5000, the equilibrium region of h and β is depicted in Fig 12.16. As shown, the conditions for hub topologies to achieve Nash equilibrium is more strict than those for edgeless and fully connected topologies (Fig. 12.13 and 12.14).

274


E

h |C|

Fig. 12.16 With |N| = 25 and |N| R = 5000, the equilibrium region of h (positive integers) and β for hub topologies. The boundaries are not included. Table 12.5 Conditions satisfied at Nash equilibria in network formation games. Condition Degree

Number of Agents |C| |N| |C| |N| |C| |N|

(A)

∀i ∈ N : di = 0

∀i ∈ N : |Ci | =

(B)

∀i ∈ N : di = |N| − 1

∀i ∈ N : |Ci | =

(C)

∀i ∈ N : di = d ∀i ∈ N : |Ci | = dhub |N| − 1 |Chub | |N| |C| = = (2h+1)|N|−h(h+1) dchild h |Cchild | h+1

(D)

Link Cost β

β >0 |N|−2 |C| |N|(|N|−1) |N| R > β |C| d (d+1)(d+2) |N| R < β h h+2

Intelligent Decision Systems in Large-Scale Distributed Environments

Designing Distributed Environments with Intelligent Software Agents

Decision Support Systems and Intelligent Systems 7th

Intelligent Environments

Distributed intelligent systems: a coordination perspective

Distributed Intelligent Systems: A Coordination Perspective

Decision-making in Complex Environments

Next Generation Intelligent Environments: Ambient Adaptive Systems

Distributed Intelligent Systems: A Coordination Perspective

Distributed Intelligent Systems: A Coordination Perspective

Advanced intelligent environments

Designing Distributed Learning Environments with Intelligent Software Agents

Distributed Systems -An Algorithmic Approach Distributed Systems

Intelligent Systems Modeling And Decision Support in Bioengineering

Distributed Systems

Intelligent Systems

Intelligent Systems

Fault Tolerance in Distributed Systems

Distributed Decision Making and Control

Advances in Intelligent Information Systems

Innovations in hybrid intelligent systems

Intelligent Distributed Video Surveillance Systems (Professional Applications of Computing)

Advances In Decision Support Systems

Distributed Database Systems

Distributed Energy Systems

Distributed Operating Systems

Distributed Event-Based Systems

Distributed Database Systems

Distributed Operating Systems

Fuzzy Reasoning in Information, Decision and Control Systems (Intelligent Systems, Control and Automation: Science and Engineering)

Intelligent Decision Systems in Large-Scale Distributed Environments

Designing Distributed Environments with Intelligent Software Agents

Decision Support Systems and Intelligent Systems 7th

Intelligent Environments

Distributed intelligent systems: a coordination perspective

Distributed Intelligent Systems: A Coordination Perspective

Decision-making in Complex Environments

Next Generation Intelligent Environments: Ambient Adaptive Systems

Distributed Intelligent Systems: A Coordination Perspective

Distributed Intelligent Systems: A Coordination Perspective

Advanced intelligent environments

Designing Distributed Learning Environments with Intelligent Software Agents

Distributed Systems -An Algorithmic Approach Distributed Systems

Intelligent Systems Modeling And Decision Support in Bioengineering

Distributed Systems

Intelligent Systems

Intelligent Systems

Fault Tolerance in Distributed Systems

Distributed Decision Making and Control

Advances in Intelligent Information Systems

Innovations in hybrid intelligent systems

Intelligent Distributed Video Surveillance Systems (Professional Applications of Computing)

Advances In Decision Support Systems

Distributed Database Systems

Distributed Energy Systems

Distributed Operating Systems

Distributed Event-Based Systems

Distributed Database Systems

Distributed Operating Systems

Fuzzy Reasoning in Information, Decision and Control Systems (Intelligent Systems, Control and Automation: Science and Engineering)

Recommend Documents