Recent Advances in Artificial Life: Sydney, Australia 5 - 8 December 2005

Recent Advances in Artificial Life ADVANCES IN NATURAL COMPUTATtON Series Editor: Xin Yao (University of Birmingham,...

Author: H. A. Abbass; et al

10 downloads 391 Views 19MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

Recent Advances in Artificial Life

ADVANCES IN NATURAL COMPUTATtON Series Editor:

Xin Yao (University of Birmingham, UK)

Assoc, Editors: Hans-Paul Schwefel (University of Dortmund, Germany) Byoung-Tak Zhang (Seoul National University, South Korea) Martyn Amos (University of Liverpool, UK)

Published

Vol. 1:

Applications of Multi-Objective Evolutionary Algorithms Eds: Carlos A. Coello Coello (CINVESTAV-IPN, Mexico) and Gary B. Lamont (Air Force Institute of Technology, USA)

Vol. 2:

Recent Advances in Simulated Evolution and Learning Eds: Kay Chen Tan (National University of Singapore, Singapore), Meng Hiot Lim (Nanyang Technological University, Singapore), Xin Yao (University of Birmingham, UK) and Lip0 Wang (Nanyang Technological University, Singapore)

Recent Advances in Artificial Life Advances in Natural Computation - Vol. 3 Sydney, Australia

5 - 8 December 2005

editors

H. A. Abbass University of N e w South Wales, Australia

T. Bossomaier Charles Sturt University, Australia

J. Wiles The University of Queensland, Austalia

N E W JERSEY

*

LONDON

u:s

World Scientific - SINGAPORE - B E l J l N G - S H A N G H A I H O N G KONG *

*

TAIPEI

*

CHENNAI

Published by

World Scientific Publishing Co. F?e. Ltd. 5 Toh Tuck Link, Singapore 596224

USA ofice: 27 Warren Street, Suite 401-402, Hackensack, NJ 07601 UK ofice: 57 Shelton Street, Covent Garden, London WCW 9HE

British Library Cataloguing-in-PublicationData A catalogue record for this book is available from the British Library.

RECENT ADVANCES IN ARTIFICIAL LIFE Advances in Natural Computation Vol. 3

-

Copyright Q 2005 by World Scientific Publishing Co. Re. Ltd. All rights reserved. This book, or parts there% may not be reproduced in any form or by any means, electronic or mechanical, including photocopying, recording or any information storage and retrieval system now known or to be invented, without written permissionfrom the Publisher.

For photocopying of material in this volume, please pay a copying fee through the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA. In this case permission to photocopy is not required from the publisher.

ISBN 98 1-256-615-5

Printed in Singapore by World Scientific Printers (S)Pte Ltd

Preface

This book arose from the set of papers that were submitted and accepted at the Australian Conference on Artificial Life (ACAL’05), 5-9 December, 2005. ACAL’05 is the second in a series of Australian artificial life conferences at which the advances in Artificial Life are reported. All papers submitted to the conference were peer reviewed by at least two experts in the relevant area and only those that were accepted are included in this proceedings. ACAL’O5 received 40 papers, out of which 25 papers only were selected to appear in this book. The conference attracted submissions from Australia, China, Japan, New Zealand, the Netherlands, UK, and the USA. The conference hosted a number of events including the Second Workshop on Patterns. ACAL’05 witnessed a number of invited speakers including Prof. David Goldberg, from the University of Illinois, Urbana-Champaign, and Prof. Mark Bedau, from Reed College. We wish to acknowledge the role of the advisory and program committees of the conference. The advisory committee included: Mark Bedau (Reed, USA); Eric Bonabeau (Icosystem, USA); David Fogel (Natural Selection, USA); Peter Stadler (Leipzig, Germany); Masanori Sugisaka (Oita, Japan). The program committee included: Alan Blair (UNSW, Australia); Xiaodong Li (RMIT, Australia); Stephen Chalup (Newcastle, Australia); Xavier Llor (UIUC, USA); Tan Kay Chen (NUS, Singapore); F’rederic Maire (QUT, Australia); Vic Ciesielski (RMIT, Australia); Bob Mckay (UNSWQADFA, Australia); David Cornforth (CSU, Australia); Naoki Mori (Osaka Prefecture University, Japan); Alan Dorine (Monash, Australia); Akira Namatame (Defence Academy, Japan); Daryl Essam (UNSWQADFA, Australia); Chrystopher Nehaniv (Herts, UK); David Green (Monash, Australia); David Newth (CSIRO, Australia); Tim Hendt-

V

vi

Preface

lass (Swinburne, Australia); Stefan0 Nolfi (CNR, Italy); Christian Jacob (Calgary, Canada); Marcus Randall (Bond, Australia); Ray Jarvis (Monash, Australia); Alex Ryan (DSTO, Australia); Graham Kendall (Nottingham, UK); Russell Standish (UNSW, Australia); Kevin Korb (Monash, Australia); Jason Teo (Universiti Malaysia Sabah, Malaysia). The editors also wish to acknowledge the efforts of the enthusiastic team of World Scientific Publishing, who worked hard to get high quality manuscript and to make it available for the authors at the conference. We also wish to thank the series editor, Prof. Xin Yao, for accepting our proposal and making this publication possible. ACAL runs biannually and we look forward to ACAL 2007. We hope that this event benefited all researchers who attended the conference.

Hussein A . Abbass, Terry Bossomaier, and Janet Wiles (Editors)

Contents

Preface

V

1. Recreating Large-Scale Evolutionary Phenomena

1

P.-M. Agapow 1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 A framework for recreating evolution . . . . . . . . . . . 1.2.1 Simulation . . . . . . . . . . . . . . . . . . . . . . . 1.2.2 Data analysis and manipulation . . . . . . . . . . 1.2.3 Practical issues . . . . . . . . . . . . . . . . . . . . 1.3 Life out of balance . . . . . . . . . . . . . . . . . . . . . . 1.3.1 Key innovations . . . . . . . . . . . . . . . . . . . . 1.3.2 Methods. . . . . . . . . . . . . . . . . . . . . . . . 1.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . .

.

.

2 . Neural Evolution for Collision Detection & Resolution in a 2D Free Flight Environment

1 3 3 4 5 5 6 6 9

13

S . Alam. M . McPartland. M . Barlow. P . Lindsay. and H . A . Abbass 2.1 Background . . . . . . . . . . . . 2.2 Modelling the Problem . . . . . . 2.2.1 Collision Detection . . . . 2.3 The Neural Network Structure . 2.4 Preliminary Experimental Setup 2.4.1 Preliminary Results . . . vii

. . . . . .

. . . . . .

. . . . . .

.. .. .. .. .. ..

. . . . . .

. . . . . .

14 15 18 18 20 21

...

Contents

Vlll

2.5 2.6 2.7 2.8

Main Experiment Setup . . . . . . . . . . . . . . . . . . . Fitness Function . . . . . . . . . . . . . . . . . . . . . . . Main Results and Analysis . . . . . . . . . . . . . . . . . . Conclusion & Future Work . . . . . . . . . . . . . . . . .

3. Cooperative Coevolution of Genotype-Phenotype Mappings to Solve Epistatic Optimization Problems

21 23 24 26

29

L . T. Bui. H . A . Abbass, and D . Essam Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . The use of co-evolution for GPM . . . . . . . . . . . . . . The proposed algorithm . . . . . . . . . . . . . . . . . . . A comparative study . . . . . . . . . . . . . . . . . . . . . 3.4.1 Testing scenario . . . . . . . . . . . . . . . . . . . . 3.4.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.3 Fitness landscape analysis . . . . . . . . . . . . . . 3.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . .

3.1 3.2 3.3 3.4

4 . Approaching Perfect Mixing in a Simple Model of the Spread of an Infectious Disease

29 31 33 36 36 37 39 42

43

D . Chu and J . Rowe Introduction . . . . . . . . . . . . . . . . . . . . . . Description of the Model . . . . . . . . . . . . . . . Behavior of the Model in the Perfect Mixing Case . Beyond perfect Mixing . . . . . . . . . . . . . . . . 4.4.1 No Movement: The Static Case . . . . . . . 4.4.2 In Between . . . . . . . . . . . . . . . . . . 4.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . 4.6 Conclusion & Future Work . . . . . . . . . . . . .

4.1 4.2 4.3 4.4

. . . . . . . .

. . . . . . . .

.. .. .. .. .. .. .. ..

5. The Formation of Hierarchical Structures in a PseudoSpatial Co-Evolutionary Artificial Life Environment

43 44 45 47 47 48 49 53

55

D . Cornforth. D . G. Green and J . Awburn 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Themodel . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.1 Genotype to phenotype mapping . . . . . . . . . . 5.2.2 Selection mechanism . . . . . . . . . . . . . . . . .

55 57 57 58

Contents

5.2.3 Reproduction and genetic operators . . . . . . . . 5.2.4 Memetic evolution . . . . . . . . . . . . . . . . . . 5.2.5 Global parameters . . . . . . . . . . . . . . . . . . 5.3 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . 6. Perturbation Analysis: A Complex Systems Pattern

ix

59 60 61 61 62 66 69

N . Geard. K . Willadsen and J . Wiles 6.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 Applicability . . . . . . . . . . . . . . . . . . . . . . . . . 6.3 Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4 Participants . . . . . . . . . . . . . . . . . . . . . . . . . . 6.5 Collaborations . . . . . . . . . . . . . . . . . . . . . . . . 6.6 Consequences . . . . . . . . . . . . . . . . . . . . . . . . . 6.7 Implementation . . . . . . . . . . . . . . . . . . . . . . . . 6.8 Sample code . . . . . . . . . . . . . . . . . . . . . . . . . . 6.9 Knownuses . . . . . . . . . . . . . . . . . . . . . . . . . . 6.10 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.11 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 7. A Simple Genetic Algorithm for Studies of Mendelian Populations

70 71 71 71 73 74 75 77 81 82 83

85

C. Gondro and J.C.M. Magalhaes 7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2 Genetic Algorithms . . . . . . . . . . . . . . . . . . . . . . 7.2.1 Search operators . . . . . . . . . . . . . . . . . . . 7.3 Conceptual Model of Mendelian Populations . . . . . . . . 7.3.1 Virtual organisms as a simple genetic algorithm . . 7.4 Nardy-Weinberg Equilibrium in a Virtual Population . . . 7.5 Conclusions and Future Work . . . . . . . . . . . . . . . . 8. Roles of Rule-Priority Evolution in Animat Models

86 88 88 89 91 94 97 99

K . A . Hawick, H . A . James and C.J.Scogings 8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2 Rule-Based Model . . . . . . . . . . . . . . . . . . . . . .

99 100

Contents

X

8.3 8.4 8.5

8.6 8.7

8.2.1 Our Predator-Prey Model . . . . . . . . . . . . . . Resultant Behaviours from Prioritisation . . . . . . . . . . Behavioural Metrics and Analysis . . . . . . . . . . . . . . An Evolutionary Survival Experiment . . . . . . . . . . . 8.5.1 Evolution Procedure . . . . . . . . . . . . . . . . . 8.5.2 Survivability . . . . . . . . . . . . . . . . . . . . . Generalising the Approach . . . . . . . . . . . . . . . . . . Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . .

9 . Gauging ALife: Emerging Complex Syst.ems

102 103 109 110 110 112 113 114 117

K . Kitto 9.1 Life and ALife . . . . . . . . . . . . . . . . . . . . . . . . . 9.1.1 Development . . . . . . . . . . . . . . . . . . . . . 9.1.2 ALife and Emergence . . . . . . . . . . . . . . . . . 9.1.3 Complexity and Contextuality . . . . . . . . . . . . 9.2 Incorporating Context into our Models . . . . . . . . . . . 9.2.1 The Baas Definition of Emergence . . . . . . . . . 9.2.2 Quantum Mechanics . . . . . . . . . . . . . . . . . 9.2.3 Gauge Theories . . . . . . . . . . . . . . . . . . . . 9.3 The Recursive Gauge Principle (RGP) . . . . . . . . . . . 9.3.1 Cellular Automata and Solitons . . . . . . . . . . . 9.3.2 BCS Superconductivity and Nambu-Goldstone modes . . . . . . . . . . . . . . . . . . . . . . . . . 9.3.3 Returning to Development . . . . . . . . . . . . . . 10. Localisation of Critical Transition Phenomena in Cellular Automata Rule-Space

117 118 119 121 122 123 124 124 127 129 130 130

131

A . Lafusa and T. Bossomaier 10.1 10.2 10.3 10.4 10.5 10.6

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 131 Automatic classify rules with input-entropy . . . . . . . 132 Parameterisation of cellular automata rule-space . . . . . 134 Experimental determination of the edge-of-chaos . . . . . 134 Definition of a unique critical parameter . . . . . . . . . 136 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . 144

Contents

xi

11. Issues in the Scalability of Gate-Level Morphogenetic Evolvable Hardware

145

J . Lee and J . Sitte 11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Scaling with Morphogenesis . . . . . . . . . . . . . . . . 11.3 Evolving One Bit Full Adders . . . . . . . . . . . . . . . 11.3.1 Experimental Setup . . . . . . . . . . . . . . . . 11.3.2 LUT Encoding . . . . . . . . . . . . . . . . . . . 11.3.3 Fitness Evaluation . . . . . . . . . . . . . . . . . 11.3.4 Experiment Results . . . . . . . . . . . . . . . . 11.3.5 Further Experiments . . . . . . . . . . . . . . . 11.4 Analysing Problem Difficulty . . . . . . . . . . . . . . . . 11.4.1 Experiment Difficulty Comparisons . . . . . . . 11.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . .

12. Phenotype Diversity Objectives for Graph Grammar Evolution

145 146 148 149 150 151 152 153 154 155 158 159

M .H . Luerssen 12.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 12.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . 12.2.1 Evolution and Development . . . . . . . . . . . 12.2.2 Graph Ontogenesis . . . . . . . . . . . . . . . . 12.2.3 Evolving a Graph Grammar . . . . . . . . . . . 12.2.4 Diversity Objectives . . . . . . . . . . . . . . . . 12.3 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . 12.3.1 Measures of Phenotype Diversity . . . . . . . . . 12.3.2 Evaluation . . . . . . . . . . . . . . . . . . . . . 12.3.3 Results and Discussion . . . . . . . . . . . . . . 12.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . 13. An ALife Investigation on the Origins of Dimorphic Parental Investments

159 160 160 161 161 163 165 165 167 168 170

171

S. Mascaro. K .B . Korb and A .E . Nicholson 13.1 Introduction . . . . . . . . . . . . . 13.2 ALife Simulation . . . . . . . . . . 13.3 Prior investment hypothesis . . .

............ 171 ............ 173 . . . . . . . . . . . . . 175

Contents

xii

13.4 13.5 13.6 13.7 13.8

Desertion hypothesis . . . . . . . . . . . . . . . . . . . . Paternal uncertainty hypothesis . . . . . . . . . . . . . Association hypothesis . . . . . . . . . . . . . . . . . . Chance dimorphism hypothesis . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . .

. . .

14. Local Structure and Stability of Model and Real World Ecosystems

177 180 182 183 185

187

D . Newth. and D . Cornforth 14.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 14.2 Ecological stability and patterns of interaction . . . . . . 14.2.1 Community Stability . . . . . . . . . . . . . . . 14.2.2 Local patterns of interaction . . . . . . . . . . . 14.3 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . 14.3.1 Stability properties of motifs . . . . . . . . . . . 14.3.2 Motif frequency . . . . . . . . . . . . . . . . . . 14.3.3 Community food web data . . . . . . . . . . . . 14.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.4.1 Stability properties of motifs . . . . . . . . . . . 14.4.2 Stability and occurrence of three node motifs . . 14.4.3 Stability and occurrence of four node motifs . . 14.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . 14.6 Closing comments . . . . . . . . . . . . . . . . . . . . . . 15. Quantification of Emergent Behaviors Induced by Feedback Resonance of Chaos

188 188 189 190 191 191 192 193 193 193 193 195 196 198

199

A . Patti. M . Lungarella. and Y. Kuniyoshi 15.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 15.2 Model System . . . . . . . . . . . . . . . . . . . . . . . . 15.2.1 Dynamical Systems Approach . . . . . . . . . . 15.2.2 Feedback Resonance . . . . . . . . . . . . . . . . 15.2.3 Coupled Chaotic Field . . . . . . . . . . . . . . 15.3 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.4 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . 15.5 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.5.1 Analysis 1: Body movements . . . . . . . . . . .

199 201 201 201 202 204 204 207 208

Contents

xiii

15.5.2 Analysis 2: Neural coupling . . . . . . . . . . . . 209 15.6 Discussion and Conclusion . . . . . . . . . . . . . . . . . 210 15.7 Acknowledgements . . . . . . . . . . . . . . . . . . . . . 21’3 16. A Dynamic Optimisation Approach for Ant Colony Optimisation Using the Multidimensional Knapsack Problem

215

M . Randall 16.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 16.2 Adapting ACO to Dynamic Problems . . . . . . . . . . . 16.2.1 Overview . . . . . . . . . . . . . . . . . . . . . . 16.2.2 The Solution Deconstruction Process . . . . . . 16.2.2.1 Event Descriptors . . . . . . . . . . . 16.3 Computational Experience . . . . . . . . . . . . . . . . . 16.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . 17. Maintaining Explicit Diversity Within Individual Ant Colonies

215 217 217 218 221 222 225 227

M . Randall Introduction . . . . . . . . . . . . . . . . . . . . . . . . . Ant Colony System . . . . . . . . . . . . . . . . . . . . . Explicit Diversification Strategies for ACO . . . . . . . . Maintaining Intra-Colony Diversity . . . . . . . . . . . . Computational Experience . . . . . . . . . . . . . . . . . 17.5.1 Experimental Design . . . . . . . . . . . . . . . 17.5.2 Implementation Details . . . . . . . . . . . . . . 17.5.3 Problem Instances . . . . . . . . . . . . . . . . . 17.5.4 Results . . . . . . . . . . . . . . . . . . . . . . . 17.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . .

17.1 17.2 17.3 17.4 17.5

227 229 230 232 233 233 234 235 236 237

18. Evolving Gene Regulatory Networks for Cellular Morphogenesis 239

T. Rudge and N . Geard 18.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 18.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . 18.2.1 Leaf Morphogenesis . . . . . . . . . . . . . . . . 18.2.2 Previous Models . . . . . . . . . . . . . . . . . . 18.3 The Simulation Framework . . . . . . . . . . . . . . . . . 18.3.1 The Genetic Component . . . . . . . . . . . . .

239 240 240 241 242 243

xiv

Contents

18.3.2 The Cellular Component . . . 18.3.3 Genotype-Phenotype Coupling 18.3.4 The Evolutionary Component 18.4 Initial Experiments . . . . . . . . . . . . 18.4.1 Method . . . . . . . . . . . . . . 18.4.2 Results . . . . . . . . . . . . . . 18.5 Discussion and Future Directions . . .

.......... .......... .......... ......... ......... ......... ..........

19. Complexity of Networks

244 245 247 247 248 249 251 253

R . K . Standish 253 19.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 19.2 Representation Language . . . . . . . . . . . . . . . . . . 255 19.3 Computing w . . . . . . . . . . . . . . . . . . . . . . . . 257 19.4 Compressed complexity and Offdiagonal complexity . 19.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . .

. . 258

20. A Generalised Technique for Building 2D Structures with Robot Swarms

262

265

R.L. Stewart and R.A. Russell 20.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 20.2 Background Information . . . . . . . . . . . . . . . . . .

265 266

20.3 A New Technique for Creating Spatio-temporal Varying Templates . . . . . . . . . . . . . . . . . . . . . . . . . . 268 20.3.1 Calibration . . . . . . . . . . . . . . . . . . . . . 268 20.3.2 Experimental Procedure . . . . . . . . . . . . . 270 20.3.3 Building a Radial Wall With and Without a Gap . . . . . . . . . . . . . . . . . . . . . . . . 271 20.4 Solving the Generalised 2D Collective Construction Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . 273 20.4.1 Experimental Procedure . . . . . . . . . . . . . 274 20.4.2 Building Structures of Greater Complexity . . . 274 20.5 General Discussion . . . . . . . . . . . . . . . . . . . . . 275 20.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . 277

Contents

21. H-ABC: A Scalable Dynamic Routing Algorithm

xv

279

B . Tatomir and L .J.M. Rothkrantz 21.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 21.2 The Hierarchical Ant Based Control algorithm . . . . . . 21.2.1 Network model . . . . . . . . . . . . . . . . . . . 21.2.2 Local ants . . . . . . . . . . . . . . . . . . . . . 21.2.3 Backward ants . . . . . . . . . . . . . . . . . . . 21.2.4 Exploring ants . . . . . . . . . . . . . . . . . . . 21.2.5 Data packets . . . . . . . . . . . . . . . . . . . . 21.2.6 Flag packets . . . . . . . . . . . . . . . . . . . . 21.3 Simulation environment . . . . . . . . . . . . . . . . . . 21.4 Test and results . . . . . . . . . . . . . . . . . . . . . . . 21.4.1 Low traffic load . . . . . . . . . . . . . . . . . . 21.4.2 High traffic load . . . . . . . . . . . . . . . . . . 21.4.3 Hot spot . . . . . . . . . . . . . . . . . . . . . . 21.4.4 Overhead . . . . . . . . . . . . . . . . . . . . . . 21.5 Conclusions and future work . . . . . . . . . . . . . . . . 22 . Describing DNA Automata Using an Artificial Chemistry Based on Pattern Matching and Recombination

279 281 281 283 284 285 286 286 287 289 290 290 291 292 293

295

T. Watanabe. K . Kobayashi. M . Nakamura. K . Kishi. M . Kazuno and K . Tominaga 22.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 22.2 An Artificial Chemistry for Stacks of Character Strings . 22.2.1 Elements and objects . . . . . . . . . . . . . . . 22.2.2 Patterns . . . . . . . . . . . . . . . . . . . . . . 22.2.3 Recombination rules . . . . . . . . . . . . . . . . 22.2.4 Sources and drains . . . . . . . . . . . . . . . . . 22.2.5 Dynamics . . . . . . . . . . . . . . . . . . . . . . 22.3 Implementation of Finite Automata with DNA . . . . . 22.4 Describing the Implementation with the Artificial Chemistry . . . . . . . . . . . . . . . . . . . . . . . . . . 22.4.1 Describing the automaton with two states . . . . 22.4.2 Describing an automaton with three states . . . 22.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . 22.6 Comparison with Related Works . . . . . . . . . . . . . .

295 296 296 297 297 298 298 298 300 300 303 305 306

xvi

Contents

Concluding Remarks . . . . . . . . . . . . . . . . . . . .

307

23. Towards a Network Pattern Language for Complex Systems

309

22.7

J . Watson. J . Hawkins. D . Bradley. D . Dassanayake. J . Wiles and J . Hanan 23.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 310 311 23.2 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . 23.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313 23.3.1 Development of the Network Diagram pattern . 313 23.3.2 Development of the Synchronous System State Update pattern . . . . . . . . . . . . . . . 314 23.3.3 Development of the Discrete Statespace Trajectory pattern . . . . . . . . . . . . . . . . . 315 23.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . 316 316 23.5 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . 24 . The Evolution of Aging

319

0. G. Woodberry. K . B . Korb an& A . E . Nicholson

24.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 24.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . 24.2.1 Group Selection . . . . . . . . . . . . . . . . . . 24.2.2 Kin Selection . . . . . . . . . . . . . . . . . . . . 24.2.3 Price Equation . . . . . . . . . . . . . . . . . . . 24.2.4 Mitteldorf’s Aging Simulation . . . . . . . . . . 24.3 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . 24.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24.4.1 Simulations Replicating Mitteldorf’s Results . . 24.4.1.1 Without Aging: . . . . . . . . . . . . 24.4.1.2 With Aging: . . . . . . . . . . . . . . 24.4.2 Simulation Without Kin Selection . . . . . . . . 24.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . 25 . Evolving Capability Requirements in WISDOM-I1

320 321 321 322 323 325 326 327 328 328 328 328 332 335

A . Yang. H.A. Abbass. M . Barlow. R . Sarker. and N . Curtis 25.1 Introduction 25.2 WISDOM-I1

....................... .......................

336 337

Contents

25.3 Experimental setup . . . . . . . . . . . . . . . . . . . . . 25.3.1 Chromosome representation . . . . . . . . . . . 25.3.2 Objectives and evolutionary computation setup 25.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25.5 Conclusion and future work . . . . . . . . . . . . . . . .

xvii

338 340 341 344 347

Appendix

349

Bibliography

359

Index

387

This page intentionally left blank

Chapter 1

Recreating Large-Scale Evolutionary Phenornena

P.-M. Agapow VieDigitale Limited, 20 Matthias Ct, 119 Church Rd. Richmond, London T W10 6LL, United Kingdom E-mail: [email protected] Any study of the evolutionary past is hampered by the size and type of events involved. Change on such a grand scale cannot be observed directly or manipulated experimentally, and so to date it has been deduced from clues left in the present day. This limitation can be overcome by replaying the course of evolution and observing what results and whether it matches what we know of present day biology. Here I present MeSA, a sophisticated framework for the simulation of large-scale evolution, and demonstrate how this approach can be used to investigate key innovations, a putative cause of patterns of biodiversity. 1.1

Introduction

It has often been said about evolution that “the present is the key to the past” [368], that studying observable microevolutionary process and facets of extant biology can tell us about the the history of life on Earth. Given our necessarily limited view of paleontological life (due to the sparseness of the fossil record, and the subsequent difficulty in deducing the paths of evolution) until recently this forensic approach dominated evolutionary discussion. The unknowable past was a subject for study, but not a source of information in itself.

1

2


It is obvious that studying past via the present has limitations. It assumes that the evolutionary game has not changed substantially over the course of life’s history, that while the fine details of forces and patterns behind biodiversification may vary, the broad picture is the same. This may not be true, with possible explosions of organismal variety occurring in the past [154] and anthropogenic extinctions shaping the present [315]. Second, the scale and size of macroevolution (potentially trillions of organisms over millions of years) defies direct examination and manipulation. In contrast, the scale of most contemporary ecological observations is less than a single square meter over less than a year. The gulf between the two prevents all but the most general conclusions. Conversely, there is a great deal that a “knowable past” can tell us about the present. Evolutionary history is a smoking gun. Its course and shape are a rich trove of information about how organisms arise, prosper and die. In addition, it can serve as a large pool of samples for statistical testing of hypotheses, each diverging population acting as a replicate. Thus by examining the family tree sketched by evolution, it is possible (for example) to study what characteristics make organisms more susceptible to extinction, how body-size changes when the environment changes, what organismal characteristics co-evolve. In this way phylogenies have been used for insights into epidemiology [319], conservation [228], development [96] and many other fields [182]. Unfortunately, there are several obstructions in the path of such investigation. Different workers use a wide variety of metrics or tests to assess how organismal traits change and correlate. It is unclear which tests are best, how tests perform under different evolutionary scenarios [314], or even what such measurements actually mean in terms of the behaviour of real organisms. Given the large number of possible interactions between organisms and their environment, macroevolution may be irreducibly complex [152]. Certainly, no general analytical formulation can possibly capture its many properties. To complicate matters further, phylogenies are at best only estimates of the actual pattern of evolution and may be inaccurate or even incomplete. In summary, the current analysis of large-scale evolution is unable to cope with the complexities of real life. ISome specialised terms are required for this paper and they are grouped here for convenience. Microevolution refers to those processes that shape and shape a population from generation to generation. Macroevolution in contrast describes the longer term behaviour that leads to the formation of species and higher groups of organisms. A phylogeny is a evolutionary family tree, a branching pattern that describes how species give rise to other species or die out. A trait is here used t o abstractly refer to some heritable organismal trait, including bodyform, behaviour, senses etc.. A key innovation is the evolution of a novel organismal trait within a species that grants it an advantage over other species in terms of giving rise to other species.

Recreating Large-Scale Evolutionary Phenomena

3

Below, I present a general simulation solution to the problem of macroevolutionary analysis, and demonstrate its application on a perplexing issue in macroevolution, the causes of diversity.

1.2

A framework for recreating evolution

If analysis is problematic, the generating system is complex and inference uncertain, how can we study macroevolution? An alternative approach is to replay the process evolution. Put in its simplest form: by observing the results of a model that recreates macroevolution and comparing them to what we know of contemporary biology, conclusion can be made about the verisimilutude of the model. Further, the statistical support for these conclusions can be made by compiling the results of many runs. Use of a synthetic model also allows total knowledge of the system unlike real world data. To this end, I present MeSA (Macroevolutionary Analysis and Simulation), a portable and extensible software program and framework for macroevolutionary investigation. Evolutionary hypotheses may be tested by recreating them within MeSA's simulation framework and analysing the results for comparison to the products of terrestrial evolution, contemporary organisms, and what knowledge we have of their ancestors.

1.2.1

Simulation

MeSA incorporates a discrete-event simulation of the salient events of macroevolution - the creation and extinction of species, and organismal trait evolution within species - which it uses to recreate and grow synthetic phylogenies. The factors controlling these events are expressed as a set of rules. Each rule defines the resultant event, its instantaneous rate, and how this rate might vary as a result of the current state of the entire evolutionary system or the particular species for which the rate is being calculated.For example, speciation can be invoked as a number of rule types including Markovian / equal rates (the rate is constant for all species at all times), latency (the rate falls immediately after speciation for a period and by an amount calculated by a' constant function), dependent (the rate varies according to one or more trait values of the possessing species), density dependence (speciation rate alters according the total number of species in the system) and so on. A similar variety of rules exists for extinction, and also extends to a variety of mass extinction rules that may randomly or selectively target species.

4

Recent Advances i n Artificial Life

Organismal traits may be discrete or continuous and may evolve in a punctuational or saltational manner (2. e. where change occurs only at speciation events or in a smooth continuous manner respectively). The direction and velocity of trait evolution can be brownian, directed brownian or dependent on other trait values. Where traits are evolving punctuationally, trait evolution of the two new daughter taxa can be set to be symmetrical (i.e., they inherit traits under the same rules) or asymmetrical (i.e., they inherit different rules). Asymmetrical trait inheritance can be used to model situations like master-slave evolution, where a single rapidly speciating master species gives rise repeatedly to slowly speciating slave species, as is seen in RNA viruses [103]. In order to simulate different periods of evolution in which conditions may change, rules can be grouped together in epochs. An epoch is a period within which those rules hold until an end-condition is reached. These conditions can be related to time or number of species. Thus periods of climatic change or the intervals between mass extinction can be simulated. A discrete event simulation implies that no change occurs in the system between events. Several evolutionary models may violate this assumption. Examples include anagenesis or phyletic transformation, where taxa traits vary continuously across time between speciations and extinctions. In many such cases, this change does not effect other entities or events in the simulation and MeSA sums the change and updates their state at logical points in the simulation (ie. speciation, extinction, the end of the simulation). In other cases these continuous changes interact with and affect the probability of other events. Examples include speciation with latency (where the probability of speciation changes with increasing taxa age), species senescence and stability (where vulnerability to extinction changes with increasing taxa age), or other cases where the probability of extinction or speciation is dependent on traits that vary continuously through time. Wholly realistic simulations of such systems are difficult to construct and computationally expensive. Where necessary, MeSA can discretize these continuous changes across time into a series of small differences that approximate smooth and continuous change. 1.2.2

Data analysis and manipulation

A wide range of macroevolutionary analyses are built into MeSA. While outside the scope of the current discussion, these include measures of phylogeny shape and balance, distance and diversity. To facilitate further analyses, MeSA can also report various information gathered across the nodes of a tree including the total number of taxa, the number of extant taxa, the age, time since speciation, number of children, number of siblings and number of


5

ancestors ultimately subtended. These analyses (where appropriate) can be calculated over all nodes, or selectively over terminal nodes, internal nodes or extant taxa. Also, investigation of the effects of inaccuracy on tests often requires distortions in phylogenetic data to represent these uncertainties. To this end, MeSA allows systematic distortion of data, including equalising or randomising phylogenies branch lengths, randomising character states, and reducing phylogenies from paleontological to neontological forms. A thorough comparison of a range of evolutionary scenarios can prove to be a long and tedious task. The detail and repetition required may discourage researchers from testing as many cases as necessary or lead to the making of mistakes. To this end, MeSA allows all the actions of the program - data manipulation, analysis, simulation and saving of results to be linked together in a queue and executed as one. Furthermore, such actions can be embedded in loops that repeat those actions a given number of times or over the available set of trees. Thus a user can program the queue to repeatedly generate trees under a variety of conditions and then loop over those trees analysing them with the same set of metrics. While it is not as powerful or as flexible as these as full scripting approach, I feel its simplicity and robustness make up for this fact. 1.2.3

Practical issues

Finally, MeSA solves some prosaic issues en passant. Different file formats are often used for storing phylogenetic and organismal information and the (often manual) task of converting between them can be so error-prone and laborious as to prevent the practical use of datasets. Analytical tests are scattered across a variety of software packages (on a number of different platforms), hampering comparison. MeSA uses NEXUS [255] as its primary data format, because it is the most prevalent. Furthermore MeSA can read and write CAIC [318;81, PHYLIP [124] and simple tab-delimited format files. Combined with existing translators this allows the program to easily interface with most forms of data. MeSA may be found on the web at . As it is written in standard C++ and eschews a graphical interface, it is highly portable across platforms and fast.

1.3

Life out of balance

Biology is skewed. It is clear from many studies that biodiversity and phylogenies show a non-random distribution of species. Some groups of

6


organisms are richly fertile, having divided repeatedly to give rise to many, many ancestral species (e.g. beetles). Other groups of similar or greater age are depauperate, with few ancestors (e.g. pangolins, tuataras). Different taxa seem to follow different macroevolutionary clocks of wildly varying speeds. The meaning of this has long been a puzzle of great controversy (e.g. [107;170; 345; 287; 314; 283; 3531). Even when confounding problems - such as artefacts from reconstruction, our limited sampling of the planets evolutionary history, and the random nature of evolution - are accounted for, this pattern remains. Persistent and unidentified factors are systematically distorting the shape of evolution. 1.3.1

Key innovations

The search for putative causes of this imbalance has a substantial recent history (e.g., [29;971). However, given the wide array of processes that could operate at a macroevolutionary level and the aforementioned complexity of macroevolution, interpretation is necessarily complex. In response, a bewildering array of methodologies has arisen and created a spectrum of sometimes contradictory hypotheses about the factors promoting speciation and diversity. Confusion over the factors behind imbalance has thus been augmented by confusion over which methodologies are best under which circumstances. One plausible explanation is the theory of key innovations. This argues that the chance evolution of certain novel morphological and functional traits is associated with new lineages and rapid diversification [401] and may be what creates major taxonomic groups [240]. Put another way, the “discovery” of a novel strategy, bodyform, behaviour etc. by a species gives it - and its descendants - a decisive advantage over other species. This places the idea at the heart of macroevolution, but the paucity of evidence (or approaches for examining it) makes it still “just potentially interesting” [97]. For example, one putative innovation in the plants is a shift in the mode of seed dispersal. Logically, the various patterns of dispersal (e.g., animals carriers, wind, etc.) should have a great effect on the isolation of individual plants and subsequent possibilities for speciation [386]. However several studies failed to show any relationship (e.g., [333;387]), while one indicated a possible complex interaction [97]. 1.3.2

Methods

Conventional forensic analysis having failed, here I attack the problem from the reverse perspective. Rather than search hopefully for correlations between any particular putative key traits and imbalance, instead the shape


7

and imbalance of published plant, arthropod and vertebrate phylogenies shall be compared to those generated by MeSA under a range of evolutionary models in which speciation is determined by key innovations or by other factors (species age or random chance). The aim is to determine which scenarios yield phylogenies of realistic shape. Phylogeny shape and imbalance is assessed with recently-developed imbalance measures [317;9] that are based on the degree of asymmetry in individual nodes. The technical details of these metrics can be found in referenced publications, but briefly imbalance is calculated as the discrepancy between diversification in sister species. For example, a parent species gives rise to two daughter species that (between them) give rise to 10 contemporary species. Maximal imbalance occurs when the split of descendants between the daughters is 1 and 9 (ie., one daughter is the ancestor to all but one of the eventual descendants). Maximal balance occurs when the daughters give rise to the same number of descendants. An imbalance metric is calculated by dividing the observed imbalance by the maximal possible imbalance such that 1 is maximal imbalance, 0 is minimal and 0.5 the expected value under a markovian model (ie., when all species divide at the same rate). Figure 1.1 shows the imbalance signature of a dataset of 208 phylogenies collected from the literature [52]; while Figure 1.2 shows the signatures for arthropods, plants and vertebrates separately. Note that imbalance is significantly greater than the 0.5 value expected under random speciation ( p < 0.0001); in fact it rises significantly when the older, more ancestral species are tested (data not shown). Further, the imbalance signatures of different types of organisms cannot be distinguished statistically, either across all nodes (comparison of slopes p > 0.9; comparison of intercepts p > 0.4). Can key innovations explain these patterns? A number of possible scenarios were constructed in MeSA. In each of these 100 phylogenies were grown until they contained 1000 species:

Key innovation In this model, a discrete trait X has two states, 0 and 1, that confer very different speciation rates. The ancestral state, 0, gives a speciation rate of 0.01, whereas the derived state, 1 (a key innovation), gives a tenfold higher speciation rate of 0.1. X was set to switch between states 0 and 1 with a per-lineage rate of 0.0005 per unit time. In these models, the overall speciation rate was initially low but rose as X changed to a 1 in some lineages. Gradual trait-dependent Here, a continuous trait X evolves by gradual brownian motion, with the speciation rate being directly proportional to X. The starting value of was 100 and the instantaneous

8

Recent Advances in Artijicial Life

Fig. 1.1 Imbalance signature for combined phylogenies. Error bars are 1 standard error. Filled circles = arthropods; hollow circles = plants; hollow triangles = vertebrates. There are no significant differences among the three lines.

+

rate of speciation was determined as a . X c where a and c are 0.0001. Thus, given the initial value of X, the speciation rate was low. There was no upper bound set on X, but if X evolved to a negative value it was set to zero Punctuational trait-dependent Speciation rates were determined by the values of a punctuationally evolving trait, ie. one that changed only at speciation time. In other respects, this was set as per gradual trait-dependent model. Previous research [362] has shown that punctuational change can lead to greater imbalance than gradual change.This is because lineages can be "stuck" with values of X associated with low speciation rates - their values of X cannot change until they speciate, which they are unlikely to do. While to some extent the exact parameter values (and functions) chosen for these three models are arbitrary, they can still be interpreted as the general trends that would be seen in these models. This has been confirmed by experimentation (data not shown). Figure 1.3 shows their imbalance signatures. While details differ among the models and depend upon parameter values, in every case, the trend is for the larger (ie. deeper and older) nodes to show an increasing imbalance. In contrast the smaller nodes are indistinguishable from the null model expectation of 0.5, very different from the pattern seen in the empirical data. In retrospect this is unsurprising. As time increases, the possible divergence in speciation rate between descendants increases. Species that speciate rapidly give rise to more species


9

9 r 9 _.

0

c09 0

Q)

0

Eh, U

$2 $ 2

r .-ul

0

t 0

r! 0

2

4

6

8

10

12

14

In(node size) Fig. 1.2 Imbalance signature for select phylogenies. Error bars are 1 standard error. Filled circles = arthropods; hollow circles = plants; hollow triangles = vertebrates. There are no significant differences among the three lines.

that speciate rapidly, giving the characteristic ((ski-jump” profile seen in the plots.

1.4 Discussion Although the main point of this paper is a description of the approach, the biological findings deserve some consideration. While they do not dispose of key innovations as a possible cause of evolutionary imbalance, it is now arguable that key innovations are a seductively simple idea with many problems. If there were such a things as innovations, then species should be engaged in an ((innovationarms-race” with the velocity of macroevolution increasing steadily through time as more innovative traits are discovered and spread. There is no evidence for this, with recent studies showing rates of speciation have little or no heritability [91; 3321. If key innovations are occurring, they are soon lost or lose their adaptive edge. Perhaps the apparent advantages some organism enjoy over others can be explained not in terms of the organism but their environment. If a species is in the right situation to exploit a new environment, a new niche, a change in climate etc., it and some descendants may enjoy a macroevolutionary advantage before the status quo is restored. This may help to explain why despite much research the list of demonstrated correlates of diversity is short. Also, some other different models of speciation may fit the observed data better than innovations. Patency models [316] describe

10


Fig. 1.3 Imbalance signatures from simulations. Diamonds = key innovations; triangles = gradual evolution model; inverted triangles = punctuational model. Points are averages from 100 simulations of each model. Bars showing 1 standard error would be hidden by the plotting symbols except at the largest node size.

situations where newly created species have a tendency to speciate again, creating rapidly forming chains of species. Other researchers have recently proposed models based on an abstract “niche-space” in which each species occupies a position in a multidimensional grid where the axes are characteristics of the species or environment in occupies [131]. Speciation occurs by species colonizing randomly generated new niches in the space. This produces a similar effect to patency as above because, as the space becomes occupied, the species radiates out from its origin into unpopulated nichespace. It is unclear as yet exactly what realworld properties these models represent. However they do apparently have certain features in common with “real” macroevolution: (1) closely related species can differ markedly in speciation rate, and (2) these rates are not strongly heritable. As mentioned, phylogenies are but estimates and sometimes bad estimates. Could a systematic bias in constructing phylogenies have created the patterned seen? While there have been some reports linking poor quality data to imbalance, no such link was found in a more recent, larger survey [362]. Finally, one recent study has provocatively suggested that some imbalance is the result of human biases in categorization. As always, much work remains to be done. Within biological rsearch there is a long history of model systems, using small manipulatable surrogates for complex realities. This of course has been very sucessful, with evolution being studied by manipulating bacterial populations in a test-tube, studying ecological interactions in small


11

controlled environments, and animal behaviour in artificial populations of rodents and birds. Yet, there are many questions that cannot be handled by these approaches. Processes on the scale of months and metres cannot tell us about millions of years and continents. Bacterial evolution cannot tell us about megafauna, microevolution cannot tell us about macroevolution. This, I think, demonstrates a clear need for artificial life and complex simulation approaches like MeSA in the biological sciences. Such techniques should not be seen as something fundamentally different, but as an extension of the use of model systems. While these methods are not unknown in biology, they are as yet underused. With them we can study problems at scales that were not previously accessible, recreating rather than (0ver)simplifying scenarios. Further, they have certain advantages. By directly recreating experimental questions rather than adapting pre-existing biology, no factor is silently incorporated into the experiment. Thus the effects of hidden states and variables are avoided. Finally, experiments can be easily replicated for statistical analysis, which cannot be done with many biological experiments. Note these advantages do not eliminate the need for rigor. If phenomena are observed in a synthetic model, researchers must always question whether these are the result of peculiarities of the model including implementation details. This, arguably, is the greatest barrier facing the wider use of artificial life systems. As regards MeSA, it has already proven highly useful in exploring a wide variety of macroevolutionary questions, including the assessing methods to measuring phylogenetic imbalance [317], extrapolating the impact of current extinction trends [315] and patterns of viral evolution. Obviously the simulation framework could be extended to encompass other modes of extinction and speciation. The aforementioned niche-space model is an obvious candidate. Another direction for expansion would the development of modes including abiotic, non-evolving factors such as climate, latitude and resources. Such data is often available from stratigraphic records, and may allowing the testing of hypotheses that relate diversity to extrinsic (environmental) as opposed to intrinsic (organismal) factors. A valid criticism of artificial life and simulation models, where attempts are made to replicate realworld biological phenomena, is that goodness-of-fit can be hard to measure. When one researcher sees punctuated equilibrium in a model, another may find only a fleeting resemblance. The experiments above escape this problem thanks to well defined metrics for imbalance. For less strictly defined phenomena assessing a match can be highly subjective. More sophisticated statistical models (such as Bayesian inference) would be useful. In this way, the match between synthetic model and realworld can be properly calculated.

12

Recent Advances an Artificial Life

Acknowledgement Some of this work was published in [316] but is included here for clarity. I am grateful to the Institute for Animal Health for their support during the preparation of the manuscript. The early part of this work was supported by the Natural Environment Research Council (U.K.) through grant GR3/11526.

Chapter 2

Neural Evolution for Collision Detection & Resolution in a 2D Free Flight Environment S. Alaml, M. McPartlandl, M. Barlowl, P. Lindsay2, and H. A. Abbassl ' A R C Center for Complex Systems, The Artificial Life & Adaptive Robotics Laboratory, Australian Defence Force Academy, University of New South Wales, Canberra, Australia { 2314 7403,23153140, spike, abbass} @itee. adfa. edu. au ARC Center for Complex Systems, School of Information Technology and Electrical Engineering. The University of Queensland St Lucia Queensland 4072 Australia paloitee. uq. edu. au During the last decade, Air Traffic movements worldwide have experienced a tremendous growth with new concepts such as Free Flight. Under Free Flight, current procedures of Airways and Waypoints for maintaining separation wouldn't be there. In the absence of Airway structure and ground based tactical support, automated conflict detection and resolution tools will be required t o ensure safe and smooth flow of Air traffic. The main challenge is to develop robust and efficient conflict detection and resolution algorithms to achieve real time performance for complex scenarios of conflicts in a Free Flight Airspace. This paper investigates preliminary design and implementation issues in two dimension application of evolutionary techniques for collision detection and resolution. The preliminary results demonstrate that an artificial neural network (ANN) using evolutionary techniques can be trained not only follow optimum trajectories, but also to detect and avoid collisions in two dimensions.

13

14

2.1

Recent Advances in Artijcial Life

Background

Collision detection and resolution (CD&R) is a fundamental problem to many mission critical & real time applications. It is also of prime importance in Air Traffic Control, Vehicle Navigation, Robotics, and Maritime operations. A literature review revealed that most of the CD&R techniques discussed assumes the availability of intent information of all the other aircrafts in the vicinity for the purpose of trajectory prediction. However in the real life scenario it’s highly unlikely that long term intent information can be made available to predict conflicts with accuracy. Free flight concept lets the pilot decide the course; altitude, air speed and other parameters dynamically in real time. This makes it extremely hard for conventional CD&R algorithms to work in such environment. Use of linear programming techniques were discussed in the SOM (Stream Operation Manager) model [291],it suggests techniques for automated integration of aircraft separation, merging and stream management using linear programming techniques. The SOM input requires quite a few data to be input before hand including static Aircraft performance data, which given the dynamic nature of a flight plan may change dramatically en-route. One of the key parameters given as input in case of a conflict resolution is pilot preference, which may override the Aircraft performance envelop while negotiating a conflict with other aircraft compromising the safety. Another mathematical modeling technique, multi-point boundary value problem with ordinary differential equations, to be solved numerically with the multiple shooting method was discussed by Rainer Lachner (Collision Avoidance as a Differential Game) [230]. However the unavoidable analytical or numerical calculation of hundreds, thousands or even ten thousands of optimal trajectories to obtain the optimal strategies representation is generally a difficult task given the time criticality and lack of high computing power onboard. A geometrical approach to the problem of CD&R was investigated by Ki-Yin Chang and Gene Parberry [75] by using 4 Geometry Maze routing algorithm (A modified version of 2D Maze Lee Algorithm) to obtain the particular route of each naval ship that have potential to collide, which is detected by simulating the particular routes with ship domains. The algorithm provides linear time complexity and guarantees to find an optimal path if exists. However the algorithm is suitable for navigational situations at sea characterised by slow cruise speed and large time window for heading change. It assumes that trajectory and speed of the naval ships remains unchanged which is highly unlikely in a free flight environment. A rule based approach for solving CD&R problem is described by I. Hwang and C. Tomlin (Protocol Based CD&R for ATC control) [203]. It uses multiple conflict detection model and detects collision in 2D horizontal plane. It

Neural Evolution for Collision Detection

15

uses rule based approach for conflict resolution. The algorithm discussed by the authors is robust to uncertainties in aircraft position, heading & velocity. However the experiments performed by the authors using fixed 20 minutes lookahead time window for conflict prediction gave rise to lots of false alerts and act as a limiting factor to extend the model further. Free flight model where uncertainty in trajectory in inherent makes the detection of conflict between aircrafts very complex task, A. Chakravarthy and D. Ghose [74] proposed collision cone approach as an aid to collision detection and avoidance between irregularly shaped moving objects with unknown trajectories. This mathematical model was restricted to CD&R between two objects only and is discussed mathematically with out any simulation and experimental results. This uncertainty in intent information and its complexity in detecting conflict in a free routing environment was also investigated by Prandini, M. [309]. The 2-D, two Aircraft CD&R algorithm uses probabilistic framework, thus allowing uncertainty in the aircraft positions to be explicitly taken in to account when detecting a potential conflict. The authors use the fight plans of the two aircraft, generate pairs of aircraft trajectories over a 20 minutes time horizon according to the discretized version of the stochastic differential equation, and do the computations for conflict detection. High false alert rate(l8%) shows that algorithm needs further improvements and sensitivity analysis on the part of crossing angles, minimum deterministic distance, and time of minimum distance, in order to set a value for the threshold which is appropriate for the typically encountered path configuration. Many of these 2D CD&R algorithms are justifiable for a completely known environment. A partially known dynamic environment like a free flight airspace where long term trajectories cannot be predicted, requires an entirely different approach. An evolutionary algorithm based model may handle the flexibility required in free routing model and may handle additional constraints. These facts and also the relative simplicity of dealing with the two dimensional case have caused our approach proposed in this paper to focus on 2D conflict detection, and horizontal resolution maneuvers.

2.2

Modelling the Problem

It is assumed for modeling the problem that two Aircraft are flying at a constant speed and altitude, in a 2-D free flight environment. The Aircrafts (referred to as agents from here onwards) explore the environment trying to reach their destination in a given time interval. The agents have to minimize the off track error (The difference between the planned trajectory and actual trajectory) and to detect & resolve collision with other agents. Our

16

Recent Advances in Artajkial Life

objective is to build and train an artificial neural network (ANN) applying Evolutionary techniques which are able to modify the heading of the Agents to avoid the conflict while keeping the agent nearest to its optimal trajectory and directing them to reach their destinations. The experimental environment (airspace) is made up of 2-dimensional cells and is discrete. The preliminary experiment used a 10x10 environment with 2 agents. For experimental purpose the starting position of an agent A and the starting position of an agent B are chosen such that their mission trajectories cross each other ensuring a collision scenario, if the agents maintain planned paths and do not use avoidance. Each Agent has proximity sensors for the neighboring eight cells and they also emit probability signals into the surrounding cells for likelihood of occupation on their next move. These signals are directly proportional to the Euclidian Distance from the cell position to the destination. Any obstacles like terrain proximity, Bad weather; Special Use Airspace is read by 8 obstacle sensor. The ANN has to deal with the issue of Agents reaching the edges of the discrete environment, for that a wrap-around environment was implemented. An Agent can wrap-around the environment from left to right, top to bottom as well as the four diagonals. For example if an Agent is at position (0’10) and decides to move East, the Agent’s new position is (29,lO). As wrap-around behavior is not desired, Agents that perform it are penalized. In conceptual terms wrapping around the environment is a greater distance, and therefore time, than moving one cell within the environment’s boundaries. Agents’ movements are updated asynchronously to ensure no particular Agent is biased. At each time step the environment is updated according to the following steps. (1) Sense: Compute the Euclidian distance to destination from each neighboring cell. Compute the probability of collision for each neighboring cell based on the other Agent proximity signals. Compute the probability of collision for each neighboring cell for the presence of Obstacles.

(2) Make a decision: Based on the objective function Set the inputs to the ANN ( proximity signals, obstacles, distance) (3) Move: Update the Agent’s position based on the ANN output. (4) Update: Update the neighboring cells of the new position for probability of occupancy in the next move.

Neural Evolution f o r Collision Detection

17

Fig. 2.1 Representation of the cellular environment with two Agents, their destination targets and their paths. X marks the destination target of the Agents; the shaded area represents the paths of the Agents.

Fig. 2.2 Representation of the cell occupancy signals model based on the destination target as shown in Figure 1, the depth of shading is indicative of the distance to the target.

Each Agent emit occupancy signals for each 8 adjacent neighboring cell at every time step indicating the normalized Euclidean distance to its destination according to the following equation.

18


After each step the occupancy signals are re-assigned. The Agents are equipped with 18 sensors: (a) 1 distance to destination sensor; (b) 9 proximity sensors; and, (c) 8 obstacle sensors. The distance sensor indicates the Euclidean distance from the current position to the Agent’s destination position. The proximity sensors detect the combined Agent’s occupancy signal values in the adjacent cells to the current position. The obstacle sensor’s act similarly to the signal values, by detecting obstacles in adjacent cells (for the preliminary experiments, the Agent’s are the only obstacles and are represented by 1. 0 if present and 0. 0 if not). At the end of each run, the Euclidian distance from the agent’s end position to destination is computed for fitness. The collision counter maintains the number of times an agent collided with another. In this preliminary experimentation, if the agent reaches its destination then its position is not updated further. 2.2.1

Collision Detection

Collisions among the agents are detected according to the following rules as shown in Fig 3: (1) Agent A and B occupy the same cell (2) Agent A and B have switched cells and, (3) Agent A and B’s paths have crossed over in the same time step.

Fig. 2.3

2.3

Collision Scenarios between two agents in a 2D environment

The Neural Network Structure

A three layer ANN architecture is used (Fig 4). The input layer has 18 inputs based on the Agent sensors as mentioned in the section above, the middle (hidden) layer contains a fixed number of nodes and are varied as 2, 5, 10 and 12. The third layer, the output layer has three binary outputs which denote the direction of movement i.e. 23 = 8 possible moves.


19

The ANN topology that was used was a feed forward network with input to output connections and recurrence on the hidden neurons (see Fig 4). Recurrent connections were used to attempt to stop the Agents from getting ‘stuck’ in a two-step movement (moving back and forth in two cells only). This problem will occur, particularly in feed-forward topologies, when the inputs to the network are identical in the cells. For example if the ANN decides to move to the cell directly East, then based on the new inputs for the Easterly cell decides to move back to the original cell, the original inputs are identical to the first time the Agent was here (assuming another Agent is not nearby). Using recurrent connections adds a dimension of time to the ANN, so this problem is less likely to occur. Note if the context neurons are not being used by the ANN this problem may still occur. Classical back propagation cannot be used in our case because conflict free trajectories are not known in advance. We have used the Self-Adaptive Pareto Artificial Neural Network (SPANN-R) algorithm [l;3801 for evolving the weights of the ANN.

Fig. 2.4

Type 3 - ANN topology with recurrence in the hidden nodes

20

2.4


Preliminary Experimental Setup

Two fitness functions were used. The first fitness function attempts to minimize the off track distance travel by the Agents and reduce the time to find the target destination represented by the following equation.

.c N

fl=

D(Cur,, Dest,)

+ T, + a.P,

n=l

where N is the number of Agents, D is the Euclidean distance between the current position of the Agent and its destination, T is the time the Agent found its destination (this will be the total time steps if the Agent does not reach its destination). T can be seen as a penalty term which is used to set a pressure on each Agent to get from origin to destination in shortest possible time. If the Agent is not able to find the destination at the end of a run, a larger value is assigned for T . a! is the wrap around penalty and P is the number of times the agent wrapped around the environment, the more times the agent wraps around an environment, the higher is the P value. The second fitness function is the total number of collisions that occurred in the run.

f2

=

c

(2.3)

where C is the total number of collisions detected in the run. The evolutionary runs were performed on a population size of 100 chromosomes for 1000 generations. The initialization of the ANN weights in SPANN-R is done using a Gaussian N(0,l). The crossover rate and mutation rate are assigned random values according to a uniform distribution between [0, 11. These functions were designed to guide the evolution to avoid the other Agents as well as to find their targets in the shortest possible path. Direct encoding is used here as it is easy to implement and simple to understand encoding scheme. The mutation rate and crossover rate are assigned when the random initial population of potential solutions is created. However,the representation scheme of SPANN-R algorithm allows for the self adaptation of crossover and mutation rate during optimization. It is recognized that there are issues with the design in that it does not promote generalization of the networks (i.e. changing the environment will cause unexpected results). The preliminary experiments were designed to test the initial theory of target finding and collision avoidance behavior in a static scenario.


21

Table 2.1 Environment Parameter set up for preliminary runs Parameter World Dimensions Number of Agents Agent A Origin Agent A Destination Agent B Origin Agent B Destination Generations Time Steps Population size Hidden Nodes

Value lox10 2 191 8,8 872 23 1000

40 100 2,5,10,12

2.4.1 Preliminary Results The preliminary experiments and results show that an ANN can be trained for target finding and collision avoidance. The results are recorded from the best performing ANN in fitness 1 with the lowest scoring fitness 2; or, in other words we only consider solutions where no collisions occurred. From the results below it can be seen that an ANN with 10 hidden nodes has a better chance of finding a good solution as seen by the low average of the solution set. It is also seen that ANNs with hidden nodes as low as 2 are still able to find a good solution but it seems that the ruggedness of the landscape is high. The best overall solution is found in the experiment with 12 hidden nodes (value of 55) which is the lowest possible value that can be obtained without a collision occurring. From these results it was decided to use 10 hidden nodes for the main experiments. The implementation of the wrap aiound behavior adds additional complexity to the problem. From observation of initial generations the agents were found to wrap around the environment and find their destination positions in a couple of steps. Due to the wrap around penalty, the evolution eventually finds better solutions which do not perform the wrap around behavior. This adds a level of complexity to the solution space.

2.5

Main Experiment Setup

For the main experiment, the Agents have their mission trajectories embedded in them and one of the components of the modified fitness function is to try to minimize the off track movement in the environment while still trying to reach the destination. The trajectories are generated from an equation of an ellipse whose major and minor axis points are given.

Recent Advances in ArtiJcial Life

22

Table 2.2 Preliminary results with various hidden units and random seeds. In all results, there was no collision Randseed 101 102 103 104 105 106 107 108 109 110

2 Nodes 56. 00 128. 00 102. 35 105. 76 69. 23 128. 00 128. 00 118. 24 104. 76 120. 45

5 Nodes 128. 00 115. 94 128. 00 119. 89 69. 00 56. 00 88. 63 104. 76 56. 00 83. 65

10 Nodes 56. 00 68. 00 55. 00 68. 00 128. 00 56. 00 115. 94 69. 23 68. 00 58. 00

12 Nodes 122. 42 68. 00 68. 00 120. 31 55. 00 55. 00 76. 21 128. 00 115. 43 68. 00

Fig. 2.5 5a: The agents moving in the environment towards their target with the shaded area representing their occupancy sensor and shows the two agents coming in close proximity of each other, detecting the conflict. 5b: The agent resolving the conflict by changing the direction heading. 5c: shows the final path taken by two agents at the end of simulation.


23

The center of the ellipse is at 20, yo, assuming b > a, b is called the minor axis and a is called the major axis. From the start position and end position for an Agent, the center of the ellipse is computed and the elliptical mission trajectory is generated. This is to ensure that the ANN can learn to follow an elliptical path rather then just moving the Agents in a straight line. The ANN is trained to move the Agents towards their destination in desired time steps as computed by their elliptical mission trajectory. If the Agent reaches its destination early, there will be a penalty associated in terms of the extra distance which it will move away from its destination in the remaining time steps. To compensate for any collision avoidance manoeuvres extra time steps are allowed to an agent. If an Agent doesn’t reach its destination in the desired time steps, then there will be a penalty cost based on the remaining dynamic distance to its destination.

2.6

Fitness Function

For the main experiments a powerful class of technique known as dynamic programming is used [37]. A particular sub-category known as dynamic time warping (DTW), that has been successfully utilized in automatic speech processing [320] is employed for computing the Fitness Function. DTW is a method for calculating the distance between two time-varying sets of values. The method seeks the best temporal alignment between the two samples under various constraints - the alignment can be visualized as ‘stretching’ (repeating) certain portions of each set at certain times. Given that alignment - which ensures start and finish alignment, and that all values from each set are used - a minimum distance between the two sets is found. This technique suits our test environments given the Agents mission path and dynamic nature of the actual path they take while exploring the environment. The agents may deviate from their mission path for a variety of reasons like avoiding a collision, implementing wrap around the environment etc. For computing the first fitness as the area between the two paths (mission path and actual path) this technique ‘compensates’ for relatively minor temporal differences, while still ’accentuating’ significant temporal (and raw value) differences between the two paths. The first fitness is given by f l = D ( M ,N )

(2.5)

24

Recent Advances an Artzjcial Life Table 2.3 Parameters used for the main experimental setup Parameter World Dimensions Number of Agents Generations Time Steps Population size Hidden Nodes

Value 10x10 2 1000 40

100 10

M is the mission path = b l , bz, b, and N is the actual path = a l l a2,..a,. D(1,l) = d(b1, a l ) where d is the Euclidian distance D ( i , j ) = r n i n { ~ (-i l , j ) ,~ (- 1i , j - i),~ ( i ,-j 1)) d(bi,aj) m e r e d(bs', = J ( b s , - at,)2 (bs, - at,)2 at position z, y. The second fitness function

+

+

atj

remains the same as total number of collisions that occurred in the run.

f2 = c

(2.6)

where C is the total number of collisions detected in the run. The fitness function for the main experiment is

F

= min(fl+

a.P,

+ f2)

(2.7)

where a.P, is the wraparound penalty as is described in Equation 2 To ensure generalization, four different scenarios were taken for each run. Each scenario has different start and end position for the Agents, these scenarios ensure that the agents have colliding trajectories. For each run each scenario gives its own fitness which is then averaged out. This helps the ANN learn motion, avoidance and target acquisition beyond symmetrical paths.

2.7 Main Results and Analysis The experiments and results with the new fitness function and generalization show that an ANN can be very efficiently trained for multi objective scenarios, viz. following an elliptical trajectory, Conflict Detection and Resolution, finding the target and avoiding the wraparound behavior. The results are recorded from the best performing ANN in fitness 1 with the lowest scoring fitness 2; we analyzed only those solutions where no collisions occurred. The best overall solution found in the experiment with Fitness Function Value 10. 82. This ANN keeps the off track error to its minimum, detects collision, avoids the collision and drives the agents towards their target. Hinton diagram display the output behavior of the hidden nodes with every time step of the run.

Neural Evolution f o r Collision Detection

25

Table 2.4 Results of Experiment Best Case Worst Case AveraEe Case

Fitness F1 10. 82 144. 70

Fitness F2 0. 0 0. 0

Fitness Value(FlSF2) 10. 82 144. 70

I 55 in9 163 217 271 325 379 433 487 541 595 ~ 4 703 3 757 811 865 919 973 Iknerdim Fig. 2.6 Evolution Graph showing the best Fitness values in a population set of 100 for 1000 generations

It appears from Figure 2.7 that hidden node 2 activation state drives the agent towards its trajectory as well as collision detection. The Hidden node 1 is activated during collision resolution, hidden node 1 and 6 activates to bring the agent back to its trajectory following an off course resolution maneuver and hidden node 5 is activated when destination is reached. Hidden nodes 4 and 8 remain inactive during the run. F'rom Figure 2.7 it appears also that that hidden node 5 drives the agents to their trajectory path and node 3,lO and 4 activates during conflict detection, resolution and resume own navigation respectively. Hidden Node land 7 remain inactive during the simulation run. From Figure 2.7 it also appears that hidden nodes 7 and 3 initially drives the agent to its planned trajectory and then collision detection and resolution are regulated by hidden nodes 6 and 5 respectively. Node 3 drives the agent to resume its own navigation. Hidden nodes 10 remain inactive during the run.

26

Recent Advances an Artificial Liie

15

15

10

10

5

3

2

4

E

8

1

3

3

2

4

6

~

1

0

Hidden Nodes 15

15

10

10

5

Z

4

8

B

I

b

2

4

6

8

1

0

Hidden Nodes Fig. 2.7 Hinton diagrams showing the output behavior of the hidden nodes with every time step of the run

2.8

Conclusion & Future Work

Experiments and results shows that an ANN can be trained efficiently using Evolutionary techniques for collision detection and resolution in a 2-D environment using horizontal manoeuvre techniques. With the new fitness function and generalization after 1000 generations the Neural Network not only learns well t o guide the Agents in a 2D environment to reach their desired destination while minimizing the cross track error (deviation from optimal trajectory) but also detects and resolves collisions with other agents in the environment. On of the principle requirement of future Free Flight system will be robust CD&R mechanism. Since detecting conflicts with aircrafts on random routes is more difficult than if the air traffic were on structured-airways, the pilots/controller will have to rely on an automated system to detect problems and to provide solutions. Such a system can only be implemented by developing a robust and efficient CD&R algorithm.


27

Fig. 2.8 A single agent movement in the environment showing that an ANN can be trained to follow an elliptical path, the light shade lines denotes the original mission trajectory and dark shade denotes the actual trajectory

Fig. 2.9 The initial position of agents in the 2D environment for the main experiment setup with their destination marked as X. The elliptical trajectories displayed are optimal path to destination. The shaded rectangle shows a potential conflict zone

Recent Advances in Artajcial Life

28

I ! ! ! ! ! ! ”

Fig. 2.10 A (top): Two scenarios showing the agents approaching each other and detecting a collision B(midd1e): Two scenarios showing the agents in collision resolution by change of heading. C(bottom): Two scenarios showing the agents reaching their destination without colliding with each other

Future work involves extending the model to three dimensions and to add other parameters for collision resolutions viz. speed, heading and vertical manoeuvres. Moving the environment from discrete to the continuous domain will bring new challenges in training and testing the ANN. Future developments on the 3-D environment with continuous domain will certainly affect the architecture of ANN and increase the complexity of the system,and may give a deeper insight in understanding its behavior.

Acknowledgements This work is supported by the Australian Research Council (ARC) Centre for Complex Systems grant number CE00348249.

Chapter 3

Cooperative Coevolution of Genotype-Phenotype Mappings to Solve Epistatic Optimization Problems L. T. Bui, H. A. Abbass, and D. Essam The ARC Centre for Complex Systems, The Artificial Life and Adaptive Robotics Laboratory, School of ITEE, UNSW@ADFA, Canberra, Australia E-mail: { 1. bui,h.ab bass, d. essam} @adfa.edu.au Genotype-phenotype mapping plays an important role in the evolutionary process. In this paper, we argue that an adaptive mapping could help to solve a special class of highly epistatic problems known as rotated problems. Our conjecture is that co-evolving the mapping represented by a population of matrices in parallel with the genotypes will overcome the problem of epistasis. We use the fast evolutionary programming (FEP) algorithm which is known to be unsuitable for rotated problems. We compare the results against the traditional FEP and a conventional co-evolutionary algorithm. The results show that, in tackling rotated problems, both FEP and the co-evolutionary FEP were inferior to the proposed model.

3.1

Introduction

The biological evolution can arguably and debatably be seen as an optimization process in which the fittest individuals of a species survive from the competition with others throughout the evolutionary chain. This feature attracts much attention, particularly in the field of optimization. To simulate evolution, a population of individuals are distributed randomly in a search space. They then evolve and compete overtime, and the population gradually approaches the area of optimality. 29

30


In biology, to clarify the structure of an individual (or organism), geneticists distinguish between the concepts of genotype and phenotype, or the genetic structure and the physical characteristics of the organism. The genotype is the genetic material inherited from the parents, while the phenotype is the expressed traits of an organism [65]. The mechanism by which genetic variations are mapped onto phenotypic variations has a strong impact on the evolvability of the organism [66]. In general, the genotype-phenotype mapping (GPM) has .an important role in the interaction between the evolutionary process and the process whereby the organism interacts with the surrounding environment (learning process). Despite that GPM correlates the phenotype of an organism to its genotype, organisms with identical genotypes can express variations in their phenotypes. The Baldwin effect reveals that the behaviors learnt on the level of organisms have a high impact on the evolution of species [132]. The Baldwin effect works through phenotypic plasticity and genetic/environment canalization [293;400; 4041. In phenotypic plasticity, the external environment can contribute to the formation of the phenotype; thus the phenotype does not depend solely on the instructions encoded in the genotype. Through learning, an individual may adapt to a mutation that would be useless to the individual without the extra learning step. Thus, if the fitness of the individual increases as a result of this mutation+learning component, the mutated gene may proliferate in the population. Evolution, however, cannot depend on this costly phenotypic plasticity alone. Evolution may then play a role in maintaining the learnt behavior through genetic/environment canalization. In genetic canalization, specific genes become resistent to genetic mutations; thus persist to exist in the population. In environment canalization, a stablizing natural selection process may favor genes which reduce environmental variance of a trait. The Baldwin effect differs from Lamarckian evolution in that there is no direct alteration in the genotype as a result of the learning occurring on the phenotypic level, while in Lamarckian evolution, a direct alteration in the genotype exists. Given the importance of GPM in biology, we hypothesize that this mapping can play an important role in optimization as well. In particular, we have considered an adaptive GPM scheme to solve rotated optimization problems. In rotated optimization problems, a number of almost independent variables are rotated. The resultant variables thus become highly dependent and the optimization problem is defined on those rotated variables This problem is very difficult under the following two assumptions: (1) We do not have access to the independent variables, otherwise, the optimization problem becomes trivial. Thus, due to the dependency

Cooperative Coevolution of GPM

31

(interaction, epistasis) between the variables, the problem represents a major challenge to genetic algorithms. (2) The optimization problem is black box, thus it is not possible to analyze the objective function analytically and explicit calculation of exact gradient information is not possible. We assume that the GPM is neither unique or fixed; thus as natural selection favors genes, it also favors successful mappings. We co-evolve populations of genotypes and mappings (matrices). In this paper, we only look at linearly rotated problems, where the original space is linearly transformed to a new space. A genotype is mapped to it’s phenotype by multiplying it with a select6d matrix. Since we do not know the mapping, the genotypic space simulates the dependant variable space, and hence the population of matrices can simplify the problem by simulating the inverse of the mappings. Thus, a phenotype in this paper is the product of a chromosome in the genotype population and a matrix in the mapping population. As both populations co-evolve, the inverse effect of rotation emerges to relax the difficulty of the epistasis. To validate this model, we have compared its performance to that of conventional Fast Evolutionary Programming (FEP) [439] and a cooperative co-evolutionary version of FEP called CFEP [246]. We chose these two methods because the genetic operators of evolutionary programming are known not to work for rotated problems. Thus, if the proposed method converges, it will converge mainly because of the mappings, not because of the operators. In the rest of the paper, we first review work on co-evolutionary GPM, and then explain the proposed model, and the setup for the comparative study. The paper concludes with results, and discussions.

3.2

The use of co-evolution for GPM

There are several pieces of work on the integration of co-evolution with the process of GPM. In general, by allowing multiple populations that are interactively (cooperatively or competitively) co-evolved, GPM could be evolved in parallel with the evolution of genotypes. A typical example is the co-evolutionary model in the work of Potter and Dejong [307;3081 where the phenotype is built by collecting individuals from each sub population. Individuals could be either selected at random or from the best of each sub population. However, although this is a kind of of indirect encoding, the mapping structure here is fixed and non-evolved. A thorough study about the co-evolutionary approach was carried out by Paredis [299]. The author proposed a symbiotic co-evolution system,

32


called SYMBIOT, to solve 3-bit deceptive problems. It involved two cooperative populations, one is the population of solutions and the other is the population of permutations. The mapping between a permutation and a solution is implemented by using a permutation to define the order of genes in a solution. This ordered solution will be applied to the target function in order to get the fitness value of the solution. Each encounter between solutions and permutations involves two solutions and a permutation. These two solutions are used to generate a child. The permutation is used for both the parents and the child. For it, each encounter gives the permutation a pay-off value that is the average fitness of the parents divided by the fitness of the child. Each permutation has a history list of 20 most recent encounters. This list is updated continuously over generations. The sum of pay-off values in the history will be set to be the fitness of the corresponding permutation. In this way, the fitness becomes continuous and partial. This evaluating technique is called Life-Time Fitness Evaluation (LTFE). Murao et a1 [285] proposed an idea to apply this approach to engineering design using genetic algorithms. The authors hypothesized that a GPM could help designers to explore the search space to find a possible solution candidate, but that the lack of prior knowledge of the problem poses a difficulty to the design of a proper GPM. In order to overcome this, it is possible to evolve the GPM along with the evolution of genotypes. This will therefore help to relax the aforementioned difficulty. Basically, the system is designed similarly to the SYMBIOT model described in Paredis’s work [299]. In Murao’s system, two populations are built, one is the population of sequences of genes, while the other is the population of sequences of permutations of positions in the genome. A genome is generated by the permutation of all the genes in the first population (the population of seeds) in the order defined by an individual (a rule) of the second population. However, instead of using LTFE, the fitness of a seed is determined by first playing the seed against all rules (permutations), and then the maximum value obtained is given to the fitness of the seed. The fitness of a rule is evaluated in the same manner. The empirical experiments showed that the co-evolutionary approach outperformed a conventional GA on a 3-bit deceptive problem. Although the system was limited to the binary domain, it gives a strong implication of the usefulness of the co-evolutionary approach in helping the GPM process. As stated by Murao [285], an adaptive mapping would be convenient for engineering design problems where it is difficult or impossible to obtain the prior knowledge of the problem to determine a suitable mapping. In this situation, adaptive mapping can help to relax this difficulty. By adaptive mapping, we mean that a genotype is mapped indirectly to a phenotype. Although, to date, it is not clear that it is better to allow adaptive mapping


33

in combination with or separated from the genotype, the application of coevolution has great potential to adaptive mapping. The analysis of evolving populations of combined mappings and genotypes is outside the scope of this paper. The above work shows that for a certain problem and with a suitable design, co-evolution can help search algorithms to progress better towards an optima than comparable conventional EAs. It would be ideal if we can find some way to effectively combine the adaptive mapping with coevolutionary features. In previous work, a number of problems were considered by co-evolved adaptive mapping systems, such as a binary 3-bit deceptive problem where the mapping was the permutations of gene loci in the chromosome. Potter and De Jong [307;3081 proposed another scheme for adaptive mapping. For it, each gene was evolved cooperatively in differing populations, and combination of a single representative from each population defined a phenotype vector. However, this approach for realparameter optimization is most suitable when variables do not interact. When high epistasis exists, the interaction between variables is much harder to capture.

3.3

The proposed algorithm

Undoubtedly GPM has an important role in both biological and computational evolution. It is the bridge for two parallel processes: evolution and learning. It is possible to use GPM as a tool to impose bias and to implement strategies to control the interaction between evolution and learning. One of these possibilities is the evolution of a GPM. Instead of a static representation, a mapping can be adapted to be better suited for a given problem. This corresponds to the situation where the mapping evolves under the effect of both the interaction of genetic materials, as well as with the surrounding environment. For rotated problems, the genotype always has to undergo a transformation operation with a fixed rotated matrix before assessing its fitness against the target function. If the matrix is known in advance, it is easy to generate the inverse matrix and therefore the optimization process could more easily progress. However, in the case of the absence of information about the rotation matrix, the rotation becomes difficult. Therefore, if an adaptive mapping is applied, it should support this inverse process. Our conjecture is focussed on two aspects. Firstly, a mapping is defined in the way that a phenotype is generated by multiplying a genotype with an evolved matrix. Secondly, the mapping will be evolved in parallel with the evolution of the genotypes in order to adapt matrices towards the inverse matrix. This is

34


1-2

Fitness

Fig. 3.1

T

The general framework of the proposed model.

an incremental development process in which good mappings are developed incrementally based on previous mappings. In order to implement the evolution of mappings, we co-evolve the population of genotypes with a population of matrices, called the population of mappings, in a hope to evolve the inverse of the rotation matrix. The population of genotypes are implemented as normal except for the calculation of their fitness values. The evaluation of an individual’s fitness is carried out by taking the phenotype resulting from mapping the genotype through a selected matrix, and then measuring how well it performs on the target function. The same applies for the population of matrices; the phenotype used to evaluate the fitness of a matrix is generated from mapping a selected genotype with the matrix. In this way, we integrate a cooperative approach into the proposed model (See Fig. 3.1). The fitness of an individual in the genotype population is partially dependent on the performance of individuals in the mapping population and vice verse. A good convergence to the inverse of the rotation matrix will help to reduce the distortion caused by the rotation matrix and a diverse population of genotypes will benefit the convergence of the mapping population. There are a number of possible fitness assignment schemes, we adopt the “best” strategy to assign the fitness value for each individual in both populations. According to this strategy, an individual in a population will play with the best one in the other population. In other words, the mapping always takes into account a pair of a selected individual and the current best one in the other population. It is clear that the fitness of an individual is dependent on two aspects: the collaboration of the individual with the other and the convergence of the mapping population. In order to run the model, we implemented the Fast Evolutionary Programming (FEP) as the underlying evolutionary algorithm, due to its successful performance on real-valued optimization problems [439]. FEP is known to perform badly on problems with high epistasis as our experiment below confirms. In general, FEP and EP are quite similar. The only difference is in the way to mutate the chromosome. In EP, the mutation is


35

carried out by adding a Gaussian random value to each gene in the chromosome, while FEP uses a Cauchy distribution. As stated by Yao et a1 [439], the Cauchy distribution is better in exploring the search space than its Gaussian counterpart. A description of FEP is given in Algorithm 3.1.

Algorithm 3.1 The pseudo code for the Fast Evolutionary Programming Require: Population P: p individuals (pairs of real-valued vectors (x, 7)) Ensure: Evolve P 1: repeat 2: Create a population Q of p children from P by modifying each individual in P in the following way 3: for all individuals (each individual i has n members) do 4: Generate R = N(0,1), a Gaussian random generated value 5: Generate 6 = a Cauchy random generated value 6: Generate a child (xz,$): 7: x: = xi ~ i and 6 T$ = qiexp(7’R 7N(O,1)) 8: 7’ = (-)-’ and T = (&)-l 9: Evaluate the fitness values for the child 10: end for 11: Use tournament selection to select p individuals from P+Q 12: Replace all members in P by these new individuals 13: until Given conditions are satisfied 14: return New population P

+

+

In detail, the model can be described by its major steps as follows: 1. Initialize: both populations are initialized uniformly within prespecified ranges. For the genotype population, the ranges of genes are similar to the ranges of variables of a specific problem. Meanwhile, the values of the matrices are set initially in the range of [-1,1]. The phenotype that is used to evaluate the fitness of an individual is generated by mapping between the individual and a randomly selected one in the other population. 2. Apply FEP to the genotype population: The evolution of the genotype population is controlled directly by FEP. The fitness of a new individual is evaluated by using the current best mapping. The outcome of this step is a new population of genotypes. 3. Apply FEP to the mapping population: The Operation of FEP on the mapping population is the same as in Step 2. A new population of mappings are generated as a result. 4. Examine the termination condition: A pair of new populations from Steps 2 and 3 is denoted as a generation. If the error rate or the

36


number of evaluations reaches a certain level, the co-evolutionary process is terminated, otherwise the algorithm returns to Step 2.

3.4

3.4.1

A comparative study Testing scenario

To validate the model, this system was tested on a rotated Rastrigin problem with 10 variables [377]. This is a minimized multi-modal optimization problem with a huge number of local optima. Further, it is a shifted, rotated, non-separable and scalable problem. Figure 3.2 visualizes the problem in both cases: with and without rotation. The problem is defined formally as in Equation 3.1. D

f ( z )=

C(z?- locOs(27rzi) + 10) + fbias

(3.1)

i=l

in which z=M*(x-o), M is the rotation matrix, D is dimension, o is shifting vector, and z E [ - 5 , 5ID. Global optimum x*=o, and f(z*)=fbias(lO)=-330 We also tested this problem with two versions of FEP: a conventional FEP and a version of co-evolutionary FEP called CFEP. The CFEP is implemented similarly to CCGAl in Potter and De Jong’ work [308]. It uses FEP as the underlying EA instead of using conventional GA. The setting of all approaches are given in Table 3.1. In terms of the genotypephenotype mapping, FEP uses a direct encoding scheme, while CFEP and the proposed model employ indirect schemes. CFEP’s mapping structure is fixed because the way it converts the genotypes to phenotypes is kept unchanged over time, while the proposed model uses an adaptive mapping scheme. All algorithms were tested with 20 separate runs in which each run was associated with a different initial random seed.

Fig. 3.2 Rastrigin function with and without rotation.


37

Table 3.1 The settings for the proposed model, F E P and CFEP

Parameters Population size Initial 11 Tournament size Number of evaluations

3.4.2

The proposed model 50 O.l(for the population of mappings) 10 100000

-

3.0

CFEP 50 3.0 10 100000

FEP 50 3.0 10 100000

Results

From Table 3.2, it is clear that the proposed model is leading with a mean of -315.879, while the equivalent values of CEFP and FEP are just -271.588 and -310.618, respectively. This finding continues to be true when we examines the best, seventh, median, and worst (25th) runs. In all cases, including those not shown, the proposed model outperformed the others. The matrices seem to have helped the system to relax the difficulty of the rotation. Once again, the advantage of adaptive mapping is confirmed over fixed mappings. However, it is interesting to note that CFEP is inferior to FEP. This is another case in which an indirect scheme is outperformed by a direct one. To understand this, we examined the structure of CFEP in which each gene is evolved independently with a separate population. This kind of structure does not have any advantage in the case of rotation, as was pointed out by Potter and De Jong [308]. We visualized the fitness Table 3.2 Fitness values achieved after 100000 evaluations

Parameters 1st 7th 13th 25th Mean STD

The proposed model -325.740 -319.055 -315.075 -307.581 -315.879 5.187

CFEP -320.050 -288.650 -274.283 -194.001 -271.588 27.505

FEP -323.035 -316.071 -312.091 -286.222 -310.618 9.904

values over time on a convergence graph (Fig. 3.3). In the figure, we plot the fitness values recorded overtime at every cycle of 1000 evaluations. The graph shows that both CFEP and FEP converge quickly to their optima in the first 10000 evaluations, then stagnate. Meanwhile, the proposed model gradually approaches the optima. The fitness variance graph indicates that for the proposed model, the variance between the runs gets smaller over time, while for the others, this value quickly becomes unchanged. We note that the fitness is measured on the phenotype space. Thus, for CFEP, a fixed variance implies that the individual populations do not change. As for FEP, the mutation step somehow becomes low, thus solutions move around in a valley or a small neighborhood.

38


Fig. 3.3 Convergence graph: fitness and its variance (for 25 runs) over time.

Fig. 3.4 The averaged phenotype of the best individual overtime for the proposed model (left), CFEP (middle) and FEP (right). Each phenotype contains 10 variables.

This is more clear when we examine the vector of decision variables for the best individual over time (Fig. 3.4). In the proposed model, this vector changes frequently; indicating high genetic activity. Meanwhile for FEP and CFEP, stagnation is strongly apparent. The above phenomena can be explained by the way that the rotation directs the search to arbitrary local optima. CFEP and FEP rely only on the power of searching in the genotype space and have no control over the phenotype space. On the other hand, the proposed model can control the mappings by adapting them to the specific structures which facilitate the search by reducing the effect of rotation on the phenotypes. This seems to be the reason that the model gradually overcomes local optima to approach the global one. We hypothesize that the cooperation strategy also plays an important role in preserving the diversity in the populations over time. This diversity helps our system to get out of the local optima. For FEP, diversity is lost when it gets stuck at the local optima, as the mutation operator does not generate enough diversity to resist the effect of the rotation. Although, CFEP is supported by the cooperative ceevolutionary mechanism, it seems to be trapped in a local Nash equilibrium as stated in [308].


39

Fig. 3.5 The exploration of the phenotype space for the proposed model (left), CFEP (middle) and FEP (right): the lStline is the averaged values of 10 variables and the second is the corresponding variances.

In order to understand the dynamics of the system and to consequently explain the convergence of each technique, we investigated how the techniques explored the phenotype space. The averaged values of each variable in the phenotype space are recorded overtime and they are plotted in Fig. 3.5. We find that CFEP and FEP quickly explore a large part of the space and then remain fixed in this area, while in the proposed model, variables keep changing over time; thus the adaptive mapping gives the proposed model the ability to continue exploring the phenotype space. Our technique also shows very good exploration in the genotype space (Fig. 3.6).

3.4.3 Fitness landscape analysis In this section, we analyze the fitness landscape. This analysis helps us to understand the dynamics of the system [2]. The fitness landscape is constituted by the three components: genotype representation, fitness values and the operator to generate the neighborhood of related solutions. In our comparison, the three models have different techniques to build the fitness landscape; this is because they use different types of mapping: direct for FEP, fixed indirect for CFEP and adaptive mapping for our model.

40


Fig. 3.6 The exploration of the genotype space for the proposed model: the averaged values of variables (left) and their variances (right) over time.

Figure 3.7 depicts the fitness histograms of the respective models. The proposed model clearly has a better distribution in the fitness space, while the others focus only on a small area. The distribution of the proposed model implies that there is a good collaboration between mapping and genotype populations. It also maintains diversity from generation to generation in the genotype space, and therefore in the fitness space as well.

Fig. 3.7 Fitness histograms for our model (left), CFEP (middle), and F E P (right)

Further, we have employed entropic measures from information theory to analyze the landscapes [402]. The entropic measurement not only determines the shapes of landscapes, but also helps to identify the diversity of the populations overtime. The information contents (IC) (based on Shannon entropy) is used as a measure of the ruggedness of a landscape, and the partial information contents (PIC) measure is used to scrutinize the modality of the fitness paths (time series). All obtained information is given in Table 3.3. The parameter E in the table is used to determine the measures on different levels of the flatness of the landscape. If E =0, the calculation of measures is very sensitive to the difference between fitness values. If E is maximal (the difference between the maximal and minimal fitness values in the path), the landscape is seen as flat and the measures become zero.


41

Table 3.3 Information contents (IC) and partial information contents (PIC) for the test problem using the proposed model, CFEP and FEP. E

0 100 400 1000

The proposed method IC PIC 0.476 0.379 0.247 0.202

0.619 0.143 0.088 0.067

IC

CFEP PIC

0.425 0.119 0.001 0

0.177 0.015 0.00003 0

IC 0.294 0.098 0.002 0

FEP PIC 0.026 0.005 0.00006

0

Obviously, for all E values, our model always has a greater value for its IC. This means that the proposed model's landscape is more rugged than that of other models. It shows the strength of the proposed model's capability to explore the search space. By having this capability, the model must maintain a certain level of diversity in the phenotype space. That is why in the phenotype exploration graph, the proposed model's variables keep changing overtime. Meanwhile, CFEP and FEP quickly explore the phenotype space and remain unchanged. Therefore, their landscape's ruggedness are effected. As E increases, their landscapes are quickly flattened out: with E = 1000, their information contents become zero, while the proposed model's value is 0.202. This fact also indicates the higher value of the information stability of the proposed model's landscape in comparison with the others. It is worth noting that FEP has the worst value of its information contents. This is because FEP does not have support from the co-evolutionary process in contrast to that our model and CFEP do. The exploration capacity of the algorithms has also been assessed by the ability of each algorithm to discover local optima. This concept can be quantified by using the partial information contents (PIC) measure. In general, PIC helps to identify the modality of the landscape. The proposed model shows a strong ability to explore local optima. 0.36

I

Fig. 3.8 The averaged entropy value (left) and its standard derivation (right) obtained over time for the proposed model, CFEP, and FEP.


42

Lastly, we investigated the values of IC over cycles of time using 1000 evaluations for each cycle (Fig. 3.8). All algorithms started with almost identical values of information content ( x 0.4).In the first few time cycles, while the proposed model keeps this value quite stable, CFEP and FEP increased quickly. However, these values dropped sharply in the cases of CFEP and FEP (especially, FEP dropped to nearly zero) as time passed. Meanwhile, the proposed model continued to stay stable and even increased at the end. This fact once again confirms our above analysis that CFEP and FEP quickly become trapped in local optima and lose diversity. For the proposed model, the adaptive mapping drives the search mechanism to escape the local optima and approach the global one. The figure showing their variances is evidence for the above finding. For CFEP, the variance stays fixed after being trapped in a local optima. This indicates that the individual populations do not change. As for FEP, this figure reduces significantly over time. The reason is that the mutation step becomes low somehow, thus it cannot help FEP to escape from the local optima. 3.5

Conclusion

In this paper, we proposed a new model of adaptive mapping to solve rotated problems. GPM was implemented by using cooperative coevolution on two populations: one is the population of genotypes and the other is the population of matrices or mappings. The phenotype was generated by transforming selected genotypes by selected matrices. To validate the model, we compared its performance with that of the conventional FEP and a version of cooperative co-evolutionary FEP on the rotated Rastrigin problem. The results show that with the help of adaptive mapping, the proposed model clearly outperformed CFEP and FEP in all aspects such as the best achieved fitness value, convergence, and the diversity. We have also carried out an analysis on the fitness landscape to verify these findings. In future work, we will continue to improve the representation and computational cost of the mapping.

Acknowledgement This work is supported by the University of New South Wales grant PS04411 and the Australian Research Council (ARC) Centre for Complex Systems grant number CE00348249.

Chapter 4

Approaching Perfect Mixing in a Simple Model of the Spread of an Infectious Disease D. Chu and J. Rowe School of Computer Science, The University of Birmingham, B15 2TT, Birmingham, UK E-mail: (D.Chu, J.E.Rowe) @cs.bham.ac.uk In this article we present an agent-based simulations of the spread of a vector borne disease in a population with limited mobility. The model assumes two types of agents, namely “vectors” and “people agents” ; infections can only be transmitted between agents of different type. We discuss how the infection levels of the population depend on the mobility of agents.

4.1

Introduction

Recent outbreaks of infectious diseases, such as the SARS virus and the avian flu in south east Asia or recurrent outbreaks of Ebola in Africa underline the need to understand how diseases spread in a population. A major practical problem connected to those diseases is how to contain local outbreaks and prevent them to cause a global pandemic. A major problem here is of course the global mobility of people, especially infected people. A commonality of the above mentioned diseases is that an infected person can directly infect another susceptible individual through some type of interaction (what type depends on the pathogen in question). The focus of interest of the present contribution is another type of infection that requires a mediating agent; so called vector borne diseases cannot directly be transmitted on from one infected individual to another, but require a vector (typically an insect of some sort) as an intermediate carrier of the pathogen. An example of such a vector borne disease is Malaria. Malaria parasites 43

44


are passed on from one person to a mosquito if the mosquito feeds on the person’s blood. Similarly, the next person to be bitten by the mosquito will then get infected. Malaria (and other vector borne diseases) are important killers far outstripping more widely publicized diseases such as Ebola or AIDS in the number of fatalities they cause. There is a large body of mathematical theory modeling how infectious diseases (vector borne and otherwise) spread in a population. Mathematical approaches to model the spread of diseases usually assume that the population mixes perfectly. In the context of a discrete time model perfect mixing means the following: At every time step the probability that two members of the population meet is independent of who they encountered in the previous time steps. Clearly, real populations normally do not fulfill this criterion, but are spatially structured. In practice perfect mixing nevertheless turns out to be a very good approximation. This is so because a limited amount of local mixing might be sufficient t o generate the global effect of perfect mixing. While it is fairly well understood how diseases spread in a perfectly mixing population, there are relatively few attempts to study the spread of diseases in populations where this condition does not hold[211; 226; 2951, that is in population where the local mixing is not sufficient for the population as a whole to approach global perfect mixing. In such cases the mathematics necessary to describe the systems tends to become very involved. In those situations, agent-based computer simulations[72; 197; 1931 are a valuable tool, as they are very adept a t modeling populations with limited mobility. In this article we will describe a simplified model of the spread of a vector borne disease in a population. The aim of this article is to investigate how the infection levels in the population depend on the mobility of the agents. We describe our model in section 4.2; section 4.3 presents previous results describing a mathematical result of how the model behaves in the perfect mixing case. Section 4.4 thereafter describes the results of computational simulations for the case of limited movement. We provide a discussion and conclusion in sections 4.5 and 4.6 respectively.

4.2

Description of the Model

The model we describe in this article is not meant to be realistic with respect to the behavior of any real system. Instead we aim to study how infection levels depend on agent mobility in a bare-bone model of the spread of a vector borne disease. Once this basic understanding of the maximally simple case is reached, it will be possible to add further detail to the model.

Perfect Mixing in a Model of the Spread of a n Infectious Disease

45

ABMs are best described in order of their major components: 0 0

0

Environment Agents Interactions

The environment of the model is a 2 dimensional continuous space of size 2L x 2L, where L is measured in some unit. The environment has no features other than that it provides a topology for the agents. There are two types of agents in the model, “vectors” and “people”. Those agents are mainly distinguished by their infection period, movement rules, and mode of infection transmission. An infection can only be transmitted between agents of different type. The number of agents of each type is held constant during a simulation run. Agents are thus characterized by their position in the environment and by their internal state, which can be either “infected” or “healthy”. At each time-step the agents take a random position within a square of linear size 2M centered around their current position. M is a parameter of the model and set independently for people and vectors; throughout this article we will refer to M as step-size of the agent. Movement is always subject to the constraint that the agents stay within the boundaries of the environment. The only form of interaction between agents is transmission of an infection. At each time-step each vector interacts simultaneously with all people agents that are at most one unit away from it. If the vector is infected then all agents it interacts with will also be infected from the following time-step on. If the vector is not infected, but at least one of the people in its “bite area” is infected, then the vector will be infected from the next time-step on. In all runs presented here the bite area is a circle of radius 1 centered around the vector. Throughout all simulations presented here, vectors keep their infection for two time steps and people keep their infection for 40 time-steps. However, the model has re-infection, that is whenever an agent interacts with another infected agent, while it is already infected, then its remaining infection period is reset to its full value. So, for example, if a people agent is re-infected 39 time-steps after it has been infected the last time, then it will still have to wait for 40 more time-steps until it loses its infection again.

4.3 Behavior of the Model in the Perfect Mixing Case Standard epidemiologicalmodels are mostly dealing with the case of what is called “perfect mixing.” In the case of agent-based models perfect mixing


46

V P V

P

total number of vectors total number of people number of infected vectors number of infected people

R, time for vector to recover

RP b W

time for person to recover biting area of a vector world size (area)

+

is achieved if for each agent it is true that its position at time t 1 is independent of its position at time t. In the real world perfect mixing is hardly ever realized. Many systems are nevertheless well approximated by a perfect mixing approach: The present system seems to behave as though it perfectly mixes for relatively modest step sizes. If the agents’ step size is 9 (or greater) the infection levels are negligible different from the perfect mixing case. This observation presumably generalizes to a large class of systems. Hence, examining models that assume perfect mixing is useful also for systems that are relatively far from the ideal of perfect mixing. In [79] we showed that for the perfect mixing case the equilibrium of the present model is described by the following set of equations: 21 = 1 - (1V

P -1-(1P

(4.1)

Here the function q(z) is defined as: q(z) = 1 - exp(-bz/(W))

Figure 4.1 shows some simulation results for the perfect mixing case; in this case we realized perfect mixing by assigning to each people agent at each time step a random position. Figure 4.1 shows the proportion of agents that are infected for various system and population sizes. As expected, the infection levels vary from zero (no infection) to one (all people agents infected). The infection levels of vectors (data not shown) are similar. A thorough discussion of the perfect mixing case including a discussion of how the infection spreads for vectors can be found in [79].

47

Perfect Mixing in a Model of the Spread of an Infectious Disease

0 50

100

150

200

250

300

350

400

# people agents x i 0

Fig. 4.1 The proportion of infected agents for various systems sizes. Along the xaxis the size of the people population is increased. The number of vectors has been kept constant at 10000 in all simulations. Every point marks the infection level in the population once the equilibrium has been reached; each data point has been obtained from the time average over 1000 time-steps of one simulation run of the system.

4.4

Beyond perfect Mixing

In this section we will describe the spread of a vector borne disease in a population that does not perfectly mix. 4.4.1

No Movement: The Static Case

The next simplest case to perfect mixing is the case where there is no movement at all. In the context of the present model it would make no sense to keep both agents and vectors fixed in space. Throughout this article, we will therefore only vary the mobility of the people agents while keeping the mobility of vectors constant at step size 1. In this subsection we will consider the case where the agents do not move at all; we will henceforth refer to this case as the static case. Generally, the infection levels in this case are substantially lower than in the case of perfect mixing. For the population densities considered here we observe substantial infection levels only for the smallest system size ( L = 100). When we increase the system size to L = 140 the infection level never surpasses 0.2. For even greater systems, the infection never

48

Recent Advances an Artificial Lije

1.2

Syssize: I W SysSize: 140 SysSize: 1M) SysSize: 170

-

+ X X 0

-

++++++++++++ ++++++++

0.4

t

++ +

++

+++ ++ +

+++ I

0.2

0

50

I00

150

200

250

350

400

# people agents x10

Fig. 4.2

Same as figure 4.1, but the agents are not allowed to move.

establishes itself. 4.4.2

In Between

The comparison between the perfect mixing case and the static case showed that there is a strong quantitative difference in the infection levels in the population between the perfect mixing case and the static case (i.e. no agent movement). In this section we will investigate the behavior of the system in between those two extremes. The question we will ask is how the system approaches perfect mixing. Figure 4.3 illustrates the approach of the system to perfect mixing as the mobility of the agents increases. The first observation is that for small increases of the step size (up to a step size of 3) the infection levels increase as the step size increases. In the static model and a world size of 170 the infection cannot establish itself in the parameter range considered here. Once people agent mobility is introduced the infection levels increase up to a step size of 3; at this point the infection levels are clearly higher than in the perfect mixing case, particularly for low densities of people agents (fewer than 3000 people agents; see figure 4.3). Increasing the step size beyond 3 will not lead to a further increase of the infection levels. Figure 4.3 shows that the infection levels in the perfect mixing case are lower than the infection levels in the case of a step size

49

Perfect Mixing in a Model of the Spread of a n Infectious Disease

1.2

step=l

-

step=3

+

X

step=5 step=g step=pei-f. mix

1 -

-

0.8 -

0.6 -

0.4

-

0.2 -

0 #people agents x i 0

Fig. 4.3 The transition from the static model to the perfect mixing model in a system of size 170.

1. The same qualitative dependence of the infection levels on the step size was observed for a wide range of system sizes (data not shown). The step size at which the infection levels begin to decrease depend on the specific parameter settings. 4.5

Discussion

The dependence of the infection levels as shown by the present model is surprising. In this section we will explore possible explanations for the observed effect. The low infection levels in the static case (if compared to the perfect mixing case) are readily explained by the following observation: In the case of no agent movement, a vector can only infect an agent, if the following two conditions are fulfilled: (1) It picks up an infection from another people agent (2) It can travel the distance from this infected agent to the people agent in question without losing its infection Given the restrictions on the movements of vectors (step size l),in the present model the second condition can only be fulfilled if the distance

50


Cluster Size

Fig. 4.4 The distribution of clusters for three system sizes. In all experiments there were 3000 agents in the system.

between the people agent to be infected and an already infected people agent is at most 3 units. This is so because the vector remains infected for only 2 time steps if no re-infection occurs; hence it could not carry an infection over distances larger than 3 units. This then leads to a natural definition of clusters of neighboring agents: We say that two agents are neighbors if they are at most 3 units away from each other; two people agents are then defined to be in the same cluster if they are neighbors. In the case of no movement, the spread of the disease is restricted to clusters. If two agents are not in the same cluster, by definition one cannot cause the infection of the other, neither directly nor indirectly. Conversely, once there is neither an infected people agent nor an infected vector left in the cluster, the cluster will remain free of infection for all times. The graphs in figure 4.4 show the distribution of cluster sizes for a system of 3000 agents. For these parameters the clustering is sub-critical (see [159;251) and the cluster size distribution is exponential. For the smallest system size ( L = 100) the maximal cluster size of about 60 is considerably larger than in the case of the larger systems where clusters are never bigger than about 25. Also, by far the majority of all clusters in the larger system consist of only very few people agents.

Perfect Mixing in a Model of the Spread of an Infectiow Disease

,

.

Histogram:Penod spent In neighborhoodof at least one agent

,.....,

. . ......,

.

. ......,

.

.

stepsize: 0.1

.......

+

step-size: I step-size: 9

51

A

. .

:

A IWOW n r

g 5

1WW

1000

I

* 100

10

1

Length of period

Fig. 4.5 The distribution of the contiguous periods spent in the neighborhood of at least one agent. The system size is L = 170.

In the case of no movement of the agents it can be expected that the infection can only be sustained in the larger clusters. Smaller clusters, particularly clusters of size one, can in principle sustain an infection, but will be much more vulnerable to random fluctuations; those will over time wipe out infections in smaller clusters. Hence, the main contribution to infections mainly comes from the large clusters. The larger the system size, the smaller the population density and maximal cluster size; furthermore, the proportion of the population contained in large clusters increases. The change of the cluster size distribution is therefore the main reason for the rapid fall of the fraction of infected agents with increasing system size. The density of agents in the case of L = 170 is too low to sustain an infection if people agents are not mobile. This is in strong contrast to the perfect mixing case (fig 4.1) that shows considerable levels of infection in the population in the corresponding parameter range. We will now discuss the transition from the static case to perfect mixing. There are two antagonistic effects that determine the infection in the case of a non-static population. Firstly, as discussed above, if agents do not move, then the infection cannot spread to larger parts of the population, thus essentially limiting the possibilities for the spread of the infection. As the step size increases, agents have more opportunity to find themselves in the neighborhood of other agents, and therefore also in the neighborhood of other infected agents. Overall, this greatly facilitates the spread of the disease. Hence, from this perspective one would expect that agent mobility is positively correlated with the infection level in the population.


52

.

lW000

, . . . . . I

1

.

. . . . . . I

,

. ......,

,

,

step-size: o 1 slep-slze: 1 step-size: 9

10000 -

p

-:

......

+

2

.

AA A 1000

:

+

Im 100

-

* 10 -

1

-

10000

Fig. 4.6 Lengths of periods spent outside the neighborhood of at least one infected agent. The system size is L = 170.

On the other hand, increased movement also leads to a decrease of the typical number of contiguous time units spent in the neighborhood of a specific agent. More precisely, the overall time spent in the neighborhood of a specific agent is independent of the step size (for simulation times large enough), yet the contiguous time periods spent in the neighborhood of a specific agent strongly depend on the step-size. Figure 4.5 shows, for different step sizes, the distribution of the times agents have at least one neighbor. In the case of very limited mobility (step size of 0.1) agents spend very long time periods in clusters; in the present simulations they spend up to the entire duration of the simulation in a cluster. At a step size of 1, agents are more mobile. Hence, correlations between successive locations decrease and clusters of agents are shorter lived. At this degree of mobility agents spend in a cluster is maximally of the order of magnitude of 100 time steps. At a step size of 9 the same number is under 20 (see figure 4.5). It should be noted, however, that the overall time spent in the neighborhood of at least one other agent is independent of the step size. The effect of shorter periods in the neighborhood of agents is thus counteracted by more frequent visits with no net change (taken over sufficiently long simulation periods). The time spent in clusters is thus not relevant for the change of infection levels the step size varies. A related measure is a better indicator of what is going on. Figure 4.6 shows the distribution of the time periods spent outside the neighborhood of infected agents. This is relevant because infection transmission is

Perfect Mixing i n a Model of the Spread of an Znfectiow Disease

53

limited to areas in the neighborhood of infected agents. Figure 4.6 shows for the case of a system of size 170 and 1500 people agents that both the overall time spent outside a cluster and the distribution of the times spent outside the neighborhood of an infected agent increases with the step size. This explains the decrease of the infection levels as the step size increases. The explanation is as follows: Outside the neighborhood of infected agents, agents cannot get infected; on the other hand, agents remain infected only for a fixed amount of time steps. Extended periods outside the neighborhood of other infected agents will therefore necessarily lead to a loss of infection. 4.6

Conclusion & Future Work

In this article we reported some simulation results of how a vector borne disease spreads in a population of mobile agents. We found that for moderate levels of agent mobility the infection levels are highest, decreasing both if the mobility is increased and decreased. We also provided a qualitative explanation for this effect. Future work will need to provide a quantitative explanation for this effect. A mathematical model of mobile agents that do not mix perfectly is needed in order to further elucidate the nature of the observed effect.


Chapter 5

The Formation of Hierarchical Structures in a Pseudo-Spatial Co-Evolutionary Artificial Life Environment D. Cornforthl, D. G. Green2 and J. Awburn3 ‘School of Information Technology and Electrical Engineering, UNSWQADFA, Canberra ACT 2600, Australia. Email: d. cornforthQadfa.edu.au. Faculty of Information Technology, Monash University, Clayton VIC, 3800, Australia. Email: [email protected]. School of Environmental and Information Sciences, Charles Sturt University, PO Box 789, Albury, NSW 2601, Australia Enumeration of the factors underlying the formation of modules and hierarchical structures in evolution is a major current goal of artificial life research. Evolutionary algorithms are often pressed into service, but it is not clear how the various possible features of such models facilitate this goal. We address this question by using a model that allows experimentation with several candidate model features. We show how the notions of variable length genotype, variable genotype to phenotype mapping, pseudospatial environment, and memetic evolution can be combined. We quantify the effects of these features using measures of module size, and show that information shared between individuals allows them to build modules and combine them to form hierarchical structures. These results suggest an important role for phase changes in this process, and should inform current artificial life research.

5.1

Introduction

One of the key questions in the study of Artificial Life is to understand “open-ended complexity”. That is, how do increasingly complex structures 55

56

Recent Advances in Artzficial Life

and behaviour arise in natural systems? In particular, is it possible to capture this phenomenon within a simulation model [34]? Complex behavior arises from interactions between simple elements of a system [156], and includes clustering, modularity, and phase changes. Many models have successfully demonstrated emergent complex behavior from simple systems, for example, cellular automata [427] and Tierra [328]. These models have been able to demonstrate emergent effects such as the spontaneous appearance of parasitic organisms. However, they have generally not examined the interaction between open-ended evolution, local phenomena and global constraints. In particular, evolutionary models, which are often used in such investigations, are often limited to a fixed-length genotype. In contrast Harvey [180] suggested that a variable length genotype is essential for open-ended evolution, and Angeline and Pollack [15] have shown the efficacy of the same for the emergence of modules, and their combination to produce higher levels of problem abstraction, in conjunction with a co-evolving representation language. Many evolutionary models employ a fixed genotype to phenotype mapping, while deJong [lo41 has shown the advantages of a variable mapping, and its role in building modularity. Furthermore, Channon and Damper [76] have suggested a co-evolving environment as a necessary feature for the discovery of truly novel behavior. Green has shown [157] that phase changes have an important role in explaining how more complex structures arise in complex systems. In his book, The Blind Watchmaker, Richard Dawkins [92] illustrates the power of natural selection to produce complex results through the example of a child typing Shakespeare. Taking the line “methinks it is like a weasel “ out of the play Hamlet, he points out that the chances of a child typing this line are virtually zero. However, if each time the child types a correct letter, and that letter becomes fixed, then the child’s typing will quickly converge on the targeted line. However, this model has selection directed towards a fixed target. Natural evolution does not proceed in such a simple linear way, but behaves more like a growing bush [153], with branching and pruning producing a host of different forms. Modularity is one of the most prevalent ways of organizing complexity, both in natural and artificial systems [158]. Modules provide building blocks that simplify the construction of large systems. Plants for instance, are built of repeating modules including branches, buds and leaves. By restricting interactions, modules also simplify control of large complex systems. In order to provide a test bench for the investigation of open ended evolution, we developed the Weasel World model [86]. In that work, we were

Hierarchical Structures in a Co-Evolutionary Artificial Lije Environment

57

able to demonstrate the development of phase changes, clustering and selforganization, as well as showing the importance of the interaction between local phenomena and global constraints. In this work, we go further and investigate how modules can be combined into hierarchical structures. In the next section, we describe the Weasel World model. Section 5.3 describes the experiments and section 5.4 the results. Section 5.5 summarizes our conclusions.

The model

5.2

Weasel World is an implementation of an evolutionary algorithm featuring a fixed size population Pgof individuals placed in a one-dimensional spatial environment. Many environments could be suggested, but because of our inspiration from the work of Richard [92], the environment consists of the entire text of Shakespeare’s Hamlet [350],with upper case characters converted to lower case, and periodic boundary conditions to avoid edge effects. Individuals are spatially located within the text, and evolve to match their phenotype to the local text. An important feature of our model is that there is no fixed goal or ‘kolution”, but individuals are free to match as much text, and to grow as large, as they are able. In this (limited) sense our model is “open ended”. The model consists of the following components: a genotype representation a genotype to phenotype mapping a selection mechanism reproduction and genetic operators a co-evolving memetic population a pseudo-spatial environment We have adapted these features to suit the purposes of this work, as well as adding the concept of territory of individuals, and allowing the population Pg to share knowledge discovered about the environment by contributing to and accessing a shared information repository, similar to the notion of memetic evolution.

5.2.1

Genotype t o phenotype mapping

As individuals evolve to match the text at their location, the phenotype is represented as a sequence of symbols pi drawn from the same set H as those for the text of Hamlet. The set H includes all the letters of the English alphabet, plus punctuation and the space character,

58


H = { a .. . z , .‘ - [I; ?!() : ”&} , where the ellipsis . . .indicates all the characters between a and z. For a phenotype of length n,we define the phenotype to be of the form 071 , p 2 , . . . ,pn): pi E H . The obvious representation for the genotype is also a string of symbols with the same definition, so that the genotype to phenotype mapping is the identity operator. However, in this new version of our model we introduce a variable mapping by using instead the set of integers K from zero up to some limit Nk. A symbol in the genotype gi E H is mapped using the ASCII numeric to character conversion, while a symbol gi $ H undergoes a variable mapping defined by an individual in a co-evolving memetic population, if a suitable individual exists. Otherwise, the symbol is regarded as “junk DNA”, and is not represented in the phenotype. For example, if the genotype consists of the integers {119,2,10,115,101,108}, this is mapped as follows. The first, 199, is a member of H so the ASCII mapping is used and the phenotype becomes w. The next, 2, is not a member of H . However, assume that an individual having a code of 2 exists in the memetic population, and represents the mapping 2 -+ ea. The phenotype becomes wea. Assume that integer 10 has no individual in the memetic population, so is not expressed. The remaining integers in the genotype are ASCII codes for lower case letters s , e , l , and are simply copied into the phenotype to produce the complete string weasel. This mapping means that genotype length and phenotype length may be (and often are) different. In our model, individuals have a fixed genotype length, but their offspring may acquire a longer genotype by adding a random gene during reproduction (within certain constraints described below). Additionally, offspring may have a shorter genotype length if the environment is unable to sustain them. The evolutionary process thus may lead to individuals of differing genotype lengths. In turn this means variation in phenotype length, with longer phenotypes able to achieve greater competitiveness by discovering potentially more building blocks. 5.2.2

Selection mechanism

Selection of individuals to reproduce is implemented using tournament selection, with the requirements of a fixed population size. An individual is chosen at random from Pg and examines its territory. If there are no other neighbors (individuals in its territory), it reproduces asexually. This means that there is no selection mechanism when individuals are small or located away from each other. If the individual has one neighbor, they mate to produce two offspring. If there is more than one neighbor, two are selected at random and compared

Hierarchical Structures in a Co-Evolutionary Artificial Life Environment

59

using two measures. The first is the length of the largest block of contiguous matches at any position in the rival’s local environment or territory. For example, a phenotype “metweel” with local environment “methinks it is like a weasel” will score only 3 matches for “met”, as the longest contiguous block. The second measure is the number of matches divided by the phenotype length. Either of these measure may be normalised as follows:

where ~ p 1is the measure of the first individual. Competition proceeds using a modified Pareto technique as follows: If one rival is dominant in both measures it wins. If one measure is equal in both rivals, comparison is based on the other. If neither rival is clearly dominant, both measures are normalised, using equation 5.1. The measure with the largest normalised value is selected, and rivals are compared on that measure alone. If there is still no winner after the preceding comparisons, one of the rivals is randomly selected. The first individual selected mates with the winner of the tournament, producing two children. This process is repeated until the new generation reaches the fixed size. In the case of sparsely distributed individuals with small, nonoverlapping territories, reproduction will be completely asexual. However, as the phenotype length of individuals increases, their territory size will increase so that they gradually come into contact with others. The reproduction will become a mixture of asexual/sexual and then finally entirely sexual as sub-populations merge. 5.2.3

Reproduction and genetic operators

Mutation occurs by replacing a randomly selected gene with a random integer selected from the set K . Rather than a fixed mutation rate, the rate is specified relative to genotype length, which Harvey [5] suggests is more appropriate for variable length genotypes. In our model, the mutation rate is the number of mutations per genotype, rather than a fixed rate. For example with a genotype length 5 and mutation parameter 1,each gene will have a 1 in 5 chance of being mutated. A standard single point crossover was used for sexual reproduction. The increase-length operator, suggested by Harvey [5],increases genotype length by exactly one character. A random gene is generated and


60

inserted into the offspring at a random position in the genotype. The decrease-length operator removes a random gene from the offspring at a random location. 5.2.4

Memetic evolution

We wish to study the formation of modules and their combination, so here we define a structure for modules. It should be noted that we do not define what the modules will contain or how they combine to form larger structures, but leave that up to the evolution of the model. We define a module as any group of two or more adjacent symbols in the environment, and facilitate the discovery of frequently occurring modules by maintaining a list separate to the population. This is similar to the co-evolving representation described in [ 6 ] . Modules are stored in a fked size population of-memesP,, that evolves over time, where individuals are subject to competition and selection. This is distinct from the genetic population Pg. Memes consist of a module plus a unique integer key. When a gene from an individual in Pg matches a key in P,, the module is added to its phenotype. At each generation, individuals in Pg identify the largest contiguous text fragment matched by their phenotype. This becomes a new candidate individual for the memetic population P,, along with a fitness value for insertion given by the length of the fragment multiplied by the number of times it was encountered within the territory of the individual. The memetic population Pm reproduces a new generation by selecting memes from Pgwith the highest insertion fitness value. This is illustrated in Figure 5.1. Memes from P, are used during the genotype to phenotype mapping, if a match is found between the gene and the key associated with the meme. The memetic population also loses memes by deletion, according to a deletion fitness value, defined as the length of the fragment times the number of times it has been used for mapping. Genetic population

I

Memetic population P,

Pg

I

Genotype: 19 10 I 3 2 I l l 9 12 I115 I101 I108 Phenotype: “they weasel” Territory: “of ot&r nations; thev clip us’’ Longest contiguous match: “the”, frequency 2

kev module

B-B

Fig. 5.1 Memes are selected from the population according to their fitness


5.2.5

61

Global parameters

Two global variables affect the evolutionary process: nutrient and territory ratio. Nutrient represents a global common resource in the environment, and is implemented in the model as a single variable that is increased by a fixed amount at each generation, and reduced according to the total of the phenotype length of all individuals in the population Pg.This represents individuals from Pgconsuming resources. If the nutrient amount falls below zero, reproduction entails a reduction in size of the genotype at a rate controlled by a parameter. Otherwise, genotype lengthening takes place at reproduction. Territory ratio controls the size of the territory relative to the individuals’ phenotype length. An individual exists in the centre of the territory, i.e. if the phenotype length is 5 and territory ratio 100, the territory will consist of 250 characters to the left of the position, the 5 characters it currently occupies, and 250 characters to its right.

Fig. 5.2 Illustrating the territory of individuals of varying phenotype length.

Given the different length phenotypes, territory size may be different for each individual. When the territories of two individuals grow large enough for them to meet, they are able to commence sexual reproduction, and competition for mates. This is illustrated in Figure 5.2. The individual with phenotype “deedh” is able to find one neighbor with “anfhg”, so reproduces with it. The individual with phenotype “anfhg”, although having the same size neighborhood as “deedh”, has two neighbors, due to the juxtaposition of individuals in the environment. In this case, the two neighbors would compete for the right to reproduce. The individual with phenotype “thejgknmhh” has more neighbors due to its larger territory size.

5.3

Experiments

Our research question is to determine the effects of various features of the model upon the size of modules formed, and the types of higher order structures. The features to be assessed are variable length genotype, variable genotype to phenotype mapping, memetic population and interaction with global parameters. To quantify the effect of these on module formation,


62

Table 5.1 A summary of the parameter values used in the four experiments.

ExDeriment 2 3 4 1.0 1.0 1.0 5 5 5 1.0 1.0 1.0 10 10 10 0 100 100 0 0 10,000 I

Parameter Lengthening rate Initial no. of characters Mutation rate (per genotype) Territory ratio Size of memetic population Nutrient

1 0 100 1.0 10 0 0

~

~

~~~~~

a measure is needed that can be applied to compare the operation of the model with or without these features. A direct measure of the size of and distribution of modules is inappropriate, for example, since it cannot be measured when the memetic evolution is disabled, as no modules will be produced. In this work we have chosen to measure the size of modules formed using phenotype length per gene, and the usefulness of those modules using the number of matches per gene. These measures are proxies for the average module size and frequency of use. All experiments were performed with a population Pg of 100 individuals placed at a random location within the text: Periodic boundary conditions were used, so that the end of the Hamlet text was considered to be adjacent to the beginning for the purposes of calculating proximity. All experiments consisted of 500 iterations, and were repeated 10 times. The first experiment is to establish a baseline, and uses a fixed length genotype, fixed genotype to phenotype mapping, no memetic population P, and no nutrient effects. In order for the results to be comparable with other experiments with variable length genotype, the genotype length was set to 100. In the second experiment, we introduced the variable length genotype. In the third experiment, we introduced the variable genotype to phenotype mapping, and a memetic population. In the fourth experiment, we introduced a nutrient limit. Parameters for these experiments are shown in Table 5.1. 5.4

Results

The results of all four experiments are summarised in Table 5.2, which shows mean and standard error of selected measures at the conclusion of experimental runs. Experiment 1 is a baseline test with no variable length genotype, no variable mapping and no nutrient. Figure 5.3 shows phenotype length per


63

Table 5.2 A summary of results, showing mean (standard error) for each measure. Experiment 4 clearly shows increased values for the size of modules formed (phenotype length per gene), and the usefulness of those modules (matches pre gene).

Experiment Measure genotype length phenotype length pheno. length / gene matches matches per gene

1 100 (0) 34.0 (0.8) 0.34 (0.008) 1.906 (0.036) 0.019 (0.00036)

2 505 (0) 170.0 (3.16) 0.337 (0.006) 2.0 (0) 0.00396 (0)

3 505 (0) 773.5 (30) 1.53 (0.059) 2.0 (0.0034) 0.00397 (0)

4 3.2 (0.53) 24.5 (4.18) 7.86 (0.62) 7.64 (0.81) 3.46 (0.95)

gene and matches per gene, averaged over 10 repeated runs of the model. There are no observable trends: these values reflect the underlying probability of 41 valid letters selected from 121(0.34), and of finding 2 contiguous letters in the individual’s neighborhood with 100 genes. Any increase in either measure due to selection is balanced by mutation. In this experiment there was no opportunity for building modules or forming higher structures. In the second experiment, the variable length genotype was introduced, and the genotype grew at the rate of one gene per iteration, reaching a final value of 505 (Table 5.2). The phenotype length reached 170, as the phenotype length per gene was roughly constant at around 0.34 (Figure 5.4). The number of matches quickly increased from near zero to 2 (Table 5.2), as favorable amendments suggested by the lengthening operator were selected. Subsequently, the mutation, selection and lengthening operators balanced to keep this constant, so the matches per gene then diminished (Figure 5.4). Adding a variable length genotype by itself does not provide any opportunity for building modules or higher structures. In the third experiment we introduced a co-evolving representation in a memetic population. The genotype length grew as before, but the phenotype length was able to grow larger than in experiment 2, because of the presence of modules adapted to the environment (Table 5.2). This is reflected in a higher phenotype length per gene, shown in Figure 5.5. The number of matches, although rising more rapidly than experiment 2, did not achieve a higher value at the end of the run (Table 5.2). In fact the matches per gene had a behavior similar to experiment 2. Although the variable genotype to phenotype mapping succeeded in allowing the formation of modules, a limit on their size was very quickly reached, so this does not represent a convincing mechanism for building hierarchical structures. In the fourth experiment, we introduced a nutrient limit that had the effect of producing oscillations in the genotype length of individuals. Because of this environmental pressure, the genotype length at the end of the run was only 3.2 (Table 5.2). However, the phenotype length was 24.5,


64

1.6 a,

1.4

i1.2 i

1

f 0.8 - 0.6

-pheno/gene _-.. match/gene

1

I

T 0.16 0.14 o.12g 0.1

9)

0)

0.065

$ 2 0.4

n

:: Q)

0.08

0.04g

0.2 ..................................................................................

0

1 0

100

I

200

300

400

~- 0.02

10 500

Iteration Fig. 5.3 Results from experiment 1, using a fixed length genotype, fixed genotype to phenotype mapping, no memetic population Pm and no nutrient effects.

-phendgene .- -.matchlgene

1.6

0.16

1.4

0.14

Q)

C

$1.2

0.12

i

0.1

1

;

e 9)

0.08

0.8

9)

- 0.6 9)

0.065

$ 2 0.4

-,..-..-._ ..........

n

0.2

0.04g

-.._ .............

.........................................

0 0

100

200

300

400

0.02 0 500

Iteration Fig. 5.4

Results from experiment 2, after introducing the variable length genotype.

and the phenotype length per gene continued to rise during the run (Figure 5.6). Modules formed represent increasingly longer segments of the text, and provided the opportunity for higher order structures to form. The average number of matches reached 7.64 (Table 5.2), as matches per gene continued to grow during the run up to 4. Table 5.3 provides some examples of the five most used modules at

65

Hierarchical Structures in a Co-Evolutionary Artijcial Life Environment

1.6

0.16

1.4

0.14

al

3.2

0.12 2

g 1

0.1

:: al

0.8

0.08

-0 0.6

0.06g

2 0.4

0.04=

al

0)

(R

C

n

0.2

0.02

0 0

1 00

200

300

400

0 500

Iteration Fig. 5.5 Results from experiment 3, using a variable genotype t o phenotype mapping and a memetic population..

lo

-pheno/gene --.-match/gene

1

0

100

200

300

T5

400

500

Iteration Fig. 5.6 Results from experiment 4, after introducing the nutrient limitation.

the end of a typical result of experiments 3 and 4. The module includes the symbols inside square brackets, separated by c o m a s . Numbers refer to other modules, so the “end result” column provides a full decoding of the module (what would be added to the phenotype). Notice that the modules are generally larger for the results from experiment 4, suggesting that the phase changes experienced have enabled the formation of larger

66

Recent Advances an ArtiJcial Life Table 5.3 A comparison of some modules formed during experiments 3 and 4. Experiment 3 Module End result [35, g] “. g” “r.” b.1 ‘( P” [ PI [62, k] “nd! k” “g. ” [g, 351

Experiment 4 End result [49, hl “1 work on h” i82, 2i1 “ient as’’ [135, 641 ‘‘ work on hi” “ work on ” [75, 15, 541 [ 671 “ madnes”

I Module I

modules. Notice also the prevalence of symbols gi E H , which require further decoding, indicating hierarchical structures.

5.5

Discussion

One of the aims of artificial life research is to construct computer models that reproduce in some sense the complexity of biological systems [ 6 ] . Although evolutionary algorithms are often used, it is not clear which features are appropriate. In this work we have utilized a novel artificial life model to incorporate features that have not previously appeared together, allowing a comparative study of how they affect the formation of modules and hierarchical structures. This model incorporates a variable length genotype, a variable genotype to phenotype mapping, a separate memetic population encoding a collection of modules, and situates individuals in a pseudo-spatial environment to examine local effects between individuals as well as the interaction of global parameters on individuals. The results indicate that although the variable length genotype, variable genotype to phenotype mapping, and the co-evolving population of memes increase the size of modules formed, the most dramatic effect upon module size is caused by phase changes, where the model is forced to oscillate between growth and consolidation phases. These results have implications for the understanding of the formation of hierarchical structures in evolution. They imply a role in evolution of phase changes, although at present further work is required to identify exactly what systems this would apply to, and how it could be exploited. However, this should alert researchers to the consequences of choosing model features that can either restrict or enable desired model behavior. Finally, it has not escaped our notice that this model provides a potential research tool for investigating patterns within data of various kinds, including text and protein sequences.

Hierarchical Structures in a Co-Evolutionary Artzficial Life Environment

67

Acknowledgements This work was supported in part by a grant from Charles Sturt University under the Early Career Researcher scheme, The simulations were carried out on CSU’s Cluster Computing Centre. David Green’s work was supported by the Australian Research Council and the Australian Centre for Complex Systems.


Chapter 6

Perturbation Analysis: A Complex Systems Pattern N. Geard, K. Willadsen and J. Wiles

ARC Centre for Complex Systems, School of Information Technology and Electrical Engineering, The University of Queensland, QLD 4072 Australia. E-mail: { nic,kaiw,janetw}@itee.uq.edu.au Patterns are a tool that enables the collective knowledge of a particular community to be recorded and transmitted in an efficient manner. Initially developed in the field of architecture and later developed by software engineers [138], they have now been adopted by the complex systems modelling community [417]. It can be argued that, while most complex systems models are idiosyncratic and highly specific to the task for which they are constructed, certain tools and methodologies may be abstracted to a level at which they are more generally applicable. This paper presents one such pattern, Perturbation Analysis, which describes the underlying framework used by several analytical and visualisation tools to quantify and explore the stability of dynamic systems. The format of this paper follows the outline specified in [417].

Pattern name Perturbation Analysis Classification Dynamics, State Space Intent The Perturbation Analysis pattern provides a quantifiable measure of the stability of a dynamic system. Also known as None

69

70

6.1


Motivation

A complex dynamic system is one consisting of multiple elements, where the future state of the system is determined by a function f of its current state,

where s ( t ) is the state of the system at time t. The typical feature of interest of complex dynamic systems is their asymptotic behaviour as t 4 m. The set of states towards which a system converges under these conditions is known as an attractor. Attractors may be fixed points, limit cycles, or non-repeating ‘chaotic’ attractors. Systems may contain single or multiple attractors. The set of initial states of a system that converge to a given attractor forms the basin of attraction of that attractor. Complex dynamic systems are widespread: genetic networks, economies and ecosystems are all examples. One of the emergent features of these types of distributed systems is their robustness or stability. Systems in the real world operate in noisy environments and are subject to perturbations from a wide variety of internal and external sources. In many cases, despite short term fluctuations, the long term behaviour of these systems is remarkably stable. When modelling such systems, it is useful to be able to quantify this level of stability. For example, in a genetic regulatory system, where basins of attraction have been equated to cell behaviours [218], the stability of a system may reflect the phenomena of cell differentiation during development. Early in the developmental process, cells are sensitive to signals from their environment: transplantation experiments have demonstrated how embryonic cells can adopt the fate of their new neighbours rather than their original fate. As development progresses, the stability of cell types increases, and almost all fully differentiated cells will retain their original fate when transplanted [428]. The differentiation process itself is robust to fluctuations in external factors, such as temperature variation and nutrient levels, as well as internal factors, such as the stochastic nature of many genetic and cellular processes [225] The Perturbation Analysis pattern provides a general framework for measuring the effect of changes to a system’s current state on its long-term behaviour. These measurements may then be used as the basis for calculating more specific quantities, such as the rate of convergence or divergence of two nearby trajectories, or the probability of a perturbation causing a system to switch between different attractors.

PeTtUTbatiOn Analysis: A Complex Systems Pattern

6.2

71

Applicability

The Perturbation Analysis pattern requires a dynamic system, consisting Of:

a finite set of elements, e x h of which may take a discrete or continuous value; and a deterministic updating function.

0

0

The Perturbation Analysis pattern is useful in the following situations:

(1) A dynamic system is subject to some intrinsic or extrinsic perturbation, and it is desirable to stochastically or systematically explore and quantify the effects of these perturbations. (2) A dynamic system can settle down into one of several possible behaviours and it is desirable to know either the likelihood of a system reaching a specific stable behaviour, or the probability of a system switching from one stable behaviour to another. (3) A dynamic system is being used for prediction and it is desirable to know how far into the future its behaviour can be confidently predicted if there is some uncertainty as to its initial state. 6.3

Structure

The relationships between the classes involved in the Perturbation Analysis pattern are detailed in Figure 6.1. 6.4

Participants

State stores a state of the system s , represented as a vector the values of each of the n elements, s = (so,. . . , & I ) System applies an update function f to update the values of each element of a state, s(t

+ 1)= f(s(t))

Perturber applies a perturbation function p to create a new state from an old state in a systematic fashion, s' = p ( s )


72

Pelturber

System

-perturbFn: filnctioi

-@at eFn : function

tPerturb(s:Star.e): State

tUpdatels :State): State

State Get0

)-state: (-bcev tGet(1:

State +Set(s:State): void

)

Measurer I-conpar eFn: function

I

tConpare(s:State,s’:Statel: Distance Fig. 6.1 A class diagram describing- the types _ . of objects involved in the Perturbation Analysis pattern and the relationships that exist between them. Each object lists the private data variables it contains (indicated by a minus), and the public functions it provides (indicated by a plus), together with their arguments and return values.

Measurer quantifies the distance d between two states according to some metric rn,

d = m ( s ,s’) Concrete examples of the update and perturbation functions, and of the distance metric, are provided below, in the Implementation Section.

Perturbation Analysis: A Complex Systems Pattern

73

6.5 Collaborations

A dynamic view of the interactions between objects in the Perturbation Analysis pattern is shown in Figure 6.2.

I

A

Get

Update

Set Get

I

I

c

I I I I I

b

b

-L

b

Update

I I I

I I

Set

I I I

b

I

I

I I I I I I I I I

I I I I I I I I I

I

I I I I

Get

I

I

I

I I I

-

Distance

Fig. 6.2 A sequence diagram of the interactions between objects in the Perturbation Analysis pattern. Time runs vertically from top to bottom. The activation bars in the lifeline of each object indicate when that object is active in the interaction. Horizontal arrows indicate interactions - in this case function calls.

74


(1) Note that two States are maintained at all times: one ( s ) corresponding to the original system trajectory and another (s’) to the perturbed trajectory. (2) Perturber sets the value of the perturbed State according to the application of the Perturb function to the original State. (3) Measurer uses the Distance metric to calculate the distance between the original and perturbed trajectories. (4) System uses the Update function to advance each of the states by one iteration (or time step). 6.6

Consequences

The Perturbation Analysis pattern has the following benefits and limitations: (1) The pattern facilitates perturbation of a system and collation of distance measurements which can then be analysed using other methods. Two examples of the type of context in which the Perturbation Analysis pattern can be applied are provided below in the Sample Code Section. (2) The pattern allows for a range of perturbation functions, distance metrics and system updating functions, each of which can be varied independently. Examples of these functions and metrics are provided below in the Implementation Section. (3) Because the pattern only specifies a single iteration of the perturb and measure cycle, it supports the investigation of both annealed and quenched systems. In a quenched system, the structure of the system and the update function are static through time: measurements of a quenched system are specific to that particular instance of the system. In an annealed system, the basic parameters of the system (level of connectivity and type of updating function) are static, but the specific pattern of connectivity and set of updating functions are generated anew at each time step: measurements of an annealed system reflect basic properties of an entire class of systems. (4) One limitation of the pattern as described here is that it requires a deterministic system updating function. While there is no reason that the pattern could not be applied to a stochastic system, doing so raises several issues that have not been addressed here relating to the structure of state spaces and the nature of attractors (see, e.g., [lsl]).

Perturbation Analysis: A Complex S y s t e m Pattern

6.7

75

Implementation

The Perturbation Analysis pattern is generally applied as an iterative procedure. That is, a large number of perturbations and measurements are carried out in order to provide an estimation of the stability of a particular system or class of systems. A single iteration of perturbation and measurement may be described (in pseudocode) as follows: # Set the initial state of the system. State s = initialstate # Perturb the current state. State s ’ = Perturber.Perturb

(9)

# Measure the distance between the original and perturbed states startDist = Measurer.Distance ( s , s ’ ) # Update both the original and perturbed states. s = System.Update (s) s ’ = System.Update ( s ’ )

Measure the distance between the updated states. enmist = Measurer.Distance ( s , s ’ )

#

The main variables in this procedure are the nature of the update and perturbation functions and the distance metric. Each of these aspects may be varied independently.

(1) Defining an update function. The update function is defined by the dynamic system to which the Perturbation Analysis pattern is being applied. An example of an updating function in a discrete dynamic system is provided by Kauffman’s Random Boolean Network model [218]. In this model, the value of a state element, un,at time t 1 is some random Boolean function, fn, of its K inputs at time t ,

+

fn(t

+ 1) = fn(an, ( t ) ,. . .

1

CnK

(t))

An example of a continuous update function is the sigmoid function used in many neural network applications,

where x is a weighted sum of the inputs to a particular element.

76


( 2 ) Defining a perturbation function. Perturbation functions on discrete states can be either systematic or stochastic. A systematic perturbation function varies state elements systematically (e.g., by incrementing or decrementing an integer value, or by negating a Boolean value). A stochastic perturbation function varies state elements without regard for the sequential nature of the element values (e.g., randomly assigning a new integer value within the allowable range). A single application of either type of perturbation function will involve the alteration of one or more elements. The properties of the perturbation function are therefore: the number of elements being varied; and the mechanism (Le., systematic or stochastic) used in their variation. Several possibilities exist for perturbing continuous state element values. Unlike the discrete case, it is not possible to systematically explore the set of all possible perturbations. Therefore perturbations are generally applied in a stochastic fashion, by the addition of noise generated according to some distribution (e.g., Uniform or Gaussian) to some or all of the elements. The properties of the perturbation function that can be modified are: the number of elements modified by the addition of noise; and the parameters of the distribution used to generate the noise, for example, the mean and standard deviation of a Gaussian distribution. Another possibility for perturbing continuous state elements is to define a discrete-valued structure embedded within the continuous state space and then systematically perturb the system within the bounds defined by this structure. For example, consider a system with three elements, in which the values of each element are constrained to the range [0,1]. It is possible to define a three-dimensional cube within this space and constrain the initial states and perturbations to the vertices of the cube. If greater resolution is desired, the cube may be subdivided to introduce the midpoints of the edges and the centre of the cube. This method enables a continuous space to be explored and perturbed in a systematic fashion. ) (3) Defining a distance function. In the case of discrete states, a distance function applies some transformation to the set of distances between individual elements of the state. The most common transformation performed in discrete systems is summation (e.g., Hamming distance), though other transformations such as the average or the sum of squares may be used.


77

When the values of the state elements are continuous, the standard distance metric is the Euclidean distance between the original and perturbed states. For a system with N elements, the Euclidean distance m between states s and s’ is given by, m(s,s’) =

”C(S’~ - si)

i=l

where si is the value of the ith element of state s. (4) Alternative distance measures. Some methods for measuring the effects of perturbations do not require the initial distance measurement indicated above. An example method is the standard basin of attraction stability measurement commonly used in Random Boolean Network models [330;141 - in this analysis the distance comparison is based solely on the final basins of attraction of the perturbed and unperturbed states.

6.8

Sample code

Boolean network attractor stability

A common application of the Perturbation Analysis pattern is to estimate the stability of an attractor in a Boolean network through either stochastic or systematic perturbation of attractor states. The typical unit of measurement in this usage case is whether or not the perturbed state reaches the same basin of attraction as the unperturbed state. Repeated trials are used to provide an estimation of the stability of a given basin of attraction, where stability is defined as the probability that a perturbation to a state does not change the basin of attraction. One standard approach to obtaining such a stability measurement is to look only at the states in the attractor [181;330; 141. In this situation, the resulting measurement is the probability that the perturbation of an attractor state will move the system to a different basin of attraction. Choose a state s in attractor a. Perturb s to obtain s’. This step is generally performed by flipping n elements of the Boolean state, where n is a small integer (frequently one). Iterate the trajectory starting at s’ until it reaches an attractor a’. Store the value a = a’. Repeat steps 1 to 4 some number of times and calculate the average of the values stored in step 4.

78

Recent Advances in Artzjcial Lzfe

An interesting characteristic of Boolean networks is the change in their behaviour as the degree of connectivity of the network ( K ) varies.

0

2

4

6

0

10

12

Network Connectivity (K) Fig. 6.3 Variation of attractor stability with increasing degree of connectivity in an N = 12 random Boolean network.

By repeated application of the above procedure, an approximation of the stability of the attractor states can be obtained for a range of connectivity values. Figure 6.3 shows the results of measuring stability in the above manner on a Random Boolean Network model with N = 12, by iterating through N perturbations of all 2 N system states and recording the probability of the target attractors of the original and perturbed states being


79

different. This observed decrease in system stability is consistent with general expectations of the behaviour of the Random Boolean Network model.

1;yapunov

characteristic exponents

One purpose for which the Perturbation Analysis pattern may be applied is estimating the largest Lyapunov exponent in order to determine the stability of an attractor. The Lyapunov exponents of a system measure the exponential rate of convergence or divergence of two nearby trajectories. If the largest Lyapunov is negative, the attractor is stable. If the largest Lyapunov is positive, the attractor is chaotic, and the magnitude of the exponent gives an indication of the time scale on which the future behaviour of the system becomes unpredictable. The Lyapunov exponent X is given by,

where 6xt is the separation of the original and perturbed trajectories at time

t. While methods do exist for determining the Lyapunov exponent directly from the equations describing a system’s dynamics, it is also possible to approximate the value from a series of data points. The procedure for estimating the largest Lyapunov exponent is as follows [361]: (1) Choose an initial system state. (2) Iterate the state s until it is located on an attractor. (3) Perturb s to obtain s’. This step is generally performed by adding a small amount of Gaussian noise (mean 0, standard deviation 1 x lo-’) to each of the state elements. (4) Calculate the Euclidean distance do between s and s’. ( 5 ) Iterate both trajectories. (6) Calculate the new Euclidean distance dl . (7) Calculate and store the value logl21. (8) Perturb s to obtain s‘ such that distance between them is do in the direction of dl . This step can be carried out by adjusting each element i of state s’ such that,

80

Recent Advances in Artificial Lije

(9) Repeat steps 5 to 8 some number of times and calculate the average of the values stored in step 7. The number of iterations required to reach an attractor in step 2 and the number of iterations of steps 5 to 8 required for the value of X to converge may vary. Similarly, it can be useful to repeat the calculation process using different initial states (step 1) and different initial perturbations (step 3). It is important t o note that if a system contains more than one attractor, then the value of X will be specific to the particular basin of attraction that contains the initial state. One way to analyse the behaviour of a dynamical system is to explore the behaviour of a family of parameterised functions. For example a family of linear systems may be described by the function fm(x) = ma: where m is varied over the real numbers. It is then possible to observe how the dynamics of the system change as the function is changed. The same technique may be applied to investigate the behaviour of more complex systems, such as neural networks, by the inclusion of a gain parameter g that scales the net input into the update function f ,

As g affects the slope of the sigmoid function, modifying g from very small to very large results in a sweep from the linear range, through the nonlinear range to the Boolean range when the function is saturated. By calculating the value of the Lyapunov exponent (using the same initial state each time) for each value of g, the range of dynamic behaviours of a particular system can be visualised. Figure 6.4 shows how perturbation analysis may be used to visualise the dynamics of a recurrent neural network [120]. The network used in this example consisted of 20 fullyconnected nodes, with weights drawn from a Gaussian distribution with mean 0 and standard deviation 1. The gain parameter g was varied from 0.2 and 40 with increments of 0.2. An initial system state I was generated by setting the activation of each node to a value in the range [0, 11. For each value of g the system was initialised to I and the procedure described above was used to estimate the Lyapunov exponent (Figure 6.4, top). In addition, the average activation of the network was recorded for each iteration of the calculation, providing an alternative visualisation of network dynamics (Figure 6.4, bottom). These two complementary views provide a comprehensive picture of the dyanmics of a system across a range of weight scales, revealing such features as bifurcations, fixed point and cyclic attractors, and chaotic behaviour.


.......................................................................................................................

.e

81

I

_ ...........................................................................................................................

........

. . .................... .......................... . ........................ 5

a

I

Wa'ght Scale

Fig. 6.4 Lyapunov exponent (top) and activation diagram (bottom) for a fully connected 20 node network as g is scaled from 0 to 40. Note the correlation between fixed point and cyclic attractors, indicated by single or multiple discrete points on the battom chart, with negative Lyapunov values. In contrast chaotic attractors, with positive Lyapunov, values appear as as 'smears' of points.

6.9

Known uses

The concept of perturbation analysis as an exploratory tool was first formalised in the realm of Discrete Event Dynamic Systems, where it was


82

developed to estimate the gradient of performance measures with respect to variation in system control parameters (see [191] for a history and overview of perturbation analysis in this context). Within the field of complex systems, perturbation analysis has been used on an ad hoc basis by numerous researchers as a means of exploring the stability of genetic regulatory systems (e.g., [330]). Perturbation analysis has also been employed in a more principled fashion, to generate theoretical results about system stability: Derrida’s annealed approximation method [lo51 illustrates the use of the Perturbation Analysis pattern on an annealed version of the Random Boolean Network model. This analytic tool uses an annealed random Boolean updating function, a stochastic perturbation process involving all N state elements and a state distance metric based on the normalised overlap of the states’ values. The annealed approximation method was used to show that K = 2 connectivity in the Random Boolean Network model described a phase transition between the ordered and chaotic behaviour of the system. The annealed approximation method has since been used in different situations to identify phase transitions in the behaviour of networks of multi-state automata [357], and Boolean networks with scale-free topologies [14]. Lyapunov exponents have been used by mathematicians as an indicator of chaotic systems for some time. During the 1980s, several approaches were developed to allow the Lyapunov exponent to be determined from time series data [426], allowing the recognition of chaos in systems whose generating equations were unknown. Subsequent studies introduced the use of neural networks as general models of dynamic systems, typically for econometric and financial time series prediction tasks (e.g., [102]). More recently, simulations of high dimensional neural networks and systematic measurement of Lyapunov exponents has been used to investigate routes to chaos in high dimensional nonlinear systems [ll].Finally, the techniques described here have been extended and used to develop intuitions about the formation and stability of attractors in network models of gene regulation [141].

6.10

Summary

This paper has used the formal framework of patterns to describe a standard technique for analysing the stability of complex dynamical systems. The Perturbation Analysis pattern can be applied to a variety of discrete and continuous systems, as demonstrated by the random Boolean network and neural network examples detailed above. This form of stability analysis allows the effects of intrinsic and extrinsic perturbations on the dynamics


83

of a system to be quantified. This paper also serves as an example of how the software engineering concept of patterns can be used to formalise modelling techniques and strategies for effective communication within a research community. 6.11

Acknowledgements

This pattern was developed during a Patterns Workshop held by the ARC Centre for Complex Systems on 6-7 July 2005.


Chapter 7

A Simple Genetic Algorithm for Studies of Mendelian Populations C. Gondro and J.C.M. Magalhaes CRC for Cattle and Beef Quality, Animal Science, University of New England, Armidale, NSW 2351, Australia cgondro @ m e .edu.au Departament of Genetics, Universidade Federal do Parana, Curitiba, PR 81540-970, Brazil - C.P. 19071, [email protected] Evolutionary processes and the dynamics of Mendelian populations result from the complex interactions of organisms with other organisms and with their environment. Through simulations of virtual organisms the basic dynamics of these populations can be emulated. A conceptual model is used to define the universe, the hierarchical structures and a small set of rules that govern the basic behavior of these virtual populations. At the organism level a simple genetic algorithm is used to model the genotype of the entities and the Mendelian genetic processes. The model is implemented in an educational simulator called Sigex. From a small set of low level rules a t the organism level, higher-order population and environmental interactions emerge that are in accordance to those postulated by the theory of population genetics.

85

86

Recent Advances in Artzjicial Lzfe

7.1 Introduction The two fundamental principles of Evolutionary Theory are that hereditary variability is the result of biological processes commonly referred to as factors of evolution; and, individuals who are more successful in survival and reproduction are differentially selected. These principles can only be explained taking into account the underlying genetic processes that operate on populations [334]. Genetic processes have classically been studied in Mendelian populations which consist of communities of potentially interbreeding, bisexually reproducing organisms. The main implication of the definition is that a Mendelian population consists of a single discrete species reproductively isolated from other species; thus genetic material can flow within the population whilst preserving its genetic identity from other species. Mendelian populations are important because higher animals, including humans, fall under this category and an understanding of population structures has immediate practical applications in medicine, animal breeding, ecology and wildlife management [1791. Further, the dynamics of allelic and genotypic frequencies in these populations can be studied in biological time scales (months or years) and help understand how changes can occur in evolutionary time scales (millions of years). Evolutionary processes and the dynamics of Mendelian populations result from the complex interactions of organisms with other organisms and with their environment. Population genetics is the field of science that tries to explain these relationships or, more formally, it is the study of the effect of genetic processes on entire populations and how evolution factors modify species through time [179]. As a classic numeric branch of genetics, it has usually used a reductionist approach [269] operating mainly through mathematical models. These models are necessarily simplifications of reality, abstracted of the phenomena’s complexity while trying to emphasize some of their aspects; for instance models of selection acting on the frequency of alleles at a single locus do not consider the effects of genes of other loci on fitness neither linkage or epistasis among genes. This is necessary since models can become very complex, as more parameters are included. Simpler and more tractable models can be overly distant from reality, whilst more realistic models can become too complex to be tested experimentally. Further, not infrequently population data is sparse and costly to gather, when not unobtainable; as for example the fossil record which is usually very fragmented. Within this context computer simulations can be an important tool to link theoretical abstractions with the complexity found in nature [72]. A sound scientific theory relies on a small number of hypotheses or axioms to generate complex models [269]. Analogously, computational models

A Simple Genetic Algorithm for Studies of Mendelian Populations

87

based on a small number of rules can exhibit complex behaviors. Phenomena that seem to be operating in nature, given the underlying constraints, can be emulated through computer simulations allowing data generation, model testing, experimental planning, emergence and investigation of complex phenomena not easily derived from theoretical postulates [72]. A particularly interesting approach to address complexity issues at workable levels is through Artificial Life [3] where computational simulations have been used to test new approaches into theoretical biology topics with promising results [238]. Holland [196] [197] introduced Echo models as a means of understanding complex adaptive systems which evolve by natural selection through the interaction of agents among themselves and with the environment. An Echo model is an empirical model in the sense that it encapsulates the mechanisms deemed the most relevant of the system. Even if holding similarities to classifier systems, Echo uses less abstract ruleactivating messages making interpretation of the system easier [196]. The best known implementation of Echo was developed by Hraber et al. [200]. Other computational systems for studying complex systems include the seminal Tierra [327], the more generic Swarm system (www.swarm.org) and Sugarscape [ 12 11. Evolutionary factors such as drift, mutation, migration and selection seldom act in isolation and studying the individual effect of these factors in natural populations can be a daunting task. In this paper we present a model along the general lines of Echo to simulate Mendelian populations using virtual organisms which allows studying different genetic processes and how the allelic and genotypic frequencies are affected over time. We use conceptual modeling [77] to define the universe, the hierarchical structures and a small set of rules that govern the basic behavior of these virtual populations. At the organism level a simple genetic algorithm [195] is used to model the genotype of the virtual entities and the Mendelian genetic processes. Our model is implemented in a freely available simulation package called Sigex. The remainder of the paper is structured as follows. Section 2 briefly reviews the basics of genetic algorithms. In section 3 we overview the conceptual model of the virtual organisms, introduce the simulator Sigex and describe with some detail the genetic algorithm used to constructed the genetic structure of the agents. In section 4 a simple simulation example of the Hardy-Weinberg principle using Sigex is discussed. In section 5 we present some conclusions and future work.

88

7.2


Genetic Algorithms

The most widely disseminated Evolutionary Computation (EC) branch, Genetic algorithms (GAS) date back to Hollands [195]seminal work. GAS are the EC class of optimization heuristics which most closely mimic evolutionary processes at a genetic level; traditionally organisms are represented as linear bitstrings which are referred to as chromosomes; this is the canonical GA [195]). The value in each position of the bitstring is an allele (0 or 1) and the position itself is a gene or locus. The combination of values (alleles) in the bitstring (chromosome) maps to a phenotypic expression, such as a parameter to be optimized. From the above it is clear that GAS operate at two structural levels: a genotypic and a phenotypic one. Selection operators are carried out based on the overall chromosome value (phenotype) while search operators act on the genotype, modifying the chromosome which may or may not change the phenotypic expression. The main search operators are recombination and mutation.

7.2.1

Search operators

Recombination is a search operator that does not generate new sources of variability in the populations albeit introducing new variation. It operates by combining parts from two or more parents to generate one or more offspring. The drive behind recombination is to generate new variability in the population by manipulating the component sources of variation to explore new combinations. Figure 7.1 illustrates a one-point crossover in a binary GA. Briefly, two parents are selected for recombination, a breakpoint in the chromosome is randomly determined, and from the breakpoint onwards the two chromosomes swap the remainder of their bitstrings. One-point crossover breakpoint I Parent A

mq

Parent B Offspring

Fig. 7.1 One-point crossover in a binary genetic algorithm. A breakpoint is randomly selected and the two chromosomes swap bitstrings after the breakpoint. Recombin& tion is a search operator which explores available population variability by testing new combinations. No new allelic variability is generated through recombination but it does generate new variation in fitness values.


89

In contrast to recombination, mutation generates new allelic variability in the population. The general principle is that new offspring are created by a stochastic change to a single parent. Figure 7.2 shows a point-mutation bit-flip in which an allele of a parent is randomly selected to be flipped. The most common approach is to assign a small uniform probability that mutation will occur and test each position of the bitstring; if the mutation operator returns true the bit at the position is flipped. One-point bit-flip mutation

Fig. 7.2 Point-mutation bit-flip in a binary genetic algorithm. New offspring are produced by a random change to the parent. In the example allele at position 4 was selected for mutation and flipped from zero to one. Mutation is a source of new variability in a population.

7.3

Conceptual Model of Mendelian Populations

Our universe of virtual organisms was designed as a conceptual model, meaning that the key components and low-level interactions of the system were empirically determined. The key aspect of the model was to abstract the main mechanistic properties of the biological population. In order to do this the model was structured according to the following steps: Description of the components of the system, properties and low-level interactions. Definition of analogies between the biological system and the virtual environment. Definition of the hierarchical structures of organization with each level establishing a set of elements and relations. Implementation from lower to higher order levels. Once the elements and relations of the model were defined these were grouped in hierarchical levels and implementation was restricted to the lower level elements and relations. Higher level interactions emerge from lower level orders. These steps are clearer if we look at the model itself. Our model consists of a much reduced abstraction of reality formed by the universe (biosphere) which is the uppermost hierarchical level and two subsets forming the environment and the population. The environment has a single level set of three

90


elements, namely food, barriers and space each with its own set of properties. The population has three tiers: organisms (tier 3), chromosomes (tier 2) and genes (tier 1),each with its own set of elements and relations. Figure 7.3 depicts an abstract three level hierarchical structure with elements and the interactions between levels. At the intersections of the layers the properties of the higher level tiers emerge from the interactions of the lower level elements.

Fig. 7.3 Elements and relations in a three layer structure. At the intersection the properties of the higher level tiers emerge form the interactions of the lower level elements.

Differently from Echo [200] which uses haploid agents with a single chromosome and asexual reproduction, our virtual organisms are an abstraction of Mendelian populations, meaning that they are a single species of freely interbreeding diploid organisms with two sexes on an X Y system. There are two genes in the sex chromosomes and seven genes distributed in a variable number of autosomes (between 1 and 7). Each gene has between two and four allelic variants with user-defined phenotypic expressions within a certain interval limit. The genes through their phenotypic expressions express characteristics that intimately relate to the universe ensuring a rapid evolution of the population. For example the gene for vision determines the line of sight of the organism which is an important trait for searching for food in the environment and finding a partner for reproduction. The genes not only relate to the environment but they also relate to other organisms, as for instance the fight gene which defines the level of aggressiveness of an organism. An exception to this aspect is a neutral gene that has no phenotypic expression in any allelic combination. This gene is important in drift and migration studies. The only genetic processes included in the model were mutation, segregation, recombination and reproduction. This


91

model was implemented in an educational software package - Sigex - that is used for teaching population genetics and evolution subjects at undergraduate and graduate levels (figure 7.4). Sigex allows simulation studies using virtual organisms and the generation of population data files that can be studied with the Analysis module that covers the main topics of population genetics. A full description of the model and the simulation tool are available from www.sigex.com.br/genetics.

Fig. 7.4 Sigex - an educational package for studies of population genetics and evolution. The program consists of four modules: a simulator of virtual organisms, a genotype editor, a data analysis tool and a manual/tutorial of population genetics and evolution.

7.3.1

Virtual organisms as a simple genetic algorithm

Each organism is formed by two structures: an identification structure and a genetic structure. The first structure simply stores the organisms information with a unique identifier that allows tracking its activities in the environment and retrieving parental information. The genetic structure defines the organism itself. It is based on a simple genetic algorithm that represents the organisms genotype. It consists of two

92

Recent Advances in Artificial Lafe

homologous bitstrings, in an analogy to DNA molecules. Each bitstring is defined as a chromosome. Here we define that if M is a chromosome; M I and M2 form the homologous chromosome pair and m, is a gene and the phenotypic expression resulting from the genes action, where x is an integer that stands for the position (locus) of the gene in the chromosome and nzy are the alleles of gene m, established in such way that y is the number of bits in locus x and n indicates the number of zeros found in this particular section of the bitstring. Consider that a chromosome M codes traits mj (x = 1, 2, 3, .., j ) and each locus has its y number of bits. A pair of chromosomes ( M I and M2) represents an organism and each phenotype is a result of the interaction between the two alleles n X ycorresponding to locus x of chromosomes M I and M2. So each phenotypic expression of trait m, results from the number of zeros from both chromosomes in the position corresponding to locus x. This does not exclude the possibility that certain phenotypes can be originated by the interaction of genes from different loci. Table 7.1 Possible coding levels for a gene with 2 bits (y = 2).

MI

MZ

0

0

0

0

0

1 2 3 4 5 6 7 8 9 A B C

0 0 0 0 0 0 0 1 1 l l l

0 0 0 1 1 1 1 0 0 O O l

0 1 1 0 0 1 1 0 0 l l O

1 0 1 0 1 0 1 0 1 O l O

D l l O l E F

l 1

l 1

1 1

0 1

Zeros 4 3 3 2 3 2 2 1 3 2 2 1 2

1 1 0

The different combinations of values for each locus x can be classified according to the number of zeros as shown in table 7.1, representing a locus x for y = 2. From table 7.1 it is evident that in this case there are five coding levels. For y = 1 there would be three coding levels. More bits in y imply in more available coding levels.

A Sample Genetic Algorithm for Studies of Mendelian Populations

93

The higher the value of y used the lower is the probability of certain combinations appearing. Table 7.2 presents the associated probabilities for y = 2 (1:4:6:4:1). Table 7.2 Levels and probabilities for y = 2.

Zeros

Probability

2 1 3 0 4

6/16 = 0.3750 4/16 = 0.2500 4/16 = 0.2500 1/16 = 0.0625 1/16 = 0.0625

We can thus define that the set of possible combinations of zeros and ones form the set of alleles of trait m,. The set of m, genes forms chromosome M j , the haplotype. For every gene m, in MI there is a homologous of the same size and in the same position in Mz. Chromosomes M I and M2 form the genotype. In this manner binary structures with any number of genes and each gene with any number of alleles for diploid organisms can be very easily created. Table 7.3 shows a binary structure with two chromosomes M1 and Mz and six genes. Table 7.3

Binary structure of an organism.

Genes mz

Chromosome A41

Chromosome M2

1 2 3 4 5 6

00 01 11 01 0

10 11 01 11 1 0

1

The main genetic operators: segregation, recombination, mutation and reproduction operate on the binary structures. These operators are executed before a new organism is formed. Each operator is discussed below. We have so far mentioned only a pair of chromosomes ( M I and M2). Obviously natural organisms may have more than a single pair of chromosomes. To obtain independent segregation between loci belonging to different chromosomes it is not necessary to use more than a single binary string for each haplotype. Independent segregation can be obtained specifying a recombination ratio of 0.5 between the last locus of a chromosome and the first locus of the following chromosome. In this manner it is

94


possible to emulate the existence of several binary strings (chromosomes) in a single string. Recombination is processed in the same manner as independent segregation, with the difference that the recombination ratio between the loci is below 0.5. To obtain absolute linkage between loci a 0.0 recombination ratio can be determined. Operationally, recombination and independent segregation are obtained by defining breakpoints between loci, testing the probability that a recombination will or not occur against a computer generated random number. If the recombination should occur the string is swapped with its pair after the breakpoint (figure 7.1). The main differences to a conventional GA are that (1) recombination occurs only within an organism and not between organisms and (2) since organisms are diploid, allelic interactions (such as dominance) are possible. Recombination within a locus follows the same procedure described above, thus generating new alleles. Mutation is also a probabilistic event in the binary string of the organism. If the event returns true, the bit(s) will be inverted (1-+ 0 or 0 + 1). Mutation rates can be modeled for each allele, locus, chromosome or organism. It is important to establish appropriate mutation rates ensuring that these occur within the desired frequencies. The use of two homologous binary strings is a natural approach to emulating diploid organisms. It also makes reproduction a simple procedure, being necessary only to randomly select one string (chromosome) from the father and one from the mother to form a new organism, after the previous operators have been executed. Offspring are generated applying the genetic operators to the structure of the parents according to the following steps:

(1) Define temporary structures to store the bitstrings of the parents modified by the operators. (2) Segregation and recombination between the binary strings of both selected parents. (3) Selection of one of the strings from the parents. (4) Mutation. (5) Creation of the new organism. (6) Discard of the temporary structures

7.4

Hardy-Weinberg Equilibrium in a Virtual Population

To illustrate the use of our model, the Hardy-Weinberg law was selected, which is one of the basic principles of population genetics. This principle


95

essentially states that in a large (theoretically infinite) random mating population, in the absence of evolution factors - mutation, migration, genetic drift and selection - the relation between allelic and genotypic frequencies remains constant from generation to generation and the genotypic frequencies are determined by the allelic frequencies [179]. Mathematically for a single locus with two alleles this relationship is expressed by the simple equation:

(p

+ q ) 2 = p2 + 2pq + q2 = 1

where p and q are alleles. The immediate implication of the principle is that it allows estimation of genotypic frequencies in cases where not all genotypes are phenotypically distinguishable. It also allows checking the equilibrium of the population which is the first step to identify if evolution factors are altering the population structure. To test for Hardy-Weinberg equilibrium the assumptions of the model have to be taken into account. In Sigex the natural choice is the neutral gene which has no phenotypic expression and is thus not subject to selective pressures. We ran a simulation using discrete generations over 20 generations. In Sigex discrete generations are simulated by having the parents lay eggs instead of juveniles; when a user-defined number of eggs has been generated, the parental population dies out and the eggs hatch to form the new generation. Each generation consisted of 1000 organisms. The initial population was randomly generated, mutation rate was set at zero and the neutral gene was placed on an individual chromosome not to be affected by selection on linked genes. The resource food was supplied at a level that it did not exert a high selective pressure on inferior phenotypes (500 food units, 200 calories per unit, replacement ratio 1 : 1). Figure 7.5 shows the allelic and genotypic frequencies for the neutral gene over 20 generations. The dashed lines correspond to the allelic frequencies of the two allelic variants n(p) and N(q) and the solid lines are the genotypic frequencies n n ( p 2 ) , Nn(2pq) and N N ( q 2 ) . Since there is no selection, mutation or migration on this gene, the frequency changes over the generations are essentially due to genetic drift. The effect of drift on frequencies is evident observing the frequencies of the homozygotes in comparison to the frequencies of the heterozygotes, since the frequencies of the former, in populations in Hardy-Weinberg equilibrium, are defined by p 2 and q2 of the allelic frequencies p and q making them more susceptible to changes in the allelic frequencies than the heterozygotes which change by 2Pq. Table 7.4 summarizes the allelic and genotypic frequencies. The traditional method for testing Hardy-Weinberg equilibrium is the x2 test. The

96


Fig. 7.5 Allelic and genotypic frequencies for the neutral gene over 20 generations in a population of 1000 organisms.

probability values associated with the x2 are shown in bold in table 7.4. From these it can easily be observed that even though the frequencies vary across the generations, the population remains in Hardy-Weinberg equilibrium over time. The frequency fluctuations can largely be attributed to drift. Even though population sizes of 500 and above are considered sufficiently large to minimize the effect of drift over a small number of generations (over many generations even populations of lo6 organisms show drift effects), the actual breeding population is much smaller ( N e ) , increasing the effect of drift on the frequencies [179]. This simple example illustrates the use of our model for population genetics studies. For validation studies that we carried out to test Sigex the reader is referred to [150]. A study comparing the adequacy of different methods for estimating effective population size ( N e ) using population data derived from the simulator was presented in [379].


97

Table 7.4 Allelic and genotypic frequencies for a population of 1000 organisms over 20 generations. P-values are the probabilities associated to the chi-square test for Hardy- Weinberg equilibrium.

Generation Parental 1 2 3 4 5 6 7 8 9 10 11 12

13 14 15 16 17 18 19 20

7.5

Allelic frequencies n(p) N(d 0.5190 0.4810 0.3275 0.6725 0.4490 0.5510 0.4420 0.5580 0.4470 0.5530 0.4050 0.5950 0.4185 0.5815 0.4820 0.5180 0.4675 0.5325 0.4445 0.5555 0.4580 0.5420 0.3675 0.6325 0.3375 0.6625 0.3035 0.6965 0.3375 0.6625 0.3835 0.6165 0.3485 0.6515 0.2836 0.716p 0.3660 0.6@0 0.4000 0.6000 0.4717 0.5282

G e n o t y p i c frequencies nn(p2) N n ( 2 p d "(q2) 0.2540 0.5300 0.2160 0.1000 0.4550 0.4450 0.0260 0.4860 0.3080 0.1970 0.4900 0.3130 0.2020 0.4900 0.3080 0.1650 0.4800 0.3550 0.1670 0.5030 0.3300 0.2410 0.4820 0.2770 0.2200 0.4950 0.2850 0.2050 0.4790 0.3160 0.2120 0.4920 0.2960 0.1410 0.4530 0.4060 0.1210 0.4330 0.4460 0.0980 0.4110 0.4910 0.1010 0.4730 0.4260 0.1520 0.4630 0.3850 0.1120 0.4730 0.4150 0.0820 0.4031 0.5148 0.1290 0.4740 0.3970 0.1580 0.4840 0.3580 0.2203 0.5028 0.2768

p-values

0.0516 0.2974 0.5739 0.8338 0.7792 0.8980 0.2900 0.2718 0.8545 0.3419 0.7758 0.4187 0.3157 0.3784 0.0679 0.5098 0.1880 0.4033 0.4994 0.7922 0.8134

Conclusions and Future Work

In this paper a simple genetic algorithm was presented for simulating Mendelian populations of virtual organisms. Using homologous bitstrings and adapting conventional GA search operators, the basic Mendelian genetic processes were implemented in virtual entities at low hierarchical levels. This structure at the organism level is sufficient to manifest at a higher level, the dynamics of populations. The simple conceptual model we designed and implemented in Sigex was extensively tested through custom developed simulations evidencing a good fit of the resulting population data to the current theory of population genetics. Within the limits of the model several topics in population genetics were addressed: Hardy-Weinberg equilibrium, selection, genetic drift, mutation, migration, linkage disequilibrium, fitness and adaptation [150]. Sigex was primarily designed for educational purposes to allow students to design, run and interpret population studies, whilst at the same time emphasizing that evolution factors do not operate in isolation but together [151].

98


It is clearly noticeable that as for real biological systems, lower hierarchical level components interact to originate complex higher-level properties. Even though the model is constructed informally it is designed to be a model of the evolutionary theory. This means that it is a set of objects, properties and relations that must satisfy an abstract structure (axiomatics) as the one presented by Magalhaes and Krause [256], in the same way that a real biological system must, in principle, satisfy. This informal approach was adopted due to its approximation with the methods of experimental science; thus the model was built based on the intuitive perception of the biological system from which it was abstracted. The virtual populations had to reflect the same collective properties of natural Mendelian populations and this was the main validation criterion. Nevertheless, once it has been extensively tested, it can be used in new situations as a prospective method. We are currently interested in classic experiments in population genetics, applying the original analysis methods on data obtained from simulations. The results of these experiments should help determine the suitability of virtual populations as surrogates of natural populations for population genetics studies. An example of a classic experiment with Drosophila is found in Buri’s [60] drift experiment. Frequency variations were used to estimate the effective population size, Ne. This parameter, which allows estimation of the occurrence of genetic drift and inbreeding in finite populations, can be estimated through other methods using different sources of data [62]. All these methods can be applied since full datasets of the population dynamics are generated, including the genealogical history of each organism and its genotype. Virtual populations can be a useful tool for research in population genetics. Even though computational simulations cannot accurately depict the dynamics of natural populations, for theoretical studies and as an initial approach to test a new model artificial data can be used prior to obtaining experimental data. This approach not only allows testing on rapidly obtainable, controlled data but can also help in determining which experimental data is relevant, assisting in the design of the experiment.

Acknowledgement

We thank Mr. Omar Achraf for the C++ modules in Sigex and Professor Brian Kinghorn for his comments on this manuscript.

Chapter 8

Roles of Rule-Priority Evolution in Anirnat Models K.A. Hawick, H.A. James and C.J. Scogings

Computer Science, IIMS, Massey University - Albany North Shore 102-904, Auckland, New Zealand Email: { k. a. hawick, h. a.james, c.scogings} @massey.ac. nz Tel: t-64 9 414 0800 Fax: +64 9 441 8181 Evolutionary behaviour in “animat” or physical-agent models has been explored by several researchers, using a number of variations of the genetic algorithmic approach. Most have used a bio-inspired mutation/evolution of low-level behaviours or model properties and this leads to large and mostly “uninteresting” model phase-spaces or fitness landscapes. Instead we consider individual animats that evolve their priorities amongst shortlists of high-level behavioural rules rather than of lower-level individual instructions. This dramatically shrinks the combinatorial size of the fitness landscape and focuses on variations within the “interesting” regime. We describe a simple evolutionary survival experiment, which showed that some rule-priorities are drastically more successful than others. We report on the success of the rule-priority evolutionary approach for our predator-prey animat model and consider how it would apply to more general agent-based models.

8.1

Introduction

Evolutionary behaviour in physical agent or animat models [424] is both a philosophically intriguing problem and also a computationally demanding one. Other workers have demonstrated exciting emergent [116] properties of animat models, starting from a set of very low-level instructions or microscopic behaviours [326;4; 1941. It now seems indeed reasonable 99

100

Recent Advances in Arti$cial Life

to describe collections of such animats as artificial life [241;64] systems. We have previously reported on emergent macroscopic “life-forms” or large scale patterns in a predator-prey model with some carefully chosen microscopic behaviours [209;210; 1861. Introducing evolutionary behaviour into our model to evolve a better predator or a “longer surviving prey” is interesting but involves what is essentially a brute force exploration of the model’s phase space. In this paper we report on what happens when we do not evolve microscopic behaviour or instructions which are all assumed to have a priori probabilities, but rather what happens when we assume the micro-animat behaviour to be made up of a pre-selected set of high-level rules. We apply evolutionary algorithms to exploring combinations of preevolved rule priorities. We believe this is an important mechanism in real life, in that presumably many of the high level features in real life-forms, once established, can be reused in interesting ways. We consider the analogy with bio-programming and real programming is that we are experimenting with different combinations of relatively small numbers of sub-programs rather than arbitrary large combinations of micro-instructions. In this paper we briefly summarise and review our predator-prey animat model in section 8.2. We show some pictorial results of different rule combinations in section 8.3 and discuss some parametric and statistical metrics to characterise the overall results of simulations and model behaviour in section 8.4. We report on an evolutionary survival experiment in section 8.5 and explore how the formulation and application of this hierarchical rule prioritisation approach might be generalised for other animat simulations in section 8.6.

8.2

Rule-Based Model

Our model [209;210; 1861 is based around a set of rule-controlled individual animats which can move, breed, eat and die in a “flatland” of discrete x,y integer coordinates. More than one animat can exist on a single physical cell and the boundaries are not fixed: the size of our model world simply expands as animats explore it. In all our experiments we initialise the model with a block of animats near the origin and timestep the rules so each animat has the opportunity to take one action at each time step. Two well-known systems with similar objectives to ours are Tierra [326] and Avida [4]. Within both Tierra and Avida each animat (cell) is represented by a set of integers that represent a command string, consisting of low-level instructions for the individual cell. Each cell has an internal maximum command string length but not all instruction locations may be filled or

Roles of Rule-Priority Evolution in Animat Models

101

valid. Instructions are commands such as “ n ~ p ” “if-not-0” , , “inc”, “dec”, “push” and “pop” for the simple accumulator-based cells. They are a kind of simple cell programming language and include instructions to divide cells and attempt to inject instruction sequences into other cells. Cells are contained within a structure called a GeneBank. One of the fundamental differences between Tierra and Avida is that cells are considered to be ‘in a soup’ in Tierra, whereas in Avida cells interact only with their (nearest or next-nearest) neighbours on a periodic 2-dimensional grid structure. Depending on the characteristics of the system under consideration in Tierra and Avida, cells can either undergo reproduction or random mutations. In reproduction the user has control over how the candidate that will reproduce is chosen: either the oldest or largest can be chosen, the one with empty space near it, or a random cell. Users can choose to enable point mutations of the entire population and mutations on certain operations such as copying cell instruction sets, dividing instruction sets between parent and daughter cells and deletion of instructions from cells. If point mutation is enabled in the system, then at the end of each evolutionary step inside both systems the total population of cells inside the GeneBank is considered as targets for mutation. Using point mutations, the number of mutations that are to occur is given as the product of the number of active cells inside the GeneBank multiplied by the maximum creature size multiplied by the probability of a point mutation within the system. Once the total number of mutations has been calculated, then random cells are chosen from within the GeneBank, and inside those cells a random int (defining the cell’s instruction sequence) is changed, thus introducing a new instruction. Note that it is possible that a single cell may be hit with repeated mutations, making it evolve more quickly than other cells in the system. Thus, the rules in both these systems are very low level. Furthermore, after the evolution process of a cell has completed, there are no guarantees that its instruction set will produce any meaningful set of operations, let alone a “better” individual. There are no simple ways, in Avida or Tierra, to “pre-package” a group of low-level instructions into a high-level “routine” for easily manipulation. This is precisely the approach that we take in our exploration of how we can better evolve a predator-prey system: we have defined a small number of operations that we believe characterise the behaviours of our predators and their prey. Our model has some interesting spatial properties and we are presently working to compare it to other spatial models and iterated games such as the Prisonner’s dilemma [297].

102

8.2.1


Our Predator-Prey Model

We first developed our model with only two very definite animat types. Predators (known as “foxes”) and prey (known as “rabbits”) coexist in the “flatland” simulated by the model. Within our predator-prey model there are several redundant parameters that control microscopic details of the animats. These rules fall into two broad categories: (a) rules affecting the environment such as how far a rabbit can “see” or how long a fox takes to get hungry; and (b) the set of rules that govern the behaviour of an individual animat. We describe choices for category-a parameters in previous work, but generally most of the model behaviours are insensitive to these. In this paper we focus on permutations of category-b rules that govern behaviour of individual animats. These rules are typical of the type of rule that is changed in a system with animats that are evolving by following a genetic algorithm such as Tierra or Avida. We are exploring how the model varies by changing the rule priority permutation that animats use, rather than changing the basis set of possible rules themselves. In our previous work, we have used the same set of rules for every animat. These rules were chosen by us as a good base set providing a rich set of pattern formation and are stable against small variations. An outline of these rules is: A fox will: 0 eat rabbit if adjacent; 1 move towards rabbit if hungry; 2 breed if adjacent to fox; move towards fox if not hungry; 3 4 move randomly.

A rabbit will: 0 move away from adjacent fox; 1 breed if adjacent t o rabbit; 2 move towards another rabbit; 3 move randomly.

Note that foxes and rabbits can only move towards each other if they are within the radius of perception. Outside of this distance, animals cannot be seen. The order of these rules is important. Each rule has a condition and if that condition is true, the rule is executed and any later rules will be ignored. Thus rule 0 has a higher priority than rule 1 and so forth. For example, if a fox is not hungry and is adjacent to another fox but not adjacent to a rabbit, than rule 2 (breeding) will be executed and rules 3 and 4 will be ignored. Note that the conditions for rules 0 and 1 will be false in this case. The “move randomly” rule is a catch-all and does not have a condition. Thus if all else fails the animat will move randomly. We can therefore denote the behaviour of a particular animat i a t time t by &(t)which will result in the action expressed by the rule priorities in effect. Suppose the animat i has a current rule list Ri = [ T O ,T I , rz, ...,r j ] ; j= O , l , ..., NR - 1 Rules are evaluated in strict order and the first that can

Roles of Rule-Priority Evolution an Animat Models

103

be applied is actioned. In this paper we restrict the list length N L to being identical to the number of possible rules N R and so each of our rules appears precisely once in an animat’s list. Our rules are formulated as a “first matching” so that it is not useful to duplicate rules in the list, but it would be possible to omit some possible rules randomly or otherwise, so that N L O.l’), while each leaf node at the bottom of the tree consists of a probability distribution over the actions an agent can take. Thus, passing the observations down the tree will trigger a single leaf node. Once a leaf node is triggered, the distribution it holds is sampled to determine which action the agent will perform. Parental investment. For this investigation on the origins of dimorphic parental investment, the key agent properties are, obviously, the amount of parental investment, pi, and the parental investment term, it. In the simulations reported in this paper, agents can either evolve their parental investment or their investment term, but not both. Each agent stores genetic information about what (or for how long) it invests in offspring, and genes for both male and female investment are stored. A child inherits both these genes from a randomly chosen parent, but only expresses the gene corresponding to its own sex (of course). These genes are mutated by a mutation variable - itself, stored with each agent. This is so that the system can meta-mutate these mutation variables, allowing for adaptive mutation levels to evolve. Statistics. The main statistics in the following experiments involve averages of pi and it. Another important statistic is the action rate, which is defined as follows: (13.1) where e is a given epoch, ak is one of the acts available to agents, an is the act of interest, and Count,(a) is a count of the number of times the act a was performed in the epoch e. As noted earlier, an epoch is simply a period of cycles in which statistics are gathered. One last important statistic is

Origins of Dimorphic Parental Investments

175

reproductive success, which is the number of offspring that an agent has. Usually, average statistics will be collected from a Tzln set - that is, a set of runs with identical parameters, that differ only in the initial random number seed. The run sets here consist of 15, 30 or 50 runs, as indicated. Some of the graphs for run sets are displayed with confidence intervals these use the between run variance of a parameter, not the within run variance of the agent population.

13.3

Prior investment hypothesis

The first hypothesis we investigate is the one implied by Trivers in his seminal essay on parental investment [396]. Namely, that the sex that commits the most investment has the more to lose - and thus is the sex more likely to evolve further investment. Therefore, if correct, an arbitrary initial difference in parental investments may lead to greater differences of the same kind. In this paper, we call this hypothesis the prior investment hypothesis. While at first this may seem a plausible hypothesis, it was criticised by Dawkins and Carlisle [93], who noted that it involved fallacious reasoning - of the sort used to justify continued spending on a project based on how much has been invested, rather than what future investment will likely return. They used the then topical example of government spending on a supersonic airliner based on past spending, and the fallacy is now often referred to as the ‘Concorde fallacy’.

Method. To test the prior investment hypothesis, we set up the simulation as follows. An agent can invest in just one way - that is, by transferring some of its health to its offspring at birth. We call this investment total parental investment (or t p i ) . As noted earlier, there is a t p i for each sex t p i f and t p i , - the genes for which each agent inherits from a randomly chosen parent. The test of the hypothesis is then quite simple: we initially set t p i f > t p i m for all agents at t = 0 (i.e. t p i f , o > tpi,,~), and then allow them to evolve. If the prior investment hypothesis holds, then t p i f - t p i , measured late in the simulation should be greater than the same difference at the beginning. Results. Using an initial male investment of zero (tpim,o = 0), we ran two experiments with different initial female investment, t p i f , o = 20 and t p i f , o = 100. The evolution of the male and female health investments ( t p i ) for these experiments is shown in Figure 13.l(a) and (b) respectively. Clearly, regardless of the initial settings for t p i , t p i f - t p i , does not evolve to be greater than it was at first. Indeed, quite the opposite happens -

176


Health investments by each sex

Health investments by each sex

$if=

fpif= 20 health units

100 healthunits

Female investment Male investment

-

60

Iwx)

Cycles

zoo00

3oooo

40000

Cycles

Distribution of offspring numbers Fern s d 12 265, Male s d =2 244

02 0 16 0 12

0 08 0 04 0 0 1 2 3 4 5 6 7 8 9 10111213

Number of offspring Fig. 13.1 (top left) Evolved tpi made by males and females when tpim,o = 0 and t p i f , o = 20 and (top right) tpim,o = 0 and t p i f , o = 100. Also, (bottom) distribution of reproductive success by sex for the run set shown in b. (Average of 15 runs.)

that is, sexually dimorphic investment disappears entirely. We also ran experiments with different initial values for tpif and t p i , (ranging from 0 to 100) with the same result. As noted earlier, Bateman identified a key idea in parental investment theory: that the sex that invests more will evolve to have less variance in its reproductive success [30]. In contrast, we would expect there to be no difference in reproductive variability if both sexes invest equally. We check this prediction in Figure 1 3 . 1 which ~ ~ is taken from the last 7000 cycles of the tpif,~ = 100 run set. The graph is a frequency distribution of the number of offspring agents have, split by sex. As we can see, the distributions are near identical. While the distributions are significantly different on a chi-square test, (x2= 161, p < 0.001), the Kullback-Leibler (KL) distance between the distributions is negligible (7.5 x In addition to parental investment, we can also see whether any sexually dimorphic behaviour is evolving by looking at action rates, as shown in Table 13.1. The top row shows the female minus male difference in action


177

Table 13.1 Female minus male action rates for the prior investment experiments. Female action rates in parentheses.

tpif,o = 20 tpif,o= 100

Eat 1.1% 0.5 (81.4%) -0.54% 0.5 (81.7%)

Mate -1.2% 0.5 (15.3%) 0.54% 0.5 (15.6%)

rates that evolves for the t p i f ,= ~ 20 run set, and the bottom row shows the same for the tpif,o= 100 run set (the numbers in parentheses are the female action rates alone). As we can see, females evolve to eat 82% of the time, while they evolve to mate 16% of the time (the remaining 2% is due to resting). Further, there is little to no dimorphism in both run sets. In fact, there is an initial rapid move toward dimorphism in both eating and mating (not shown), with males mating more, and females instead eating more. This is almost certainly due to the sexual difference in investments a t the beginning of the simulations. However, this dimorphism disappears, resulting in no stable dimorphism by the end.' While no stable dimorphism develops for the prior investment hypothesis, we will see an example of stable dimorphism at the end of the next section on the desertion hypothesis.

13.4 Desertion hypothesis The desertion hypothesis was born from Dawkins and Carlisle's criticism of Trivers' prior investment hypothesis [93]. Dawkins and Carlisle noted that dimorphic investments may evolve when exactly one parent is required to raise a viable offspring. In particular, if one sex has a chance to desert the offspring first, then it will. Dawkins and Carlisle cited parental investment amongst fish as an example of this: in many species of fish, it is the male who looks after the offspring. They suggested that this is because females spawn their eggs first and males fertilize them after - by which time, of course, the female has the opportunity to leave. In contrast, male mammals fertilize female eggs internally, producing zygotes that are stored within the female. Thus, the male clearly has the first opportunity to desert, potentially explaining why maternal care (which occurs after birth, of course) is predominant amongst mammals. 'We also ran an experiment in which we set a minimum - non-evolvable - amount that females must make. This simulates investment methods such as gestation, which, once evolved, are difficult to evolve away. We then left e x h sex free to evolve additional investment. On doing this, we found that females did not evolve to make greater additional investments. In some cases, both sexes evolved the same additional investments, while in others, males evolved the greater additional investments.

178


Length of investment term

fi

$

Distribution of offspring numbers

pcpl 0 5

3

0

Fem s d =3 032, Male s d =3 029

0 35 03 0 25 02 0 15

25

2

15

2

10

Female investment term Male investment term

g 5 - 0

0

5Mx)

C

-2

15Mx)

2oo00

0 1 2 3 4 5 6 7 8 9 10111213

Cycles

Number of offspnng



M

fi

loo00

01

0 05 0

pep1 = 4

0

Fern s d =2 056, Male s d =2 15

02

Male investment term

p

0

5000

C

10000

15ooO

0 16

0 I 2 3 4 5 6 7 8 9 10111213

20Mx)

U

Cycles

Number of offspnng

fi


Distnbution of offspring numbers

I

pcpr 16

4

0

Fem s d =I 466, Male s d =I 8

0 14 Male investment term

01

0 08 0 06 0 04 0 02 0

* 10 $

J C

U

0 0

5000

loo00

Cycles

15ooO

20000

0 I 2 3 4 5 6 7 8 9 I0111213

Number of offspnng

Fig. 13.2 The evolved eit for males and females when (top) pcpi = 0.5, (middle) pcpi = 4 and (bottom) pcpi = 16. Distributions of reproductive success by sex for the simulations in (top right) top left, (middle right) middle left and (bottom right) bottom left. (Average of 30 runs.)

Method. To test this hypothesis, we allow parents to invest for an evolvable period after birth (the evolvable investment term, or eit). For females, we set the minimum eitf to 5 cycles; in contrast, males have no minimum


179

Table 13.2 Female minus male action rates for the desertion experiments. Female action rates in parentheses.

Eat pcpi = 0.5 pcpi = 16

0.38% 0.5 (59.1%) 7.5% 0.5 (71.6%)

Mate -0.52% 0.5 (29.6%) -7.2% 0.5 (20.1%)

period (other than 0, of course). The child needs a minimum investment of 32 cycles - so if both parents invest for the same terms, they would each invest at least 16 cycles. If one parent quits investing before 16 cycles, the other parent is forced to make up the other parent’s investments. We force the remaining investment for simplicity, rather than try to produce environments in which full investment by a t least one parent is needed. Finally, we fix the per cycle purentul investment (or pcpi) as a parameter of each run set. In the simulations shown here, the pcpi takes on one of 3 values: 0.5, 4 and 16 health units per cycle.

Results. Figure 13.2 shows the results of our tests of the desertion hypothesis. When the pcpi is lowest, no dimorphic investments result (Figure 13.2a). In this case, relatively high periods of investment are needed from both parents: each tries to invest for -25 cycles, which results in -50 cycles of combined investment - well above the minimum 32 cycles of investment needed by the child. Thus, the female’s minimum eit of 5 cycles becomes irrelevant. In the reproductive success distributions for this run set, shown in Figure 13.2b, we can see that no substantial sexual difference exists (KL distance of 4.8 x lop5). Furthermore, there is no sexually dimorphic behaviour evident either (first row, Table 13.2). For the run set in which pcpi sits a t the higher level of 4 health units per cycle, the result is very different. Here, e i t f reaches an average of 15 cycles, while eit, reaches an average of -10 cycles. Since eit, < 16, females must make up the remaining investment, so that females invest for the greater of e i t f = 15 and 16+ (16 - eit,) = 22 - which, of course, is the latter. It is interesting that the minimum e i t f of 5 cycles can have an effect here. In fact, the average standard deviations of eit, and e i t f (not shown in the graphs) fall between 5 and 7 cycles, allowing the minimum e i t f t o influence the evolution of investments.2 Note that Figure 13.2d shows that dimorphism in reproductive success begins t o develop in this run set. Finally, in the pcpi = 16 run set shown in Figure 13.2e, eit, reaches 5 cycles and e i t f reaches 15. That is, females come to invest for -27 cycles. This establishes strong conditions for dimorphism, which indeed evolves as can be seen quite obviously in Figure 13.2f and the bottom row of ’Keep in mind that the confidence intervals in the graphs only use the variance in runs, not the variance in the underlying populations.

180


Table 13.2. This dimorphism is exactly the kind that parental investment theory predicts - that is, that the sex that invests less will evolve to try mating more often. Of course, trying does not equate with succeeding - males (and females) must average 2 offspring in a stable population. Instead, the eagerness of males leads some to greater success, and this in turn causes others to have lesser success; which is exactly what we see in Figure 13.2f.

13.5

Paternal uncertainty hypothesis

The paternal uncertainty hypothesis is again due to Trivers [396]. He suggested that males are often in a situation of being uncertain about their parentage, particularly in species where females go through a gestation period. In contrast, uncertain female parentage is very unlikely. In that case, it may pay males to spend less effort on an offspring, and instead spend more effort trying to mate. There is some evidence in humans that paternal uncertainty has an effect on how parents and their families interact. Daly and Wilson report that the mother’s family will make comments about how similar the child looks to the father more frequently than reciprocal comments are made by the father’s family [89]. Further, Fox and Bruce report that paternal certitude affects how fathers take to their roles as fathers [129].

Method. We test the paternal uncertainty hypothesis by fixing the probability of paternity, pp, as a parameter of the simulation. In particular, females always invest in their own offspring; in contrast, females choose males from the neighbourhood to invest in their offspring according to pp. At one extreme, if pp = 1 for a simulation, the chosen male is certainly the father; at the other extreme, if pp = 0 for a simulation, the chosen male is never the father. We set p p to 101 equally spaced values between 0 and 1 inclusive. As for the prior investment experiments, the parental investments that both sexes make, tpif and tpi,, are free to evolve. Results. Figure 13.3a shows the main result of this experiment. Each point in the scatter plot represents the average tpi, of the last 1000 cycles (of 7000 total) in a single run. The horizontal axis shows the setting of the p p parameter for each run, and the vertical axis indicates the investment amount. The result here is clear: the lower the probability of being the actual father, the less males invest in the offspring. Indeed, p p and tpi, have a correlation coefficient of 0.848 (t(100) = 15.81, p < 0.001). Thus, this result provides strong support for the paternal uncertainty hypothesis. We can also see how females evolve tpif for different p p from Fig-


Male investment vs paternal probability

181

Female investment vs paternal probability

correl=0.8476

50 Y

$40

2>

.s

g

30

20

b)

10

n 0 0 1 02 0 3 0 4 05 06 0 7 08 0 9 I

0 0 1 02 0 3 04 05 06 0 7 08 0 9 I

Probability of paternity

Probability of patermty


Distribution of offspring numbers Ferns d =I 986, Males d =2 M)1

Ferns d =2 141, Males d =2 219

02

0 16

0 16

0 14 0 12 01 0 08 0 06 0 04 0 02 0

0 12 0 08 0 04

0 0 I 2 3 4 5 6 7 8 9 101ll2l3

Number of offspnng

0 I 2 3 4 5 6 7 8 9 10111213

Number of offspnng

Fig. 13.3 (top left) Evolved investments made by males as a function of p p . (top right) a s per top left, but for females. Distributions of reproductive success by sex for (bottom left) p p < 0.1 and (bottom right) p p > 0.9. (1 run per graph point.)

Table 13.3 Female minus male action rates for the paternal uncertainty experiments. Female action rates in parentheses.

I pp pp

< 0.1 > 0.9

Eat 3.0% 0.5 (67.2%) 0.72% 0.5 (66.9%)

Mate -3.1% 0.5 (19.2%) -0.86% 0.5 (19.4%)

ure 13.3b. As p p increases, and therefore as males invest an increasing amount, tpif falls away slightly. The negative correlation is not large (T = -0.265), but is significant (t(100) = -2.72, p < 0.004). Thus, the more males invest, the more females take advantage by investing less. To assess the level of dimorphism (in behaviour and reproductive success) that evolves in these runs, we take the runs in which p p < 0.1 as one group and p p > 0.9 as another. The former should exhibit more dimorphic behaviour, while the latter should exhibit less. Figure 1 3 . 3 ~ and


182

Figure 13.3d shows the reproductive success distributions for the last -1200 cycles of runs with p p < 0.1 and p p > 0.9, respectively. We can see that there is a slight dimorphism evident in the p p < 0.1 runs (KL distance of 0.0011) that is not evident in the p p > 0.9 runs (KL distance of 0.00019). More tellingly, we can see a reasonably strong behavioural dimorphism in Table 13.3 for the p p < 0.1 runs, that is much reduced in the p p > 0.9 runs. 13.6

Association hypothesis

The association hypothesis or, more generally, the pre-adaptation hypothesis was suggested by Williams [418]. He noted the perhaps obvious point that if only one sex remains in the vicinity of the offspring after birth - due to some pre-adaptation of that sex - then that sex has the opportunity to evolve parental care, while the other sex does not.

-

fi

$ C

‘G 15

Length of investment terms

c

Length of investment terms

Equal mobility

c

More mobile males



-

C H

Cycles

c

C

H

Cycles

Fig. 13.4 The evolved after birth investment terms for both males and females for (on left) no sex differences and (on right) males as the more mobile sex. (Average of 50 runs.)

Method. As it stands, the association hypothesis is almost tautological. However, this need not be so: the sex that does not remain with the offspring - which we will take to be the male in these experiments - could evolve to return every so often to make parental investments. There is no a priori reason why males cannot continue investing. Nevertheless, males will find it harder t o invest in offspring. ‘Harder’ here simply means that males have to do more to invest a t the same rate as females. In this case, it is not immediately obvious that females will invest more than males, though we would expect them to do so since they find investment easier. We choose to test this form of the hypothesis by having a non-evolvable ‘Move’ action that causes males to move about more actively. Specifically,

183

Origins of Dimorphic Parental Investments Table 13. 4 Female minus male action rates for the association experiments.Female action rates in parentheses.

Equal mobility More mobile male

Eat

Mate

0.20% 0.5 (68.0%) 1.1%0.5 (66.2%)

-0.18% 0.5 (23.8%) -1.1%0.5 (25.2%)

males move about randomly in a 9x9 neighbourhood with 0.6 probability each cycle, while females move about randomly in a 3x3 neighbourhood with 0.2 probability each cycle. In addition, we established that parental investments have a certain ‘efficiency’, dependent on the distance from the child. That is, the closer one is to a child, the more of one’s investment reaches the child. The function of efficiency, e , over distance, d , that we used is a simple linear inverse function of distance from the parent: e = 1- if d < 20 and e = 0 otherwise. The distance is the minimum number of cells in either the horizontal or vertical direction. Similar t o experiments in previous sections, the investments are in the form of per-cycle investments after birth. Here, pcpi = 8 and agents are free to evolve the term for which they invest (the eit).

&

Results. Figures 13.4a and b show the results of 2 run sets, the first in which the ‘Move’ action is the same for both sexes, and the second in which the ‘Move’ action makes males more mobile. The graphs show the eit for both sexes. In the first graph, no dimorphism evolves - as we would expect. In contrast, the second graph shows that females - the sex that can invest more efficiently - evolve to invest for longer periods. Surprisingly, the degree of behavioural dimorphism that evolves is very slight. The bottom row of Table 13.4 shows that a difference of only 1% in action rates evolves - in contrast to experiments in previous sections that showed differences of between 3% and 7%. Further, dimorphism in reproductive success (not shown) is not evident (KL distance of 6.6 x for the more mobile male run set).

13.7

Chance dimorphism hypothesis

All of the above hypotheses on the evolution of sexual dimorphism assume that there is a pre-existing sexual difference. But there may be cases in which there is no pre-existing difference or, perhaps more likely, that an existing difference is not sufficient to cause the evolution of further dimorphism. Trivers suggested that the sexes differentiated very early on due to positive selective pressure acting on gametes whose sizes fell in the tails of the normal curve [396]. That is, smaller, mobile, gametes would be selected

184

Recent Advances in Artificial Life Table 13.5 Averages and standard deviations of degree of dimorphism (dd) for runs o f differing population sizes. (Average of 50 runs.) Avg stable pop’n size

Mean dimorphism (dd)

S.D.o f dd

237 409 648 901

p = 44.32 p = 29.19 p = 9.25 p = 11.19

u = 83.96 u = 54.98 0 = 11.91 u = 17.12

for since they can fertilize other cells more easily, while larger, immobile, gametes would be selected for since they increase the probability of a viable offspring. In contrast, those with intermediate sizes would not fare so well. Trivers does not seem to regard this process as occurring anew in new species, but rather occurring amongst progenitor species, from which dimorphism is inherited. However, perhaps it is possible, as Gould might hold, that sexual differences in parental investment can arise by chance. If a chance difference in investments persisted for long enough, the sexes may adapt to the difference. This could then ‘lock them in’ - that is, chance reductions in dimorphic investment could cause agents to become less fit, and thus be selected against. We would expect such events to be most likely amongst small populations, since the genetic variance within such populations will be small, while the genetic variance between such populations will be large.3

Method. To see if dimorphism may arise at all, we run several runs in which the sexes are initially identical, and then see whether substantial dimorphic investments (leitf - eit,I) and behaviour can develop. Further, to discover if the size of the population has an effect on the frequency with which dimorphism develops, we run the simulations with different sized populations - which we achieve by regulating the food supply. To assess the degree of dimorphism, dd, for a single run, we take the mean Ieitf-eit,I in that run, and divide by the pooled standard deviation of eitf and eit, within that run; this is so as to counter the run to run differences. In essence, dd is the number of standard deviations of difference between e i t f and eit,. Results. Figure 13.5 summarises the results of run sets, each with different average population sizes. The table shows the mean dd for a run set with a given population size (along with the standard deviation). The first thing to note is that dimorphism evolves quite regularly. If we focus on those cases in which there are 3 standard deviations or more of difference (i.e. dd 2 3), we note that dimorphism results in half or more of all cases 3This is similar to the argument supporting peripatric speciation.


185

(not shown). Further, there seems to be an inverse correlation between the size of the population and the average dd that evolve^.^ There also seems to be an inverse correlation between the variance of the dd and population size.

13.8

Conclusion

We have explored various hypotheses on the origins of sexually dimorphic investments through simulation, and found support for those that we would expect to be correct. Our simulation results concur with the view that the prior investment hypothesis is wrong, given initial sex differences in investments (and also minimum sex differences in investments). We found strong support for the desertion hypothesis and for the paternal uncertainty hypothesis. While our results also agreed with the association hypothesis, the level of dimorphic behaviour and reproductive success in these experiments was lowest. Finally, we had little difficulty in finding simulations that produced dimorphism by chance, and confirmed that smaller populations do indeed lead to greater levels of dimorphism.

4The last run set here defies this trend, but runs that we further tested, using larger populations, continue the correlation.


Chapter 14

Local Structure and Stability of Model and Real World Ecosystems lD. Newth, and 2D. Cornforth CSIRO Centre for Complex Systems Science GPO Box 284 Canberra, ACT 2601, Australia Email: [email protected] 2School of Environmental and Information Sciences Charles Stud University PO Box 789, Albury, NSW 2601, Australia Email: [email protected] For over a century, the analysis of community food webs has been central to ecology. Community food webs describe the feeding relationships between species within an ecosystem. Over the past five years, many complex systems -including community food webs- have been shown to exhibit similar global statistical properties (such as higher than expected degree of clustering) in the arrangement of their underlying components. Recent studies have attempted to go beyond these global features, and understand the local structural regularities specific to a given system. This is done by detecting nontrivial, recurring patterns of interconnections known as motifs. Theoretical studies on the complexity and stability of ecosystems generally concluded that model ecosystems tend to be unstable. However this is contradicted by empirical studies. Here we attempt to resolve this paradox by examining the stability of common motifs, and show that the most stable motifs are most frequently encountered in real ecosystems. The presences of these motifs could explain why complex ecosystems are stable and able to persist.

187

188


14.1 Introduction Community food webs describe who eats whom within an ecosystem. For almost a century, they have been central to community ecology as they provide a complex yet tractable description of biodiversity, ecosystem structure, function [115] and fragility [358]. For over fifty years there has been an ongoing debate over the relationship between the stability and complexity of community assemblages [270]. MacArthur [254] was one of the first to suggest that the more complex an ecosystem was, the more likely it was to be stable, as population fluctuations have a greater chance of being corrected. However, theoretical studies on the matter usually conclude that systems with more species, and stronger interactions between species, are more likely to be unstable than those with fewer species and weaker interactions [139;266; 267; 335; 304; 305; 3951. Recently there has been a renewed effort towards the understanding of food web topology. This interest was sparked by the finding that many complex networks share common topological features [12;408; 4071. In previous work [288;2901, we have argued that complex dynamical systems like community food webs evolve through the process of invasion (where new species are added) and collapse (where species become extinct). The collapse of a food web results in a “stable core” around which a more complex system can evolve. In this paper we examine the dynamical stability properties of small sub-networks, or “motifs”, that represent small ensembles of species that are candidates for these stable cores. We will then examine real world food webs for the occurrence of stable motifs. The remainder of this paper is structured as follows. The following section outlines the theory of stability applicable to the study of ecosystems and of motifs. Section 14.3 describes the experiments used to determine the stability properties of motifs, and their frequency within real ecosystems. Section 14.4 provides the results of these experiments. A discussion of the implications and significance of this study is given in section 14.5. Finally section 14.6 provides some closing comments and possible future directions.

14.2 Ecological stability and patterns of interaction The behavior of an ecosystem is subject to external and internal perturbations. External perturbations arise from macroscopic uncertainties in the environment. Internal perturbations arise from spontaneous noise processes from microscopic fluctuations in the environment. Much of ecological theory is based on the underlying assumption of equilibrium population dynamics. An ecosystem’s stability properties are determined by the way in

Local Stmcture and Stability of Model and Real World Ecosystems

189

which it responds to internal and external perturbations. A system is said to be stable if the system returns to an equilibrium point after being perturbed away from its steady state. In the following section we will outline linear stability analysis, a common approach for determining the stability of a community assemblage.

14.2.1

C o m m u n i t y Stability

Landmark studies by May [266] introduced notions of linear stability analsysis into theoretical ecology. Typically, models of ecological interactions are inspired by equations similar to that proposed in [247]:

dNi - - F i ( N l ( t ) N2(t), , * . * ,Nn(t)), dt

(14.1)

where Fi is an empirically inspired, nonlinear function of the effect on the ith population on the remaining n populations within the ecosystem. Most commonly the function Fi takes on the form of the Lotka-Volterra equations:

(14.2)

where Ni is the biomass of the ith species; bi is the rate of change of the biomass of species Ni in the absence of prey and predators; and aij is the per unit effect of species j ’ s biomass on the growth rate of species i ’ s biomass. Of particular interest in ecology is steady state of the system, in which all growth rates are zero, giving the fixed point or steady state populations N,*. This occurs when: 0 = F i ( ( N l ( t ) , N z ( t ).,. . , N n ( t ) ) .

(14.3)

The local population dynamics and stability in the neighborhood of the fixed point can be determined by expanding equation 14.1, in a Taylor series about the steady state populations,

(14.4) where x i ( t ) = Ni(t)-N: and * denotes the steady state. Since Fil* = 0, and close to the steady state xi are small, all terms that are second order and

190


higher, need not be considered in determining the stability of the system. This gives a linearized approximation that can be expressed in matrix form as : k ( t ) = Az(t)

(14.5)

where z ( t ) is an n x 1 column vector of the population deviations from the steady state, and the community matrix A has the elements aij (14.6) which represents the effects of population j on the rate of change of population i near the steady state. As May [267] demonstrates, solving the algebraic eigenvalue problem for A reveals the systems dynamical response to perturbations . Of particular interest here is the stability of the system near the stead states. A system will return to a steady state if (and thus said to be stable) if all the real parts of the eigenvalues associated with A are negative.

14.2.2 Local patterns of interaction Over the past 10 years, it has been shown that many natural systems including community food webs share a number of common statistical properties. For example the distribution of links within a network are often found to follow a power-law distribution, in which some nodes are more highly connected than other nodes. Other common patterns found in many complex networks include the small-world properties, of high clustering and short path-lengths [408]. While many systems may share global characteristics, they may vary greatly in their local patterns of interaction. Recently it has been found that many networks display local patterns of interaction or “network motifs” at a much higher frequency than expected in random networks [280;351]. A motif is a small sub-graph that defines the local interactions between a set of nodes [280]. Figure 14.1 illustrates a series of three node motifs. Previous studies have shown that different types of complex networks are constructed from different combinations of motifs. For example social systems tend to have a high number of complete three node subgraphs, while linguistic networks tend to have highly expressed tree structures [280]. Over expressed motifs can be interpreted as structures that arise because of the special constraints under which the network has evolved. In previous work it has been shown that local food web assemblages change as a function of latitude, environment, and habitat [289].


191

Fig. 14.1 The seven three node motifs, where each node interacts with each other node.

Here we are concerned with the frequency of occurrence of structural motifs and their stability properties. In general a motif defines a dynamical system, where the nodes are state variables and the links define relationships between the state variables and link weights define the rate of change of a state variable. We will attempt to reconcile the stability properties of a given motif (via the use of the linear stability criteria outlined above), with its frequency of occurrence in real world food webs.

14.3 Experiments In this section we describe the experiments used to determine the stability properties of motifs, and to determine the frequency of these motifs in real ecosystems.

14.3.1

Stability properties of motifs

To determine the most stable motifs, we have devised a simple systematic numerical experiment that tests the stability of each motif. The stability of a motif is influenced by three factors: (1) the minimum on-diagonal term (which determines the lower bound for all eigenvalues); (2) the interaction strengths between elements (which determines the rate of change of a state variable); and (3) the loop structure (which influences the sign of the real part of the eigenvalues). Each of the motifs defines a particular loop structure, so it is necessary to test systematically the effects of varying interaction magnitudes and self-regulatory terms. For each motif, the on-diagonal terms were systematically varied between 0 and -1 in steps of 0.05. All non-diagonal terms were randomly drawn from uniform distribution centered around zero, and systematically varied in increments of 0.05. The range of the non-diagonal term was [-1,1]. This results in 441 parameter combinations for each of the motifs. To gain a probability of stability,

192


A

B

Fig. 14.2 Network randomization procedure. (A) T h e original network. (B) An ensemble of four networks that have the same degree distribution as (A), however the local patterns of interactions are destroyed.

each parameter configuration was executed 50,000 times. To determine the stability of a given motif, we have calculated the eigenvalues for each of the 50,000 randomly generated weights for a specific motif configuration. The probability of stability was calculated as the number of times the motif was found to be stable across all parameter configurations.

14.3.2

Motif frequency

To measure the frequency of the motifs making up the networks, we have implemented an algorithm for detecting recurring patterns of interconnections, or motifs. A detailed overview of the algorithm used here and its application to a gene regulation network was presented in [351]. Each community food web (see 514.3.3) was scanned for all possible n-node sub graphs (in the present study, n = 3 and 4), and the number of occurrences of each sub graph was recorded. To focus on those motifs that are likely to be important, we compared the frequency of occurrence of these motifs with their occurrence in randomized networks. The randomization procedure employed here preserves the degree distribution of the original network; that is, each node in the randomized ensemble has the same in- and outdegree as the corresponding node in the original network. This allows for a stringent comparison between the randomized and the observed networks, as the randomized ensembles account for patterns that appear because of the degree distribution. Fig 14.2 depicts the result of the randomization procedure for a simple 16 species food web. The ratio between the motif frequencies in the real network and the randomized ensemble provides a measure of how over or under expressed each motif is.

Local Structure and Stability of Model and Real World Ecosystems

193

14.3.3 C o m m u n i t y food web data To reconcile the stability properties of motifs to their frequency in real world ecosystems, we analyzed a set of 184 community food web. A majority of the food webs were taken from the data set compiled by Briand and Cohen [57]. This database contains 113 food webs, from a wide range of habitats including: salt marshes, deserts, swamps, costal areas, estuaries, lakes and pack ice zones. Other communities include the Grasslands in Great Britain [260]; Silwood Park, England [275]; two variations of the Ythan Estuary, in Scotland [175;2021; Little Rock Lake, in Wisconsin [259]; Chesapeake Bay [24]; St Marks Seagrass, an estuarine seagrass community in Florida [78]; St Martin Island in the Caribbean [148]; Skipwidth pond, England [406]; Coachella valley, in southern Californian desert [306]; 10 grassland communities surrounded by pastures in southern New Zealand [394]; and 50 New York Adirondack lake food webs [183].

14.4 Results In this section we describe the results of the two experiments: stability properties of motif, and frequency of motifs in real ecosystems.

14.4.1 Stability properties of motifs Figure 14.3 shows the probability of a given motif being stable for a given parameter configuration for a fully connected three cycle (14.3A), and a feed forward loop (14.3B). As can be seen, the fully connected three cycle is only stable when the self regulatory term is sufficing high, and the interaction terms are low. By contrast, the feed forward loop is completely stable regardless of the self regulatory and interaction terms. The feed forward loop could be considered to be sign stable. That is regardless of the nature and magnitude of the interactions within the motif, the system is always stable. These results demonstrate that certain motifs are only stable under certain conditions, while other motifs exhibit stability over a wide range of the interaction terms. This results may allow us to speculate on the nature of the interactions taking place when these motifs are observed in nature.

14.4.2

Stability and occurrence of three node motifs

In this section we will compare the stability properties of the community food webs with their occurrence in community assemblages and their rate of expression. Figure 14.4 compares these results. In Figure 14.4 the top graph

194


Fig. 14.3 The relationship between stability and magnitude of the matrix elements aij for two motifs, a fully connected three cycle (A), and a feed forward loop (B)

shows the probability of stability of the various 3 species motifs, with more unidirectional vertices on the left, and more bidirectional vertices on the right. It can be seen from this figure that the presence of cycles in the graph is associated with instability. The second, fourth, sixth and seventh motifs contain cycles, and have lower probability of being stable. The middle graph in figure 14.4 shows the percentage of real food webs examined in this study that contained the respective motif. Although the most stable motifs (see Figure 14.4 Top) tend to be expressed more frequently than more unstable ones, these frequencies are not normalized for the effects of the link (degree) distribution of the food webs. Figure 14.4 bottom shows the rate of expression of each motif in actual ecosystems compared to those in randomized networks, so these figures have been normalized for the effects of the degree distribution. It can be seen that a number of motifs that are less stable tend to be under expressed, as is the case with the second, fourth and sixth motif. However the seventh motif, the fully connected three cycle, (while only present in a small fraction of real world food webs) is highly over expressed by a factor of almost two. This motif is highly unstable. So why is it so highly over expressed in real-world food webs? An inspection of the data reveals that this motif is the result of the life cycle of a trophic species. Given two species A and B, species B eats the young of species A. However when species A matures, it eats species B. In this case the aggregation of species life stages into a single trophic species may appear to be theoretically unstable, but within natural systems this is a stable configuration. This implies that ecosystems models should take account the life stages of species.


195

AAAAAAA Fig. 14.4 Properties of three species motifs. Top: stability of motif, middle: relative frequency of motif in actual food webs, bottom: normalized occurrence, compared to randomized controls.

14.4.3 Stabilitp and occurrence of four node motifs We now turn our attention to 4-node motif stability and frequency. Figure 14.5 shows the results of this analysis. Again we see that simple motifs are more stable than those with elaborate loop structures. Like the feed forward loop of the 3-node motifs, the bi-parallel feed forward, and the biparallel fan (first and second motifs) are sign stable. It is interesting to note that these motifs are found in a wide cross section of community food webs (see Figure 14.5 middle). In contrast, other more complex structures (with the exception of the fully connected 4-node motif) are only found in a small fraction of communities. This may indicate that these motifs are specific to a particular type of habitat or they may provide a specific ecosystem function. Again we see that the most complex motif (the fully connected 4-node motif) is the least stable, but is found more in many community assemblages. This may be the result of the aggregation of life stages into a single trophic species. Again this warrants further investigation. Figure 14.5 bottom illustrates the rate of expression of each motif. It is interesting to note that almost all appear to be expressed with approximately the frequency found in the randomized controls, although in most cases slightly higher. This suggests that many of the local patterns can be attributed to the degree distribution of the community assemblage. The fully connected 4-node motif occurs more frequently than expected.

Recent Advances in ArtiJcial Life

196

40

c 35

1 F

f

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

15

'L

10

Fig. 14.5 Properties of four species motifs. Top: stability of motif, middle: relative frequency of motif in actual food webs, bottom: normalized occurrence, compared to randomized controls.

14.5

Discussion

We are remarkably ignorant of the dynamics that govern the natural systems that surround us. The southern oceans around Antarctica; for example, produce about 3% of the world's phytoplankton. An insufficient amount, it would seem, to support complex ecosystems; yet they do. Even

Local Structure and Stability of Model and Real World Ecosystems

197

more surprisingly, these ecosystems are notoriously top-heavy. Some 15 million Crab eater seals-possibly the world’s second most abundant large animal after humans-, two million Weddel seals, half a million Emperor penguins and four million Adelie penguins all live on the pack ice around Antarctica. While seeming to be awkwardly top-heavy these ecosystems work, and more importantly they persist. Earlier we noted that theoretical studies of community food webs based on randomly assembled systems with Lotka-Volterra type dynamics generally find these systems to be mostly unstable [139; 266; 267; 335; 304; 305; 3951, but studies of the structural properties of real world ecosystems suggest that they are not randomly assembled [115;24; 289; 358; 4191. More recent theoretical studies based on the analysis of small species models show trends where certain topological regularities are preferred -in some sense selected for- over others [290;419]. The results presented in this paper also support this notion. It appears that the more stable motifs are being selected for, and as a result occur more frequently, and across a wide range of communities. It is also important to note that the environment in which an ecosystems is embedded places constraints on resources - s u c h as food, water, space and temperature. These constraints drive the evolution of the community towards a robust topology. Knowledge of the context of an environment coupled with an understanding of the dynamics and the structure of the interactions may be able to shed light on the natural laws that govern complex ecological communities, and explain why the seemingly infeasible systems such as the Antarctic pack ice food web described above could persist. From the analysis presented here we can speculate on some heuristics that govern the assemblage of community food webs. First, those motifs with no cycle components are the most stable (we will call these structures tree like). Of the tree like structures feed forward loops, (along with other linear structures) are the most stable and appear most frequently in community assemblages. This result is also supported by the analysis of the spectra of graphs, which show these structures are the most linearly stable. Second, long cycles seem to be important. Those motifs that have long cycles tend to be more stable. Again, this is supported by more detailed eigenvalue analysis that suggests that long weakly coupled loops have a stabilizing influence on dynamical systems. While loops within loops seem to have a stabilizing effect, this is only up until a certain extent. Fully connected motifs seem to be less stable over certain parameter ranges. This notion also suggests that the inclusion of a treelike component that breaks several loops, or joins loops in a certain way, plays a key role in promoting stability. These results suggest that there is a trade-off between stability and complexity, but if we are clever about the arrangement of the interac-


198

tions within a system, large complex community assemblages are feasible. 14.6

Closing comments

In this paper we have attempted to reconcile the stability properties of small groups of interacting species, with observed local patterns of interaction (motifs) in community food webs. It appears that those motifs which are most stable are most frequently encountered in community food webs. This suggests that these stable motifs may be the basic building blocks of complex ecosystems. The most dramatic structural change that many ecosystems face is from human-driven biodiversity loss. Following our natural feeding patterns and needs for material resources, we have historically tended to impact on higher trophic levels through over fishing and hunting. Often the exhaustion of a natural species population results in a string of extinctions that cascade through the community. In this paper we have investigated the stability properties of fixed motifs. One future avenue for further research is to investigate how the stability properties of a motif change as nodeslarcs are removed, and the nature of the interactions change. Such studies could provide insights into the possible effects of human impact (and natural extinction) on ecosystem services. Finally, the findings presented here open at least three lines of further inquiry. First, how can these ecological generalizations be explained in terms of the large scale behaviour and population dynamics of individual and collective species ensembles? Second, do these topological features of ecological organization explain other significant feature of food webs, such as species turnover? Third, what structural and dynamical characteristics of the individual communities account for their deviations from these overall trends? Understanding these questions in the context of the structural properties of community food webs may provide insights into the dynamical behavior they can support.

Acknowledgements The authors wish to thank Ross Thompson, Karl Havens and David Raffaelli for providing the data upon which this study was based.

Chapter 15

Quantification of Emergent Behaviors Induced by Feedback Resonance of Chaos A. Pitti, M. Lungarella, and Y . Kuniyoshi Lab. for Intelligent Systems and Informatics Dept. of Mechano-Informatics The University of Tokyo, Tokyo, Japan Email: { alex,maxl,kuniyosh} @isi.imi.i.u-tokyo.ac.jp We address the issue of how an embodied system can autonomously explore and discover the action possibilities inherent to its body. Our basic assumption is that the intrinsic dynamics of a system can be explored by perturbing the system through small but well-timed feedback actions and by exploiting a mechanism of feedback resonance. We hypothesize that such perturbations, if appropriately chosen, can favor the transitions from one stable attractor to another, and the discovery of stable postural configurations. To test our ideas, we realize an experimental system consisting of a ring-shaped mass-spring structure driven by a network of coupled chaotic pattern generators (called coupled chaotic fields). We study the role played by the chaoticity of the neural system as the control parameter governing phase transitions in movement space. Through a frequency-domain analysis of the emergent behavioral patterns, we show that the system discovers regions of its state space exhibiting notable properties.

15.1

Introduction

How do infants explore their bodies and acquire motor skills? How do humans and other animals adapt to unexpected contingencies and opportunities in a dynamic and ever changing environment such as the real world? Or more in general, what are the mechanisms that allow a complex embodied system consisting of a multitude of coupled and potentially heterogenous el199

200

Recent Advances in Artijicial Lge

ements to autonomously explore, discover, and select possibilities for action and movement? These are difficult issues whose answers despite intensive efforts still elude us. The main goal of this paper is to shed some light on how body dynamics might be explored. Exploration and emergence represent important first steps towards gaining further insights into how higher level cognitive skills are bootstrapped. Nikolaus Bernstein was probably the first to address in a systematic way the question of how humans purposefully exploit the interaction between neural and body-environment dynamics to solve the complex equations of motion involved in the coordination of the large number of mechanical degrees of freedom of our bodies [41]. In the last decade or so, Bernstein’s degrees of freedom problem has been tackled many times through the framework of dynamical systems (e.g. [214;3811). Such research has three important implications which are relevant for this paper. First, movements are dynamically soft-assembled by the continuous and mutual interaction between the neural and the body-environment dynamics. Second, embodiment and task dynamics impose consistent and invariant (i.e. learnable) structure on sensorimotor patterns. Third, when the neural dynamics of a system is coupled with its natural intrinsic dynamics, even a complicated body can exhibit very robust and flexible behavior, mainly as a result of mutual entrainment (e.g. neural oscillator based biped walking [378] and pendulation 12521). In this paper, we pursue further the idea of a network of chaotic units used as a model for exploration of body dynamics [229]. One of the core features of our model is that it allows to switch between different attractors while maintaining adaptivity. We make two main contributions: The first one is the introduction of a mechanism of feedback resonance of chaos in our model. The second contribution is a set of tools for analyzing the resulting spatio-temporal dynamics. In the following section, we will present the three pillars on which our augmented model rests: (a) dynamical systems approach, in particular the notions of global dynamics and interaction dynamics; (b) mechanism of feedback resonance thanks to which the neural system tunes into the natural frequencies of the intrinsic dynamics of the mechanical system; and (c) concept of coupled chaotic fields which is responsible for exploration of the neural dynamics. We then introduce a set of methods used to analyze the spatio-temporal dynamics of the neural and mechanical system. Subsequently, we describe our experimental setup and report the outcome of our experiments. In the last section, we discuss our results and conclude by pointing to some future work.

Emergence of Behavioral Patterns Through Feedback Resonance of Chaos

15.2

201

Model System

In this section, we introduce the three key elements of our model of exploration. We briefly describe dynamical systems theory, and the concepts of feedback resonance and coupled chaotic field.

15.2.1

Dynamical Systems Approach

Dynamic systems theory is a well-established framework primarily geared towards modeling and describing nonlinear phenomena that involve change over time [372;3831. From a dynamical point of view, systems are typically described in terms of their evolution over time, their robustness against internal and external perturbations, number and type of attractors and repellors, as well as bifurcations, that is, qualitative changes of the dynamics of the system occurring at critical states. A dynamical system - initialized in a particular stable attractor state and affected by noise - fluctuates irregularly inside it despite internal and external perturbations. The system cannot evolve to a new state until the pertubations reach a certain level triggering a transition to a new (possibly more stable) attractor. Here, we conceive of perturbations (internal and external ones) (a) as a means to explore and discover stable as well as unstable action possibilities of a mechanical system, and (b) as a mechanism of adaptation against environmental changes. Our approach is reminiscent of the process of chaotic itinerancy which can be defined as the chaotic transition dynamics resulting from a weak instability of the attractor landscape of the dynamical system [214]. By contrast, control theoretical approaches (e.g. [360]),including adaptive methodologies, are typically framed as abstract mathematical problems and avoid exploiting any nonlinear physical aspect in the control process such as interactions, perturbations, body dynamics, and entrainment.

15.2.2

Feedback Resonance

The second key element of our approach is feedback resonance [130;2981. This mechanism indicates that small but timely feedback actions can dramatically affect the dynamics of a nonlinear system, e.g. turn chaotic motions into periodic ones. The rationale is that by having the feedback actions occur at a specific instant in time, it is possible either to entrain a system to the action or to destabilize it and induce a transition to a new behavioral pattern. The phenomenon can be conceptualized as the energyefficient excitation through resonance of the many degrees of freedom of a system. Resonance is pivotal because when the frequency of oscillation

202


of the system matches its natural vibration frequency, the system absorbs injected oscillatory energy more effectively. Action on the system using feedback resonance is described as:

) the dynamics of the system (here, the where Fi(t) >> y ~ i ( t expresses force acting on motor i at time t ) ,and ~ i ( tis)the controlled variable (here, the neural excitation) scaled by a parameter y. By explo'iting resonance, the system can amplify the small perturbations and dramatically affect the global dynamics of the system inducing bifurcations and new postural configurations (in the case of a mechanical system). The resonant forces are used to transfer energy to new behaviors. The general idea is akin to the concepts of global dynamics and intervention introduced by [434]. The work done in physics has mainly focused on idealized pendular systems and chaotic models [130]. By contrast, our framework explicitly includes information about the morphology of the embodied system (i.e. distribution and type of actuators and sensors), the properties of the environment, as well as the coupling between body and neural system. The dynamics of the body embedded in the environment is exploited and natural resonant states are discovered.

15.2.3

Coupled Chaotic Field

We used a chaotic pattern generator as an internal source of perturbation (external perturbations are gravity, impactive forces acting on the body, and so on). As a model of neural activity we chose the globally coupled map (GCM) which is a network of chaotic elements instantiating a minimal model that has local chaotic dynamics but also global interaction between the elements [214]. Although simple, such maps exhibit a rich and complex dynamics. A coupled chaotic field (CCF) is essentially a globally coupled map in which every chaotic element is connected to the embodied system through sensors (supplying the input) and motors (to which the output is relayed). In other words, the global coupling between the chaotic units is provided by the environmental interaction (see Fig. 15.1). Formally, the neural system can be described as follows:


I

.

203

.

Env i ronment Fig. 15.1 Outline of our model.

1 - a ~ i ( t>> ) ~qFi(t) with a ~]0.0,2.0] and

E

~]0.0,0.4]

The parameters a and E are the two control parameters of the CCF, and fa(z)realizes the chaotic dynamics of the neural unit at time t. The variable Q determines the chaoticity of each neural unit and E controls the level of synchronization among the units. In our experiments] the only control parameter was the chaoticity. The level of synchronization was fixed to a small number ( E < 0.05) so as not to bias the emergence of synchronized and coherent states. In fact, rather than initializing the synchronization parameter to a specific value among the units and have a static coupling among the chaotic system, the coupling was dynamically altered through internal and external perturbations (here, the CCF was perturbed by the output of force sensors located in the joints). The coupling constant q (eq. 15.2.3) was selected empirically. Its value was small enough so as not too affect too much the intrinsic dynamics of the chaotic units. The output of the CCF was then fed - after appropriate scaling - to the motors (eq.15.1). By letting neural and body-environment dynamics interact many instances of mutual entrainment among the two dynamical systems could be observed. This kind of mutual interaction has been called global entrainment] and has been hypothesized to generate stable and flexible movements in a self-organized manner despite unpredictable changes in environmental conditions [378].


204

15.3 Methods

In this section, we introduce the tools used to analyze the data generated in our experiments. The main purpose of our methods is to quantitatively assess behavioral patterns. We describe three methods: (1) the “Spectral Bifurcation Diagram, (2) the “Wavelet Transform”, and (3) a novel visualization method inspired by methods 1 and 2, which we shall refer to as the “Wavelet Bifurcation Diagram.” The Spectral Bifurcation Diagram is a recently introduced method for investigating the qualitative changes of the dynamics of high-dimensional systenis [296]. Essentially, it displays the power density spectrum of a system variable as a function of a control parameter of the system (the power spectra of the individual variables are superposed). The control parameter is a variable to which the behavior of the system is sensitive and that moves the system through different (attractor) states. The representation illustrates how the neural system affects the coordination among the different parts of the body in frequency space. This method allows to identify resonant states which are characterized by sharp frequency components, chaotic states having rather broad power spectra, as well as bifurcations, that is, abrupt transitions from one attractor state to another. The second method used was the Wavelet transform [257]. In the Wavelet space, one variable of the system is projected onto a space spanning time and frequency in which it is possible to identify changes of behavior at different time scales, short-range as well as long-range temporal correlations. The Wavelet transform thus allows to analyze temporal bifurcations, and consequently the evolution of the dynamics of each unit of the embedded neural system. Because we are also interested in understanding the spatial correlations between neural units while performing a certain movement, as well as the type of interactions at different spatio-temporal scales, we introduced a novel tool for the analysis of high-dimensional systems spanning frequency, time, and index of the neural unit. In this space, we are not only able to identify spatio-temporal correlations but also changes over time of each unit or groups of units as well as bifurcations in the dynamics of systems with a large number of degrees of freedom. We refer to this method as Wavelet Bifurcation Diagram.

15.4

Experiments

For our experiments we used a ring-shaped mass-spring robotic system composed of 10 prismatic elements connected by 10 force-controlled sliding


Y

E

r]

0.001

0.001

0.01

density 2.0

width 0.1

damping 0.3

205

stiffness 0.5

joints (Fig. 15.2). Each element was connected to its two neighbours by compression springs. The spring-motor complex had 3 degrees of freedom (DOF), and the complete mechanical system had a total of 30 DOF. The neural and body dynamics were mutually coupled as explained previously (Eqs. 15.1 and 15.3). The robot was placed in a plane devoid of obstacles and of other external perturbations except gravity, ground reaction forces, and friction. We realized our ring-shaped robot using a physicsbased simulator (see [355]). Table 15.1 shows the parameters we used for all our simulations. It is important to stress that despite its simplicity, our robot model can display a sufficiently large set of behaviors, and is therefore appropriate for investigating the mechanisms underlying the emergence of stable as well as as unstable behavioral patterns. Our analysis was mainly focused at understanding how the body dynamics evolved in time as a function of the chaoticity a of the neural units. The initial value of a was zero. For this value the output of the neural system was a constant, and the system did not move. By increasing by a small amount the chaoticity of the neural units ( a ~]O.0,0.1]), the “ring”

Mars

\

Motor

Spring and Damper

Fig. 15.2 Schematic representation of our robot model. The masses are prisms and are connected to each other by force-controlled compliant linear actuators. The system has 30 degrees of freedom.

206


started to vibrate almost unperceivably at a very small spatial scale. After a certain amount of time, the seemingly random movements converged to a slow rhythmical rocking movement, that is, the system's dynamics had found a stable attractor. Figure 15.3 shows the phase space trajectories of

i

l

Fig. 15.3 Left: Snapshots of different postures for different a; (a) balance (a = 0.05), (b) rolling (a = 0.10), (c and d) quick and unstable movement ( a = 1.0 and a = 1.5), and (e) uncoordinated movement pattern (a= 1.9). Center: Time series of one joint force pressure (unit: N). Right: Tim+delayed phase potraits of two arbitrarily selected neighboring joints as a function of a.


207

two arbitrarily selected neighbouring joints for different values of a. It plots ( t )- Xi(t - t’) against Xj( t )- Xj(t - t’) giving some information concerning the spatial correlations (Xivs. X j ) and the temporal correlations between joint variables (d = 10). Up to a very specific level of chaoticity ( a = 0.097477), the system stroke a perfectly poised balanced posture, and did not have sufficient energy to start rolling. By slightly increasing the control parameter ( a = O.l), a phase transition occurred, and the system started to roll. Interestingly, for values of Q < 0.15, the system oscillated quite unpredictably between rolling and balancing. We hypothesize that the emergence of particular movement patterns depends on the presence or absence of entrainment between neural and body-environment dynamics. For levels of neural chaoticity Q E [0.15;0.41, novel behaviors and patterns of locomotion emerged. The neural system seemed to exploit the natural dynamics of the ring-shaped body to balance, rock, roll, accelerate and decelerate, and in rare occasions, even to jump. We suggest that the unstabilities in the movements patterns were mainly caused by micro-scale perturbations acting on the neural dynamics with a consequent disruption of the entrainment between neural and body-environment dynamics and the emergence of new locomotion patterns. In the range a E [0.4; 1.21, the perturbations were larger and thus had a more pronounced effect on the system’s dynamics. While still displaying coherence (that is, rather strong correlations between neighbouring segments of the ring), the movements were generally more complex and characterized by abrupt changes. Most notable was the high sensitivity of the system to internal (neural) and external perturbations (due to sensory feedback) , and the emergence of a number of different behavioral patterns. The ring rolled quickly, accelerated, decelerated, and displayed many unstable postural configurations (such as balancing on an edge). Finally, for Q in the interval [1.2;21, the system did not display any coherent or organized movement patterns. The activation levels of the chaotic system were too large to be influenced (perturbed) by sensory feedback causing a disruption of entrainment between neural and body-environment dynamics.

xi

15.5

Analysis

One immediate implication of our experiments is that in an embodied system, the exploration of the space of possible coherent and stable postural modes is induced by the mutual adaptation between neural and bodyenvironment dynamics. In other words, the neural system, the body, and

208


the environment are all responsible for the emergence of particular movement patterns. Note that this global view contrasts with the one that sees only neural parameters responsible for exploration of the movement space.

15.5.1

Analysis 1: Body movements

The Spectral Bifurcation Diagram for varying levels of chaoticity is reproduced in Figure 15.4. Low levels of chaoticity (a< 0.05) are characterized by sharp peaks in the power spectral density of the force sensors located in the joints, and given a particular value of Q! the resonance response is close to the one of a damped oscillator. The low frequency component around 10 H z dominates the interaction dynamics between the neural system and the ring. This frequency corresponds to the fundamental mode of the coupled system, that is, its eigen-frequency. For this frequency, the joints are highly synchronized and the system displays a high degree of coordination. A minimal amount of energy is required to move the system and to transfer energy to the different parts of the body. When the chaoticity increases higher harmonics appear introducing discontinuities in the resonance response. The main resonance persists for all values of a but we observe abrupt changes and bifurcations in the magnitude of other peaks. The new harmonic peaks are located at integer and fractional multiples of the first eigen-frequency [383]. The latter peaks are caused by small damped actions of the chaotic system and affect the joint properties, in the sense that a change of chaoticity of the neural system can induce a change in the stiffness of the springs in the joints. As a result the system is able to generate a large variety of patterns (stable, weakly stable, and unstable). When the amplitude of the harmonics is too large, it negatively affects the groups formed in different regions of the body generating decoherence and destroying stable activity patterns. Note that the harmonic states are intrinsic to the coupling between neural, body, and environmental dynamics, and even if the spectral patterns seem complex, they should not be considered to be the outcome of yet another kind of neural noise. We have previously suggested that behavioral changes are a complex function of the coupling between neural and body-environment dynamics. By using the Spectral Bifurcation Diagram we can now shed light on the patterns of neural activity leading to such changes. For example for a level of chaoticity a = 0.097477 (that is, when the ring starts to roll) t,he power spectrum has a second harmonic which disappears in the interval [0.1,0.13] (that is, when the system present difficulties to roll again) see 15.4. The more harmonics there are, the more complex the behavior, despite preservation of coherence of behavior.


209

Fig. 15.4 Spectral Bifurcation Diagram. Inset shows spectral peaks for low values of neural chaoticity. The control parameter is a.

15.5.2

Analysis 2: Neural coupling

We are interested in the spatio-temporal interaction patterns that emerge in the neural system. To get a better grasp on these patterns, we applied the Wavelet Transform to the activity of an arbitrarily selected chaotic unit for different values of the control parameter (see Figure 15.5). In the Wavelet space, the activities of the units disclose temporal correlations at different scales. The larger is the value of the control parameter, the higher is the complexity of the temporal patterns. Further analysis reveals a scale-free fractal-like structure in the neural activity and temporal coherence bridging multiple time scales. This result can be easily explained by considering the harmonics produced through feedback resonance. Long-range movements resulting from highly chaotic neural activity are composed of short-range movements triggered by lower values of chaoticity (Figure 15.5 c). One surprising effect is that the higher harmonics can dynamically alter the stiffness of the springs and thus their temporal responses. “Positive” resonance hardens the springs, and ‘‘negative”resonance softens them (see also [383]). We note also that the temporal scale of correlations seem to be quantified with respect to the control parameter (see horizontal lines in Figures 15.5 a-d). In other words, changes of “locomotion pattern” occur for specific values of neural chaoticity. The Wavelet Bifurcation Diagrams of the neural activity for different values of the control parameter are shown in Figures 15.6 and 15.7. Through the mutual interactions between the neural sytem and the body, groups of neural units synchronize at multiple spatial scales (vertical axes of the figures). As in the case of the Wavelet Transform, the scale at which synchronization takes place and the type of emerging patterns depends on the amount of chaoticity in the neural system.

210

Recent Advances in ArtQicial Lafe

iW

c)

lime [mr)

200

3CC

4W

5W

Iime(ms)

Fig. 15.5 Wavelet Transform. The spectro-temporal correlations between chaotic units are displayed. The control parameter CY varies linearly in the time interval 0-500msec from (a) [0.0;0.5], (b) [0.5;1.0], (c) [1.0;1.5]and (d) [1.5;2.0].

Interactions between neural system and body for low values of chaoticity form long-range correlations at a low spatial scale (rolling and balancing behavior). The “low-scale” spatio-temporal patterns correspond to disconnected short-range movements and the “high-scale’’ ones correspond to long-range movements. For higher chaoticity values, the same fractal-like spatio-temporal organizations are formed. Note that the control parameter is correlated to the complexity degree of the emergent behaviors. For low values of a , stable activity groups of neurons are generated in the chaotic system. For higher values of a , we can observe chaotic itinerancy in the system with an higher spatio-temporal complexity structure in the units. The groups are unstable and bifurcate to new transient configurations.

15.6 Discussion and Conclusion In this paper, we introduced and discussed a novel framework for exploring action possibilities of complex mechanical systems. In particular, we studied (a) how chaotic neural activity can drive the exploration of movement patterns, and (b) how feedback resonance can be used to “tune into” particularly efficient movements. We also provided a set of tools to quantitatively measure the spatio-temporal organization of the neural system,


211

time (ms)

Scale 2 ~

~

Scale 1

Scale 0

time (ms)

Fig. 15.6 Wavelet bifurcation diagram at different scales. In all three scale-plots, the horizontal axis denotes time (0-500 msec), and the vertical axis is the index of the chaotic unit.

and the stability of the emerging behavioral patterns. We suggest that resonance plays a pivotal role for learning to control our bodies. Resonant states act as some kind of amplifier guiding the exploration and discovery of intrinsic modes of the body dynamics. One important side-result is the reduction of the number of degrees of freedom despite an increase in the overall complexity of the system. Another result is that resonance pushes the compliant actuators composing the body to dynamically alter their properties (e.g. stiffness) and to cooperate. In a sense, resonance also satisfies the principle of cheap design [303]. This principles states that when designing a system it is better to exploit physics and

212


10 20 30 Chaotic iltlitS

).

10 20 Chaotic Units

3

10

20

Chaotic Units

30

10

20

30

Chaotic Units

10

20

Chaotic Units

Chaotic [Jnits

30

In 20 30 Chaotic IJnits

Fig. 15.7 Wavelet bifurcation diagram at different time instants. Snapshots of different neural configurations taken at different time instants for varying levels of chaoticity: a) a = 0.5, b) a = 1.0, c) a = 1.5, d) a = 1.9. The horizontal axis denotes the index of the chaotic unit, the vertical one is the scale (adimensional).

the dynamics of the system-environment interaction. Mapped onto our case study it means that resonance guarantees the emergence of energy-efficient movement patterns. As for learning or planning, this property can also be useful to understand when to increase or decrease the coupling between parts of the body, and to understand which parts have to be linked rather than testing all possible combinations. In addition, critical states (e.g. corresponding to unstable activity patterns or states where bifurcations occur) can also be identified and analyzed. We hypothesize that a mechanism of

Emergence of Behavioral Patte rn Through Feedback Resonance of Chaos

213

feedback resonance is responsible for combining unstable short-range patterns into stable long-range ones. In future works we intend to implement our exploration model in a real robot situated in a dynamic environment. The robot will hopefully autonomously explore its body, and over time acquire a repertoire of complex, adaptive, and highly dynamic movements. 15.7 Acknowledgements

The authors would like to thank the Japan Society for the Promotion of Science for funding, as well as K. Shiozumi and S. Suzuki for valuable discussions.


Chapter 16

A Dynamic Optimisation Approach for Ant Colony Optimisation Using the Multidimensional Knapsack Problem M. Randall School of Information Technology Bond University, QLD 4229, Australia E-mail: [email protected] Meta-heuristic search techniques have been extensively applied to static optimisation problems. These are problems in which the definition and/or the data remain fixed throughout the process of solving the problem. Many real-world problems, particularly in transportation, telecommunications and manufacturing, change over time as new events occur, thus altering the solution space. This paper explores methods for solving these problems with ant colony optimisation. A method of adapting the general algorithm to a range of problems is presented. This paper shows the development of a small prototype system to solve dynamic multidimensional knapsack problems. This system is found to be able to rapidly adapt to problem changes.

16.1

Introduction

Many industrial optimisation problems are solved in non-static environments. These problems are referred to as dynamic optimisation problems and are characterised by an initial problem definition and a series of “events” that occur over time. An event defines some change either to the data of the problem or its structural definition. A dynamic operating environment can be modelled in two ways. Either a series of events may be determined a priori in which case the optimum solution can be pre-determined, or a probabilistic discrete event simulator is used to 215

216

Recent Advances in Artajicial Life

create events. While the latter mimics real world problems more closely, the former is better for testing and development purposes. In comparison to static optimisation problems, dynamic optimisation problems often lack well defined objective functions, test data sets, criteria for comparing solutions and standard formulations [32;114;3411. At the present time, the main strategies used to solve these problems have been specialised heuristics, operations research and manual solving [33;3411. Commercial implementations of operations research software can also be used by providing additional constraints at each problem change in order to lock some existing solution components in place. An example of manual solving is given by aircraft controllers at London’s Heathrow airport determining appropriate aircraft landing schedules [33]. Evolutionary algorithms such as genetic algorithms have been modified to accommodate dynamic optimisation problems. A survey of these approaches is given by Branke [56]. However, for a group of meta-heuristic strategies known collectively as Ant colony Optimisation (ACO) [110], relatively little work has been done in this area. This is despite the fact that ACO offers possibilities in this direction. Natural ants are adaptive agents that must source food in a continually changing environment and supply this back to a nest. Food sources constantly change, obstacles appear and disappear and new routes become available, while old routes become cut off or impractical to traverse. One of the major differences between ACO meta-heuristics and other evolutionary algorithms is that they are constructive, i.e., they build solutions a component’ at a time. This paper therefore develops new generalisable strategies that are applicable to ACO. In terms of artificial ant systems, some successful work (in the main) has already been done on producing specialised ACO implementations for the travelling salesman problem [16;1231 and telecommunication and routing problems (see Dorigo, Di Car0 and Gambardella [lll]for an overview). This paper, however, looks at the issue related to adapting the standard algorithm to suit a wide range of combinatorial optimisation problems. It is an initial investigation demonstrating the effective implementation of a system to solve dynamic multidimensional knapsack problems . Another attempt to generalise ACO for dynamic problems has been made. Population ACO (P-ACO) [165;1661 is an ACO strategy that is capable of processing dynamic optimisation problems. It achieves this by using a different pheromone updating strategy. Only a set of ‘elite’ solutions are used as part of the pheromone updating rules. At each iteration, one solution leaves the population and a new one (from the current l A solution component is the building block of the solution. Two examples are a city for the travelling salesman problem and a facility for the quadratic assignment problem.

A Dynamic Optimisation Approach for Ant Colony Optimisation

217

iteration) enters. The candidate solution to be removed can be selected on age, quality, a probability function or a combination of these factors. The authors argue that this arrangement lends itself to dynamic optimisation as extensive adjustments (due the problem change) need not be made to the pheromone matrix. Instead, a solution modified to suit the new problem is used to compute the new pheromone information. This modification process is a heuristic called KeepElite [167] that works for full solutions only and is tailored for particular problems. The approach proposed herein allows for the automatic adaptation of a solution to suit a changed version of the problem. The main difference between this and P-ACO is that it incorporates a general process that adapts partial solutions to new problem definitions. This has the advantage that the solver can process a change to the problem at any time without having to wait for the ants’ constructive process to finish. The remainder of the paper is organised as follows. Section 16.2 describes how standard ACO may be adapted to solve dynamic optimisation problems. Preliminary results of this approach (using the dynamic multidimensional knapsack problem) are given in Section 16.3 while Section 16.4 gives the conclusions of this work. Note that a thorough description of the ACO meta-heuristic (and its variants) is given by Dorigo and Di Caro [110].

16.2 Adapting ACO to Dynamic Problems 16.2.1

Overview

Beasley, Krishnamoorthy, Shariah and Abramson [32] describe a generic displacement model that allows a static integer linear program (ILP) to be transformed into another static ILP, given that some change takes place to the original ILP. The model encourages (by the use of penalty functions) new solutions to be close to the previous solution. The rationale is that high quality solutions to the new problem should be relatively close to those of the old problem as usually only a small structural/data change to the problem has been made. The ideas presented as part of the displacement problem can be adapted to form part of a generic strategy that is suitable for ACO. The following stages define a framework to solve dynamic optimisation problems in a real-time/production environment for constructive meta-heuristics (in particular ACO). The following is a set of generic rules, the details of which (such as when to use improved solutions in the production environment) may vary according to industry and management strategy.

(1) While no changes occur to alter the problem definition/data, allow

218


the meta-heuristic to solve the problem. Improved solutions are sent to the production environment. For instance, at an airport the production system would correspond to the control tower while for dynamic vehicle routing it would correspond to the dispatch section of the depot. As it may be costly or impractical to “reset” the production environment too frequently, the above may be done in a number of ways. For instance, the system may receive the revised solution at certain time intervals or when a significant improvement has been found (bounded by either an absolute or relative amount). (2) Suspend Stage 1 if either a number of events has occurred or an event of sufficient magnitude has occurred. This is very environment specific and in an industrial setting would need to be decided with management input. For instance, given the problem of scheduling aircraft to land on a runway, how many aircraft would need to enter the airspace of an airport before landing schedules need to be changed? (3) Use a solution deconstruction process to determine which components of an ant’s solution need to be discarded so that the feasibility of the “new” part of the problem is not violated. The ACO process is then restarted from this partial solution. It is best only to perform this deconstruction process on one solution as executing it on a population of solutions may yield different length partial solutions. This is potentially computationally costly and means that ants could not synchronously complete their solutions within an iteration. The most reasonable solution on which to perform this process would be the best solution from the current colony. This new partial solution is then copied to all the ants in the colony for the next iteration. This process is described in detail in Section 16.2.2. (4) Go back to Stage 1. It is critical that Stage 3 is very efficient. This is because the production environment must stop or be put in some ‘holding pattern’ while the solution deconstruction process and new set of solutions are being computed and evaluated respectively. However, the advantage is that the solver system does not have to be stopped and restarted from the beginning. The existing “best” solution to the problem should serve as a good starting point to the new problem (as has been found by Beasley et al. [32]). 16.2.2

The Solution Deconstruction Process

When an event occurs, it is probable that current solutions will become infeasible. However, as an event will typically change only part of the problem data/structure, it is likely that current solutions will only require

A Dynamic Optimisation Approach for A n t Colony Optimisation

219

small modifications to make them feasible [165; 1661. Rather than restarting the solution construction process, it is possible to modify a partial solution instead. In this way, it is similar to the Beasley et al.’s [32] process as it ensures that the solution to the modified problem will be relatively close to the previous best solution. The solution deconstruction technique progressively removes components from a solution until it becomes a feasible solution to the changed problem. In the generalised form presented herein, deconstruction is applicable to a wide range of dynamic combinatorial optimisation problems (some examples being dynamic travelling salesman and vehicle routing problems). The first part of the deconstruction process determines which constraints need to be satisfied at each step of the construction process and which constraints need only be satisfied on the termination of the process [321]. More specifically, the two types of constraints can be described as :

(1) Constraints that must be satisfied at each step. An example of such a N constraint is a knapsack capacity constraint of the form of Cixi < b. Regardless of the number of steps that have elapsed within an iteration, it is always necessary to satisfy such a constraint (given positive C coefficients and a positive value of b). (2) Constraints that cannot be satisfied at each step of the algorithm. Consider the case where a constraint of the form of CE1Cixi > b is present in the problem model. It is apparent that in the beginning stages of the construction, the length of the solution vector is small and it is unlikely that this constraint will be satisfied. Hence, these types of constraints must be treated as special. Only once these types of constraints have been satisfied can the algorithm consider the termination of the solution augmentation process. The analysis to determine which constraints fall into which categories can be done before the dynamic optimisation problem is solved (i.e., before Stage 1 from Section 16.2). If an event occurs that adds a constraint (within Stage 2), this analysis will need to be partially redone. Algorithm 16.1 shows the generic deconstruction algorithm. In this algorithm, only constraints of type 1 need to be processed. This algorithm simply takes the most recently added solution component and removes it, calculating the amount of feasibility violation. If this increases the infeasibility, the component is added back and the next newest component is tried. This process is repeated until a feasible solution to the new problem is produced (i.e., the value of the constraints’ violation is 0 ) . The algorithm has a complexity of O ( a M ) ,where a is the length of the partial solution and A4 is the number


220

A l g o r i t h m 16.1 The solution deconstruction process. Note: ‘constraints’ refer to constraints of type 1 and X is the solution. 1: c = Calculate the amount of constraint violation of X 2: if c > 0 then 3: a = the length of X 4: XI = x 5: w h i l e c > 0 do 6: component = X l ( a ) 7: XI = Remove component from X’ 8: Cprev = c 9: c = Calculate the amount of constraint violation of XI 10: if c > cprev then X’ = Add component to XI 11: 12: end if 13: a=a-1 14: end while 15: = X’ 16: end if

x

of constraints. Constraint violation is calculated according to the relational operators that are present in the constraints [321]. For instance, if the sign of a constraint is 5 and the left-hand side is larger than the right-hand side, the net difference is the amount of constraint violation. This is shown in Equation 16.1. The constraint violations (16.2-16.6) for the other signs are calculated in a similar manner. The sum of the constraints’ violations is given in Equation 16.7.

()

ci = MAX(0, Zhsi - rhsi

ci

+ 1)

= MAX(O, rhsi - Zhsi)

ci = MAX(0, rhsi - Zhsi

+ 1)

(16.2)

(16.3)

(16.4)

A Dynamic Optimisation Approach for Ant Colony Optamisation

(f)

ci=

{

(rhsi = Zhsi) 1 otherwise.

221

(16.6)

M

~ = C c i i=l

(16.7)

Where:

M

is the number of constraints,

la1 is the absolute value of a,

MAX(a, b) returns the larger value of a and b, ci is the constraint violation of constraint i , 1 5 i 5 M , Zhsi is the evaluation of the left hand side of constraint i , 1 5 i 5 M , rhsi is the evaluation of the right hand side of constraint i, 1 5 i 5 M and u is the total amount of constraint violation. 16.2.2.1 Event Descriptors Algorithm 16.1 is a naive heuristic that does not take into account the type of change made to the problem. As such, it is likely that that the algorithm will remove more components than is necessary to provide a new feasible partial solution. In order to ensure that deconstruction is efficient, the nature of possible events must be defined. These event descriptors can be used in conjunction with the algorithm in Algorithm 16.1 to efficiently adapt a solution to suit the altered problem (i.e., determine the most appropriate component to remove). An event descriptor may be any of 0

0 0 0 0 0

ADD-COMPONENT REMOVE-COMPONENT ADD-CONSTRAINT REMOVE-CONSTRAINT MODIFY-CONSTRAINT MODIFY-OBJECTIVE

The last descriptor is superfluous (but is present for completeness) as such an event will not change the feasibility of the solution, only its cost. An example of an event descriptor for the previously described knapsack event

222

Recent Advances in Artificial Lafe

Algorithm 16.2 The modified solution deconstruction process to include event descriptors. 1: c = Calculate the amount of constraint violation of X 2: if c > 0 then 3: a = the length of X 4: XI = x 5: while c > 0 do 6: component = evaluate event descriptor(event,X ) 7: XI = Remove component from X’ 8: Cprev = c c = Calculate the amount of constraint violation of X’ 9: 10: if c > cprev then 11: X‘ = Add component to X‘ 12: end if 13: a=a-1 14: end while 15: x = X‘ 16: end if would be: “On REMOVE-COMPONENT a,remove +a from X ” . This has the advantage of being an 0(1)operation. The modified deconstruction algorithm is given in Algorithm 16.2. In a general solver system, the description could be implemented as a set of high level rules, without the need to reprogram the solver.

16.3

Computational Experience

In order to test the concepts outlined in the previous section, a relatively straightforward problem has been chosen rather than an industrial application (which is in the future development of this project). The multidimensional knapsack problem (MKP) [SO] is an extension of the classical knapsack problem. In it, a mix of items must be chosen that satisfy a series of weighting constraints whilst maximising the collective utility of the items. Equations 16.8 - 16.10 show the 0-1 ILP model. N

Maximise

Pixi i= 1

s.t.

(16.8)


223

(16.10) Where:

xi is 1 if item i is included in the knapsack, 0 otherwise, Pi is the profit of including item i in the knapsack, N is the total number of items, wij is the weight of item j in constraint i, M is the number of constraints, and bi is the total allowable weight according to constraint i. In order to make this problem dynamic, MKP problem instances from OR-Library [31]have been adapted (hereafter referred to as variants). This has been done by taking the instance of the problem and modifying it in the following six ways:

(a) No Change - The original problem. (b) Modify the capacity of each constraint - This is achieved by setting each capacity to 80% of its original level. (c) Remove a constraint - Removes a particular constraint. (d) A d d a constraint - Adds a particular constraint. (e) Remove an item - Removes a particular item from the item mix. (f) A d d an item - Adds a particular item to the item mix. Any and all of these changes occur within a particular run of the ACO solver. They take place at the step level of the ACO algorithm and the likelihood of a problem change is given by a Poisson distribution. To calculate the number of events that will occur within an interval (defined by a number of ACO steps) C, the average number of events that will occur within this period, A, is required. From this, the probability of an event occurring at each ACO step can be calculated. Within the solver system, this probability is generated each C steps. The simulated environment notifies the solver (i.e., the ACO engine) if a change to the problem has been made. As the problem variants are fixed, the results can be compared to a standard (static) ACO solver. The problem instance descriptions' are as follows: *These are available online at http: //www.it.bond.edu. au/randall/dynmkp.tar

224


Problem Name mknapl mknap2 mknap3 mknap4 mknap5 mkna~6

N 6 20 28 39 50 60

M 10 10 10 5 5 5

Optimal Value 3800 6120 12400 10618 6339 6954

The only event descriptor that will be required is: “On REMOVE-COMPONENT, remove+N 1 from X ” . It is necessary because the identifier number of each knapsack item will be decremented for all the items above the item dropped. Therefore, if item N 1 (where N is the number of items in the new problem) is present in the solution, it is no longer valid and must be removed to ensure a feasible solution. The computing platform used to perform the experiments is a Sun Ultra 5 (rated at 400 MHz). Each problem instance is run across 10 random seeds. The Ant Colony System (ACS) approach [112] will be used as the solver. Its parameter settings are as follows: TO = O . O l , P = 2 , y = 0 . 1 , ~ = O.l,qo = 0.9,C = 200,X = 5,ants = 10,iteration.s = 3000. Note that the ACO parameters TO,y,p, Q and QO are defined in Dorigo and Gambardella [112]. In order to describe the range of objective costs gained by these experiments, the minimum (denoted “Min”), median (denoted “Med”), maximum (denoted “Max”) and Inter Quartile Range (denoted “IQR”) are given. Non-parametric descriptive statistics are used as the data are highly non-normally distributed. The first half of Table 16.1 shows the results of solving each of the five variants (and the original) of the test problems. This was done using the static version of the ACS solver. The second half shows the results of the dynamic system. In it, the best objective cost achieved for each variation of each problem is recorded. To determine the degree of adaptation from one problem variant to the next, the number of ACS iterations between each change and attaining the best objective cost (within that period) is also recorded. The results from Table 16.1 indicate that the dynamic ACS engine can very quickly adapt to a new problem, usually within a small number of iterations. In some cases, after the solution deconstruction procedure has been performed, ACS constructs a solution whose cost is the best known for that particular problem within that iteration. Unlike the other problems, improved objective costs were received for mknap5 and mknap6 using the dynamic solver over the static solver. A possible reason for this is that the change in pheromone values due to changing problem definitions may allow ants to become free of attractive

+

+


225

Table 16.1 The results of solving each of the test problems (and their variants) using

the static and dynamic solvers. Problem

mknapl

Variants

a b C

mknap2

d e f a b C

mknap3

mknap4

d e f a b c d e f a b C

d e

mknap5

f a b C

mknap6

d e f a b C

d e f

Min 3800 3300 3300 2400 3700 3900 6120 5200 4475 6450 6120 6120 12380 10750 10310 9380 11950 13070 9909 4260 9936 9909 9723 11533 6285 5739 5389 6285 6110 6711 6923 6161 6923 6665 6765 7101

Static Solver Objective Costs Med Max IQR 3800 3800 0 3300 3300 0 3300 3300 0 2400 2400 0 0 3700 3700 3900 3900 0 6120 6120 0 5200 5200 0 4475 4475 0 6450 6450 0 6120 6120 0 6120 6120 0 12380 12380 0 10750 10750 0 10310 10310 0 9440 9440 0 11970 11970 0 13070 13070 0 9981.5 10130 97 4260 4260 0 10080 10202 178.25 10061 10157 90 9730 9771 5.25 11546 11600 47 6285 6285 0 5739 5739 0 5389 5389 0 6285 6285 0 6110 6110 0 6711 6711 0 6923 6923 0 6176 6193 17 6923 6923 0 6665 6665 0 6765 6765 0 7101 7101 0

Best Cost 3800 3300 3300 2400 3700 3900 6120 5200 4475 6450 6120 6120 12380 10750 10310 9380 11970 13070 10120 4260 10084 10141 9777 11534 6339 5810 5389 6339 6110 6765 6954 6250 6954 6783 6765 7101

Dynamic Solver Trials Min Med Max 0 1 10 0 1 11 0 1 12 0 2 20 0 1 11 0 1 18 0 0 4 0 1 7 0 0 2 0 0 3 0 0 4 0 0 3 0 0 1 0 0 4 0 0 1 0 0 5 0 0 1 0 0 2 0 0 1 1 3 41 0 0 1 0 0 0 0 0 1 0 0 2 0 0 1 0 0 1 0 0 3 0 0 1 0 0 1 0 0 1 0 0 1 0 0 1 0 0 1 0 0 1 0 1 1 0 0 1

IQR 2

2 2 3 1

1 1 1 0

1 1 1 0

0 0 1 0

0 0 4 0 0

0 0 0

0 1 0 0 0 0 0 0 0

0 0

local optima. A similar effect has been observed in Kawamura, Yamamoto and Ohuchi [220].

16.4 Conclusions This paper is a preliminary investigation that has presented methods for generalising ACO so that it can solve dynamic optimisation problems. It was shown that a modified ACS engine, that incorporates solution deconstruction and event descriptors, can solve dynamic multidimensional

226

Recent Advances in Artaficial Life

knapsack problems, within the framework given in Section 16.2. In many cases, the ACS engine can adapt to a change in problem definition within an ant colony iteration. The dynamic solver also managed to receive better solution objective costs than the static solver on the largest problems. The solution deconstruction and subsequent adaptation is very efficient for this class of problems. Due to the complimentary nature of P-ACO and the new approach, it would be interesting to combine P-ACO’s pheromone updating mechanism with solution deconstruction and event descriptors. Implicit in this would be to conduct performance comparisons between the new approach, P-ACO and the hybrid of both. Beyond this, the ultimate aim is to apply these techniques to industrial dynamic problems using a probabilistic event simulator. A suitable application would be the aircraft landing problem [32;3221. From here, a generic system capable of processing a range of problem definitions and utilising other constructive meta-heuristics (such as GRASP [125]) will be developed.

Chapter 17

Maintaining Explicit Diversity Within Individual Ant Colonies M. Randall School of Information Technology Bond University, QLD 4229 Australia E-mail: [email protected] Natural ants have the property that they will follow one another along a trail between the nest and the food source (and vice versa). While this is a desirable biological property, it can lead to stagnation behaviour within artificial systems that solve combinatorial optimisation problems. Although the evaporation of pheromone within local update rules, mutating pheromone values or the bounding of pheromone values may alleviate this, they are only implicit forms of diversification within a colony. Hence, there is no guarantee that stagnation will not occur. In this paper, a new explicit diversification measure is devised that balances between the restriction and freedom of incorporating various solution components. In terms of the target applications, the travelling salesman problem and quadratic assignment problem, this form of diversification allows for the comparison of sequences of common solution components. If an ant is considered too close to another member of the colony, it is explicitly forced to select another component. This restriction may also be lifted if necessary as part of the aspiration criteria. The results reveal improved performance over a control ant colony system.

17.1 Introduction Ant Colony Optimisation (ACO) represents a group of powerful techniques for solving combinatorial optimisation problems (COPS) [110]. A property 227

228


of these systems is that ants will be attracted to select solution components1 that are associated with a combination of large pheromone value and high quality. While this helps to exploit promising areas of the solution space, it may also lead to a lack of diversity within explored solutions. Ensuring that solutions within a colony are sufficiently different in order to adequately sample the search space of a COP is essential to the operation of ant colony techniques. If this is not achieved, ants will tend to produce the same small set of solutions, thus leading to stagnation behaviour. In essence, this becomes an inefficient utilisation of computational resources. ACO meta-heuristics have in some way addressed this problem by using implicit means based on pheromone. For example, ant colony system (ACS) has a local update rule that evaporates the pheromone of frequently incorporated solution components while M A X - M I N Ant System limits the allowed range of possible trail strengths between problem specific constants 7,in and T~~~ [374]. Additionally, M A X - M I N Ant System uses a technique known as pheromone trail smoothing. This goes further by redistributing pheromone when the trail strength limit has been exceeded [375]. Another ant colony meta-heuristic, Best-Worse Ant System [84;851, mutates the pheromone matrix with a low probability. Over time the strength of the mutation increases to encourage diversity. Additionally, it will periodically reset values of the pheromone matrix to the initial value if the makeup of the best and worse solutions becomes too close. In this paper, a new form of explicit diversification within colonies is proposed. It uses the tabu search notion of balancing the restriction and freedom of incorporating components into solutions to achieve this. For example, a solution component (for an ant) becomes tabu if it gives the same partial solution as another ant. The restriction can be lifted (through an aspiration function) if the partial solution is better than that obtained previously or if no other choices are available. The experimental evidence herein shows that more diversification is achieved within individual colonies and subsequently improved solution costs are obtained (over a control strategy). The remainder of the paper is organised as follows. Section 17.2 gives a technical description of one of the ACO meta-heuristics, ACS, while Section 17.3 describes some of the existing diversification approaches that have been used for ACO. Section 17.4 outlines the mechanics of a new colony diversification measure while Section 17.5 describes an implementation of the scheme that is suitable for the travelling salesman problem (TSP) and quadratic assignment problem (QAP). Additionally, a comparison is made against a control technique. Finally, Section 17.6 provides the conclusions 'A solution component is the building block of the solution. Two examples are a city for the travelling salesman problem and a facility for the quadratic assignment problem.

229

Maintaining Explicit Diversity

and suggests some extensions to this new technique.

17.2

Ant Colony System

ACS is one of the common varieties of ACO that has been shown to be a robust optimisation technique [112; 1881. The following is a description of the operation of ACS in terms of one of the test problems, the TSP. Consider a TSP with N cities. Each pair of cities r and s is separated by distance d(r, s). Place m ants randomly on these cities. In discrete time steps, all ants select their next city s and then simultaneously move to their next city. Ants deposit a substance known as pheromone to communicate the utility (goodness) of the edges to the rest of the colony. The quality of a solution component is given by the visibility heuristic q(r,s) (for this problem q(r,s) = d(r,s)). Denote the accumulated strength of pheromone on edge ( r ,s) by ~ ( rs)., At the commencement of each time step, Equations 17.1 and 17.2 are used to select the next city s for ant k currently at city r. Equation 17.1 is a greedy selection technique that will choose the city that has the best combination of short distance and large pheromone levels. Using the first branch of Equation 17.1 exclusively will lead to sub-optimal solutions due to its greediness. To compensate, there is a probability that Equation 17.2 will be used to select the next city instead. This equation generates a probability for each candidate city and then roulette wheel selection is used to select s. s={

argmax,€J,(r) Equation 17.2

{7(~,4[V(V4l0}

if 4 I40 otherwise

(17.1)

Note that 4 E [0,1] is a uniform random number, 40 is a parameter and R represents the roulette wheel selection function. To maintain the restriction of unique visitation, ant k is prohibited from selecting a city that it has already used. The cities that have not yet been visited by ant k are indexed by J k ( r ) . As TSP is a minimisation problem, p is negative so that shorter edges are favoured. The use of T(r,s) ensures preference is given to edges that are well traversed (i.e., have a high pheromone level). The pheromone level on the selected edge is updated according to the local updating rule in Equation 17.3. This has the effect of decreasing the pheromone level

230


slightly on the selected edge to ensure some implicit measure of diversification amongst the members of the colony is achieved.

Where: (rls) is an edge within the TSP graph,

< p < 1 and the initial amount of pheromone deposited on each of the edges.

p is the local pheromone decay parameter, 0 TO is

Upon conclusion of an iteration (i.e., once all ants have constructed a solution), global updating of the pheromone takes place. Edges that compose the best solution (so far) are rewarded with an increase in their pheromone level while the pheromone on the other edges is evaporated (decreased). This is expressed in Equation 17.4. T(T,

s)

+-

(1- y) * T ( T , s)

A T ( T , s=)

+ y . AT(T,s)

3 if (r1s) E s 0 otherwise.

(17.4)

(17.5)

Where:

AT(T, s ) is used to reinforce the pheromone on the edges of the iteration best solution (see Equation 17.5), L is the length of the best (shortest) tour to date while Q is a constant, y is the global pheromone decay parameter, 0 < y < 1 and S is the set of edges that comprise the best solution found to date. It is typical that a local search phase is performed on each ant’s solution, before global pheromone updating takes place.

17.3 Explicit Diversification Strategies for ACO There have been a few attempts to explicitly incorporate diversification strategies into ACO techniques. Gambardella, Taillard and Dorigo [137] propose a hybrid ant system in which the usual constructive component is replaced by local search instead. It has been implemented for the QAP and is subsequently known as HAS-QAP. It incorporates simple intensification and diversification processes into the algorithm. HAS-QAP begins each iteration with a complete solution (rather than constructing it). In


231

diversification phases, both the pheromone matrix and the initial solution are reinitialised. In the case of the latter, the solution is a random solution (i.e., a random permutation for the QAP). Blum [48] uses a similar approach to the above by having a number of restart phases and resetting the pheromone values to random levels. Hendtlass’ 11871 Ant Multi Tour System (AMTS) achieves a measure of diversification by allowing ants to retain a memory of TSP tours they constructed in previous generations. A weighted term in the solution component selection equations is used to discourage ants from choosing previously incorporated components. In a case study of a 14 city problem, it was shown that AMTS could outperform implementations of ACS and MAX - M Z N Ant System. Randall and Tonkes [325] outline a scheme based on the ACO metaheuristic ACS in which the characteristic component selection equations (Equations 17.1 and 17.2 herein) are modified so that the level of pheromone, in relation to the heuristic information, is varied. The premise is that solution components having higher pheromone levels have shown in the past to be attractive and vice-versa. During a diversification phase, components with large amounts of pheromone are actively discouraged. There was however no statistically significant difference in solution quality between the intensification/diversification schemes and a control ACS strategy. Meyer I2771 has extended this idea by introducing the concept of “a-annealing” based on simulated annealing. Like simulated annealing, diversity is highly encouraged at the beginning of the search process by controlling the relative weighting of the pheromone information. The paper reports encouraging initial results for small benchmark TSPs. Instead of modifying trail strength importance, Randall [323] examines the frequency of incorporation of solution components. During diversification, frequently incorporated components are discouraged while less frequently occurring components are encouraged. This strategy was inspired by a classic tabu search intensification/diversification strategy. Intensification, diversification and normal phases in the search are triggered dynamically. The results showed that improved performances (over a control strategy) on large TSPs could be obtained. Nakamichi and Arita [286] define a simple diversification strategy for the TSP. At each step of the ant algorithm, each ant has a probability of selecting a city at random, without regard to pheromone or heuristic (cost) information. This allows the search to diversify, however, the results were far from conclusive as only one relatively small problem instance (ei151, see TSPLIB [331]) was used.

232


Algorithm 17.1 This algorithm checks the tabu status of a component. Note that k is the current ant and p is the position of the solution component.

tabu = FALSE 2: for i = 1 to k - 1 do 3: if x i p = x k p then 4: if Cik 2 tabu-threshold then 5: tabu = TRUE 6: end if 7: end if 8: end for 9: end 1:

17.4

Maintaining Intra-Colony Diversity

Tabu search principles of the balance between the restriction and freedom of the incorporation of various solution components [146] can be extended to form the basis of an intra-colony diversification strategy. The overall strategy is referred to as divers herefrom. As previously mentioned, an undesirable property of ACO techniques is that a group of ants within a particular colony can take the same path through search space and subsequently produce the same solution. This is commonly referred to as stagnation behaviour [374]. The approach described below allows ants to achieve greater exploration within each iteration while still maintaining the ability to exploit good components via standard pheromone mechanisms. To illustrate this, consider the TSP. The exploration of solution space is impaired if two or more ants within a colony select the same sequence of cities within their tours. Therefore, it may be considered tabu for an ant to have a number of sequentially visited cities in common with another ant. If this occurs, one of the ants is forced to choose another city. This requires the creation of a separate memory, c, that stores the number of components (cities) that ants have in common with one another. Formally, c is a matrix in which cij contains the length of the common sequence that is shared between ants i and j (1 5 i , j 5 m). Note that this is a symmetric matrix such that cij = cji. The tabu status of a component is checked after it has been selected by either Equation 17.1 or 17.2. Algorithm 17.1 is used to determine the tabu status. Algorithm 17.1 inspires two important questions, a) how is tabu-threshold set? and b) how can the tabu status of a component be lifted (i.e., aspiration)? In terms of the former, there are two simple ways


233

in which this can achieved: (1) tabu-threshold is made a constant (regardless of problem size) or (2) tabu-threshold = round(an), where a is a proportion (0 5 a 5 l), n is the number of solution components (e.g., N , the number of cities in a TSP) and round() is a rounding function. Implementation specific details of the setting of tabu-threshold are provided in Section 17.5. The application of aspiration can be achieved in either of the following ways: (1) If all the non-tabu solution components are unavailable (as, for instance, they have already been incorporated into the solution), the elements of the tabu list are used instead. (2) If by adding a tabu solution component to a partial solution, the cost of this partial solution is better than the best solution’s partial cost (at that point within the constructive process), the component becomes non-t abu . The c matrix is updated after every step of the ant algorithm. If ant i has the same component in the same position as ant j, then cij = cji = cij 1. If it is different, then cij = cji = 0. The latter condition ensures that the value of each element of c is the number sequential solution components rather than just the number of common components. As the diversity scheme consists, in the main, of array lookups and updates, it should not significantly impact on the algorithm’s computational requirements.

+

17.5

Computational Experience

The computing platform used to perform the experiments is a 2.6GHz Red Hat Linux (Pentium 4) PC with 512MB of RAM. The experimental programs are coded in the C language and compiled with gcc. The experimental work is designed to test the operation of the new strategy as well as to contrast it against a control ACS meta-heuristic implementation. The following describes the design of the computational experiments; the implementation details for each problem; the benchmark problem instances and finally the results.

17.5.1

Experimental Design

The experiments are divided into two stages:

234

0

Recent Advances in Artzjicial Life

Stage 1: In this stage, the mechanics and parameters of the diversification strategy are tested. The key parameter is tabu-threshold. In order to encourage diversity, these values will be kept small relative to the problem size. Constant values of 5, 10 and 15 (for TSP) and 3, 5, 7 (for QAP) will be trialed. For the linear proportional model, a will be set as 0.05, 0.1 and 0.15. Additionally, the effectiveness of the second aspiration criterion will be tested by turning it on and off. The combination of these settings will be tested on a subset of the problems, namely hk48, st70, nug30 and tai35a. The number of iterations per run is set as 1000. The Kruskal-Wallis statistical procedure will be used to determine if there is a significant difference between the combinations and if so, which combination performs the best overall. It is used as there are multiple factors to test and the data are non-normally distributed. Stage 2: The performances of three solvers (for both TSP and QAP) will be compared. These are a) the divers strategy (using the best combination of parameters as identified from Stage 1),b) the divers strategy (without the local pheromone updating rule (Equation 17.3)) which is referred to as divers-nlp and c) the control strategy (which is a standard ACS implementation as outlined in Section 17.2 and is referred to as control). divers-nlp is trialed as the divers strategy performs the same function as the local updating rule. Each run will constist of 3000 iterations, so as to give the solvers sufficient opportunity to find good solutions.

The standard ACS parameters have been set as { p = -2,y = 0 . 1 , = ~ O . l , q o = 0.9,m = 10) for all experiments as these values have been found to be robust by Dorigo and Gambardella [112]. The value of p is set to 0 to implement diver-nlp. Each problem instance is run across ten random seeds.

17.5.2 Implementation Details Local search will not be used in the first stage as it alters individual solutions and will hence confound the results. However, it will be used for divers, divers-nlp and control in Stage 2. It is applied, for each ant, at each iteration, in the following manner. The transition operators used for the TSP and QAP are inversion and 2-opt respectively. These operators have been found by Randall [324] to provide good performance for these problems. For each operator, the entire neighbourhood is evaluated at each step of the local search phase. The phase is terminated when a better solution cannot be found, guaranteeing a local minimum. Thus it is a variable

235


depth local search scheme. For the visibility heuristic, the TSP uses the distance measure between the current city and the potential next city. For the QAP, there are a nuinber of choices for this heuristic, such as the use of a range of approximation functions and not using them at all [373]. The definition used here is given by Equation 17.6.

Where:

w is the current location, ~ ( wis)the facility assigned to w, j is the the potential facility to assign to z(w), u ( i ,w) is the distance between locations i and w and b ( i , j ) is the flow between facilities i and j .

17.5.3

Problem Instances

Twelve TSP and QAP problem instances are used to test the effectiveness of the three solvers. These problems are from TSPLIB [331]and QAPLIB [61] respectively and are given in Table 17.1. Table 17.1 Problem instances used in this study (TSPs on the left and QAPs on the right). “Size” for the TSP and QAP is recorded in terms of the number of cities and facilities/locations respectively. Name

Size

hk48 ei151 st70 ei176 kroAlOO bier127 d198 ts225 gi1262 pr299 lin318 ocb442

48 51 70 76 100 127 198 225 262 299 318 442

Best-Known Cost 11461 426 675 538 21282 118282 15780 126643 2378 48191 42029 50778

I

Name

Size

n u-a l 2 nugl5 nug20 tai25a nug30 tai35a ste36a tho40 sko49 tai50a sko56 sko64

12 15 20 25 30 35 36 40 49 50 56 64

Best-Known Cost 578 1150 2570 1167256 6124 2422002 9526 240516 23386 494 1410 3445s 48498


236

17.5.4

Results

Stage 1 tested for the best combinations of settings for divers. Using the Kruskal-Wallis procedure and a significance level of 0.05 revealed that having a! = 0.15 with the second aspiration criteria turned off produced the overall best overall results. As the results were not statistically significant, the combination was chosen on the Kruskal-Wallis rankings. Further testing showed that using the second aspiration criteria was significantly worse than not using it. Using the best settings from Stage 1 (no second aspiration criteria and a = 0.15), Tables 17.2 and 17.3 give the results of the new solvers (divers and divers-nlp) and the control strategies for TSP and QAP respectively. In order to describe the range of objective costs obtained by these experiments, the minimum (denoted “Min”), median (denoted “Med”) and maximum (denoted “Max”) are given. Non-parametric descriptive statistics are used as the data are highly non-normally distributed. The cost results are reported as the relative percentage deviation (RPD) from the best known x 100 where E is the result cost solution cost. This is calculated as and F is the best known cost. The runtime is recorded as the number of CPU seconds required to obtain the best solution within a particular run. In terms of the runtime, there is no noteworthy difference between the control and diversity strategies (see Section 17.4). Considering each problem type separately for the objective cost results revealed that: 0

In terms of the TSP, the diversity strategies are significantly better than control. In terms of the QAP, there is no significant difference between the three solvers.

A possible explanation of the latter can be formed by examining the trials/recorded time at which best solutions were found. Unlike the TSP, all the solvers regularly found the best solution later in the run. If both problem types are considered together, there is a statistically significant difference between the three solvers. Post-hoc testing (using Scheffes’s test) showed that both divers and divers-nlp are significantly better than control. No overall difference is recorded between divers and divers-nlp. This indicates that for both TSP and QAP the use of local pheromone (an implicit form of intra-colony diversification) is unnecessary provided that the explicit scheme is used. In light of the success of the new strategy for TSPs, it was decided to test it on two larger problem instances. Solving the problems d657 (657 cities) and rat783 (783 cities) under the same conditions showed that the


237

Table 17.2 Results for all the solvers on the TSP. Problem

Solver

Cost R P D

Min dives

divers-nlp

control

hk48 ei151 st70 ei176 kroAlOO bier127 d198 ts225 gi1262 pr299 lin318 pcb442 hk48 ei151 st70 ei176 kroAlOO bier127 d198 ts225 gi1262 pr299 lin318 pcb442 hk48 ei151 st70 ei176 kroAlOO bier127 d198 ts225 gi1262 pr299 lin318 pcb442

Med

Max

0

0

0

0

0.23 0 0 0 0 0.03

0.23 0 0.74 0 0.32 0.13 0 0.34 0.38

0 0 0 0 0 0

0

0 0 0.13 0.32 0 0

0.06 0.08 0.32 0.69 0 0.23

0

0 0 0 0

0 0 0 0

0.15 0.36

0.03 0 0.23 0.07 0.3 0.69

0

0

0

0.23 0.37 0.09 0.2 0.29 0.17 0 0.69 0.64 1.03 1.68

0 0 0

0

0 0 0

0.04 0 0.29 0.32 0.73 0.95

1.33

1.14 0 0.23 0 0 0

0.15 0.09 0 0.84 0.18 0.43 0.77 0.11 0.47 1.33

0.74 0.46 0.75 0.8 0.13 1.39 1.47 2.09 2.51

Min 0.02 0.02 0.33 0.13 0.3 2.5 151.68 12.46 42.7 196.88 328.09 2746.7 0.04 0.04 0.35 0.3 0.45 2.65 235.39 31.56 111.07 391.86 660.86 4246.9 0.02 0.03 0.28 0.29 0.25 2.44 14.56 6.24 11.12 21.37 43.08 81.63

Time (seconds) Med Max 0.12 0.16 39.92 0.15 26.49 1.49 2.21 8.17 4.7 0.99 887.83 8.69 3741.14 1157.75 62.68 23.51 6735.24 476.19 10651.66 1304.07 12286.94 2997.45 11241.24 5570.13 1.47 0.19 0.71 0.17 89.37 2.1 130.47 6.29 6.41 3.4 1020.8 48.96 4254.66 622.34 264.4 82.97 5008.39 628.51 9700.76 2523.08 16584.53 3892.62 42211.08 24747.65 27.07 0.13 0.21 0.38 0.6 2.22 1.16 3.35 1.27 5.97 665.57 7.97 62.86 1546.57 17.87 71.15 32.3 2769.19 45.35 9202.87 1141.83 85.98 161.71 6268.72

new diversification strategy again produced very good solutions. Typically these were within 0.5% of the best known cost.

17.6

Conclusions

This paper has described an explicit intra-colony diversification mechanism for ACO. The strategy ensures that ants within a colony produce different partial solution sequences. The results show that the strategies based on this diversification notion outperform a control ACO implementation. This is particularly evident for the TSP in which both divers and divers-nlp consistently find solutions within a percent of optimal - even for the largest


238

Table 17.3 Results for all the solvers on the QAP. Problem

Solver Min

divers

divers-nlp

control

nugl2 nugl5 nug20 tai25a nug30 tai35a ste36a tho40 sko49 tai50a sko56 sko64 nugl2 nugl5 nug2O tai25a nug30 tai35a ste36a tho40 sko49 tai50a sko56 sko64 nugl2 nugl5 nug2O tai25a nug30 tai35a ste36a tho40 sko49 tai50a sko56 sk064

0 0

0 0.37 0 1.12 0

0.01 0.05 2 0.02 0.02 0 0 0 0.37 0 0.94 0

0.01 0.03 2.08 0.02 0 0 0

0 0.71 0 1.11 0 0.05 0

2.1 0.02 0

Cost RPD Med Max 0 0 0 0.76 0 1.54 0.25 0.43 0.14 2.34 0.13 0123 0 0 0 0.87 0 1.49 0.25 0.21 0.07 2.36 0.16 0.07 0 0 0 1.14 0 1.69 0.24 0.37 0.1 2.24 0.15 0.17

0 0 0

1.43 0.07 1.87 1.47 0.61 0.27 2.55 0.34 0.31 0 0 0

1.22 0.07 1.74 1.47 0.34 0.26 2.55 0.3 0.35 0

0 0

1.24 0.07 2 1.18 0.66 0.25 2.6 0.32 0.34

Time (seconds) Med Max 0 0.02 0.03 0.01 0.02 0.23 0.12 0.94 10.69 49.77 158.79 221.86 7 21.26 761.47 3.24 245.76 1038.54 414.9 1013.62 1815.62 279.6 2359.55 4199.35 492.11 3472.31 7408.42 500.35 1836.43 5931.69 2423.35 14095.43 22218.96 20083.71 30350.97 47165.03 0 0.02 0.17 0.01 0.02 0.17 0.11 0.79 20.47 16.31 121.32 214.7 234.89 264.86 286.81 3.28 716.81 1187.37 232.26 502.46 2477.66 733.01 2494.46 3770.59 1039.54 2929.4 9371.47 773.71 4680.92 6705.23 1315.99 6840.7 2 1409.46 7148.14 33504.2 41416.52 0 0.02 0.33 0.01 0.03 0.57 0.15 0.74 5.38 11.99 89.71 230.97 17.44 136.18 835.52 62.63 508.87 984.4 729.08 2202.54 1452.94 10.41 1446.31 3795.78 2237.63 10967.64 9188.76 184.94 5049.98 3292.33 1731.5 9328.02 16282.59 1781.22 17777.08 46135.64 Min

problems. All strategies performed equally well on the QAP. The reason for this may be attributed to the fact the ACO generally found its best solution later in the run. Running the new solver across more problem types will help to verify this. This work naturally leads to a range of possible extensions. The most important of which is allowing the diversification scheme to continue across iterations. An implementation of this would ensure that all the solutions generated within a run would better sample the solution space. Additionally, the mechanics of the tabu-threshold need to be refined and trialed across a range of combinatorial problems. Dynamic varying of this parameter is also an option that should be considered.

Chapter 18

Evolving Gene Regulatory Networks for Cellular Morphogenesis T. Rudge and N. Geard

The ARC Centre for Complex Systems, School of Information Technology and Electrical Engineering, The University of Queensland, QLD 4072, Australia. E-mail: {timrudge,nic}@itee.uq. edu.au The generation of pattern and form in a developing organism results from a combination of interacting processes, guided by a programme encoded in its genome. The unfolding of this programme involves a complex interplay of gene regulation and intercellular signalling, as well as the mechanical processes of cell growth, division and movement. In this study we present an integrated modeling framework for simulating multicellular morphogenesis that includes plausible models of both genetic and cellular processes, using leaf morphogenesis as an example. We present results of an experiment designed to investigate the contribution that genetic control of cell growth and division makes to the performance of a developing system.

18.1

Introduction

The generation of pattern and form in a developing organism results from a combination of interacting processes, guided by a programme encoded in its genome. The unfolding of this programme involves a complex interplay of gene regulation and intercellular signalling, as well as the morphogenetic processes of cell growth, division and movement [428]. Recently, computers have enabled these multi-scale developmental systems to be simulated, revealing new insights into the emergence of pattern and form [192;3431. 239


240

In this study we present an integrated modeling framework for simulating multicellular morphogenesis that includes plausible models of both genetic and cellular processes, using leaf morphogenesis as an example. Leaf forms display a wide variety of morphological features, the development of which provide excellent examples of robust control of shape formation. We focus here on the role played by genetic control of cell growth and division orientation in the generation of specific shapes, and how this interacts with the physical constraints on cell shape. The format of this report is as follows: Section 18.2 presents background material on leaf morphogenesis and previous computational models of gene regulation and development. Section 18.3 describes our simulation framework, which consists of a gene network model, a physical cell model, a procedure for coupling these two components, and an evolutionary search algorithm for investigating model parameters. Section 18.4 presents initial results obtained using this simulation framework and Section 18.5 concludes with a discussion of future directions for this research.

18.2 18.2.1

Background Leaf Morphogenesis

Morphogenesis, the formation of shapes and structures in plants and animals, occurs by three processes: 1. Tissue growth; 2. Cell movement; 3. Cell death (apoptosis) [83]. Active cell movement does not occur in plants and so morphogenesis is coordinated by tissue growth - determined by cell shape, growth, and proliferation - and cell death. Variation in these behaviours across tissues and over developmental time causes the development of specific forms. The term patterning is applied to the coordinated differential expression of genes over space and time. It is these gene expression patterns that give rise to the variation in cell behaviour that drives morphogenesis. Patterning provides positional information that guides cell behaviour and although cell lineage also plays some role, it seems this positional information is of primary importance in plant development [log;1271. The relationship between patterns of gene expression and the specification of tissue and organ shape is not well characterised. [83] cite the difficulty in measuring morphogenetic effects and the need for quantitative analysis as possible reasons for this gap. Another difficulty is understanding the tight coupling of morphogenesis and patterning: the patterns develop along with, and are embedded in the forms to which they give rise. Formation of leaf shape is tightly regulated, and evidence exists for

Evolving Gene Regulatory Networks for Cellular Morphogenesis

241

both cell-division dependent and cell-division independent regulatory mechanisms [127]. That is, final leaf form and size is to some extent independent of both cell number and cell size. There is also evidence that overall leaf shape is unaffected by cell division orientation [127]. This suggests regulation of cell behaviour depends on feedback of organ-level information [399]. The nature of this regulation is at present unknown. However, hormone (e.g. auxin) transport has been implicated in organ shape regulation [127; 2221, and [221] suggest that hormones may also regulate organ shape by affecting cell expansion and/or by modulating the cell cycle. Noting that leaf size is dependent on whole-plant physiology, [399] suggests that a sourcesink relationship within the plant (e.g. of nutrients) might limit leaf size. [433] and [221] present results of experiments in which cell division rates, cell division orientation, and cell growth rates were perturbed both locally and across the leaf. These results provide evidence for the involvement of the above mentioned processes in regulating leaf morphogenesis, but how these multiple mechanisms interact and their relative importance are still unknown. 18.2.2

Previous Models

[313] identifies three categories of plant development models, focusing on plant architecture, individual organs, and the underlying mechanics of gene regulation, respectively. The first of these is well established, with Lsystems being the dominant modelling framework; the latter two areas are still active areas of research - [313] and [83] provide an overview of recent developments. These categories occupy very different temporal and spatial scale ranges, and a full understanding of development requires the integration of multiple scales. Developmental issues have been addressed by the Artificial Life research community [367]. An early attempt to integrate multiple scales of developmental mechanism into a single model included cells with complex internal dynamics that communicated with each other via chemical and electrical signals as well as physical interactions 11261. One of the findings of this study was that, while the multiple mechanisms enabled the robust production of interesting phenotypes, it also made the design of specific phenotypes more difficult. Later research demonstrated that this difficulty could be addressed by using a representation of the regulatory network that could be artificially evolved [119]. [192] also used a model that combined mechanisms at multiple scales of description - gene regulation, development and evolution - to investigate the interactions between evolutionary dynamics and morphology. Her evolutionary process was aimed a t maximising cell type diversity, rather

242

Recent Advances an Artificial Lzfe

than achieving a specific morphological shape. However, she found that some features of morphology, such as engulfing, budding and elongation, were relatively ‘generic’, that is, they appeared ‘for free’ in systems that satisfied certain prerequisites. The formation of patterns across a fixed field of autonomous cells has also been studied in some detail [344;359]. Here too, it was found that patterns were a common emergent feature of interacting gene networks, although again, selection was for ‘pattern complexity’ rather than a specific phenotypic target. Later research [343] focused on the issue of how pattern formation processes interact with growth processes, specifically with reference to the evolution of tooth development. In general, the models mentioned above do not include individual cell morphology: Cells are represented as circles or squares, of equal or varying size. The model used by [192] does support anisotropic cell shapes; however, these result from the algorithm used to calculate cell boundaries, rather than reflecting anisotropy in the underlying growth process. In plants, cell behaviour is frequently anisotropic, with the axes of both growth and division under a degree of cellular and genetic control. As described above, control of cell morphology is intimately connected with the production of leaf form, therefore a detailed and flexible model of cell shape is of fundamental importance in any approach to modeling leaf morphogenesis.

18.3

The Simulation Framework

Our integrated model of plant morphogenesis brings together plausible representations of cell shape, genetic regulation, and cell-cell signalling. Cell shape is determined by growth and division activity as well as external physical forces, and the combination of the shapes of all cells determines the overall phenotypic form. Each cell is autonomous and its behaviour is regulated by its own copy of the organism’s gene network, which also responds to signals received from neighbouring cells. The genetic network thus indirectly specifies phenotypic morphology. As noted earlier, due to the complexity of cellular developmental systems, a search and optimisation approach is favoured when examining their properties in silico. We have chosen to use an evolutionary algorithm to search for systems with particular shape formation capabilities. Our approach is to decide on a target phenotypic shape, specify initial conditions, and then artificially evolve gene networks which come close to producing the desired shape. In the following we describe the primary components of the model: the genetic component, consisting of a network embedded within each cell, and


243

the spatial model, consisting of an arrangement of cells that constitutes the phenotype. Following that, the coupling between these two components is described. Finally, the evolutionary algorithm used to explore the parameter space of these networks is outlined.

18.3.1

The Genetic Component

In this study, we used a dynamic recurrent gene network (DRGN) model for the genetic component of the framework [141]. The DRGN model is based on a widely studied class of artificial neural network models known as recurrent neural networks [120],and has previously been used to investigate the generation of developmental cell lineages [141]. An advantage of a recurrent network representation is that it enables the model to express a complex range of gene interactions while abstracting away from the specific biological processes that underly those interactions.

INPUT GENES

W

REGULATORY GENES

OUTPUT GENES

Fig. 18.1 The structure of the DRGN model. The network is partitioned into three categories: input genes that detect the presence of morphogens produced by other cells or the environment; regulatory genes that interact with one another to perform the computational tasks of the cell; and output genes producing morphogens that can be transmitted to other cells or that trigger events such as growth and division.

In the DRGN model, a genetic system is defined as a network of N interacting nodes (see Figure 18.1). Depending on the level of abstraction, each node can be considered to represent either a single gene, or a cluster of co-regulated genes. In this study we generally consider a node to be a equivalent to a single gene. The activation state of each node is a continuous variable in the range [0,1],where 0 represented a completely inactive gene and 1 a fully expressed gene. Nodes can be divided into three classes: input genes that detect the presence of morphogens; regulatory genes that interact with each other to carry out the computational task of the network; and output genes that produce morphogen signals.

Recent Advances in Artijcial Lafe

244

The network is updated synchronously in discrete time steps. To capture the potential complexity of the interacting factors involved in gene expression, we have used a network in which each input gene is connected to each regulatory gene, all regulatory genes are connected to each other and themselves, and each regulatory gene is connected to each output gene. Thus an individual link in the network does not necessarily represent a direct physical interaction, but rather the degree of influence that the expression of the source gene at time t has on the expression of the target gene at time t+l. These interactions can be summarised in a weight matrix, in which the entry at row i, column j specifies the influence that gene j has on gene i . These entries may be positive or negative, depending on whether the product of gene j is an activator or a repressor in the regulatory context of gene i. A zero entry indicates that there is no interaction between the two genes. The inclusion of self-connections (i.e. from node i to node i) allows for the possibility of genes influencing their own regulation. The state of the network is updated synchronously, with the activation of node i at time t 1, ai(t l),given by

+

+

+ 1) = (.

N,

C w i j a j ( t )- ei)

(18.1)

j=1

where N, is the number of regulatory nodes, wij is the level of the interaction from node j to node i, 8i is the activation threshold of node a , and a(.) is the sigmoid function, given by

).(. 18.3.2

=

1

~

1

+ e-"

(18.2)

The Cellular Component

We use a 2-dimensional spatial model of the cellular arrangement. This is based on linear cell boundary elements (walls), which are modelled as elastic springs. The approach is similar to that of [198], however we also consider some more complex cell dynamics such as anisotropic growth. Cellcell signalling is considered in the form of chemical diffusion, as in [126]. This approach has previously been used to examine rule-based control of plant morphogenesis [340]. Cell: The genome of our artificial organism is represented as a DRGN. Each cell is defined by its DRGN, a set of dynamic state parameters and a closed boundary. The DRGNs contained by each cell in a phenotype have identical structure and weights, reflecting the genetic homogeneity of an individual organism. The activation levels of the DRGN nodes in each cell,


245

however, are independent and represent the variation in gene expression across the phenotype. The cell state parameters include passively received information such as morphogen levels and cell volume, and behavioural states like growth rate and morphogen production rates. As part of their state, the cells also maintain polarity vectors that are used to direct anisotropic growth, to orient the division plane, and to asymmetrically divide the cells morphogens between its daughters, according to the behavioural state parameters. The state of the cell determines its behaviour at any point in time. Cell dynamics are expressed as the transformation of cell state parameters to proceed to a new state. Behavioural states are transformed by the DRGN, with the inputs and outputs of the DRGN defined by a fked mapping onto the cell state parameters. The passive state parameters are transformed by physical simulation of the cells’ environment, including its own boundary shape and interactions with neighbours. Spatio-Mechanical Model: The boundary of the cell describes its shape, and is decomposed into a set of walls. Each wall is the interface between two cells. Morphogens diffuse from one cell to the other via the wall, providing a cell-cell signalling mechanism. The walls are considered to be two linearly elastic elements (springs), one for each adjacent cell, bound together a t the end points (vertices). Each of the adjacent cells influences the properties of only one of these springs. Each spring has stiffness K and natural length L, determined from the state parameters of the appropriate cell [340]. Each cell exerts a turgor force perpendicular to each of its walls in an outward direction with respect to the cell, extending the springs, which then exert an opposing tension force. At each time-step these simulated forces are accumulated at the vertices, and the vertex positions are adjusted to find the equilibrium configuration. Cell growth is achieved by increasing the natural lengths of each cells’ springs t o varying degrees (see [340]for details). Division consists of inserting a dividing wall across the centre of the cell, and redefining the daughter cell boundaries. When a cell divides, its DRGN (including current node activation levels) is copied into the two daughter cells. 18.3.3

Genotype-Phenotype Coupling

The system integrates multiple scales of model into a single framework. Figure 18.2 shows an overview of the way in which the levels of the model interact. Starting at the micro level, the DRGN transforms the cell state. The cell state is expressed as local behaviours such as growth, which then affect the entire phenotype via simulated mechanical forces and diffusion

246


processes. This global effect is then transduced back into local information to each cell, and from there transformed into micro level input to the DRGN.

Fig. 18.2 Scheme of interactions between different levels in the model, from microscopic (left) to macroscopic (right). Circular arrows indicate faster time scale processes running multiple time steps between cell state updates.

The flow of control is therefore from micro to macro level, and the flow of information or feedback is from macro to micro. The nature of the coupling between information feedback and phenotypic output is ultimately determined by the structure and weights of the DRGN, as well as the mapping from DRGN to cell state - i.e. the genotype. The dynamics of cell behaviour, such as growth and division, gene expression, and transmission of mechanical forces, occur on very different time scales. In general, variation in cell behaviour occurs most slowly and equilibration of forces occurs most quickly. We assume that mechanical equilibrium is reached instantaneously when relevant parameters such as growth rate change. The DRGN can be used to model genetic regulation on several levels. Each node may represent a single gene or a cluster of genes, and each node update may represent one or many regulatory events. In order to incorporate this flexibility we allow the DRGN to update multiple times before affecting the cell state. The procedure that produces a cellular phenotype from the DRGN genotype is thus: (1) Determine cell states from initial conditions (2) Map DRGN inputs from cell states (3) Update DRGN by some number of time steps (4) Map cell states from DRGN outputs (5) Compute cell shapes and morphogen difision (6) Repeat from 2 until stopping condition met


247

The stopping condition may be chosen arbitrarily according to the experiment. We used a maximum number of time steps of 250 in our experiments. 18.3.4

The Evolutionary Component

The evolutionary component, which enables a population of DRGNs to be artificially “evolved” towards some particular target, serves two purposes. At a methodological level, it provides a useful machine learning technique for searching the parameter space of networks. At a theoretical level, it facilitates questions about the evolutionary dynamics of morphogenesis [192]. A simple evolutionary search strategy called the 1+1ES was used [23]. Initially, a single DRGN was generated with weights randomly drawn from a Gaussian distribution with mean 0 and standard deviation 4. This DRGN was used to develop a phenotype, as described in Section 18.3.3. A fitness value for this phenotype was calculated as described in Section 18.4.1 below and stored. A new DRGN was derived from the existing DRGN by adding Gaussian noise (mean 0, standard deviation 0.01) to each of the node interactions. A new phenotype was developed and evaluated and the fitness value for the modified DRGN was compared t o that of the original DRGN. The DRGN producing the phenotype with the greatest fitness was retained and used as the basis for the creation of a further new DRGN. This process was repeated until the stopping conditions were met. We used a maximum number of generations of 15,000 in our experiments.

18.4 Initial Experiments

To investigate the role of genetic control of growth and development in morphogenesis, we ran three sets of comparative evolutionary trials: random growth and division orientation, regular growth and division orientation and genetically controlled growth and division orientation. We set the DRGN the task of generating a circular shaped final phenotypic form. (1) Random orientation In the first set of trials, there was no control of growth and division orientation - they were each chosen randomly at each time step. Only one output node was utilised, the morphogen controlling the decision to grow and divide. (2) Regular orientation In the second set of trials, the orientation of growth and division of each cell was chosen to be opposite to that of its parent cell - giving alternating axial and lateral growth and division each generation. The DRGN was not able to change this sequence of


248

orientations; however, it was able to make use of the regularity of the predetermined sequence. (3) Controlled orientation In the final set of trials, two additional output nodes were used, producing morphogens that controlled the orientation of growth and division respectively. Therefore the DRGNs had the capability to coordinate the two processes. 18.4.1

Method

DRGN Coupling: Three inputs were provided to the network. The first input responded to the concentration of a morphogen that was initialised to a concentration of 1.0 in the initial cell, and was not produced after that. Therefore, as the volume of the phenotype increased due to growth and division, the concentration of this morphogen decreased. The second and third inputs responded to morphogens related to the position of the cell. These were externally supplied as the (z, y) position of the cell centre. This may be considered as incorporating information supplied by underlying cell layers. The phenotypes were initialised as a single unit square cell with unit morphogen concentration, and DRGN outputs p j mapped to cell behaviour as follows:

0

If po > 0.5 then set growth rate to 0.2, and divide if volume > 2. If po 5 0.5 then set growth rate to 0 and do not divide. If p l > 0.5 set division orientation to axial otherwise set to lateral. If p2 > 0.5 set growth orientation to axial otherwise set to lateral.

Fitness function: The task used for this study was to evolve a DRGN capable of generating a circular arrangement of cells of a given radius. Fitness was calculated for each phenotype at each time step based on the current cell arrangement - specifically, the absolute distance of each exterior (marginal) cell from the centre of mass of the phenotype, given by: Ti

= [Xi- CI

(18.3)

for cell i, where xi is the cell’s centre of mass, and c is the centre of mass of the whole phenotype. The error of the phenotype from a circle radius R at any time point, is calculated from the distance of each cell from the circle dri = (ri- RI:

(18.4)


249

where the sum is over the N exterior cells. The first term is the mean distance error, and the second is the standard deviation in the distance error. We used a target radius of 5 units in our experiments. The overall phenotypic fitness was calculated from the cumulative error over all time steps { 1 , 2 , 3...T } and scaled by a constant factor such that the maximum possible fitness is approximately 1.O:

(18.5) Summary: In summary, three sets of evolutionary trials were run, each corresponding to one of the control conditions described in Section 18.4. Each evolutionary trial was run for up to 15,000 generations, with snapshots of the best phenotype being recorded at 500 generation intervals. In each generation, the DRGN was run for 250 developmental time steps, with it’s fitness evaluated over this period as described above. 18.4.2

Results

The DRGNs that had explicit control of the growth and division orientation were able to generate considerably more accurate phenotypes than the those supplied with either a regular or random sequence of orientations (Table 18.1). Table 18.1

Regular Controlled

0.555 0.642

The evolutionary history of the most successful evolutionary trial from the Controlled set displays a level of continual innovation typical of highly evolvable systems (Figure 18.3). By contrast, the most successful trials from the Regular and Random sets (not shown), reached their peak fitness early (around generation 4,000), and failed to improve any further. The phenotypes produced at different stages of evolution provide some clues to explain these differences (Figure 18.4).

(1) Random orientation (Figures 18.4.a- 18.4.d): Regulation of size appears relatively early, and is consistent throughout the course of evolution. However control of phenotypic shape has not evolved. The randomness of the sequence of orientations prevents the DRGN from

250


0.640 -

0.6350.630In

s

0.6250.620 -

0.6150.6100.605

0

2

4

6

8

10

12

14

6

Fig. 18.3 The evolutionary history of the fittest system found in the Controlled set of trials, in which the DRGN had explicit control of the orientation of growth and division.

being able to successfully coordinate the development of a stable shape. The examples shown here represent only one instance of a set of possible outcomes for a given DRGN. (2) Regular orientation (Figures 18.4.e- 18.4.h): When the DRGN is able to rely on a regular sequence of growth and division orientations, greater control of phenotypic shape is achieved. Very early in this evolutionary trial, a strategy emerged in which a group of growing cells is surrounded by non-growing cells. However, while ensuring a reasonably circular phenotype, this approach proves too strong a constraint, limiting any further improvement. (3) Controlled orientation (Figures 18.4i.- 18.4.1): With full control over division and growth orientations, the evolutionary algorithm was able to explore a much broader range of developmental possibilities. In the example shown, the DRGNs found early in the evolutionary history developed by first growing and dividing laterally and then switching to axial division, resulting in the phenotype fanning out. The largest jump in fitness (Figure 18.3, around generation 11,000) occurred when a DRGN was discovered in which the fanning out process was inhibited by a cap of quiescent cells. The resulting “stem and bud” arrangement was refined in successive stages of evolution by increasing the roundness of the bud, and reducing the length of the stem.

251


G = 0; f

= 6.25

x 10-5

4 G = 14000; f

b)

-

c)

= 0.308

-

G = 2000; f = 0.502

G = 4000; f = 0.534

e) G = 9 0 0 0 ; f =0.555

f)

= 0.535

G = 11000; f

G = 15000; f = 0.641

G = 2 0 0 0 ; f =0.610

i)

g)

G = 10000; f

G = 3000; f

= 0.314

4 G = 6000; f

G = 2000; f = 0.302

= 0.616

k)

= 0.617

-

-

Fig. 18.4 Fittest phenotypes at key stages (generation G ) in artificial evolution, where f is fitness: [a,d] Random growth/division, [e,h] Regular growth/division, [i,l] DRGN control. Shading shows DRGN output on a grey scale, white(0) to dark grey(1): [a,h] cell growth and division trigger, [ill]division orientation. Scale bar is 10 units.

18.5 Discussion and Future Directions In all three sets of trials, DRGN evolved that were capable of controlling phenotype size. With full DRGN control over development a significant degree of shape control evolved, using a variety of developmental approaches. With regular cell growth and division, DRGNs were able to control the

252


phenotype shape to a similar extent, but the range of strategies for doing this was more limited. With random growth and division, little control of phenotypic shape emerged. While preliminary, these results suggest that positional information only provides sufficient information to enable generation of stable phenotypic forms in the presence of predictable growth and division orientation. It would appear that the claim by [127], that leaf shape is, to a degree, independent of division orientation, requires the presence of more complex cell-cell signalling than simple positional cues. One strong possibility is that cell-cell communication plays a vital role in the robust development of form. Future work will involve the investigation of inductive interactions between cells and how this additional level of communication may facilitate more robust development.

Acknowledgments This study was funded by the ARC Centre for Complex Systems (http://www. accs edu. au). We thank Jim Hanan and Janet Wiles for stimulating discussions, helpful suggestions and guidance in carrying out this research.

.

Chapter 19

Complexity of Networks

R. K. Standish Mathematics, University of New South Wales E-mail:[email protected] http://parallel.hpc.unsw.edu.au/rks Network or graph structures are ubiquitous in the study of complex systems. Often, we are interested in complexity trends of these system as it evolves under some dynamic. An example might be looking at the complexity of a food web as species enter an ecosystem via migration or speciation, and leave via extinction. In this paper, a complexity measure of networks is proposed based on the complexity is information content paradigm. To apply this paradigm to any object, one must fix two things: a representation language, in which strings of symbols from some alphabet describe, or stand for the objects being considered; and a means of determining when two such descriptions refer to the same object. With these two things set, the information content of an object can be computed in principle from the number of equivalent descriptions describing a particular object. I propose a simple representation language for undirected graphs that can be encoded as a bitstring, and equivalence is a topological equivalence. I also present an algorithm for computing the complexity of an arbitrary undirected network.

19.1

Introduction

In [363], I argue that information content provides an overarching complexity measure that connects the many and various complexity measures proposed (see [117] for a review). The idea is fairly simple. In most cases, 253

254

Recent Advances i n Artijicial Life

there is an obvious prefix-free representation language within which descriptions of the objects of interest can be encoded. There is also a classifier of descriptions that can determine if two descriptions correspond to the same object. This classifier is commonly called the observer, denoted O (x). To compute the complexity of some object x , count the number of equivalent descriptions w(C, x ) = of length L that map to the object z under the agreed classifier. Then the complexity of x is given in the limit as C -+ 00:

C ( x ) = lim ClogN - logw(C,s)

(19.1)

e-00

where N is the size of the alphabet used for the representation language. Because the representation language is prefix-free, every description y in that language has a unique prefix of length s ( y ) . The classifier does not care what symbols appear after this unique prefix. Hence w(C, O ( y ) ) 2 Ne-3(Y), 0 I C ( O ( y ) ) I s ( y ) and so equation (19.1) converges. The relationship of this algorithmic complexity measure to more familiar measures such as Kolmogorov (KCS) complexity, is given by the coding theorem[242, Thm 4.3.31. Equation (19.1) corresponds to the logarithm of the universal a priori probability. The difference between these measures is bounded by a constant independent of the complexity of x . Many measures of network properties have been proposed, starting with node count and connectivity (no. of links), and passing in no particular order through cyclomatic number (no. of independent loops), spanning height (or width), no. of spanning trees, distribution of links per node and so on. Graphs tend to be classified using these measures - small world graphs tend to have small spanning height relative to the number of nodes and scale free networks exhibit a power law distribution of node link count. Some of these measures are related to graph complexity, for example node count and connectivity can be argued to be lower and upper bounds of the network complexity respectively. However, none of the proposed measures gives a theoretically satisfactory complexity measure, which in any case is context dependent (ie dependent on the observer 0, and the representation language). In this paper we shall consider only undirected graphs, however the extension of this work to directed graphs should not pose too great a problem. In setting the classifier function, we assume that only the graph’s topology counts - positions, and labels of nodes and links are not considered important. Clearly, this is not appropriate for all applications, for instance in food web theory, the interaction strengths (and signs) labeling each link is crucially important. The issue of representation language, however is far more problematic. In some cases, eg with genetic regulatory networks, there may be a clear


255

representation language, but for many cases there is no uniquely identifiable language. However, the invariance theorem[242, Thm 2.1.11 states that the difference in complexity determined by two different Turing complete representation languages (each of which is determined by a universal Turing machine) is at most a constant, independent of the objects being measured. Thus, in some sense it does not matter what representation one picks one is free t o pick a representation that is convenient, however one must take care with non Turing complete representations. In the next section, I will present a concrete graph description language that can be represented as binary strings, and is amenable to analysis. The quantity w in eq (19.1) can be simply computed from the size of the automorphism group, for which computationally feasible algorithms exist[271]. The notion of complexity presented in this paper naturally marries with thermodynamic entropy S[235]:

where Smax is called potential entropy, ie the largest possible value that entropy can assume under the specified conditions. The interest here is that a dynamical process updating network links can be viewed as a dissipative system, with links being made and broken corresponding to a thermodynamic flux. It would be interesting to see if such processes behave according the maximum entropy production principle[l06] or the minimum entropy production principle[312]. In artificial life, the issue of complexity trend in evolution is extremely important [34]. I have explored the complexity of individual Tierran organisms[364; 3651, which, if anything, shows a trend to simpler organisms. However, it is entirely plausible that complexity growth takes place in the network of ecological interactions between individuals. For example, in the evolution of the eukaryotic cell, mitochondria are simpler entities than the free-living bacteria they were supposedly descended. A computationally feasible measure of network complexity is an important prerequisite for further studies of evolutionary complexity trends.

19.2

Representation Language

One very simple implementation language for undirected graphs is to label the nodes 1..N, and the links by the pair ( i , j ) ,i < j of nodes that the links connect. The linklist can be represented simply by a N(N - 1)/2 length bitstring, where the a j ( j - 1) ith position is 1 if link ( i , j ) is present, and 0 otherwise. We also need to prepend the string with the value of N in order to make it prefix-free - the simplest approach is t o interpret the

+

256

Recent Advances in Artificial Lzfe

Network

/. A Y

Bitstring description 1110100,1110010,1110001 1110110,1110101,1110011 11110110100,11110101010,11110011001,11110000111

+

number of leading 1s as the number N,which adds a term N 1 to the measured complexity. Some example 3 and 4 node networks are shown in table 19.1. One can see how several descriptions correspond to the same topological network, but with different node numberings. A few other properties are also apparent. A network A that has a link wherever B doesn’t, and vice-versa might be called a complement of B. A bitstring for A can be found by inverting the 1s and 0s in the linklist part of the network description. Obviously, w(A,L ) = w(B,L ) . The empty network, and the fully connected network have linklists that are all 0s or 1s. These networks are maximally complex at 1 C = -N(N+1)+1 2

(19.3)

bits. This, perhaps surprising feature, is partly a consequence of the definition we’re using for network equivalence. If instead we ignored unconnected nodes (say we had an infinite number of nodes, but a only a finite number of them connected into a network), then the empty network would have extremely low complexity, as one would need to sum up the ws for N = 0,1,. . .. But in this case, there would no longer be any symmetry between a network and its complement. It is also a consequence of not using a Turing complete representation language. Empty and full networks are highly compressible, therefore we’d expect a Turing complete representation language would be able to represent the network in a compressed form, lowering the measured complexity. Networks of 3 nodes and 4 nodes are sufficiently simple that it is possible enumerate all possibilities by hand. It is possible to numerically enumerate larger networks using a computer, however one will rapidly run into diminishing returns, as the number of bitstrings to consider grows as 2 3 N ( N - 1 ) . I have done this up to 8 nodes, as shown in Fig. 19.1.

257


Network

1. ..

Network 0

I

Complement

Iw I

A

3

Complement

.

w r XI w

C

5.42

W -

C

1

11

6

8.42

12

7.42

3

9.42

12

7.42

c .

..

c . . c1

t Y 19.3

same

Y

4 -

9

Computing w

The first problem to be solved is how to determine if two network descriptions in fact correspond to the same network. We borrow a trick from the field of symbolic computing, which is to say we arrange a canonical labeling of the nodes, and then compare the canonical forms of each description. Brendan McKay [271] has solved the problem of finding canonical labelings of arbitrary graphs, and supplies a convenient software library called nauty' that implements the algorithm. The number of possible distinct descriptions is given by N ! (the number of possible renumberings of the nodes), divided by the number of such renumberings that reproduce the canonical form. As a stroke of good fortune, nauty reports this value as the order of the automorphism group, and Available from http://cs.anu.edu.au/-bdm/nauty.

258

Recent Advances i n Artijcial Life

is quite capable of computing this value for networks with 10s of thousands of nodes within seconds on modern CPUs. So the complexity value C in equation (19.1) is computationally feasible, with this particular choice of representation.

19.4

Compressed complexity and Offdiagonal complexity

I have already mentioned the issue of non Turing completeness of the proposed bitstring representation of a network. This has its most profound effect for regular networks, such as the empty or full networks, where C is at a maximum, yet contained a great deal of redundancy in the expression. To get a handle on how much difference this might make, we can try a compression algorithm of the all the equivalent bitstring representations, choosing the length most compressed representation as a new measure I call zcomplexity. Inspired by the brilliant use of standard compression programs (gzip, bzip2, Winzip etc.) to classify texts written in an unknown language[38], I initially thought to use one of these compression libraries. However, all of the usually open source compression libraries were optimised for compressing large computer files, and typically had around 100 bits of overhead. Since the complexities of all networks studied here are less than around 50 bits, this overhead precludes the use of standard techniques. So I developed my own compression routine, based around run length encoding, one of the simplest compression techniques. The encoding is simple to explain: Firstly a “wordsize” w is chosen such that log, N 5 w 5 log, N logz(N - 1) - 1. Then the representation consists of w 1 bits, followed by a zero, then w bits encoding N , then the compressed sequence of links. Repeat sequences are represented by a pair of w bit words, which give the repeat count and length of a sequence, followed by the sequence to be repeated. As an example, the network:

+

1111110101010101010101 can be compressed to

111 0 110 000 010 10 . v vvvv w

N

rpt

len

seq

Here 000 represents 8, not 0, as a zero repeat count makes no sense! Also, since the original representation is prefix free, the extra 0 that the compressed sequence adds to the original is ignored.


259

By analogy with equation (19.1) define zcomplexity as

where b iterates over all bitstring representations of the network we're measuring, and c(b) is the compressed length of b, using the best w, by the aforementioned compression algorithm. The extra 1 takes into account a bit used to indicate whether the compressed or uncompressed sequence is used, so C, 5 C 1. The optimal w for the empty (or full) network w = [log, N ] , and zcomplexity can be readily computed as

+

C, = 2 + 3 w + [

N ( N - 1) 2w+1 1

Compared with equation (19.3), we can see that it makes a substantial difference. Already at N = 5, C, = 13 and C = 16, with the difference increasing with N . To compute the zcomplexity for an arbitrary graph, we need to iterate over all possible bit representations of a graph. There are two obvious ways to do this, since the number of nodes N , and number of links Z are identical in all representations: Start with the bitstring with the initial 1 bits of the linkfield set to 1, and the remaining bits 0. Then iterate over all permutations, summing the right hand term into a bin indexed by the canonical representation of the network. This algorithm computes C, for all networks of N nodes and 1 links. This algorithm has complexity

( N ( N 1)12)= N ( N - 1 ) . . . ( N - Z)/l! Take the network, and iterate over all permutations of the node labels. Some of these permutations will have identical bitstring representations as others - as each bitstring is found, store it in a set to avoid double counting. This algorithm has complexity N !

In my experiments, I calculate zcomplexity for all networks with linkcount 1 such that

(N ( N

greater link counts.

l)")

< N ! , then sample randomly networks with

260


20

22

24

26

28

30

32

34

36

38

C

Fig. 19.1 C, plotted against C for all networks of order 8. Note the empty/full network lying in the lower right hand corner

Fig. 19.1 shows C, plotted against C for all networks of order 8, which is about the largest size network for which an exhaustive computation of C, is feasible. Unfortunately, without a smarter way of being able to iterate over equivalent bitstring representations, zcomplexity is not a feasible measure, even it more accurately represents complexity. The disparity between C, and C is greatest for highly structured graphs, so it would be interest to know when we can use C, and when a more detailed calculation is needed. Claussen[81] introduced a measure he calls ofldiugonul complexity, which measures the entropy of the distribution of links between different node degree. Regular graphs will have zero offdiagonal complexity, as the node degree distribution is sharply peaked, and takes on moderate values for random graphs (where node degree distribution is roughly exponential) and is extremal for scale-free graphs. Since the discrepancy between C and C, was most pronounced with regular graphs, I looked at offdiagonal complexity as a predictor for this discrepancy. Figure 19.2 shows the compression error (defined as plotted as a function of offdiagonal complexity and C. The dataset falls clearly into two

q)


261 links < 7 random

compression error

0.2 0.15 -

ni Oa5 0 -0.05

52

0

diagonal complexity

Fig. 19.2 Compression error as a function of C and offdiagonal complexity for networks with 10 nodes. All networks with link count less than 7 were evaluated by method 1, and 740 graphs with more than 7 links were selected at random, and computed using method 2. The separation between the two groups is due t o compressibility of sparse networks.

groups - all sparse networks with link count less than 7, and those graphs sampled randomly, corresponding to the two different methods mentioned above. The sparse networks are expected to be fairly regular, hence have high compression error, whereas randomly selected networks are most likely to be incompressible, hence have low compression error. Figure 19.3 shows the results of a linear regression analysis on offdiagonal complexity with compression error. The correlation coefficient is -0.87. So clearly offdiagonal complexity is correlated (negatively) with compression error, much as we expected, however it is not apparently a good test for indicating if the compression error is large. A better distinguishing characteristic is if C is greater than the mean random C (which can be feasibly calculated) by about 3-4 bits. What remains to be done is to look at networks generated by a dynamical process, for example Erdos-RQnyi random graphs[l22], or Barabbsi-Albert preferential attachment[26] to see if they fill in the gap between regular and algorithmically random graphs.

Recent Advances in Artificial Lzfe

262

0.4 .-

+4

0.3~7

1.

+

+

0.2 -

2

-

0.1

i

0 -

-0.1

-

-0.2-

-0.3

I

I

I

I

Off diagonal complexity

Fig. 19.3 Compression error as a function of offdiagonal complexity. A least squares linear fit is also shown

19.5 Conclusion In this paper, a simple representation language for N-node undirected graphs is given. An algorithm is presented for computing the complexity of such graphs, and the difference between this measure, and one based on a Turing complete representation language is estimated. For most graphs, the measure presented here computes complexity correctly, only graphs with a great deal of regularity are overestimated. A code implementing this algorithm is implemented in C++ library, and is available from version 4.D17 onwards as part of the EccEab system, an open source modelling framework hosted at http://ecolab.sourceforge.net. Obviously, undirected graphs is simply the start of this work - it can be readily generalised to directed graphs, and labeled graphs such as food webs (although if the edges a labeled by a real value, some form of discretisation of labels would be needed). Furthermore, most interest is in complexities of networks generated by dynamical processes, particularly evolutionary processes. Some of the first processes that should be examined are the classic Erdos-RCnyi random graphs and BarabQsi-Albert preferential attachment.


263

Acknowledgements

I wish to thank the Australian Centre for Advanced Computing and Communications for a grant of computer time, without which this study would not be possible.


Chapter 20

A Generalised Technique for Building 2D Structures with Robot Swarms R.L. Stewart and R.A. Russell

Intelligent Robotics Research Centre, Monash University, Clayton, VIC 3800, Australia E-mail: (robert.stewart, andy.russell)@eng.rnonash.edu.au Templates (or patterns) in the environment are often used by social insects to guide their building activities. Robot swarms can also use templates during construction tasks. This paper develops a generalised technique for generating templates that can vary in space and with time. It is proposed that such templates are theoretically sufficient to facilitate the loose construction of any desired planar structure. This claim is supported by a number of experimental trials in which structures are built by a real robot swarm.

20.1

Introduction

Collective construction is a relatively new problem domain for swarm-based robotics . Thus far, robot swarms have demonstrated the ability to build walls [409;261; 274; 3701, circular nests [300]and annular structures [423]. The question of how more complex structures might be constructed by swarms of minimalist robots is problematic. In addressing this question, the work in this paper is specifically directed a t how complex structures, of a designer’s choosing, might be constructed. We define our generalised two-dimensional (2D) collective construction problem as follows:

How might a robot collective be designed such that, when provided with adequate building material, the robots are capable of loosely constructing any given planar structure? 265


266

While partial answers to similar questions have recently been proposed [412], a real robot swarm capable of such a task has not (to the best of our knowledge) been demonstrated. In order to increase the potential generality of solutions to this problem, a number of restrictions are imposed. Namely, (i) robots should be designed with a minimalist philosophy, (ii) radio-communication between robots is not permitted, (iii) a global coordinate system is unavailable, and, (iv) an environment map is unavailable. In this paper, a solution to this generalised 2D collective construction problem is detailed. The solution relies on the use of spatietemporal varying templates (patterns that change in space and with time) and a distributed feedback mechanism.

20.2

Background Information

The individuals of a social insect colony are relatively simple, yet together, through their combined actions they have the capability to solve difficult problems. One such problem is nest construction. A number of mechanisms are thought to be employed by social insects during construction. One of these is the template mechanism. A template is a pattern in the environment that is “used to construct another pattern” [50]. Templates are often in the form of chemical, temperature, humidity and light heterogeneities [382] and insects use these patterns to guide their building activities. Previous work has investigated how templates might also be used to guide the building activities of robots. An early study revealed how a minimalist robot could build doughnut shaped structures using a template already present in the environment [369]. Following on from this, a robot swarm was designed and constructed with the ability to create templates of its own [370]. These templates were created by a light source mounted on a mobile organiser robot. The light source was set up so that it only projected light over an angle of 30 This situation is represented in Fig. 20.1. By generating and then using a spatio-temporal varying template, the robot swarm was able to construct a loose linear wall structure. Equipped with light sensors, builder robots were programmed to deposit building blocks when they were in a certain window of light intensities (equivalent to a window of sensed voltages, Vmin 5 V I Vmax).This window of light intensities corresponded to a spatial region, r1 5 r r2, in the beam called the deposition window (the hatched region in Fig. 20.1). Here, the term beam refers to that region of the light beam defined by Varnbient

5 v 5 vmax.

The organiser robot drove in steps along a straight line, waiting a set time (called the latency time) at each stop. This resulted in a path being

A Generalased Technique for Building 2D Structures with Robot Swarms

267

m s o u r c e voltage = S

Fig. 20.1 A light source mounted on an organiser robot projects light over a fixed angle of 30 '. The deposition window is indicated by hatching.

traced out by the deposition windows that fell along a straight line. Because the builder robots deposited blocks when they were within the deposition window, a loose linear wall structure resulted. The latency time was chosen through trial and error t o give a good trade-off between the number of blocks deposited and the total trial time. In an environment, which may be changing, choosing an appropriate latency time becomes a difficult problem. To address this problem the robot system was then modified to incorporate a distributed feedback mechanism that allowed for an adaptable choice of latency time [371]. In the new system (shown in Fig. 20.2), builder robots assessed the current state of the deposition window using a cue. If they perceived the window to be full (with blocks) they became frustrated by their inability to make a deposit. After two consecutive failed attempts to deposit a block they displayed their frustration by producing a flash of high intensity light. This burst of light acted as a signal to the organiser robot. The organiser robot waited to receive a number of these signals, possibly from different robots, before moving to the next stop. Having moved to a new stop, the deposition window was then devoid of any blocks. This meant that builder robots were again able to make deposits and their frustration level decreased. A moving organiser robot carrying a light source of fixed intensity created the spatio-temporal varying template. Because the window limits, Vminand V,,,, were fixed, the distance of the deposition window from the


268

Fig. 20.2 The small swarm of robots, called The Robotemites, with important hardware features highlighted. The organiser robot is on the far left.

organiser robot was also fixed. In this paper a new technique is proposed, and verified, that allows for variable deposition window placement without the need for the organiser robot to employ complex motion patterns itself. This is achieved through a systematic intensity variation of the light source mounted on top of the organiser robot. Both techniques are then combined to allow more complex structures to be constructed.

20.3

20.3.1

A New Technique for Creating Spatio-temporal Varying Templates Calibration

To understand how light intensity variation might lead to variable deposition window placement, a calibration routine was undertaken. The calibration routine involved driving a builder robot away from the organiser robot’s light source along a radial line. The starting position of the builder robot was such that the initial distance between the organiser robot’s light source and the builder robot’s absolute light sensor was 0.265m. The builder robot was stopped every O.lm and drove a total distance of 1.5m. At each stop the intensity of the organiser robot’s light bulb (light source) was varied by automatically adjusting the voltage applied to it. This light source voltage, S, was varied from 30 to 100 normalised units in increments of 5. For each different light source voltage, the light level sensed by the builder robot was recorded (in terms of the N

N

N

A Generalised Technique for Building 2 0 Structures with Robot S w a m

269

h

20

; $10

+--

U

---@--e ~~

0.265 0

---t--c----J,

moo

rzioo

1.7677

distance from light source (m)

Fig. 20.3 Plots of two characteristic curves for different light source voltages (namely, S = 60 and S = 100). Also shown are two lines that define the deposition window, 90 5 v 5 120.

sensed voltage, V ) . Note that the higher the value for S, the more intense the light source was. Two typical data curves obtained during this process are shown in Fig. 20.3. As can be seen, each curve is a monotonically decreasing function of distance as expected. Also evident from the entire set of data curves (not shown in Fig. 20.3) was that for any given distance from the light source, the sensed voltage was a monotonically increasing function of light source voltage. To understand how the deposition window placement ( T I 5 T 5 7-2) can be varied, consider the situation where builder robots are, as in previous trials, programmed t o deposit building material when they are in a certain window of sensed voltages, Vmin 5 V 5 V,,,. Consider again Fig. 20.3 which shows the two example data curves that correspond to the light = 120 source voltages S = 60 and S = 100. Two lines for V1 = V,, and V2 = Vmin = 90 are also shown on the same graph representing a n arbitrarily chosen deposition window definition (90 5 V 5 120). The two lines are seen to intersect each curve at different locations. If the source intensity is low ( S = 60) the deposition window placement is close to the light source (T1,60 5 T 5 T2,60). If on the other hand, the source intensity is

270


high (S = 100) the deposition window placement is further away from the light source ( ~ 1 , 1 0 05 T 5 ~ 2 , 1 0 0 ) It . is therefore clear that by systematically varying the light source intensity the location of the deposition window can be controlled. By re-expressing the obtained data, it was possible to determine appropriate values for Vmin,Vm,, and Vambient (namely, 50, 70 and 15 respectively). A number of factors had to be considered when deciding upon values for these parameters. For brevity, details regarding this design process will be provided elsewhere. With the design parameters chosen, it is possible to determine the light source voltage required to give a deposition window placement that starts at some desired position ( T I ) , as well as the depth of the deposition window (7-2 - T I ) . Due to the non-linear nature of the data curves (e.g. Fig. 20.3), the deposition window depth does increase somewhat with distance. This could be regarded as a loss of resolution with distance and is intrinsic t o this system.

20.3.2

Experimental Procedure

The trials in this section (20.3) were conducted with a robot swarm consisting of one organiser robot and two builder robots. The two builder robots were known to possess similar absolute sensor responses to light intensity so that calibration only had to be performed once. As in previous experiments, the organiser robot had a light source mounted on top of it while builder robots possessed an absolute light sensor, a gripper and a flash bulb (for producing bursts of high intensity light). All robots had a light sensor ring, infra-red (IR) proximity sensors and a radiomodem (for data logging purposes). The complete robot swarm is shown in Fig. 20.2. A rectangular test area (with inner dimensions of 2.56m x 2.16m) was used for the trials. The organiser robot was stationary for the duration of these trials and the builder robots roamed freely. As in previous experiments, two layers of building blocks were arranged outside the inner perimeter of the main test area. Blocks taken from the outermost layer during the trials were replaced manually by the observer. If blocks were pushed aside in the outermost layer so that a large gap developed, blocks were added to a third layer where needed. The organiser and builder robots were programmed with the rule sets that are shown in Tables 20.1 and 20.2 respectively. The If-Then rules appearing first receive higher precedence. The feedback mechanism described earlier (Section 20.2) is evident in these rules (allowing for an adaptable choice of latency time).

A Generalised Technique for Building 2D Structures with Robot Swarms

271

Table 20.1 The rule set used by the organiser robot in Trials 1 and 2. (1) If (number of deposition windows completed equals the total number of deposition windows required) Then (turn off light bulb) (2) If (3 flashes have been detected for the current deposition window or the trial has just been started) Then (change light source voltage to the next value in the sequence

1)

(an

(3) Count flash signals

NOTE: In Trial 1 the sequence (a,) = (Sl,Sz, 5’3, S4, S5, &) and in Trial 2 the sequence (a,) = ( s i , s z , S 6 ) where S1 = 41.5, Sz = 48.2, S3 = 56.4, S4 = 67.4, S5 = 81.6 and s6 = 100.0.

Table 20.2 The rule set used by the builder robots in all trials. (1) If (not holding block and not in beam and object detected) Then(attempt to pick up object) (2) If (holding block and in deposition window) Then(deposit block behind another block or at inner window limit and reset frustration counter) (3) If (holding block and have just arrived in beam) Then (drive further into beam avoiding obstacles if detected) (4) If (frustration counter equals 2) Then (flash light and reset frustration counter) (5) If (holding block and in beam and obstacle detected) Then (increment frustration counter, turn randomly f 1 3 5 O and then drive forwards) (6) If (holding block and in beam) Then (drive up beam towards light) (7) If (obstacle detected) Then avoid obstacle (8) 8. Drive forwards

20.3.3

Building a Radial Wall With and Without a Gap

The first two trials undertaken demonstrate that the deposition window placement, and hence block placement, can indeed be varied by altering the light source intensity. From the calibration results and chosen design parameters, different light source voltages (equivalent to source intensities) were chosen to create a certain number of deposition windows that lined up with one another. This was achieved by making the inner limit placement of each deposition window equal to the outer limit placement of the proceeding deposition window. Figure 20.4 shows the placement of the different deposition windows and the corresponding light source voltages.

272

Recent Advances an Artijicaal Life

s, = 56.4 S, = 67.4 Ss= 81.6

s, = 100.0 Fig. 20.4 Deposition window placements (numbered from 1 to 6) and the corresponding light source voltages used to create these placements.

In the first trial (Trial l), the organiser robot was programmed to have an initial light source voltage of S1. After waiting a certain latency time (determined by the feedback system) the organiser robot was programmed to then change its light source voltage to the next value, S2, and wait again for a new latency time. This process continued until all light source voltages in the sequence (a,) = (Sl,,572, S3,S4, Sg, SS) = (41.5,48.2,56.4,67.4,81.6,100.0)had been applied and the appropriate latency time waited for each. Note that the definition for latency time has now been broadened to mean the time spent waiting by the organiser robot for the current deposition window to be deemed full. Because the deposition windows fall along a radial line from the light source (Fig. 20.4), it was expected that deposited blocks would also fall along a radial line. Figure 20.5.a shows the structure constructed by the robot swarm in this trial. Blocks were deposited in the expected manner allowing a radial wall structure to grow out from a starting point. A natural extension to the first trial was to create a discontinuity in the radial wall structure. In a second trial (Trial 2) the organiser robot was given a reduced sequence of light source voltages (a,) = (Sl,S2,Ss)= (41.5,48.2,100.0) so that the 3 r d , 4th and 5th deposition windows in Fig. 20.4 would be absent. From this choice of light source voltages it was expected that a radial wall similar to that in Fig. 20.5.a would be constructed but that this time there would be a gap in the wall. As expected, a radial wall with a gap was constructed by the robot

A Genemlised Technique for Building 2D Strzlctures with Robot S w a m

273

Fig. 20.5 The configuration of the test area (with robots removed) after the completion of (a) Trial 1 and (b) Trial 2. The organiser robot's position during each trial is indicated by a circle and the structures are located in the graphically enhanced lighter areas. In the first trial a radial wall has been constructed and in the second trial a radial wall with a gap has been constructed.

swarm (see Fig. 20.5.b). For this to happen an appropriate jump in light source intensity was all that was required. Again the structure was observed to grow outwards away from the light source as was dictated by the sequence of light source voltages. The results from both trials confirm that the proposed technique for generating a spatio-temporal varying template is indeed viable. 20.4

Solving the Generalised 2D Collective Construction Problem

Thus far two techniques have been found for generating spatio-temporal varying templates. In general terms, these are (i) a moving source of fmed intensity [370] and (ii) a stationary source of variable intensity (Section 20.3). By combining these we arrive at the more general technique for generating spatio-temporal varying templates, namely, (iii) a moving source of variable intensity. It is through this generalised technique that it should be possible to realise any desired 2D structure without violating the problem restrictions. By establishing a Cartesian coordinate system relative to the organiser robot's direction of movement (Fig. 20.6), it becomes clear that for any given deposition window placement centred on (xi,yi), there exists a corresponding source intensity (equivalent to a light source voltage, Si = j-'(yi)) and organiser displacement (xi). Any given 2D structure can then be decomposed into a building program that consists of a sequence of linear displacement values and light source voltages that govern the spatial location of deposition windows. If each deposition window is filled with

274


Y’

Yi --

”&&

Oganrscr robot at ( x

x,

~-

__c_

Organiwr robol’s direction 01 inoveincnt

Fig. 20.6 A conceptual coordinate system established relative to the organiser robot’s initial position and its direction of intended movement.

blocks by the builder robots then we would expect the final built structure to be a loosely constructed realisation of the intended structure design. To verify this generalised technique, a number of trials were proposed and undertaken . 20.4.1

Experimental Procedure

The entire 5-member robot swarm was used in the remaining trials (Fig. 20.2). Calibration was performed for all builder robots. The design parameters chosen for each builder robot ensured they all had a consistent quantifiable response to light. The physical dimensions for the test area set-up were kept the same (as in Section 20.3) as was the rule set followed by the builder robots (see Table 20.2). The organiser robot’s rule set (Table 20.1) was modified somewhat to allow for the possibility of movement after the completion of each deposition window (Table 20.3). This gave the organiser robot, the ability to reach all required displacement values (xi in Fig. 20.6). 20.4.2

Building Structures of Greater Complexity

Three trials were undertaken to verify the generalised technique for building structures using spatio-temporal varying templates. Sequences of light source voltages (a,) and movement distances (b,) were programmed into

A Genemlised Technique for Building 2 0 Structures with Robot Swarms

275

Table 20.3 The modified rule set used by the organiser robot in Trials 3, 4 and 5 .

If (number of deposition windows completed equals the total number of deposition windows required) Then (turn off light bulb) If (3 flashes have been detected for the current deposition window or the trial has just been started) Then (move/rotate a distance given by the next value in the sequence bn and change the light source voltage to the next value from the sequence an 1

Count flash signals

the organiser robot. These sequences specified the placement of deposition windows relative to the organiser robot. Deposition window placements were ordered so that previous construction did not inhibit future construction. This was achieved by ensuring structural growth was always away from the organiser robot. In each trial the feedback system enabled the transition between deposition windows. Figure 20.7 gives a representation of three different spatio-temporal varying template designs and the resulting structures for each of these that were actually built by the robot swarm. In the first trial (Trial 3, 20.7.a and 20.7.b), a diagonal wall was constructed to show how linear motion and variable source intensity can readily be combined. In the second trial (Trial 4,20.7.c and 20.7.d), a cross shape structure was built to demonstrate the capacity for the swarm to build intersecting walls. In the third trial (Trial 5, 20.7.e and 20.7.f), the organiser robot rotated incrementally and varied its intensity so that two concentric arcs were constructed. This trial showed that rotational motion can be used instead of linear motion.

20.5

General Discussion

In each of the trials detailed in this paper, the builder robots did not need to discern changes made to the light source intensity by the organiser robot. Instead, they wandered about searching for the beam and the fixed conditions (namely, a sensed voltage in the range Vmin V V,,,) under which they would make a deposit. That is, there was no need for the builder robots to be informed directly that the spatial location of the deposition window was being varied. It was enough for the organiser robot to make changes to a pattern in the environment that indirectly affected the actions undertaken by builder robots. The generalised technique for building 2D structures has a number

<

Recent Advances in Artificial Life: Sydney, Australia 5 - 8 December 2005

Recent Advances in Otolaryngology 8

Progress in Artificial Intelligence: 12th Portuguese Conference on Artificial Intelligence, EPIA 2005, Covilha, Portugal, December 5-8, 2005,

Recent Advances in Intrusion Detection, 8 conf., RAID 2005

Advances in Artificial Life: 8th European Conference, ECAL 2005, Canterbury, UK, September 5-9, 2005, Proceedings

Recent advances in artificial neural networks

AI 2006: Advances in Artificial Intelligence: 19th Australian Joint Conference on Artificial Intelligence, Hobart, Australia, December 4-8, 2006: Proceedings

Recent Advances in Technologies

Recent Advances in Mechatronics

Recent Advances in Mechanics

Recent Advances in Surgery

Recent Advances in Constraints

Recent Advances in Optimization

Recent Advances in Physiotherapy

Recent Advances in Mechatronics

Recent Advances in Optimization

Real and Complex Singularities: Proceedings of the Australian-Japanese Workshop, University of Sydney, Australia, 5-8 September 2005

Advances in Artificial Intelligence

Frommer's Australia 2005

Recent Advances in Anaesthesia and Intensive Care 022 (Recent Advances)

Global Finance (December 2005)

Web Engineering: 5th International Conference, ICWE 2005, Sydney, Australia, July 27-29, 2005, Proceedings

php|architect (December 2005)

Recent Advances in Computational Terminology

Recent Advances in Epilepsy Research

Recent Advances in Hydride Chemistry

Recent Advances in Phototrophic Prokaryotes

Recent Advances in Plant Biotechnology

Recent advances in nonsmooth optimization

Recent Advances in Reinforcement Learning

Recent Advances in Applied Probability

Recent Advances in Artificial Life: Sydney, Australia 5 - 8 December 2005

Recent Advances in Otolaryngology 8

Progress in Artificial Intelligence: 12th Portuguese Conference on Artificial Intelligence, EPIA 2005, Covilha, Portugal, December 5-8, 2005,

Recent Advances in Intrusion Detection, 8 conf., RAID 2005

Advances in Artificial Life: 8th European Conference, ECAL 2005, Canterbury, UK, September 5-9, 2005, Proceedings

Recent advances in artificial neural networks

AI 2006: Advances in Artificial Intelligence: 19th Australian Joint Conference on Artificial Intelligence, Hobart, Australia, December 4-8, 2006: Proceedings

Recent Advances in Technologies

Recent Advances in Mechatronics

Recent Advances in Mechanics

Recent Advances in Surgery

Recent Advances in Constraints

Recent Advances in Optimization

Recent Advances in Physiotherapy

Recent Advances in Mechatronics

Recent Advances in Optimization

Real and Complex Singularities: Proceedings of the Australian-Japanese Workshop, University of Sydney, Australia, 5-8 September 2005

Advances in Artificial Intelligence

Frommer's Australia 2005

Recent Advances in Anaesthesia and Intensive Care 022 (Recent Advances)

Global Finance (December 2005)

Web Engineering: 5th International Conference, ICWE 2005, Sydney, Australia, July 27-29, 2005, Proceedings

php|architect (December 2005)

Recent Advances in Computational Terminology

Recent Advances in Epilepsy Research

Recent Advances in Hydride Chemistry

Recent Advances in Phototrophic Prokaryotes

Recent Advances in Plant Biotechnology

Recent advances in nonsmooth optimization

Recent Advances in Reinforcement Learning

Recent Advances in Applied Probability

Recommend Documents