Advances in Swarm Intelligence, Part II

Lecture Notes in Computer Science Commenced Publication in 1973 Founding and Former Series Editors: Gerhard Goos, Juris...

Author: Ying Tan | Yuhui Shi | Yi Chai | Guoyin Wang

146 downloads 4994 Views 15MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

Lecture Notes in Computer Science Commenced Publication in 1973 Founding and Former Series Editors: Gerhard Goos, Juris Hartmanis, and Jan van Leeuwen

Editorial Board David Hutchison Lancaster University, UK Takeo Kanade Carnegie Mellon University, Pittsburgh, PA, USA Josef Kittler University of Surrey, Guildford, UK Jon M. Kleinberg Cornell University, Ithaca, NY, USA Alfred Kobsa University of California, Irvine, CA, USA Friedemann Mattern ETH Zurich, Switzerland John C. Mitchell Stanford University, CA, USA Moni Naor Weizmann Institute of Science, Rehovot, Israel Oscar Nierstrasz University of Bern, Switzerland C. Pandu Rangan Indian Institute of Technology, Madras, India Bernhard Steffen TU Dortmund University, Germany Madhu Sudan Microsoft Research, Cambridge, MA, USA Demetri Terzopoulos University of California, Los Angeles, CA, USA Doug Tygar University of California, Berkeley, CA, USA Gerhard Weikum Max Planck Institute for Informatics, Saarbruecken, Germany

6729

Ying Tan Yuhui Shi Yi Chai Guoyin Wang (Eds.)

Advances in Swarm Intelligence Second International Conference, ICSI 2011 Chongqing, China, June 12-15, 2011 Proceedings, Part II

13

Volume Editors Ying Tan Peking University Key Laboratory of Machine Perception (MOE) Department of Machine Intelligence Beijing, 100871, China E-mail: [email protected] Yuhui Shi Xi’an Jiaotong-Liverpool University Department of Electrical and Electronic Engineering Suzhou, 215123,China E-mail: [email protected] Yi Chai Chongqing University Automation College Chongqing 400030, China E-mail: [email protected] Guoyin Wang Chongqing University of Posts and Telecommunications College of Computer Science and Technology Chongqing, 400065, China E-mail: [email protected] ISSN 0302-9743 e-ISSN 1611-3349 ISBN 978-3-642-21523-0 e-ISBN 978-3-642-21524-7 DOI 10.1007/978-3-642-21524-7 Springer Heidelberg Dordrecht London New York Library of Congress Control Number: 2011928465 CR Subject Classification (1998): F.1, H.3, I.2, H.4, H.2.8, I.4-5 LNCS Sublibrary: SL 1 – Theoretical Computer Science and General Issues

© Springer-Verlag Berlin Heidelberg 2011 This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typesetting: Camera-ready by author, data conversion by Scientific Publishing Services, Chennai, India Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com)

Preface

This book and its companion volume, LNCS vols. 6728 and 6729, constitute the proceedings of the Second International Conference on Swarm Intelligence (ICSI 2011) held during June 12–15, 2011 in Chongqing, well known as the Mountain City, the southwestern commercial capital of China. ICSI 2011 was the second gathering in the world for researchers working on all aspects of swarm intelligence, following the successful and fruitful Beijing ICSI event in 2010, which provided a high-level international academic forum for the participants to disseminate their new research findings and discuss emerging areas of research. It also created a stimulating environment for the participants to interact and exchange information on future challenges and opportunities in the field of swarm intelligence research. ICSI 2011 received 298 submissions from about 602 authors in 38 countries and regions (Algeria, American Samoa, Argentina, Australia, Austria, Belize, Bhutan, Brazil, Canada, Chile, China, Germany, Hong Kong, Hungary, India, Islamic Republic of Iran, Japan, Republic of Korea, Kuwait, Macau, Madagascar, Malaysia, Mexico, New Zealand, Pakistan, Romania, Saudi Arabia, Singapore, South Africa, Spain, Sweden, Chinese Taiwan, Thailand, Tunisia, Ukraine, UK, USA, Vietnam) across six continents (Asia, Europe, North America, South America, Africa, and Oceania). Each submission was reviewed by at least 2 reviewers, and on average 2.8 reviewers. Based on rigorous reviews by the Program Committee members and reviewers, 143 high-quality papers were selected for publication in the proceedings with an acceptance rate of 47.9%. The papers are organized in 23 cohesive sections covering all major topics of swarm intelligence research and development. In addition to the contributed papers, the ICSI 2011 technical program included four plenary speeches by Russell C. Eberhart (Indiana University Purdue University Indianapolis (IUPUI), USA), K. C. Tan (National University of Singapore, Singapore, the Editor-in-Chief of IEEE Computational Intelligence Magazine (CIM)), Juan Luis Fernandez Martnez (University of Oviedo, Spain), Fernando Buarque (University of Pernambuco, Brazil). Besides the regular oral sessions, ICSI 2011 had two special sessions on ‘Data Fusion and Swarm Intelligence’ and ‘Fish School Search Foundations and Application’ as well as several poster sessions focusing on wide areas. As organizers of ICSI 2011, we would like to express sincere thanks to Chongqing University, Peking University, Chongqing University of Posts and Telecommunications, and Xi’an Jiaotong-Liverpool University for their sponsorship, to the IEEE Computational Intelligence Society, World Federation on Soft Computing, International Neural Network Society, and Chinese Association for Artificial Intelligence for their technical co-sponsorship. We appreciate the Natural Science Foundation of China for its financial and logistic supports.

VI

Preface

We would also like to thank the members of the Advisory Committee for their guidance, the members of the International Program Committee and additional reviewers for reviewing the papers, and members of the Publications Committee for checking the accepted papers in a short period of time. Particularly, we are grateful to the proceedings publisher Springer for publishing the proceedings in the prestigious series of Lecture Notes in Computer Science. Moreover, we wish to express our heartfelt appreciation to the plenary speakers, session chairs, and student helpers. There are still many more colleagues, associates, friends, and supporters who helped us in immeasurable ways; we express our sincere gratitude to them all. Last but not the least, we would like to thank all the speakers and authors and participants for their great contributions that made ICSI 2011 successful and all the hard work worthwhile. June 2011

Ying Tan Yuhui Shi Yi Chai Guoyin Wang

Organization

General Chairs Russell C. Eberhart Dan Yang Ying Tan

Indiana University - Purdue University, USA Chongqing University, China Peking University, China

Advisory Committee Chairs Xingui He Qidi Wu Gary G. Yen

Peking University, China Tongji University, China Oklahoma State University, USA

Program Committee Chairs Yuhui Shi Guoyin Wang

Xi’an Jiaotong-Liverpool University, China Chongqing University of Posts and Telecommunications, China

Technical Committee Chairs Yi Chai Andries Engelbrecht Nikola Kasabov Kay Chen Tan Peng-yeng Yin Martin Middendorf

Chongqing University, China University of Pretoria, South Africa Auckland University of Technology, New Zealand National University of Singapore, Singapore National Chi Nan University, Taiwan, China University of Leipzig, Germany

Plenary Sessions Chairs Xiaohui Cui James Tin-Yau Kwok

Oak Ridge National Laboratory, USA The Hong Kong University of Science and Technology, China

Special Sessions Chairs Majid Ahmadi Hongwei Mo Yi Zhang

University of Windsor, Canada Harbin Engineering University, China Sichuan University, China

VIII

Organization

Publications Chairs Rajkumar Roy Radu-Emil Precup Yue Sun

Cranfield University, UK Politehnica University of Timisoara, Romania Chongqing University, China

Publicity Chairs Xiaodong Li Haibo He Lei Wang Weiren Shi Jin Wang

RMIT Unversity, Australia University of Rhode Island Kingston, USA Tongji University, China Chongqing University, China Chongqing University of Posts and Telecommunications, China

Finance Chairs Chao Deng Andreas Janecek

Peking University, China University of Vienna, Austria

Local Arrangements Chairs Dihua Sun Qun Liu

Chongqing University, China Chongqing University of Posts and Telecommunications, China

Program Committee Members Payman Arabshahi Carmelo Bastos Christian Blum Leandro Leandro dos Santos Coelho

University of Washington, USA University of Pernambuco, Brazil Universitat Politecnica de Catalunya, Spain

Pontif´ıcia Universidade Cat´ olica do Parana, Brazil Carlos Coello Coello CINVESTAV-IPN, Mexico Oscar Cordon European Centre for Soft Computing, Spain Jose Alfredo Ferreira Costa UFRN Universidade Federal do Rio Grande do Norte, Brazil Iain Couzin Princeton University, USA Xiaohui Cui Oak Ridge National Laboratory, USA Swagatam Das Jadavpur University, India Prithviraj Dasgupta University of Nebraska, USA Kusum Deep Indian Institute of Technology Roorkee, India Mingcong Deng Okayama University, Japan Haibin Duan Beijing University of Aeronautics and Astronautics, China

Organization

Mark Embrechts Andries Engelbrecht Wai-Keung Fung Beatriz Aurora Garro Licon Dunwei Gong Ping Guo Walter Gutjahr Qing-Long Han Haibo He Lu Hongtao Mo Hongwei Zeng-Guang Hou Huosheng Hu Guang-Bin Huang Yuancheng Huang Hisao Ishibuchi Andreas Janecek Zhen Ji Changan Jiang Licheng Jiao Colin Johnson Farrukh Aslam Khan Arun Khosla Franziska Kl¨ ugl James Kwok Xiaodong Li Yangmin Li Fernando Buarque De Lima Neto Guoping Liu Ju Liu Qun Liu Wenlian Lu Juan Luis Fernandez Martinez Wenjian Luo Jinwen Ma Bernd Meyer

IX

RPI, USA University of Pretoria, South Africa University of Manitoba, Canada CIC-IPN, Mexico China University of Mining and Technology, China Beijing Normal University, China University of Vienna, Austria Central Queensland University, Australia University of Rhode Island, USA Shanghai Jiao Tong University, China Harbin Engineering University, China Institute of Automation, Chinese Academy of Sciences, China University of Essex, UK Nanyang Technological University, Singapore Wuhan University, China Osaka Prefecture University, Japan University of Vienna, Austria Shenzhen University, China Kagawa University, Japan Xidian University, China University of Kent, UK FAST-National University of Computer and Emerging Sciences, Pakistan National Institute of Tech. Jalandhar, India ¨ Orebro University, Sweden Hong Kong University of Science and Technology, China RMIT University, Australia University of Macau, China Polytechnic School of Pernambuco, Brazil University of Glamorgan, UK Shandong University, China Chongqing University of Posts and Communications, China Fudan University, China University of Oviedo, Spain University of Science and Technology of China, China Peking University, China Monash University, Australia

X

Organization

Martin Middendorf Mahamed G. H. Omran Jeng-Shyang Pan Pan Shaoning Pang Bijaya Ketan Panigrahi Thomas Potok Radu-Emil Precup Guenter Rudolph Gerald Schaefer Yuhui Shi Michael Small Jim Smith Ponnuthurai Suganthan Norikazu Takahashi Kay-Chen Tan Ying Tan Ke Tang Peter Tino Christos Tjortjis Frans Van Den Bergh Ba-Ngu Vo Bing Wang Guoyin Wang Hongbo Wang Jiahai Wang Jin Wang Lei Wang Ling Wang Lipo Wang Benlian Xu Pingkun Yan Yingjie Yang Hoengpeng Yin Peng-Yeng Yin Dingli Yu Jie Zhang Jun Zhang Lifeng Zhang Qieshi Zhang Qingfu Zhang

University of Leipzig, Germany Gulf University for Science and Technology, Kuwait National Kaohsiung University of Applied Sciences, Taiwan, China Auckland University of Technology, New Zealand IIT Delhi, India ORNL, USA Politehnica University of Timisoara, Romania TU Dortmund University, Germany Loughborough University, UK Xi’an Jiaotong-Liverpool University, China Hong Kong Polytechnic University, China University of the West of England, UK Nanyang Technological University, Singapore Kyushu University, Japan National University of Singapore, Singapore Peking University, China University of Science and Technology of China, China University of Birmingham, UK The University of Manchester, UK CSIR, South Africa The University of Western Australia, Australia University of Hull, UK Chongqing University of Posts and Telecommunications, China Yanshan University, China Sun Yat-sen University, China Chongqing University of Posts and Telecommunications, China Tongji University, China Tsinghua University, China Nanyang Technological University, Singapore Changshu Institute of Technology, China Philips Research North America, USA De Montfort University, UK Chongqing University, China National Chi Nan University, Taiwan, China Liverpool John Moores University, UK Newcastle University, UK Waseda University, Japan Renmin University of China, China Waseda University, Japan University of Essex, UK

Organization

Dongbin Zhao Zhi-Hua Zhou

Institute of Automation, Chinese Academy of Science, China Nanjing University, China

Additional Reviewers Bi, Chongke Cheng, Chi Tai Damas, Sergio Ding, Ke Dong, Yongsheng Duong, Tung Fang, Chonglun Guo, Jun Henmi, Tomohiro Hu, Zhaohui Huang, Sheng-Jun Kalra, Gaurav Lam, Franklin Lau, Meng Cheng Leung, Carson K. Lu, Qiang Nakamura, Yukinori Osunleke, Ajiboye

Qing, Li Quirin, Arnaud Saleem, Muhammad Samad, Rosdiyana Sambo, Francesco Singh, Satvir Sun, Fuming Sun, Yang Tang, Yong Tong, Can V´ azquez, Roberto A. Wang, Hongyan Wang, Lin Yanou, Akira Zhang, Dawei Zhang, X.M. Zhang, Yong Zhu, Yanqiao

XI

Table of Contents – Part II

Multi-Objective Optimization Algorithms Multi-Objective Optimization for Dynamic Single-Machine Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Li Nie, Liang Gao, Peigen Li, and Xiaojuan Wang

1

Research of Pareto-Based Multi-Objective Optimization for Multi-vehicle Assignment Problem Based on MOPSO . . . . . . . . . . . . . . . . . Ai Di-Ming, Zhang Zhe, Zhang Rui, and Pan Feng

10

Correlative Particle Swarm Optimization for Multi-objective Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yuanxia Shen, Guoyin Wang, and Qun Liu

17

A PSO-Based Hybrid Multi-Objective Algorithm for Multi-Objective Optimization Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Xianpeng Wang and Lixin Tang

26

The Properties of Birandom Multiobjective Programming Problems . . . . Yongguo Zhang, Yayi Xu, Mingfa Zheng, and Liu Ningning

34

A Modified Multi-objective Binary Particle Swarm Optimization Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ling Wang, Wei Ye, Xiping Fu, and Muhammad Ilyas Menhas

41

Improved Multiobjective Particle Swarm Optimization for Environmental/Economic Dispatch Problem in Power System . . . . . . . . . Yali Wu, Liqing Xu, and Jingqian Xue

49

A New Multi-Objective Particle Swarm Optimization Algorithm for Strategic Planning of Equipment Maintenance . . . . . . . . . . . . . . . . . . . . . . . Haifeng Ling, Yujun Zheng, Ziqiu Zhang, and Xianzhong Zhou

57

Multiobjective Optimization for Nurse Scheduling . . . . . . . . . . . . . . . . . . . . Peng-Yeng Yin, Chih-Chiang Chao, and Ya-Tzu Chiang

66

A Multi-objective Binary Harmony Search Algorithm . . . . . . . . . . . . . . . . . Ling Wang, Yunfei Mao, Qun Niu, and Minrui Fei

74

Multi-robot, Swarm-robot, and Multi-agent Systems A Self-organized Approach to Collaborative Handling of Multi-robot Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tian-yun Huang, Xue-bo Chen, Wang-bao Xu, and Wei Wang

82

XIV


An Enhanced Formation of Multi-robot Based on A* Algorithm for Data Relay Transmission . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Zhiguang Xu, Kyung-Sik Choi, Yoon-Gu Kim, Jinung An, and Suk-Gyu Lee WPAN Communication Distance Expansion Method Based on Multi-robot Cooperation Navigation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yoon-Gu Kim, Jinung An, Kyoung-Dong Kim, Zhi-Guang Xu, and Suk-Gyu Lee

91

99

Relative State Modeling Based Distributed Receding Horizon Formation Control of Multiple Robot Systems . . . . . . . . . . . . . . . . . . . . . . . Wang Zheng, He Yuqing, and Han Jianda

108

Simulation and Experiments of the Simultaneous Self-assembly for Modular Swarm Robots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hongxing Wei, Yizhou Huang, Haiyuan Li, and Jindong Tan

118

Impulsive Consensus in Networks of Multi-agent Systems with Any Communication Delays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Quanjun Wu, Li Xu, Hua Zhang, and Jin Zhou

128

Data Mining Methods FDClust: A New Bio-inspired Divisive Clustering Algorithm . . . . . . . . . . . Besma Khereddine and Mariem Gzara

136

Mining Class Association Rules from Dynamic Class Coupling Data to Measure Class Reusability Pattern . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Anshu Parashar and Jitender Kumar Chhabra

146

An Algorithm of Constraint Frequent Neighboring Class Sets Mining Based on Separating Support Items . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Gang Fang, Jiang Xiong, Hong Ying, and Yong-jian Zhao

157

A Multi-period Stochastic Production Planning and Sourcing Problem with Discrete Demand Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Weili Chen, Yankui Liu, and Xiaoli Wu

164

Exploration of Rough Sets Analysis in Real-World Examination Timetabling Problem Instances . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . J. Joshua Thomas, Ahamad Tajudin Khader, Bahari Belaton, and Amy Leow Community Detection in Sample Networks Generated from Gaussian Mixture Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ling Zhao, Tingzhan Liu, and Jian Liu

173

183


XV

Efficient Reduction of the Number of Associations Rules Using Fuzzy Clustering on the Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Amel Grissa Touzi, Aicha Thabet, and Minyar Sassi

191

A Localization Algorithm in Wireless Sensor Networks Based on PSO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hui Li, Shengwu Xiong, Yi Liu, Jialiang Kou, and Pengfei Duan

200

Game Theoretic Approach in Routing Protocol for Cooperative Wireless Sensor Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Qun Liu, Xingping Xian, and Tao Wu

207

Machine Learning Methods A New Collaborative Filtering Recommendation Approach Based On Naive Bayesian Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kebin Wang and Ying Tan

218

Statistical Approach for Calculating the Energy Consumption by Cell Phones . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Shanchen Pang and Zhonglei Yu

228

Comparison of Ensemble Classifiers in Extracting Synonymous Chinese Transliteration Pairs from Web . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chien-Hsing Chen and Chung-Chian Hsu

236

Combining Classifiers by Particle Swarms with Local Search . . . . . . . . . . . Liying Yang

244

An Expert System Based on Analytical Hierarchy Process for Diabetes Risk Assessment (DIABRA) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mohammad Reza Amin-Naseri and Najmeh Neshat

252

Practice of Crowd Evacuating Process Model with Cellular Automata Based on Safety Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Shi Xi Tang and Ke Ming Tang

260

Feature Selection Algorithms Feature Selectionfor Unlabeled Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chien-Hsing Chen Feature Selection Algorithm Based on Least Squares Support Vector Machine and Particle Swarm Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . Song Chuyi, Jiang Jingqing, Wu Chunguo, and Liang Yanchun Unsupervised Local and Global Weighting for Feature Selection . . . . . . . . Nadia Mesghouni, Khaled Ghedira, and Moncef Temani

269

275 283

XVI


Graph-Based Feature Recognition of Line-Like Topographic Map Symbols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Rudolf Szendrei, Istv´ an Elek, and M´ aty´ as M´ arton

291

Automatic Recognition of Topographic Map Symbols Based on Their Textures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Rudolf Szendrei, Istv´ an Elek, and Istv´ an Fekete

299

Using Population Based Algorithms for Initializing Nonnegative Matrix Factorization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Andreas Janecek and Ying Tan

307

A Kind of Object Level Measuring Method Based on Image Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Xiaoying Wang and Yingge Chen

317

Pattern Recognition Methods Fast Human Detection Using a Cascade of United Hogs . . . . . . . . . . . . . . . Wenhui Li, Yifeng Lin, and Bo Fu The Analysis of Parameters t and k of LPP on Several Famous Face Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sujing Wang, Na Zhang, Mingfang Sun, and Chunguang Zhou

327

333

Local Block Representation for Face Recognition . . . . . . . . . . . . . . . . . . . . . Liyuan Jia, Li Huang, and Lei Li

340

Feature Level Fusion of Fingerprint and Finger Vein Biometrics . . . . . . . . Kunming Lin, Fengling Han, Yongming Yang, and Zulong Zhang

348

A Research of Reduction Algorithm for Support Vector Machine . . . . . . . Susu Liu and Limin Sun

356

Fast Support Vector Regression Based on Cut . . . . . . . . . . . . . . . . . . . . . . . Wenyong Zhou, Yan Xiong, Chang-an Wu, and Hongbing Liu

363

Intelligent Control Using Genetic Algorithm for Parameter Tuning on ILC Controller Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Alireza rezaee and Mohammad jafarpour jalali

371

Controller Design for a Heat Exchanger in Waste Heat Utilizing Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jianhua Zhang, Wenfang Zhang, Ying Li, and Guolian Hou

379


XVII

Test Research on Radiated Susceptibility of Automobile Electronic Control System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Shenghui Yang, Xiangkai Liu, Xiaoyun Yang, and Yu Xiao

387

Forgeability Attack of Two DLP-Base Proxy Blind Signature Schemes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jianhong Zhang, Fenhong Guo, Zhibin Sun, and Jilin Wang

395

Other Optimization Algorithms and Applications Key Cutting Algorithm and Its Variants for Unconstrained Optimization Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Uthen Leeton and Thanatchai Kulworawanichpong

403

Transmitter-Receiver Collaborative-Relay Beamforming by Simulated Annealing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dong Zheng, Ju Liu, Lei Chen, Yuxi Liu, and Weidong Guo

411

Calculation of Quantities of Spare Parts and the Estimation of Availability in the Repaired as Old Models . . . . . . . . . . . . . . . . . . . . . . . . . . Zhe Yin, Feng Lin, Yun-fei Guo, and Mao-sheng Lai

419

The Design of the Algorithm of Creating Sudoku Puzzle . . . . . . . . . . . . . . Jixian Meng and Xinzhong Lu Research and Validation of the Smart Power Two-Way Interactive System Based on Unified Communication Technology . . . . . . . . . . . . . . . . . Jianming Liu, Jiye Wang, Ning Li, and Zhenmin Chen

427

434

A Micro Wireless Video Transmission System . . . . . . . . . . . . . . . . . . . . . . . . Yong-ming Yang, Xue-jun Chen, Wei He, and Yu-xing Mao

441

Inclusion Principle for Dynamic Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Xin-yu Ouyang and Xue-bo Chen

449

Lie Triple Derivations for the Parabolic Subalgebras of gl(n, R) . . . . . . . . Jing Zhao, Hailing Li, and Lijing Fang

457

Non-contact Icing Detection on Helicopter and Experiments Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jie Zhang, Lingyan Li, Wei Chen, and Hong Zhang

465

Research on Decision-Making Simulation of “Gambler’s Fallacy” and “Hot Hand” . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jianbiao Li, Chaoyang Li, Sai Xu, and Xue Ren

474

An Integration Process Model of Enterprise Information System Families Based on System of Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yingbo Wu, Xu Wang, and Yun Lin

479

XVIII


Special Session on Data Fusion and Swarm Intelligence A Linear Multisensor PHD Filter Using the Measurement Dimension Extension Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Weifeng Liu and Chenglin Wen

486

An Improved Particle Swarm Optimization for Uncertain Information Fusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Peiyi Zhu, Benlian Xu, and Baoguo Xu

494

Three-Primary-Color Pheromone for Track Initiation . . . . . . . . . . . . . . . . . Benlian Xu, Qinglan Chen, and Jihong Zhu Visual Tracking of Multiple Targets by Multi-Bernoulli Filtering of Background Subtracted Image Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Reza Hoseinnezhad, Ba-Ngu Vo, and Truong Nguyen Vu

502

509

Mobile Robotics in a Random Finite Set Framework . . . . . . . . . . . . . . . . . . John Mullane, Ba-Ngu Vo, Martin Adams, and Ba-Tuong Vo

519

IMM Algorithm for a 3D High Maneuvering Target Tracking . . . . . . . . . . Dong-liang Peng and Yu Gu

529

A New Method Based on Ant Colony Optimization for the Probability Hypothesis Density Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jihong Zhu, Benlian Xu, Fei Wang, and Qiquan Wang

537

Special Session on Fish School Search - Foundations and Application A Hybrid Algorithm Based on Fish School Search and Particle Swarm Optimization for Dynamic Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . George M. Cavalcanti-J´ unior, Carmelo J.A. Bastos-Filho, Fernando B. Lima-Neto, and Rodrigo M.C.S. Castro Feeding the Fish – Weight Update Strategies for the Fish School Search Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Andreas Janecek and Ying Tan Density as the Segregation Mechanism in Fish School Search for Multimodal Optimization Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Salom˜ ao Sampaio Madeiro, Fernando Buarque de Lima-Neto, Carmelo José Albanez Bastos-Filho, and Elliackin Messias do Nascimento Figueiredo

543

553

563

Mining Coherent Biclusters with Fish School Search . . . . . . . . . . . . . . . . . . Lara Menezes and André L.V. Coelho

573

Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

583

Table of Contents – Part I

Theoretical Analysis of Swarm Intelligence Algorithms Particle Swarm Optimization: A Powerful Family of Stochastic Optimizers. Analysis, Design and Application to Inverse Modelling . . . . . Juan Luis Fern´ andez-Mart´ınez, Esperanza Garc´ıa-Gonzalo, Saras Saraswathi, Robert Jernigan, and Andrzej Kloczkowski

1

Building Computational Models of Swarms from Simulated Positional Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Graciano Dieck Kattas and Michael Small

9

Robustness and Stagnation of a Swarm in a Cooperative Object Recognition Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . David King and Philip Breedon

19

Enforced Mutation to Enhancing the Capability of Particle Swarm Optimization Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . PenChen Chou and JenLian Chen

28

Normalized Population Diversity in Particle Swarm Optimization . . . . . . Shi Cheng and Yuhui Shi

38

Particle Swarm Optimization with Disagreements . . . . . . . . . . . . . . . . . . . . Andrei Lihu and S ¸ tefan Holban

46

PSOslope: A Stand-Alone Windows Application for Graphical Analysis of Slope Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Walter Chen and Powen Chen

56

A Review of the Application of Swarm Intelligence Algorithms to 2D Cutting and Packing Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yanxin Xu, Gen Ke Yang, Jie Bai, and Changchun Pan

64

Particle Swarm Optimization Inertia Weight Adaption in Particle Swarm Optimization Algorithm . . . . Zheng Zhou and Yuhui Shi Nonlinear Inertia Weight Variation for Dynamic Adaptation in Particle Swarm Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Wudai Liao, Junyan Wang, and Jiangfeng Wang

71

80

XX


An Adaptive Tribe-Particle Swarm Optimization . . . . . . . . . . . . . . . . . . . . . Yong Duan Song, Lu Zhang, and Peng Han

86

A Novel Hybrid Binary PSO Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Muhammd Ilyas Menhas, MinRui Fei, Ling Wang, and Xiping Fu

93

PSO Algorithm with Chaos and Gene Density Mutation for Solving Nonlinear Zero-One Integer Programming Problems . . . . . . . . . . . . . . . . . . Yuelin Gao, Fanfan Lei, Huirong Li, and Jimin Li A New Binary PSO with Velocity Control . . . . . . . . . . . . . . . . . . . . . . . . . . . Laura Lanzarini, Javier L´ opez, Juan Andrés Maulini, and Armando De Giusti

101

111

Adaptive Particle Swarm Optimization Algorithm for Dynamic Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Iman Rezazadeh, Mohammad Reza Meybodi, and Ahmad Naebi

120

An Improved Particle Swarm Optimization with an Adaptive Updating Mechanism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jie Qi and Yongsheng Ding

130

Mortal Particles: Particle Swarm Optimization with Life Span . . . . . . . . . Yong-wei Zhang, Lei Wang, and Qi-di Wu

138

Applications of PSO Algorithms PSO Based Pseudo Dynamic Method for Automated Test Case Generation Using Interpreter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Surender Singh Dahiya, Jitender Kumar Chhabra, and Shakti Kumar Reactive Power Optimization Based on Particle Swarm Optimization Algorithm in 10kV Distribution Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chao Wang, Gang Yao, Xin Wang, Yihui Zheng, Lidan Zhou, Qingshan Xu, and Xinyuan Liang

147

157

Clustering-Based Particle Swarm Optimization for Electrical Impedance Imaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Gang Hu, Min-you Chen, Wei He, and Jin-qian Zhai

165

A PSO- Based Robust Optimization Approach for Supply Chain Collaboration with Demand Uncertain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yutian Jia, Xingquan Zuo, and Jianping Wu

172

A Multi-valued Discrete Particle Swarm Optimization for the Evacuation Vehicle Routing Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Marina Yusoff, Junaidah Ariffin, and Azlinah Mohamed

182


A NichePSO Algorithm Based Method for Process Window Selection . . . Wenqi Li, Yiming Qiu, Lei Wang, and Qidi Wu

XXI

194

Efficient WiFi-Based Indoor Localization Using Particle Swarm Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Girma S. Tewolde and Jaerock Kwon

203

Using PSO Algorithm for Simple LSB Substitution Based Steganography Scheme in DCT Transformation Domain . . . . . . . . . . . . . . Feno Heriniaina Rabevohitra and Jun Sang

212

Numerical Integration Method Based on Particle Swarm Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Leila Djerou, Naceur Khelil, and Mohamed Batouche

221

Identification of VSD System Parameters with Particle Swarm Optimization Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yiming Qiu, Wenqi Li, Dongsheng Yang, Lei Wang, and Qidi Wu

227

PSO-Based Emergency Evacuation Simulation . . . . . . . . . . . . . . . . . . . . . . . Jialiang Kou, Shengwu Xiong, Hongbing Liu, Xinlu Zong, Shuzhen Wan, Yi Liu, Hui Li, and Pengfei Duan

234

Training Spiking Neurons by Means of Particle Swarm Optimization . . . Roberto A. V´ azquez and Beatriz A. Garro

242

Ant Colony Optimization Algorithms Clustering Aggregation for Improving Ant Based Clustering . . . . . . . . . . . Akil Elkamel, Mariem Gzara, and Hanêne Ben-Abdallah Multi-cellular-ant Algorithm for Large Scale Capacity Vehicle Route Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jie Li, Yi Chai, Penghua Li, and Hongpeng Yin Ant Colony Optimization for Global White Matter Fiber Tracking . . . . . Yuanjing Feng and Zhejin Wang

250

260 267

Bee Colony Algorithms An Efficient Bee Behavior-Based Multi-function Routing Algorithm for Network-on-Chip . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Junhui Wang, Huaxi Gu, Yintang Yang, and Zhi Deng

277

Artificial Bee Colony Based Mapping for Application Specific Network-on-Chip Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Zhi Deng, Huaxi Gu, Haizhou Feng, and Baojian Shu

285

XXII


Using Artificial Bee Colony to Solve Stochastic Resource Constrained Project Scheduling Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Amin Tahooneh and Koorush Ziarati

293

Novel Swarm-Based Optimization Algorithms Brain Storm Optimization Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yuhui Shi

303

Human Group Optimizer with Local Search . . . . . . . . . . . . . . . . . . . . . . . . . Chaohua Dai, Weirong Chen, Lili Ran, Yi Zhang, and Yu Du

310

Average-Inertia Weighted Cat Swarm Optimization . . . . . . . . . . . . . . . . . . Maysam Orouskhani, Mohammad Mansouri, and Mohammad Teshnehlab

321

Standby Redundancy Optimization with Type-2 Fuzzy Lifetimes . . . . . . . Yanju Chen and Ying Liu

329

Oriented Search Algorithm for Function Optimization . . . . . . . . . . . . . . . . Xuexia Zhang and Weirong Chen

338

Evolution of Cooperation under Social Norms in Non-Structured Populations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Qi Xiaowei, Ren Guang, Yue Gin, and Zhang Aiping Collaborative Optimization under a Control Framework for ATSP . . . . . . Jie Bai, Jun Zhu, Gen-Ke Yang, and Chang-Chun Pan Bio-Inspired Dynamic Composition and Reconfiguration of Service-Oriented Internetware Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Huan Zhou, Zili Zhang, Yuheng Wu, and Tao Qian A Novel Search Interval Forecasting Optimization Algorithm . . . . . . . . . . Yang Lou, Junli Li, Yuhui Shi, and Linpeng Jin

347 355

364 374

Artificial Immune System A Danger Theory Inspired Learning Model and Its Application to Spam Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yuanchun Zhu and Ying Tan

382

Research of Hybrid Biogeography Based Optimization and Clonal Selection Algorithm for Numerical Optimization . . . . . . . . . . . . . . . . . . . . . Zheng Qu and Hongwei Mo

390

The Hybrid Algorithm of Biogeography Based Optimization and Clone Selection for Sensors Selection of Aircraft . . . . . . . . . . . . . . . . . . . . . . . . . . . Lifang Xu, Shouda Jiang, and Hongwei Mo

400


A Modified Artificial Immune Network for Feature Extracting . . . . . . . . . Hong Ge and XueMing Yan

XXIII

408

Differential Evolution Novel Binary Encoding Differential Evolution Algorithm . . . . . . . . . . . . . . Changshou Deng, Bingyan Zhao, Yanling Yang, Hu Peng, and Qiming Wei

416

Adaptive Learning Differential Evolution for Numeric Optimization . . . . Yi Liu, Shengwu Xiong, Hui Li, and Shuzhen Wan

424

Differential Evolution with Improved Mutation Strategy . . . . . . . . . . . . . . Shuzhen Wan, Shengwu Xiong, Jialiang Kou, and Yi Liu

431

Gaussian Particle Swarm Optimization with Differential Evolution Mutation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chunqiu Wan, Jun Wang, Geng Yang, and Xing Zhang

439

Neural Networks Evolving Neural Networks: A Comparison between Differential Evolution and Particle Swarm Optimization . . . . . . . . . . . . . . . . . . . . . . . . . Beatriz A. Garro, Humberto Sossa, and Roberto A. V´ azquez

447

Identification of Hindmarsh-Rose Neuron Networks Using GEO metaheuristic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lihe Wang, Genke Yang, and Lam Fat Yeung

455

Delay-Dependent Stability Criterion for Neural Networks of Neutral-Type with Interval Time-Varying Delays and Nonlinear Perturbations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Guoquan Liu, Simon X. Yang, and Wei Fu Application of Generalized Chebyshev Neural Network in Air Quality Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Fengjun Li

464

472

Financial Time Series Forecast Using Neural Network Ensembles . . . . . . . Anupam Tarsauliya, Rahul Kala, Ritu Tiwari, and Anupam Shukla

480

Selection of Software Reliability Model Based on BP Neural Network . . . Yingbo Wu and Xu Wang

489

Genetic Algorithms Atavistic Strategy for Genetic Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dongmei Lin, Xiaodong Li, and Dong Wang

497

XXIV


An Improved Co-evolution Genetic Algorithm for Combinatorial Optimization Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Nan Li and Yi Luo

506

Recursive Structure Element Decomposition Using Migration Fitness Scaling Genetic Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yudong Zhang and Lenan Wu

514

A Shadow Price Guided Genetic Algorithm for Energy Aware Task Scheduling on Cloud Computers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Gang Shen and Yan-Qing Zhang

522

A Solution to Bipartite Drawing Problem Using Genetic Algorithm . . . . Salabat Khan, Mohsin Bilal, Muhammad Sharif, and Farrukh Aslam Khan

530

Evolutionary Computation Evaluation of Two-Stage Ensemble Evolutionary Algorithm for Numerical Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yu Wang, Bin Li, Kaibo Zhang, and Zhen He

539

A Novel Genetic Programming Algorithm For Designing Morphological Image Analysis Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jun Wang and Ying Tan

549

Fuzzy Methods Optimizing Single-Source Capacitated FLP in Fuzzy Decision Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Liwei Zhang, Yankui Liu, and Xiaoqing Wang

559

New Results on a Fuzzy Granular Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . Xu-Qing Tang and Kun Zhang

568

Fuzzy Integral Based Data Fusion for Protein Function Prediction . . . . . Yinan Lu, Yan Zhao, Xiaoni Liu, and Yong Quan

578

Hybrid Algorithms Gene Clustering Using Particle Swarm Optimizer Based Memetic Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Zhen Ji, Wenmin Liu, and Zexuan Zhu Hybrid Particle Swarm Optimization with Biased Mutation Applied to Load Flow Computation in Electrical Power Systems . . . . . . . . . . . . . . . . . Camila Paes Salomon, Maurilio Pereira Coutinho, Germano Lambert-Torres, and Cl´ audio Ferreira

587

595


Simulation of Routing in Nano-Manipulation for Creating Pattern with Atomic Force Microscopy Using Hybrid GA and PSO-AS Algorithms . . . Ahmad Naebi, Moharam Habibnejad Korayem, Farhoud Hoseinpour, Sureswaran Ramadass, and Mojtaba Hoseinzadeh Neural Fuzzy Forecasting of the China Yuan to US Dollar Exchange Rate—A Swarm Intelligence Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chunshien Li, Chuan Wei Lin, and Hongming Huang

XXV

606

616

A Hybrid Model for Credit Evaluation Problem . . . . . . . . . . . . . . . . . . . . . . Hui Fu and Xiaoyong Liu

626

Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

635

Multi-Objective Optimization for Dynamic Single-Machine Scheduling Li Nie, Liang Gao*, Peigen Li, and Xiaojuan Wang The State Key Laboratory of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan, Hubei 430074, People’s Republic of China [email protected]

Abstract. In this paper, a multi-objective evolutionary algorithm based on gene expression programming (MOGEP) is proposed to construct scheduling rules (SRs) for dynamic single-machine scheduling problem (DSMSP) with job release dates. In MOGEP a fitness assignment scheme, diversity maintaining strategy and elitist strategy are incorporated on the basis of original GEP. Results of simulation experiments show that the MOGEP can construct effective SRs which contribute to optimizing multiple scheduling measures simultaneously. Keywords: multi-objective optimization; gene expression programming; dynamic scheduling; single-machine.

1 Introduction Production scheduling problem is one of the most important tasks carried out in manufacturing systems and has received considerable attention in operations research literature. In this area, it is usually assumed that all the jobs to be processed are available at the beginning of the whole planning horizon. However, in many real situations, jobs may arrive over time due to transportation etc. There are many approaches have been proposed to solve production scheduling problem, such as branch and bound [1], genetic algorithms [2], tabu search [3] etc. However, these methods usually offer good quality solutions with the cost of a huge amount of computational time. Furthermore, these techniques are not applicable in dynamic or uncertain conditions because it is needed to frequently modify the original schedules to respond to the changes of system status. Scheduling with scheduling rules (SRs) that defines only the next state of the system is highly effective in such dynamic environment [4]. Due to inherent complexity and variability of scheduling problem, a considerable effort is needed to develop suitable SRs for the problem at hand. Many researchers have investigated the use of genetic programming (GP) to create problem specific SRs [4][5][6]. In our previous work, we have applied gene expression programming (GEP), a new evolutionary algorithm, on dynamic single-machine scheduling problem (DSMSP) with job release dates and demonstrated that GEP is more promising than GP to create efficient SRs [7]. All the *

Corresponding author.

Y. Tan et al. (Eds.): ICSI 2011, Part II, LNCS 6729, pp. 1–9, 2011. © Springer-Verlag Berlin Heidelberg 2011

2

L. Nie et al.

work mentioned above has only concentrated on single-objective optimization. Several objectives usually must be considered simultaneously in the real-world production situation and these objectives often conflict with each other. It is not possible to have a single solution which simultaneously optimizes all objectives. To tradeoff between these objectives is necessary, which makes the multi-objective optimization problems (MOPs) more difficult than the single-objective optimization problems. There have been many multi-objective optimization evolutionary algorithm (MOEA) proposed [8][9][10][11][12][13]. However, they can not be employed to construct SRs for DSMSP. In this paper, we propose a multi-objective evolutionary algorithm based on gene expression programming (MOGEP) and apply it on optimizing of several objectives simultaneously for DSMSP. In MOGEP, (1) a fitness assignment scheme which combines Pareto-dominating relation and density information is proposed to guide the search to approximate the Pareto optimal solutions; (2) a diversity maintain strategy is used to adjust non-dominated set of each generation in order to keep the diversity of the non-dominated set; (3) an elitist strategy is used to guarantee the convergence of search. The remainder of the paper is organized as follows. In Section 2, DSMSP with job release date is described. In section 3, the fundamental concepts of multi-objective optimization are stated briefly. In Section 4, MOGEP and its application on DSMSP are elaborated. In Section 5, the experiments are presented. The final conclusions are given in Section 6.

2 Problem Description The DSMSP with job release dates is described as following: The shop floor consists of one machine and n jobs, which are released over time and are processed once on the machine without preemption. The attributes of a job, such as processing time, release date, due date, are unknown in advance till the job is currently available at the machine or arrive in the immediate future. It is assumed that the machine is available all the time and cannot process more than one job simultaneously. The task of scheduling is to determine a sequence of jobs on the machine in order to minimize several optimization criteria simultaneously, in our case, makespan, mean flow time, maximum lateness and mean tardiness. The four performance criteria are defined below.

F1 = Cmax = max(ci , i = 1,..., n) .

F2 = F =

1 n ∑ (ci − ri ) . n i =1

F3 = Lmax = max(ci − d i , i = 1,..., n) .

F4 = T =

1 n ∑ max(ci − di , 0) . n i =1

(1) (2) (3) (4)

Where, ci, ri and di denote the completion time, release date and due date of job i, respectively. n denotes the number of jobs. Cmax, F , Lmax and T denote makespan, mean flow time, maximum lateness and mean tardiness, respectively.

Multi-Objective Optimization for Dynamic Single-Machine Scheduling

3

3 Basic Concepts of Multi-Objective Optimization In the section, we shortly describe several basic concepts of multi-objective optimization and Pareto-optimality those are intensively used in the literature [14]. The multi-objective optimization problem is generally formulated as follow:

minimize X ∈Ω

F ( X ) = ( F1 ( X ), F2 ( X ),..., FL ( X )) .

(5)

Where X is a possible solution, Ω is the feasible solution space, F (i ) is the objective function and Fr (i ) is the rth objective function (for 1 ≤ r ≤ L ). A solution a dominates a solution b (or b is dominated by a), if the following conditions are satisfied:

Fi (a ) ≤ Fi (b), ∀i ∈{1, 2,..., L} .

(6)

Fi (a ) < Fi (b), ∃i ∈{1, 2,..., L} .

(7)

A solution a is indifferent to solution b, if a can not dominate b and b can not dominate a. A solution is called non-dominated solution, if it is not dominated by any other one. Pareto-optimal set is constituted of non-dominated solutions. Pareto-optimal frontier is allocations in the objective space corresponding to the Pareto-optimal set. The goal of multi-objective optimization is to find or approximate the Pareto-optimal set. It is usually not possible to have a single solution which simultaneously optimizes all objectives, therefore, an algorithm that gives a large number of alternative solutions lying on or near the Pareto-optimal front is of great practical value.

4 MOGEP for DSMSP GEP is a new technique of creating computer programs based on principle of evolution, firstly proposed by Ferreira [15]. GEP has been applied in different field, e.g. functions discovery [16], classification rules discovery [17], time series prediction [18], digital elevation model generation [19], and it shows powerful ability to solve complex problems. However, original GEP can only optimize one objective. If several objectives are demanded to be optimized concurrently, some extra steps should be designed specially. Based on original GEP, MOGEP is equipped with a fitness assignment scheme, diversity maintaining strategy and elitist strategy. 4.1 Flow of MOGEP MOGEP is executed in the following steps: Step 1: Iteration counter iter = 0. An initial population Pt which consists of N individuals are randomly generated and an empty external archive NDSet, whose size is M (M dis(θa , θb ), then remove θa and insert η. Else if minθ∈N P ∧θ=θb dis(η, θ) > dis(θa , θb ), then remove θb and insert η. Else choose a closet z ∈ N P to η; if minθ∈N P ∧θ=z dis(η, θ) > dis(η, z), then remove z and insert η. 6. Else discard η.

1. 2. 3. 4. 5.

A New Multi-Objective Particle Swarm Optimization Algorithm Table 1. Parameter setting in the algorithms, where M = total number of equipment

m

i=1

63

B C xA i + xi + xi is the

|N P |max kmax p c1 c2 wmax w min w1max w1min MOPSO-A max( M/10, 50) M/2 m/10 0.8 1.0 0.9 0.1 0.9 0.1 MOPSO-B max( M/10, 50) M/2 m/10 0.8 1.0 0.9 0.1 MOTS max( M/10, 50) M Algorithm

Table 2. Computational experiments conducted on the test problem instances m

M

MOPSO-A t |N P |

4

I

∗

MOPSO-B C

∗

t |N P |

I

∗

MOTS C

∗

t |N P |

I∗

C∗

50

300

0.1

5 30.6 11.4

0.1

5 30.6 11.4

0.1

5 30.6 11.4

50

900

0.2

9 75.2 35.7

0.2

9 75.2 35.7

0.1

9 75.2 35.7

100 1800

0.7

13 24.3 89.0

0.7

13 24.3 89.0

0.6

13 24.3 89.0

100 9750

2.4

28 45.9 12.6

2.7

26 40.1 12.6

2.2

30 45.9 14.3

200 3800

2.6

18 14.9 12.1

2.7

17 14.6 12.1

2.6

18 14.9 12.1

200 17200

21.8

36 92.5 42.2

26.1

39 79.0 42.2

42.2

40 73.6 42.2

300 4900

22.2

20 80.4 89.6

26.6

23 72.1 93.2

40.5

23 72.1 87.5

300 23600

63.6

39 54.2 44.7

60.5

33 52.3 43.8 181.7

37 53.9 46.7

500 9500

72.1

27 36.7 25.0

74.1

29 36.3 25.2 208.0

30 36.9 28.1

500 45800 292.4

46 50.0 12.5 329.2

50 48.7 12.8 1480.9

48 49.9 12.7

800 13650 283.3

36 34.9 22.2 280.7

36 34.9 22.2 1675.8

35 29.6 22.0

800 64000 2791.2

49 75.3 48.8 3039.4

50 70.1 48.8

1000 15000 2846.6

38 29.4 14.2 3022.3

38 27.3 14.2

1000 82000 6558.9

50 69.3 60.0

Computational Experiments

The presented MOPSO algorithm (denoted by MOPSO-A) has been tested on a set of SEMP problem instances and compared with two other algorithms: – A basic MOPSO algorithm (denoted by MOPSO-B) where the gbest is randomly selected from N P . – A multi-objective tabu search algorithm (denoted by MOTS) proposed in [19].

64

H. Ling et al.

The experiments are conducted on a computer of 2 × 2.66GHz AMD Athlon64 X2 processor and 8GB memory. The basic parameter values are given in Table 1. The performance measures include the CPU time t (in seconds), the number of non-dominated solutions |N P |, the result maximum mission capability I ∗ and minimum maintenance cost C ∗ . In order to improve the clarity of comparison, all the values of I ∗ and C ∗ are scaled into the range (0, 100). The summary of experimental results are presented in Table 2 (the maximum running time on every instance is 2 hours, and that “ ” denotes the algorithm fails to stop within the time). As we can see from the computational results, for small-size problem instances where m ≤ 100 and M ≤ 1800, all the three algorithms reach the same Paretooptimal front; but with the increasing of instance size, two PSO algorithms exhibit significant performance advantage over the tabu search algorithm; for large-size problems, MOPSO-A also exhibits certain performance advantage over MOPSO-B. On the other hand, the result I ∗ obtained by MOPSO-A is always no less than that obtained by MOPSO-B, and C ∗ obtained by MOPSO-A is always no more than that obtained by MOPSO-B except for one case (which is italicized in Table 2). This demonstrate that our strategy from global best selection plays an important role for improving the quality of result solutions.

5

Conclusion

The paper presents an effective multi-objective particle swarm optimization (PSO) algorithm for solving the SEML problem model. Our algorithm employs an objective leverage function for global best selection and preserves the diversity of non-dominated solutions based on the measurement of minimum pairwise distance, and thus decreases the computational cost and improve the quality of result solution set. As demonstrated by the experimental results, the proposed algorithm are quite efficient even for large-size problem instances. We are now extending the algorithm by introducing non-dominated sorting method [11] which will increase the computational cost but can evolve the swarm more close to the true Pareto front, and thus is more appropriate for medium-size problem instances. Further research will also include the fuzziness of maintenance costs and mission capability to decrease the sensitivity of the model and improve the adaptivity of the algorithm.

References 1. Ai, B., Wu, C.: Genetic and simulated annealing algorithm and its application toequipment maintenace resource optimization. Fire Control & Command Control 35(1), 144–145 (2010) 2. Clerc, M.: Particle Swarm Optimization. ISTE, London (2006) 3. Coello, C.A.C., Lechuga, M.S.: MOPSO: A proposal for multiple objective particle swarm optimization. In: Proceedings of Congress on Evolutionary Computation, vol. 2, pp. 1051–1056. IEEE Press, Los Alamitos (2002)

A New Multi-Objective Particle Swarm Optimization Algorithm

65

4. Fletcher, J.D., Johnston, R.: Effectiveness and cost benefits of computer-based decision aids for equipment maintenance. Comput. Human Behav. 18, 717–728 (2002) 5. Hajek, J., Szollos, A., Sistek, J.: A new mechanism for maintaining diversity of Pareto archive in multi-objective optimization. Adv. Eng. Softw. 41, 1031–1057 (2010) 6. Ho, S.-J., Ku, W.-Y., Jou, J.-W., Hung, M.-H., Ho, S.-Y.: Intelligent particle swarm optimization in multi-objective problems. In: Ng, W.-K., Kitsuregawa, M., Li, J., Chang, K. (eds.) PAKDD 2006. LNCS (LNAI), vol. 3918, pp. 790–800. Springer, Heidelberg (2006) 7. Jayakumar, A., Asgarpoor, S.: Maintenance optimization of equipment by linear programming. Prob. Engineer. Inform. Sci. 20, 183–193 (2006) 8. Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of IEEE International Conference on Neural Networks, Perth WA, Australia, pp. 1942–1948 (1995) 9. Kleeman, M.P., Lamont, G.B.: Solving the aircraft engine maintenance scheduling problem using a multi-objective evolutionary algorithm. In: Coello, C.C., Aguirre, A.H., Zitzler, E. (eds.) EMO 2005. LNCS, vol. 3410, pp. 782–796. Springer, Heidelberg (2005) 10. Laskari, E.C., Parsopoulos, K.E., Vrahatis, M.N.: Particle swarm optimization for integer programming. In: Proceedings of Congress on Evolutionary Computing, pp. 1582–1587. IEEE Press, Los Alamitos (2002) 11. Li, X.: A non-dominated sorting particle swarm optimizer for multiobjective optimization. In: Cant´ u-Paz, E., Foster, J.A., Deb, K., Davis, L., Roy, R., O’Reilly, U.-M., Beyer, H.-G., Kendall, G., Wilson, S.W., Harman, M., Wegener, J., Dasgupta, D., Potter, M.A., Schultz, A., Dowsland, K.A., Jonoska, N., Miller, J., Standish, R.K. (eds.) GECCO 2003. LNCS, vol. 2723, pp. 37–48. Springer, Heidelberg (2003) 12. Liu, D., Tan, K., Goh, C., Ho, W.: A multiobjective memetic algorithm based on particle swarm optimization. IEEE Trans. Syst. Man. Cybern. B 37, 42–50 (2007) 13. Parsopoulos, K.E., Vrahatis, M.N.: Particle dwarm optimization method in multiobjective problems. In: Proceedings of the 2002 ACM Symposium on Applied Computing, pp. 603–607. ACM Press, New York (2002) 14. Verma, A.K., Ramesh, P.G.: Multi-objective initial preventive maintenance scheduling for large engineering plants. Int. J. Reliability Quality & Safety Engineering 14, 241–250 (2007) 15. Xu, L., Han, J., Xiao, J.: A combinational forecasting model for aircraft equipment maintenance cost. Fire Control & Command Control 33, 102–105 (2008) 16. Yang, Y., Huang, X.: Genetic algorithms based the optimizing theory and approaches to the distribution of the maintenance cost of weapon system. Math. Prac. Theory 24, 74–84 (2002) 17. Yu, G., Li, P., He, Z., Sun, Y.: Advanced evolutionary algorithm used in multiobjective constrained optimization problem. Comput. Integ. Manufact. Sys. 15, 1172–1178 (2009) 18. Zhang, Z., Wang, J., Duan, X., et al.: Introduction to Equipment Technical Support. Military Science Press, Beijing (2001) 19. Zheng, Y., Zhang, Z.: Multi-objective optimization model and algorithm for equipment maintenance palnning. Comput. Inter. Manufact. Sys. 16, 2174–2180 (2010)

Multiobjective Optimization for Nurse Scheduling Peng-Yeng Yin*, Chih-Chiang Chao, and Ya-Tzu Chiang Department of Information Management, National Chi-Nan University Nantou 54561, Taiwan [email protected]

Abstract. It is laborious to determine nurse scheduling using human-involved manner in order to account for administrative operations, business benefits, and nurse requests. To solve this problem, a mathematical formulation is proposed where the hospital administrators can set multiple objectives and stipulate a set of scheduling constraints. We then present a multiobjective optimization method based on the cyber swarm algorithm (CSA) to solve the nurse scheduling problem. The proposed method incorporates salient features from particle swarm optimization, adaptive memory programming, and scatter search to create benefit from synergy. Two simulation problems are used to evaluate the performance of the proposed method. The experimental results manifest that the proposed method outperforms NSGA II and MOPSO in terms of convergence and diversity performance measures of the produced results. Keywords: cyber swarm algorithm, adaptive memory programming, scatter search, multiobjective optimization, nurse scheduling.

1 Introduction Nurse scheduling, which is among many other types of staff scheduling, intends to automatically allot working shifts to available nurses in order to maximize hospital value/benefit subject to relevant constraints including governmental regulations, nurse skill requirement, minimal on-duty hours, etc. There are several solution methods proposed in the last decade for dealing with the nurse scheduling problem. These methods can be divided into three categories: mathematical programming, heuristics, and metaheuristics. Most of the methods aimed to solve a single-objective formulation, only few of them [1-4] addressed a more complete description of real-world hospital administration and attempted multiobjective formulation of nurse scheduling. Nevertheless, due to the high complexity of multiobjective context, the authors of [1-3] converted the multiobjective formulation into a single-objective program by the weighting-sum technique. The weighting-sum technique fails to identify optimal solutions if the Pareto front is non-convex and the value of the weights used to combine multiple objectives is hard to determine. This paper proposes a cyber swarm algorithm (CSA) for the Multi-Objective Nurse Scheduling Problem (MONSP). The CSA is a new metaheuristic approach which marries the major features of particle swarm optimization (PSO) and scatter search. *



Multiobjective Optimization for Nurse Scheduling

67

The CSA has been shown to be more effective than several state-of-the-art methods for single-objective optimization [5]. The contribution of this paper includes the following. (1) We devise a multiobjective version for the CSA. The proposed method, named MOCSA, is general and can be employed to solve many classes of problems with multiobjective context; (2) we show the effectiveness of MOCSA in tackling the generic multiobjective nurse scheduling problem. The non-dominated solutions obtained by MOCSA are superior to those produced by other competing methods in terms of the dominance strength and the diversity measure on the solution front; and (3) the multi-dimensional asymptotic Pareto front is shown in the objective space to illustrate the comparative performances of competing methods. The remainder of this paper is organized as follows. Section 2 presents a literature review of existing methods for the nurse scheduling problem and introduces the central concepts of multiobjective optimization. Section 3 describes the problem formally and articulates the proposed method. Section 4 presents experimental results together with an analysis of their implications. Finally, concluding remarks and discussions are given in Section 5.

2 Related Works To assist various operations performed in a hospital, a work day is normally divided into two to four shifts (for example, a three-shift day may include day, night, and late shift). Each nurse is allocated to a number of shifts during the scheduling period with a set of constraints. A shift is fulfilled by a specified number of nurses with different medical skills depending on the operations to be performed in the shift. The adherent constraints with nurse scheduling are necessary hospital regulations when taking into account the wage cost, execution of operations, nurses’ requests, etc. The constraints can be classified as hard constraints and soft constraints. Hard constraints should be strictly satisfied and a schedule violating any hard constraints will not be acceptable. Soft constraints are desired to be satisfied as much as possible and a schedule violating soft constraints is still considered feasible. The objective could involve the reduction of the human resource cost, satisfaction of nurses’ request, or minimization of violations to any soft constraints. Most existing works seek to optimize one objective, only few consider multiple objectives when search for solutions. Berrada et al. [1] proposed the first attempt to find a nurse schedule optimizing several soft constraints simultaneously. The lexico-dominance technique is applied where the priority order of the soft constraints is pre-specified and is used to determine the quality of solutions. Burke et al. [3] applied the weighting sum technique but the weight values are determined by the priority order of objectives obtained after close consultation with hospitals. Burke et al. [4] proposed a simulated annealing multiobjective method which generates the non-dominated solutions to obtain an approximate Pareto front. A widely accepted notion in decision science field for multiobjective optimization is to search the Pareto-optimal solutions which are not dominated by any other solutions. A solution x dominates another solution y, denoted x ; y , if x is strictly better than y in at least one objective and x is no worse than y in the others. The plots of objective values for all Pareto-optimal solutions form a Pareto front in the objective space. It is usually hard to find the true Pareto front due to the high complexity of the problem nature. Alternatively, an approximate Pareto front is searched for. The

68

P.-Y. Yin, C.-C. Chao, and Y.-T. Chiang

quality of this front is evaluated by two measures: (1) The convergence measure indicates how close the approximate front is converging to the true front, and (2) the diversity measure favors the approximate front whose plots are evenly spread on the front. Classical multiobjective optimization methods include lexico-dominance, weighting sum, and goal programming. However, multiple runs of the applied method are needed to obtain a set of non-dominated solutions. Recently, metaheuristic algorithms have been introduced as a viable technique for multiobjective optimization. Notable applications have been proposed using Strength Pareto Evolutionary Algorithm (SPEA II) [6], Non-dominated Sorting Genetic Algorithm (NSGA II) [7], and Multi-Objective Particle Swarm Optimization (MOPSO) [8].

3 Proposed Method This paper deals with the MONSP on a shift-by-shift basis. Each working day is divided to three shifts (day, night, and late shift), and the total shifts in a scheduling period are numbered from 1 to S (1 indicates the day-shift of the first day, 2 indicates the night-shift of the first day, etc). Assume that there are M types of nurse skills, and skill type m is owned by Tm nurses. The aim of the MONSP is to optimize multiple objectives simultaneously by allotting appropriate nurses to the shifts subject to a set of hard constraints. By using the notations introduced in Table 1, we present the mathematical formulation of the addressed MONSP as follows. Table 1. Notations used in the addressed MONSP formulation Lmj Umj Wm Rm Cmj Pmij xmij

Min. number of nurses having skill m required to fulfill shift j Max. number of nurses having skill m required to fulfill shift j Min. number of shifts a nurse having skill m should serve in a scheduling period Max. number of consecutive working days that a nurse having skill m can serve Cost incurred by allotting a nurse having skill m to shift j Pmij = 1 if nurse i having skill m is satisfied with shift j assignment; Pmij = 1 if unsatisfied; and Pmij = 0 if no special preference xmij = 1 if nurse i having skill m is allotted to shift j; otherwise, xmij = 0

－

Minimize

M

Tm

S

∑∑∑ x

f1 =

m =1 i =1 j =1

f2 =

Minimize

f3 =

(1)

(2)

∑ ∑ ∑ x (1 − P )

(3)

S

Tm

∑ ∑ ⎜⎜ ∑ x m =1 j =1

Minimize

C mj ⎞ − L mj ⎟⎟ ⎠

M

⎛

mij

M

Tm

⎝

mij

i =1

S

m =1 i =1 j =1

mij

mij

Subject to S

∑x j =1

mij

≥ W m ∀m, i

(4)

mij

≥ L mj

∀m, j

(5)

Tm

∑x i =1

Multiobjective Optimization for Nurse Scheduling Tm

∑x i =1

≤ U mj

∀m, j

(6)

≤1

r = 1, 4, 7, …, S-2 ∀m, i

(7)

r = 1, 4, 7, …, S-2 ∀m, i

(8)

mij

r+2

∑x

mij

69

j =r

r + 3 ( Rm +1 )−1

∑x

mij

≤ Rm

j=r

x mij ∈ {0, 1}

∀m, i, j

(9)

The first objective (Eq. (1)) intends to minimize the cost incurred by performing the nurse schedule. The second objective (Eq. (2)) tries to minimize the deviation between the minimum number of required nurses for a shift and the number of nurses really allotted to that shift. The third objective originally intends to maximize the total nurses’ preference Pmij about the schedule, it is converted to a minimization objective by using 1 Pmij (Eq. (3)). The first constraint (Eq. (4)) stipulates that the number of shifts fulfilled by a nurse having skill m should be greater than or equal to a minimum threshold Wm. Eq. (5) and Eq. (6) describe that the number of nurses having skill m which are allotted to shift j should be a value between Lmj and Umj. The fourth constraint (Eq. (7)) indicates any nurse can only work for at most one shift during any working day. Finally, the fifth constraint (Eq. (8)) requests that the nurse having skill m can serve for at most Rm consecutive working days.

－

Fig. 1. The conception diagram of the MOCSA

One of the notable PSO variants is the Cyber Swarm Algorithm (CSA) [5] which facilitates the reference set, a notion from scatter search [9], keeping the most influential solutions. To seek the approximate Pareto optimal solutions for the MONSP problem, we propose the multiobjective version of the CSA, named MOCSA. Fig. 1 shows the conception diagram of the MOCSA which consists of four memory components. The swarm memory component is the working memory where a population of swarm particles evolve to improve their solution quality. The individual memory reserves a separate space for each particle and stores the pseudo non-dominated solutions by reference to all the solutions found by this designated particle only. Note that the

70


pseudo non-dominated solutions could be dominated by the solutions found by other particles, but we propose to store the pseudo non-dominated solutions because our preliminary results show that these solutions contain important diversity information along the individual search trajectory and they assist in finding influential solutions that are overlooked by just using global non-dominated solutions. The global memory tallies the non-dominated solutions that are not dominated by any other solutions found by all the particles. The solutions stored in the global memory will be output as the approximate Pareto optimal solutions as the program terminates. Finally, the reference memory taking the notion of reference set from scatter search [9] selects the most influential solutions based on objective values and diversity measures. The MOCSA exploits the guiding information by the manipulations on different types of adaptive memory. The details of the features of MOCSA are presented as follows. Particle Representation and Fitness Evaluation. Given S working shifts to be fulfilled, there are at most 2 S possible allocations (without considering scheduling constraints) for assigning a nurse to available shifts. Hence, a nurse schedule can be encoded as a value between [0, 2 S 1]. Assume a population of U particles is used, where a particle Pi = {pij}, indicating the schedule for all the nurses. The fitness of the ith particle is a four-value vector (f1, f2, f3, f4). The objective values evaluated using Eqs. (1)-(3) are referred to as the first three fitness values (f1, f2, f3). The fourth fitness value f4 serves as a penalty which computes the amount of total violations incurred by any constraint (Eqs. (4)-(8)). We assume that a feasible solution always dominates any infeasible solution.

－

Exploiting guiding information. The CSA extends the learning form using pbest and gbest by additionally including another solution guide which is systematically selected from the reference set, storing a small number of reference solutions, denoted RefSol[m], m = 1, 2, …, RS, observed by all particles by reference to fitness values and solution diversity. For implementing the MOCSA, the selecting of solution guides is more complex because multiple non-dominated solutions can play the role of pbest, gbest and RefSol[m]. Once the three solution guides were selected, particle Pi updates its positional vector in the swarm memory by the guided moving using Eqs. (10) and (11) as follows. ⎛ ⎛ ω1ϕ1 pbestij + ω 2ϕ 2 gbest j + ω3ϕ3 RefSol[m] j ⎞ ⎞ ,1≤m≤RS (10) vijm ← K ⎜⎜ vij + (ϕ1 + ϕ 2 + ϕ 3 )⎜⎜ − pij ⎟⎟ ⎟⎟ ω ϕ + ω ϕ + ω ϕ 1 1 2 2 3 3 ⎝ ⎠⎠ ⎝

Pi←non-dominated

{ ( f (P + v ) 1 ≤ k ≤ 4) k

i

m i

m ∈ [1, RS ]

}

(11)

where K is the constriction factor, ω and ϕ are the weighting value and the cognition coefficient for the three solution guides pbest, gbest and RefSol[m]. As RefSol[m], 1≤m≤RS is selected in turn from the reference set, the process will generate RS candidate particles for replacing Pi. We choose the non-dominated solution from the RS candidate particles. If there exist more than one non-dominated solutions, the tie is broken at random. Nevertheless, all the non-dominated solutions found in the guided moving are used for experience memory update as noted in the following. Experience memory update. As shown in Fig. 1, experience memory consists of individual memory, global memory and reference memory, where the rewarded


71

experience pbest, gbest and RefSol[m] are stored and updated. The individual memory tallies the personal rewarded experience pbest for each individual particle. Because there may exist more than one non-dominated solution in the search course of a particle (here, the non-dominance only refers to all the solutions found by this particle), we save all these solutions in the individual memory. Any solutions in the individual memory can serve as pbest in the guided moving, and we’ll present the Diversity strategy [10] for selecting pbest from the individual memory. By contrast to individual memory, the global memory stores all the non-dominated solutions found by the entire swarm. Hence, the content of the global memory is used for the final output of the approximate Pareto-optimal solutions. During the evolution, the solutions in the global memory are also helpful in assisting the guided moving of particles by serving as gbest. The Sigma strategy [11] is employed in our method for selecting gbest from the global memory. The reference memory stores a small number of reference solutions selected from individual and global memory. According to the original scatter search template [9], we facilitate the 2-tier reference memory update by reference to the fitness values and diversity of the solutions. Selecting solution guides. First, the Diversity strategy for selecting pbest is employed where each particle selects from its individual memory a non-dominated solution as pbest that is the farthest away from the other particles in the objective space. Thus, the particle is likely to produce a plot of objective values equally-distanced to those of other particles, improving the diversity property of the solution front. Second, we apply the Sigma strategy for selecting gbest from the global memory. For a given particle, the Sigma strategy selects from the global memory a non-dominated solution as gbest which is the closest to the line connecting the plot of the particle’s objective values to the origin in the objective space, improving the convergence property of the solution front. Finally, the third solution guide, RefSol[m], m = 1, 2, …, RS, is systematically selected from the reference memory. These reference solutions have good properties of convergence and diversity, so their features should be fully explored in the guided moving for a particle.

4 Result and Discussion We have intensively consulted administrators and senior staffs at the Puli Christian Hospital (http://www.pch.org.tw/english/e_index.html). A dataset consisting of two problem instances was thus created for assessing the objective values of the nurse schedules produced by various algorithms. The first problem instance (Problem I) requires to determine the optimal scheduling of 10 nurses with two levels of skills in a planning period of one week, while the second problem instance (Problem II) consists of 25 nurses with three different skills to be scheduled in a period of four weeks. Among others, NSGA II and MOPSO are two notable methods and are broadly used as performance benchmarks. We thus choose these two methods for performance comparison. All the algorithms were coded using C# language, and the following experiments were conducted on a 2.4GHz PC with 1.25GB RAM. The quality of the final solution front is evaluated in two aspects: the convergence of the produced front to the true Pareto front and the diversity of the produced front manifesting the plots of objective values that are evenly spaced on the front. The convergence measure named Hypervolume calculates the size of the fitness space covered by the produced front.

72


To prevent the bias preferred to a less number of efficient points, the Hypervolume is normalized by the final number of solutions produced. The solutions with a smaller Hypervolume value is more desired because they are closer to the true Pareto front. The diversity measure named Spacing which estimates the variance of the distance between adjacent fitness plots. The solutions with a smaller Spacing value are more desired because these solutions exhibit a better representation of a front. As all the competing algorithms are stochastic, we report the average performance index values over 10 independent runs. Each run of a given algorithm is allowed with a period of duration of 80,000 fitness evaluations. Table 2 lists the values of the performance indexes for the solution fronts produced by the competing algorithms. For Problem I, the MOCSA gives the smallest Hypervolume value indicating the produced solution front converges closer to the true Pareto front than the other two algorithms. The Spacing value for the MOCSA is also the smallest among all which discloses that the non-dominated solutions produced by MOCSA spread more evenly on the front. On the other hand, the NSGA II produces the greatest values (worst performance) for both Hypervolume and Spacing, while the MOPSO generates the intermediate values. The experimental outcome for Problem II is slightly different with the previous case. The NSGA II gives the smallest Hypervolume value (best performance) although its spacing value indicates that the produced solutions are not well distributed on the front. The MOCSA produces the second smallest Hypervolume value and the smallest Spacing value among all competitors, supporting the claim that the MOCSA is superior to the other two algorithms. The MOPSO generates the worst Hypervolume value and a median Spacing value. Fig. 2(a) shows the plots of the multiobjective values of all the solutions for Problem I obtained by different algorithms. It is seen that the front produced by MOCSA is closer to the origin. We can also observe that the spread of the solutions are better Table 2. The values of performance indexes obtained by competing algorithms

MOCSA NSGA II MOPSO

Problem I Hypervolume Spacing 2.42E+07 1.41 9.45E+07 2.37 6.17E+07 2.01

(a)

Problem II Hypervolume Spacing 9.86E+07 3.40 8.37E+07 7.82 1.35E+08 4.24

(b)

Fig. 2. The multiobjective-valued front for simulation problems


73

distributed on the front than those produced by the other two methods. The front generated by the MOPSO is next to that produced by the MOCSA by reference to the visual distance to the origin. The front generated by the NSGA II is the farthest to the origin and the obtained solutions are not evenly distributed on the front. For Problem II as shown in Fig. 2(b), we can see the front produced by the NSGA II is the closest to the origin although the obtained solutions are still not evenly distributed on the front. The MOCSA produces the front next to that of NSGA II, but better spacing is observed. Finally, MOPSO front is the furthest to the origin where the distribution of the obtained solutions on the front is also better than that produced by the NSGA II.

5 Conclusions In this paper, we have presented a multiobjective cyber swarm algorithm (MOCSA) for solving the nurse scheduling problem. Based on a literature survey, we propose a mathematical formulation containing three objectives and five hard constraints. In contrast to most existing methods which transform multiple objectives into an integrated one, the proposed MOCSA method tackles the generic multiobjective setting and is able to produce approximate Pareto front. The experimental results on two simulation problems manifest that the MOCSA outperforms NSGA II and MOPSO in terms of convergence and diversity measures of the produced fronts.

References 1. Berrada, I., Ferland, J., Michelon, P.: A multi-objective approach to nurse scheduling with both hard and soft constraints. Socio-Economic Planning Sciences 30, 183–193 (1996) 2. Azaiez, M.N., Al Sharif, S.S.: A 0-1 goal programming model for nurse scheduling. Computers & Operations Research 32, 491–507 (2005) 3. Burke, E.K., Li, J., Qu, R.: A Hybrid Model of Integer Programming and Variable Neighbourhood Search for Highy-Constrained Nurse Rostering Problems. European Journal of Operational Research 203, 484–493 (2010) 4. Burke, E.K., Li, J., Qu, R.: A Pareto-based search methodology for multi-objective nurse scheduling. Annals of Operations Research (2010) 5. Yin, P.Y., Glover, F., Laguna, M., Zhu, J.X.: Cyber swarm algorithms – improving particle swarm optimization using adaptive memory strategies. European Journal of Operational Research 201, 377–389 (2010) 6. Zitzler, E., Laumanns, M., Thiele, L.: SPEA2: Improving the strength pareto evolutionary algorithm. Technical Report 103, ETH, Switzerland (2001) 7. Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Transaction on Evolutionary Computation 6, 42–50 (2002) 8. Coello Coello, A.C., Pulido, G.T., Lechuga, M.S.: Handling multiple objectives with particle swarm optimization. IEEE Trans. on Evolutionary Computation 8, 256–279 (2004) 9. Laguna, M., Marti, R.: Scatter Search: Methodology and Implementation in C. Kluwer Academic Publishers, London (2003) 10. Branke, J., Mostaghim, S.: About selecting the personal best in multi-objective particle swarm optimization. In: Runarsson, T.P., Beyer, H.-G., Burke, E.K., Merelo-Guervós, J.J., Whitley, L.D., Yao, X. (eds.) PPSN 2006. LNCS, vol. 4193, pp. 523–532. Springer, Heidelberg (2006) 11. Mostaghim, S., Teich, J.: Strategies for finding local guides in multi-objective particle swarm optimization (MOPSO). In: Proceedings of the IEEE Swarm Intelligence Symposium 2003 (SIS 2003), Indianapolis, Indiana, USA, pp. 26–33 (2003)

A Multi-Objective Binary Harmony Search Algorithm Ling Wang, Yunfei Mao, Qun Niu, and Minrui Fei Shanghai Key Laboratory of Power Station Automation Technology, School of Mechatronics and Automation, Shanghai University, Shanghai, 200072 [email protected]

Abstract. Harmony Search (HS) is an emerging meta-heuristic optimization method and has been used to tackle various optimization problems successfully. However, the research of multi-objectives HS just begins and no work on binary multi-objectives HS has been reported. This paper presents a multi-objective binary harmony search algorithm (MBHS) for tackling binary-coded multiobjective optimization problems. A modified pitch adjustment operator is used to improve the search ability of MBHS. In addition, the non-dominated sorting based crowding distance is adopted to evaluate the solution and update the harmony memory to maintain the diversity of algorithm. Finally the performance of the proposed MBHS was compared with NSGA-II on multi-objective benchmark functions. The experimental results show that MBHS outperform NSGA-II in terms of the convergence metric and the diversity metric. Keywords: binary harmony search, multi-objective optimization, harmony search.

1 Introduction Harmony Search (HS) is an emerging global optimization algorithm developed by Geem in 2001 [1]. Owing to its excellent characteristics, HS has drawn more and more attention and dozens of variants have been proposed to improve the optimization ability. On the one hand, the control parameters of HS were investigated and several adaptive strategies were proposed to achieve better performance. Pan et al [2] proposed a self-adaptive global best harmony search algorithm in which the harmony memory consideration rate and pitch adjustment rate were dynamically adapted by the learning mechanisms. Wang and Huang [3]presented a self-adaptive harmony search algorithm which used the consciousness to automatically adjust parameter values. On the other hand, various hybrid harmony search algorithms were proposed where additional information extracted by other algorithms was combined with HS to improve the optimization performance. For instances, Li and Li [4]combined HS with the real valued Genetic Algorithm to enhance the exploitation capability. Several hybrid HS algorithms combined with Particle Swarm Optimization (PSO) were developed to optimize the numerical problem [5], pin connected structures [6] and water network design [7]. Other related works include the fusion of HS with Simplex Algorithm [8] or Clonal Selection Algorithm [9]. Y. Tan et al. (Eds.): ICSI 2011, Part II, LNCS 6729, pp. 74–81, 2011. © Springer-Verlag Berlin Heidelberg 2011

A Multi-Objective Binary Harmony Search Algorithm

75

Now HSs have been successfully applied in a wide range of optimization problems in the scientific and engineering fields. However, most of these works focused on the single-objective optimization problems in the continuous or discrete space; and so far just several researches are concerned with the binary-coded problems or multiobjective optimization problems. On binary-coded optimization problems, Geem [10] firstly used HS to solve the water pump switching problem where the candidate value for each decision variable is “0” or “1”. Then Greblicki and Kotowski [11] analyzed the properties of HS on the one dimensional binary knapsack problem and the optimization performance of HS is unsatisfactory. Afterwards, Wang et al[12] pointed out that the pitch adjustment rule of HS cannot performs its function for binary-coded problems which is the root of the poor performance. To make up for it, Wang proposed a binary HS algorithm in which a new pitch adjustment operator was developed to ameliorate the optimization ability. On the multi-objective optimization problems, Geem and Hwangbo [13] studied the satellite heat pipe design problem which need simultaneously consider two objectives, i.e., the thermal conductance and the heat pipe mass. However, the authors transformed this multi-objective problem into a single objective function by minimizing the sum the individual error between current function value and optimal value. And Geem [14] later used HS to tackle the multi-objective time-cost optimization problem for scheduling a project. In this work, the dominance-based comparison for selection was adopted to achieve the trade-off of the time and cost. As far as we know, there is no work reported on the multi-objective binary HS (MBHS). To extend HS to tackle the multi-objective binary-coded problems, a new Pareto-based multi-objective binary HS is proposed in the work. This paper is organized as follow. Section 2 briefly introduces the standard HS algorithm. Then the proposed MBHS is described in Section 3 in details. Section 4 presents the experimental results of MBHS on the benchmark functions and the comparisons with NSGA-II are also given. Finally, some conclusions are drawn in Section 5.

2 Harmony Search Algorithm Harmony Search Algorithm is inspired by the improvising process of musicians. HS mimics this process by keeping a matrix of the best solution vectors named the Harmony Memory (HM). The number of vectors that can be simultaneously remembered in the HM called the Harmony Memory Size (HMS). These memory vectors are initialized with HMS solutions randomly generated for each decision variable. The search procedure after initialization is called improvisation which includes three operators, i.e., harmony memory considering operator, pitch adjusting operator and random selection operator. The harmony memory considering rate (HMCR), which is between 0 and 1, controls the balance between exploration and exploitation during improvisation. A random number is generated and compared with HMCR during search process for each decision variable. If it is smaller than HMCR, the memory vector in HM is taken into consideration for generating the new value; otherwise a value is randomly selected from the possible ranges of the decision variable. Each decision variable of the new solution vector obtained from the HM is examined to determine whether it should be pitch adjusted. The pitch adjusting rate (PAR) decides the ratio of

76

L. Wang et al.

pitch-adjustment. Another random number between 0 and 1 is generated and the pitch-adjustment operating as Eq. (1) is executed if it is not bigger than PAR. new ⎪⎧ xi + rand ().BW in continuous space xi new = ⎨ new in discrete space ⎪⎩ xn

(1)

Here xi new is the i-th element of the new harmony solution vector; rand() is the random number; BW is an arbitrary distance band width; and xn new is a neighboring value of xi new . If the new harmony vector is better than the worst solution vector in HM in terms of the fitness value, it will be included in the HM and the existing worst harmony solution vector is excluded from HM. This process runs iteratively till the terminated rules are satisfied.

3 Multi-Objective Binary Harmony Search Algorithm The standard multi-objective HS can be used to deal with binary-coded multiobjective optimization problems, but the disfunction of the pitch adjustment operator in binary space will spoil the performance greatly. So the multi-objective binary harmony search algorithm (MBHS) is proposed in this paper to achieve the satisfactory optimization ability. In MBHS, the harmony vector is formed by the binary-string. For a N-dimension problem, the HM with the size of HMS can be represented as Eq. (2) and initialized randomly,

x HM

⎡ x1,1 ⎢ ⎢ x2,1 ⎢... =⎢ ⎢... ⎢x ⎢ HMS −1,1 ⎢⎣ xHMS ,1

x1,2

...

x1, N −1

x2,2

...

x2, N −1

... ... ... ... xHMS −1,2 ...

... ... xHMS −1, N −1

xHMS ,2

xHMS , N −1

...

⎤ ⎥ x2, N ⎥ ⎥ ... ⎥ ... ⎥ xHMS −1, N ⎥ ⎥ xHMS , N ⎥⎦

x1, N

(2)

where xi , j ∈ {0,1} is the j-th element of i-th harmony memory vector. Like the standard HS, MBHS also uses three updating operators, that is, harmony memory consideration operator, pitch adjustment operator and random selection to generate the new solutions. 3.1 Harmony Memory Consideration Operator and Radom Selection Operator

In MBHS, harmony memory consideration operator (HMCO) and random selection operator (RSO) are used to perform the global search. MBHS performs HMCO with the probability of HMCR, i.e., picking a value in HM; while it runs RSO with the rate of (1-HMCR), i.e., choosing a feasible value not limited to HM, which means that the bit is re-initialized stochastically to be “0” or “1”. The process of HMCO and RSO can be described as Eq. (3-4)


⎧⎪ xk , j , k ∈ {1, 2,....HMS} r1 ≤ HMCR xj = ⎨ otherwise ⎪⎩ RSO ⎧1 RSO = ⎨ ⎩0

r2 ≤ 0.5 otherwise

77

(3)

(4)

where x j is the j-th bit of the new harmony solution vector; r1 and r2 are two independent random number between 0 and 1. 3.2 Pitch Adjustment Operator

If the element of the new harmony comes from the HM, it need to be adjusted by pitch adjustment operator (PAO) with the probability PAR. However, in binary space, the value of the each element in HM is bound to be “0” or “1”, so the standard definition of PAO in HS will be degraded to mutation operation [12]. If we simply abandon the PAO, the algorithm will lack the operator to perform local search. To remedy it, the pitch adjustment operator as Eq. (5) is used in MBHS. ⎧⎪ B j xj = ⎨ ⎪⎩ x j

r ≤ PAR otherwise

(5)

where r is a random number; B j is the j-th element of the best harmony solution vector in HM. The PAO executes a local search based on the current solution and the optimal solution which will help MBHS find the global optima effectively and efficiently. 3.3 Updating of HM

The new generated harmony vector is added into the HM. Then all the solutions in HM are sorted according to the fitness values and the solution with the worst fitness is removed from HM. In the multi-objective optimization problems, the two major goals of Pareto-based optimizer are to pursue the convergence to the Pareto-optimal set as well as maintain the diversity. To achieve it, the non-domination sort strategy based on crowding distance is adopted to sort the HM vectors.

4 Result and Discussion Following the previous work, five multi-objective optimization functions, i.e., SCH*[15], FON[16] and Deb*[17], are chosen as benchmark problems. 4.1 Performance Measures

In this work, the convergence metric γ and the diversity metric Δ proposed in [18] are adopted to evaluate the performance.

78

L. Wang et al.

(1) convergence metric γ The convergence metric γ is used to measure the closeness of solutions in obtained Pareto-optimal set to true Pareto-optimal set and it is calculated as Eq.(6): | p*|

di = min j =1

⎛ f m (hi ) − f m ( p j ) ⎞ ⎟ f m max − f m min ⎠ m =1 ⎝ k

∑⎜

(6)

|H |

γ =

∑d i =1

i

(7)

|H |

where p* = ( p1 , p2 ,...... p| p* | ) is the true Pareto-optimal set, H = ( h1 , h2 ,......h| A| ) is the obtained Pareto-optimal set, f m max is the maximum of the m-th objective function and f m min is the minimum of the m-th objective function. In this work, a set of | p* |= 400 uniformly distributed Pareto-optimal solutions is used to calculate the convergence metric γ . (2) diversity metric Δ The diversity metric is computed as Eq. (8): Δ=

d f + dl +

HMS −1

∑ i =1

−

di − d

(8)

d f + dl + ( HMS − 1)

where di is the distance between two successive solutions in the obtained Paretooptimal set; d is the mean value of all the di ; d f and d l are the two Euclidean distances between the extreme solutions and the boundary solutions of the obtained non-dominated set. 4.2 Result and Discussion

For MBHS, a reasonable set of parameter values are adopted, i.e., HMCR=0.9, and PAR=0.03; each decision variable are coded with 30 bits. For a comparison, NSGA-II [18] with the default parameters is used to solve these problems as well. MBHS and NSGA-II both ran with 50000 function evaluations. Table 1-2 list the optimization results of MBHS and NSGA-II and box plots of γ and Δ are given in Fig.1 and Fig.2. According to the results in Table 1-2, it is reasonable to claim that the proposed MBHS is superior to the NSGA-II. Fig.1 indicated that MBHS generally achieved solutions with higher quality in comparison with NSGA-II in terms of convergence metric. And in Fig.2, the comparison of the diversity metric indicated that MBHS is able to find a better spread of solutions and obviously outperforms NSGA-II in all problems.


-3

x 10

-3

FON

x 10

DEB2

DEB1

1.25

6 4

Convergence

1.2

Convergence

Convergence

8

0.2

1.15 1.1 1.05

0.15 0.1 0.05

1

2

0

0.95 MBHS

MBHS

NSGA-II -4

x 10

-4

SCH1

x 10

NSGA-II

SCH2

8

Convergence

10.5

Convergence

MBHS

NSGA-II

10 9.5

7.5

7

9 6.5 MBHS

MBHS

NSGA-II

NSGA-II

Fig. 1. Box plot of the convergence metrics γ obtained by MBHS and NSGA-II

FON 0.65

0.4

0.9

0.6

Diversity

0.6

Diversity

Diversity

DEB2

DEB1

0.8

0.55

0.8 0.7 0.6

0.2

0.5 0.5 MBHS

MBHS

NSGA-II

NSGA-II

SCH2

SCH1 1.1

0.5

Diversity

0.4

Diversity

MBHS

NSGA-II

0.3

1.05

1

0.2 0.95

0.1 MBHS

NSGA-II

Fig. 2. Box plot of the diversity metrics

MBHS

NSGA-II

△ obtained by MBHS and NSGA-II

79

80

L. Wang et al. Table 1. Mean and Variance of the convergence metric γ MBHS Mean

FON SCH1 SCH2 DEB1 DEB2

NSGA-II Variance

Mean

Variance

1.9534481E-003

2.5725898E-003

1.9009196E-003

1.9787263E-004

9.7508949E-004

5.9029912E-005

9.7769396E-004

6.9622480E-005

7.3687049E-004

5.6615711E-005

7.4402367E-004

5.2879053E-005

1.0286786E-003

5.8010990E-005

1.0697121E-003

6.6791139E-005

8.3743810E-003

1.5211841E-002

9.8603419E-002

1.0217030E-001

Table 2. Mean and Variance of the diversity metric Δ NSGA-II

MBHS Mean FON SCH1

9.6845154E-002 1.1668542E-001

Variance

Mean

Variance

6.2345711E-002

7.8416829E-001

2.9294262E-002

1.0841259E-002

4.2701519E-001

3.5264364E-002

SCH2

9.4714113E-001

1.6775193E-003

1.0347253E+000

2.7413411E-002

DEB1

4.7516338E-001

5.5063477E-003

6.3378683E-001

1.9689019E-002

DEB2

6.6037039E-001

1.8871529E-001

6.8131960E-001

1.1085579E-001

5 Conclusion This paper presented a new multi-objective binary harmony search algorithm for tackling the multi-objective optimization problems in binary space. A modified pitch adjustment operator is used to perform a local search and improve the search ability of algorithm. In addition, the non-dominated sorting based on crowding distance is adopted to evaluate the solution and update the HM which insures a better diversity performance as well as convergence of MBHS. Finally the performance of the proposed MBHS was compared with NSGA-II on five well-known multi-objective benchmark functions. The experimental results show that MBHS outperforms NSGAII in terms of the convergence metric and the diversity metric.

Acknowledge This work is supported by Research Fund for the Doctoral Program of Higher Education of China (20103108120008), the Projects of Shanghai Science and Technology Community (10ZR1411800 & 08160512100), ChenGuang Plan (2008CG48), Mechatronics Engineering Innovation Group project from Shanghai Education Commission, Shanghai University “11th Five-Year Plan” 211 Construction Project and the Graduate Innovation Fund of Shanghai University.


81

References 1. Geem, Z., Kim, J., Loganathan, J.: A new heuristic optimization algorithm: harmony search. J. Simulations 76, 60–68 (2001) 2. Pan, Q., Suganthan, P., Tasgetiren, M., Liang, J.: A self-adaptive global best harmony search algorithm for continuous optimization problems. Applied Mathematics and Computation 216, 830–848 (2010) 3. Wang, C., Huang, Y.: Self-adaptive harmony search algorithm for optimization. Expert Systems with Applications 37, 2826–2837 (2010) 4. Li, H., Li, L.: A novel hybrid real-valued genetic algorithm for optimization problems. In: International Conference on Computational Intelligence and Security, pp. 91–95 (2008) 5. Omran, M., Mahdavi, M.: Global-best harmony search. Applied Mathematics and Computation 198, 643–656 (2008) 6. Li, L., Huang, Z., Liu, F., Wu, Q.: A heuristic particle swarm optimizer for optimization of pin connected structures. Computers & Structures 85, 340–349 (2007) 7. Geem, Z.: Particle-swarm harmony search for water network design. Engineering Optimization 41, 297–311 (2009) 8. Jang, W., Kang, H., Lee, B.: Hybrid simplex-harmony search method for optimization problems. In: IEEE Congress on Evolutionary Computation, pp. 4157–4164 (2008) 9. Wang, X., Gao, X.Z., Ovaska, S.J.: A hybrid optimization method for fuzzy classification systems. In: 8th International Conference on Hybrid Intelligent Systems, pp. 264–271 (2008) 10. Geem, Z.: Harmony search in water pump switching problem. In: Wang, L., Chen, K., S. Ong, Y. (eds.) ICNC 2005. LNCS, vol. 3612, pp. 751–760. Springer, Heidelberg (2005) 11. Greblicki, J., Kotowski, J.: Analysis of the Properties of the Harmony Search Algorithm Carried Out on the One Dimensional Binary Knapsack Problem. In: Moreno-Díaz, R., Pichler, F., Quesada-Arencibia, A. (eds.) EUROCAST 2009. LNCS, vol. 5717, pp. 697– 704. Springer, Heidelberg (2009) 12. Wang, L., Xu, Y., Mao, Y., Fei, M.: A Discrete Harmony Search Algorithm. Communications in Computer and Information Science 98, 37–43 (2010) 13. Geem, Z., Hwangbo, H.: Application of harmony search to multi-objective optimization for satellite heat pipe design. Citeseer, pp. 1–3 (2006) 14. Geem, Z.: Multiobjective Optimization of Time Cost Trade off Using Harmony Search. Journal of Construction Engineering and Management 136, 711–716 (2010) 15. Schaffer, J.: Multiple objective optimization with vector evaluated genetic algorithms. In: Proceedings of the 1st International Conference on Genetic Algorithms, pp. 93–100 (1985) 16. Fonseca, C., Fleming, P.: Multiobjective optimization and multiple constraint handling with evolutionary algorithms. II. Application example. IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and Humans 28, 38–47 (2002) 17. Deb, K.: Multi-objective genetic algorithms: Problem difficulties and construction of test problems. Evolutionary Computation 7, 205–230 (1999) 18. Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multi-objective genetic algorithm: NSGA-II. IEEE Trans. on Evolutionary Computation 6, 182–197 (2002)

A Self-organized Approach to Collaborative Handling of Multi-robot Systems Tian-yun Huang1,2, Xue-bo Chen2, Wang-bao Xu1,2, and Wei Wang1 1

Faculty of Electronic Information and Electrical Engineering, Dalian University of Technology, 116024 Liaoning, China 2 School of Electronics and Information Engineering, Liaoning University of Science and Technology, 114051 Liaoning, China [email protected], [email protected], [email protected], [email protected]

Abstract. The purpose of this paper is to develop a general self-organized approach to multi-robot’s collaborative handling problem. Firstly, an autonomous motion planning graph (AMP-graph) is described for individual movement representations. An individual autonomous motion rule (IAM-rule) based on “free-loose” and “well-distributed load-bearing” preferences is presented. By establishing the simple and effective individual rule model, an ideal handling formation can be formed by each robot moving autonomously under their respective preferences. Finally, the simulations show that both the AMP-graph and the IAMrule are valid and feasible. On this basis, the self-organized approach to collaborative hunting and handling with obstacle avoidance of multi-robot systems can be further analyzed effectively. Keywords: Self-organized, Collaborative handling, Formation control.

1 Introduction Collaborative handling, as one of tasks of multi-robot systems, plays an important role in the research on collaborative control of complex system. It begins with the research on ‘two industrial robots handling a single object’ of Zheng, Y.F. and J.Y.S. Luh [1], continues in the work of Y Kume [2] as ‘multiple robots’, and receives maturity in recent times in the work of a motion-planning method of multiple mobile robots in a three-dimensional environment. (see, for example, [3] ). In the Early Stage of research most of the classic approaches to collaborative handling are centralized control which may be effective only when the number of controllers is usually limited within a certain range [4][5][6]. Decentralized control is an effective method by which each robot is controlled by its own controller without explicit communication among robots, and the method usually employ the leader-following relational mode by assigning to a leader who obtain the motion information of the object [2][7]. However, it may not be the best choices because of explicit relational mode and the bottleneck in communication and computing of leader robot. Self-organized approach is Y. Tan et al. (Eds.): ICSI 2011, Part II, LNCS 6729, pp. 82–90, 2011. © Springer-Verlag Berlin Heidelberg 2011

A Self-organized Approach to Collaborative Handling of Multi-robot Systems

83

a good train of thought for collaborative handling of multi-robot systems, even swarm systems [8][9][10]. The main objective of this paper is to initiate a study on selforganized approach to multi-robot’s collaborative handling problem. For individual movement representations, an autonomous motion planning graph (AMP-graph) is described. An individual autonomous motion rule (IAM-rule) including two kind of “free-loose” and “well-distributed load-bearing” preferences is presented. By establishing the simple and effective individual rule model, an ideal handling formation can be formed by each robot moving autonomously under their respective preferences. The simulations show that both the AMP-graph and the IAM-rule are valid and feasible. Considering many uncertain factors in the handling process, before continuing any further we will make three necessary assumptions: First, the handling process happen in the ideal plane. Second, the rim of object exist any solid handling points which hardly produce deformation. Lastly, handling robots with strong bearing capacity don’t sideslip and deflect in the handling process. Based on these assumptions, a selforganized approach will be design.

2 Autonomous Motion Planning Model Definition 1. Based on local sensing, each robot can complete collaborative handling task only through their own simple rules, we call it multi-robot self-organized handling (MSH). Based on Definition 1, we will make three assumptions [11], which are the base of autonomous motion planning model of handling robots: Assumption 1. Each handling robot can obtain location information of object. Assumption 2. Each handling robot has a local sensor, by which the robot can obtain the position information of a finite number of its neighboring robots. Assumption 3. There are always some simple rules, by which each robot can autonomously move under the respective preferences to form an ideal handling formation. Next, the details of autonomous motion planning model will be described. Definition 2. In the absolute coordinates XaOaYa, the robot Ri can obtain four location information denoted by T0=(xt0,yt0), Ri=(xi,yi), Rpi=(xpi,ypi), Rqi=(xqi,yqi), which are of any target point T0 within the object, the robot Ri and two neighboring robots Rpi, Rqi of the robot Ri. A target constraint line clti is the position vector from the robot Ri to the target T0, denoted by clti=(xt0-xi)+i(yt0-yi). A target constraint angle θ ti is the angle from X-axis to clti, denoted by θ ti =arctan((yt0-yi)/(xt0-xi)). Two interconnected constraint lines clpi and clqi is the position vector from the robot Ri to its neighboring robot Rpi and Rqi, denoted by clpi=(xpi-xi)+i(ypi-yi), clqi=(xqi-xi)+i(yqi-yi). Two interconnected constraint angles θ pi and θ qi is the deflection angle from X-axis to clpi and clqi, denoted by θ pi =arctan((ypi-yi)/(xpi-xi)), θ qi =arctan((yqi-yi)/(xqi-xi)). And two interconnected deflection angle θpti and θqti are the angle from clti to clpi and clqi, denoted by θ pti = θ pi - θ ti , θ qti = θ qi - θ ti . The desired linear velocity vei is decomposed into a vertical component vtdi in the direction of clti and a horizontal component vtpi in the direction perpendicular to clti. The desired deflection angle θ eti is the angle from vtdi to vei, denoted by θ eti = θ ti - θ ei .

84

T.-y. Huang et al.

T0 clti

vei vtdi

Rpi clpi

θ ei

Rqi θ ti θ pi vtpi clqi θ tpi θ qi Ri

Fig. 1. The AMP-graph

Then, an autonomous motion planning graph (AMP-graph) is formed by location information of any target point T0 and two neighboring robots Rpi, Rqi, shown in Fig. 1.

3 Individual Autonomous Motion Rule Model From the above discussion, we note that the key of MSH problem is how to design some simple rules by which each robot can autonomously determine the direction of motion at every moment, and by which all the handling robots can be distributed evenly to various points around the edge of the target within a finite time. There are two parts in the moving process: (1) aimed at collision avoidance when the robots move from initial points to target; (2) aimed at well-distributed load-bearing when all the robots reach the edge of the target. Considering the parameters in definition 2 and two different motion processes, an individual autonomous motion rule (IAM-rule) with the “freeloose” and “well-distributed load-bearing” preferences will be designed. 3.1 The IAM-Rule Based on the “Free-Loose” Preference Given in Definition 2, the desired linear velocity vei is a vector sum of vtdi in the direction of clti and vtpi in the direction perpendicular to clti. For the sake of simplicity, only the target T0, two neighbors Rpi, Rqi of the ith robot Ri are taken into account in the IAM-rule based on the “free-loose” preference. Ensure that vtpi points to the “freeloose” space while vtdi always point in the direction of cl0i and two constraint conditions vtdi= fd(|clti|,|clpi|,|clqi|) and vtpi=fp(|clpi|,|clqi|) are satisfied, where fd, fp are the vertical and horizontal potential functions. In the process of moving to the target, the rule will make all the robots coordinate into the ideal formation and all the robots tend towards scatter each other and towards gather relative to the target, therefore we call it the IAM-rule based on the “free-loose” preference. The “free-loose” space modeling. Let us consider the robot Ri in the relative coordinates in which Y-axis always point to the target T0, the first and forth quadrants are defined as the positive quadrants since θ pti , θ qti are positive within them and the second and forth quadrants are defined as the negative quadrants since θ tpi , l

negative within them, then the “free-loose” space can be described:

θ qti are


85

The direction of the “free-loose” space points to 1) the opposite direction of the space two neighbors belong to, when two neighbors lay together in the positive or negative quadrant. 2) the direction of the space the interconnected constraint line with greater X-axis component belong to, when two neighbors lay respectively in the positive or negative quadrant. Thus, the description can be expressed mathematically as follows:

⎧⎪Cli = cl pi sin θ pti + clqi sin θ qti ⎨ ⎪⎩Cli = cl pi sgn(θ pti ) + clqi sgn(θ qti ) l ⎧θ tpi = θ ti ⎪ sgn(θ pti )+ sgn(θ qti ) ⎨ π l 2 ⎪θ tpi = θ ti + (−1) ⋅ sgn(Cli ) ⋅ 2 ⎩

clti ≠ 0 clti = 0

(3.1)

Cli ≤ ε Cli > ε

(3.2)

where ε is a permissible error. Because Cli covers all of the information to determine autonomous motion of Ri, we call it interconnected characteristics parameter with “free-loose” feature. Cli denotes the vector sum of the X-axis components of two interconnected direction lines clpi and clqi if the robot Ri has not reach the edge of the target, or the vector sum of clpi and clqi if the robot Ri has reach. Similarly, because

θ tpil covers all the possible direction of “free-loose” space of Ri, therefore we call it

autonomous motion direction angle with “free-loose” feature. Specially, the desired linear velocity vei point in the direction of θ ti if “free-loose” space do not exist, that is, ∃ε , θ tpi = θ ti , if |Cli| ≤ ε . l

We know that the arrow of the Y-axis represents the direction that all the robots tend towards gather relative to the target and the arrow of the X-axis represents the direction that all the robots tend towards scatter each other on the edge of the target. Therefore, the desired angle θ ei at every moment of autonomous motion with the IAM-rule based on the “free-loose” preference can be obtained as follow:

⎧θ ei ⎪ ⎪θ ei ⎨ ⎪θ ei ⎪ * ⎩θ ei

= θ ti

Cli ≤ ε

= θ ti + arctan(vtpi / vtdi ) l = θ tpi

= θ ti

Cli > ε and clti ≠ 0 Cli > ε and clti = 0

(3.3)

Cli ≤ ε and clti = 0

Eq. (3.3) describes every process of multi-robot self-organized handling. According to Definition 2, the desired angle θ ei is the deflection angle of vei and xa if two interconnected constraint lines exist and the robot does not reach the edge of the

86

T.-y. Huang et al.

target, that is, |clti| ≠ 0 and |Cli| > ε . Specially, when two interconnected constraint lines do not exist, that is |Cli|=0, the desired angle θ ei coincides with the target constraint angle θ ti . When the robot reaches the edge of the target and the interconnected constraint lines exist, that is, |clti|=0 and |Cli| > ε , the desired angle

θ ei coincides

with θ . When the robot reaches the edge of the target and the interconnection can be l ti

negligible, that is, |clti|=0 and |Cli| angel θ coinciding with θ ti .

≤ ε , the robot obtain a stable desired

* ei

Now, we turn to the second motion process of multi-robot self-organized handling. 3.2 The IAM-Rule Based on the “Well-Distributed Load-Bearing” Preference After a uniform dispersed formation is formed by autonomous motion with the IAMrule based on the “free-loose” preference, that is, |clti|=0 and |Cli| ≤ ε , all the handling robots smoothly lift up the object together to measure the load bearing data which are used as the parameter of the IAM-rule based on the “well-distributed loadbearing” preference. Similar to the “free-loose” preference, only the load-bearings of the two nearest neighbors at both left and right sides of Ri are taken into account. Ensure that Ri always move along the edge of the object and in the direction of neighbor with larger load-bearing, the IAM-rule will make the load-bearing of all the robots tending towards average, therefore we call it the IAM-rule based on the “welldistributed load-bearing” preference. The “well-distributed load-bearing” space modeling. Similar to the “free-loose” preference, the robot Ri in the relative coordinates in which Y-axis always point to the target T0, the first and forth quadrants are defined as the positive quadrants since θ pti , θ qti are positive in them and the second and forth quadrants are defined as the negative quadrants since θ pti ,

θ qti are negative in them, then the “well-distributed

load-bearing” space can be described: The direction of the “well-distributed load-bearing” space points to the direction of the space the neighbor with larger load-bearing belong to. Corresponds to the "free-loose" preference model, the description can be expressed mathematically as follows:

Cbi = G pi sgn(θ pdi ) + Gqi sgn(θ qdi )

(3.4)

b ⎧θ ei = θ tpi = θ ti Cbi ≤ ε ⎪ π ⎪ b Cbi > ε ⎨θ ei = θ tpi = θ ti + sgn(Cb ) ⋅ 2 ⎪ ⎪Gei* = G0 / n all Cbi ≤ ε ⎩

(3.5)

where Gpi and Gqi are the load-bearing of the two nearest neighbors at both left and right sides of Ri. Because Cbi covers all of the information to determine the direction


87

of autonomous motion of Ri, we call it interconnected characteristics parameter with

“well-distributed load-bearing” feature. Similarly, because θ tpi covers all the possible b

direction of “well-distributed load-bearing” space of Ri, therefore we call it autonomous motion direction angle with “well-distributed load-bearing” feature. Specially, if |Cbi| ≤ ε , i=1,2…n, then all the robots receive the weight equally denoted by G*i=G0/n. 3.3 General Remarks on Multi-robot Self-organized Handling Remark 1. Effective sensing range is the maximum range within which the omnidirection sensor of each handling robot can detect a target, denoted by Rs. If the minimum distance between the robot and the object beyond effective sensing range Rs, the robot follows any given point T0=(xt0,yt0) within the object, or the robot follows the point T0i=(xt0i,yt0i) located nearest from the edge of the object. Remark 2. By setting the parameters of the potential field function, the IAM-rule can maintain collision avoidance between any two robots. When the distance between the robots is smaller, the potential field function make the interconnected deflection angle increasing rapidly to produce greater repulsive interaction. Specially, when the “freeloose” spaces in all directions do not exist, the robot is forced to remain stationary and wait for a chance to autonomous motion. Remark 3. Effective interconnected radius δ is the maximum value within which the interaction between any two robots Rpi, Rqi exists, that is, ∃ δ i , |clpq|=|clpq| if |clpq| ≤ δ i , or |clpq|=0 if |clpq| > δ i ,p ≠ q ∈ {1,2,…,n}.

4 Simulations and Analysis In order to test the validity and feasibility of the IAM-rule based on the “free-loose” preference, two simulations are carried out, in which 8 handling robots are present. The group of robots is required to start from a disordered state and to then form the relatively well-distributed formation around the edge of the object described as an ellipse, where the parametric equations of the ellipse are x=3cos(t)-3, y=1.5sin(t)+1. Each robot obtains the same motion parameters, denoted by r the radius of robot, by Rs effective sensing radius, by δ effective interconnected radius, by ε the minimum distance difference between two robots, by T0 any given point within the object and by λ the step factor. The parameter values of the trajectory control are shown in Table 1 and the initial position information of all the handling robots are shown in Table 2. Table 1. The parameter values of the trajectory control Parameter Value

r 0.2

¤ 0.1

Rs 8

G

O

H

4

0.3

0.5

88

T.-y. Huang et al. Table 2. The initial position information of all the handling robots

X Y

R1 R2 R3 R 4 R5 R6 R7 R8 T0 0 -2.2 1.1 -0.2 3.9 2.6 -0.3 -4.8 -4.0 -0.8 -4.6 -2.8 -1.3 0.2 -1.6 -7.2 -4.9 1.0

Fig. 2. The moving process of 8 handling robots with IAM-rule (36 steps)

From Fig. 2, we observe that after 36 steps all the handling robots distribute uniformly around the edge of the target, so the IAM-rule based on the “free-loose” preference can effectively make multi-robot systems form the ideal handling formation corresponding to formation control [12][13][14]. In the initial period R7 follows the known point T0 within the object, since the object from which the initial position of R7 farther away can not be perceived, coincides with Remark 1. Due to the smaller distance between R1 and R4 in the initial period, R1 and R4 obtain two larger desired deflection angles θ et1 and θ et 4 , coincide with Remark 2. In addition, although the robot R2, R7 and R8 are neighbors each other, the interaction between them are negligible in the initial period because of the greater distances each other, and then the re-establishment of the interaction makes their trajectories deflected during the autonomous motion, coincides with Remark 3. It is to be noted that because each robot is always fond of moving in the direction of “free-loose” space, the robots in the periphery of the group possess more dispersity and make the ones within the group pulled by the “free-loose” space to spread to the periphery gradually, thus the relatively dispersed characteristics for the group are formed finally. If each robot satisfies local collision avoidance conditions under the special circumstances of Remark 2, we might as well call it “strict collision avoidance”.


89

5 Conclusion and Future Work The self-organized approach with IAM-rules has the following advantages over other methods. Firstly, a simple and effective individual autonomous motion rule(IAM-rule) model is established, by which an ideal handling formation can be formed by each robot moving autonomously under their respective preferences. Compared with the centralized control used for multi-robot collaborative handling, the self-organized approach with IAM-rule is simple and has fewer bottlenecks in communication and computing caused by the centralized data processing and leader guiding. For the robot itself, if the information of the target and two neighbors are obtained by local perception, it will determine their own desired speed, thus less information processing are beneficial to making rapid judgment. Secondly, the self-organized approach with IAM-rule has the good characteristics for strict collision avoidance which provides a solution for coordination problem of swarm systems. The IAM-rule can be applied to explanation and resolution of group behaviors since the “free-loose” preference coincides with individual behavior of real system. Thirdly, it may provide a novel train of thought for emergence control modeling, which is verified by the simulation that the system can be controlled to produce certain characteristic and function of emergence by constructing simple individual rules. The paper is the basement of research on emergence of multi-robot collective behavior. Future work include: 1.On this basis, self-organized approach to multi-robot systems’ collaborative hunting and handling with obstacle avoidance can be further analyzed effectively. 2. More rules with certain preferences can be designed to jointly complete more complex function of swarm systems. 3. Based on the IAM-rule, leader emergence can be further discussed. Acknowledgments. Supported by the National Natural Science Foundation of China (Grant No. 60874017).

References 1. Kim, K.I., Zheng, Y.F.: Two Strategies of Position and Force Control for Two Industrial Robots Handling a Single Object. Robotics and Autonomous Systems 5, 395–403 (1989) 2. Kosuge, K., Oosumi, T.: Decentralized Control of Multiple Robots Handling an Object. In: IEEE/ RJS Int.Conf. on Intelligent Robots and Systems, vol. 1, pp. 318–323 (1996) 3. Yamashita, A., Arai, T., et al.: Motion Planning of Multiple Mobile Robots for Cooperative Manipulation and Transportation. IEEE Transactions on Robotics and Automation 19(2) (2003) 4. Koga, M., Kosuge, K., Furuta, K., Nosaki, K.: Coordinated Motion Control of Robot Arms Based on the Virtual International Model. IEEE Transactions on Robotics and Autonomous Systems 8 (1992) 5. Wang, Z., Nakano, E., Matsukawa, T.: Cooperating Multiple Behavior-Based Robots for Object Manipulation. In: IEEE /RSJ/GI International Conference on Intelligent Robots and Systems IROS 1994, vol. 3, pp. 1524–1531 (1994)

90

T.-y. Huang et al.

6. Huang, T.-y., Wang, X.-n., Chen, X.-b.: Multirobot Time-optimal Handling Method Based on Formation Control. Journal of System Simulation 22, 1442–1465 (2010) 7. Kosuge, K., Taguchi, D., Fukuda, T., Sakai, M., Kanitani, K.: Decentralized Coordinated Motion Control of Manipulators with Vision and Force Sensors. In: Proc. of 1995 IEEE Int. Conf. on Robotics and Automation, vol. 3, pp. 2456–24162 (1995) 8. Jadbabaie, A., Lin, J., Morse, A.S.: Coordination of Groups of Mobile Autonomous Agents Using Nearest Neighbor Rules. IEEE Transactions on Automatic Control 48, 988–1001 (2003) 9. Turgut, A.E., Çelikkanat, H., Gökçe, F., Şahin, E.: Self-organized Flocking in Mobile Robot Swarms. Swarm Intelligence 2, 97–120 (2008) 10. Gregoire, G., Tu, H.C.Y.: Moving and Staying Together Without a Leader. Physica D 181, 157–170 (2003) 11. Xu, W.B., Chen, X.B.: Artificial Moment Method for Swarm Robot Formation Control. Science in China Series F: Information Sciences 51(10), 1521–1531 (2008) 12. Balcht, T., Arkin, R.C.: Behavior-based Formation Control for Multi-robot Teams. IEEE Transactions on Robotics and Automation 14, 926–939 (1998) 13. Lawton, J.R., Beard, R.W., Young, B.J.: A Decentralized Approach to Formation Maneuvers. IEEE Transactions on Robotics and Automation 19, 933–941 (2003) 14. Das, A.K., Fierro, R., et al.: A vision-based formation control framework. IEEE Transactions on Robotics and Automation 18, 813–825 (2002)

An Enhanced Formation of Multi-robot Based on A* Algorithm for Data Relay Transmission Zhiguang Xu1, Kyung-Sik Choi1, Yoon-Gu Kim2, Jinung An2, and Suk-Gyu Lee1 1

Department of Electrical Eng. Yeugnam Univ., Gyongsan, Gyongbuk, Korea 2 Daegu Gyeongbuk Institute of Science & Technology, Daegu, Korea [email protected], {robotics,sglee}@ynu.ac.kr, {ryankim9,robot}@dgist.ac.kr

Abstract. This paper presents a formation control method of multi-robot based on A* algorithm for data relay transmission. In our system, we choose Nanotron sensor and compass sensor to execute the tasks of distance measurement, communication and obtaining moving direction. Since there exists data disturbance from Nanotron sensor when there is an obstacle between two robots. Therefore, we embed path planning algorithm information control. The leader robot (LR) knows the whole information of environment, and sends its moving information and corner information as a node to FRs. The FRs regard the node information which received from LR as temporary target to increase the efficiency of multi-robot formation by optimal path. From the simulations and experiments, we will show desirable results of our method. Keywords: multi-robot, formation, path planning, data relay transmission.

1

Introduction

In mobile robotics, robots execute their own tasks in unknown environment by navigation, path planning, communication and etc. Recently, researchers focus on the navigation in multi-robot system to deal with cooperation [1], efficient path planning [2] [3], keeps the stability of navigation [4], and collision avoidance [5]. They have got respectable results through simulations and some experiments. For path planning algorithms, such as Genetic Algorithm, Ant Colony System, A* Algorithm, Neural Network [6]-[9] are very favored by the researchers. Neural network algorithm [9] implements path planning for multiple mobile robots to coordinate with other avoiding moving obstacles. A* algorithm as one of graph search algorithm provides the fastest search of shortest path under the same heuristic. In [3], A* algorithm utilizes function to accelerate searching and reduce computational time. In multi-robot system, the robots are not only required to avoid obstacles, but also need to avoid collision among each other. To solve this problem, [11] adopted a reactive multi-agent solution with decision agent and obstacle agent on a linear configuration. The avoidance decision strategy will be acquired from the timely observations of the decision agents’ organization and calculating the trajectories interacted with other decision agents and obstacle agents. [5] developed a step forward approach for collision avoidance in multiple robot system. They built Y. Tan et al. (Eds.): ICSI 2011, Part II, LNCS 6729, pp. 91–98, 2011. © Springer-Verlag Berlin Heidelberg 2011

92

Z. Xu et al.

techniques from omni-directional vision systems, automatic control, and dynamic programming. This strategy is avoiding static obstacles and dynamic objects from reestablishing positions of each robot. In our system, we assume LR knows whole information of environment. FRs follow LR or forward FRs. We consider that the FRs use Nanotron sensor to obtain distance information. Robots keep in certain distance with each other to avoid collision between each robot. FRs follow the frontal robot in given distance range and plan the path based on the knowledge of nodes which are received from LR. To get shortest trajectory, FRs also apply A* algorithm. The paper is organized as follows. Section 2 derives the mathematical descriptions for embedding path planning algorithm in our system. In section 3, simulation results which coded by Matlab show good performance of our proposed method. In section 4, experiment result validates proposed method.

2 Related Works In multiple robots system, there are three main control approaches, leader-follower based approach [12], behavior-based approach [13] and virtual structure approach [14]. Several control algorithms, such as EKF [15], I/O linearization [16], sliding mode control method [17] are common used to control each robot. In practical, computer process should be loaded if we use the control approaches and control methods above to do multi-robot formation. However, the MCU of our system is AVR, so it is difficult to make our system commercialize and industrialize. In our robots’ system, each robot just utilizes on-board sensors to localization, but redundant sensor data will make a great burden on the aim controller. Consequently, we adopt a more realistic practical application to achieve our control goal which implant path planning algorithm to each robot so as to reduce the computational burden and satisfy our control system requirements. 2.1

System Structure

The system structure for the homogeneous robot is shown in Fig.1. There are two robots for real experiment: one leader robot (LR) and one follow robot (FR). There are two missions for FR. One is to maintain the given distance with LR, the other is to determine the ideal temporary target based on A* algorithm when LR change its moving direction. Generally, mobile robot measures the distance by motor encoders or some kinds of distance measurement sensors while robots explore freely in an experimental environment. We consider a new approach to for team robots navigation based a wireless RF module. The used wireless RF module is a Nanotron sensor node, Ubi-nanoLOC, which is developed by HANBACK Electronics©[18]. The WPAN module is based on IEEE 802.15.4a protocol for high aggregate throughput communications with a precision ranging capability in this system. Since the measured distance from the wireless module may also include considerable error according to ambient environments, the system adopted Kalman filter to reduce the localization error. LR knows the whole information of the experiment environment, and realizes communication between two robots byadhoc routing application among multiple wireless communication modules.

An Enhanced Formation of Multi-robot Based on A* Algorithm

93

Fig. 1. System Structure of the Homogeneous Robot

We utilize a MCU, ATmega128 which adopts 8 bits control system which developed by Atmel©[19], to control robot navigation. We also provide the direct movement by compass sensor XG1010 which developed by Microinfinity©[20].

3 3.1

Algorithm Description State Function and System Flow Chart

The motion of each robot is described in terms of P = (x, y, θ)T, where x, y and θ are the x coordinate, the y coordinate and the bearing respectively. The trajectory of each robot has the form of (x, y) with the velocity v and the angular velocity ω. The model of robot Ri as the form of:

x& (t ) = v(t ) cosθ y& (t ) = v(t ) sin θ θ&(t ) = ω (t )

(1)

Fig. 2 shows flow chart of LR process which knows the whole information of environment. If LR does not reach its destination, LR will send MI to rear robots at each time step, such as moving distance, heading angle and node information. When LR arrives at a corner, it will turn 90 degree and regard the next step position of LR as a node. Fig. 3 describes flow chart of FR maintaining a given distance range. To reduce the steps from start point to goal point and maintain the communication with LR, the FRs use A* algorithm to plan the path, where the FRs make use of the information nodes received from LR.

Fig. 2. Flow chart of LR

94

Z. Xu et al.

Fig. 3. Flow chart of FRs for maintaining a given distance with LR

For the FRs, the node is target which received from the LR. And when LR moves in the environment, there is not only just one node. So the FRs must reach every node as target. However, if two nodes are very close, to increase the efficiency of navigation, the FRs will use A* algorithm to obtain a shortest path and eliminate useless nodes. 3.2

Path Planning

A* algorithm is widely used in graph searching algorithm which includes heuristic function and evaluation function to sort the nodes. This algorithm expressed as f(n) which consists of two functions as follows: g(n) is defined as the cost to go, and h(n) is the cost remaining after having gone, and it is chosen as the Euclidean distance from the other end of the new candidate edge to the destination. The searching process of our system is as follows: (1) Marking the initial node and expanding the unmarked subsequent nodes or the child nodes; (2) Calculating the evaluation function value for each subsequent node, and sorting by evaluation function. Then identifying and marking the node of the minimum evaluation function value; (3) Iterating upper steps to recording the shortest path until the current node same as the goal node [2] [8]. f(n) = g(n) + h(n).

(2)

Fig. 4 is pseudo code of A* in the simulation. In the pseudo code, EXP means horizontal and vertical position on nodes, evolution function, cost function, and heuristic value function, respectively. OPEN and CLOSE sets are to store available path information and disable path information, respectively. To search evolution function with minimum value, there is the comparison step between EXP and OPEN.


1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11.

12.

95

If Have a new node(goal) A*(current position, goal) Closed set = the empty set Open set = includes current position(CP) Store node(start) in OPEN While Open set != empty & path movable Do calculate N’s G, H, F and save in EXP Compare EXP and OPEN IF F is minimum then flag=1 Else flag=0 and Add node(n) in OPEN Update path cost and CLOSE End while

Fig. 4. Pseudo code of A* algorithm

4 4.1

Simulation and Experiment Simulations

We use Matlab 2008b to do the simulation of the multi-robot formation. The simulation describes the moving trajectory of one LR and several FRs. We give the whole map information and path to LR. FRs follow the LR, and planning the path to keep the formation at the same time. The distance range is constrained from 1m to 3m. The environment is separated by cells; one cell size is 1m by 1m. Fig. 5 (a) describes the trajectories of one LR and three FRs which without A* algorithm. The three FRs follow the leader robot by the same trajectory of leader robot. In order to obtain accurate distance measurement data, the LR have to wait the rear FRs until the distance between these two robots is in the minimize value when the LR arrive a corner. It may also reduce the efficiency of formation. However, in Fig. 5 (b), the black circles are denoted as the information nodes sending by leader robot. When the distance measurement data is disturbed, especially the robots go through a corner, LR will send the node information to the robots which moving behind it. Then the FRs use A* algorithm to plan the shortest path to reach its node (as target) within minimize steps. This method not only increases the efficiency of formation but also prevent the effectiveness from data disturbance in Nanotron sensor. Fig. 6 shows the step comparison histogram of one leader robot with different number of follower robots from start point to goal point using A* algorithm or not. If the leader robot just has one follower robot, the step of follower robot using A* algorithm is 39, it is 12 steps less than the follower robot without A* algorithm. If the leader robot has two follower robots using A* algorithm, the step of follower robots is 48, however, if the follower robots do not use A star algorithm, the step is 57. In three robots case, the three robots use 48 steps from start point to goal point using A* algorithm. If without A* algorithm, the three followers use 57 steps. From the comparison, we get the result that the steps using A* algorithm is much less than the steps of follower robots without A* algorithm. The follower robots are able to reach their goal points using A* algorithm more efficiently.

96

Z. Xu et al.

Fig. 5. Trajectories of one LR and three FRs, (a) without A* algorithm, (b) using A* algorithm

Fig. 6. Step comparison histogram of one leader robot with different number of follower robots using A* algorithm or not

4.2 Experiments In the experiment, we embed whole map information to LR, such as the distance between target and corner information. FR navigates autonomously and follows the LR within a given distance to keep required form. When LR arrive at the corner, it will send the corner information to FR for executing A* algorithm. Each robot in the team executes localization using motor encoder and Nanotron sensor data based on Kalman Filter. From some paper, the researchers obtain θ calculated from the relationship between motor encoder and the length of robot’s two wheels. However, heading angle data from compass sensor is more accurate than the data calculated from encoder, we get θ value from XG1010 and let robot go straight using XG1010. Robots are in indoor environment and there is just one corner. The initial distance between LR and FR is 3 meter. When LR robot move 3 meter, it will


97

experiment result:one leader robot and one follower robot

700 leader robot follower robot

600 500

y-axis(cm)

400 300 200 100 0 -100 -100

0

100

200 300 400 x-axis(cm)

500

600

700

Fig. 7. Experiment result about LR and FR moving trajectories

turn left and send the node information to rear FR by Nanotron sensor. At this time, FR plans optimal path to the temporary target based on A* algorithm to keep the required form with LR. We get the robot’s position values and orientation value at each step. And when robots go straight, from its real trajectory, we measure the error in x axis and y axis is less than 1 centimeter. Then we use Matlab draw the trajectories of each robot as Fig. 7.

5

Conclusion

In multiple mobile robots system, it is so important to share the moving information of every robot to increase the efficiency of cooperation. The FRs move to its node (as target) with A* path planning algorithm using the information node which is achieved from LR. Basically, the proposed method obtains a respectable result as we want. The steps of FRs using A* path planning algorithm are much less than the steps of FRs without A* algorithm. The simulation and experiment results show that the robots embedded A* algorithm could obtain better performance on efficiency. For future research, we are going to realize this multi-robot formation control among more number of robots. And we will consider more complex environment, such as exist some obstacles.

Acknowledgment This research was carried out under the General R/D Program of the Daegu Gyeongbuk Institute of Science and Technology (DGIST), funded by the Ministry of Education, Science and Technology (MEST) of the Republic of Korea.

98

Z. Xu et al.

References 1. Farinelli, A., Iocchi, L., Nardi, D.: Multi-robot Systems: A Classification Focused on Coordination. IEEE Transactions on Systems, Man, and Cybernetics, Part-B: Cybernetics 34(5), 2015–2028 (2004) 2. Wang, K.H.C., Botea, A.: Tractable Multi-Agent Path Planning on Grid Maps. In: Int. Joint Conf. on Artificial Intelligence, pp. 1870–1875 (2009) 3. Seo, W.J., Ok, W.J., Ahn, J.H., Kang, S., Moom, B.: An Efficient Hardware Architeture of the A-star Algorithm for the Shortest Path Search Engine. In: Fifth Int. Joint Conf. INC, IMS and IDC, pp. 1499–1502 (2009) 4. Scrapper, C., Madhavan, R., Balakirsky, S.: Stable Navigation Solutions for Robots in Complex Environments. In: Proc. IEEE Int. Workshop on Safety, Security and Rescue Robotics (2007) 5. Cai, C., Yang, C., Zhu, Q., Liang, Y.: Collision Avoidance in Multi-Robot Systems. In: Proc. IEEE Int. Conf. on Mechatronics and Automation, pp. 2795–2800 (2007) 6. Castillo, O., Trujillo, L., Melin, P.: Multiple objective optimization genetic algorithms for path planning in autonomous mobile robots. Int. Journal of Computers, Systems and Signals 6(1), 48–63 (2005) 7. Li, W., Zhang, W.: Path Planning of UAVs Swarm using Ant Colony System. In: Fifth Int. Conf. on Natural Computation, vol. 5, pp. 288–292 (2009) 8. Yao, J., Lin, C., Xie, X., Wang, A.J., Hung, C.C.: Path planning for virtual human motion using improved a star algorithm. In: Seventh Int. Conf. on Information Technology, pp. 1154–1158 (2010) 9. Li, H., Yang, S.X., Biletskiy, Y.: Neural Network Based Path Planning for A Multi-Robot System with Moving Obstacles. In: Fourth IEEE Conf. on Automation Science and Engineering (2008) 10. Otte, M.W., Richardson, S.G., Mulligan, J., Grudic, G.: Local Path Planning in Image Space for Autonomous Robot Navigation in Unstructured Environments. Technical Report CU-CS-1030-07, University of Colorado at Boulder (2007) 11. Sibo, Y., Gechter, F., Koukam, A.: Application of Reactive Multi-agent System to Vehicle Collision Avoidance. In: Twentieth IEEE Int. Conf. on Tools with Artificial Intelligence, pp. 197–204 (2008) 12. Consolini, L., Morbidi, F., Prattichizzo, D., Tosques, D.: A Geometric Characterization of Leader-Follower Formation Control. In: IEEE International Conf. on Robotics and Automation, pp. 2397–2402 (2007) 13. Balch, T., Arkin, R.C.: Behavior-based Formation Control for Multi-robot Teams. IEEE Trans. on Robotics and Automation 14, 926–939 (1998) 14. Lalish, E., Morgansen, K.A., Tsukamaki, T.: Formation Tracking Control using Virtual Structures and Deconfliction. In: Proc. of the 2006 IEEE Conf. on Decision and Control (2006) 15. Schneider, F.E., Wildermuth, D.: Using an Extended Kalman Filter for Relative Localisation in a Moving Robot Formation. In: Fourth Int. Workshop on Robot Motion and Control, pp. 85–90 (2004) 16. Desai, J.P., Ostrowski, J., Kumar, R.V.: Modeling formation of multiple mobile robots. In: Proc. of the 1998 IEEE Int. Conf. on Robotics and Automation, Leuven, Belgium (1998) 17. Sánchez, J., Fierro, R.: Sliding Mode Control for Robot Formations. In: Proc. of the 2003 IEEE Int. Symposium on Intelligent Control, Houston, Texas (2003) 18. Hanback Electronics, http://www.hanback.co.kr/ 19. Atmel Corporation, http://www.atmel.com/ 20. MicroInfinity, http://www.minfinity.com/

WPAN Communication Distance Expansion Method Based on Multi-robot Cooperation Navigation Yoon-Gu Kim1, Jinung An1, Kyoung-Dong Kim2, Zhi-Guang Xu2, and Suk-Gyu Lee2 1

Daegu Gyeongbuk Institute of Science and Technology, 50-1, Sang-ri, Hyeonpung-myeon, Dalseong-gun, Daegu, Republic of Korea 2 Department of Electrical Engineering, Yeungnam University, 214-1, Dae-dong, Gyongsan, Gyongbuk, Republic of Korea {ryankim9,robot}@dgist.ac.kr, [email protected], [email protected], [email protected]

Abstract. Over the past decade, an increasing number of researches and developments for personal or professional service robots are attracting considerable attention and interest in industry and academia. Furthermore, the development of intelligent robots is strongly promoted as a strategic industry. To date, most of the practical and commercial service robots are controlled remotely. The most important technical issue of remote control is wireless communication, especially in indoor and unstructured environments where communication infrastructure may be hampered. Therefore, we propose a multi-robot cooperation navigation method for securing the communication distance extension of the remote control based on wireless personal area networks (WPANs). The concept and implementation of following navigation are introduced, and performance verification is carried out through navigation experiments in real or test-bed environments. Keywords: WPAN, Communication distance expansion, Multi-robots, Remote control.

1 Introduction In fire-fighting and disaster rescue situations, fire fighters always face unpredictable situations. The probability of unexpected accidents is increased as they cannot effectively cope with such events, owing to which they experience mental and physical strain. In contrast, a robot can be put in dangerous environments because it can be controlled or navigated autonomously in a global environment. The use of robots to accomplish fire-fighting missions can reduce much of the strain experienced by the fire fighters. This is the reason for the development and employment of professional robots for fire fighting and disaster prevention. Incidentally, fire sites are considered in either the local or the global environment. If robots are placed in a global setting, they have to secure reliable communication among themselves and the central control system. Therefore, we approached the robot application from the point of view of fire fighting and disaster prevention, which require reliable communication and highly accurate distance measurements information. Y. Tan et al. (Eds.): ICSI 2011, Part II, LNCS 6729, pp. 99–107, 2011. © Springer-Verlag Berlin Heidelberg 2011

100

Y.-G. Kim et al.

The Kalman filter, a well-known algorithm widely applied in the robotics field, is based on the linear mean square error filtering for state estimation. A set of mathematical equations in the Kalman filter is implemented adequately as a compensator and an optimal estimator for some types of noises. Therefore, it has been used for stochastic estimation of measurements with noisy sensors. This filter can minimize the estimated error covariance when the robot is placed under presumed conditions. For the given spectral characteristics of an additive combination of signal and noise, the linear operation based on these input yields the best results with minimum square error of the signal from the noise. The distinctive feature of the Kalman filter, described in its mathematical formulation in terms of a state space analysis, is that its solution is computed recursively. Park [1] approached the recognition of position and orientation of a mobile robot using encoders and ubiquitous sensor networks (USNs). For this, the USNs are consisted of four fixed nodes and a mobile node. The robot is based on the fuzzy algorithm using information from the encoder and the USNs. Incidentally, this proposal has errors in the recognition of a USN when considering the exploration of each robot without fixed nodes. In addition, the noises caused by the friction between the road surface and the wheels and the control error of the motor affect the localization estimation acquired from the encoders. In addition, the measured errors accumulate while a robot navigates. In order to solve these problems, we proposed a localization and navigation system, which is based on the IEEE 802.15.4a protocol, to measure the distance between the robots and a compass sensor to obtain the heading angle of each robot. The IEEE 802.15.4a protocol allows for high aggregate throughput communication with a precision ranging capability. Nanotron techniques developed their first Chirp spread spectrum (CSS) smart RF module—smart nanoLOC RF with ranging capabilities. The proposed method is based on a modified Kalman filter, which is adapted in our system to improve the measurement quality of the wireless communication module, and the compass sensor for reducing the error in the localization and navigation process. This paper is organized as follows. Section 2 introduces related works and discusses localization approaches and the application of the IEEE 802.15.4a protocol to our system. Section 3 presents the proposed multi-robot-based localization and navigation. Section 4 explains and analyzes the experimental results. Finally, Section 5 presents the conclusion of this research and discusses future research directions.

2 Related Works 2.1 Localization Approaches In general, localization is divided into relative localization and absolute localization. Relative localization is the process of estimating a mobile robot’s state or pose (location and orientation) relative to its initial one in the environment; it also called dead reckoning (DR). Generally, an encoder, a gyroscope, and an inertial measurement unit (IMU) are used for localization by DR. It is easy and economically efficient to implement DR localization; however, DR has a critical drawback in that it is easily affected by external noises, resulting in error accumulation.

WPAN Communication Distance Expansion Method

101

Absolute localization is based on telemetric or distance sensors and may avoid the error accumulation of relative localization. Absolute localization is a global localization using which it is possible to estimate the current pose of the mobile robot even if the conditions of the initial pose are unknown and the robot is kidnapped and teleported to a different location [2]. The basic principle of absolute localization is based on probabilistic methods and the robot’s belief or Bayes’ rule. The former is a probability density function of the possible poses. The latter updates the belief according to the information. Taking into account the problem of approximating the belief, we can classify localization into Gaussian filter-based localization and non-parametric filter-based localization. The extended Kalman filter (EKF) [4] and the unscented Kalman filter (UKF) [3] are included in the former. Markov localization [5] and Monte Carlo localization [2] are included in the latter. EKF localization represents the state or pose of the robot as Gaussian density to estimate the pose using EKF. UKF localization addresses the approximation issues of the EKF. The basic difference between the EKF and the UKF stems from the manner in which Gaussian random variables (GRV) are represented for propagating through system dynamics [3]. In the EKF, state distribution is approximated by GRV, which is then propagated analytically through the first-order linearization of a nonlinear system. This can introduce large errors in the true posterior mean and the covariance of the transformed GRV, which may lead to sub-optimal performance and sometimes divergence of the filter. The UKF addresses this problem by using a deterministic sampling approach. The state distribution is also approximated by GRV. In contrast, it is now represented using a minimal set of carefully chosen sample points. These sample points completely capture the true mean and covariance of the GRV, which are propagated through the true nonlinear system. The EKF achieves only first-order accuracy. Neither the explicit Jacobian nor the Hessian calculation is necessary for the UKF. Remarkably, the computational complexity of the UKF is of the same order as that of the EKF [3]. Markov localization approximates the posterior pose of a robot using a histogram filter over a grid decomposition of the pose space. Hence, it is called grid localization. Monte Carlo localization approximates the posterior pose of a robot using a particle filter that represents the pose of the robot by a set of particles with important weight. This non-parametric filter-based localization can resolve the global localization and kidnap problem through multi-modal distribution. 2.2 IEEE 802.15.4A IEEE 802.15 related to the wireless personal area network (WPAN) is the standard protocol developed by many task groups (TG) in IEEE. In particular, IEEE 802.15.4 is the standard of the low power for driving devices, the low cost for establishment, and the available industrial, scientific, and medical (ISM) band. In addition, IEEE 802.15.4a provides enhanced ranging information among nodes through its adaptation of wireless communication. As a result, we decided to use this protocol for sensor networking. IEEE 802.15.4a was standardized in August 2007 based on the low complexity, low cost, and low energy in a WPAN environment and its capability to simultaneously allow for correspondence and distance measurement. IEEE 802.15.4a chooses two PHY techniques, namely, the ultra-wide band (UWB) method and the

102

Y.-G. Kim et al.

chirp spread spectrum (CSS) method with the centre of Samsung and Nanotron[6, 7]. UWB is a technique used for local distance communication and it is used for communicating signals with shorter pulse width in the baseband without a carrier. Owing to the extremely short pulse width, the applied frequency bandwidth is long. Therefore, it appears as though normal noise exists in channels of low output power. It does not affect the wireless device. However, it is difficult in long distance communication because it is a baseband communication and the output has low voltage. Its frequency range is 3.4 GHz~10 GHz. CSS was developed in the 1940s, and it is referred to as the dolphin and bat communication. It has been typically used in radars because it has some advantages such as strong interference and availability to long distance communication. After 1960, it was expanding into industrialization, and grafted linear sweep into chirp signal to get the significant information. CSS uses its entire allocated bandwidth to broadcast a signal, making it robust to channel noise. Moreover, even in the low voltage case, multi-path fading will not be much affected. The frequency of the CSS method is the 2.4 GHz ISM band.

3 System Architecture Figure 1 shows the proposed WPAN communication expansion scenario, which is based on the multi-robot cooperative robot navigation. There are two robots in this system: the leader robot (LR) and the follow robot (FR). Figure 2 shows the system architecture of the proposed scenario. In the proposed system, an operator controls the navigation of the LR by a remote controller and the FR navigates autonomously but also following the LR within a certain distance, which is limited to 2~3m. The ultimate mission of the FR is to secure the reliability of wireless communication among multiple communication nodes. Therefore, when the communication distance between the FR and the remote controller exceeds a valid communication distance or when communication is unstable, the FR stops following the LR and its mission changes to that of a communication relay node. Normally, localization and navigation research regarding a mobile robot has been using the distances measured by the motor encoders while the robots explore freely in an experimental environment. However, progressive error accumulation cannot be ignored and the result is a very serious but also unanticipated navigation error, especially in fire-fighting or disaster conditions. Therefore, we considered a new approach to the optimal navigation in rough terrains, which is a wireless RF-module-based navigation. The used wireless RF module is a Nanotron sensor node, i.e., Ubi-nanoLOC, which is developed by Hanbeak electron©[8]. The WPAN module is based on the IEEE 802.15.4a protocol for high aggregate throughput communication with a precision ranging capability. The measured distance from the wireless module may also include considerable error in an ambient environment. Because of this reason, the system adopted the Kalman filter to reduce the measurement error. LR can communicate with a remote controller through an ad-hoc routing application among multiple wireless communication modules, and the FR can undertake the assignment of securing the reliability of communication.


103

We utilized a micro-controller, i.e., the ATmega128, which adopts an 8-bit control system developed by Atmel© for controlling robot navigation. The driving performance of the DC motors was enhanced by PID control, and the accuracy of the compass was secured by fusing two sensors, namely, an AMI302 compass sensor, developed by Aichi Steel© [9], and an XG1010 gyro sensor, developed by Micro-Infinity© [10]. Fig. 3 shows the operation flow of the LR and FR systems. The leader robot is operated by a remote control. When the robot receives navigation commands such as forward, backward, and turn left and right, it moves depending on the commands. Moreover, it has more abilities such as acceleration and deceleration based on the PID control. The operation flow of the FR system maintains a valid distance between the robots through the ranging information from the LR. As mentioned above, the measured distance from the WPAN has various error factors because each robot always moves. Here, the applied Kalman filter works for compensating for the errors.

Fig. 1. Wireless communication expansion scenario based on a multi-robot system

Fig. 2. System architecture

104

Y.-G. Kim et al.

Fig. 3. Operation flowcharts of leader robot and follower robot

4 Experimental Results For this experiment, we placed two robots at certain distances in the linear corridor. Figure 4 shows the measured distance errors with keeping each distance interval between the FR and the LR. This experiment shows that the error for maintaining a specific interval is decreased when the Kalman filter is applied to the distance measurement. The Kalman filter estimates a more correct distance between the predicted encoder distance information and the measured WPAN distance information, as summarized in equations (1)–(7). We tried to simulate how well the FR follows LR by the leader following operation, which is based on the WPAN distance measurement. Figure 5 shows the simulation results of the leader following navigation of a follower robot. The simulation is the navigation results of the FR following the LR and navigating in a 10 m × 10 m area. The RF sensor data and compass sensor data have uncertainty error factors. Therefore, the objective of the proposed system is to achieve accuracy in the WPAN sensor network system by using the Kalman filter. However, the Kalman filter requires a considerable amount of data for the estimation. Even in this case, the system cannot move perfectly when the measurement data are dispersed. To solve this problem, we have to ignore the dispersed data. Therefore, the system cannot avoid resulting in errors. Figure 6 shows an experiment of the multirobot cooperation navigation for valid wireless communication distance expansion.

xˆ k−+1 = xˆ k + u k + wk

(1)

,

d k +1 = d k − ( Δt vk cosθ k )2 + ( Δt vk sinθ k )2

,

(2)


θ k +1 = θ k +

Δt vk tanφk L ,

Pk−+1 = Pk + σ w2 k K =

105

(3)

(4)

,

Pk−+1

2 Pk−+1 + σ RF k +1

(5) ,

xˆ k +1 = xˆ k−+1 + K ( z k +1 − xˆ k−+1 ) , Pk +1 = Pk−+1 (1 − K )

(6)

(7)

.

−

Fig. 4. Measured distance errors with keeping each interval b/t FR and LR

Fig. 5. Simulation of the leader robot following navigation by a follower robot

106

Y.-G. Kim et al.

Fig. 6. Multi-robot cooperation navigation for wireless communication distance expansion

5 Conclusion We proposed a multi-robot cooperation navigation method for securing a valid communication distance extension of the remote control based on the WPAN. The concept and implementation of the LR following navigation were introduced, and performance verification was carried out through navigation experiments in real or test-bed environments. The proposed multi-robot cooperation navigation method verified the effect and reliability of securing valid wireless communication and expanding the valid communication distance in an indoor and special-purpose service robot. Acknowledgments. This research was carried out under the General R/D Program sponsored by the Ministry of Education, Science and Technology(MEST) of the Republic of Korea and the partial financial support by the Ministry of Knowledge Economy(MKE), Korea Institute for Advancement of Technology(KIAT) and DaeguGyeongbuk Leading Industry Office through the Leading Industry Development for Economic Region.

References 1. Jong-Jin, P.: Position Estimation of a Mobile Robot Based on USN and Encoder and Development of Tele-operation System using the Internet. The Institute of Webcasting Internet and Telecommunication (2009) 2. Sebastian, T., Dieter, F., Wolfram, B., Frank, D.: Robust Monte Carlo Localization for Mobile Robots. Artificial Intelligence 128, 99–141 (2001) 3. Wan, E.A., van der Merwe, R.: Kalman Filtering and Neural Networks. In: The Unscented Kalman Filter, ch. 7. Wiley, Chichester (2001) 4. Greg, W., Gary, B.: An Introduction to the Kalman Filter. Technical Report: TR 95-041, University of North Carolina at Chapel Hill (July 2006) 5. Dieter, F., Wolfram, B., Sebastian, T.: Active Markov Localization for Mobile Robots in Dynamic Environments. Journal of Artificial Intelligence Research 11(128), 391–427 (1999) 6. Jeon, H.S., Woo, S.H.: Adaptive Indoor Location Tracking System based on IEEE 802.15.4a. Korea Information and Communications Society 31, 526–536 (2006)


107

7. Lee, J.Y., Scholtz, R.A.: Ranging in a Dense Multipath Environment using an UWB Radio Link. IEEE Journal on Selected Areas in Comm. 20(9) (2002) 8. http://www.hanback.co.kr/ 9. http://www.aichi-steel.co.jp/ 10. http://www.minfinity.com/

Relative State Modeling Based Distributed Receding Horizon Formation Control of Multiple Robot Systems* Wang Zheng1,2, He Yuqing2, and Han Jianda2 1 2

Graduate School of Chinese Academy of Sciences, Beijing, 100049, P.R. China State Key Laboratory of Robotics, Shenyang Institute of Automation, Shenyang, 110016, P.R. China {wzheng,heyuqing,jdhan}@sia.cn

Abstract. Receding horizon control has been shown as a good method in multiple robot formation control problem. However, there are still two disadvantages in almost all receding horizon formation control (RHFC) algorithms. One of them is the huge computational burden due to the complicated nonlinear dynamical optimization, and the other is that most RHFC algorithms use the absolute states directly while relative states between two robots are more accurate and easier to be measured in many applications. Thus, in this paper, a new relative state modeling based distributed RHFC algorithm is designed to solve the two problems referred to above. Firstly, a simple strategy to modeling the dynamical process of the relative states is given; Subsequently, the distributed RHFC algorithm is introduced and the convergence is ensured by some extra constraints; Finally, formation control simulation with respect to three ground robots is conducted and the results show the improvement of the new given algorithm in the real time capability and the insensitiveness to the measurement noise. Keywords: multiple robot system, formation control, distributed receding horizon control, relative state model.

1 Introduction Formation control, multiple robot systems working together as a fixed geometry configuration, has been widely researched in the past decades. And a great deal of strategies have been introduced and presents their great validity in both theory and reality, such as leader-following[1], behavior based[2], and virtual structure [3], etc. Receding horizon control (RHC), also called model predictive control (MPC), with the abilities of handling constraints and optimization, has been paid more and more attentions in the field of formation control in most recent. However, there are. One of the huge disadvantages in almost all existing receding horizon formation control (RHFC) is the huge computational burden due to the required online optimization algorithm. In order to solve this problem, distributed RHFC (DRHFC) seems a good method and some researching works have been published [4-9]. *

This work is supported by the Chinese National Natural Science Foundation: 61005078 and 61035005.


Relative State Modeling Based Distributed Receding Horizon Formation Control

109

However, there are some problems in DRHFC algorithms in most practical applications: 1) Absolute states of each individual robot are difficult to be obtained by other robots since intercommunication is lack of reliability in poor environment. 2) Most DRHFC algorithms use the absolute states directly while relative states between two robots are more accurate and easier to be measured in many applications[16]. Relative state model, i.e., which determines the relative motion law between two robot systems considering each individual model simultaneously in detail, is a new concept originated from the multiple satellite formation control[10]. Both relative kinematics model[11] and relative dynamics model[12] described this kind of relative motion. And, these relative state models have been applied to many distributed formation problems recently. In this paper, a new DRHFC strategy is proposed by introducing relative state model to deal with the above disadvantages. And the remainder of this paper is organized as follows: First, in section 2, the relative state model between two robot systems and a whole formation model are derived. Second, the formation strategy and distributed control law is realized in section 3. Subsequently, in section 4, a simulation results are presented to verify the validity of the proposed algorithm. Finally, the conclusions are given in section 5.

2 System Modeling 2.1 Relative Model We consider the formation control problem of N (N≥2) robot systems, and each individual robot’s dynamical model can be denoted as follows,

xi0 = fi 0 ( xi0 , ui )

(1)

where xi0 ∈ \ n (i=1,2,…,N) and ui ∈ \ m are the state vector and control input vector of the ith robot, respectively; f i 0 (⋅) are some nonlinear smooth functions with predefined structure. Generally, Eq.(1) describes the motion of the robot system in the global coordinate fixed with the earth [14-15]. Thus, xi0 is often called absolute states. Actually, for most member robot system in a formation, only relative states information are necessary to keep a high precise formation, so it is necessary to obtain the dynamical equation considering relative states between two interested robot systems. In this paper, we denote the relative model of robot i and robot j as follows,

x ij = f ji ( xij , ui , u j )

(2)

where xij ∈ \n is the relative state vector with the same dimensions of individual state xi and x j . ui , u j ∈ \ m are the control input of robot i and j, respectively. The methods for modeling relative state equations can be founded in [11] and [12].

110

W. Zheng, H. Yuqing, and H. Jianda

2.2 Formation Model In a formation control problem, suppose that every robot i has ni neighbor robots (neighbors of the ith robot mean the robots which can exchange information with robot i), and all the neighbors of robot i consist of a set Ni. There are two roles in our formation architecture, Na (Na≤N) leaders and N-Na followers. Leaders mean these robots which know their own desired states profile. While Followers denote these robots have no a priori-knowledge about their own desired states profile, and they can only follow their neighbor robots to keep the formation. Thus, the leader robot can be modeled using absolutely state equation and the follower robot can be modeled as several relative state equations with his neighbor robots. Thus, each robot’s state equation combined to its neighbors can be denoted as follows, ⎡ xi0 ⎤ ⎡ fi 0 ( xi0 , ui ) ⎤ ⎢ ⎥ ⎢ ⎥ # ⎢# ⎥=⎢ ⎥ ⎢ x ij ⎥ ⎢ f ji ( xij , ui , u j ) ⎥ ⎢ ⎥ ⎢ ⎥ # ⎣⎢ # ⎦⎥ ⎣⎢ ⎦⎥

(3.a)

# ⎡#⎤ ⎡ ⎤ ⎢ x i ⎥ = ⎢ f i ( x i , u , u ) ⎥ ⎢ j⎥ ⎢ j j i j ⎥ ⎢⎣ # ⎥⎦ ⎢⎣ ⎥⎦ #

(3.b)

where vector xi = [ xi " xij "]T and xi = [" xij "]T denote the leader and follower’s states, respectively. For the purpose of simplification, Eq.(3.a) and Eq.(3.b) can be transformed uniformly as xi = fi ( xi , ui , u−i )

(4)

where u−i = ["u j "]T is all the neighbors’ control inputs. Combining all the system states and models, the whole formation system’s formation model should be expressed as follow, x = f ( x, u )

(5)

where x = [ x1 ," xN ]T is the total states of all robots, and u = [u1 ,", uN ]T the total control input. f ( x, u ) = [" fi ( xi , ui , u−i )"]T is the summation of all the individual robots’ model (4).

3 Distributed Receding Horizon Formation Control 3.1 Cost Function

Before introducing the distributed receding horizon formation control algorithm, we first give some notations used in the following sections. For any vector x ∈ \ n , x


denotes the vector norm. x

2 P

111

= xT Px is P-weight 2-norm of vector x, where P is an

arbitrary positive-definite real symmetric matrix. Also, λmax ( P ) and λmin ( P) denote the largest and smallest eigenvalue of P, respectively. xijc , xi0c , xic and

x c = [ x1c ," xNc ]T are the desired states. In general, the following cost function is used in RHFC algorithm, N N ⎧ ⎪ L( x, u ) = ∑ L ( xi , ui ) = ∑ ⎨γ xi0 − xi0 c i =1 i =1 ⎪ ⎩

2 Qi0

1 + (1 − γ ) ∑ x ij − xij c 2 j ∈N i

2 Q ij

+ ui

2 Ri

⎫⎪ ⎬ ⎪⎭

(6)

where

for i ∈ {1,… , Na} (robot i is a leader) for i ∈ {N − Na,… , N } (robot i is a follower)

⎧1 ⎩0

γ =⎨

is a positive constant for distinguishing leader and follower. Weighted matrix Qi0 , Qij and Ri are all positive definite matrixes, and Qij = Qi j . Let Q = diag ("Qi0 "Qij ") and R = diag (" Ri ") , the integrated cost function can be equivalently rewritten as

L ( x, u ) = x − x c

2 Q

+ u

2

(7)

R

Splitting the cost function (7) as following distributed cost function for each individual robot,

Li ( xi , ui ) = xi − xic

2 Qi

+ ui

2 Ri

= γ xi0 − xi0 c

2 Qi0

1 + (1 − γ ) ∑ xij − x ij c 2 j ∈Ni

2 Qij

+ ui

2 Ri

(8)

Then, the distributed formation control problem can be described as: Design some distributed controllers ui = ki ( xi ) by solving a optimal control problem with respect to the distributed const function (8) for each individual robot i to make the formation system (5) converge to the desired formation state x c . 3.2 Algorithm

Since some cost Li(xi,ui) depends upon the relative states xij , which is subject to dynamics model (2), robot i must predict some relative trajectories according to ui and ui over each prediction horizon. That means, during each update, robot i will receive an assumed control trajectories uî (⋅; tk ) from its neighbors[9]. Then, by solving the optimal control problem using model (2), the assumed relative state trajectories can be computed. Likewise, robot i should transmit an assumed control to all neighbors for their own behavior optimization. Thus, the optimal control problem for each individual robot system can be denoted as

112


Problem 1. For every robot i ∈ {1,… , N } and at any update time tk, give initial conditions xi(tk), and assumed controls uˆ−i (⋅; tk ) , for all s ∈ [tk , tk + T ] find J i∗ ( xi (tk )) = min J i ( xi (tk ), ui (⋅; tk )) ui ( ⋅ )

(9)

where J i ( xi (tk ), ui (⋅; tk )) = ∫

tk + T

tk

Li ( xi ( s; tk ), ui (s; tk ))ds + M i ( xi (tk + T ; tk ))

subject to dynamics constrains xi (s; tk ) = f i ( xi (s; tk ), ui (s; tk ), u− i (s; tk )) input constrains ui ( s; tk ) ∈ U terminal constrains xi (tk + T ; tk ) ∈ Ωi (ε i )

(10)

and compatibility input constrains

ui ( s; tk ) − uî ( s; tk ) ≤ δ 2κ where terminal set is defined as Ω i (ε i ) = { xi | xi − xic

(11) 2

≤ ε i } , given the constants

κ , ε i ∈ (0, ∞) . Constraint (11) is used to reduce the prediction error due to the difference between what a robot plans to do and what neighbors believe that robot will plan to do. Details about defining constraint (11) can be found in [9]. Terminal function Mi(.) should be chosen to drive the terminal state enter the terminal set (10) so that system close-loop stability can be guaranteed. By solving Problem 1, the optimal control solution, we can obtain the optimal control profile ui∗ (τ ; tk ) , τ ∈ [tk , tk + T ] . And, the close-loop system for which stability is to be guaranteed is x(τ ) = f ( x(τ ), u∗ (τ )), τ ≥ t0

(12)

with the applied distributed receding horizon control law

u∗ (τ ; tk ) = (u1∗ (τ ; tk ),", uN∗ (τ ; tk )) for τ ∈ [tk , tk + 1) , and the receding horizon control law is updated when each new initial state update x(tk ) ← x(tk +1 ) is available. Following the succinct presentation in [9], we state the control algorithm. Algorithm 1. At time t0 with initial state xi(t0), the distributed receding horizon controller for any robot i ∈ {1,… , N } is as follows,


113

Data: xi(t0), T ∈ (0, ∞) , δ ∈ (0, T ] . Initialization: At time t0, solve Problem 1 for robot i, setting uî (τ ; t0 ) = 0 and uˆ− i (τ ; t0 ) = 0 for all τ ∈ [t0 , t0 + T ] and removing constraint (11). At every update interval, (1) Over any interval [tk, tk+1): a) Apply ui∗ (τ ; tk ) , τ ∈ [tk , tk +1 ) , b) Compute uî (τ ; tk +1 ) = uî (τ ) as ⎧⎪u ∗ (τ ; tk ) τ ∈ [tk +1 , tk + T ) uî (τ ; t k +1 ) = ⎨ i τ ∈ [tk + T , tk +1 + T ] ⎪⎩0

c) Transmit uî (τ ; tk +1 ) to neighbors and receive uˆ− i (τ ; tk +1 ) from neighbors. (2) At any time tk: a) Measure current state xi(tk), b) Solve Problem 1 for robot i, yielding ui∗ (τ ; tk ) , τ ∈ [tk , tk + T ] . 3.3 Stability Analysis

In this section, the stability analysis of algorithm 1 is given and the main result is somewhat similar to the work in reference [9]. So, the primary lemmas and theorems will be given with a simple explanation. Lemma 1. For a given fixed horizon time T>0, and for the positive constant ξ defined by

ξ = 2ρmax λmax (Q)ANT δ 2κ The function J*(.) satisfies N

tk +1

i =1

tk

J ∗ ( x(tk +1 )) − J ∗ ( x(tk )) ≤ − ∑ ∫

Li ( xi∗ ( s; tk ), ui∗ ( s; tk )) ds + ξδ 2

(13)

for any δ ∈ (0, T ] . In (13), ρ max ∈ (0, ∞) is a positive constant for restricting the state boundary, such that xi ( s; tk ) − xic ≤ ρ max for all s ∈ [tk , tk + T ] . Constant A ∈ (0, ∞) , restricting the boundary of uncontrollable input, satisfies x ij1 − xij 2 ≤ A u j1 − u j 2

at invariant ui

*

subject to relative model (2). Lemma 1 shows that J (.), the optimal value function, decreases from one update to the next along the actual closed-loop trajectories by properly choosing the update interval δ . That is, sufficient small δ will ensure the monotonically decreasing characteristic of objective function J*(.), satisfies J ∗ ( x(τ )) − J ∗ ( x(tk )) ≤ − x(tk ) − x c

2 Q

(14)

Theorem 1. For a given fixed horizon time T>0 and for any state x (t0 ) ⊂ X at initialization, if there exist an proper update time δ satisfies (14), then the formation can converge to x c asymptotically.

114


A small fixed upper bound on δ is provided that guarantees all robots have reached their terminal constraint sets via the distributed receding horizon control. After applying the previous lemmas, J*(.) is shown to be a Lyapunov function for the closed-loop system and the remainder of the proof follows closely along the lines of the proof of Theorem 1 in [13].

4 Simulation In this section, we will conduct some simulations to verify the supposed algorithm. Considering two dimensional bicycle-style robot system, shown an Fig.1, and its absolute and relative state model are stated as, ⎡ xi ⎤ ⎡υi cos θi ⎤ ⎢ y ⎥ ⎢υ sin θ ⎥ i ⎥ ⎢ i⎥ =⎢ i ⎢θi ⎥ ⎢ ui1 ⎥ ⎢ ⎥ ⎢ ⎥ ⎣υi ⎦ ⎣ ui 2 ⎦

(15.a)

⎡ x ij ⎤ ⎡υ j cos θ ij − υi + y ij ui1 ⎤ ⎢ i⎥ ⎢ ⎥ i i ⎢ y j ⎥ = ⎢ υ j sin θ j − x j ui1 ⎥ ⎢θij ⎥ ⎢ ⎥ −ui1 + u j1 ⎢ ⎥ ⎢ ⎥ ui 2 ⎢⎣ υi ⎥⎦ ⎢⎣ ⎥⎦

(15.b)

Simulation of three robots formation is presented. Robot-1 is a leader robot. As two followers, robot-2 follows robot-1 by the measured relative states, and robot-3 simultaneously follows robot-1 and robot-2. Set update interval δ =0.2s and predict horizon T=1s. At initial time, there robots are located at (2, 2), (1, 3) and (3, 1) in the global coordinate, respectively, and the desired formation is at initial time instant. In order to do some comparisons, we conducts the simulations using both absolute state based DRHFC method (DRHFC-A) and the supposed algorithm in this paper (DRHFC-B). The simulations are conducted using Matlab Optimization Toolbox solver at a PC (Intel(R) Core(TM)i5, M450 @ 2.40GHz). υj

υj

θ ij

θj

( x ij , y ij )

(x j , y j )

υi ( xi , yi )

θi

υi ( xi , yi )

Fig. 1. Absolute and relative modeling of robots

θi


115

Simulation 1: Time consuming There robots, keeping initial formation geometry, move along x axis with velocity 1m/s during the first 5 seconds. At the time 5s, an abrupt change of the leader robot’s desired position in the direction of y axis happens, i.e., the desired trajectory of the leader robot is as follows, ⎧[2 + t , 2], t ∈ [0,5] ⎨ ⎩[2 + t ,3], t ∈ [5,10] The whole simulation takes 10 second, and the trajectories of the robots are shown in Fig.2 where the five dashed circles of each individual robot denote their five different predictive states in every time interval. The relative position between robot-1 and robot-2 is shown at Fig. 3, where the dashed line denote the simulation results of DRHFC-A (algorithm in reference [9]) and the solid line describe the results of DRHFC-B (the proposed algorithm in this paper). From Fig.3, it can be seen that the precision of these two algorithms is similar. Time=5.4s

Time=9.6s Leader Follower1 Follower2

8

8

4

4 Y(m)

6

Y(m)

6

Leader Follower1 Follower2

2

2

0

0

-2

-2 0

5

10

15

0

5

X(m)

10

15

X(m)

Fig. 2. Trajectories of three robots formation at 5.4s and 9.6s respectively

Since DRHFC-B takes one relative model instead of two absolute models while solving the optimal problem at every interval. The computing time will be naturally reduced. Computing time of the two algorithms is shown in Fig.4, with average cost time Time(DRHFC-A)=3.18 and Time(DRHFC-B)=1.81. That means DRHFC-B is more effective than DRHFC-A. Also, comparisons are conducted in different simulation environment as shown in Table 1, and the similar results can be concluded. Relative Positions 14

1 10

0.5

0

1

2

3

4

5

6

7

8

9

10

-0.5

y12(m)

Time(A)=3.1848 Time(B)=1.8053

12

CPU Time(s)

x12(m)

1.5

8

6

4 -1 2

-1.5

0

1

2

3

4

5 Time(s)

6

7

8

9

10

Fig. 3. Relative positions of robot 1 and 2

0 -2

0

2

4

6

8

10

12

Time(s)

Fig. 4. Computing time at every update interval

116

W. Zheng, H. Yuqing, and H. Jianda Table 1. Computing time in different environments

Environment Hareware Solve method Intel(R) Core(TM)2 Line-Search Duo [email protected] L-M method Intel(R) Core(TM) i5 Line-Search M450 @ 2.40GHz L-M method Intel(R) Core (TM)2 Line-Search Duo 6300 @ 1.86GHz L-M method AMD Athlon(TM)64×2 Line-Search Dual Core 4000+ L-M method

Computing time DRHFC-B 1.63 4.07 1.81 4.47 3.93 4.69 3.61 4.28

DRHFC-A 2.82 5.34 3.18 5.90 6.32 7.42 6.21 7.48

Saving time 42.20% 23.78% 41.90% 24.24% 37.82% 36.79% 41.87% 42.78%

Simulation 2: Insensitiveness to measurement noise There robots keep initial formation geometry stationary for 10 seconds, this time desired trajectory of the leader robot changes to be [2, 2], t ∈ [0,10] Since there is no filter in controllers, and σ2=0.01m2 Gaussian white noise contained in every measured absolute and relative states, robots’ formation are interfered dramatically as shown in Fig.5. We chose object function J*(.)>0 to measure the stationary noise disturbance, larger J*(.) representing stronger disturbance. Fig.6 displays the compared cost function with average J*(A)=0.01196 and J*(B)=0.00351. That means DRHFC-B has less disturbance than DRHFC-A. Relative Positions 0.035 1.2 x12(m)

1.1

J(A)=0.011959 J(B)=0.0035076

0.03

1 0.025

0.9

0

1

2

3

4

5

6

7

8

9

10

Cost(m 2)

0.8 0.02

0.015

-0.8 y12(m)

-0.9

0.01

-1 0.005

-1.1 -1.2 0

1

2

3

4

5 Time(s)

6

7

8

9

10

Fig. 5. Relative positions of robot 1 and 2

0 -2

0

2

4

6

8

10

12

Time(s)

Fig. 6. Effect of noise disturbance

5 Conclusion In this paper, a new decentralized receding horizon formation control based on relative state model was proposed. The new designed algorithm has the following advantages: 1) The relative states, instead of the absolute states are used, since the latter is the only requirement for most member robots in a formation and easier to be measured; 2) Computation burden and influence from measurement noise is reduced. However, as a classical leader-follower scheme, some disadvantages will still exist in the proposed algorithm, which is common in most DRHFC algorithm. Such as, how to select proper parameters as the receding horizon time T and update period δ .


117

References 1. Das, A.K., Fierro, R., Kumar, V.: A vision-based formation control framework. J. IEEE Transactions on Robotics and Automation 18(5), 813–825 (2002) 2. Balch, T., Arkin, R.C.: Behavior-based formation control for multi-robot teams. J. IEEE Transactions on Robotics and Automation 14(6), 926–939 (1998) 3. Lewis, M.A., Tan, K.H.: High precision formation control of mobile robots using virtual structures. J. Autonomous Robots 4(4), 387–403 (1997) 4. Camponogara, E., Jia, D., Krogh, B.H., Talukdar, S.: Distributed model predictive control. J. IEEE Control Systems Magazine 22(1), 44–52 (2002) 5. Motee, N., Sayyar-Rodsari, B.: Optimal partitioning in distributed model predictive control. In: Proceedings of the American Control Conference, pp. 5300–5305 (2003) 6. Jia, D., Krogh, B.H.: Min-max feedback model predictive control for distributed control with communication. In: Proceedings of the American Control Conference, pp. 4507–4512 (2002) 7. Richards, A., How, J.: A decentralized algorithm for robust constrained model predictive control. In: Proceedings of the American Control Conference, pp. 4261–4266 (2004) 8. Keviczy, T., Borrelli, F., Balas, G.J.: Decentralized receding horizon control for large scale dynamically decoupled systems. J. Automatica 42(12), 2105–2115 (2006) 9. Dunbar, W.B., Murray, R.M.: Distributed receding horizon control for multi-vehicle formation stabilization. J. Automatica 42(4), 549–558 (2006) 10. Inalhan, G., Tillerson, M., How, J.P.: Relative dynamics and control of spacecraft formations in eccentric orbits. J. Guidance, Control, and Dynamics 25(1), 48–59 (2002) 11. Chen, X.P., Serrani, A., Ozbay, H.: Control of leader-follower formations of terrestrial UAVs. In: Proceedings of Decision and Control, pp. 498–503 (2003) 12. Wang, Z., He, Y.Q., Han, J.D.: Multi-unmanned helicopter formation control on relative dynamics. In: IEEE International Conference on Mechatronics and Automation, pp. 4381– 4386 (2009) 13. Chen, H., Allgower, F.: Quasi-infinite horizon nonlinear model predictive control scheme with guaranteed stability. J. Automatica 34(10), 1205–1217 (1998) 14. Fukao, T., Nakagawa, H., Adachi, N.: Adaptive tracking control of a nonholonomic mobile robot. J. IEEE Transactions on Robotics and Automation 16(5), 609–615 (2002) 15. Béjar, M., Ollero, A., Cuesta, F.: Modeling and control of autonomous helicopters. J. Advances in Control Theory and Applications 353, 1–29 (2007) 16. Leitner, J.: Formation flying system design for a planer-finding telescope-occulter system. In: Proceedings of SPIE the International Society for Optical Engineering, pp. 66871D-10 (2007)

Simulation and Experiments of the Simultaneous Self-assembly for Modular Swarm Robots Hongxing Wei1, Yizhou Huang1, Haiyuan Li1, and Jindong Tan2 1

School of Mechanical Engineering and Automation, Beijing University of Aeronautics and Astronautics, 100191, Beijing, China [email protected] 2 Electrical Engineering Department, Michigan Technological University, 49931, Houghton, USA [email protected]

Abstract. In our previous work, we have proposed a distributed self-assembly method based on Sambot platform. But there have interference of the infrared sensors between multiple Sambots. In this paper, two interference problems with multiple DSAs are solved and a novel simultaneous self-assembly method is proposed to enhance the efficiency of the self-assembly of modular swarm robots. Meanwhile, the simulation platform is established; some simulation experiments for various configurations are made and the results are analyzed for finding out evidence for further improvement. The simulation and physical experiment results verify the effectiveness and scalability of the simultaneous self-assembly algorithm which is more effective to shorten the assembly time. Keywords: swarm, self-assembly, modular robot.

1 Introduction The self-assembly has been paid special attention to in modular robot field, and has made a remarkable progress. Self-assembly can realize autonomous construction of configurations which refers to organizing a group of robot modules into a target robotic configuration through self-assembly without human interventions [1]. Because in modular swarm robotic field, the basic modules in most cases usually can not move on their own or only have very limited ability of autonomous locomotion, their initial configuration is generally manually assembled. However, once the robotic configuration is established, the number of modules is fixed, leading to difficulties to add new modules without external direction [2]. The self-assembly provides an efficient way for autonomous construction for modular swarm robots [3]. A group of modules or individual robots with the same function through self-assembly are connected into robotic structures, which have higher capabilities of locomotion, perception and operation. Bojinov [4], Klavins [5], Grady [6] et al respectively proposed some self-assembly control methods in different ways. We have designed a newly developed robotic module named as Sambot, which is an autonomous mobile robot having the characteristics of chain-type and mobile selfreconfigurable robots. Each Sambot has one active docking interface and four passive Y. Tan et al. (Eds.): ICSI 2011, Part II, LNCS 6729, pp. 118–127, 2011. © Springer-Verlag Berlin Heidelberg 2011

Simulation and Experiments of the Simultaneous Self-assembly

119

docking interfaces. It can move fully autonomously and dock with another Sambot from four directions. Through docking with each other, multiple Sambots can organize into a collective robot [7]. The algorithm for self-assembly is complex and because of high cost of hardware experiment, a simulation platform for Sambot robot is required. Using Microsoft Robotics Studio (MSRS), we design a simulation platform according to physical Sabmot system and some simulation experiments of autonomous construction for various configurations are conducted. In our previous work [7], [8], we have proposed a distributed self-assembly method based on Sambot platform. There have three types of Sambots, including Docking Sambots (DSA), SEED and Connected Sambots (CSA). Single DSA experiments for some configurations have been conducted. But because there have interference of the infrared sensor between multiple Sambots, the simultaneous self-assembly have not been realized. In this paper, two interference problems in Wandering and Locking phase are found out and solved. A simultaneous self-assembly method is designed to enhance the efficiency of the self-assembly of modular swarm robots. Meanwhile, the simultaneous docking of multiple Sambots in Locking phase has been realized. The simulation and physical experiment results show that the simultaneous self-assembly control method is more effective for the autonomous construction of swarm robots. The paper is organized as follows. In section 2, the overall structure of the Sambot robot is described and simulation platform of Sambot is introduced. In section 3, two interference problems in Wandering and Locking phase are analyzed and a simultaneous self-assembly control method is proposed. In section 4, based on the Sambot simulation platform, some simulation experiments are demonstrated to verify the selfassembly algorithm suitable for autonomous construction of various configurations. The simulation results are provided and analyzed. In section 5, physical experiments are implemented and the results are discussed. Finally, conclusions are given and the ongoing work is pointed out.

2 The Sambot Robot and Simulation Platform 2.1 Overall Structure of Sambot The Sambot is an autonomous mobile and self-assembly modular robot including a power supply, microprocessors, sensors, actuators, and a wireless communication unit, which is composed of a cubic main body and an active docking interface, as shown in Fig. 1 (a). The control system of each Sambot is composed of a main microcontroller and four salve microprocessors. The Sambot has two types of communications: the ZigBee wireless communication and the CAN bus communication. The former can be used to achieve global wireless communication among multiple Sambots but here it is not used. The latter takes effect only after two or multiple Sambots finish docking. In autonomous construction phase, CAN bus is adopted to communicate the information and command between Sambots and share the parameters. The bus can support 128 nodes at the most which is large enough for most engineering application.

120

H. Wei et al.

(a)

(b)

(c)

(d)

Fig. 1. The structure of Sambot. (a) a Sambot robot ; (b) simulated Sambot module; (c) simulated cross quadruped configuration; (d) simulated parallel quadruped configuration.

2.2 Simulation Platform While some researches are being performed, we use Microsoft Robotics Studio (MSRS) to build our simulation platform for more complex strcuture and large quantity of swarms. The simulation model is shown in Fig. 1(b). To realize physics-based simulation, we should design a class which contains inspection module, control module and execution module (as shown in Fig. 2). The inspection module contains gyroscope, infrared sensor and bumper sensor. Control module works as ports in simulation environment which receives message from inspection module and makes decision according to the information. Then robot carries out performance according to these decisions. In Fig. 1 (c) and (d), the simulated cross quadruped configuration and simulated parallel quadruped configuration have been demonstrated. Analysis of Configuration

Gyroscope Infrared sensor

Inspection module

Bumper sensor

Simulation engine AGEIA XNA PhysX render

Control module Execution module

Wander

Navigati on

Dock

Fig. 2. Frame of simulation platform

3 The Simultaneous Self-assembly Algorithms This section presents three types of roles of different Sambot robots in the selfassembly control model and a newly improved simultaneous self-assembly algorithm. In our previous work [8], a control model consisiting of SEED, CSA, DSA, and CCST (configuration connection state table) is proposed. However in phase of selfassembly, only a single docking Sambot can enter the platform and finish docking once, which is obviously efficient for large quantity of swarms. But, here, a simultaneous self-assembly algorithm for various configuration is designed for improving the assembly. These are carried out based on a bounded experimental platform.


121

3.1 Interferences of Self-assembly with Multiple DSAs In our previous work [8], in order to avoid the collision of simultaneous docking, the DSA Sambots are added onto the experimental platform one by one. The DSA is an “isolated” wander and does not have the information of the target configuration and the global coordination. Its controller works according to a series of behaviors of the DSA, including Wandering, Navigation, Docking and Locking. Obviously, if simultaneous docking is available, efficiency of self-assembly would be improved. Self-assembly with multiple DSAs has two interference problems to solve. One appears in Wandering phase, and the other in Locking phase.

SEED Sambo t

DSA (1)

DSA (1)

Left

DS (2 A )

DSA (2)

(a)

Back

SEED Sambo Front t Right

(b)

Fig. 3. Two interference situations. (a) DSA’s detecting infrared sensors are interfered by another DSA. (2) Information conflict of simultaneous docking for multiple DSAs.

1. In the phase of Wandering, when there is only one DSA to dock with the current configuration, the DSA searches for the Docking_Direction (infrared emitters) without another DSA’s interference. However, if there are multiple DSAs wandering simultaneously, interference would occur from other Sambots’ infrared emitters. In such cases, the DSAs might mistake anther DSA as the current configuration and then miss the target. As shown in Fig. 3 (a), in the process of searching SEED or CSA, detecting sensors of DSA (2) detect DSA (1) before find SEED and DSA (1) is mistaken as current configuration. Then it will navigate around DSA (1). Although DSA (2) still can get away from DSA (1) after DSA (1) is not within the perception of DSA (2), this process is unprofitable. So it is necessary to distinguish current configuration and DSA. 2. In the Locking phase, for simultaneous docking of multiple Sambots, information transmitting conflict can cause the deadlock. Because of CAN bus characteristics and sensors’ limitation, the bus is shared simultaneously by two or more Docking Sambots. When two docking interfaces of current configuration are docked with Sambot A and B meanwhile, Sambot A waits for the record end of Sambot B while Sambot B waits for record end of Sambot A. For example, in Fig. 3 (b) DSA (1) and DSA (2) are docking simultaneously with the SEED, the SEED needs to communicate with them. However, in previous self-assembly algorithm, docking time difference is used to recognize which interface is docked with and further define DSA’ node number in connection state table, here which is unavailable and need to be improved. 3.2 Solution of Interference and Simultaneous Self-assembly Algorithm To achieve simultaneous self-assembly with multiple DSAs, We proposed the improved algorithm to solve the interference problems for obtain better efficiency of self-assembly with multiple DSAs.

122

H. Wei et al.

SEED Sambo t

DSA

(a)

SEED Sambo t

A fixed angle

SEED Sambo t

DSA

SEED Sambo t

DSA

DSA

(b)

(c)

(d)

Fig. 4. Operation scenario of DSA detecting current configuration (here only SEED) A fixed angle Anothe r DSA

DSA

(a)

Anothe r DSA

Anothe r DSA

DSA

Anothe r DSA

DSA

DSA

(b)

(c)

(d)

Fig. 5. Operation scenario of DSA detecting another DSA

1. In order to avoid infrared sensors’ interference in the phase of Wandering, noting that the DSAs are wandering but the current configuration always remains static. Therefore, when detecting infrared sensors of a DSA receive signals, the object detected by the DSA may be the current configuration or another DSA. At that moment, the DSA moves forward a short step and then rotates around it by certain angle. After this moment, if the signal disappears, the object in the front must be another DSA. Otherwise, it might be the current configuration. However, a possible exception is that two or more DSAs might be interfered simultaneously, which may lead to a wrong judgments. In this situation, the DSA would be in endless deadlock. A function is designed to monitor this sutiation periodically to terminate the deadlock. The Wandering algorithm for multiple DSAs is improved as the following example. If the object detected by DSA is the current configuration, operation scenario is as shown in Fig. 4. First, detecting infrared sensors of DSA have input signal reflected by SEED (a). DSA rotates to the right by a certain angle (b). After that, DSA takes a certain distance forward (c). Then, DSA rotates turns left by a fixed angle and detects SEED again (d). However, as shown in Fig. 5, if the detected object is another DSA, the object will move away from the original place in movement process of DSA and at last is not within perception of DSA (Fig. 5 (d)). Therefore, this method can be used to distinguish current configuration and DSA and solve sensors’ interference problem. 2. Referring to the ordered resource allocation policy to prevet the deadlock by Havender, solution to avoid information conflict is designed. To solve the problem, an ordered conmunication process is introduced. Here, the four interfaces of the same Sambot belongs a group and the smaller the Samot’s node number in connection state table the smaller group number. Meanwhile, the four interfaces in the same group as the suquence of the front, left, back and right are respectively defined as 1, 2, 3 and 4(shown in Fig. 6). Fig 3 (b) gives a possible deadlock. Once deadlock happens,


123

interface of a lower number (here back) is delayed, until the information of high number has been transmitted and deadlock is removed, that is, comunication is running as an ordered allocation. Two improved algorithms to solve the corresponding interference problems are added to self-assembly control method. Multiple DSAs are able to simultaneously self-assembly into the target configuration according to design requirement. Obviously, it will shorten the assembly time, which would be analyzed in next sections through simulation and physical experiments.

the ordered resource allocation Left front right back Group 1

Communication bus

Group 2 Left front right back

Fig. 6. Solution to avoid the information conflict using ordered resource allocation policy

4 Simulation and Analysis 4.1 Simulation of Snake-Like and Quadruped Configuration On the simulation platform, we try to construct a snake-like configuration and a cross quadruped configuration. We will take experiments with 5, 6 and 7 robots as examples to show the distribution of completion time. Fig. 7 shows the process of the self-assembly experiments of snake-like and cross quadruped configuration. Fig. 8 shows the distribution of completion time. As shown in these graphs, as the number of robots increases, the completion time grows quickly. To show the variation trend of completion time, we expand the number of robots to 11 robots. As shown in Fig. 9, the curvilinear path of completion time is almost like quadratic curve whose slope grows. When the number of robots grows to a certain value, the completion time becomes unacceptable. However, the slope of expected curve should stay the same or even goes down. To explain the phenomenon, we can focus on a single robot. For each robot, the time of docking with other robot stays the same. Then we should pay attention to wandering and navigating state. In wandering state, as the number of robots grows, the probability of interference from other robots increases. It becomes more difficult to find the SEED or CSA. In navigating state, there are two main reasons: one is the distance for a robot to navigate increases as the configuration grows. The second is the interference from other robots when one robot is in navigating state, it can be brought back to wandering state by other robot. So to reduce the completion time, we should improve the wandering and navigating algorithms.

124

H. Wei et al.

(a) snake-like configuration

(b) cross quadruped configuration

Fig. 7. The self-assembly experiments of the snake-like and cross quadruped configuration on simulation platform

(a) Snake-like configuration

(b) Cross quadruped configuration

Fig. 8. Completion time of snake-like and cross quadruped configuration on simulation platform

Fig. 9. Average time of snake-like and cross quadruped configuration

4.2 The Simulation of Complex Configuration Fig. 10 shows the process of the self-assembly experiments of H-form and parallel quadruped configuration on simulation platform. The Fig.11 shows the distribution of completion time.


125

Fig. 10. The self-assembly experiments on the H-form and quadruped configuration on simulation platform

Fig. 11. Distribution of completion time of the H-form and quadruped configurations on simulation platform

5 Physical Experiments

ⅹ

Based on Sambot modules, on a platform of 1000 mm 1000 mm, we conduct the simultaneous self-assembly experiments with multiple DSAs for both the snake-like and the quadruped configurations. The SEED is also located at the platform center, but the DSAs are put randomly at the four corners. 1. The simultaneous self-assembly of the snake-like configuration with multiple DSAs is shown in Fig. 12. As for linear configuration, in simultaneous self-assembly process, simultaneous docking conflict doesn’t exit but DSA’s sensors are possible to be interfered by another DSA. 2. The simultaneous self-assembly of the quadruped configuration with multiple DSAs. As indicated by the red arrows in Fig. 13, all the four lateral interfaces of the SEED are Docking-Directions which remarkably enhance the experimental efficiency. Transmitting information conflict to the deadlock and sensor interference are possible to happen. However, the simultaneous self-assembly algorithm can deal with the problems. The experimental results verify the effectiveness of the algorithm.

126

H. Wei et al.

Fig. 12. The self-assembly experiment of the snake-like configuration with multiple DSAs

Fig. 13. The self-assembly experiment of the quadruped configuration with multiple DSAs

6 Conclusions and Future Work This paper proposed a simultaneous self-assembly control algorithm based on our novel self-assembly modular robot, Sambot. That can be used to realize reconfiguration by autonomous construction. Each module of Sambot is a fully self-contained, mobile robot that has the characteristics of both the chain-type and the mobile swarm robots. In distributed state, each DSA is an autonomous mobile robot; the control model has distributed characteristics. A simultaneous self-assembly algorithm is proposed to enhance the docking efficiency by solving transmitting information conflict and sensor interference. On simulation platform, we make simultaneous self-assembly experiments of various configuration and analyze the efficiency. We succeed in autonomously constructing the snake-like and the quadruped configurations with five Sambots on physical platform which verify simultaneous self-assembly control algorithm comparing the previous researches.


127

Some ongoing researches still deserve studying. It is significant that wandering and navigating algorithm still needs further improvement using evolutionary algorithm. Moreover, it is necessary to establish an autonomous control system for the selfassembly of some given configurations, the movement of the whole configuration, the evolutionary reconfiguration to another arbitrary robotic structure and so on.

Acknowledgments This work was supported by the 863 Program of China (Grant No. 2009AA043901 and 2009AA043903), National Natural Science Foundation of China (Grant No. 60525314), Beijing technological new star project (Grant No. 2008A018).

References 1. Whitesides, G.M., Grzybowski, B.: Self-Assembly at All Scales. J. Science 295, 2418– 2421 (2002) 2. Christensen, A.L., Grady, R.O., Dorigo, M.: Morphology Control in a Multirobot System. J. IEEE Robotics & Automation Magzine 14, 18–25 (2007) 3. Anderson, C., Theraulaz, G., Deneubourg, J.L.: Self-assemblages in Insect Societies. J. Insectes Sociaux 49, 99–110 (2002) 4. Bojinov, H., Casal, A., Hogg, T.: Multiagent Control of Self-reconfigurable Robots. J. Artificial Intelligence 142, 99–120 (2002) 5. Klavins, E.: Programmable Self-assembly. J. IEEE Control Systems Magazine 27, 43–56 (2007) 6. Christensen, A.L., O’Grady, R., Dorigo, M.: Morphology Control in a Multirobot System. J. IEEE Robotics & Automation Magazine 14(4), 18–25 (2007) 7. Hongxing, W., Yingpeng, C., Haiyuan, L., Tianmiao, W.: Sambot: a Self-assembly Modular Robot for Swarm Robot. In: The 2010 IEEE Conference on Robotics and Automation, pp. 66–71. IEEE Press, Anchorage (2010) 8. Hongxing, W., Dezhong, L., Jiandong, T., Tianmiao, W.: The Distributed Control and Experiments of Directional Self-assembly for Modular Swarm Robot. In: The 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4169–4174. IEEE Press, Taipei (2010)

Impulsive Consensus in Networks of Multi-agent Systems with Any Communication Delays Quanjun Wu1, , Li Xu1 , Hua Zhang2 , and Jin Zhou2 1

Department of Mathematics and Physics, Shanghai University of Electric Power, Shanghai, 200090, China [email protected] 2 Shanghai Institute of Applied Mathematics and Mechanics and Shanghai Key Laboratory of Mechanics in Energy Engineering, Shanghai University, Shanghai, 200072, China

Abstract. This paper considers consensus problem in directed networks of dynamic agents having communication delays. Based on impulsive control theory on delayed dynamical systems, a simple impulsive consensus protocol for such networks is proposed, and a generic criterion for solving the average consensus problem is analytically derived. Compared with some existing works, a distinctive feature of this work is to address average consensus problem for networks with any communication delays. It is shown that the impulsive gain matrix in the proposed protocol play a key role in seeking average consensus problems. Simulations are presented that are consistent with our theoretical results. Keywords: average consensus; impulsive consensus; directed delayed networked multi-agent system; fixed topology; time-delay.

1

Introduction

Recently, the distributed coordination in dynamic networks of multi-agents has emerged as a challenging new research area. The applications of multi-agent systems are diverse, ranging from cooperative control of unmanned air vehicles, formation control of mobile robots, control of communication networks, design of sensor-network, to flocking of social insects, swarm-based computing, etc., [1,2,3,4]. Agreement and consensus protocol design is one of the important problems encountered in decentralized control of communicating-agent systems. To achieve cooperative consensus, a series of works have been performed recently [1,2,3,4,5,6]. Jadbabaie et al. provided a theoretical explanation for the consensus behavior of the Vicsek model using graph theory [1]. Fax et al. emphasized the role of information flow and graph Laplacians and derived Nyquist-like criterion for stabilizing vehicle formations [2]. Olfati-Saber et al. investigated a systematical framework of consensus problem in networksof agents. Three consensus problems were discussed: directed networks with fixed topology, directed


Y. Tan et al. (Eds.): ICSI 2011, Part II, LNCS 6729, pp. 128–135, 2011. c Springer-Verlag Berlin Heidelberg 2011

Impulsive Consensus in Delayed Networks of Multi-agent Systems

129

networks with switching topology, as well as undirected networks with communication time-delays and fixed topology [3]. Moreau used a set-valued Lyapunov approach to study consensus problems with unidirectional time-dependent communication links [4]. Ren et al. extended the results to unidirectional communication and relaxed the connectivity assumption to the assumption that the union of the communication graphs has a spanning tree [5]. Time-delays often occur in such systems as transportation and communication systems, chemical and metallurgical processes, environmental models and power networks [7,8,9]. In many scenarios, networked systems can possess a dynamic topology that is time-varying due to node and link failures/creations, packet-loss, asynchronous consensus, state-dependence, formation reconfiguration, evolution, and flocking. There has been increasing interest in the study of consensus problem in dynamic networks of multi-agents with time-delays in the last several years [3,7,8,9]. It has been noticed the existing studies on consensus problem are predominantly to give some consensus protocols for networks of dynamic agents having communication delays with various network topology. However, these consensus protocols are only valid for some specific small communication delays [3,7,8,9]. For example, Olfati-Saber et al. discussed average consensus problems in undirected networks having a common constant communication delay with fixed topology and switching topology. They presented the following main result (See Theorem 10 in [3]): A sufficient and necessary condition for seeking average consensus in an undirected connected network is that the communication delays are less than a positive threshold. Therefore, this motivates the present investigation of average consensus problems in networks of dynamic agents for any communication delays particularly regarding practical engineering applications. This present paper considers consensus problem in directed networks of dynamic agents with fixed topology for any communication delays. It can generalize to the case of switching topology. The primary contribution of this work is to propose a novel yet simple impulsive consensus protocol for such networks, which is the generalization of corresponding results existing in the literature. A generic criterion for solving the average consensus problem is derived based on impulsive control theory on delayed dynamical systems. It is demonstrated that average consensus in the networks is heavily dependent on impulsive gain matrix in the proposed consensus protocol. Finally, simulations are presented that are consistent with our theoretical results. The paper is organized as follows. A simple impulsive consensus protocol is proposed in Section 2. In Section 3, we focus on the average consensus problem in directed delayed networks of dynamic agents with fixed topology. Some simulation results are provided in Section 4. Finally, we summarize the main conclusions in Section 5.

2

Consensus Algorithms

Let R = (−∞, +∞) be the set of real numbers, R+ = [0, +∞) be the set of nonnegative real numbers, and Z + = {1, 2, · · · } be the set of positive integer

130

Q. Wu et al.

numbers. For the vector x = [x1 , · · · , xn ] ∈ Rn , x denotes its transpose. Rn×n stands for the set of n × n real matrixes, for the matrix A = [aij ]n×n ∈ Rn×n , A denotes its transpose, As = (A + A )/2 stands for the symmetric part of A. The spectral norm of A is defined as A = [λmax (AA )]1/2 . E is the identity matrix of order n. In this paper, we are interested in discussing average consensus problem in directed delayed networks of dynamic agents with fixed topology, where the information (from vj to vi ) passes through edge (vi , vj ) with the coupling timedelays 0 < τ (t) ≤ τ . Here we assume that the communication topology of G is balanced and has a spanning tree. Moreover, each agent updates its current state based upon the information received from its neighbors. As L is a balanced matrix, an average consensus is asymptotically reached and α = ( i xi (0))/n = Ave(x). The invariance of Ave(x) allows decomposition of x according to the following equation: x = α1 + η, (1) where η = (η1 , · · · , ηn )T ∈ Rn satisfies 1T η = 0. Here, we refer to η as the (group) disagreement vector. The vector η is orthogonal to 1 and belongs to an (n − 1)-dimensional subspace. Let xi be the state of the ith agent. Suppose each node of a graph is a dynamic integrator agent with dynamics: x˙ i (t) = ui (t), i = 1, 2, · · · , n. (2) where ui (t) is the control input (or protocol) at time t. In [3], Olfati-Saber and Murray presented the following linear time-delayed consensus protocol: ui (t) = aij xj (t − τ ) − xi (t − τ ) . (3) vj ∈Ni

They presented the following main result [3]: Proposition 1. Assume the network topology G is fixed, undirected, and connected. Then, the protocol (3) globally asymptotically solves the average-consensus problem if and only if the following condition is satisfied: (i) τ ∈ (0, τ ∗ ) with τ ∗ = π/2λn , λn = λmax (L). Obviously, the consensus protocol (3) is invalid for any τ ≥ τ ∗ . The main objective of this section is to design and implement an appropriate protocol such that (2) uniformly asymptotically solves the average consensus problem for any communication delays. This is to say, limt→+∞ xi (t) − xj (t) = 0, for τ ∈ (0, +∞) and all i, j ∈ Z + . Based on impulsive control theory on delayed dynamical systems, we propose the following impulsive consensus protocol: aij xj (t − τ (t)) − xi (t − τ (t)) ui (t) = vj ∈Ni

+

+∞ m=1 vj ∈Ni

bij (xj (t) − xi (t))δ(t − tm ),

(4)


131

where bij ≥ 0 are constants called as the control gain, δ(t) is the Dirac function [9,10]. Remark 1. If bij = 0 for all i, j ∈ n, then the protocol (4) becomes a linear consensus protocol (3) corresponding to the neighbors of node vi . Clearly, consensus protocol (4) is the generalization of corresponding results existing in the literature [3,7,8,9]. It should be noted that the latter part of the impulsive consensus protocol (4) has two aims. On one hand, if τ (t) < τ ∗ , we can utilize it to accelerate the average consensus of such systems. On the other hand, if τ (t) ≥ τ ∗ , it can solve average consensus for any communication time-delays. This point will be further illustrated through the numerical simulations. Under the consensus protocol (4), the system (2) has the following form x(t) ˙ = −Lx(t − τ (t)), t = tm , t ≥ t0 , Δx(t) = x(t) − x(t− ) = −M x(t), t = tm , m ∈ Z + ,

(5)

where n M = (mij )n×n is a Laplacian defined by mij =s −bij , j = i, and mij = k=1,k=i bik , j = i. The eigenvalues of the matrix M can be ordered as 0 = λ1 (M s ) < λ2 (M s ) ≤ · · · ≤ λn (M s ). Moreover, η evolves according to the (group) disagreement dynamics given by η(t) ˙ = −Lη(t − τ (t)), t = tm , t ≥ t0 , (6) (E + M )η(t) = η(t− ), t = tm , m ∈ Z + In what follows, we will consider the average consensus problem of (5) with fixed topology. We will prove that under appropriate conditions the system achieves average consensus uniformly asymptotically.

3

Main Results

Based on stability theory on impulsive delayed differential equations, the following sufficient condition for average consensus of the system (5) is established. Theorem 1. Consider the delayed dynamical network (5). Assume there exist positive constants α, β > 0, such that for all m ∈ N , the following conditions are satisfied: s (A1 ) 2 + 2λ2 (M ) + λ2 (M M ) · L ≤ α; (A2 ) ln 1 + 2λ2 (M s ) + λ2 (M M ) − α(tm − tm−1 ) ≥ β > 0. Then the delayed dynamical network (5) achieve average consensus uniformly asymptotically. Proof. Since the graph G has a spanning tree, by using Lemma 3.3 in [5], then its Laplacian M has exactly one zero eigenvalue and the rest n− 1 eigenvalues all have positive real-parts. Furthermore, M s is a symmetric matrix and has zero row sums. Thus, the eigenvalues of matrices M s and M M can be ordered as 0 = λ1 (M s ) < λ2 (M s ) ≤ · · · ≤ λn (M s ),

132

Q. Wu et al.

and

0 = λ1 (M M ) < λ2 (M M ) ≤ · · · ≤ λn (M M ).

On the other hand, since M s and M M are symmetric, by the basic theory of Linear Algebra, we know η (t)M s η(t) ≥ λ2 (M s )η (t)η(t),

1 η = 0.

η (t)M M η(t) ≥ λ2 (M M )η (t)η(t),

1 η = 0.

(7) (8)

Let us construct a Lyapunov function of the form V (t, η(t)) =

1 η (t)η(t). 2

(9)

When t = tm , for all η(t) ∈ S(ρ1 ), 0 < ρ1 ≤ ρ, we have − η (tm )(E + M )(E + M )η(tm ) = η (t− m )η(tm ),

By (7) and (8), we get − 1 + 2λ2 (M s ) + λ2 (M M ) η (tm )η(tm ) ≤ η (t− m )η(tm ), that is V (tm , η(tm )) ≤

1 − V (t− m , η(tm )) 1 + 2λ2 (M s ) + λ2 (M M )

(10)

1 t, then ψ(t) is strictly increasing and 1 + 2λ2 (M s ) + λ2 (M M ) ψ(0) = 0 with ψ(t) < t for all t > 0. Hence, the condition (ii) of Theorem 1 in [10] is satisfied. For any solutions of Eqs. (6), if Let ψ(t) =

V (t − τ (t), η(t − τ (t))) ≤ ψ −1 (V (t, η(t)).

(11)

Calculating the upper Dini derivative of V (t) along the solutions of Eqs. (6), and by using the inequality x y + y x ≤ εx x + ε−1 y y, we can get that D + V (t) = −η Lη(t − τ (t)) ≤ L · V (t, η(t)) + sup V (s, η(s)) t−τ ≤s≤t

≤ 2 + 2λ2 (M s ) + λ2 (M M ) · LV (t, δ(t)) ≤ αV (t, η(t)). Letting g(t) ≡ 1 and H(t) = αt. Thus, the condition (iii) of Theorem 1 in [10] is satisfied. The condition (A2 ) of Theorem 1 implies that μ tm ds − g(s) ds ψ(μ) H(s) tm−1 1

1 = ln μ − ln[ μ] − (tm − tm−1 ) s α 1 + 2λ2 (M ) + λ2 (M M ) =

ln[1 + 2λ2 (M s ) + λ2 (M M )] β − (tm − tm−1 ) ≥ > 0. α α


133

The condition (iv) of Theorem 1 in [10] is satisfied. Let w1 (|x|) = w2 (|x|) = |x2 |/2, so the condition (i) of Theorem 1 in [10] is satisfied. Therefore, all the conditions of Theorem 1 in [10] are satisfied. This completes the proof of Theorem 1. Remark 2. Theorem 1 shows that, average consensus of the delayed dynamical network (5) not only depends on the topology structures of the entire network, but also is heavily determined by the impulsive gain matrix M and the impulsive interval tm − tm−1 . In addition, the conditions of Theorem 1 are all sufficient conditions but not necessary, i.e., the dynamical networks can achieve average consensus uniformly asymptotically, although one of the conditions of Theorem 1 may fail.

4

Simulations

As an application of the above theoretical results, average consensus problem for delayed dynamical networks is worked out in this section. Meanwhile, simulations with various impulsive gains matrices are given to verify the effectiveness of the proposed impulsive consensus protocol, and also visualize the impulsive gain effects on average consensus problem of the delayed dynamical networks. Here we consider a directed network with fixed topology G having 100 agents as in Fig. 1. It is easy to see that G has a spanning tree. Matrix L is given by ⎞ ⎛ 2 −1 0 · · · −1 ⎜ −1 2 −1 · · · 0 ⎟ ⎟ ⎜ ⎟ ⎜ L = ⎜ 0 −1 2 · · · 0 ⎟ . ⎜ .. .. .. . . .. ⎟ ⎝ . . . . . ⎠ −1 0

0 ··· 2

100×100

Fig. 1. A directed network with fixed topology having 100 agents

For simplicity, we consider the equidistant impulsive interval tm − tm−1 ≡ Δt. It is easy to verify that if the following conditions hold, 2 + 2λ2 (M s ) + λ2 (M M ) × 4 ≤ α; and ln 1 + 2λ2 (M s ) + λ2 (M M ) − α(tm − tm−1 ) ≥ β > 0.

134

Q. Wu et al. 12

xi(t),(i=1,2,...,100)

10

8

6

4

2

0

0

5

10 t

15

20

10

10

9

9

8

8

7

7 x (t),(i=1,2,...,100)

6 5 4

6 5 4

i

xi(t),(i=1,2,...,100)

Fig. 2. The change process of the state variables of the delayed dynamical network (5) without impulsive gain in case τ (t) = τ ∗ = π/8

3

3

2

2

1 0

1

0

0.5

1 (a) t

1.5

2

0

0

0.5

1 (b) t

1.5

2

Fig. 3. Average consensus process of the agents state of the delayed dynamical network (5) with different impulsive gains matrices in case τ (t) = 1.0

then all the conditions of Theorem 1 are satisfied, which means the delayed dynamical network (5) achieve average consensus uniformly asymptotically. Let the equidistant impulsive interval be taken as Δt = 0.02. Fig. 2 is the simulation result corresponding to change process of the state variables of the delayed dynamical network (5) having the communication delay τ (t) = τ ∗ = π/2λn = π/8 with the impulsive gain matrix M = 0 in time interval [0, 20]. This clearly shows that average consensus is not asymptotically reached, which is consistent with the result of Proposition 1. Fig. 3 demonstrates the change process of the state variables of the delayed dynamical network (5) having the communication delay τ (t) = 1 with different impulsive gain mij = −0.015, i = j, mij = 1.485, i = j, α = 30, β = 2.7322 and mij = −0.018, i = j, mij = 1.782, i = j, α = 36, β = 2.9169 in time interval [0, 2], respectively, which satisfy the conditions of Theorem 1. It can be shown that impulsive average consensus is finally achieved, and the impulsive gain matrix heavily affect consensus of the delayed dynamical network.

5

Conclusions

This paper has developed a distributed algorithm for average consensus in directed delayed networks of dynamic agents. We have proposed a simple impulsive consensus protocol for such networks for any communication delays, and some generic sufficient conditions under which all the nodes in the network achieve


135

average consensus uniformly asymptotically have been established. It has been indicated that average consensus in the networks is heavily dependent on communication topology of the networks and impulsive gain. Finally, numerical results have been used to show the robustness and effectiveness of the proposed impulsive consensus protocol.

Acknowledgment This work was supported by the National Science Foundation of China (Grant Nos. 10972129 and 10832006), the Specialized Research Foundation for the Doctoral Program of Higher Education (Grant No. 200802800015), the Innovation Program of Shanghai Municipal Education Commission (Grant No. 10ZZ61), the Shanghai Leading Academic Discipline Project (Project No. S30106), and the Scientific Research Foundation of Tongren College (Nos. TS10016 and TR051).

References 1. Jadbabaie, A., Lin, J., Morse, A.S.: Coordination of Groups of Mobile Autonomous Agents Using Nearest Neighbor Rules. IEEE Trans. Autom. Contr. 48, 988–1001 (2003) 2. Fax, J.A., Murray, R.M.: Information Flow and Cooperative Control of Vehicle Formations. IEEE Trans. Autom. Contr. 49, 1465–1476 (2004) 3. Olfati-Saber, R., Murray, R.M.: Consensus Problems in Networks of Agents with Switching Topology and Time-Delays. IEEE Trans. Autom. Contr. 49, 1520–1533 (2004) 4. Moreau, L.: Stability of Multiagent Systems with Time-Dependent Communication Links. IEEE Trans. Autom. Contr. 50, 169–182 (2005) 5. Ren, W., Beard, R.W.: Consensus Seeking in Multiagent Systems Under Dynamically Changing Interaction Topologies. IEEE Trans. Autom. Contr. 50, 655–661 (2005) 6. Hong, Y.G., Hu, J.P., Gao, L.X.: Tracking Control for Multi-Agent Consensus with an Active Leader and Variable Topology. Automatica 42, 1177–1182 (2006) 7. Sun, Y.G., Wang, L., Xie, G.M.: Average Consensus in Networks of Dynamic Agents with Switching Topologies and Multiple Time-Varying Delays. Syst. Contr. Lett. 57, 175–183 (2008) 8. Lin, P., Jia, Y.M.: Average Consensus in Networks of Multi-Agents with both Switching Topology and Coupling Time-Delay. Physica A 387, 303–313 (2008) 9. Wu, Q.J., Zhou, J., Xiang, L.: Impulsive Consensus Seeking in Directed Networks of Multi-Agent Systems with Communication Time-Delays. International Journal of Systems Science (2011) (in press), doi:10.1080/00207721.2010.547630 10. Yan, J., Shen, J.H.: Impulsive Stabilization of Functional Differential Equations by Lyapunov-Razumikhin Functions. Nonlinear Anal. 37, 245–255 (1999)

FDClust: A New Bio-inspired Divisive Clustering Algorithm Besma Khereddine1,2 and Mariem Gzara1,2 1

Multimedia InforRmation systems and Advanced Computing Laboratory (MIRACL) Sfax, Tunisia 2 Institut supérieur d’informatique et de mathématique de Monastir [email protected], [email protected]

Abstract. Clustering with bio-inspired algorithms is emerging as an alternative to more conventional clustering techniques. In this paper, we propose a new bio-inspired divisive clustering algorithm FDClust (Artificial Fish based Divisive Clustering algorithm). FDClust takes inspiration from the social organization and the encounters of fish shoals. In this algorithm, each artificial fish (agents) is identified with one object to be clustered. Agents move randomly on the clustering environment and interact with neighboring agents in order to adjust their movement directions. Two Groups of similar objects will appear through the movement of agents in the same direction. The algorithm is tested and evaluated on several real benchmark databases. The obtained results are very interesting in comparison with Kmeans, Slink, Alink, Clink and Diana algorithms. Keywords: Clustering, data mining, hierarchical clustering, divisive clustering, swarm intelligence, fish shoals.

1 Introduction Clustering is an important data mining technique that has a wide range of applications in many areas like biology, medicine, market research and image analysis among others. It is the process of partitioning a set of objects into different subsets. The goal is that the object within a group be similar (or related) to one another and different from (or unrelated to) the objects in other groups. Many clustering algorithms exist in the literature. At a high level, we can divide these algorithms into two classes: partitioning algorithms and hierarchical algorithms. Given a database of n objects or data tuples, a partitioning method constructs k partitions of the data, where each partition represents a cluster. Whereas, hierarchical clustering presents data in the form of a hierarchy over the entity set. In hierarchical clustering methods, the number of clusters has not to be specified a priori, and there are no initializations to be done. Hierarchical clustering is static, and data affected to a given cluster in the early stages cannot be moved between clusters. There are two approaches to build a cluster hierarchy: (i) agglomerative clustering that builds a hierarchy in the bottom up fashion by starting from smaller clusters and sequentially Y. Tan et al. (Eds.): ICSI 2011, Part II, LNCS 6729, pp. 136–145, 2011. © Springer-Verlag Berlin Heidelberg 2011

FDClust: A New Bio-inspired Divisive Clustering Algorithm

137

merging them into parental nodes (ii) divisive clustering that builds a top-down hierarchy by splitting greater clusters into smaller ones starting from the entire data set. Researchers seek to invent new approaches to enhance the resolution of the clustering problem and to achieve better results. Recently, research on and with the bio-inspired clustering algorithms has reached a very promising state. The basic motivation of these approaches stems from the incredible ability of social animals and other organisms (ants, bees, termites, birds, fish, etc) to solve complex problems collectively. These algorithms use a set of similar and rather simple artificial agents (ant, bee, individual, etc) to solve the clustering problem. These algorithms can be divided into three main categories according to data representation [1]: (i) an agent represents a potential solution to the clustering problem to be optimized such as genetic [2,3] and particle swarm optimization clustering algorithms [4,5], (ii) data points which are objects in the universe, are moved by agents in order to form clusters. Examples of such approaches are ant-based clustering algorithms [6] [7], (iii) each artificial agent represents one data set. These agents move on the universe to form groups of similar entities, for example Antree [8] and AntClust [9]. In this work, we propose a new bio-inspired divisive clustering algorithm: artificial Fish based Divisive Clustering algorithm (FDClust). This algorithm takes inspiration from the social organization and the encounters of fish shoals phenomena. Several studies have shown that fish shoals are assorted according to several characteristics [10][11]. During fish shoals encounters, an individual fish decides to join or to leave a group according to its common characteristics with the already existing group members [12][13]. Shoals encounters may result in the fission of the group into two homogenous shoals. Thus real fish are able to solve the sorting problem. These phenomena can be easily adapted to solve the clustering problem. In FDClust, an artificial fish represents an object to be clustered. The encounters of two artificial shoals results in the fission of the group into two clusters of similar objects. FDClust builds a binary tree of clusters. It applies recursively this process to split each node into two homogenous clusters. The reminder of the paper is organized as follows. Section 2 first describes the social organization of fish species and then the encounter phenomenon of fish shoals. In section 3 we present the FDClust algorithm in details. Experimental results are presented and discussed in section 4. Section 5 concludes the paper and gives suggestions for future work.

2 Social Organization and Encounters of Fish Shoals Fish are strikingly social organisms [14]. Several biological studies have observed and developed theoretical models to understand the fish shoals structures. In [13] the authors have stated that fish shoals are not random aggregations of individuals but they are instead assorted with respect to several factors including species and size. Croft et al [14] provided evidence that juveniles display assortative shoaling based on color pattern. Shoaling by color has also been reported in mollies [12]. Shoaling by species and body length was observed in several species [13][11]. The homogeneity of group composition has associated benefits such as anti-predator defense and foraging efficiency.

138

B. Khereddine and M. Gzara

Fig. 1. Diagram showing the two forms of fission events that were recorded a) a rear fission event, b) a lateral fission event [14]

Shoals membership is not necessarily stable over time. Individuals are exchanged between groups [14]. Fish shoals are thus open groups (groups where individuals are free to leave and join). Theoretical models of open groups assert that socials animals make adaptive decisions about joining groups on the basis of a number of different phenotypic traits of existing group members. Hence, individuals prefer to associate with similar conspecifics, those of similar body length and those free of parasite [13]. Active choice of shoal mates has been documented for many fish species. During shoals encouters individuals may actively choose neighboring fish that are of a similar phenotype. Fish have limited vision and then cannot interact with the total group members but only with perceived ones. Thus, shoals encounters provide an individual based mechanism for shoal assortment. Since individuals can make decisions based on the composition of available shoals, other group members are a source of information about the most adaptive decisions [15]. Group living is likely to be based on a continuous decision-making process, with individuals constantly evaluating the profitability of joining, leaving or staying with others, in each encounter with other groups. The encounters of fish shoals result in shoal fission or fusion. Fission (but not fusion) events are shown to be an important mechanism in generating phenotypic assortment [14]. Shoal fission events are divided into two categories (figure 1): (i) rear fission events where the two resulting shoals maintained the same direction of travel and fission occur due to differential swimming speeds, (ii) lateral fission events where the two resulting shoals are separated due to different directions of travel [14]. The social organization of fish shoals is based on the phenotypic similarity. The continuous decision-making process is based on the maintenance of social organization with neighboring group members. The behavior of real fish during shoals encounters makes them able to solve collectively the sorting problem. Our study of these phenomena (particularly the fission events) from a clustering perspective results in the development of a clustering model for solving the divisive clustering problem. The core task in such a problem is to split a candidate cluster into two distant parts. In our model, this task is achieved by the simulation of the encounters of two groups of artificial fish. The model is described in the next section.

3 FDClust: A New Bio-inspired Divisive Clustering Algorithm The FDClust algorithm constructs a binary tree of clusters by starting from all objects in the same cluster and splitting recursively each candidate cluster into two


139

sub-clusters until each object form one cluster. At each step the cluster with the highest diameter among those not yet splitted is partitioned into two sub-clusters. To achieve the partitioning of a group of objects into two homogenous groups, FDClust applies a bipartitioning procedure that takes inspiration from the shoals encounters phenomenon. During shoals encounters, real fish are able to evaluate dynamically the profitability of joining, leaving or staying with neighboring agents. This decision making process is based on the maintenance of social organization of the entire group. Fish shoals are phenotypically assorted by color, size and species. Shoals encounters may result in the fission of the group into two well-organized groups (assorted groups). In lateral fission, groups are separated due to two different directions of swimming. To achieve the division of the candidate cluster into two sub-clusters, we use two artificial fish shoals. The encounter of these two groups of artificial fish results in a lateral fission of the group into two homogenous groups. Artificial fish (agents) are initially randomly scattered on the clustering environment. Each agent is an object to be clustered. Each agent is randomly associated a direction left or right. Since real fish have only local vision, artificial agents interact only with neighboring agents to make adaptive decisions about joining or leaving a group. Each agent has to make a binary decision whether to move to the left or to the right. Agents take the same direction as most similar agents in their neighborhood. Artificial fish join finally their appropriate group composed with similar agents. The initial group is then separated into two sub-groups of similar objects due to the two directions of travel left and right. Two groups of agents are formed those having the left direction and those having the right direction. 3.1 Clustering Environment and Agents Vision The clustering environment is a rectangular 2D grid G. Its width is w= ⎡ n ⎤ and length is L = ⎡ n + 2 A ⎤ , where A is a positive parameter and n is the number of objects. Objects are initially scattered randomly in the central square of the grid of size w×w (figure 2). Two objects cannot occupy initially the same cell of the grid. Each agent has initially a random direction left (←) or right (→). Artificial agents have a limited vision. An agent can perceive only s×s neighboring cells (figure 2). Agents are allowed to move to a cell still occupied by other agents. Let n i be the

(a)

(b)

Fig. 2. FDClust: a) Clustering environment b) Agents vision

140


number of agents in the neighborhood v ( p i ) of the agent p i . If n ≤ v ( p ) = s × s then the agent p interacts with all his neighbors, else it interacts only with n v = s × s neighbors chosen randomly among those situated in his neighborhood. We note pv ( p i ) the set of agents with which the agent p i can interact. i

i

i

3.2 Agents Movements Each agent has an initial preferable direction left (←) or right (→). This direction is initially fixed randomly. Agents move with identical speed. In one step, an agent can move to one of its neighboring cells whatever the left one or the right one. It chooses actively its travel direction through the interactions with its neighboring agents. An agent interacts with at most n v nearest neighbors among those situated in his local neighborhood. In fact, agents can occupy the same cell as other agents. To take the decision about the next travel direction, the agent p i evaluates its similarity with agents from

pv ( p i )

that have the direction left (→) (respectively right (←)). These

two similarities are calculated as follow: p

sim

( pi,→ ) = 1 −

j

p

m *

j

pv

∈ pv ( p ∈ pv ( p j

i

) / dir

∑

i

( p 2

j

d

) / dir

) =→

( pi, p

( p

j

) =→

∈ v ( p i ) / dir ( p

p j ∈ pv ( p i ) / dir ( p j ) =← 2

sim ( p i , ← ) = 1 −

∑

d

p j ∈ pv ( p i ) / dir ( p

m * p

j

( pi, p

j

) =←

∈ pv ( p i ) / dir ( p

j

)

j

) =→

)

j

j

(1)

(2)

) =←

Where m is the number of attribute considered to characterize the data. An agent has the tendency to have the same direction of travel as its most similar neighbors. If agents in pv ( p ) that have the left direction are more similar to p than those having the right direction, than p will move to the cell at its left and vice versa. An agent will apply the following rules: i

i

i

• •

If | If |

•

If pv ( p ) ≠ to the right. If pv ( p ) ≠ to the left.

•

pv ( p

pv ( p

i

)

i

)

i

i

|=0 then the agent will stand by. |≠0 and sim ( p i , → ) = sim ( p i , ← ) then the agent will stand by. and

0

0

and

sim

( p

i

, →

) >

sim

( p

i

, ←

)

sim ( p i , → ) < sim ( p i , ← )

then the agent will move then the agent will move

3.3 The Algorithm FDClust FDClust starts by all objects gathered in the same cluster. At each step it applies the bi-partitioning algorithm to the cluster to be splitted until each object constitutes one cluster. It is a hierarchical divisive clustering algorithm (figure 3).


1. 2. 3. 4. 5.

141

Initially, the universal cluster C containing all objects is to be splitted. Bi-partitioning(C). Eliminate C from the list of clusters to be splitted, and add Cr and Cl to this list. Select the cluster C with the highest diameter among those not yet splitted. If |C|=1 stop else go to step 2. Fig. 3. The algorithm FDClust

Input: number of objects N, the size of the perception zone s*s, the movement step p and the number of iterations T. Output: Cl and Cr 1. Scatter objects of cluster C in the central square of the grid 2. Associate random direction (ĺ or ĸ) to each object 3. For t=1 to T do 4. For i=1 to N do 5. If(| pv ( p ) |=0) then stand by else 6. compute sim ( p i , → ) and sim ( p i , ← ) i

7.

If

8.

direction ( p )= ĺ and move to the right. If sim ( p , → ) < sim ( p , ← ) then

9.

direction ( else stand by

sim

( p

i

, →

) >

sim

( p

i

, ←

)

then

i

i

10. for i=1 to N do 11. if direction ( p )= ĺ then 12. Else p i ∈ Cl 13. end 14. end 15. Return Cl et Cr i

i

p

i

p

i

)=ĸ and move to the left.

∈ Cr

Fig. 4. Bi-partitioning algorithm

The bi-partitioning algorithm (figure 4) receives as a parameter a cluster C composed of n objects, the size of the perception zone s×s and the number of iterations T. The output of the algorithm is two clusters Cl and Cr. It assigns randomly to each object its corresponding coordinate on the grid and its initial direction left or right. The algorithm consists of T iterations. At each iteration, each agent evaluates its similarity with neighboring ones having the left direction (respectively the right direction), takes the decision on its following direction and computes its new coordinates on the grid. After T iterations, two clusters will be formed Cl and Cr, where Cl (respectively Cr) is the set of objects having the left direction (respectively right direction). The computational complexity of the bi-partitioning procedure is Ο(T n v n) with T the number of iteration, n v the maximum number of neighboring agents and n the number of objects in the candidate cluster.

142


4 Tests and Results To evaluate our algorithm, we have used real databases issued from the machine learning repository [16]. Table 1. Real data bases Data base Iris Glass Thyroid Soybean Wine Yeast

N 150 214 215 47 178 1484

M 4 9 5 35 13 8

K 3 6 3 4 3 10

The main features of the databases are summarized in Table 1. In each case the number of attributes (M), the number of classes (K) and the total number of objects (N) are specified. To evaluate our algorithm we have used the following measures:

The intra clusters inertia: used to determine how homogonous the objects in clusters are with each others (where, Gi is the center of the cluster I, d is the Euclidean distance): I =

1 K

¦

K

i =1

¦

xi ∈ C

d ( x i , Gi ) 2

(3)

The recall, the precision and the F-measure: are based on the idea of comparing a resulting partition with a real or a reference partition. The relative recall (respectively precision and F-measure) of the reference class Ci to the resulting class Cj are defined as follows: n ij

recall ( i , j ) =

N

precision (i , j ) =

i

n ij Nj

F (i, j ) = 2

precision precision

( i , j ) * recall ( i , j ) + recall

(i, j ) (i, j )

Where n ij is the number of objects or individuals present in the reference class Ci and in the resulting class Cj. Ni and Nj represent respectively the total number of objects in the class Ci and Cj. To evaluate the entire class Ci, we just choose the maximum of values obtained within Ci: recall ( i ) = max ( recall ( i , j )) precision ( i ) = max ( precision ( i , j )) F ( i ) = max ( F ( i , j )) j

j

j

The global value of the recall (r) , the precision (p) and F-measure (F) for all classes will be respectively ( p i is the weight of the class Ci): r =

∑ i

p i × recall (i )

p = ∑ pi × précision(i) i

F =

∑ i

p i × F (i )

where p i =

Ni (4) ∑ Nk k

In table 2, we present the obtained results for FDClust, kmeans, Alink, Clink, Slink and Diana algorithms. Since FDClust and kmeans are stochastic, we give the min, the max the mean and the standard deviation of 100 runs.

FDClust: A New Bio-inspired Divisive Clustering Algorithm Table 2. FDClust: experimental results Iris I R P F Glass I r p F Thyroid I r p F Soybean I r p F Wine I r p F Yeast I r p F

FDClust min 0 .047 max 0.051 mean 0.05 sd 0.01 min 0.86 max 0.92 mean 0.88 sd 0.01 min 0.85 max 0.92 mean 0.89 sd 0.01 min 0.85 max 0.92 mean 0.88 sd 0.01 FDClust min 0,09 max 0.14 mean 0.1 sd 0.03 min 0,32 max 0.75 mean 0.49 sd 0.05 min 0.41 max 0.7 mean 0.57 sd 0.07 min 0.37 max 0.61 mean 0.48 sd 0.05 FDClust min 0.02 max 0.11 mean 0.034 sd 0.01 min 0.54 max 0.72 mean 0.66 sd 0.03 min 0.59 max 0.74 mean 0.71 sd 0.01 min 0.49 max 0.68 mean 0.64 sd 0.03 FDClust

Kmeans min 0.05 max 0.052 mean 0.051 sd 0.01 min 0.89 max 0.92 mean 0.92 sd 0.009 min 0.66 max 0.89 mean 0.86 sd 0.08 min 0.7 max 0.88 mean 0.85 sd 0.06 Kmeans min 0,08 max 0.1 mean 0.83 sd 0.04 min 0,52 max 0.85 mean 0.63 sd 0.06 min 0.52 max 0.78 mean 0.63 sd 0.05 min 0.53 max 0.74 mean 0.6 sd 0.04 Kmeans min 0.03 max 0.056 mean 0.049 sd 0.03 min 0.86 max 0.78 mean 0.78 sd 0.01 min 0.71 max 0.9 mean 0.9 sd 0.02 min 0.77 max 0.87 mean 0.87 sd 0.01 Kmeans

min 1.42 max 1.57 mean 1.43 sd 0.02 min 0.62 max 0.8 mean 0.79 sd 0.03 min 0.71 max 1 mean 0.94 sd 0.04 min 0.75 max 0.97 mean 0.93 sd 0.04 FDClust min 0.32 max 0.34 mean 0.32 sd 0.03 min 0.86 max 0.98 mean 0.94 sd 0.05 min 0.68 max 0.93 mean 0.87 sd 0.05 min 0.73 max 0.93 mean 0.87 sd 0.04 FDClust min 0.011 max 0.012 mean 0.011 sd 0.001 min 0.21 max 0.64 mean 0.37 sd 0.09 min 0.29 max 0.47 mean 0.34 sd 0.03 min 0.22 max 0.52 mean 0.34 sd 0.05

min 1.57 max 1.64 mean 1.61 sd 0.02 min 0.53 max 1 mean 0.93 sd 0.07 min 0.51 max 1 mean 0.9 sd 0.11 min 0.41 max 1 mean 0.89 sd 0.11 Kmeans min 0.27 max 0.32 mean 0.28 sd 0.05 min 0.8 max 0.96 mean 0.95 sd 0.01 min 0.56 max 0.96 mean 0.95 sd 0.03 min 0.65 max 0.96 mean 0.94 sd 0.02 Kmeans min 0.032 max 0.035 mean 0.033 sd 0.0001 min 0.32 max 0.45 mean 0.39 sd 0.02 min 0.57 max 0.82 mean 0.65 sd 0.03 min 0.35 max 0.53 mean 0.48 sd 0.02

Slink 0.07

Alink 0.047

Clink 0.047

Diana 0.046

0.99

0.88

0.88

0.88

0.66

0.9

0.91

0.89

0,77

0,87

0,88

0,88

Slink 0,2

Alink 0,091

Clink 0,1

Diana 0,082

0,97

0,86

0,87

0,84

0.53

0.56

0.56

0.56

0.62

0.62

0.62

0.62

Slink 0.09

Alink 0.054

Clink 0.054

Diana 0.049

0.97

0.92

0.96

0.88

0.8

0.87

0.67

0.9

0.87

0.88

0.75

0.87

Slink

Alink

Clink

Diana

1.57

1.39

1.57

1.39

0.95

1

0.95

1

0.88

1

0.88

1

0.9

1

0.9

1

Slink 0.52

Alink 0.29

Clink 0.52

Diana 0.32

0.98

0.93

0.98

0.87

0.58

0.93

0.58

0.82

0.67

0.93

0.67

0.81

Slink 0.07

Alink 0.07

Clink 0.042

Diana 0.035

0.98

0.98

0.5

0.47

0.67

0.67

0.72

0.54

0.72

0.72

0.54

0.53

143

144


For the database Iris, our algorithm generates the best results according to all considering measures in comparison with other algorithms. For the data bases Glass and thyroid, FDClust encounters some difficulty in the determination of real cluster structure, but the obtained clusters are homogenous. For the data base soybean, all algorithms generate good partitions and results are nearby. For the data base Wine FDClust generates a partition of a good quality in term of inertia, recall, precision and F_measure in comparison with those obtained by the other algorithms. For the data base yeast, FDClust generates the best partition in term of intraclusters inertia but like Kmeans it has a difficulty in detecting real clusters structures. Comparing with other algorithms, we note that for all data bases FDClust has recorded good performances. Moreover FDClust has the advantage of having lower complexity than the other hierarchical algorithms.

5 Conclusion Bio-inspired clustering algorithms are an appropriate alternative to traditional clustering algorithms. Research on bio-inspired clustering algorithms is still an on-going field of research. In this paper we have presented a new approach for divisive clustering with artificial fish. It is based on the shoal encounters and social organization of fish shoals phenomena. The obtained results are encouraging. As prospects, we attempt to extend our algorithm by considering more than two directions of travels. A candidate cluster may be divided into more than two sub-clusters.

References 1. Bock., H., Gaul, W., Vichi, M.: Studies in classification, data analuysis, and knowldge organization (2005) 2. Falkenauer, E.: A new representation and operators for genetic algorithms applied to grouping problems. Evolutionary Computation 2(2), 123–144 (1994) 3. Maulik, U., Bandyopadhyay, S.: Genetic algorithm-based clustering technique. Pattern Recognition 33, 1455–1465 (2000) 4. Sandra Cohen, C.M., Leandro de Castro, N.: Data Clustering with Particle Swarms. In: IEEE Congress on Evolutionary Computations 2006 (2006) 5. Chen, C.-Y., Ye, F.: Particle swarm optimization algorithm and its application to clustering analysis. In: Proceedings of IEEE International Conference on Networking, Sensing and Control, pp. 789–794 (2004) 6. Lumer, E., Faieta, B.: Diversity and adaptation in populations of clustering ants. In: Cliff, D., Husbands, P., Meyer, J., W., S. (eds.) Proceedings of the Third International Conference on Simulation of Adaptive Behavior, pp. 501–508. MIT Press, Cambridge (1994) 7. Gzara., M., Jamoussi., S., et Elkamel, A., Ben Abdallah, H.: L’algorithme CAC: des fourmis artificielles pour la classification automatique. Accepté à paraitre dans la revue d’intelligence artificielle (2011) 8. Azzag, H., Guinot, C., Oliver, A., Venturini, G.: A hierarchical ant based clustering algorithm and its use in three real-world applications. In: Dullaert, W., Marc Sevaux, K.S., Springael, J. (eds.) European Journal of Operational Research (EJOR). Special Issue on Applications of Metaheuristics (2005)


145

9. Labroche, N., Monmarché, N., Venturini, G.: A new clustering algorithm based on the chemical recognition system of ants. In: van Harmelen, F. (ed.) Proceedings of the 15th European Conference on Artificial Intelligence, pp. 345–349 (2002) 10. Krause, J., Butlin, R.K., Peuhkuru, N., Prichard, V.: The social organization of fish shoals: a test of the predictive power of laboratory experiments for the field. Biol. Rev. 75, 477– 501 (2000a) 11. McCann, L.I., Koehn, D.J., Kline, N.J.: The effects of body size and body markings on nonpolarized schooling behaviour of zebra fish (Brachydanio rerio). J. Psychol. 79, 71–75 (1971) 12. Krause, J., Godin, J.G.: Shoal choice in the banded killifish (Fundulus diapha-nus, Teleostei, Cyprinodontidae) – Effects of predation risk, fish size, species compo-sition and size of shoals. Ethology 98, 128–136 (1994) 13. Crook, A.C.: Quantitative evidence for assortative schooling in a coral reef. Mar. Ecol. Prog. Ser. 179, 17–23 (1999) 14. Theodorakis, C.W.: Size segragation and effects of oddity on predation risk in minnow schools. Anim. Behav. 38, 496–502 (1989) 15. Croft, D.P., Arrowsmith, B.J., Bielby, J., Skinner, K., White, E., Couzin, I.D., Margurran, I., Ramnarine, I., Krausse, J.: Mechanisms underlying shoal composition in Trinidadian guppy (Poecilia). Oikos 100, 429–438 (2003) 16. Blake, C.L., Merz, C.J.: UCI repository of machine learning databases (1998)

Mining Class Association Rules from Dynamic Class Coupling Data to Measure Class Reusability Pattern Anshu Parashar1 and Jitender Kumar Chhabra2 1

2

Haryana College of technology & Management Kaithal,136027, India Department of Computer Engineering, National Institute of Technology, Kurukshetra, Kurukshetra 136 119, India [email protected]

Abstract. The increasing use of reusable components during the process of software development in the recent times has motivated the researchers to pay more attention to the measurement of reusability. There is a tremendous scope of using various data mining techniques in identifying set of software components having more dependency amongst each other, making each of them less reusable in isolation. For object-oriented development paradigm, class coupling has been already identified as the most important parameter affecting reusability. In this paper an attempt has been made to identify the group of classes having dependency amongst each other and also being independent from rest of the classes existing in the same repository. The concepts of data mining have been used to discover patterns of reusable classes in a particular application. The paper proposes a three step approach to discover class associations rules for Java applications to identify set of classes that should be reused in combination. Firstly dynamic analysis of the Java application under consideration is performed using UML diagrams to compute class import coupling measure. Then in the second step, for each class these collected measures are represented as Class_Set & binary Class_Vector. Finally the third step uses apriori (association rule mining) algorithm to generate Class Associations Rules (CAR’s) between classes. The proposed approach has been applied on sample Java programs and our study indicates that these CAR’s can assist the developers in the proper identification of reusable classes by discovering frequent class association patterns. Keywords: Coupling, Data Mining, Software Reusability.

1 Introduction Object oriented development has become widely acceptable in the software industry. It provides many advantages over the traditional development approaches [17] and is intended to enhance software reusability through encapsulation and inheritance [28]. In object-oriented concept, classes are basic building blocks and coupling between classes is well-recognized structural attribute in OO software engineering. Software Reuse is defined as the process of building or assembling software applications from previously developed software [20]. Concept of reuse has been widely used by the software industry in recent times. The present scenario of development is to reuse Y. Tan et al. (Eds.): ICSI 2011, Part II, LNCS 6729, pp. 146–156, 2011. © Springer-Verlag Berlin Heidelberg 2011

Mining Class Association Rules from Dynamic Class Coupling Data

147

some of the already existing quality components and development of new highly reusable component. The reuse of software components in software development leads to increased productivity, quality, maintainability etc [3,23]. The success of reusability is highly dependent on proper identification of whether a particular component is really reusable or not. These measures help to develop, store and identify reusable components [21]. Reuse of Class code has been frequent in practice. It is essential & tricky to identify a set of needed classes to reuse together or alone. Hence it is always desirable to find out the classes along with their associated classes [17]. Class coupling plays a vital role to measure reusability and selecting classes for reuse in combination because the highly coupled classes are required to be reused as a group [7]. One can define a class Ca related to class Cb if Ca must use Cb in all future reuse. So group of dependent classes should be reused together for ensuring the proper functioning of the application [22]. There is a thrust of software metrics especially reusability metric as an active research area in the field of software measurement. Software metric is a quantitative indicator of an attribute of a software product or process. There are some reuse related metric models like cost productivity, return on investment, maturity assessment, failure modes and reusability assessment etc [20]. For developer who wants to reuse components, reusability is one of the important characteristic. It is necessary to measure the reusability of components in order to recognize the reuse of components effectively. So classes must be developed as Reusable to effectively reuse them later. Developers should be trained or facilitated to use reusable components e.g. classes because it is hard to understand the structure of classes developed by others [24]. If developers do not have any prior knowledge about the coupling of classes they want to reuse, then they need to spend few time to understand the association pattern of classes. So there is a need to develop some mechanism that helps to know what combination of classes to reuse. By viewing class association rules and patterns, developer can predict required set of classes and can avoid unnecessary, partial class reuse. So for reuse, issues like maintaining class code repository, deciding what group of classes should be incorporated into repository & their association patterns and identifying exact set of classes to reuse, need to be addressed. It will reduce some reuse efforts. To discover the class association rules data mining can be used. By using data mining technology, one can find frequently used classes and their coupling pattern in a particular java application. 1.1 Data Mining and Its Usage in Reusability Data mining is the process of extracting new and useful knowledge from large amount of data. Mining is widely used to solve many business problems such as customer profiling, customer behavior modeling, product recommendation, fraud detection etc [25]. Data mining techniques can be used to analyze software engineering data to better understand the software and assist software engineering tasks. It also helps in programming, defect detection, testing, debugging, maintenance etc. In component reuse, mining helps in numerous ways such as to decide which components we should reuse, what is the right way to reuse, which components may often be reused in combinations etc [25]. The general approach of mining software engineering data consists of following steps:

148

A. Parashar and J.K. Chhabra

a) b) c) d)

Identify the software engineering problem to be solved by mining Identify and study data source Extract & preprocess data Mine the data e.g discover association rules

Due to the popularity of open source concept large amount of source code of classes is available on Internet as software repositories. Some also exists in large software companies where developer in one group may reuse classes written by other groups. For this reason, it is desirable to have mining tools that tell explicitly the class association patterns. Finding associations provides a distinct advantage in highly reusable environment. By searching for class patterns with high probability of repetitions we can correlate one set of classes with other set of classes. Also class associations will help developer to know which classes are likely to be reused together. The process of selecting required set of classes to reuse is complicated and requires some fundamental knowledge about the class structure, relationship or interaction with other classes. For this either software developer learns the reuse pattern of classes by their continuous experience or reading documentations/ manuals or by browsing such mined class association rules. The latter being feasible practically. In this paper, we explore market basket analysis technique to mine class association rules (CAR’s) from vast collection of class coupling data for particular project/program. This can be helpful in reusing classes by capturing association rules between classes. By querying or browsing such association rules a developer can discover patterns for reusing classes. For this purpose firstly dynamic analysis of java application is done using UML diagrams to collect class import coupling data. Then in second step, these collected data are represented as Class Set & Binary Class Vector. Then finally in the third step market basket analysis (apriori) technique is applied on Class Set representation to find frequently used classes and association rules between them. Further Class Vector representation is used to measure cosine similarity between classes. The measured values are analyzed to compare import coupling pattern between classes. The rest of the paper is organized as follows. Section 2 discusses the related works. Section 3 describes the proposed methodology to mine class association rules and class coupling behavior. Section 4 shows example case study to illustrate our approach .Section 5 presents results and discussion. Finally, Section 6 concludes this paper.

2 Related Works For object-oriented development paradigm, class coupling has been used as an important parameter effecting reusability. Li et.al. [19], Yacoup et. al [18] , Arisholm et. al.[2] proposed some measures for coupling. Efforts have been made by the researchers to measure reusability through coupling and cohesion of components [5]. Gui et al [6,7] and Choi et al [4] provided some reusability measures based on coupling and cohesion. ISA [8] methodology has been proposed to identify data cohesive subsystems. Gui et al [10] proposed a new static measure of coupling to assess and rank the reusability of java components. Arisholm et. al.[2] have provided a method for identifying import coupled classes with each class at design time using UML


149

diagrams. Data mining is focused on developing efficient techniques to extract relevant information from very large volumes of data that may be exploited, for example, in decision making, to improve software reliability and productivity [9]. Association rule discovery from large databases is an important data mining task. Few algorithms have been proposed for mining the association rules like market basket analysis [1],apriori[11],Ranked Multilabel Rule (RMR) [12], CAR[13], CMAR [14],ARMC[13]. Michail[28] considered the problem of discovering association rules that identify library components that are often reused in combination in the ET++ application framework . Yin et al[15] proposed a Classification approach CPAR based on Predictive Association Rules, which combines the advantages of both associative classification and traditional rule-based classification. Cosine similarity (between two vectors) and Jaccard similarity coefficient are often used to compare documents in text mining[30]. We find the association mining approach proposed by Agrawal et al[1,11,3] and cosine similarity measure very simple and go well with our idea. So to predict class reusability pattern of a particular java application, we are using cosine similarity measure and association mining approach.

3 Proposed Methodology The concepts of data mining have been used to discover patterns of reusable classes in a particular application. These patterns are further helpful in reusing the classes. Association rules between classes and class coupling behaviour are used to identify the class reusability patterns. For this purpose, association mining algorithm [1, 11] is used to mine class association rules (CAR) from class import coupling data. To know the class coupling behaviour the cosine similarity measure can be applied on class import coupling data. Our approach to mine class association rules and class coupling behavior consists of three steps: 1. 2. 3.

Collection of Class import coupling data through UML. Representation of Collected Data. Mining of Class Association Rules (CAR) & Prediction of class import coupling behavior. The steps are described in section 3.1 to 3.3. 3.1 Collection of Class Import Coupling Data through UML Dynamic analysis of a program is a precondition for finding the association rules between classes. Dynamic analysis of programs can be done through UML diagrams [27]. Significant advantages of using UML are its language independence and computation of dynamic metrics based on early design artifacts. Erik Arisholm[2] referred UML models to describe dynamic coupling measures as a way to collect for each class its import coupled classes. They used following formula for calculating class import coupling IC_OC (Ci). I C _ O C ( c 1)

{ ( m 1 , c 1 , c 2 ) | ( ( o 1 , c 1 ) R o c ) ( ( o 1 , c 2 ) R o c | N ) c 1 z c 2 ( o 1 , m 1 | o 2 , m 2 ) M E }

150


IC_OC (Ci) counts the number of distinct classes that a method in a given object uses. This formula can be used to measure dependency of one class to other classes in terms of its import coupling. 3.2 Representation of Collected Data Data thus collected in step one should be represented in some suitable intermediate representation, so that mining algorithm can be applied easily to find the class associations. In this we propose to represent data in two forms: 3.2.1 Class Set Representation Class import coupling data of each class can be represented by class set representation. IC_Class_Set represents classes coupled (import) with a class. For example , Let C= {C1,C2,C3,C4,C5} is set of classes of an application , IC_Class_Set(C1)= {C1,C3,C4 } means that class C1 is coupled(import) with classes C3 and C4.Class C1 itself is included in its import coupled class set to have complete set of classes used . For an application, the collection of IC_Class_Set of all classes is called as IC_SET (application). 3.2.2 Class Vector Representation Class vector of a class also represents the import coupled classes with a given class but it is in vector form and is w.r.t. all classes of the application. Suppose C is the ordered set of classed in a particular application , then for a class Ci , class vector is represented as C_V(Ci)=[1,0,1,1,0] . Here 1 at place j indicates that Ci class is coupled (import) with class j and 0 at place k indicates no coupling (import) of class Ci with class Ck . From above two representations, IC_SET (application) is used to mine class association rules through apriori approach and C_V(Ci) is used for measuring class coupling behavior by cosine similarity measure. 3.3 Mining of Class Association Rules and Prediction of Class Import Coupling Behavior 3.3.1 Mining Class Association Rules To mine class association rules for java programs basic apriori approach proposed by Agrawal et al[1,11,3] has been used. They have used the apriori approach (Market basket analysis) to analyze marketing transaction data for determining which products customers purchase together. The concept of support and confidence has been used to find out association rules for the purchased products. The support of an itemset is defined as the proportion of transactions in the transaction data set which contain the itemset and confidence of a rule X→Y is defined as an estimate of the probability P(Y | X), the probability of finding the RHS of the rule in transactions under the condition that these transactions also contain the LHS[29].Based on that we are considering a collection of class import coupling data for an application i.e. IC_SET (application) to find the set of Class Association Rules (CAR). Association rules are required to satisfy a user-specified minimum support and minimum confidence at the same time. Association rule generation is usually split up into two separate phases. In First phase, minimum support min_sup is applied to mine all frequent classes called as Frequent Class Combination Set (FCCS). The process to find FCCS is as follows:


151

1: i=1 2: Create candidate class set CSi having all classes and their Support. (The Support for a class Ci is the frequency of occurrence of that class in IC_SET (application). 3: Create large class set Li by eliminating Class Set from CSi having Support sup<min_sup 4: Repeat 5: i=i+1 6: Create candidate class set CSi having Cartesian product of sets in Li-1 and calculate their support from IC_SET (application). 7: Create Large set Li by eliminating Class Set from CSi having sup <min_sup 8: Until (no scope to built large class set) So Class Set in CSi give frequent Class Combination Set (FCCS).After this in Second phase, FCCS and minimum confidence min_conf constraint are used to form CAR. The support and confidence values for each pair of classes in FCCS are calculated using below mentioned formulas 1 &2 [1,11,3]: support(Ci → Cj) =

confidence(Ci → Cj) =

no of tuples containing both Ci & C j total no of tuples no of tuples containing both Ci & C j no of tuples containing Ci

(1)

(2)

So a rule Ci → Cj holds in the IC_SET with confidence cf% if cf% tuples of ICOUP_SET contain Ci also contain Cj . The rule Ci → Cj has support sp% if sp% tuples of IC_SET contain Ci U Cj. So the association rule Ci → Cj on IC_SET is worth considering if set of tuples in IC_SET holds this rule with sup (Ci → Cj) >min_sup & conf (Ci → Cj) >min_conf. As a result of this, the distinction can be made between classes that are being used together often and classes that are not. These rules suggest which classes can be reused as a group and repository designer can then use these rules to put frequently used classes in the repository. 3.3.2 Measuring Class Coupling Behavior The class vector representation C_V(C) of classes is used to compute Cosine Similarity between classes on a scale of [0, 1]. The Cosine Similarity [16] of two class vectors C_V (Ci) & C_V (Cj) is defined as:

Cos_Sim(Ci,Cj) =

Ci .C j Ci C j

The value 1 means that the coupling pattern of classes Ci & Cj is identical, 0 means completely different[16,26].So using Cosine similarity one can analyze which classes have similar, nearly similar or completely different coupling pattern. In next section, we demonstrate our approach of Mining Class Association Rules and Measuring Class Coupling Behavior of a sample application.

152


4 Example Case Study We are using a small example to illustrate our approach for Mining Class Association Rules and measuring class coupling behavior. We consider example java application MYSHAPES. Application MYSHAPES has class set C(MYSHAPES)= { myshape, circle, square, shape }. In the first step import coupling data are collected for MYSHAPES using UML approach. In second step these collected values are represented asIC_SET(MYSHAPES). The next sections 4.1 & 4.2 show the third step of our approach. We assume min_sup>25%. 4.1 Mining Class Association Rules As a first part of third step of methodology, the method given in section 3.3.1 is applied on IC_SET (MYSHAPES) to find Frequent Class Combination Set (FCCS) and is shown in figure 1.Then the output FCCS is used to form Class Association Rules(CAR) having min_conf ≥90%. Table 1 lists the CAR with confidence more than 90% for the application MYSHAPES. 4.2 Measuring Class Coupling Behavior To find out the behavior of each class in terms of class coupling pattern we use class vector representation (C_V) of MYSHAPES (table 2) and compute Cosine similarity measure between classes as mentioned in section 3.3.2. Following table 3 shows the computed Cos_Sim(Class1,Class2).

5 Results and Discussion We can measure the reusability pattern of classes by analyzing their association rules and import coupling patterns. CAR’s of application MYSHAPES (figure 2) suggest whenever a class on the left hand side of rule is to be reused, there is strong probability with 100% confidence that classes on right side of the rule will also be reused. From figure 2 it is observed that whenever class square is to be reused class shape will also be reused. From figure 2 it is observed that the cosine similarity between classes circle and shape is 1 and myshape and square is.71. It suggests that import coupling behavior of classes circle & shape are exactly similar i.e. they are always used together while classes myshape and square are sometimes import coupled to some common classes. Our study shows that FCCS, CAR’s and Cos_sim between classes can be helpful for a repository designer/user to predict which classes are required to be reused in combination and what is the coupling pattern of classes. The effectiveness of Class association rule is dependent on type of coupling attributes used to know import coupling between classes, ways to represent coupling data and accuracy of association mining algorithm applied to it.


IC_SET (MYSHAPES)

CS1

Class_Id

IC_Class_Set

Class_Set

sup

myshapes

myshape, circle, square, shape

myshapes

01

circle

circle, square, shape

circle

03

square

04

square

square , shape

shape

04

shape

circle , square , shape

L1

CS2 Class_Set

sup

{ circle ,square }

03

{ circle ,shape }

03

{ square shape }

03

Class_Set

sup

circle

03

square

04

shape

04

CS3 L2 Class_Set

sup

{ circle ,square }

03

{ circle ,shape }

03

{ square shape }

03

Class_Set

sup

{ circle ,square, shape }

03

Frequent Class Combination Set(FCCS) { circle ,square, shape }

Fig. 1. Frequent Class Combination Mining Steps

153

154

A. Parashar and J.K. Chhabra Table 1. List of CAR & FCCS

Application

Frequent Class Combinations Set (FCCS)

MYSHAPES

{circle , square, shape}

CAR

1.

circleĺsquare

(sup=75% , conf=100%)

2.

circleĺshape

(sup=75% , conf=100%)

3.

shapeĺsquare

(sup=100% , conf=100%)

4.

squareĺshape

(sup=100% , conf=100%)

5.

circleĺshape, square

Table 2. Class vector of MYSHAPES

(sup=75% , conf=100%)

Table 3. Cosine similarity between of MYSHAPES

myshape

circle

square

shape

C_V(myshape)

1

1

1

1

C_V(circle)

0

1

1

1

C_V(square)

0

0

1

1

C_V(shape)

0

1

1

1

Cos_Sim(Class1,Class2)

Scale

Cos_Sim(myshape,circle)

.87

Cos_Sim(myshape,square)

.71

Cos_Sim(myshape,shape)

.87

Cos_Sim(circle,square)

.81

Cos_Sim(circle,shape)

1

Cos_Sim(square,shape)

.87

1 0.8

squareĺshape

SCALE

circleĺshape, square

0.6 0.4

CAR’s

0.2

shapeĺsquare

0

circleĺshape circleĺsquare 0% 20% 40% 60% 80%100% SUPPORT

Fig. 2. CAR and their Support

Cos Sim(Class1,Class2)

Fig. 3. Cosine Similarities between Classes


155

6 Conclusions In this paper, an attempt has been made to determine class reusability pattern from dynamically collected class import coupling data of java application. Our initial study indicates that basic technique of market basket analysis (apriori) and cosine similarity measure can be constructive to find out class association rules (CAR’s) and class import coupling behaviour. Currently, we have deduced CAR’s for a sample java application. However, the approach can also be applied on larger java applications. Moreover, other association mining and clustering algorithms can be explored to apply on class coupling data for finding class reusability patterns.

References 1. Agrawal, R., Imielinski, T., Swami, A.: Mining Association Rules between Sets of Items in Large Databases. In: ACM, SIGMOD, pp. 207–216 (1993) 2. Arisholm, E.: Dynamic Coupling Measurement for Object-Oriented Software. IEEE Transactions on Software Engineering 30(8), 491–506 (2004) 3. Negandhi, G.: Apriori Algorithm Review for Finals, http://www.cs.sjsu.edu 4. Choi, M., Lee, J.: A Dynamic Coupling for Reusable and Efficient Software System. In: 5th IEEE International Conference on Software Engineering Research, Management and Applications, pp. 720–726 (2007) 5. Mitchell, A., Power, F.: Using Object Level Run Time Metrics to Study Coupling Between Objects. In: ACM Symposium on Applied Computing, pp. 1456–1462 (2005) 6. Gui, G., Scott, P.D.: Coupling and Cohesion Measures for Evaluation of Component Reusability. In: ACM International Workshop on Mining Software Repository, pp. 18–21 (2006) 7. Taha, W., Crosby, S., Swadi, K.: A New Approach to Data Mining for Software Design. In: 3rd International Conference on Computer Science, Software Engineering, Information Technology, e-Business, and Applications (2004) 8. Montes, C., Carver, D.L.: Identification of Data Cohesive Subsystems Using Data Mining Techniques. In: IEEE International Conference on Software Maintenance, pp. 16–23 (1998) 9. Xie, T., Acharya, M., Thummalapenta, S., Taneja, K.: Improving Software Reliability and Productivity via Mining Program Source Code. In: IEEE International Symposium on Parallel and Distributed Processing, pp. 1–5 (2008) 10. Gui, G., Scott, P.D.: Ranking reusability of software components using coupling metrics. Elsevier Journal of Systems and Software 80, 1450–1459 (2007) 11. Agrawal, R., Srikant, R.: Fast Algorithms for Mining Association Rules. In: 20th International Conference on Very Large Data Bases, pp. 487–499 (1994) 12. Thabtah, F.A., Cowling, P.I.: A greedy classification algorithm based on association rule. Elsevier journal of Applied Soft Computing 07, 1102–1111 (2007) 13. Zemirline, A., Lecornu, L., Solaiman, B., Ech-Cherif, A.: An Efficient Association Rule Mining Algorithm for Classification. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2008. LNCS (LNAI), vol. 5097, pp. 717–728. Springer, Heidelberg (2008) 14. Li, W., Han, J., Pei, J.: CMAR: Accurate and Efficient Classification Based on Multiple Class-Association Rules. In: International Conference on Data Mining, pp. 369–376 (2001)

156


15. Yin, X., Han, J.: CPAR: Classification based on Predictive Association Rules 16. Cosine Similarity Measure, http://www.appliedsoftwaredesign.com/cosineSimilarityCalculat or.php 17. Lee, Y., Chang, K.H.: Reusability and. Maintainability Metrics for Object-Oriented Software. In: ACM 38th Annual Southeast Regional Conference, pp. 88–94 (2000) 18. Yacoub, S., Ammar, H., Robinson, T.: Dynamic Metrics for Object-Oriented Designs. In: IEEE 6th International Symposium Software Metrics, pp. 50–61 (1999) 19. Li, W., Henry, S.: Object Oriented Metrics that predict Maintainability. In: Technical Repot, Virginia Polytechnic Institute and State University (1993) 20. Shiva, S.J., Shala, L.A.: Software Reuse: Research and Practice. In: Proceedings of the IEEE International Conference on Information Technology, pp. 603–609 (2007) 21. Bhatia, P.K., Mann, R.: An Approach to Measure Software Reusability of OO Design. In: Proceedings of the 2nd National Conference on Challenges & Opportunities in Information Technology, pp. 26–30 (2008) 22. Eickhoff, F., Ellis, J., Demurjian, S., Needham, D.: A Reuse Definition, Assessment, and Analysis Framework for UML. In: International Conference on Software Engineering (2003), http://www.engr.uconn.edu/~steve/Cse298300/eickhofficse2003s ubmit.pdf 23. Caldiera, G., Basili, V.R.: Identifying and Qualifying Reusable Software Components. IEEE Journal of Computer 24(2), 61–70 (1991) 24. Henry, S., Lattanzi, M.: Measurement of Software Maintainability and Reusability in the Object Oriented Paradigm. In: ACM Technical Report (1994) 25. Xie, T., Pei, J.: Data mining for Software Engineering, http://ase.csc.ncsu.edu/dmse/dmse.pdf 26. Cosine Similarity, http://en.wikipedia.org/wiki/Cosine_similarity 27. Gupta, V., Chhabra, J.K.: Measurement of Dynamic Metrics Using Dynamic Analysis of Programs. In: Proceedings of the Applied Computing Conference, pp. 81–86 (2008) 28. Michail, A.: Data Mining Library Reuse Patterns in User-Selected Applications. In: 14th IEEE International Conference on Automated Software Engineering, pp. 24–33 (1999) 29. Associations Rule, http://en.wikipedia.org/wiki/Association_rule_learning 30. Jaccard Index, http://en.wikipedia.org/wiki/Jaccard_index

An Algorithm of Constraint Frequent Neighboring Class Sets Mining Based on Separating Support Items Gang Fang, Jiang Xiong, Hong Ying, and Yong-jian Zhao College of Mathematics and Computer Science, Chongqing Three Gorges University Chongqing 404000, P.R. China [email protected], [email protected], [email protected], [email protected]

Abstract. For the reasons that present constraint frequent neighboring class sets mining algorithms need generate candidate frequent neighboring class sets and have a lot of repeated computing, and so this paper proposes an algorithm of constraint frequent neighboring class sets mining based on separating support items, which is suitable for mining frequent neighboring class sets with constraint class set in large spatial database. The algorithm uses the method of separating support items to gain support of neighboring class sets, and uses up search to extract frequent neighboring class sets with constraint class set. In the course of mining frequent neighboring class sets, the algorithm only need scan once database, and it need not generate candidate frequent neighboring class sets with constraint class set. By these methods the algorithm reduces more repeated computing to improve mining efficiency. The result of experiment indicates that the algorithm is faster and more efficient than present mining algorithms when extracting frequent neighboring class sets with constraint class set in large spatial database. Keywords: neighboring class set; constraint class set; separating support items; up search; spatial data mining.

1 Introduction Geographic information databases is an important and typical spatial database, mining spatial association rules from geographic information databases is one important part of spatial data mining and knowledge discovery, which is all known as spatial co-location pattern written in [1]. Spatial co-location pattern are some implicit rules expressing construct and association of spatial objects in geographic information databases, and also expressing hierarchy and correlation of different subsets of spatial association or spatial data in geographic information databases written in [2]. At present, in spatial data mining, there are mainly three kinds methods of mining spatial association rules written in [3], such as, layer covered based on clustering written in [3], mining method based on spatial transaction written in [2, 4, 5 and 6] and mining method based on non-spatial transaction written in [3]. We use the first two methods to extract frequent neighboring class set written in [4, 5 and 6], but AMFNCS written in [4] and TDA written in [5] are not able to efficient extract frequent neighboring Y. Tan et al. (Eds.): ICSI 2011, Part II, LNCS 6729, pp. 157–163, 2011. © Springer-Verlag Berlin Heidelberg 2011

158

G. Fang et al.

class set with constraint class set, and MFNCSWCC written in [6] need generate many candidates and have a lot of repeated computing when it uses iterative search to generate frequent neighboring class set with constraint class set. Hence, this paper proposes an algorithm of constraint frequent neighboring class sets mining based on separating support items, denoted by CMBSSI, which need not generate candidate when mining frequent neighboring class sets with constraint class set.

2 Definition and Problem Description Spatial data set is made up of each spatial object in spatial domain. We use this data structure as to save every object. Here, Class Identification distinguishes from different class in spatial data set, Object Identification distinguishes from different spatial object of the same class, Spatial Location expresses location coordinate of spatial object. Each object is regarded as an instance of corresponding class, and so spatial data set is made up of these instances of spatial Class Identification. Sets of Class Identification are thought as a spatial class set as C = {C1, C2…Cm}, it means there are m different classes. Definition 1. Neighboring Class Set, it is a subset of spatial class set in spatial data set, which is expressed as {Ct1, Ct2…Ctk} (tk ≤ m) denoted by NCS. Let I = {it1, it2… itk} be an instance of neighboring class set as NCS = {Ct1, Ct2…Ctk}, here, itj is an instance of Ctj (j ∈ 1, 2…k). Example, let {V, W, Y} be a NCS, and I = {V3, W1, Y2} is an instance of NCS. Definition 2. Neighboring Class Set Length, its value is the number of class in neighboring class set. If the length of NCS is k, it is expressed as k-NCS. Definition 3. Right Instance of Neighboring Class Set, let I = {it1, it2… itk} be an instance of NCS, if ∀ ip and iq (ip, iq ∈ I), and distance (ip, iq) ≤ d, and then we think I be a right instance of NCS. Here, d is the minimal distance used by deciding two spatial objects are close to each other, distance (ip, iq) expresses Euclidean distance. Definition 4. Constraint Class Set, it is a proper subset of neighboring class set, which is made up of class given by user. Definition 5. Class Weight, it is an integer as 2 Nok −1 , here No k is the sequence number of class in spatial class set as C = {C1, C2…Cm}. Definition 6. Neighboring Class Set Vector, it is a vector, whose each component is Class Weight as 2 Nok −1 in neighboring class set. Example let C = {C1, C2… Cm} be a spatial class set, there is a neighboring class set as {Ct1, Ct2… Ctk}, and then its Vector is expressed as NCSV = ( 2 t1−1 , 2 t2 −1 ... 2 tk −1 ). Definition 7. Neighboring Class Set Identification, it is an integer whose value is the sum of all components of NCSV, denoted by ∑ Lj=1 2 No j −1 , L is Length of NCS. According to above example, Neighboring Class Set Identification is ∑kj =1 2 t j −1 .

An Algorithm of Constraint Frequent Neighboring Class Sets Mining

159

Definition 8. Neighboring Class Set Support, it is the number of right instance of neighboring class set, which is denoted by support (NCS). Definition 9. Frequent Neighboring Class Set, its support is not less than the minimal support given by user. As above knowledge, mining frequent neighboring class set with constraint class set is expressed as follows: Input: (1) Spatial class set as C = {C1, C2… Cm}, instance set as I = {i1, i2… in}, each ik (ik ∈ I) is expressed as above defined data structure. (2) Minimal distance as d. (3) Minimal support as s. (4) Constraint class set. Output: Frequent neighboring class set with constraint class set.

3 Mining Constraint Frequent Neighboring Class Sets 3.1 The Method of Separating Support Items In order to not generate candidate when mining frequent neighboring class sets, the algorithm introduces the method of separating support items. The method is used to separate all items or itemsets from spatial transaction to compute support, namely, extracting all itemsets supported by a spatial transaction to compute support. Let C = {C1, C2…Cm} be a spatial class set, we regard NCS = {Ct1, Ct2…Ctk} as a transaction, and the method is expressed as follows: Step1, according to definition 5 and 6, computing Neighboring Class Set Vector as NCSV = ( 2 t1 −1 , 2 t2 −1 ... 2 tk −1 ). Step2, let every itemsets supported by the NCS be new Neighboring Class Set, and computing index interval as [1,2 k − 1] , we use this interval to generate Neighboring Class Set Identification as NCSIx of these new Neighboring Class Sets. Step3, computing NCSIx = B x ⋅ NCSV T , x ∈ [1,2 k − 1] , component of vector B x is k bit of integer as x. Example C = {U, V, W, X, Y, Z} is a spatial class set, NCS = {V, X, Y}. We use the method of separating support items to extract all itemsets supported by the NCS. Step1, we compute Neighboring Class Set Vector as NCSV = (22-1, 24-1, 25-1) = (2, 8, 16). Step2, we compute index interval as [1,23 − 1] , namely, [1, 7]. Step3, we extract all itemsets supported by the NCS as follows: NCSI1 = B1 ⋅ NCSV T = (0, 0, 1) · (2, 8, 16) T =16, corresponding NCS1= {Y}. NCSI2 = B 2 ⋅ NCSV T = (0, 1, 0) · (2, 8, 16) T =8, corresponding NCS2 = {X}. NCSI3 = B3 ⋅ NCSV T = (0, 1, 1) · (2, 8, 16) T =24, corresponding NCS3 = {X, Y}. NCSI4 = B4 ⋅ NCSV T = (1, 0, 0) · (2, 8, 16) T =2, corresponding NCS4 = {V}. NCSI5 = B5 ⋅ NCSV T = (1, 0, 1) · (2, 8, 16) T =18, corresponding NCS5 = {V, Y}.

160

G. Fang et al.

NCSI6 = B6 ⋅ NCSV T = (1, 1, 0) · (2, 8, 16) T =10, corresponding NCS6 = {V, X}. NCSI7 = B7 ⋅ NCSV T = (1, 1, 1) · (2, 8, 16) T =26, corresponding NCS7 = {V, X, Y}. Obviously, we use the method to gain all NCSk supported by NCS = {V, X, Y}, and so compute once support of all NCSk supported by NCS = {V, X, Y}. 3.2 The Process of Mining Constraint Frequent Neighboring Class Sets Input: (1) Spatial class set as C = {C1, C2… Cm}, instance set as I = {i1, i2… in}, each ik (ik ∈ I) is expressed as above defined data structure. (2) Minimal distance as d. (3) Minimal support as s. (4) Constraint class set. Output: Frequent neighboring class set with constraint class set. The algorithm uses these data structure as follows: Structure neighboring class set { Char ncs; // saving neighboring class sets as NCS Int count; // saving the number of right instance belong to NCS} NCS Array [2m-1]; // saving the number of similar neighboring class set, namely, support of neighboring class set F [2m-l-1]; // saving frequent neighboring class set, l is length of constraint class set Step1, Computing the entire right instance as I’ from instance set as I by the minimal distance as d. Step2, Gaining neighboring class set as NCS after scanning once right instance set as I’. Step3, Scanning NCS[j] (j=0), according to NCS[j].ncs, computing Neighboring Class Set Length as Lengthj and Neighboring Class Set Vector as NCSVj. Step4, via the method of separating support items in chapter 3.1, the algorithm computes Neighboring Class Set Identification as NCSIx of every NCSx separated from NCS[j].ncs, and further executes the operation as follows: Array [NCSIx -1] = Array [NCSIx -1] + NCS [j].count. Step5: j = j + 1, and repeated executing these operations form step3 to step4. Step6: Scanning Array [p - 1] by up search, namely, p is ascending, if Array [p - 1] > s and p c = c (here c is Neighboring Class Set Identification of constraint class set, is logical “and” operation), and writing p to F after deleting q (p q = q, q ∈ F ). Step7: Outputting frequent neighboring class set with constraint class set from F according to definition 5 and 7.

∧

∧

∧

4 The Analysis and Comparing of Capability At present, there are little documents of research frequent neighboring class set. AMFNCS written in [4] uses numerical variable to generate candidate, this method is simple, and uses logic operation to compute support, and this is also very simple. In mining process the algorithm hasn’t superfluous NCS, but it isn’t able to extract frequent neighboring class set with constraint class set.


161

TDA written in [5] adopts top-down strategy to generate candidate frequent neighboring class set, which is made of three stages, firstly, computing the 1st mcandidate frequent neighboring class set which contains all classes, and then generating (m-1)-candidate frequent neighboring class set, let (m-1) be k, and generating all (k-1)-frequent neighboring class set (k>3) by iteration. But it isn’t also able to extract frequent neighboring class set with constraint class set. MFNCSWCC written in [6] via iterative search efficiently extracts short frequent neighboring class set with constraint class set, but it need generate many candidates and have a lot of repeated computing to restrict the efficiency. CMBSSI presented by this paper need not generate candidate to extract frequent neighboring class set with constraint class set, which is suitable for mining short frequent neighboring class set with constraint class se. But it is able to reduce many candidates frequent neighboring class sets and more repeated computing to improve the efficiency. 4.1 The Analysis of Capability Let C = {C1, C2…Cm} be a spatial class set, and let I = {i1, i2…in} be an instance set, let nk (n=∑nk) be the number of instance of Ck, The length of constraint class set is l. Time complexity. Computing of CMBSSI mainly includes three parts which are expressed as computing right instance, separating support items of NCS and search frequent NCS. Time complexity is expressed as follows: ( 2 m −l − 1)[n 2 C m2 / m 2 + 2 m −l −1 − 1] .

Space complexity. Space complexity of CMBSSI is expressed as O ( α ⋅ 2 m ), α is parameter about support and length of constraint class set. If right instances in spatial data set are not uniform distribution, space utilization ratio of CMBSSI is too low. 4.2 The Comparing of Experimental Result Now we use result of experiment to testify above analyses. Two algorithms MFNCSWCC and CMBSSI are used to generate frequent neighboring class set with constraint class set from 12267 right instances, whose Neighboring Class Set Identification are from 3 to 8191, neighboring class set does not include any simple class, namely, it has two classes at least, the number of spatial class is 13, Neighboring Class Set Identification of constraint class set is 9, the number of right instance included by these neighboring class set observe the discipline which is expressed as follows: Neighboring Class Set Identification as integer 8191 has one right instance. Neighboring Class Set Identification as integer 8190 has two right instances. Neighboring Class Set Identification as integer 8189 has one right instance. Neighboring Class Set Identification as integer 8188 has two right instances. ... Our experimental circumstances are expressed as follow: Intel(R) Celeron(R) M CPU 420 1.60 GHz, 1.24G, language of the procedure is Visual C# 2005.NET, OS is Windows XP Professional.

＠

162

G. Fang et al.

Runtime(ms) )

50000 40000

MFNCSWCC

30000

CMBSSI

20000 10000 0 8.15

4.08

1.63

0.82

0.41

0.25

0.16

0.08

Support(%)

Fig. 1. Comparing the runtime of two algorithms as support changes

Runtime(ms) )

50000 40000

MFNCSWCC

30000

CMBSSI

20000 10000 0 3

4

5

6

7

8

9

10

Length

Fig. 2. Comparing the runtime of two algorithms as length changes The runtime of two algorithms is expressed as Figure 1 as support of neighboring class set changes, and the runtime of two algorithms is expressed as Figure 2 as length of neighboring class set changes. According to these two figures, we knew that CMBSSI is faster and more efficient than MFNCSWCC when mining frequent neighboring class set with constraint class set in large spatial database.

5 Conclusion This paper proposes an algorithm of constraint frequent neighboring class sets mining based on separating support items, which is suitable for mining frequent neighboring class sets with constraint class set in large spatial database. In the future, we need further discuss how to improve space utilization ratio. Acknowledgments. This work was fully supported by science and technology research projects of Chongqing Education Commission (Project No. KJ091108), and it was also supported by science and technology research projects of Wanzhou District Science and Technology Committee (Project No. 2010-23-01) and Chongqing Three Gorges University (Project No. 10QN-22, 24 and 30).


163

References 1. Ma, R.H., Pu, Y.X., Ma, X.D.: GIS Spatial Association Pattern Ming. Science Press, Beijing (2007) 2. Ma, R.H., He, Z.Y.: Mining Complete and Correct Frequent Neighboring Class Sets from Spatial Databases. Journal of Geomatics and Information Science of Wuhan University 32(2), 112–114 (2007) 3. Zhang, X.W., Su, F.Z., Shi, Y.S., Zhang, D.D.: Research on Progress of Spatial Association Rule Mining. Journal of Progress in Geography 26(6), 119–128 (2007) 4. Fang, G.: An algorithm of alternately mining frequent neighboring class set. In: Tan, Y., Shi, Y., Tan, K.C. (eds.) ICSI 2010. LNCS, vol. 6146, pp. 588–593. Springer, Heidelberg (2010) 5. Fang, G., Tu, C.S., Xiong, J., et al.: The Application of a Top-Down Algorithm in Neighboring Class Set Mining. In: International Conference on Intelligent Systems and Knowledge Engineering, pp. 234–237. IEEE press, Los Alamitos (2010) 6. Fang, G., Xiong, J., Chen, X.F.: Frequent Neighboring Class Set Mining with Constraint Condition. In: International Conference on Progress in Informatics and Computing, pp. 242– 245. IEEE press, Los Alamitos (2010)

A Multi-period Stochastic Production Planning and Sourcing Problem with Discrete Demand Distribution Weili Chen, Yankui Liu , and Xiaoli Wu College of Mathematics & Computer Science, Hebei University Baoding 071002, Hebei, China [email protected], [email protected], [email protected]

Abstract. This paper studies a new class of multi-period stochastic production planning and sourcing problem with minimum risk criteria, in which a manufacturer has a number of plants or subcontractors and has to meet the product demands according to the service levels set by its customers. In the proposed problem, demands are characterized by stochastic variables with known probability distributions. The objective of the problem is to minimize the probability that the total cost exceeds a predetermined maximum allowable cost, where the total cost includes the sum of the inventory holding, setup and production costs in the planning horizon. For general demand distributions, the proposed problem is very complex, so we cannot solve it by conventional optimization methods. To avoid this difficulty, we assume the demands have finite discrete distributions, and derive the crisp equivalent forms of both probability objective function and the probability level constraints. As a consequence, we turn the original stochastic production planning problem into its equivalent integer programming one so that the branch-and-bound method can be used to solve it. Finally, to demonstrate the developed modeling idea, we perform some numerical experiments via one 3-product source, 8-period production planning problem. Keywords: Stochastic production planning, Minimum risk criteria; Probability service level, Integer programming.

1

Introduction

Production planning and the sourcing problem is viewed as the manufacturer’s decision variables that are how much to produce, when to produce, where to produce, and how much inventory to carry in each period. In the literature, the deterministic production planning problem has received much attention [1,2,3,4,5]. However, due to the uncertainty usually presents in a complex decision system, the stochastic production planning problems has also been studied widely in the field of production planning management. For example, Bitran and Yanasse [6]



A Multi-period Stochastic Production Planning and Sourcing Problem

165

dealt with a stochastic production planning problem with a service level requirement, and provided non-sequential and deterministic equivalent formulations of the model; Z¨ apfel [7] claimed that MRP II systems can be inadequate for the solution of production planning problems with uncertain demand because of the insufficiently supported aggregation process, and proposed a procedure to generate an aggregate plan and a consistent disaggregate plan for the master production schedule, and Kelly et al. [8] considered randomness in demand for a single-product, single-machine line with setups in the process industry, and proposed a model that incorporates mean and standard deviation of demand in the planning horizon time periods to set production runs. Though only one product was being made, start-ups after periods of idleness required significant setups. On the basis of fuzzy theory, the production planning problems have also been studied in fuzzy community. In this respect, the interested reader may refer to Lan et al. [9,10], and Sun et al. [11,12]. The purpose of this paper is to study a realistic production planning model. We consider production, setup, and inventory carrying costs and minimum service level constraints at each time period, in which the demands are stochastic with known probability distributions. Most of the stochastic production planning models in the literature may formulate the model to minimize the expected sum of all costs [6,8,13]. In the current development, we minimize the probability that the total cost exceeds a predetermined maximum allowable cost, where the total cost includes the sum of the inventory holding, setup and production costs in the planning horizon. For general demand distributions, the proposed problem is very complex, so we cannot solve it by conventional optimization methods. To avoid this difficulty, we assume the demands have finite discrete distributions, and derive the crisp equivalent forms of both probabilistic objective function and the probability level constraints. As a consequence, the proposed production planning problem is turned into its equivalent integer programming problem. Since there is no “one-size-fits-all” solution that is effective for all integer programming problems, we adopt the branch and bound method to solve our equivalent integer production planning problem. The rest of this paper is organized as follows. In Section 2, we formulate a new class of stochastic production planning models with probability objective subject to service level constraints. In Section 3, we assume the demands have finite discrete probability distributions, and deal with the equivalent formulation of original stochastic production planning problem. Section 4 is devoted to the discussion of the branch and bound solution method for the equivalent integer production planning problem. Section 5 performs some numerical experiments via one 3-product source, 8-period production planning problem to demonstrate the developed modeling idea. Finally, we draw our conclusions in Section 6.

2

Formulation of Problem

In this section, we will develop a new class of stochastic minimum risk programming models for a multi-period production planning and sourcing problem. Assume that there is a single product and N types of production sources (plants

166

W. Chen, Y. Liu, and X. Wu

and subcontractors). The demand for this specific product in each period is characterized by a random variable with known probability distribution. The costs in the objective function consist of production cost, inventory holding cost and setup cost. The objective of the problem is to minimize the probability that the total cost exceed a predetermined maximum allowable cost. Constraints on the performance (related to backorders) of the system are imposed by requiring service levels which forces the probability of having no stock out to be greater than or equal to a service level requirement in each period. In addition, we adopt the following notation to model our production planning problem: i is the index of sources, i = 1, 2, . . . , N ; t is the index of periods, t = 1, 2, . . . , T ; cit is the unit cost of production at source i in period t; ht is the unit cost of inventories in period t; I0 is the initial inventory; It is the inventory level at the end of period t; sit is the fixed cost of setup at source i in period t; yit is 1 if a setup is performed at source i in period t, and 0 otherwise; Mit is the capacity limitation of source i at time period t; dt is the stochastic demand in period t; αt is the service level requirement in period t; ϕ is the maximum allowable cost, and xit is the production quantities at source i in period t. Using the notation above, a minimum-risk stochastic production planning model with probability service levels is formally built as ⎧ T N ⎪ min Pr{ t=1 (ht (It )+ + i=1 (sit yit + cit xit )) > ϕ} ⎪ ⎨ s.t.: Pr{It ≥ 0} ≥ αt , t = 1, 2, . . . , T (1) ⎪ xit ≤ Mit yit , i = 1, 2, . . . , N, t = 1, 2, . . . , T ⎪ ⎩ n xit ∈ Z+ , yit ∈ {0, 1}, i = 1, 2, . . . , N, t = 1, 2, . . . , T, where (It )+ = max{0, It }, t = 1, 2, . . . , T , are the real inventory levels. For each period t the inventory balance is It = It−1 +

N i=1

xit − dt = I0 +

t N τ =1 i=1

xiτ −

t

dτ ,

(2)

τ =1

where the set of demand quantities {dt , t = 1, 2, . . . , T } are assumed to be mutually independent random variables. If demands dt , t = 1, 2, . . . , T , have general probability distributions, then stochastic integer production planning problem (1) is very complex, so we cannot solve it by conventional optimization methods. To find efficient solution method for problem (1), we assume that the demands have finite discrete distributions, and turn the original problem (1) into its equivalent integer programming problem. This issue will be addressed in the next section.

3

Handing Probability Objective and Level Constraints

In this section, we discuss the equivalent deterministic production planning model of original problem (1). For this purpose, we assume that the discrete distribution demand d is characterized by


d = d1 . . . dT

T

∼

dˆ1 dˆ2 · · · dˆK p1 p2 · · · pK

167

,

(3)

where dˆk = (dˆk1 , dˆk2 , · · · , dˆkT ) is the kth realization of demand d during T periods, pk > 0 for all k such that K k=1 pk = 1. In this case, the tth probability level constraint

t t N Pr I0 + xiτ − d τ ≥ 0 ≥ αt (4) τ =1 i=1

τ =1

can be turned into the following equivalent deterministic form I0 +

t N τ =1 i=1

t where Q−

t xiτ ≥ Q−

τ =1

dτ

(αt ),

(5)

(αt ) is the left end-point of the closed interval of αt -quantiles of probability distribution F tτ =1 dτ (t) of random demand tτ =1 dτ . Furthermore, we define a binary vector z whose components zk , k ∈ K, take 1 if the corresponding set of constraints has to be satisfied and 0 otherwise. In particular, for each scenario k, we may introduce a number M large enough so that the following inequality holds τ =1

T t=1

dτ

(ht (I0 +

N t τ =1 i=1

xiτ −

t τ =1

dˆkτ )+ ) − M zk ≤ ϕ −

T N

(sit yit + cit xit ). (6)

i=1 t=1

As a consequence, the original stochastic integer production planning problem (1) is equivalent to the following nonlinear integer programming problem: ⎧ K pk zk ⎪ ⎪ min k=1 N ⎪ t − ⎪ ⎪ s.t. (I + 0 ⎪ τ =1 i=1 xit ) ≥ Q tτ =1 dτ (αt ), t = 1, 2, . . . , T ⎪ ⎪ ⎪ ⎨ xit ≤ Mit yit , i = 1, 2, . . . , N, t = 1, 2, . . . , T t N T xi,τ − tτ =1 dτ )+ − M zk t=1 ht (I0 + τ =1 i=1 ⎪ N T ⎪ ⎪ ⎪ ≤ ϕ − i=1 t=1 (sit yit + cit xit ), k = 1, 2, . . . , K ⎪ ⎪ ⎪ ⎪ zk ∈ {0, 1}, k = 1, . . . , K ⎪ ⎩ n xit ∈ Z+ , yit ∈ {0, 1}, i = 1, 2, . . . , N, t = 1, 2, . . . , T.

(7)

By introducing auxiliary variables ltk , k = 1, 2, . . . , K, t = 1, 2, . . . , T , we can turn problem (7) into the following integer linear programming problem: ⎧ min K p k zk ⎪ k=1 ⎪ N ⎪ t − ⎪ ⎪ s.t.: (I + 0 ⎪ τ =1 i=1 xit ) ≥ Q tτ =1 dτ (αt ), t = 1, 2, . . . , T ⎪ ⎪ ⎪ ⎪ xit ≤ Mit yit , i = 1, 2, . . . , N, t = 1, 2, . . . , T ⎪ ⎨ N T T ht ltk − M zk ≤ ϕ − i=1 t=1 (sit yit + cit xit ), k = 1, . . . , K (8) t=1 t N t ⎪ ⎪ I0 + τ =1 i=1 xi,τ − τ =1 dˆkτ ≤ ltk , t = 1, 2, . . . , T, k = 1, . . . , K ⎪ ⎪ ⎪ k ⎪ lt ≥ 0, t = 1, 2, . . . , T, k = 1, 2, . . . , K ⎪ ⎪ ⎪ ⎪ z k ∈ {0, 1}, k = 1, 2, . . . , K ⎪ ⎩ n xit ∈ Z+ , yit ∈ {0, 1}, i = 1, 2, . . . , N, t = 1, 2, . . . , T.

168


Furthermore, we have the following result: Theorem 1. Problem (7) and problem (8) are equivalent. Proof. If (¯ xit , y¯it , z¯k , i = 1, . . . , N, t = 1, . . . , T, k = 1, . . . , K) is a feasible solution to (7), then, for all t and k, we take ltk

= (I0 +

N t τ =1 i=1

xiτ −

t

dˆkτ )+ .

(9)

τ =1

Hence, (¯ xit , y¯it , z¯k , ¯ltk , i = 1, . . . , N, t = 1, . . . , T, k = 1, . . . , K) is a feasible solution to (8) with equal objective value. Conversely, let (ˆ xit , yît , zˆk , ˆltk , i = 1, . . . , N, t = 1, . . . , T, k = 1, . . . , K) be a feasible solution to (8). Then, for each k, the following inequality holds T t=1

ht (I0 +

N t τ =1 i=1

xîτ −

t τ =1

dˆkτ )+ − M zˆk ≤

T

ht ˆltk − M zˆk .

(10)

t=1

Consequently, (ˆ xit , yît , zˆk , i = 1, . . . , N, t = 1, . . . , T, k = 1, . . . , K) is a feasible solution to (7), and the corresponding objective value in (7) equals to the one in (8). The proof of the theorem is complete. From the reformulation of production planning problem (7), we can see that even for a small size of the random vector, the number of K can be very large. In addition, problem (8) consists of integer and binary decision variables. Thus, problem (8) belongs to the class of NP-hard problems. In the next section, we discuss the solution method of (8) by a general purpose optimization software.

4

Solution Method

The equivalent production planning problem (8) is an integer programming one that may be solved by a pure enumeration scheme. However, the scheme would not allow to compute large size of the random vector. Standard branch and bound scheme uses enumeration ingeniously and it is considered the classical method to solve the purely integer and the mixed-integer programming problem, and it is one of the successful methods to solve this kind of programming problems at present. All commercially available integer programming software packages employ a linear programming based branch and bound scheme. In order to use integer programming software packages effectively, it is required to understand the use of lower and upper bounds on the optimal objective value in an linear programming based branch-and-bound algorithm. For a comprehensive exposition on integer-programming algorithms we may refer to Nemhauser and Wolsey [14], and Wolsey [15].


169

There is no “one-size-fits-all” solution method that is effective for all integer programming problems. Therefore, to handle situations in which the default settings do not achieve the desired performance, integer-programming systems allow users to change the parameter settings, and thus the behavior and performance of the optimizer. By Lingo software, we employ the standard branch and bound algorithm to solve the equivalent production planning problem (8).

5

Numerical Experiments

In this section, we perform some numerical experiments via the following example. A manufacturer supplies his products to a retailer, suppose that the manufacturer has three product sources, N = 3, and eight production periods, T = 8. Each plant and subcontractor has different setup cost, product capacity and unit production cost. Suppose sit , Mit , cit , ht , αt , ϕ are all predetermined by the actual situation. The manufacturer has to meet the demands for different products according to the service level requirements set by its customers. Let Table 1. The Data Set of Production Planning Problem sit Periods source 1 2 3 Mit Periods source 1 2 3 cit Periods source 1 2 3 dt Periods p p Periods ht αt

1

2

3

4

5

6

7

8

1500 1200 2500

1450 1280 2000

2000 1300 1880

1600 1850 1600

1200 1600 1980

1250 1650 1500

2200 1480 1660

1800 2000 1750

1

2

3

4

5

6

7

8

5000 6000 6500

4000 5500 6500

4500 5500 5500

4500 4500 4000

4500 4800 4000

4800 3800 3800

5000 4000 3800

5000 4000 3500

1

2

3

4

5

6

7

8

2 2.5 3

3 3 3.5

2.5 3 2

2.5 4 2.5

3.5 4.5 2.2

2.5 1.6 2.8

2.5 3 5

2.5 1.8 3.5

1 3800 0.4 3290 0.6 1 4 0.95

2 3760 0.3 4300 0.7 2 5 0.8

3 4800 0.5 5200 0.5 3 5.5 0.9

4 4500 0.45 5000 0.55 4 4 0.92

5 4890 0.35 6100 0.65 5 4.5 0.88

6 3200 0.6 5740 0.4 6 3 0.9

7 3450 0.55 4880 0.45 7 3.5 0.92

8 3990 0.2 4100 0.8 8 6 0.95

170

W. Chen, Y. Liu, and X. Wu Table 2. The Optimal Solution of Production Planning xit 1 2 3

1 3800 0 0

2 0 4100 0

3 0 0 5500

4 4500 0 0

5 2100 0 4000

6 1830 3800 0

7 4590 0 0

8 0 4000 0

us assume that the demand dt has a finite integer discrete distribution and it is meaningful when the products are not indivisibly. We assume, for the sake of simplicity, that the initial inventory level is 0, I0 =0, and the data used for this test are collected in Table 1. Due to T = 8 and each period demand has two realizations, one has K = 256. Let ϕ = 1.8 × 105 , and M = 106 . We employ the Lingo 8.0 to solve the equivalent production planning problem (8). The obtained optimal suction of the production planning problem is reported in Table 2, and the corresponding optimal value is 0.1412070. From Table 2 we get the production quantities at each source in each period. The production quantity is nonzero as the binary variables yit = 1. From the numerical experiment, we can see that even for a small size of the random vector, the number of K can be very large, and because of introducing auxiliary variables, the scale of this numerical example is also rather large. Furthermore, more numerical experiments for this example have been performed with different values of parameter ϕ. Figure 1 shows that how the optimal objective value varies with the predetermined maximum allowable cost ϕ. Lower values of ϕ allow bigger probability that the total cost exceeds the maximum allowable cost. Nevertheless, the choice of ϕ is up to the capability of a decision maker. In reality life, a manufacturer who has lower acceptable costs may suffer higher risk than the ones who have higher acceptable costs. So the manufacturer should make a decision according to the relationship between an acceptable maximum cost and the suffered risk.

6

Conclusions

When optimal production decisions must be reached in a stochastic environment, in order to give to the optimization problem its appropriate form, the formulation of the decision model requires a deeper probing of the aspirations criteria. In addition, the computational obstacles should be overcome to find optimal production decisions. In these two aspects, the major new contributions of the current development are as follows. (i) On the basis of minimum risk criteria, we have presented a new class of stochastic production planning problem with probability objective subject to service level constraints, in which product demands are characterized by random variables. In addition, a manufacturer has a number of plants and subcontractors and has to meet the product demands according to various service levels prescribed by its customers.


171

1 0.9 0.8 0.7 0.6

risk

0.5 0.4 0.3 0.2 0.1 0 0.8

1

1.2

1.4

1.6

maximum allowable cost

1.8

2 5

x 10

Fig. 1. Trade-off between Maximum Allowable Cost and Risk

(ii) For general demand distributions, the developed stochastic production planning problem (1) is very complex, so we cannot solve it by conventional optimization methods. So, we assumed the demands have finite discrete distributions, and derived the crisp equivalent forms of both probability objective function and the probabilistic level constraints. As a consequence, we turn the original production planning problem (1) into its equivalent integer programming model (7) so that the branch-and-bound method can be used to solve it. The equivalent alternative formulation about integer production planning problem (7) has also been discussed (see Proposition 1). (iii) To demonstrate the developed modeling idea, a number of numerical experiments has been performed via one numerical example with three product sources and eight production periods. By changing the value of parameter ϕ, we get the trade-off between an acceptable maximum cost and the suffered risk (see Figure 1). This relationship is considered as a guidance for investment that is meaningful in reality production processing. Acknowledgments This work was supported by the National Natural Science Foundation of China under Grant No.60974134, the Natural Science Foundation of Hebei Province under Grant No.A2011201007, and the Education Department of Hebei Province under Grant No.2010109.

172


References 1. Candea, D., Hax, A.C.: Production and Inventory Management. Prentice-Hall, New Jersey (1984) 2. Das, S.K., Subhash, C.S.: Integrated Approach to Solving the Master Aggregate Scheduling Problem. Int. J. Prod. Econ. 32(2), 167–178 (1994) 3. Dzielinski, B.P., Gomory, R.E.: Optimal Programming of Lot Sizes, Inventory and Labor Allocations. Manag. Sci. 11, 874–890 (1965) 4. Florian, M., Klein, M.: Deterministic Production Planning with Concave Costs and Capacity Constraints. Manag. Sci. 18, 12–20 (1971) 5. Lasdon, L.S., Terjung, R.C.: An Efficient Algirithm for Multi-Echelon Scheduling. Oper. Res. 19, 946–969 (1971) 6. Bitran, G.R., Yanasee, H.H.: Deterministic Approximations to Stochastic Production Problems. Oper. Res. 32(5), 999–1018 (1984) 7. Z¨ afel, G.: Production Planning in the Case of Uncertain Individual Demand Extension for an MRP II Concept. Int. J. Prod. Econ. 119, 153–164 (1996) 8. Kelly, P., Clendenen, G., Dardeau, P.: Economic Lot Scheduling Heuristic for Random Demand. Int. J. Prod. Econ. 35(1-3), 337–342 (1994) 9. Lan, Y., Liu, Y., Sun, G.: Modeling Fuzzy Multi-Period Production Planning and Sourcing Problem with Credibility Service Levels. J. Comput. Appl. Math. 231(1), 208–221 (2009) 10. Lan, Y., Liu, Y., Sun, G.: An Approximation-Based Approach for Fuzzy MultiPeriod Production Planning Problem with Credibility Objective. Appl. Math. Model. 34(11), 3202–3215 (2010) 11. Sun, G., Liu, Y., Lan, Y.: Optimizing Material Procurement Planning Problem by Two-Stage Fuzzy Programming. Comput. Ind. Eng. 58(1), 97–107 (2010) 12. Sun, G., Liu, Y., Lan, Y.: Fuzzy Two-Stage Material Procurement Planning Problem. J. Intell. Manuf. 22(2), 319–331 (2011) 13. Yıldırım, I., Tan, B., Karaesmen, F.: A Multiperiod Stochastic Production Planning and Sourcing Problem with Service Level Constraints. OR Spectrum 27(2-3), 471–489 (2005) 14. Nemhauser, G.L., Wolsey, L.A.: Integer and Combinatorial Optimization. John Wiley & Sons, New York (1988) 15. Wolsey, L.A.: Integer Programming. John Wiley & Sons, New York (1998)

Exploration of Rough Sets Analysis in Real-World Examination Timetabling Problem Instances J. Joshua Thomas, Ahamad Tajudin Khader, Bahari Belaton, and Amy Leow School of Computer Sciences, University Sains Malaysia & KDU College Penang [email protected], {tajudin,bahari}@cs.usm.my

Abstract. The examination timetabling problem is widely studied and a major activity for academic institutions. In real world cases, an increasing number of student enrolments, variety of courses throw in the growing challenge in the research with a wider range of constraints. Many optimization problems are concerned with the best feasible solution with minimum execution time of algorithms. The aim of this paper is to propose rough sets methods to investigate the Carter datasets. Two rough sets (RS) approaches are used for the data analysis. Firstly, the discretization process (DP) returns a partition of the value sets into intervals. Secondly the rough sets Boolean reasoning (RSBR) achieves the best decision table on the large data instances. The rough sets classified datasets are experimented with an examination scheduler. The improvements of the solutions on Car-s-91 and Car-f-91 datasets are reported. Keywords: Examination Timetabling, Rough sets, discretization.

1 Introduction Examination timetabling is a problem of allocating a timeslot for all exams in the problem instances within a limited number of permitted timeslots, in such a way that none of the specified hard constraints are violated. In most cases, the problem is highly constrained and, moreover, the set of constraints which are required to be satisfied is different from one institution to another as reported by Burke et al. [1]. In general, the most common hard constraint is to avoid any student being scheduled for two different exams at the same time. In practice, each institution usually has a different way of evaluating the quality of the developed timetable. In many cases, the quality is calculated based on a penalty function which represents the degree to which the constraints are satisfied. Over the years, numerous approaches have been investigated and developed for exam timetabling. Such approaches include constraint programming, graph colouring, and various metaheuristic approaches including genetic algorithms, tabu search, simulated annealing, the great deluge algorithm, and hybridized methods which draw on two or more of these techniques. Some recent important papers which reflect this broad range of activity are [2, 3, 4, 5]. The earlier work by authors focus on interaction on the scheduling data and, this is a continuous research on the data analysis on the same problem. These approaches can be found in paper [6]. Y. Tan et al. (Eds.): ICSI 2011, Part II, LNCS 6729, pp. 173–182, 2011. © Springer-Verlag Berlin Heidelberg 2011

174

J. Joshua Thomas et al.

In examination timetabling problem the exams are ordered prior to assignment to a timeslot, have been discussed by several authors including Boizulmault et al. [8], Brailsford et al. [9], Burke et al. [10], Burke and Newall [11], Burke and Petrovic [12] and Carter et al. [16]. Carter et al. [16] reports the use of four ordering criteria to rank the exams in decreasing order to estimate how difficult it is to schedule each of the exams. Each one of these techniques has its own properties and features including their ability of finding important rules and information that could be useful for the examination timetabling domain. However, no literature is discussed on the rough sets based methodology to address the problem instances. Rough set theory [17,18,19] is a comparatively innovative intelligent technique that has been applied to the real-world cases, and is used for the discovery of data dependencies, discovers the patterns of data, and seeks the minimum subset of values. One advantage of the rough set is the creation of readable if-then rules. Such rules have a potential to reveal new patterns in the data material. More advanced and intelligent techniques have been used in data analysis such as neural network, Bayesian classifier, genetic algorithms, decision trees, fuzzy theory, and rough set however rough set methods are not popular yet used for scheduling datasets. It offers a problem-solving tool between the precision of classical mathematics and the natural vagueness of the real world. Other approaches like case-based reasoning and decision trees [20,21] are also widely used to solve data analysis problems. The objective of the investigation has been to develop intervals that could rank the dataset on the basis of rough set discretization process and make decision. The newly created rough sets based dataset will then inject to the examination evaluator to generate quality feasible solution. The structure of the paper is as follows. Section 2 discusses the rough sets data analysis method in detail. The characteristics of the benchmark dataset are presented in section 3. The modeling process based on rough sets is briefly described in section 4. Experimental analysis and Results are in Section 5. Finally, the conclusion is presented in Section 6.

2 Rough Sets Methods Rough sets methods is to analyze datasets of the Carter, Laporte and Lee [15, 16] with a set of 12 real-world exam timetabling problems from 3 Canadian highs schools, 5 Canadian, 1 American, 1 UK and 1 mid-east universities. The methods used in this study consist of two main stages: Discretization process (DP) and Rough sets Boolean reasoning (RSBR) data processing. Preprocessing stage (DP) includes discretization. Data Processing (RSBR) includes the generation of preliminary knowledge, such as computation of object ranking from data, and the classification processes. The final goal is of generating rules from the information or decision system for the benchmark datasets. Figure 1 shows the overall steps in the proposed rough sets data analysis methods.

Exploration of Rough Sets Analysis in Real-World Examination

175

Fig. 1. Rough Set Data Analysis method

3 Dataset Many researchers are using the benchmarking Carter dataset [16] to applying the methods and test the results with quality feasible solution. There are two standard dataset used by the examination timetabling community Carter dataset and the ITC(International Timetabling Competition) dataset[ 14]. Everybody in the scientific community uses in order to test any proposed algorithms. The Carter dataset were introduced in 1996 by Carter, Laporte and Lee in a paper published in the Journal of the Operational Research Society. One of the major drawbacks of most articles in the timetabling literature is that testing is limited to randomly generated problems and perhaps to one practical example. The formulation for the Carter dataset as follows: ─ The room capacities for the examination rooms (which is why is it considered an uncapacitated problem). ─ The fact that two consecutive examinations, which are on different days, is better than two consecutive examinations on the same day. Both of these scenarios would give the same penalty cost using the usual objective function used in the Carter dataset, even though the student would have the evening (indeed, all night) to revise, as opposed to no time at all if the examinations were truly consecutive. Indeed, each instance in the dataset just has a number of timeslots. There is concept of different days.

176

J. Joshua Thomas et al. Table 1. Carter Examination Timetabling problem instances

The recent examination timetabling review paper [13] has explained the two versions of the datasets and the modifications. However, the contributions of the works are not with the data values, but on the problem instances. Few works were done on modifying with respective of real-world scenario provided by the institutions. Table 1 shows Carter dataset with the problem instances.

4 Pre-processing In real world examination timetabling, many decisions are required to take into account several factors simultaneously under various sets of constraints (soft constraints). Usually it is not known which parameter(s) need to be emphasized more in order to generate a better solution or decision. In many cases, a tradeoff between the various potentially conflicts on assignment of exams into timeslots. Rough sets usually employ a dataset is represented as a table, where each row represents an object. Every column represents a variable, an observation that can be evaluated for each object. This table is called an information system. The following ordering criteria were considered when selecting which exam should be scheduled first: ─ Number of conflict exams, largest degree (LD) ─ Number of student enrolled, largest enrollment (LE) ─ Number of available slot, saturation degree (SD) In each case, two out of the three criteria above were selected as input variables. More formally, a pair U, A ,where U is a non-empty finite set of objects called the universe and A is a non-empty finite set of attributes such that : for every . The set is called the value set of a.


177

Table 2. Sample Information system

Table 3. Decision Table on the car-s-91 data instances

An example of simple information system is shown in Table 2. There are 12 cases or variable objects, and two conditions (Course and Enrollment). The cases of variable objects x4 and x5 as well as x10 and x12 have exactly the same values of conditions. A decision system is any information system of the form U, A d , where is the decision conditions or criteria’s. A small example decision table can be found in Table 3. The table has the same 12 cases or variable objects as in the previous example, but one decision attribute number of student enrollment (LE) with two three possible outcomes has been added. The reader may again notice that cases x4 and x5 as well as x10 and x12 still have exactly same values of conditions, but the second pair has a different outcome. The definition to be synthesized from the decision tables is in of the rule form: IF Course = 0004 and Enrollment = 73 then LE = Medium It is assumed that a decision table expresses all the knowledge about the model. Same objects may be represented several times or objects overflows. The notion of equivalence must be considered first. A binary relation (RB) R which is reflexive, a value is in relation with itself (x RB x), symmetric (if x RB y then y RB x) and transitive if (x RB x and yRB z then xRB z) is called an equivalence relation. Let , be an information system, then with any there is associated an equivalence relation

178


=

|

,

(1)

is called the indiscernibility relation. For instance Table. 2 define an indiscernibility relation. The subsets of the conditional attributes are [Course], [Enrollment]. If for instance, [No of students Enrollment (LE)] only, objects x4 and x5 belongs to the same equivalence class and defines three equivalence class indiscernible. We look at the relation identified below. = {{ x1,}, { x2,}, { x3,}, { x4,}, { x5,}, { x6,}, { x7,}, { x8,}, { x9,}, { x10,}, { x11,}, { x12,}} = {{ x4 , x5 }, { x10, x12}} = {{ x1, x2}, { x3, x6, x7, x10, x11, x12}, { x4, x5}, { x8, x9}} 4.1 Data Completion and Discretization of Data Values The rough set approach requires only indiscernibility it is not necessary to define an order or distance when the values of different kinds are combined (e.g. courses, enrollment). The discretization step determines how roughly the data to be processed. We called this as “pre-processing”. For instance, course or enrollment values have to establish cut-off points. The intervals might be refined based on good domain knowledge. Setting the cut-off points are computationally expensive for the large datasets and that domain expert to prepare discretization manually. Let be an information system with n objects. The discernibility matrix of is as given below. Each entry thus consists of symmetric n matrix with entries the set of values upon which objects and differ. |

} for i, j = 1,….,n

for an information system The discrenibility function m Boolean variables (a1………..am). ……, The discernibility function is , ,

|1 , , ,

, , ,

is a Boolean function ,

,

(2)

0

,

,

, , , Where “,” stands for disjunction in the Boolean expression, after simplification, the , , , where (e denotes enrollment and r denotes rank the function is data values). 4.2 Data Processing Processing stage includes generating knowledge, such as computation of object from data, split intervals, ranking and classification. These stages lead towards the final goal of generating rules from information or decision system of the Carter dataset.


179

Let , be given. The cardinality of the data d(U) = {k|d(x) = k, x } is called the rank of d and is denoted by r(d). Assume the set of values of decision d is equal to …… }. Quite often the rank is of two Boolean values (e.g. Y, N) or it can be an arbitrary number, in the above example, we could have four ranks if the decision had values in the set {rank 3, rank 2, rank 1, rank 0}.The decision d determines a partition . , …… |

for 1

}

(3)

is called the classification of objects in determinded by the decith sion d. The set , is called the i- decision class of . Fig 2 explains the RSBR discretization algorithm applied on the dataset. Table 4 shows the intervals and ranking of the dataset. Input: Information table (T) created from the dataset value column for each column value. Output: Information table (DT) with discretized real value column. 1. For do 2. 3. 4. 5.

and n is the no. of interval

Define Boolean variables B = ∑ , …… End For where ∑ correspond to a set of partition defined on the variables of column v. Create a new Information table (DT) by using the set of partition. Find the objects that discerns in the decision class.

Fig. 2. Rough sets Boolean Reasoning discretization Table 4. Interval & Ranking of Carter dataset

For instance, the Table. 4 shows the interval and cut-off points used in the Carter dataset problem instances. The column count explains the large, average, medium and low intervals a set on the standard dataset where the number of student enrolled largest enrollment (LE). Searching the reducts form a decision table is NP-complete. Fortunately, Carter dataset has no reducts, and the work directions with setting intervals, ranking with classification on the dataset.

180


5 Experiment and Result The algorithm was developed using java based object oriented programming. The experiments were run on a PC with a 2.8 GHz Core(2) Duo and 2GB of RAM. Carter’s (1996) publicly available exam timetabling datasets were used in the experiments as shown in Table 1. In this work, we evaluate the performance of our approach on twelve instances. In order to test our work modification to the sequential construction method previously developed by Carter et al. [16], the algorithm was initially run with implementing rough set discretization. The experiment works on the exams in the problem instances are in single interval criteria. (E.g. LE) From Table 5 it can be seen that the initial rough sets methods produced comparable results to the single interval criteria. Slightly modified problem instance of Car-f-91 and Car-f-92 are tested with the algorithm. The algorithm produced similar but better results for the Car-f-91 and Car-f-92 dataset, hec-s-92 and kfu-s-93 other datasets were compared with the standard results. Table 5. Experimental results for the rough sets discretization approach that were implemented

6 Conclusion In this paper, we have presented an intelligent data analysis approach based on rough sets theory for generating classification rules from a set of observed 12- real world problem instances as a benchmarking dataset for the examination timetabling community. The main objective is to investigate the problem instances/dataset and with minor modification to obtained better timetables. To increase the classification process rough sets with Boolean reasoning (RSBR) discretization algorithm is used to discretize the data. Further work will be done to minimize the experiment duration in order to get better results with the rough set data analysis.


181

References [1] Burke, E.K., Elliman, D.G., Ford, P.H., Weare, R.F.: Examination timetabling in British Universities – a survey. In: Burke, E., Ross, P. (eds.) PATAT 1995. LNCS, vol. 1153, pp. 76–90. Springer, Heidelberg (1996) [2] Burke, E.K., Elliman, D.G., Weare, R.F.: A hybrid genetic algorithm for highly constrained timetabling problems. In: Proceedings of the 6th International Conference on Genetic Algorithms (ICGA 1995), Pittsburgh, USA, July 15-19, pp. 605–610. Morgan Kaufmann, San Francisco (1995) [3] Burke, E.K., Bykov, Y., Newall, J., Petrovic, S.: A time-predefined local search approach to exam timetabling problems. IIE Transactions on Operations Engineering, 509–528 (2004) [4] Caramia, M., Dell’Olmo, P., Italiano, G.F.: New algorithms for examination timetabling. In: Näher, S., Wagner, D. (eds.) WAE 2000. LNCS, vol. 1982, pp. 230–241. Springer, Heidelberg (2001) [5] Carter, M.W., Laporte, G., Lee, S.Y.: Examination timetabling: Algorithmic strategies and applications. Journal of the Operational Research Society, 373–383 (1996) [6] Joshua, J., et al.: The Perception of Interaction on the University Examination Timetabling Problem. In: McCollum, B., Burke, E., George, W. (ed.) Practice and Theory of Automated Timetabling, ISBN 08-538-9973-3 [7] Al-Betar, M., et al.: A Combination of Metaheuristic Components based on Harmony Search for The Uncapacitated Examination Timetabling. In: McCollum, B., Burke, E., George, W. (eds.): Practice and Theory of Automated Timetabling, ISBN 08-538-9973-3 (PATAT 2010, Ireland, Aug, selected papers) for Annals of operational research [8] Boizumault, P., Delon, Y., Peridy, L.: Constraint logic programming for examination timetabling. The Journal of Logic Programming 26(2), 217–233 (1996) [9] Brailsford, S.C., Potts, C.N., Smith, B.M.: Constraint satisfaction problems: Algorithms and applications. European Journal of Operational Research 119, 557–581 (1999) [10] Burke, E.K., de Werra, D., Kingston, J.: Applications in timetabling. In: Yellen, J., Gross, J.L. (eds.) Handbook of Graph Theory, pp. 445–474. Chapman Hall, CRC Press (2003) [11] Burke, E.K., Newall, J.P.: Solving examination timetabling problems through adaption of heuristic orderings. Annals of Operations Research 129, 107–134 (2004) [12] Burke, E.K., Petrovic, S.: Recent research directions in automated timetabling. European Journal of Operational Research 140, 266–280 (2002) [13] Qu, R., Burke, E.K., McCollum, B., Merlot, L.T.G., Lee, S.Y.: A Survey of Search Methodologies and Automated System Development for Examination Timetabling. Journal of Scheduling 12(1), 55–89 (2009), online publication (October 2008), doi: 10.1007/s10951-008-0077-5.pdf [14] McCollum, B., Schaerf, A., Paechter, B., McMullan, P., Lewis, R., Parkes, A., Di Gaspero, L., Qu, R., Burke, E.: Setting The Research Agenda in Automated Timetabling: The Second International Timetabling Competition. INFORMS Journal on Computing 22(1), 120–130 (2010) [15] Carter, M.W.: A survey of practical applications of examination timetabling algorithms. Operation Research 34(2), 193–202 (1986) [16] Carter, M.W., Laporte, G., Lee, S.Y.: Examination timetabling: Algorithmic strategies and applications. Journal of the Operational Research Society 47, 373–383 (1996) [17] Pawlak, Z.: Rough sets. International Journal of Computer and Information Science 11, 341–356 (1982)

182


[18] Pawlak, Z.: Rough Sets Theoretical Aspect of Reasoning about Data. Kluwer Academic, Boston (1991) [19] Pawlak, Z., Grzymala-Busse, J., Slowinski, R., Ziarko, W.: Rough sets. Communications of the ACM 38(11), 89–95 (1995) [20] Ślęzak, D.: Various approaches to reasoning with frequency-based decision reducts: a survey. In: Polkowski, L., Tsumoto, S., Lin, T.Y. (eds.) Rough Sets in Soft Computing and Knowledge Discovery: New Developments. Physica Verlag, Heidelberg (2000) [21] Pal, S.K., Polkowski, L., Skowron, A.: Rough-Neuro Computing: Techniques for Computing with Words. Springer, Berline (2004)

Community Detection in Sample Networks Generated from Gaussian Mixture Model Ling Zhao1 , Tingzhan Liu2 , and Jian Liu3 1

3

Beijing University of Posts and Telecommunications, Beijing 100876, P.R. China 2 School of Sciences, Communication University of China, Beijing 100024, P.R. China LMAM and School of Mathematical Sciences, Peking University, Beijing 100871, P.R. China [email protected]

Abstract. Detecting communities in complex networks is of great importance in sociology, biology and computer science, disciplines where systems are often represented as networks. In this paper, we use the coarse-grained-diffusion-distance based agglomerative algorithm to uncover the community structure exhibited by sample networks generated from Gaussian mixture model, in which the connectivity of the network is induced by a metric. The present algorithm can identify the community structure in a high degree of efficiency and accuracy. An appropriate number of communities can be automatically determined without any prior knowledge about the community structure. The computational results on three artificial networks confirm the capability of the algorithm. Keywords: Community detection, Gaussian mixture model, Coarsegrained diffusion distance, Agglomerative algorithm, k-means.

1

Introduction

The modern science of networks has brought significant advances to our understanding of complex systems [1,2,3]. One of the most relevant features of graphs representing real systems is community structure, i.e. the organization of vertices in clusters, with many edges joining vertices of the same cluster and comparatively few edges joining vertices of different clusters. Such communities can be considered as fairly independent compartments of a network, playing a similar role like the tissues or the organs in the human body [4,5]. Detecting communities is of great importance, which is very hard and not yet satisfactorily solved, despite the huge effort of a large interdisciplinary community of scientists working on it over the past few years [6,7,8,9,10,11,12,13]. On a related but different front, recent advances in computer vision and data mining have also relied heavily on the idea of viewing a data set or an image as a graph or a network, in order to extract information about the important features of the images [14]. In our previous work [12], we extend the measure of diffusion distance between nodes in a network to a generalized form on the coarse-grained network with data Y. Tan et al. (Eds.): ICSI 2011, Part II, LNCS 6729, pp. 183–190, 2011. c Springer-Verlag Berlin Heidelberg 2011

184

L. Zhao, T. Liu, and J. Liu

parameterization via eigenmaps. This notion of proximity of meta-nodes in the coarse-grained networks reflects the intrinsic geometry of the partition in terms of connectivity of the communities in a diffusion process. Nodes are then grouped into communities through an agglomerative hierarchical clustering technique [15] under this measure and the modularity function [7,8] is used to select the best partition of the resulting dendrogram. A widely used simulated example is the sample network generated from a Gaussian mixture model [10,11]. This model is related the concept of “random geometric graph” proposed by Penrose [16] except that we take Gaussian mixture here instead of uniform distribution. First we generate n sample points {xi } in two K dimensional Euclidean space subject to a K-Gaussian mixturedistribution K k=1 qk G (µk , Σk ), where {qk } are weights that satisfy 0 < qk < 1, k=1 qk = 1, while µk and Σk are the mean positions and covariance matrices for each component, respectively. Here we pick node set Tk in group k, and with this choice, approximately qk = |Tk |/n. Next, we generate the network with a thresholding strategy. That is, if |xi − xj | ≤ dist, we assign an edge between the i-th and j-th node; otherwise they are not connected. With this strategy, the connectivity of the network is induced by a metric. We are interested in the connection between our network clustering and the traditional clustering in the metric space. To evaluate our result obtained by the algorithm proposed above, we can compare the clustering result with the original partition {Tk }. Notice that {Tk } is independent of the topology of the network, which can be only considered as a reasonable reference value but not an exact object. Another choice is to compare our result with those obtained from k-means algorithm [15] since the metric is known in this case. The rest of the paper is organized as follows. In Section 2, we briefly introduce the coarse-grained random walk and coarse-grained diffusion distance [12]. After reviewing the definition of modularity, we describe the algorithm in detail. In Section 3, we apply the algorithm to three artificial examples mentioned before. Finally we make the conclusion in Section 4.

2 2.1

Framework of Coarse-Grained-Diffusion-Distance Based Agglomerative Algorithm Construction of Coarse-Grained Diffusion Distance

We will start with a brief review of the basic idea in [12]. Let G(S, E) be a network with n nodes and m edges, where S is the nodes set, E = {e(x, y)}x,y∈S is the weight matrix and e(x, y) is the weight for the edge connecting the nodes x and y. We can relate this network to a discrete-time Markov chain with stochastic matrix P with entries p1 (x, y) given by p1 (x, y) = e(x,y) z∈S e(x, z), d(x) , d(x) = where d(x) is the degree of the node x [3]. The process is driven by the P t = {pt (x, y)}x,y∈S , where pt (x, y) represents the probability of going from node x to node y through a random walk in t time steps. This Markov chain has stationary distribution μ(x) = d(x)d(z) and it satisfies the detailed balance z∈S

Community Detection in Sample Networks

185

condition μ(x)p1 (x, y) = μ(y)p1 (y, x). The diffusion distance Dt (x, y) between x and y is defined as the weighted L2 distance Dt2 (x, y) =

pt (x, z) − pt (y, z) 2 , μ(z)

(1)

z∈S

where the weight μ(z)−1 penalize discrepancies on domains of low density more than those of high density. As is well known, the transition matrix P has a set of left and right eigenvectors and a set of eigenvalues 1 = λ0 ≥ |λ1 | ≥ · · · ≥ |λn−1 | ≥ 0, and naturally P ϕi = λi ϕi , ψiT P = λi ψiT , i = 0, 1, · · · , n − 1. Note that ψ0 = μ, ϕ0 ≡ 1 and ψiT ϕj = δij . The left and right eigenvectors are related according to ψi (x) = ϕi (x)μ(x). The spectral decomposition of P t is given by pt (x, y) =

n−1

λti ϕi (x)ψ(y) =

i=0

n−1

λti ϕi (x)ϕ(y)μ(y),

(2)

i=0

then the diffusion distance (1) can be reduced to Dt2 (x, y) =

n−1

2 λ2t i (ϕi (x) − ϕi (y)) .

(3)

i=0

N We take a partition of S as S = k=1 Sk with Sk Sl = Ø if k = l, and regard each set Sk in the state space S = {S1 , · · · , SN } as corresponding to the ˆ Et ), where Et = {ˆ nodes of a N -nodes network G(S, et(Sk , Sl )}Sk ,Sl ∈S , and the weight e ˆ (S , S ) on the edge that connects S and S ˆt (Sk , Sl ) = t k l k l is defined as e μ(x)p (x, y), where the sum involves all the transition probabilities t x∈Sk ,y∈Sl between x ∈ Sk and y ∈ Sl . From the detailed balance condition, it can be verified that eˆt (Sk , Sl ) = eˆt (Sl , Sk ). By setting μ ˆ(Sk ) = z∈Sk μ(z), one can ˆ define a coarse-gained Markov chain on G(S, Et ) with stationary distribution μ ˆ and transition probabilities pˆt (Sk , Sl ) = N

eˆt (Sk , Sl )

ˆt (Sk , Sm ) m=1 e

=

1 μ ˆ(Sk )

μ(x)pt (x, y).

(4)

x∈Sk ,y∈Sl

It can be easily shown that pˆt is a stochastic matrix on the state space S and satisfies a detailed balance condition with respect to μ ˆ. More generally, we define coarse-grained versions of ψ in a similar way by summing over the nodes in a i partition ψî (Sk ) = ψ (z) and as above, coarse-grained versions of ϕi z∈Sk i ˆ according to the duality condition ψi (Sk ) = ϕî (Sk )ˆ μ(Sk ) is defined as ϕî (Sk ) = î (Sk ) ψ 1 = μˆ(Sk ) z∈Sk ϕi (z)μ(z). Then the coarse-grained probability pˆt can be μ ˆ (Sk ) written in a similar way as (2) in the spectral decomposition form as follows pˆt (Sk , Sl ) =

n−1 i=0

λti ϕ î (Sk )ϕî (Sl )ˆ μ(Sl ).

(5)

186


This can be considered as an extension version of (2). This leads to the diffusion distance between community Sk and Sl given by ˆ t2 (Sk , Sl ) = D

n−1

λti λtj (ϕ î (Sk ) − ϕ î (Sl ))(ϕˆj (Sk ) − ϕ ˆj (Sl ))

i,j=0

N

ψî (Sm )ϕˆj (Sm ).

m=1

(6) This notion of proximity of communities in the coarse-grained networks reflects the intrinsic geometry of the set S in terms of connectivity of the meta-nodes in a diffusion process. This metric is thus a key quantity in the design of the following algorithm that is based on the preponderance of evidences for a given hypothesis. 2.2

Modularity Maximization and Its Main Limits

In recent years, a concept of modularity proposed by Newman [7,8] has been widely used as a measure of goodness for community structure. A good division of a network into communities is not merely one in which the number of edges running between groups is small. Rather, it is one in which the number of edges between groups is smaller than expected. These considerations lead to the modularity Q defined by Q = (number of edges within communities) − (expected number of such edges). It is a function of the particular partition of the network into groups, with larger values indicating stronger community structure. Let pE (x, y) be the expected number of edges between x and y, and then for a given partition {Sk }N k=1 , the modularity can be written as Q=

N 1 e(x, y) − pE (x, y) , 2me k=1 x,y∈Sk

pE (x, y) =

d(x)d(y) 2me

(7)

and me is the total weight of edges given by x,y∈S e(x, y)/2. Some existing methods are presented to find good partitions of a network into communities by optimizing the modularity over possible divisions, which has proven highly effective in practice [7,8,11,12]. 2.3

The Algorithm

Agglomerative clustering algorithms begin with every observation representing a singleton cluster. At each of the n − 1 steps the closest two (least dissimilar) clusters are merged into a single cluster, producing one less cluster at the next higher level [15]. Here we make use of the process of agglomerative hierarchical clustering technique for network partition and choose the coarse-grained diffusion distance as the measure of dissimilarity between communities at each step. The maximal value of modularity determines the optimal partition of the network. Given a distance measure between points, the user has many choices for how to define intergroup similarity in traditional clustering literature [15]. However, different choices have different benefits and shortages. The advantage of our algorithm is that the proposed measure of coarse-grained diffusion distance overcomes the weaknesses of the traditional linkage techniques based on node-to-node


187

Table 1. The parameters for construction of the three sample networks generated from the Gaussian mixture model Networks 1 2 3

n

K

µT1

µT2

µT3

150 300 320

3 3 4

(0.0,3.0) (1.0,4.0) (1.0,5.0)

(1.5,4.5) (3.0,6.0) (3.0,5.5)

(-0.5,5.0) (0.5,6.5) (1.0,7.0)

µT4

σ

dist

(3.0,7.5)

0.15 0.25 0.15

0.9 0.8 0.8

Table 2. The computational results obtained by our method. Here CR1 and CR2 are the correct rates compared with the original partition {Tk } and those obtained from k-means algorithm, respectively. Networks 1 2 3

n

N

t

Q

CR1

CR2

150 300 320

3 3 4

1 3 3

0.6344 0.6547 0.7301

0.9867 0.9867 0.9969

1 1 1

dissimilarity mentioned above, since it takes into account all the information relating the two clusters. The only parameter our computation is the time step t and increasing t corresponds to propagating the local influence of each node with its neighbors. 123

6

142

107 108 121 149 112 110 127 115 147 143 114 124 102106 140 139 120 128 133 130 144 129 118 132 122 109 104116 138 101 137 146 99 131 113136111 103 9380 119 145 150 125 141 105 62 135 148 117 134 69 126

5.5 5 4.5

y

4 3.5

3 42 24 34 13 12 33 15 41 2644 18 43 29 40 31 4930 8 162 25 20 46 736 11 4538 21 28 23 1 37 27 19 5 22 32 47 39 50 17 10 14 6

3 2.5

Samples in group 1 Samples in group 2 Samples in group 3

2 −1

0

1

x

(a)

66 76 89 60 78 70 85 57 83 67 5955 96 72 82 77 56 71 51 52 84 75100 91 81 86 88 87 68 73 64 53 7494 92 98 9754 65 95 63 61 58 79 90

9

4

2 48

35

(b)

Fig. 1. (a)150 sample points generated from the given 3-Gaussian mixture distribution. The star symbols represent the centers of each Gaussian component. The circle, square and diamond shaped symbols represent the position of sample points in each component, respectively. (b)The network generated from the sample points in Figure 1(a) with the parameter dist = 0.9.

188


3

Experimental Results

As mentioned in Section 1, we generate n sample points {xi } in two dimensional Euclidean space subject to a K-Gaussian mixture distribution K k=1 qk G (µk ,Σk ). Here we pick nodes n(k − 1)/K + 1 : nk/K in group Tk for simplicity, and with this choice, approximately qk = n/K, k = 1, · · · , K. The covariance matrices are set in diagonal form Σk = σI. The other parameters for construction of the three sample networks generated from the Gaussian mixture model are list in Table 1. The computational results obtained by our method are shown in Table 2. Here CR1 and CR2 are the correct rates compared with the original partition {Tk } and those obtained from k-means algorithm, respectively. We can see that the number of communities are in accordance with the number of components in the corresponding Gaussian mixture models and the two kinds of correct rates

0.6

t=1 t = 1000 t = 1200 t = 1500 t = 2000

Modularity function

0.6

0.5

0.4

Modularity Q

0.4

Q

0 20

0.3

0.2

0.1

0 0

50

100

150

Number of communities

(a) 123

142

107 121 108 149 112 110 127 115 147 143 114 124 102106 140 139 120 128 133 130 144 129 118 132 122 109 104116 138 101 137 146 99 131 113136111 103 9380 119 145 150 125 141 105 62 135 148 117 134 69 126

3 4224 34 13 12 33 15 41 2644 18 43 29 40 31 49 162 8 25 20 46 30 736 11 38 45 21 28 23 1 37 27 19 5 22 32 47 39 50 17 10 14 6

66 76 89 60 78 70 85 57 83 67 5955 96 72 82 56 71 77 51 52 84 75 81 86 100 88 91 87 68 73 64 53 7494 92 98 9754 65 95 63 61 58 79 90

9

48

35

(b)

4

=0.6344

max

0.2

40

60

80

100

120

140

27 35 4 33 31 43 29 134 48 23 47 39 17 50 32 22 10 5 3 11 40 8 14 6 19 41 18 44 30 16 45 25 28 21 38 49 46 20 26 15 42 24 34 12 13 9 36 7 2 37 1 140 132 141 117 148 135 105 123 142 126 150 125 124 139 128 127 112 149 115 110 108 107 146 131 145 144 113 104 130 118 106 120 114 147 143 102 133 129 116 136 138 122 119 103 109 111 137 101 121 93 80 97 69 90 76 99 83 85 70 66 62 65 67 63 79 60 78 92 91 72 61 58 68 89 81 71 98 94 73 64 74 95 54 53 59 57 96 100 82 55 77 56 51 87 84 75 52 88 86

(c)

Fig. 2. The computational results for the sample network with 150 nodes detected by our method. (a)The modularity changing with number of communities in each iteration for different time parameter t. (b)The community structure identified by setting t = 1, corresponds to 3 communities represented by the colors. (c)The dendrogram of hierarchical structures and the optimal partition with a maximal modularity Q = 0.6344 is denoted by a vertical dash line.


7.5

t=1 t=2 t=3 t = 10 t = 20

0.6

7

Modularity function

6.5 6

y

5.5 5 4.5 4

3 0

1

2

3

0.5

0.4

190 271

0.3

268 0.2

149 0.1

Samples in group 1 Samples in group 2 Samples in group 3

3.5

189

0 0

4

50

100

150

200

250

300


x

(a)

(b)

(c)

Fig. 3. (a)300 sample points generated from the given 3-Gaussian mixture distribution. (b)The modularity changing with number of communities in each iteration for different time parameter t. (c)The community structure identified by setting t = 3, corresponds to 3 communities represented by the colors.

8.5 8

Modularity function

0.6

7 6.5 y

t=1 t=2 t=3 t = 10 t = 100

0.7

7.5

6 5.5 5 4.5

Samples in group 1 Samples in group 2 Samples in group 3 Samples in group 4

4 0

0.5

1

1.5

2

2.5

3

3.5

x

(a)

4

4.5

0.5 0.4 0.3 0.2 0.1

17 0 0

50

100

150

200

250

300


(b)

(c)

Fig. 4. (a)320 sample points generated from the given 4-Gaussian mixture distribution. (b)The modularity changing with number of communities in each iteration for different time parameter t. (c)The community structure identified by setting t = 3, corresponds to 4 communities represented by the colors.

indicate that our method can produce accurate results when the time parameter t is properly chosen. The visualization of the partitioning result and the dendrogram of hierarchical structures are shown in Figure 2. The same goes for the other two sample networks and the clustering results can be seen in Figure 3 and Figure 4, respectively.

4

Conclusions

In this paper, we use the coarse-grained-diffusion-distance based agglomerative algorithm to uncover the community structure exhibited by sample networks generated from Gaussian mixture model. The present algorithm can identify the community structure in a high degree of efficiency and accuracy. An appropriate number of communities can be automatically determined without any prior

190


knowledge about the community structure. The computational results on three artificial networks generated from Gaussian mixture model confirm the capability of the algorithm. Acknowledgements This work is supported by the Project of the Social Science Foundation of Beijing University of Posts and Telecommunications under Grant 2010BS06.

References 1. Albert, R., Barab´ asi, A.L.: Statistical mechanics of complex networks. Rev. Mod. Phys. 74(1), 47–97 (2002) 2. Newman, M.: The structure and function of complex networks. SIAM Review 45(2), 167–256 (2003) 3. Newman, M., Barab´ asi, A.L., Watts, D.J.: The structure and dynamics of networks. Princeton University Press, Princeton (2005) 4. Barab´ asi, A., Jeong, H., Neda, Z., Ravasz, E., Schubert, A., Vicsek, T.: Evolution of the social network of scientific collaborations. Physica A 311, 590–614 (2002) 5. Ravasz, E., Somera, A., Mongru, D., Oltvai, Z., Barab´ asi, A.: Hierarchical organization of modularity in metabolic networks. Science 297(5586), 1551–1555 (2002) 6. Girvan, M., Newman, M.: Community structure in social and biological networks. Proc. Natl. Acad. Sci. USA 99(12), 7821–7826 (2002) 7. Newman, M., Girvan, M.: Finding and evaluating community structure in networks. Phys. Rev. E 69(2), 026113 (2004) 8. Newman, M.: Modularity and community structure in networks. Proc. Natl. Acad. Sci. USA 103(23), 8577–8582 (2006) 9. E, W., Li, T., Vanden-Eijnden, E.: Optimal partition and effective dynamics of complex networks. Proc. Natl. Acad. Sci. USA 105(23), 7907–7912 (2008) 10. Li, T., Liu, J., E, W.: Probabilistic Framework for Network Partition. Phys. Rev. E 80, 26106 (2009) 11. Liu, J., Liu, T.: Detecting community structure in complex networks using simulated annealing with k-means algorithms. Physica A 389(11), 2300–2309 (2010) 12. Liu, J., Liu, T.: Coarse-grained diffusion distance for community structure detection in complex networks. J. Stat. Mech. 12, P12030 (2010) 13. Fortunato, S.: Community detection in graphs. Phys. Rep. 486, 75–174 (2010) 14. Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intel. 22(8), 888–905 (2000) 15. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, New York (2001) 16. Penrose, M.: Random Geometric Graphs. Oxford University Press, Oxford (2003)

Efficient Reduction of the Number of Associations Rules Using Fuzzy Clustering on the Data Amel Grissa Touzi, Aicha Thabet, and Minyar Sassi Université de Tunis El Manar Ecole Nationale d’Ingénieurs de Tunis Bp. 37, Le Belvédère 1002 Tunis, Tunisia {amel.touzi,minyar.sassi}@enit.rnu.tn, {[email protected]}

Abstract. In this paper, we are interested in the knowledge discovery methods. The major inconveniences of these methods are: i) the generation of a big number of association rules that are not easily assimilated by the human brain ii) the space memory and the time execution necessary for the management of their data structures. To cure this problem, we propose to build rules (meta-rules) between groups (or clusters) resulting from a preliminary fuzzy clustering on the data. We prove that we can easily deduce knowledge about the initial data set if we want more details. This solution reduced considerably the number of generated rules, offered a better interpretation of the data and optimized both the space memory and the execution time. This approach is extensible; the user is able to choose the fuzzy clustering or the extraction rules algorithm according to the domain of his data and his needs.

1 Introduction Nowadays, we notice a growing interest for the Knowledge Discovery in Databases (KDD) methods. One of the important reasons to that fact is the increasing volume of the accumulated data by organizations that are under-exploited extensively. Several solutions were proposed, they are based on neural networks, trees, concept lattices, association rules, etc. [1]. Several algorithms for mining association rules were proposed in the literature. The existing generation methods are combinative and generate a big number of rules (even when departing from sets of reasonable size) that are not easily exploitable [2], [3]. Several approaches of reduction of this big number of rules have been proposed like the use of quality measurements, syntactic filtering by constraints, and compression by the representative or Generic Bases [4]. These bases constitute the reduced sets of rules permitting to preserve the most relevant rules, without any loss of information. In our opinion, the big number of the generated rules is due to the fact that these approaches try to determine rules departing from the enormous data set. In this paper, we propose to extract knowledge taking in consideration another degree of granularity into the process of knowledge extraction. We propose to define rules (meta-rules) between classes resulting from a preliminary fuzzy clustering on the data. We call the knowledge yet extracted "Meta-Knowledge". Indeed, while classifying data, we construct homogeneous groups of data having the same properties, so Y. Tan et al. (Eds.): ICSI 2011, Part II, LNCS 6729, pp. 191–199, 2011. © Springer-Verlag Berlin Heidelberg 2011

192

A. Grissa Touzi, A. Thabet, and M. Sassi

defining rules between clusters implies that all the data elements belonging to those clusters will be necessarily dependent on these same rules. Thus, the number of generated rules is smaller since one processes the extraction of the knowledge on the clusters which number is relatively lower compared to the initial data elements. We prove that we can easily deduce knowledge about the initial data set if we want more details. The rest of the paper is organized as follows: section 2 presents the basic concepts of discovering association rules. Section 3 presents problems and limits of the existing knowledge discovery approaches. Section 4 defines the theoretical foundation of this approach. Section 5 contains the principles of the new approach that we proposed. Section 6 enumerates the advantages of the proposed approach. Section 7 validates the proposed approach and gives an experimentation example. We finish this paper with a conclusion and a presentation of some future works.

2 Basic Concepts In this section, we present the basic concepts of discovering association rules. 2.1 Discovering Association Rules Association rules mining have been developed in order to analyze basket data in a marketing environment. Input data are composed of transactions: each transaction consists of items purchased by a consumer during a single visit. Output data is composed of rules. An example of an association rule is “90% of transactions that involve the purchase of bread and butter also include milk” [5]. Even if this method was introduced in the context of Market Business Analysis, it has many applications in other fields, like webmining or textmining. It can also be used to search for frequent cooccurrences in every large data set. The first efficient algorithm to mine association rules is APriori [6]. Other algorithms were proposed to decrease the count of reads of the database and to improve computational efficiency. Among them, we mention CLOSED [7], CHARM [8], TITANIC [9],[10], GENALL [11], PRINCE [12]. Several varieties of lattice have been introduced with these algorithms, like Iceberg Concept lattices [10] where the nodes are frequent closed itemsets ordered by the inclusion relation, Minimal Generators Lattice [12], where the nodes are the minimal Generators (called key itemsets) are ordered by the inclusion relation. In these cases we don't construct the FCA on the data but on the found itemsets. For more detail the reader can see [12]. 2.2 Classification and Knowledge Extraction The only interesting work, that used the data classification as a prior step of the generation of association rules, applied in the industry, is the one of Plasse et al. [13]. The technique proposed was to carry out a preliminary classification of the variables in order to obtain homogeneous groups of attributes then to seek the association rules inside each one of these groups. They obtained groups of variables, more restricted and homogeneous. Besides the rules obtained are fewer and simpler.

Efficient Reduction of the Number of Associations Rules

193

To validate their approaches, they proceeded to a research of association rules considering several partitions obtained at the end of either proc VARCLUS or of proc CLUSTER from SAS. The various tests achieved showed that the research of rules inside classes permits to decrease their number. We notice that even with this solution, the number of rules is not negligible; the user finds difficulties in assimilating this big number.

3 Problems and Motivations Several algorithms try to trace the decision tree or the FCA or one of these extensions to extract the association rules. In this case, researchers always focus on giving an optimum set of rules modeling in a faithful way the starting data unit, after having done a data cleansing step and an elimination of invalid-value elements. To our point of view, limits of these approaches consist in extracting the set of rules departing from the data or a data variety like the frequent itemsets or the frequent closed itemsets, which may be huge. Thus we note the following limits: 1) these approaches require a big space memory and an important execution time for data modeling because of the data structures required for these algorithms such as the trees or the graphs or the lattices; 2) the rules generated from these data are generally redundant rules; 3) these algorithms generated a very big number of rules, almost thousands, that the human brain cannot even assimilate; 4) some previous works demonstrated that the behaviour of these algorithms of association rules extraction varies strongly according to the features of the used data set [14]. The number of the generated association rules in general varies from several ten of thousands to several millions [3], [15]. The Execution times obtained strongly vary according to the used algorithm [16]; 5) generated rules by these algorithms don't take into account the data semantics nor the importance of an attribute in relation to another in the data description, according to the specific domain of this data set; and (6) Generally, the goal to extract a set of rules is to help the user to give semantics of data and to optimize the information research. This fundamental constraint is not taken into account by these approaches. To cure all these problems, we propose a new approach for knowledge extraction using a preliminary fuzzy clustering on the data. We start by presenting the theoretical foundations of the proposed approach.

4 Theoretical Foundation of the Knowledge Extraction In this part, we present the theoretical foundations of the proposed approach, based on the following properties and theorems. We start with an example. Example Given a set of students S1, S2, S3, S4, S5, S6, S7, S8, S9 and S10 according to their preferences for the following modules: Databases (DB), Programming Languages(PL), NeTworks(NT), LItterature(LI) and Another Topic(AT). Table 1 puts in correspondence the student and his marks in these different modules. The fuzzy clustering operation (FCM algorithm) applied to this example generates 3 fuzzy partitions. The result of the fuzzy clustering algorithm is the matrix of member-ship described in Table 2.

194


We define the cut, noted α-Coupe, on the fuzzy context as being the reverse of the number of clusters obtained. We can consider two possible strategies for the application of the α-Coupe: Binary strategy (resp. Fuzzy strategy): This strategy defines a binary membership (resp. fuzzy membership) of the objects to the different clusters. We propose to leave the fuzzy formal context, to apply an α-Coupe to the set of the degrees of membership, to replace these last by values 1 and 0 and to deduce the binary reduced formal context.

Table 1. Example of students marks DB

PL

NT

LI

AT

15

14

12

14

10

S1

Table 2. Result of clustering C1

C2

C3

S1

0.092

0.804

0.104

S2

0.091

0.708

0.201

S3

0.041

0.899

0.060

S4

0.071

0.100

0.829

14

15

9

8

10

16

13

12

12

7

7

10

14

12

8

11

5

18

15

14

S5

0.823

0.070

0.107

12

11

10

10

10

S6

0.090

0.548

0.362

17

6

14

15

14

S7

0.810

0.108

0.082

9

10

12

11

10

S8

0.036

0.066

0.898

5

6

10

6

10

S9

0.157

0.179

0.664

13

7

12

14

13

S10

0.231

0.388

0.381

S2 S3 S4 S5 S6 S7 S8 S9 S10

In our example α-Coupe = 1/3. Table 3 presents the binary reduced formal context after application of the α-Coupe to the fuzzy formal context presented in table 2. Table 4 represents the fuzzy reduced formal context after application of the α-Coupe to the fuzzy formal context presented on Table 2. Generally, we can consider that the attributes of a formal concept, best-known as the concept intention, are the description of the concept. Thus, the relationships between the object and the concept should be the intersection of the relationships between the objects and the attributes of the concept. Since each relationship between the object and an attribute is represented as a set of membership values in the fuzzy formal context, the intersection of these member-ship values should be the minimum of these membership values, according to fuzzy theory. Thus, we defined the fuzzy formal concept from the fuzzy formal context. Properties − The number of clusters generated by a clustering algorithm is always lower than the number of starting objects to which one applies the clustering algorithm − All objects belonging to one same cluster have the same characteristics. These characteristics can be deduced easily knowing the center and the distance from the cluster.


Table 3. Reduced binary formal context C1 S1

0

195

Table 4. Reduced fuzzy formal context

C2

C3

C1

C2

C3

1

0

S1

-

0.804

-

S2

0

1

0

S2

-

0.708

S3

0

1

0

S3

-

0.899

-

S4

0

0

1

S4

-

-

0.829

S5

1

0

0

S5

0.823

-

-

S6

0

1

1

S6

-

0.548

0.362

S7

1

0

0

S7

0.810

-

-

S8

0

0

1

S8

-

-

0.898

S9

0

0

1

S9

-

-

0.664

S10

0

1

1

S10

-

0.388

0.381

Notation Let C1 and C2 be two clusters generated by a fuzzy clustering algorithm. The rule C1 = > C2 with a coefficient (CR) will be noted C1 = > C2 (CR) If the coefficient CR is equal to 1 then the rule is called an exact rule. Theorem 1 Let C1, C2 be two clusters, generated by a fuzzy clustering algorithm and verifying the properties p1 and p2 respectively. Then the following properties are equivalent: C1 ⇒ C2 (CR) ⇔ - ∀ object O1 ∈ C1 => O1 ∈C2 (CR) - ∀ object O1 ∈ C1, O1 checks the property p1 of C1 and the property p2 of C2. (CR)

Theorem 2 Let C1, C2 and C3 be three clusters generated by a fuzzy clustering algorithm and verifying the properties p1, p2 and p3 respectively. Then the following properties are equivalent: C1 and C2 = > C3 (CR) ⇔ ∀ object O1 ∈ C1 ∩ C2 = > O1 object ∈C3 (CR) ∀ O1 object ∈ C1 ∩ C2 then O1 checks the properties p1, p2 and p3 with (CR)

The proof of the two theorems rises owing to the fact that all objects which belong to a same cluster check necessarily the same property as their cluster. Classification of data We regroup customers that check the same property in a class (only one property). Using this type of fuzzy clustering, we have the following properties: - The number of clusters in this case will be equal to the number of attributes. - The Class i will contain all the objects which check only one same property. For example in basket data in a marketing environment we regroups all the customers who bought the same product x.

196


C1 Customers who bought Bread

C2 Customers

who

bought

Fig. 1. Example of overlapping between two clusters

From this matrix, we can generate rules giving associations between the different clusters. The Figure1 models an example of classification result modeling an overlapping between the two clusters C1 and C2. We notice that the intersection of the two clusters gives the customers who bought bread and Chocolate. Generation of the knowledge departing from Meta knowledge Definition Each Class generated by a fuzzy clustering algorithm will be modeled by a predicate having two arguments: the first is the object; the second carries the name of the corresponding criteria of clustering. Example The cluster C1 will be modeled by the predicate buys(x, Bread). In this case, we deduce that if we have the rule: C1 = > C2 (CR), it will be transformed in the following rule: buys (x, Bread) = > buys (x, Chocolate) (CR ) We can simplify this notation while only specifying: Bread ⇒ Chocolate (CR)

Thus we can generate the different rules on data from generated rules on clusters.

5 New Approach for the Knowledge Discovery The principle of our approach is based on the following ideas: − While applying a fuzzy clustering algorithm to the elements of a data source to divide them into clusters, we obtain the matrix of adherence of each of these elements to a given cluster, − We consider the obtained matrix as a formal context where objects are elements to classify and attributes are clusters. Then, we deduce the reduced binary formal context of the obtained matrix. − From this formal context, we apply an algorithm of generation of association rules on clusters to generate the set of Meta knowledge. − We deduce the set of knowledge from this Meta knowledge This principle for the extraction of knowledge from the data proceeds in two phases: 1. Phase of clustering, to organize the data into groups, using the algorithms of fuzzy clustering, 2. Phase of extraction of knowledge, using any algorithm of generation of association rules. This process can be summarized in the following steps:


197

Begin Step 1: Introduce a data set (any type of data) Step 2: Apply a fuzzy clustering algorithm to organize the data into the different groups (or clusters). Step 3: Determine the fuzzy formal context (Object/Cluster) of the matrix obtained in the step 2 Step 4: Deduce the reduced binary formal context of the matrix obtained in the step 3 Step 5: Apply an algorithm of generation of association rules on clusters to generate the set of Meta knowledge in the form of association rules between clusters Step 6: Generate knowledge of the data set in the form of association rules End

Fuzzy Classification

Set of clusters generated

Context Matrix (Objects / Clusters)

Analysis

Meta knowledge Set of Rules on the clusters Cluster 1 ⇒ Cluster 2 (CR)

Extraction of Meta Knowledge’s

Knowledge set of Rules on data Objet1 ⇒ Objet2 (CR)

Generation of Knowledge

Fig. 2. Process of knowledge extraction

6 Advantages of the New Approach Different advantages are granted by the proposed approach: 1) Extensibility of the proposed approach: 1) Our approach can be applied with any fuzzy clustering algorithm to classify the initial data. 2) The stage of generation can be applied with any rules generation algorithm. In the literature, the studies showed that such algorithm is optimum than another according to the field of the data used. It means that we can apply the optimum method according to the field of data set. 3) We can generate the maximum of knowledge on our initial data set; it’s enough to modify the choice of the clustering criteria. These criteria can be chosen by the user like parameters of entry according to the importance of the attribute in its applicability. 4) We can classify our data according to different criteria and obtain different clusters which generate a different set of Meta knowledge 2) The definition of the Meta knowledge concept: This definition is in our opinion very important, since the number of rules generated is smaller. Besides, the concept of Meta knowledge is very important to have a global view on the data set which is very voluminous. This models a certain abstraction of the data that is fundamental in the case of an enormous number of data. In this case, we define the set of association rules between the classes. That can generate automatically the association rules between the data, if we want more details.

198


7 Validation of the Proposed Approach To validate the approach proposed, we chose: 1) The FCM algorithm for a fuzzy clustering of the data set, and 2) The PRINCE algorithm, presented in [12], permits to extract the generic bases. This algorithm is based on the FCA. It takes as input an extraction context K, the minimum threshold of support minsup and the minimum threshold of confidence minconf. It outputs the list of the frequent closed itemsets and their associated minimal generators as well as generic bases of association rules. Thus, Prince operates in three successive steps: (i) Minimal generator determination (ii) Partial order construction (iii) Extraction of generic rule bases [12]. A free version of this algorithm exists on internet1. The choice of FCA method and Generic Bases is justified by the fact that these Bases constitute reduced sets of informative and generic rules permitting to preserve the most relevant rules, without loss of information. Thus, the major problems in the other methods such as memory capacity and execution time will be more interesting because we operate the knowledge extraction on clusters which number is negligible compared to the initial data. Results that we obtained are encouraging. We are now to test our approach on different types of data. A platform of fuzzy clustering and knowledge extraction of data is implemented in our laboratory. It offers in particular the possibility to: 1) Compare the generated rules by traditional methods and those generated by our new approach, following a fuzzy clustering operation of data; 2) Visualize the different generated lattices; and 3) Model the different generated clusters. This platform is extensible, it offers different fuzzy clustering algorithms as C-moyennes, FCM, etc; and different association rules generation algorithms as PRINCE, CLOSE, etc.

8 Conclusion In this paper, we presented a new approach that permits to extract knowledge from a preliminary fuzzy clustering of the data. Generally, all the methods in this field applied to the data (or data variety) which is huge. It generates consequently a big number of association rules that are not easily assimilated by the human brain. The space memory and the time execution necessary for the management of these lattices are important. To resolve this problem, we propose to build rules (meta-rules) between groups (or clusters) resulting from a preliminary fuzzy clustering of the data. This approach is based on the following main ideas: while classifying data, we construct homogeneous groups or clusters of data having each one the same properties. Consequently, defining rules between clusters implies that all data belong to those will be necessarily dependent on these same generated rules. To validate this approach, we have chosen the FCM (Fuzzy C-Means) algorithm which allows a fuzzy clustering to generate clusters. We have chosen the Prince algorithm that permits to extract Generic Bases modeling the Meta Knowledge from initial data and we deduce the data set’s knowledge. We have implemented a fuzzy clustering and a data knowledge extraction platform. It is extensible, it offers different fuzzy clustering algorithms and different generation association rules algorithms. 1

www.cck.rnu.tn/sbenyahia/software_release.htm


199

In the future, we propose to define obtained rules in an Expert System and offer to the user the possibility to dialogue with this system to satisfy his needs.

References 1. Goebel, M., Gruenwald, L.: A Survey of Data Mining and Knowledge Discovery Software Tools. SIGKDD, ACM SIGKDD 1(1), 20–33 (1999) 2. Zaki, M.: Mining Non-Redundant Association Rules. In: Data Mining and Knowledge Discovery, vol. (9), pp. 223–248 (2004) 3. Stumme, G., Taouil, R., Bastide, Y., Pasquier, N., Lakhal, L.: Intelligent structuring and reducing of association rules with formal concept analysis. In: Baader, F., Brewka, G., Eiter, T. (eds.) KI 2001. LNCS (LNAI), vol. 2174, pp. 335–350. Springer, Heidelberg (2001) 4. Pasquier, N.: Data Mining: Algorithmes d’Extraction et de Réduction des Règles d’Association dans les Bases de Données. Thèse, Département d’Informatique et Statistique, Faculté des Sciences Economiques et de Gestion, Lyon (2000) 5. Agrawal, R., Imielinski, T., Swami, A.: Mining Association Rules between sets of items in large Databases. In: Proceedings of the ACM SIGMOD Intl. Conference on Management of Data, Washington, USA, pp. 207–216 (June 1993) 6. Agrawal, R., Skirant, R.: Fast algoritms for mining association rules. In: Proceedings of the 20th Int’l Conference on Very Large Databases, pp. 478–499 (June 1994) 7. Pasquier, N., Bastide, Y., Taouil, R., Lakhal, L.: Efficient Mining of Association Rules Using Closed Itemset Lattices. Information Systems Journal 24(1), 25–46 (1999) 8. Zaki, M.J., Hsiao, C.J.: CHARM: An Efficient Algorithm for Closed Itemset Mining. In: Proceedings of the 2nd SIAM International Conference on Data Mining, Arlington, pp. 34– 43 (April 2002) 9. Stumme, G., Taouil, R., Bastide, Y., Pasquier, N., Lakhal, L.: Fast Computation of Concept Lattices Using Data Mining Techniques. In: Bouzeghoub, M., Klusch, M., Nutt, W., Sattler, U. (eds.) Proceedings of 7th Intl. Workshop on Knowledge Representation Meets Databases (KRDB 2000), Berlin, Germany, pp. 129–139 (2000) 10. Stumme, G., Taouil, R., Bastide, Y., Pasquier, N., Lakhal, L.: Computing Iceberg Concept Lattices with TITANIC. J. on Knowledge and Data Engineering (KDE) 2(42), 189–222 (2002) 11. Ben Tekaya, S., Ben Yahia, S., Slimani, Y.: Algorithme de construction d‘un treillis des concepts formels et de déte rmination des générateurs minimaux. ARIMA Journal, 171–193 (Novembre 2005); Numéro spécial CARI 2004 12. Hamrouni, T., Ben Yahia, S., Slimani, Y.: Prince: Extraction optimisée des bases génériques de règles sans calcul de fermetures. In: Proceedings of the Intl. INFORSID Conference, Editions Inforsid, Grenoble, France, May 24-27, pp. 353–368 (2005) 13. Plasse, M., Niang, N., Saporta, G., Villeminot, A., Leblond, L.: Combined use of association rules mining and clustering methods to find relevant links between binary rare attributes in a large data set. Computational Statistics & Data Analysis 52(1), 596–613 (2007) 14. Pasquier, N., Bastide, Y., Touil, R., Lakhal, L.: Pruning closed itemset lattices for association rules. In: Proceedings of 14th International Conference Bases de Données Avancées, Hammamet, Tunisia, October 26-30, pp. 177–196 (1998) 15. Zaki, M.J.: Generating Non-Redundant Association Rules. In: Proceedings of the 6th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Boston, MA, pp. 34–43 (August 2000) 16. Bastide, Y., Taouil, R., Pasquier, N., Stumme, G., Lakhal, L.: Mining frequent patterns with counting inference. SIGKDD Explorations 2(2), 66–75 (2000)

A Localization Algorithm in Wireless Sensor Networks Based on PSO Hui Li, Shengwu Xiong, Yi Liu, Jialiang Kou, and Pengfei Duan School of Computer Science and Technology, Wuhan University of Technology, Wuhan 430070, China

Abstract. Node localization is a fundamental and important technology in wireless sensor networks. In this paper, a localization algorithm in wireless sensor networks based on PSO is proposed. Unlike most of the existing location algorithm, the proposed algorithm figures out the rectangular estimation range of unknown node by bounding box algorithm and takes one value as the estimated coordinates of this node, then it has been optimized by PSO, so got the more precise location of unknown nodes. Simulation results show that this optimized algorithm outperforms traditional bounding box on the positioning accuracy and localization error. Keywords: Node localization, wireless sensor networks, PSO, bounding box, localization error.

1 Introduction Wireless sensor network integrates multifold subjects including microelectronic, wireless communication and wireless network, and is widely used in industrial controlling, military, medical assistance fields [1-3]. In most applications, determining the physical positions of sensor nodes is the basic requirements. With regard to localization algorithms used for estimation location, there are two categories: range-based and range-free [4, 5]. Because of the hardware limitations of WSN devices, solutions in range-free localization are being pursued as a cost-effective alternative to more expensive range-based approaches. In this paper, we firstly describe the Particle swarm optimization [6]; then propose a localization algorithm in wireless sensor networks based on PSO; at last analysis and simulation show that the proposed algorithm can improve the accuracy of the estimated location.

2 A Localization Algorithm in WSNs Based on PSO In this section, a range-based localization algorithm based on PSO for wireless sensor networks is described. A similar bounding-box localization algorithm is used to calculate a rectangular estimation area of unknown nodes, and then the coordinates of nodes in the area is generated randomly, these coordinates are regarded as initial population and are optimized by PSO, and we can get the more precise coordinates of unknown nodes. Y. Tan et al. (Eds.): ICSI 2011, Part II, LNCS 6729, pp. 200–206, 2011. © Springer-Verlag Berlin Heidelberg 2011

A Localization Algorithm in Wireless Sensor Networks Based on PSO

201

2.1 Communication Model In this paper, RSS (Received Signal Strength) [7] is used to measure distance between two nodes. The radio range is a circle in this model. The distance between the nodes is measured according to the attenuation of signal broadcasted in medium. Mathematical model of wireless communication channel is showed as follows:

⎛d ⎞ PL(d ) = PL(d 0 ) − 10n lg ⎜ ⎟ − xσ ⎝ d0 ⎠

(1)

Where d denotes the distance between transmitter and receiver; d0 denotes the reference node; n denotes the channel attenuation index and can be read out value from 2 to 4. Xσ denotes a variable of Gauss random noise; PL(d0) denotes the signal strength which is d0 away from the transmitter; PL(d) denotes signal strength which is d away from transmitter; PL(d0) can be got from experience or definition of hardware criterion. In this formula, the distance d is calculated by signal strength PL(d). 2.2 Assumptions

In order to research conveniently, some assumptions are put forward: (1) Each sensor node has a unique ID, sensor nodes are deployed irregularly; (2) Signal transmission model of sensor nodes is the ideal space of sphere; (3) All ordinary sensor nodes are homogeneous and the power and computing capacity are limited; (4) Anchor nodes can get two-dimensional coordinates information by means of GPS [8, 9] devices or other devices; (5) All sensor nodes and the anchor nodes are time-synchronized. 2.3 Algorithm Description

In general, most localization algorithms adopt the communication model which deems that the communication region of a node in a two dimension space is a circle. However, the bounding box algorithm uses squares instead of circles to bound the possible positions of a node. An example of this algorithm is depicted in Fig.1. For each anchor node i, a bounding box is defined as a square with its center at the position of this node (xi,yi), with sides of size 2h (the side length of the internal connecting square of the circle mentioned above) and with four corners’ coordinates of ( xi − h, yi − h), ( xi − h, yi + h),( xi + h, yi − h), ( xi + h, yi + h) respectively. The intersection of all bounding boxes can be easily computed without any need for floating point operations by taking the maximum of the low coordinates and the minimum of the high coordinates of all bounding boxes, expressed by formula (5) and denoted by the shaded rectangle in Fig.1.

y floor = max( yi − h);

yceiling = min( yi + h);

xleft = max( xi − h); xright = min( yi + h);

(2)

202

H. Li et al.

Fig. 1. Sketch map of bounding box algorithm

So we can figure out the rectangular estimation range of unknown node from the formula (6) as follows:

EstimateScopei = [ xleft xright y floor yceiling ]; i = 1, 2,..., N

(3)

With the scope of the EstimateScope, get the xrandom between the xleft and the xright，get the yrandom between the yfloor and the yceiling，so we can figure out：

EstimateCoordinatei = [ xrandom yrandom ]; i = 1, 2,..., N

(4)

Therefore, each unknown node has a value of EstimateCoordinate. If there are N numbers of Unknown nodes, we can randomly get a group of EstimateCoordinate value at a time, randomly do this on that way, M times there will be M groups. Each group is regarded as a particle, all of the M particles is optimized by PSO, we can figure out the more precise coordinate value of the unknown nodes. Despite the final error of this algorithm, computing the scope of rectangles uses fewer processor resources than computing the intersection of circles.

3 Simulation and Analysis 3.1 Simulation Environment and Parameters

In this set of experiments, we deployed 300 sensor nodes including anchor nodes and unknown nodes randomly distributed in a two-dimension rectangular area of 1000*1000m2(shows in figure 2). We assume the transmission range is fixed by 200m for both unknown nodes and anchor nodes. In the simulation of the position algorithm, we assume the channel attenuation index n of 4. Figure 3 shows the connectivity relation of sensor nodes, which leads to an average connectivity of 31.8933 and adjacent anchor node numbers of 6.5533. In the graph, points represent nodes and edges represent the connections between neighbors who can hear each other.


203

Fig. 2. Node distribution

Fig. 3. Connectivity relation

3.2 Performance Comparison

In this section, traditional bounding box and optimized bounding box localization algorithm will be simulated in same parameters environment (original node coordinates, ratio of anchor nodes, node density and communication radius) and the different performance of positioning accuracy and localization error will be analyzed. Figure 4 shows the localization error of different nodes before optimization and after optimization. All of the unknown nodes have been estimated by bounding box algorithm.

204

H. Li et al.

Fig. 4. Localization error of different nodes

Fig. 5. comparison between non-optimization and optimization

The green represent the localization error of different nodes before optimization and then the blue represent the localization error of different nodes after optimization. We can see clearly from Figure 4 that localization error of almost all nodes decrease correspondingly by optimization. This is due to the fact that in the bounding box algorithm, the final position of the unknown node is then computed as the center of the intersection of all rectangular estimation range, however, after estimated coordinates are optimized by PSO, we can get the more precise coordinate value of the unknown nodes.


205

Fig. 6. Original position and optimized position of Unknown nodes

Figure 5 shows the average location accuracy of the traditional bounding box algorithm compared with optimized algorithm, which shows us that the latter is more preferable to the former. Figure 6 shows original position and optimized position of unknown nodes. The circles represent the true location of the nodes, and the squares represent the estimated location of the nodes which have been optimized by PSO. The longer the line, the larger the error is. In this graph, we can see that the estimated location of the nodes is more near to the true location after optimization.

4 Conclusion Localization is an important issue for WSNs. To reduce the localization error and improve the accuracy of the estimated location, a localization algorithm in WSNs based on PSO is proposed. In this paper, after the rectangular estimation range of unknown node is calculated by bounding box, the final position of the unknown node is not computed by the center of the intersection of all rectangular estimation range, but got by the position of the unknown node randomly within rectangular estimation range and then optimized by PSO. Analysis denotes that this scheme requires small amount of calculation and simulation results shows that optimized algorithm is superior to the traditional bounding box on the positioning accuracy and localization error.

206

H. Li et al.

Acknowledgements. This work was supported in part by the National Science Foundation of China under grant no. 40971233.

References 1. Akyildiz, I.F., Su, W., Sankarasubramaniam, Y., Cayirci, E.: A Survey on Sensor Networks. IEEE Commun. Magn. 40, 102–114 (2002) 2. Tubaishat, M., Madria, S.: Sensor Networks: An Overview. IEEE Potentials 22(2), 20–23 (2003) 3. Basagni, S., Carosi, A., Melachrinoudis, E., Petrioli, C., Wang, Z.M.: Protocols and model for sink mobility in wireless sensor networks. ACM SIGMOBILE Mobile Computing and Communications Review 10, 28–30 (2006) 4. He, T., Huang, C., Blum, B.M., Stankovic, J.A., Abdelzher, T.: Range-Free Localization Schemes for Large Scale Sensor Networks. In: 9th Annual International Conference on Mobile Computing and Networking, pp. 81–95. IEEE Press, San Diego (2003) 5. You, Z., Meng, M.Q.-H., Liang, H., et al.: A Localization Algorithm in Wireless Sensor Networks Using a Mobile Beacon Node. In: International Conference on Information Acpuisition, pp. 420–426. IEEE Press, Jeju City (2007) 6. Eberhart, R.C., Kennedy, J.: A New Optimized using Particle Swarm Theory. In: 6th International Symposium on Micromachine and Human Science, pp. 39–43. IEEE Press, Piscataway (1995) 7. Chen, H., Ping, D., Xu, Y., Li, X.: A Novel Localization Scheme Based on RSS Data for Wireless Sensor Networks. In: Advanced Web and Network Technologies, and Applications, pp. 315–320. IEEE Press, Harbin (2008) 8. Bulusu, N., Heidemann, J., Estrin, D.: GPS-less low-cost outdoor localization for very small devices. IEEE Personal Communications 7(5), 28–34 (2000) 9. Capkun, S., Hamdi, M., Hubaux, J.P.: GPS-Free Positioning in Mobile Ad-Hoc Networks. In: 34th Annual Hawaii International Conference on System Sciences, pp. 255–258. IEEE Press, Maui (2001)

Game Theoretic Approach in Routing Protocol for Cooperative Wireless Sensor Networks Qun Liu, Xingping Xian, and Tao Wu College of Computer Science, Chongqing University of Posts and Telecommunications, Chongqing, China [email protected], [email protected], [email protected]

Abstract. A game theoretic method, called the first price sealed auction game, was introduced to control routing overhead in wireless sensor networks in this paper. The players of the game are the wireless nodes with set of strategies (forward or not). The game is played whenever an arbitrary node in the network forwards packets. In order for the game to function, a multi-stage pricing game model is established, this provides the probability that the wireless nodes forward the receiving packets, and the payoff of all nodes can be optimize through choosing the best neighbour node. The simulations in NS2 showed that the pricing routing game model improves performance, not only decreasing the energy consumption, but also prolonging network life time. Finally the numerical analysis about nodes’ payoff is given through Matlab. Keywords: Game Theory, Packet Forwarding, Routing, Auction, Incentive Mechanism.

1 Introduction Wireless sensor networks(WSNS) have received significant attention in recent years. The main features of WSNS are that of low-cost nodes with limited resources both in terms of computational power and battery whose purpose is sensing the environment. In order to decrease the energy consumption, Numerous routing protocols have been introduced for wireless sensor networks. Our approach in this paper falls into selectively balancing the forwarding overhead on nodes by applying Game Theory. Game theory is a mathematical method that attempts to mathematically capture and analyze the behavior in strategic situations, in which an individual's success in making choices depends on the choices of others. It ensures that the desired global objective is achieved in the presence of selfish agents. Game Theory is not new to the area of telecommunications and wireless networks. It has been used to model the interaction among users, to solve routing and resource allocation problems in a competitive environment, it provides incentive mechanism for high energy nodes to cooperate with other nodes in transferring information in networks. In [1-4], authors gave the relative researches on these problems. In this paper, a routing model based pricing game is presented. In order to use first price sealed auction game model to select relay node, we first organize the node and its neighbour nodes into an incomplete information auction game, the node is buyer Y. Tan et al. (Eds.): ICSI 2011, Part II, LNCS 6729, pp. 207–217, 2011. © Springer-Verlag Berlin Heidelberg 2011

208

Q. Liu, X. Xian, and T. Wu

and its neighbour nodes are sellers. Then, we evolve the auction game as a multi-stage game model. Finally, we get a reliable routing path through relay node selection in each stage game. In each stage game, the Sink offers to the buyer a payment which compensates for its energy consumption and service reward (Sink-pay). The buyer purchases service from sellers and selects the relay node by evaluating each seller, the selected seller will get a payment for the forwarding service (relay-reward). The stability of the routing path is very important. So the choice of the relay node is a key problem. From a game theory perspective, such a stable configuration corresponds to Nash Equilibrium [5]. In our algorithm, we first choose the best node to forward packets in each stage, and then all the selected nodes form a stable path through multi-stage game. So our algorithm decreases the energy consumption and prolongs network life time. The rest of this paper is organized as follows. Section 2 surveys the related network. Section 3 introduces the network model and game model. Section 4 provides the design of pricing routing game model. Section 5 elaborates the proposed algorithm. Section 6 evaluates the algorithm with numerical analysis and simulation. At last, we conclude the paper in the section 7.

2 System Model 2.1 Network Model Consider a multi-hop networks consisting of one buyer node and several seller nodes, we model the WSNs as an undirected graph G =< V , E > as illustrated in Fig. 1, where V denotes the node set and E represents the link set. Each link between nodes indicates whether each buyer-seller pair can communicate with each other. If they could, there is a link between them. Otherwise, there is no link between the two nodes. ei denotes the residual energy of node vi, hvi denotes the hop count from the node vi to the Sink node, and h(vs, v3) denotes the minimum hop count from vs to the Sink node through node v3. Each node saves local network information, including its minimum hop count to the Sink node, its residual energy, its neighbour nodes’ minimum hop count to the Sink node and its neighbour nodes’ residual energy. We present the node of vs information matrix in Table 1.

Fig. 1. Network topology

Game Theoretic Approach in Routing Protocol for Cooperative WSNs

209

Table 1. The node of VS information matrix Notate vs

Explanation The node vs’s information

Value

Explanation

(h(vsˈv1),es)=(3,es) (h(vsˈv3),es)=(3,es)

The minimum hop count of vs to the sink node through v1 and the residual energy of vs are (3, es). The minimum hop count of vs to the sink node through v3 and the residual energy of vs are (3, es).

Nvs (v1)

The neighbour node v1’s information of vs

(h(v1ˈv6),e1)=(3,e1) (h(v1ˈv2),e1)=(2,e1) (h(v1ˈv4),e1)=(2,e1)

The minimum hop count of neighbour node v1 to the sink node through v6 and the residual energy of v1 are (3, e1). The minimum hop count of neighbour node v1 to the sink node through v2 and the residual energy of v1 are (2, e1). The minimum hop count of neighbour node v1 to the sink node through v4 and the residual energy of v1 are (2, e1).

Nvs (v3)

The neighbour node v3’s information of vs

(h(v3ˈv2),e3)=(2,e3) (h(v3ˈv5),e3)=(2,e3)

The same with the above description.

2.2 Problem Description

In most cases, the key problem that gets energy efficiency and reliability is to design an efficient incentive for accelerating cooperation. Most of nodes would like to choose cooperation strategy in order to improve their payoff. Some of the nodes are likely to break their agreement and drop the packets after they get the payoff. Some existing mechanisms can't solve this problem well, and these mechanisms can expend many network resources due to the overhead packets’ transmitting frequently [5]. In WSNs, most of the nodes will serve as routers, the whole forwarding process is depicted as multi-stage game, each stage is composed of one buyer and several sellers, the buyer is the sending node and the sellers are receiving nodes. We model this network based on sealed auction game model, the whole auction game process in network is shown in Fig. 2. At first stage, the Sink needs the packets of source node, the source node acts as the buyer, and he will pay a certain price to buy the forwarding service from his neighbour nodes. After the selected neighbour node receives the packets and gets the profit, the auction game enters the second stage with new state and the sellers in stage one become buyers. The auction game is a typical strategic game with incomplete information, because the buyer knows the valuation of all the sellers, but any seller does not know valuation about others except his own.

Fig. 2. Auction game model

210


3 Pricing Routing Game Model We will state the first price sealed auction game model at this section, In each stage game, the players of the game are composed of one node and its neighbor nodes, the node is buyer and its neighbor nodes are sellers. The Sink will pay a certain price to the buyer who will send packets to the Sink, and the buyer will pay certain price to sellers to compensate for the forwarding service. Multi-sellers compete against each other to forward packets and improve their own profit. Definition 1. Define the link quality as r (ei (t ), hi , j ) = hi , j / ei (t ) , where ei (t ) denotes the residual energy at time t of node i . hi , j is the hop count of node i to the Sink node through node j . The smaller the value of r (ei (t ), hi , j ) , the higher the reliability of the link. Definition 2. Define the forwarding success rate of node i at time t as α i (t ) , α i (t ) = Psi (t ) / Pri (t ) , Psi (t ) is the number of the forwarded packets, and Pri (t ) is the total number of the received packets in a time unit before time t . In the cooperative multiuser forwarding game model, an important question needs to be considered: which node can forward packets reliably. Each player will get the reward of value b for sending packets. In an auction game, buyer and sellers give

their own price and quote at the same time. Define N i is the neighbor node set of node i, define the bidding function of node j in N as β j (hj ,k (t), r(ej (t ), hj ,k )) , given by i

β j ( r (e j (t ), h j , k )) = b ⋅ r (e j (t ), h j , k )

(1)

where h j , k is the hop count of node j to the sink node through neighbor node k , b is the payoff for forwarding packets, r (e j (t ), h j , k ) is the link quality of node j . Because the buyer knows the residual energy and the hop count of his neighbor nodes, he will give an average price according to the valuation information of all the neighbor nodes. The bidding function of buyer node i can be expressed as

ρi (r (ei (t ), hi , j )) = b ⋅ ∑ ( r (ei (t ), hi , j )) / Φ ( N i )

(2)

j∈ N i

where

∑ (r (e (t ), h i

i, j

)) is the total evaluation of all the neighbor nodes’ link quality,

j∈ N i

Φ ( N i ) is the number of i ’s neighbor nodes,

∑ (r (e (t ), h i

i, j

)) / Φ ( N i ) is the average

j∈ N i

link quality of each neighbor node. After the buyer and sellers quote price respectively according to their link quality, we can get the deal price ( ϕ (i , j ) ) of buyer i and seller j , given by

⎧β j (hj ,k , r(ej (t), hj ,k ))

ϕ(i, j) = ⎨ ⎩0

ρ(ei (t ), hi, j )) > β j (hj ,k , r(ej (t), hj ,k )) others

(3)


211

where ϕ (i , j ) is the deal price of buyer i and seller j . According to (3), when the quoted price of buyer i is larger than seller j , the buyer i buy forwarding service at j ’s price, or else, the deal price is 0, which means i doesn’t choose j as relay node. When there are multiple sellers, the seller with the lowest price is chosen by i , which is depicted as

ϕ ( i , j ) = M IN β j ( h j , k , r ( e j ( t ), h j , k ))

(4)

j∈ N i

3.1 The Payoff Function of Source Node as Buyer at First Stage Auction

Sink will give a price to the source nodes which send the packets to it, and it is the payment to compensate for energy consumption and service reward of source node. We define the payoff function of source node s at time t is

us (t ) = α s (t )[( hs , j ) 2 ⋅ b − ϕ ( s, j ) − ess (t ) / es (t )]

(5)

where ess (t ) is the energy consumed in sending packets of source node s , es (t ) is the residual energy of source node s .

( hs , j )

2

⋅b

is the payment to source node s

given by Sink, it makes the source node (buyer) give the lowest pay to sellers, then it can get the highest possible profit. We use ( h s , j ) 2 control the pay, since we must guarantee the value of payoff that source node gets is greater than zero, and ensure each selected seller get the fair payoff. As sensor node is rational and selfish, he refuses to send all packets or send part of packets to his next seller after he obtains the payment, and get more extra profit along with reducing energy consumed. In order to control this situation, forwarding success rate is introduced to control the node’s payoff, if the node wants to maximize its own payoff, it should forward packets as much as possible. If he refuses to give the price to his neighbor node, the neighbor node will drop all the packets which he sends, and then the number of forwarded packets is falling as well as the profit. According to the definition, we need that for source node vs , his utility ui (t ) ≥ 0. This implies 0
hop1) then Update(hop1); /*Insert the value of hop1 to Vi . hop*/ End if Else insert(Vi , hop1); End if hop2 = hop1++; Send(Vj , hop2 , BSHello); /*relay the information to other nodes.*/ Algorithm 2. Build Up and Update Energy Information Before each cluster head selection, for each node Vi ∈ N, sending a Hello information with node id, residual energy (Hello, Vi ,energy) to his neighbours, Send(Hello, Vi ,energy(ei)). 2. For each node Vi ∈ N receives a Hello information, recv(Hello, Vj Ni , ej ). 3. Find(Vj ); Update(Vj , ej);/*Update the energy of Vj*/ When the current cluster selection time is over, jump to step 1.

∈

The neighbour node table is built up at the cluster head selection stage. In the process of sending packets, each node will choose the proper relay node to forward packets according the neighbour node table. The buyer node and seller nodes will launch an auction for the forwarding service, and the seller nodes compete with each other to get profit by sending buyer node’s packets. The pricing routing game algorithm is given in algorithm 3.

5 Numeric and Simulation Results 5.1 Numerical Analysis

We use Matlab to analyse the node’s payoff which is affected by the forwarding success rate and link quality, and we use source node and his neighbour nodes game model to calculate source node’s payoff. We set the number of nodes is 50, the number of BS nodes is 1 and the mobility of nodes is static, the value of reward b is 1, the hop count of source node to Sink is 10, and the energy consumed of sending packets is 0.1. In Fig.3, it shows the payoff of source node at different forwarding success rate α . We can know that the payoff at α = 0.8 is higher than the payoff at α = 0.4 and α = 0.1 . The experiment results

214


show that larger the forwarding success rate, higher the payoff, and also show the source node’s payoff increases with his neighbour node’s residual energy increasing. Therefore, each node would like to choose the maximum energy neighbour node as his relay node to send packets to destination. Algorithm 3. Price Routing Game For each source node Vk select the relay node Vj ∈ Nk, and sends a information to his neighbour nodes. The source node estimates price ρ ( r ( e (t ), h

k

k , j )) , and his neighbours Nk estimate their

k

price, for node Vj ∈ Nk, the price is β j ( r ( e j (t ), h j , l )) Neighbour nodes send the price information to the source node, and source node judge the deal price by

ρ k (r (ek (t ), hk , j )) ≥ β j ( r (e j (t ), h j , l )) , finding the proper node as relay node, and give

the deal price ϕ ( k , j ) to the selected node. Sends data to Vj ,calculate the payoff u k (t ) of Vk. For each node Vj , if (Vj is the selected node) then Buy the link quality from neighbour nodes, give the deal price to the selected node; Send data to Vl Calculate the payoff of Vj, u j (t ) = α j (t )[ h j , k ) ⋅ b + ϕ ( k , j ) − ϕ ( j , l ) − e js (t ) / e j (t )] 2

Else The payoff u j (t ) =0 End if If the node is not the one which is described above, go to sleep. 80

α =0.8 70 60 50

payoff

α =0.4 40 30 20

α =0.1 10 0

0

1

2

3

4

5

energy

6

7

8

9

10

Fig. 3. The payoff of source node at different Fig. 4. The payoff of different hops of neighbour nodes and node’s energy forwarding success rate

The source node’s payoff with different hop count and different residual energy of neighbour nodes is given in Fig.4. We assume the value of forwarding success rate is 1 ( α =1), the hop count of source node is 10, the energy consumed of sending packets is 0.1, and the residual energy of source node is 8. Fig.4 indicates that the smaller the hop count of neighbour nodes and the larger the residual energy of neighbour nodes


215

are, the higher the payoff of source node is. We also conclude that the larger the hop count and the smaller the residual energy of neighbour nodes are, and the payoff of source node will decrease rapidly and nearly close to 0. In order to increase its payoff, the source node will choose the minimum hop count and the largest residual energy neighbour node as his relay node. 5.2 Simulation Results

In this section, we perform extensive simulation experiments to analyze the performance of our algorithm on NS-2. All the nodes randomly distribute in the network scene, the BS location is in the center of the network field, and all of the simulation parameters used in the simulations are summarized in Table 2. Fig. 5 shows the results of the residual energy profit of a randomly selected node in WSNs using LEACH protocol and our proposed algorithm respectively. Simulation results show that the randomly selected node’s energy consumes very quickly when the network uses LEACH protocol, and our algorithm is much better than the LEACH protocol on energy saving. This is because of LEACH protocol’s single hop communication strategy. In LEACH protocol, every node sends packets to its cluster head node, the cluster head communicates directly with the remote base station. Because the cluster head is far from the base station, the transmission power is large, and leads to excessive energy consumption [18]. We know that the energy consumption of single hop network is worse than the multi-hop network. We transmit packets through multi-hop communication to save the nodes’ energy, so the residual energy of the nodes with our algorithm is much greater than the LEACH protocol. Simulation results in Fig. 6 exhibit the profit of the sum of all active nodes’ residual energy in a period time. We can know that the sum of all active nodes’ the residual energy decays more rapidly in LEACH protocol than in our algorithm, and the variation of the sum of the residual energy is more stable in our algorithm. The LEACH protocol consumes more amounts of energy than our algorithm. In order to illustrate the stability of energy consumption in our algorithm well, we collect nine nodes randomly and describe their residual energy in a short period of time, as depicted in Fig.7. We can see from Fig.7 that the nine nodes’ residual energy with LEACH protocol is extremely unstable. Because cluster head selection is random in LEACH protocol, the number of the cluster head and their location are random. On the other hand, the cluster head communicates directly with the remote base station, the transmission power is large, and leads to excessive energy consumption, which overburden a small number of sensor nodes. however, in our algorithm, we transmit packets through multi-hop communication, and we use game model to select the best relay node, so the residual energy is steady in network with our algorithm. Therefore, the life time of sensor network with LEACH protocol is shorter than the network with our pricing routing game algorithm.

216

Q. Liu, X. Xian, and T. Wu Table 2. Simulation parameters

Parameter name Initial energy Pt_ Sensing Power_ Processing Power Simulation time BS energy Network area size Control packet length

Value 10 J 0.281838 w 0.015J 0.024J 200 s 100J 100×100 m2 200bits

10

1000 LEACH pricing routing game

9

princing routing game LEACH

900 800

8

600

energy

residual energy

700 7 6 5

500 400

4

300

3

200

2

100 0

5

10

15

20 25 30 simulation time

35

40

45

50

0

0

10

20

30

40

50

60

70

simulation time

Fig. 5. The residual energy profit of a randomly selected node

Fig. 6. The profit of the sum of all active nodes’ residual energy

10 9 8

residual energy

7 6 5 4 3 2

LEACH pricing routing game

1 0 15

16

19

24

35

37

40

41

49

node id

Fig. 7. The randomly selected node’s residual energy

6 Conclusion In this paper, we considered an energy constrained cooperative wireless sensor networks, and proposed a pricing routing game model based on the first price sealed auction game model. Through the pricing routing game model, we encourage the relay node to forward packets, each node aims at maximizing his payoff by choosing the optimal relay node. Compared to the LEACH protocol, our algorithm can enhance network’s life time effectively. In the next paper, we will discuss the network performance under the influence of the dishonest nodes and cooperative nodes respectively.


217

Acknowledgements. This work is supported by the National Natural Science Foundation of China under GrantNo. 60903213. Natural Science Foundation of Chongqing under GrantNo.CSTC2007BB2386.

References 1. Machado, R., Tekinay, S.: A survey of game-theoretic approaches in wireless sensor networks. Computer Networks 52, 3047–3061 (2008) 2. Liu, Q., Liao, X.F., et al.: Dynamics of an inertial two-neuron system with time delay. Nonlinear Dynamics 58(3), 573–609 (2009) 3. Komathy, K., Narayanasamy, P.: Best neighbor strategy to enforce cooperation among selfish nodes in wireless ad hoc network. Computer Communications 30(18), 3721–3735 (2007) 4. Jun, C., Xiong, N.X., Yang, L.T., He, Y.: A joint selfish routing and channel assignment game in wireless mesh networks. Computer Communications 31, 1447–1459 (2008) 5. Liu, H., Krishnamachari, B.: A price-based reliable routing game in wireless networks. In: Proceedings of the First Workshop on Game Theory for Networks, GAMENETS 2006 (2006) 6. Heinzelman, W.B., Chandrakasan, A.P., Balakrishnan, H.: An application-specific protocol architecture for wireless microsensor networks. IEEE Transactions on Wireless Communications 1, 660–670 (2002) 7. Zhong, S., Chen, J., Yang, Y.R.: A simple, cheat-proof, Credit-based System for Mobile Ad hoc Networks. In: Proceeding of IEEE INFOCOM, pp. 1987-1997 (2003) 8. Marti, S., Giuli, T.J., Lai, K., Baker, M.: Mitigating Routing Misbehaviour in Mobile Ad Hoc Networks. In: Proceedings of the Sixth Annual International Conference on Mobile Computing and Networking, MobiCom 2000 (2000) 9. Lu, Y., Shi, J., Xie, L.: Repeated-Game Modeling of Cooperation Enforcement in Wireless Ad Hoc Network. Journal of Software 19, 755–776 (2008) 10. Altman, E., Kherani, A.A., Michiardi, P., Molva, R.: Non-cooperative Forwarding in Adhoc Networks, Technical Report INRIA Report No.RR-5116 (2004) 11. Wang, B., Han, Z., Liu, R.: Stackelberg game for distributed resource allocation over multiuser cooperative communication networks. IEEE Trans. Mobile Computing 8(7), 975– 990 (2009) 12. Shastry, N., Adve, R.S.: Stimulating cooperative diversity in wireless ad hoc networks through pricing. In: Proc. IEEE Intl. Conf. Commun. (June 2006) 13. Zhong, S., Li, L., Liu, Y., Yang, Y.R.: On designing incentive-compatible routing and forwarding protocols in wireless ad-hoc networks an integrated approach using game theoretical and cryptographic techniques, Tech. Rep. YALEU/DCS/TR-1286, Yale University (2004) 14. Huang, J., Berry, R., Honig, M.: Auction-based spectrum sharing. ACM/Springer J. Mobile Networks and Applications 11(3), 405–418 (2006) 15. Chen, J., Lian, S.G., Fu, C., Du, R.Y.: A hybrid game model based on reputation for spectrum allocation in wireless networks. Computer Communications 33, 1623–1631 (2010) 16. Huang, J., Han, Z., Chiang, M., Poor, H.V.: Distributed power control and relay selection for cooperative transmission using auction theory. IEEE J. Sel. Areas Commun. 26(7), 1226–1237 (2008) 17. Chen, L., Szymanski, B., Branch, W.: Auction-Based Congestion Management for Target Tracking in Wireless Sensor Networks. In: Proceedings of the 2009 IEEE International Conference on Pervasive Computing and Communications (PERCOM 2009), Galveston, TX, USA, 9-13, pp. 1–10 (2009)

A New Collaborative Filtering Recommendation Approach Based on Naive Bayesian Method Kebin Wang and Ying Tan Key Laboratory of Machine Perception (MOE), Peking University Department of Machine Intelligence, School of Electronics Engineering and Computer Science, Peking University, Beijing, 100871, China [email protected], [email protected]

Abstract. Recommendation is a popular and hot problem in e-commerce. Recommendation systems are realized in many ways such as content-based recommendation, collaborative filtering recommendation, and hybrid approach recommendation. In this article, a new collaborative filtering recommendation algorithm based on naive Bayesian method is proposed. Unlike original naive Bayesian method, the new algorithm can be applied to instances where conditional independence assumption is not obeyed strictly. According to our experiment, the new recommendation algorithm has a better performance than many existing algorithms including the popular k-NN algorithm used by Amazon.com especially at long length recommendation. Keywords: recommender system, collaborative filtering, naive Bayesian method, probability.

1

Introduction

Recommendation systems are widely used by e-commerce web sites. They are a kind of information retrieval. But unlike search engines or databases they provide users with things they have never heard of before. That is, recommendation systems are able to predict users’ unknown interests according to their known interests[8],[10]. There are thousands of movies that are liked by millions of people. Recommendation systems are ready to tell you which movie is of your type out of all these good movies. Though recommendation systems are very useful, the current systems still require further improvement. They always provide either only most popular items or strange items which are not to users’ taste at all. Good recommendation systems have a more accurate prediction and lower computation complexity. Our work is mainly on the improvement of accuracy. Naive Bayesian method is a famous classification algorithm[6] and it could also be used in the recommendation field. When factors affecting the classification results are conditional independent, naive Bayesian method is proved to be the solution with the best performance. When it comes to the recommendation field, naive Bayesian method is able to directly calculate the probability of user’s possible interests and no definition of similarity or distance is required, while in Y. Tan et al. (Eds.): ICSI 2011, Part II, LNCS 6729, pp. 218–227, 2011. c Springer-Verlag Berlin Heidelberg 2011

A New Collaborative Filtering Recommendation Approach

219

other algorithms such as k-NN there are usually many parameters and definitions to be determined manually. It is always fairly difficult to measure whether the definition is suitable or whether the parameter is optimal. Vapnik’s principle said that when trying to solve some problem, one should not solve a more difficult problem as an intermediate step. On the other side, although Bayesian network[7] have good performance on this problem, it has a great computational complexity. In this article, we designed a new collaborative filtering algorithm based on naive Bayesian method. The new algorithm has a similar complexity to naive Bayesian method. However, it has an adjustment of the independence which makes it possible to be applied to the instance where conditional independence assumption is not obeyed strictly. The new algorithm provides us with a new simple solution to the lack of independence other than Bayesian networks. The good performance of the algorithm will provide users with more accurate recommendation.

2 2.1

Related Work Recommendation Systems

As shown in Table 1, recommendation systems are implemented in many ways. They attempt to provide items which are likely of interest to the user according to characteristics extracted from the user’s profile. Some characteristics are from content of the items, and the corresponding method is called content-based approach. In the same way, some are from the user’s social environment which is called collaborative filtering approach[12]. Content-based approach reads the content of each item and the similarity between items is calculated according to characteristics extracted from the content. The advantages of this approach are that the algorithm is able to handle brand new items, and the reason for each recommendation is easy to explain. However, not all kinds of items are able to read. Content-based systems mainly focus on items containing textual information[13], [14], [15]. When it comes to movies, the content-based approach does not work. Therefore in this problem, we chose collaborative filtering approach. Compared to content-based approach, collaborative filtering approach does not care what the items are. It focuses on the relationship between users and items. That is, in this method, items in which similar users are interested are considered similar[1],[2]. Here we mainly talk about collaborative filtering approach. Table 1. Various recommendation systems recommendation systems

content-based

collaborative filtering model-based memory-based

220

2.2

K. Wang and Y. Tan

Collaborative Filtering

Collaborative filtering systems try to predict the interest of items for a particular user based on the items of other users’ interest. There have been many collaborative systems developed in both academia and industry[1]. Algorithms for collaborative filtering can be grouped into two-general classes, memory-based and model-based[4], [11]. Memory-based algorithms essentially are heuristics that make predictions based on the entire database. Values deciding whether to recommend the item is calculated as an aggregate of the other users’ records for the same item.[1] In contrast to memory-based methods, model-based algorithms first built a model according to the database and then made predictions based on the model[5]. The main difference between model-based algorithms and memorybased methods is that model-based algorithms do not use heuristic rules. Instead, models learned from the database provide the recommendations. The improved naive Bayesian method belongs to the model-based algorithms while the k-NN algorithm which appears as a comparison later belongs to the memory-based algorithms. 2.3

k-NN Recommendation

k-NN recommendation is a very successful recommendation algorithm used by many e-commerce web sites including Amazon.com[2], [9]. The k-NN recommendation separates into item-based k-NN and user-based k-NN. Here we mainly talk about item-based k-NN popularized by Amazon.com. First an item-to-item similarity matrix using cosine measure is built. For each pair of items in the matrix, the similarity is defined as the cosine value of two item-vectors. The item-vectors’ M dimensions corresponding to the M users is one, which means the user is interested in the item, or zero otherwise. The next step is to infer each user’s unknown interests using the matrix and his known interests. The items most similar to his known interests will be recommended according to the matrix.

3 3.1

Improved Naive Bayesian Method Original Naive Bayesian Method

For each user, we are supposed to predict his unknown interests according to his known interests. User’s unknown interest is expressed in such a way. p(mx |mu1 , mu2 , · · ·)

(1)

When considering the user’s interest on item mx , we have mu1 , mu2 · · · as known interests. Of course, mx is not included by the user’s known interests. The


221

conditional probability means the possibility of the item mx being an interest of the user whose known interests are mu1 , mu2 , etc. In our algorithm, the items of higher conditional probability have higher priority to be recommended and our job is to compute the conditional probability of each item for each user. p(mx |mu1 , mu2 , · · ·) =

p(mx ) · p(mu1 , mu2 , · · · |mx ) p(mu1 , mu2 , · · ·)

(2)

We have the conditional independence assumption that p(mu1 , mu2 , · · · |mx ) = p(mu1 |mx ) · p(mu2 |mx ) · · · ·

(3)

In practice, comparison only occurred among the conditional probabilities of the same user where the denominators of equation (2) p(mu1 , mu2 , · · ·) are all the same and have no influence on the final result. Therefore its calculation is simplified as (4). p(mu1 , mu2 , · · ·) = p(mu1 ) · p(mu2 ) · · · ·

(4)

So the conditional probability can be calculated in this way. p(mx |mu1 , mu2 , · · ·) = p(mx ) · q,

(5)

where q=

3.2

p(mu1 , mu2 , · · · |mx ) p(mu1 |mx ) p(mu2 |mx ) = · · ··· p(mu1 , mu2 , · · ·) p(mu1 ) p(mu2 )

(6)

Improved Naive Bayesian Method

In fact, the conditional independence assumption is not suitable in this problem. Because the relevance between items is the theory foundation of our algorithm. p(mx ) in (5) shows whether the item itself is attractive, and q shows whether the item is suitable for the very user. In our experiment, it is revealed that the latter has more influence than it deserved because of the lack of independence. To adjust the bias we have p(mx |mu1 , mu2 , · · ·) = p(mx ) · q

cn n

(7)

n is the number of the user’s known interests and cn is a constant between 1 and n. The transformation makes the influence of the entire n known interests equivalent to the influence of cn interests, which will greatly decrease the influence of the user’s known interests. Actually, cn represents how independent the items are. The value of cn is calculated by experiments and for most of the n’s the value is around 3.

222

3.3

K. Wang and Y. Tan

Implementation of Improved Naive Bayesian Method

Calculation of prior probability. First we calculate the prior probability p(mi ). The prior probability is the possibility that the item mi is interesting to all the users. The algorithm 1 shows how we do the calculation. foreach item i in database do foreach user that interested in the item do ti = ti + 1; end p(mi ) = ti / TheNumberOfAllUsers; end Algorithm 1. Calculation of prior probability

Calculation of conditional probability matrix. In order to calculate the conditional probability, first the joint probability is calculated and then the joint probability is turned into conditional probability. The algorithm 2 shows how we do the calculation. foreach user in database do foreach item a in the user’s known interests do foreach item b in the user’s known interests do if a is not equal to b then ta,b = ta,b + 1; end end end end foreach item pair (a,b) do p(ma , mb ) = ta,b / TheNumberOfAllUsers; p(ma |mb ) = p(ma , mb )/p(mb ); end Algorithm 2. Calculation of conditional probability matrix

Making recommendation. Now we have the prior probability for each item and the conditional probability for each pair of items. The algorithm 3 will show how we make the recommendations. How to compute cn . As mentioned before, cn is calculated by experiments. That is, the database is divided into different groups according to the size of user’s known interest. For each group we use many cn ’s to do the steps above and choose the one with the best result. 3.4

Computational Complexity

The offline computation, in which prior probability and conditional probability matrices are calculated, has a complexity of O(LM), where L is the length of log


223

foreach user that needs recommendation do foreach item x do r(mx ) = p(mx ); foreach item ui in user’s known interests do p(mx |mui ) cn r(mx ) = r(mx ) × ( p(m )n ; x) end p(mx |mu1 , mu2 , · · ·) = r(mx ); end end Algorithm 3. Making recommendation

in which each line represent an interest record of a user and M is the number of items. The online computation which gives the recommendation of all users, also has a complexity of O(LM). Therefore the total complexity is O(LM) only.

4

Experiment

Many recommendation algorithms are in use nowadays. We have nonpersonalized recommendation and k-NN recommendation mentioned before to be compared with our improved naive Bayesian. 4.1

Non-Personalized Recommendation

Non-Personalized recommendation is also called top-recommendation. It presents the most popular items to all users. If no relevancy is there between user’s interests and the user, the Non-Personalized will be the best solution. 4.2

Data Set

The movie log from Douban.com is used in the experiment. It has been a nonpublic dataset up to now. The log includes 7,163,548 records of 714 items from 375,195 users. It is divided into matrix-training part and testing part. Each user’s known interest of testing part is divided into two groups. One of them is considered known and is used to infer the other which is considered unknown. The Bayesian method ran for 264 seconds and the k-NN for 278 seconds. Both of the experiments are implemented in Python. 4.3

Evaluation

We have F-measure as our evaluation methodology. F-measure is the harmonic mean of precision and recall[3]. Precision is the number of correct recommendations divided by the number of all returned recommendations and recall is the number of correct recommendations divided by the number of all the known interests supposed to be discovered. A recommendation is considered correct if it is included in the group of interests which is set unknown. It is to be noted that the value of our experiment result shown later is doubled F-measure.

224

4.4

K. Wang and Y. Tan

Comparison with Original Naive Bayesian Method

As it is shown in Figure 1, the improvement on naive Bayesian method has a fantastic effect. Before the improvement it is even worse than the non-personalized recommendation. After the improvement, naive Bayesian method’s performance is obviously better than the non-personalized recommendation at any length of recommendation.

Fig. 1. comparison with original naive Bayesian method

4.5

Comparison with k-NN

As it is shown in Figure 2, before the peak k-NN and improved naive Bayesian method have almost the same performance. But when more recommendations are made, k-NN’s performance declines rapidly. At the length larger than 45, k-NN is even worse than the non-personalized recommendation while improved naive Bayesian method still has a reasonable performance. 4.6

Analysis and Discussion

It is noticed that though there are great difference between different algorithms, the performances of all these algorithms turn out to have a peak. Moreover, the value of F-measure increases rapidly before the peak and decreases slowly after the peak. The reason for the rapid increase is that the recall rises and the precision is almost stable, while the reason for the slow decrease is that the precision reduces but the recall hardly increases.


225

Fig. 2. Comparison with k-NN

According to our comparison between ordinary and improved naive Bayesian method, the improvement on naive Bayesian method has an excellent effect. The result of ordinary naive Bayesian method is even worse than that of nonpersonalized recommendation. However, after the improvement the performance is obviously better than the non-personalized recommendation. It is concluded that there is a strong relevance between user’s known and unknown interests. The performance of non-personalized recommendation tells that the popular items are also very important to our recommendation. When a proper combination between two aspects is made, as it is in the improved naive Bayesian method, performance of the algorithm should be satisfactory. When the combination is not proper, it may lead to a terrible performance as it is shown in the ordinary naive Bayesian method. The comparison of improved naive Bayesian method and k-NN shows that the improved naive Bayesian method has a better performance than the popular kNN recommendation especially when it comes to long length recommendation. It is worth notice that the performance of two different algorithms are fairly close at short length recommendation, which leads to the conjecture that the best possible performance may have been approached though it calls for more proofs. Unlike short length recommendation, the performance of k-NN recommendation declines rapidly after the peak. It is even worse than the non-personalized recommendation at the length larger than 45. It is concluded that Bayesian method’s good performance is because of its solid theory foundation and better

226

K. Wang and Y. Tan

obedience of Vapnik’s principle while k-NN’s similarity definition may not be suitable for all the situations, which leads to the bad performance at long length recommendation.

5

Conclusion

In this article, we provide a new simple solution to the recommendation topic. According to our experiment, the improved naive Bayesian method has been proved able to be applied to instances where conditional independence assumption is not obeyed strictly. Our improvement on naive Bayesian method greatly improved the performance of the algorithm. The improved naive Bayesian method has shown its excellent performance especially at long length recommendation. On the other hand, we are still wondering what the best possible performance of a recommendation system is and whether it has been approached in our experiment. The calculation of cn is still not satisfactory. There may be a more acceptable way to get cn , which is not by experiments. All of these call for our future work. Acknowledgments. This work was supported by National Natural Science Foundation of China (NSFC), under Grant No. 60875080 and 60673020, and partially supported by the National High Technology Research and Development Program of China (863 Program), with Grant No. 2007AA01Z453. The authors would like to thank Douban.com for providing the experimental data, and Shoukun Wang for his stimulating discussions and helpful comments.

References 1. Adomavicius, G., Tuzhilin, A.: The next generation of recommender systems: A survey of the state-of-the-art and possible extensions. IEEE Transactions on Knowledge and Data Engineering (2005) 2. Linden, G., Smith, B., York, J.: Amazon.com recommendations: Item-to-item collaborative filtering. IEEE Internet Computing (2003) 3. Makhoul, J., Kubala, F., Schwartz, R., Weischedel, R.: Performance measures for information extraction. In: Proceedings of Broadcast News Workshop 1999 (1999) 4. Breese, J.S., Heckerman, D., Kadie, C.: Empirical Analysis of Predictive Algorithms for Collaborative Filtering. In: Proc. 14th Conf. Uncertainty in Artificial Intelligence (July 1998) 5. Hofmann, T.: Collaborative Filtering via Gaussian Probabilistic Latent Semantic Analysis. In: Proc. 26th Ann. Int’l ACM SIGIR Conf. (2003) 6. Kotsiantis, S.B., Zaharakis, I.D., Pintelas, P.E.: Machine learning: a review of classification and combining techniques. Artificial Intelligence Review (2006) 7. Yuxia, H., Ling, B.: A Bayesian network and analytic hierarchy process based personalized recommendations for tourist attractions over the Internet. Expert System With Applications (2009) 8. Resnick, P., Varian, H.R.: Recommender systems. Communications of the ACM (March 1997)


227

9. Koren, Y.: Factorization Meets the Neighborhood: a MultifacetedCollaborative Filtering Model. ACM, New York (2008) 10. Schafer, J.B., Konstan, J.A., Reidl, J.: E-Commerce Recommendation Applications. In: Data Mining and Knowledge Discovery. Kluwer Academic, Dordrecht (2001) 11. Pernkopf, F.: Bayesian network classifiers versus selective k-NN classifier. Pattern Recognition (January 2005) 12. Balabanovic, M., Shoham, Y.: Fab: Content-Based, Collaborative Recommendation. Comm. ACM (1997) 13. Rocchio, J.J.: Relevance Feedback in Information Retrieval. In: Salton, G. (ed.) SMART Retrieval System-Experiments in Automatic Document Processing, ch. 14. Prentice Hall, Englewood Cliffs (1979) 14. Pazzani, M., Billsus, D.: Learning and Revising User Profiles: The Identification of Interesting Web Sites. Machine Learning 27, 313–331 (1997) 15. Littlestone, N., Warmuth, M.: The Weighted Majority Algorithm. Information and Computation 108(2), 212–261 (1994)

Statistical Approach for Calculating the Energy Consumption by Cell Phones Shanchen Pang and Zhonglei Yu College of Information Science and Engineering, Shandong University of Science and Technology, Qingdao 266510, China [email protected]

Abstract. Energy consumption by cell phones has great effect on energy crisis. Calculating and optimizing the method of service that provided by the cell phone is essential. In our solution, we build up three main models. Transition model reflects the relationship between the change of energy and time; next we give the function of energy consumption during the steady state. Optimization approach structures the function of energy consumption and constructs the function with convenience degree to emphasize the convenience of cell phones. Using waste model we obtain the waste functions under different situations and get the total waste energy. Keywords: energy consumption, transition function, steady state, optimization approach, waste model.

1 Introduction Recently, the use of mobile computing devices has increased in computation and communication. With the development of cell phones, landline telephones are eventually given up. We have noticed that people’s charger stays warm even it is not charging the phone. All of these drain electricity. It’s not just wasting your money, but also adding to the pollution created by burning fossil fuels [1]. According to investigation, only 5% of the power drawn by cell phone chargers is actually used to charge phones. The other 95% is wasted when you leave it plugged into the wall, but not into your phone [2]. It is no doubts that calculate the energy consumption of cell phone and optimize the method of service that provided by the landline and the cell phone is significant for coping with the energy crisis. Although increases in the perceived likelihood of an energy shortage had no effect, increments in the perceived noxiousness or severity of energy crisis strengthened intentions to reduce energy consumption [3]. Over the last decades, in order to reduce the energy waste of communication Equipments, many academics and politicians had put forward some algorithms, methods, models and arguments for energy consumption. In literature [4], several authors compare the power consumption of an SMT (DSP) with a CMP (DSP) under different architectural assumptions; they find that the SMT (DSP) uses up to 40% less power than the CMP (DSP) in our target environment. To reduce the idle power, Eugene Shih, Palaver Buhl introduces a technique to increase the battery lifetime of a PDA-based phone by reducing its idle power, the power a device consumes in a Y. Tan et al. (Eds.): ICSI 2011, Part II, LNCS 6729, pp. 228–235, 2011. © Springer-Verlag Berlin Heidelberg 2011

Statistical Approach for Calculating the Energy Consumption by Cell Phones

229

"standby" state. Using this technique, we can increase the battery lifetime by up to 115%. In the paper, they describe the design of "wake-on-wireless" energy-saving strategy and the prototype device they implemented [5]. In this paper, we use available data to build up a transition model and interpret the steady state to study consequences of the change of electricity utilization after landlines are replaced by cell phones. Then we consider an optimal way of phone service through discussing three different cases and discover the convenience using cell phones instead of landlines. Besides, we use a population growth model and an economic growth model to predict the energy consumption by cell phones combining with the population and economy in the future. This paper is organized as follows. In section two, we design models of energy consumption by cell phones. Besides, we analyze and discuss the relationship between the energy consumption and the habit of people using cell phones. Section three is an application of the models for “Pseudo US”. We find that Americans waste plenty of oil because of their bad habits. In section four, we make a conclusion.

2 Design of Model With the development of technology, cell phone usage is mushrooming, and many people are using cell phones and giving up their landline telephones [6], [12]. Our model just involves the “energy” consequences of the cell phone revolution. Here, we make an assumption that every cell phone comes with a battery and a charger. We design models in consider with the current US, a country of about 300 million people. 2.1 Relative Definitions In this paper, we develop a model to analyze how the cell phone revolution impacts electricity consumption at the national level. The basic component of our model is the household. A household can exist in one of three disjoint states at a time. The three states are as follows: (1) Initial State: a household only uses landline telephones. (2) Acquisition State: when a household acquires its first cell phone. (3) Transition State: all household members have their own cell phones but the landline is retained. Definition 1. In order to describe the quality of countries’ phone system, we introduce the concept of “Convenience”. The phone’s convenience degree has a relationship with the mobile phone ownership. The larger the country’s mobile phone ownership, the more convenient the country’s phone system is. Definition 2. W e define “waste” in a simple way, it’s just the misuse of energy with nothing utilization. Take the “waste” of the electricity into three different cases: charging the cell phone when it turns on, continuing to charge cell phone after it has fully charged and leaving the charger plugged in but not charging the device. Definition 3. Compare with “waste”, we define “consumption” is that the common use of the energy, no matter with high or low utilization, there is no waste of energy.

230

S. Pang and Z. Yu

2.2 Transition Model If all the landlines are replaced by cell phones, there exists a change of electricity utilization. Here, we assume that each family only has one landline phone and each member just has one cell phone [7], and the energy consumption of the average cell phone remains constant. We except for those who don’t have cell phones that they belong to the family that owns a landline phone. If one loses his cell phone, he buys new cell phone immediately. The energy consumed by cell phones is calculated through the following formula:

W (t ) = ( H (t )mP1 − H (t ) P2 ) × t .

(1)

Where H (t ) is the number of landline users at time t, it also symbolize the amount of total family at time t, m is the average number of family members in the United States.

P1 is the average power of cell phones in the U.S. market. P2 is the average power of a single landline. As the change of W (t ) follows time t, it is possible that the energy consumption can reach a steady state. Here, the “steady state” means that the growth of energy consumption remains unchanged all the time. In mathematics, we calculate the derivate of function (1), expressed as W ′(t ) .Just because

H (t ) = H (t0 )e ρ ×t .

(2)

H (t0 ) Symbolizes the amount of total family at time t, we can consider it as a constant at time t0 . ρ is the population growth rate of mobile phone users. Based on both function (1) and (2), we can draw a conclusion that W ′(t ) . Generally, when W ′(t ) equals zero, the system reaches the steady state. Here, it is sure that H (t0 ) ≠ 0 and m ≥ 1 . For that, we conclude that W ′(t ) cannot equal to zero. Consequently, only when all landline users transform into mobile phone users, it can reach steady state. 2.3 The Optimal Way of Providing Phone Service Consider a second “Pseudo US” [8], a country with about the same economic status as the current US. However, this emerging country has neither landlines nor cell phones. We need to find an optimal way of providing them with phone service. 2.3.1 Excluding the Social Consequences We discuss the broad and hidden consequences of having only landlines, only cell phones, or a mixture of the two from an energy perspective. Firstly, we don’t consider the convenience of the cell phone. We can get the total energy consumption:


W = ω0 × ( P3 β0 + P4 β1 ) × T + There

ω0

ω1 m

× P5 × T .

is the population of America who own cell phones.

don’t have cell phones, so the sum of

231

(3)

ω1 is the population who

ω0 and ω1 is the total population. T

is the time

after charging from the beginning through to the next complete depletion when the power in the mobile phone completely exhausted. P3 is the power of a cell phone when maintain a cell phone call.

P4 is the power of a cell phone when it is not used. P5 is the

average power of a single landline.

β0

is the percentage of T when maintain a cell

β1 is the percentage of T when the cell phone is not used. There are three different conditions when ω0 , ω1 takes different values: (1) When ω0 = 0 , ω1 ≠ 0 . All of the people use landlines. At this time, ω1

phone call.

W=

m

×P 5 ×T .

(2) When ω0

≠ 0 , ω1 ≠ 0 . That shows that some people use landlines while others use

cell phones. We can express the whole enery consumption as function (3). (3) When ω0 ≠ 0 , ω1 = 0 . All of the people use cell

phones.

W = ω0 × ( P3 β 0 + P4 β1 ) . 2.3.2 Including the Social Consequences As the cell phone has many social consequences that landline phones don’t allow. So we take the convenience of the cell phone into account. Here is the function:

C = c0 × ω0 + c1 ×

ω1 m

, c0 + c1 = 1

(4)

c0 is the convenient degree of mobile phones. c1 is the convenient degrees of landline. We consider that min(W ) / max(C ) is the optimal way of providing phone

Where

service. 2.4 Modeling for Energy Wasted by Cell Phones Considering people who waste electricity in many ways, we divide them into three basic situations. In every situation, we can conclude a waste function, so we can calculate the accurate energy consumption. Here are the details: 2.4.1 Charge the Cell Phone When It Turns on Take it for granted that charging the cell phone when it turns on is a waste of electrical energy. From that, we have a waste function.

232

S. Pang and Z. Yu

w = p 3 × t1 × N ( t ) × γ 1

.

(5)

p3 the rated power of cell phones, t1 is the phone standby time. N (t ) is the total population of the United States at time t . γ 1 is the proportion of Americans who Where

charge the cell phone with it turns on. In order to find the functional relationship be-

w and t , the relationship between the populations N (t ) and t have to be confirmed firstly. Here, we can estimate the United States population N (t ) using a tween

Logistic model. 2.4.2 Charge the Cell Phone after It Is Fully Charged There is a proportion of Americans who continue to charge cell phones after they are fully charged [9]. We need to calculate the energy wasted by these people. We know that two-thirds electronic energy was wasted by chargers, so we have the second waste function:

2 w = × p 4 × t2 × N ( t ) × γ 2 . 3

(6)

p4 is the power of cell phone while continue to charge cell phone after it is fully charged.

t2 is the time of fully charged cell phones continue to charge per day. N (t )

is the same as above.

γ2

is the proportion of Americans who continue to charge cell

phone after it is full. 2.4.3 Charger Plugged in without Device Some people prefer to plug the charger in whether they need to be recharged or not. In this case, we construct the model for wasted energy as follows:

w = p5 × t3 × N ( t ) × γ 3 Where

.

(7)

p5 is the power of cell phone while leaving the charger plugged in but not

charging the device. the device per day.

t3 is the time of leaving the charger plugged in but not charging

γ3

is the proportion of people who left the charger plugged in but

not charging the devices. We can simply take the power of charger as the main power while continuing to charge after the cell phone is fully charged.

3 Application of Model According to energy assumption that, the growth rate of mobile phone users is as same as the economic growth rate. The above discussion is the current situation. Now consider population and economic growth over the next 50 years.


233

3.1 Prediction Model For each 10 years for the next 50 years, predict the energy needs for providing phone service based upon your analysis in the first three requirements. Again, assume electricity is provided from oil. Interpret your predictions in term of barrels of oil. We use the population and economic growth model to explain the energy needs for providing phone service based upon the analysis in the first three parts. Solow neoclassical model of economic growth adds to the quality of labor and capital elements of the quality of the elements use the construction of Cobb-Douglas production function model [10], we can get the model:

α ( λ ) = α (1 + λ ) Using

α (λ )

.

(8)

and the recursive type, we get data as follows.

Fig. 1. Energy Consumption

Fig.1 shows the energy needs over the next 50 years, and signifies the energy consumption in terms of oil. 3.2 An Application of the Models for “Pseudo US” We consider a second “Pseudo US”-a country of about 300 million peoples with about the same economic status as the current US. Cell phones periodically need to be recharged. However, many people always keep their charger plugged in. Additionally, many people charge their phones every night, whether they need to be recharged or not. This causes a large quantity of energy consumption. Assume that the Pseudo US supplies electricity from oil. Take the situation that people continue to charge their cell phones after they are fully charged as an example, we can calculate the wasted energy

234

S. Pang and Z. Yu

according to formula (6). For a particular mobile phone, the battery capacity of cell phone C=850(mA) [11], [12], and the voltage of cell phone battery V=3.7 (v). Thus,

p 4 = ( C × V ) / 1000 = 3.145 w . Take t = 2009 , t2 =5 , γ 2 =5% [13]. Then we get the result w = 1.2721×10

8

J . According to oil calculation, American people waste B = w / w4 = 12.085 barrels per day through this way. Similarly, for the other two situations, they are 7.411 and 20.804 barrels respectively. Thus, American waste 40.3 barrels oil per day.

4 Conclusions From the models we build, with the landlines are replaced by cell phones, there exists a change of electricity utilization. We make a transition model to estimate the consumption of enery, and get the steady model that the growth of energy consumption remains unchanged. We realize that the amount of energy consumption of phones is very large. At present, the energy crisis becomes more and more serious. So, we have to make the most use of the energy and save the energy. However, our models still exist weaknesses. Our model doesn’t examine all household member dynamics, i.e., members getting born, growing old enough to need cell phones, moving out, starting households of their own, etc. Another is ignores Infrastructure. We do not examine the energy cost of cellular infrastructure (towers, base stations, servers, etc.) as compared to the energy cost of landline infrastructure (i.e. telephone lines and switchboards).

References 1. Robert, L.H.: Mitigation of Maximum World Oil Production: Shortage scenarios. Energy Policy 36, 881–889 (2008) 2. Mayo, R.N., Ranganathan, P.: Energy Consumption in Mobile Devices: Why Future Systems Need Requirements–Aware Energy Scale-Down. In: Falsafi, B., VijayKumar, T.N. (eds.) PACS 2003. LNCS, vol. 3164, pp. 26–40. Springer, Heidelberg (2005) 3. Hass, J.W., Bagley, G.S., Rogers, R.W.: Coping with the Energy Crisis: Effects of Fear Appeals upon Attitudes toward Energy Consumption. Journal of Applied Psychology 60, 754–756 (1975) 4. Stefanos, K., Girija, N., Alan, D.B., Zhigang, H.: Comparing Power Consumption of an SMT and a CMP DSP for Mobile Phone Workloads. In: The 2001 International Conference on Compilers, Architecture, and Synthesis for Embedded Systems (2001) 5. Eugene, S., Paramvir, B., Michael, J.S.: Wake on Wireless: An Event Driven Energy Saving Strategy for Battery Operated Devices. In: 8th Annual International Conference on Mobile Computing and Networking, pp. 160–171 (2002) 6. Singhal, P.: Integrated Product Policy Pilot Project. Nokia Corporation (2005) 7. Paolo, B., Andrea, R., Anibal, A.: Energy Efficiency in Household Appliances and Lighting. Springer, New York (2001) 8. Tobler, W.R.: Pseudo-Cartograms. The Am. Cartographer 13, 40–43 (1986)


235

9. Sabate, J.A., Kustera, D., Sridhar, S.: Cell-phone Battery Charger Miniaturization. In: Industry Applications Conference, pp. 3036–3043 (2000) 10. Meeusen, W., Broeck, J.: Efficiency Estimation from Cobb-Douglas Production Functions with Composed Error. International Economic Review 9, 435–444 (1977) 11. Toh, C.: Maximum Battery Life Routing to Support Ubiquitous Mobile Computing in Wireless ad hoc Networks. IEEE Communications, 138–147 (2001)

Comparison of Ensemble Classifiers in Extracting Synonymous Chinese Transliteration Pairs from Web Chien-Hsing Chen1 and Chung-Chian Hsu2 1

Department of Information Management, Hwa Hsia Institute of Technology, 111 Gong Jhuan Rd., Chung Ho, Taipei, Taiwan 2 Department of Information Management, National Yunlin University of Science and Technology, 123 University Road, sec. 3, Douliou, Yunlin, Taiwan [email protected], [email protected]

Abstract. There is no transliteration standard across all Chinese language regions, including China, Hong Kong, and Taiwan, and variations in Chinese transliteration have thus arisen in the process of transliterating foreign languages (English, for instance) into the Chinese language. In this paper, we compare several ensemble classifiers in confirming a pair, that is, a transliteration and another term, whether it is synonymous. We construct a new confirmation framework to confirm whether a pair of a Chinese transliteration and another Chinese term is synonymous. The presented framework is applied to extract synonymous transliteration pairs from a real-world Web corpus; this is valuable to build a new database of synonymous transliterations or support search engines so that they can return much more complete documents as Web search results to increase the usages in practice. Experiments show that our integrated confirmation framework is effective and robust in confirming and extracting pairs of Chinese transliteration following the collection of synonymous transliterations from the Web corpus. Keywords: Chinese transliteration variation, pronunciation-based approach, ensemble scheme, boosting, and bagging.

1 Introduction There is no transliteration standard across all Chinese language regions; thus, many different Chinese transliterations can arise. As an example, the Australian city “Sydney” also produces different transliterations of (xi ni), (xue li) and (xue li). Someone who uses the Chinese language may never know all these different Chinese synonymous transliterations; hence, this level of Chinese transliteration variation leads readers to mistake transliterated results or to retrieve incomplete results when searching the Web for documents or pages if a trivial transliteration is submitted as the search keyword in a search engine such as Google or Yahoo. Moreover, while variations in Chinese transliteration have already emerged in all Chinese language regions, including China, Hong Kong and Taiwan, we still lack effective methods to address this variation. Most research focuses on machine transliteration across two different

悉悉


雪雪

雪雪

Comparison of Ensemble Classifiers in Extracting Synonymous Chinese Transliteration

237

languages; in contrast, fewer efforts in the literature have focused on confirming a pair comprised of a Chinese transliteration term and a Chinese term (or another Chinese transliteration) as to whether it is synonymous. In this paper, we compare several ensemble classifiers in confirming a pair whether this pair is “synonymous” or “not synonymous”. We first construct an integrated confirmation framework (ICF) with considering the use of a majority voting scheme and a boosting scheme [1] together to robustly confirm pairs, since majority voting and boosting have been used to reduce noise and over fitting in terms of training classifiers. Then, the well-known ensemble classifiers, boosting [1] and bagging [2] are applied on this classification problem. The contribution of this research lies in that the results of the confirmation framework can be applied to construct a new database of synonymous transliterations, which can then be used to increase the size of the transliterated vocabulary, making it useful to expand an input query in search engines such as Google and Yahoo. This could alleviate the problem of incomplete search results stemming from the existence of different transliterations of a single foreign word.

2 Decision-Making 2.1 Similarity Evaluation among Sequences Based on Pronunciation Approaches Two major steps are included in the framework for the sake of confirming whether a pair is synonymous. First, we study two Romanization transcription systems, including the National Phonetic System of Taiwan (BPMF system) and the Pinyin system, to transcribe Chinese characters into sound alphabets. The BPMF system is used to transcribe a Chinese character into a phonetic sequence for the use of CSC [3] and LC [4]; the Pinyin system is used for ALINE [5], FSP [6]and PLCS [7]. Measuring similarity for two sets of sound alphabet sequences produces a similarity score between two transliterations. Assume that we have two Chinese transliterations A ={a1, …, an, …, aN} and B ={b1, …, bm, …, bM}, where an is the nth character of A and bm is the mth character of B. N may not be equal to M. The characters an and bm are formed into sound alphabet sequences an ={ an,1, …, a n,i, …, a n,I} and bm ={ bm,1, …, b m,j, …, b n,J}, respectively. The alphabets an,i and bm,j are generated by either the BPMF system or the Pinyin system. Second, we use a dynamic programming-based approach to obtain the similarity score for a given Chinese pair; that is, a Chinese transliteration versus another Chinese term. To acquire the maximum similarity score between two sets of sound alphabet sequence (formed from A and B, respectively), which is represented as score(A,B), a dynamic programming-based approach can be used to acquire the largest distortion between A and B by adjusting the warp on the axis of T(n,m) of sim(an,bm), which represents the similarity between an and bm. The recursive formula (1) is defined as follows. ܶሺ݊ െ ͳǡ ݉ െ ͳሻ ൅ ‫݉݅ݏ‬ሺܽ௡ ǡ ܾ௠ ሻ ܶሺ݊ǡ ݉ሻ ൌ ቐܶሺ݊ െ ͳǡ ݉ሻ ܶሺ݊ǡ ݉ െ ͳሻ

(1)

238

C.-H. Chen and C.-C. Hsu

where the base conditions are defined as { ,0 = 0 and 0, = 0. To avoid longer transliterations that appear to be more similar and thus acquire higher T(N,M), the similarity score must be normalized, taking into account the average length of the transliterations, as defined below. ,

, /

(2)

where the formula respects the similarity range [0,1]; accordingly, the two normalized scores in the above examples are 0.87 and 0.71, respectively. 2.2 Definition and Decision-Making Using a Similarity Entity be a pair consisting of Let X be a dataset containing a set of n data pairs, and let a transliteration and another Chinese term, which corresponds to class label yj Y, representing a synonymous pair or not a synonymous pair. Let M = {m1, …, mI} be a set of pronunciation-based approaches, where mi is the ith approach in M. For a pair , let scorej = {scorej,1, …, score j,I} be a set of similarity scores, where scorej,i is measured by mi (to use formula (2)) for xj, and then let vj = {vj,1, …, v j,I} be a set of decisions, where vj,i is a decision (i.e., a vote) taken from scorej,i. In particular, a pair xj has three entities, namely, yj, vj and scorej. The similarity entity scorej drives the decision entity vj. Most studies in the literature often take a vote, represented as vj,i that is accepted when scorej,i ≧ θi, whereas vote vj,i is rejected when scorej,I < θi. The parameter θi is a threshold. Determining a higher value for θi often brings higher precision but lower recall, whereas determining a lower value θi often brings lower precision but higher recall. Nevertheless, the determination of the appropriate parameters is usually empirical in many applications of information retrieval. , we use the K-nearest neighbor algorithm Instead of requiring the parameters to obtain vi with the help of scorej, because it provides a rule that xj can be classified according to its K nearest neighbor pairs; by the same token, the vote vj,i is assigned by with respect to , where “j → k” a majority vote on , , = yr in represents the kth nearest neighbor training pair of xj. Initially, we denote , advance if xr is a training pair. Since a majority-voting scheme is a well-known integrated voting approach to generate a final decision, it is applied to obtain a class label. The class label yj is determined using a majority-voting scheme on vj. In particular, the voting function h(xj) and is determines a predicted class label via a majority vote of , written as ∑ δ , argmax (3) where the function δ returns a Boolean value. 2.3 Hypotheses Combination on the Confirmation of Chinese Transliteration Pairs The ensemble framework proposed in this paper considers the use of multiple learning be approaches M = {m1, …, mI} and multiple data fractions X1, X2, …, XT. Let


239

a set of learning approaches based on the pronunciation model. Let be a set of training datasets generated by the use of a boosting scheme [8] fitting variant data distributions of a participative training dataset for accurate confirmation performance. Following the generation of variant data distributions, Xt is evolved from Xt-1 using a bootstrapping strategy in the training process. It is worth mentioning that a pair must be learned more frequently when it is not easy to confirm. In other words, a pair xj will appear much more possible in Xt while acquiring the wrong predication class label in Xt-1. In contrast, xj, while contributing the correct class label in Xt-1, may not appear in Xt. The final voting function , which is to integrate multiple votes of number T as a final vote, is performed while allowing T rounds. Thus, a T-dimensional voting vector is made for each xj and given via . Additionally, a learning round acquiring an accuracy rate lower than the random guess accuracy (be 1-1/|y|) will not contribute to the final vote. The function Hfin for xj is written as argmax

∑

|

.

δ

(4)

where ht represents an integrated vote for at the tth round, and the function δ returns a Boolean value. We extend h(.), which was mentioned in formula (6), as a weighted majority-voting function ht(.) related to the various contributions of the set of at the tth round. In addition, the extended formula must take the approaches parameter t into account. The extended equation is written as argmax

∑

δ

,

(5)

Providing different voting confidences for a repeatable learning procedure is indeed necessary. In other words, it is quite understandable that have different weights with respect to their capabilities in their corresponding learning spaces . Two weighted ; in addition, the capabilities of comparison approaches entities wt and are learned from the learning property for round t. We write (6) where

is the probability of training error at the tth round. In addition, we also write (7)

where is the probability of training error in the comparison approach mi at the tth round. The entities are good candidates for driving the data distribution for Xt. The xj obtaining the correct vote at round t will receive a lower probability value Dj(t+1) and will be less likely to be drawn at round t+1. Dj(t+1) is expressed as

‫ܦ‬௝ ሺ‫ ݐ‬൅ ͳሻ ൌ ൝

‫ܦ‬௝ ሺ‫ݐ‬ሻ݄௧ ሺ‫ݔ‬௝ ሻ ് ‫ݕ‬௝ ‫ܦ‬௝ ሺ‫ݐ‬ሻ ൈ

ఌ೟ ଵିఌ೟

(8)

240


3 Experiments 3.1 Preparation of the Training Dataset The data source is selected from the study in [3] in which the dataset contains a total of 188 transliterations collected from Web news sources. These transliterations are proper names, including geographic, entertainment, sport, political and some personal names. They are built as a set of pairs, some of which are synonymous and others of which are not synonymous pairs. In other words, the class label of each pair is known in advance. The pairs are constructed as a training dataset and are used for decision-making. In particular, a total number of 17,578 unique pairs (C ) is obtained. However, we only allow the length diversity of the pair to be one because the length of differentiation between a Chinese transliteration and its actual synonym is often at most one [3]. From this point of view, many pairs can be ignored without allowing the length diversity to exceed one; thus, we retain a total of 12,006 pairs, which include 436 actual-synonymous pairs and 11,570 pseudo-synonymous pairs (i.e., pairs that are not synonymous). In order to reduce the likelihood of participative training data driving confirmation performance as well as to ignore the influences of an imbalanced training dataset, we perform a validation task involving ten different datasets selected from the training data by sampling without replacement and thus ensure the number of positive pairs is the same as the number of negative ones. Therefore, ten training datasets, each of which includes 436 positive pairs and 436 negative ones, are used for the experiments. 3.2 Description of the Input Transliterations Two datasets, D50 and D97, are used for the experiments in [9] and contain transliterations. The second dataset, referred to as D97, is from the 2008 TIME 100 list of the world's most influential people. There are a total of 104 names in the list, since four entries include two names. Ninety-seven names are retained for the experiment. Seven names are ignored, namely, Ying-Jeou Ma, Jintao Hu, Jeff Han, Jiwei Lou, Dalai Lama, Takahashi Murakami, and Radiohead. The first five have Chinese last names that have standard Chinese translations. The sixth term is a Japanese name for which translation is usually not done using transliteration. The last name is that of a music band; its translation to Chinese is not according to its pronunciation, but its meaning. 3.3 Constructing Pairs from the Web In this experiment, we input the transliterations in D50 and D97 to collect their synonyms from a real-world Web corpus using the integrated confirmation framework proposed in this paper. For each transliteration, we collected Web snippets by submitting a search keyword to the Google search engine. The search keyword is used to retrieve Web snippets; however, it does not contribute information to the confirmation framework, which determines whether a pair is synonymous. To construct a pair, we use the original term of the given transliteration as a search keyword, because the original term is able to retrieve appropriate Web documents in which the transliteration’s synonyms appear. Let a transliteration (abbreviated as TL)


241

be an entry. The TL’s original term (abbreviated as ORI), which is treated as the search keyword for the search engine, is represented as QOri and is submitted to retrieve search result Web snippets, represented as DOri. The set DOri is limited to Chinese-dominant Web snippets. The procedure of returning a pair by collecting Web snippets from the Google search engine is as follows. A. For each TL in D50 and D97, we use QORI to download Web snippets DORI. In particular, we set |DORI | to 20 for each TL because the snippets appearing at the head of the returned snippets are often more relevant to the research keyword. The size of the downloaded DORI for D50 is 1,000, whereas the size of the downloaded DORI for D97 is 1,940. B. We delete known vocabulary terms with the help of a Chinese dictionary for DORI and apply an N-gram algorithm to segment Chinese n-gram terms for the remaining fractional sentences in DORI. Furthermore, most synonymous transliterations (TLs with their STs) have the same length, but some of them have different lengths of at most one [3]. Therefore, we retain the Chinese terms from DORI while controlling for length. Each Chinese term of length N is retained, with N=|TL|-1 to N=|TL|+1 and N ≥ 2.The number of remaining pairs for D50 is 9,439, whereas that for D97 is 19,263, where the pair consists of a given TL and a remaining Chinese n-gram term. C. However, some pairs have similarities that are not high enough and thus are never considered synonymous pairs. We set a similarity threshold to ignore those pairs. According to the findings in [3], a lower similarity threshold can be set to 0.5 by using the CSC approach to cover effectively all examples of synonymous transliterations. After discarding the pairs with similarities lower than 0.5, a total of 2,132 and 5,324 pairs are retained for D50 and D97, respectively. These pairs are confirmed by the use of the framework proposed in this paper and will be discussed in next section. 3.4 Confirmation of Synonymous Transliterations and Performance Analysis The experiments demonstrate whether the proposed framework is effective in extracting synonymous transliterations from the Web. The following nine approaches are employed for comparison in the experiments. z The integrated confirmation framework (ICF): This is the ensemble framework proposed in this paper. z The majority-voting approach (MV): This is a simple ensemble approach. We use equation (6) to perform this approach. z The individual approach: There are five approaches, CSC, LC, ALINE, FSP and PLCS, each of which is performed individually in the experiment. A feature vector with five dimensions generated using these five approaches can be performed for the experiments; hence, a classification-learning algorithm can be applied to predict the class label for each 5-tuple pair. The following two approaches are popular for analyzing experiments in the literature and are employed for comparison in this paper.

242


z Bagging [2]: This combines multiple classifiers to predict the class label for a pair by integrating their corresponding votes. The base algorithm for the classification we used is KNN with k=5 due to its simplicity. z Boosting [1, 8]: This requires a weak learning algorithm. We use KNN with k=5 in this study. ICF, bagging and boosting are the same in that they determine a parameter T, the number of iterations. One study [8] set the parameter T to 10 to use the boosting scheme. We follow the same setting for our experiments. A total of ten results are obtained for the testing data in the experiment, since we have ten training datasets involved in the validation process. The evaluator used for the experiment is the accuracy measure, which is common in a classification task. Moreover, we use a box-plot analysis to graphically employ a total of nine approaches, including ICF, boosting, bagging, MV, and five individual approaches (CSC, LC, ALINE, FSP and PLCS). The results are shown in Figure 1.

(a) D50

(b) D97

Fig. 1. Box-plot analysis for nine approaches in the testing datasets (a) D50 and (b) D97

In Figure 2, the experimental results show that the average accuracy in the confirmation of Chinese transliteration pairs for three ensemble approaches (namely, ICF, boosting, and bagging) is higher than that of the other individual approaches. This is because the three ensemble approaches allow repeated learning of the variant data distributions, whereas the individual approaches only perform the experiments once, driven by the participative training datasets. In addition, ICF achieves an average accuracy of 0.93 in D50 and 0.89 in D97 and is the best among nine approaches, because it considers several individual approaches together in evaluating variant data distributions. Otherwise, CSC achieves an average accuracy of 0.88 in D50 and 0.85 in D97 and is the best of the five individual approaches. Moreover, a shorter distance between the top and the bottom in a box-plot analysis demonstrates that ICF produces a much more stable performance than the others do; in contrast, bagging produces the most unstable performance among all ensemble approaches. This is because ICF best achieves learning objectives with variant data distributions. Otherwise, all five individual approaches produce a less stable performance than do the ensemble approaches, because they are seriously affected by the training datasets.


243

4 Conclusions In this paper, we propose a new ensemble framework for confirming Chinese transliteration pairs. Our framework confirms and extracts pairs of synonymous transliteration from a real-world Web corpus, which is helpful to support search engines such as Google and Yahoo for retrieving much more complete search results. Our framework considers the use of the majority-voting scheme and the boosting scheme together at the same time. The experimental results were evaluated according to the proposed framework in this paper, comparing boosting, bagging, general majority voting, and five individual approaches. The experimental results demonstrate that the proposed framework is robust for improving performance in terms of classification accuracy and stability.

References 1. Freund, Y., Schapire, R.E.: Experiments with a new boosting algorithm. In: Proceedings of the 13th International Conference on Machine Learning, pp. 148–156 (1996) 2. Breiman, L.: Bagging Predictors. Machine Learning 24, 123–140 (1996) 3. Hsu, C.C., Chen, C.H., Shih, T.T., Chen, C.K.: Measuring similarity between transliterations against noise data. ACM Transactions on Asian Language Information Processing 6, 1–20 (2007) 4. Lin, W.H., Chen, H.H.: Similarity measure in backward transliteration between different character sets and its applications to CLIR. In: Proceedings of Research on Computational Linguistics Conference XIII, Taipei, Taiwan, pp. 97–113 (2000) 5. Kondrak, G.: Phonetic alignment and similarity. Computers and the Humanities 37, 273–291 (2003) 6. Connolly, J.H.: Quantifying target-realization differences. Clinical Linguistics & Phonetics, 267–298 (1997) 7. Gao, W., Wong, K.-F., Lam, W.: Phoneme-based transliteration of foreign names for OOV problem. In: Su, K.-Y., Tsujii, J., Lee, J.-H., Kwong, O.Y. (eds.) IJCNLP 2004. LNCS (LNAI), vol. 3248, pp. 110–119. Springer, Heidelberg (2005) 8. Sun, Y., Wang, Y., Wong, A.K.C.: Boosting an associative classifier. IEEE Transactions on Knowledge and Data Engineering 18, 988–992 (2006) 9. Hsu, C.C., Chen, C.H.: Mining Synonymous Transliterations from the World Wide Web. ACM Transactions on Asian Language Information Processing 9(1), 1–28 (2010)

Combining Classifiers by Particle Swarms with Local Search Liying Yang School of Computer Science and Technology, Xidian University, Xi’an, 710071, China [email protected]

Abstract. Weighted combination model with appropriate weight vector is very effective in multiple classifier systems. We presented a method for determining the weight vector by particle swarm optimization in our previous work, which called PSO-WCM. A weighted combination model, PSO-LS-WCM, was proposed in this paper to improve the classification performance further, which obtained the weighted vector by particle swarm optimization with local search. We describe the algorithm of PSO-LS-WCM in detail. Seven real-world problems from UCI Machine Learning Repository were used in experiments to justify the validity of the approach. It was shown that PSO-LS-WCM is better than PSO-WCM and the other six combination methods in literature. Keywords: Multiple Classifier Systems, Combination Method, Particle Swarm Optimization, Local Search.

1 Introduction Combining classifiers is one of the most prominent techniques currently used to augment the accuracy of learning algorithms. Instead of evaluating a set of different algorithms against a representative validation set and selects the best one, Multiple classifier systems (MCS) are to integrate several models for the same problem. MCS came alive in the 90’s of last century, and almost immediately produced promising results [1][2]. From this beginning, research in this domain has increased and grown tremendously, partly as a result of the coincident advances in the technology itself. These technological developments include the production of very fast and low cost computers that have made many complex pattern recognition algorithms practicable [3]. A large number of combination schemes have been proposed in the literature [4]. Majority vote is the simplest combination method and has been a much-studied subject among mathematicians and social scientists. In majority vote, each individual has the same importance. A natural extension to majority vote is to assign weight to different individual. Thus weighted combination algorithm was obtained. Since under most circumstances, there is difference between individuals, weighted combination algorithm provides a more appropriate solution. The key to weighted combination algorithm is the weights. Two weighted combination models based on particle swarm optimization were proposed in our previous work [5][6]. In order to avoid the local optimum in PSO-WCM, a new weighted combination model is proposed in this paper, which cooperate PSO with local search to combine multiple classifiers. Y. Tan et al. (Eds.): ICSI 2011, Part II, LNCS 6729, pp. 244–251, 2011. © Springer-Verlag Berlin Heidelberg 2011

Combining Classifiers by Particle Swarms with Local Search

245

2 Particle Swarm Optimization with Local Search 2.1 Particle Swarm Optimization Everhart and Kennedy introduced Particle Swarm Optimization (PSO) in 1995, in which candidate solutions are denoted by particles [7][8]. Each particle is a point in the search space and has two attribute values: fitness determined by the problem and velocity to decide the flying. Particles adjust their flying toward a promising area according to their own experience and the social information in the swarm. Thus they will at last reach the destination through continuous adjustment in the iteration. Given a D-dimension search space, N particles constitute the swarm. The i-th particle is denoted by xi = ( xi1 , xi 2 ,..., xiD ), i = 1, 2,..., N . Taking xi into the objective function, the fitness for the i-th particle can be work out, which could tell the quality of current particle, i.e. the current solution. The current velocity and the best previous solution for the i-th particle are represented by vi = (vi1 , vi 2 ,..., viD ) and

pi = ( pi1 , pi 2 ,..., piD ) . The best solution achieved by the whole swarm so far is denoted by p g = ( p g1 , p g 2 ,..., p gD ) . In Everhart and Kennedy’s original version, particles are manipulated according to the following equations:

vid (t +1) = vid (t ) + c1r1 ( pid − xid (t ) ) + c2 r2 ( pgd − xid (t ) ) .

(1)

xid (t +1) = xid (t ) + vid ( t +1) .

(2)

where t is the loop counter;

i = 1,..., m ; d = 1,..., D ; c1 and c2 are two positive

constants called cognitive learning rate and social learning rate respectively;

r1 and

r2 are random numbers in the range [0,1]. The velocity vid is limited in [−vmax , vmax ] with vmax a constant determined by specific problem. The original version of PSO lacks velocity control mechanism, so it has a poor ability to search at a fine grain [9]. Many researchers devoted to overcoming this disadvantage. Shi and Eberhart introduced a time decreasing inertia factor to equation (1) [10]:

vid (t +1) = μ vid ( t ) + c1r1 ( pid − xid (t ) ) + c2 r2 ( pgd − xid ( t ) ) .

(3)

Where μ is inertia factor which balances the global wide-range exploitation and the local nearby exploration abilities of the swarm. Clerc introduced a constriction factor a into equation (2) to constrain and control velocities magnitude [11]:

xid (t +1) = xid (t ) + avid (t +1) .

(4)

The above equations (3) and (4) are called classical PSO, which is much efficient and precise than the original one by adaptively adjusting global variables.

246

L. Yang

2.2 Hill-Climbing Used as Local Search Algorithm in PSO Hill-climbing is a typical local search algorithm used in many fields, partially due to its easier implement and flexible transform in the particles. Aimed to avoid some demerits in classical PSO, such as relapsing into local extremum and low convergence precision in the late evolutionary, we adopted a hybrid algorithm of particle swarm optimization and hill-climbing algorithm, PSO-LS called in [12]. In PSO-LS, each particle has a chance of self-improvement by applying hill-climbing algorithm before it exchanges information with other particles in the swarm. Hill-climbing used as local search algorithm in our work is executed as follows. Procedure of hill-climbing algorithm for local search

Step 1. Initialization: select an initial solution w0 and specify the neighborhood function as follows.

wnew = wcurrent + r (1 − 2rand ()) .

(5)

where r represents the changing range of original solution. Rand() is random values between 0 and 1. Step 2. Set the max loop times T1 as a big enough number or according circumstance and the loop counter t1=1, set wcurrent=w0; Step 3. While t1 V max V min ≤ V i ( k + 1) ≤ V max V i ( k + 1) < V min

3.4 Updating Position

Updating position follows the Eq.(2) which adds the position vector and the velocity vector. And then the result is rounded. Finally, the result modulo 2 maps 0 and 1. 3.5 Fitness of Particle

The purpose of feature selection is to find the feature subset which has stronger classification ability. Fitness is the scale for evaluating the feature subset which is denoted by a particle. Fitness is composed by two parts: (a) testing accuracy. (b) numbers of selected features. For each particle h, the fitness is as follows:

fit ( h ) = 10 4 × (1 − acc ( h )) + k × ones ( h ) where the acc(h) is the classification accuracy. The classifier is constructed by the features selecting according to h. Ones(h ) is the number of 1 in h. The PSO is to find the global minimum. The higher accuracy means lower fitness. k is the parameter which balances the accuracy and feature number. The larger k means that the feature number is important. 3.6 Classifier Selection

For the better classification and generality ability, we consider selecting the support vector machines as classifier. The least square support vector machines converse the inequality constrain to equality constrain. So it is easy to solve. LS-SVM is selected as classifier in this paper.

280

S. Chuyi et al.

3.7 Algorithm Description

The steps of the proposed algorithm are as follows: Step 1. Set the size m of colony, set the maximum velocity Vmax and the minimum velocity Vmin . Step 2. Initialize the colony: For each particle, generate the number of 1 randomly and scatter 1 to the position vector. Generate the initial velocity according to Eq. (12) Step 3. For each particle, construct the LS-SVM classifier according to the selected feature. Calculate the accuracy and then calculate the fitness. Step 4. Compare each particle’s fitness value with that of the best position Pi that has been experienced, if it is better than the old one it will be selected to be the present best position Pi . Step 5. Compare each particle’s fitness with that of the present best global position Pg , if it is better than the current Pg , the index of Pg will be reset. Step 6. Update the velocity and position of each particle by Eq. (1) and Eq. (2). Step 7. If the maximum number of iterations is reached or the designated fitness is achieved, the process is stopped. Otherwise go to step 3.

4 Numerical Experience We use two datasets to demonstrate the performance of the proposed algorithm. The datasets are obtained from http://sdmc.lit.org.sg/GEDatasets/Datasets.html. Table 1 shows the details of the two datasets. The experiments are run on a Lenovo personal computer, which utilizes a 3.0GHz Pentium IV processor with 1GB memory. This computer runs Microsoft Windows XP operating system. All the programs are written in C++, using Microsoft's Visual C++ 6.0 compiler. We use the original datasets without normalization. The parameters that should be predetermined are as follows: The kernel ψ (⋅) for LS-SVM is chosen as a linear function ψ ( x , x i ) = ϕ ( x ) T ϕ ( x ) = x iT x .k=0.45 in Eq(13). The size m of colony is 100. The value of

Vmax in each dimension is 10 and

the value of Vmin in each dimension is 0.1. The performance of the proposed algorithm is summarized in Table 2. Table 1. Information of datasets

Datasets

Number of gens

ALLAML Leukemia

7129

Lung Cancer

12533

Number of train samples 38 ALL AML 27 11 32 MPM ADCA 16 16

Number of test samples 34 ALL AML 20 14 149 MPM ADCA 15 134

Feature Selection Algorithm Based on Least Squares Support Vector Machine

281

Table 2. Performance of the proposed algorithm

Datasets ALLAML Leukemia Lung Cancer

Number of selected gens

Testing error(%)

157

6.7628

135

4.2974

It can be seen from Table 2 that 2.20% genes are selected from ALLAML Leukemia dataset (157 genes are selected from 7129 genes). 1.07% genes are selected from Lung Cancer dataset (135 genes are selected from 12533 genes).

5 Conclusion A feature selection algorithm based on PSO and LS-SVM is proposed in this paper. LS-SVM performs well for classification problem. PSO is easy to solve and robust. The proposed algorithm combines the advantages of LS-SVM and PSO. PSO is used to select feature and LS-SVM is used to construct classifier. The accuracy for classification is the main part in fitness function. Numerical experiences show that this algorithm decreases the dimension of samples and improve the efficiency for classification. There are some further improvements for the proposed algorithm. For example, the distance between two positions, the added method for position vector and velocity vector, the initial colony method and so on. The further improvement algorithm will achieve better performance.

Acknowledgment This work was supported by the funds from National Natural Science Foundation of China (NSFC) (61073075, 60803052 and 10872077), the National High-Tech R&D Program of China (2009AA02Z307), Jilin University ("985" and "211" project, Scientific Frontier and interdisciplinary subject project (200903173)), Inner Mongolia Autonomous Region Research Project of Higher Education (NJ10118 and NJ10112).

References 1. Shoorehdeli, M.A., Teshnehlab, M., Moghaddam, H.A.: Feature Subset Selection for Face Detection Using Genetic Algorithms and Particle Swarm Optimization. In: Proceedings of the 2006 IEEE International Conference on Networking, Sensing and Control, pp. 686–690 (2006) 2. Huang, R., He, M.Y.: Feature Selection Using Double Parallel Feedforward Neural Networks and Particle Swarm Optimization. In: IEEE Congress on Evolutionary Computation, pp. 692–696 (2007) 3. Yu, H.L., Gu, G.C., Zhu, C.M.: Feature Gene Selection by Combining an Improved Discrete PSO and SVM. Journal of Harbin Engineering University 30(13), 1399–1403 (2009)

282

S. Chuyi et al.

4. Qiang, L.Y., Peng, X.Y., Peng, Y.: BPSO-SVM Wrapper for Feature Subset Selection. Acta Electronica Sinica 34(3), 496–498 (2006) 5. Dai, P., Li, N.: A Fast SVM-based Feature Selection Method. Journal of Shandong University (Engineering Science) 40(5), 60–65 (2010) 6. Kennedy, J., Eberhart, R.C.: Particle Swarm Optimization. In: IEEE International Conference on Neural Networks, pp. 1942–1948. IEEE Service Center, Piscataway (1995) 7. Shi, X.H., Wan, L.M., Lee, H.P., et al.: An Improved Genetic Algorithm with Variable Population-size and a PSO-GA Based Hybrid Evolutionary Algorithm. In: Second International Conference on Machine Learning and Cybernetics, pp. 1735–1740 (2003) 8. Eberhart, R.C., Kennedy, J.: A Discrete Binary Version of the Particle Swarm Algorithm. In: IEEE Conference on Systems, Man, and Cybernetics, vol. 5, pp. 4104–4109. IEEE Press, Orlando (1997) 9. Suykens, J.A.K., Vandewalle, J.: Least Squares Support Vector Machine Classifiers. Neural Processing Letter 9, 293–300 (1999) 10. Chua, K.S.: Efficient Computations for Large Least Square Support Vector Machine Classifiers. Pattern Recognition Letters 24, 75–80 (2003) 11. Ma, H.M., Ye, C.M., Zhang, S.: Binary Improved Particle Swarm Optimization Algorithm for Knapsack Problem. Journal of University of Shanghai for Science and Technology 28(1), 31–34 (2006)

Unsupervised Local and Global Weighting for Feature Selection Nadia Mesghouni, Khaled Ghedira, and Moncef Temani University of Tunis, LI3 Laboratory, ISG TUNIS, 92, Avenue 9 avril 1938, Tunis – 1007, Tunisia [email protected], [email protected], [email protected]

Abstract. In this paper we will describe a process for selecting relevant features in unsupervised learning paradigms using a new weighted approachs: local weight observation “OBS-SOM”, and global weight observation “GObs-SOM” This new methods are based on the self organizing map (SOM) model and feature weighting. These learning algorithms provide cluster characterization by determining the feature weights within each cluster. We will describe extensive testing using a novel statistical method for unsupervised feature selection. Our approach demonstrates the efficiency and effectiveness of this method in dealing with high dimensional data for simultaneous clustering and weighting. These models are tested on a wide variety of datasets, showing a better performance for new algorithms or classical SOM algorithm. We can also show that through deferent means of visualization, OBS-SOM, and GObs-SOM algorithms provide various pieces of information that could be used in practical applications. Keywords: Self Organizing Map, unsupervised learning, local weighting observation, global weighting observation.

1 Introduction Feature selection for clustering or unsupervised feature selection aims at identifying the feature subsets so that a model describing accurately the clusters can be obtained from unsupervised learning. This improves the interpretability of the induced model, as only relevant features are involved in it, without degrading its descriptive accuracy. Additionally, the identification of relevant and irrelevant variables with SOM [1, 2] learning provides valuable insight into the nature of the group-structure. Features selection or variables selection for clustering is difficult because, unlike supervised learning [3], there are no class labels for the dataset and no obvious criteria to guide the search. The important issue of feature selection in clustering is to provide variables which give the Memory based Weighted Topological Clustering "best" homogeneous clustering [4]. Therefore, we use the weight and prototype vector π[j] and w[j] provided by our proposed weighting approaches to cluster the map and to characterize each cluster with relevance variable. For map clustering we use Y. Tan et al. (Eds.): ICSI 2011, Part II, LNCS 6729, pp. 283–290, 2011. © Springer-Verlag Berlin Heidelberg 2011

284

N. Mesghouni, K. Ghedira, and M. Temani

traditional hierarchical clustering coupled to Davides-Bouldin index [5] to choose optimal partition. Thus, to select variables, we use original method which is based on statistical criteria named Scree Test to choose the most important variables for cell and cluster of cell [6]. In the following, we present a version of local variables weighting using SOM: weighting the distances. Firstly, we propose adaptive weighting approaches which were inspired from the w–k means and ω-SOM algorithms. After, we minimize the objective SOM function based on analytical method which allows us to obtain the SOM batch version; and we will present our proposed feature weighted approach for the adaptive SOM version distance weighing and weighing observation. Finally, we tested our model on a wide variety of datasets, showing a better performance for the OBS-SOM, GObs-SOM or classical SOM algorithm.

2 Adaptive Weighting SOM We proposed to use the principle of thewLVQ2 weighting technique and to adapt it to the Self-Organizing Maps using the stochastic version of the SOM algorithm. The minimization of the objective function is doing using the gradient descent techniques looking for a local minimum. This type of approaches is more efficient than the analytical type weighting, and this is because we use the adaptive weighting for the clustering characterization. We propose four types of adaptive approaches: local weighting observations; local weighting distance; global weighting observations, and global weighting distance. We proposed to use the principle of thewLVQ2 weighting technique and to adapt it to the Self-Organizing Maps using the stochastic version of the SOM algorithm. The minimization of the objective function is doing using the gradient descent techniques looking for a local minimum. This type of approaches is more efficient than the analytical type weighting, and this is because we use the adaptive weighting for the clustering characterization. We propose four types of adaptive approaches: local weighting observations; local weighting distance; global weighting observations, and global weighting distance.

3 Weighting Observations Weighting the observations during the learning process is a technique which allows giving more importance to the relevant features of the weighted observation. Consider the dataset X={x1, x2, x3, x4} and suppose that the observation x2 has a bigger relevance in the X. In this case the weighting approach must be able to assign a highest weight value to this one comparing to others three observations. For this type of approach we propose the both local and global weighting described in the next sections. 3.1 Local Weighting Observations: OBS-SOM We based our method on initial work describing the supervised modelw-LVQ2 [7]. This approach adapts weights to filter the observation during the learning process. Using this model, we weighted observations x using weight vectors π before computing the distance. So, the weight matrix will be considered like a filtering process for the observations. The objective function was rewritten as follows:

Unsupervised Local and Global Weighting for Feature Selection | | ∑| | R Obs-SOM ( ,W,π) = ∑

||

||²

285

(1)

Minimization of R OBS-SOM (χ,W,Π) was performed by iterative repetition of the following three steps until stabilization. The initialization step determines the prototype set W and the set of associated weights Π, at each training step (t+ 1). An observation xi is then randomly chosen from the input dataset and the following operations are repeated: Minimize R OBS-SOM (χ, , ) with respect to χ by fixing W and Π. Each weighted observation (πj, xi) is assigned to the closest prototype wj using the assignment function, defined as follows: (xi) = arg minj (||πjxi-wj||²)

(2)

Minimize R OBS-SOM (χ, , ) with respect to W by fixing χ and Π. The prototype vectors are updated using the gradient stochastic expression: Wj(t+1)= wj(t) + (t) Kjx(xi) (πjxi-wj(t))

(3)

Minimize R OBS-SOM (χ, , ) with respect to Π by fixing χ and W. The update rule for the feature weight vector πj (t+1) is: Πj(t+1)=πj(t)+ (t) Kjx(xi) (πjxi-wj(t))

(4)

As in the traditional stochastic learning algorithm of Kohonen, we denote the learning rate at time t by (t). The training is usually performed in two phases. In the first phase, a high initial learning rate (0) and a large neighborhood radius Tmax are used. In this c on d phase, a low learning rate and small neighborhood radius are used from the beginning. So, for a map there is a associated matrix of weights trained during the learning algorithm. If we want to obtain a global weighting observations algorithm (GObs-SOM) one changes this matrix in a vector. In this case we do not take care of the importance of each variable for assigned to each observation. The relevance vector do not depends on the cell and variables, but it depends on the map C: π= (π1,...,πd). We show the adaptive local process in the algorithm 1. Algorithm 1. The Obs-SOM learning algorithm Input: Data set X; Iter - number of iterations Initialization Phase: Randomly initialize the prototype matrix W; Randomly initialize the weight matrix Π; For t= 1 to Iter do Learning Phase: Present a learning example x and find the BMU computing the Euclidian distance; Updating Phase: Compute the new prototypes w using the expression (3); Compute the weights π using the expression (4). End for

286


3.2 Global Weighting Observations: GObs-SOM Weighting the clustering algorithm in a global way will allow us to weight the entire map with the same vector of weight. This is useful when we do not see to find the relevant features for each cluster, but we want to detect them for entire dataset or for the obtained map. The objective function is the same like for the adaptive learning Obs-SOM, changing only the weight matrix π in a vector of weights: RGObs-SOM( ,W,π)=∑

| |

∑|

|

(i) |i| |j| j (x ))||πx -w ||²

(5)

For each feature we have a corresponding numerical weight, where: Πj(t+1)=π(t)+ (t)Kjx(xi)(πxi–wj(t))

(6)

4 Experimental Results for the Cluster Characterization (Using the Adaptive Approaches: G/Obs-SOM) We have performed several experiments on five known problems from the UCI Repository of machine learning data bases: waveform, spam base, madelon, isolet and Wisconson cancer data base [8]. To evaluate the quality of clustering, we compared results to a "ground truth". We used the clustering accuracy form assuring the clustering results. In general, the results of clustering are usually assessed on the basis of some external knowledge about how clusters should be structured. The only way to assess the usefulness of a clustering result is in direct validation, where by clusters are applied to the solution of a problem and the correctness is evaluated. A gain to objective external knowledge. This procedure is defined by [9] as "validating clustering by extrinsic classification", and has been used in many other studies. To use this Approach we therefore need labeled data sets, where the external (extrinsic) knowledge is the class information provided by labels. Thus, the identification of significant clusters in the data, by Obs-SOM, will be reflected by the distribution of classes. A purity score can thus be expressed as the percentage of elements in a cluster that have been assigned a particular class. We also validated our approaches in supervised case learning paradigms. We used the K-fold cross validation technique, repeated s times for s= 5 and K= 3, to estimate the performance of G/ Obs-SOM. For each run, the dataset was split into three disjoint subsets of equal size (15 runs for each dataset). We used two subsets for training and then tested the model on there mining subset using all features and selected features (selected on the cells or on clusters). The labels generated were compared to the real labels of the test set for each run. We used the purity index to evaluate the quality of map segmentation. This index shows the correspondence between the class of data and cluster label, which is computed using the majority vote rule. A high value for this measure indicates a high level of homogeneous clustering. A purity index value close to 0 is indicative of poor clustering, whereas an index value close to 1 is indicative of a good clustering result. 4.1 Results on the Waveform Dataset for G/OBS-SOM We used this dataset to show a good level of performance for both algorithms (Dis-SOM and Obs-SOM) for simultaneous clustering and feature weighting. All

Unsupervised Local and Global Weighting for Feature Selection

287

observations were used to generate a map with 26×14 cells dimension. Both learning algorithms provided two vectors for each cell: the referent vector wj= (w1 j,w2j,...,wdj) and weight vector πj= (π1j,π2j,...,πdj), where d= 40. Preparing data for clustering requires some preprocessing, such as normalization or standardization. In the first experimentation step, we normalized the initial dataset (Figure 1(a)) to obtain more homogenous data (Figure 1 (b)). We used variance normalization, representing a linear transformation that scales the values such that their variance is equal to 1. We created 3D representations of the referent vector and weight vector provided by classical SOM and by our methods (G/Obs-SOM). The axes X and Y indicate the features and the referent indexes, respectively. The amplitude indicates the mean value of each component. Examination of the two graphs (3(c), 4(b)) shows that the noise represented by features 19 to 40 may be clearly detected with low amplitudes. This visual analysis of the results clearly shows that Obs-SOM algorithm provides the best results. Both graphs of weights Π and prototypes W show that features associated to noise is irrelevant with low amplitude. Visual analysis of the weight vectors (Figure 2(d)) showed the weight vectors obtained with Obs-SOM to give a more accurate. The Obs-SOM algorithm provides good results because the weight vectors work as a filter for observations and estimates the referents that result from this filtering. We applied the selection task to all parameters of the map before and after map clustering to check that it was possible to automatically select the features using our algorithms. This task involves detecting major changes for each input vector represented as a signal graph. We used hierarchical classification [10] for clustering the map. After Obs-SOM map clustering, we obtained three clusters with a purity index equal to 0.7076. This demonstrates that when there is no cluster (labels) information, feature weighting can be used to find and characterize homogeneous clusters. The importance of this index is that it can give us information about each clusters in a visually mode. The plot founded on the left part of the figure shows the wrong labeled observations. In the case of both global weighting algorithms, we can see that some noise features has a high value, and even for the Obs-SOM the first features (1-20) do not describe well the waves. This disadvantage compared to local weighted approaches is because the global weighting technique uses only a vector of weights for all the data, and respectively each sample vector will be weighted with the same vector of weights. After Obs-SOM map clustering with the referents W, which are already weighted, we obtain 3 clusters. The characterization of clusters with the "Scree Test" algorithm is provided in Table 1for each algorithm, we present the features selected for each cluster. Both techniques (Obs-SOM, GObs-SOM) provided three clusters characterized by different features. By contrast, segmentation of the map using classical SOM provided six clusters with a purity index value of 0.662. Map segmentation was performed using hierarchical clustering with all the features. For clusters cl1, cl2 and cl3, the features selected using Obs-SOM. We found that the algorithm Obs-SOM identified relevant and informative features, giving more accurate results than classical SOM. The new and classical methods were compared after segmentation of the map. We investigated the effect of selected features before and after, or without segmentation by testing this selection process in the supervised paradigm and computing the accuracy index for each method. In the case of global weighting approaches (G/Obs-SOM) we are not able to characterize each cluster because the weight vector are the same for all the prototypes, but we can detect the

288


relevant features for the whole map (dataset). We can see that the set of selected features using these global weighting algorithms (Table 1) represent the union of relevant features obtained with the local weighting approach for all the clusters. Table 1. Comparison of the selected variables using traditional and our approaches (G/ObsSOM). [i− j] indicates the set of selected variables. Database Wave-form

Real cluster 3

Gobs-SOM [3-20]

Obs-SOM Cluster1:[3-8;11-16] Cluster2:[8-11,14-19] Cluster3[3-20]

In order to evaluate the relevance of variable selected, we compute purity score by running a 3-fold cross-validation five times. Figure 3 shows the box plot indicating the purity scores calculated for each run with learning data set, using SOM, GObsSOM and Obs-SOM, We show also the result after clustering the corresponding map using hierarchical clustering. We observe that the Obs-SOM method has significantly better score compared to traditional SOM. The score is degraded after clustering the map, but Obs-SOM is still significantly better than traditional SOM. In the second time we evaluate the variable selected by assigning the data set (test part) during the cross validation task. Figure 3 shows the purity score using traditional SOM, and ObsSOM. The classification task are tested using all variables, the variables selected by cell and the variables selected after clustering map

(a)

Waveform database

(b) Normalized Waveform dataset

(c) W provided by SOM

Fig. 1. Waveform dataset

Unsupervised Local and Global Weighting for Feature Selection

(a) (W provided by GObs-SOM)

(c)Π provided by Gobs-SOM

289

(b) (W provided by Obs-SOM)

(d) Π provided by Obs-SOM

Fig. 2. 3D visualization of the referent vector and weight vector. The axes X and Y indicate features and the referent index values, respectively. The amplitude indicates the mean value of each component of map 26×14 (364 cells).

Fig. 3. Comparison of purity score (classification accuracy with learning dataset) using SOM, GObs-SOM and Obs-SOM before and after clustering map

290


5 Conclusion In this paper, we have described a process for selecting relevant features in unsupervised learning paradigms using these new weighted approaches. These new methods are based on the SOM model and feature weighting. Both learning algorithms Obs-SOM, and Gobs-SOM provide cluster characterization by determining the feature weights within each cluster. We described extensive testing using a novel statistical method for unsupervised feature selection. Our approaches demonstrated he efficiency and effectiveness of this method in dealing with high dimensional data for simultaneous clustering and weighting. The models proposed in this paper were tested on a wide variety of datasets, showing a better performance for the Obs-SOM, and Gobs-SOM algorithms or classical SOM algorithm. We also showed that through different means of visualization, Obs-SOM, and Gobs-SOM, algorithms provide various pieces of information that could be used in practical applications. The global weighted approaches are used in the case of analysis of the entire clustering result and not each cluster separately.

References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11]

Kohonen, T.: Self-organizing Maps. Springer, Berlin (1995) Vesanto, J., Alhoniemi, E.: Clustering of the selforganizing map. IEEE Neural Networks Kohonen, T.: Self-organizing Maps. Springer, Berlin (2001) Frigui, H., Nasraoui, O.: Unsupervised learning of prototypes and attribute weights. Pattern Recognition 37(3), 567–581 (2004) Yacoub, M., Bennani, Y.: Features selection and architecture optimization in connectionist systems. IJNS 10(5) (2000) Cattell, R.: The scree test for the number offactors. Multivariate Behavioral Research 1, 245–276 (1966) Yacoub, M., Bennani, Y.: Features selection and architecture optimization in connectionist systems. IJNS 10(5) (2000) Asuncion, A., Newman, D.J.: Uci machine learning repository (2007) Jain, A.K., Murty, M.N., Flynn, P.J.: Data clustering: a review. ACM Computing Surveys 31(3), 264–323 (1999) Vesanto, J., Alhoniemi, E.: Clustering of the self-organizing map. IEEE Transactions on Neural Networks 11(3), 586–600 (2000) Yacoub, M., Bennani, Y.: Une mesure de pertinence pour la sélection de variables dans les perceptrons multicouches. RIA, Apprentissage Connexionniste, pp. 393–410 (2001)

Graph-Based Feature Recognition of Line-Like Topographic Map Symbols Rudolf Szendrei, Istv´ an Elek, and M´ aty´ as Márton ELTE University, Faculty of Informatics, Budapest [email protected], {elek,matyi}@map.elte.hu http://team.elte.hu/

Abstract. Paper-based raster maps are primarily for human consumption. Todays computer services in geoinformatics usually require vectorized topographic maps, while the usual method of the conversion has been an error-prone, manual process. The system in development separates the recognition of point-like, line-like and surface-like objects, and the most successful approach appears to be the recognition of these objects in a reversed order with respect to their printing. During the recongition of surfaces, homogeneous and textured surfaces must be distinguished. The most diverse and complicated group constitute the line-like objects. In this article, a possible method of the conversion is discussed for line-like topographic map objects. The results described here are partially implemented in the IRIS project, but further work remains. This emphasizes the tools of digital image processing and knowledge-based approach. Keywords: Geoinformatics, topographic maps, raster-vector conversion.

1

Introduction

Paper-based raster maps are primarily appropriate for human usage. They always require a certain level of intelligent interpretation. In GIS applications vectorized maps are preferred. Especially, government, local authorities and service providers tend to use topographic maps in vectorized form. It is a serious challenge in every country to vectorize maps that are available in raster format. This task has been accomplished in most countries — often with the use of uncomfortable, “manual” tools, taking several years. However, it is worth dealing with the topic of raster-vector conversion. On one hand, some results of vectorization need improvement or modification. On the other hand, new maps are created that need vectorization. The theoretical background of an intelligent raster-vector conversion system has been studied in the IRIS project [2]. Several components of a prototype system has been elaborated. It became clear very early that the computer support of conversion steps can be achieved at quite different levels. For example, a map symbol can be identified by a human interpreter, but the recognition can be Y. Tan et al. (Eds.): ICSI 2011, Part II, LNCS 6729, pp. 291–298, 2011. c Springer-Verlag Berlin Heidelberg 2011

292

R. Szendrei, I. Elek, and M. M´ arton

attempted with a software, using the tools of image processing [3]. A computer system can be fairly valuable and usable even if every important decision of interpretation is made by the expert user. However, the system designed and developed by the authors is aimed at to automatize the raster-vector conversion of line-like symbols as much as possible. This aim gives an emphasis to a knowledge-based approach. This paper deals with a part of raster-vector conversion applied in cartography, with knowledge-based approach [1]. The line-like map symbols used in topographical maps will be introduced, together with the algorithms used to recognize them. The organization of expertise into knowledge base will also be presented. The following must be considered in connection with good quality and automated vectorization. Raster maps can be adequately understood only by human expert. After the vectorization, the relationships used for interpretation are no more contained in the vectorized map — it consists only of numerical and descriptive data. Automatic interpretation of image contents requires sophisticated image processing tools, whiches are not comparable to human perception in the majority of cases. Therefore, the level of automatic recognition must also be appropriately determined.

2

Line-Like Map Symbols

The topic of this article is how to interpret printed variant of line-like map symbols and how to represent them in computer systems. This process is considered basically as the result of interpretation and processing of map symbols. To accomplish this task it is very important to understand maps, and specifically, map symbols. To gain a comprehensive survey, refer to [5]. Although human cognition can not be completely understood, it is necessary to know to a certain extent how the human expert interprets graphical information. Regarding human perception, primarily points, lines (see Fig. 1) and textured surfaces are sought and distinguished. It must be realized that human perception may reveal finer or hidden information, for example how roads crossing at different levels hide each other. Human mind is also capable of abstraction, for example when it disregards the actual texture of surface, and investigates only its shape. Human eye can make some corrections, for example in the determination of shades of color layers printed over each other. Map interpretation process and the complexity of knowledge based object recognition can be visualized via any example of the four different object types — that is, point, line, surface and inscription. Line-like elements can cover larger area on map than their real size. For instance in the case of a highway, a zero-width center line can represent the theoretical position of the road in the database. Beyond the graphical properties of lines the database may contain real physical parameters, such as road width, carrying capacity, coating (concrete, asphalt) etc. Hiding is a very inherent phenomenon in maps when line-like objects, landmarks (typically roads, railways

Graph-Based Feature Recognition of Line-Like Topographic Map Symbols

293

and wires) located at different elevations intersect. This results in discontinuity of objects in map visualization. However, in map interpretation continuity must be assumed.

3

Recognition of Line-Like Symbols

Line-like symbols are usually the trace of a road like object, or edge of a polygon with a given texture/attribute. c)

a)

f)

d)

b)

12

e)

Fig. 1. Examples for line-like symbols: a) railway b) railway network at a railway station, c) highway under construction d) highway with emergency phone. e) road with a specified width f) bridge above a canal

Recognition of line-like symbols is one of the most difficult tasks of rastervector conversion. These symbols are often complex, and it is permitted for two symbols to differ only in their size, to cross each other or to join to form a single object (see Fig. 1a, b, respectively). Difficulties are posed by parallel lines belonging to the same object (see Fig. 1d, e) versus lines running in parallel which belong to separate objects. Further difficulties are the discontinuous symbols (see Fig. 1c, e, f). It is beyond the aim of the current article to solve all the difficulties mentioned above, so for the purpose of this paper we assume that line-like symbols 1. do not cross each other, 2. do not join to form a single object, and 3. are continuous. A classic way of line-like symbol vectorization is introduced in [4], where cadastral maps in binary raster image format are vectorized. The additional features of color topographic maps, like road width, capacity, coating etc. can not be recognized the classical way [6]. Each of these features are represented by a corresponding graphics, color and structure. The following method is able to recognize the trace of the line-like symbols of a topographic map. 1. 2. 3. 4.

Do image segmentation and classify each pixel. Create a binary map, where each black pixel belong to eg. road. Apply thinning and morphological thinning on the binary map. Vectorize the one pixel thin skeletons.

The first step is the segmentation, which works as follows. Define an object color set O and a surface color set S. The amount of colors in each color set is approx.

294


a)

0 0 0 0 0 b) 1 1 0 1 1 1 1 1

1 c) 1 1 1 d)

1 1 1

Fig. 2. a), b) structuring elements of morphological thinning based skeletonization, c), d) structuring elements of morphological fork detection on binary images. Values of the elements can be: 0 - background, 1 - foreground or undefined.

5-7 in the case of topographic maps. We assume that on a printed map each pixel color can be defined as a linear combination of a surface and an object color. In optimal case, this can be written as a c = α ∗ co + (1 − α) ∗ cs equation, where c is the value of the current pixel, and co , cs are the respective object and surface colors, so the segmentation can be done by solving the minimalization task min |c − α ∗ co + (1 − α) ∗ cs | for each pixel. o∈O,s∈S

As the second step is a simple selection on the segmented pixels, it can be done easily. The third step consists of two different thinning methods. A general thinning method is used first to avoid creating unneeded short lines by morphological thinning. The general thinning can be described as it iteratively deletes pixels inside the shape to shrink it without shortening it or breaking it apart. Because the result of the general thinning algorithm may contain small pixel groups, a morphological thinning should be performed. This morphological thinning can be done by using the structuring elements shown in Fig. 2. At each iteration, the image is first thinned by the left hand structuring element (see Fig. 2 a) and b) ), and then by the right hand one, and then with the remaining six 90◦ rotations of the two elements. The process is repeated in cyclic fashion until none of the thinnings produces any further change. As usual, the origin of the structuring element is at the center. The skeletonized binary image can be vectorized in the following way. Mark all object pixels black and surface pixels white. Mark those black pixels red, where N(P 1)> 2, and then mark the remaining black fork points blue by using structuring elements c) and d) of figure 2 in the same way as structuring elements are used in morphological thinning. The red fork points are connecting lines, while blue fork points are connecting other fork points. Mark green each black pixel, if at most one neighbour of it is black (end point of line segment). It can be seen that a priority is defined over the colors as white < black < green < red < blue. The following steps vectorize the object pixels 1. Select a green point, mark white and create a new line segment list, which contains that point. 2. Select a black neighbour if it exists and if the current point is also black. Otherwise select a higher priority point. Mark white the point and add to the end of the list. 3. Go to Step 2, while a corresponding neighbour exists. 4. Go back to the place of the first element of the list and go to Step 2. Be careful that new points should be added now to the front of the list. (This step processes points in the opposite direction.)


295

5. Go to Step 1, while a green point exists. 6. Select a black point, mark white, and create a new line segment list, which contains that point. 7. Select a black neighbour of the current point, mark white, and put it at the end of the list. 8. Go to Step 7, while a black neighbour exists. 9. Select a red point p, mark white and create a new line segment list, which contains that point. Let N eighbourSelect = RedSelect = Counter = 0, BlueF irst = false, where = back, q = p. 10. Let P revP oint = q. 11. If the N eighbourSelectth neighbour r of q exists, let q = r, let BlueF irst = (Steps = 0 and where=back), let n = q, and increment N eighbourSelect by 1. Put q into the list at where and go to Step 13. 12. If the RedSelectth neighbour r of q exists, (a) If q and n are neighbours and where =front, then let q = P revP oint and increment RedSelect by 1. Go to Step 10. (b) Put q into the list at where, mark q white, let N eighbourSelect = 0 and increment Counter by 1. Go to Step 10. 13. If where=back, then let where=front, q = p and go to Step 10. 14. Go to Step 9, while a red point exists. Although, the algorithm above vectorizes all the objects, it merges the several object types and colors. Hence, pixels of a given object color are copied onto a separate binary image before they are vectorized. We introduce an approach, which is able to recognize the features of line-like objects, so the corresponding attributes can be assigned to them. This assumes that the path of each object exists in the corresponding vector layer. In order to recognize a specific feature, its properties should be defined for identification. Two properties of vectorised symbols are recognized: forks (F ∼ F ork), and end-points (E ∼ End). Both are well known in fingerprint recognition where they are called minutiae. In the case of fingerprints, a fork means an end-point in the complement-pattern, so only one of them is used for identification. In our case, we can not define a complement-pattern, so both forks and end-points are used. Representation of line-like symbols is based on weighted, undirected graphs. An EF -graph is an undirected graph with the following properties: – Nodes are either of type E or F . The color of a node is determined by the corresponding vector layer. – Two nodes are connected if the line-segment sequence connecting the nodes in the corresponding vector layer does not contain additional nodes. Edges running between nodes of different colors can be defined by the user (in case of multicolor objects). The weight of the edge is equal to the length of the road connecting the two nodes, and it has the color of the corresponding symbol part. – There are special nodes, denoted by an index P , which occur on the trace of a line object. These will be used to produce the final vector model.

296


E

E

4, Black

EP

4, Black

FP

4, Black

4, Black

E

E

4, Black

FP

8, Black

4, Black

E

4, Black

FP

4, Black

EP

4, Black

E

E

E

4, Black

FP

4, Black

FP

4, Black

4, Black

E

4, Black

FP

4, Black

FP

4, Black

E

Fig. 3. The EF graph and the elementary EF graph of a double railway line. Distances are relative values and refer to the scale of the map. The dashed line in the elementary EF graph represents its cyclical property.

An EF -graph can also be assigned to the vectorised map, not only to the vectorised symbols, where line-like symbols are not separated to their kernels. For recognition we use the smallest units of the symbol, called the kernel. The smallest unit is defined as the one which can be used to produce the entire symbol by iteration. In the EF -graph there is usually only two nodes participating in the iteration; these are type F with only a single edge, so become the entry and exit points to the graph. In the very few cases, where the entry and exit points of the smallest unit can not be identified, the kernel of the line-like object is itself. Smallest unit can not be defined for the whole vectorised map. Figure 3 shows how a symbol is built up from its smallest units by iteration. Weights represent proportions and depend on the scale of the map. Beside weights, we can assign another attribute to edges, their color. In the figure almost all edges are coloured black. The recognition of line-like objects is reduced to an extended subgraph isomorphism problem: we try to identify all the occurrences of the EF graph of the symbol (subgraph) in the EF graph of the entire map. The weights of the EF graphs are normalized with respect to the scale of the map, and the collection is sorted in decreasing order of node degrees. Call this collection of sorted EF graphs S. Since the EF graphs created to maps do not contain the edges those connecting nodes with different colors, this case should be handled. In this article, the potential edges are identified by searching the corresponding neighbour on its own layer in the given distance of the node. The validity of a found potential edge is verified by comparing the color of the edge and the color of the segmented image pixels lying under the edge. Subject to the conditions above it is possible to design an algorithm for the recognition of subgraphs. While processing the map, recognized objects are removed, by recoloring the corresponding subgraph. Two colors, say blue and red, can be used to keep track of the progress of the algorithm and to ensure termination.


297

The following algorithm stops when there are no more red nodes left in the graph. 1. Choose an arbitrary red node U with the highest degree from the EF graph of the map. 2. Attempt to match the subgraph at node U against S, that is the sorted collection of EF graphs, until the first successful match in the following way: (a) Perform a “parallel” breadth-first search on the EF graph of the map and the EF graph of kernel of the current symbol with origin U . This is called successful if both the degree of all nodes match, and weights are the same approximtely. (b) In the case of success, all matching nodes become blue, otherwise they remain red. Upon successful matching the EF -graph of the symbol is deleted from the EF graph of the map. Entry and exit points must not be deleted unless they are marked as E, and the degree of remaining F nodes must be decreased accordingly. The equality of edge weights can only be approximate, due to the curvature of symbols. The algorithm above can be implemented, as it keeps track the given object by using the line segments as paths in vector data. The other difficulty is that edges with differently colored nodes are not directly defined by the vector layers. In practice, we have created a spatial database, which has contained the vectorized line segments and their color attribute. The potential edges was determined by a query, looking for existence of a neighbour with the right color in a given distance from the corresponding node.

4

Results

In this article a feature extraction method is introduced for line-like symbol vectorization in the IRIS projects. The project aims to automate and support the recognition of raster images of topographic maps, with the combination of digital image processing and a knowledge-based approach. The interpretation of line-like symbols is the most difficult issue in topographic map vectorization. An EF graph representation is developed, which is used for the recognition of curved, line-like objects with regular patterns. Fork-free line, Recognized pattern, Recognized circle, Unrecognized pattern

Fig. 4. Recognition results of a Hungarian topographic map line networks at map scale 1:10000 and scanning resolution of 300dpi

298


The method was tested on a 6km × 4km section (6990 × 4680 pixels) of a large scale 1:10 000 topographic map (see Fig. 4). In our experience some spatial filters, like Kuwahara and conservative smoothing improved the quality of segmentation. During a large number of tests, symbols completely appearing on maps were identified at a high rate (> 90%), while symbols disappearing partially, (like junctions and discontinuities) remained mostly unidentified. In the latter case, the level of identification can be enhanced with some heuristics using the neighbour segments.

Acknowledgement The work was supported by the European Union and co-financed by the European Social Fund (grant agreement no. TAMOP 4.2.1./B-09/1/KMR-20100003).

References [1] Corner, R.J.: Knowledge Representation in Geographic Information Systems. Ph.D. thesis, Curtin University of Technology (December 1999) [2] Dezs˝ o, B., Elek, I., M´ ari´ as, Z.: IRIS, Development of Automatized Raster-Vector Conversion System. Tech. rep., E¨ otv¨ os Lor´ and University and IKKK (November 2007) (in Hungarian) [3] Dezs˝ o, B., Elek, I., M´ ari´ as, Z.: Image processing methods in raster-vector conversion of topographic maps. In: Karras, A.D., et al. (eds.) Proceedings of the 2009 International Conference on Artificial Intelligence and Pattern Recognition, pp. 83–86 (July 2009) [4] Janssen, R.D.T., Vossepoel, A.M.: Adaptive vectorization of line drawing images. Computer Vision and Image Understanding 65(1), 38–56 (1997) ´ F¨ [5] Klinghammer, I., Papp-V´ ary, A.: old¨ unk t¨ ukre a térkép (Map, mirror of the Earth). Gondolat (1983) [6] Liang, S., Chen, W.: Extraction of line feature in binary images. IEICE Transactions on Fundamentals of Electronics Communications and Computer Sciences E91A(8), 1890–1897 (2008)

Automatic Recognition of Topographic Map Symbols Based on Their Textures Rudolf Szendrei, István Elek, and Istv´ an Fekete ELTE University, Faculty of Informatics, Budapest {swap,elek,fekete}@inf.elte.hu http://team.elte.hu/

Abstract. The authors’ research goal is to automatize the raster-vector conversion of topographic maps. To accomplish this goal, a software system is currently, under development. It separates the recognition of pointlike, line-like and surface-like objects. The first of these three topics is discussed in details in this paper. It is assumed that a topographic map and its vectorized form (possibly in rough form) are given. In this paper a method is introduced that is able to recognize the pointlike symbols of the map and to assign them as attributes to the corresponding polygon of the vectorized map. It means that point-like symbols should not appear as polygons in the vectorized data model. Instead, symbols appear as polygon attributes. The method presented here is also able to “clean” unnecessary polygons of symbols from vectorized map. This method is implemented by optimized pattern matching on the raster image source of the map, where the symbols are handled as special textures. This method will be improved by using a raw vector model and a set of kernel symbols. Keywords: Geoinformatics, map symbol recognition, image processing, pattern matching.

1

Introduction

This paper1 describes a method that recognizes symbols within the raster-vector conversion of maps [4]. Maps that contain topographic symbols are made from vector data models, because photos and remote sensing images contain map symbols only in raster form. If a map symbol is identified, then two transformation steps can be made automatically [1, 3]. First, the vectorized polygon of the map symbol will be removed from the vectorized map if it was mistakenly recognized as a line or polygon object. Next, the meaning of the removed symbol will be assigned as an attribute to the polygon of the corresponding real object i.e. surface in the vector data model. For instance, after removing the symbol “vineyard”, this attribute will be added to the boundary polygon of the “real” vineyard (see Fig. 1a ). In practice, the attributes of the polygons are stored in a GIS database. 1

´ This research was supported by the project TAMOP-4.2.1/B-09/1/KMR-2010-003 of E¨ otv¨ os Lor´ and University.


300

R. Szendrei, I. Elek, and I. Fekete

Fig. 1. a) Recognizing the symbol of vineyard, b) result of Prewitt filter

2

Main Steps of Raster-Vector Conversion

The raster-vector conversion of maps consists of three main steps. In the first step, the original colors appearing in the image will be transformed into a palette with reduced number of colors, which belong to the interpretation categories. This process can be set up as a series of image filters. These filters reduce the errors of the image, emphasize dominant color values or increase the distance between color categories. After these filters were applied, the intensity values of pixels are classified into color categories by clustering methods. During these steps the palette is evolved in such a way that the probability of the false pixel classification is minimal. The second step determines all edge vectors in the map with reduced number of colors. Edge filters and edge detectors, like Canny, Laplace or Prewitt methods, are frequently used to solve this problem. Using these filters local edges can be found and their direction can be determined. If a pixel is not considered as a local edge, it can be dropped or represented by a null vector. The last step is the processing of vectors. This means the extraction of all possible structural information and storing them with the vectorized map in the database. This step will build polygons or polylines based on the vectors determined for each pixel in the previous step. Experience shows that the most difficult part of raster-vector conversion is the third step. As an illustration, let us consider the case of roads on a map. The width of their polylines can be different according to their types. In this case, most software interpret them as polygons which have edges on both sides of roads because of their width. The width of a line on the map does not correspond to the width of the road that is represented by the line. This kind of false classification is well known, and even the recent applications do not yield complete solution to this problem.

3

Symbol Recognition

It is important to recognize those objects of the map that represent a symbol even if they look like lines or polygons. The texture based pattern matching algorithm developed by the authors will directly recognize these symbols. This algorithm also determines the positions of symbols on the map. The position is needed in order to query its corresponding polygon from the vector model. This polygon will be removed from the vector model and its attribute property (e.g. “vineyard”) will be assigned to the underlying polygon. A second query is required to determine the line or polygon that belongs to the symbol [2].

Automatic Recognition of Topographic Map Symbols

301

The attribute represented by the symbol has to be assigned to the corresponding object (a polyline or a surface) of the vectorized map. In order to do this, polylines and polygons should be handled differently. All segments of the polyline should inherit the attribute of the polyline symbol. The assignment to polygons is more sophisticated, because both the border and the interior of a polygon have to receive the attribute. The decision whether the attribute information is stored implicitly or explicitly is user dependent. The explicit and implicit storage means the assignment of attribute information only to the polygon or to all the polyline segments of the polygon border, respectively. Character recognition is a special case of symbol recognition [5]. It is assumed that maps have a legend of symbols on the map sheet or the map interpreter identifies the map symbols (see Fig. 2). A map can be represented as an m × n matrix, where each pixel described by a k-tuple of color components. It is assumed that a part of the map represents the symbol as a u × v matrix. It is possible that symbols are not rectangular. This difficulty can be handled by using another u × v matrix that represents a bitmask. This matrix determines which pixels of the symbol will be used during pattern matching. The following two sections will show a simple and an improved pattern matching method.

4

A Simple Pattern Matching Method

The basic method applies a brute force pattern matching as it tries to match the matrix of the symbol to each u × v matrix of the map. This is an inefficient solution, because it determines for each pixel of the map whether the pixel is a part of a symbol or not. Each map pixel can be covered by a u × v matrix in u ∗ v different ways. This leads to a number of u ∗ v pattern matching where each costs u ∗ v pixel comparisons. Thus, the runtime in pixel comparisons will be Tbf(m, n, u, v, k) = Θ((m ∗ n) ∗ (u ∗ v)2 ∗ k). In addition, this method works only if the symbols on the map have the same orientation as in the symbol matrix. Unfortunately, polylines mostly have transformed symbols in order to follow the curves of a polyline. Symbols on a map can be transformed in several ways that makes the matching more difficult. In the least difficult case an affin transformation was made to a symbol, e.g. it was rotated. However, it can be much more difficult to recognize the non-located symbols (e.g. railroads which continously follow the curves of the track). In this project only the problem of rotated symbols was treated. Without additional concrete or contextual information the rotated symbols can be identified if the matching symbol is rotated too. If there is no knowledge of the orientations of symbols, a number of directions has to be defined as possible positions for rotated pattern matching. Refining the rotations makes the recognition more accurate. A correct pattern matching algorithm without any knowledge has to test at least 20-30 directions. If the symbol is asymmetric, it may be necessary to do the pattern match with the mirrored symbol too (e.g. country borders)

302

R. Szendrei, I. Elek, and I. Fekete

As the maps are often distorted or defected, statistical methods should be applied instead of regular pattern matching methods. Several tests are known for statistical pattern matching depending on the problem class and they mainly use the mean and variance of a matrix. This paper uses a simple statistical comparison called similarity function. It takes two u × v matrices as parameters and calculates the variance of their difference matrix. The pattern matching algorithm uses the variance as a measure of similarity. In practice, the user defines or the software calculates a threshold value which will be used for pattern matching decisions. Each map pixel covered by the u × v matrix of the symbol is part of the symbol when the value of the similarity function is less than the threshold.

5

Efficient Pattern Matching

Some commercial software support the raster-vector conversion process. The embedded algorithms are well known, and most of them are filters (e.g. edge and corner detectors, edge filters). Despite the large number of filters, the Gauss and Laplace filters are used most often in digital image processing as edge filters, while Canny and Prewitt (see Fig. 1b) methods as edge detectors. Our task is to enhance the efficiency of symbol recognition. As a starting point, the vector data model is needed in an uninterpreted raw format, which naturally contains redundant vectors. The goal is to create the model which is as similar to the raster image as only possible. From this model, those datas are required, which describe the presence of a vector and the directon of the vector (when it exists) at a given point. If a vector exists at a pixel of the map, then the pixel belongs to an edge, which is represented by a vector with direction d. If a vector does not exist at a point, no pattern matching is required there. In other words no symbol is recognized at this point. The pattern matching is much more efficient if only those map pixels and symbol pixels will be matched which sit on a line. Namely, these points have a vector in the vector data model. It is assumed that total length of edges in the map is l ≤ m ∗ n, and the number of edge pixels in the symbol is ls ≤ u ∗ v. The cost of pattern matching in a fixed position remain unchanged (u ∗ v pixel comparisons). The estimated runtime of the improved matching process is then Teff (m, n, u, v, k) = Θ(l ∗ (u ∗ v) ∗ ls ∗ k). The total length of the edges may be u ∗ v at worst case. In this case the runtime can reach asymptotically the Tbf runtime of the brute force algorithm. The effective runtime of this algorithm is certainly significantly less, because, in practice the total length of the symbol edges is a linear function of the diameter of symbols. It is proven in [4] that the “speed up factor” of the improved method is m ∗ n ∗ l2 ∗ k l2 Teff s s =O = O . Tbf m ∗ n ∗ (u ∗ v)2 ∗ k (u ∗ v)2

Automatic Recognition of Topographic Map Symbols

6

303

Finding the Kernel of the Pattern

Certain symbols are used as a tile in maps and this tile is called kernel. This often happens when the user selects a part of the map that is larger than the symbol. This part includes the symbol at least one occurence and may also contain the symbol partially. In this case the pattern matching is less efficient. The optimized algorithm uses the smallest tile (see Fig. 2). If a kernel K is a uK × vK matrix and S is a u × v symbol matrix, then |S(i, j) − K(i mod uK , j mod vK )| F

4

270.674

0.06

0.9932

Within groups

1096589.16

245

4475.87412

Total

1097671.86

249

4408.32071

5 Conclusion This paper reviews the definition of concepts such as the gambler's fallacy, hot hand and their performance in reality, and then summarizes the generation causes of the four effects. On this basis, we analyze the occurring numbers and the revenues of the four effects by describing the decision-making preferences of heterogeneous individuals with the method of computer simulation. We can conclude that the occurring numbers

478

J. Li et al.

and the revenues of the four effects are indifferent from the analysis of Stata 10.0. There exist some differences in the beliefs and behavior of the real decision-making. People's behavior is influenced by their own preferences and beliefs. Preference is the driver of behavior, and the belief is the understanding of the relation between the behavior and the result, which also can be considered as an expectation of future events. People in different countries form their own unique behavior preferences because of the distinct national culture and institutional environment. Different decision-making beliefs will be generated in different situations, and the beliefs of these recency effects such as the gambler's fallacy, hot hand can be interchangeable under certain conditions. People in different countries, with different preferences, prefer a certain kind of effect, resulting in "collective effect" of accumulation actions. Thus, decision-making analysis in cross-cultural perspective is a valid way to study on the collection of the false beliefs. Acknowledgment. The National Natural Science Foundation of China (70972086, 70672029, Director: Professor Li Jianbiao), Project supported by Humanities and social sciences key research base from the Ministry of Education of China (05JJD630023, Director: Professor Xing Xiaolin, Professor Li Jianbiao), Major Program of the National Natural Science Foundation of China (70532001, Director: Professor Li Weian).

References 1. Sundali, J., Croson, R.: Biases in casino betting: The hot hand and the gambler’s fallacy. Judgment and Decision Making 1(1), 1–12 (2006) 2. Sun, Y., Wang, H.: Gambler’s fallacy, hot hand belief, and the time of patterns. Judgment and Decision Making 5(2), 124–132 (2010) 3. Ayton, P., Fischer, I.: The gambler’s fallacy and the hot-hand fallacy: Two faces of subjective randomness. Memory and Cognition (32), 1369–1378 (2004) 4. Guryan, J., Kearney, M.S.: Gambling at Lucky Stores: Empirical Evidence from State Lottery Sales. American Economic Review 98(1), 458–473 (2008) 5. Rabin, M.: The Gambler’s and Hot-Hand Fallacies: Theory and Applications. Review of Economic Studies 77, 730–778 (2010)

An Integration Process Model of Enterprise Information System Families Based on System of Systems Yingbo Wu1, Xu Wang2, and Yun Lin2 1

School of Software Engineering [email protected] 2 School of Mechanical Engineering, Chongqing University, Chongqing, China {wx921,linyun313}@163.com

Abstract. Based on the theory of system of systems (SoS), an integration process model of an enterprise information system family is discussed. The model is stratified into two levels, the top-level sub-processes of SoS and the bottom-level sub-processes of component systems, and the mapping relations between activities in sub-processes are classified as horizontal and vertical, which all contribute to better internal consistency. To support the dynamic integration environment of an enterprise information system family, the model is also designed to be an iterative process. Finally, based on the proposed model, we present an example of an enterprise information system family integration process in an auto industry chain. Keywords: Integration Process, Enterprise Information System Family, System of Systems.

1 Introduction System of systems (SoS) is a set of theories and methods proposed and developed to solve complex system family issues. Before integration, the constituent system is in spatial distribution and complementary in terms of capabilities. By effective integrating the constituent systems, SoS enhances problem-solving and responsiveness to challenging opportunity without changing the existing system working environment. SoS is highly adaptable to dynamic and unstable external environment [1]. Andrew P. Sage et al pointed out many of today’s systems are no longer engineered as stand alone systems by an individual institution, but as part of an integrated system of systems, or a federation of systems or systems family [2]. With the increasing demand on integrative supply chain and enterprise collaborative management, an enterprise information system family needs to be integrated as a SOS among the enterprises along the industry chain and within an enterprise itself to facilitate collaboration between enterprises or within an enterprise itself. An integration process study of an enterprise information family using SoS theories and methods will cast light on how to integrate an enterprise information SoS, and how to systemize, regulate and guide integration of an enterprise information system family. Y. Tan et al. (Eds.): ICSI 2011, Part II, LNCS 6729, pp. 479–485, 2011. © Springer-Verlag Berlin Heidelberg 2011

480

Y. Wu, X. Wang, and Y. Lin

At present, research on the integration of an enterprise information system family is mainly focused on integration technologies, frameworks or standards and so on. Though some scholars made useful exploration in applying SoS to enterprise information system engineering [3-5]. But little is done on the integration process of an enterprise information system family based on SoS.

2 SOS and Enterprise Information System Family The concept of SoS derives from early research on and understanding of issues of multi-system integration, which was later expounded by scholars in different research areas from different perspectives. Maier [6] first described SoS in terms of its general attributes, which were later reduced to the following five main features by Sage & Cuppan [7]: 1) Operational Independence of the Individual System. A SoS is composed of systems that are independent and useful in their own right. 2) Managerial Independence of the Individual System. The component systems in a system of systems not only can operate independently; they generally do operate independently to achieve an intended purpose. 3) Geographical Distribution. Geographic dispersion of component systems is often large. 4) Emergent Behavior. The SoS performs functions and carries out purposes that are not necessarily associated with any component system, leading to behaviors that are emergent properties of the entire SoS and not the behavior of any component system. 5) Evolutionary Development. Development of a SoS is generally evolutionary over time. Components of structure, function, and purpose are added, removed, and modified as experience with the system grows and evolves over time. In Boardman & Sauser’s thesis [8], these five main features were summarized as follows: Autonomy, Belonging, Connectivity, Diversity and Emergence. Based on these studies, we believe that a SoS is a aggregation of interactive component systems in a dynamic environment, and the component systems themselves within that aggregation should have a certain degree of autonomy in order to have better adaptive and collaborative capabilities, while the interactions between component systems is loosely coupled and a SoS should possess functional features not necessarily associated with its any component systems. The emergence properties of a SoS are important and necessary, because they distinguish a SoS from a system family. A SoS must demonstrate its unique functional features unavailable in its component systems, as opposed to a system family formed by simply combining independent systems. An enterprise information system family is an information system set established to serve corporate business management. Compared with a SoS, the component systems within an enterprise information system family also possess features of autonomy, belonging, connectivity and diversity. But in order to possess emergence properties unavailable in an existing individual enterprise information system, an enterprise information system family has to be effectively integrated as a SoS. The emergence properties are realized as information, functions, and interactiveness unavailable in component systems at a micro-level, better collaboration, adaptability

An Integration Process Model of Enterprise Information System Families

481

and flexibility at a macro-level. Therefore, the integration of an enterprise information system family should be viewed as constructing a SoS. Only in so doing, can we achieve greater comprehensive integration benefits and corporate business and management integration objectives.

3 Integration Process Model According to the traditional system engineering perspective, an integration process of an enterprise system family can be divided into four phases, namely objectives planning, needs definition, design & development, and implementation. Each of the four phases includes a number of specific integration activities. In contrast, according to SoS, a SoS integrated from an enterprise information system family should be viewed as a whole in terms of its objectives and the emergence properties. And the objectives and emergence properties should be reflected in the integration activities. In addition, because the integration of an enterprise information system family often occurs in a dynamic environment where new systems are added, and the old systems removed, replaced or modified into a new system; it should be viewed as a dynamic process. And this process favors the evolution of SoS. The integration process proposed in this paper is an iterative process in which each integration is seen as a component system of SoS, which in turn, can be integrated into a larger SoS. This whole iterative process makes the continuous dynamic evolution of SoS a reality. A SoS can be logically divided into a top-level SoS and a bottom-level of component systems [1], and then we propose the integration process model which be divided into top-level integration sub-processes of SoS and button-level integration sub-processes of component systems. And each sub-process is further sub-divided into four stages. Each time after the completion of an integration process, it waits next in line based on the need for a SoS evolution. The overall process framework is shown in Fig. 1.

Fig. 1. The integration process model of an enterprise information system family based on SoS

3.1 Top-Level Integration Sub-processes of SoS The top-level SoS provides an integration overview of an enterprise information system family. Integration sub-processes for a SoS involve four stages. In the first two stages, the integration objectives and requirements are planned and defined, and emergence function feature set is clearly defined. And in the next stage, the structure

482


of a SoS is designed based on integration objectives and requirements, and corresponding integration specifications and standards of a SoS are devised. The final stage—system implementation involves developing integration implementation plan, and preparing the necessary basic integration conditions and environments. The subprocess model is shown in Fig. 2.

Fig. 2. Top-level Integration Sub-processes of SoS

3.2 Bottom-Level Integration Sub-processes of Component Systems The sub-processes which is shown in Fig. 3 also involve four stages. In the first two stages, component system integration objectives and requirements are planned and defined based on SoS integration sub-processes. In the next two stages, component systems combination & connection schemes are developed, and component systems integration is implemented by devising specific implementation schemes of engineering organization, management and technologies.

Fig. 3. Bottom-level Integration Sub-processes of Component Systems

3.3 Iterative Processes That Favor Dynamic Evolution of Integration As enterprise information system family changes with its integration, the SoS thus integrated evolves dynamically. In the process model, each time after the completion


483

of integration process, it waits next in line based on the need for a SoS evolution. And the activities in the process are based on a decomposition principle, for example a SoS objectives can be decomposed into sub-SOS objectives activities, thus each iterative integration can be viewed as a construction process of a next SoS. An episode of the iterative processes is shown in Fig. 4.

Fig. 4. An episode of the iterative processes that favor dynamic evolution of integration

3.4 Mapping Relations between Sub-process Activities In the process model, the activities in integration sub-processes at the two differentlevels form mapping relations, which can be divided into horizontal and vertical relations based on mapping directions. Mapping relations of sub-process activities are shown in Fig. 5.

Fig. 5. Mapping relations between sub-process activities

484


In terms of horizontal mappings of sub-process activities, SoS function and performance feature sets can be obtained by SoS objectives, features and performance decompositions, and SoS function and performance can be more clearly defined by SoS objectives & functions definitions, and performance designing. Horizontal mapping also holds water for component system integration sub-process activities.

4 An Integration Process Example A sampled auto industry chain centers on a vehicle manufacturer, where strong crossenterprise business collaboration is formed among upstream parts suppliers, vehicle manufacturers, and downstream sales and third-party logistics agents. The enterprise information system family of this auto industry chain is integrated to address issues of information sharing and business collaboration among enterprises. Based on the process model proposed in this paper, we first conduct SoS needs analysis and function definitions based on the SoS objective in the collaborative industry chain, and completed collaborative performance designing between procurement and production, between production and sales, and between procurement, sales and logistics. In the system integration process, we clearly define system integration objectives and functions based on SoS collaborative objectives, functions and performance, devise the integration tasks & implementation schemes based on datamiddleware and service-oriented framework. The integration process model proposed in this paper is proved to be an effective tool in developing an integration process catering to an enterprise information system family.

5 Conclusion The integration process of an enterprise information system family, which differs from constructing an individual system, is concerned with a complex comprehensive SoS. The concept and method of SoS will help us solve complex multi-system integration issues. Based on SoS, this paper made useful exploration by proposing a process model that guides and regulates enterprise information system family integration. There is still much work to be done in integration process of an enterprise information system family based on SoS like process refinement, assessment, and optimization.

References 1. Songbao, Z., Weiming, Z., Zhong, L., et al.: Research of Structure Analyzing and Modeling for Complex System of Systems. Journal of National University of Defense Technology 28(1), 62–67 (2006) 2. Sage, A.P., Biemer, S.M.: Process for System Family Architecting, Design, and Integration. IEEE System Journal 1(1), 5–16 (2007) 3. Carlock, P.G., Fenton, R.E.: System of Systems (SoS) Enterprise Systems Engineering for Information-intensive Organizations. System Engineering 4(4), 242–261 (2001) 4. Morganwalp, J., Sage, A.P.: A System of Systems Focused Enterprise Architecture Framework. Information, Knowledge System Management 3(2), 87–105 (2003)


485

5. Stephenson, S.V., Sage, A.P.: Architecting for Enterprise Resource Planning. Information, Knowledge, System Management 6(1-2), 81–121 (2007) 6. Maier, M.: Architecting Principles for Systems-of-systems. System Engineering 1(4), 267– 284 (1998) 7. Sage, A.P., Cuppan, C.D.: On the Systems Engineering and Management of Systems of Systems and federation of systems. Information Knowledge System Management 2(4), 325–345 (2001) 8. Boardman, J., Sauser, B.: System of Systems - the Meaning of. In: Proceedings of the 2006 IEEE/SMC International Conference on System of Systems Engineering, pp. 118–123. IEEE Press, New York (2006)

A Linear Multisensor PHD Filter Using the Measurement Dimension Extension Approach Weifeng Liu and Chenglin Wen Hangzhou Dianzi University, Hangzhou Zhejiang 310018, China {liuwf,wencl}@hdu.edu.cn

Abstract. The common probability hypothesis density (PHD) fiter is derived under the single sensor condition. The multisensor PHD (MPHD) filter is remarkably complex and thus is impractical to use. Mahler proposed a MPHD filter under the assumption of independence of all senors. This paper studies the linear multisensor-multitarget system. We propose a linear multisensor probability hypothesis density (LMPHD) filter. By combining measurement dimension extension (MDE) approach, we consider linear correlation of all sensors. A simulation is finally proposed to verify the effective of the L-MPHD filter.

1

Introduction

Since it was proposed by Mahler [1], the probability hypothesis density (PHD) filter has been widely studied, especially in target tracking. These researches can be classified two classes according to their development: the PHD based algorithms and the cardinality PHD (CPHD) based algorithms. In the PHD based class, for the PHD filter is a nonlinear function, the particle-PHD filter was first given in references [2,3,4]. The particle-PHD filter can deal with the nonlinear tracking system, but it needs more computational load. The further important work is the Gaussian mixture PHD (GM-PHD) filter due to Vo et al [5]. The GM-PHD filter can estimate the target states without state clustering algorithms. To improve the state estimation, Nandakumaran et al proposed the PHD smoother [8]. Combining the interacting multiple model, Kirubarajan et al proposed the multiple-model PHD filter for tracking maneuvering targets [6]. In the CPHD based aspect, since the value of expected number of targets is very unstable in the presence of missed detections and/or significantly large false alarm densities in the PHD filter propagation process [9,10], Mahler further proposed the CPHD filter [10]. The CPHD filter propagates not only the PHD but also the entire probability distribution on number of targets. The analytic solutions of the CPHD was proposed by Vo et al [11]. Nevertheless, The most PHD and CPHD based algorithms are based on the single-sensor observation. Mahler investigated the multisensor PHD filter in reference [1] and pointed out that the resulting PHD formula is impractical due to its complexity in multisensor case. He proposed an approximation multisensor PHD algorithm through the product of individual sensor PHDs. In this algorithm, the sensors are independence from respective observation spaces. This Y. Tan et al. (Eds.): ICSI 2011, Part II, LNCS 6729, pp. 486–493, 2011. c Springer-Verlag Berlin Heidelberg 2011

A LMPHD Filter Using the Measurement Dimension Extension Approach

487

assumption is impractical in many cases. For example, the multisensor may observe the same targets and the multisensor thus be dependent in this case. Besides, in sensor network environment, individual sensors communicate information and thus the observations coming from the sensors are also dependent. For reducing difficulty, in this paper we confine our problem to the multisensor with a linear correlation. We propose the linear multisensor PHD (LMPHD) filter by adopting the measurement dimension extension (MDE) approach. This approach method not only effectively reduces the difficulty but also deal with the sensor correlation. This paper is organized as follows. Section 2 is about background and problem description. Section 3 shows the proposed LMPHD filter. Section 4 gives a three target tracking simulation. Last section 5 is our conclusion.

2

Background and Problem Description

2.1

Review on the PHD Filter

In the single-sensor case, the PHD filter is iteratively proceeded by the following time-update and measurement-update steps. Predicted step: Dk+1|k (x) = γk+1|k (x) + [(Ps (u)fk+1|k (x|u) + βk+1|k (x|u))Dk (x|u)]du (1) Update step: Dk+1 (x|Zk+1 ) = Fk+1 (Zk+1 |x)Dk+1|k (x) (2) Pd (x)gk+1 (z|x) Fk+1 (Zk+1 |x) = 1 − PD (x) + (3) λck+1 (z) + Pd (x)gk+1 (z|x)Dk+1|k (x)dx z∈Z k+1

where γk+1|k (x) is the target birth intensity at time step k + 1, PD (u) is the detection probability, βk+1|k (x|u) is the target spawning intensity, which has state u at time step k and has state x at time step k + 1, PS (x) denotes target surviving probability, λ is the average clutter intensity, ck+1 (z) is the clutter density for each clutter point, and gk+1 (z|x) is the measurement likelihood function. Mahler proposed an approximation for the multisensor PHD filter as follows. [1] [1] [s] [s] Dk+1 (x) ∼ = Fk+1 (Zk+1 |x) · · · Fk+1 (Zk+1 |x)Dk+1|k (x)

(4)

where s is the number of sensors. It can be seen from (4) that the PHD Dk+1 (x) is a s + 1-fold application of the Poisson approximation [1]. This update process can not be used in the sensor correlation case. 2.2

Multisensor Multitarget System Description

The linear multisensor-multitarget system is defined by

488

W. Liu and C. Wen

xk+1 = F k xk + Gk w k

(5)

[j] y k+1 [j] z k+1

(6)

=

[j] H k+1 xk+1

+

[j] v k+1

= Ak+1 yk+1 + bk+1

y k+1 =

[1] [y k+1 , · · ·

(7)

[s] , y k+1 ]T

[j]

Where zk+1 denotes the measurement of the jth sensor. Obviously, individual [j]

observations {zk+1 } are dependent. Equation (6) is the measurement model for each sensor. Accordingly, equation (7) is the proposed linear fusion model of each sensor. We aim at deriving the PHD intensity conditional on the measure[1] [s] [j] ment set Zk+1 , i.e., D(xk+1 |Zk+1 ), where Zk+1 = {Zk+1 , · · · , Zk+1 }T ,Zk+1 = [j]

[j]

{zk+1,1 , · · · , zk+1,nj } By augmenting measurements, we rewrite the measurement function (7) as follows z k+1 = Ak+1 H k+1 xk+1 + Ak+1 v k+1 + bk+1 [1] [s] z k+1 = [z k+1 , · · · , z k+1 ]T [1] [s] H k+1 = [H k+1 , · · · , H k+1 ]T [1] [s] v k+1 = [v k+1 , · · · , v k+1 ]T

(8) (9) (10)

Given this, we sum up the linear multisensor-multitarget system functions as follows xk+1 = Fk xk + Gk w k A zk+1 = H A k+1 xk+1 + v k+1

(11) (12)

A Where H A k+1 = Ak+1 H k+1 , v k+1 = Ak+1 v k+1 + bk+1 . Our next main task is to obtain the corresponding LMPHD filter after the MDE.

3

The Linear Multisensor PHD Filter

The proposed LMPHD filter in the predicted step is same as the original PHD filter. Therefore, we focus on the Bayesian update PHD in the measurementupdate step. 3.1

The LMPHD Bayesian Update

We first consider the LMPHD filter using the common approach. Suppose that [1] [s] individual sensor RFSs are {Σk , · · · , Σk }, in the Bayesian update step, the proposed LMPHD filter is derived by probability generating function (PGL) F [g1 , · · · , gs , h] [1]


Dk+1 (x|Zk+1 ) =

δ ms F · · · δ m1 F δF [0, · · · , 0, 1] fk+1 (zk+1 |Zk ) δ ms z [s] · · · δ m1 z [1] δx 1

489

(13)

s

F [g1 , · · · , gs , h] = [1] [s] [1] [s] g Z · · · g Z hX fk+1 (Z [1] |X) · · · fk+1 (Z [s] |X)fk+1|k (X|Zk )δZ [1] · · · δZ [s] δX (14) δ ms F δ ms z [s] = m [s] s δ z δz ms · · · δz 1

(15)

Obviously, it is intractable to obtain the update PHD Dk+1 (x|Zk+1 ). We here adopt the MDE approach (9) z k+1 . Thus the above multisensor PHD reduce to the single-sensor PHD [1](pp: 1173, equations (110),(111)), but here the extension measurements consist of total combination of all sensor measurements. That is 1 δ Ls +1 F Dk+1 (x|Zk+1 ) = [0, 1] (16) fk+1 (zk+1 |Zk ) δz Ls · · · δz 1 δx s Where Ls is the total combination number. It equals to Ls = l=1 ml , ml is the number of measurements of the lth sensor. Fk+1 (Zk+1 |x) = 1 − PD (x) + [1]

[z k+1 ,··· ,z [s] ]T ∈Zk+1

Pd (x)gk+1 ([z [1] , · · · , z [s] ]T |x) (17) λs ck+1 ([z [1] , · · · , z [s] ]T ) + PD (x)gk+1 (z|x)Dk+1|k (x)dx

Equation (17) is the same as a single sensor in appearance. Nevertheless, it describes a linear multisensor system and also considers the correlation among the sensors. We thus derive the LMPHD filter like the single sensor PHD update approach. However, some parameters which is different from the single-sensor PHD filter are required to be calculated. 3.2

Derivation of Some Parameters

In the proposed LMPHD update equation, several parameters need to be calculated, including clutter density ck (·), clutter intensity λ and extension covariance RA k+1 . We here investigate how to get these parameters. Clutter Density and Clutter Intensity. Assume that the lth sensor to be a space surface Sl . Thus, the multisensor can be described to be a super-cylinder C consisting of these sensor spaces, i.e., C S1 × · · · × Ss . Therefore, the clutter intensity is proposed by: λ = λ1 + · · · + λs (18) where λ1 , · · · , λs are the clutter intensities of the sensors. The clutter density of each clutter point is proposed as follows ck ([z [1] , · · · , z [s] ]T ) =

λ V (C)

Where V (C) = V (Sl × · · · × Ss ) denotes the volume of the super-cylinder.

(19)

490

W. Liu and C. Wen

The Extension Measurement Covariance. Assume that the process noise [j] wk and measurement noise {v k } follow Gaussian noise with mean 0 and co[j] variances Qk and Rk , respectively. The measurement error covariance RA k+1 = A cov(v A , v ) of the measurement function in (12) is proposed as follows: k+1 k+1 T RA k+1 = Ak+1 diag[Rk+1 , · · · , Rk+1 ]Ak+1 [1]

v ¯k+1 =

E(v A k+1 )

[s]

= bk+1

(20) (21)

Given the three parameters, the LMPHD filter can be proceeded like the singlesensor PHD filter. Similarly, the GM-PHD filter can also be used in LMPHD filter.

4

Simulation

In this section, three targets with CV movements are proposed in x-y coordination plane. We suppose two sensors observing the same region [−1000, 1000] × [−1000, 1000]m2 . These two sensors are placed in different positions and have a difference in precision. Some system parameter are proposed as follows: initial states are [250m, 5m/s, 250m, −12m/s],[−250m, 12m/s, −250m, −5m/s] and [−250m, 12m/s, −250m, −5m/s] respectively for target 1, 2 and 3. Detection probability is PD = 0.9. The process covariance is Qk = diag(25, 25)m2 . The measurement covariances are respectively R1 = diag(25, 25)m2 and R2 = diag(50, 50)m2 for sensor 1 and 2. That is, the sensor 1 has a better performance than sensor 2. The measurement functions are position observation. We assume in the simulation that the two sensors have the same reference coordinations. Accordingly, we fuse these two sensors as follows: y k = H k xk + v k z k = Ak y k + bk

⎤ 0.8 0 0.2 0 ⎢ 0 0.8 0 0.2 ⎥ 1000 ⎥ , Ak = ⎢ Hk = ⎣ 0.6 0 0.4 0 ⎦ , bk = 0 0010 0 0.6 0 0.4

⎡

The GM-PHD filter is here proposed to track the targets. The Gaussian terms whose weights are greater than threshold 0.5 are selected as the estimations. Fig.1(a) and Fig.1(b) suggest that the proposed algorithm is effective in target tracking. We further compare the proposed algorithm with the PHD filter in Fig.2 and 3. It can be seen from Fig.2 that in the number of targets the proposed algorithm is stable than the PHD filter. Fig.3(a) is the Wasserstein distances of these two algorithms. It shows that the PHD filter has something fluctuation due to the estimation of target number. This point can be also found in Fig.3(b) where the OSPA distance is adopted. However, the proposed algorithm needs some more computing time than the PHD filter. The reason is that the proposed


400

1000 True track Estimation track: the proposed algorithm

500 x(m)

200

0 −500

0

y(m)

−1000

True track Estimation track: the proposed algorithm

0

10

20

30 Time step(s)

40

50

60

10

20

30 Time step(s)

40

50

60

−200 500 −400 y(m)

0 −600

−500

−800 −800

−600

−400

−200

0 x(m)

200

400

600

−1000

800

(a) The proposed linear MPHD filter: true tracks and estimation tracks in x,y plane

0

(b) Tracks in x,y coordinations

Fig. 1. Target tracking trajectorys 5 True number of targets Estimation number of targets: the proposed algoirthm Estimation number of targets: the original PHD filter

4.5 4

Number of targets

3.5 3 2.5 2 1.5 1 0.5 0

0

10

20

30 Time step(s)

40

50

60

Fig. 2. The number of targets against time 400

100 The proposed algorithm The original PHD filter

350

80

300

70

250

60 200

OSPA

Wasserstein distance

The proposed algorithm The original PHD filter

90

150

50 40

100

30

50

20

0 −50

10 0

10

20

30 Time step(s)

40

(a) The Wasserstein against time

50

60

0

0

10

20

30 Time step(s)

40

50

60

distances (b) The OSPA distances against time

Fig. 3. Tracking performance comparison

491

492

W. Liu and C. Wen

algorithm requires to deal with more extension measurements and the higher dimension vectors. Here, the number of the measurements is Mk = m1 × · · ·× ms and the dimension is d = d1 + · · · + ds , where d1 , · · · , dm are the measurement dimensions for individual sensors.

5

Conclusion

We proposed a LMPHD filter by using the MDE approach. Though the new filter has the same form as the original PHD filter, it can deal with the case where the sensors are linear correlation. We also proposed the derivation of some parameters for the new extension measurement in the LMPHD filter. Future works may focus on two aspects: computational complexity and nonlinear correlation. The proposed filter needs to update the product of all sensor measurements compared with the sum of the measurements in single sensor PHD filter. Secondly, Radar plays an important role in target tracking, and radar networks is a new tendency in future application. Nevertheless, radar is a nonlinear sensor system. How to solve the nonlinear multisensor will be a popular topic.

References 1. Mahler, R.P.S.: Multitarget Bayes Filtering via First-Order Multitarget Moments. IEEE Transactions on Aerospace and Electronic systems 39(4), 1152–1178 (2003) 2. Vo, B., Singh, S., Doucet, A.: Sequential Monte Carlo implementation of the PHD filter for multi-target tracking. In: Proceedings of the International Conference on Information Fusion, Cairns, Australia, pp. 792–799 (2003) 3. Sidenbladh, H.: Multi-target particle filtering for the probability hypothesis density. In: Proceedings of the International Conference on Information Fusion, Cairns, Australia, pp. 800–806 (2003) 4. Zajic, T., Mahler, R.: A particle-systems implementation of the PHD multitarget tracking filter. In: Signal Processing, Sensor Fusion, and Target Recognition XII, pp. 291–299 (2003) 5. Vo, B.-N., Ma, W.-K.: The Gaussian Mixture Probability Hypothesis Density Filter. IEEE Transactions on signal processing 54(11), 4091–4104 (2006) 6. Punithakumar, K., Kirubarajan, T., Sinha, A.: Multiple-model probability hypothesis density filter for tracking maneuvering targets. IEEE Transactions on Aerospace and Electronic Systems 44(1), 87–88 (2008) 7. Vo, B.N., Pasha, A., Tuan, H.D.: A Gaussian mixture PHD filterr for nonlinear jump Markov models. In: Proceedings of the 45th IEEE Conference on Decision and Control, pp. 3162–3166. IEEE, San Diego (2006) 8. Nandakumaran, N., Punithakumar, K., Kirubarajan, T.: Improved multi-target tracking using probability hypothesis density smoothing. In: Drummond, O.E. (ed.) Proc. Signal and Data Processing of Small Targets, vol. 6699 (August 2007)


493

9. Erdinc, O., Willet, P., Bar-Shalom, Y.: A Physical-Space Approach for the Probability Hypothesis Density and Cardinalized Probability Density Filters. In: Signal and Data Processing of Small Targets, Proc. of SPIE, vol. 6236, pp. 1–12 (2006) 10. Mahler, R.: PHD filters of higher order in target number. IEEE Trans. Aerosp. Electron. Syst. 43(3), 1523–1543 (2007) 11. Vo, B.-T., Vo, B.-N., Cantoni, A.: Analytic Implementations of the Cardinalized Probability Hypothesis Density Filter. IEEE Transactions on Signal Processing 55(7), 3553–3567 (2007)

An Improved Particle Swarm Optimization for Uncertain Information Fusion Peiyi Zhu1,2, Benlian Xu2, and Baoguo Xu1 1

College of IoT Engineering, Jiangnan University, Lihu Road, Wuxi City, Jiangsu, China 2 School of Electrical and Automation Engineering, Changshu Institute of Technology, Nansanhuan Road, Changshu, Jiangsu, China [email protected]

Abstract. Multi-sensor information fusion is used to carry on synthesizing excellently to the multi-source information, make verdict of people more accurate and credible. But the influences of uncertainties on the safety/failure of the system and on the warranty costs exist. The new method to deal with the uncertain information fusion based on improved Dempster-Shafer (D-S) evidence theory has been proposed, and set up the concept of weight of sensor evidence itself and evidence distance based on a quantification of the similarity between sets to acquire the reliability weight of the relationship between evidences. Next an improved particle swarm optimization (PSO) is used to computer sensor weight to modify D-S evidence theory. Finally, numerical experiments are adopted to prove its effectiveness. Keywords: D-S theory, evidence distance, uncertain, particle swarm optimization.

1 Introduction The integration of uncertain multi-sensor information is commonly deal by information fusion algorithm [1-2]. Information often contains uncertainties, which are usually related to physical constrains, detection algorithms, and the transmitting channel of the sensors. Whilst the intuitive approaches, such as Dempster-Shafer Fusion, Dezert Samarandche Fusion, and Smets’ Transferable Belief Model [3-5] are to aggregate all available information, these approaches do not always guarantee optimum results. Acknowledging these measurement techniques have associated measurement costs, the essence is to derive a fusion process to minimize global uncertainties. Nowadays, the system is increasingly relying on information fusion techniques to automate processes and make decisions. An informed decision maker, meanwhile, often relies on various forms of data fusion models to assist him/her to assess the current situations. The D-S evidence theory is an excellent method of information fusion, its adoption belief degree function but not probability is a generous character, no request to make binary mutex assumption to the indetermination affairs, having Y. Tan et al. (Eds.): ICSI 2011, Part II, LNCS 6729, pp. 494–501, 2011. © Springer-Verlag Berlin Heidelberg 2011

An Improved Particle Swarm Optimization for Uncertain Information Fusion

495

more catholicity to the indetermination. Because of the difference which is caused by sensors in the credibility or reliability, the important degree of the proof provided is different. So it is essential to deal with the evidence with a method of weighed, adapting to a weighed evidence synthetic technique. People have already put forward a variety of methods of weighed. But up to now the study on how to obtain better weight value is also rarely covered in the weight D-S evidence theories. Presently the methods that select weight value currently: From the expert of fusion or the result of the covariance etc. [6]. But it is hard to obtain the better weight value through these methods. Considering of the disadvantages of the weighed D-S theory, we present a method to solve optimization weight value, and the thought of the method is explained, then use improved PSO algorithm to work out the weight value in the established optimization model. Compared with some other methods [7], this evidence theory proves more effective and advanced by making simulation.

2 An Improved Particle Swarm Optimization Since the original version of PSO proposed in 1995[8], lots of work has been done to develop it. According to the Standard PSO (SPSO), the weight ω is a fixed parameter, so the algorithm easily falls into premature convergence. So this paper presents an improved strategy which is based on Dynamical Weight (DW) that the particles have the adaptive changes the diversity of groups to adjust their speed. And the particles effectively jump out form the best local to avoid premature convergence. In the standard PSO algorithm, the particles have own thinking and share information and cooperate with each other in the process of continuous evolution, so it can be divided into two parts: the evolution degree of particles speed and the aggregation degree of particles. Definition 1: xi (t ) | (i ∈ {1, 2,..., N }) denotes ith particle in tth-generation population,

the particle best fitness can be expressed as f i , individual extreme value records at presents denotes f ibest (t ) , so individual extreme value records at (t − 1) time denotes f ibest (t − 1) . And the evolution degree of particles speed E( x) can be represented as the followings: E( x) = fibest (t ) / f ibest (t − 1)

(1)

Definition 2: xi (t ) | (i ∈ {1, 2,..., N }) denotes ith particle in tth-generation population,

the particle best fitness can be expressed as f i , individual extreme value records at presents denotes f ibest (t ) , f avg denotes all particle average fitness at present, so the aggregation degree of particles A( x) can be represented as the followings: A( x) = f ibest (t ) / f avg (t )

(2)

According to defines 1 and 2, we can clearly see the particle swarm optimization process. If adjusts the weight according to the particle velocity evolution degree and the granule extent of polymerization, we can combine the weight with the particles optimization process, thus adjust population diversity by adjusting the weight.

496

P. Zhu, B. Xu, and B. Xu

Therefore the weight changes along with changing the particle velocity evolution and aggregation degree, which situation can solve the particles premature problem from the mathematical model. When E( x) is big the velocity evolution is quick, the algorithm may continue in the big space to search, namely the swarm optimizes in wide range. When E( x) is small, may reduce ω to cause the swarm searched in a small scope, thus found the optimum value quickly. When A( x) is small the swarm is quite scattered, not easy to fall into local optimum. The swarm easily to fall into local optimum with A( x) increasing, at this moment, we will increase ω to increases the swarm search space, and improve the swarm global search capability. In conclusion, ω increases with the particle velocity evolution E( x ) reduce or increases with the particles aggregation degree A( x) increases. Therefore

ω

is decided by

E( x) and A( x) , so the functional relations may express the equation below: w = f ( E ( x), A( x)) = w0 − 0.55* E ( x) + 0.15* A( x)

＝

(3)

Where ω0 is initial value of ω , generally speaking, ω0 0.9 ; by definition we can see 0 < E (x ) ≤ 1 and 0 < A (x ) ≤ 1 , so ω0 − 0.55 < ω < ω0 + 0.15 , ensure ω < 1 the convergence requirements.

3 An Improved d-s Theory for Uncertain Information Fusion 3.1 Basic of D-S Theory

D-S theory is developed as an attempt to generalize probability theory by introducing a rule for combining distinct bodies of evidence [9]. In D-S theory, a finite nonempty set of mutually exclusive alternatives is called the frame of discernment, denoted by Θ. The power set 2Θ is the set of all the subsets of Θ including itself. The basic probability assignment (BPA) reflects a degree of belief in a hypothesis or the degree to which the evidence supports the hypothesis. DS evidence theory has been widely used in uncertainty information fusion, and provides a strong theoretical basis on dealing with uncertain information. However, D-S evidence theory would arrive at conclusion contrary to common sense when high confliction existed in evidences. Facing the high conflict, many scholars put forward their own amendments to this disadvantage, such as Yager rule [10], Dubois-Prade rule [11], the average distribution rule and the weighted rule. However, these combination rules have disadvantages in robustness and effectiveness. For the above amendment combination rules, from the practical application point of view, Haenni pointed out some problems as follows: weight distribution of designated evidence does not meet the exchange law and integration law and the issue of complex computation in practical application. 3.2 The Distance between Weighted Evidences

Since the aim of this paper is to define a meaningful metric distance for BPAs, let w1 , w2 is the corresponding weight of the evidence m1 , m2 in information fusion,


497

m1 , m2 is two BPAs on the frame of discernment Θ, containing n mutually exclusive and exhaustive hypotheses [12]. The distance between m1 and m2 is:

JJG 2 JJG 2 JJG JJG 1 ( w1 m1 + w2 m2 − 2 < w1 m1 , w2 m2 >) (6) 2 Where the index 1/2 is needed to normalize d BPA and to guarantee that JJG JJG 0 ≤ d BPA ≤ 1 ; and < m1 , m2 > is the scalar product defined by Eq. (7): 2n 2n JJG JJG | Ai ∩ Aj | (7) < m1 , m2 > = ∑∑ m1 ( Ai )m2 ( Aj ) | Ai ∪ Aj | i =1 j =1 d BPA ( w1m1 , w2 m2 ) =

3.3 The New Uncertain Information Fusion

In order to make the amended evidence overall between closer and the final decision is more reasonable, weighted evidence distance function is established, this function is about evidence weight function. In this paper, we will take this function as optimized object. According to optimization theory, the optimization model can be represented as the followings: 2 F ( w) = min(d BPA ) n n (8) G G 2 = min ∑∑ d BPA (mi , m j ) i =1 j =1

It can be easily seen that ∑ wi = 1 , thus, the weight shows the relative importance of the collected evidence. We can use traditional optimization methods to solve Eq. (8). Considering real-time demand for information fusion, and accuracy and validity fusion results in conflict evidence, the more optimal solution based on improved PSO should be given to meet these requirements. For the summary of the new uncertain information fusion see Table 1. Table 1. The procedure of the new uncertain information fusion algorithm

The weight of evidence’ owns reliability wi can be acquired by the above analysis, so we can revise evidence theory by wi . Considering the evidence source itself has different importance, therefore, the revised evidence not be a simple average, the new evidence source probability assigned is defined as follows: n

mae(m) = ∑ ( wi mi ) i =1

(9)

498

P. Zhu, B. Xu, and B. Xu

Then, the new probability assignment is acquired, thus, we can combine it using D-S combination rule.

4 Numerical Experiments The new method have analyzed above, this section mainly give two numerical examples to further understanding the method proposed above.

A. Initial experiments The experimental arrangement, two groups of common data to acquire the basic probability assignment, this problem is solved by using two different analysis types namely case 1 using D-S rules and case 2 using Yager’s rule to combine evidence at sub-system level for each type. We used modified PSO algorithm to solve this optimization problem along with D-S rule of combined evidence program in the developed matlab program to solve this problem. For the PSO parameters, namely, population size=15, tmax = 500 , c1 = c2 = 2.0 , ω0 = 0.9 , stopping convergence criterion (in terms of change in the objective function value) = 10-8 for over 200 continuous iterations. Assuming a discernment frame Θ = { A, B, C} , source S1 , S 2 , S3 , S 4 , such that Data 1: S1 : m( A) = 0.7 m( B) = 0.1 m(C ) = 0.1

m( A, C ) = 0.1

S2 : m( A) = 0.1 m( B) = 0.8 m(C ) = 0.05 m( A, C ) = 0.05 S3 : m( A) = 0.4 m( B ) = 0.3 m(C ) = 0.2 m( A, C ) = 0.1 S4 : m( A) = 0.4 m( B) = 0.2 m(C ) = 0.1 m( A, C ) = 0.3

Data 2: S1 : m( A) = 0.001 m( B ) = 0.199 m(C ) = 0.8 S2 : m( A) = 0.9 S3 : m( A) = 0.3

m( B ) = 0.05 m(C ) = 0.05 m( B ) = 0.6

S4 : m( A) = 0.4 m( B) = 0.4

m(C ) = 0.1 m(C ) = 0.2

Table 2. Experimental result of Data 1

Table 3. Experimental result of Data 2


499

The results of combination are shown in Table 2 by three different rules, we can see that when the confliction between evidences are relatively small, this paper’s method is slightly better than D-S rules and Yager rules, which reflects the basic status of D-S evidence theory, so there is not much difference among effectiveness of sensor fusion methods of linear combination, In this paper, but if the identification object is multi-element set, this paper’s method makes it more reasonable and simple to compute. The numbers in the table 3 confirm our conclusion made in the following. We can see that when the confliction between evidences is relatively large, the experimental results are much better than Yager rules, and even better than the method of D-S rules. Because there is any contradiction among the evidences from different sources, the Yager rule treats that contradiction as coming from ignorance. If there is any additional knowledge, then this contradiction might be resolved. So the Yager rule is more conservative than the D-S rule. Therefore, we have any reason to believe that this paper’s method has the best combination result between three algorithms.

B. Evidence conflict and robustness In this section, we consider two evidence source combinations by the classic DS evidence theory and the improved algorithm. Assuming a discernment frame Θ = { A, B, C} , source S1 and S2, such that Data 3:

m1 : m1 ( A) = 0.99, m1 ( B ) = 0.01, m1 (C ) = 0 m2 : m2 ( A) = 0, m1 ( B ) = 0.01, m1 (C ) = 0.99 m1' : m1' ( A) = 0.98, m1' ( B) = 0.01, m1' (C ) = 0.01 The results of combination are shown in Table 4 by two different rules. Table 4. Experimental result of Data 3

We can see that the evidence is high conflicting in this test. In these two evidence, the focal element A and the focal element C intuitively obtain a higher support degree, should be approximately 50%, which should get more support after combination. However, combination based on DS combination rule causes that the focal element B gets almost certainly support, which is clearly unreasonable. We proposed algorithm by modifying conflict evidence, get more reasonable results, therefore, the improved algorithm can effectively avoid the defects of the traditional DS combination rules when there is conflict evidence. If the evidence m1' is substitute by m1 , the combination results are shown in Table 5.

500

P. Zhu, B. Xu, and B. Xu Table 5. Experimental result of Data 3

Contrasting Table 4 and Table 5, the combination results based on DS combination rule have great changes when the evidence m1' is substitute by m1 . This situation shows that DS rules is sensitive to the change of the focus element probability distribution, robustness is poor. But the combination results based on the improved algorithm is little change, robustness is better.

5 Conclusion For traditional D-S evidence theory’s problems of high conflict between evidences, this paper proposed a new method to solve it. Firstly, we deal with the evidence with a method of weighted D-S theory. Then an optimize model of obtaining sensor weights has been set up. At last, we use the improved PSO to acquire the reliability weight of the relationship between evidences to modify D-S theory. Numerical experiments show that this new method is more effective. Acknowledgements. This research is supported by a grant from the National Natural Science Foundation of China No. 60804068 & No. 30971689, the Natural Science Foundation of Jiangsu Province No. BK2010261, the Key Technology Support Program of Jiangsu province No. BE2010627, the Natural Science Fundamental Research Project of Colleges and Universities in Jiangsu province No. 10KJD510001.

References [1] Wan, S.: Fusion Method for Uncertain Multi-sensor Information. In: International Conference on Intelligent Computation Technology and Automation, vol. 1, pp. 1056– 1060 (2008) [2] Chen, L., Huang, J.: Research of Uncertainty. Journal of Circuit and System 9(3), 105–111 (2004) [3] Shafer, G.: A mathematical theory of evidence, pp. 19–63. Princeton University Press, Princeton (1976) [4] Dezert, J., Smarandache, F.: DSmT: A New Paradigm Shift for Information Fusion. In: COGnitive systems with Interactive Sensors International Conference, Paris, March 2006, pp. 1–11 (2006) [5] Ristic, B., Smets, P.: Target Classification Approach Based On the Belief Function Theory. IEEE Transactions on Aerospace and Electronics Systems 41(2), 1097–1103 (2005) [6] Capelle, A.S., Fernandez-Maloigne, C., Colot, O.: Introduction of Spatial Information within the Context of Evidence Theory. In: IEEE International Conference On Acoustics, Speech, and Signal Processing, vol. 2, pp. 785–788 (2003)


501

[7] Wu, Z., Wu, G.: A new improvement of evidence combination. Computer and Modern 12, 116–117 (2007) [8] Eberhart, R.C., Kennedy, J.: A new optimizer using particle swarm theory. In: Proceedings of the sixth International Symposium on Micro and Human Science, Nagoya, Japan, pp. 39–43 (1995) [9] Sentz, K., Ferson, S.: Combination of evidence in Dempster Shafer theory. TR 0835, Sandia National Laboratories, Albuquerque, New Mexico (2002) [10] Yager, R.: On the Dempster-Shafer framework and new combination rules. Information Sciences 41, 93–137 (1987) [11] Dubois, D., Prade, H.: Representation and combination of uncertainty with belief functions and possibility measures. Computational Intelligence 4, 244–264 (1998) [12] Jousselem, A., Grenier, D., Bosse, E.: A new distance between two bodies of evidence. Information Fusion 2, 91–101 (2001)

Three-Primary-Color Pheromone for Track Initiation Benlian Xu1, Qinglan Chen2, and Jihong Zhu1 1

School of Electrical & Automatic Engineering, Changshu Institute of Technology, 215500 Changshu, China 2 School of Mechanical Engineering, Changshu Institute of Technology, 215500 Changshu, China {xu_benlian,chen_qinglan,djyzhjh}@yahoo.com.cn

Abstract. We propose a novel ant system with a “subtractive” color mixing model of three-primary-color for jointly estimating the number of tracks to be initiated and their individual tracks in the multi-sensor multi-target system. In our algorithm, each ant deposits cyan, magenta, or yellow pheromone all the time, and ant’s decision depends on the colored pheromone similarity comparison with candidates to be visited. On the basis of it, a mixture optimization function on a three dimensional parameter space is proposed to exploit best solutions by the following ants. Simulation results are presented to support obtained favorable performance of our algorithm. Keywords: Track initiation, Ant colony optimization, Three-primary-color.

1 Introduction Multi-target tracking (MTT) has received considerable interest over last decade years, and its application focuses on civil and military areas [1,2]. In general, MTT includes phases of track initiation, data association and state estimate, among which track initiation can determines the number of targets as well as the initial state estimate for state estimator, and track initiation under consideration may result in target loss or increase computation burden. So far, four popular track initiation techniques are generally used in radar tracking, namely, the rule-based method, the logic-based method, the Hough transform and the modified Hough transform method [3]. In this work, however, we focus on the bearings-only track initiation problem in the sonar based tracking of submarines. Since measurement candidates grows exponentially with the number of sensors ( m ) or polynomially with the number of targets, many attempts have been made including various evolutionary algorithms, among which Ant Colony Optimization (ACO) approach is recognized a competitive one [4,5]. However, these algorithms need to be improved for practical tracking scenario due to the assumption that the number of tracks is known a prior. Biologist reports that there are near 20,000 species of ants that vary in size, color, and ways of life. Most are a dull, drab color such as brown, rust, or black, but some are yellow, green, blue, or purple. Inspired from these colored ants, we propose an ant system with three primary colors to jointly identify the number of tracks to be initiated and their individual tracks. Y. Tan et al. (Eds.): ICSI 2011, Part II, LNCS 6729, pp. 502–508, 2011. © Springer-Verlag Berlin Heidelberg 2011

Three-Primary-Color Pheromone for Track Initiation

503

2 Background In the generic ACO algorithm, ants communicate with each other indirectly by the stigmergy, and such behavior can be replaced by a more direct communication means called color similarity of pheromone. Color similarity comparison can be conducted through the following two steps in general: color space conversion and color difference computation. Since there are many techniques for color space conversions, the adopted conversion strategy is generally conditional on various applications. In this work, to increase the color discrimination ability of each ant in the “subtractive” color mixing model, the following conversion steps are employed: Step 1): From CMY to standard RGB space. Since standard RGB component values vary between 0 and 1, a cheap and simple transform from CMY space to standard RGB space is adopted, namely, R = 1 − C , G = 1 − M , and B = 1 − Y , respectively, where each component of CMY lies in the range of [0,1] as well. Step 2): From standard RGB to CIE XYZ. According to human vision tristimulus, the conversion law from standard RGB to CIE XYZ is introduced as [6]

⎡X ⎤ ⎡ 0.412453 0.357580 0.180432 ⎤ ⎡ R ⎤ ⎢ Y ⎥ = 100 ⎢ 0.212671 0.715160 0.072169 ⎥ ⎢G ⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢⎣ Z ⎥⎦ ⎢⎣ 0.019334 0.119193 0.950227 ⎥⎦ ⎢⎣ B ⎥⎦

(1)

Step 3): From CIE XYZ to CIE LAB. Because the CIE LAB is more perceptually uniform than the CIE XYZ, i.e. a change of the same amount in a color value can produce a change of about the same visual importance, the color difference comparison is conducted preferably in the CIE LAB space, which can be computed as

L∗ = 116 f ( Y Yn ) − 16

⎧t 1/3 t > (6 / 29) 3 ⎪ a = 500 ⎡⎣ f ( X X n ) − f (Y Yn ) ⎤⎦ where f (t ) = ⎨ 1 (2) 4 2 otherwise ⎪ ( 29 6 ) t + ∗ 29 ⎩3 b = 200 ⎡⎣ f ( Y Yn ) − f ( Z Z n ) ⎤⎦ ∗

where typical ranges for above three values are L* ∈ [0,100] , a* ∈ [−100,100] , and b* ∈ [−100,100] , respectively; ( X n , Yn , Z n ) denotes the tristimulus value with reference to white point in the CIE XYZ space, and is given by (95.017,100.0,108.813) . Step 4): Color difference computation. Consequently, given two colors w1 = ( L1∗ , a1∗ , b1∗ ) and w2 = ( L∗2 , a2∗ , b2∗ ) obtained from Eq. (2) in the CIE LAB space, the corresponding color difference in terms of the metric of Euclidean distance is

ΔE ( w1 , w2 ) =|| w1 − w2 ||2 = ( L∗1 − L∗2 )2 + (a1∗ − a2∗ ) 2 + (b1∗ − b2∗ ) 2

(3)

504

B. Xu, Q. Chen, and J. Zhu

3 Three-Primary-Color Pheromone for Track Initiation 3.1 Track Construction

For a multi-sensor multi-target bearings-only tracking system, the sampling data from the first four scans are generally utilized to initiate tracks, thus we obtain four spaces, denoted by Ω1 , Ω 2 , Ω3 and Ω 4 , respectively, and each is formed through intersecting bearing measurements (or Line of sights) of the same scan, as shown in Fig. 1. Since our algorithm is based on the three primary colors, we consider three groups of equal number of ants accordingly. Initially, three groups of ants are mixed together and placed randomly on position candidates in the first search space Ω1 . Afterwards, each ant will visit the position candidates in the next search space probabilistically. Suppose that an ant with s 1 pheromone is now located at position i in Ω k ( k = 1, 2,3 ), then the ant will visit position j in the next search space by applying the following probabilistic formula: Pi ,( sj ) = e

−α ⋅ΔE ( ws , w j )

ηi , j

∑eα

ηi , l

− ⋅ΔE ( ws , wl )

l ∈Ωk +1

(4)

−α ⋅ΔE ( w , w )

s j where e denotes the pheromone color similarity between the current ant and the path from i to j ; ηi , j denotes the problem dependent heuristic function; while

α is an adjustable positive parameter whose value determines the degree of pheromone color similarity among candidates. It is noted that the smaller the −α ⋅ΔE ( ws , w j ) , the bigger the color difference. To eliminate pheromone color similarity e the effect of outliers and simplify the computation of each ant, ηi , j is defined as below ⎪⎧1

ηi , j = ⎨ ⎪⎩κ

if r1 ≤ Di , j ≤ r2 , j ∈ γ otherwise

(5)

where κ is a constant and takes a value between 0 and 1, Di , j is the distance between i and j , and γ is denotes an annular gate region whose inner and outer radiuses are determined respectively by r1 =|| vmin ||iT and r2 =|| vmax ||iT with sampling interval T . While walking from i to j , the ant deposits its corresponding color pheromone with a given amount τ 0s as

τ i , j ← τ i , j + τ 0s

(6)

where τ 0s is the added local pheromone amount with color s , and τ i , j is the resulting pheromone through mixing three primary colors with their individual amounts. Once all ants at a given iteration have finished their individual tours, the pheromone amount on some established tracks will be updated globally, which will directly result in the color changes on these tracks. We consider respectively three 1

Without loss of generality, s = 1, 2,3 represents cyan, magenta, and yellow, respectively.


505

Y

Line of sight from sensor 2

Ω1

Line of sight from sensor 1

i Ω2

Ω3

Ω4

Sensor 2

Sensor 1

X

Fig. 1. The obtained search spaces θ

Y a 1 b

1 2

V

p

p

3 c

2

3 d O

ρ

O

X (a)

(b)

Δ Δ E p (1, 2 ) Δ E p ( 2, 3) Δ E p (1, 3)

θ 1 3 2

ρ

O (c)

Fig. 2. Objective function in ρ −θ −Δ space

groups of solutions to update the corresponding pheromone trails. Moreover, each group is equally composed of L best-so-far-solutions found by ants. Given a track p in the s th group, the pheromone update is conducted according to the following rule

τ ip, j ← τ ip, j + Δτ is,,jp .

(7)

where Δτ is,,jp is the amount of pheromone with color s added on the segment ij of track p . In our ant system, Δτ is,,jp is defined as follow Δτ is,,jp = Q0s J p

(8)

s

where Q0 is a adjustment constant related to the s type of three primary colors, and J p is the objective function to be discussed below.

3.2 Objective Function in a Three Parameter Space

Hough transform (H-T) has been recognized as a robust technique for line or curve detection in the image detection field, and it in is essence a transform from a point

506


( x, y ) in a Cartesian coordinate system to a curve in a ρ − θ parameter space, and it obeys the following basic law [7]

ρ = x cos θ + y sin θ

(9)

where ρ is the distance from the line through ( x, y ) to the coordinate origin, and θ is the angle to the normal with the x axis. In our proposed ant system, the objection function is designed in a three dimensional parameter spaces denoted by ρ − θ − Δ , where notation “ Δ ” represents the color difference. As a result, the obtained objective function for track p is a combined form and described as J p = ξ J dp + (1 − ξ ) J Δp

(10)

where J dp denotes the distance difference sub-objective function, J Δp is the color difference sub-objective function for track p , and parameter ξ ( 0 ≤ ξ ≤ 1 ) is to balance the importance between the two sub-objective functions. The cluster idea is inherited to calculate J dp in the ρ − θ − Δ space as Ck

J dp = ∑ || x ( k , p ) − V p ||2

(11)

k =1

where x ( k , p ) is the k th point corresponding to some segment on track p , denoted by 1, 2, or 3 in Fig.2(a) to 2(b); Ck is the number of intersections; and V p is the cluster Ck

prototype with the value V p = ∑ x ( k , p ) Ck , as shown in Fig. 2(b). J Δp can be k =1

computed as J Δp =

∑

1≤ i ≠ j ≤ 3

ΔE p ( w j , wi )

(12)

As shown in Fig. 2(c), any two segments on track p are selected to compute the corresponding color difference, so total three terms are required to be calculated.

4 Numerical Simulation Numerous simulations with different cases are conducted on a DELL 6GHz processor with 1.99 GB RAM, we only present the case of three track initiation due to the layout restrictions. Figs.3 (a) and 4 (a) present the obtained colored board in clutterfree and clutter environment, respectively. Since our goal is to select all potential tracks of nearly cyan, magenta, or yellow, through using both color and distance difference thresholds ( ε c = 60, ε d = 400 ) the extracted tracks are extracted and plotted in Figs.3 (b) and 4 (b), and each is nearly in cyan, magenta, or yellow. Besides, simulation results indicate that our proposed algorithm enjoys robust track initiation performance both in clutter-free and clutter environment.


9

x 10

4

507

4

9

x 10

8

7

7

Y-Coordinate in (m)

Y-Coordinate in (m)

track 1 track 2 track3 ghosts 8

6

5

4

6

5

4

3 4000

5000

6000

7000

8000

9000

10000

11000

12000

3 4000

13000

5000

6000

7000

X-Coordinate in (m)

8000

9000

10000

11000

12000

13000

X-Coordinate in (m)

a) The obtained color “board”

b) The extracted tracks

Fig. 3. Track extraction results in clutter-free environment with ε c = 60, ε d = 400 4

x 10

9

8

8

7

7

Y-Coordinate in (m)

Y-Coordinate in (m)

9

6

5

5

4

3

3

5000

6000

7000

8000

9000

10000

11000

12000

13000

X-Coordinate in (m)

a) The obtained color “board”

14000

4

6

4

2 4000

x 10

2 4000

5000

6000

7000

8000

9000

10000

11000

12000

13000

14000

X-Coordinate in (m)

b) The extracted tracks

Fig. 4. Track extraction results in clutter environment with ε c = 60, ε d = 400

Acknowledgments. This work is supported by National Natural Science Foundation of China (No. 60804068), Natural Science Foundation of Jiangsu Province (No.BK2010261), and Cooperation Innovation of Industry, Education and Academy of Jiangsu Province (No. BY2010126).

References 1. Cheng, H.-Y., Hwang, J.-N.: Adaptive particle sampling and adaptive appearance for multiple video object tracking. Signal Processing 89(9), 1844–1849 (2009) 2. Vo, B.-N., Singh, S., Doucet, A.: Sequential Monte Carlo methods for multi-target filtering with random finite sets. IEEE Trans. On Aerospace & Electronic Systems 41(4), 1224– 1245 (2005) 3. Hu, Z., Leung, H., Blanchette, M.: Statistical performance analysis of track initiation techniques. IEEE Transactions on Signal Processing 45(2), 445–456 (1997)

508


4. Xu, B., Chen, Q., Wang, Z.: Ants for Track Initiation of Bearings-Only Tracking. Simulation Modelling Practice and Theory 16(6), 626–638 (2008) 5. Xu, B., Wang, Z.: A Multi-objective-ACO-Based Data Association Method for BearingsOnly Multi-Target Tracking. Communications in Nonlinear Science and Numerical Simulation 12(8), 1360–1369 (2007) 6. Albers, J.: Interaction of Color. Revised and Expanded edn. Yale University Press, New Haven (2006) 7. Bhattacharya, P., Rosenfeld, Weiss, A.I.: Point-to-line mappings as Hough transforms. Pattern Recognition Letters 23, 1705–1710 (2002)

Visual Tracking of Multiple Targets by Multi-Bernoulli Filtering of Background Subtracted Image Data Reza Hoseinnezhad1 , Ba-Ngu Vo2 , and Truong Nguyen Vu3 1

3

RMIT University, Victoria, Australia [email protected] 2 The University of Western Australia, WA, Australia [email protected] Vietnam Academy of Science and Technology, Ho Chi Minh City, Vietnam

Abstract. Most visual multi-target tracking techniques in the literature employ a detection routine to map the image data to point measurements that are usually further processed by a filter. In this paper, we present a visual tracking technique based on a multi-target filtering algorithm that operates directly on the image observations and does not require any detection nor training patterns. Instead, we use the recent history of image data for non-parametric background subtraction and apply an efficient multi-target filtering technique, known as the multi-Bernoulli filter, on the resulting grey scale image data. In our experiments, we applied our method to track multiple people in three video sequences from the CAVIAR dataset. The results show that our method can automatically track multiple interacting targets and quickly finds targets entering or leaving the scene. Keywords: visual tracking, Bayesian estimation, multi-target tracking, random finite sets, multi-Bernoulli.

1

Introduction

Single-view visual tracking techniques invariably consist of detection followed by filtering. A detection module generates point measurements from the images in the video sequence which are then utilised as inputs by a filtering module, which estimates the number of targets and their states (properties such as location and size). Detection is an integral part of single-view visual tracking techniques. There is a large body of literature on models and techniques for detecting targets based on various background and foreground models. One of the most popular approaches is the detection of targets based on matching colour histograms of rectangular blobs [1,2]. Other recent methods include a game-theoretic approach [3], using human shape models [4,5], multi-modal representations [6], sample-based detection [7], range segmentation [8] and a multi-step detection scheme including median filtering, thresholding, binary morphology and connected components analysis [9]. Y. Tan et al. (Eds.): ICSI 2011, Part II, LNCS 6729, pp. 509–518, 2011. c Springer-Verlag Berlin Heidelberg 2011

510

R. Hoseinnezhad, B.-N. Vo, and T.N. Vu

Detection compresses the information on the image into a finite set of points measurements, and is efficient in terms of memory as well as computational requirements. However, this approach may not be adequate when the information loss incurred in the detection process becomes significant. Another problem with using detection is the selection of a suitable measurement model for the filtering algorithm. Modelling the detection process in a computationally tractable manner is a difficult problem. In practice, the selection of the measurement model is done on an ad-hoc basis and requires the manual tuning of model parameters. Using random finite set (RFS) theory, a tractable framework for tracking multiple targets from video data without detection was recently introduced in [10]. This work led to a novel method for tracking multiple targets in video and has been successfully demonstrated on sport players tracking [11]. However, this method requires prior information about the visual appearance of the targets to be tracked, and is most useful in cases where visual target model is available either a priori or from training data. In many applications, such as people surveillance, there is no prior information about the visual appearance of the targets and a new algorithm is needed. This paper presents a novel algorithm that tracks multiple moving targets directly from the image without any training data. Our proposed algorithm gradually learns and updates a probabilistic background model based on kernel density estimation. This resulting background model is then substracted to generate a grey scale foreground image from which the multi-target posterior distribution can be computed analytically using the multi-Bernoulli update of [10]. A sequential Monte Carlo implementation of the multi-Bernoulli filter is detailed and demonstrated through case studies involving people tracking in video sequences.

2

Background

In the context of jointly estimating the number of states and their values, the collection of states, referred to as the multi-target state, is naturally represented as a finite set. The rationale behind this representation traces back to a fundamental consideration in estimation theory–estimation error, see for example [10]. Since the state and measurement are treated as realisations of random variables in the Bayesian estimation paradigm, the finite-set-valued (multi-target) state X is modelled as a random finite set (RFS). Mahler’s Finite Set Statistics (FISST) provides powerfull yet practical mathematical tools for dealing with RFSs [12], [13], based on a notion of integration and density that is consistent with the wellestablished point process theory [14]. FISST has attracted substantial interest from academia as well as the commercial sector with the developments of the Probability Hypothesis Density (PHD) and Cardinalized PHD filters [12],[14], [15], [16], [17]. Let us denote the frame image observation by y = [y1 . . . ym ]. Then, using the FISST notion of integration and density, we can compute the posterior probability density π(·|y) of the multi-target state from the prior density via Bayes rule:

Visual Tracking of Multiple Targets by Multi-Bernoulli Filtering

π(X|y) =

g(y|X)π(X) g(y|X)π(X)δX

511

(1)

where g(y|X) is probability density (likelihood) of observation y given the multitarget state X and the integral over the space of finite sets is defined as follows: f (X)δX

∞ 1 f ({x1 , . . . , xi })dx1 . . . dxi . i! i=0

(2)

In this paper, the finite set of targets, X, is modelled by a multi-Bernoulli RFS which is defined as the union of M independent RFSs X (i) where M is the maximum number of targets. X=

M i=1

X (i) .

In this representation, each X (i) is either empty or a singleton with probabilities 1 − r(i) and r(i) , respectively. In the case where X (i) is a singleton, its only element is distributed according to a probability density p(i) (·). Thus, a complete representation of the multi-target state is given by {(r(i) , p(i) )}M i=1 . The Bayes update (1) is computationally intractable in general. Fortunately, it was shown in [10] that if the likelihood function has the following separable form: g(y|X) = f (y) g(x, y) (3) x∈X

and the multi-target RFS has a multi-Bernoulli prior distribution {(r(i) , p(i) )}M i=1 , then the posterior distribution of X, given by Bayes rule (1), is also multi(i) (i) Bernoulli with the parameters {(rupdated, pupdated)}M i=1 where: (i)

rupdated = (i)

pupdated(·) =

r(i) p(i) (·), g(·, y) 1 − r(i) + r(i) p(i) (·), g(·, y)

(4)

p(i) (·)g(·, y) p(i) (·), g(·, y)

(5)

and f1 , f2 denotes the standard inner product f1 (x)f2 (x)dx. In the next section, we show that the likelihood function for the image after background subtraction satisfies the above separable form. Using the update results (4) and (5), we detail a recursive filtering scheme that takes the background subtracted images as input to directly track multiple targets.

3

Visual Likelihood

Using background subtraction, each frame image is transformed into a grey scale image in which each pixel value is the probability density of the pixel belonging to the background. The background subtraction method used in this work

512


is based on kernel density estimation which has been quite popular in visual tracking [18,19,20]. The resulting grey scale image is then used as input to the multi-target filter. For simplicity of notation, we will use the y and yi symbols for the background subtracted grey scale image and its pixel values (which are indeed the probability density values of the actual pixel in the colour image to belong to background). We also assume that the yi values are normalised to the interval [0, 1]. 3.1

Background Subtraction

It is assumed that the pixel i in the k-th colour image frame of the video has an RGB colour denoted by [Ri (k) Gi (k) Bi (k)] . We first convert the RGB colour to chromaticity (rgI) colours by: ri (k) = Ri (k)/ (Ri (k) + Gi (k) + Bi (k))

(6)

gi (k) = Gi (k)/ (Ri (k) + Gi (k) + Bi (k))

(7)

Ii (k) = (Ri (k) + Gi (k) + Bi (k)) /256

(8)

where the denominator 256 applies to 8-bit colour quantisation. It is observed in [19] that chromaticity colour is more robust to ambience light variations and shadows. Note that the above colour components all vary within the interval [0, 1]. To compute the kernel density estimate of the i-th pixel to belong to background, we keep a stack of N0 image frames (each in the form of a 3-D array including all rgI colours of the pixels) and update the contents of the stack regularly after every K0 frames. The interpretation of the parameter K0 can be explained via an example: if the frame rate of the video is 25 and we are looking for moving targets that are not stationary for more than 5 seconds, we can choose K0 in the range of 5 × 25 = 125. The stack of images will initially contain all pixel values recorded at the sampling times 0, K0 , 2K0 , . . ., (N0 −1)K0 . This stack will be then updated (first at the time k = N0 K0 then in each K0 frames) by removing the first image from the bottom of the stack and appending the most recently recorded image (e.g. at the sampling time N0 K0 ). More precisely, at time k (for k ≥ N0 K0 ), the stack will contain the rgI values of all pixels at the times K0 k/K0 , K0 (k/K0 − 1), . . . , K0 (k/K0 − N0 + 1). The kernel density estimate of the likelihood of the event that the i-th pixel belongs to the background is then given by: pi (k) =

N0 −1 1 N di (k); di (K0 (k/K0 − )), σd2 N0

(9)

=0 d=r,g,I

2

1 0) where N (x; x0 , σ) √2πσ exp(− (x−x 2σ2 ) and σr , σg and σI are the bandwidth of Gaussian kernels for the rgI colours and are user-defined parameters chosen


513

between 0 and 1. To normalise the pi (k) values to vary within [0,1], the density 1 normalisation factors √ are removed which results in the following 3 (2π) σr σg σI

normalised yi values:

N0 −1 1 [di (k) − di (K0 (k/K0 − ))]2 yi (k) = exp − . N0 2σd2

(10)

=0 d=r,g,I

3.2

Likelihood Model

Having obtained a grey scale image with pixel values yi , we need to compute its likelihood for a given set of multi-target state X = {xj |j = 1, . . . , n}. Each target region is denoted by T (xj ) within which the average (or a weighted average) of all pixel values can be computed: yj = yi m j (11) i∈T (xj )

where mj = |T (xj )| is the number of pixels within the region T (xj ) defined by the state xj . The likelihood of the region T (xj ) to include a target is expressed as a function of y j denoted by gF (y j ). This function should be a strictly decreasing function in [0,1]. An appropriate choice of such function is gF (y j ) = ζF exp(−y j /δF ) where δF is a control parameter to tune the sensitivity to large average pixel values, and ζF is a normalising constant – see Fig. 1(a). Based on independence assumptions, the likelihood of all elements of the state set X to include target regions in the background-subtracted image is given by nj=1 gF (y j ). The rest of the pixels in the image that do not belong to any of the regions T (xj ) (j = 1, . . . , n), are highly likely to belong to background. This is an important condition as otherwise, there might be more than n targets in the scene and this violates the premise that there are n targets. Let us denote the rest of the image by: y−X (y) −

n

{yi |i ∈ T (xj )}

(12)

j=1

where (y) is the result of mapping the matrix y to a set containing all the pixel values. We also construct a new image by filling up all the target regions with background pixels (all yi values equal to 1), and denote the set of its pixel values by (y; X) which can also be expressed as below: ⎡ ⎤ n ⎢ ⎥ (y; X) = ⎣ {1, · · · , 1 }⎦ y−X . (13) j=1

mj times

The average (or weighted average) of pixels belonging to (y; X) is given by: ⎛ ⎞ m n 1 ⎝ yB = yi + (1 − yi )⎠ . (14) m i=1 j=1 i∈T (xj )

514

R. Hoseinnezhad, B.-N. Vo, and T.N. Vu § y g F ( y j ) ] F exp¨¨ j © GF

0

§y g B ( y B ) ] B exp¨¨ B © GB

· ¸¸ ¹

1

yj

0

(a)

· ¸¸ ¹

1

yB

(b)

Fig. 1. (a) Foreground likelihood model (b) Background likelihood model

This average is within [0,1] and expected to be very close to 1. Indeed, if there are any targets existing in the image but not included in the hypothesised state X, the low values of the pixels belonging to that target region will decrease y B . If the average target size is small relative to the whole image, this decreasing effect can be small. Therefore, the likelihood of y B to represent background region should be large only for y B values that are very close to 1. It is important to note that scattered noise (e.g. salt and pepper noise) in the background-subtracted image may cause reduction in y B , similar to the effect of small size targets. To prevent this, we remove such tiny noise and other areas of image (containing small-values pixels) by morphologically closing the image (erosion followed by dilation of the image using a small structural element). We denote the likelihood of y B to represent an all background region by gB (y B ) which is expected to be an increasing function of y B in [0,1]. We choose the exponential function gB (y B ) = ζB exp(y B /δB ) where δB is a control parameter to tune the sensitivity to deviations of the average pixel value from 1, and ζB is a normalising constant – see Fig. 1(b). As we will see later, the exponential form is a necessity here to provide a separable form for the total likelihood. Replacing y B from equation (14), we derive: m n i=1 yi + j=1 i∈T (xj ) (1 − yi ) gB (y B ) = ζB exp (15) m δB n m (1 − y ) i y j=1 i∈T (x ) i j i=1 = ζB exp exp (16) m δB m δB m n mj − i∈T (xj ) yi i=1 yi = ζB exp exp (17) m δB m δB j=1 m n mj (1 − y j ) i=1 yi = ζB exp exp . (18) m δB m δB j=1 Finally, the total likelihood of the image y for the given set of states X is given by: n g(y|X) = gB (y B ) gF (y j ). (19) j=1


515

By substituting the foreground and background likelihood functions, we derive the following separable form: m ! n mj (1 − yj ) i=1 yi g(y|X) = ζB exp exp gF (y j ) . (20) m δB m δB j=1 f (y)

4

gy (xj )

Monte Carlo Implementation

Our implementation is based on the method presented in [10], adapted to the likelihood function defined in (20) for multi-target visual tracking. Suppose that Mk−1 (i) (i) (i) at time k − 1, the posterior density {rk−1 , pk−1 }i=1 is given and each pk−1 (i,j)

(i,j) L

(i)

k−1 is represented by a set of weighted samples (particles) {wk−1 , xk−1 }j=1 . More precisely, (i)

Lk−1 (i)

pk−1 (x) =

j=1

(i,j)

w(k−1) δx(i,j) (x).

(21)

k−1

We assume a constant survival probability PS , and consider a predefined model (i) (i) Γ for birth particles denoted by known parameters {rΓ , pΓ,k }M i=1 where the density

Γ pΓ,k is represented by the particles {wΓ,k , xΓ,k }L j=1 . In our experiments, we assume that with a constant probability of 0.02, one target appears in each of the four quarters of the image planes, with the location of the target being (1) (4) uniformly distributed within the quarter. Thus, MΓ = 4, rΓ = · · · = rΓ = 0.02 and the birth particles are sampled with uniform distribution and weights. Similar to many other particle filtering schemes, in each iteration, the particles are predicted then updated. In the prediction step, the birth particles are generated according to the birth model parameters. The multi-Bernoulli parameters (i) (i,j) (i,j) from the previous iteration, {rk−1 , wk−1 , xk−1 }, are propagated forward:

(i)

(i,j)

(i,j)

(i,j)

(i,j)

(i)

(i)

(i,j)

(i,j)

xk|k−1 ∼ fk|k−1 (·|xk−1 ) ; rk|k−1 = PS rk−1 ; wk|k−1 = wk−1 .

(22)

The proposal density equals the state transition density fk|k−1 (·|xk−1 ). In our experiments, the targets are modelled by rectangular blobs and the target state is a 4-tuple vector comprising the x and y location and width and height. The target dynamic is modelled by x(k+1) = x(k)+e(k) where e(k) is a 4-dimensional 2 Gaussian variable with zero mean and variance Σ = diag(σx2 , σy2 , σh2 , σw ). Thus, fk|k−1 (x|xk−1 ) = N (x, Σ). In the update step, the predicted multi-Bernoulli parameters are updated using the likelihood function (20) and update formulas (4) and (5) which translate to: " # (i) (i) (i) (i) (i) (i) rk = rk|k−1 k / 1 − rk|k−1 + rk|k−1 k (23) (i,j)

wk (i)

where k =

L(i) k|k−1 j=1

(i,j)

(i,j)

(i)

= wk|k−1 gyk (xk|k−1 )/k (i,j)

(i,j)

wk|k−1 gyk (xk|k−1 ) [10].

(24)

516


Similar to the MeMBer filter [21], the updated particles are resampled with the number of particles reallocated in proportion to the probability of existence as well as restricted between a minimum Lmin and maximum Lmax . To reduce the growing number of multi-Bernoulli parameters, those with probabilities of existence less then a small threshold (set at 0.01) are removed. In addition, the targets with substantial overlap are merged. Finally, the number of targets and their states are estimated via finding the multi-Bernoulli parameters with existence probabilities larger than a threshold (set at 0.5 in our experiments). Each target state estimate is then given by the weighted average of the particles of the corresponding density.

5

Tracking Experiments

We demonstrate our method for tracking moving people in three video sequences from the CAVIAR dataset1 which is a benchmark for visual tracking experiments. The tracking results are available to download and view from our home page.2 The first video shows two persons each entering the lobby of a lab in INRIA and leaving the environment. The second video shows people walking in a shopping centre and occasionally visiting a shop that is in the front view of the camera. The third video shows four people entering the same place as in the first video, walking together and leaving the lobby. Except for a small number of frames, the four people are relatively accurately detected and tracked at all times. In this video, we also show the background subtracted (grey scale) images to give an indication of how our tracking method uses the results of background subtraction. Figure 2 shows snapshots of the third video. It demonstrates that in general, our method can accurately track multiple targets in the video. The tracking results in the frames shown in Fig. 2 also present the ability of our tracking technique in detecting the arrival of new targets into the scene and tracking them while moving and interacting with other targets, and detecting their departure from the scene.

6

Conclusions

A novel algorithm for tracking multiple targets directly from image observations has been presented. Using kernel density estimation, the proposed algorithm gradually learns and updates a probabilistic background model which is then used to generate a grey scale foreground image. A separable likelihood function has been derived for the grey scale foreground image, which enabled an efficient multi-target filtering technique called multi-Bernoulli filtering to be applied. The method has been evaluated in three tracking scenarios from the CAVIAR datasets, showing that multiple persons can be tracked accurately. 1 2

http://homepages.inf.ed.ac.uk/rbf/CAVIARDATA1/ Video 1: www.dlsweb.rmit.edu.au/eng1/Mechatronics/Case01.mpg Video 2: www.dlsweb.rmit.edu.au/eng1/Mechatronics/Case02.mpg Video 3: www.dlsweb.rmit.edu.au/eng1/Mechatronics/Case03.mpg


Frame 83 out of 491

Frame 152 out of 491



50

50

50

100

100

100

100

150

150

150

150

200

200

200

200

250

250

250

100

200

300

100


200


50

250 100

300

200

300

100


50

50

50

50

100

100

100

150

150

150

150

200

200

200

200

250

250

250

200

300

100

200

300

300

250 100


200


100

100

517

200

300

100

200

300


50

50

100

100

150

150

200

200 250

250 100

200

300

100

200

300

Fig. 2. Tracking of up to four people in a video sequence from CAVIAR dataset. The selected frames show that the method is capable of detecting and tracking multiple moving objects as they enter the scene, interact and leave the scene.

Acknowledgement This work was supported by ARC Discovery Project grant DP0880553. Authors thank Dr Branko Ristic from DSTO, Australia for his contribution.

References 1. Okuma, K., Taleghani, A., De Freitas, N., Little, J., Lowe, D.: A boosted particle filter: Multitarget detection and tracking. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3021, pp. 28–39. Springer, Heidelberg (2004) 2. Kristan, M., Per, J., Pere, M., Kovacic, S.: Closed-world tracking of multiple interacting targets for indoor-sports applications. Computer Vision and Image Understanding 113(5), 598–611 (2009) 3. Yang, M., Yu, T., Wu, Y.: Game-theoretic multiple target tracking. In: ICCV 2007, Rio de Janeiro, Brazil (2007), http://dx.doi.org/10.1109/ICCV.2007.4408942 4. Wu, B., Nevatia, R.: Detection and tracking of multiple, partially occluded humans by Bayesian combination of edgelet based part detectors. IJCV 75(2), 247–266 (2007) 5. Zhao, T., Nevatia, R., Wu, B.: Segmentation and tracking of multiple humans in crowded environments. PAMI 30(7), 1198–1211 (2008) 6. Apewokin, S., Valentine, B., Bales, R., Wills, L., Wills, S.: Tracking multiple pedestrians in real-time using kinematics. In: CVPR 2008 Workshops, Anchorage, AK, United states (2008), http://dx.doi.org/10.1109/CVPRW.2008.4563149

518


7. Zhu, L., Zhou, J., Song, J.: Tracking multiple objects through occlusion with online sampling and position estimation. Pattern Recognition 41(8), 2447–2460 (2008) 8. Parvizi, E., Wu, Q.J.: Multiple object tracking based on adaptive depth segmentation. In: Canadian Conference on Computer and Robot Vision – CRV 2008, Windsor, ON, Canada, pp. 273–277 (2008) 9. Abbott, R., Williams, L.: Multiple target tracking with lazy background subtraction and connected components analysis. Machine Vision and Applications 20(2), 93–101 (2009) 10. Vo, B.N., Vo, B.T., Pham, N.T., Suter, D.: Bayesian multi-object estimation from image observations. In: Fusion 2009, Seatle, Washington, pp. 890–898 (2009) 11. Hoseinnezhad, R., Vo, B.N., Suter, D., Vo, B.T.: Multi-object filtering from image sequence without detection. In: ICASSP, Dallas, TX, pp. 1154–1157 (2010) 12. Mahler, R.: Multi-target bayes filtering via first-order multi-target moments. IEEE Trans. Aerospace & Electronic Systems 39(4), 1152–1178 (2003) 13. Mahler, R.: Statistical multisource-multitarget information fusion. Artech House, Boston (2007) 14. Vo, B.N., Singh, S., Doucet, A.: Sequential Monte Carlo methods for multi-target filtering with random finite sets. IEEE Tran. AES 41(4), 1224–1245 (2005) 15. Vo, B.N., Ma, W.K.: The Gaussian mixture probability hypothesis density filter. IEEE Trans. Signal Proc. 54(11), 4091–4104 (2006) 16. Mahler, R.: Phd filters of higher order in target number. IEEE Trans. Aerospace & Electronic Systems 43(4), 1523–1543 (2007) 17. Vo, B.T., Vo, B.N., Cantoni, A.: Analytic implementations of the Cardinalized Probability Hypothesis Density filter. IEEE Trans. Signal Processing 55(7), 3553– 3567 (2007) 18. Tyagi, A., Keck, M., Davis, J.W., Potamianos, G.: Kernel-based 3D tracking. In: CVPR 2007, Minneapolis, Minnesota, USA (2007) 19. Elgammal, A., Duraiswami, R., Harwood, D., Davis, L.S.: Background and foreground modeling using nonparametric kernel density estimation for visual surveillance. Proceedings of the IEEE 90(7), 1151–1162 (2002) 20. Han, B., Comaniciu, D., Zhu, Y., Davis, L.S.: Sequential kernel density approximation and its application to real-time visual tracking. PAMI 30(7), 1186–1197 (2008) 21. Vo, B.T., Vo, B.N., Cantoni, A.: The cardinality balanced multi-target multiBernoulli filter and its implementations. IEEE Transactions on Signal Processing 57(2), 409–423 (2009)

Mobile Robotics in a Random Finite Set Framework John Mullane1 , Ba-Ngu Vo2 , Martin Adams3 , and Ba-Tuong Vo2 1 2

Nanyang Technological University, Singapore [email protected] University of Western Australia, Perth, Australia {ba-ngu.vo,ba-tuong.vo}@uwa.edu.sg 3 University of Chile, Santiago, Chile [email protected]

Abstract. This paper describes the Random Finite Set approach to Bayesian mobile robotics, which is based on a natural multi-object filtering framework, making it well suited to both single and swarm-based mobile robotic applications. By modeling the measurements and feature map as random finite sets (RFSs), joint estimates the number and location of the objects (features) in the map can be generated. In addition, it is shown how the path of each robot can be estimated if required. The framework differs dramatically from existing approaches since both data association and feature management routines are integrated into a single recursion. This makes the framework well suited to multi-robot scenarios due to the ease of fusing multiple map estimates from swarm members, as well as mapping robustness in the presence of other mobile robots which may induce false map measurements. An overview of developments thus far is presented, with implementations demonstrating the merits of the framework on simulated and experimental datasets. Keywords: mobile robotics, Bayesian estimation, random finite sets, Probability Hypothesis Density.

1

Introduction

Mobile robotics is becoming an increasingly popular field of research, especially as embedded computing technology matures and gets ever more sophisticated. The goal of a mobile robot (or a swarm of mobile robots) is typically to travel through an unknown environment autonomously, while being continuously aware of both its surroundings and its position in relation to those surroundings. To measure the area of operation, robots are equipped with a suite of sensors such as lasers, radar, sonar or cameras which are inherently prone to sensing and data association error. In addition, the control inputs applied to a robots actuators introduce further uncertainty into the robot’s position due to noise and kinematic modeling errors of the robot. Due to these multiple sources of uncertainty, Bayesian approaches have become widely popular [1], [2], [3], [4], which adopt Y. Tan et al. (Eds.): ICSI 2011, Part II, LNCS 6729, pp. 519–528, 2011. c Springer-Verlag Berlin Heidelberg 2011

520

J. Mullane et al.

probabilistic robot and sensor models and attempt to extract optimal estimates of the map and robots. By far the most common approach to the problem is to use a random vector framework, in which the map and robot paths are modeled as random vectors containing positional information about the features and robot locations respectively [1]. While this model is the basis for the majority of existing mobile robotics algorithms, it requires independent data association and map management routines to respectively assign measurements to features and to estimate the number of features in the map [5], [6]. Recently, a new framework has been developed using Random Finite Set (RFS) models [7], [8], [9], which alleviates the need for independent routines and unifies the stochastic mobile robotics framework into a single Bayesian recursion. This new approach admits numerous benefits such as removal of data association, increased robustness to measurement error, integration of map management, straightforward fusion of multiple robot map estimates, expected map estimation and can be readily applied to single or multiple robot scenarios. This paper advocates a fully integrated Bayesian framework for mobile robotics under DA uncertainty and unknown feature number. The key to this formulation is the representation of the map as a finite set of features. Indeed, from an estimation viewpoint, it is argued in section that the map is indeed a finite set and not a vector. Using Random Finite Set (RFS) theory, mobile robotics is then posed as a Bayesian filtering problem in which the posterior distribution of the set-valued map is propagated forward in time as measurements arrive. In the case of an unknown robot path, the joint density including the robot trajectory can be propagated. A tractable solution which propagates the first order moment of the map, its Probability Hypothesis Density (PHD), is presented. The PHD construct can also be interpreted in terms of occupancy maps [9], [10]. In this paper, both the map estimation from a known robot path and joint map/ trajectory estimation from an unknown robot path are examined separately. In particular, mapping robustness to multiple robots, which may interfere with the map building process, is demonstrated.

2

Background

Map estimation is closely related to the multi-target filtering problem, where the aim is to jointly estimate the time-varying number of targets (features) and their states from sensor measurements in the presence of data association uncertainty, detection uncertainty, clutter and noise. The first systematic treatment of this problem using random set theory was conceived by Mahler in 1994 [11], which later developed into Finite Set Statistics and the Probability Hypothesis Density (PHD) filter in 2003 [12]. A detailed treatment can be found in [13]. The mobile robotics problem was first formulated in an RFS framework in [7], with mapping and localisation algorithms presented in [8]. The approach modeled the joint vehicle trajectory and map as a single RFS, and recursively propagates its first order moment. Stemming from the popular FastSLAM algorithm [6], a factored

Mobile Robotics in a Random Finite Set Framework

521

approach to RFS-SLAM was proposed in [14] however vector-based approximations of RFS integrals were used which invalidate the approach. A theoretically correct factored approach was presented in [15], [9], which propagates the posterior PHDs of multiple trajectory-conditioned maps and the posterior distribution of the vehicle trajectory, with implementations on marine based radar data also appearing in [16].

3

The Robotics RFS Filtering Framework

This section details both the RFS map estimation filter and the RFS SLAM filter. 3.1

The Robotic Mapping Problem

As was first proposed in [7] and [8], let M be the RFS representing the entire unknown environment comprising static map features and multiple mobile robots and let Mk−1 be the RFS representing the subset of the map that has passed through the field-of-view (FoV) of the on-board sensor with trajectory X0:k−1 = [X0 , X1 , . . . , Xk−1 ] at time k − 1, i.e. Mk−1 = M ∩ F oV (X0:k−1 ). Given this representation, Mk−1 evolves in time according to, ¯ Mk = Mk−1 ∪ F oV (Xk ) ∩ Mk−1

(1)

(2)

¯ k−1 = M − Mk−1 , i.e the set of features that are not in Mk−1 . If where M fk|k−1 (Mk |Mk−1 , Xk−1 ) then represents the RFS feature map state transition density, the generalised Bayesian RFS robotic mapping recursion can be written [17], pk|k−1 (Mk |Z0:k−1 , X0:k ) = fk|k−1 (Mk |Mk−1 , Xk )× pk−1 (Mk−1 |Z0:k−1 , Xk−1 )δMk−1 pk (Mk |Z0:k , X0:k ) =

gk (Zk |Xk , Mk )pk|k−1 (Mk |Z0:k−1 , X0:k ) gk (Zk |Xk , Mk )pk|k−1 (Mk |Z0:k−1 , X0:k )δMk

(3)

(4)

where gk (Zk |·) denotes the likelihood of the RFS measurement and δ denotes a set integral. Integration over the map, requires integration over all possible feature maps (all possible locations and numbers of features). 3.2

The Joint Robotic Mapping and Localisation Problem

To jointly estimate the map and the robot trajectory, denoted by the random vector X1:k , the posterior density of (4) can be modified to get,

522

J. Mullane et al.

pk (Mk , X1:k |Z0:k , U0:k−1 , X0 ) = pk (X1:k |Z0:k , U0:k−1 , X0 )pk (Mk |Z0:k , X0:k ). (5) where U0:k−1 denotes the robot control inputs random vector. Note that the second term is exactly equivalent to the posterior of (4). The first term can be calculated via, pk|k−1 (Mk |Z0:k−1 , X0:k ) pk (Mk |Z0:k , X0:k ) pk|k−1 (X1:k |Z0:k−1 , U0:k−1 , X0 ) × gk (Zk |Z0:k−1 )

pk (X1:k |Z0:k , U0:k−1 , X0 ) = gk (Zk |Mk , Xk ) ×

(6)

Further details can be seen in [9], [10], [15], [16]. 3.3

First Order Moment Approximation

As with the classical approaches, the previous Bayesian recursions are numerically intractable and sensible approximations are required. In this work, the predicted and posterior RFS maps of (3) and (4) are approximated by Poisson RFSs with PHDs vk|k−1 (m|Z0:k−1 , X0:k ) and vk (m|Z0:k , X0:k ). In essence, this approximation assumes that features are iid and the number of features is Poisson distributed, i.e., vk|k−1 (m|X0:k ) m∈Mk pk|k−1 (Mk |Z0:k−1 , X0:k ) ≈ (7) exp vk|k−1 (m|X0:k )dm vk (m|X0:k ) m∈Mk . pk (Mk |Z0:k , X0:k ) ≈ (8) exp vk (m|X0:k )dm This PHD approximation has been proven to be effective in multi-target tracking [13]. As shown in [12], [17], the recursion of (3) and (4) can equally be reduced to a predictor corrector form. The PHD predictor equation is, vk|k−1 (m|X0:k ) = vk−1 (m|X0:k−1 ) + b(m|Xk )

(9)

where b(m|Xk ) is the PHD of the new feature RFS, B(Xk ). The PHD corrector equation is then, vk (m|X0:k ) = vk|k−1 (m|X0:k ) 1 − PD (m|Xk )+

Λ(m|Xk ) (10) ck (z|Xk ) + Λ(ζ|Xk )vk|k−1 (ζ|X0:k )dζ z∈Z k

where, PD (m|Xk ) is the probability of detecting a feature at m, ck (z|Xk ) is the PHD of the clutter RFS and Λ(m|Xk ) = PD (m|Xk )gk (z|m, Xk ). The PHD recursion is far more numerically tractable than propagating the RFS map densities of (4). In addition, the recursion can be readily extended to incorporate


523

Fig. 1. An example of a map PHD superimposed on the true map represented by black dots. The peaks of the PHD represent locations with highest concentration of expected number of features. The PHD on the left is at time k − 1, and that on the right is at time k.

multiple sensors / swarms of robots by sequentially updating the map PHD with the measurement from each robot. In addition, using the same Poisson approximations, (6) can be readily evaluated. A graphical depiction of a PHD approximated by a Gaussian Mixture before and after an update is shown in figure 1. The PHD construct is also beneficial to the multi-robot submapping problem [18], where the global map may be recovered by simply adding (and averaging the over-lap) the PHD sub-maps.

4

Filter Implementations

This section outlines a Gaussian Mixture implementation of the proposed filters. For the Robotic Mapping filter of section 3.1,let the predicted map PHD, Jk−1

vk−1 (m|Xk−1 ) =

j=1

(j) (j) (j) ηk−1 N m; μk−1 , Pk−1 (j)

(j)

(11)

(j)

which is a mixture of Jk−1 Gaussians, with ηk−1 , μk−1 and Pk−1 being the corresponding predicted weights, means and covariances respectively for the j th Gaussian component of the map PHD. Let the new feature intensity, b(m|Zk−1 , Xk ), also be a Gaussian mixture of the form, b(m|Zk−1 , Xk ) =

Jb,k

j=1

(j) (j) (j) ηb,k N m; μb,k , Pb,k

(12)

where, Jb,k defines the number of Gaussians in the new feature intensity at (j) (j) (j) time k and ηb,k , μb,k and Pb,k are the corresponding components. The predicted intensity is therefore also a Gaussian mixture, Jk|k−1

vk|k−1 (m|Xk ) =

j=1

(j) (j) (j) ηk|k−1 N m; μk|k−1 , Pk|k−1

(13)

524

J. Mullane et al.

which consists of Jk|k−1 = Jk−1 + Jb,k Gaussians representing the union of the prior map intensity, vk−1 (m|Xk−1 ), and the proposed new feature intensity, according to (9). Since the measurement likelihood is also of Gaussian form, it follows from (10) that the posterior map PHD, vk (m|Xk ) is then also a Gaussian mixture given by,

Jk|k−1

(j) vk (m|Xk ) = vk|k−1 (m|Xk ) 1 − PD (m|Xk ) + vG,k (z, m|Xk ) . z∈Zk

j=1

The components of the above equation are given by, (j)

(j)

(j)

(j)

vG,k (z, m|Xk ) = ηk (z|Xk )N (m; μk|k , Pk|k )

(14)

(j)

(j) ηk (z|Xk )

=

PD (m|Xk )ηk|k−1 q (j) (z, Xk ) Jk|k−1

() c(z) + PD (m|Xk )ηk|k−1 q () (z, Xk )

(15)

=1

(j) (j) z; Hk μk|k−1 , Sk .

where, q (z, Xk ) = N The terms μk|k , Pk|k and Sk can be obtained using any standard filtering technique such as EKF or UKF. In this paper, the EKF updates are adopted. The clutter RFS, Ck , is assumed Poisson distributed [2] in number and uniformly spaced over the mapping region and Gaussian management methods are carried out as in [19]. For the Robotic Mapping filter of section 3.2, the location density of (6) can be propagated via Particle Filtering techniques [6], [15]. If the vehicle transition density is chosen as the proposal distribution, the weighting for the ith particle becomes, (i) (i) )w(i) . w k = gk (Zk |Z0:k−1 , X (16) 0:k k−1 (j)

Then for each hypothesised robot trajectory particle, an independent robotic mapping filter of equations (17)-(22) is executed.

5

Results and Analysis

The benchmark algorithms used in the analysis are the EKF-based mapping filters [4] and the FastSLAM localisation [6] algorithm with maximum likelihood data association, using a mutual exclusion constraint and a 95% χ2 confidence gate. An arbitrary simulated feature map and vehicle trajectory are used as shown in figure 2. Existing approaches to mobile robotics typically deal with interfering measurements through ‘feature management’ routines, primarily via the landmark’s (feature’s) quality (LQ) [4] or a binary Bayes filter [6]. These operations are typically independent of the main filtering update, whereas the proposed approach unifies feature management, data association and state filtering into a single Bayesian update giving it a more robust performance in the presence of multiple mobile robots.


525

Fig. 2. Left: The simulated environment showing point features (green circles). A sample measurement history plotted from the robot trajectory (green line) is shown. Right: Comparison of mapping error vs. measurement noise for the proposed filters and classical vector EKF solutions.

Figure 2 also shows the filter performance in increasing measurement noise. Figure 3 shows a comparison of the map estimate for each filter at differing noise inflation values, γ. In a mobile robot swarm, mapping robots should exclude the moving robots from the static feature map. As such, figure 3 also depicts the suitability of the proposed approach to swarm based mapping, as the approach is robust to an increasing number of other robots. The proposed mapping framework performs well in the presence of increased measurement uncertainty and other mobile robots.

Fig. 3. Left: Feature mapping error vs. clutter density for vector based NN-EKF and JCBB-EKF approaches and the proposed PHD framework, with the PHD approach seen to perform well in high clutter. Right: Comparison of the map estimation error in the presence of increasing densities per square meter of mobile robots.

By assuming an unknown vehicle trajectory, and applying random control inputs, this section analyses the proposed joint mapping and localisation filter in comparison to the popular FastSLAM algorithm [6]. Figure 5 demonstrates the ability of the trajectory estimation filter to estimate the location of the robot at each time step, showing less error than conventional methods. In terms of estimating the map from an unknown path, figure 4 show how the RFS framework and proposed filters maintain accurate estimates of the number of features in the map and their locations, even in the presence of false measurements from other robots

526

J. Mullane et al.

Fig. 4. Left: The average estimated number of features in the map vs. ground truth for each approach. The feature number estimate from the proposed approach can be seen to closely track that of the ground truth. Right: A comparative plot of the mean and standard deviation of the map estimation error vs. time. Note that the ‘ideal’ error converges to zero, an important property for robotic mapping filters. 7

Average Positional RMSE (std)

GPS

RB−PHD−SLAM NN−FastSLAM-FE NN−FastSLAM-LQ

6

5

Wireless Modem

Industrial PC

FMCW MMWR

4

LMS 200

LMS 200 E-Stop

Steering Encoder

Motor Speed Controllers

3

2

1

Wheel Encoders 0

0

500

1000

1500

2000

2500

3000

3500

4000

4500

Time Index

Fig. 5. Left: The mean and standard deviation of the expected trajectory estimates of the proposed RFS approach versus that of FastSLAM over 50 MC runs. LQ refers to an implementation with the ‘landmark quality’ method of [4]. Right: The Autonomous Robot used in experimental trials.

Fig. 6. A: Raw radar measurements and noisy vehicle path. B: The scan map plotted from the GPS path. C: Posterior Estimate from FastSLAM, D: Posterior Estimate from PHD-SLAM.


527

and mapping noise. Experimental results based on a 77GHz millimeter wave radar mounted on an autonomous robot, as seen in figure 5, are shown in 6.

6

Conclusion

This paper shows that from a fundamental estimation viewpoint that a featurebased map is a finite set and subsequently presented a Bayesian filtering formulation as well as a tractable solution for the feature-based mobile robotics problem. The framework outlined here presents a new direction of research for the multiple mobile robot community, which naturally encapsulates the inherent system uncertainty. Both a mapping-only and joint robot trajectory / map filter was introduced and analysed. In contrast to existing frameworks, the RFS approach to mobile robotics jointly estimates the number of features in the map as well as their individual locations in the presence of data association uncertainty and clutter. It was also shown that this Bayesian formulation admits a number of optimal Bayes estimators for mobile robotics problems. Analysis was carried out both in a simulated environment through Monte Carlo trials, demonstrating the robustness of the proposed filter, particulary in the presence of large data association uncertainty and clutter, illustrating the merits of adopting an RFS approach a swarm-based robotics applications.

Acknowledgements This research is funded in part by the Singapore National Research Foundation through the Singapore-MIT Alliance for Research and Technology CENSAM. The second author is supported in part by discovery grant DP0880553 awarded by the Australian Research Council.

References 1. Smith, R., Self, M., Cheeseman, P.: Estimating uncertain spatial relationships in robotics. In: Autonomous Robot Vehicles, pp. 167–193 (1990) 2. Makarsov, D., Durrant-Whyte, H.: Mobile vehicle navigation in unknown environments: a multiple hypothesis approach. In: IEE Proceedings of Contr. Theory Applict., vol. 142 (July 1995) 3. Thrun, S.: Particle filter in robotics. In: Uncertainty in AI (UAI) (2002) 4. Dissanayake, G., Newman, P., Durrant-Whyte, H., Clark, S., Csorba, M.: A solution to the simultaneous localization and map building (SLAM) problem. IEEE Transactions on Robotic and Automation 17(3), 229–241 (2001) 5. Guivant, J., Nebot, E., Baiker, S.: Autonomous navigation and map building using laser range sensors in outdoor applications. Journal of Robotic Systems 17(10), 565–583 (2000) 6. Montemerlo, M., Thrun, S., Siciliano, B.: FastSLAM: A Scalable Method for the Simultaneous Localization and Mapping Problem in Robotics. Springer, Heidelberg (2007)

528

J. Mullane et al.

7. Mullane, J., Vo, B., Adams, M., Wijesoma, W.: A random set formulation for bayesian SLAM. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, France (September 2008) 8. Mullane, J., Vo, B., Adams, M., Wijesoma, W.: A random set approach to SLAM. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) workshop on Visual Mapping and Navigation in Outdoor Environments, Japan (May 2009) 9. Mullane, J., Vo, B., Adams, M., Vo, B.: A random finite set approach to bayesian SLAM. IEEE Transactions on Robotics 27(2), 268–283 (2011) 10. Mullane, J., Vo, B., Adams, M., Vo, B.: Random Finite Sets for Robot Mapping & SLAM. Springer Tracts in Advanced Robotics (to appear) 11. Mahler, R.: Global integrated data fusion. In: Proc. 7th Nat. Symp. on Sensor Fusion, vol. 1, pp. 187–199 (1994) 12. Mahler, R.: Multi-target bayes filtering via first-order multi-target moments. IEEE Transactions on AES 4(39), 1152–1178 (2003) 13. Mahler, R.: Statistical Multisource Multitarget Information Fusion. Artech House (2007) 14. Kaylan, B., Lee, K., Wijesoma, W.: FISST-SLAM: Finite set statistical approach to simultaneous localization and mapping. International Journal of Robotics Research 29(10), 1251–1262 (2010), Published online first in October 2009 15. Mullane, J., Vo, B., Adams, M.: Rao-blackwellised PHD SLAM. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Alaska, USA (May 2010) 16. Mullane, J., Keller, S., Rao, A., Adams, M., Yeo, A., Hover, F., Patrikalakis, N.: X-band radar based SLAM in singapore’s off-shore environment. In: Proceedings of the 11th IEEE ICARCV, Singapore (December 2010) 17. Vo, B., Singh, S., Doucet, A.: Sequential monte carlo methods for multi-target filtering with random finite sets. IEEE Transactions on Aerospace and Electronic Systems 41(4), 1224–1245 (2005) 18. Shoudong, H., Zhan, W., Dissanayake, G.: Sparse local submap joining filter for building large-scale maps. IEEE Transactions on Robotics 24(5), 1121–1130 (2008) 19. Vo, B., Ma, W.: The gaussian mixture probability hypothesis density filter. IEEE Transactions on Signal Processing 54(11), 4091–4104 (2006)

IMM Algorithm for a 3D High Maneuvering Target Tracking Dong-liang Peng and Yu Gu Institute of Information and Control, Hangzhou Dianzi University, Hangzhou, 310018, China {dlpeng,guyu}@hdu.edu.cn

Abstract. A major challenge posed by target tracking problems is the target flying at high speeds and performing “high-g” turns in the 3D space. In this situation, horizontally or decoupled models may lead to an unacceptable accuracy. To address this problem an IMM algorithm that includes a 3D constant velocity model (CV), a 3D “current” statistic model (CSM), and 3D constant speed coordinated turn model (3DCSCT) with the kinematic constant for constant speed targets is proposed. The tracking performance of the proposed IMM algorithm is compared with that of an IMM algorithm utilizing CV, 3DCSCT, and constant acceleration model (CA) or singer model. Simulation results that demonstrate the algorithm is feasible and practical for a 3D high maneuvering target tracking. Keywords: 3D constant speed coordinate turn model, high maneuvering target tracking, IMM.

1 Introduction The problem of tracking maneuvering targets has been studied extensively since the mid 1960s. However, it’s still a challenge to track targets that flying at high speeds, particularly performing “high-g” turns in the 3D space. When targets maneuver in a horizontal plane with nearly constant speed and turn rate and have little or limited vertical maneuver(such as civilian aircraft in ATC system), many of 2D horizontal models and algorithms can lead to an acceptable accuracy [1]. In practice, targets (such as military aircraft) always perform an arbitrary trajectory in the 3D space with high speed or coordinated-turn. In this situation, horizontally or decoupled models may lead to an unacceptable accuracy. So it’s necessary to investigate an algorithm for a 3D maneuvering target tracking. The multiple model approach has been observed to be a successful method for maneuvering target tracking [2]. The results of previous investigations also indicate that the IMM algorithm is the superior technique for tracking maneuvering targets when the computational requirements of the technique are considered [1, 2, 3]. The IMM algorithm uses model (Markov chain state) probabilities to weigh the inputs and outputs of a bank of parallel Kalman filters (maybe other filter) at each time instant. The key matter of the IMM algorithm is how to select models to match the real motion mode. In other words, the combination of models in the IMM algorithm plays an important role for the final tracking accuracy. Y. Tan et al. (Eds.): ICSI 2011, Part II, LNCS 6729, pp. 529–536, 2011. © Springer-Verlag Berlin Heidelberg 2011

530

D.-l. Peng and Y. Gu

If the target flying at high speeds and performing coordinated-turns in 3D space, an algorithm use single model (such as constant acceleration model, singer model, “current” statistical model and so on) can exhibit considerable model error [4]. The 3D constant speed coordinate turn model (3DCSCT) with the kinematic constraint for constant speed targets is considered in the IMM algorithm in order to match the coordinated-turn mode. The turning rate, as a parameter in 3DCSCT model, is computed as the magnitude of the target acceleration divided by its speed. So the precision of this model partly depends on the estimation of acceleration and velocity. Three model combinations that include CA-CV-3DCSCT, Singer -CV-3DCSCT, and CSM-CV-3DCSCT are given to track the 3D high maneuvering target. Simulation results show that the IMM algorithm of CSM-CV-3DCSCT is more effective than other two combinations for a maneuvering target with a high acceleration. This paper is structured as follows. Four 3D motion models are reviewed in Section 2. In Section 3, the IMM estimator for target tracking with r modes is outlined. In Section 4, the performance of the proposed IMM algorithm is illustrated and the analysis of the simulation results is given. Finally in Section 5, we make some concluding remarks.

2 3D Motion Models 2.1 Constant Acceleration Model (CA) CA model assumes that the acceleration derivative is a white noise, i.e.

a (t ) = w(t )

(2.1)

The corresponding 3D discrete-time model is

x(k) = diag[F(k), F(k), F(k)]x(k −1) + w(k) with state

(2.2)

x = [ x, x, x, y, y, y, z, z, z ] , and where ⎡1 T T 2 / 2 ⎤ ⎢ ⎥ F (k ) = ⎢ 0 1 T ⎥ ⎢0 0 1 ⎥⎦ ⎣

(2.3)

2.2 Singer Model (Singer) Singer model is explored by singer in 1970 [4]. It assumes that the target acceleration a (t ) is a zeros-mean first-order stationary Markov process with autocorrelation [5]

R a (τ ) = E { a ( t ) a ( t + τ )} = σ a2 e − α τ

(a ≥ 0)

(2.4)

where a is the reciprocal of the maneuver time constant and depends on how long the maneuver lasts.

σ a2

is the “instantaneous variance” of the acceleration.

Such a process a (t ) is the state process of a linear time-invariant system

IMM Algorithm for a 3D High Maneuvering Target Tracking

a (t ) = −α a (t ) + w(t ) (α > 0) where w(t ) is zero-mean white noise with power spectral density 2 aσ The corresponding 3D discrete-time model is

531

(2.5) 2 . a

x(k) = diag[F(k), F(k), F(k)]x(k −1) + w(k)

(2.6)

with state x = [ x, x, x, y, y, y, z , z , z ] , and where

⎡1 T (αT −1+ e−αT ) / α 2 ⎤ ⎢ ⎥ F (k) = ⎢0 1 (1− e−αT )/ α ⎥ ⎢0 0 ⎥ e−αT ⎣ ⎦

(2.7)

2.3 “Current” Statistical Model (CSM) In practice, if a target maneuvers with a specific acceleration at time k, the acceleration at time k+1 should be correlated with the “current” acceleration. Considering this fact, Zhou H R proposed a “current” statistical model [6]. It assumes the acceleration has a non-zero mean, such a non-zero-mean acceleration satisfies

a (t ) = −α a (t ) + α a (t ) + w(t ) where

(2.8)

a (t ) is the mean of the acceleration, assumed to be constant over each

sampling interval. So the estimate aˆ( k ) of a ( k ) is taken to be the “current” value of the mean a (k + 1) , and this is available online information. The corresponding 3D discrete-time equivalent can be represented as

x(k ) = diag[ F (k ), F (k ), F (k )]x(k − 1) + diag[U (k ), U (k ), U (k )]a + w(k ) with the state

(2.9)

x = [ x, x, x, y, y, y, z, z, z ] , and where

⎡1 T ⎢ F (k ) = ⎢ 0 1 ⎢0 0 ⎣

⎡1 αT 2 1 − e−αT ⎤ ( − T + + ) ⎢α α ⎥ 2 (α T − 1 + e−α T ) / α 2 ⎤ ⎢ ⎥ ⎥ 1 − e−αT ⎥ (1 − e −α T ) / α ⎥ , U (k ) = ⎢ T− ⎢ ⎥ α ⎥ ⎢ ⎥ e −α T −αT ⎦ 1− e ⎢ ⎥ ⎢ ⎥ ⎣ ⎦

(2.10)

2.4 Constant Speed Coordinate Turn (CSCT) The constant speed coordinated turn model [7, 8] assumes a circular target moves at constant turn rate in a plane (for a constant speed motion, the acceleration vector is orthogonal to the velocity vector). For an arbitrary plane of maneuver, the acceleration can be described as

a = Ω×v

(2.11)

532


where Ω is the (constant) turn rate vector and Ω = 0 , v is the velocity vector. Taking the derivative of (2.12) lead to the following equivalent

a = (Ω ⋅ v )Ω − ( Ω ⋅ Ω )v Using the fact that as where

ω

(2.12)

v is orthogonal to Ω , that is, Ω ⊥ v , (2.13) can be reformulated a = −ω 2 v

(2.13)

is defined as

ω

Ω =

a

(2.14)

v

If the acceleration perturbations modeled as white noise expressed as

w , (2.13) can be

a = −ω 2 v + w

(2.15)

The corresponding 3D discrete-time model is

x(k) = diag[F(ω), F(ω), F(ω)]x(k −1) + w(k) where

sin ωT ⎡ ⎢1 ω ⎢ F (ω ) = ⎢0 cos ωT ⎢ ⎢ ⎢0 −ω sin ωT ⎣⎢

1 − cos ωT ⎤ ⎥ ω2 ⎥ sin ωT ⎥ ⎥ ω ⎥ cos ωT ⎥ ⎥⎦

(2.16)

(2.17)

and T is the sampling period.

3 IMM Estimator Here we consider a typical linear dynamic system, it can be represented as

X (k ) = F (k ) X (k − 1) + W (k )

(3.1)

Z (k ) = H (k ) X (k ) + V (k )

(3.2)

X = [ x, x, x, y, y, y, z , z, z ] is the system state that are and Z ( k ) is the output measurement. F ( k ) is the state transition matrix. H ( k ) is

where X(k) denoted by

the measurement matrix. W ( k ) ∼ N (0, Q ( k )) and V ( k ) ∼ N (0, R ( k )) are the Gaussian noises that are used to descript the system disturbance and the measurement noise, respectively. The IMM algorithm uses model (Markov chain state) probabilities to weigh the inputs and outputs of a bank of parallel Kalman filters (maybe other filters) at each time instant.


533

The main steps of the IMM estimator [3, 9, 10, 11] are as follows: Step 1- Model Interaction or Mixing The mode-conditioned state estimate and the associated covariances from the previous iteration are mixed to obtain the initial condition for the mode-matched filters. The initial condition in cycle k for the Kalman filter matched to the j-th mode is computed using

Xˆ 0j (k −1)

∑

r

i =1

Xˆ i (k −1)μi| j (k −1)

(3.3)

and

Pj0 (k −1) = ∑i =1 μi| j (k −1){Pi (k −1) +[Xˆ i (k −1) − Xˆ 0j (k −1)][Xî (k −1) − Xˆ 0j (k −1)]'} r

(3.4)

where r is the number of model-matched filters used. The state estimates and their covariance matrix at time k-1 conditioned on the i-th model are denoted by Xî (k−1) and Pi (k −1) , respectively; μi| j (k − 1) are the mixing probabilities and can be described as

{

μi| j (k − 1) P m(k − 1) = i m(k ) = j, Z k −1 =

pij μi (k − 1) r

∑ p μ (k −1) l =1

lj

(i, j = 1, 2,

}

r)

(3.5)

l

where m(k) is the index of the model in effect in the interval (k-1, k]. μi ( k ) is the probability that the model i (i=1, 2, 3…r) is in effect in the above interval and can be expressed as

μi (k ) P{m(k ) = i Z k }

(3.6)

The cumulative set of measurements up to and including scan k is denoted by Zk. pij is the model transition probability and is defined as

pij

P{m( k ) = j m(k − 1) = i}

(3.7)

The definitions of m(k-1), μi ( k − 1) and Zk-1 are similar to the definitions of m(k), k μi ( k ) and Z . Step 2- Model-conditioned Filtering 0 According to the outline of the Kalman filtering, the mixed state Xˆ j (k − 1) and the

associated covariance matrix Pj0 ( k − 1) are matched to each model to yield the model-conditioned state estimate Xˆ j ( k ) and its covariance matrix Pj ( k ) at time k . In addition, the likelihood function Λ j (k ) of each model at time k can be computed using

534


Λ j (k )

p{Z (k ) m(k ) = j , Z k −1} = N [υ j (k ); 0, S j ( k )]

(3.8)

where υ j (k ) and S j (k ) are the measurement residual and its covariance.

N [υ j (k );0, S j (k )] denotes the normal pdf with argument υ j (k ) , mean zero and covariance matrix S j (k ) . Step 3- Model Probability Update The model probabilities are updated based on the likelihood function of each model using

μ j ( k ) = Λ j ( k ) ∑ l =1 plj μi ( k − 1) c r

where

(3.9)

c is a normalization constant and can be computed using r

r

c = ∑ ∑ Λ i ( k ) pli μ l ( k − 1)

(3.10)

i =1 l =1

Step 4- Estimate and Covariance Combination The model-conditioned estimates and covariances are combined to find the overall estimate Xˆ (k ) and its covariance matrix P(k) are obtained as follows: r Xˆ ( k ) = ∑ j =1 Xˆ j ( k ) μ j ( k )

P (k ) = ∑ j =1 μ j (k ){Pj (k ) +[Xˆ j (k) − Xˆ (k)][Xˆ j (k) − Xˆ (k)]'}

(3.11)

r

(3.12)

4 Implementation and Simulation 4.1 Description of Simulation Scenarios The simulation scenario is following: (1) A 3D maneuvering target trajectory is considered. The target is located in A (30km, 30km, 30km) at time t=0 s and moves with constant velocity v equal to (300m/s, 300m/s, 300m/s) for time 0~50 s. The target executes a clockwise coordinate turn B with a initial acceleration (10m/s2, -12m/s2, 2 m/s2) during 50~100 s, and such acceleration vector to be given is orthogonal to the velocity vector in order to maintain the orthogonal property ( a ⋅ v = 0 ) in 3DCSCT model. During 200~250s, the target performs an anti-clockwise coordinate turn C with an arbitrary initial acceleration (16 m/s2, 28 m/s2, -14 m/s2). The target moves with constant velocity during 250~300s. (2) The X-Y-Z positions of the target are measured and three measurement standard deviations are all 400m. The process noise is assumed to be a white noise. We denote CV-CA-3DCSCT as an IMM algorithm that includes a CV model, a CA model, and a 3DCSCT model. The same signs are used to CV-Singer-3DCSCT and CV-CSM-3DCSCT, respectively. All those algorithms have the same initial model probabilities μ0 = [ 0.8 0.1 0.1] and the same model switching probabilities:


0 ⎤ ⎡ 0.97 0.03 ⎢ pij = ⎢ 0 0.75 0.25⎥⎥ ⎢⎣ 0.05 0 0.95⎥⎦

535

(4.1)

According to [8] R u is (800(0.92)k + 20) m2 /s4 when the kinematic constraint is used for 3DCSCT model. 4.2 Simulation Results and Analysis When the target perform turns B and C, the RMSE of the three algorithms in X, Y, Z respectively are demonstrated in Fig. 1. Fig.2 is the specific of the RMSE of three algorithms between 200~250s, i.e., Fig. 2 is the magnified version of Fig.1 between 200~250s. It’s clearly shown that when the target is non-maneuvering the performance of three algorithms is almost the same. At the turn B, CV-Singer-3DCSCT and CV-CSM-3DCSCT have almost the same RMSE and are slightly better than CV-CA3DCSCT in tracking accuracy. When the assumption of 3DCSCT model is slightly violated, such as turn C, CV-Singer-3DCSCT and CV-CSM-3DCSCT also have almost the same RMSE, however, they are much better than CV-CA-3DCSCT.

Fig. 1. RMSE of three IMM algorithms for X-Y-Z

Fig. 2. RMSE of three IMM algorithms at turn C

5 Conclusions The benefits of using the CSM and 3DCSCT model in IMM algorithm to track a 3D high maneuvering target have been clearly demonstrated in this paper. When the target perform “high-g” turn in 3D space, this IMM algorithm utilizing CSM is better

536


than other two IMM algorithms, which Singer and CA are included. However, how to choose the parameter in models and filters is an important issue to be addressed in future study.

References 1. Li, X.R., Jilkov, V.P.: Survey of maneuvering target tracking. In: Part V: multiple-models. SPIE, vol. 4048, pp. 212–236 (2000) 2. Blom, H.A., Bar-Shalom, Y.: The interacting multiple model algorithm for systems with markovian switching coefficient. IEEE Transactions on Automatic Control 33(8), 780–783 (1988) 3. Watson, G.A., Blair, W.D.: IMM algorithm for tracking targets that maneuver through coordinated turn. In: SPIE, vol. 1698, pp. 236–247 (1992) 4. Nabaa, N., Bishop, R.H.: Validation and comparison of coordinated turn aircraft maneuver models. IEEE Transactions on Aerospace and Electronic Systems 36(1), 250–259 5. Singer, R.A.: Estimating optimal tracking filter performance for manned maneuvering targets. IEEE Transactions on Aerospace and Electronic Systems 6(4), 473–483 (1970) 6. Zhou, H.R., Jin, Z.L., Wang, P.D.: Maneuvering target tracking, pp. 135–145. National Defence Industry Press, Beijing (1991) 7. Tahk, M., Speyer, J.L.: Target tracking problems subject to kinematic constraints. IEEE Transactions on Automatic Control 35(3), 324–326 (1990) 8. Alouani, A.T., Blair, W.D.: Use of a kinematic constraint in tracking constant speed, maneuvering targets. IEEE Transactions on Automatic Control 38(7), 1107–1111 (1993) 9. Bar-Shalom, Y., Li, X.R., Kirubarajan, T.: Estimation with applications to tracking and navigation: theory, algorithms, and software, pp. 453–457. Wiley, New York (2001) 10. Li, X.R., Jilkov, V.P.: Survey of maneuvering target tracking. Part V: multiple-model methods. IEEE Transactions on Aerospace and Electronic Systems 41(4), 1255–1321 (2005) 11. Kadirkamanathan, V., Li, P., Kirubarajan, T.: Sequential Monte Carlo filtering vs. the IMM estimator for fault detection and isolation in nonlinear systems. In: SPIE, vol. 4389, pp. 263–274 (2001)

A New Method Based on Ant Colony Optimization for the Probability Hypothesis Density Filter* Jihong Zhu1, Benlian Xu2, Fei Wang2, and Qiquan Wang1 1

School of Automation, NanJing University of Science & Technology, NanJing, 210094, China 2 School of Electric and Automatic Engineering, ChangShu Institute of Technology, ChangShu, 215500, China {djyzhjh,xu_benlian,wangleea,wangzqwhz}@yahoo.com.cn

Abstract. A new approximating estimate method based on ant colony optimization algorithm for probability hypothesis density (PHD) filter is investigated and applied to estimate the time-varying number of targets and their states in clutter environment. Four key process phases are included: generation of candidates, initiation, extremum search and state extraction. Numerical simulations show the performance of the proposed method is closed to the sequence Monte Carlo PHD method. Keywords: Multi-target tracking, Probability hypothesis density, Ant colony optimization, extremum search.

1 Introduction Multi-target tracking (MTT) is regarded as a classic but an intractable problem in a wide variety of contexts. According to recent literature [1-5], data association (DA) problems form the main stream in MTT. But due to its combinatorial nature, the DA problem makes up the bulk of the computational load in MTT filed. The random finite sets (RFS) which avoids explicit associations between measurements and tracks becomes an alternative formulation in recent decade. Especially, the probability hypothesis density (PHD) [6], a novel RFS-based filter, and its implementations have generated substantial interest. The PHD filter operates on the single-target state space and avoids the combinatorial problem that arises from DA problem. This salient feature renders the PHD filter extremely attractive. However, the PHD recursion involves multiple integrals that have no closed form solutions in general. Fortunately, two methods have been successfully developed for approximating the PHD filter so that it can be implemented [7-8], i.e., the sequence Monte Carlo PHD method (SMCPHD) [7] and the Gaussian mixture PHD method (GMPHD) [8]. Hundreds of papers based on methods in [7-8] are proposed in recent decade. But most of them are applied directly with two methods in different fields or modified with traditional DA algorithm. *

This work is supported by national natural science foundation of China (No.60804068) and by national science foundation of Jiangsu province (No.BK2010261) and by cooperation innovation of industry, education and academy of Jiangsu province (No.BY2010126).


538

J. Zhu et al.

So far, there are few reports on the ant-based applications to the parameter estimate field or multi-target tracking except [9-11]. In this work, a novel approximating method based on ant colony optimization (ACO) for PHD filter is proposed. The remainder of this paper is organized as follows. Section 2 presents the background on the PHD filter. Section 3 describes the principle of the proposed method for PHD filter. Numerical simulations are conducted and corresponding results are analyzed in Section 4. Finally, conclusions are drawn in Section 5.

2 The Probability Hypothesis Density Filter The more details for PHD filter are introduced in [6]. Here, only main formulas are given. For a RFS X on χ with probability distribution P , its first-order moment or intensity is a function v : χ → [ 0, ∞ ) such that for each region S ⊆ χ

∫ v( x)dx = ∫ | X ∩ S | P(dX ) S

(1)

where | X | denotes the cardinality of a set X . In other words, the integral of v over any region S gives the expected number of elements of X that are in S . This intensity is commonly known in the tracking literature as PHD. Let vk and vk |k −1 denote the respective intensities associated with the multi-target posterior density pk and the multi-target predicted density pk |k −1 . So the PHD filter can be shown that the posterior intensity can be propagated in time via the PHD recursion (2) and (3). vk |k −1 ( x) = ∫ ps , k (ς ) f k |k −1 ( x | ς )vk −1 (ς )d ς

+ ∫ β k |k −1 ( x | ς )vk −1 (ς )d ς + γ k ( x)

(2)

vk ( x) = ⎡⎣1 − pD , k ( x) ⎤⎦ vk | k −1 ( x) +

∑κ

Z ∈Z K

pD , k ( x) g k ( z | x)vk | k −1 ( x ) k

(3)

( z ) + ∫ pD , k (ξ ) g k ( z | ξ )vk |k −1 (ξ )d ξ

where γ k (i) denotes the intensity of the birth RFS Γ k at time k , β k |k −1 (i| ς ) denotes the intensity of the RFS Bk |k −1 (ς ) spawned at time k by a target with previous state

ς , ps , k (ς ) denotes the probability that a target still exists at time k given that its previous state is ς , f k |k −1 (i| ς ) denotes the transition probability density of individual targets, pD , k ( x) denotes the probability of detection given a state x at time k , g k ( z |i) denotes the likelihood of individual targets and γ k (i) denotes the intensity of

the clutter RFS Κ k at time k .

A New Method Based on Ant Colony Optimization for the PHD Filter

539

3 Approximating Method Based on ACO for PHD Filter As mentioned in (1), the expected number of elements of X , Nˆ = ∫ v( x)dx can be

used as an estimate for the number of targets. The local maximum of the intensity are points in χ with the highest local concentration of expected number of targets and hence can be used to generate estimates for the elements of X . So the idea that a method may extract the peaks of intensity function directly has been generated, i.e., the solution problem is transfer to find all extreme points of intensity function. Obviously, the Ant colony optimization (ACO) [12] provides an alternative to solve this task, due to its successful application on many combinatorial optimization problems and continuous-space optimization problems. The approximating method based on ACO for PHD filter, which includes four phases, i.e., generation of candidates, initiation, extremum search and state extraction, is depended on [13], but made major changes with that in their work. In the first phase, the state of candidates is generated as same as that in the particle filter [7]. Without loss of generality, let xt(i−)1 denotes the state of candidate i at time t − 1 , which is represented by the position ( xt(−i )1 , yt(−i )1 ) and the velocity ( xt(−i )1 , y t(−i )1 ) as xt(i−)1 = [ xt(−i )1 , yt(−i )1 , xt(−i )1 , yt(−i )1 ]T . In the second phase, the value of intensity function and some parameters should be initiated. The number of ants N ant is fixed with the number of candidates, and the

pheromone of candidate i is set to be τ ( i ) = 1 .Given the importance density pk (i| xk −1 ) , rk (i| xk −1 ) and qk (i| xk −1 , Z k ) , the (2) can be transformed by

vk |k −1 ( x) = ∫ ps , k (ς ) +∫

f k | k −1 ( x | ς ) pk ( x | xk −1 )

β k | k −1 ( x | ς ) rk ( x | xk −1 )

pk ( x | xk −1 )vk −1 (ς )dς

)rk ( x | xk −1 )vk −1 (ς )dς +

γ k ( x) qk ( x | xk −1 )

(4) qk ( x | xk −1 )

The local maximum of the intensity are points with the highest local concentration of expected number of targets, in other words, the true target always distributes around these local maximum points. The value of intensity function of each candidate is computed by formula (5). vk |k −1 ( x) ≈ pk ( x | xk −1 )rk ( x | xk −1 ) + qk ( x | xk −1 )

(5)

In the third phase, the extremum search process is executed. Suppose the value of candidate i is denoted by vk( i|k) −1 , the value of its neighbors is denoted by [vk(i|k−−1)1 , vk(i|k+−1)1 ] . If ant am locates on the candidate i , it will move to this left neighbor or right neighbor, four moving behaviors are designed as follows: z

If vk( i|k−1)−1 < vk( i|k) −1 and vk( i|k) −1 < vk( i|k+−1)1 holds, ant am will move to candidate i + 1 .

z

If vk( i|k−1)−1 > vk(i|k) −1 and vk( i|k) −1 > vk(i|k+−1)1 holds, ant am will move to candidate i − 1 .

540

z

J. Zhu et al.

If vk( i|k−1)−1 ≤ vk( i|k) −1 and vk( i|k) −1 ≥ vk( i|k+−1)1 holds, ant am will select candidate i + 1 or i − 1 with given probability threshold P0 .

z

If vk( i|k−1)−1 > vk(i|k) −1 and vk( i|k) −1 < vk( i|k+−1)1 holds, ant am will select candidate i + 1 or i − 1 with probability P , which is given by −ηij / C1

τ i( m ) e ∑ τ in( m) e−η

Pij ( m ) =

in

(6)

/ C1

n∈[ i −1, i +1]

where

C1

i.e., ηij = v

(i ) k | k −1

is −v

a

( j) k | k −1

given

positive

constant,

the ηij

is

heuristic

value,

, j ∈ [i − 1, i + 1] , which implied that the ant will move to its

neighbor which the value of function is bigger than that of it. When all ants finish their tour, the pheromone update process is executed. Suppose ant m move on candidate i , the ant will release pheromone on candidate i , and the pheromone amount is denoted by Δτ im (t ) Δτ im (t ) = C2 (vk( m|k )−1 − vk(i|k) −1 )

(7)

where C2 is a given positive constant. If the number of ants has moved to the candidate i is l at iteration t , the pheromone on candidate i is updated as following formula l

τ i (t ) = (1 − ρ )τ i (t ) + ∑ Δτ im (t )

(8)

m =1

Meanwhile, all ants will stay on the points with local maximum intensity function value. But no all of these points are originated from true targets, so in the final phase, the state extraction of targets is executed depending on the measurement at each time step. Given the importance density g k′ ( z | x) , the formula (3) can be defined by vk ( x) ≈ ⎣⎡1 − pD , k ( x) ⎦⎤ vk |k −1 ( x) +

∑κ

Z ∈Z K

pD , k ( x) g k′ ( z | x)vk |k −1 ( x) k

(9)

( z ) + C3 pD ,k ( x) g k′ ( z | x)vk |k −1 ( x)

where C3 is a given positive parameter. Each candidate where ants stay on will be computed the value based on formula (9), if vk ( xi ) of candidate i is smaller than a given parameter ε , all ants staying on candidate i will die and the candidates with surviving ants will be regarded the state originated from true target. And these candidates will be utilized as the beginning process at next time step.

4 Numerical Simulations For illustration purposes, two dimensional scenario with an unknown and time varying number of targets observed in clutter over the surveillance region

A New Method Based on Ant Colony Optimization for the PHD Filter

541

[1km,3km] × [14km,16km] are considered, the Poisson birth RFS Γ k with intensity

γ k ( x ) = 0.1N ( x, mr , Pr ) , mr = [2000,50,14816, −50]T , Pr = diag ([100,10,100,10]T ) and other parameters are set to be as same as in [10]. The importance density used are pk = f k |k −1 , qk = N (i, x , Q) and g k′ = g k ( z | x) . Additionally, the ant-based parameters are set to be as follows: N iteration = 500 , C1 = 0.5 , C2 = 100 , C3 = 1.0 , ρ = 0.2 , ε = 1e − 20 . 4

1.6

3000 True tracks Measurements

1.58

2000

1.56

1500

1.54

5

10

15

20

25

30

35

40

45

50

time step 4

1.6

x 10

1.5 1.48

1.55

1.46

1.5

1.44

y (m)

True tracks Measurements

1.52

1000

y (m)

x (m)

2500

x 10

1.45

1.42

1.4

5

10

15

20

25

30

35

40

45

1.4 1000

50

1200

1400

1600

1800

time step

2000

2200

2400

2600

2800

3000

x (m)

3000

3000

2500

2500

x(m)

x(m)

Fig. 1. True target tracks and measurements

2000 Ant PHD estimates True tracks

1500 1000

5

10

15

20

25

30

35

40

45

2000 Particle PHD estimates True tracks

1500 1000

50

5

10

15

20

time 1.6

35

40

45

50

1.6

30

35

40

45

50

x 10

1.55

y(m)

y(m)

30

4

1.55 1.5 1.45 1.4

25

time

4

x 10

1.5 1.45

5

10

15

20

25

30

35

40

45

1.4

50

5

10

15

20

time

25

time

Fig. 2. Position estimates of proposed method and SMCPHD method 3 Target number The estimated target number

3

No. of targets

No. of targets

4

2 1 0

5

10

15

20

25

30

35

40

45

1

0

50

Target number The estimated target number

2

5

10

15

20

1.69

10

1.6

10

5

10

15

20

25

30

time step

25

30

35

40

45

50

35

40

45

50

time step

35

40

45

50

OSPA distance (in m)

OSPA distance (in m)

time step 1.6

10

1.1

10

5

10

15

20

25

30

time step

Fig. 3. Target number estimate and OSPA distance of proposed method and SMCPHD

542

J. Zhu et al.

Figure 1 shows the true target tracks in clutter environment. Figure 2 shows that the position estimates based on the proposed method and SMCPHD (500 particles). Figure 3 shows target number estimate and OSPA distance of proposed method and SMCPHD. From Figure 2 and 3, it can be observed that the performance of proposed method is closed to the SMCPHD in such kind of scenario. But our method is simpler than SMCPHD method, due to the approximating represent.

5 Conclusions A new approximating estimate method based on ACO algorithm for PHD filter is proposed. The main idea is composed of four process phases, and the key idea of the proposed method is that the extremum search method based on ACO deals with the approximating recursive function. Simulations show that the proposed method can be closed to the SMCPHD according to the OSPA distance metric. And the proposed method is simpler than SMCPHD. Future work will focus on the estimate accuracy of the proposed method and extend the method into maneuvering targets tracking cases.

References 1. Lee, M.S., Kim, Y.H.: New Data Association Method for Automotive Radar Tracking. IEE Proc.-Radar Sonar Navig. 148(5), 297–301 (2001) 2. Li, X.R., Bar-Shalom, Y.: Tracking in Clutter with Nearest Neighbor Filters: Analysis and Performance. IEEE Trans. On Aerospace and Electronic Systems 32(3), 995–1010 (1996) 3. Li, X.R.: Tracking in Clutter with Strongest Neighbor Measurements –Part I: Theoretical Analysis. IEEE Trans. On Automatic Control 43(11), 1560–1578 (1998) 4. Fortmann, T., Bar-Shalom, Y., Scheffe, M.: Sonar Tracking of Multiple Targets Using Joint Probabilistic Data Association. IEEE Journal of Oceanic Engineering, OE 8, 173–183 (1983) 5. Blackman, S.S.: Multiple Hypothesis Tracking for Multiple Target Tracking. IEEE A&E Systems Magazine 19(1), 5–18 (2004) 6. Mahler, R.: Multi-target Bayes Filtering via First-order Multi-target Moments. IEEE Trans. AES 39(4), 1152–1178 (2003) 7. Vo, B., Singh, S., Doucet, A.: Sequential Monte Carlo Implementation of the PHD Filter for Multi-target Tracking. In: Proc. Int’l Conf. on Information Fusion, Cairns, Australia, pp. 792–799 (2003) 8. Vo, B., Ma, W.K.: The Gaussian Mixture Probability Hypothesis Density Filter. IEEE Trans. Signal Processing 54(11), 4091–4104 (2006) 9. Nolle, L.: On a Novel ACO-Estimator and its Application to the Target Motion Analysis problem. Knowledge-Based Systems 21(3), 225–231 (2008) 10. Xu, B.L., Vo, B.: Ant Clustering PHD Filter for Multiple Target Tracking. Applied Soft Computing 11(1), 1074–1086 (2011) 11. Xu, B.L., Chen, Q.L., Zhu, J.H., Wang, Z.Q.: Ant Estimator with Application to Target Tracking. Signal Processing 90(5), 1496–1509 (2010) 12. Dorigo, M., Maniezzo, V., Colorni, A.: Positive Feedback as a Search Strategy. Technical Report 91-016, Dipartimento di Elettronica, Politecnico di MILANO, Milan, Italy (1991) 13. Pang, C.Y., Li, X.: Applying Ant Colony Optimization to Search All Extreme Points of Function. In: 5th IEEE Conf. on industrial Electronics and Applications, pp. 1517–1521 (2009)

A Hybrid Algorithm Based on Fish School Search and Particle Swarm Optimization for Dynamic Problems George M. Cavalcanti-J´ unior, Carmelo J.A. Bastos-Filho, Fernando B. Lima-Neto, and Rodrigo M.C.S. Castro Polytechnic School of Pernambuco, University of Pernambuco, Recife, Brazil {gmcj,cjabf,fbln,rmcsc}@ecomp.poli.br

Abstract. Swarm Intelligence algorithms have been extensively applied to solve optimization problems. However, some of them, such as Particle Swarm Optimization, may not present the ability to generate diversity after environmental changes. In this paper we propose a hybrid algorithm to overcome this problem by applying a very interesting feature of the Fish School Search algorithm to the Particle Swarm Optimization algorithm, the collective volitive operator. We demonstrated that our proposal presents a better performance when compared to the FSS algorithm and some PSO variations in dynamic environments.

1

Introduction

The optima solutions for many real-world problems may vary over the time. For example, the optimal routes for a computer network can change dynamically due to nodes failures or due to unavailable links. Therefore, optimization algorithms to solve real-world problems should present the capability to deal with dynamic environments, in which the optima solutions can change along the time. Many bio-inspired optimization algorithms have been proposed in the last two decades. Among them, there are the swarm intelligence algorithms, which were conceived based on some collective behaviors. In general, swarm algorithms are inspired in groups of animals, such as flocks of birds, schools of fish, hives of bees, colonies of ants, etc. Although a lot of swarm-based algorithms were already proposed, just some few were designed to tackle dynamic problems. One of the most used swarm intelligence algorithms is the Particle Swarm Optimization (PSO). Despite the fast convergence capability, the vanilla version of the PSO can not tackle dynamic optimization problems. It occurs because the entire swarm often increases the explotation around a good region of the search space, reducing the overall diversity of the population. However, some variations of the PSO have been created in order to increase the capacity to escape from regions in the search space where the optimum is not located anymore [1,2,3]. On the other hand, another swarm intelligence algorithm proposed in 2008, the Fish School Search algorithm (FSS) [4,5,6], presents a very interesting feature that can be very useful for dynamic environments. FSS presents an operator, called Y. Tan et al. (Eds.): ICSI 2011, Part II, LNCS 6729, pp. 543–552, 2011. c Springer-Verlag Berlin Heidelberg 2011

544

G.M. Cavalcanti-J´ unior et al.

volitive operator, which is capable to auto-regulate the exploration-exploitation trade-off during the algorithm execution. Since the PSO algorithm converges faster than FSS but can not auto-adapt the granularity of the search, we believe the FSS volitive operator can be applied to the PSO in order to mitigate this PSO weakness and improve the performance of the PSO for dynamic optimization problems. Based on this, we propose in this paper a hybrid algorithm, called Volitive PSO. This paper is organized as follows. Section 2 provides the background on PSO and FSS, also including a brief explanation of a well known PSO variation to tackle dynamic problems, called Charged PSO. Section 3 describes our proposal, which is a FSS-PSO hybrid algorithm. Section 4 presents the simulation setup. Section 5 is divided in two sub-sections and depicts some results. The former presents a parametrical analysis of our proposal and the latter shows a comparison between our proposal and some other approaches. In Section 6 we give our conclusions and we present some ideas for future works.

2 2.1

Background PSO (Particle Swarm Optimization)

Particle Swarm Optimization is a population-based optimization algorithm inspired by the behavior of flocks of birds. It was firstly introduced by Kennedy and Eberhart [7] and it has been largely applied to solve optimization problems. The standard approach is composed by a swarm of particles, where each one → has a position within the search space − xi and each position represents a solution for the problem. The particles fly through the search space of the problem → searching for the best solution, according to the current velocity − vi , the best −−−→ position found by the particle itself (Pbesti ) and the best position found by the −−−→ entire swarm during the search so far (Gbest ). According to the approach proposed by Shi and Eberhart [8] (this approach is also called inertia PSO), the velocity of a particle i is evaluated at each iteration of the algorithm by using the following equation: −−−→ → −−−→ → − → → vi (t + 1) = w− vi (t) + r1 c1 [Pbesti − − xi (t)] + r2 c2 [Gbesti − − xi (t)],

(1)

where r1 and r2 are numbers randomly generated in the interval [0, 1]. The inertia weight (w) controls the influence of the previous velocity and balances the exploration-exploitation behavior along the process. It generally decreases from 0.9 to 0.4 during the algorithm execution. c1 and c2 are called cognitive and social acceleration constants, respectively, and weights the influence of the memory of the particle and the information acquired from the neighborhood. The position of each particle is updated based on the velocity of the particle, according to the following equation: − → → → xi (t + 1) = − xi (t) + − vi (t + 1).

(2)

A Hybrid Algorithm Based on FSS and PSO for Dynamic Problems

545

The communication topology defines the neighborhood of the particles and, as a consequence, the flow of information through the particles. There are two basic topologies: global and local. In the former, each particle shares and acquires information directly from all other particles, i.e. all particles use the same social memory, called Gbest . In the local topology, each particle only share information with two neighbors and the social memory is not the same within the whole swarm. This approach, called Lbest , helps to avoid a premature attraction of all particles to a single spot point in the search space. 2.2

Charged PSO

Since the standard PSO can not tackle dynamic problems due to the the low capacity to increase the diversity after the entire swarm has converged to a single region of the search space, many efforts to overcome this weakness have been made. The simplest idea is to restart the particles every time the search space changes. However, all the previous information obtained from the problem during the search process is lost in this case. An interesting approach introduced by Blackwell and Bentley [1] is the Charged PSO, which uses the idea of electrostatic charges. Some particles are charged (they repeal themselves) and some others are neutral. In general, the neutral particles tend to exploit towards a single sub-region of the search space, whereas the charged particles never converges to a unique spot. Nevertheless, the charged particles are constantly exploring in order to maintain diversity. In order to consider the effect of the charged particles, the velocity equation receives a fourth term, as shown in the equation (3). This term is defined as the → acceleration of the particle i (− a i ) an can be seen in equation (4). −−−→ → −−−→ → − → → → vi (t + 1) = w− vi (t) + r1 c1 [Pbesti − − xi (t)] + r2 c2 [Gbest − − xi (t)] + − a i (t). − → a i (t) =

Qi Qj − → r ij (t), − i=j → r ij (t)3

0,

→ if Rc ≤ − r ij (t) ≤ Rp , otherwise ,

(3)

(4)

→ → → where − r ij (t) = − x i (t) − − x j (t), Qi is the charge magnitude of the particle i, Rc is the core radius and Rp is the perception limit of the particle. Neutral particles have charge value equal to zero, i.e. Qi = 0. 2.3

FSS (Fish School Search)

The Fish School Search (FSS) is an optimization algorithm based on the gregarious behavior of oceanic fish. It was firstly proposed by Bastos-Filho et al in 2008 [4]. In the FSS, each fish represents a solution for the problem. The success of a fish during the search process is indicated by its weight. The FSS has four operators, which are executed for each fish of the school at each iteration: (i) individual movement, which is responsible for local search stepind ; (ii) feeding, which updates the fish weights indicating the degree of success or failure during

546


the search process so far; (iii) collective-instinctive movement, which makes all fish moves toward a resultant direction; and (iv) collective-volitive movement, which controls the granularity of the search. In this paper, as we are dealing with dynamic environments, only the feeding and collective-volitive movement operators are used to build the proposed hybrid algorithm. Feeding operator The feeding operator determines the variation of the fish weight at each iteration. One should notice that a fish can increase or decrease its weight depending, respectively, on the success or failure during the search process. The weight of the fish is evaluated according to the following equation: Wi (t + 1) = Wi (t) +

Δfi , max(|Δf |)

(5)

where Wi (t) is the weight of the fish i, Δfi is the variation of the fitness function between the new position and the current position of the fish, max(|Δf |) is the absolute value of the greatest fitness variation among all fish. There is a parameter wscale that limits the maximum weight of the fish. The weight of each fish can vary between 1 and wscale and has an initial value equal to wscale . 2 Collective-volitive movement operator This operator controls the granularity of the search executed by the fish school. When the whole school is achieving better results, the operator approximates the fish aiming to accelerate the convergence toward a good region. On the contrary, the operator spreads the fish away from the barycenter of the school and the fish have more chances to escape from a local minimum. The fish school expansion or contraction is applied as a small drift to every fish position regarding the school barycenter, which can be evaluated as shown below: N − → → − x i (t)Wi (t) B (t) = i=1 . (6) N − → i=1 x i (t) We use equation (7) to perform the fish school expansion (use sign +) or contraction (use sign −). → − → − x i (t) − B (t) − → → x i (t + 1) = − x i (t) ± stepvol r1 , − → → d(− x i (t), B (t))

(7)

→ − → where r1 is a number randomly generated in the interval [0, 1]. d(− xi , B ) evaluates the euclidean distance between the particle i and the barycenter. stepvol is called volitive step and controls the step size of the fish. The stepvol is bounded by two parameters (stepvol min and stepvol max ) and decreases linearly from stepvol max to stepvol min along the algorithm iterations. It helps the algorithm to initialize with an exploration behavior and change dynamically to an exploitation behavior.


3

547

Volitive PSO

This section presents the proposed algorithm, called Volitive PSO, which is a hybridization of the FSS and the PSO algorithms. Our proposal is to include two FSS operators in the Inertia PSO, the feeding and the collective-volitive movement. In the Volitive PSO, each particle becomes a weighted particle, where the weight is used to indicate the collective-volitive movement, resulting in expansion or contraction of the school. In our proposal, the stepvol does not decrease linearly, it decreases according to equation (8). The parameter volitive step decay percentage (decayvol ) must be in the interval [0, 100]. stepvol (t + 1) = stepvol (t)

100 − decayvol . 100

(8)

The stepvol is reinitialized to stepvol max when a change in the environment is detected. We use a sentry particle [9] to detect these changes. The fitness of the sentry particle is evaluated in the end of each iteration and in the beginning of the next iteration. The Algorithm 1.1 shows the Volitive PSO pseudocode. Algorithm 1.1: Volitive PSO pseudocode Initialize parameters and particles; while the stop condition is not reached do foreach particle of the swarm do Evaluate the fitness of the particle; − −− → −−−→ Evaluate Pbest and Lbest ; end if an environment change is detected then Initialize stepvol ; end foreach particle of the swarm do Update the velocity and the position of the particle; Evaluate the fitness of the particle; end Execute feeding operator; Execute collective-volitive movement operator; foreach particle of the swarm do − −− → −−−→ Evaluate Pbest and Lbest ; end Update stepvol and w; end

4

Simulation Setup

In this section we present the benchmark function, the metric to measure the quality of the algorithms and the values for the parameters used in the simulations.

548

4.1


Benchmark Function

We used the DF1 benchmark function proposed by Morrison and Jong [10] in our simulations. DF1 is composed by a set of random peaks with different heights and slopes. The number of peaks, their heights, slopes, and positions within the search space are adjustable. The function for a N -dimensional space is defined according to the equation (9). → → → f (− x ) = maxi=1,2,...,P [Hi − Si (− x −− xi )2 ], (9) → where P is the number of peaks (peak i is centered in the position − xi ), Hi is the peak height and Si is the peak slope. The values for xid , Hi and Si are bounded. The dynamic components of the environment are updated using discrete steps. The DF1 uses a logistic function to control the generation of different step sizes. The parameter used to calculate the steps is adjusted according to the equation (10). ei = rei−1 [1 − ei−1 ], (10) where r is a constant in the interval [1,4]. As r increases, more simultaneous results for e are achieved. As r gets closer to 4, the behavior becomes chaotic. The dynamics of the environment is specified using the following parameters: Npeaks is the number of peaks in motion; rh is the r value for height dynamics; rs is the r value for slope dynamics; rxd is the r value for position dynamics in dimension d; It is necessary to have a scaling factor for each r value. 4.2

Performance Metric

The mean fitness metric was introduced by Morrison [11]. He argued that a representative performance metric to measure the quality of an algorithm in a dynamic environment should reflect the performance of the algorithm across the entire range of environment dynamics. The mean fitness is the average over all previous fitness values, as defined below: T Fmean (T ) =

t=1

Fbest (t) , T

(11)

where T is the total number of iterations and Fbest is the fitness of the best particle after iteration t. The advantage of the mean fitness is that it represents the entire algorithm performance history. We also used the collective mean fitness [11], that is simply the average value of the mean fitness at the last iteration over a predefined number of trials. 4.3

Parameters Settings

All results presented in this paper are the average values after 30 trials. We used 10,000 iterations for all algorithms. We performed the experiments in two situations: (i) 10 dimensions and 10 peaks and (ii) 30 dimensions and 30 peaks.


549

In this paper, only the peak positions are varied along the iterations. The heights and slopes of the peaks were initialized randomly within the predefined interval. The parameters used for the DF1 function are Hbase = 40, Hrange = 20, Hscale = 0.5, rh = 3.2, Sbase = 1, Srange = 7, Sscale = 0.5, rs = 1.2, xbase id = −10, xrange id = 20, xscale id = 0.7, rxd = 3.2. For all PSO algorithms, we used 50 particles, local topology, c1 and c2 equal to 1.494 [12] and w decreasing linearly from 0.9 to 0.4 along 100 iterations. We set up w = 0.9 every time an environment change is detected. We chose the local topology since it helps to avoid premature convergence to a local optimum, which is good for optimization in dynamic environments. The Charged PSO was tested empirically with 30%, 50% and 70% of charged particles, and for Q = 4, Q = 8, Q = 12 and Q = 16. In both scenarios, the best results were achieved for 30% of charged particles and Q = 12. Hence, these values were used. For the FSS, we used 50 fish, Wscale = 500, initial and final individual step equal to 2% and 0.01%, and initial and final volitive step equal to 40% and 0.1%. stepind and stepvol decreases linearly along 100 iterations and are reinitialized when a change in environment occurs. For the Volitive PSO, we used wscale = 500, and stepvol min = 0.01%.

5 5.1

Results Analysis of the Parameters

This section presents an analysis of the influence of the parameters decayvol and stepvol max in the performance of the Volitive PSO. As preliminary results showed that the algorithm is more sensible to the decayvol parameter and high values for decayvol do not present good performance, we tested the following decayvol values: 0%, 10% and 25%. For each decayvol value, we varied the stepvol max value and the box plots of the mean fitness at the last iteration are shown in the Figure 1. For the case 1 (10 dimensions and 10 peaks), the average mean fitness for different stepvol max are not so different (as shown in Figures 1(a), 1(c) and 1(e)). However, slightly better results can be observed for decayvol = 10%. Nevertheless, for the case 2 (30 dimensions and 30 peaks), the best results were achieved for decayvol equal to 0%. It indicates that is better to not diminish the stepvol for spaces with higher dimensionality. The best results for the case 2 were achieved when stepvol max = 40% and decayvol = 0%. Hence, we used these values for the comparison presented in the next sub-section. 5.2

Comparison with Other Approaches

In this section we present a brief performance comparison among the Volitive PSO, Inertia PSO, Restart PSO (simply reinitialize the particles when a change in the environment is detected), Charged PSO and FSS. Figure 2 depicts the average values of fitness for each algorithm. As can be seen, the Volitive PSO achieved better results in average than the other algorithms in both cases.

550


(a) decayvol = 0%, 10d and 10 peaks.

(b) decayvol = 0%, 30d and 30 peaks.

(c) decayvol = 10%, 10d and 10 peaks.

(d) decayvol = 10%, 30d and 30 peaks.

(e) decayvol = 25%, 10d and 10 peaks.

(f) decayvol = 25%, 30d and 30 peaks.

Fig. 1. Analysis of the parameters decayvol and stepvol max of the Volitive PSO algorithm


(a) 10 dimensions and 10 peaks.

551

(b) 30 dimensions and 30 peaks.

Fig. 2. Comparative evolution of the algorithms on the DF1 function Table 1. Collective Mean Fitness - Average (standard deviation) after 10, 000 iterations (a) 10 dimensions and 10 peaks. (b) 30 dimensions and 30 peaks. PSO Restart PSO Charged PSO FSS Volitive PSO

39.207 46.528 42.249 31.032 47.168

(6.533) (5.590) (4.542) (9.742) (4.517)

PSO Restart PSO Charged PSO FSS Volitive PSO

24.827 32.493 22.039 20.192 41.854

(5.486) (6.088) (6.965) (7.340) (4.521)

Table 1 shows the collective mean fitness (and standard deviation in parenthesis) after 10, 000 iterations. One can observe that the Volitive PSO also achieved lower standard deviation in both cases.

6

Conclusion

In this paper we proposed a hybrid FSS-PSO algorithm for dynamic optimization. We showed that the collective-volitive movement operator applied to the PSO can help to maintain diversity when the search space is varying over the time, without reducing the exploitation capability. Some preliminary results showed that the volitive step must not decay quickly. It indicates the important hole of the FSS-operator to generate diversity after environmental changes. Further research includes a deeper analysis of the Volitive PSO and more tests varying the peaks height and slopes. Also, we intend to analyze the dynamics of the swarm within the search space.

Ackonowledgments The authors acknowledge the financial support from CAPES, CNPq and University of Pernambuco for scholarships, support and travel grants.

552


References 1. Blackwell, T.M., Bentley, P.J.: Dynamic Search with Charged Swarms. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 19–26 (2002) 2. Rakitianskaia, A., Engelbrecht, A.P.: Cooperative charged particle swarm optimiser. In: Congress on Evolutionary Computation, CEC 2008, pp. 933–939 (June 2008) 3. Nickabadi, A., Ebadzadeh, M.M., Safabakhsh, R.: Evaluating the performance of DNPSO in dynamic environments. In: IEEE International Conference on Systems, Man and Cybernetics, pp. 2640–2645 (October 2008) 4. Bastos-Filho, C.J.A., Neto, F.B.L., Lins, A.J.C.C., Nascimento, A.I.S., Lima, M.P.: A novel search algorithm based on fish school behavior. In: IEEE International Conference on Systems, Man and Cybernetics, pp. 2646–2651. IEEE, Los Alamitos (October 2009) 5. Bastos-Filho, C.J.A., Neto, F.B.L., Sousa, M.F.C., Pontes, M.R.: On the Influence of the Swimming Operators in the Fish School Search Algorithm. In: SMC, pp. 5012–5017 (October 2009) 6. Bastos-Filho, C.J.A., de Lima Neto, F.B., Lins, A.J.C.C., Nascimento, A.I.S., Lima, M.P.: Fish school search. In: Chiong, R. (ed.) Nature-Inspired Algorithms for Optimisation. SCI, vol. 193, pp. 261–277. Springer, Heidelberg (2009) 7. Kennedy, J., Eberhart, R.C.: Particle swarm optimization. In: Proceedings of IEEE international conference on neural networks, vol. 4, pp. 1942–1948 (1995) 8. Shi, Y., Eberhart, R.: A modified particle swarm optimizer. In: The 1998 IEEE International Conference on Evolutionary Computation Proceedings, IEEE World Congress on Computational Intelligence, pp. 69–73 (1998) 9. Carlisle, A., Dozier, G.: Applying the particle swarm optimizer to non-stationary environments. Phd thesis, Auburn University, Auburn, AL (2002) 10. Morrison, R.W., Jong, K.A.D.: A test problem generator for non-stationary environments. In: Proc. of the 1999 Congr. on Evol. Comput., pp. 2047–2053 (1999) 11. Morrison, R.W.: Performance Measurement in Dynamic Environments. In: GECCO Workshop on Evolutionary Algorithms for Dynamic Optimization Problems, pp. 5–8 (2003) 12. Eberhart, R.C., Shi, Y.: Particle Swarm Optimization: Developments, Applications and Resources. In: Proceedings of the IEEE Congress on Evolutionary Computation, CEC 2001 (2001)

Feeding the Fish – Weight Update Strategies for the Fish School Search Algorithm Andreas Janecek and Ying Tan Key Laboratory of Machine Perception (MOE), Peking University Department of Machine Intelligence, School of Electronics Engineering and Computer Science, Peking University, Beijing, 100871, China [email protected], [email protected]

Abstract. Choosing optimal parameter settings and update strategies is a key issue for almost all population based optimization algorithms based on swarm intelligence. For state-of-the-art optimization algorithms the optimal parameter settings and update strategies for different problem sizes are well known. In this paper we investigate and compare different newly developed weight update strategies for the recently developed Fish School Search (FSS) algorithm. For this algorithm the optimal update strategies have not been investigated so far. We introduce a new dilation multiplier as well as different weight update steps where fish in poor regions loose weight more quickly than fish in regions with a lot of food. Moreover, we show how a simple non-linear decrease of the individual and volitive step parameters is able to significantly speed up the convergence of FSS.

1

Introduction

The Fish School Search (FSS) algorithm [1, 2, 3] is a recently developed swarm intelligence algorithm based on the social behavior of schools of fish. By living in swarms, the fish improve survivability of the whole group due to mutual protection against enemies. Moreover, the fish perform collective tasks in order to achieve synergy (e.g. finding locations with lots of food). Comparable to real fish that swim in the aquarium in order to find food, the artificial fish search (swim) the search space (aquarium) for the best candidate solutions (locations with most food ). The location of each fish represents a possible solution to the problem – comparable to locations of particles in Particle Swarm Optimization (PSO, [4]). The individual success of a fish is measured by its weight – consequently, promising areas can be inferred from regions where bigger ensembles of fish are located. As for other heuristic search algorithms we consider the problem of finding a “good” (ideally the global) solution of an optimization problem with bound constraints in the form: minx∈Ω f (x), where f : RN → R is a nonlinear objective function and x is the feasible region. Since we do not assume that f is convex, f may possess many local minima. Solving such tasks for high dimensional real world problems may be expensive in terms of runtime if exact algorithms were used. Various nature inspired algorithms have shown to be able Y. Tan et al. (Eds.): ICSI 2011, Part II, LNCS 6729, pp. 553–562, 2011. c Springer-Verlag Berlin Heidelberg 2011

554

A. Janecek and Y. Tan

to preform well with these difficulties. Even though if these algorithms are only meta-heuristics, i.e. there is no proof that they reach the global optimum of the solution, these techniques often achieve a reasonably good solution for the given task at hand in a reasonable amount of time. Related work. The FSS algorithm was introduced to the scientific community in 2008 [1]. This paper was extended to a book chapter [2] where FSS has been evaluated and compared to different variants of PSO. Results indicate that FSS is able to achieve better results as PSO on several benchmark functions, especially on multimodal functions with several local minima. In another study [3] the same authors analyzed the importance of the swimming operators of FSS and showed that all operators have strong influences on the results. Although for some benchmarks the individual operator alone sometimes produced better results than all operators together, the results using only the individual operator are highly sensitive to the initial and also final values of stepind and stepvol . Moreover, it was shown that a rather large initial value for stepind (stepind initial = 10%) generally achieved the best results. In a very recent study FSS has been used successfully to initialize the factors of the non-negative matrix factorization (NMF) [5]. In this work we aim at investigating the influence of newly developed weight update strategies for FSS as well as the influence of a non-linear decrease of the step-size parameters stepind and stepvol . We introduce and compare weight update strategies based on a linear decrease of weights, as well as a fitness based weight decrease strategy. Moreover, we introduce a combination of (i) this fitness based weight decrease strategy, (ii) the non-linear decrease of the step-size parameters, and (iii) a newly introduced dilation multiplier which breaks the symmetry between contraction and dilation but can be useful in some situations to escape from local minima. Experimental evaluation performed on five benchmark functions shows that especially the non-linear decrease of the stepsize parameters is an effective and efficient way to significantly speed up the convergence of FSS and also to achieve better fitness per iteration results.

2

The Fish School Search Algorithm

FSS is based on four operators which can be grouped into two classes: feeding and swimming. Feeding represents updating the weight of the fish based on the successfulness of the current movement. The swimming operators (individual movement, collective instinctive movement, and collective volitive movement) move the fish according to the feeding operator. FSS is closely related to PSO and other population based algorithms such as Genetic Algorithms [6], Differential Evolution [7], and the Firework Algorithm [8]. The main difference compared to PSO is that no global variables need to be logged in FSS. Some similarities and differences of FSS to other population based algorithms are given in [2]. FSS operators. In the following we briefly review the basic operators of the Fish School Search algorithm as presented in [3]. A pseudo code of the FSS algorithm can also be found in [3]. The algorithm starts with all fish initialized at random positions and equal weight wi (0) set to 1.

Feeding the Fish – Weight Update Strategies for the FSS Algorithm

555

A. Individual movement: In each iteration, each fish randomly chooses a new position which is determined by adding to each dimension j of the current position a random number multiplied by a predetermined step (stepind ). nj (t) = xj (t) + randu(−1, 1) ∗ stepind ,

(1)

where randu(−1, 1) is a random number from a uniform distribution in the interval [−1, 1]. The movement only occurs if the new position n has a better fitness than the current position x, and if n lies within the aquarium boundaries. Fitness difference (Δf ) and displacement (Δx) are evaluated according to Δf = f (n) − f (x),

(2)

Δx = n − x.

(3)

If no individual movement occurs Δf = 0 and Δx = 0. The parameter stepind decreases linearly during the iterations stepind (t + 1) = stepind (t) −

stepind initial − stepind f inal . number of iterations

(4)

B. Feeding: Fish can increase their weight depending on the success of the individual movement according to wi (t + 1) = wi (t) + Δf (i)/max(Δf ),

(5)

where wi (t) is the weight of fish i, Δf (i) is the difference of the fitness at current and new location, and max(Δf ) is the maximum Δf of all fish. An additional parameter wscale limits the weight of a fish (1

Advances in Swarm Intelligence, Part II

Advances in Swarm Intelligence, Part I - ICSI 2011

Swarm intelligence

Swarm Intelligence

Innovations in swarm intelligence

Advances in Computational Intelligence, Part II - IWANN 2011

Swarm Intelligence in Data Mining

Nanocomputers and Swarm Intelligence

Advances in Computing and Communications, Part II

Advances in Multimedia Modeling, Part II

Innovations in Swarm Intelligence (Studies in Computational Intelligence, Volume 248)

Swarm Intelligence: Introduction and Applications

Physicomimetics: Physics-Based Swarm Intelligence

Swarm Intelligence (The Morgan Kaufmann Series in Artificial Intelligence)

Advances in Artificial Intelligence

Swarm, Evolutionary, and Memetic Computing, Part II - SEMCCO 2011

Artificial Intelligence and Computational Intelligence. AICI 2011 Proceedings Part II (Lecture Notes in Artificial Intelligence)

Modern Approaches in Applied Intelligence, Part II - IEA AIE 2011

Advances in Artificial Intelligence, Part I - MICAI 2011

Advances in Computational Intelligence, Part I - IWANN 2011

Advances in Knowledge Discovery and Data Mining, Part II

Advances in Soft Computing, Part II - MICAI 2011

Advances in Applied Artificial Intelligence

Advances in Neural Networks, Part II - ISNN 2011

Advances in Artificial Life, Part II - ECAL 2009

Advances in Artificial Intelligence SBIA

Swarm

Ant Colony Optimization And Swarm Intelligence

Swarm

Swarm

Swarm

Advances in Swarm Intelligence, Part II