Advances in Neural Network Research and Applications (Lecture Notes in Electrical Engineering, 67)

Lecture Notes in Electrical Engineering Volume 67

Zhigang Zeng · Jun Wang (Eds.)

Advances in Neural Network Research and Applications

ABC

Prof. Zhigang Zeng Department of Control Science and Engineering Huazhong University of Science and Technology Wuhan, Hubei, 430074 China E-mail: [email protected] Prof. Jun Wang Department of Mechanical & Automation Engineering The Chinese University of Hong Kong Shatin, New Territories Hong Kong Tel.: +852 26098472 Fax: +852-26036002 E-mail: [email protected]

ISBN 978-3-642-12989-6

e-ISBN 978-3-642-12990-2

DOI 10.1007/978-3-642-12990-2 Library of Congress Control Number: 2010926919 c 2010 Springer-Verlag Berlin Heidelberg  This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typeset & Coverdesign: Scientific Publishing Services Pvt. Ltd., Chennai, India. Printed on acid-free paper 987654321 springer.com

Preface

This book is a part of the Proceedings of the Seventh International Symposium on Neural Networks (ISNN 2010), held on June 6-9, 2010 in Shanghai, China. Over the past few years, ISNN has matured into a well-established premier international symposium on neural networks and related fields, with a successful sequence of ISNN series in Dalian (2004), Chongqing (2005), Chengdu (2006), Nanjing (2007), Beijing (2008), and Wuhan (2009). Following the tradition of ISNN series, ISNN 2010 provided a high-level international forum for scientists, engineers, and educators to present the state-of-the-art research in neural networks and related fields, and also discuss the major opportunities and challenges of future neural network research. Over the past decades, the neural network community has witnessed significant breakthroughs and developments from all aspects of neural network research, including theoretical foundations, architectures, and network organizations, modeling and simulation, empirical studies, as well as a wide range of applications across different domains. The recent developments of science and technology, including neuroscience, computer science, cognitive science, nano-technologies and engineering design, among others, has provided significant new understandings and technological solutions to move the neural network research toward the development of complex, large scale, and networked brain-like intelligent systems. This long-term goals can only be achieved with the continuous efforts from the community to seriously investigate various issues on neural networks and related topics. Toward this end, the ISNN 2010 provided a great platform for the community to share their latest research results, discuss critical future research directions, stimulate innovative research ideas, as well as facilitate worldwide multi-disciplinary collaborations. ISNN 2010 received numerous submissions from about thousands of authors in about 40 countries and regions (Australia, Belgium, Brazil, Canada, China, Colombia, Czech Republic, Finland, France, Germany, Hong Kong, India, Iran, Italy, Japan, Lebanon, Lithuania, Macao, Malaysia, Mexico, Netherlands, New Zealand, Norway, Peru, Poland, Portugal, Singapore, Slovenia, South Africa, South Korea, Spain, Sri Lanka, Taiwan, Thailand, Tunisia, Uganda, United Arab Emirates, United Kingdom, and USA) across six continents (Asia, Europe, North America, South America, Africa, and Oceania). Based on the rigorous peer-reviews by the program committee members and the reviewers, 108 high-quality papers were selected for publications in Lecture Notes in Electrical Engineering (LNEE) Proceedings. These papers cover all major topics of the engineering designs and applications of neural network research. In addition to the contributed papers, the ISNN 2010 technical program included four plenary speeches by Andrzej Cichocki (RIKEN Brain Science Institute, Japan), ChinTeng Lin (National Chiao Tung University, Taiwan), DeLiang Wang (Ohio State University, USA), Gary G. Yen (Oklahoma State University, USA). We would like to take this opportunity to express our sincere thanks to the Shanghai Jiao Tong University, The Chinese University of Hong Kong, Microsoft Research Asia, and the National Natural Science Foundation of China for their

VI

Preface

sponsorship, to the IEEE Shanghai Section, IEEE Computational Intelligence Society, International Neural Network Society, Asia Pacific Neural Network Assembly, European Neural Network Society, and International Association for Mathematics and Computers in Simulation for their technical co-sponsorship. We would also like to sincerely thank the chairs for their contributions to the symposium, members of the Advisory Committee and Steering Committee for their guidance, and the members of the Organizing Committee, Special Sessions Committee, Publication Committee, Publicity Committee, Finance Committee, Registration Committee, and Local Arrangements Committee for all their great efforts and time in organizing such an event. We would like especially to express our deepest gratitude to the members of the International Program Committee and all reviewers for their professional review of the papers; their expertise guaranteed the high quality of the technical program of the ISNN 2010. Furthermore, we would also like to thank Springer for publishing the proceedings in the prestigious series of Lecture Notes in Electrical Engineering. Meanwhile, we also would like to express our heartfelt appreciations to the plenary and panel speakers for their vision and discussions on the latest research development in the field as well as critical future research directions, opportunities, and challenges. Finally, we would also like to thank all the speakers, authors, and participants for their great contribution and support that made ISNN 2010 a great success.

Zhigang Zeng Jun Wang

ISNN 2010 Organization

Sponsor and Organizer Shanghai Jiao Tong University

Co-organizers The Chinese University of Hong Kong Microsoft Research Asia

Technically Co-sponsors IEEE Shanghai Section IEEE Computational Intelligence Society International Neural Network Society European Neural Network Society Asia Pacific Neural Network Assembly International Association for Mathematics and Computers in Simulation

Financial Co-sponsor National Natural Science Foundation of China

General Chairs Jun Wang Bao-Liang Lu

Hong Kong, China Shanghai, China

Organizing Committee Chair Jianbo Su

Shanghai, China

Program Committee Chairs Liqing Zhang Zhigang Zeng James T.Y. Kwok

Shanghai, China Wuhan, China Hong Kong, China

VIII

Organization

Special Sessions Chairs Si Wu Qing Ma Paul S. Pang

Shanghai, China Kyoto, Japan Auckland, New Zealand

Publications Chairs Hongtao Lu Yinling Wang Wenlian Lu

Shanghai, China Shanghai, China Shanghai, China

Publicity Chairs Bo Yuan Xiaolin Hu Qingshan Liu

Shanghai, China Beijing, China Nanjing, China

Finance Chairs Xinping Guan Xiangyang Zhu

Shanghai, China Shanghai, China

Registration Chairs Fang Li Gui-Rong Xue Daniel W.C. Ho

Shanghai, China Shanghai, China Hong Kong, China

Local Arrangements Chairs Qingsheng Ren Xiaodong Gu

Shanghai, China Shanghai, China

Advisory Committee Chairs Xiaowei Tang Bo Zhang Aike Guo

Hangzhou, China Beijing, China Shanghai, China

Organization

IX

Advisory Committee Members Cesare Alippi, Milan, Italy Shun-ichi Amari, Tokyo, Japan Zheng Bao, Xi'an, China Dimitri P. Bertsekas, Cambridge, MA, USA Tianyou Chai, Shenyang, China Guanrong Chen, Hong Kong Andrzej Cichocki, Tokyo, Japan Ruwei Dai, Beijing, China Jay Farrell, Riverside, CA, USA Chunbo Feng, Nanjing, China Russell Eberhart, Indianapolis, IN, USA David Fogel, San Diego, CA, USA Walter J. Freeman, Berkeley, CA, USA Kunihiko Fukushima, Osaka, Japan Xingui He, Beijing, China Zhenya He, Nanjing, China Janusz Kacprzyk, Warsaw, Poland Nikola Kasabov, Auckland, New Zealand Okyay Kaynak, Istanbul, Turkey Anthony Kuh, Honolulu, HI, USA Frank L. Lewis, Fort Worth, TX, USA Deyi Li, Beijing, China Yanda Li, Beijing, China

Chin-Teng Lin, Hsinchu, Taiwan Robert J. Marks II, Waco, TX, USA Erkki Oja, Helsinki, Finland Nikhil R. Pal, Calcutta, India Marios M. Polycarpou, Nicosia, Cyprus José C. Príncipe, Gainesville, FL, USA Leszek Rutkowski, Czestochowa, Poland Jennie Si, Tempe, AZ, USA Youxian Sun, Hangzhou, China DeLiang Wang, Columbus, OH, USA Fei-Yue Wang, Beijing, China Shoujue Wang, Beijing, China Paul J. Werbos, Washington, DC, USA Cheng Wu, Beijing, China Donald C. Wunsch II, Rolla, MO, USA Youlun Xiong, Wuhan, China Lei Xu, Hong Kong Shuzi Yang, Wuhan, China Xin Yao, Birmingham, UK Gary G. Yen, Stillwater, OK, USA Nanning Zheng, Xi'an, China Yongchuan Zhang, Wuhan, China Jacek M. Zurada, Louisville, KY, USA

Program Committee Members Haydar Akca Alma Y. Alanis Bruno Apolloni Sabri Arik Vijayan Asari Tao Ban Peter Baranyi Salim Bouzerdoum Martin Brown Xindi Cai Jianting Cao Yu Cao Jonathan Chan

Chu-Song Chen Liang Chen Sheng Chen Songcan Chen YangQuan Chen Yen-Wei Chen Zengqiang Chen Jianlin Cheng Li Cheng Long Cheng Zheru Chi Sung-Bae Cho Emilio Corchado

X

Organization

Jose Alfredo F. Costa Ruxandra Liana Costea Sergio Cruces Baotong Cui Chuanyin Dang Mingcong Deng Ming Dong Jixiang Du Andries Engelbrecht Meng Joo Er Jufu Feng Chaojin Fu Wai-Keung Fung John Gan Junbin Gao Xiao-Zhi Gao Xinping Guan Chen Guo Chengan Guo Ping Guo Abdenour Hadid Honggui Han Qing-Long Han Haibo He Hanlin He Zhaoshui He Akira Hirose Daniel Ho Noriyasu Homma Zhongsheng Hou Chun-Fei Hsu Huosheng Hu Jinglu Hu Junhao Hu Sanqing Hu Guang-Bin Huang Tingwen Huang Wei Hui Amir Hussain Jayadeva Minghui Jiang Tianzi Jiang Yaochu Jin Joarder Kamruzzaman Shunshoku Kanae Qi Kang Nik Kasabov

Okyay Kaynak Rhee Man Kil Kwang-Baek Kim Sungshin Kim Mario Koeppen Rakhesh Singh Kshetrimayum Edmund Lai Heung Fai Lam Minho Lee Chi-Sing Leung Henry Leung Chuandong Li Fang Li Guang Li Kang Li Li Li Shaoyuan Li Shutao Li Xiaoli Li Xiaoou Li Xuelong Li Yangmin Li Yuanqing Li Yun Li Zhong Li Jinling Liang Ming Liang Pei-Ji Liang Yanchun Liang Li-Zhi Liao Wudai Liao Longnian Lin Guoping Liu Ju Liu Meiqin Liu Yan Liu Hongtao Lu Jianquan Lu Jinhu Lu Wenlian Lu Jian Cheng Lv Jinwen Ma Malik Magdon Ismail Danilo Mandic Tiemin Mei Dan Meng Yan Meng

Organization

Duoqian Miao Martin Middendorf Valeri Mladenov Marco Antonio Moreno-Armendáriz Ikuko Nishkawa Stanislaw Osowski Seiichi Ozawa Shaoning Pang Jaakko Peltonen Vir V. Phoha Branimir Reljin Qingsheng Ren Tomasz Rutkowski Sattar B. Sadkhan Toshimichi Saito Gerald Schaefer Furao Shen Daming Shi Hideaki Shimazaki Michael Small Qiankun Song Jochen J. Steil John Sum Roberto Tagliaferri Norikazu Takahashi Ah-hwee Tan Ying Tan Toshihisa Tanaka Dacheng Tao Ruck Thawonmas Xin Tian Christos Tjortjis Ivor Tsang Masao Utiyama Marc Vanhulle Bin Wang Dan Wang Dianhui Wang Lei Wang Liang Wang Rubin Wang

Wenjia Wang Wenwu Wang Xiaoping Wang Xin Wang Yinglin Wang Yiwen Wang Zhanzhan Wang Zhongsheng Wang Zidong Wang Hau-San Wong Kevin Wong Wei Wu Cheng Xiang Hong Xie Songyun Xie Rui Xu Xin Xu Guirong Xue Yang Yang Yingjie Yang Yongqing Yang Jianqiang Yi Dingli Yu Jian Yu Xiao-Hua Yu Bo Yuan Kun Yuan Pong C Yuen Xiaoqin Zeng Changshui Zhang Jie Zhang Junping Zhang Kai Zhang Lei Zhang Nian Zhang Dongbin Zhao Hai Zhao Liang Zhao Qibin Zhao Mingjun Zhong Weihang Zhu

Reviewers Ajith Abraham Alma Y. Alanis NG Alex

Jing An Sung Jun An Claudia Angelini

Nancy Arana-Daniel Nancy Arana-Daniel Kiran Balagani

XI

XII

Organization

Tao Ban Simone Bassis Anna Belardinelli Joao Roberto Bertini Junior Amit Bhaya Shuhui Bi Xuhui Bo Salim Bouzerdoum N. Bu Qiao Cai Xindi Cai Hongfei Cao Yuan Cao Jonathan Chan Wenge Chang Benhui Chen Bo-Chiuan Chen Chao-Jung Chen Chu-Song Chen Cunbao Chen Fei Chen Gang Chen Guici Chen Junfei Chen Lei Chen Min Chen Pin-Cheng Chen Sheng Chen Shuwei Chen Tao Chen Xiaofen Chen Xiaofeng Chen Yanhua Chen Yao Chen Zengqiang Chen Zhihao Chen Jianlin Cheng K. H. Cheng Lei Cheng Yu Cheng Yuhu Cheng Seong-Pyo Cheon Zheru Chi Seungjin Choi Angelo Ciaramella Matthew Conforth

Paul Christopher Conilione Paleologu Constantin Jose Alfredo F. Costa Ruxandra Liana Costea Fangshu Cui Zhihua Cui James Curry Qun Dai Xinyu Dai Spiros Denaxas Jing Deng Xin Deng Zhijian Diao Ke Ding Jan Dolinsky Yongsheng Dong Adriao Duarte Doria Neto Dajun Du Jun Du Shengzhi Du Wei Du Qiguo Duan Zhansheng Duan Julian Eggert Yong Fan Chonglun Fang Italia De Feis G C Feng Qinrong Feng Simone Fiori Chaojin Fu Jun Fu Zhengyong Fu Zhernyong Fu Sheng Gan Shenghua Gao Fei Ge Vanessa Goh Dawei Gong Weifeng Gu Wenfei Gu Renchu Guan Chengan Guo Jianmei Guo Jun Guo Ping Guo

Xin Guo Yi Guo Juan Carlos Gutierrez Caceres Osamu Hasegawa Aurelien Hazart Hanlin He Huiguang He Lianghua He Lin He Wangli He Xiangnan He Zhaoshui He Sc Ramon Hernandez Esteban Hernandez-Vargas Kevin Ho Xia Hong Chenping Hou Hui-Huang Hsu Enliang Hu Jinglu Hu Junhao Hu Meng Hu Sanqing Hu Tianjiang Hu Xiaolin Hu Zhaohui Hu Bonan Huang Chun-Rong Huang Dan Huang J. Huang Kaizhu Huang Shujian Huang Xiaodi Huang Xiaolin Huang Zhenkun Huang Cong Hui GuoTao Hui Khan M. Iftekharuddin Tasadduq Imam Teijiro Isokawa Mingjun Ji Zheng Ji Aimin Jiang Changan Jiang Feng Jiang

Organization

Lihua Jiang Xinwei Jiang Gang Jin Ning Jin Yaochu Jin Krzysztof Siwek Yiannis Kanellopoulos Enam Karim Jia Ke Salman Khan Sung Shin Kim Tae-Hyung Kim Mitsunaga Kinjo Arto Klami Mario Koeppen Adam Kong Hui Kong Qi Kong Adam Krzyzak Jayanta Kumar Debnath Kandarpa Kumar Sarma Franz Kurfess Paul Kwan Darong Lai Jiajun Lai Jianhuang Lai Wei Lai Heung Fai Lam Paul Lam Yuan Lan Ngai-Fong Law N. K. Lee Chi SingLeung Bing Li Boyang Li C. Li Chaojie Li Chuandong Li Dazi Li Guang Li Junhua Li Kang Li Kelin Li Li Li Liping Li Lulu Li Manli Li

Peng Li Ping Li Ruijiang Li Tianrui Li Tieshan Li Xiaochen Li Xiaocheng Li Xuelong Li Yan Li Yun Li Yunxia Li Zhenguo Li Allan Liang Jinling Liang Pei-Ji Liang Li-Zhi Liao Wudai Liao Hongfei Lin Qing Lin Tran Hoai Lin Bo Liu Chang Liu Chao Liu Fei Liu Hongbo Liu Jindong Liu Lei Liu Lingqiao Liu Nianjun Liu Qingshan Liu Wei Liu Xiangyang Liu Xiwei Liu Yan Liu Yanjun Liu Yu Liu Zhaobing Liu Zhenwei Liu Jinyi Long Jinyi Long Carlos Lopez-Franco Shengqiang Lou Mingyu Lu Ning Lu S F Lu Bei Lv Jun Lv

XIII

Fali Ma Libo Ma Singo Mabu Danilo Mandic Qi Mao Tomasz Markiewicz Radoslaw Mazur Tiemin Mei Bo Meng Zhaohui Meng Marna van der Merwe Martin Middendorf N. Mitianoudis Valeri Mladenov Alex Moopenn Marco Moreno Loredana Murino Francesco Napolitano Ikuko Nishkawa Tohru Nitta Qiu Niu Qun Niu Chakarida Nukoolkit Sang-Hoon Oh Floriberto Ortiz Stanislaw Osowski Antonio de Padua Braga Antonio Paiva Shaoning Pang Woon Jeung Park Juuso Parkkinen Michael Paul Anne Magály de Paula Canuto Zheng Pei Jaakko Peltonen Ce Peng Hanchuan Peng Jau-Woei Perng Son Lam Phung Xiong Ping Kriengkrai Porkaew Santitham Prom-on Dianwei Qian Lishan Qiao Keyun Qin Meikang Qiu

XIV

Organization

Li Qu Marcos G. Quiles Mihai Rebican Luis J. Ricalde Jorge Rivera Haijun Rong Zhihai Rong Tomasz Rutkowski Jose A. Ruz Edgar N. Sanchez Sergio P. Santos Renato José Sassi Chunwei Seah Nariman Sepehri Caifeng Shan Shiguang Shan Chunhua Shen Furao Shen Jun Shen Yi Shen Jiuh-Biing Sheu Licheng Shi Qinfeng Shi Xiaohu Shi Si Si Leandro Augusto da Silva Angela Slavova Sunantha Sodsee Dandan Song Dongjin Song Doo Heon Song Mingli Song Qiang Song Qiankun Song Kingkarn Sookhanaphibarn Gustavo Fontoura de Souza Antonino Staiano Jochen Steil Pui-Fai Sum Jian Sun Jian-Tao Sun Junfeng Sun Liang Sun Liming Sun Ning Sun

Yi Sun Shigeru Takano Mingkui Tan Ke Tang Kecheng Tang Y Tang Liang Tao Yin Tao Sarwar Tapan Ruck Thawonmas Tuan Hue Thi Le Tian Fok Hing Chi Tivive Christos Tjortjis Rutkowski Tomasz Julio Tovar Jianjun Tu Zhengwen Tu Goergi Tzenov Lorenzo Valerio Rodrigo Verschae Liang Wan Min Wan Aihui Wang Bin Wang Bo Hyun Wang Chao Wang Chengyou Wang Dianhui Wang Guanjun Wang Haixian Wang Hongyan Wang Huidong Wang Huiwei Wang Jingguo Wang Jinghua Wang Lan Wang Li Wang Lili Wang Lizhi Wang Min Wang Ming Wang Pei Wang Ruizhi Wang Xiaolin Wang Xiaowei Wang Xin Wang

Xu Wang Yang Wang Ying Wang You Wang Yunyun Wang Zhanshan Wang Zhengxia Wang Zhenxing Wang Zhongsheng Wang Bunthit Watanapa Hua-Liang Wei Qinglai Wei Shengjun Wen Young-Woon Woo Ailong Wu Chunguo Wu Jun Wu Qiang Wu Si Wu Xiangjun Wu Yili Xia Zeyang Xia Cheng Xiang Linying Xiang Shiming Xiang Xiaoliang Xie Ping Xiong Zhihua Xiong Fang Xu Feifei Xu Heming Xu Jie Xu LinLi Xu Rui Xu Weihong Xu Xianyun Xu Xin Xu Hui Xue Jing Yang Liu Yang Qingshan Yang Rongni Yang Shangming Yang Wen-Jie Yang Wenlu Yang Wenyun Yang Xubing Yang

Organization

Yan Yang Yongqing Yang Zi-Jiang Yang John Yao Jun Yao Yingtao Yao Keiji Yasuda Ming-Feng Yeh Xiao Yi Chenkun Yin Kaori Yoshida WenwuYu Xiao-Hua Yu Kun Yuan Weisu Yuan Xiaofang Yuan Zhuzhi Yuan Zhuzhu Yuan P C Yuen Masahiro Yukawa Lianyin Zhai Biao Zhang Changshui Zhang Chen Zhang

Secretariat Jin Gang Kan Hong Qi Kong Qiang Wang Qiang Wu Rong Zhou Tianqi Zhang

Dapeng Zhang Jason Zhang Jian Zhang Jianbao Zhang Jianhai Zhang Jianhua Zhang Jin Zhang Junqi Zhang Junying Zhang Kai Zhang Leihong Zhang Liming Zhang Nengsheng Zhang Nian Zhang Pu-Ming Zhang Qing Zhang Shaohong Zhang Tao Zhang Teng-Fei Zhang Ting Zhang Xian-Ming Zhang Yuyang Zhang Hai Zhao Qibin Zhao

Xiaoyu Zhao Yi Zhao Yongping Zhao Yongqing Zhao Ziyang Zhen Chengde Zheng Lihong Zheng Yuhua Zheng Caiming Zhong Mingjun Zhong Shuiming Zhong Bo Zhou Jun Zhou Luping Zhou Rong Zhou Xiuling Zhou Haojin Zhu Song Zhu Wenjun Zhu Xunlin Zhu Yuanming Zhu Wei-Wen Zou Xin Zou Pavel Zuñiga

XV

Table of Contents

Prediction and Forecasting A Novel Prediction Mechanism with Modified Data Mining Technique for Call Admission Control in Wireless Cellular Network . . . . . . . . . . . . . . Chen-Feng Wu, Yu-Teng Chang, Chih-Yao Lo, and Han-Sheng Zhuang

1

The Study of Forecasting Model of Rock Burst for Acoustic Emission Based on BP Neural Network and Catastrophe Theory . . . . . . . . . . . . . . . Yunyun Xu and Dongqiang Xu

11

Application of Wavelet Neural Network to Prediction of Water Content in Crude Oil . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Huifen Niu, Cuiling Liu, Jinqi Wang, and Xiaowen Sun

21

Prediction of Urban Heat Island Intensity in Chuxiong City with Backpropagation Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Wujun Xi and Ping He

29

Research on the Fouling Prediction of Heat Exchanger Based on Wavelet Relevance Vector Machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lingfang Sun, Rina Saqi, and Honggang Xie

37

Electricity Price Forecasting Using Neural Networks Trained with Entropic Criterion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jianhua Zhang, Jingyue Wang, Rui Wang, and Guolian Hou

47

Least Square Support Vector Machine Ensemble for Daily Rainfall Forecasting Based on Linear and Nonlinear Regression . . . . . . . . . . . . . . . . Jiansheng Wu, Mingzhe Liu, and Long Jin

55

Estimating Portfolio Risk Using GARCH-EVT-Copula Model: An Empirical Study on Exchange Rate Market . . . . . . . . . . . . . . . . . . . . . . . . . . Zongrun Wang, Yanbo Jin, and Yanju Zhou

65

Forecasting Financial Time Series via an Efficient CMAC Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chi-Jie Lu and Jui-Yu Wu

73

Forecasting Daily Cash Turnover of Bank with EWMA and SVR . . . . . . Wei-min Ma and Wei Lu Financial Distress Prediction Model via GreyART Network and Grey Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ming-Feng Yeh, Chia-Ting Chang, and Min-Shyang Leu

83

91

XVIII

Table of Contents

Risk Assessment Model Based on Immune Theory . . . . . . . . . . . . . . . . . . . . Tao Liu, Li Shang, and Zhifeng Hu

101

Short-Term Load Forecasting Based on Bayes and RS . . . . . . . . . . . . . . . . Yanmei Li

109

Fuzzy Neural Networks Nonlinear System Modeling with a New Fuzzy Model and Neural Compensation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cruz Vega Israel and Wen Yu

117

A Research of Fuzzy Neural Network in Ferromagnetic Target Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Caipeng Wu, Jiahao Deng, and Yanli Yang

129

Multiple T-S Fuzzy Neural Networks Soft Sensing Modeling of Flotation Process Based on Fuzzy C-Means Clustering Algorithm . . . . . . Jiesheng Wang, Yong Zhang, and Shifeng Sun

137

A Using Reliability Evaluation Model for Diesel Engine Based on Fuzzy Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ying-Kui Gu and Kai-Qi Huang

145

Fuzzy Sliding Mode Control with Perturbation Estimation for a Piezoactuated Micromanipulator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Qingsong Xu and Yangmin Li

153

A Credit Risk Rating Model Based on Fuzzy Neural Network . . . . . . . . . . Ke-Jun Zhu, Pin-Chang Chen, and Yu-Teng Chang

161

Interval-Valued Fuzzy Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Wenyi Zeng and Jiayin Wang

173

Research on Fuzzy Preference Relations-Based MAS for Decision Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Weijin Jiang and Qing Jiang

185

Optimization and Planning Study of Stochastic Demand Inventory Routing Problem with Soft Time Windows Based on MDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Wei Zeng and Qian Zhao

193

An Agent-Based Approach to Joint Procurement Modeling with Virtual Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chih-Yao Lo, Hung-Teng Chang, Yu-Teng Chang, and Hsiu-Yun Hu

201

Table of Contents

Interactive Hybrid Evolutionary Computation for MEMS Design Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ying Zhang and Alice M. Agogino

XIX

211

Genetic Algorithms for Traffic Grooming in Optical Tree Networks . . . . . Shu-tong Xie and Li-fang Pan

219

Multi-sensor Multi-target Tracking with OOSM . . . . . . . . . . . . . . . . . . . . . . Cheng Cheng and Jinfeng Wang

227

Hopfield Neural Network Guided Evolutionary Algorithm for Aircraft Penetration Path Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Nan Wang, Lin Wang, Xueqiang Gu, Jing Chen, and Lincheng Shen Fuzzy Material Procurement Planning with Value-at-Risk . . . . . . . . . . . . . Gao-Ji Sun Radial Basis Function Network for Endpoint Detection in Plasma Etch Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Shu-Kun Zhao, Min-Woo Kim, Yi-Seul Han, Se-Youn Jeon, Yun-Keun Lee, and Seung-Soo Han A Novel Cellular Neural Network and Its Applications in Motion Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yan Cao, Feng Zhang, Xuewu Wu, Sheng Lu, Yi Li, Lei Sun, and Shuai Li Evaluation of Enterprise ERP System Based on Neural Network Optimized by Ant Colony . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hanmei Wang, Dongxiao Niu, and Chengkai Cai

235 245

253

265

275

Pattern Recognition Visual Attention-Based Ship Detection in SAR Images . . . . . . . . . . . . . . . . Ying Yu, Zhenghu Ding, Bin Wang, and Liming Zhang

283

Recognizing Multi-ships Based on Silhouette in Infrared Image . . . . . . . . Jun-Wei Lv, Bo Wang, and Dong-Mei Wang

293

SAR Images Feature Extraction and Recognition Based on G2DCDA . . . Liping Hu, Hongwei Liu, and Ping Zhou

301

Bridge Detection and Recognition in Remote Sensing SAR Images Using Pulse Coupled Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Zhenming Peng, Shijun Liu, Guiyou Tian, Zhang Chen, and Tao Tao

311

Approaches to Robotic Vision Control Using Image Pointing Recognition Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tian-Ding Chen

321

XX

Table of Contents

A Hybrid Evolutionary Approach to Band Selection for Hyperspectral Image Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hao Wu, Jiali Zhu, Shijin Li, Dingsheng Wan, and Lin Lin Impulsive Environment Sound Detection by Neural Classification of Spectrogram and Mel-Frequency Coefficient Images . . . . . . . . . . . . . . . . . . Peerapol Khunarsa, Chidchanok Lursinsap, and Thanapant Raicharoen Classification and Control of Cognitive Radios Using Hierarchical Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sheng Chen, Xiaochen Li, Qiao Cai, Nansai Hu, Haibo He, Yu-Dong Yao, and Joseph Mitola Identifying Spatial Patterns of Land Use and Cover Change at Different Scales Based on Self-Organizing Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hao Wu, Xiaoling Chen, Zhan Li, Sheng Wang, Wei Cui, and Qian Meng Colon Tumor Microarray Classification Using Neural Network with Feature Selection and Rule-Based Classification . . . . . . . . . . . . . . . . . . . . . . Narissara Eiamkanitchat, Nipon Theera-Umpon, and Sansanee Auephanwiriyakul

329

337

347

355

363

Signal and Image Processing Dual Channel Speech Denoising Based on Sparse Representation . . . . . . . Gongxian Sun, Feng Gao, Jun Lv, and Ming Xiao Frequency-Domain Blind Separation of Convolutive Speech Mixtures with Energy Correlation-Based Permutation Correction . . . . . . . . . . . . . . . Li-Dan Wang and Qiu-Hua Lin A Blind Broadband Beamforming Method for Speech Enhancement . . . . Dongxia Wang, Fuliang Yin, and Fuming Sun

373

381

391

Algorithm and Simulation Research for Blind Nonlinear System Identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hongqiu Teng and Weizheng Ruan

397

Study on Digital Image Correlation Using Artificial Neural Networks for Subpixel Displacement Measurement . . . . . . . . . . . . . . . . . . . . . . . . . . . . Xiao-yong Liu, Qing-chang Tan, and Rong-li Li

405

Tree Modeling through Range Image Segmentation and 3D Shape Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mingrui Dai, Hongjun Li, and Xiaopeng Zhang

413

Table of Contents

XXI

Combining Bag of Words Model and Information Theoretic Method for Image Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Xue Bai and Siwei Luo

423

Fast Fuzzy c-Means Clustering Algorithm with Spatial Constraints for Image Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yanling Li and Gang Li

431

Application of Support Vector Machines to Airborne Hyper-Spectral Image Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ming-Der Yang, Kai-Siang Huang, Ji-Yuan Lin, and Pei Liu

439

Research for the Identification Method of the Image Definition Based on a W-N Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Miaofen Zhu, Guojin Chen, Yongning Li, and Wanqiang Wang

445

Analysis of Texture Images Generated by Olfactory System Bionic Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chen Fang, Jin Zhang, Shangwu Zhu, Guang Li, and Rulong Wang

453

Do Neural Networks Have True Power for Natural Language Processing? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Qing Ma

461

Robust Channel Identification Using FOCUSS Method . . . . . . . . . . . . . . . Zhaoshui He and Andrzej Cichocki

471

Human Head Modeling Using NURBS Method . . . . . . . . . . . . . . . . . . . . . . Songyun Xie, Ningfei Li, and Zhuo Lv

479

Risk Sensitive Unscented Particle Filter for Bearing and Frequency Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Peng Li, Shenmin Song, and Xinglin Chen

485

Fully Complex Multiplicative Neural Network Model and Its Application to Channel Equalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kavita Burse, R.N. Yadav, and Sushil Chandra Shrivastava

493

Robotics and Control Visual Navigation of a Novel Economical Embedded Multi-mode Intelligent Control System for Powered Wheelchair . . . . . . . . . . . . . . . . . . . Jizhong Liu, Xuepei Wu, Jiating Xia, Guanghui Wang, and Hua Zhang Neural Networks L2 -Gain Control for Robot System . . . . . . . . . . . . . . . . . . Zhi-gang Yu, Yong-liang Shen, Shen-min Song, and Da-wei Zhang

503

513

XXII

Table of Contents

Neural Network Control of Spacecraft Formation Using RISE Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Shicheng Wang, Haibo Min, Fuchun Sun, and Jinsheng Zhang A Simplified Modular Petri Net for the Walking Assistant Robot . . . . . . . Zhen Zhang, Qixin Cao, Chuntao Leng, and Peihua Chen

521

529

Omni-directional Vision Based Tracking and Guiding System for Walking Assistant Robot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Xiaoxiao Zhu, Qixin Cao, Hongbing Tan, and Aolin Tang

537

Dynamic Eye-in-Hand Visual Servoing with Unknown Target Positions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hesheng Wang, Weidong Chen, and Yun-hui Liu

545

Optimum Motion Control for Stacking Robot . . . . . . . . . . . . . . . . . . . . . . . . Xiaoming Zhang, Nan Luan, Zhong Dong, and Liming Chen Passive Target Tracking Using an Improved Particle Filter Algorithm Based on Genetic Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yue Liang, Zhong Liu, and Guodong Zhang Large-Scale Structure Assembly by Multiple Robots Which May Be Broken . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Masayuki Otani, Kiyohiko Hattori, Hiroyuki Sato, and Keiki Takadama Real-Time Five DOF Redundant Robot Control Using a Decentralized Neural Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ramon Garcia-Hernandez, Edgar N. Sanchez, Maarouf Saad, and Eduardo Bayro-Corrochano

551

559

567

577

Improving Transient Response of Adaptive Control Systems Using Multiple Neural Network Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Xiao-li Li, Xiang Yu, and Yan Zhang

587

An Information Theoretic Approach for Design MIMO Networked Control Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jianhua Zhang and Hong Wang

595

An Engineering Solution for Decoupling Control of Aircraft Motion Using Affine Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tsurng-Jehng Shen

603

Black-Box Input-Output Identification of a Class of Nonlinear Systems Using a Discrete-Time Recurrent Neurofuzzy Network . . . . . . . . . . . . . . . . Marcos A. Gonz´ alez-Olvera and Yu Tang

615

Table of Contents

XXIII

Passivity Analysis of Stochastic Neural Networks with Mixed Time-Varying Delays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Qinggao He and Qiankun Song

623

A Novel Recovering Initial Conditions Method from Spatiotemporal Complex Dynamical System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Longquan Dai, Xiaoyun Kang, and Minfen Shen

631

An Intelligent Control Scheme for Nonlinear Time-Varying Systems with Time Delay . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sun Zhou, Guoli Ji, Wei Lin, and Zijiang Yang

639

Master-Slave Chaos Synchronization of Uncertain Nonlinear Gyros Using Wavelet Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chien-Jung Chiu, Chun-Fei Hsu, Tsu-Tian Lee, and Jang-Zern Tsai

647

Transportation Systems WNN-Based Intelligent Transportation Control System . . . . . . . . . . . . . . . Chiu-Hsiung Chen, Ya-Fu Peng, and Tsai-Sheng Kao

655

Incident Detection in Urban Road . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Wu Bing-Fei, Kao Chih-Chung, Chen Chao-Jung, Li Yen-Feng, Chen Ying-Han, and Yang Cheng-Yen

663

An Efficient Web-Based Tracking System through Reduction of Redundant Connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Bing-Fei Wu, Ying-Han Chen, Chao-Jung Chen, Chih-Chung Kao, and Po-Chia Huang An Embedded All-Time Blind Spot Warning System . . . . . . . . . . . . . . . . . Bing-Fei Wu, Chao-Jung Chen, Yen-Feng Li, Cheng-Yen Yang, Hai-Chang Chien, and Chia-Wei Chang Design of Autonomous Parallel Parking Using Fuzzy Logic Controller with Feed-Forward Compensation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Bo-Chiuan Chen, Yi-Wen Huang, Shiuh-Jer Huang, and Bo-Jhao Liu Telematics Services through Mobile Agents . . . . . . . . . . . . . . . . . . . . . . . . . . Jonathan Lee, Shin-Jie Lee, Hsi-Min Chen, and Wen-Tin Lee Multi-agent System Model for Urban Traffic Simulation and Optimizing Based on Random Walk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yu Cheng, Tao Zhang, and Jianfei Wang Vehicle Detection Using Bayesian Enhanced CoBE Classification . . . . . . . Zhong Zheng, Shen Xu, and Yi L. Murphey

671

679

687

695

703 713

XXIV

Table of Contents

Vibration Analysis of a Submarine Model Based on an Improved ICA Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Wei Cheng, Zhousuo Zhang, and Zhengjia He

721

A Hierarchical Salient-Region Based Algorithm for Ship Detection in Remote Sensing Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Fukun Bi, Feng Liu, and Lining Gao

729

Industrial Applications Turning Tool Wear Monitoring Based on Fuzzy Cluster Analysis . . . . . . . Hongtao Chen, Sui Huang, Dengwan Li, and Pan Fu Part-Machine Clustering: The Comparison between Adaptive Resonance Theory Neural Network and Ant Colony System . . . . . . . . . . . Bo Xing, Wen-Jing Gao, Fulufhelo V. Nelwamondo, Kimberly Battle, and Tshilidzi Marwala

739

747

Fault Diagnosis of Bearings Based on Time-Delayed Correlation and Demodulation as Well as B-Spline Fuzzy Neural Networks . . . . . . . . . . . . . Pan Fu, Li Jiang, A.D. Hope, and Weiling Li

757

Fast and Noninvasive Determination of Viscosity of Lubricating Oil Based on Visible and Near Infrared Spectroscopy . . . . . . . . . . . . . . . . . . . . Lulu Jiang, Yu Zhang, Fei Liu, Lihong Tan, and Yong He

765

Chattering-Free Adaptive Wavelet Neural Network Control for a BLDC Motor via Dynamic Sliding-Mode Approach . . . . . . . . . . . . . . . . . . . . . . . . . Chun-Fei Hsu and Chih-Hu Wang

773

A New BPSO Algorithm and Applications in Interruptible Load Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ping Huang, Pengcheng Li, Yao Zhang, Jinyang Yu, and Yongquan Yuan

783

Force Identification by Using Support Vector Machine and Differential Evolution Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Zhichao Fu and Wei Cheng

793

Soft Sensor Modeling of Ball Mill Load via Principal Component Analysis and Support Vector Machines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jian Tang, Lijie Zhao, Wen Yu, Heng Yue, and Tianyou Chai

803

An Approach Based on Hilbert-Huang Transform and Support Vector Machine for Intelligent Fault Diagnosis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chang Che and Dan Hu

811

Table of Contents

XXV

Real-World Applications Study on Factors of Floating Women’s Income in Jiangsu Province Based on Bayesian Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Yingyu Ge, Chunping Li, and Qin Yin

819

Variation Trend Analysis of Groundwater Depth in Area of Well Irrigation in Sanjiang Plain Based on Wavelet Neural Network . . . . . . . . . Hong Ding, Dong Liu, and Fei-fei Zhao

829

A Petri-Net Modeling Method of Agent’s Belief-Desire-Intention and Its Application in Logistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Weifeng Zhu and Qi Fei

837

Supply Chain Flexibility Assessment by Multivariate Regression and Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ananda S. Jeeva and William W. Guo

845

An Intelligent Mobile Location-Aware Book Recommendation System with Map-Based Guidance That Enhances Problem-Based Learning in Libraries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chin-Ming Chen and Yu-Chieh Yang

853

Applying Least Squares Support Vector Regression with Genetic Algorithms for Radio-Wave Path-Loss Prediction in Suburban Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kuo-Ping Lin, Kuo-Chen Hung, Jen-Chang Lin, Chi-Kai Wang, and Ping-Feng Pai

861

A Probabilistic Neural Network Approach to Modeling the Impact of Tobacco Control Policies by Gender . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Xiaojiang Ding, Chung-Hsing Yeh, and Susan Bedingfield

869

The BPNN-Fuzzy Logic Pre-control of an Underground Project in City Center of Shanghai . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Zhenzhen Yin, Youliang Chen, and Peng Wang

877

Optimal Parameter Inversion of Marine Water Quality Model Using a BPNN Data-Driven Model — A Case Study on DIN . . . . . . . . . . . . . . . . . Mingchang Li, Bin Zhou, Shuxiu Liang, and Zhaochen Sun

885

Determination of Sugar Content of Instant Milk-Tea Using Effective Wavelengths and Least Squares-Support Vector Machine . . . . . . . . . . . . . . Fei Liu and Yong He

893

Sports Video Summarization Based on Salient Motion Entropy and Information Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Bo-Wei Chen, Jhing-Fa Wang, Jia-Ching Wang, and Chen-Yu Chen

901

XXVI

Table of Contents

A Neural Network Based Algorithm for the Retrieval of Precipitable Water Vapor from MODIS Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Shenglan Zhang, Lisheng Xu, Jilie Ding, Hailei Liu, and Xiaobo Deng A Neural Network Based Approach to Wind Energy Yield Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Piers R.J. Campbell, Faheem Ahmed, Haydar Fathulla, and Ahmad D. Jaffar

909

917

Research on New Intelligent Business-Oriented Decision-Making Model Based on MA and GA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Weijin Jiang and Qing Jiang

925

Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

933

A Novel Prediction Mechanism with Modified Data Mining Technique for Call Admission Control in Wireless Cellular Network Chen-Feng Wu, Yu-Teng Chang, Chih-Yao Lo, and Han-Sheng Zhuang Department of Information Management, Yu Da University, No. 168, Hsueh-fu Rd, Tanwen Village, Chaochiao Township, Miaoli County, 361 Taiwan {cfwu,cyt,jacklo,97104504}@ydu.edu.tw

Abstract. It is an important issue to allocate appropriate resources to mobile calls for wireless cellular networks owe to scarce wireless spectrums. The call admission control (CAC) will maintain better performance metrics of mobile call such as call dropping probability (CDP) and call blocking probability (CBP) if the future utilization of wireless spectrums can be predicted and provided to the decision of CAC. Therefore, a prediction mechanism which can predict most information such as system utilization is proposed in this paper. The techniques of data mining and pattern matching which adopts gradient to fuzz time series data for representations of chain code are applied to mining a possible repetitive pattern. Our proposed prediction mechanism can provide prediction information in advance whether the repetitive time series pattern of information exists or not. Furthermore, an update of confident level will be conducted continuously for performing each prediction in the proposed scheme. Our proposed mechanism is developed and tested with four cases which can be regarded as using scenarios of wireless cellular network. The experimental results show that the proposed scheme can capture repetitive time series patterns and perform following predictions with these repetitive time series patterns. Besides, the required storage is less than traditional schemes and lower computation power is required for the proposed scheme. Keywords: Call Admission Control (CAC), data mining, time series, pattern matching, prediction.

1 Introductions Although there has been a rapid development in wireless cellular communications, the QoS guarantee remains one of the most challenging issues [1]. One of the key elements in providing QoS guarantees is an effective CAC policy, which not only has to ensure that the network meets the QoS of the newly arriving calls if accepted, but also guarantees that the QoS of the existing calls does not deteriorate. The variable user mobility has made that it becomes more complex to predict the appropriate cell for handoff. The past research [2] showed the impact of mobility on cellular network and provided a modeling method for configuring cellular networks to study the dynamics of mobility. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 1–9. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

2

C.-F. Wu et al.

The improvement of radio bandwidth is always thought as a dynamic channel (code) allocation problem in literatures [3]. Although there were some schemes for bandwidth reservation proposed to reduce the CDP, such as the study [4], seldom literature has developed to satisfy QoS and lower CBP issues at the same time. Furthermore, the past researches mentioned at the above focused on individual mobility prediction, and they may cause mass load focusing on the MSC. According to the study [5], final information is a prediction of users’ number in a given cell, and it leads to use a global approach that only observes variations of system utilization and users’ flows. There are many advantages for [5]: it does not require any control message and additional load for the MSC; cells generate their own statistics independently from others; it is sensitive to geographical constrains and to users’ common habits. According to [12], the final information is a quantity prediction of users in a given cell, and it leads to use a global approach that only observes variations of system utilization and users’ flows. There are many advantages of the scheme presented in [12]: it does not require any control message and additional load for the MSC; cells generate their own statistics independently from others; it is sensitive to geographical constraints and to users’ common habits. Besides, the concept of aggregated history has been also applied to acquire user mobility profile in [13] so that a user mobility profile framework is developed for estimating service patterns and tracking mobile users, including descriptions of location, mobility, and service requirements. Therefore, in order to provide suitable statistical prediction information which may be system utilization, CBP, or other system resources to the CAC, a prediction mechanism which can predict most information is proposed in this paper.

2 Related Works A suitable prediction mechanism can help the CAC scheme to do better decision for call admission, thus the integration of CAC and prediction will effectively reduce the CDP to meet the QoS requirement. 2.1 CAC Overviews Although CAC schemes can be classified into centralized ones that make the decision of call admission in the MSC and distributed ones that make the decision of call admission in individual cell locally, most of prior researches [14], [15] that were based on different theories focused on the developments of centralized CAC schemes. Instead of using centralized CAC schemes, distributed CAC schemes have been proposed in [16], [17]. Based on the local information, Zhang et al. [16] have applied two prediction mechanisms that were historic data and time series prediction to CAC scheme in noncollaborative manner. Unfortunately, the overheads that are caused by predicting the appropriate information for making CAC decision have been still existed. A clustering prediction scheme that is based on data mining concept for user mobility prediction has been proposed in [17], but the related information from neighboring cells is still required to participate in mobility prediction and more computation loads and storage for data mining process are also required. Although the above researches exhibit that the distributed CAC schemes can reduce the traffic loads

A Novel Prediction Mechanism with Modified Data Mining Technique

3

concentrating on the MSC indeed, some drawbacks, such as an efficient prediction scheme, still exist for themselves. 2.2 Time Series Prediction One of the typical statistic models that based on statistical concepts and principles is Box-Jenkins forecasting model [6]. Although a wide spectrum of time series behavior can be modeled by such models, they can not model well when there are noises in the time series such as inaccurate or incomplete data. In the research [7], neural networks that are one of the well-known artificial intelligent techniques have been claimed to be able to achieve better prediction compared to statistical models. However, the study [8] shows that there exit some drawbacks for neural networks, such as hard to pre-select the system architecture, spending much training time and lacking knowledge representation techniques. Besides, the study [9] applies the data mining concept to perform time series prediction. Both adaptive association rule and pattern matching techniques are employed by [9] to mine for repetitive patterns, which can be used to understand the underlying utilization of system utilization, in the time series. Although this model can perform life long adaptation and reinforcement learning for continuous prediction, the disadvantage is the requirement for intensive computation power and huge storage. The proposed prediction module that is based on time series prediction can provide the appropriate information for the CAC algorithm to make the decision of call admission. Comparing to the traditional data mining process for time series prediction scheme, the proposed prediction method reduces calculation overheads and storage requirements significantly.

3 Methodology Data mining is always applied to perform the prediction by analyzing data and look for data behavior pattern owing to its four basic goals: prediction, identification, classification, and optimization. There exist many knowledge representations, and the association rule is commonly used. Association rules correlate the presence of a set of items with another range of values for another set of variables. 3.1 The Proposed Prediction Method The process flow of the proposed prediction method can be summarized in four steps, as shown as follows: Procedure of Prediction Begin Step 1: Time-series data analysis; Step 2: Time-series prediction; Step 3: Compensation prediction; Step 4: Prediction selection; End

4

C.-F. Wu et al.

Step 1: Time Series Data Analysis This step is response for performing data pre-processing and data mining on time series data. The purpose of data pre-processing is to convert the time series data into gradient code which is defined by the application characteristics of prediction. The data mining is to generate chain code pattern, and the learning results are stored into database which will be used for prediction task. The time series data is represented in a chain code format to keep track of the rate-of-change of time series. The rate-of-change of a time series is obtained by calculating the difference between each two temporal point. The following phase of the analysis step performs data mining to detect the periodicity of patterns and other relative information. The process of data mining is based on Eq. (1) to find iterative pattern. ⎛ R (d ) = set ⎜ ⎝

i+ n −1

∑

t=i

⎞ G (t + d ) − G (t ) ⎟ ⎠

(1)

The set function decides whether a repetitive pattern exists or not. If there exists a repetitive pattern of length n that is represented as the size of the pattern window, the n differences between each gradient vectors must be all zero with distance d, where d is the delay (d >= n) and i is the time of the first point in the preceding pattern. Pattern with different distances between the pattern length and the delay are managed in different list. The occurrence times of periodic patterns and the average of compensation values will be recorded in the database. Step 2: Time-Series Prediction After data mining, the prediction step will make use of periodic patterns to predict possible time series data in future. If a segment of a repetitive time series pattern matches with the inputs, it may happen that the following segment of the particular repetitive pattern has the possibility of reoccurrence. The prediction result with highest probability which is decided by counts of these patterns will be adopted. Because the prediction value is only a gradient code, the final output should be adjusted by the average compensation value and decoded. Fig. 1 illustrates the process of performing prediction task. The example that is with prefix pattern [1, 2, -4, 3] of gradient code wants to predict the value G(t+1), and G(t+1) = -2 with the highest probability 54% will be predicted. Moreover, the prediction gradient code will be passed through the compensation and decoding process to obtain the prediction times series data X(t+1).

Fig. 1. An example of performing prediction

A Novel Prediction Mechanism with Modified Data Mining Technique

5

Step 3: Compensation Prediction In order to provide more accurate prediction information, the compensation prediction that is based on Eq. (2) to perform prediction generates different prediction values to supply selection step for choosing the most appropriate one. C P ( t + 1) = α × ( C P ( t ) + ε t ) + (1 − α ) × X ( t )

ε

t

=

1 t

t

∑

i=1

ε

i

ε i = X (i) − i X (i )

(2) (3) (4)

For Eq. (2), the compensation prediction is concerned with the actual time series data X(t) at current time t and the previous prediction result CP(t) at time t-1. The parameter α is used to decide the weight of the value CP(t), and ε t is average difference that is the distance from actual value X(i) to prediction value i X (i ) and is defined by Eq. (3). Step 4: Prediction Section This step is responsible for determining the final prediction result between time series prediction and compensation prediction and updating the confident levels of pattern lists. Because the proposed scheme can dynamically remind the confidence level of pattern lists as time goes on, the pattern lists with low confidence level will be gradually neglected.

Fig. 2. An example of updating the pattern list

The process of updating the pattern list is illustrated in Fig. 2. At time t+1, the actual time series data X(t+1) that is with former pattern [1, 2, 4, -3] is received and transferred into G(t+1) by pre-processing process. If the actual value of G(t+1) is -2, 1, or 6, the count value of the existing pattern list will increase, and the corresponding confident level will be risen at next prediction. A new pattern will be added to the pattern list, if G(t+1) is not an expected value, such as 8. The prediction selection step dynamically adopts one of the prediction results that are from both compensation prediction and time series prediction. For compensation prediction step, the compensation algorithm generates several results with various coefficients αs, and the best prediction result will be chosen according to the average distance.

6

C.-F. Wu et al.

3.2 Design Considerations There are two defined thresholds, the distance threshold and the count threshold, dominating the accuracy of the time series prediction. If the difference between the prediction value and the actual value is larger than the distance threshold, the prediction will be regarded as a prediction miss. The time series prediction will be thought of being inaccurate when the number of continuous prediction misses is more than the count threshold. For compensation prediction step, the compensation algorithm generates several results with various coefficients of αs, and the best prediction result will be chosen according to the average distance. If there has not been any periodic pattern found or the accuracy of the time series prediction is too low, then the best prediction value that is generated by the compensation prediction will be regarded as the output. Otherwise, the prediction result of the time series patterns will be selected and sent out, and the prediction process is also completed simultaneously by the proposed prediction system. For reducing the consumption of computation capability and the demand of storage, some techniques are applied to the implementation. A list that records the possible pattern length is maintained in the database. The pattern lengths with the higher occurrence probability will earlier become the window size of performing pattern detection, and it will avoid running the detecting algorithm for every single length pattern. All of the system average values are computed by Eq.(5) instead of Eq.(6). A v g (n ) =

n − 1 × A v g (n − 1) + n

A v g (n ) =

1 × n

1 × v a lu e ( n ) n

(5)

n

∑

i = 1

v a lu e (i)

(6)

Where Avg(n) is the average of n number of values, and value(n) is the last value in the data stream. Eq.(6) needs to record all of the n values in database; on the contrary, Eq.(5) reduces the storage demand and less computation is needed.

4 Experimental Results There are four artificial test cases designed for verifying the accuracy of the proposed prediction module for system utilization for a given cell, and the time scale is in minute for all simulations. The purposes of these four test cases are to test how well the prediction method can detect the periodic patterns and perform prediction using these patterns, even though the periodic pattern is not found. The simulations start at 00:00, and advanced time unit is 00:01. The simulation results are shown in Fig. 3 to Fig. 6. The prediction method can provide the next three prediction value X(t+1), X(t+2), and X(t+3) at the time t. For the first case, a periodic pattern is successfully found at 00:08, as shown in Fig. 3, since the pattern ‘4, 2,-1, -4’ has appeared twice. The prediction value which produced by the compensation prediction step with the most suitable α is adopted by the prediction method before the pattern detected. While the periodic pattern has been detected, the time series prediction step applies it to perform prediction.

A Novel Prediction Mechanism with Modified Data Mining Technique

7

For the second case, the simulation results of the second case that is the first case of time series added a little bit of random oscillation is shown in Fig. 4. At the same time 00:08, the same pattern has been detected and the prediction value from compensation step is also replaced by the result of time series prediction step at the following time. Obviously, the compensation values of each item in the pattern list are different, and the prediction step adjusts the expected prediction value with the average compensation values. The third case is applied to test a suddenly continuous scenario as shown in Fig. 5. The front part of this time series is the same as the first test case except at 00:09 the utilization increases 10% suddenly and lasts in the following time series. The identical pattern can be detected but the prediction value that is derived from the periodic pattern causes one miss. Whatever, the prediction module can restore the accuracy immediately. In the fourth case, the time series prediction sub-module identifies successfully no existence of periodic pattern found. Thus, the compensation prediction step is instead of time series prediction step to provide the prediction value in the prediction method. Fig. 6 shows that the compensation prediction step can provide better prediction values at the part with fixed and smaller variation. According to the simulation results of the four test cases, the proposed prediction method can perform well in all kinds of time series data. Once the repetitive patterns exist in the time series, the step of time series data analysis can identify the pattern quickly, and accurate prediction values can be provided by the time series prediction step. Contradictorily, if there is not any periodic pattern existing, the proposed prediction module not only successfully identifies that the time series is random but also provides continuously the prediction values in the following times. Actual Value Predict at t-1 Predict at t-2 Predict at t-3

Time series data for utilization (simple case) 100 90

Utilization

80 70 60 50 40

00 :1 8

00 :1 9

00 :1 6

00 :1 7

00 :1 4

00 :1 5

00 :1 2

00 :1 3

00 :1 1

00 :0 9

00 :1 0

00 :0 8

00 :0 7

00 :0 6

00 :0 5

00 :0 3

00 :0 4

00 :0 1

00 :0 2

00 :0 0

30

Time

Fig. 3. The simulation result for system utilization of simple case Actual Value Predict at t-1 Predict at t-2 Predict at t-3

Time series data for utilization (thrashing case) 100 90

70 60 50 40

00 :1 9

00 :1 7

00 :1 8

00 :1 6

00 :1 5

00 :1 4

00 :1 3

00 :1 2

00 :1 0

00 :1 1

00 :0 9

00 :0 8

00 :0 7

00 :0 6

00 :0 5

00 :0 4

00 :0 3

00 :0 2

00 :0 1

30

00 :0 0

Utilization

80

Time

Fig. 4. The simulation result for system utilization of thrashing case

8

C.-F. Wu et al. Actual Value Predict at t-1 Predict at t-2 Predict at t-3

Time series data for utilization (occupation case-2) 100 90

Utilization

80 70 60 50 40

00 :1 9

00 :1 7

00 :1 8

00 :1 6

00 :1 4

00 :1 5

00 :1 3

00 :1 1

00 :1 2

00 :1 0

00 :0 9

00 :0 7

00 :0 8

00 :0 6

00 :0 4

00 :0 5

00 :0 3

00 :0 1

00 :0 2

00 :0 0

30

Time

Fig. 5. The simulation result for system utilization of occupation case Actual Value Predict at t-1 Predict at t-2 Predict at t-3

Time series data for utilization (random case)

oatin iliztU

100 90 80 70 60 50 40 30 20 10 0

:00 00

:01 00

:02 00

:03 00

:04 00

:05 00

:06 00

:07 00

:08 00

:09 00

:10 00

Time

:11 00

:12 00

:13 00

:14 00

:15 00

:16 00

:17 00

:18 00

:19 00

Fig. 6. The simulation result for system utilization of random case

5 Conclusions The simulation results of the prediction method show that the proposed scheme works well for the four different kinds of test cases. According to the simulation results of the time series prediction method, the proposed prediction method can estimate the required resources accurately in advance, so it could be applied in several fields that need to estimate time series based information beforehand for further manipulations, such as applications in finance, Internet traffic, grid computing, and so on.

Acknowledgements This work was supported in part by the National Science Council of Taiwan under the Grants NSC97-2221-E-412-001.

References 1. Akyildiz, I.F., McNair, J., Ho, J., Uzunaliogluh, H., Wang, W.: Mobility management in next-generation wireless systems. In: Proc. of IEEE ICON 1999, vol. 87, pp. 1347–1384. IEEE Press, Los Alamitos (1999) 2. Zaidi, Z.R., Mark, B.L.: Mobility estimation for wireless networks based on an autoregressive model. In: Proc. of IEEE GLOBECOM 2004, vol. 6, pp. 3405–3409. IEEE Press, Los Alamitos (2004)

A Novel Prediction Mechanism with Modified Data Mining Technique

9

3. Kim, Y.C., Lee, D.E., Lee, B.J., Kim, Y.S.: Dynamic channel reservation based on mobility in wireless ATM networks. IEEE Communication magazine, 47–51 (1999) 4. Tajima, J., Imamura, K.: A Strategy for Flexible Channel Assignment in Mobile Communication Systems. IEEE Transactions on Vehicular Technology 37 (1988) 5. Perato, L., Agha, K.A.: Handover Prediction: User Approach versus Cell Approach. In: Proc. of Mobile and Wireless Communications Network 4th International Workshop, pp. 9–11. IEEE Press, Los Alamitos (2002) 6. William, W., Wei, S.: Time Series Analysis: Univariate and Multivariate Methods. Addison Wesley Publication, Reading (1999) 7. Kamitsuji, S., Shibata, R.: Effectiveness of Stochastic Neural Network for Prediction of Fall or Rise of TOPIX. Research Paper, Keio University (2004) 8. Kasabov, N.: Evolving Connectionist Systems. Springer-Verlag London Limited (2003) 9. Yaik, O.B., Yong, C.H., Haron, F.: Time Series Prediction Using Adaptive Association Rules. In: Proc. Of the First International Conference on Distributed Frameworks for Multimedia Applications (DFMA 2005), pp. 310–314. IEEE Press, Los Alamitos (2005) 10. Chen, H., Kumar, S., Kuo, C.-C.J.: Dynamic Call Admission Control Scheme for QoS Priority Handoff in Multimedia Cellular System. In: Proc. of IEEE Wireless Communications and Networking Conference WCNC 2002, vol. 1, pp. 114–118. IEEE Press, Los Alamitos (2002) 11. Gaasvik, P.-O., Cornefjord, M., Svensson, V.: Different methods of giving priority to handoff traffic in a mobile telephone system with directed retry. In: Proc. Of 41st IEEE Vehicular Technology Conference ‘Gateway to the Future Technology in Motion’, pp. 549–553. IEEE Press, Los Alamitos (1991) 12. Hong, D., Rapport, S.S.: Traffic model and performance analysis for cellular mobile radiotelephone systems with prioritized and nonprioritized handoff procedures. IEEE Trans. on Vehicular Technology 35, 77–92 (1986) 13. Akyildiz, I.F., Wang, W.: The predictive user mobility profile framework for wireless multimedia networks. IEEE/ACM Transactions on Networking 12(6), 1021–1035 (2004) 14. Levine, D.A., Akyildiz, I.F., Naghshineh, M.: A resource estimation and call admission algorithm for wireless multimedia networks using the shadow cluster concept. IEEE/ACM Transactions on Networking 5(1), 1–12 (1997) 15. Hou, J., Yang, J., Papavassiliou, S.: Integration of pricing with call admission control to meet QoS requirements in cellular networks. IEEE Transactions on Parallel and Distributed System 19(9), 898–910 (2002) 16. Zhang, T., et al.: Local predictive resource reservation for handoff in multimedia wireless IP networks. IEEE Journal on selected areas in Communications 19(10), 1931–1941 (2001) 17. Tsiligaridis, J., Acharya, R.: A clustering prediction scheme for wireless cellular network. In: Proc. of the 2005 International Symposium on Collaborative Technologies and Systems, pp. 298–304. IEEE Press, Los Alamitos (2005)

The Study of Forecasting Model of Rock Burst for Acoustic Emission Based on BP Neural Network and Catastrophe Theory Yunyun Xu1,2 and Dongqiang Xu2 1 2

Department of Civil Engineering of Tangshan College, Hebei Tangshan 063009, China College of Civil Engineering, Hebei University of Technology, Tianjin 300132, China

Abstract. Forecasting model of the rate of rock burst acoustic emission time series has been established by BP neural network, and it is combined with the catastrophe theory to determine whether the rock burst. And then the experimental data recorded are used for examining the model. The results show that the degree of prediction accuracy is high, and it proves that the prediction model of rock burst is feasible. Keywords: rock burst, acoustic emission, forecasting model, BP neural network, catastrophe theory.

1 Introduction Rock burst is a usual geologic disaster, which occurs in rock of underground works of high ground stress. The reason for rock burst is that stress field in wall rock redistributes, caused by unloading in the process of excavation, and it results in elastic strain energy stored in hard and brittle wall rock being released suddenly, and wall rock being burst, loosen, peeled, ejected or even thrown[1]. The process of rock burst preparation and development is that surrounding rocks in mining space or roadway deform rapidly and fracture, from flow to sudden change. Rock acoustic emission is an intuitive performance of rock micro breakdown, and it is caused by rock crack propagation under load [2]. Acoustic emission method is to predict the occurrence of rock burst based on the parameters in each stage of deformation such as acoustic emission rate, energy rate, and wave shape, and so on, which is the most direct means of detecting and predicting rock burst. Prediction of rock burst is to mainly determine the possibility of rock burst in a large area in the next period[3]. For a long time, the analysis of acoustic emission time series monitored and forecasting trends has become the primary means of predicting rock burst [4-5]. Gray system and the neural network model are the most important prediction methods [6]. But prediction results show that the gray model is suitable for exponential growth in time-series, and in dealing with geotechnical engineering timeseries prediction problem, gray model should be used with caution [7-8].In this paper, the rock burst prediction method by means of neural network theory and catastrophe theory is discussed on the basis of the test data of rock burst of acoustic emission. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 11–19. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

12

Y. Xu and D. Xu

2 The Establishment of Forecasting Model of Time Series Neural Network 2.1 The Introduction of BP Neural Network Error Back Propagation(BP) neural network is one of the more mature and widely used models. It is a former feed-forward network, mainly composed of input layers, hidden layers and output layers [9]. Network structure is shown in Fig.1.

Fig. 1. BP network topology

BP network is a kind of guiding learning network, which relies on adjusting the connection weights between layers of neurons (nodes) to "memory" the learning samples. And it automatically creates the highly nonlinear mapping relationship between the inputs and outputs by studying a large number of samples. It has been proved that a three-layer BP network includes m input nodes, 2m +1 hidden layer nodes and n output nodes, which can achieve any non-linear mapping from Rm to Rn [9]. 2.2 Process of Network Training Learning process of BP algorithm is composed of front-propagation and backpropagation. In the process of front-propagation, the input information is dealt with the hidden units layer by layer and transmitted to the output layers. If the output layer can not get the desired output, the back-propagation process is transferred, and the errors between actual value and network output return along the path to the original links. By modifying the link weights of every layer’s neurons, the errors reduce, and then the process of front- propagation is transferred, iterating until the error is less than the value given. (1) Design network structure to adapt to the issue (network layers and nodes of network layers), and choose reasonable network learning parameters (learning rate η and impulse parameters α ) . (2) Set the initial value of network connection weights and thresholds w ji (0) , θ j (0) , which are the random number in the range of [-1,+1].

The Study of Forecasting Model of Rock Burst for Acoustic Emission

13

p ( p = 1,2, " , K ) by the steps of (4) - (8). Sample data include input vector x = ( x1 , x 2 , " , x n ) and desired output vector (3) Deal with every study sample

t p = (t p1 , t p 2 , " , t pm ) . (4) Calculate the actual outputs of nodes layer by layer in forward direction. n

O pj = f j (∑ w ji O pj + θ j ) i =1

，

f ( x) =

1 1 + e−x

(1)

In the Formula 1, f (x ) is an active function of the S-type. (5) Calculate output network errors.

⎤ 1 k ⎡1 m E = ∑ ⎢ ∑ (t pj − O pj ) 2 ⎥ k p =1 ⎣ 2 j =1 ⎦ (6) If

(2)

E ≤ EG (the system average error margin) or iterations number reaches the

specified number, the study ends. Otherwise, the back-propagation turns to step (7). (7) Calculate reversely the error signal of each node in network layer by layer.

δ pj = O pj (1 − O pj )(t pj − O pj )

(3)

m

δ pj = O pj (1 − O pj )∑ δ pl w jl

(4)

l =1

(8) Modify the network connection weights and node thresholds.

Δw ji (t + 1) = η × δ pj × O pj + αΔw ji (t ) Δθ j (t + 1) = η × δ pj + αΔθ j (t )

，

(5)

(9) Turn to step (3). 2.3 Establishment of Time Series Neural NetworkForecasting Model Suppose that there is a time-series X (i ) , i = 1,2, " , N ; and N is the number of observation points. Geotechnical engineering time-series forecasting model can be described as the following:

X (t ) = ϕ[ X (t − 1), " , x(t − p )] In the formula, ϕ [.] is a non-linear function, and p is the order of the model. The building process of geotechnical engineering time-series forecasting model is to find ϕ [.] [10].

14

Y. Xu and D. Xu

From the time series X (i ) ,

i = 1,2, " , N , N − p samples can be constructed.

Input Layer Vector No1 Sample No t − p Sample No N − p Sample

Output Layer Vector

X (1), " , X ( p − 1), X ( p ) X (t − p ), " , X (t − 2), X (t − 1) X ( N − p ), " , X ( N − 2), X ( N − 1)

X ( p + 1) X (t ) X (N )

N − p samples constructed are used for the network shown in Fig.1 for learning and training, and then a stable network structure, connection weights and thresholds can be got. The time series forecasting model of geotechnical engineering is established based on artificial neural network. The input vectors are inputted into the network, using the stable network structure, connection weights and thresholds, so that the samples can be predicted.

3 Catastrophe Cusp Model and Yield Mechanism Cusp theory of the catastrophe theory is the most widely used in the range of mechanics [11]. It has two control variables and a state variable. Its regular function of potential function in the form is as the following:

v ( x) = x 4 + px 2 + qx

(6)

So phase space is three-dimensional, in which x is the state variable, p and q are control variables, and x is the time variable. The corresponding set M of all critical points for E, is called the equilibrium surface (shown in Fig. 2). M satisfying:

grad xV = 4 x 3 + 2 px + q = 0

Fig. 2. Balance surface and control surface

(7)

The Study of Forecasting Model of Rock Burst for Acoustic Emission

15

The graphics of M in ( x, p, q ) space is a smooth surface with a fold, so in different regions, the number of equilibrium position is different. It is easy to be proved that the potential function corresponding to the middle lobe takes the maximum (that is to say, grad x ( grad xV ) < 0 ), whose equilibrium position is unstable; the potential function corresponding to the upper and lower lobe takes the minimum, whose equilibrium position is stable. Clearly, there are vertical tangent points at the curve surface, that is, near the point to satisfy Formula 8 in Fig.2, the number of equilibrium position is different. These points are called sudden change points or singular points, whose projection at the control variables p, q plane constitutes a bifurcation set B (also called bifurcation). This set includes all the state variables, and makes x produce a sudden change.

8 p 3 + 27q 2 = 0 grad x ( grad xV ) = 12 x 2 + 2 p = 0 Eliminate

(8)

x from Formula7 and Formula 8, and the equation is got: 8 p 3 + 27q 2 = 0

(9)

Through the above analysis, the instability discriminant can be drawn as the following:

F = 8 p 3 + 27 q 2

(10)

If control point ( p, q ) changes, corresponding point position changes on the curve surface accordingly. But when the control points trajectory cross over the bifurcation set 8 p + 27 q = 0 , corresponding points must jump across the middle of leaves, that is, rock mass is unstable. The relation is described as the following: 3

If If If

2

F > 0 , the rock mass near measuring point is steady; F = 0 , the rock mass near measuring point is critical; F < 0 , the rock mass near measuring point is unstable.

From Formula 7, it can be seen that the number sequences increase in the form of an exponent, but this kind of exponential function is not suitable for the requirements of catastrophe theory. Because any single variable function can be spread by the Taylor formula, v(t ) can be spread by the Taylor series, and truncated to 5. Then we can get: 4

v(t ) = ∑ ai t i i =0

(11)

16

Y. Xu and D. Xu

∂ i v(t ) In Formula 11, a i = | t =0 , and suppose x = t − n by using of variable ∂t i substitution, we can get:

v( x) = x 4 + px 2 + qx

(12)

For the consistency of the neural network model and the mutation model, it can be obtained:

p = −6n 2 + a 2 / a 5 n = a3 / 5a5

(13)

p = −6 n 2 + a 2 / a 5

(14)

q = 8n 2 − 2na 2 / a 5 + a1 / a 5

(15)

Inserting AE parameters at the moment to predict in Formula 12, adopting the leastsquares method, we can get a i . And then, inserting a i in Formula 13 to Formula 15, we can get the value of p and q , which can be inserted in Formula 10 to calculate the value of F. At last, we can forecast the rock burst circumstances based on the value of F.

4 Application of BP Neural Network – Mutation Model in Rock Burst Prediction of Acoustic Emission To verify the accuracy of the neural network - mutation model in rock burst prediction, we use the data recorded on the experimental to test the model. The white marble specimens, whose integrity is good, are used for testing. In the entire failure process of the specimen, the acoustic emission event rate has a bimodal nature and when the stress reaches about 80% of ultimate stress, it reaches high-value soon until the stress reaches 95% ultimate stress or so. During this time from 80% to 95% ultimate stress, the acoustic emission event rate fluctuations little and has an upward trend. The acoustic emission parameters of white marble specimens after 85% of ultimate stress for some time are shown in Table 1 for every 5 seconds. The corresponding data of rate of acoustic emission time series at 190-260second are used as a model; and the corresponding data of rate of acoustic emission time series at 265-270second are used for testing. We insert the modeling data in the network to operate. The model order is 3, the input layer nodes number using the network is 2, hidden layer node number s is 5, and output layer nodes number is 1. After 15000 iterations, the network converges, and the steady network structure,

The Study of Forecasting Model of Rock Burst for Acoustic Emission

17

connection weights and threshold are got. The fitted and measured values are compared in Table 1. Table 1. Comparison of fitted and measured values Monitoring Time (s)

Monitoring value-AE event rate (time/m)

Predicting value-AE event rate (time/m)

Relative error (%)

190

85

87

2.35

195

90

93

-3.23

200

100

100

0

205

109

103

5.50

210

98

105

-7.14

215

105

107

-1.90

220

111

108

-0.93

225

113

110

2.70

230

109

112

-2.75

235

115

113

1.74

250

115

118

2.60

255

118

120

-1.69

260

124

126

-1.61

Table 2. Predictions comparison Monitoring time (s)

Measured value

Prediction value

Relative error%

265

129

133

-3.10

270

135

132

2.22

From Table 1, the neural network fitting is good, and the relative average error is only 2.62%. The rates of acoustic emission at 265s and 270s are predicted, and compared with the record values to test the correctness of the model. Those results are shown in Table2. It can be seen that for the samples without training, the neural network model has very high prediction accuracy and it is an effective way to solve the problem of rock burst time series. Use AE parameters shown in Table1 at the moment to calculate the value of F for predicting the burst of rock. The result is shown in Table 3.

18

Y. Xu and D. Xu Table 3. Rock burst prediction

Time(s)

p

q

F

Rock burst prediction

245

-5.7

51.69

56096.9

No burst

250

-5.65

-58.55

91091.9

No burst

255

-6.65

-29.30

20837.2

No burst

260

-7.695

-105.36

290512.1

No burst

265

-98.81

-35.08

-7686526.1

burst

From the values shown in Table 3, it can be seen that rock may burst in holes at 265 seconds when the specimens is under load. In reality, at 269 seconds the rock fragments in holes appear, followed by rupture of the specimens. It shows that the forecasting results are very consistent with the actual situation. Several other specimens are used to predict, and the forecast is successful.

5 Conclusion The studies show that artificial neural network method for prediction of rock burst has provided a new and effective method. In this paper, the forecasting model of the rate of rock burst acoustic emission time series has been established by using BP neural network, and it was combined with the catastrophe theory to determine whether the rock burst. The results show that the degree of prediction accuracy is high, and it proves that the prediction model of rock burst is feasible, and has important application value. The model described in this article can forecast the occurrence of rock burst, but it can not predict the level and scale of rock burst. Many factors must be considered comprehensively to predict rock burst, so it is necessary to develop rock burst prediction experts system to grasp comprehensively and accurately the laws of rock burst, and to take appropriate measures to prevent rock burst.

References 1. Wang, W., W.E.Y.: Summary and Development Analysis of Rock Burst Prediction Technology (in Chinese). Mining Safety and Environmental Protection 30(4), 12–15 (2003) 2. Zhang, Y.B., Kang, Z.Q., Jiang, G.H., Xu, D.Q.: Rock Burst Mechanism Based on Rock Damage and Acoustic Emission Theory (in Chinese). Metal Mine 12, 79–82 (2007) 3. Pan, C.L., Cao, X.B., Cao, P.: Rock Burst Prediction Method (in Chinese). Non-ferrous Mining and Metallurgy 13(6), 3–5 (1997)

The Study of Forecasting Model of Rock Burst for Acoustic Emission

19

4. Tang, Q.B., Cao, P.: The Analysis of Rock Burst Prediction Method (in Chinese). Jiang Xi Nonferrous Metals 6, 2–4 (1997) 5. Yang, T., Li, G.W.: Study Based on Prior Knowledge of the Rock Burst Prediction (in Chinese). Chinese Journal of Rock Mechanics and Engineering 19(4), 429–431 (2000) 6. Peng, Q., Zhang, R., Xie, H.P.: Prediction Model for Rock Burst Based on Acoustic Emission Time Series (in Chinese). Rock and Soil Mechanic 30(5), 1436–1440 (2009) 7. Lu, X.Q., Liang, X.H., Lu, X.J.: Neural Network and Its Application to Predication of Nonlinear Time Sequence (in Chinese). Systems Engineering-theory and Practice 6, 97–99 (1997) 8. Zhang, Y.X., Lu, S.L.: Application of Artificial Neural Network to Engineering Time Sequence Forecasting Problem (in Chinese). Journal of NanJing University of Science and Technology 21(6), 522–525 (1997) 9. Li, X.A., Zhang, X.H.: Introduction to Neural Network and Neural Computation (in Chinese). Northwestern Polytechnical University Press, Xi an (1994) 10. Zhang, Y.X.: Primary Research on Forecasting Problem of Time Sequence in Geotechnical Engineering (in Chinese). Chinese Journal of Rock Mechanics and Engineering 17(5), 552–558 (1998) 11. Ling, F.H.: Catastrophe Theory and Its Application (in Chinese). Shanghai Jiaotong University Press, Shang Hai (1987)

Application of Wavelet Neural Network to Prediction of Water Content in Crude Oil Huifen Niu1, Cuiling Liu1, Jinqi Wang2, and Xiaowen Sun1 1

Computer and Information Engineering College, Beijing Technology and Business University, 100048 Beijing, China 2 Earth Sciences College, Daqing Petroleum Institute, 163318 Daqing, China [email protected], [email protected], [email protected], [email protected]

Abstract. In the course of oilfield development, accurate measurement of water content in crude oil has always been playing important role in practicing development adjustment and enhancing the effects of stimulation operation, and moreover it determines the development perspectives of oilfield. After research of the method of measuring the rate of water content, and the non-linear mapping relation between the rate of water content and impact factors of crude oil, a model of predicting the rate of water content in crude oil about vertical well based on wavelet neural network is proposed. Results of the simulation in MATLAB indicated that the way of WNN (wavelet neural network) has better convergent rate, prediction precision, learning ability and generalization ability than the traditional BP neural network. The method of WNN can predict the water content in crude oil with high precision and it owns much more powerful theoretical guide and much better application effects. It will have a broad application in the future. Keywords: wavelet neural network, BP neural network, crude oil, water content, prediction model.

1 Introduction The water content of oil is the parameter which is not only important in oil gas and oilfield measure of production, but also indispensable in researching the development perspectives of oilfield. If the inspection accuracy of the rate of water content in crude oil can be improved, the production parameters will be more optimized and production accounting will be more reasonable during the storage and transportation process of the oilfield and oil gas[1]. Lots of work on the rate of water content in crude oil has been researched by many domestic and foreign scholars for a long time. In order to improve the measure precision, many kinds of methods have been proposed and many types of equipment about testing the rate of the water content have been designed. But it is hard to set up accurate mathematical models for all situations due to the complex nonlinear relationship of the factors which cause the effect of measuring the rate of the water content in crude and complicated work conditions. With the rapid development of computer and artificial intelligence technology, the prediction models about the rate of Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 21–27. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

22

H. Niu et al.

the water content in crude oil based on ANN(artificial neural network) have been put forward recently[2]. The ANN is suitable for the non-linear prediction, and in theory it can approach the non-linear sequences in any precision. But the ANN is hard to decide the structure of the net in the application, and it like slow convergence of learning and liability of dropping into local minimum [3]. The WNN has been a new analysis method of modeling in recent years, which is established with the combination of the wavelet transformation and the traditional ANN thoughts. It is the progression formed by transformation of shifting and dilation with wavelet decomposition, which has the same attribute of approximating function as wavelet decomposition. Meanwhile, because of introducing two new variables named non-integer dilation factor and shift factor, it has more freedom of motion than wavelet transform, more flexible and effective function approximation capability. It can effectively overcome some defects of normal ANN models. In this paper, after intensive analysis and research on the data of flux and the water content which get from multi-sensors, a prediction model of water content in crude oil based on WNN is established, and compared with the result of the traditional BP neural network.

2 Water Content in Crude Oil Measurement System The oil-water mixture flows between the inner conductor and the outer conductor of the coaxial line sensor, which is used as the measuring device. It also becomes the carrier of the spread of electromagnetic energy. Choosing appropriate parameters of the sensor can ensure the electromagnetic wave in the coaxial line propagate in the form of TEM (Transverse Electromagnetic). It can be got the message of water content by measuring the phase characteristics of the electromagnetic waves spreading in the coaxial line sensor, changing the characteristics of the voltage signal into a frequency signal, and then using the signal amplification circuit and A/D converter. The object of the measurement is well mixed wet crude, whose water content is from 0% to 100% and its gas content is certain[4-6]. The Coaxial-Phase Technology crude oil water content measurement system diagram is shown in Fig. 1.

Fig. 1. Schematic Diagram of Water Content in Crude Oil Measurement System

Application of Wavelet Neural Network to Prediction of Water Content in Crude Oil

23

3 Wavelet Neural Network Model 3.1 Basic Theory The wavelet neural network uses the shifting and dilation of the mother wavelet to constitute the wavelet base, and takes the wavelet as the excitation function of a hidden lay. This network takes the attribute of evaluation index which is used to describe the attribute of the evaluation object as the input vector of the network. And it takes the corresponding evaluation as the output vector of the network. It can batchprocessing by using the conjugate gradient method, and adjust the wavelet exponential and neural weight by self-adapting. Use enough samples to training the neural network, and through the study of the samples the wavelet neural network like a “black box”, it saved the experience of the expertise and some mathematical inference mechanism. And it established the nonlinear mapping relation from the attribute of the evaluation object to the output. In this way, we can use the wavelet which has been trained to evaluate the new object. 3.2 Wavelet Neural Network If towards ψ ( t ) ∈ L2 ( R ) and Ψ ( ω ) which is the Fourier transformation of ψ (t ) meet the admissibility condition of wavelet function[7]: cψ

=

∫

+∞

ω

− ∞

− 1

Ψ (ω )

2

dω

< ∞

(1)

Then ψ (t ) is be called basic wavelet, and it also referred to as mother wavelet. After

the ψ (t ) shift b, then flex a in different delay, it generates the Wavelet bases: ψ

a ,b

(t ) =

⎛ t − b ⎞ ⎟ a ⎝ ⎠

1

ψ ⎜

a

(2)

Where, a, b——the dilation factor and shift factor ofψ a ,b (t ) , a, b ∈ R , a ≠ 0 . Given non-linear time sequence transformation function as wavelet transformation is: 1

WT x ( a , b ) ≤ x (t ),ψ a ,b (t ) ≥

a

∫

+∞

−∞

⎛t −b⎞ x (t )ψ ⎜ ⎟dt ⎝ a ⎠

x(t ) ∈ L2 ( R) , the

(3)

The water content in crude oil sequence can be obtained through the method which is linear superposition, use the finite term of the wavelet levels to approach the real testing data of the water content of crude oil: g (x) =

L

∑

k =1

⎛ ⎜ w kψ ⎜ ⎜ ⎜ ⎝

n

∑

i=1

u

ki

⎞ x (i) − bk ⎟ ⎟ ⎟ ak ⎟ ⎠

(4)

24

H. Niu et al.

Where, g(x)—the prediction value of y; u ki ,

wk —weight coefficient; ak , bk —the

dilation factor and shift factor; n—dimension of input vector; L—number of wavelet basis. The structural diagram of wavelet neural network is shown in Fig. 2 (combined with the actual needs, here gives the situation of two input vectors and one output vector).

Fig. 2. Structural Diagram of Wavelet Neural Network

For N samples of the input and output that is (xt, yt)( t=1, 2, …N), the aim is to

u ki , wk , ak , bk and L to make the gt(x) and the yt u is best fitting. The network parameter ki can be optimized by the energy function of

make sure the network parameter

minimum mean-square error: E

= 0 . 5 [y t − g t ( x ) ]

2

t

(5)

The wavelet basis adopt the Morlet mother wavelet which is : ψ ( T ) = cos( 1 . 75 T ) exp( − 0 . 5 T

2

(6)

)

This wavelet is gaussian wave which is cosine modulation, it has high precision and strong antijamming feature. The calculating method in detail of this network is introduced as follow: 1. 2. 3.

Initialization of the network parameters; Input the training samples xt, and matching desired output value yt; The network self-learning, use the current network parameter to figure out the output of the network:

g

t

( x ) =

L

∑

k = 1

w

k

ψ

⎛ ⎜ ⎜ ⎜ ⎜ ⎝

n

∑

u

ki

x (i ) −

i = 1

a

k

b

k

⎞ ⎟ ⎟ ⎟ ⎟ ⎠

(7)

Application of Wavelet Neural Network to Prediction of Water Content in Crude Oil

4. 5. 6.

Figure out the instantaneous value of gradient vector; Back propagation, modify the network parameter; Calculation the error N

E = 0 .5 ∑

t =1

7.

25

[g t ( x ) −

yt

]2

(8)

If E < E max (one positive value has be preset) or it gets the maximum train epochs, then stop training; else order E=0, return to step (2).

4 The Prediction Model Simulation Prepare oil samples with calibration water content using artificial ration, and then use the three-phase flow dynamic mixed devices to simulate the actual oil well. According to the nonlinear mapping relation between the water content and its influencing factors, taking the frequencies of the coaxial line water detector based on phase and the measured value of the turbine flowmeter as the input vectors of the WNN, taking corresponding known water content in crude oil as the output vector. Get the node number of latent layer by auto-optimization function of the WNN. So the node number of latent layer is intended to be 8 wavelet function to build the WNN prediction model. In order to increase the precision of prediction, we need normalization processing the data firstly by using function (7), because the water content is between 0~1, so it does not need to normalization[8]. x i' =

x i − x min + 0 . 1 x max − x min + 0 . 1

(9)

Choose the former 44 groups of the known data as the training samples, and the remaining 11 groups as the testing samples. Momentum coefficient λ =0.5, learning rate η =0.3, learning error E=0.01, and simulate in matlab7.1. In order to make known the effects of the prediction of WNN, this paper also use traditional BP to predict the water content in crude oil, and compared these two methods. Fig. 3 and Fig. 4 are the fitting chart between the real data and the data predicted by BP and the fitting chart between the true value and the prediction value of WNN. From these two figures we can see that the data predicted by WNN is much more approach the real data. Tab. is the simulated result of BP and the simulated result of WNN. From table 1, we can see that the relative error and mean square error (MSE) of the prediction model based on WNN is much smaller than the relative error and mean square error (MSE) of the prediction model based on BP, the precision of prediction has been remarkably improved. So, we can find out that the predict precision and the data fitting of the prediction model of water content in crude oil based on WNN are both better than the model based on BP neural network.

Ⅰ

26

H. Niu et al.

Fig. 3. The fitting chart between the true value and the prediction value of BP

Fig. 4. The fitting chart between the true value and the prediction value of WNN

Application of Wavelet Neural Network to Prediction of Water Content in Crude Oil

27

Table 1. The Simulated Result of BP Neural Network and The Simulated Result of WNN

number

Measured of water content%

1 2 3 4 6 6 7 8 9 10 11

0 10 20 30 40 50 60 70 80 90 100

BP neural network Prediction Relative of water error% content% 3.10 ∞ 13.56 35.6 24.46 22.3 40.32 34.4 48.62 21.5 53.75 7.5 57.73 5.4 63.16 9.8 70.13 12.3 78.13 13.2 102.16 2.2

MSE

0.5224

WNN neural network Prediction Relative of water error% content% 0 0.0 10. 90 9.0 17.39 13.1 33.12 11.0 37.08 7.3 52.32 4.6 63.12 5.2 69.18 1.2 82.96 3.7 92.12 2.4 101.14 1.14

MSE

0.0562

5 Conclusion This paper proposed the prediction model of water content in crude oil based on wavelet neural network, use the nonlinear wavelet basis replace the excitation function of the neuron in neural network. It combines advantage of the wavelet and the traditional neural network, and through the calculation function such as flexible and shifting to multi-scale analysis the data. The result of the prediction shows that the wavelet neural network prediction model has the following advantages: much better convergence and higher prediction precision than BP network, and have stronger capability of study and popularize. So it can provide the on-line prediction and has a broad application.

References 1. Ming-yi, C., Xin-rong, Y.: Information Merge Technology of Multiple Transducers and Its Application in Moisture Measurement of Oil Products. J. Process Automation Instrumentation 23(8), 2–3 (2002) 2. Shun-tian, L., Yang, S.: The Systems Analysis and Design Based on Matlab-Neural Network. Xidian University Press, Xi’an (1999) 3. Tsaih, R., Hsu, Y.-s., Lai, C.C.: Forecasting S&P 500 Stock Index: Futures with a Hybrid AI System. J. Decision Support System, 161–174 (1998) 4. Jin-qi, W., Xi-fu, Q., Kui-yong, Z.: Coaxial Transmission Line Phase Method for Measuring Water Content of Oil Well. J. Chinese Journal of Scientific Instrument 23(1), 75–76 (2002) 5. Jin-qi, W., Xi-fu, Q., Jian-ming, C.: Fhase measurement of water content in oil well. J. Journal of Harbin Institute of Technology 34(2), 245–247 (2002) 6. Jin-qi, W., Xi-fu, Q., Ying-hua, Y.: Test Study of Water Cut Tool in Oil Well Based on Phase Method. Acta Metrologica Sinica 25(4), 366–368 (2004) 7. Xin, W., Lu, Z., Xiang, L.: Simulation and Application of MATLAB Neural Network. Science Press (2003) 8. Li-qun, H.: Artifical Neural Network Course Book. Beijng University of Posts and Telecommunications Press, Beijing (2006)

Prediction of Urban Heat Island Intensity in Chuxiong City with Backpropagation Neural Network Wujun Xi and Ping He Department of Geography and Tourism Management, Chuxiong Normal University, Chuxiong, China [email protected], [email protected]

Abstract. In order to improve prediction accuracy of urban heat island intensity, we chose 9 main influencing factors from 1981 to 2006 and predicted urban heat island intensity in Chuxiong city in 2006 with backpropagation neural network. The predicted value was 2.1815 . Compared with its measured value, residual error was 0.1042, relative error was 4.5588%. It was superior to results of SCGM model and GM(1,1) model. The result shows that backpropagation neural network is effective to predict urban heat island intensity.

℃

Keywords: Backpropagation neural network, Prediction, Urban heat Island intensity, Chuxiong.

1 Introduction An urban heat island (UHI) is an urban area which is significantly warmer than its surrounding rural areas [1]. Urban heat island is one of the most important features of urban climate and atmospheric environment, urban human factors and local meteorological condition cause urban heat island to form [2]. In 1820, The British chemist Lake Howard put forward urban heat island firstly, by observing the urban centre and surrounding town areas [3]. After that, many researchers paid great attention to it, and the attention increased in recent years (Figure 1).

Fig. 1. Academic attention to urban heat island effect in China from 1996 to 2008

Data source: www.cnki.net. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 29–36. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

30

W. Xi and P. He

Correlation study included urban heat island intensity[4-5], Sequential variation of urban heat island[6-7], spatial distribution characteristic and pattern of urban heat island [8-9], monitoring method of urban heat island[10-11], influencing factors of urban heat island[12-15], formation mechanism of urban heat island[16], influence of urban heat island and countermeasures[17-18]. However, the researches of prediction of urban heat island intensity were relatively less, and the method was mainly regression and gray system [18-21]. Professor Zhi CHEN predicted urban heat island intensity in Xian with SCGM model, the relative error was 5.0% [20]. Professor Ping HE predicted urban heat island intensity in Chuxiong with GM(1,1) model, the relative error was 7.97%[21]. When they predicted urban heat island intensity by using regression and gray system, they commonly used urban heat island intensity data, but not considered other factors. In order to remedy the monotony of them, we used backpropagation neural network to synthesize the relevant influencing factors to predict urban heat island intensity in Chuxiong city.

2 A Survey of Chuxiong City 2.1 Natural Situation of Chuxiong City Chuxiong city locates on North-central of Yunnan Province, 101.32°E, 25.01°N, the land area of 4,433 square kilometers, among which, the built-up areas is 23 square kilometers. It has a population of 500,000. The altitude is between 1700 meters to 2300 meters. Its mean annual temperature is 15.6 , and average annual rainfall is 864 millimeters. Chuxiong belongs to subtropical monsoon climate [22].

℃

2.2 Sequential Variation of Urban Heat Island in Chuxiong City According to the professor Ping He's research [6], in the years prior to 1988, the urban heat island intensity in Chuxiong city was below 1 , since then, the intensity was going up over 1 . It increased quickly and reached 2 in 2006 (Figure 2).

℃

℃ ℃

Fig. 2. Variation of annual heat island intensity in Chuxiong city from 1959 to 2000

The maximum value of heat island intensity in Chuxiong city from 1980 to 2000 happened in spring (from March to April), the minimum happened in summer (from June to July) (Figure 3).

Prediction of Urban Heat Island Intensity in Chuxiong City

31

Fig. 3. Variation of monthly heat island intensity in Chuxiong city

3 Prediction of Urban Heat Island Intensity in Chuxiong City with Backpropagation Neural Network 3.1 Influencing Factors of Urban Heat Island in Chuxiong City Because urban human factors and local meteorological condition cause urban heat island to form, so influence factors of urban heat island include relative humidity, air pressure, aqueous vapor pressure, wind velocity, low cloud amount, total cloud amount, sunshine duration, prevailing wind, greenhouse gases, atmospheric pollution, urban underlying surface, urban population, power consumption, coal consumption, number of motor vehicles, industrial pattern, land use type, density of population, water area, road area, building density and urban green coverage ratio, etc[20,21,24,25]. Considering influencing factors’relevancy and data’s acquisition, we chose 4 meteorological elements (relative humidity, wind velocity, total cloud amount, sunshine duration) and 5 human factors (urban population, urban underlying surface area, power consumption, number of motor vehicles and urban green coverage ratio) to analyze urban heat island intensity in Chuxiong city. 3.2 Prediction Process with Backpropagation Neural Network Data preparation. Convert 9 factors and urban heat island intensity data in Chuxiong city from 1981 to 2006 into non-dimensional quantities, all factors’ measured values as reference value 1. Table 1 was the processing results. Table 1. The influencing factors’ values of urban heat island intensity in Chuxiong city from 1981 to 2006 Year

Urban heat island intensity

Urban population

Urban underlying surface area

Urban green coverage ratio

Number of motor vehicles

1981

1.0000

1.0000

1.0000

1.0000

1.0000

1982

1.0000

0.9677

0.9362

1.0784

0.9928

1983

1.0000

1.0622

0.8511

1.0963

1.0870

1984

0.8571

1.2191

1.0851

0.9940

1.1726

1985

0.8571

1.1660

1.0638

1.0273

1.4039

1986

1.4286

1.2674

1.0638

1.1671

1.7280

32

W. Xi and P. He Table 1. (continued) 1987

2.2857

1.2885

1.0638

1.0554

1.9845

1988

2.0000

1.3226

1.3617

1.0656

2.3234

1989

2.1429

1.4746

1.3617

1.0656

2.4559

1990

2.1429

1.5572

1.3191

1.0656

2.8267

1991

2.1429

1.5499

1.7234

1.0656

3.0661

1992

2.1429

1.6971

1.7234

1.2447

3.4622

1993

2.8571

1.7372

2.6809

1.1679

3.9689

1994

2.7143

1.9483

2.8085

1.1765

4.0659

1995

2.5714

1.8281

2.8936

2.1381

4.8725

1996

2.8571

1.9296

2.8723

2.0946

6.1463

1997

3.2857

1.9751

3.0000

2.1475

6.8072

1998

3.0000

2.0821

2.8723

1.8824

7.0374

1999

2.8571

2.1664

2.8723

2.0350

8.0073

2000

2.8571

2.2027

3.5745

1.9241

8.2236

2001

2.5714

2.3715

3.0851

2.0716

8.4146

2002

3.0000

2.2386

3.6170

2.1824

9.0101

2003

2.2857

2.3707

3.6383

2.1483

9.9578

2004

1.8571

2.3615

3.8298

2.1560

11.6458

2005

2.1429

2.4351

4.4681

2.2165

14.0174

2006

2.2857

2.7815

4.2553

2.1876

15.1821

Year 1981

Power consumption 1.0000

Sunshine duration 1.0000

Wind velocity 1.0000

Relative humidity 1.0000

Total cloud amount 1.0000

1982

0.9954

0.8590

1.0000

1.0290

0.9474

1983

1.1382

0.7948

1.0625

1.0145

1.0877

1984

1.1648

0.9771

1.0625

1.0000

0.8421

1985

1.2453

0.8484

0.9375

1.0435

1.0351

1986

1.4415

0.9158

0.9375

0.9565

0.9123

1987

1.3941

0.9151

1.0000

0.9565

0.8772

1988

1.5710

0.8775

1.0000

0.9565

1.0351

1989

1.6595

0.8588

0.7500

0.9565

0.9825

1990

1.8474

0.6352

0.6875

1.0580

1.1053

1991

1.9153

0.8053

0.6875

0.9855

0.9123

1992

2.3170

0.7183

0.8750

1.0000

0.9649

1993

2.1374

0.7223

0.7500

0.9855

0.9649

1994

2.4260

0.8171

0.7500

1.0000

0.8947

Prediction of Urban Heat Island Intensity in Chuxiong City

33

Table 1. (continued) 1995

2.5505

0.7704

0.9375

0.9710

1.0702

1996

3.0175

0.7759

0.8750

1.0435

0.9649

1997

2.8170

0.8852

0.8125

0.9710

1.0000

1998

3.0634

0.7654

0.6875

0.9710

0.9474

1999

3.4123

0.7150

0.6250

0.9855

1.0877

2000

3.3011

0.6787

0.6250

0.9710

1.0702

2001

3.4886

0.6144

0.5625

1.0145

1.2105

2002

3.9863

0.6812

0.4375

0.9855

0.9649

2003

3.9070

0.8734

1.3750

0.9420

0.8070

2004

4.5524

0.6433

1.3750

0.9130

1.0526

2005

5.7407

0.6713

1.5625

0.8841

1.0526

2006

7.3056

0.7999

1.3750

0.9275

1.0175

Data source: Chuxiong city yearbook (1994-2007), Lucheng town statistical station.

Prediction with backpropagation neural network. Analysis tool was DPS Data processing system [23], it provided a rapid way to compute with backpropagation neural network. First, we imported all non-dimensional quantities into DPS Data processing system, then made backpropagation neural network analyze with 9 influencing factors and urban heat island intensity from 1981 to 2005. After repeated experiments, we finally determined essential parameters: the number of hidden network Layer was 1, minimum training ratio was 0.1, dynamic parameter was 0.4, sigmoid was 0.9, error excepted was 0.0001, maximum iteration number was 5000. When iteration number reached about 2300, the result matched error excepted, and the computation stopped, fitting residual error was 0.00094(Figure 4).

Fig. 4. Backpropagation neural network fitting curve

34

W. Xi and P. He

We predicted urban heat island intensity in Chuxiong city with 9 influencing factors in 2006 as observed data, the predicted value was 2.1815 . Compared with the measured value of urban heat island intensity in Chuxiong city in 2006, residual error was 0.1042, relative error was 4.5588%.

℃

3.3 Predicted Result and Analysis Compared with these researchers’ predict results[20,21], our predicted relative error was only 4.5588%. It showed that backpropagation neural network is effective to predict urban heat island intensity. The result is superior to results of SCGM model and GM(1,1) model.

4 Conclusion Prediction the development tendency of urban heat island intensity is more important now. However, the prediction accuracy was not very satisfied. In order to improve prediction accuracy of urban heat island intensity, we chose 9 influencing factors of urban heat island as main influencing factors by analyzing the urban heat island situation in Chuxiong city, the factors included relative humidity, wind velocity, total cloud amount, sunshine duration, urban population, urban underlying surface area, power consumption, number of motor vehicles and urban green coverage ratio. We made our backpropagation neural network analysis with 9 influencing factors and urban heat island intensity from 1981 to 2005, and predicted urban heat island intensity in Chuxiong city with these factors in 2006 as observed data, the predicted value was 2.1815 . Compared with the measured value of urban heat island intensity in Chuxiong city in 2006, the residual error was 0.1042, and the relative error was 4.5588%. Compared with the mentioned researchers’ predict results, our result is superior to the results of SCGM model and GM(1,1) model. It shows that backpropagation neural network is effective to predict urban heat island intensity, and it used many kinds of influencing factors synthetically to analyze urban heat island intensity. It has remedied the defect of predicting with unitary urban heat island intensity values, and has promoted the prediction accuracy.

℃

Acknowledgments. This study was supported by the National Natural Science Foundation of China (Grant No. 40965002), and Natural Science Foundation of Yunnan province (Grant No. 2005D0077M). And Hui Chen made a contribution to data collection.

References 1. Han, S., Guo, J., Huang, S., Bian, H.: Study on the Evolution of Urban Heat Island in Tianjin city (in Chinese). Ecology and Environment 16(2), 280–284 (2007) 2. Zhou, S., Shu, J.: Urban Climatology (in Chinese), p. 315. China Meteorological Press, Beijing (1994) 3. Howard, L.: Climate Deduced from Meteorological, vol. 1(3), pp. 1–24. Harvey and Darton, London (1833)

Prediction of Urban Heat Island Intensity in Chuxiong City

35

4. Wang, J., Wang, K., Wang, P.: Urban Heat (or Cool) Island over Beijing from Modis Land Surface Temperature (in Chinese). Journal of Remote Sensing 11(3), 330–339 (2007) 5. Li, F., Ma, A., Ding, Y., Yang, J., Jiao, J., Liu, L.: Research on Urban Heat Island Effect Based on Landsat Data (in Chinese). Remote Sensing Technology and Application 24(4), 553–558 (2009) 6. He, P., Li, H., Shu, J., Lu, X., Sun, J.: The Analysis of the Urban Climate in Chuxiong of Yunnan: Comparison of Urban Heat Island Effect in China (in Chinese). ACTA Geographica Sinica 58(5), 712–720 (2003) 7. Song, Y., Zhang, S.: The Study on Heat Island Effect in Beijing During Last 40 Years. Chinese Journal of Eco-Agriculture 11(4), 126–129 (2003) 8. Zhang, Y., He, Y., Ma, Y., Liu, Y., Li, Y., Dou, J., et al.: Characteristics of Vertical Distribution of Urban Heat Island Effect in Kunming City (in Chinese). Plateau Meteorology 21(6), 604–609 (2002) 9. Zheng, Z., Liu, W., Wang, Y.: Distributive Character of Urban Heat Island Effect in the Beijing Region (in Chinese). Journal of Nanjing Institute of Meteorology 29(5), 694–699 (2006) 10. Jian, W., Samei, S.: Research on the Method of Monitoring the Urban Heat Island by Remote Sensing Technique of Satellite (in Chinese). Modern Scientific Instruments (12), 27–29 (2004) 11. Rongbo, X., Ouyang, Z., Zhaoming, Z., Xiaoke, W., Weifeng, L., Hua, Z.: Advances in Methodology of Urban Heat Island (in Chinese). Meteorological Monthly 31(11), 3–6 (2005) 12. Cheng, C., Wu, N., Guo, S., Li, S., Liu, D.: A Study on the Interaction Between Urban Heat Island and Vegetation Theory, Methodology, and Case Study (in Chinese). Research of Soil and Water Conservation 11(3), 172–174 (2004) 13. Huang, J., Liu, X., Li, Q.: The Study Of Relationship Between Heat Island Effect and Population in Cities Over South of China (in Chinese). Journal of Tropical Meteorology 20(6), 713–722 (2004) 14. Ma, Y., Tiyip, T., Huang, Y., Yang, J.-L.: Effects of Landscape Pattern Change on Urban Heat-island Effect in Arid Areas—A Case Study in Urumqi (in Chinese). Arid Zone Research 23(1), 172–176 (2006) 15. Zhou, K., Ye, Y., Peng, S., Su, J.: Total suspended particulates and its effects on urban heat island (in Chinese). Ecology and Environment 15(2), 381–385 (2006) 16. Dai, X., Zhang, L., Guo, Z., Wu, J., Li, X., Zhu, Y.: Mechanism of formation of urban heat island effect and its spatial pattern in Shanghai (in Chinese). Acta Ecologica Sinica 29(7), 3995–4004 (2009) 17. Zhu, Z., Wang, M.: The Harms and Countermeasures of Urban Heat Island Effect (in Chinese). Pollution Control Technology 22(2), 94–96 (2009) 18. Wen, J., Bao, J., Zhang, Z.: Ecological Measure Analysis of Reducing Urban Heat Island (in Chinese). Ecological Economy (2), 151–153 (2008) 19. Wei, H., Zhao, Y., Chen, Y.: Grey Assessment and Prediction of the Urban Heat Island Effect in Lanzhou City (in Chinese). Journal of Desert Research 29(3), 571–573 (2009) 20. Zhi, C., Bingfeng, Y., Wangyang, H., Xilian, L., Linxiang, Q.: Grey Assessment and Prediction of the Urban Heat Island Effect in City (in Chinese). Journal of Xi’an Jiaotong University 38(9), 985–988 (2004) 21. He, P., Chen, H., Li, H., Xi, W.: Grey Analysis of the Urban Heat Island Effect Factors of the Medium-sized City of Chuxiong on Yunnan Plateau (in Chinese). Progress in Geography 28(1), 25–32 (2009)

36

W. Xi and P. He

22. Chuxiong Online . A Survey of Chuxiong, http://www.cxs.gov.cn/file_read.aspx?id=14944 23. Tang, Q.Y., Feng, M.G.: DPS Data processing system: Experimental design, statistical analysis, and data mining (in Chinese), pp. 932–935. Science Press, Beijing (2007) 24. Li, Z., Gong, C., Hu, Y., Yin, Q., Kuang, D.: The Progress of the Remote Sensing Research on Urban Heat Island (in Chinese). Remote Sensing Information (8), 100–105 (2009) 25. Wang, X., Bai, J.: The Development Prospect of the Urban Heat Island Effect (in Chinese). Journal of Foshan University (Natural Science Edition) 26(1), 53–56 (2008)

Research on the Fouling Prediction of Heat Exchanger Based on Wavelet Relevance Vector Machine Lingfang Sun, Rina Saqi, and Honggang Xie School of Automation Engineering, Northeast Dianli University, Jilin City, 132012, China [email protected]

Abstract. Based on the relevance vector machine and wavelet theory, a new machine learning method–wavelet relevance vector is introduced. The method performs wavelet transform to decompose the original series into some filtered series, and a relevance vector machine whose kernel functions is wavelet function models each of them. Then, this method is used to predict fouling thermal resistance of Heat exchanger. Construction of wavelet relevance vector machine prediction model is presented. Simulations show that wavelet relevance vector machine requires dramatically fewer kernel functions and it can get high prediction precision, and work in the paper offers a new method for the research of heat exchanger fouling. Keywords: Fouling thermal resistance, Wavelet Transform, Relevance Vector Machine, Prediction.

1 Introduction Fouling exists in nature, daily life and most of the industrial process. Accumulation of fouling not only degrades heat transfer efficiency of heat exchanger, increases greatly initial capital and maintenance costs, but also causes overheat on certain local areas of heat exchanger, and threaten the safety of heat exchange equipment severely. The economic losses caused by fouling are as high as 0.25 of Gross National Product in a developed country [1]. In our country, the partial fouling losses have reached 0.11 of Gross Domestic Product only in electric power system [2], if considering other industry, the whole fouling losses is more than the developed country’s. So minimizing the harm of fouling becomes an effective avenue of save energy. As the foundation of the fouling research, research on the fouling prediction has become of one main direction since the late 1980s, however, for the many influence factors, fouling prediction based on the traditional mechanics analysis can not satisfy the actual needs. In recent years, the artificial intelligence is applied to fouling research, such as neural network [3,4], support vector machine (SVM) [5] and so on, and gets great results, but these methods also show some limitations at the same time. Tipping improved on the SVM and formulated the relevance vector machine (RVM) [6,7]. The RVM is a probabilistic model similar to the widespread SVM, but the training takes place in a Bayesian framework and predictive distributions of the outputs instead of point estimates are obtained. Relevance vector machine draws Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 37–45. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

38

L. Sun, R. Saqi, and H. Xie

hyper-parameters into run, implements automatic estimation of ‘nuisance’ parameters, and its kernel function doesn’t need to satisfy ‘Mercer’ condition, its calculate complexity is dramatically reduced and the operating speed is advanced. So the RVM has been applied to much industrial process recently [8,9]. In the paper, we constructs the wavelet relevance vector machine(WRVM) regression prediction model with the wavelet transform and relevance vector machine, and uses to analyze and predicts the heat exchanger fouling increase.

2 The Principles of Wavelet Relevance Vector Machine 2.1 Wavelet Transform Wavelet analysis is a multi-resolution analysis method both in time domain and frequency domain. On the low frequency part, the frequency resolution of wavelet transform is high, but the time domain resolution is low. On the high frequency part, the time domain resolution is high, but the frequency resolution of wavelet transform is low. So wavelet analysis is called a mathematic microscope. Wavelet function ϕ (t ) is called mother wavelet.  a,b(t) can be obtained through compressing and expanding ϕ (t ) : −1

ψ a ,b ( t ) = a

2

ψ(

t−b ) a

(1)

Where a,b(t) is a successive wavelet; a is scale or frequency factor, a time factor; b R, R is the domain of real number. Wavelet transform is defined as:

∈

WTx (a,τ ) =

1 a

∫

+∞

−∞

x(t )Ψ ∗ (

∈R, a≠0; b is

t −τ )dt a

(2)

The equivalent formula in frequency domain: WTx (a ,τ ) =

a 2π

∫

+∞

−∞

X (ω )Ψ ∗ ( aω )e jωτ d ω

(3)

Fourier transformation of x(t) and ¸ (t) are X(¹) and ¸(¹) respectively. Let frequency factor a and time factor b be discrimination: a=2j and the discrete binary wavelet transform is written as: WT f ( j , k ) = 2

−

j 2

∫ f (t ) Ψ

∗

⎡t − 2 j k ⎤ ⎢ 2 j ⎥dt ⎣ ⎦

τ=2 k. So j

(4)

Where, k is translation factor, and j is wavelet decomposition level. Its binary wavelet reconstruction is formed as follows: f (t ) = ∑ c j ,k ϕ j ,k (t ) + ∑∑ d j , k φ j , k (t ) = A j (t ) + ∑ D j (t ) k

k

j

j

(5)

Research on the Fouling Prediction of Heat Exchanger

39

Where, Aj and Dj is the trend part (background information) and detail part (detail information) respectively. The trend series contains the slowest dynamic, and the detail series contains the dynamic at a certain intermediate scale. 2.2 Relevance Vector Machine

∈

∈

Given a training set {xn , tn }nN=1 , with xi Rn for the input, yi R for the target output, N for the number of training sample. Form this ‘training’, the regression problem is to find a mapping f from the input space to the output space. The purpose of the RVM is to find a relevance regression model based on the Bayesian framework:

t=¶w+¦

(6)

Here, ¹ is parameter vector, ¶ is the N×(N+1) design matrix with Φ = [φ1 , φ2 ,..., φM ]

(7)

Φ nm = K ( xn , xm )

(8)

Wherein

Due to the Bayesian inference, the likelihood of the complete date set can be written as: p(t w , σ 2 ) = (2πσ 2 ) − N 2 exp{− t − Φw

…

2

(2σ 2 )}

(9)

…,

wN). Where, t = (t1, t2, , tN), w = (w1, w2, Generally, with multi-parameter in the regression model, its maximum likelihood estimation may lead to severe over-fitting. And that, on the relevance vector machine, owing to the Bayesian inference makes model more general. The Gaussian prior distributions are chosen: M

p( w, α ) = (2π )− M 2 ∏ α m1 2 exp(−(α m wm2 ) 2) m −1

(10)

Where α = {α1 , α 2 ,...α M } is a vector of hyper parameters that governs the prior over the weight. The value of the hyper parameter is the key to the model sparsity and it also controls the strength of influence of the prior distribution on the each parameter. On the basis of prior distribution and marginal likelihood, the posterior distribution over the weights can be written using Bayes inference: p(w t ,α ,σ 2 ) = = (2π ) − ( N +1) 2 Σ

p ( y | w, σ 2 ) p ( w | α ) ~ N ( μ , Σ) p( y | α , σ 2 ) −1 2

exp{−( w − μ )T Σ −1 ( w − μ ) 2}

(11)

40

L. Sun, R. Saqi, and H. Xie

Where, the posterior covariance and mean are respectively given by: Σ = (σ −2 Φ T Φ + A) −1

(12)

μ = σ −2ΣΦT

(13)

With A = diag (α 0 , α1...α M ) . In practice, we can find that the majority of parameters are automatically set to zero during the leaning process. And the corresponding basis function can thus be ‘pruned’. To the RVM regression model, the non-zero parameters are the relevance vectors (like SV). By integrating the parameters, we obtain the marginal likelihood for the hyper parameters: p (t α , σ 2 ) = N (0, C )

(14)

Here, C = σ 2 I + ΦA−1ΦT . For a new input x∗ and output t∗ , we can then compute the predictive distribution is Gaussian: μ∗ = F μ

(15)

σ ∗2 = σ op2 + F T ΣF

(16)

With F = [Φ1 ( x∗ ), Φ 2 ( x∗ ),..., Φ M ( x∗ )]T . We can consider that μ∗ is the prediction to t∗ , and the distribution of t∗ is given, so we can obtain the Confidence interval under some conspicuous level. If given α ∗ , we can gain its 1 − α ∗ Confidence interval:

( μ∗ − t∗α / 2 ( N − k )σ ∗ , μ∗ + t∗a / 2 ( N − k )σ ∗ )

(17)

With N is the number of the sample and k is the number of parameter.

3 Construction of WRVM Prediction Model WRVM is to hang together the wavelet transform and wavelet kernel function with relevance vector machine. The construction of wavelet relevance vector machine have four steps: (1) decompose the original series into some filtered series with the resolution level 3 (6 time series); (2) reconstruct each sub-time series; (3) let wavelet function to be the kernel function of RVM; (4) predict each sub-time series (trend part and detail part), and combine the results of every series, and then we can acquire the final prediction result. The diagrammatic sketch is shown as Fig. 1. Due to that every sub-time series has its own characteristic, so the each predict model corresponding to the sub-time series is distinct.

Research on the Fouling Prediction of Heat Exchanger

41

Original series

x Wavelet decomposition

d1…

aJ

…

dJ

Wavelet Reconstruction

D1…

AJ

…

DJ

Wavelet Kernel RVM

YD1…

YAJ

…

YDJ

Prediction Result

Fig. 1. Method of WRVM

The capability of non-liner regression model can be evaluated by many measurements. The most important measurement to evaluate the quality of model is the accuracy. In this work, we use mean relatively error: ∧

MAPE =

1 n yt − yt ∑ n t =1 yt

(18)

4 Fouling Thermal Resistance Prediction Based on WRVM 4.1 Data Acquisition and Pretreatment Analyzing the system that we need to predict, make sure the characteristic parameters that can reflect the system’s state. We select the fouling thermal resistance as a way to detect the fouling degree. The data we acquired used for the research were from online monitoring equipment [10]. According to the methods in the reference [1], we found that six measuring parameters influenced fouling, i.e. input temperature of working fluid, output temperature of working fluid, flow velocity of working fluid and three temperatures of heat exchanger tube wall. So we use these parameters forming the inputs of WRVM. The data we collected were from the IDCB893 for every one minute, altogether 180 series sample consisted the discrete fouling thermal resistance data. In the same way, another group data were completed, for 180 series sample, for the prediction. These parameters were chosen to be the inputs of the SVM network. They represent themselves for different significations, so they need to be normalized. The process can be done with the function ‘Prestd’ in the Matlab, it is as below, [pn, meanp, stdp] = prestd (x) Here, ‘pn’ stands for the normalized input vector or matrix; ‘meanp’ and ‘stdp’ stand for the mean and the standard deviation of the x. The sample data were divided into two groups; the former 120 data were used for training, the rest ones were used for predicting. The second group was all used for prediction.

42

L. Sun, R. Saqi, and H. Xie

Hyper parameters and deviation was initialized respectively as follows: initAlpha=( 1 ) 2 N

initBeta=var[t] × 0.1

4.2 Structure of Kernel Function The performance of RVM is mainly decided by its kernel function. How to construct suitable kernel function relating on complicated practical application is an important aspect of RVM all the way. Two wavelet functions are chosen in this paper, Mexico wavelet function and Morlet wavelet function respectively. Mexico wavelet function d

xi − xi2

i =1

a2

K ( x, x ' ) = ∏ [(1 −

) exp( −

xi − xi'

2

)]

(19)

xi − xi' xi − xi' )exp(− )) a 2a2

(20)

2a 2

Morlet wavelet function: 2

d

K (x, x' ) = ∏(cos(1.75× i =1

4.3 Results and Discussion The simulation based on wavelet relevance vector machine includes 5steps. Step 1: Apply the function ‘wavedec’ of Matlab, and then decompose the original time series to 4 subsidiary time series. [ci,li]=wavedec (x1c (:,i),2,'db3') Step 2: Apply the function ‘wrcoef’ of Matlab and carry out wavelet reconstruction to each subsidiary time series. A(:,i) = wrcoef('a',ci,li,'db3',2) Step 3: Let Mexico wavelet function and Morlet wavelet function to be the kernel function of relevance vector machine. Through repeatedly experiments, we can select the most contented output. Step 4: In order to cater for the characteristic of the fouling time series, the parameters is setting as follow: The regression time-delay :d=3 The width of wavelet function: a=0.8 The variance of noise: σ 2 = var( y ) × 0.1 The other parameters:

itermax = 50 ,

α max = 1.0e5 , {α } = (ones (1.N + 1) × ( 1 )) 2 2

Step 5: Predict each sub-time series, and combine the results of every series, we can acquire the final prediction result. Corresponding experiment data and prediction results are shown as Fig. 2 – Fig. 8.

Research on the Fouling Prediction of Heat Exchanger

Fig. 2. Prediction curve of trend part (*) and real fouling cure (-)

Fig. 3. Prediction curve of detail part cd1 (*) and real fouling cure (-)

Fig. 4. Prediction curve of detail part cd2 (*) and real fouling cure (-)

Fig. 5. Prediction curve of detail part cd3 (*) and real fouling cure (-)

43

44

L. Sun, R. Saqi, and H. Xie

Fig. 6. Prediction fouling curve (-) and real fouling cure (*)

Fig. 7. The contrastive curve with 95% confidence interval

Fig. 8. Relative error curve

Through simulation, we can obtain that wavelet theory was used two parts, wavelet kernel functions and wavelet transform respectively. As a result, the precision of wavelet relevance vector is higher than standard relevance vector machine, shown as table 1. So the method reported in this paper plays an important role in improving the precision of relevance vector machine in forecasting time series. Table 1. Performance analysis WRVM The number of RVs/SVs Mean relative error

10 0.000457

RVM

SVM

10

92

0.000465

0.000598

Research on the Fouling Prediction of Heat Exchanger

45

5 Conclusions A research on modeling and Predicting the fouling of heat exchanger based on WRVM was introduced in this work. Simulation analysis indicates that the result of prediction has high prediction precision. The main reason is that the wavelet transform was used to extract the trend series and detail series, and make models respectively. It can avoid the over fitting in short term fluctuate. This method took even short time and it has excellent ability and generalization performance and robustness under the noises background. Acknowledgments. Supported by National Basic Research Program of China (2007CB206904)and the Eleventh Five-Year Science & Technology Project of Jilin Province Office of Education ([2007]NO.53).

References [1] Yang, S.R., Xu, Z.M., Sun, L.F.: Heat Exchanger Fouling and Its Countermeasure, 2nd edn. Science Press, Beijing (2004) (in Chinese) [2] Xu, Z.M., Yang, S.R.: Approach to Costing the Utility Boiler Fouling in China. J. of the CSEE. 24, 196–200 (2004) (in Chinese) [3] Radhakrishnan, V.R., Ramasamy, M.: Heat Exchange Fouling Model and Preventive Maintenance Scheduling Tool. Applied Thermal Engineering 27, 2791–2802 (2007) [4] Sun, L.F., Cai, H.D., Zhang, Y.Y., Yang, S.R., Qin, Y.K.: Research on the Fouling Prediction of Heat exchanger Based on Wavelet Neural Network. In: 3rd IEEE International Conference On Cybernetics and Intelligent Systems, Chengdu, pp. 961–964 (2008) [5] Sun, L.F., Zhang, Y.Y., Zheng, X.P., Yang, S.R., Qin, Y.K.: Research on the Fouling Prediction of Heat exchanger Based on Support Vector Machine. In: 2008 International Conference on Intelligent Computation Technology and Automation, Changsha, pp. 240– 244 (2008) [6] Tipping, M.E.: The Relevance Vector Machine. In: Proc. of Advances in Neural Information Processing Systems. MIT Press, Cambridge (2000) [7] Nikolaev, N.: Sequential Relevance Vector Machine Learning from Time Series. In: Proceedings of International Joint Conference on Neural Networks, Montreal (2005) [8] Lexander, S., Richard, E.: Smooth Relevance Vector Machine: A Smoothness Prior Extension of the RVM. Machine Learning 68, 107–135 (2007) [9] Clodoaldo, A.M., Lima, A.L.V., Coelho, S.C.: Automatic EEG Signal Classification for Epilepsy Diagnosis with Relevance Vector Machines. Expert Systems with Application 36, 10054–10059 (2009) [10] Yang, S.R., Xu, Z.M., Sun, L.F.: Automatic On-line Monitor of Fouling Thermal Resistance and Its Application. J. of Engineering Thermophysics 22, 150–152 (2001) (in Chinese)

Electricity Price Forecasting Using Neural Networks Trained with Entropic Criterion Jianhua Zhang, Jingyue Wang, Rui Wang, and Guolian Hou Beijing Key Laboratory of Industrial Process Measurement & Control New Technology and System, North China Electric Power University, Beijing 102206, China

Abstract. Accurate electricity price forecasting is critical to market participants in wholesale electricity markets. The problem becomes more complex because the acquired data series are non-linear and non-Gaussian. In this paper, Multi Layer Perceptrons (MLP) trained with minimizing error entropy (MEE) algorithm is utilized to forecast electricity price. Compared with the conventional MLP with mean square error (MSE) criterion, the proposed approach can achieve better performance in simulated examples. Keywords: Electricity price forecasting, Neural network, Entropy.

1 Introduction Recently, a group of algorithms have been developed to analyze and forecast electricity prices with different aims and time horizons. Arciniegas and Rueda [1] used Takagi-Sugeno-Kang (TSK) fuzzy inference system in forecasting day-ahead prices in Ontario electricity market. Alicia and Antonio [2] introduced an input-output hidden markov model (IOHMM) by finding the probabilistic model of prices and market information, such as the loads, generation kinds and prices. Rodriguez and Anders [3] invented a hybrid forecast model by back propagation neural networks and neuro-fuzzy system for predicting hourly market clearing price in the energy market. In addition, support vector machine (SVM) was also applied into electricity price forecasting field in [4]. And the forecasting method using rough set theory based on collective concept dealing with uncertainty was also proposed in [5]. Among these different techniques of price forecasting, artificial neural networks has been applied by many researchers because of its ability to learn complex and non-linear relationships that are difficult to model with conventional techniques. The mean square error (MSE) has been almost exclusively employed to train ANN systems [6,7]. However, MSE which concentrates on second order statistics was under the linearity and Gaussian distribution assumptions. Most real problems are governed by nonlinear equations and most random phenomena are far from being normally distributed. In this context, the higher-order statistical behavior of the systems is required. Recently, minimizing error entropy (MEE) has been utilized in the training of some adaptive systems [8-10]. As such, minimization of error entropy is employed as a more robust criterion when MLP is utilized to predict electricity prices. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 47–53. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

48

J. Zhang et al.

2 MLP Trained with MEE Criterion A schematic representation of the MLP with MEE criterion is shown in Fig.1. The weights between different layers of MLP are updated by minimizing the entropy of the error e( k ) between the estimated output and the targets.

y ( n)

r ( n)

^

y ( n)

e(n)

Fig. 1. Training MLP using MEE criterion

In this work, the quadratic Renyi entropy Hˆ 2 (e) of error e( k ) between the desired and estimated output is employed to be the performance index, and it can be obtained from data samples {e1(k), e2 (k),..., eN (k)} by utilizing Parzen window with Gaussian kernel function.

⎡ 1 Hˆ 2 (e) = − log ⎢ 2 ⎣N where the kernel function

κσ ( x) =

N

⎤

N

∑∑ κσ (e (k ) − e (k )) ⎥ i

i =1 j =1

1 2πσ

j

⎦.

(1)

− x2

e

2σ 2

.

For the purpose of computing, the estimation of quadratic Renyi entropy at time k can be calculated by a sliding window of samples [8].

⎡1 Hˆ 2 (e) = − log ⎢ ⎣W where

V ( e) =

1 W

k -1

⎤

∑ κσ (e(i) − e(k )) ⎥⎦ = − log V (e)

(2)

i = k -W

k -1

∑ κσ (e(i) - e(k)) is named as information potential in [9], and

i = k -W

W is the width of sliding window.

Electricity Price Forecasting Using Neural Networks Trained with Entropic Criterion

49

In this work, a q − p − l MLP is used to forecast electricity prices. The weights can be updated by using the steepest descent algorithm as follows.

wk (n) = wk (n − 1) + η ⋅ where

∂V (e(n)) ∂wk

(3)

∂V (e(n)) is the gradient of the information potential with respect to the ∂wk

weights and η the learning rate. Once the weight updating rules are obtained, the MLP can be trained by sampled electricity prices series. The electricity prices can be predicted by generalizing the trained MLP.

3 Price Forecasting by MLP with MEE Criterion The factors that influence the electricity price were investigated in [11, 12] , for example, power demands, transmission and generation outages, system or regional reserves, bidding strategy and market power, system congestion, ancillary services and market rules and so on. The main factors can be extracted by correlation analysis and experience in order to improve the efficiency of forecasting. The correlation coefficient ρ is defined as follows: −

n

ρ=

−

∑ ( x − x)( y − y) i

i =1

i

−

n

−

n

∑ ( x − x) ∑ ( y − y ) 2

i

i =1

−

x= −

y=

i

i =1

1 n

n

∑x

i

(5)

1 n ∑ yi n i =1

(6)

i =1

−

where

(4) 2

−

xi , yi are the ith observations and x , y the corresponding mean value of n

observation series. The correlation coefficients between the physical variables and prediction variables are listed in Table 1 (The historical price and demand data are selected from a real power market. P and L represent the electricity price and load in different days i and different times t). The MLP used to forecast daily electricity prices is shown in Fig.2.

50

J. Zhang et al. Table 1. Correlation coefficients between physical variables and price

Variables P(iˈt-1) P(iˈt-2) P(iˈt-3) P(iˈt-4) P(i-1ˈt) P(i-2ˈt) P(i-7ˈt) P(i-1ˈt-1) P(i-1ˈt+1)

correlation with price P(iˈt) 0.9235 0.8396 0.7451 0.6224 0.8646 0.7078 0.9770 0.7343 0.6249

Variables L(iˈt-1) L(iˈt-2) L(iˈt-3) L(iˈt-4) L(i-1ˈt) L(i-2ˈt) L(i-7ˈt) L(iˈt) L(i-1ˈt-1)

correlation with price P(iˈt) 0.9217 0.8172 0.7420 0.6178 0.8723 0.7738 0.9711 0.9747 0.7248

Fig. 2. MLP applied for forecasting

Several error indicators are analyzed to evaluate the accuracy of the proposed approaches to forecasting electricity prices. They are the mean absolute percentage error (MAPE), absolute percentage error (APE) and sum squared error (SSE). The MAPE criterion is given by:

δ MAPE =

∧ 1 n Y − Y ( ∑ i i ) Y × 100% n i =1

Y=

(7)

1 n ∑ Yi n i =1

(8) ∧

where

n is the total number of hours, Yi is the actual price at hour i, Yi is the

forecasted price at that hour.

Electricity Price Forecasting Using Neural Networks Trained with Entropic Criterion

51

The absolute percentage error (APE) is defined as follows: ∧

δ APE = (Yi − Yi ) Yi ×100%

(9)

The SSE criterion is the sum of the n square differences between the forecasted and the actual ones. The SSE criterion is given by: n

∧

δ SSE = ∑ (Yi − Yi )2

(10)

i =1

4 Simulation and Analysis In order to test the validity of the proposed algorithm, a 9-21-1 MLP is constructed with the historical data of electricity price and demand in certain region from July 15 to August 15, and then the trained MLP is utilized to investigate the daily price for next day-August 16. In this simulation, the learning rate η = 0.9 , the width of sliding window W = 32 , the total number of hours n = 24 . The information potential during training phase is shown in Fig. 3, it can be seen that the entropy of error is minimized gradually. Compared with the conventional MLP (using MSE criterion), the forecasting electricity prices are shown in Fig. 4 and the corresponding APEs are shown in Fig.5. It is clear that APE with MEE criterion is almost smaller than that one with MSE. In addition, Table 2 shown the value of MAPE and SSE during the whole day, it can be seen that the proposed MLP with MEE criterion can obtain better forecasting than conventional MLP.

Fig. 3. Information potential of error

52

J. Zhang et al.

Fig. 4. Forecasting by MLP-MEE and MLP-MSE

Fig. 5. The APE of MLP-MEE and MLP-MSE Table 2. Statistical Analysis of Errors

Criteria Error MAPE SSE

MLP-MEE

MLP-MSE

5.0609% 7.8988

8.1125% 14.1206

Electricity Price Forecasting Using Neural Networks Trained with Entropic Criterion

53

5 Conclusions In this paper, a MLP with MEE criterion is utilized to predict the next 24 hour electricity prices in the real electricity market. The predicted price can be used for obtaining bidding strategies. Acknowledgments. This work is partially supported by National Natural Science Foundation of China under grants (No. 60674051, 60974029). These are gratefully acknowledged.

References 1. Arciniegas, A.I., Rueda, I.E.A.: Forecasting Short-term Power Prices in the Ontario Electricity Market (OEM) with a Fuzzy Logic Based Inference System. Utilities Policy 16, 39–48 (2008) 2. Gonzalez, A.M., Roque, A.M.S., Gonzalez, J.G.: Modeling and Forecasting Electricity Prices with Input/output Hidden Markov Models. IEEE Trans. Power Syst. 20, 13–24 (2005) 3. Rodriguez, C.P., Anders, G.J.: Energy Price Forecasting in the Ontario Competitive Power System Market. IEEE Trans. on Power Syst. 19, 366–374 (2004) 4. Sun, W., Lu, J.-C., Meng, M.: Application of Time Series Based SVM Model on Next-day Electricity Price Forecasting under Deregulated Power Market. In: Proc. 5th Int. Conf. Machine Learning and Cybernetics, Dalian, China, pp. 13–16 (2006) 5. Xiao, Z., Ye, S.J., Zhong, B., Sun, C.X.: BP Neural Network with Rough Set for Short Term Load Forecasting. Expert Systems with Applications 36, 273–279 (2009) 6. Szkuta, B.R., Sanabria, L.A., Dillon, T.S.: Electricity Price Short-term Forecasting Using Artificial Neural Networks. IEEE Trans. Power Syst. 14, 851–857 (1999) 7. Mandal, P., Senjyu, T., Urasaki, N., Funabashi, T., Srivastava, A.K.: A Novel Approach to Forecast Electricity Price for PJM Using Neural Network and Similar Days Method. IEEE Trans. on Power Systems. 22, 2058–2065 (2007) 8. Erdogmus, D., Principe, J.C., Kim, S.P., Sanchez, J.C.: A Recursive Renyi’s Entropy Estimator. In: Proceedings of IEEE Neural Networks for Signal Processing, Martigny, Valais, Switzerland, pp. 209–217 (2002) 9. Erdogmus, D., Principe, J.C.: An Error–entropy Minimization Algorithm for Supervised Training of Nonlinear Adaptive Systems. IEEE Transactions on Signal Processing 50, 1780–1786 (2002) 10. Erdogmus, D., Principe, J.C.: Comparison of Entropy and Mean Square Error Criteria in Adaptive System Training Using Higher Order Statistics. In: Proc. Second International Workshop on Independent Component Analysis and Blind Signal Separation, Helsinki, Finland (2000) 11. Zhu, J.Z., Jordan, G., Ihara, S.: The Market for Spinning Reserve and Its Impacts on Energy Prices. In: Proc. IEEE Power Engineering Society Winter Meeting, Singapore, pp. 1202–1207 (2000) 12. Kian, A., Keyhani, A.: Stochastic Price Modeling of Electricity in Deregulated Energy Markets. In: Proc. 34th Hawaii Int. Conf. on System Sciences, Hawaii, USA, pp. 832–838 (2001)

Least Square Support Vector Machine Ensemble for Daily Rainfall Forecasting Based on Linear and Nonlinear Regression Jiansheng Wu1 , Mingzhe Liu2 , and Long Jin3 1

Department of Mathematical and Computer Sciences, Liuzhou Teachers College, Liuzhou, 545004, Guangxi, China [email protected] 2 Institute of Information and Mathematical Sciences, Massey University, Albany, 0629, Auckland, New Zealand [email protected] 3 Guangxi Climate Center, Naning, 530022, Guangxi, China [email protected]

Abstract. Accurate forecasting of rainfall has been one of the most important issues in hydrological research. In this paper, a novel nonlinear regression ensemble model is proposed for rainfall forecasting. The model employs Least Square Support Vector Machine (LS-SVM) based on linear regression and nonlinear regression. Firstly, Projection Pursuit (PP) technology and Particle Swarm Optimization (PSO) algorithm are used to obtain the main factors of the rainfall, which optimize projection index from high dimensionality to a lower dimensional subspace. Secondly, using different linear regressions extract linear characteristics of the rainfall system, and using different Neural Network (NN) algorithms and different network architectures extract nonlinear characteristics of the rainfall system. Finally, LS-SVM regression is used for nonlinear ensemble model. This technique is implemented to forecast daily rainfall in Guangxi, China. Empirical results show that the prediction by using the LS-SVM ensemble model is generally better than those obtained using other models presented in this study in terms of the same evaluation measurements. The results suggest that our nonlinear ensemble model can be extended to meteorological applications in achieving greater forecasting accuracy and improving prediction quality. Keywords: Projection Pursuit, Particle Swarm Optimization, Neural Network, Least Square Support Vector Machine Ensemble.

1

Introduction

Rainfall prediction is a challenging task in the climate dynamics and climate prediction theory [1]. Accurate rainfall forecasting is one of the most important issues in hydrological research because early warnings of severe weather can help prevent casualties and damages caused by natural disasters [2,3]. With the Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 55–64. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com 

56

J. Wu, M. Liu, and L. Jin

development of science and technology as well as intelligent computing in recent decades, many emerging techniques, such as Neural Networks (NNs), have been used in the rainfall forecasting. These efforts enhanced our understanding of intelligent computing and rainfall forecasting [4,5,6,7]. Short-term precipitation in the climate process is a very complex system, which it is not a simple linear or non-linear system. Therefore, it is difficult to effectively establish a model rainfall forecasting by the pure linear or pure nonlinear model. In other words, it is difficult to determine an effective predictor of the impact of weather systems due to the large dimension and noise in the weather system. The model will be complicated and the forecasting ability will reduce if there are many factors. Currently, modelling such a system focuses on: 1. how to extract sufficient feature information and how to reduce irrelevant information from the spatial data for the rainfall forecasting model; 2. how to set the effective forecasting rainfall modeling. In this paper, a novel method is presented for daily rainfall forecasting. This approach combines Least Square Support Vector Machine (LS–SVM), linear and nonlinear regressions. The Projection Pursuit technology and Particle Swarm Optimization (PP–PSO) are used to select the rainfall factors. Three linear regressions extract linear features and three NNs extract nonlinear features. The rainfall data in Guangxi is used as a case study for the development of daily rainfall forecasting model. The rest of the paper is organized as follows. Section 2 describes the model building process. For further illustration, this work employs the method to set up a prediction model for daily rainfall forecasting in Section 3. Finally, some concluding remarks are drawn in Section 4.

2

The Building Process of the LS-SVM Ensemble Model

Once data are collected, determining variables for modelling becomes possible. A large number of meteorological factors depends each others, and too much variables can enlarge the model scale, increase the complexity of the model. Therefore, the best way is to identify the variables that will save modelling time and reduces the problem space. Other reasons for the difficulties in forecasting rainfall are, such as the complexity of atmosphere-ocean interactions and the uncertainty of the relationship between rainfall and hydro-meteorological variables. The characteristic of rainfall is complex system dynamics under the influence of different meteorological factors. 2.1

Feature Extraction Used by PP–PSO

In the late 1970s, the international statistical society has developed a class of new statistical method to deal with and analysis high-dimensional data, which is called Projection Pursuit (PP) [8], which is suitable for dealing with nonlinear, non-normal distribution data to avoid “dimension disaster”, and it has succeeded in practice in many areas [9]. The basic idea of the method lies in: that

LS–SVM Ensemble for Daily Rainfall Forecasting

57

the computer technology is used to project high-dimensional data into the low dimensional subspace by numerically maximizing a certain objection function or projection index, which can extract the original data structure or characteristics, so as to achieve the goal of the study and analysis of high dimensional data. In rainfall system, X={xij , i = 1, 2, · · · , m, j = 1, 2, · · · , n} are m-dimensional factor of weather,where n is the number of the sample. A linear projection from Rm to Rp ,The matrix of the projectional direction is described as follow A = {ast > 0, s = 1, 2, · · · , p, t = 1, 2, · · · , m}

(1)

where p(p < m) is the number of new dimension for the factor matrix, and the row vector of A are orthogonal to each other. Then the new factor matrix has the form Z = A ∗ X = {zsj , i = 1, 2, · · · , p, j = 1, 2, · · · , n} (2) PP searches for a projection A maximizing a certain objection function. While synthesizing the projection value, the dispersal characteristic of projection value Z requests to be: the partial projection point should be as intensive as possible, it can be best to be some point zone, and in the total the point zone should be as wide as possible. The objective function can be written as  max Q(a) = S(a)D(a) (3) s.t. A · AT = 1 Because of the complex nonlinear optimization, it is very difficult to solve the solution projectional direction by the traditional method. In order to extract the information, reduce dimension and decrease noise from the original variables, the following optimization models are used to find the solution projectional direction parameters with PSO. The fitness function of PSO is defined as follows: F itness = Q(a)

(4)

The results can be solved by a standard PSO algorithm for Equation 4. 2.2

Extraction of Linear Features by Three Traditional Linear Regression

Perhaps the most popular mathematical model for making predictions is the multiple linear regression model in the traditional mathematical method, which estimates one random variables using known values of multiple variables. There is a continuous random variable called the dependent variable Y , and a number of independent variables, x1 , x2 , · · · , xp . The goal of regression models is to fit a set of data with an equation, the simplest being a linear equation. The linear regression model is given by yˆ = β0 + β1 x1 + β2 x2 + · · · + βn xp + ε

(5)

where ε , the “noise”variable, is a normally distributed random variable with mean equal to zero and standard deviation σ, {βt , t = 1, 2, · · · , p} are the values

58

J. Wu, M. Liu, and L. Jin

of the coefficients. Unknown parameters values should be estimated from the smallest error sum of squares of the samples. Three types of linear regression models have been made in this paper for parsimony variables, such as stepwise multiple linear (SML) regression, partial least squares (PLS)regression and multi recursive regression method (MRR) to capture the linear characteristics of rainfall system. More detailed introductions to the three linear regression model are referred to the related literature [10,11,12]. 2.3

Extraction of Nonlinear Features by Three NNs Methods

NNs provide an interesting technique that theoretically can approximate any nonlinear continuous function on a compact domain to any designed of accuracy. The network learns by adjusting the interconnections (called weights) among layers. When the network is adequately trained, it is able to generalize relevant output for a set of input data. A valuable property of neural networks is that of generalization, whereby a trained neural network is able to provide a correct matching in the form of output data for a set of previously unseen input data. Learning typically occurs by example through training, where the training algorithm iteratively adjusts the connection weights (synapses). With the work about bias-variance trade-off of Bretiman [13], an NNs regression model consisting of diverse models with much disagreement is more likely to have a good generalization [14]. Therefore, how to generate diverse models is a crucial factor. In this paper, there are three methods for generating diverse models, such as multi–layer perceptions based on the Levenberg–Marquard algorithms (MLP–NN), the radial basis function network network (RBF–NN) and General Regression Neural Network (GR–NN) for nonlinear regression model. Readers interested in a more detailed introduction about these neural network algorithms are referred to the related literature [15]. 2.4

Nonlinear Ensemble Based on LS-SVM

LS-SVM is a least squares modification to the Support Vector Machine [16]. The major advantage of LS-SVM is that it is computationally very cheap while it still possesses some important properties of the SVM. In this section we will briefly discuss the LS-SVM method for a regression task. For more detailed information see [16]. We let {xi , i = 1, 2, · · · , N } be the output of linear and nonlinear forecasting predictors, {yi , i = 1, 2, · · · , N } be the aggregated output and the goal is to estimate a regression function f . Basically we define a N -dimensional function space by defining the mapping ϕ = [ϕ1 , ϕ2 , · · · , ϕN ]T according to the measured ˆ = ω T ϕ(x) + b where ω is a weight points. The LS-SVM model is of the form f (x) vector and b is a bias term. The optimization problem is the following: ⎧ N  ⎨ min J(ω, ) = 12 ω T ω + γ 12 2i (6) i=1 ⎩ s.t. yi = ω T ϕ(xi ) + b + i , i = 1, 2, · · · , N

LS–SVM Ensemble for Daily Rainfall Forecasting

59

where the fitting error is denoted by i . Hyper-parameter γ controls the tradeoff between the smoothness of the function and the accuracy of the fitting. This optimization problem leads to a solution: ˆ = f (x)

N 

αi K(x, xi ) + b

(7)

i=1

where αi is for the coefficients and K(x, xi ) = ϕT (x)ϕ(xi ) is the kernel. A common choice for the kernel is the Gaussian RBF: K(x, xi ) = e

x−xi 2 2σ2

(8)

The above-mentioned method can be summed up as follows: firstly, this paper is to extract the feature of the original meteorological factors from high dimension space to lower dimension subspace,and construct a forecasting variable by PSO algorithm optimization projection index. Secondly, three traditional linear regression methods are used to capture rainfall series linear characteristics. Thirdly, three different NNs models is trained to capture nonlinear patterns. Finally, the LS-SVM method are used for ensemble, which incorporate Linear model with Nonlinear model in an adaptive manner(LS–SVM–LNE). The basic flow diagram can be shown in Fig.1.

Randomly initialize population position and velocities

Initial value of projection directions

Calculate fitnessvalure by the original datd sets

Data sets (the main factor of weather) Yes

S atis fy s toping criteria?

Generate new factors of rainfall

Stepwise Muliple Regression Partial Leas t S quares Regres s ion

Multi Recursive Regression Levenberg-Marquard Neural Network

LS-SVM Ensemble

No Update particle best Update global best

Radial Basis Function Network Network

Update particle velocity Update particle position

General Regression Neural Network

Fig. 1. A Flow Diagram of The Proposed Nonlinear Ensemble Forecasting Model

3

Experiments Analysis

In this paper, PSO parameters are set as follows: the iteration times are 100; the population is 100; the minimum inertia weight is 0.1; the maximum inertia weight is 0.9 and the learning rate is 2.0. NNs parameters are set as follows: the number of neurons in the hidden layer range from 3. The learning rate is 0.9; the momentum factor is 0.7; the iteration times are 1000; the global error is 0.001.

60

3.1

J. Wu, M. Liu, and L. Jin

Empirical Data

Real–time ground rainfall data have been obtained in June from 2003 to 2008 in Guangxi by observing 89 stations, which 144 samples are modeled from 2003 to 2007 in June, other 30 samples are tested modeling in June of 2008. Method of modeling is one-step ahead prediction, that is, the forecast is only one sample each time and the training samples is an additional one each time on the base of the previous training. Due to the complex terrain of Guangxi and inhomogeneous rainfall, the region has been divided into three regional precipitation based on historical precipitation data by the cluster analysis method to reduce the difficulty of forecasting. Statistics for each district in the average daily precipitation is used as the forecasting object. Fig.2 shows three region maps.

Fig. 2. The Group Average Region Map of Guangxi Rainfall

In the first region as an example to show the process of modeling, it is very important to select the appropriate forecasting factors for model, different climatic variability and its effect have been discussed many times in the literature. In this paper, first of all, the candidate forecasting factors are selected from the numerical forecast products based on 48h forecast field, which includes:(1) the 17 conventional meteorological elements and physical elements from the T213 numerical products of China Meteorological Administration, the data cover the latitude from 15◦ N to 30◦ N, and longitude from 100◦ E to 120◦ E, with 1◦ ×1◦ resolution, altogether there are 336 grid points.(2) the fine-mesh precipitation data from the East Asia of Japanese Meteorological Agency, the data cover the latitude from 15◦ N to 30◦ N, and longitude from 100◦E to 120◦ E, with 1.25◦ × 1.25◦ resolution, altogether there are 221 grid points. We can get 60 variables as the main forecasting factors. Finally, a new information matrix can be extract form 60 variables by PP– PSO so we get 3 variables as the predictors in the first region. Fig.3 shows the data–sets of the main impact factors, which 60 variables and 144 samples. Fig.4 shows the data–sets of the predictors including 3 variables and 144 samples by PP–PSO extraction.

LS–SVM Ensemble for Daily Rainfall Forecasting

61

1

1

0.5

0.5

0

0

−0.5 −0.5

−1 1 1

0.5 0.5

0 0

−0.5

0.5 0

−0.5 −1

−1

−1

Fig. 3. The Main Factors Data–sets

3.2

−1 1

−0.5

0

0.5

1

Fig. 4. The Predictors Data–sets

The Performance Evaluation of the Model

In order to measure effectiveness of the proposed method, five types of errors, such as, the Maximum Absolute Error (MAE), the Root Mean Squares Error(RMSE), the Pearson Relative Coefficient (PRC), the Error Value more than 25mm (F1 ) and the Error Value less than 5mm (F2 ) which have been found in many papers, are also used here. Interested readers can be referred to [4] for more details. For the purpose of comparison, we have also built three other ensemble forecasting models: (1) Simple Average Ensemble (SAE-LE) - all the available linear forecasting output; 1 Tˆ1 = (y1 + y2 + y3 ) (9) 3 (2) Simple Average Ensemble (SAE-NE) - all the available nonlinear forecasting output; 1 Tˆ2 = (y4 + y5 + y6 ) (10) 3 (3) Simple Average Ensemble (SAE-LNE) - all the available linear and nonlinear forecasting output; 1 Tˆ3 = (y1 + y2 + y3 + y4 + y5 + y6 ) 6 3.3

(11)

Analysis of the Results

Fig.5. shows the fitting results of all model for 144 training samples, we can see that the fitting results of SL-SVM-LNE model are best in all models, the SAE–LE can basically fit the historical precipitation trends,the fitting results of the SAE-NE and the SAE-LNE are in good agreement with the actual values. The more important factor to measure performance of a method is to check its forecasting ability.

62

J. Wu, M. Liu, and L. Jin 80

140

Rainfall(mm)

Rainfall(mm)

Actural Rainfall SAE−LE Fitting SAE−NE Fitting SAE−LNE Fitting LS−SVM−LNE Fitting

70

60

Actural Rainfall SAE−LE Forecasting SAE−NE Forecasting SAE−LNE Forecasting LS−SVM−LNE Forecasting T213 Forecasting

120

100

50 80

40 60

30 40

20

20

10

0

0

20

40

60

80

100

120

140

5

10

15

20

Day

Fig. 5. Fitting of 144 Samples in June

25

30 Day

Fig. 6. Forecasting of 30 Samples in June

From Table 1, for the rainfall testing case in June, the MAE and RMSE of the LS–SVM–LNE model is also the smallest in all model. The minimum values of MAE and RMSE indicate that the deviations between original values and forecast values are very small. At the same time, from Table 1, the PRC of the LS–SVM–LNE model is the highest in all model. The accurate efficiency of the proposed model is measured as PRC, The higher values of PRC (maximum value is 1) indicate that the forecasting performance of the LS–SVM–LNE model is effective, which can capture the average change tendency of the cumulative rainfall data. The effective information of the proposed rainfall forecasting model is measured by F1 and F2 . If the Error Value is more than 15mm between the original value and the forecast value, the model predictions don’t provide effective information, if the Error Value is less than 5mm, the model predictions provide a piece of very important effective information for the future forecasting. From Table 1, the LS–SVM-LNE model has no unreliable information. Similarly, the LS–SVM-LNE model has reference information at least 86.67%. These results indicate that the proposed model can provide more useful information, avoid invalid information for the future forecasting. Table 1. A Comparison of Result of Ensemble Models about 30 Testing Samples Results Errors MAE RMSE PRC F1 (F1 /30) · 100% F2 (F2 /30) · 100%

SAE–LE

SAE–NE

SAE–LNE

LS–SVM–LNE

T213

79.4627 24.5914 0.5728 12 40.00% 9 30.00%

68.4590 17.7559 0.7543 6 20.00% 13 43.33%

73.7623 18.5157 0.7062 8 26.67% 16 53.33%

49.8746 13.5746 0.8957 0 0.00% 26 86.67%

62.7980 35.4308 0.0129 11 36.67% 10 33.33%

LS–SVM Ensemble for Daily Rainfall Forecasting

63

Furthermore, the same method is used to train precipitation data and predict precipitation at the other two regions. The experimental results also show that LS–SVM–LNE method is better than other methods. From the experiments presented in this paper, the LS–SVM ensemble forecasting model is superior to the traditional linear regression model, the individual NNs model as well as the linear ensemble models for the test cases of daily rainfall.

4

Conclusions

Accurate rainfall forecasting is crucial for a frequent unanticipated flash flood region to avoid life losing and economic loses. This paper proposes a LS–SVM ensemble forecasting model that combines three traditional linear regression model and three different NNs model to predict daily rainfall based on PP–PSO. In terms of the different forecasting models, empirical results show that the nonlinear ensemble model performs the best for the test cases of three regions Guangxi, China on the basis of different criteria. In the LS–SVM ensemble model test cases, the MAE and RMSE are the lowest and the PRC is the highest. This indicates that the LS–SVM ensemble forecasting model can be used as an alternative tool for daily rainfall forecasting to obtain greater forecasting accuracy and improve the prediction quality further in view of empirical results, and can provide more useful information, avoid invalid information for the future forecasting.

Acknowledgment This work was supported in part by the Key Natural Science Foundation of Guangxi under Grant No.0832019Z, and in part by the Department of Guangxi Education under Grant No. 200807MS098.

References 1. Jiangsheng, W., Long, J.: Forecast Research and Applying of BP Neural Network Based on Genetic Algorithms. Mathematics in Practice and Theory 35(1), 83–88 (2005) 2. Jiansheng, W., Long, J., Mingzhe, L.: Modeling Meteorological Prediction Using Particle Swarm Optimization and Neural Network Ensemble. In: Wang, J., Yi, Z., ˙ Zurada, J.M., Lu, B.-L., Yin, H. (eds.) ISNN 2006. LNCS, vol. 3973, pp. 1202–1209. Springer, Heidelberg (2006) 3. Nasseri, M., Asghari, K., Abedini, M.J.: Optimized Scenario for Rainfall Forecasting Using Genetic Algorithm Coupled with Artificial Neural Network. Expert Systems with Application 35, 1414–1421 (2008) 4. Jiansheng, W., Enhong, C.: A Novel Nonparametric Regression Ensemble for Rainfall Forecasting Using Particle Swarm Optimization Technique Coupled with Artificial Neural Network. In: Yu, W., He, H., Zhang, N. (eds.) ISNN 2009. LNCS, vol. 5553, pp. 49–58. Springer, Heidelberg (2009)

64

J. Wu, M. Liu, and L. Jin

5. Yingni, J.: Prediction of Monthly Mean Daily Diffuse Solar Radiation Using Artificial Neural Networks and Comparison with other Empirical Models. Energy Policy 36, 3833–3837 (2008) 6. Govindaraju, R.S.: Artificial Neural Network in Hydrology, I: Preliminary Concepts. Journal of Hydrologic Engineering 5(2), 115–123 (2000) 7. French, M.N., Krajewski, W.F., Cuykendal, R.R.: Rainfall Forecasting in Space and Time Using a Neural Network. Journal of Hydrology 137, 1–37 (1992) 8. Friedman., J.H., Turkey, J.W.: A Projection Pursuit Algorithm for Exploratory Data Analysis. IEEE Transaction On Computers 3(9), 881–889 (1974) 9. Filzmoser, P., Serneels, S., Croux, C., Van Espen, P.J.: Robust Multivariate Methods: The Projection Pursuit Approach. In: Spiliopoulou, M., Kruse, R., N¨ urnberger, A., Borgelt, C., Gaul, W. (eds.) From Data and Information Analysis to Knowledge Engineering Proceedings of the 29th Annual Conference of the Gesellschaft f¨ ur Klassifikation e.V., University of Magdeburg, vol. 8, pp. 270–277. Springer, Heidelberg (2006) 10. Johenson., D.: Applied Ultivariate Methods for Data Analysts, 2nd edn. Thomson Learning Press, UK (1998) 11. Helland, I.S.: PLS Regression and Statistical Models. Scandivian Journal of Statistics 179, 97–114 (1990) 12. Kenneth, D.S.: Using recursive regression to explore nonlinear relationships and interactions: A tutorial applied to a multicultural education study. Practical Assessment Research and Evaluation 14(3), 1–13 (2009) (Retrieved March 3, 2009) http://pareonline.net/getvn.asp 13. Jiansheng, W.: A Novel Artificial Neural Network Ensemble Model Based on K–nn Nonparametric Estimation of Regression Function and Its Application for Rainfall Forecasting. In: 2nd Internatioal Joint Conference on Computational Sciences and Optimization, pp. 44–48. IEEE Computer Society Press, New York (2009) 14. Benediktsson, J.A., Rveinsson, J., Ersoy, O.K., Swain, P.H.: Parallel Consensual Neural Neural Networks. IEEE Transactions on Neural Networks 8, 54–64 (1997) 15. Fredric, M.H., Ivica, K.: Principles of Neurocomputing for Science & Engineering. McGraw-Hill, New York (2001) 16. Suykens, J., Van Gestel, T., De Brabanter, J., De Moor, B., Vandewalle, J.: Least Squares Support Vector Machines. World Scientific Publishing Press, Singapore (2002)

Estimating Portfolio Risk Using GARCH-EVT-Copula Model: An Empirical Study on Exchange Rate Market Zongrun Wang1, Yanbo Jin2, and Yanju Zhou1 1

2

School of Business, Central South University, Changsha 410083, China Department of Finance, California State University, Northridge 91325, USA

Abstract. This paper introduces GARCH-EVT-Copula model and applies it to study the portfolio risk of exchange rates. Multivariate Copulas including Gaussian Copula, t Copula and Clayton Copula were used to describe the structure and extend the analysis from bivariate to any n-dimension. We apply this methodology to study the returns of a portfolio of four major foreign currencies in China. Our results suggest that the optimal investment allocations are similar across different Copula and confidence levels and the optimal investment concentrates in the USD investment. Generally speaking, t Copula and Clayton Copula can better portray the correlation structure of multiple assets than Normal Copula. Keywords: GARCH-EVT-Copula model, Exchange rate, Portfolio risk, VaR and CVaR.

1 Introduction Value-at-Risk (VaR) is one of the most important measures of the market risk and Conditional VaR (CVaR) has emerged as a supplement to VaR. It provides a more conservative measure of losses relative to VaR. Extreme value theory (EVT) focuses directly on the tails and could therefore potentially give us better estimates and forecasts of risk. But applying EVT to the return series is inappropriate as they are not independent and identically distributed. Thus following the approach of McNeil and Frey [1], we use GARCH model to fit the return series and apply EVT to the residuals rather than to the return series. Several researchers home and abroad have made great progress using this method to measure VaR, such as Hans N. E. Bystrom [2], Malay Bhattacharyya [3], Xiao [4] and so on. On the other hand, when measuring the risk of a multi-asset portfolio, one needs to model the relation between the risk factors. This has become an active line of research in the finance field in recent years. Copulas are widely used along this line of research, because it is able to extract the dependence structure from the joint probability distribution function and, simultaneously, to isolate such dependence structure from the univariate marginal distributions. However, most empirical studies have focused on the equity risk using Gaussian Copula and t Copula. Among the few studies that did focus on the exchange rate risk. The Archimedean Copula is rarely used because of implementation issues and the Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 65–72. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

66

Z. Wang, Y. Jin, and Y. Zhou

analysis was limited to two-dimension [5,6]. Tursunalieva [7] extends [5,6] and shows that the SJC-Copula captures the tail dependence between the exchange rates in pre-euro and post-transition periods, while the rotated Gumbel Copula captures the dependence during the transition period. In China, Wu [8] was the first to analyze two-asset portfolio of foreign exchanges using Archimedean Copula. Sun [9] considered the EUR/RMB and JPY/RMB exchange rates, using variety of Archimedean Copula to fit the data and derived the optimal Copula to capture the two variable’s correlation. Wang [10] uses EVT to measure the exchange rate risk and builds Copula-EVT model and calculates the VaR for the exchange rate portfolio of EUR/CNY and JPY/CNY by Monte Carlo simulation. On the other hand, some researchers combine GARCH model and Copula to dynamically study the correlation and risk of financial assets, such as Jondeau [11,12], Tae-Hwy lee [13], Wu[14] and so on. In this paper, we introduce Clayton Copula to apply to the exchange rate portfolio. The contributions of our paper are two folds. First, in addition to Gaussian Copula and t Copula, we use the multivariate Archimedean Copula, in particular the Clayton Copula, to analyze the asymmetric dependence structure among financial asset returns. Second, we extend the bivariate copula methods in a trivariate context, and use GARCH nested copulas with EVT margins to estimate the risk of fourdimensional exchange rate portfolios.

2 GARCH-EVT Model and Copula Parameter Estimates 2.1 GARCH-EVT Model Recognizing the heteroskedasticity and volatility clustering nature of time series, we first use GARCH (1,1) to model the time series. The model is:

⎧⎪ rt = μ + ε t = μ + σ t zt ⎨ 2 2 ⎪⎩σ t = w + αε t −1 + βσ t −1

(1)

Where w > 0, α ≥ 0, β ≥ 0, α + β < 1 . rt is the actual return, μ is the expected return and σ t is the volatility of the returns on day t . zt comes through the stochastic variables, which are the residuals or the innovations of the process. Conventionally these residuals are assumed to follow a normal distribution. However, the conditional distribution of GARCH models has been shown to have a heavier tail than that of a normal distribution, which provides better approximation to actual financial time series. Furthermore, EVT appears to be an appropriate approach for modeling the tail behavior. However, applying EVT to the random variable rt is inappropriate as rt is not i.i.d. Therefore, our marginal model is built on GARCH-EVT model, in which we use GARCH model to fit the historical return data, then model the new series by the GPD distribution in the lower or upper tail and by the empirical distribution in the remaining part. The distribution of innovation is followed by (see details in Stuart Coles [15]):

Estimating Portfolio Risk Using GARCH-EVT-Copula Model −1/ ξ ⎧N ⎧ L ⎫ ⎪ u L ⎨1 + ξ L u − zt ⎬ zt < u L ⎪ N ⎩ βL ⎭ ⎪⎪ F ( zt ) = ⎨φ ( z ) u L < zt < u R ⎪ −1/ ξ R R ⎪ Nu R ⎧ R u − zt ⎫ zt > u R ⎬ ⎪1 − N ⎨1 + ξ R β ⎩ ⎭ ⎪⎩

67

L

(2)

Where β , ξ are the scale and shape parameter respectively, u R ( u L ) is the upper (lower) threshold. In order to get β , ξ exactly, we must choose the proper threshold. 2.2 Copula Parameter Estimates

The following theorem is known as Sklar’s Theorem. It is the most important theorem about copula functions because it is used in many practical applications. Theorem: Let F be an n-dimensional c.d.f. with continuous margins F1 , F2 , it has the following unique copula representation:

xn ) = C ( F1 ( x1 ), F2 ( x2 ),

F ( x1 , x2 ,

Fn ( xn ))

Fn .Then

(3)

We can see that, for continued multivariate distribution, the univariate marginal distribution and the multivariate dependence structure can be separated. Assuming that the dependence structure does not change with time, we select Gaussian Copula, t Copula and Clayton Copula to estimate the joint distribution. For n-variable Gaussian Copula, Let φ∑ be the standardized multivariate normal distribution with correlation matrix ∑ . It can be defined as follows:

C Gu (u1t , u2t ,

, unt ) = φ∑ (φ −1 (u1t ), φ −1 (u2t ),

, φ −1 (unt ))

(4)

Where φ −1 denotes the inverse of the normal cumulative distribution function. From (2) we know the marginal distribution is Fi ( z ) . Based on the historic data {z1t , z2t znt }, t = 1, 2 T ,we set: ut = (u1t , u2t

unt ) = ( F1 ( z1t ), F2 ( z2t )

ξt = (φ (u1t ), φ (u2t ), φ (unt )) −1

−1

−1

Fn ( znt ))

(5)

So we can get the simple expression: C Gu (ut ) = φ∑ (ξt ) . And using MLE method, ∑ is estimated as follows: T ˆ = 1 ξξ ' ∑ ∑ t t T t =1

(6)

To capture the fat tail property, we introduce the multivariate student’s t Copula which shows more observations in the tails than the Gaussian. Let tv , ∑ be the

68

Z. Wang, Y. Jin, and Y. Zhou

standardized multivariate t distribution with correlation matrix ∑ and v degrees of freedom. The t Copula can be defined as follows: C t (u1t , u2t ,

−1

−1

unt ) = tv , ∑ (tv (u1t ), tv (u2t ),

−1

(7)

tv (unt ))

−1

Where tv denotes the inverse of the Student’s t cumulative distribution function. Based on the historic data and the given degree of freedom v , we set: ut = (u1t , u2t −1

unt ) = ( F1 ( z1t ), F2 ( z2t )

ζ t = (tv (u1t ), t (u2t ), −1 v

Fn ( znt ))

(8)

−1 v

t (unt ))

So we have C t (ut ) = tv , ∑ (ζ t ) . The parameter matrix ∑ is also estimated using MLE method. The initial matrix is the correlation coefficient matrix of multivariate normal ˆ Copula function made from (6). We can get the correlation coefficient matrix ∑ of n +1

multivariate t Copula through the following iterative calculation method:

ζ tζ t ' 1 ⎛v+n⎞ T ˆ k = 1, 2, ∑ k +1 = ⎜ ⎟⋅∑ T ⎝ v ⎠ t =1 1 + 1 ζ ' ∑ ˆ ζ t k t v

(9)

ˆ =∑ ˆ . In addition to the fat tails, we can try to Repeat the above process until ∑ n +1 n check whether asymmetry exists in the tails. We use the Clayton Copulas, one of the Archimedean families better known in capturing left tail dependence. The Clayton Copula’s cumulative distribution functions are defined by the following:

C clayton (u1 , u2 ,

, un ) = (u1−θ + u2 −θ +

+ un −θ − n + 1)

−

1

θ

,θ ≥ 0

(10)

Base on the historic data and (2), we have ui = Fi ( zit ) and can directly estimate the parameter θ of n-variable Clayton Copula using MLE method.

3 Simulation Algorithm and Portfolio Risk Analysis 3.1 Simulation Algorithm

ˆ First, for ∑

derived from (6), we do Cholesky decomposition and then

have ∑ = A ' A . Second, draw independent n-dimensional vectors from the multiple normal distribution x = ( x1 , x2 ,

, xn ) ' , xi ∼ N (0,1) , set y = A ' x ,then

z = ( F (φ ( y1 )), F2−1 (φ ( y2 )),

Where Fi −1

， i = 1, 2,

−1 1

, F1−1 (φ ( yn )))

(11)

, n is the inverse of distribution of Fi in (2). Third, repeat the

above step and simulate M times, we can get vector ( z1m , z2 m , , znm ) ' m = 1, 2 M . Then restoring it into (1) we can get M returns at the time t+1. The returns residuals’

Estimating Portfolio Risk Using GARCH-EVT-Copula Model

69

joint distribution is this Gaussian Copula. The returns can be defined by rt +1 = (r1m , r2 m , , rnm ) ' = ( μ1 + z1mσ 1,t +1 , μ 2 + z2 mσ 2,t +1 , , μ n + znmσ n ,t +1 ) ' , Where σ i ,T +1 , μi , i = 1, 2,

, n are calculated by GARCH (1,1) model. ˆ derived from (9) we firstly do Cholesky decomposition. On the other hand, for ∑ After that, generate n random vectors which are i.i.d, then produce random vector s subjecting to Chi-square distribution and independent from x , and set: z = ( F1−1 (t (

，

y1 s/v

y2

)), F2−1 (φ (t (

s/v

))),

, Fi −1 (φ (

yi s/v

)))

(12)

Where Fi −1 i = 1, 2, , n is the inverse of distribution of Fi in (2). Finally, repeat the above step M times, we can also get the vector and the simulating returns via t Copula at the time t+1. A hurdle for practical implementation of any multivariate Archimedean Copula was the absence of an efficient method to generate returns. Marshall and Olkin proposed an alternative method, which is computationally straightforward than the conditional distribution approach. It requires the generation of an additional variable. 1 1 ˆ −1 = t − θ − 1 , Considering the inverse function of Gamma( ,1) ’s LT transformation Ga

θ

it is different from the generator φ (t ) =

1

θ

(t −θ − 1) of Clayton Copula just a constant

and it does not affect generating Clayton Copula. So the algorithms of simulating returns via Clayton Copula are as follows: First, Generate M random variables Ym , m = 1, 2 M which obey the function 1 Gamma( ,1) , where the parameter θ is estimated using MLE. Second, simulate

θ

independent

uniform

random

variables y1 , y2 ,

, yn .Third,

Let uim = Ga(

− ln( yi ) ) , i = 1, 2, Ym

u = (u1 , u2 ,

, un ) is the n-dimension Clayton Copula with the parameter θ , then

, n, m = 1,

M , then the joint distribution of

−1

zim = Fi (uim ), i = 1, 2, , n, m = 1, 2, , M . Finally, similar to the last step of simulating via Gaussian Copula or t Copula above, we can also get M returns via Clayton Copula at the time t+1. 3.2 Portfolio Risk Analysis 3.2.1 VaR and CVaR of the Equally-Weighted Portfolio As we know, the above algorithms have simulated the returns at time t+1 of ndimension time series using the three Copula functions to describe the correlation structure. Now suppose we want to compute the empirical VaR and CVaR of an equally-weighted portfolio. Assuming there are n-dimension assets we can invest and

70

Z. Wang, Y. Jin, and Y. Zhou

1 1 1 the weight in each asset is the same, that is w = [ , , , ]' , according to the M n n n returns samples simulated above, we can get M total returns of the portfolio: Re turns = rt +1 * w = (r1m , r2 m , , rnm ) * w . As a result, we can get the empirical distribution and thus the VaR and CVaR of the equally-weighted portfolio. 3.2.2 Optimal Portfolio with Minimum Risk The discussion above focused on estimating the VaR and CVaR of an equallyweighted portfolio. However, for individual investors, one of the major concerns is to minimize risk of the investment portfolio. As a result, the following algorithm computes the optimal weights of each asset which minimizes the portfolio risk. Assuming that the weight in individual assets within a portfolio is w = [ w1 , w2 , , wn ]' ,

where 0 ≤ wi ≤ 1 and w1 + w2 + + wn = 1 . Needless to say, there are many weight vectors that satisfy these conditions. For each given investment weight vector w , we can get M returns of the portfolio; that is, Re turns = rt +1 * w = (r1n , r2 n , , rnm ) * w and compute VaR of portfolio under a given confidence. After getting all VaR values corresponding to all the weight vectors, we can find the minimum VaR and return back to get the weight in each asset; that is, min VaRα = risk ( w1 , w2 , , wn ) , also we can get the CVaR value under this given investment proportion.

4 Empirical Studies in Foreign Exchange Market We choose four foreign exchange rates including USD/CNY, EUR/CNY, JPY/CNY, HKD/CNY to demonstrate the application of GARCH-EVT-Coupla model. As we know, China has started implementing the new exchange rate policy since July 21,2005. The data series used here are from July 25, 2005 to July 25, 2008.1. We use the daily logarithmic returns defined as rt = ln Pt − ln Pt −1 . The data series give us 732 observations for each exchange rate. 4.1 Summary Statistics

The descriptive statistics of daily logarithmic returns show that all series have heavy tails and they do not follow normal distribution. Take USD/CNY as an example, the ADF test result of USD/CNY indicates that there is no unit root and that the series follow a stationary stochastic process. Then we perform ARCH LM test to find that USD/CNY is a stationary stochastic process, but with heteroskedasticity. And the analysis on the other three exchange rates shows similar properties. As a result, we can use GARCH (1,1) model as in (1) to study the exchange rate movement. 4.2 GARCH-EVT-Copula Application

We first establish GARCH-EVT model to the each RMB exchange rate series and get the marginal distribution of residuals. Second use Multivariate Gaussian Copula, t 1

Data source: http://www.safe.gov.cn/model_safe/index.html

Estimating Portfolio Risk Using GARCH-EVT-Copula Model

71

Copula and Clayton Copula to describe the correlation of the residuals and obtain the Copula parameters. Finally adopt Monte Carlo to simulate returns considering the correlation structure, calculate the risk value under the same investment proportion and find the optimal investment proportion under the given confidence. As described in earlier sections, we can determine the threshold μ . The other parameters can be obtained using the method of MLE. We find that GPD fits the residuals of all series perfectly. After obtaining the parameters of GPD and the residuals i = 1, 2,3, 4, t = 1, 2 T , we substitute zit into (2) and get the marginal distribution ui = f ( zi ) . According to the parameter estimation method in the section 2.1 we can get the parameters of Copulas. That is the correlation matrix ∑ of Gaussian Copula, ∑ and freedom degree v of t Copula, the parameter θ of Clayton Copula. Applying the simulation algorithm described in section 2.2 above, we can simulate the returns at the time t+1 considering correlation structure using the three types of Copula. When the investment proportion of the four foreign exchanges were equal we can calculate VaR and CVaR of the portfolio. In addition, we can find the optimal investment proportion to minimize the portfolio risk. The results indicate that under the same confidence level, VaR(CVaR) calculated from t Copula and Clayton Copula is less than that from Gaussian Copula. That is because t Copula and Clayton Copula consider the tail correlation but the Gaussian Copula does not. In addition, the optimal portfolio weights under minimum portfolio risk are similar across different Copula and confidence levels. The investment concentrates on USD, although the price of USD/CNY has decreased since 2007 and the volatility has increased. However, the volatility of USD/CNY is still smaller than that of EUR/CNY and JPY/CNY, as a result, during the sample period of July 2005-July 2008, USD investment accounted for the largest proportion in the optimal investment under minimum portfolio risk. With higher confidence level, the proportion of USD investment becomes smaller and the proportions of the other foreign currencies become larger. When α = 0.9 , VaR using Clayton Copula is larger than those using t Copula and Gaussian Copula. However, with higher confidence level, Clayton Copula better captures the correlations and results in VaR that is similar to the result using t Copula. As a result, under the higher confidence level, we can choose t Copula and Clayton Copula to calculate the portfolio risk.

，

5 Conclusions In this paper we introduce the GARCH-EVT-Copula model and use it to measure the risk of foreign exchange portfolio with multiple foreign exchanges. When modeling the correlation structure we not only apply the conventional Gaussian Copula and t Copula, but also the multivariate Clayton Copula which was rarely used because the complexity involved in programming. We find that for an equally-weighted foreign exchange portfolio, VaR(CVaR) calculated from t Copula and Clayton Copula is less than that computed from the Gaussian Copula. This is because t Copula and Clayton Copula model the tail correlation but the Gaussian Copula does not. On the other hand, when minimizing portfolio risk, optimal portfolio weights are similar across different copula and

72

Z. Wang, Y. Jin, and Y. Zhou

different confidence levels. The optimal investment tends to concentrate in the investment of USD. With higher confidence level, weight in USD decreases while weight in EUR and JPY increases. In addition, we show that t Copula and Clayton Copula provide better approximation to capture portfolio risk compared to Gaussian Copula. These findings can help investors better manage the exchange rate risk. Acknowledgments. The work was supported by the National Natural Science Foundation of China under the grant No. 70973145 and 70771114.

References 1. Frey, R., McNeil, A.J.: Estimation of tail-related risk measures for heteroscedastic financial time series: an extreme value approach. Journal of Empirical Finance 7, 271–300 (2000) 2. Bystrom, H.N.E.: Managing extreme risks in tranquil and volatile markets using conditional extreme value theory. International Review of Financial Analysis 13, 133–152 (2004) 3. Malay, B., Gopal, R.: Conditional VaR using EVT- Towards a planned margin scheme. International Review of Financial Analysis 17, 382–395 (2008) 4. Xiao, Z., Fu, X.X., Zhang, B.: Dynamic VaR Risk Measures Based on EVT-POTFIGARCH. Nankai management review 11(4), 100–104 (2008) (in Chinese) 5. Patton, A.J.: Modelling time-varying exchange rate dependence using the conditional Copula. Working Paper of London School of Economics & Political Science (2001) 6. Patton, A.J.: Modeling asymmetric exchange rate dependence. International Economic Review 2(47), 527–556 (2006) 7. Tursunalieva, A., Silvapulle, P.: Assessing and modeling the changes in dependence between exchange rates. Working Paper of Monash University (2007) 8. Wu, Z.X., Ye, W.Y.: Risk Analysis of Foreign Exchange Markets by Copula. Chinese management science 4, 1–5 (2004) (in Chinese) 9. Sun, L.J., Bai, M.Y.: The Research of Optimal Archimdean Copula Based Exchange Rate Correlation. Data Analyses 3(2), 91–102 (2008) 10. Wang, Z.R., Chen, C.: The exchange rate risk of Chinese yuan: using VaR and ES based on extreme value theory. Journal of Applied Statistics 37(2), 265–282 (2010) 11. Eric, J., Micheal, R.: Copula-GARCH model of Conditional Dependency of Financial Series: the Copula-GARCH Model. Working Paper (2002) 12. Eric, J., Micheal, R.: The Copula-GARCH model of Conditional Dependency: An international stock market application. Journal of International Money and Finance 25, 827–853 (2006) 13. Lee, T.H., Long, X.D.: Copula-based multivariate GARCH model with uncorrelated dependent errors. Journal of Econometrics 150, 207–218 (2009) 14. Wu, Z.X., Chen, M., Ye, W.Y.: Risk Analysis of Portfolio by Copula-GARCH. Journal of Systems Engineering Theory and Practice 2(8), 45–52 (2006) (in Chinese) 15. Coles, S.: An Introduction to Statistical Modeling of Extreme Values. Series in Statistics, New York (2001)

Forecasting Financial Time Series via an Efficient CMAC Neural Network Chi-Jie Lu1 and Jui-Yu Wu2,* 1

Department of Industrial Engineering and Management, Ching Yun University, Taiwan [email protected] 2 Department of Business Administration, Lunghwa University of Science and Technology, Taiwan [email protected]

Abstract. Cerebellar model articulation controller neural network (CMAC NN) has many advantages, such as very fast learning, reasonable generalization capability and robust noise resistance. Thus, CMAC NNs are conventionally used in robot control. To solve financial time series forecasting, this paper presents an efficient CMAC NN scheme. The proposed CMAC NN transforms continuous values of input variables to discrete indexes by using a quantization operator. To enhance generalization ability, the CMAC NN employs high quantization resolution and a large generalization size. To perform many-to-few mappings, the CMAC NN uses an efficient and fast hashing code based on bitwise XOR operator. The proposed CMAC NN was used to Nikkei 225 closing cash indexes collected from Japanese stock market. The forecasting results of the proposed CMAC NN were compared with those of support vector regression (SVR), which is statistical/ machine learning algorithm. Experimental results indicate that the performance of the CMAC NN was better than SVR in the tested case. Therefore, the CMAC NN may be considered as an efficient tool for forecasting financial time series. Keywords: cerebellar model articulation controller, neural network, stock price prediction, support vector regression.

1 Introduction Over the past three decades, there has been a growing interest in financial time series forecasting [1, 2]. Stock index forecasting is regarded as a challenging task of the financial time series prediction process since the stock market is a complex, evolutionary, and nonlinear dynamic system [3]. Many techniques have been applied to the domain of time series prediction, and are traditionally categorized as statistical methods and spectral analysis [4, 5]. Zhang et al. [6] found that the traditional statistical methods based on linear models, such as Box-Jenkins approach, have difficulty in measuring the performance of real systems with nonlinear behavior. Rigoze et al. [7] compared the advantages and drawbacks of four classical spectral analytical methods. These approaches are difficult to apply for general practitioners; Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 73–82. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

74

C.-J. Lu and J.-Y. Wu

that is, prior knowledge of signal processing is required to adjust specific parameters for these spectral analytical methods. Artificial neural networks (ANNs) which can accurately model nonlinear systems. Unlike traditional statistical models, NNs are data-driven and non-parametric models. A cerebellar model articulation controller (CMAC) is a supervised NN. Albus [8, 9] first introduced the CMAC NN based on the functions of the human cerebellum. The CMAC NN has the advantages of very fast learning, reasonable generalization ability and robust noise resistance [10]. It has been successfully used in many applications, such as control [11, 12], fault detection [13, 14] and classification [15, 16]. Moreover, CMAC NNs have been used to predict, such as electricity price forecasting and power system marginal price forecasting [17, 18]. However, this paper attempted to extend the application field to stock price forecasting. This paper presented a CMAC NN scheme for stock index prediction. Moreover, this paper compared the performance of the CMAC NN and support vector regression (SVR), which is a novel machine learning algorithm, and is increasing attention to solve nonlinear regression estimation problems.

2 Literature Review 2.1 CMAC NN A CMAC NN comprises five cells, input space ( X ), sensory cell ( S ), association cell ( A ), physical memory cell ( P ) and output cell ( Y ), and transforms input values into output values using a series of mappings, as follows: (1) X → S A quantization operator is used to transform components of input vectors into discrete quantization indexes. Each component xi of input pattern j is quantized individually by:

⎡⎛ N sij = ⎢⎜ max i min ⎜ ⎣⎢⎝ xi − xi

⎤ ⎞ min ⎥ − 1, i = 1, 2,… , n j = 1, 2,… , Dtotal ⎟⎟ × xij − xi ⎥⎦ ⎠

(

)

sij = 0, if sij < 0 where sij = quantization index of component xi of input pattern j xij = component xi of input pattern j ximin = minimum value of input vector xi ximax = maximum value of input vector xi Ni = quantization resolution of input vector xi n = dimension of input space Dtotal = total number of input patterns

[⋅]

= rounded number

(1)

(2)

Forecasting Financial Time Series via an Efficient CMAC Neural Network

75

The indexes sij depends on ximin , ximax and Ni . Larger values of Ni improve the accuracy of representation, but require a large weight table w . (2) S → A A sensory cell comprises random tables [19]. The size of each random table is calculated by: Ci = Ni + g − 1, i = 1, 2,… , n

(3)

where

Ci = size of the random table of input vector xi g = generalization size The indexes sij are mapped to random table i , and segment mappings are created based on g . The quantization and segment mappings generate natural interpolation, and give the CMAC NN the ability to generalize [20]. A large g increases generalization ability, but reduces the approximation accuracy. (3) A → P → Y A vector a in cell A corresponds to a table w in cell P . The table w stores many weights. An actual output value in cell Y can be computed by the matrix operation:

yo = aT w

(4)

where ⎡ a1 ⎤ ⎢ ⎥ a a = ⎢ 2⎥ , w = ⎢ ⎥ ⎢ ⎥ ⎣ ak ⎦

⎡ w1 ⎤ ⎢ ⎥ ⎢ w2 ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ wk ⎦

y o = actual output, k = size of w and T = transpose of a matrix. 2.2 SVR The SVR is an adaptation of recently introduced statistical/ machine learning theory based classification paradigm namely, support vector machines. For illustrating the concept of SVR, a typical regression problem is formulated. Consider a dataset G = ( xij , yd , j ) ( i = 1, 2,..., n , j = 1, 2,..., Dtotal ), where xij are input patterns and yd , j is desired output. The goal of the regression analysis is to determine a function f (x) , so as to predict accurately the yd , j . The regression problem can be classified as linear and nonlinear regression problems. As the nonlinear regression problem is more difficult to deal with, SVR was mainly developed for dealing with the nonlinear regression problem. SVR algorithms have been successfully applied in time series prediction, such as and financial time series forecasting [21, 22]. When constructing a

76

C.-J. Lu and J.-Y. Wu

forecasting model, one major concern is increasing the accuracy of predictions. This would be especially beneficial in areas like stock price forecasting, where even the smallest improvement in prediction accuracy can have a positive impact on investments. For building SVR forecasting model, the LIBSVM package proposed by Chang and Lin [23] is used in this paper since the package can be easy to find and use.

3 Method A CMAC NN was developed and used to Nikkei 225 closing cash index dataset collected from Japanese stock market. Figure 1 shows the pseudo-code of the proposed CMAC NN, which is described below. begin Training stage: h←0 Step 1: quantize and normalize Step 2: create random tables and weight table While h < epochmax do for j = 1,2,…, Dtrain do Step 3: generate address table Step 4: calculate of actual output Step 5: learning rule endFor Step 6: evaluate training accuracy h ← h +1 endWhile Testing stage: Step 7: measure testing accuracy end

Fig. 1. The pseudo-code of the proposed CMAC NN

Step 1: Quantize and normalize Equations (1) and (2) are used to quantize components of input patterns. Each desired output

yd , j ( j = 1, 2,… , Dtotal ) undergoes normalization, as follows: yd' , j =

yd , j − ydmin ydmax − ydmin

( Emax − Emin ) + Emin ,

j = 1, 2,… , Dtotal

where

yd' , j = normalized desired output j ydmin = minimum value of desired output vector Yd = [ yd ,1 , yd ,1 ,… yd , Dtotal ]T ydmax = maximum value of Yd Emin = minimum value of expected output Emax = maximum value of expected output The values [ Emin , Emax ] are generally set to [0.2, 0.8].

(5)

Forecasting Financial Time Series via an Efficient CMAC Neural Network

77

Step 2: Create random tables and weight table w Equation (3) is used to compute the size of each random table, and the random tables are then created. To control the range of the index of generated address table, each random table consists of uniform random numbers created from the interval [ 0, k / 2 ]. A table w is generated based on k as defined in Section 2, with initial weights set to “0”. To reduce the probability of hash collision, this paper used a large value of k = 10000. Step 3: Generate address table To implement many-into-few mapping, the proposed CMAC NN used a hash coding based on bitwise XOR operator [24]. The detail description of the hash coding can be found in the literature [15]. Step 4: Calculate actual output Equation (4) is used to measure the values yo, j ( j = 1, 2,… , Dtrain ). Step 5: Learning rule Equation (6) is employed to modified the weights in w , as follows: wl (h + 1) = wl (h) +

β ( yd − yo ) g

, l = 1, 2,… , k

h = 1, 2,… , epochmax

(6)

where β = learning rate Step 6: Evaluate training accuracy A normalized root mean square error (NRMSE) is used as a performance index to evaluate a training accuracy, as follows: Dtrain

∑

NRMSE =

j =1

( yd' , j − yo, j )2

(7)

Dtrain

where Dtrain = number of input patterns for training Steps 3–6 are repeated until the termination condition epochmax is met. Step 7: Measure testing accuracy Input pattern j ( j = Dtrain + 1, Dtrain + 2,… , Dtotal ) is used for testing. The actual outputs of recall of the CMAC NN can be calculated using the w obtained in the training stage (Steps 1−6), and then the generalization accuracy of the CMAC NN can be measured by a testing NRMSE.

4 Experimental Results For evaluating the performance of the proposed CMAC NN forecasting model, the daily Nikkei 225 closing cash index dataset is used in this paper. In forecasting Nikkei

78

C.-J. Lu and J.-Y. Wu

225 closing cash index, the Nikkei 225 index future prices are used as forecasting variables since the changes of the future prices lead price changes of the cash market [25, 26]. Four forecasting variables (Singapore Exchange-Derivative Trading Limited, Osaka Securities Exchange, Chicago Mercantile Exchange and the previous day’s cash market closing index) are used for predicting the Nikkei 225 closing cash index. The daily data of futures and cash prices from December 6, 2004 to December 30, 2008 of the Nikkei 225 cash index provided by Bloomberg are collected in this paper. There are totally 1000 data points in the dataset and the daily Nikkei 225 closing cash prices are shown in Fig. 2. The first 800 data points (80% of the total sample points) are used as the training sample while the remaining 200 data points (20% of the total sample points) are used as the testing sample.

19000 17000 Prices

15000 13000 11000 9000 7000 5000 1

100

200

300

400

500

600

700

800

900

1000

Data points

Fig. 2. The daily Nikkei 225 closing cash prices from December 6, 2004 to December 30, 2008

The proposed CMAC NN was coded in MATLAB. This paper employed epochmax = 200 as a termination condition, β = 0.1 and k = 10000. The optimal settings of parameters ( Ni , g ) for the proposed CMAC NN was evaluated by using Ni = {30, 40, 50} and g = {10, 20, 30, 40, 50}. Table 1 lists the model selection results of the proposed CMAC NN. When using a large value of Ni = 50, increasing g decreased the training accuracy, but enhanced the generalization ability of the CMAC NN. When using small values of Ni = 10 and Ni = 20 against various g = {10, 20, 30, 40, 50}, the poor training and generalization accuracies were found. As shown in Table 1, the CMAC NN obtained the best training and testing RMSEs by using the parameter settings ( Ni , g ) = (50, 40), which are high quantization and a large generalization size. For building SVR forecasting model, the LIBSVM package proposed by Chang and Lin [23] is used in this paper. The original dataset is first scaled into the range of [−1.0, 1.0] when using LIBSVM package. The parameter setting of regularization constant (C) and precision parameter ( ε ) which generate the minimum forecasting root mean square error (RMSE) is considered as the best parameter setting. In the modeling of the SVR model, C=1.25 and ε =0.0019 can be obtained. Since C=1.25 is near C=21 and ε =0.0019 is close to ε =2-9, the parameter set (C=21, ε =2-9) is used as the starting point of grid search for searching the best parameters. The testing results

Forecasting Financial Time Series via an Efficient CMAC Neural Network

79

of the SVR model with combinations of different parameter sets are summarized in Table 2. From Table 2, it can be found that the parameter set (C=23, ε =2-9) gives the best forecasting result (minimum testing RMSE) and is the best parameter set for SVR model in forecasting Nikkei 225 closing cash index. Table 1. Model selection results of the proposed Table 2. Model selection results of the SVR CMAC NN model-Nikkei 225 forecasting model-Nikkei 225

The prediction performance is evaluated using the following performance measures, namely, the root mean square error (RMSE), mean absolute difference (MAD), mean absolute percentage error (MAPE), directional accuracy (DA), correct up trend (CP) and correct down trend (CD). The definitions of these criteria can be found in Table 3. RMSE, MAD and MAPE are measures of the deviation between actual and predicted values. Table 3. Performance measures and their definitions

Metrics

Calculation

Metrics

Dtotal

RMSE

RMSE =

∑ ( yd , j − yo , j ) 2 j =1

Dtotal Dtotal

MAD

MAD =

∑ j =1

∑

MAPE =

yd , j − yo , j

j =1

CP

yd , j

⎧1 dj = ⎨ ⎩0 CD =

yd , j − yo , j Dtotal

⎧1 dj = ⎨ ⎩0 CP =

Dtotal Dtotal

MAPE

DA

Calculation 100 DA = Dtotal

CD

⎧1 dj = ⎨ ⎩0

Dtotal

∑ d j , where j =1

(yo , j − yo , j −1 )( yd , j − yd , j −1 ) ≥ 0 otherwise 100 Dtotal ∑ dj n1 j =1

, where

(yo , j − yo , j −1 )( yd , j − yd , j −1 ) ≥ 0 otherwise 100 Dtotal ∑ dj n2 j =1

, where

(yo , j − yo , j −1 )( yd , j − yd , j −1 ) ≥ 0 otherwise

* n1 is number of data points belong to up trend and n2 is number of data points belong to down trend.

80

C.-J. Lu and J.-Y. Wu

The Nikkei 225 closing cash index forecasting results using the SVR and proposed CMAC NN models are computed and listed in Table 4. From Table 4, it can be found that the RMSE, MAD and MAPE of the CMAC NN model are, respectively, 55.36, 35.50 and 0.35%. It can be observed that these values are smaller than those of SVR models. It indicates that there is a smaller deviation between the actual and predicated values using the proposed CMAC NN model. Moreover, compared to the SVR models, the CMAC NN model has the highest DA (directional accuracy), CP (correct up trend) and CD (correct down trend) ratios which are 81.58%, 83.33% and 79.62%, respectively. DA, CP and CD provide a good measure of the consistency in prediction of the price direction. Thus, the proposed CMAC NN model provides better forecasting results than the SVR model in terms of prediction error and prediction accuracy in the tested dataset. Table 4. The Nikkei 225 closing cash prices forecasting results using the proposed the SVR and proposed CMAC NN models Metrics

RMSE

MAD

MAPE

DA

CD

CP

SVR

61.58

43.20

0.39%

78.95%

81.67%

75.93%

CMAC NN

55.36

35.50

0.35%

81.58%

83.33%

79.62%

Models

5 Conclusion Stock index forecasting is a challenging task and has drawn serious attention during past three decades. This paper developed an efficient CMAC NN to forecast Nikkei 225 closing cash index. This paper compared the forecasting results of the proposed CMAC NN with SVR. Experimental results indicate that the CMAC NN was superior to SVR for the tested dataset. Moreover, the CMAC NN is much easier to use than traditional statistical and spectral analytical methods. Thus, the CMAC NN may be considered as financial tool for stock price prediction. Future work is to compare the performance of the CMAC NN with SVR using the dataset TAIEX closing cash indexes collected from Taiwanese stock market.

Acknowledgements The authors would like to thank the National Science Council of the Republic of China, Taiwan for financially supporting this research under Contract No. NSC 982221-E-262-014-.

References 1. De Gooijer, J.G., Hyndman, R.J.: 25 Years of Time Series Forecasting. International Journal of Forecasting 22, 443–473 (2006) 2. Lawrence, M., Goodwin, P., O’Connor, M., et al.: Judgmental Forecasting: A Review of Progress over the Last 25 Years. International Journal of Forecasting 22, 493–518 (2006)

Forecasting Financial Time Series via an Efficient CMAC Neural Network

81

3. Atsalakis, G.S., Valavanis, K.P.: Surveying Stock Market Forecasting Techniques - Part II: Soft Computing Methods. Expert Systems with Applications 36, 5932–5941 (2009) 4. Chatfield, C.: Time-Series Forecasting. Chapman & Hall/CRC, New York (2001) 5. Priestley, M.B.: Spectral Analysis and Time Series. Academic Press, New York (1981) 6. Zhang, G., Patuwo, B.E., Hu, M.Y.: Forecasting with Artificial Neural Networks: The State of Art. International Journal of Forecasting 14, 35–62 (1998) 7. Rigozo, N.R., Echer, E., Nordemann, D.J.R., et al.: Comparative Study between for Classical Spectral Analysis Methods. Applied Mathematics and Computation 168, 411– 430 (2005) 8. Albus, J.S.: A New Approach to Manipulator Control: The Cerebellar Model Articulation Controller (CMAC). ASME Journal of Dynamic Systems, Measurement, and Control 97, 220–227 (1975) 9. Albus, J.S.: Data Storage in the Cerebellar Model Articulation Controller (CMAC). ASME Journal of Dynamic Systems, Measurement, and Control 97, 228–233 (1975) 10. Wong, Y.F., Sideris, A.: Learning Convergence in the Cerebellar Model Articulation Controller. IEEE Transactions on Neural Networks 3, 115–121 (1992) 11. Lin, C.M., Chen, L.Y., Chen, C.H.: RCMAC Hybrid Control for MIMO Uncertain Nonlinear Systems Using Sliding-Mode Technology. IEEE Transactions on Neural Networks 18, 708–720 (2007) 12. Peng, Y.F.: Robust Intelligent Sliding Model Control Using Recurrent Cerebellar Model Articulation Controller for Uncertain Nonlinear Chaotic Systems. Chaos, Solitons & Fractals 39, 150–167 (2009) 13. Wang, S., Jiang, Z.: Valve Fault Detection and Diagnosis Based on CMAC Neural Networks. Energy and Buildings 36, 599–610 (2004) 14. Hung, C.P., Wang, M.H.: Diagnosis of Incipient Faults in Power Transformers Using CMAC Neural Network Approach. Electric Power Systems Research 71, 235–244 (2004) 15. Wu, J.Y., Lu, C.J.: Applying Classification Problems via a Data Mining Approach Based on a Cerebellar Model Articulation Controller. In: 1st Asian Conference on Intelligent Information and Database Systems, pp. 61–66. Dong Hoi City, Vietnam (2009) 16. Lin, C.J., Lee, J.H., Lee, C.Y.: A Novel Hybrid Learning Algorithm for Parametric Fuzzy CMAC Networks and Its Classification Applications. Expert Systems with Applications 35, 1711–1720 (2008) 17. Zhou, H., Chen, J., Wu, H., et al.: CMAC-Based Short-Term Electricity Price Forecasting. In: Sixth International Conference on Advances in Power System Control, Operation and Management, Hong Kong, pp. 348–353 (2003) 18. Qiaolin, D., Jing, T., Jianxin, L.: Application of New FCMAC Neural Network in Power System Marginal Price Forecasting. In: The 7th International Power Engineering Conference, Singapore, pp. 1–57 (2005) 19. Lee, J.: Measurement of Machine Performance Degradation Using a Neural Network Model. Computers in Industry 30, 193–209 (1996) 20. Handelman, D.A., Lane, S.H., Gelfand, J.J.: Integrating Neural Networks and KnowledgeBased Systems for Intelligent Robotic Control. IEEE Control Systems Magazine 10, 77–87 (1990) 21. Huang, C.L., Tsai, C.Y.: A Hybrid SOFM-SVR with a Filter-Based Feature Selection for Stock Market Forecasting. Expert Systems with Applications 36, 1529–1539 (2009) 22. Lu, C.J., Lee, T.S., Chiu, C.C.: Financial Time Series Forecasting Using Independent Component Analysis and Support Vector Regression. Decision Support Systems 47, 115– 125 (2009)

82

C.-J. Lu and J.-Y. Wu

23. Chang, C.C., Lin, C.J.: Libsvm: A Library for Support Vector Machines, http://www.csie.ntu.edu.tw/~cjlin/libsvm 24. Zobrist, A.L.: A New Hashing Method with Application for Game Playing. Technical report, Computer Sciences Department, University of Wisconsin (1969) 25. Lee, T.S., Chen, N.J.: Investigating the Information Content of Non-Cash-Trading Index Futures Using Neural Networks. Expert Systems with Applications 22, 225–234 (2002) 26. Lee, T.S., Chiu, C.C.: Neural Network Forecasting of an Opening Cash Price Index. International Journal of Systems Science 33, 229–237 (2002)

Forecasting Daily Cash Turnover of Bank with EWMA and SVR Wei-min Ma and Wei Lu School of Information Technology of Beijing Normal University Zhuhai Campus, Zhuhai 519085, China

Abstract. This paper present one forecasting method with Exponential Weighted Moving Average(EWMA) and Support Vector Regression (SVR). The daily cash turnover of the banks is time-serial data, banks need to forecasting daily cash turnover for banking reserve. First, the time series is preprocessed with EWMA method. The EWMAs with different coefficients are selected for forecasting features. And then SVR is used in the transformed dataset with EWMA for forecasting. The experimental result shows that the EWMA can improve the forecasting accuracy, and the SVR is more effective than other method such as 1NN and MLP. Statistical correlation of SVR between the forecasted and actual values is much higher than other method such as 1-NN and MLP. Keywords: Time series, EWMA, SV-regression, SRM.

1

Introduction

Forecasting time-series is important in several domains[1][2] and a large number of studies have used classical statistical method for forecasting series behavior. Advanced methods such as neural networks[3], genetic algorithms[4], Markov model[5] and fuzzy methods have also been frequently used[6]. The one nearest neighbor(1-NN) method is one local approximation technique. Unlike global approximators, local approximation techniques are dependent on the number of free parameters. The well-known methods of local approximation for time-series forecasting include chart representations, nearest neighbor methods[7] and pattern imitation techniques. [8] applied local approximation method in dynamic time-serial forecasting. The Multi-Layer Perceptron(MLP)[9] is a kind of neural net that is capable of generating arbitrary decision boundaries. But when the data change remarkably, the forecast precision of these method will descend obviously. This paper present one forecasting method with Exponential Weighted Moving Average(EWMA)[10]and Support Vector Regression(SVR)[11][12] [13] which can forecast the time series steadily. Firstly we use exponential weighted moving average method(EWMA) to preprocess the time series data. Exponential weighted moving average lends more weight to recent data than the simple moving average, which actually drops old data from its sample. And we also calculated the ratio of original data to 

Corresponding author.

Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 83–90. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com 

84

W.-m. Ma and W. Lu

different EWMA. By comparing, we find that using ratio data can improve the forecasting precision. After the preprocessing, we use SVR method for forecasting. SVMs(Support Vector Machines) are a recently developed class of classifier architectures, derived from the structural risk minimization principle by Vapnik[11]. A promising aspect of the SVM classifiers is that, SVM training involves selecting a small subset of critical data points(known as support vectors) from the original training data base. SVMs are also used to estimate real-valued functions, which is usually denote as SVR. By experiment, we find that SVR is better than 1-NN and MLP. The rest of paper is organized as following: section 2 describes the preprocessing of the original data and forecasting method with EWMA and SVR; the section 3 gives experiment result; and section 4 gives conclusion and future work.

2 2.1

Forecasting Method with EWMA and SVR Domain Description

Our problem is to model the current daily cash turnover of the time-series with past states.The data example is: Table 1. The example of data CDVH     …

OULJLQDO Bank Transaction 5DQN No. Date Weekday Records    Saturday     Sunday     Monday      Wednesday ... … …. … …

… … … … … …

Daily Cash Turnover     …

We denote our data as the form (x, y), where x represent some effective factors such as date, bank No., debit or credit, and so on, y represent daily cash turnover. If we choose to represent daily cash turnover of time-series as vector y = {y1 , y2 , · · · , yn } where n is the total number of points in the series. Often, we also represent such a series as a function of time, e.g. yn = yt , yn−1 = yt−1 , and so on. Then the state of the time-series represents its current daily cash turnover value yt . One simple method of forecasting may be based on identifying the closest neighbor of yt in the past data, say yj , and forecasting yt+1 on the basis of yj+1 . In this paper, we will also refer to states as patterns used in the matching process. The technique of matching structural primitives is based on the premise that the past repeats itself. The dynamic behavior of time-series can be efficiently forecasted by using local approximation. For this purpose, a map between current states and the nearest neighbor past states can be generated for forecasting. The aim of a pattern matching algorithm is to find the closest match of patter ρ in the historical data and use this for forecasting yt+1 . 2.2

Exponential Weighted Moving Average

Exponential smoothing is the most common model-free means of forecasting a future realization of a time series[10]. It requires the specification of a smoothing

Forecasting Daily Cash Turnover of Bank with EWMA and SVR

85

factor which is usually chosen from the data to minimize the average squared residual of previous one-step ahead forecasts. The most basic exponential smoother is the exponentially weighted moving average(EWMA). It has the attraction of being model-free and based on a simple algebraic formula, in terms of a smoothing factor, and therefore enjoys common usage as an ad hoc forecasting procedure. The goal is to estimate the underlying trend in a scatterplot without the use of restrictive models. The exponentially weighted moving average has the advantage of giving the most weight to the most current value. Also it only requires maintaining the value of the previous day’s average rather than the data for the entire period being averaged, so it is very compact. Following above, let y = {y1 , y2 , · · · , yt } be a time series observed of daily cash turnover at equally-spaced time points T1 , T2 , · · · , Tt . We consider the problem of using these data to forecast yt+1 . One of the simplest model-free approaches to this problem is the exponentially weighted moving average(EWMA). It forecasts yt+1 using a weighed average of past observations with geometrically declining weights: t−1  y t+1 = (1 − ω) ω i yt−i (1) i=0

where 0 < ω < 1 is commonly referred to as the smoothing constant. An advantage of using geometric weights is that equ. (1) admits the recurrence formula y t+1 = ωyt + (1 − ω)yt

(2)

with y 1 = y1 . Notice that the sum of the weights in (1)is 1 − ω t which is close to 1 when it is large enough. We use formula (2) to preprocess our bank data. The ω is called EXPONENT, which is the element in the formula that determines the period of the moving average, can set different value. The calculation of the EXPONENT is done by dividing 2 by the number of days in the period being averaged. For example, the EXPONENT for a 200-day moving average is determined by dividing 2 by 200. The result is 0.01 (2/200=0.01). Because our data capacity is not very large, so we let ω takes five values:0.1, 0.2, 0.25, 0.3, 0.4, corresponding to 20, 10, 8, 20/3, 5 days of the moving average. We notate the calculating result of (2) as y(1) for ω = 0.1, y(2) for ω = 0.2, and so on. In respecting that the value of original data or exponential weighted moving average data is very big, we consider using the ratios of them for forecasting. For example, we can create new data y1 = {y11 = y1 /y(1)1 , y21 = y2 /y(1)2 , · · · , yt1 = yt /y(1)t }. We can see that it improves the forecasting precision by latter experiment. 2.3

Support Vector Regression Algorithm

Support vector regression is based on the Structural Risk Minimization(SRM) principle from statistical learning theory [11]. The idea of SRM is to find a hypothesis for which one can guarantee the lowest probability of error. In SVR, this

86

W.-m. Ma and W. Lu

principle is translated to find a separating hyperplane with the largest distance to the closest training samples. Given the training data (xi , yi ), i = 1, · · · , n, xi ∈ Rd , yi ∈ R. SVR maps the input vectors x into a high-dimensional feature space H through some mapping function φ : Rd → H, and constructs an optimal separating hyperplane in this space. In the linear case, φ(x) is an identity function; in the nonlinear case, the mapping φ(·) is performed by a kernel function K(·, ·) which defines an inner product in H. In ε-SV regression Vapnik[11][13], our goal is to find a function f (x) that has at most ε deviation from the actually obtained targets yi for all the training data, and at the same time, is as flat as possible. The function given by an SVR is: f (x) = ω · φ(x) + b. and therefore f (x) =

l 

(αi − α∗i )k(xi , x) + b.

(3)

(4)

i=1

where the threshold b is computed as follows: b = yi − ω · φ(xi ) − ε, αi ∈ (0, C) b = yi − ω · φ(xi ) + ε, α∗i ∈ (0, C).

(5)

The SVR acts well in estimating real-valued functions, regression estimation and signal processing[11] [14]. 2.4

The Forecasting Flow

We first preprocess original data with some basic operations, such as separating the data according to bank No., removing outlines, processing missing value, and so on. So the whole processing flow is shown in Figure 1:

2ULJLQDO'DWD

%DVLF3UHSURFHVV

(:0$SURFHVV

0HDVXUH5HVXOWV

11 0/37HVWLQJ 690 7HVWLQJ GDWD 7UDLQLQJGDWD 11

0/37UDQLQJ 690

Fig. 1. Forecasting processing flow chart

Forecasting Daily Cash Turnover of Bank with EWMA and SVR

3 3.1

87

Experiment Results Experimental Data and Performance Measures

The data comes from the bank, we do experiment for every branch, and forecast the daily cash turnover for the debit and credit respectively. We select randomly 4/5 of the data as training set, the rest as testing set. We use the following measures for estimating the accuracy of the forecasting.In these equations, we denote f as the forecasted value , a the actual value, and a the average of the actual value. 

i (fi −f )(ai −a) (1) The correlation coefficient SSFFSAA . where SF A = , SF = n−1  2  2 f −f ( ) i (a −a) i i , SA = i n−1 . It measures the statistical correlation between the n−1 forecasted and actual values. (2) Mean absolute error.

|a1 − f1 | + |a2 − f2 | + · · · + |an − fn | n

(6)

It is the average of the difference between forecasted and actual value in all test case; it is the average forecasting error. (3) Root mean-squared error.  (a1 − f1 )2 + (a2 − f2 )2 + · · · + (an − fn )2 (7) n If this number is significantly greater than the mean absolute error, it means that there are test cases in which the forecasting error is significantly greater than the average forecasting error. (4) Relative absolute error. |a1 − f1 | + |a2 − f2 | + · · · + |an − fn | |a1 − a| + |a2 − a| + · · · + |an − a|

(8)

It is the total absolute error made relative to what the error would have been if the forecasting simply had been the average of the actual values. (5) Root relative squared error.  (a1 − f1 )2 + (a2 − f2 )2 + · · · + (an − fn )2 (9) (a1 − a)2 + (a2 − a)2 + · · · + (an − a)2 It is the total squared error made relative to what the error would have been if the forecasting had been the average of the absolute value. In latter four error measurements, a lower value means a more precise model, with a value of 0 depicting the statistically perfect model.

88

3.2

W.-m. Ma and W. Lu

Primary Results

First we compare the results of different preprocessing data. The results are showed on the table 2. Table 2. The measures of different data Original ratio1 ratio2 &RUUHODWLRQFRHIILFLHQW    5HODWLYHDEVROXWHHUURU    5RRWUHODWLYHVTXDUHGHUURU   

ratio3   

ratio4   

ratio5   

In table 2, row original represents using original data, row ratio1 represents using data y1 , row ratio2 of using data y2 , and so on. From the table, we can see that the statistical correlation using ratios increase remarkably than using original data, the error measures using ratios reduce much. And using data y1 can obtain best result, corresponding to take EXPONENT ω = 0.1 in formula (2). It is showed that taking 20 days for smoothing region acts best on our data. We can also show the results of 1-NN, MLP and SVR methods on table 3. Table 3. The measures of different methods 1-NN MLP SVR (SMO) &RUUHODWLRQFRHIILFLHQW    0HDQDEVROXWHHUURU    5RRWPHDQVTXDUHGHUURU    5HODWLYHDEVROXWHHUURU    5RRWUHODWLYHVTXDUHGHUURU   

We select the parameter of SVR algorithm, and the table is the best result. From the table, we can see the the forecasting method with EWMA and SVR is much better than other two methods. The correlation of using SVR is much higher than using other two methods, and the error measures of using SVR is the lowest. This prove SVR has better generalization than 1-NN and MLP, for it’s based on Structural Risk Minimization, and other methods are based on Experimental Risk Minimization. The precisions of different methods on some testing data are showed in figure 2: In these figures, the serial number represent testing data set, and forecasting error is defined by error = |yf − ya |/ya (10) where yf is forecasting value and ya is actual value. The forecasting error can reflect the effect of the methods, we can see that the points represent forecasting error of SVR is below these points represent 1-NN and MLP on most data, which proves the SVR does better than the other two methods.

Forecasting Daily Cash Turnover of Bank with EWMA and SVR

89

IRUHFDVWLQJHUURU

  

11 0/3 695

                    VHULDOQXPEHU

Fig. 2. The results of three method

4

Conclusion and Future Work

In this paper we present one forecasting method with EWMA and SVR in the bank cash. The experimental results have showed that the SVR works better than 1-NN and MLP methods, and the forecasting precision with SVR is much better. The correlation of SVR is much higher than the other two methods, and the error measures of SVR is the lowest. This is mainly because SVR is based on Structural Risk Minimization, and the other two methods are based on Experimental Risk Minimization. For the data of every branch of bank is small, about 100 records. And SVR is good for small sample problem, so it has better generalization than the other two methods on our data. We also compare the forecasting effect of using original data and ratio data preprocessed by EWMA methods. The experimental results show that the EWMA methods can smooth the data, consequently improve the precision. The statistical correlation of using ratios increase remarkably, and the error measures reduce much. In the future, we will consider adjusting the parameter of the SVR so that we can obtain better forecasting precision. Also we will apply more preprocessing and learning algorithms on our data, and use some statistic measures to compare the results. Acknowledgments. This paper is supported by the teaching quality engineering foundation of Beijing Normal University Zhuhai Campus.

References 1. Last, M., Klein, Y., Kandel, A.: Knowledge Discovery in Time Series Databases. IEEE Transactions on Systems, Man, and Cybernetics 31B (2001) 2. Han, J., Dong, G., Yin, Y.: Efficient Mining of Partial Periodic Patterns in Time Series Database. In: Fifteenth International Conference on Data Engineering, pp. 106–115. IEEE Computer Society, Sydney (1999)

90

W.-m. Ma and W. Lu

3. Refenes, A.N., Burgess, A.N., Bentz, Y.: Neural Neworks in Financial Engineer: A study in Methodology. IEEE Transactions on Neural Networks 8(6), 1222–1267 (1997) 4. George, S.I.: A Search for Hidden Relationships: Data Mining with Genetic Algorithms 5. Tim, O., Firoiu, L., Cohen, P.: Clustering Time Series with Hidden Markov Models and Dynamic Time Warping. In: IJCAI-1999 Workshop on Sequence Learning (1999) 6. Hadjimichael, M., Arunas, P., Kuciauskas, L., Brody, R.: MEDEX: A Fuzzy System for Forecasting Mediterranean Gale Force Winds. In: IEEE Int. Conf. On Fuzzy Systems, pp. 529–534. IEEE press, New Orleans (1996) 7. Bottou, L., Vapnik, V.N.: Local Learning Algorithms. Neural Computation 4(6), 888–900 (1992) 8. Singh, S., McAtackney, P.: Dynamic Time-Series Forecasting Using Local Approximation. In: Proceedings of the IEEE Tenth International Conference on Tools with Artificial Intelligence, pp. 392–399. IEEE press, Los Alamitos (1998) 9. Hampshire II, J.B., Perlmutter, B.A.: Equivalence Proofs for Multilayer Perceptron Classifiers and the Bayesian Discriminant Function. In: Proceedings of the 1990 Connectionist Models Summer School, San Mateo (1990) 10. Gijbels, I., Pope, A., Wand, M.: Automatic Forecasting via Exponential Smoothing: Asymptotic Properties (1997) (manuscript) 11. Vapnik, V.: The Nature of Statistical Learning Theory. Springer, NY (1995) 12. Cortes, C., Vapnik, V.: Support-Vector Networks. Machine Learning 20(3), 273– 297 (1995) 13. Smola, A., Sch, B.: A Tutorial on Support Vector Regression. NeuroCOLT2 Technical Report NC2-TR-1998-030 (1998) 14. Sch¨ olkopf, B., Smola, A.J.: Learning with Kernels. MIT Press, Cambridge (2002)

Financial Distress Prediction Model via GreyART Network and Grey Model Ming-Feng Yeh1, Chia-Ting Chang2, and Min-Shyang Leu1 1

Department of Electrical Engineering, Lunghwa University of Science and Technology, Taoyuan, 33327 Taiwan {mfyeh,unit484}@mail.lhu.edu.tw 2 Taipei Municipal Wan Fang Senior High School, Taipei, 11696 Taiwan [email protected]

Abstract. This study attempts to use GreyART network and grey model to construct a financial distress prediction model. The inputs used to train the network are the historical data containing 17 different financial ratios of 22 healthy and 5 distressed Taiwan’s listed banks. With the help of the developed performance index, this study also proposes a growing extraction method for financial variables not only to further improve the classification ability in the training and testing phases, but also to use fewer extracted variables to build the financial distress prediction model. Simulation results show that the optimal condition is the one using four extracted variables as inputs and the vigilance threshold of 0.80. Under this condition, the proposed method generates only two clusters with corresponding classification hit rates of 96.30% and 95.24% for the training and testing results, respectively. Keywords: Financial distress prediction model, Grey model, Grey relational analysis, Adaptive resonant theory, GreyART network.

1 Introduction Since Beaver’s pioneering work in 1966, there are a large amount of studies using financial ratios to predict financial failure. Among these researches, Altman was the first one to use discriminant analysis in predicting bankruptcy by combining financial ratios into a single predictor of corporate bankruptcy [1]. The main appeal of this approach was its ability to reduce a multidimensional problem to a single score. Although discriminant analysis was widely used in the financial distress prediction problem, it has the following potential problems: 1) the distressed and viable firms used to distinguish must be linearly separable, and 2) ratios are treated as completely independent [2-3]. An alternative methodology for the analysis of financial distress is based on neural networks. Owing to the learning capability, neural networks could create a perfect mapping between the input and output values for a set of training data. Once the training process is completed, the patterns found by the neural networks can be used to forecast situations where the outcome (e.g., a firm is in distress or health) is unknown. This also demonstrates that the use of neural networks is possible without any of circumscription that binds discriminant analysis. Research studies on using neural networks for bankruptcy prediction started in 1990 [4], and Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 91–100. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

92

M.-F. Yeh, C.-T. Chang, and M.-S. Leu

are still active now. There have been lots of published literatures in the area of financial distress prediction [5-8], which supported the use of the neural networks over the statistical methods. GreyART network [9], a neural-network architecture combining grey relational analysis [10] and Adaptive resonant theory (ART) [11] network as shown in Fig. 1, is designed to find the underlying structure of a given dataset. Owing to the architecture of ART-based network with flexibility and stability, and GreyART network without the problem of selecting an appropriate learning rate, this study adopts GreyART network to construct the proposed financial distress prediction model. Generally speaking, the testing patterns of a neural network are usually given in advance and could be used to ‘post-examine’ the generalization ability of the network. Although those patterns have never been trained in the training phase, the results cannot actually reflect the forecasting ability of the network. Grey model [10] provides a simple and fast scheme to generate the forecasting values from the current and past data. This study adopts the forecasting values obtained by grey model as the testing patterns to solve this problem. Besides, GreyART network in [9] only focuses on determining the optimal clustering results for a given data set and it might not perform well in classifying the financial health of a firm. This study, therefore, develops a new performance index to solve the classification problem by determining the best result the GreyART network can attain in the sense of that performance index. This study also proposes a growing extraction method for financial variables such that the financial distress prediction model could be constructed by some extracted variables, but has better classification ability in the training and testing phases.

2 Preliminaries 2.1 Grey Relational Analysis Grey relational analysis is a similarity measure for finite sequences with incomplete information [10]. Denote the ith N-dimensional input pattern by xi = (xi(1), xi(2), …, xi(N)) and the weight vector of node j in an ART-based network by yj = (yj(1), yj(2), …, yj(N)), j ∈ {1, 2, …, m}. If xi is viewed as a specific reference pattern and all weight vectors are regarded as the comparative patterns, then the grey relational coefficient between xi and yj at the kth component, k ∈ {1, 2, …, N}, is defined as follows:

(

)

r x i (k ), y j (k ) =

Δ i ,min + ξ ⋅ Δ i , max Δ ij (k ) + ξ ⋅ Δ i ,max

(1)

where Δ ij (k ) = xi (k ) − y j (k ) , Δ i , max = max j max k Δ ij (k ), Δ i ,min = min j min k Δ ij (k ), and

ξ ∈ (0, 1] is the distinguishing coefficient controlling the resolution between Δi,max and Δi,min. The corresponding grey relational grade is g (x i , y j ) =

∑ [wk ⋅ r (x j (k ), y j (k ) )] N

k =1

(2)

Financial Distress Prediction Model via GreyART Network and Grey Model

93

where wk ≥ 0 is the weighting factor and ∑kN=1 wk = 1 . Normally, we select it as wk = 1/N for all k. The best comparative weight vector to the input is the one with the largest relational grade. Note that no matter how large the input pattern is, we can guarantee that 0 < g(xi, yj) ≤ 1. 2.2 GreyART Network GreyART network is a kind of ART-based network that incorporates grey relational analysis into ART network [11]. If each input pattern is viewed as a reference while the templates (weight vectors) of all categories are regarded as comparative ones, then the best comparative weight vector can be easily obtained by grey relational analysis. The node corresponding to the best comparative weight vector is denoted as a winner. Additionally, the vigilance test is designed to check the similarity measurement (the grey relational grade) of the winner. The algorithm of GreyART network is presented as follows: Step 1: Present a new training pattern, say xi. By using grey relational analysis, we can select the winner, which has the largest relational grade maxj{g(xi, yj)} among all weight vectors. The winner is denoted as j*. Step 2: The node j* passes the vigilance test if and only if the grey relational grade g (x i , y j ∗ ) ≥ ρ , where ρ is the vigilance threshold.

Step 3: If the winner fails the vigilance test, a new node unit k is created with weight vector yk = xi. Step 4: If the winner passes the vigilance test, adjust the weight vector of the winner j* by ) y (jnew = ∗

(

(

)

) Num cluster j(∗old ) ⋅ y (jold + xi ∗

(

)

Num cluster j(∗old ) + 1

)

) where Num cluster j(old denotes the number of members in cluster j ∗ . ∗

Fig. 1. The configuration of GreyART network

(3)

94

M.-F. Yeh, C.-T. Chang, and M.-S. Leu

After the learning process is terminated, the final weight vectors are regarded as the estimated cluster centroids. Since 0 < g(xi, yj) ≤ 1, the vigilance threshold ρ is therefore limited to the range (0, 1]. 2.3 Grey Model Let x(0) = (x(0)(1), x(0)(2), …, x(0)(N)) be a raw sequence and x(0)(k) ≥ 0, k = 1, 2, …, N. Then the grey differential equation of GM(1,1) model, i.e., the first order and single variable grey model, is defined as [11]

x (0 ) (k ) + az (1) (k ) = b, k = 2, 3, …, N,

(4)

where z (1) (k ) = 12 [ x (1) (k ) + x (1) (k − 1)], k = 2, 3, …, N, and x (1) (k ) = ∑km=1 x ( 0) (m), k = 1, 2, …, N. The unknown parameters a (the developing coefficient) and b (the grey input) in (4) can be obtained by the least square method as ⎡a ⎤ T −1 T ⎢ ⎥ = (B B ) B z , ⎣b ⎦

(5)

T

T ⎡− z (1) (2) − z (1) (3) " − z (1) ( N )⎤ ( 0) ( 0) ( 0) where B = ⎢ ⎥ and z = [x (2) x (3) " x ( N )] . " 1 1 ⎣ 1 ⎦ Once the developing coefficient and the grey input are obtained, the forecasting values of x(0)(k) and x(1)(k), denoted by xˆ ( 0 ) (k ) and xˆ (1) (k ), respectively, are determined by

b b xˆ (1) (k ) = [ x ( 0) (1) − ]e −a ( k −1) + , k = 1, 2, … a a

(6)

⎧ xˆ (1) (k ) − xˆ (1) (k − 1), k = 2, 3, ... xˆ ( 0) (k ) = ⎨ (0 ) k = 1. ⎩ x (1),

(7)

3 Financial Distress Prediction Model This study adopts GreyART network and GM(1,1) model to construct the proposed financial distress prediction model. The inputs applied to GreyART network for classifying the financial health of a firm or bank are the historical financial data. Each resultant centroid indicates a subclass, rather than a class, and several different subclasses could make up each class. Once the training phase is completed, the final clustering results can be used to indicate the distressful conditions in currently viable banks. 3.1 Data Collection and Variable Selection

The data used in this study are collected from the Taiwan Economic Journal (TEJ) Data Bank [12] covering the period 2004-08. Two classes of listed banks are drawn: a

Financial Distress Prediction Model via GreyART Network and Grey Model

95

class identified as financially distressed banks and a class identified as financially healthy banks, namely the ‘distressed class’ and the ‘healthy class,’ respectively. Additionally, the dataset is divided into two separate sets. Data for the 4th quarter of 2004 to 2006 are used as the training set to train the GreyART network, while data for the 4th quarter of 2007 and the 2nd quarter of 2008 are used as the testing (validation) set. The training set contains 27 banks with 5 banks in the distressed class, while the testing set consists of 21 banks with 1 bank classified as in distress in each year. Seventeen financial ratios used in this study are collected from TEJ Data Bank, which reflect the following five dimensions: structure, management, solvency, growth, and profitability. Table 1 contains a detail description of all selected variables, where xi,j represent the raw data of the jth financial ratio for the ith bank. 3.2 Data Preprocess

To reduce the number of training patterns, data for the training set are generated as follows. Let xi,j(t) represent the raw data of the jth financial ratio for the ith bank at year t, and yi = (yi,1, yi,2, …, yi,N) denote the generated pattern of the ith bank. Then each element of yi can be obtained by the next two strategies: i) Healthy class: If the ith bank is in the healthy class, then its jth element is the average value of all corresponding ratios in this bank, i.e.,

(

)

yi , j = xi , j (t1 ) + xi , j (t 2 ) + " + xi , j (t k ) k ,

(8)

Table 1. Description of research variables Dimensions Structure Management Solvency

Growth

Profitability

Description Debt ratio stockholder / total assets Total assets turnover Equity Turnover Current ratio Revenue growth rate Income from operation growth rate Earning before taxes growth rate Total assets growth rate Stockholder’s equity growth rate Total assets turnover growth rate Return on asset Return on equity rate Gross profit rate Operating profit rate Earning before taxes rate EPS (earning per share)

Denotation x i,1 x i,2 x i,3 x i,4 x i,5 x i,6 x i,7 x i,8 x i,9 x i,10 x i,11 x i,12 x i,13 x i,14 x i,15 x i,16 x i,17

96

M.-F. Yeh, C.-T. Chang, and M.-S. Leu

ii) Distressed class: If the ith bank is in the distressed class, then its jth element is the value when the bank occurred the financial distress, i.e., yi , j = xi , j (t d ),

(9)

where td denotes the year that the ith bank occurred the financial distress. After generating the input patterns by (8) or (9), we uses the data preprocessing method proposed by Hsia and Wu [13] to enable each input pattern to be comparable and, at the same time, normalize each pattern. The data preprocessing formula for yi,j is stated as follows. ⎧ yi , j − Δ j ,min , if the larger, the better ⎪ ⎪ Δ j ,max − Δ j ,min ~ yi , j = ⎨ ⎪ Δ j ,max − yi , j , if the smaller, the better ⎪ Δ j ,max − Δ j ,min ⎩

(10)

where Δj,max = maxiyi,j and Δj,min = miniyi,j. After the data preprocessing, we can guarantee that 0 ≤ ~ yi , j ≤ 1. Note that, in Table 1, a financially healthy banks is normally to pursue higher ratios, except the debt ratio (xi,1). 3.3 Learning and Testing Phases

In the learning phase, the inputs applied to GreyART network are those training patterns after data preprocessing. While the learning phase of the network is completed, some templates could be obtained and regarded as the cluster centroids. Thereafter, the cluster centroids are respectively recognized by auditors, experts, or managers to label them as ‘health’ or ‘distress’. Assume that cα,β represents the cluster centroid of the βth cluster belonging to class α. Then an input pattern, say ~ y i , is in * the class α if g (c ,~ y ) = max{g (c , ~ y )}. (11) α ∗ ,β ∗

i

α ,β

α ,β

i

That is to say, an input pattern is in the class that one of its subclasses has the largest relational grade. In the testing phase, all resultant cluster centroids obtained in the learning phase remain changeless and only the first step of GreyART algorithm is required. Each testing pattern is identified as the class that the corresponding template belongs to. The testing patterns described in Sec. 3.1 have never been used to train the network in the training phase. They could be used to post-examine the ‘generalization’ ability of the network. Since those patterns are actual values, the classification results cannot actually reflect the ‘forecasting’ ability for the case of one- or two-year-ahead. As indicated in [14], constructing a GM(1,1) model requires at least four data, i.e., x (0) = (⊗, x(0)(2), x(0)(3), x(0)(4)), where ⊗ represents a arbitrarily positive number. Let x(0)(k) = xi,j(k+2002), k = 2, 3, and 4. That is to say, the raw sequence x (0) in this study is formed by three ‘historical’ data: xi,j(2004), xi,j(2005) and xi,j(2006). Then using (5) to (7) could obtain one- and two-step ahead forecasting values of x (0),

Financial Distress Prediction Model via GreyART Network and Grey Model

97

i.e., xˆi , j (2007) = xˆ (0 ) (5) and xˆi , j (2008) = xˆ ( 0 ) (6). Once the forecasting values are obtained, those forecasting data are regarded as the testing patterns of GreyART network. By this way, the corresponding classification results can actually represent the forecasting results of the proposed financial distress prediction model. Fig. 2 demonstrates the training and testing accuracies with respect to the vigilance threshold, where the number above the point on the training accuracy indicates the number of clusters resulted from that threshold and the testing accuracy includes the results of the actual and forecasting values. As can be seen, the training accuracy is 96.30% for each vigilance threshold. Also, the larger the value of the threshold, the more the created clusters. In one- or two-step ahead prediction case, generally speaking, the testing result with the actual data as inputs (the generalization ability) is better than that with the forecasting values as inputs (the forecasting ability). Besides, the best forecasting result is 57.14% in both cases. It is a worse result. This study therefore proposes a growing extraction method to extract some significant variables from the given 17 variables to improve the forecasting ability. 3.4 Growing Extraction Method In order to overcome the trade-off between the clustering performance and the classification ability, this study utilizes the following index to evaluate the performance of the clustering results. Let Accuracy(ρ) denote the classification accuracy resulted from the vigilance threshold ρ and PQ(ρ) represent the partition quality caused by the same threshold ρ, where the partition quality (PQ) is defined in Hamerly and Elkan [15] as

Fig. 2. Training and testing accuracies with 17 financial ratios

PQ =

nclass ncluster

nclass

α =1 β =1

i =1

∑ ∑ p(α , β ) 2 ∑ p(α ) 2 ,

(12)

98

M.-F. Yeh, C.-T. Chang, and M.-S. Leu

where nclass is the true number of classes, ncluster is the number of clusters found by the clustering algorithm, p(α,β) and p(α) represent the frequency based probability of a point vector in cluster β belonging to class α, and the actual class probability, respectively. Then, the performance index (PI) is defined as follows. PI ( ρ ) = 100 × Accuracy( ρ ) × PQ( ρ ).

(13)

Among those thresholds, the one with the maximal performance index is termed an optimal threshold in this study. The corresponding cluster centroids and the number of clusters are the optimal results in the sense of the performance index (13). The proposed growing extraction method starts with the training input containing only a single variable. That is to say, in the beginning, the input pattern used to train yi = (~ yi , j ), j = 1, 2, …, 17. Hereafter the jth the GreyART network is of the form ~ variable is termed the active variable. Denote the performance index due to such an input form and a specific vigilance threshold ρ as PIj(ρ). Among all possible active variables, the extracted variable is the one with the maximal performance index. In other words, if the j*th variable is the extracted variable, then it must satisfy PI j∗ ( ρ ) = max j PI ( ρ ).

(14)

Assume that the first extracted variable is the ‘return on equity rate’, i.e., j* = 13. To extract the second variable from the remaining variables, the form of input pattern y i = ( ~yi ,13 , ~ yi , j ), j ≠ 13. According to (14) again, the second extracted becomes ~ variable can be determined. Assume that it is the ‘total assets growth rate’, i.e., j* = 9. Subsequently, the 9th variable is inserted into the second component of the input, i.e., ~ yi = ( ~ yi ,13 , ~yi ,9 , ~ yi , j ), j ≠ 9, 13, for consideration. The remaining extraction process could be done by the similar method. Since the number of components is successively increased, that is the reason why the proposed scheme is termed the growing extraction method. Besides, the final input form also could reveal the significance of a variable in order in which it is extracted.

3.5 Results and Discussion The growing extraction method given in Section 3.4 is under a specific vigilance threshold. To search of an optimal result, the vigilance threshold starts from 0.70 and increases by 0.01 in the simulations. Fig. 3 shows the training and testing accuracies corresponding to the vigilance threshold ρ with the value of 0.80. The ‘variable sequence’ given in the top figure represents the extraction variables in order in which they are given. The x-axis represents the number of variables, that is, if the number of variables is n, then it represents that the first n variables in sequence are selected. Besides, the number above the point on the training accuracy indicates the number of resultant clusters resulted from that number of variables. Fig. 3 reveals that no matter how large the number of extracted variables is, the training accuracy rate is 96.30%, i.e., one financially distressed bank is misclassified into the healthy class. However, the larger the number of variables, the more the created clusters. While using the forecasting data as the testing patterns, it can be seen that the forecasting ability with

Financial Distress Prediction Model via GreyART Network and Grey Model

99

fewer extracted variables is generally better than that with more variables in both oneand two-year ahead prediction cases. In our simulations, the conditions resulting in the optimal result are ρ = 0.80 and the inputs with the form constructed by the first four extracted variables (i.e., return on equity rate, total assets growth rate, stockholder’s equity growth rate, and revenue growth rate). Under these conditions, there are only two clusters, one for the healthy class and the other for the distressed class, being generated with the training accuracy of 96.30%. Both one- and two-year ahead prediction cases can attain the same forecasting accuracy of 95.24% (one distressed bank is misclassified into the healthy class) which is much better than the best forecasting result obtained by use of 17 variables as inputs, i.e., 57.14%. That is to say, the forecasting ability is indeed improved with the proposed variable extraction method.

4 Conclusions The proposed financial distress prediction model is based on GreyART network and grey model. In order to further improve the classification ability in the training and testing phases of GreyART network, this study also proposes a growing extraction method for financial variables to extract some important variables. Simulation results show that the best result the proposed method can attain is used by four extracted variables with the vigilance threshold of 0.80. For this case, there are only two clusters, one for each class, being generated with the corresponding classification hit rates of 96.30% and 95.24% for the training and forecasting results, respectively.

Fig. 3. Training and testing accuracies with the growing extraction method

100

M.-F. Yeh, C.-T. Chang, and M.-S. Leu

Acknowledgments. This work was supported by the National Science Council, Taiwan, Republic of China, under Grant NSC 98-2221-E-262-004.

References 1. Altman, E.I.: Financial ratios, discriminant analysis and the prediction of corporate bankruptcy. J. Financ. 23(4), 589–609 (1968) 2. Deakin, E.B.: A discriminant analysis of predictors of business failure. J. Account. Res. 10(1), 167–179 (1972) 3. Blum, M.: Fail company discriminant analysis. J. Account. Res. 12(1), 1–25 (1974) 4. Odom, M.D., Sharda, R.: A neural network model for bankruptcy prediction. In: Proc. Int. Joint Conf. Neural Netw., vol. 2, pp. 163–168. IEEE Press, New York (1990) 5. Atiya, A.F.: Bankruptcy prediction for credit risk using neural networks: a survey and new results. IEEE Trans. Neural Netw. 12(4), 929–935 (2001) 6. Kumar, P.R., Ravi, V.: Bankruptcy prediction in banks and firms via statistical and intelligent techniques – a review. Eur. J. Operation Res. 180, 1–28 (2007) 7. Tsai, C.F., Wu, J.W.: Using neural network ensembles for bankruptcy prediction and credit scoring. Expert Syst. Appl. 34, 2639–2649 (2008) 8. Chen, W.S., Du, Y.K.: Using neural networks and data mining techniques for the financial distress prediction model. Expert Syst. Appl. 36, 4075–4086 (2009) 9. Yeh, M.F., Chiang, S.S.: GreyART network for data clustering. Neurocomputing 67, 313– 320 (2005) 10. Deng, J.L.: Introduction to grey system theory. J. Grey Syst. 1(1), 1–24 (1989) 11. Carpenter, G.A., Grossberg, S.: ART 2: self-organization of stable category recognition codes for analog input patterns. Appl. Optics. 26(23), 4919–4930 (1987) 12. Taiwan Economic Journal Data Bank, http://www.tej.com.tw/ 13. Hsia, K.H., Wu, J.H.: A study on the data preprocessing in grey relational analysis. J. Chinese Grey Syst. 1(1), 47–54 (1998) 14. Li, B.Q.: Three-data modeling of grey system theory. J. Grey Syst. 2(1), 11–20 (1990) 15. Hamerly, G., Elkan, C.: Learning the k in k-means. In: Proc. 17th Ann. Conf. Neural Inform. Process. Syst. MIT Press, Cambridge (2003)

Risk Assessment Model Based on Immune Theory Tao Liu, Li Shang, and Zhifeng Hu Jiangsu Research & Development Centre for Modern Enterprise Information Software Engineering, Suzhou, China [email protected], {sl0930,huzf}@jssvc.edu.cn

Abstract. As new field of computer intelligence research, Artificial Immune System inspired by biological immune system, provides a strong paradigm for information processing and problem solving. Artificial immune network theory is an important theory of AIS and has already wildly applied in the fields of data clustering, data analysis and robot control. This research proposes firstly to apply artificial immune network theory into risk assessment of managerial field. According to the comparability of the e-commerce risk problem and the biological immune system, presents specific model construction process relating to case study and testifies the result with data from real test. Keywords: Risk, Assessment, Model, Immune network.

1 Introduction This Artificial Immune Systems (AIS) are the information processing technology, computer technology and other intelligence systems based on the theory and application of biological immune system [1]. The study of AIS has become a focus in the research of intelligence system as well as neural networks, fuzzy logic and genetic algorithms. AIS provide a strong paradigm for information processing and problem solving in computer intelligence research [2][3]. Immune theory [4][5]suggested systematic hypothesis on the activity of immune cell, generation of antibody, immune tolerance, self and non-self identification, immune memory and immune system evolution and defined immune system as adjustment network composed by immune cell or molecule. Inspired by immune theory, researchers constructed many artificial immune model [6-8] and implied them into fields like data clustering, data analysis, robust controlling, computer safety, etc [1-2]. However, artificial immune theory is not used in risk fields. This research provides specific model construction process with the study of risk assessment in risk field. Theoretical analysis and test results show that it is an effective new method of risk appraisal.

2 Design of Immune Network Risk appraisal model proposed in this research is constructed on the immune network. In our adaptive immune response network (AIRN) [8], antigen is the original data and antibody is feature of pattern which displays itself as a data item in the network; the system structure is the population of antibody. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 101–108. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

102

T. Liu, L. Shang, and Z. Hu

2.1 Initialization of Network and Antigen Presentation The antigen in the system can be described in detail as AG

= (agi ...ag I )T , and agi

represents the i-th antigen; agij represents the j-th vector of the i-th antigen. And we have

agij ∈ [0,1]m , i = 1,..., I ; j = 1,..., m . The following vector is generated

randomly:

AB = (ab1 ⋅⋅⋅ abk )T , and abij ∈ [0,1]m , i = 1,..., k ; j = 1,..., m are the

new antibodies. In the expression, abi represents the i-th antigen, and abij represents the j-th vector of the i-th antibody. Designate each element in the antigen to each element in the antibody. For any antigen agi and any antibody abj,, the distance between the two can be calculated as follows: 1/ 2

⎛ m ⎞ d (i, j ) = ⎜ ∑ (agit − ab jt )2 ⎟ ⎝ t =1 ⎠

(1)

.

For any pair of antibodies abi and abj, the distance between the two can be calculated as follows: 1/ 2

⎛ m ⎞ s (i, j ) = ⎜ ∑ (abit − ab jt ) 2 ⎟ ⎝ t =1 ⎠ For any antigen

ab j (ab j ∈ AB, j = 1,

ag i (agi ∈ AG, i = 1,

(2)

.

, k ) , the stimulation sl(i, j) it receives from the

, I ) is:

⎧⎪1 d (i, j ) sl (i, j ) = ⎨ ⎪⎩0

d (i, j ) < Tsl

.

(3)

other

In (3), Tsl is the threshold value of stimulation from antigen to antibody, and I

k

Tsl = ksl ∑∑ d (i, j ) [( I × k ) / 2] , Tsl represents the threshold of the stimulation i =1 j =1

from the antigen to the antibody, and

ksl ∈ (0,1] is a constant.

2.2 Clone and Mutation When the stimulation to antibody from antigen arrives certain degree, the immune system is activated and begins to clone antibody. The number of cloned antibody abi stimulated by antigen agi is:

⎧⎪ ⎢ sl (i, j ) ⎥⎦ d (i, j ) > Tsl . c1 (i, j ) = ⎨ ⎣ other ⎪⎩0

(4)

Risk Assessment Model Based on Immune Theory

In (4):

103

⎣⎢i ⎦⎥ represents the closest natural number to sl(i, j). Abji is the set of clones of

the antibody abj as a result of the stimulation from the antigen agi, and the set of clones of the antibody abj as a result of the stimulation from the antigens is:

C j = Ab j1 ∪ Ab j 2 ∪

∪ Ab jc1 (i , j ) =

c1 ( i , j )

∪

Ab jt .

(5)

t =1

The corresponding set of antibodies AB generates the set of new antibodies C through cloning:

C = C1 ∪ C2 ∪

k

∪ Ct = ∪ Ct .

(6)

t =1

The clone cell needs mutation process to bestow the newly born cell higher affinity with the selected antigen. Suppose Mut (i, j ) = d (i, j ) is the new antibody mutation rate from antibody abj stimulated by antigen agi. Suppose the mutation set of C ′j , then antibody abj stimulated by antigen group is

c′j = c j + mut (i, j ) × (c j − agi ) .

Suppose the mutation set of antibody abj

stimulated by antigen group is C ′j , then the antibody set from mutation is:

C ′ = C1′ ∪ C2′ ∪

k

∪ Ct′ = ∪ Ct′ .

(7)

t =1

The mutation ensures the diversity in the newly generated antibodies. This is an inherent characteristic of the learning mechanism of the immune system. The system improves its recognition rate of antigens through repeated cycling of the above process. The clone suppression starts right after clone mutation is completed. For the set C ′ , the intimacy level between the antigen and the antibody and the antibody cells are computed again after the mutation. Meanwhile, the suppression threshold between the antigen and the antibody Td, and the suppression threshold between the antibodies, Ts , are computed as follows: Td p

constant,

= kd

1 p min iI=1 d (i, j ), kd ∈ (0,1] , and kd is a ∑ p j =1

p

Ts = k s ∑∑ s (i, j ) [ p × ( p − 1) / 2] k s ∈ (0,1] and ks is a constant. i =1 j =1

During the clone suppression stage, the antibody cells with higher values of the intimacy level (as compared with the threshold Td) are deleted at first, and then the memory cells with smaller intimacy levels (as compared with the threshold Ts) are deleted according to the intimacy levels between the antibody cells. The set of

104

T. Liu, L. Shang, and Z. Hu

antibodies upon the completion of this stage is C ′′ , whose elements satisfy the following conditions:

⎧ab j ab j = ⎨ ⎩0

∃agi d (i, j ) < Td

⎧ab j ab j = ⎨ ⎩0

∃abi s (i, j ) > Ts

other

other

.

(8)

.

(9)

After the stage of clone suppression, the new set of antibody C ′′ is smaller than C ′ . Clone selection is then carried out at this stage. Among the antibodies in C ′′ , ξ % of the mutated cells are chosen to act as the memory cells for the antibody, according to their intimacy level with the antigen. ξ % can be specified beforehand. The chosen mutated cells are then put into M, the set of memory cells. 2.3 Termination of Network Training Among the antibodies after clone mutation, the ones with higher antigen stimulation level morph into the memory cells with longevity, while the ones with lower antigen stimulation level die out as the antigens die out. Since the recognition among the antibody cells themselves could activate the autoimmune response, the antibodies that are capable of self recognition have to be suppressed each time after new antibody memory cells are added in order to suppress autoimmune. Denote the number of elements in the memory cell set M (M={m1,m2,…,mz; M ∈ ℜ }) as p1. Define the intimacy maturity threshold of the memory cell as: Z

p1

p1

Ts′ = k s′ ∑∑ s (i, j ) [ p1 × ( p1 − 1) / 2] .

(10)

i =1 j =1

In (10):

k s′ ∈ (0,1] and is a constant. Therefore, after the process of the intimacy

level maturity, the elements in the set of memory cells M should satisfy the following conditions:

⎧⎪ab ab j = ⎨ j ⎪⎩0

s (i, j ) > Ts′ other

i =1,2,..., p1 ; j =1,2,..., p1 .

(11)

During the network training, a certain number of iterations could be specified as the termination condition. As an alternative, the stimulation level between the antigen and the memory cell can be measured upon the completion of each iteration to serve as the termination condition. In other words, for any given antibody agi, an antibody abj can be found in the set of memory cells so that the condition d (i, j ) ≤ Ts′ is satisfied.

Risk Assessment Model Based on Immune Theory

105

3 Definition of Risk Measure Risk assessment of managerial field is quite similar with those in the biological immune system (BIS), both needs to sustain the stability of the system in a dynamic environment. Considering the corresponding relationship between antibody consistence and pathogeny invasion degree, to establish mathematical models of elements such as self body, non-self body, antibody and antigen, establish the risk appraisal model based on antibody consistence. With this model, the type and degree of attacks to risk can be calculated. There are diversified risks and different risk sources would bring different damages. Therefore, when assessing the risks, all the factors and their corresponding importance must be considered. Moreover, risk is calculated according to the antibody information under the stimulation of all kinds of risk factors. When there is no risks, number of antibody is stable; when risk increases, number of antibody increase sharply and vice versa. Thus risk can be judged by the number of antibody stimulated by risk factors. Suppose r or R is the facing risk and 0≤r(R) ≤1. when r(R) =1, it indicates the most great risk for current system; when r(R)=0 means no risk for current system, and bigger r(R) implies higher risks. The degree of r(R) can provide information for antirisk policy-making. Define ni is the number of antibody caused by i risk found by the system, βi is the degree of danger of i attack, ci is the number of antibody under normal network. Risk calculation of the system under i risk:

ri = 1 −

1 . 1 + lg( ni − ci + 1)

(12)

Risk calculation of the whole system:

Ri = 1 −

1 1 + lg( β i ∑ ( ni − ci ) + 1)

.

(13)

i

There exist many indicators which are hard to explain exactly in the risk appraisal system. However, this research solved the problem.

4 Case Study This research used e-commerce risk assessment as an example. E-commerce risk appraisal indicator system is a serial of integrated indicators which are inter-related and can sensitively reflect the situation and problems. The design of the system follows principles of sensitivity, flexibility, integrity and with combination of quantitative and qualitative indicators. With the literature review, this research constructs e-commerce risk appraisal indicator system based on AIRN theory Risk quotient of e-commerce risk is the top level of the system (A). And then the ecommerce risk appraisal. indicator system has four first level indicators: indicator for

106

T. Liu, L. Shang, and Z. Hu

risk caused by the incompleteness of national control system (B1), indicator for the risk in the food production and processing channel (B2), indicator for the risk in the food distribution channel (B3) and indicator for the risk in the food consuming process (B4). All these four first level indicators can be assessed by its second level indicators (C) respectively. Risk appraisal indicators are in Table 1. Table 1. E-commerce risk appraisal indicator system

Final Result (A)

Risk quotient of e-commerce

First Level Indicators (B)

Second Level Indicators (C)

Risk in network(B1)

Physical security (C11) Threaten of network virus(C12) attack of hacker (C13)

Risk caused by transaction (B2)

Validity of electronic contract (C21) Flaw of electronic payment (C22) Preservation of virtual property (C23)

Risk in supply chain collaboration (B3)

Risk in law (B4)

… …

price fluctuations (C31) Uneven in profit distribution (C32)

…

confusion caused by unification policy (C41) Non-performance of market regulatory functions (C42) Financial services imperfect (C43)

…

The value of risk appraisal indicator can be decided by expert evaluation and statistic. Expert evaluation is to select a certain number of consultants and experts who would appraise every indicator according to its level and give specific reference value; statistic method would refer to all the data and draw the value of risk indicator by statistical instrument. In this research, 5 indicators such as Law and Regulation and Operation of Government Body are decided by expert method while the others are by statistic method. Relevant e-commerce risk indicator data from 2008 to 2009 in a company are divided into 45 groups. After test, these data is normally distributed. The other data are also generated and satisfy the normal distribution. Firstly, normalize data. Then calculate the risk indicator of e-commerce risk using the presented model. Threshold value of immune network is set as ksl=1,kd=1,ks=1,k’s=1. Fig.1(a) is the threat degree of risk happened in the attack of hacker; Fig.1(b) is the threat degree of risk happened in the confusion caused by unification policy; Fig.1(c) is the e-commerce risk degree appraisal when there is no threat for risk sources; Fig.1(d) is the e-commerce risk degree appraisal when there is threat for risk sources.

Risk Assessment Model Based on Immune Theory

107

Fig. 1. (a) Threat degree of risk only caused by network virus; (b) Threat degree of risk only caused by Flaw of electronic payment; (c) E-commerce risk degree appraisal when there is no threat for risk sources; (b) commerce risk degree appraisal when there is threat for risk sources

From Fig.1(a)- (d), it can be learned that risk appraisal model based on AIRN can reflect the risk of e-commerce risk and dynamically calculated its degree. Compared with neural network appraisal, this appraisal does not need the training sample of risk in advance. While compared with ambiguous appraisal, this can avoid subjective judgment in the process.

5 Conclusions Artificial immune theory has developed into a new branch of computer intelligence research. This research explores the feasibility of constructing risk assessment model based on immune network theory, designs plans for risk assessment and constructs ecommerce risk assessment model with the study of risk assessment in e-commerce risk field. The result indicated that this model is practical, reliable and capable of providing strategic decision-making tool for policy maker of e-commerce risk supervision. Further, this research abstracts and expands the conception of antibody consistence in biological immune system and applies into e-commerce risk appraisal. Lastly this research establishes e-commerce risk appraisal model based on immune network. Test results show that this method is an effective new method of ecommerce risk appraisal. The risk appraisal model in this research has deep influence to self-nonself, antigen, antibody, immune cell, clone selection, immune memory. It can not only be applied in the e-commerce risk appraisal and restructuring of mathematic model but also in pattern recognition and combinatorial optimization.

Acknowledgment This research was supported by the grants of the National Science Foundation of China (No.60970058), the grants of Natural Science Foundation of Jiangsu Province

108

T. Liu, L. Shang, and Z. Hu

of China (No.BK2009131) and is also sponsored by Qing Lan Project of Jiangsu Province.

References 1. Timmis, J., Hone, A., Stibor, T., Clark, E.: Theoretical Advances in Artificial Immune Systems. Theoretical Computer Science 1, 11–32 (2008) 2. Dasgupta, D.: Advances in Artificial Immune Systems. Computational Intelligence Magazine 4, 40–49 (2006) 3. Castro, D.E., Zuben, V.: The Clonal Selection Algorithm with Engineering Applications. In: The Genetic and Evolutionary Computation Conference, pp. 36–37. Morgan Kaufmann Publishers, San Fransisco (2002) 4. Jerne, N.K.: The immune System. Scientific American 1, 51–60 (1973) 5. Jerne, N.K.: Towards a Network Theory of the Immune System. Annual Immunology 125C, 373–389 (1974) 6. Li, Y.Y., Jiao, L.C.: Quantum-inspired Immune Clonal Algorithm. In: 4th International Conference on Artificial Immune Systems, pp. 304–317. IEEE Press, New York (2005) 7. Mo, H.W., Zuo, X.Q., Xu, L.f.: Immune Algorithm Optimization of Membership Functions for Mining Association Rules. In: Garijo, F.J., Riquelme, J.C., Toro, M. (eds.) ICNC 2006. LNCS, vol. 4222, pp. 92–99. Springer, Heidelberg (2006) 8. Tao, L.: Study on Risk Management of E-business Based on Immune Principle. Report of Post Doctorate, Huazhong University of Science and Technology (2007)

Short-Term Load Forecasting Based on Bayes and RS Yanmei Li School of Business and Administration, North China Electric Power University, Baoding 071003, China [email protected]

Abstract. In the many influencing factors of load forecasting, some are related to each other, and some are independent, so it is not very necessary of all factors to the conclusions. The paper adaptively selects input features making use of Bayes method and Rough Set, and chose a group of input characteristic set which report most out the reason of output change to train the BP Neural Network model for forecasting, which has been testified to be valid. Keywords: Bayes, Load Forecasting, Rough Set, BP Neural Network.

1 Introduction When building up the short-term load forecasting model, we should first make sure the input of the model.The factors and datas which influence the forecasting result are very much, and not all the influencing factors are necessary.In the factors, some are interrelated, and others are irrelated. How to select the factors from the large numbers of infecting factors and datas which have strongger influences on forecasting value and constitute an effective inputting variable set function are the chief matter before establishing the forecasting model. The paper selects the inputting characteristic value of forecasting model through Bayes and RS to predigest the inputting characteristic values. After simplfying and abstracting the samples, making sure the number of neurons of the inputting values,then optimizing the neurons of hidden layer, through which the training speed and accuracy can be increased. The method has been proved to be valid through an example.

2 Bayes Decision Theory Bayes decision theory concept and method are used for fields such as engineering, management science already by broad field, adaptively selects input features step with Bayes method as follows: 2.1 Ascertains a Priori Probability Distribution The priori probability P (ω j ) represents the estimation to the probability distribution of the variable ω j , it has reflected a priori knowledge to the variable, has included Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 109–116. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

110

Y. Li

practical experience and subjective judgment etc. And that a priori probability scatters in load forecasting , is what be remained to be chosen influencing factor and their probability aggregation , restricting condition is whose necessary probability greater than zero, whose combination is 1. The load at the same time, forecasting (namely "big close small distant" distance according to load forecasting nearest segment of period affects maximal) and "identical date load type similarity" characteristic, the priori probability distribution is Pper = αP dis ⎧ ⎪ P equ = βPdiv . ⎪⎪ P per = γPequ ⎨ ⎪ P + P + P + P =1 ⎪∑ per ∑ dis ∑ equ ∑ div ⎪⎩ Pper > 0, Pdis > 0, Pequ > 0, Pdiv > 0

(1)

In the formula, Pper and Pdis represent the distance from influencing factor of forecasted point far or close; and respectively, Pequ and Pdiv represent the priori probability of influencing factor that having identical or different date type; Take α , β , γ value range being (1, ∞ ) , may look at concrete conditions but fix. When α = 1, β = 1, γ = 1 , the priori probability distribution is an uniform one. 2.2 Ascertains a Likelihood Function The likelihood function has been a condition probability essentially it has reflected sample information, whose function value has been called likelihood rates, has been.

(

)

P x1 , x 2 ," , x N ω j . During the period of load forecasting, if the input variable is selected as ω j , we may make use of N samples x n ( n = 1, " , N ) to calculate the Error E according to the follow formula. Ei =

Rvalid _ j [ f train ] =

1 n ∑ Rvalid _ j [ f train ] . n j =1

1 N valid

∑

x j ∈Z valid

[y

j

(2)

( )] × 100% .

− f train x j yj

(3)

Rvalid _ j [ f train ] be relative mean error in style, from training collection to get regression function f train relativity on effective set; N valid be an effective set Z valid all together sample book number; y j be that actual load value; f train x j be forecasting load value.

( )

Short-Term Load Forecasting Based on Bayes and RS

111

2.3 Calculates Posterior Probability

(

)

The formula of posterior probability P ω j x1 , x 2 ,", x N is

(

)

P ω j x1 , x 2 , ", x N =

(

)( ) . ω )P (ω )

P x1 , x 2 ," , x N ω j P ω j

∑ P(x , x j

1

2

i =1

,", x N

j

(4)

j

Among them, a priori probability and likelihood function action input vectors, a posteriori probability is output vector. Because a posteriori probability has synthesized a priori knowledge and sample book information, ultimateness being to decide which group of influencing factors to choose being input vector is standard. A posteriori probability is increasingly big, the probability that input vector is pitched on is increasingly big. 2.4 Chooses Input Vector ω j According to the actual characteristic being unlike area load, choose M may affect bigger factor composition to load waiting for choosing influencing factor, again out of, the random chooses different influencing factor combination, forms input vector ω j .

3 Rough Sets Reduction Algorithm 3.1 Use the Dicernibility Matrix to Get the Least Reduction The Discernibility Matrix is proposed by the Poland Warsaw famous mathematician Skowron. Utilizing this tool, we can give expression to the whole cannot distinguish relations which consist in the complicated information system. One presupposition to use the dicernibility matrix to get the least reduction is to deal with the antipathic register at first at the data in table pretreatment stage, that is, the discermibility matrix not deal with the incompatibility record. With the pretreatment methods, if the records of the conflict by dividing the number of the total number of records, we can get a measurement of the roughness and this measurement can be a character of the data sheet. Suppose S = (U , R , V , f ) is a information system, R=CYD is property collection, subset C = {ai / i = 1,2, " , m} and D = {d } is Conditional attribute set and decisionmaking attribute set, U = {x1 , x2 ," , x n } is on domain, a k (x j ) is sample, on the

j

a k . Definition system get the dicernibility matrix is

x j is the value

[ ]

M (S ) = mij

n×n

, it’s

i and

the list element is ⎧a k ∈ C , a k (xi ) ≠ a k (x j ) ∧ D(xi ) ≠ D(x j ) . mij = ⎨ i, j = 1,2,", n ⎩ φ , D( xi ) = D(x j )

(5)

112

Y. Li

So in the dicernibility matrix the element collection to the

mij is capable of distinct the whole

xi and x j ; But if xi and x j belong to the same decision-making

class, the value of the

mij is φ in the dicernibility matrix.

For every dicernibility matrix M (S ) corresponding the only distinguish function

f M ( S ) (a1 , a 2 , " , a m ) = ∧{∨ mij ,1 ≤ j < i ≤ n, mij ≠ φ }.

(6)

3.2 Reduction Algorithm Based on the Attribute Dependability Strike all about a history of NP-hard problem, so enlightening information to use simplified calculation to identify the optimal or sub-optimal reduction can be achieved is obviously a method. Many basic steps of heuristic algorithms are incepted by the nuclear of the information systems or table, then some measure of the importance of attributes, choose the most important attributes to the nuclear, until the termination conditions, and we have the information systems or a decision table reduction. To be more exact, includes a reduction of the attribute set. A information system for all the attributes of the decision-making is not as important as in the rough set theory, attribute importance of the dependencies can be reflected. Decision-making property D to the property R (R belongs to C), the dependencies γ (R, D ) defined: γ ( R, D ) =

card (POS R (D )) card (POS C (D ))

(7)

Obviously, 0 ≤ γ (R, D ) ≤ 1, γ (R, D ) give a measure of the D decision-making on the attributes of the dependencies. It reflect the importance of the property R to D. Under the premise known conditions R, a property a ∈ C − R to the decision-making D’s importance SGF (a, R, D ) may be defined as follows: SGF = (a, R, D ) = γ (R + {a}, D ) − γ (R, D )

(8)

SGF = (a, R, D ) reflect that after property a plus R, the rise degree of the dependence between R and D. And in fact, the stronger for the impact of a property attribute R and D dependencies, the greater value of SGF = (a, R, D ) is. 3.3 Reduction Algorithm Based on Conditional Information Entropy This reduction algorithm does not require distinction matrix, directly from the point of view of information theory, the idea of entropy calculated using various attributes the importance of the load. Reduction act and algebra is the same in the information entropy reduction in the importance of also need to set a threshold value greater than the threshold value of the property that is affected relatively large load factors, and

Short-Term Load Forecasting Based on Bayes and RS

113

less than the threshold value of the attribute is considered to be redundant and could be deleted. In this method for the calculation of the im nt attribute set, that is, the various factors affecting load; D represent decision-making set, that is, the load of the forecast day; RC, RD represent separately the equivalence relation in the decision-making domain of the attribute set conditions C and portance of attributes ai: I (a i , D

C represent D;

)=

⎡R ⎤ H⎢ D ⎥ ⎣ RC ⎦

⎡ ⎡ RD ⎤ ⎡ RD ⎤⎤ ⎡RD ⎤ ⎢H ⎢ ⎥ − H ⎢ ⎥⎥ / H ⎢ ⎥ ⎣ RC ⎦⎦ ⎣ RC ⎦ ⎣ ⎣ RC−A ⎦

(9)

represent RD to RC’s condition entropy.

3.4 Attribute Reduction Based on Genetic Algorithm Adaptive genetic algorithm is a method of random search, the search methods, not by a unitary structure or direction, it will more than individual as a possible solution and to consider the search space within the scope of the overall sample, thus leading to the possibility of greater convergence Global Optimal Solutions, Therefore, it was the introduction of the genetic algorithm Rough Set Attribute Reduction. Algorithm by using computer simulation of biological evolution, groups continues to be optimized, in the process of change and to identify the optimal solution. In the genetic algorithm, the design of fitness function is the core of the entire GA algorithm steps, as several genetic operator rely on the adaptation of the chromosome and therefore fitness function of design objectives, determines to a large extent iteration convergence direction, but the rough set of attribute reduction is to achieve the smallest reduction attribute set. In this way, the attribute set to meet in ensuring the accuracy of certain circumstances, at least to the number of attributes that ultimately the results that are needed to meet the classification requirements of the attribute set. Therefore, the design function to the ultimate goal should contain the following two objective classification must meet quality requirements usually must be about SR. function: covered by the reduction of the number of properties to minimize. Then provisions for the fitness function: score(r ) = m − l r + k m R for which chromosome the corresponding attribute set, D for decision-making attributes, the attribute set dependence on R. This function will attribute dependence on the introduction of fitness function, and rely on the attributes of that decision attributes of the corresponding attribute set chromosome depended reflects the attributes of taxonomic capacity.

②

①

4 Case Analysis Artificial neural network has been applied extensivly in load forecasting and decisionmaking because its strong parallel processing function, approximation to arbitrary

114

Y. Li

function, self-learning, self-organization capabilities and self-adaption capabilities, and also consider many influencing factors such as climate, temperature etc. However,ANN has some deficiency and shortage. The learning speed is fixed, so the convergence rate is low and the network need longer training time. For some complicated problem, the more inputting nodes will increase the complexity of the network., which will extend the training time. Although the weight can be converged into some value, it is not necessary the minimum one of the error plane, maybe it will produce a local minimum. Rough Set may eliminate some redundancy through reduction, which is in favour of the classification and simplification of the information, but the calculation is based on some basic concept such as approximation, plus field, minus field and boundary field etc, which will bring deviation by boundary problem. The BP neural network has strong fault-tolerance, but it has no ability of screening and compression on priori knowledge,so the RS and the BP nework are complementary.The paper reduces the inputting variables og BP network by Bayes and RS, then selects training samples and bring the to the network to train, and the trained network will be used to forest. In this paper, a southern city of the 2005 data were analyzed to August 10, 2005 to August 19 points for the entire sample for the study and active load to August 20 for the entire load for the test samples were forecast in August on the 21st load. 55 selected by experience on the condition variable attributes, which is 12 load data, that is, on the 10th to the 19th day the whole point of load; The remaining 43 non-load data, including weather, the date type, sunshine duration, maximum temperature, minimum temperature, average temperature, the biggest humidity, humidity, such as minimum 43 factors, including the date and type of rest days (including weekends and the statutory rest), the weather conditions on the provision of meteorological information is divided into 17 types. The neural network input vector, using the above-mentioned were all rough intensive SR algorithm primaries to the impact of load reduction factors. Table 1 lists a variety of reduction algorithm results. Can be seen, after attribute reduction, the input vector be simplified. Table 1. Comparision of using single attribute reduction method

Method Before reduction

The left variables 42

Hidden layer units 12

Time/min

RSME/%

54

2.7317

Bayes Discernibility matrix Conditional information entropy

30

10

13

2.1478

35

11

12

1.9956

19

14

4

1.3234

Attribute dependability

28

8

10

1.6535

Genetic algorithm

27

13

8

1.8718

Short-Term Load Forecasting Based on Bayes and RS

115

Table 1 has carried out a form from the function forecasting accuracy and training time the forecast to Table 3 kinds method in two aspects comparing, has forecast accuracy among them adopt average proportional error (η ) to be the analytical index , ( P − P ) ⋅ 100 ⎤ ⎡ ⎢∑ ⎥ whose definition as follows: P ⎣ ⎦ Among them, PAi be to forecast Ai

η =

Fi n

n

value, PFi be actual value, n is to forecast number of times. We can see from Table 1, making use of Bayes reduction, the input variables reduces from 43 to 31, calculating time reduces to 13min, the average relative error decrease from 11.11% to 5.82%, not only the attribution was reduced in the maximal degree, but the computing time is shortest relatively, and the calculation error is minimum. It is shown from the training time and accuracy,. The method in the paper is better than traditional BP Neural Network, so it is suitable for the forecasting samples in this area. First of all input and output variables on the regulation of a processing, data processing interval [0,1] between the data. Regulation is one of the many ways here by the following formula: xˆ =

x − xmin . x max − x min

In a 3-layer BP neural network model to predict the hidden layer neurons of the first set to the number of n1 = n + m + a , Among them, m is the output neuron number, n is the number of input neurons, a [1,10] between the constants. Network layer neuron transfer function using S-type tangent function, the output layer neuron transfer function using S-type logarithmic function, because the output function in the interval [0,1] between the output precisely meet the requirements. The learning algorithm uses a faster convergence Levenberg-Marquardt dynamic numerical optimization algorithm. Training error indicators set to 0.01, taking into account the network structure more complicated, the number of training set for 1000, at last we get the comparison results between the way adopted in the paper and traditional BP Neural Network. It shows from the Table 1 that comparing with traditional BP Neural Network, the method based on Bayes and RS not only improves forecasting accuracy, but also iecreases training period significantly. In all the reduction ways of the paper, the conditional information entropy method has the least time and the high precision, and also the minimum forecasting error, which shows that the method put forward in the paper has some application prospect. The simulation data above is all calculated on a same computer with main frequency 2.55GHz and memory 1GB.

5 Conclusion This paper has been submitted one kind of short-term load forecasting method based on Bayes and RS. On the premise of every factor data test result being indicated, being method's turn to be able to think that the forecast is connected with load in synthesis, reach higher forecast accuracy, and be that one kind of effective short period load forecasts method within shorter training time.

116

Y. Li

References 1. Fang, R.: Short-term load forecasting of power system with rough sets and support vector regression. Journal of Huaqiao University (Natural Science) 28, 252–255 (2007) 2. Zhu, W., Shan, M.: A neural network model of risk analysis for public construction project. Journal of Harbin Engineering Unicersity 27, 142–147 (2006) 3. Ying, J., Hua, G., Liu, Y.: Power syetem load forecasting based on v-support vector regression. Journal of Northeast Danli Unicersity (Natural Science) 27, 43–47 (2007) 4. Chen, S.: Several attributes reduction algorithm analysis based on rough set. Wuhan Institute of Industrial Journal 24, 118–121 (2005) 5. Wang, L., Li, Z., Wen, J.: Short-term load forecasting based on support vector regression. Relay 33, 17–21 (2005) 6. Pan, F., Cheng, H., Yang, J.: Power system short-term load forecasting based on SVM. Power System Technology 28, 39–42 (2004) 7. Jiang, Y.: Multi-factor forecast models based on rough sets and BP neural network. Computer Engineering 33, 154–155 (2007) 8. Zhang, Q., et al.: Short-term load forecasting based on rough sets attribute reduction algorithm and support vector machine. Grid Technology 30, 56–60 (2006)

Nonlinear System Modeling with a New Fuzzy Model and Neural Compensation Cruz Vega Israel and Wen Yu Departamento de Control Automtico, CINVESTAV-IPN A.P. 14-740, Av.IPN 2508, Mxico D.F., 07360, Mxico [email protected]

Abstract. Normal fusion procedure of neural networks and fuzzy systems is to use neural learning techniques to train membership functions of fuzzy system. If no mechanistic prior knowledge can be used, fuzzy systems should be obtained from a set of data. The same data are usually used to train these fuzzy systems in the framework of fuzzy neural networks. But modeling accuracy cannot be improved extraordinary, because neural training and fuzzy modeling use the same data set. In this paper, we propose a new modeling idea, the fuzzy system will not be changed, and modeling error between real plant and the fuzzy system is compensated by a neural network. Another contribution of this paper is fuzzy model is generated automatically by kernel smoothing technique. The third contribution of this paper is a new learning approach for neural compensator is proposed, which assures stable and faster learning. Keywords: fuzzy modeling, neural compensation, satbility.

1

Introduction

Fuzzy system is a popular intelligent modeling method , which cannot only model linguistic variables based on fuzzy rules, but also is simple and highly intuitive , especially for complex industrial processes. But fuzzy model did not seem good under testing conditions for the sake of inadequate experimental data. The production rules cannot produce a good precision in case of lack of information. The structure (number of fuzzy rule) also influences identification capability and accuracy [1]. Resent results show that the fusion procedure of neural networks and fuzzy systems seems to be very effective for nonlinear system modeling [2]. A general combination method is to use neural training for a fuzzy model [3]. From prior knowledge, the structure of fuzzy system can be determined. After neural training, the membership functions of fuzzy systems are changed to match the training data. The process of fuzzy rule extraction for nonlinear systems modeling is called structure identification. Two popular methods can be used to construct fuzzy systems: 1) from experts [4], 2) from observed data [5]. In many industrial processes, no mechanistic prior knowledge can be used to construct fuzzy systems. It is difficult to obtain mathematical models for these process. A common approach is to partition input and output data, it is also called fuzzy Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 117–127. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com 

118

C.V. Israel and W. Yu

x (k )

Plant

y (k )

Final model: yˆ (k ) + Δˆ (k ) Δ (k ) e (k )

Fuzzy model

yˆ (k )

G1 ω1 G2

Kernel smoothing

Gm

ω2

Δˆ (k )

ωm RBFNN

Fig. 1. Fuzzy model with neural compensation

grid [3], such as fuzzy C-means clustering [6], mountain clustering [6], and subtractive clustering [7]. Besides clustering approaches, fuzzy models can also be obtained by neural networks method [3], genetic algorithms [4], SVD-QR [8] and support vector machine (SVM) technique [9]. Kernel smoothing is an alternative modeling technique [10], it can find a regression function such that function is best-fit match to data at those data points. Normally, modeling accuracy of kernel smoothing is better than clustering methods, it is faster than SVM and SVD-QR. And also kernel smoothing method is very simple. The idea of kernel regression is putting a set of identical weighted function called Kernel local to each observational data point. The kernel will assign weight to each location based on distance from the data point. The kernel basis function depend only to the radius or width (or variance) from the ‘local’ data point X to a set of neighboring locations x. Kernel regression is a superset of local weighted regression and closely related to moving average and K nearest neighbor (KNN), radial basis function (RBF), neural network and support vector machine (SVM). The first successful kernel smoothing approach is NadarayaWatson estimator [16], which used kernel as a weighting function to estimate f as a locally weighted average. Backpropagation and gradient algorithm are always used for neural network training. Slow convergence and local minimum are main drawbacks. Generally, we have to make some modifications such that learning processes are stable [12]. By using passivity theory, a time-varying learning rate is stable without robust modification [2]. Although the stability of neural identification is guaranteed, the optimal properties are lost. Sometimes they cannot even reach local minima. The time-varying rate defined as η (k) = kc (c is a small positive constant) can give optimally fast convergence in the sense of the misjudgment going to zero proportional to k1 , see [14]. But it results in slow convergence to bad solutions when c and k are small. Therefore, the ”search then converge” approach is pro1 posed by [13], where η (k) = η0 2 . But these two kinds of algorithms c 1+a1 k +a2 ( kc ) cannot guarantee stability.

Nonlinear System Modeling with a New Fuzzy Model

119

In this paper, a new multiple fuzzy models is given, see Fig1. First we use a windows with length of N to collect data, a kernel smoothing support method is applied to obtain an approximator in the time interval [τ1 , τ2 ]. Since the kernel smoothing model has similar form as a fuzzy system, fuzzy rules are constructed and the corresponding fuzzy systems are made. If the same data are usually used to train these fuzzy systems. The modeling accuracy cannot be improved extraordinary, because membership functions training training and fuzzy structure generation use the same data set. In the second stage, the fuzzy system will not be changed, and the modeling error between the real data and fuzzy system will be compensated by a neural network. Then we propose a new learning rate, η0 such as η (k) = 1+a1 c +G(x . It takes the advantages of [2] and [13] and assure k ) k stable and faster learning.

2

Generating Fuzzy Systems via Kernel Smoothing

Consider a nonlinear discrete plant represented by y(k) = f [x (k) , θ] + e (k) T x (k) = [y (k − 1) · · · , y (k − ny ) , u (k − d) , · · · , u (k − d − nu )] ∈ n×1

(1)

and f (·) is an unknown nonlinear difference equation representing the plant dynamics, u (k) and y (k) are measurable scalar input and output, d is the delay, θ is an unknown parameter vector associated with an appropriate but yet to be determined model structure, e (k) is a bounded observation noise, ny and nu are the lengths of output and input, ny + nu = n. In fact, (1) is NARX model [22]. A generic fuzzy model is presented as a collection of fuzzy rules in the following form (Mamdani fuzzy model [15]) Rj : IF x1 is Aj1 and x2 is Aj2 and · · · xn is Ajn THEN y is B j

(2)

We use l(j = 1, 2, · · · , l) fuzzy IF-THEN rules to perform a mapping from the input linguistic vector x = [x1 , · · · , xn ] ∈ n to the output y. From [1] we know, by using product inference, center-average and singleton fuzzifier, the output of the fuzzy logic system can be expressed as ⎛ n ⎞ ⎛ l  n ⎞ l     y = ⎝ wj μAj ⎠ / ⎝ μAj ⎠ = W ϕ [x (k)] (3) j=1

i=1

i

j=1

i=1

i

where μAj is the membership functions of the fuzzy sets Aji , wpj is the point at i which μBpj = 1. W = [w1 , · · · wl ] , ϕ is l−dimension vector function, the element ϕi =

n  i=1

μAj / i

 n l   j=1

i=1

 μAj i

(4)

120

C.V. Israel and W. Yu

If we use Takagi-Sugeno fuzzy model [21][20], the fuzzy rules are Rj : IF x1 is A1j and x2 is A2j and · · · xn is Anj THEN y = pj0 +pj1 x1 +· · · pjn xn (5) where j = 1 · · · l. The output of the fuzzy logic system can be expressed as y =

l  

piq0 + piq1 x1 + · · · piqn xn ϕi

(6)

i=1

where ϕi is defined as in (4). (6) can be also expressed in the form of the Mamdani-type (3), y = W ϕ [x (k)] (7)

1  with W (k) = p0 · · · p1n p111 · · · pl11 · · · pl0 · · · pln . Since the Takagi-Sugeno fuzzy model (6) has the same mathematical expression as the Mamdani fuzzy model (3), in this paper we only discuss Mamdani fuzzy system. Generally the fuzzy neural networks (3) can not match the given nonlinear system (1) exactly, the nonlinear system (1) can be represented as y(k) = W ϕ [V, x (k)] + ε (k)

(8)

where ε (k) is defined as modelling error. The identified nonlinear system (1) can also be written as

 y(k) = W 0 ϕ V 0 , x (k) + f(k) (9) where fk is modeling error, V 0 and W 0 are sets of known parameters chosen by   the user. In general, f(k) ≥ |ε (k)| . From (9) we know the modeling error ft depends on the structure of the fuzzy neural network. Assume we have data of observations [y (k) , x (k)] , taken at time points k ∈ [τ1 , τ2 ]. We assume that the data come from the model f (x) = φ (x) +  where φ (x) is an unknown smooth mean response curve, and  is error. Our goal is to find the kernel smooth estimate φˆ at some pre-specified time point k. Kernel regression of statistics was derived independently by Nadaraya and Watson [16] with a mathematical foundation given by Parzen’s earlier work on kernel density estimation. Kernel regression is the estimation of the functional relationship between two variables y and t on the interval {0 ≤ ti ≤ T } , i = 1···N yi = y (ti ) + ξ where ξ is a random noise variable with the mean equal to zero. The Nadaraya– Watson kernel regression estimate at a time t is   N 2 (t − ti ) 1 i yi k (t − ti ) yi =  N (10) , k (t − ti ) = √ exp − 2σ 2 σ 2π i k (t − ti )

Nonlinear System Modeling with a New Fuzzy Model

121

Here the Gaussian density of statistics is the kernel function. For nonlinear system (1) modeling, (10) is yˆ (k) = k [x (k)

N y k[x(k)−x(ti )] i N i i k[x(k)−x(ti )]   2 i ) − x (ti )] = σ√12π exp − x(k)−x(t 2 2σ

(11)

where yˆ (k) is estimated value of y(k) = f [x (k)] . Statistical analysis of the Nadaraya-Watson kernel regression is difficult because it is defined as the ratio of two random variables. In many important applications in signal processing and automatic control, which usually has a constant pdf. With these type of random variables, the summations  involving theNkernel function are equivalent to a Monte Carlo integration, N i k (t − ti ) → T . The Nadaraya-Watson kernel regression becomes Priestley-Chao regression, which is defined N N yˆ (ti ) = yi k (t − ti ) (12) T i For system identification yˆ (k) =

N N  yi k [x (k) − x (ti )] T i

(13)

The nonlinear plant y(ti ) = f [x (ti ) , θ] + e (ti ) , i = 1 · · · N has only N sparse data in the time interval τ1 ≤ ti ≤ τ2 . We use these data to construct fuzzy rules. For the data set [y(ti ), x (ti )] , we extract fuzzy product rules in the form of (2). x (ti ) is center of the Gaussian function μAj , y(ti ) is the center of the Gaussian i function μB j . Compared (13) with (3), kernel smoothing is equivalent to fuzzy modeling if each data corresponds to one fuzzy rule. However, this kind of fuzzy structure cannot be applied, because there are too many fuzzy rules from kernel smoothing when n is large. A possible way is design a wide kernel function to cover more data, such that these data correspond a rule. A practical method  is full width at half maximum. The maximum width is defined as σmax = 2 2 ln (2)σ, σmax is chosen by the total number of the data, for N example σmax = 10 , the width parameter of Gaussian function is σ = √N . 20 2 ln(2)

∗ ∗ We use center points of each Gaussian function xj , yj , j = 1 · · · v to construct the membership functions. For the data set [y (k) , x (k)] , k ∈ [1, N ] , we extract fuzzy product rules in the form of (13) or (2). x∗j is center of the Gaussian function μAj , yj∗ is the center of the Gaussian function μB j . In the time interval i [τ1 , τ2 ] , we use the following automatic kernel smoother to construct fuzzy rules. Step 1: Initialization: select a and σmax . Moreover, calculate kernel. Step 2: Solve kernel smoothing problem according to (13), which has the same form as the fuzzy system is (3). Step 3: Membership function training: we will discuss it in next section

122

C.V. Israel and W. Yu

Step 4: Check phase: check if the network construction should be changed. If no, continuous training go to Step 3. Otherwise, terminate the training precess, go to network construction phase (Step 2 ).

3

Neural Compensation for Fuzzy Models

Since there exists modeling error Δ (k) between the plant output y (k) and the fuzzy model output yˆ (k), a Radial Basis Function (RBF) neural network Δˆ (k) is used to compensate this error, see Fig.1. Define neural compensation error as e (k) = Δˆ (k) − Δ (k)

(14)

Radial Basis Function (RBF) neural networks have recently gained considerable attention. The advantages of the RBF approach, such as the linearity in the parameters and the availability of the fast and efficient training methods, have been noted in several publications, see [11]. The neural compensator is   m 2  Xi (k) − ξi  ˆ Δ (k) = (15) ωi Gi (X (k)) + b Gi [X (k)] = exp 2σi2 i=1 where m is the nodes number in hidden layer, ωi is the weight, ξi and σi are  T the center and the width of Gi (t), respectively, X (k) = x (k)T , Δ (k) . b is a threshold, it can be combined with the first term as w0 = b, G0 (x) = 1, so m  Δˆ γ (k) = ωi Gi (x). Finally, the RBF neural compensator is i=0

Δˆ (k) = Ωk G [X (k)]

(16)

where Ωk = [ω1 · · · ωm ] , G is m−dimension vector function. We use this RBF neural networks to compensate the modeling error of the fuzzy model. According to the Stone-Weierstrass theorem, this general nonlinear smooth function (14) can be written as ˆ (k) = Ω ∗ G [X (k)] − μ (k) Δ (k) − Δ where Ω ∗ is optimal weight, μ (k) is the modeling error. Since G is bounded function and the output of the plant is assumed bounded, μ (k) is bounded as μ2 (k) ≤ μ, μ is an unknown positive constant. Neural compensation error can be represented as k G [X (k)] + μ (k) e (k) = Ωk G [X (k)] − Ω ∗ G [X (k)] + μ (k) = Ω

(17)

k = Ωk −Ω ∗ . The following theorem gives a stable fast learning algorithm where Ω of the RBF compensator. The following theorem gives a stable gradient descent algorithm for neural modeling.

Nonlinear System Modeling with a New Fuzzy Model

123

Theorem 1. If we use the RBF neural networks (16) to compensate fuzzy mechanistic model error Δ (k), the following gradient updating with time-varying rate can make the final modeling error e (k) bounded (stable in an L∞ sense) Ωk+1 = Ωk − ηk G [X (k)] e (k) η0

where ηk =

2

1 + G [X (k)] + fication error satisfies

c k

, 0 < η0 ≤ 1, c > 0. The average of the identi-

J = lim sup T →∞

 where μ = max μ2 (k) .

(18)

T 1  2 2+c e (k) ≤ μ T 1 − η0

(19)

k=1

k

Proof. We select Lyapunov function as n  2      T Ω  Vk = Ω = ω i2 = tr Ω k

(20)

i=1

From the updating law (18) k − ηk G [X (k)] e (k) k+1 = Ω Ω  2  2     ΔVk = Vk+1 − Vk = Ω k − ηk G [X (k)] e (k) − Ωk  2 k G [X (k)] = ηk2 e2 (k) G [X (k)] − 2ηk e (k) Ω

Using (17) and 0 < η ≤ 1, 0 ≤ ηk ≤ η0 ≤ 1, ηk = 2

1 + G [X (k)] +

c k

η0 2

1 + G [X (k)] +

≤1+1+c

c k

,

ΔVk = ηk2 e2 (k) G [X (k)]2 − 2ηk (e (k) [e (k) − μ (k)]) = ηk2 e2 (k) G [X (k)]2 − 2ηk e2 (k) + ηk 2e (k) μ (k) 2 2 2 2 ≤ ηk2 e2 (k) G  + ηk e (k) + ηk μ (k)  [X (k)] − 2ηk e (k)

(21) 2 = −ηk e2 (k) 1 − ηk G [X (k)] + ηk μ2 (k) η0 (1 − η0 ) e2 (k) + η0 μ2 (k) ≤ −ηk (1 − η0 ) e2 (k) + ηk μ2 (k) ≤ − 2+c  2  2  2  2 Since n min w i ≤ Vk ≤ n max w i and n × max w i are K∞ i , n × min w functions, and (2 + c) e2 (k) is an K∞ -function, η02 μ2 is a K-function. From (17) and (20) we know Vk is the function of e (k) and μ (k) , so Vk admits the smooth ISS-Lyapunov function. The dynamic of the identification error is input-to-state stable. The ”INPUT” is corresponded to the second term of the last line in (21), i.e., the modeling error μ (k) , the ”STATE” is corresponded to the first term of the last line in (21), i.e.., the identification error e (k) . Because the ”INPUT” μ (k) is bounded and the dynamic is ISS, the ”STATE” e (k) is bounded. (21) can be rewritten as ΔVk ≤ −

η0 (1 − η0 ) 2 η0 (1 − η0 ) 2 e (k) + η0 μ2 (k) ≤ − e (k) + η0 μ 2+c 2+c

(22)

124

C.V. Israel and W. Yu

Summarizing (22) from 1 up to T , and by using VT > 0 and V1 is a constant, we obtain T 0) 2 VT − V1 ≤ − η0 (1−η k=1 e (k) + T η0 μ 2+c η0 (1−η0 ) T 2 2 k=1 e (k) ≤ V1 − VT + T η0 μ ≤ V1 + T η0 μ 2+c (19) is established. Remark 1. (18) is the gradient descent algorithm, which the normalizing learning rate ηk is time-varying in order to assure the identification process is stable. This learning law is simpler to use, because we do not need to care about how to select a better learning rate to assure both fast convergence and stability. No previous information is required.

4

Simulation Comparisons

The process of engine at idle has time delays that vary inversely with engine speed and is time-varying due to aging of components and environmental changes such as engine warm-up after a cold start. The measurement of system outputs occurs asynchronously with the calculation of control signals. We assume that the occurrence of plant disturbances, such as engagement of air conditioner compressor, shift from neutral to drive in automatic transmissions, application and release of electric loads, and power steering lock-up, are not directly measured. The dynamic engine model is a two-input and two-output system [17]: ·  · P˙ = kP mai − mao , N˙ p = kN (Ti − TL )  · (23) mai = 1 + km1 θ + km2 θ2 g (P ) ·

mao = −km3 Np − km4 P + km5 Np P + km6 Np P 2 The system outputs are manifold press P (kPa) and engine speed Np (rpm). The control inputs are throttle angle θ (degree) and the spark advance δ (degree). Disturbances act to the engine in the form of unmeasured accessory torque Td · · (N-m). The variable mai and mao refer to the mass air flow into and out of the manifold. mao is the air mass in the cylinder. The parameter τ is a dynamic transport time delay. The function g (P ) is a manifold pressure influence function. Ti is the engine’s internally developed torque, TL is the load torque. The parameters  for an engine model with 1.6 liter, 4-cylinder fuel in N 2 P < 50.6625 √ 1 jected are: g (P ) = + , TL = 263.17 0.0197 101.325P − P 2 P ≥ 50.6625 ·

Td , mao = mao / (120N ) , Ti = −39.22 + 325024mao − 0.0112δ 2 + 0.635δ +  2π 2 2π 0.000102N 2, kP = 42.40, kN = 54.26, km1 = 60 (0.0216 + 0.000675δ) Np − 60 0.907, km2 = 0.0998, km3 = 0.0005968, km4 = 0.0005341, km5 = 0.000001757, τ = 45/Np . Difference technique is used to get the discrete-time states of the · system (23). (23) can be written as x = f (x, u) with x = [P, Np ]T , u = T [θ, δ, Td ] .The discrete-time model for P is P (k + 1) = f1 [P (k) , Np (k) , θ (k) , δ (k) , Td (k)]

Nonlinear System Modeling with a New Fuzzy Model

125

100

50

0

0

200

400

600

800

1000

0

200

400

600

800

1000

100

50

0

Fig. 2. The training and testing results

In order to train the neural model, we select inputs as δ = 30 sin (0.06k) , θ is a square wave with amplitude 20, θ (k) = 20Square(0.04k), Td is a constant, T Td (k) = 10, x0 = [10, 500] . 6 fuzzy rules from kernel smoothing are obtained.The time interval the the hidden layer selection N is chosen as N = 300. We start from Ωk ∈ 1×15 . After then (18) is applied to update the weight Ωk . The final training result is shown in Fig.2. The testing signals are changed as δ = 10, θ (k) = 40 cos(0.05k), Td (k) = 1. The testing result is shown in Fig.2. Now we compare the performances of our fuzzy model with neural compensation (FNC) with neural networks identification (NN) [11], ANFIS fuzzy model (FS) [1], and fuzzy neural model [2] (FNN). The multilayer neural networks as in [11] is Π5,15,1 (one hidden layer with 15 nodes), and its learning rate of backpropagation is fixed as η = 0.05. For FS and FNN, we use 15 rules. The results are shown in Table 1, where the modeling error is calculated as RMS.

Table 1. Comparisons of four identification methods networks case modeling error hidden nodes rule number

NN FS FNN FNC training testing training testing training testing training testing 0.24 0.52 0.21 0.49 0.06 0.43 0.21 0.41 15

15

15

15

15

15

6

6

126

5

C.V. Israel and W. Yu

Conclusion

This paper proposed a new fuzzy model for nonlinear system modeling. Although the fuzzy system plus neural networks based method is widely used for modeling nonlinear systems, there are some new contributions in this paper: the fuzzy model is obtained from kernel smoothing, not from prior knowledge; a new fast training algorithm for the neural compensator is proven to be stable.

References 1. Wang, L.X.: Adaptive Fuzzy Systems and Control. Prentice-Hall, Englewood Cliffs (1994) 2. Yu, W., Li, X.: Fuzzy identification using fuzzy neural networks with stable learning algorithms. IEEE Transactions on Fuzzy Systems 12, 411–420 (2004) 3. Jang, J.S.: ANFIS: adaptive-network-based fuzzy inference system. IEEE Transactions on Systems, Man and Cybernetics 23, 665–685 (1993) 4. Rivals, I., Personnaz, L.: Neural-network construction and selection in nonlinear modeling. IEEE Transactions on Neural Networks 14, 804–820 (2003) 5. Leski, J.M.: TSK-fuzzy modeling based on ε-insensitive learning. IEEE Transactions on Fuzzy Systems 13, 181–193 (2005) 6. Mitra, S., Hayashi, Y.: Neuro–fuzzy rule generation: survey in soft computing framework. IEEE Transactions on Neural Networks 11, 748–769 (2000) 7. Chiu, S.L.: Fuzzy model identification based on cluster estimation. Journal of Intelligent and Fuzzy Systems 2, 159–163 (1994) 8. Chiang, J.-H., Hao, P.-Y.: Support vector learning mechanism for fuzzy rule-based modeling: a new approach. IEEE Transactions on Fuzzy Systems 12, 14–25 (2004) 9. Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines. Cambridge Univ. Press, Cambridge (2000) 10. Wand, M., Jones, M.: Kernel Smoothing. Chapman & Hall, London (1995) 11. Narendra, K.S., Cheng, X.: Adaptive control of discrete-time systems using multiple models. IEEE Transactions on Automatic Control 45, 1669–1686 (2000) 12. Suykens, J.A.K., Vandewalle, J., De Moor, B.: NLq theory: checking and imposing stability of recurrent neural networks for nonlinear modelling. IEEE Transactions on Signal Processing 45, 2682–2691 (1997) 13. Darken, C., Chang, J., Moody, J.: Learning Rate Schedules for Faster Stochastic Gradient Search. In: Proceedings of the IEEE-SP Workshop, pp. 3–12 (1992) 14. Ljung, L., Soderstrom, T.: Theory and Proactice of Recursive System Identification. The MIT Press, Cambridge (1983) 15. Mamdani, E.H.: Application of fuzzy algorithms for control of simple dynamic plant. In: IEEE Proccedings-Control Theory and Applications, vol. 121, pp. 1585– 1588 (1976) 16. Nadaraya, E.A.: On estimating regression. Theory of Probability and its Applications 9, 141–142 (1964) 17. Puskorius, G.V., Feldkamp, L.A.: Neurocontrol of nonlinear dynamics systems with Kalman filter trained recurrent networks. IEEE Transactions on Neural Networks 5, 279–297 (1994) 18. Priestley, M.B., Chao, M.T.: Non-parametric function fitting. J. Royal Statistical Soc. Ser. B 34, 385–392 (1972)

Nonlinear System Modeling with a New Fuzzy Model

127

19. Prokhorov, D., Wunsch, D.: Adaptive critic designs. IEEE Transactions. on Neural Networks 8, 997–1007 (1997) 20. Takagi, T., Sugeno, M.: Fuzzy identification of systems and its applications to modeling and control. IEEE Trans. Syst., Man, Cybern. 15, 116–132 (1985) 21. Baruch, I.S., Lopez, R.B., Olivares, J.-L., Flores, J.M.: A fuzzy-neural multi-model for nonlinear systems identification and control. Fuzzy Sets and System 159, 2650– 2667 (2008) 22. Chen, S., Billings, S., Grant, P.: Recursive hybrid algorithm for nonlinear system identification using radial basis function networks. Int. J. Control 55, 1051–1070 (1992)

A Research of Fuzzy Neural Network in Ferromagnetic Target Recognition Caipeng Wu*, Jiahao Deng**, and Yanli Yang*** National Key Laboratory of Mechatronic Engineering and Control, Beijing Institute of Technology, Beijing 100081, China [email protected], [email protected], [email protected]

Abstract. Based on deeply analyzing the characteristic of the battlefield ferromagnetic targets, according to the problems of magnetic detection system, for instance single detection pattern, low detection resolution and poor antiinterference performance, the Giant Magneto-Impedance(GMI) micro-magnetic sensor in combination with the technology of fuzzy neural networks(FNN) were carried as the core of the magnetic detection system. Take advantage of GMI sensor and FNN to realize accurate recognition of the target in the range of nanotesla magnetic field. In this paper, equable magnetization rotation ellipsoid is used to simulate the tank and military truck, taking the triaxial magnetic moments and semi-focal length, that is M x , M y , M z , c as recognition characteristic quantity , and the FNN is used to recognize the tank and military truck including the categories and motion directions. The method reaches good recognition effect through experimental verification, and it has significance to improve detection range and recognition accuracy. Keywords: Ferromagnetic target recognition, Adaptive neuro-fuzzy inference system, Giant Magneto-Impedance, Micro-magnetic sensors.

1 Introduction The tank, armored vehicle, and military truck are the important ferromagnetic target on the battlefield, and the magnetic detection system that is based on geomagnetic field disturbance generated by the ferromagnetic substance makes use of various magnetic measuring methods to locate and recognize targets. However, the majority of the magnetic fuse detection systems are confronted with the problem of neardistance detecting and poor target identification ability at present, so the early burst and error burst situations happen from time to time. *

**

***

Caipeng Wu is a Ph.D. candidate in Beijing Institute of Technology. His research area is target detection, guidance and control technology. Jiahao Deng is currently a professor in Beijing Institute of Technology. His main research interests include target detection and signal processing, sensing and control technology. Yanli Yang is a Ph.D. candidate and his research interests are target detection and signal processing.

Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 129–136. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

130

C. Wu, J. Deng, and Y. Yang

There are two problems mainly in realizing distant reliable detection according to ferromagnetic moving target on the battlefield. First, the most of the present magnetic fuse systems based on metallic eddy effect, electromagnetic induction or hall effect, exist bad effect in high temperature, poor detection accuracy and so on when conduct target detection; Second, when detection range increase, the high sensitive magnetic sensor is necessary, and the target signal is very weak, so we need reduce detection threshold to measure target signal, which is easily reached by natural or artificial disturbance, then the system has high false alarm probability. In order to solve the above two problems, this paper will design a novel magnetic detection system adopting micro-magnetic sensor based on Giant Magneto-Impedance (GMI) effect and fuzzy neural networks (FNN), then make use of it to detect and recognize ferromagnetic target on the battlefield, especially for tank.

2 Magnetic Detection System Establish The detection system mainly consists of a three-axis GMI magnetic sensor, signal conditioning circuit and the target identification module taking FPGA (Field Programmable Gate Array) as core, as shown in Fig.1.

Fig. 1. Magnetic Detection and Target Recognition System Block Diagram

Taking the three-axis GMI sensor as the core of the sensitive element, detect x (East is positive), y (North is positive), z (Upward is positive). The so-called GMI effect, is referred to a significant change in the ac impedance of a soft magnetic conductor with an ac current when subjected to an applied dc magnetic field, which is announced by Panina and Mohri for the first time in 1994. The GMI sensor initial prototype has been successfully developed abroad, the performance of which achieves the following indexes: detection accuracy reaches 1 μ Oe, reaction speed can reach 1MHz, and the average power consumption is approximately 10mW. Compared with a Giant Magneto-Resistance (GMR) that has a sensitive of ~1%/Oe, the field sensitivity of a typical GMI sensor can reach a value as high as 20~100%/Oe [1-3]. The system control and signal processing module is composed of FPGA and CPLD (Complex Programmable Logic Device). With improvement of FPGA capacity, performance, cost efficiency, and parallel processing ability, it can become an significant hardware realization method for FNN, compared with software implementation which is beneficial to improve real-time performance of the fuse system. The CPLD with design flexibility and high hardware integration is applied to assisted control logic, so it will complete chip selection signal, buffering function of A/D and FPGA, some level conversions function, and other control logic.

A Research of Fuzzy Neural Network in Ferromagnetic Target Recognition

131

3 Fuzzy Neural Network Fuzzy Neural Network, FNN for short, is referred to combine fuzzy logic with neural network appropriately, and the combination starts from 1988, absorbs the merits of the both [4]. It can fully develop the advantages of parallel processing and adaptive learning in neural network, and it has reasoning and decision function using fuzzy logic to human knowledge. If we take the FNN system as a recognition method to fuzzy and complex magnetic characteristic signals from ferromagnetic target on battlefield, we can obtain a magnetic detection system with high recognition rate. TS fuzzy neural network proposed by Takagi and Sugeno is a type of neural network that is the most widely used and deeply researched at present. The FNN based of TS type can realize Adaptive Neuro-Fuzzy Inference System(ANFIS), which is a method using data modeling, that is to say the fuzzy membership functions and fuzzy rules come from the known data’s learning, but not from experiences or intuition. For the ANFIS of the n inputs and single output, x = [ x1 x2 … xn ] T is the inputs defined by fuzzy linguistic variables, and the fuzzy rules proposed by Takagi and Sugeno can be expressed as follows [5, 6]:

： x is A

IF

1

j 1

and … and xn is Anj ，THEN: y j = q j 0 + q j1 x1 + " + q jn xn

Where Ai j (i = 1, 2," , n; j = 1, 2," , mi ) is fuzzy subset of the xi , μ

Ai

j

(1)

( xi ) is its

membership function. Every prescriptive relevance weight is shown as that:

μj = μ

j A1

( x1 ) μ

j A2

( x2 ) " μ

j An

( xn )

(2)

Then the network output can be written as follows: m

y = ∑μj yj

(3)

i =1

4 Magnetic Field Analysis of the Ferromagnetic Target In previous magnetic detection systems, the waveform characteristics of target magnetic signals, such as amplitude, waveform width and slope, were used in target detection [7]. When the projectile-target distance is relatively near, because of the higher signal-noise ratio, we can use frequency filtering and other signal preprocessing method to realize target detection effectively, however, for the situation of farther projectile-target distance and low signal-to-noise, if we only use some time-frequency characteristics of the target signal to wave filter, it is difficult to ensure high recognition accuracy of the detection system. In contrast, using modeling method to research ferromagnetic target fully utilizes the time-frequency characteristics of magnetic signal, the spatial distribution characteristic of magnetic vector and the motion features of target, so it can be called a holographic technique of the target field. If we can build the stable reliable model of ferromagnetic target, utilize the model parameter that is the most refined essential characteristic of target magnetic

132

C. Wu, J. Deng, and Y. Yang

field as the characteristic quantity of target recognition, and combine GMI micromagnetic senor as the core of detection system, the target detection distance will be improved significantly at the situation of high correct recognition rate. 4.1 Target Magnetic Field Model Building

、

The most typical ferromagnetic targets of land battlefield are tank armored vehicle and military truck, all of which are magnetic objects having a certain geometry. Taking tank as an example, the common model building methods include uniform magnetized rotary ellipsoid with array of magnetic dipole modeling method, rectangular parallelepiped with hollow rotary ellipsoid model and a single rotary ellipsoid method. The advantage of former two mixed modeling methods is that they can improve fitting precision on the near field of tank, but the deficiency is the model parameters are too complicated and not stable enough. While for a single rotary ellipsoid method, although the fitting precision on the near field of tank is poor, with distance growing, it becomes higher and higher, and the model parameters are more stable. So when the projectile-target distance is relatively far, we can use a single rotary ellipsoid to simulate the magnetic field of tank that is shown in Fig.2, and take the model parameters as the characteristic quantity of target recognition.

Fig. 2. Modeling schematic diagram for tank magnetic field

In the model of Fig.2, the parameters are defined as follows: the coordinate origin is located in the center of tank, the semi-major axis of the ellipsoid is a = L / 2 ( L is equal to the longitudinal length of the tank), the short half axis is expressed as b = W ⋅ H / 2 ( W , H respectively represent the transverse length, the height of the tank), c is the half focal length and equal to a 2 − b 2 [8].So on the ellipsoid coordinate system, the three-axis components of magnetic field intensity on point P( x, y, z ) can be expressed as: H x = M x a x + M y a y + M z a z , H y = M xbx + M y by + M z bz , H z = M x cx + M y c y + M z cz

(4)

In (4), [ M x , M y , M z ] is the magnetic moment of ellipsoid model, and the coefficient matrices are [ax , a y , az ]T [bx , by , bz ]T and [cx , c y , cz ]T that can be calculated through model geometric parameter.

A Research of Fuzzy Neural Network in Ferromagnetic Target Recognition

133

4.2 Target Characteristic Quantity Extraction

The values of magnetic moment that are obtained in different point are not the same, due to the complex shaped parts of tank and the inhomogeneous distribution of ferromagnetic substance, but we can adopt least square method to get the estimated value of model parameters. In the practical projectile target crossing process, the stepwise regression method is used to estimate the magnetic moment parameters [ M x , M y , M z ] of ferromagnetic target real-timely, and the practical projectile target

—

crossing velocity can be calculated through the method. By comparing the real-time model parameters calculated by the above method and that of knowledge base, the result of accurate target recognizing can be attained.

5 FNN Recognition System Design The ANFIS is a modeling method based on mass data, in which the membership functions and fuzzy rules come from learning the knowing data, so it is known as a recognition system based on data rules. The so-called “data rules” means unify organically the input data (target feature) and the rules that are the “IF-THEN” in the ANFIS to form the rule base, which can perform target recognition. 5.1 Fuzzy Rule Base Construction

According to the analysis above, the “ M x , M y , M z , c → y ” is the expected input-output data couple, where M x , M y , M z , c are the magnetic target model parameters according to rotary ellipsoid modeling, y is the result of target recognition. In fuzzy system, the input and output spaces should be divided into some fuzzy spaces, where there are corresponding membership functions. The input and output parameters need to be localized in the spaces having the biggest membership functions, which based on the corresponding membership of M x , M y , M z , c at different intervals, then a rule is generated from the every satisfying input-output data couple. The confidence principle can be used to solve those conflicting rules, that is selection the rules having higher confidence. Finally, the mapping relation can be determined according to combination of fuzzy rules [9-11]. 5.2 Application Example of Ferromagnetic Target Recognition

The received signals of magnetic detection system on land battlefield are mainly from tank target, common military truck, industrial interference or circuit noise, and natural interference signal, especially for the geomagnetic storm and pulsation. For the latter two interference signals, although the frequency range of signals is so wide that it can cover that of the target magnetic field completely, they can’t pass the examination of rotary ellipsoid model and the fitting accuracy is poor, because their magnetic field distribution can’t be simulated by a uniform magnetic rotary ellipsoid; whereas the

134

C. Wu, J. Deng, and Y. Yang

military truck is very similar to tank in geometry shape, material composition and weight, so the two types of targets is the distinguished emphases. Take a three bits binary array T (T1, T2 , T3 ) as the output of FNN, and there are eight possible outputs, which respectively direct at the four driving directions of tank and military truck. For improving the reliability of target recognition system, we have measured the actual magnetic field values of a certain main battle tank and a “Liberation” truck, and according to different measurement points, in the space of 20m × 15m × 10m including target, we selected 400 groups sample data that is 100 groups data selected in one direction, in which 200 groups data constitute training group, and the other 200 groups data is the testing group of the FNN. All of the data need to be preprocessed to obtain the model parameters M x , M y , M z , c at the four directions, and normalize the

—

model parameters as the inputs of FNN. Some data having been processed are shown in Table 1. Table 1. Partial training samples and the expected outputs

Model parameters(input) Mx

My

Mz

c

0.452 0.437 0.484 0.474 0.075 0.041 0.168 0.162 0.989 0.952

0.851 0.857 0.999 0.999 0.522 0.521 0.379 0.378 0.322 0.018

0.075 0.152 0.797 0.869 0.041 0.172 0.667 0.699 0.240 0.766

0.061 0.079 0.873 0.920 0.011 0.112 0.775 0.796 0.157 0.793

…

T (T1, T2 , T3 )

000 000 100 100 001 001 101 101 011 111 …

5.3 Realization Process of FNN

According to the data of Table 1, we can construct the ANFIS to complete the recognition of tank and truck targets in species and directions, which includes input layer, fuzzy layer, reference layer and conclusion layer. The inputs of FNN is M x , M y , M z and c , and the output is the recognition result. In MATLAB fuzzy logic toolbox, the function of genfis1() is used to train the ANFIS and generate the initial value of TS type ANFIS, and utilizing the function of anfis() to train the initialized network structure. The optimum parameters of membership functions can be calculated through data learning effectively, and make the designed Takagi-Sugeno type FNN simulate the expected or actual relationship of input-output. Fig.3 is the graph of membership functions about M x , M y , M z , c before and after training.

1

Before Training 1

0.5

0.5

0

c

Mx,My and Mz

A Research of Fuzzy Neural Network in Ferromagnetic Target Recognition

0

0 1 0 After Training 1

0.5

0

Mz

My

0.5 0

0.5

1

1

0.5

0.5

0

0

0.5

0

1

0.5

1

0

0.5

1

0

0.5

1

0.5 0

1

c

Mx

1

135

Fig. 3. Membership Function about M x , M y , M z and c Before and After Training

The error change curves of training and testing data is shown in Fig.4. -3

6

x 10

A B

5

training error testing error

B 4 3 2 1 0 0

A 10

20

30

40

50

60

70

80

90

100

Fig. 4. Training and Testing Error Change Curves

From Fig.4, we can see both of the training and testing errors is smaller than 0.002 after function training of 100 times, and the fitting effect is good. The testing error is bigger than the training error, but the correct recognition rate is up to 95%.

6 Conclusion In this paper, according to the problems of magnetic detection system, such as the near detection distance and poor target recognition ability, we proposed the solution that used the new generation magnetic sensor—GMI sensor having better performance as

136

C. Wu, J. Deng, and Y. Yang

the detecting element, and took fuzzy neural network as the recognition tool of ferromagnetic target. It is important to make above research for improving the distance and recognition accuracy of magnetic detection system. Acknowledgments. The work was supported by the National Natural Science Foundation of China under the grant No.60874100, and the Innovation Foundation of Aerospace Science and Technology (CAST200834).

References 1. Panina, L., Mohri, K.: Magneto-impedance effect in amorphous wires. Appl. Phys. Lett. 65, 1189–1191 (1994) 2. Mohri, K., Uchiyama, T., Panina, L.: Recent advances of micro-magnetic sensors and sensing application. Sensors & Actuators A: Physical 59, 1–8 (1997) 3. Kentaro, T., Yichi, H., Masayoshi, E.: Three-axis magneto-impedance effect sensor system for detecting position and orientation of catheter tip. Sensors & Actuators A: Physical 111, 304–309 (2004) 4. Sugeno, M., Yasukawa, T.: A fuzzy logical based approach to qualitative modeling. IEEE Trans. on Fuzzy Systems 1(1), 7–31 (1993) 5. Hairong, S., Pu, H., Lihui, Z.: A New Method to construct Fuzzy Systems Based on Rule Selecting. In: ICMLC 2004, vol. 3, pp. 1855–1858 (2004) 6. Takagi, T., Sugeno, M.: A robust stabilization problem of fuzzy control system and its application to backing up control of truck-trailer. IEEE Trans. on Fuzzy System. 2(2), 119–134 (1994) 7. Chunsheng, L., Qian, X., Shenguang, G.: A modeling algorithm for detection of moving on-water magnetic objects. Journal of China Ordnance 2(26), 192–195 (2005) 8. Pingxian, Y., Shenguang, G.: The physical field of warship, pp. 25–60. Weapon Industry Press (1992) 9. Zhenyuan, J., Ying, Z., Yingdong, S., Wenyan, T.: The application of NN in recognition of vehicles. Journal of Natural Science of Heilongjiang University 1(26), 39–42 (2009) 10. Tao, Z., Tingjin, L., Xuehai, Z.: The recognition method of small underground objective based on fuzzy neural network. Journal of Projectiles, Rockets, Missiles and Guidance 4(27), 316–319 (2007) 11. Hitoshi, L., Masafumi, H.: Adaptive fuzzy inference neural network. Pattern Recognition 10(37), 2049–2057 (2004)

Multiple T-S Fuzzy Neural Networks Soft Sensing Modeling of Flotation Process Based on Fuzzy C-Means Clustering Algorithm Jiesheng Wang, Yong Zhang, and Shifeng Sun School of Electronic and Information Engineering, Liaoning University of Science & Technology, Anshan 114044, China [email protected], [email protected], [email protected]

Abstract. Inspired by the idea of combining multiple models to improve prediction accuracy and robustness, a soft sensing modeling of flotation process based on multiple T-S fuzzy neural networks and fuzzy c-means clustering algorithm (FCM) is proposed. Firstly, the model adopts principal component analysis (PCA) to reduce dimensions of the input variables data composed of texture characteristics of floatation froth image and process variables. FCM algorithm is used for separating a whole training data set into several clusters with different centers and each subset is trained by T-S FNN. The degrees of membership are used for combining several models to obtain the finial soft sensing result. Simulation results show that the proposed modeling is effective in the prediction of indexes and meets the requirement for optimization computation for the flotation process. Keywords: Flotation Process, Texture Characteristic, Multiple T-S Fuzzy Neural Networks, FCM, Soft Sensing, PCA.

1 Introduction In the flotation process, technical indicators such as concentrate grade and flotation recovery of economic are the key to production process control targets. However, it can only be adjusted and be based on an off-line analysis. Because large-time delays, it is difficult to achieve the quality of direct closed-loop control [1]. This estimating problem of process quality indicators of off-line measurement is solved effectively by using inferred estimates (soft-sensing) technology [2]. At present, the single model structure which uses the sample data is most used in the non-linear soft sensor modeling. If we do not restrict the size of the model and the training data is sufficient, a satisfactory model structure and the approximation accuracy are got which are based on neural networks or fuzzy systems modeling. However, as the expansion of the training area, the systemic states become more and more complexity. It will result in low efficiency model optimization and the reducing of accuracy of the model prediction and the significantly ability of robustness and generalization. By considering the divide and conquer, and task decomposition, a number of model structure are proposed by many scholars in order to improve Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 137–144. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

138

J. Wang, Y. Zhang, and S. Sun

prediction accuracy and robustness, such as Multiple Neural Networks (MNN), Stacked NN (SNN) and Distributed NN (DNN), and so on [3-4]. A method which based on principal component analysis (PCA) and fuzzy c-means clustering algorithm (FCM) method of the multi-T-S model structure (the PFM T-S algorithm) is presented in this paper. Simulation studies have shown that the method can significantly improve the model accuracy and robustness.

2 FCM-Based Multiple T-S Fuzzy Neural Network Model Considering a multi-input single-output (MISO) system, the training sample set can be expressed as D = {Y , X | i = 1, 2, " , m} . Y = { y | j = 1, 2, " , n} is the output matrix ( n is the number of the training datum and m is the number of input variables). X = [ x , x , " , x ]′ (i = 1, 2, " , m ) is the input vector. If D is divided into c clusters {D , D , " , D } , it can establish c of sub-model {M , M , " , M } by using a subset {D | i = 1, 2, " , c} . Fig. 1 is the structure diagram of PFM T-S FNN model [4]. j

i

i

1i

1

2

2i

ni

c

1

2

c

i

Fig. 1. PFM T-S Fuzzy neural network model structure

2.1 FCM Clustering Algorithm FCM clustering algorithm [5] divides the feature points of the feature space X = ( x , x , " , x ) into c categories ( 2 ≤ c ≤ n ) , the i class cluster center is 1

2

expressed as

n

vi , any feature point

x

j

belong to the i

category of

membership u ( 0 ≤ u ≤ 1) , and uij Satisfy the following conditions: ij

ij

c

∑u

ij

=1

for

j = 1, 2, " , n

(1)

i =1 n

0
0 ; m ax

Step2: Center on the clustering number c = 2, 3, " , c , initialize fuzzy clustering max

Center: V = { v , v , " , v 0

10

20

c0

}.

Step 3: For t = 1, 2, " , T , according to the following formula, calculate membership degree matrix and the cluster center:

uij ,

t

⎡ 1 =⎢ ⎢ xi − v j ⎣

⎤ ⎥ 2 ⎥ ⎦

1

⎡ 1 ⎢ ∑ j =1 ⎢ x − v j ⎣ i

a −1

c

⎤ ⎥ 2 ⎥ ⎦

1

a −1

(5)

n

v j, t =

∑ [u i =1 n

ij ,t

]m xi (6)

∑ [uij,t ]m i =1

Where i = 1, 2," , n, j = 1, 2," , c . If V − V < ε , switch to the next step, otherwise repeat step 3; Step 4: According to the formula (4) Calculate V (U , c ) ; If c < c max , switch to t

t −1

p

Step 2; Otherwise, stop iteration. cb is the optimal number of clusters. cb Satisfy the following formula:

140

J. Wang, Y. Zhang, and S. Sun

Vp (U , cb ) = min{Vp (U , c )}, c = 2, 3, " , cmax

(7)

2.2 Flotation Foam Image Texture Feature Selection and Extraction

The state of the production process flotation can be reflected by flotation foam statistical characteristics of image texture, the iterative changes of different gray values which distribute in the space forming the image texture, a gray level relationship exist between the two pixels (the gray space-related features).LCM reflects the relationship of changing characteristics, which image gray on the direction, the adjacent interval and of the magnitude image. That f ( x, y ) pixel of gray-scale I for proceeding, and declination θ , distance δ , gray-scale of the j of Pixel ( x + Δx , y + Δy ) of the probability of simultaneous P (i , j , δ , θ ) .That means the joint distribution statistical form of two-pixel. It can be expressed as follow: P (i , j , δ , θ ) = {[( x , y ), ( x + Δx , y + Δy )] | f ( x , y ) = i

(8)

f (x +Δx, y +Δy) = j; x =0,1,", Nx −1; y =0,1,"Ny −1}

(9)

Where i , j = 0,1, " , L − 1 , x , y is pixel coordinates in the image, L is the gray series of the image, N and N indicated rows and columns of the image number. According to the definition above, GLCM is defined by the i row and column j element. That x

y

all the θ directions of the image and separation as δ , the gray is on the difference frequency of the i and j pixel. θ connects the two clockwise and the pixel x Axis angle. Its value is usually 0o, 45o, 90o and 135o. Three characteristic quantities of the calculation of the statistical characteristics texture energy (E), entropy (ENTS), and moment of inertia (I) are based on the gray co-occurrence matrix, are defined as follows [7]:

E=

L −1 L −1

∑∑ Pˆ (i, j ) i=0

∑∑ Pˆ (i, j) log

ENTS = −

I =

2

Pˆ (i, j )

(11)

L −1 L −1

L −1

∑n { ∑ ∑ 2

n=0

(10)

j=0

i =0 j =0

Pˆ (i , j )}

(12)

n= i− j

Three texture parameters of the flotation froth image can be calculated by using Equation (10) - (12). They can reflect the amount of pulp in the reagent addition and indirectly reflects the state of the production in the flotation process. This article use

Multiple T-S Fuzzy Neural Networks Soft Sensing Modeling of Flotation Process

141

image texture characteristics and process variables as the input variables consisting of PFM T-S fuzzy neural network soft-sensor model of the input variables. 2.3 Dimensionality Reduction Based on PCA

If input vector dimension of T-S model is too large, it would make a great network topology and training become very complex. We can use PCA on the T-S model to reduce model high-dimensional of input vector dimensionality. Using Principal Component Analysis [8], we selected the process of input vector principal component of the T-S model as follows: (1) Data collection operation, a number of the image texture statistical characteristic energy (E), entropy (ENTS), and moment of inertia (I) and the corresponding time period to the ore grade, the ore flow, the ore concentration, give mine size and pharmaceutical flow measurements X ∈ R in flotation process, m : process number of variables (in this case 8), n : a sample of data vectors. In order to avoid the impact of different dimensions of process variables on the results and ease calculate. It is , standard necessary to normalize the data processing. Using medium vector deviation vector σ , the normalized process variables are defined as follows: n ×m

μ

xij = ( xij − μ j ) / σ j (i = 1, " ,n,j = 1, " ,m)

(13)

Define X is Hutchison normalized process variable vector. Covariance matrix is ∑ . (2) Calculate the m characteristic values λ of the covariance matrix ∑ and the j

orthogonal eigenvectors

Pj

of the corresponding units;

(3) Calculate the j principal component t : j

 t j = XP j

(14)

(4) Calculate the principal component model: T T T Xˆ = t1 P1 + t2 P2 + " + t j Pj

Where

tj ∈ R

n

is the principal component,

Pj ∈ R

m

(15)

is the load vector, and

Pj

is the

characteristic roots of matrix ∑ of the matrix t arranged in descending. Each pair j , P are the corresponding eigenvectors P by the corresponding eigenvectors of

j

j

j

Eigen values λ of the descending order. The first pair intercepted the largest amount of information of the load vector of all the decomposition and principal component vectors, the rest of inferior race and so on. T = [t , " , t ] is the main element matrix, P = [ P , " , P ] is the load matrix, and then the equation above can be expressed as: j

1

1

m

m

T Xˆ = T P

(16)

142

J. Wang, Y. Zhang, and S. Sun

(5) Calculate the previous k (1 ≤ k ≤ m ) variance contribution rate ηk : k

η k = ∑ λi i =1

m

∑λ

(17)

i

i =1

Then determine the main element k of the PCA model number. Because there is a high correlation between the variables, if general previous k ( k 0 . By Optimization fuzzy model F (θ , w, p ) .we can obtain the minimum number of fuzzy rules p and the optimal model parameters (Including the rules antecedent membership function parameters θ and the rule consequent linear parameters w ), which makes the modeling error function E = y − F satisfy the inequality E (θ , w) < ε . According to s historical data sets Victoria {P1 , P2 ," , Pn } , a MISO fuzzy model can be expressed as having the following form of fuzzy rule sets:

：

Ri IF x 1 is A i1 and x 2 is A i2 " and x s is A is , Then y i = zi ( x ) Where,

x = ( x1 , x2 , " , x s ) ∈ U 1 × U 2 × " × U s

(18)

Are linguistic variables; Ai j is fuzzy sets on

the domain of discourse U ∈ R ( j = 1, 2, " , s ) ; Ri (i = 1, 2," , p ) :the i rule; yi ∈ V : the i rule output. z i (x) has the following two forms: Single-valued ( zi ( x ) = bi ), called as j

the Mamdani model; Linear function ( zi ( x ) = bi 0 + ∑ j =1 bi j x j ), called as Takagis

Sugeno (T-S) model. T-S used in this paper fuzzy neural network model diagram as shown below. The model input X r , r = 1," , 4 , each T-S model input is divided into three membership function X r u , u = 1," , 3 ; we use concentrate grade and flotation recovery as a model output ( f h , h = 1, 2 ). For each T-S sub-model, we can use gradient descent method of least squares mixed model to adopt the before and after the pieces of parameters: First, we fix antecedent parameters, prior to the calculation, and identify the last pieces by using least squares linear parameter; Second, we use gradient descent method to adjust the non-linear parameters of fuzzy rules, the algorithm in the reference [9] as described.

Multiple T-S Fuzzy Neural Networks Soft Sensing Modeling of Flotation Process

143

3 Industrial Experiments Collect historical data of production of the flotation process and choose 700 group historical data with uniform and representative. Then we divided the pre-processing datum into two parts. The first part used as the training data by using PMM T-S model and a simple model of T-S model in the paper, then we learn and train the model. The second part is used to verify the soft-sensor model performance. Fig. 2 and Fig.3 are the prediction output of concentrate grade and flotation recovery.

Fig. 2. The predictive output of concentrate grade

Fig. 3. The predictive output of flotation recovery

It can be seen from Fig. 2 and Fig.3 that the multi-TS FNN soft-sensor model is based on PCA and FCM algorithm is better than a single TS soft-sensor model at economic and technical indicators (concentrate grade and flotation recovery). It has higher prediction accuracy and generalization capabilities. The forecast error of concentrate grade is between ±0.3 percent and the flotation recovery of the forecast error is between ±0.1 percent. The simulation results show that the higher prediction accuracy of the model-based prediction model is helpful to stabilize the flotation of

144

J. Wang, Y. Zhang, and S. Sun

pharmaceutical operations and improve operational recovery rate. It has a certain practical value for the flotation process and laid the foundation for optimal control.

4 Conclusions In this paper, a multi-T-S fuzzy neural network soft-sensor model based on principal component analysis and fuzzy c-means clustering algorithm is presented. According to extrapolation of the actual operating data estimates, it shows that the method has a good tracking rate and high estimation accuracy and fully meets computing requirements of the real-time optimal flotation reagent addition system. Acknowledgments. This work is supported by the Scientific Research Foundation of the Higher Education Institutions of Education Bureau of Liaoning Province, China (Grant No. 20060432) and the Program for the Innovative Research Team of Education Bureau of Liaoning Province, China (Grant No. 2008T091).

References 1. Zhang, Y., Wang, J.S., Wang, W.: Application of Expert Control Method to Flotation Process. Control and Decision 19, 1271–1274 (2004) (in Chinese) 2. Zhu, X.F.: Soft-sensing Technique and Its Applications. Journal of South China University of Technology 30, 61–67 (2002) (in Chinese) 3. Wu, J., Xu, Y., Ou, H.Y.: Modular Fuzzy Logic System Based on Principal Component Regression Analysis. Journal of Xi’an Jiaotong University 36, 1311–1314 (2002) (in Chinese) 4. Zhong, W., Yu, J.S.: Study on Soft Sensing Modeling Via FCM-based Multiple Models. Journal of East China University 26, 83–87 (2002) (in Chinese) 5. Bezdek, J.C.: Pattern Recognition with Fuzzy Objective Function Algorithm. Plenum Press, New York (1981) 6. Chen, M.Y., Linkens, D.A.: Rule-base Self-generation and Simplification for Data-driven Fuzzy Models. Fuzzy Sets and Systems 142, 243–265 (2004) 7. Bartolacci, G., Pelletier Jr., P., Tessier Jr., J.: Application of Numerical Image Analysis to Process Diagnosis and Physical Parameter Measurement in Mineral Processes. Minerals Engineering 114, 734–747 (2006) 8. Ipek, H., Ankara, H.: The Application of Statistical Process Control. Minerals Engineering 12, 827–835 (1999) 9. Takagi, T., Sugeno, M.: Fuzzy Identification of Systems and Its Applications to Modeling and Control. IEEE Transaction on Systems, Man, and Cybernetics 15, 116–132 (1985)

A Using Reliability Evaluation Model for Diesel Engine Based on Fuzzy Neural Network Ying-Kui Gu and Kai-Qi Huang School of Mechanical and Electronical Engineering, Jiangxi University of Science and Technology, Ganzhou 341000 China [email protected]

Abstract. Using reliability evaluation for diesel engine is a complex, dynamic and uncertain process. In order to make an objective and right evaluation, and offer a stronger decision-making tool for designer and user of diesel engine, the neural-network-driven fuzzy reasoning mechanism of using reliability evaluation was developed based on the detail analysis of engine using reliability in the case that there is no sufficient quantitative information or the information is fuzzy and imprecise, where a feedforward neural network was used to replace fuzzy evaluation in the fuzzy system. Applications show that the evaluation result can be used as references for the improvement of reliability and maintainability of engines, and for the establishment of maintenance strategy. Keywords: Neural network, fuzzy logic, fuzzy reasoning, reliability evaluation, multiple attribute decision making.

1 Introduction As the momentum of automobile, engineering machine, ship and power plant units, diesel engine plays an important role in the national economy. With the fast development of science and technology, diesel engine is increasingly developing towards large scale, high speed, high precision, and its working performance has been constantly improved, the automation degree is also higher. On the one hand, the productivity of labour as well as quality of product is greatly improved, and the cost of production as well as consumption of energy is reduced. But on the other hand, once certain part or certain link of diesel engine breaks down, there will be shut down or off production, thus enormous economic loss would be directly or indirectly caused. Therefore, it is very imperative to evaluate the using reliability of diesel engine. However, it is very difficult to evaluate the using reliability of diesel engine because of the complexity, fuzziness and dynamic uncertainty of evaluation process. Fuzzy logic and neural network provide stronger tools for decision-making modeling and analyzing under the fuzzy and uncertain environment [1-3]. In this paper, we presented an approach to evaluate the using reliability of diesel engine using fuzzy logic and neural network. First, the using reliability indices for diesel engine were Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 145–152. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

146

Y.-K. Gu and K.-Q. Huang

analyzed in detail. Second, we developed a fuzzy-logic-based reasoning mechanism to analyze the using reliability under the fuzzy and imprecise design environment. Then, a neural-network-driven fuzzy reasoning system was developed to improve the reasoning efficiency, where the neural network was used to replace fuzzy evaluation in the fuzzy reasoning system. Finally, a case was used to illustrate the proposed neural-network-driven fuzzy reasoning.

2 Using Reliability Indices Analysis for Diesel Engine The main using reliability indices for diesel engine are listed as follows. (1) Mean Time to First Failure (MTTFF) The point estimate value Tˆf of MTTFF and confidence lower limit value T fL of interval estimation are as follows. 1 n Tˆ f = [∑ t i + ( N − n)t . n i =1

(1)

n

T fL =

2[∑ t i + ( N − n) ⋅ t i =1

χ 12−a (2n + 2)

.

(2)

where N is the output of diesel engine. n is the number of diesel engine that break down (not include minor failure). t i is the time to first failure of ith diesel engine, km (h). t is test cut-off distance (or time), km (h). a is significance level, the suggested value a = 0.1 . (2) Mean Time Between Failure (MTBF) The point estimate value Tˆb of MTBF and confidence lower limit value TbL of interval estimation are as follows. Nt . Tˆb = n TbL =

χ

2 1− a

2 Nt . ( 2 n + 2)

(3)

(4)

where N is the output of diesel engine. n is the number of diesel engine that break down (not include minor failure, that is not include 4 kinds of faults). t is test cut-off distance (or time), km (h). a is significance level, the suggested value a = 0.1 . (3) Use Availability (UA) N

A=

∑U

i

i =1

N

∑ (U i =1

i+

Gi )

=

MTBF . MTBF + MTTR

(5)

A Using Reliability Evaluation Model for Diesel Engine Based on Fuzzy Neural Network 147

where N is the output of diesel engine. U i is running time of ith diesel engine, h. Gi is the stopping time of ith diesel engine because of fault (include failure diagnosis time, preparing time, repairing time), h. MTBF is mean time between failure, h. MTTR is mean time to repair, h. (4) Equivalent Failure Rate (EFR) D=

1000 4 ∑ ε j rj . Nt j =1

(6)

(5) Mean Maintenance Cost Ratio of Each Year (MMCR) When reliability of diesel engine is measured from economical efficiency, it can be measured by mean maintenance cost ratio of each year. M =

C ma int enance × 100% . C price

(7)

where C ma int enance is the mean maintenance cost of each diesel engine for the first year, unit: yuan. C price is the price of diesel engine, unit: yuan. (6) Comprehensive Evaluation Value (CEV) Q = 100 −

(MTBF) 0 Nt 0

r

∑K e . i =1

(8)

i i

where (MTBF) 0 is the target value of reliability on diesel engine’s product or the MTBF value of congeneric advanced diesel engine’s product, unit: h. N is output of diesel engine . t 0 is test cut-off distance (or time), km (h). k i is coefficient of injury about ith fault, the specific value can be obtained from following table, ei is coefficient of injury about ith fault, ei =

2 ⋅ t0 , where t i is accumulative work t0 + ti

mileage of diesel engine for ith fault (or time), km (h). Each index can be obtained by reliability trial or cumulative observation, while using reliability on diesel engine can be synthetically measured by all the indices, thus hierarchy model of using reliability on diesel engine can be established as shown in Figure 1. Evaluation result

MTTFF

1# engine

MTBF

UA

2# engine

EFR

CEV

MMCR

3# engine

…

n# engine

Fig. 1. Hierarchical respresentation of using reliability evaluation for engine

148

Y.-K. Gu and K.-Q. Huang

3 The Using Reliability Evaluation Method Based on Fuzzy Logic Reliability evaluation process is a complex problem. There exist many factors, which influence the evaluation result of the engine reliability. In order to simplify the problem and also to illustrate the approach, only the following three important indices are considered in this paper, i.e. (1) use availability, (2) mean maintenance cost ratio of each year, (3) comprehensive evaluation value. All these indices are fairely vague and fuzzy and can be represented linguistically. Thus, fuzzy sets and fuzzy logic are ideally suited for representing these indices. These indices form the input of the model and the output are the desirability of the reliability evaluation results. Figure 1 illustrates the hierarchical linguistic representation. Since these representations are fuzzy and approximate, neural network learning is a necessity to improve this approximate model.

Fuzzy membership

VL

L

M

H

0.25

0.5

0.75

VH

1 0.5

0

1.0 UA, MMCR, CEV

Fig. 2. Membership functions of reliability indices

The indices are represented lingusitically in Figure 2. each index is divided into several different grades based on a universe of discourse of [0,1]. Each grade is represented by a triangular membership function. The three indices, use availability, mean maintenance cost ratio of each year and comprehensive evaluation value has the linguistic grade: VL (very low), L (low), M (medium), H (high), VH (very high) respectively. The problem is to evaluate the engine reliability based on the three indices. Notice that this is a multiple criteria problem and, in general, the three indices cannot mutually compensate. Therefore, the output variable can also be divided into different fuzzy levels. The connections between the input indices and the output evaluation results can be made by the use of fuzzy rules. Table 1 illustrates these fuzzy if-then rules. For example, Rule 1: if the use availability is VH, and the mean maintenance cost ratio of each year is VL and the comprehensive evaluation value is VH, then the grade of the output is A1. Rule 2: if the use availability is VH, and the mean maintenance cost ratio of each year is VL and the comprehensive evaluation value is H, then the grade of the output is A2.

A Using Reliability Evaluation Model for Diesel Engine Based on Fuzzy Neural Network 149 Table 1. Fuzzy if-then rules I/O Sample 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

Input (if) MMCR VL VL L L VL M M L M VL VL L L VL M M L M VL VL L L VL M M L M

UA VH VH VH VH VH VH VH VH VH H H H H H H H H H M M M M M M M M M

Out (then) Ranking Classic A1 A2 A3 A4 A5 A6 A7 A8 A9 B1 B2 B3 B4 B5 B6 B7 B8 B9 C1 C2 C3 C4 C5 C6 C7 C8 C9

CEV VH H VH H M VH H M M VH H VH H M VH H M M VH H VH H M VH H M M

The expression if-and-then is in fuzzy logic terms. To simplify the illustration, only three levels of each index with a total of 27 rules are considered in Table 1. the different output grades for these 27 rules are shown in Figure 3. The output ranking has three classes designated as A, B and C, and each class has five grades. For example, class A is divided into A1, A2, A3, A4 and A5.

Fuzzy membership

Class

C5 B5(C4) B4(C3) B3(C2) A5(B2) A4(B1) A3

A2

A1

1 0.5

0

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

Fig. 3. The membership function for output ranking score

0.1

150

Y.-K. Gu and K.-Q. Huang

The fuzzy rules or the interconnections between the input and the output can first formulated approximately based on the vague knowledge of the actual system. These approximate interconnections can be refined for any given system by actual learning using neural network. The training sample sets can be obtained from the actual system experts.

4 BP-Neural-Network-Driven Fuzzy Evaluation Model Because back-propagation neural network has the ability to learn by examples, it has been used in pattern matching, pattern classification and pattern recognition. Therefore, it can be used to establish the neural-network-driven fuzzy evaluation model for engine reliability. The BP network is constructed in three layers. Nine nodes in input layer, five nodes in hidden layer, and one node in the output layer. In the training process, a hybrid learning algorithm that combined unsupervised learning and supervised gradient-descent learning procedures was used to build the fuzzy rules and train membership functions. Through assigning values to the input variables, calculating the membership function measures of the input nodes, and achieving the output membership function using fuzzy reasoning, the correct data set can be achieved. All the fuzzy rules are involved and can be encoded in the neural network. When a set of input data is provided to the neural network, these encoded rules are activated to different degree at the same time. The output membership function can be generated by the neural network computation, and it is related to all the fuzzy rules. To obtain the training samples, the center of the area defuzzification method is used. The center of the area defuzzification equation for discrete system can be represented by [4,5]

∑ f μ (f ) n

j

F=

j =1 n

0

j

∑ μ0 ( f j )

.

(9)

j =1

where n is the number of quantization levels, and μ 0 ( f j ) is the membership at the quantized level j. using the above equation and the fuzzy if-then rules in Table 1 with the help of Figure 3, the 54 training samples can be obtained.

5 Case Study Take the reliability evaluation of YC6100 as an example to illustrate the established BP-neural-network model. There are 10 engines waiting for evaluating their reliability. The reliability index and its evaluation value for these engines were listed in Table 2.

A Using Reliability Evaluation Model for Diesel Engine Based on Fuzzy Neural Network 151 Table 2. The input and output of the training sample I/O Index sample 1# engine 2# engine 3# engine 4# engine 5# engine 6# engine 7# engine 8# engine 9# engine 10# engine

Use availability VH 0.92 0 0.88 0 0.95 0 0.86 0.78 0.75 0.65

H 0.08 0.89 0.12 0.93 0.05 0.96 0.14 0.22 0.25 0.35

M 0 0.11 0 0.07 0 0.04 0 0 0 0

Input Mean maintenance cost ratio of each year VL L M 0.85 0.15 0 0.67 0.33 0 0 0.95 0.03 0.83 0.17 0 0 0.92 0.08 0.74 0.26 0 0 0.76 0.24 0.68 0.32 0 0.65 0.35 0 0.62 0.38 0

Output Comprehensive evaluation value VH 0.85 0 0.87 0.82 0.79 0 0.75 0 0.65 0

H 0.15 0.87 0.13 0.18 0.21 0.88 0.25 0.82 0.35 0.86

M 0 0.13 0 0 0 0.12 0 0.18 0 0.14

Value

Rank

0.9321 0.8564 0.9023 0.8765 0.9125 0.8128 0.8956 0.7986 0.7887 0.7469

1 6 3 5 2 7 4 8 9 10

Based on the analysis in Section 2 and the established neural network model in Section 3 and 4, and the network precision of training parameter is 10-6 and the study rate is 0.001. The ranking order are expressed as: R1 > R5 > R3 > R7 > R4 > R2 > R6 > R8 > R9 > R10. The output result shows that the 1# engine has higher using reliability.

6 Conclusions In order to decrease the influence of the fuzzy and uncertain factors during evaluation process, the fuzzy logic and neural network were combined to establish the evaluation model for diesel engine reliability. The evaluation result of engine using reliability can be obtained by training the network. Applications show that The result can be used as references for the improvement of reliability and maintainability of engines, and for the establishment of maintenance strategy, and it also lay a theoretical foundation for studying the improvement of evaluation method for the engine reliability as well.

Acknowledgments This research was partially supported by the Jiangxi Provincial Natural Science Foundation under the contract number 2007GQC0654 and the Science Foundation of Education Commission of Jiangxi Province under the contract number GJJ09245.

References 1. Gu, Y.K., Yang, Z.Y.: TS-neural-network-based maintenance decision model for diesel engine. In: Liu, D., Fei, S., Hou, Z.-G., Zhang, H., Sun, C. (eds.) ISNN 2007. LNCS, vol. 4491, pp. 557–565. Springer, Heidelberg (2007)

152

Y.-K. Gu and K.-Q. Huang

2. Su, J., Ao, C.L., Jiao, Y.: Evaluation of tractor reliability based on Fuzzy Neural Network. Journal of Northeast Agricultural University 39, 99–103 (2008) 3. Hsu, C.H., Jiang, B.C., Lee, E.S.: Fuzzy Neural Network Modeling for Product Development. Mathematical and Computer Modelling 29, 71–81 (1999) 4. Gu, Y.K., He, X.W.: Neural-network-driven fuzzy optimum selection for mechanism schemes. In: Liu, D., Fei, S., Hou, Z., Zhang, H., Sun, C. (eds.) ISNN 2007. LNCS, vol. 4492, pp. 275–283. Springer, Heidelberg (2007) 5. Sun, J., Kalenchuk, D.K., Xue, D., Gu, P.: Design candidate identification using neural network-based fuzzy reasoning. Robotics and Computer Integrated Manufacturing 16, 383–396 (2000)

Fuzzy Sliding Mode Control with Perturbation Estimation for a Piezoactuated Micromanipulator Qingsong Xu and Yangmin Li Department of Electromechanical Engineering, Faculty of Science and Technology, University of Macau, Av. Padre Tom´ as Pereira, Taipa, Macao SAR, China {qsxu, ymli}@umac.mo http://www.sftw.umac.mo/~ yangmin/

Abstract. This paper is concentrated on control system design for an XY parallel micromanipulator with piezoelectric actuation. The decoupled property of the manipulator enables the employment of single-inputsingle-output controllers for the two working axes. To compensate for nonlinear hysteresis effect stemming from piezoelectric actuator, the dynamics with Bouc-Wen hysteresis model is derived and identified for the system. Afterwards, a real-time fuzzy sliding mode control with perturbation estimation (FSMCPE) along with fuzzy switching control is proposed to attenuate the chattering phenomenon. Experimental studies reveal that the designed FSMCPE is superior to conventional SMCPE in terms of both positioning and tracking performances of the micromanipulator system, which validates the effectiveness of the presented control system design as well. Keywords: Micropositioning, parallel manipulator, fuzzy sliding mode control.

1

Introduction

Recently, XY parallel micromanipulators have been paid extensively attentions [1,2] due to their promising applications in micro-/nano-manipulation fields such as scanning probe microscopy, biological cell manipulation, and so on. Nevertheless, majority of existing manipulators have either an uncomplicated structure along with a coupled motion or a decoupled motion which is at the expense of a complicated structure. To benefit both controller design and prototype fabrication in practice, a parallel micromanipulator with both a decoupled motion and an uncomplicated structure is desirable. Moreover, it is suggested [3] that the micromanipulator is preferred to be designed with both input and output decoupling properties in virtue of actuation isolation and decoupled output motion. This totally decoupling property is necessary for some situations where the platform is under-actuated or sensory feedback of end-effector positions is not allowed [4]. One of such XY parallel micromanipulators has been recently designed Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 153–160. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com 

154

Q. Xu and Y. Li

by the authors. The objective of the current research is to design an appropriate controller to enable a sub-micron accuracy for the micromanipulator. In view of the advantages of high stiffness as well as high output force, piezoelectric actuator (PZT) is widely used for the drives of micromanipulators. However, a piezoactuated manipulator is a highly nonlinear system due to the hysteresis and creep behaviors introduced by the PZT. In the literature, various types of approaches have been exploited to alleviate the hysteresis effect. Particularly, sliding mode control (SMC) has been employed in the piezoactuated manipulators since the SMC is an effective and simple way to deal with model imperfection and uncertainties for nonlinear systems. As a nonlinear control strategy, SMC drives the system’s state trajectory onto a specified sliding surface and maintains the trajectory on this surface for the subsequent time. However, in conventional SMC design, a priori knowledge of the bounds on system uncertainties has to be acquired. As a consequence, the controller based on the above knowledge tends to be overconservative, which may result in poor tracking performance and undesirable oscillations in control signal. To overcome such drawback, SMC with perturbation estimation (SMCPE) has been proposed [5] as an enhanced version of the conventional SMC. It is shown that the need to know the uncertainty bounds is removed in the SMCPE and is replaced with a strategy of online estimation of perturbations. Nevertheless, due to the discontinuity of the signum function, high-frequency chattering phenomenon may occur in the control input, which is a severe problem especially when the state trajectory is close to the sliding surface. To attenuate the chattering phenomenon, the boundary layer technique is usually adopted by replacing the signum function with saturation function in the switching control term. The chattering can be avoided with the saturation function. As the cost, sliding mode performances will be compromised at the same time [6]. Additionally, in selecting the boundary layer, a tradeoff between the chattering and tracking error is always required. In this research, an alternative approach employing a fuzzy SMCPE controller with fuzzy switching control is proposed to reduce the chattering effect in the conventional SMCPE. The efficiency of the presented fuzzy SMCPE controller will be demonstrated by experimental studies performed on an XY micromanipulator prototype in the following sections.

2

System Description

The experimental setup of the XY parallel micromanipulator is shown in Fig. 1, where the XY stage is fabricated from the alloy Al 7075. In particular, the stage has a compact size owing to a two-layered mechanism design. It is actuated with two 20 μm-stroke PZT (model PAS020 produced by Thorlabs, Inc.) through displacement amplifiers. A PCI-based D/A board (PCI-6703 from National Instruments Corp.) is adopted to produce an analog voltage, which is then amplified by a two-axis piezo-amplifier (BPC002 from the Thorlabs) to provide a voltage of 0–75 V for the drives of the PZTs. The output displacements of the mobile platform are measured by two laser displacement sensors (Microtrak II,

FSMCPE for a Piezoactuated Micromanipulator

Plant

Real-Time Controller

Prototype

PZT

D/A (PCI-6703)

Piezo amplifier

Laser sensor

PC (Malab/Simulink, Real-Time Windows Target Toolbox)

155

A/D (PCI-6034E)

Sensor controller

Fig. 1. Schematic diagram of apparatus connection

head model: LTC-025-02, from MTI Instruments, Inc.), whose analog voltage outputs are read simultaneously by the PC through a PCI-based data acquisition board (PCI-6034E from the National Instruments). It can be calculated that the resolution of the position detection system is 0.038 μm. Besides, the software of Matlab/Simulink with Real-Time Windows Target toolbox is employed to implement the real-time control of the micromanipulator system. Open-loop tests show that a workspace range of 138×128 μm2 can be attained by the XY micromanipulator, and the cross-talk between the two working axes is less than 5%. Thus, the two-axis motions are well-decoupled, which allows the employment of two SISO controllers for x- and y-axes, respectively.

3

Dynamic Modeling and Identification

It is known that a piezoactuated system is nonlinear mainly attributed to the complicated hysteresis effects. Due to the decoupled property of the XY stage, the x- and y-axes can be treated independently. For brevity, only the x-axis motion is described in this paper. The dynamic model of the entire micropositioning system with Bouc-Wen hysteresis model can be established as follows: m¨ x + bx˙ + kx = k(du − h) + p h˙ = αdu˙ − β|u|h ˙ − γ u|h| ˙

(1) (2)

where the parameters m, b, k, and x represent the mass, damping coefficient, stiffness, and x-axis displacement of the XY stage, respectively; d is the piezoelectric coefficient, u denotes the input voltage, and h indicates the hysteretic loop in terms of displacement whose magnitude and shape are determined by parameters α, β, and γ. Besides, p denotes the overall perturbation of the system arising from model parameter uncertainties, unmodeled dynamics, and other unknown terms. By neglecting the perturbation, the seven parameters (m, b, k, d, α, β, and γ) of the system model with Bouc-Wen hysteresis can be identified simultaneously by an optimization algorithm. Specifically, with a 0.2-Hz multi-amplitude sine-wave voltage applied to the PZT, the primary output motion of the stage is recorded and collected for the purpose of model identification. By employing

156

Q. Xu and Y. Li

the particle swarm optimization (PSO) [7], the following model parameters are identified for the x-axis motion: m = 0.1040 kg, b = 1.8625×104 N s/m, k = 1.9920×105 N/m, d = 1.8347×10−5 m/V, α = 0.0786, β = 0.0995, and γ = 0.0008. Because the Bouc-Wen model cannot represent nonsymmetric hysteresis exactly, there exist a maximal error of 7.0% between the identified model and experimental result. In order to compensate for the model errors and other uncertainties, a SMCPE is designed below.

4

Conventional SMCPE Design

Based on the perturbation estimation strategy [5], the system model (1) becomes: m¨ x(t) + bx(t) ˙ + kx(t) = k[du(t) − h(t)] + pest (t) + p(t)

(3)

with the estimated perturbation: x(t) + bx(t) ˙ + kx(t) − k[du(t − T ) − h(t − T )]. pest (t) = m¨

(4)

where p(t) = p(t) − pest (t) represents an error between the system’s real perturbation and its estimation. To design a SMCPE controller, an error coordinate is defined as: e(t) = x(t)− xd (t), where xd represents the desired position trajectory, and t denotes the time variable. In view of the linear second-order dynamic system, a first-order sliding surface is defined as s = e˙ + λe, where λ (λ > 0) is a design parameter. It can be proved that, the tracking error e satisfies lim e(t) = 0 and the sliding variable t→∞ meets the sliding condition ss˙ < 0 if the control law is given by: u = ueq + usw =

1 pest m m b ( − λ)x˙ + (x + h) − + (¨ xd + λx˙ d ) − η sgn(s) (5) kd m d kd kd

p| ε where η > | kd + kd , the last term in (5) represents the switching or corrective control usw and the rest is the equivalent control term ueq . Besides, sgn(·) represents the signum function, η is a positive switching gain, and ε is an arbitrary positive constant. Additionally, inserting (4) into (5), gives

1 m u(t) = u(t − T ) + [h(t) − h(t − T )] − [¨ e(t) + λe(t)] ˙ − η sgn(s(t)). d kd

(6)

Furthermore, due to the discontinuity of the signum function sgn(s), chattering may occur in the control input. To alleviate the chattering phenomenon, the boundary layer technique is usually adopted by replacing the signum function in switching control term usw in (6) with the saturation function:  sgn(s) for |s| > δ sat(s) = (7) s/δ for |s| ≤ δ where the positive constant δ represents the boundary layer thickness, which ensures that s is always bounded by δ. The chattering can be avoided with the

FSMCPE for a Piezoactuated Micromanipulator

157

saturation function. Nevertheless, sliding mode performances will be compromised as the cost. Besides, in the selection of parameter δ, a tradeoff between the chattering and tracking error is required. In the next section, an alternative approach employing a fuzzy SMCPE controller with fuzzy switching control is proposed to reduce the chattering effect.

5

Fuzzy SMCPE Design

In this research, the sliding variable s which measures the closeness to the sliding surface is employed as the input variable to design a fuzzy switching controller, which will replace the switching control term usw in the control signal and regulate this control term on-line to improve the control performance. A fuzzy controller consists of three modules as described below. Fuzzification: The input variable s is imported into the fuzzy controller and classified by the input membership functions in the fuzzification module. Specifically, the input space is classified into five fuzzy sets, which are specified by the selected triangular membership functions. A membership function associated with a fuzzy set maps an input value to its appropriate membership value. The degree of an input variable belonging to a fuzzy set is denoted by a membership value between 0 and 1. Particularly, the input fuzzy sets are: F (s) = {Fs1 , Fs2 , · · · , Fs5 } = {N B, N M, ZO, P M, P B}

(8)

where the labels of fuzzy sets N B, N M , ZO, P M , and P B represent negative big, negative medium, zero, positive medium, and positive big, respectively. Fuzzy inference engine and fuzzy rules: The input information is converted into the output by the fuzzy inference engine module through the fuzzy rules. In this research, the Mamdani’s fuzzy inference method is adopted. The output space is classified into five fuzzy sets: F (u) = {Fu1 , Fu2 , · · · , Fu5 } = {N B, N M, ZO, P M, P B}

(9)

where the meanings of fuzzy sets labels N B, N M , ZO, P M , and P B are the same as the input fuzzy sets. Besides, the triangular-type membership functions are chosen for the output sets. Selecting appropriate fuzzy rules is the major task in fuzzy controller design. According to the sliding surface, switching control effort is determined to satisfy the sliding condition ss˙ < 0, so that the system state is forced to move toward the origin along the sliding surface. From empirical knowledge of sliding mode controller design, a large switching control term will drive the state trajectory to approach the sliding surface s = 0 quickly. However, it tends to excite large chattering at the same time. Thus, when the sliding variable s is negative or positive large, the switching control term usw should be increased accordingly, and vice versa. Hence, the rules which determines the switching control output (6−i) can be established as Ri : If s is Fsi then usw is Fu , for i = 1, 2, · · · , 5.

Degree of membership

Q. Xu and Y. Li

NB

1

NM ZO PM

0.4

PB

0.3 (a)

0.5

0.2 0.1

0 −2

−1.5

NB

1

0.5

−1

−0.5 NM

0 s

0.5

1

ZO

PM

PB

1.5

2

usw

Degree of membership

158

0

(c)

−0.1 −0.2

(b)

−0.3 0 −0.6

−0.4

−0.2

0 usw

0.2

0.4

0.6

−0.4 −2

−1.5

−1

−0.5

0 s

0.5

1

1.5

2

Fig. 2. Input (a) and output (b) membership functions, and the control signal (c) of the fuzzy controller

The fuzzy values of the rule parameters N B, N M , ZO, P M , and P B are tuned by the trial and error procedure through experiments, and the input and output membership functions are depicted in Figs. 2(a) and 2(b), respectively. The fuzzy inference engine performs a mapping from the input fuzzy sets to output fuzzy sets based on the established IF-THEN fuzzy logic rules. Defuzzification module: The input for the defuzzification process is the output fuzzy sets obtained above, and the output is a single crisp number which represents the output of the fuzzy controller. Using the centroid defuzzification method, Mamdani (minimum function) implication in the rule base, and maximum aggregation method (μu ), the output of the fuzzy controller can be derived as: N i=1 usw μu uf s =  (10) N i=1 μu where N is the total number of fuzzy rules. The control signal of the fuzzy controller with respect to the input variable is plotted in Fig. 2(c), which exhibits a shape similar to the saturation function. By replacing the switching control in (6) with the fuzzy controller output, a fuzzy SMCPE control input can be expressed as: 1 m u(t) = u(t − T ) + [h(t) − h(t − T )] − [¨ e(t) + λe(t)] ˙ + uf s (s(t)). d kd

6

(11)

Experimental Studies

Based on the fuzzy SMCPE controller designed above, several experiments are conducted to demonstrate the performance of the micromanipulator. The implemented closed-loop sampling time is 0.003 seconds for the real-time control. Concerning the fuzzy SMCPE (FSMCPE) controller, the sliding surface is defined with λ = 2 × 104 . For comparisons, a conventional SMCPE with saturation-function switching control is also implemented with the parameters

FSMCPE for a Piezoactuated Micromanipulator

14 SMCPE

25

FSMCPE

12

Desired SMCPE FSMCPE

20

Position x (μm)

Position x (μm)

10 8 6 10.5 11

4

10

10 2

9

9.5 4

8 0

7 0

−2 0

0.5 1

4.5

4

15 10 5 0

5

1 2 3 Time (sec.)

−5 0

5

1

(a)

5

4

5

FSMCPE 1.5

Control effort (V)

Position error x (μm)

4

2 SMCPE FSMCPE

0.5 0 −0.5

1 0.5 0

−1 −1.5 0

2 3 Time (sec.)

(b)

1.5 1

159

1

2 3 Time (sec.)

(c)

4

5

−0.5 0

1

2 3 Time (sec.)

(d)

Fig. 3. (a) Step responses obtained with SMCPE and fuzzy SMCPE (FSMCPE); 0.4Hz sinusoidal motion tracking results (b), tracking errors (c), and FSMCPE control effort (d)

of λ = 2 × 104 , δ = 0.50, and η = 0.32, which gives a switching control signal shape like the one produced by the fuzzy controller as shown in Fig. 2(c). First, a 10-μm step response is experimentally tested. The control results with the two types of controllers are depicted in Fig. 3(a). It is observed that the conventional SMCPE produces a response with a longer settling time. In contrast, the fuzzy SMCPE controller gives a rapid response, which is at the expense of a larger overshoot value. In addition, at the steady-state, both controllers can maintain the positioning error within ±0.5 μm. Second, the sinusoidal motion tracking performance of the micromanipulator is verified. Given a 0.4-Hz sine-wave signal with peak-to-peak (p-p) amplitude of 20 μm, the tracking results of the micromanipulator are shown in Figs. 3(b)–(c). It is seen that the SMCPE can only suppress the p-p error within ±1.3 μm. On the contrary, the FSMCPE achieve a p-p error of ±0.8 μm, which means that the tracking error has been reduced by 38% with respect to the conventional SMCPE control. The low-level tracking error also implies that the hysteresis has been substantially compensated by the FSMCPE approach. In addition, the control effort of FSMCPE as plotted in Fig. 3(d) indicates that the chattering phenomena have been eliminated. It is noticeable that above performances for the parallel micromanipulator are obtained within the input rate of 0.4 Hz. The limitation mainly lies in the relatively low sampling frequency due to the hardware restrictions. Additional

160

Q. Xu and Y. Li

experiments show that the sub-micron accuracy of the stage is degraded as the increasing of input rate. Once a higher bandwidth is achieved, a high-rate tracking control can be implemented even with the same control method proposed in this paper.

7

Conclusions

The studies presented in this research shows that the fuzzy sliding mode control with perturbation estimation technique can improve the tracking performance of a piezo-driven XY parallel micromanipulator. Based on a relatively simple Bouc-Wen hysteresis model, the dynamic model of the system is established and identified by PSO approach. The fuzzy controller is employed to replace the conventional switching control in SMCPE so as to overcome the limitation of boundary layer technique. Experimental results show that the hysteresis effect has been significantly reduced and chattering phenomenon has been suppressed by the proposed fuzzy SMCPE substantially, which leads to a sub-micron accuracy motion tracking of the micromanipulator. In the future works, a highbandwidth control will be carried out to improve the positioning performance of the system by increasing input rate. Acknowledgments. The authors appreciate the fund support from the research committee of University of Macau under grant no.: UL016/08-Y2/EME/ LYM01/FST and Macao Science and Technology Development Fund under grant no.: 016/2008/A1.

References 1. Pham, H.H., Chen, I.M.: Evaluation of resolution of flexure parallel mechanisms for ultraprecision manipulation. Rev. Sci. Instrum. 75(9), 3016–3024 (2004) 2. Li, Y., Xu, Q.: A novel design and analysis of a 2-DOF compliant parallel micromanipulator for nanomanipulation. IEEE Trans. Automat. Sci. Eng. 3(3), 248–254 (2006) 3. Li, Y., Xu, Q.: Design and analysis of a totally decoupled flexure-based XY parallel micromanipulator. IEEE Trans. Robot. 25(3), 645–657 (2009) 4. Awtar, S., Slocum, A.H.: Constraint-based design of parallel kinematic XY flexure mechanisms. ASME J. Mech. Des. 129(8), 816–830 (2007) 5. Elmali, H., Olgac, N.: Sliding mode control with perturbation estimation: A new approach. Int. J. Contr. 56, 923–941 (1992) 6. Kim, S.W., Lee, J.J.: Design of a fuzzy controller with fuzzy sliding surface. Fuzzy Sets and Systems 71(3), 359–367 (1995) 7. Xu, Q., Li, Y.: Error analysis and optimal design of a class of translational parallel kinematic machine using particle swarm optimization. Robotica 27(1), 67–78 (2009)

A Credit Risk Rating Model Based on Fuzzy Neural Network Ke-Jun Zhu1, Pin-Chang Chen1,2, and Yu-Teng Chang2 1

China University of Geosciences, Wuhan Hubei 430074, China 2 Yu Da University, Miaoli 36143, Taiwan [email protected], {chenpc,cyt}@ydu.edu.tw

Abstract. This study integrates the characteristics of credit risk rating and artificial intelligence technology into a credit risk rating model based on fuzzy neural network. The combination of fuzzy theory and neural network provides a good foundation for credit risk rating, making this model with fewer parameters, faster learning and less training samples. This study confirms that fuzzy neural network is an effective method for credit risk rating. The results of this study can solve the shortcomings in existing credit risk rating model and provide more information for decision-making reference. Keywords: fuzzy theory, neural network, credit risk rating.

1 Introduction Credit risk has significant influence on the financial market. It can directly affect various activities in modern economy [1]. When rating the credit risk of companies or individuals, an accurate and efficient prediction model is important to financial institutions, so as to determine an effective loan solution to reduce the rate of non-performing loans [2]. This study examines the shortcomings and deficiencies in models and methods of current credit risk rating models, and combines the characteristics of credit risk rating with artificial intelligence technology to develop a credit risk rating model based on fuzzy neural network. This study is intended to construct a fuzzy neural network model through input and output data sets, and optimize the parameters in membership function through a separate back-propagation algorithm. According to the combination of fuzzy theory and neural network, the function is not based on intuition or experience, but on any given values, which is especially important for this proposed model with complex characteristics [3]. In addition, the membership function and fuzzy rules are acquired automatically, so that the application object of fuzzy inference is not limited to the models with pre-determined characteristics of some model variables [4]. The traditional neural network has no specific weight of physical meaning, but given the physical meaning of inference parameters in fuzzy theory, which increases the transparency of the black box Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 161–171. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

162

K.-J. Zhu, P.-C. Chen, and Y.-T. Chang

and expresses neural network’s inference function appropriately [5]. The fuzzy neural inference has a problem that there are too many fuzzy rules, and it is inferior to back-propagation neural network in multi-dimensional images [6]. Fuzzy neural network can effectively utilize the advantages of both fuzzy theory and neural network, and make up for their shortages. The use of fuzzy neural network for credit risk rating not only can improve the learning and expression abilities of the rating index system, but also improve the accuracy of rating at the same time, achieving the purpose of more efficient credit risk rating [7].

2 Literature Review Fuzzy theory and neural network are two distinct areas of research, but they have the same purpose because they both are used to study artificial intelligence. They are equivalent in theory, but in practice, they have their own advantages and disadvantages. Fuzzy theory focuses on human brain’s macro functions, which can be expressed with language and concept. According to the membership function and serial fuzzy inference rules defined by human, it deals with all kinds of fuzzy information with the logical inference [8]. Neural network focuses mainly on the microstructure of the human brain, and studies the generation and formation of human’s wisdom from physical structure. Through the nonlinear dynamics on neural network with self-learning, self-organizing and self-adaptive functions, it processes the information which cannot be verbalized [9]. Fuzzy theory and neural network are two new methods for information processing, they are self-adaptive estimators that are not related to the model, with a certain degree of expertise in dealing with uncertain information and the information contained noise [10]. Fuzzy theory and neural network have their own advantages and disadvantages. The advantages of neural network are its self-learning and self-adaptive abilities as well as its parallel processing capacity and good fault tolerance. However, in the expression of knowledge and interpretation of rules learned, there are obvious deficiencies and its learning speed is usually unsatisfactory. The advantages of fuzzy theory include it can expresses knowledge naturally and easily, deal with uncertain information, express knowledge with fuzzy and natural language, and easy to use expertise. Fuzzy theory can realize fuzzy inference of knowledge with simple computing. However, fuzzy theory has poor learning ability, and it is very weak in knowledge acquisition [11]. Fuzzy theory and neural network are correlated and complementary in many aspects. It is theoretically proven that fuzzy theory can approach a nonlinear function with any precision, while neural network has the imaging capacity. It means that both two have close connection. If fuzzy theory and neural network are combined, which supplements each other, field of information processing will rise to a new realm. In such a combined fuzzy neural network, neural network simulates brain’s topology, that is, the “hardware”; and fuzzy theory simulates the thinking capacity of fuzzy information processing, namely “software” [12]. As a result, fuzzy neural network absorbs the essence of the biological cybernetics, it is distributed and parallel in storage and processing of information. In this way, all kinds of neurons support and supplement information mutually based on information sharing. It can collect inputs from nearby neurons connected to the input, calculate the

A Credit Risk Rating Model Based on Fuzzy Neural Network

163

output and then pass to higher or the other neurons, thus accomplish stronger fault tolerance and association ability. The overall performance is not affected severely when partial neurons are damaged. The output is not distorted seriously if input signal is subject to a certain degree of noise pollution. It proves that fuzzy neural network has a unique ability and can be widely used in many research fields [13].

3 Methodology Due to the complexity and high nonlinearity of credit risk, there is a wide deviation between traditional linear model and actual system. This study is intended to use non-linear and generalization capacity according to fuzzy theory and neural network, establish a credit risk rating model based on fuzzy neural network, which can achieve risk assessment and prediction. The model is based on the financial risk model and company scale rating, coupled with the consideration on companies’ fundamental factors, so that the rating factors are comprehensive, and the results are more scientific and accurate. When constructing the financial risk model, due to neural network’s self-learning, self-adaptive capacity, parallel processing capability and good fault tolerance, it can obtain more accurate non-linear outputs from various financial indexes. Fuzzy theory expresses knowledge naturally and easily, which is able to process uncertain information. It can express knowledge with fuzzy and natural language, as well as use expertise to assess the company scale. Therefore, both fuzzy theory and neural network are used in credit risk rating to obtain fewer parameters, faster learning and less training samples. The research methodology of this study is described as follows: Step 1: Collect examples of industrial enterprises, substitute companies’ credit-related data into the method proposed by this study to confirm the effectiveness. Step 2: Build a financial risk model with back-propagation neural network. Compared with traditional methods of statistical analysis, neural network is more accurate and with lower predictive risk and smaller error. SPSS statistical software is used to carry out factor analysis on company balance sheet and income statement. Step 3: Build company scale rating with fuzzy theory. Based on the concept of corporate risk rating model, the company scale can be divided into four types, which are small, medium, large, and extra large. Such vague language is particularly suited to be realized with the concept of fuzzy theory. Based on expertise, parameters of the model are adjusted continuously according to the actual output, so as to determine the scope of the scale rating. Step 4: Carry out credit risk rating with fuzzy neural network. Based on the financial risk rating from Step 2 and scale rating from Step 3, the fundamental factors are added in the final credit risk rating along with the description of the index system of corporate risk rating. Such input is companies’ total score rated by the experts from quality, strength, environment, credit status, critical incidents, and leader’s quality, and then it is naturalized into a value among [0,1]. A corporate credit risk rating model is constructed based on the above-mentioned rules and fuzzy neural network. Step 5: Analyze the data and propose a conclusion.

164

K.-J. Zhu, P.-C. Chen, and Y.-T. Chang

4 Data Analysis 4.1 Acquisition and Preprocessing of Sample Data Since business environments and scopes in different industries vary greatly, their financial and non-financial indexes are not comparable. Thus, the same industry with short-term loan as sample data are selected by this study to avoid the problems mentioned above. The data of this study are collected from the credit department of a bank. A total of 250 samples are collected. The sample data has to be preprocessed first. In the actual application, the sample cannot be increased unlimitedly, so the selected samples must be representative. Since the data obtained by this study has larger capacity, there are a wider range of indicators. The double and triple standard deviation tests are conducted to remove data anomalies, and 250 sample data are obtained. Among those, 200 are selected as training sample data and 50 are test sample data. As this rating model is applied as a corporate risk rating for commercial banks through a more comprehensive system of indexes, it is obvious that application of fuzzy neural network is a relatively new attempt. However, too much input information in the fuzzy neural network will lead to explosive growth of training time. This study focuses on the financial risks and scale, and then applies fuzzy neural network to obtain the final results of credit risk rating. 4.2 Construct the Financial Risk Model Based on Back-Propagation Neural Network Compared with the traditional methods of statistical analysis, artificial neural network is more accurate with lower predictive risk and smaller error. SPSS statistical software is used to perform factor analysis on company’s balance sheet and income statement. The main purpose is to determine the smallest number of factors which can interpret the relationship between observed variables. According to the characteristics of eigenvalue criteria, under the current conditions of sample data, the number of factors is 5. Also, from the factor loading matrix, it is clear that the economic implications of each factor are obvious; the specific results are the sample data in this study. In a three-layer back-propagation neural network model, the first layer is the input layer and the number of neurons is 5. The input layer includes the data input of profitability, growth capacity, operating capacity, short-term solvency and long-term solvency, respectively. The second layer is the hidden layer. The third layer is the output layer and the number of neuron is 1. After completing the model training, it can be used to solve practical financial risk rating. The MATLAB software is applied for back-propagation neural network, the transfer function of hidden layer is tansig, the transfer function of neuron in output layer is logsig, this is because the elements in target vector are located between [0, 1], meeting the output requirements of logsig. Back-propagation neural network structure is as shown in Fig. 1.

A Credit Risk Rating Model Based on Fuzzy Neural Network

165

Fig. 1. Back-Propagation Neural Network Structure

4.3 Construct the Company Scale Rating Based on Fuzzy Inference System According to expertise, company scale has a direct impact on business credit. It is determined by the company's total income and total assets. In this study, the Mamdani fuzzy theory with fuzzy generator and eliminator is applied to build the fuzzy inference model. According to the concept of corporate risk rating model, the company scale can be divided into four types, which are small, medium, large, and extra large. Such vague language is particularly suited to realize with the fuzzy inference system. Based on expertise, the parameters of the scale rating model are adjusted continuously according to the actual output, so the scope of scale rating is determined: 0 small 0.27; 0.27<medium 0.587; 0.587 0

(1)

B. The premise for participating bids:

Per (T aution Tq ) = Cq (T aution ∪ Tq ) − Cq (Tq ) > 0

(2)

Per represents profit, C represents cost function and T represents bidding task. Because every participant is rational, in the bidding process, Bi is Among them,

inclined to offer lower bidding price (in the research,

Bi is inclined to offer l higher

bidding price). Meanwhile, because the bidders could take measures like concealment, alliance, the profit of the party who invites the bids will be affected. In fact, for the bidding prices by the bidder b = (b1 ,..., bn ) , A hopes to secure the real estimation of

Ri

Bi . Thus it could reflect the real estimation of bidding task T by each bidder and it

goes without saying the meaning. Based on “Revelation Principle”, we could give a design principle of auction mechanism. Under such principle, the bidding price by each bidder is certainly the real estimation of T . Suppose that the real estimation by the set B of n bidders is

Ri = (i = 1,2,..., n) , and satisfy the standard distribution on [0,1] . We design following auction mechanism: The bidder Bi declares that its bidding price is

bi , but the actual dealing price is b b pri = i . The probability of successful bidding for the bidder Bi is qi = n i , θ ∑ bk k =1

n

∑q

i

= 1 . Assume the bidding price bi of Bi is linear homogeneous, that

i =1

is

bi = ki Ri . Then the expected profit for the bidding price bi by Bi is E=

bi n

∑b k =1

× (R −

bi

)=

θ

k

ki Ri n

∑b

× (V −

ki Ri

θ

)

(3)

k

k =1

For a rational bidder, he hopes that his expected profit is maximum, that is

max [ ki

Ri2 n

∑b k =1

Its first order condition is obtained through

×

kiθ − ki2

θ

]

k

ki derivation to above formula. That is

(4)

206

C.-Y. Lo et al.

ki =

θ

(5)

2

= 1 , bi = Ri . Therefore, we could learn that when θ = 2 , all bidders will give their real estimation of T and not care how So called “telling the truth” by the bidder is ki

others tender their bids. Based above-mentioned formulas, we designed a new auction patter in which the bidder tender their bids individually and their probability of success bidding is a strict increase function and the dealing price is half of the bidding price. So it could prove that under such auction mechanism, a rational bidder will “tell the truth”. Step II: Using the results of value allocation model of virtual organization researched by AHP to install the module of value distribution agent system. In the process of realization of profit distribution agent, based on the principle of profit distribution of supply chain of contribution and risks-sharing, it is possible to set a value that could be accepted by related parties through the consultations between cooperative enterprises that participate in the distribution. Therefore, in the research, we firstly assume that there are n cooperative enterprises participating in the profit distribution of the virtual organization. Suppose the final distributable total amount is I that could be expressed in the form of distributable part of various products, accessories and services in the market realization, that is, the distributable part in the supply chain in which tax, necessary public expenditures of supply chain have been deducted from the final realized total income of market sales. m

The total cost

Ci paid by Enterprise i is Ci = ∑ Csj Q j + C pi . Among them, m j =1

is the type of task shouldered by Enterprise unit cost of Task

i in the virtual organization. Csj is the

j , Q j is the quantity of Task j , C pi is the extra cost paid by

Enterprise i . It is the quantification of contribution by the enterprise to the virtual organization. The extra cost could be obtained from the joint procurement plan and control process of virtual organization. Then the total operation cost of virtual n

organization could be expressed as

C = ∑ Ci . Therefore, the rate of return could be i =1

computed through the formula

ε=

I −C 100% . C

Because the purpose for enterprise to participate in a virtual organization is to acquire more profit than without participation, we introduce here the social average rate of return ε for comparisons. Based on profit distribution project of contribution and risk-sharing, the first things we should consider are the cost input by the enterprise in virtual organization and corresponding social average rate of return before distributing extra return. Let the extra return be

R , then R = 1 − (1 + ε )C . Besides, due to the

difference of the type of task, technological conditions shouldered by each cooperative enterprise in the virtual organization, the risk degrees shouldered by each cooperative

An Agent-Based Approach to Joint Procurement Modeling with Virtual Organization

207

enterprise in the virtual organization are different accordingly. Suppose the risk degree of task shouldered by Enterprise i in the virtual organization is γ i , then

γi > 0

n

,

∑γ

=1

i

(6)

i =1

Likewise, the investments by each cooperative enterprise in their own infrastructure for the purpose of integral benefit of virtual organization are different. Let λi be the ratio of investment in the field by Enterprise i in the virtual organization, then

λi > 0

n

,

∑λ

i

=1

(7)

i =1

We could conduct the evaluation of risk degree

γi

that each cooperative enterprise

should shoulder by adopting some evaluation methods through supply chain risk management and sub-system control and λi could evaluate the cooperative enterprises through the performance evaluation sub-system of virtual organization and meanwhile, further to confirm based on the actual investment. The existence of uncertain factors in the operation process of virtual organization would make the rate of return positive or negative. In a certain stage of opportunity realization, the final investment used in the distributable part could be greater or smaller than the operation cost C in virtual organization in the same stage. Therefore, the research uses following three situations to consider supply chain profit distribution strategies of opportunity contribution and risk-sharing: (1) I < C , (2)

C < I < (1 + ε )C , (3) (1 + ε )C < I . The research take the third situation, the rate of return of virtual organization is significantly greater the social average rate of return, as the method. Under the situation that the rate of return of virtual organization is significantly greater the social average rate of return, in addition to recovering cost, calculating the yield of input cost, it is necessary to distribute the extra profit of virtual organization based on the return of investment risks, the size of contribution, the size of shouldered task risk and enterprise’s own infrastructure, then Vi could be expressed as

V i Ci  H Ci 

C Ci DR  n pi ER  J iJR  Oi OR C ¦ C pi

(8)

i 1

The formula thus links the risk shouldered by the enterprise in the virtual organization and investment risk in the return of the enterprise and realizes risk diversification and equal sharing. Meanwhile, it links directly the contribution to the virtual organization by each cooperative enterprise and its profit distribution, individual activity of virtual organization in different opportunity realization periods and progress and operation revenue of virtual organization. Finally, authentically, equally distribute the profit

208

C.-Y. Lo et al.

according to the different contributions made to the virtual organization in each operation period. Step III: Use the joint procurement cycle and quantity estimation model made by back-propagation neural network algorism to install the model of enterprise demand estimation agent system and the model joint procurement agent. 1. Install the Prototype of Demand Estimation System of Back-Propagation Neural Network according to Back-Propagation Neural Network Algorism. 2. Establishment of Profit-Maximized Objective function. 3.

H / Ri

H / Ri

R =1

k =1

Max{Pi × max{ ∑ Qik , Di } − C (1 + hi ) ∑ Qik − Bi H / Ri

× max{( ∑ QiK −Di ),0} − (CTi + K ) × H / Ri } K =1

Pi = unit price, C = purchase cost, hi = yearly holding cost rate, Bi = short supply cost, CTi = transportation cost, Di = total quantity of real demand during H period, Qik = The Estimated Quantity of Order during the kth

Among them,

Replenishment Period, obtained from the estimation of back-propagation neural network, H = calculation cycle, Ri = best replenishment cycle (within H cycle),

H / Ri = replenishment times, k = cost of order 4.

5.

6.

7.

Compare the maximum profits obtained from each purchase cycle and select the cycle of the biggest profit as the best cycle for the enterprise to purchase a raw material and thus obtain the most optimal purchase batches; Adopt highest common factor method and the lowest common multiple method to select the highest common factor or the lowest common multiple as common purchase cycle before substitute the common purchase cycle into the prototype of estimation system to estimate the purchase quantity of the common purchase; Substitute common purchase cycle and quantity parameter obtained from Step 4 into the profit objective function established by Step 2 and compare the profit during the period that it had not joined the joint purchase; Look for case enterprises in a proper industry and introduce their sales history into the method introduced by the research plan to conduct empirical verification and confirm the method has a certain extent of benefit for join purchase.

4 Validation and Results As a whole, the main contributions of the research are following points: 1.

Establish a joint procurement model of multi-echelon supply chain with multiple agent conception and overcome the existing information uncertainty in practical environment and the problems of information transparency degree and openness through the functions of each module;

An Agent-Based Approach to Joint Procurement Modeling with Virtual Organization

2.

3.

4.

209

The research introduces an integral evaluation mechanism that combines the quantitative factors such as delivery dates, pricing, evaluation of history of delivery dates and qualitative factors such as production quality, partnership, cooperation intentions evaluation to conduct effective analyses for policy-decision through the operation of function. It will make order assignment policy-decision support model more complete that we could give an overall consideration and draw up the best order assignment policy-decision and help a virtual organization to plan the best order production assignment and schedule. Use enterprise’s data of production and sales history to train back-propagation neural network to obtain the best weighted value and bias value of back-propagation neural network used by the enterprise in estimating purchase quantity in different cycles; use recent enterprise’s data of production and sales and the best weighted value and bias value of back-propagation neural network to confirm the accuracy and reliability of the neural network and complete the design of back-propagation neural network algorism of the best purchase cycle and quantity for the enterprise; then substitute the estimated purchase quantity of the enterprise in different cycles into the objective function of maximum profit and find out best purchase cycle and quantity. The results of the simulated analyses of various parameters and related set situations are line with dynamic changes of real environment. Therefore, the simulation of the system could help the enterprise to conduct effective bidding and draw up the best order assignment policy-decision in the competitive and cooperative multi-echelon supply chains.

5 Conclusions The application of virtual organization and agent system is not limited to the occasion of commercial negotiation that the follow-up researcher could combine other fields, such as education, society to further extend the development. On the technological aspect, the results of the research could not only provide the references for industry in practices and virtual organization in drawing up strategies but also help the industry in cultivating talents and capability of the development of demand estimation theory and the drawing-up of joint procurement strategies. Furthermore, the results could be helpful to the establishment of agent database that will benefit follow-up researchers to find out strategy plan of virtual organization and agent system.

References 1. Mowshowitz, A.: Virtual Organization. Communications of the ACM 40(9), 30–37 (1997) 2. Hale, R., Whitlaw, P.: Towards the Virtual Organization. McGraw-Hill, London (1997) 3. Venkatraman, N., Henderson, J.C.: Real Strategies for Virtual Organizing. Sloan Management Review, 33–47 (1998) 4. Saabeel, W., Verduijn, T.M., Hagdorn, L., Kumar, K.: A Model of Virtual Organization: A Structure and Process Perspective. EJOV 4(1), 1–17 (2002) 5. Lau, C., Hennry, W., Wong, T.T.: Partner Selection and Information Infrastructure of a Virtual Enterprise Network. IJCIM 14(2), 186–193 (2002)

210

C.-Y. Lo et al.

6. Cloutier, L., Frayret, J.M., D’amours, S., Espinasse, B., Montereuil, B.: A Commitmentoriented Framework for Networked Manufacturing Co-ordination. IJCIM 14(6), 522–534 (2001) 7. Vytelingum, P., Cliff, D., Jennings, N.R.: Strategic Bidding in Continuous Double Auctions. Artificial Intelligence 172(14), 1700–1729 (2008) 8. Blumrosen, L., Nisan, N.: Informational Limitations of Ascending Combinatorial Auctions. Journal of Economic Theory (2009) (in press); Corrected Proof 9. Shim, S., Lee, B.: Applying Experimental Online Auctions in Marketing Research for Multi-channel Firms. Expert Systems with Applications 37(3), 2505–2516 (2009) 10. Bu, T.M., Sun, Q., Qi, A.W.: Unconditional Competitive Auctions with Copy and Budget Constraints. Theoretical Computer Science 393(1-3), 1–13 (2008) 11. Chang, P.C., Liu, C.H., Lin, J.L., Fan, C.Y., Ng, C.S.: A Neural Network with a Case Based Dynamic Window for Stock Trading Prediction. Expert Systems with Applications 36(3), 6889–6898 (2009)

Interactive Hybrid Evolutionary Computation for MEMS Design Synthesis Ying Zhang1 and Alice M. Agogino2 1

School of Electrical and Computer Engineering, Georgia Institute of Technology, GA, USA [email protected] 2 Department of Mechanical Engineering, University of California at Berkeley, CA, USA [email protected]

Abstract. An interactive hybrid evolutionary computation (IHC) process for MEMS design synthesis is described, which uses both human expertise and local performance improvement to augment the performance of an evolutionary process. The human expertise identifies good design patterns, and local optimization fine-tunes these designs so that they reach their potential at early stages of the evolutionary process. At the same time, the feedback on local optimal designs confirms and refines the human assessment. The advantages of the IHC process are demonstrated with micromachined resonator test cases. Keywords: interactive hybrid computation, MEMS synthesis, evolutionary computation, optimal design.

1 Introduction Evolutionary computation based on a multi-objective genetic algorithm (MOGA) has been used for microelectromechanical system (MEMS) design synthesis [1, 2]. Incorporating gradient-based local optimization at the end of the evolutionary process, referred to herein as the “standard hybrid GA process”, has been shown to improve the performance of the evolutionary MEMS design synthesis process [3]. This paper explores the integration of local optimization into intermediate generations of the evolutionary process in order to allow the refinement of promising designs at an early stage of the evolutionary process and guide search towards more promising design regions earlier. This method applies local optimization either to the whole population [4] or to individuals with the best performances [5] in intermediate generations. Applying local improvement to the whole population may be computationally prohibitive on large complex problems like MEMS design synthesis. On the other hand, limiting local optimization to the best individuals based on the quantitative fitness function does not guarantee local improvement of all designs with good patterns as these patterns have not yet reached their potential. Some of these designs Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 211–218. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

212

Y. Zhang and A.M. Agogino

could evolve to much better design solutions if fine-tuned or if given enough time to evolve in the evolutionary process. Interactive evolutionary computation (IEC) is a model of using the symbiosis of human expertise with evolutionary algorithms for a user-directed search, which has been used to evaluate objectives of MEMS designs that are difficult to model [6]. During the IEC process, human experts assess all individuals subjectively and assign fitness values based on their judgment of the potential for performance or a recognition of innovative design patterns. The reliance on human judgment to guide the evolutionary process has both potential strengths and weaknesses. On one hand, human expertise can be effectively exploited for guidance of a naive evolutionary process. On the other hand, including a human in the evolutionary loop introduces the need for rapid convergence to prevent human fatigue, and human assessment tends to have a component of subjectivity and non-linearity of focus over time [7]. To solve the human fatigue problem, Kamalian, et al., suggested a method with the human in a supervisory role, providing evaluation only every nth-generation during the IEC process [6]. To reduce the influence of the subjectivity and non-consistency of human assessment on the evolutionary process, we present an interactive hybrid computation (IHC) algorithm that uses local optimization to fine-tune all selected designs by a designer so that these designs reach their potential at early stages of the synthesis process. The performances of these locally-optimized designs can also be beneficial in confirming and refining the human’s subjective assessment of these designs over time. The advantages of the IHC process are demonstrated on the synthesis of surface-micromachined resonators.

2 Interactive Hybrid Computation (IHC) Process IHC exploits human expertise in identifying potentially good design patterns, using the results of local optimization to refine fitness values of strong designs and reduce the influence of subjectivity and non-consistency of the human evaluation. During the IHC process, a human designer evaluates all the designs or a subset of them, such as those in the Pareto optimal set for multi-objective fitness functions, before applying local improvements. To reduce human fatigue, human input is only applied at certain predefined generations during the hybrid evolutionary processes. In our MEMS resonator test cases, IHC provides the designer with multiple design concepts within a graphical user interface (Fig. 1.) at predefined generations to evaluate all the designs or a subset of them in the current generation. If the designer identifies a promising design pattern, he/she can click the ‘local_Opt’ button, to enable the MEMS synthesis process to apply gradient-based local optimization to the design. After gradient-based local optimization has been performed, the geometrical parameters and performances of the optimized designs are fed back into the genotype and the phenotype domains of the GA process to replace the original designs.

Interactive Hybrid Evolutionary Computation for MEMS Design Synthesis

213

Fig. 1. Graphical User Interface during IHC Processes

3 Evaluation of the IHC Process 3.1 Experimental Design Two experiments were conducted to evaluate the IHC process, comparing the proposed IHC process with: (1) the standard hybrid GA process; and (2) a random hybrid GA process. The random hybrid GA process randomly selects similar number of designs from the Pareto optimal set of predefined intermediate generations and applies local optimization. The second experiment is to test if the benefits of IHC are from human interaction or intermediate local optimization. A series of resonator design cases are used as test cases for the experiment. The resonator design is similar to the design used in [3] and [8] for purposes of comparison. These test cases are symmetric designs without angle constraints, which provide a relatively large design search space. The multiple design objectives considered are to: (1) minimize the error from the target resonant frequency, f_target (Designs within an acceptable error range, 5%, are considered feasible designs.); (2) minimize the design area; and (3) satisfy the stiffness ratio requirement (Kx/Ky ≥ k_target). Six test cases were designed for the comparison between the IHC and the standard hybrid GA processes. Two cases were the same as the test cases used in [3] and [8], and four other cases were developed by randomly generating design objective goals with natural resonant frequency values between 1,000Hz and 10,000Hz, and stiffness ratios between 1 and 10. The objective values of these six test cases are listed in Table 1.

214

Y. Zhang and A.M. Agogino Table 1. Design Objective Goals for the Test Cases (in addition to area minimization)

Test case 1 Test case 2 Test case 3 Test case 4 Test case 5 Test case 6

Natural resonant frequency (Hz) 10,000 10,000 2,587 5,109 8,393 9,551

Stiffness ratio requirement 1 10 5 1 5 3

All test cases have an I-shaped center mass with a fixed geometry, which has one 100μm x 100μm rectangular mass plate attached with four 30μm x 50μm beams. The comb drives connected to the center mass also have fixed geometries. Each comb drive has 11 fingers, 50μm long by 4μm wide with a 3μm gap, plus a 4μm wide spine. The parameters of springs connected to the center mass are free parameters. The parameter constraints for the springs are the same as the constraints discussed in [3] and [8] except the upper bound of the beam length is extended to 300μm for the test cases with target natural resonant frequencies higher than 5,500Hz and 800μm for the test cases with target natural resonant frequencies lower than that. All hybrid computation design synthesis processes are applied to the six test cases three times. The maximum number of generation for each process is 50 and each generation has 100 populations. The designs that are in the overall Pareto set, have a frequency deviation within 5% of the target frequency and satisfy the stiffness ratio requirement are tallied after each design synthesis process. In the first experiment, designers interact every 10th generation to select designs to apply the local improvement procedure during the IHC process (IHC_10). The objective function of the gradient-based local optimization used during the IHC process is formulated as the sum of the normalized design area and frequency deviation, plus a large penalty if the design violates the stiffness ratio requirement. In the second experiment, designers interact every 5th generation of the first 25 generations to select designs for the local performance improvement during the IHC process (IHC_5), resulting in a similar human fatigue level as that during the IHC_10 process. A total of 263 designs were selected by humans during the IHC_5 processes for all six test cases with three runs for each case, which means, on average, approximately 3 designs were selected at each computer-human interaction. To be comparable and give the random GA process a slight advantage in terms of the number of locally-optimized designs, the random hybrid GA process randomly selected 5 designs from the Pareto set every 5th generation during the first 25 generations to apply the gradient-based local optimization. At the end of all of the all evolutionary synthesis processes, another gradient-based local optimization is applied to all the tallied feasible designs generated. Since all feasible designs satisfy the following requirements: the frequency deviation is within 5% of the target frequency, and the stiffness ratio (Kx/Ky) is not less than the requirement, the local optimization only needs to minimize the design area while keeping all other requirements as inequality constraints.

Interactive Hybrid Evolutionary Computation for MEMS Design Synthesis

215

3.2 Results and Discussion The statistical analysis of the area of the feasible designs on the overall Pareto sets of the first experiment is shown in Table 2. Table 2. Statistical Analysis of the Area of Pareto Optimal and Feasible Designs Resulting from the MEMS Synthesis Processes Synthesis Minimum Average Area Area (E-07 m2) (E-07 m2) Process Standard 1.7056 4.1585 Case 1 Hybrid GA 1.6665 2.0077 IHC_10 Standard 1.8426 2.1778 Case 2 Hybrid GA 1.7997 2.0045 IHC_10 Standard 4.9434 10.1455 Case 3 Hybrid GA 3.8010 5.5189 IHC_10 Standard 4.0256 17.0716 Case 4 Hybrid GA 2.7589 3.1272 IHC_10 Standard 1.9569 2.1818 Case 5 Hybrid GA 1.8093 2.0918 IHC_10 Standard 1.7444 2.1292 Case 6 Hybrid GA 1.5723 2.0302 IHC_10 Note: Statistical significant improvements shown as shaded areas.

ANOVA pvalue 0.0234

0.2539

0.0310

0.0283

0.4902

0.6321

The results show that the IHC process is significantly better than the standard hybrid GA process using the Wilcoxon Matched-Pairs Signed-Ranks [9] (p=0.025) for both the minimum design area and the average design area. The analysis of variance (ANOVA) between the IHC final feasible populations and the standard hybrid GA final feasible populations shows that the IHC process is significantly better than the standard hybrid GA process for cases 1, 3 and 4 with p ≤ 0.031. The improvement of the IHC process is not statistically significant for cases 2, 5 and 6 using ANOVA over the entire final Pareto optimal population, although the results do show improvement for both the minimum and average areas. The reason for the lack of statistically significant improvement in the Pareto optimal populations in cases 2,5 and 6 may be due to the fact that these cases have both a high target frequency goal and a high stiffness ratio requirement, reducing the feasible search space. The best designs generated from the IHC process for different test cases have either springs turning inward or high beam densities in the rectangular bounding box around the device (not including two large anchor pads connected to comb drives). With fixed center masses, both the high target frequency and the high stiffness ratio requirement evolves to consistently push the springs outward, reducing the probability of

216

Y. Zhang and A.M. Agogino

generating novel new patterns and thus minimizing the value of the human interaction in identifying promising new patterns. Table 3. Statistical Analysis of the Area of Pareto Optimal and Feasible Designs Resulting from the Random Hybrid GA process Minimum Average Sign1* Sign2 ** area area (E-07m2) (E-07m2) Random_5 1.6854 2.1700 1 + + - IHC_5 1.6573 1.7888 Random_5 1.8568 2.1353 2 - + - IHC_5 1.6718 1.8273 Random_5 5.8068 11.1288 3 - - IHC_5 3.3859 4.9742 Random_5 2.6139 11.1723 4 + + - IHC_5 2.3948 2.9165 Random_5 1.8545 2.0142 5 + + + + IHC_5 1.8758 2.2097 Random_5 1.7534 1.9053 6 - + - + IHC_5 1.5420 1.9866 * Comparison between the random and standard hybrid GA processes. ** Comparison between the random hybrid GA process and the IHC_5 process. Note: Statistical significance shown as shaded areas. Standard hybrid GA results for minimum and average areas listed in Table 2. Case

Synthesis Process

The statistical analysis of the results of the second experiment is shown in Table 3. In the ‘sign1’ and ‘sign2’ columns, the first symbol ‘+/-’ represents the comparison between the minimum areas, and the second symbol represents the comparison between the average areas. The symbol ‘+’ is used if the area resulting from the random hybrid GA process is smaller than that resulting from the other process, sign ‘-’ is used otherwise. Based on the results of the best generated designs, the IHC_5 process is statistically better than the random hybrid GA process using the Wilcoxon Matched-Pairs Signed-Ranks (p2n. When all the states of the neurons are updated, the optimal path from the start cell to the end cell can be found quickly following the steepest gradient descent of the state values. ⎧ Di × f ( X ) / A + I / A, for end neuron X i ( k + 1) = ⎨ ⎩ Di × f ( X ) / A, other neurons

f (X ) =

∑

j < i ∩ j∈NEi

wij X j ( k + 1) +

∑

j >i ∩ j∈NEi

wij X j ( k )

(1)

(2)

3 Modeling of Penetration Factors 3.1 Terrain Mask Analysis Terrain mask is usually represented by the line-of-sight maps (also called intervisibility map) of individual threats. This paper adopts a non-ray-tracing algorithm to calculate the line-of-sight map. This algorithm is very fast and only need to be executed once to attain masked area at different fly height. Details of the algorithm are described in [10]. 3.2 Terrain Following Analysis The quality of terrain following is determined by the clearance levels. The clearance levels (Δhi) means the vertical distance between the lowest height (Hi) that the aircraft can be seen and the terrain elevation (hi). The clearance level is taken as one of the main criteria that imposed on the grids of the map, as shown in (3). ⎧ if Δhi ≤ H F ⎪1 C j =⎪ ⎨ ⎪ ⎪ ⎩ H F / Δhi else

(3)

where Cj is the clearance merit of the jth grid. 3.3 Threat Modeling As mentioned in section 1, threats are classified into known, mobile and unknown types. In this paper, the danger index at any position inside a known threat envelope is calculated as a function of the aircraft position relative to the threat site, terrain mask, as well as the difference in altitude between threat site and aircraft.

238

N. Wang et al.

For mobile and unknown threats, the “area-intervisibility” is adopted to approximately measure the danger index. The area-intervisibility of a given position implies the probability that aircraft at that position would be detected (under line-of-sight) and attacked by mobile and unknown threats. Finally, the mixed danger index of a given grid on the map is calculated as weighted sum of the danger index of known threats and the area-intervisibility, as shown in (4). This is the second main criteria that imposed on the grids of the map. j Dmix = wk × ∑ Di / N tht + wu × Pint

(4)

i

where Djmix is the mixed danger index of the jth grid; Di is the danger index of threat i; Ntht is the number of corresponding threats; Pint is the area-intervisibility of the given position; wk and wu are weights and wk + wu =1. 3.4 Other Criteria and Restrictions The typical mission requirements such as time limit and fuel usage are evaluated based on the length of the total path and the given velocity of the aircraft, which can be taken as criteria and constrains accordingly. The maneuverability of the aircraft is represented by the maximum turning angle, climb and dive angle of the aircraft, which is widely used by path planning algorithms.

4 HNN Guided EA 4.1 EA Based Path Planning Algorithm Typical EA based path planning algorithms use gene sequences to model the path of the aircraft, in which each gene represents the position or movement of the aircraft. In this paper, a fixed-length gene sequence representing the movement actions of the aircraft is adopted. The movement actions are defined as Table 1. Each gene adopts integer coding and has a value of 1-3. Examples of gene sequence under this scheme are shown in Fig. 2. Table 1. Movement actions defined in the proposed EA

Action A1 A2 A3

Meaning no change left turn right turn

Value 1 2 3

The dimensionally of the planning space of the proposed algorithm is 2.5D. This means that the actual space is two dimensional, but warped to the surface of terrain using terrain following. Path planning in 2D space is much faster than 3D space, and is more compatible with the HNN scheme mentioned above.

HNN Guided Evolutionary Algorithm for Aircraft Penetration Path Planning

239

Fig. 2. Gene sequence examples

To evaluate the criteria and constrains of the candidate movement actions, the actions is mapped into a plane trajectory using (5) and (6). ⎧ ⎪ φi (tk ) = φi (tk −1 ) − θ ⎪ ⎪ ⎪ ⎨φi (tk ) = φi (tk −1 ) + θ ⎪ ⎪ ⎪ ⎪ ⎩tk = k ×Tstep

if there is a left turn action if there is a right turn action

⎧ x(tk ) = x(tk −1 ) + v ×sin(φ (tk )) ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ y (tk ) = y(tk −1 ) + v× cos(φ(tk ))

(5)

(6)

where Φ is the its heading angle and Φ=0 means flying towards north; θ is the maximum turning angle in each step; Tstep is the time step of each movement; v is the given penetration velocity; x and y are the 2D position for the aircraft at time tk. When mapped, the plane trajectory is further transferred into a 3D trajectory by calculating the profile of the path as shown in (7). To accommodate the climb and dive angle constrains mentioned in section 3.4, the profile is calibrated using the protuberant points in z. z(tk ) = HF + Z (x(tk ), y(tk ))

(7)

where z is the fly height sequence of the aircraft profile; HF is the terrain following altitude; Z(x, y) is the terrain elevation of position (x, y). The planning procedure of the proposed EA algorithm is described as follows: Algorithm 1: EA based path planning Pre-determined phase: (1) Initialize corresponding parameters and EA arguments (2) Generate a set of M gene sequences Mutation-offspring phase: (3) Set offspring counter gm=1 (4) Repeat (5) Apply the mutation operator to the M gene sequences and generate another M sequences (6) Map the 2M sequences into 2M trajectories (7) Evaluate the criteria and constrains of the 2M trajectories based on penetration factors (8) Choose the best M out of 2M trajectories (9) gm=gm+1 (10) Until the terminal iteration condition is reached or the offspring time is used up, i.e., gm>Gm (11) Select the optimal path from the trajectories as the result

240

N. Wang et al.

4.2 HNN Guided EA Algorithm HNN has the advantage of target pursuit and obstacle avoidance in path planning, which is good for guiding the direction of EA to concentrate on the paths towards target while avoiding sharp terrain and lethal threats as obstacles. The HNN guidance occurs at the mapping step of the proposed EA. Instead of searching as the ‘left direction’ and ‘right turn’ portion in (5), the HNN is used to guide the EA as (8) shows: ⎧φi (tk −1 ) − θ if no η p & left turn ⎪ ⎪ ⎪ ⎪ φ (t ) + θ if no η p & right turn ⎪ ⎪ i k −1 φi (tk ) = ⎪ ⎨ ⎪ if η p − φi (t k −1 ) ≤ θ & turn ηp ⎪ ⎪ ⎪ ⎪ φ (t ) + θ ×sign(η p − φi (tk −1 )) if η p − φi (tk −1 ) > θ & turn ⎪ ⎪ ⎩ i k −1

(8)

where ηp is the gradient direction of the states of the HNN. If the HNN guided heading angle is within the aircraft maneuverability bounds, it is selected as the movement. If not, the mapping mechanism calculates a heading angle that towards the gradient of the HNN as much as possible. Therefore, the searching area is guided around the gradient of the HNN that improves the efficiency of pure EA algorithm in path planning. 4.3 Overall Planning Algorithm The planning procedure of the overall algorithm for penetration path planning is as follows: Algorithm 2: HNN guided EA based path planning (1) Initialize mission environment and threat parameters (2) Analyze the falls of the terrain and mark the grids with sharp falls as obstacles (3) Analyze the threats on the battlefield and identify the most lethal threats to be avoided (4) Build the HNN on the 2D data grid map (5) Analyze the terrain mask and the area-intervisibility to calculate the mixed danger index Dmix of each grid on the map (6) Perform EA pre-determined phase (7) Repeat (8) Conduct mutation-offspring (9) Map the movement actions into trajectories using the HNN guidance (10) Evaluate the criteria and constrains of the trajectories (11) Perform trajectories selection (12) Until the terminal iteration condition is reached or the offspring time is used up (12) Select the optimal path from trajectories the with the best evaluation value as the result

HNN Guided Evolutionary Algorithm for Aircraft Penetration Path Planning

241

5 Simulation Results To demonstrate the performance of the HNN guided EA algorithm, several experiments are carried out under typical mission scenarios, as shown in Fig. 3. The results of pure HNN, pure EA and the HNN guided EA under scenario (a) and (b) are shown in Fig. 4 and Fig. 5 respectively. Table 2 gives a comparison of the performance of results of the proposed algorithms.

(a)

(b)

Fig. 3. Test mission scenario, where blue rings represent threat rings, and the area filled with green color represents the intervisibility area of threats at aircraft height 200m

(a) HNN

(b) EA

(c) HNN+EA

Fig. 4. Best results of the three algorithms under mission scenario (a)

(a) HNN (no solution)

(b) EA

(c) HNN+EA

Fig. 5. Best results of the three algorithms under mission scenario (b)

242

N. Wang et al. Table 2. Comparison of results of different algorithms

(a) HNN Scenario (a) 0 0 72.5 (b) EA To Minimize Scenario (a) Obstacle constrain 0 Goal arrival constrain 20.8 Grid criteria value 8.1 (c) HNN guided EA To Minimize Scenario (a) Obstacle constrain 0 Goal arrival constrain 0 Grid criteria value 7.8 To Minimize Obstacle constrain Goal arrival constrain Grid criteria value

Scenario (b) / no solution / no solution / no solution Scenario (b) 1 23.5 20.2 Scenario (b) 0 0 19.5

The convergence performance of the proposed HNN guided EA under two scenarios tests is shown in Fig. 6 below.

Fig. 6. The convergence of the HNN guided EA (500 iterations), where the upper line in blue represents the grid criteria values, and the lower in red represents the sum of constrains

6 Conclusion In this paper, the characteristics of the aircraft penetration path planning are analyzed through factors of terrain, threat and other mission requirements and restrictions. Towards the efforts of implementing robust and efficient methods to solve the penetration path planning problem, a HNN guided EA algorithm is proposed, in which the efficiency of EA searching is improved by adding HNN guidance to the mapping step of the EA. The HNN guidance serves as heuristic knowledge of target pursuit and avoiding sharp terrain and lethal threats. Experiments demonstrate that the proposed algorithm does well in solving the penetration path planning problem compared to pure HNN and pure EA, which implies that hybrid methods may be more competitive in solving complex path planning problems.

HNN Guided Evolutionary Algorithm for Aircraft Penetration Path Planning

243

References 1. Bortoff, S.A.: Path planning for UAVs. In: Proceedings of the American Control Conference, Chicago, Illinois, pp. 364–368 (1991) 2. Weiß, B., Naderhirn, M., del Re, L.: Corporative UAV Formation Flying with Obstacle/Collision Avoidance. IEEE Transactions on Control Systems Technology 15, 672–679 (2007) 3. Kitamura, Y., Tanaka, T., Kishino, F., Yachida, M.: 3-D Path Planning in a Dynamic Environment using an Octree and an Artificial Potential Field. In: IEEE International Conference on Intelligent Robots and Systems, pp. 474–481 (1995) 4. Pettersson, P.O., Doherty, P.: Probabilistic Roadmap Based Path Planning for an Autonomous Unmanned Aerial Vehicle. In: The Workshop on Connecting Planning and Theory with Practice, ICAPS (2004) 5. Rathbum, D., Capozzi, B.: Evolutionary Approaches to Path Planning under Uncertain Environments. In: 1st Unmanner Aerospace Vehicles, Systems, Technologies, and Operations Conference and Workshop, AIAA (1994) 6. Du, Z., Qu, D., Xu, F., Xu, D.: A Hybrid Approach for Mobile Robot Path Planning in Dynamic Environments. In: Proceedings of the 2007 IEEE International Conference on Robotics and Biomimetics, pp. 1058–1063 (2007) 7. Han, J.D., Campbell, M.: Artificial Potential Guided Evolutionary Path Plan for Target Pursuit and Obstacle Avoidance. In: AIAA Guidance, Navigation, and Control Conference and Exhibit (2003) 8. Wang, N., Gu, X., Chen, J., Shen, L., Ren, M.: A Hybrid Neural Network Method for UAV Attack Route Integrated Planning. In: Yu, W., He, H., Zhang, N. (eds.) ISNN 2009, Part III. LNCS, vol. 5553, pp. 226–235. Springer, Heidelberg (2009) 9. Fan, C., Lu, Y., Liu, H., Huang, S.: Path Planning for Mobile Robot Based on Neural Networks (in Chinese). Computer Engineering and Applications 8, 86–89 (2004) 10. Leavitt, C.A.: Real-Time In-Flight Planning. In: Proceedings of the IEEE 1996 National Conference on Aerospace and Electronics, vol. 1, pp. 83–89 (1996) 11. Rana, A.S., Zalzala, A.M.S.: A Neural Networks Based Collision Detection Engine for Multi-Arm Robotic Systems. In: 5th International conference on artificial neural networks, pp. 140–145 (1997) 12. Ellips, M., Davoud, S.: Classic and Heuristic Approaches in Robot Motion Planning–A Chronological Review. Proceedings of World Academy of Science, Engineering and Technology 23, 101–106 (2007) 13. Lagoudakis, M.G.: Hopfield Neural Network for Dynamic Path Planning and Obstacle Avoidance 14. Capozzi, B., Vagners, J.: Evolving (semi)-autonomous vehicles. In: AIAA Conf. Guidance, Navigation and Control (2001) 15. Yang, G., Vikram, K.: Optimal path planning for unmanned air vehicles with kinematic and tactical constraints. In: Proceedings of the 41st IEEE Conference on Decision and Control, vol. 2, pp. 1301–1306 (2002)

Fuzzy Material Procurement Planning with Value-at-Risk Gao-Ji Sun College of Mathematical and Physical Science, Lishui University, Lishui 323000, China [email protected]

Abstract. Based on credibility theory, this paper presents a class of two-stage fuzzy programming with value-at-risk (VaR) to deal with material procurement planning (MPP) problem. Since the MPP problem usually includes continuous fuzzy variable parameters with infinite supports, it is inherently an infinite-dimensional optimization problem that can rarely be solved directly. To overcome this difficultly, this paper introduces an approximation approach (AA) which can turn the infinitedimensional optimization problem into a finite-dimensional optimization one. Furthermore, in order to solve the proposed MPP model, a hybrid algorithm is designed which combines AA, neural network (NN) and particle swarm optimization (PSO). Additionally, one numerical example is also presented to illustrate the effectiveness of the designed algorithm. Keywords: Material procurement planning, Neural network, Credibility theory, Approximation approach, Particle swarm optimization.

1

Introduction

MPP deals with the problem that purchasing the right quantity of material from the right supplier at the right time, it is an important component of production planning. Since a reasonable MPP decision can reduce the material procurement cost or avoid the risk of material procurement, it attracts many researchers’ interest. However, in the practical MPP problem, lots of parameters such as material demand, spot market material unit price have the uncertainty. In order to deal with the MPP problems under the stochastic environment, some MPP models were proposed based on probability theory. The interested readers can refer to [1,2,3], they assumed that the uncertain parameters are random variables with known probability distribution functions. It is well-known that the acquisition of probability distribution functions relies on a large number of statistical data. However, there is no enough statistical data in many practical MPP problems, these MPP problems should be treated as fuzzy optimization problems. There are some researchers handled fuzzy MPP problems which take fuzzy set [4] or possibility theory [5] as the theoretical foundation, the related literatures contain Lai & Hwang [6] and Mula & Poler [7], etc. Based on possibility measure, Liu & Liu [8] presented a self-dual set function called credibility measure, to Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 245–252. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com 

246

G.-J. Sun

measure fuzzy event. Furthermore, Liu [9] developed credibility theory in fuzzy decision systems. Recent researches about this theory included Liu [10,11], Liu & Tian [12], Sun & Liu [13] and Sun, Liu & Lan [14] etc. In addition, credibility theory has attracted much attention and been applied in many fields such as engineering optimization, economics and management problems. The purpose of this paper is to take credibility theory as the theoretical foundation of fuzzy programming, and present a class of two-stage fuzzy programming with VaR to deal with MPP problems. Because of the MPP problem usually includes fuzzy variable parameters defined through possibility distributions with infinite supports. Therefore, in order to solve the proposed MPP problem, an AA [11] is used to turn the infinite-dimensional model into a finite-dimensional one, and the AA can establish several convergence relationships between the original MPP model and the approximating one. Thence it is feasible to solve the approximating model directly instead of the original one. However, the approximating model is neither linear nor convex, conventional optimization algorithms cannot be applied. As a result, a hybrid algorithm by integrating the AA, NN and PSO is designed to solve it. One numerical example is also given to show the effectiveness of the designed hybrid algorithm. The remainder of this paper is organized as follows. Section 2 proposes the two-stage fuzzy MPP model with VaR. Section 3 applies the AA to turn the original model into an approximating one and designs a hybrid algorithm to solve the approximating MPP model. Furthermore, a numerical example is also proposed to illustrate the effectiveness of the designed algorithm. Section 4 gives the conclusions.

2

Fuzzy MPP Model with VaR

Since many real-life MPP problems include fuzzy variable parameters, the firststage decisions are often infeasibility, hence second-stage or recourse actions are required. Additionally, VaR is an important criteria to model optimization problems involving risk. Therefore, the intent of this paper is to propose a class of two-stage fuzzy MPP model with VaR. In order to propose a reasonable MPP model, we will summarize the assumptions and notations, respectively. Assumptions • The source of material contains three components: in effect contracts, pending contracts and spot market, in which in effect contracts are long-term contractual commitments; pending contracts are new supply contracts that can be selected to negotiate, and the purchaser holds the election. • The fuzzy variable parameters include spot market material unit price and material demand. They are mutually independent [15] fuzzy variables with known possibility distributions. • The in effect contracts have a fixed material unit price and supply quantity range, respectively. These assumptions are also suitable for the pending

Fuzzy Material Procurement Planning with Value-at-Risk

247

contracts. In addition, the total amount of material purchased over two stages cannot less than material demand. Notations xi : material amount purchased from the ith in effect contract in first-sage; δk : binary variable equals to 1 if the kth pending contract is signed and equals to 0 otherwise; xk : material amount purchased from the kth pending contract in first-sage; yi : material amount purchased from the ith in effect contract in second-stage; yk : material amount purchased from the kth pending contract in second-sage; y: material amount purchased from the spot market; ci : material unit price of the ith in effect contract; li : lower bound of the ith in effect contract’s supply quantity range; ri : upper bound of the ith in effect contract’s supply quantity range; ck : material unit price of the kth pending contract; lk : lower bound of the kth pending contract’s supply quantity range; rk : upper bound of the kth pending contract’s supply quantity range; ξ1 (γ): spot market material unit price under fuzzy event γ; ξ2 (γ): material demand under fuzzy event γ. We assume that there are m in effect contracts and n pending contracts. The first-sage decision variables contain (xi , δk , xk ) and the variables (yi , yk , y) are the second-stage decisions, where i = 1, 2, · · · , m; k = 1, 2, · · · , n. For convenience, we denote the first-stage decision variables (xi , δk , xk ) by vector x and the secondstage decision variables (yi , yk , y) by vector y. The parameters (ci , li , ri , ck , lk , rk ) are fixed value, and (ξ1 , ξ2 ) are fuzzy variable parameters. Using the assumptions and notations above, the two-stage fuzzy MPP model can be formulated as follows. min QαVaR (x) s.t. li ≤ xi ≤ ri , i = 1, 2, · · · , m (1a)    δk lk ≤ xk ≤ δk rk , k = 1, 2, · · · , n (1b) (1c) xi ≥ 0, xk ≥ 0, δk = 0 or 1,

(1)

    m n     ci xi + ck xk + Q (x, ξ(γ)) ≤ ϕ ≥ α , where QαVaR (x)=inf ϕ | Cr γ | i=1

i=1

α is a preselected credibility level which belongs to (0, 1), Q(x, ξ(γ)) is the optimal objective value of second-stage under fuzzy event γ, and it is obtained by the following programming m n Q(x, ξ(γ)) = min i=1 ci yi + k=1 ck yk + ξ1 (γ)y i = 1, 2, · · · , m (2a) s.t. li ≤ xi + yi ≤ ri , k = 1, 2, · · · , n (2b) δk lk ≤ xk + yk ≤ δk rk , n m   i=1 (xi + yi ) + k=1 (xk + yk ) + y ≥ ξ2 (γ) (2c) (2d) yi ≥ 0, yk ≥ 0, y ≥ 0.

(2)

248

G.-J. Sun

For a fixed budget ϕ0 , we take the maximal credibility of fuzzy procurement cost less than ϕ0 as the risk level of budget funds ϕ0 . The objective of proposed fuzzy MPP model (1)–(2) is to find the minimal budget fund such that its risk level exceeding the preselected credibility level α is met. In addition, the constraints (1a, 1b, 2a, 2b) express that the material amount purchased from each supply contract must satisfy the contract’s restrictions. The constraint (2c) indicates that the total amount of material purchased over two stages cannot less than material demand. The constraints (1c) and (2d) determine the domains of decision variables.

3

Hybrid Algorithm and Numerical Example

The intent of this section is to design an effective algorithm for solving the proposed two-stage fuzzy model (1)–(2). Furthermore, we test the designed algorithm through a numerical example. 3.1

Hybrid Algorithm

1

1

0.8

0.8 Possibility distribution

Possibility distribution

According to the two-stage fuzzy model (1)–(2), it is easy to see that for any given first-stage feasible decision x and every realization (ξ1 (γ), ξ2 (γ)), the secondstage programming (2) is a linear programming that can be solved by simplex algorithm. In order to solve the two-stage fuzzy MPP model, it is required to evaluate the function Qx : x → QαVaR (x) repeatedly. Since any computer cannot compute infinite times, the value of QαVaR (x) at x cannot be obtained when (ξ1 , ξ2 ) are continuous fuzzy vector. Therefore, we should use AA to turn the continuous fuzzy vector (ξ1 , ξ2 ) into a sequence of discrete fuzzy vector (ζp,1 , ζp,2 ) which named the discretization of original fuzzy vector. About the AA in details, the interested readers can refer to Liu [11]. Here we only compare a triangular fuzzy variable ξ = (0, 1, 2) and its discretization ζp when p = 5, p = 10 via Fig 1, in which the line represents the possibility distribution of ξ, and the points represent the possibility distribution of ζp .

0.6

0.4

0.2

0

0.6

0.4

0.2

0

0.5 1 1.5 Fuzzy variable and its discretization when p=5

2

0

0

0.5 1 1.5 Fuzzy variable and its discretization when p=10

2

Fig. 1. The comparison of possibility distribution of ξ and its discretization ζp

Fuzzy Material Procurement Planning with Value-at-Risk

249

Because of the discretization (ζp,1 , ζp,2 ) obtained via AA can converge to (ξ1 , ξ2 ), thence we can replace the continuous fuzzy vector (ξ1 , ξ2 ) by its discretization. As a result, the original infinite-dimensional optimization problem is turned into a finite-dimensional optimization one. However, for any given first-stage decision x and each realization (ζp,1 (γ), ζp,2 (γ)), we have to solve the second-stage programming (2) via simplex algorithm one time, it is a timeconsuming process. In order to speed up the solution process, we desire to replace the objective function Qx by a trained NN. In this paper, we employ the fast BP algorithm to train a feed-forward NN [16]. After the NN is well-trained, we can embed it into PSO to produce a hybrid algorithm. The PSO algorithm is a population based algorithm that invented by Kennedy & Eberhart [17]. Recently the PSO algorithm has attracted much attention and been successfully applied in the fields of evolutionary computing, unconstrained continuous optimization problems and many other fields, the interested reader can refer to [18]. PSO uses a particle to indicate a possible solution of the optimization problem, and generates pop size particles to formulate a colony. It searches for the optimal solution by updating generations. The best previous position encountered by the ith particle is denoted as Pi , the particle that attained the best previous position among all the individuals of the colony is denoted as Pg , and t to be the iteration counter. The colony is manipulated according to the following equations: Vi (t + 1) = ωVi (t) + c1 r1 (Pi (t) − Xi (t)) + c2 r2 (Pg (t) − Xi (t))

(3)

Xi (t + 1) = Xi (t) + Vi (t + 1)

(4)

where i = 1, 2, · · · , pop size; ω is called the inertia coefficient; c1 and c2 are learning rates and usually c1 = c2 = 2; r1 and r2 are are two independent random numbers generated randomly in the unit interval [0,1]. Now we only provide the summary about the process of hybrid algorithm. A Hybrid Algorithm Step 1. Generate a set of input-output data for the objective function Qx of approximating MPP model. Step 2. Train an NN to approximate the approximating objective function Qx by the generated input-output data. Step 3. Initialize pop size particles with random positions and velocities, and evaluate the objective values of all the particles by the trained NN. Step 4. Set Pi of the ith particle and its objective value equal to its current position and objective value, and set Pg and its objective value equal to the position and objective value of the best particle in the initial colony. Step 5. Update the velocity and position of each particle according to the formulas (3) and (4), respectively. In addition, calculate the objective values of all new generated particles by the trained NN. Step 6. For the ith particle, compare the current objective value with that of its Pi . If the current objective value is smaller than that of Pi , then replace Pi and its objective value by the current position and objective value.

250

G.-J. Sun

Step 7. Find the best particle of the current swarm with the smallest objective value. If its objective value is smaller than that of Pg , then renew Pg and its objective value with the position and objective value of the current best particle. Step 8. Repeat Step 5 to Step 7 until a stopping criterion is satisfied, and return the Pg and its objective value as the optimal solution and the optimal value. 3.2

Numerical Example

To show the feasibility and effectiveness of the hybrid algorithm. we provide a MPP model with the credibility level α = 0.9, the unit price of the material on the spot market is triangular fuzzy variable ξ1 = (8, 9, 10), the material demand is a trapezoidal fuzzy variable ξ2 = (1000, 1200, 1500, 1600). In addition, the fuzzy variable ξ1 and ξ2 are supposed to be mutually independent. The other related parameters are collected in Table 1. Table 1. The parameters of numerical example l1 50 r1 80 c1 10

l2 50 r2 80 c2 10

l3 80 r3 120 c3 9

l4 80 r4 120 c4 9

l5 100 r5 150 c5 8

l6 100 r6 150 c6 8

l1 50 r1 100 c1 10

l2 50 r2 100 c2 10

l3 50 r3 100 c3 9

l4 100 r4 150 c4 9

l5 100 r5 150 c5 8

l6 100 r6 150 c6 8

First, using AA with 2000 sample points, we generate 3000 input-output data for the objective function Qx . Then we use the 3000 input-output data to train an NN (12 input neurons representing the value of first-stage decision x, 15 hidden neurons and 1 output neurons representing the value of Qx ) to approximate function Qx . Finally, the well-trained NN is embedded into a PSO to search for the optimal solution of MPP model. A run of the hybrid PSO algorithm with 30 particles and 500 generations produces the following optimal solution: x1 =50.0000, x2 =50.0000, x3 =80.0000, x4 =80.0000, x5 =100.0000, x6 =100.0000, x1 =0.0000, x2 =0.0000, x3 =0.0000, x4 =0.0000, x5 =150.0000, x6 =100.0000, and its optimal value is 12952.1656. Based on general knowledge, we know that the contracts whose unit price is lower should be selected. So according to the parameters in our numerical example, we can see that the result above is reasonable, which shows that the designed hybrid algorithm is effective.

4

Conclusions

In this paper, we proposed a class of two-stage fuzzy programming to handle MPP problem with VaR criteria based on credibility theory. Since the fuzzy

Fuzzy Material Procurement Planning with Value-at-Risk

251

variable parameters had continuous possibility distribution with infinite support that cannot be solved directly. Therefore, we used the AA to turn the original infinite-dimensional optimization problem into an approximating one. However, the approximating optimization problem is a mixed-integer program that is neither linear nor convex, traditional optimization algorithms cannot be used to solve it. In order to solve the two-stage fuzzy programming problem, a hybrid algorithm which combines AA, NN and PSO, was designed. Finally, we provided one numerical example to illustrate the feasibility and effectiveness of the designed algorithm, and the computational results showed that the proposed hybrid algorithm is feasibility and effectiveness. Acknowledgments. The work was supported by the Research Project of Zhejiang Educational Commission Grant No.Y200909282.

References 1. Fabian, T., Fisher, J., Sasieni, W., Yardeni, A.: Purchasing Raw Material on A Fluctuating Market. Oper. Res. 7, 107–122 (1959) 2. Kingsman, B.G.: Purchasing Raw Materials with Uncertain Fluctuating Prices. Euro. J. Oper. Res. 25, 358–372 (1986) 3. Bonser, J.S., Wu, S.D.: Procurement Planning to Maintain Both Short-Term Adaptiveness and Long-Term Perspective. Manage. Sci. 47, 769–786 (2001) 4. Zadeh, L.A.: Fuzzy Sets. Information and Control 8, 338–353 (1965) 5. Dubois, D., Prade, H.: Possibility Theory. Plenum Press, New York (1988) 6. Lai, Y.J., Hwang, C.L.: Fuzzy Mathematical Programming: Methods and Applications. Springer, Berlin (1992) 7. Mula, J., Poler, R., Garcia-Sabater, J.P.: Material Requirement Planning with Fuzzy Constraints and Fuzzy Coefficients. Fuzzy Sets Syst. 158, 783–793 (2007) 8. Liu, B., Liu, Y.K.: Expected Value of Fuzzy Variable and Fuzzy Expected Value Models. IEEE Trans. Fuzzy Syst. 10, 445–450 (2002) 9. Liu, B.: Uncertainty Theory: An Introduction to Its Axiomatic Foundations. Springer, Berlin (2004) 10. Liu, Y.K.: Fuzzy Programming with Recource. Int. J. Uncertainty Fuzziness Knowl.-Based Syst. 13, 381–413 (2005) 11. Liu, Y.K.: Convergent Results about the Use of Fuzzy Simulation in Fuzzy Optimization Problems. IEEE Trans. Fuzzy Syst. 14, 295–304 (2006) 12. Liu, Y.K., Tian, M.: Convergence of Optimal Solutions about Approximation Scheme for Fuzzy Programming with Minimum-Risk Criteria. Comput. Math. Appl. 57, 867–884 (2009) 13. Sun, G.J., Liu, Y.K.: Fuzzy Minimum-Risk Material Procurement Planning Problem. In: 4th International Conference on Natural Computation, pp. 629–633. IEEE Press, Jinan (2009) 14. Sun, G.J., Liu, Y.K., Lan, Y.F.: Optimizing Material Procurement Planning Problem by Two-Stage Fuzzy Programming. Comput. Ind. Eng. 58, 97–107 (2010) 15. Liu, Y.K., Gao, J.: The Independence of Fuzzy Variables with Applications in Fuzzy Random Optimization. Int. J. Uncertainty Fuzziness Knowl.-Based Syst. 15, 1–20 (2007)

252

G.-J. Sun

16. Scarselli, F., Tsoi, A.C.: Universal Approximation Using Feedforward Neural Networks: A Survey of Some Existing Methods, and Some New Results. Neural Networks 11, 15–37 (1998) 17. Kennedy, J., Eberhat, R.C.: Particle Swarm Optimization. In: Proccedings of the IEEE International Conference on Neural Networks, Piscataway, NJ, pp. 1942–1948 (1995) 18. Kennedy, J., Eberhat, R.C., Shi, Y.: Swarm Intelligence. Morgan Kaufmann Publishers, San Francisco (2001)

Radial Basis Function Network for Endpoint Detection in Plasma Etch Process Shu-Kun Zhao, Min-Woo Kim, Yi-Seul Han, Se-Youn Jeon, Yun-Keun Lee, and Seung-Soo Han Digital Information Lab., 8507, Dept. of Information Engineering, Myongji University, San 28 Namdong, Yongin, Gyeonggido, 449-728, Korea [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]

Abstract. In the semiconductor manufacturing process, the endpoint of plasma etch process can be determined by the graphics based detection in order to avoid the loss of over-etching and under-etching. Our approach in current study can be conducted as one way to real-time monitor and judge the endpoint instead of observing it manually. When the endpoint occurs, this system can improve the etch processes and provide instant shutdown recommendations. This method makes use of Radial Basis Function (RBF) network’s functional approximation in time-series modeling and in pattern classification. By training with enough samples, the judge will be more accurate. All the samples are probed with optical emission spectroscopy (OES) sensor in real plasma etch process and for both network training and test. Keywords: Etch Process, OES, RBF, Neural Network, Endpoint Detection.

1 Introduction This study was on the plasma etching, which is widely used in the semiconductor manufacturing and the need for in-situ process monitoring of plasma etching is becoming greater as the technology advances. Extremely tight control of all process parameters must be maintained to increase throughput and reproducibility. The greatest need for plasma process monitoring arises in the determination of the etch endpoint for a given process, which can reduce the degree of overetching and underetching [3]. There are other techniques that are used to obtain information about reactive ion plasmas, such as ellipsometry, mass spectroscopy, Langmuir probe measurements, laser induced fluorescence, coherent anti-stokes spectroscopy, and infrared/visible region absorption spectroscopy, but these methods are primarily used for research since they are experimentally more demanding than the previous techniques. Etch is the process of selectively removing unneeded material from the wafer surface by using either chemical or physical means. The fundamental goal of etching is to accurately reproduce the mask features on the resist-coated wafer. Two basic types of etch processes are used in semiconductor manufacturing: dry etch and wet Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 253–263. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

254

S.-K. Zhao et al.

etch [5]. Dry etching is method that etching causing physical reaction in plasma state using gas. And wet etching using chemical reaction, angle of repose attains in order that chemical component and ingredient that wish to etch because material to be etched causes chemical reaction have melted. Etch rate is the speed at which material is removed from the wafer surface during etching (see Fig. 1)

Etch Rate =

ΔT t

(1)

Where:

ΔT = amount of material removed (Å or μm ) t = time elapsed during etch (typically minutes) ΔT ΔT

Fig. 1. Etch Rate

Observing formula (1.1), obviously, at the same situation, the time elapsed is a critical factor that influences the etching profile. For a typical etch process, the required film thickness is etched away and then the etch process continues until a specific over-etch time has arrived. The over-etch time must be long enough so that the etched layer has been removed at every position on the wafer surface, but must also be short enough so that the etch layer is not unacceptably eroded or the layer underlying the etch layer has not been etched [3]. Optical Emission Spectroscopy (OES) is used when monitoring etching process from optical signals into electronic signals. It is also one of the oldest and most popular methods of plasma etch monitoring. Fundamentally, OES is a bulk measure of the optical radiation of the plasma species. Since emissions can emanate from etch reactants as well as products, OES measurements are most often used to obtain the average optical intensity at a particular wavelength above the wafer [7]. While OES is an excellent tool for monitoring plasma emission intensity, a primary issue with its use is the large dimensionality of the spectroscopic data [2].

Fig. 2. General OES device. Light which come out in Plasma is frequency of very wide extent, but area that use in OES most 200 ~ 900nm.

Radial Basis Function Network for Endpoint Detection in Plasma Etch Process

255

Light flow, which comes out in plasma, needs monochromater to do dispersal and optical system and photodetector to intromit light. General OES device is as Fig.2 shown. Light which comes out in Plasma comes by spectroscope through direction or optical fiber. Then, photocurrent that is proportional in the century reaching to PM detector being spread, and only light which has decided conclusion of state examination passes aperture to be way out because diffraction occurs in spectroscope. OES that is used both number to detect change of characteristic spectrum by relative change of inside structure material. Therefore, spectrum of main material that is detected between etching detected endpoint by doing real time monitoring. As Fig.3 shown, the endpoint is the very point, after which the data will fall rapidly. It is marked with circle. As a result, use observed OES graph to trace the color changes on the wafer surface, and then can determine whether the end of the reaction. At the end of the etch process, the gas mixture again resembles its mixture prior to the start of the process. Therefore, the etch endpoint is characterized by a sharp change in the intensity of the endpoint trace.

Fig. 3. Plasma etch intensity (OES): above 4 figures are the intensity values – time graph. The normal line shows the original data, and the bold one shows the curve after filtering. (For details about the filtering, refer to the Methodology part)

Fig. 3 enumerates 4 samples, which were recorded in the real plasma etching experiments under the same condition. In this paper, we propose a novel approach to

256

S.-K. Zhao et al.

model the endpoint shape signature based on the Radial Basis Function Networks. Our approach can directly capture a process engineer’s knowledge about endpoint characteristics, for instance, the nature of the endpoint between polynomial segments, the shape of an example pattern, the expected location in time of the endpoint. This paper is structured as follows: in section 2, we explain the RBF network; in section 3, we provide the methodology details – introduction to network design, also including training data collection and choosing the target matrix; the simulation details and results when applying RBF networks are provided in section 4 and 5, while the conclusions of this study are drawn in section 6.

2 Radial Basis Function Network Radial Basis Function Network emerged as a variant of artificial neural network in late 80’s. However, their roots are entrenched in much older pattern recognition techniques as for example potential functions, clustering, functional approximation, spline interpolation and mixture models [1]. The construction of a RBF network in its most basic form involves three entirely different layers. The input layer is made up of source nodes (sensory units). The second layer is a hidden layer realizing radial basis functions with high enough dimensions, which serves a different purpose from that in a multilayer perceptron. The output layer supplies the response of the network to the activation patterns applied to the input layer. The transformation from the input space to the hidden-unit space is nonlinear, whereas the transformation from the hidden-unit space to the output space is linear. Given a set N different points of N real numbers {d i ∈ R

1

{xi ∈ R p | i = 1, 2,...., N } and a corresponding set

| i = 1, 2,...., N } , find a function F : R N → R1 that

satisfies the interpolation condition:

F ( xi ) = d i , i = 1, 2,......N

(2)

Note that for strict interpolation as specified here, the interpolating surface (i, e., function F) is constrained to pass through all the training data points. The radial basis functions technique consists of choosing a function F that has the following form (Powell M.J.D, 1985): N

F (x) = ∑ wi ⋅ ϕ ( x − xi i =1

Where

{ϕ ( x − x ) | i = 1, 2,..., N } i

)

(3)

is a set of N arbitrary (general nonlinear)

functions, known as radial basis functions, and i denotes a norm that is usually taken to be Euclidean. The known data points xi ∈ R p , i = 1, 2,..., N are taken to be the centers of the RBFs.

Radial Basis Function Network for Endpoint Detection in Plasma Etch Process

257

Inserting the interpolation conditions of (2),(3), we obtain the following set of simultaneous linear equations for the unknown coefficients (weight) of the expansion {w}:

⎡ ϕ11 ϕ12 .... ϕ1N ⎢ ⎢ ϕ21 ϕ22 .... ϕ2 N ⎢ ⎢ ⎣ ϕ N 1 ϕ N 2 .. ϕ NN

⎤ ⎥ ⎥ ⎥ ⎥ ⎦

⎡ w1 ⎤ ⎡ d1 ⎤ ⎢w ⎥ ⎢d ⎥ ⎢ 2⎥ = ⎢ 2⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ wN ⎦ ⎣ d N ⎦

(4)

where

ϕij = ϕ ( x j − xi ), j , i = 1, 2,.....N

(5)

Let

d = [d1 , d 2 ,.....d N ]T w = [ w1 , w2 ,.....wN ]T

(6)

The N-by-1 vectors d and w represent the desired response vector and linear weight vector, respectively. Let φ denote an N-by-N matrix with elements ϕij :

φ = {ϕij | j , i = 1, 2,....N }

(7)

We call this matrix the interpolation matrix. We may then rewrite (4) in the compact form

φw = x

(8)

There is a class of radial-basis functions that enjoys the following remarkable property(Light, 1992): Let x1 , x2 ,.....xN be distinct points in R p Then the N-by-N interpolation matrix

φ

whose jith element is

ϕij = ϕ ( x j − xi )

is positive definite.

This theorem is more powerful than a previous result due to Micchelli(1986) stating that the interpolation matrix φ is nonsingular. Light’s theorem applies to the following cases (among others): 1.

Inverse multiquadrics (Hardy, 1971)

ϕ (r ) =

1 for some c > 0, and, r ≥ 0 (r + c 2 )1/ 2 2

(9)

258

2.

S.-K. Zhao et al.

Gaussian functions

ϕ (r ) = exp(−

r2 ) for σ > 0 , and r ≥ 0 2σ 2

(10)

Theoretical investigations and practical results, however, seem to show that type of nonlinearity ϕ (.) is not crucial to the performance of RBF networks (Powell, 1988). Returning to the implication of Light’s theorem, we note that, provided that the data points are all distinct, the interpolation matrix φ is positive definite, and so we may solve (8) for the weight vector w, obtaining

w = φ −1d

(11)

where φ is the inverse of the interpolation matrix φ . Although in theory we are always assured a solution to the strict interpolation problem, in practice we cannot solve (8) when the matrix φ is arbitrarily closet to singular. At this point, −1

regularization theory can help by perturbing the matrix φ to φ + λ I , as we shall see in Regularization Theory. The approximation capabilities of RBF networks have been studied in [9, 11]. Due to their nonlinear approximation properties, RBF networks can build complicated mappings contrasted with multilayer perceptron [10]. Furthermore, a typical RBF network has only two weight layers and also has generalized simple learning algorithms, so the training speed is faster than multilayer perceptron. While modeling the network, it is different when approximating time-series and in pattern classification [4]. In the first case, the network inputs represent data samples at certain past time-laps, while the network has only one output representing a signal value as shown in Fig. 1. In a pattern classification application the inputs represent feature entries, while each output corresponds to a class as shown in Fig 2. The hidden units correspond to subclasses in this case [11].

3 Methodology To replace the traditional manual control to avoid bring about errors and improve performance. There are basically two ways to replace manual control: 1. Abstract and summarize from the experiences of manual control into general rules or fuzzy rules, and then construct the corresponding exclusive controller or fuzzy controller; 2. Use artificial neural network to study the behaviors of human control when it is difficult to express knowledge. That means, model the artificial control, and then take advantage of neural network controller instead of manual control. This study of traditional controller and then gradually replacing it with the neural network controller is called the neural network supervisory control. Neural network as a parallel computing model, take advantages of traditional modeling methods do not have. It also have a good ability of nonlinear mapping, in terms of that it is not necessary to ask for much empirical knowledge of modeled object. Generally, not have to foreknow about the structure parameters and dynamic

Radial Basis Function Network for Endpoint Detection in Plasma Etch Process

259

characteristics such as knowledge of objects. Instead just need the object inputs and outputs. The network itself will learn to achieve input and output mapping. First of all, observing the filtered data collection, shown in Fig. 3, can find that, if want to design a real-time monitoring system, we should filter the new probe data with previous ones after get it. After then, input the new filtered data together with a number of previous existing filtered data as input of RBF. Here, how many reference data needed, depends on the properties of the observed data: OES data fall rapid at the endpoint, and time is very short, approximately 0.5 ~ 1.0sec. The width of sample, the number of neurons in the RBF network input layer, will affect the network performance. This factor should be concerned. After input data into the RBF network, can get one output corresponds to the current data pattern and network condition. This output can be a scale number or a vector. Considering the complexity of the output in different cases, could design training data and it corresponding target output matrixes of the RBF network. Here we use a scale value as the target of each line of training input. Observing the sample data, in the rapid increase phase, mostly corresponds to the reaction beginning stages while plasma gas mixture. The training target is set at -1.0. During etch process, OES of intensity value trends in relatively flat shape. The training target is set at 0.0. And at the last of etch process, endpoint occurs, the observation value fall rapidly, while set the corresponding training target at 1.0. After then, the OES value preserves in relatively flat at lower.

Fig. 4. Concentration of blips: the samples mainly concentrated in high-value areas (about 17.0 ~ 26.0) and low-value area (about 0.0 ~ 7.0). Between them, scatters a few scattered points (besieged with smaller ellipse). The points are precisely the points corresponding to the rapid declining – endpoint.

The Fig. 4 shows that, according to the distribution of points can be considered concentrated in high-value areas and low-value areas have a similar distribution (similar to normal distribution), while in the rapid decline area, sampling points, show the distribution of uniform, and the slope of the fitted straight is very high (changes fast). Through the above generalization, we can design a system based on RBF neural network that can be suitable for real-time monitoring of plasma etch process endpoint detection.

260

S.-K. Zhao et al.

Fig. 5. System model. The 1st layer is a filter; the 2nd layer is a sequence restoring the current filtered data and a part of previous ones; the 3rd layer is a RBF network, which returns the judgment result.

Fig. 5 shows the design of the system model. The system consists of 3 layers: The first layer is a filter used to filter the system error brought by the new received data. Here we use a very simple mean filter. Namely, whenever receive a new data at time t, do the average of the measured data together with the previous n data. Then, take the mean value as the standard input data at time t. Here, call n the system of filter coefficients. If the total number of received data is less than n, average the input the data has been measured as the standard input data at time t.

Fig. 6. The mean filtering method

Next is the input layer. In this system, input layer is equivalent to a queue that has a certain length. The length of the queue is as same as the number of data which is used for training network. Actually, either training data or real data is standard input. Given the length of input l, when calculate the filtered value at time t, deliver the value to the next RBF network together with previous l – 1 filtered data. The last layer is one RBF network. RBF network use radbas function (MATLAB) as activation function:

radbas (n) = e− n

2

(12)

MATLAB supplies two functions that can be used to generate one RBF network: newrb and newrbe. Function newrbe is different from newrbe, newrbe establishes a RBF network with little error and rapidly according to the characteristics of input vectors. The functions have a same parameter SPREAD. Parameter SPREAD decides the accuracy of interpolation. With increasing SPREAD, RBF network adds more

Radial Basis Function Network for Endpoint Detection in Plasma Etch Process

261

neurons to adapt rapid changes in the curve. But a large number of neurons will take long time to train it and also might cause hard to computing. Therefore, it needs to try several SPREAD values in simulation. There are two sets of data will be input the system and validate whether the endpoint can be detected.

4 Simulation and Result Generate the Training Data Via the method introduced in Methodology, the data for training RBF network is generated. Table 1 shows the example of training data. Upper part is collected from the beginning of sample 1; and lower part is collected from the endpoint of sample 1. Table 1. Example of training data Input P matrix

Target

-3.083333333 -0.930555556 5.814814815 11.43055556 12.56111111 -0.930555556 5.814814815 11.43055556 12.56111111 18.12777778 5.814814815 11.43055556 12.56111111 18.12777778 21.88888889 11.43055556 12.56111111 18.12777778 21.88888889 21.57222222 12.56111111 18.12777778 21.88888889 21.57222222 18.22777778 18.12777778 21.88888889 21.57222222 18.22777778 17.87777778 21.88888889 21.57222222 18.22777778 17.87777778 17.31111111 21.57222222 18.22777778 17.87777778 17.31111111 17.07222222 25.45555556 24.05 22.54444444 20.05555556 17.62222222 24.05 22.54444444 20.05555556 17.62222222 13.8 22.54444444 20.05555556 17.62222222 13.8 10.4 20.05555556 17.62222222 13.8 10.4 10.4 6.316666667 17.62222222 13.8 10.4 6.316666667 5.4 13.8 10.4 6.316666667 5.4 4.688888889 10.4 6.316666667 5.4 4.688888889 2.833333333 6.316666667 5.4 4.688888889 2.833333333 3.272222222

-1 -1 -1 -1 0 0 0 0 0 0 1 1 1 1 1 0

After then, input data into the system. Follow is a part of simulation code. At each time-step, a OES value is delivered to the filter, then stored at input sequence. In final step, RBF network will judge whether current value is the endpoint. net = newrbe(P, T, 5.0); [ROW, COL] = size(test1); for i = 1 : ROW y = sim(net, (test1(i,:)')); disp(y(1,1)); if y(1,1) > 0.99999999 end end disp('ENDPOINT:'); disp(i);

break;

262

S.-K. Zhao et al.

The output will show the time-step of endpoint

Fig. 7. The detection result. In the graph, the circle besieges the area is the endpoint detected manually, and the straight line vertical to the time-step axis marks the endpoint detected by simulation result.

In some cases, if the parameter SPREAD is 1.0, the endpoint cannot be found. By increasing it to 5.0, after training, the endpoint can be detected. So if RBF is used in practice, the parameter SPREAD should be validated in several values. System Performance Evaluation If consider the strict real-time performance to the system, it is necessary to refer to the influences of filtering. That will be balanced by adjusting filtering coefficient. In current study, the filtering method will cause the detection a little delay to the endpoint, and the delay is decided by used filtering coefficient.

tdelay = α ⋅ coef filter

(13)

Where:

tdelay is the delay cause by filtering. α is the very factor reflect the relationship between delay and filtering coefficient. coef filter is the filtering coefficient used in system. And also one median RBF neural network with higher approximation capability [5] [8] will be considered and compared with current result.

5 Conclusion In this study we provide a specific application of the Radial Basis Function networks in semiconductor fabrication. Due to the marked properties, for instance, localization, functional approximation, interpolation, cluster modeling and quasi-orthogonality, RBF can be used in many applications. The feasibility of RBF network applications

Radial Basis Function Network for Endpoint Detection in Plasma Etch Process

263

into semiconductor equipment and production processes was verified. The endpoint can be detected accurately for the other two test samples. Acknowledgments. This work is financially supported by Ministry of Knowledge Economy (10031812-2008-11).

References 1. Tou, J.T., Gonzalez, R.C.: Pattern Recognition. Addison-Wesley, Reading (1974) 2. Hong, S.J., May, G.S., Park, D.C.: Neural network modeling of reactive ion etching using optical emission spectroscopy data. IEEE Transactions 16(4), 598–608 (2003) 3. Czebiniak, J.M.: End Point Detection of Plasma Etching Using Optical Methods. In: Annual Microelectronic Engineering Conference (May 2006) 4. Bors, A.G.: Introduction of the Radial Basis Function Network. University of York, UK 5. Quirk, M., Serda, J.: Semiconductor Manufacturing Technology. Pearson Education International Presses, ISBN 978-7-121-08944-2 6. Haykin, S.: Neural Networks – A Comprehensive Foundation. Prentice Hall, Englewood Cliffs (1998); ch. 7: Radial-Basis Function Networks 7. May, G.S., Spanos, C.J.: Fundamentals of Semiconductor Manufacturing and Process Control. IEEE, Wiley-Interscience 8. Bors, A.G., Pitas, I.: Median radial basis functions neural network. IEEE Trans. on Neural Networks (1996) 9. Park, J., Sandberg, J.W.: Universal approximation using radial basis functions network. Neural Computation 3 (1991) 10. Haykin, S.: Neural Networks: A comprehensive Foundation. Prentice Hall, Upper Saddle River (1994) 11. Poggio, T., Girosi, F.: Networks for approximation and learning. Proc. IEEE 78(9) (1990)

A Novel Cellular Neural Network and Its Applications in Motion Planning Yan Cao1, Feng Zhang1, Xuewu Wu1, Sheng Lu1, Yi Li1, Lei Sun2, and Shuai Li3 1

Artillery Academy, Hefei 230037, China Anhui Agriculture University, Hefei 230036, China 3 Institute of Intelligent Machines, Chinese Academy of Sciences, Hefei 230031, China [email protected] 2

Abstract. A Novel Cellular Neural Network (CNN) entitled the shortest path CNN (SP-CNN) is proposed in this paper. Compared with general CNN, it is distinguished in the network structure and neural dynamics. As a result of these distinctions, SP-CNN has a good performance in motion planning for mobile robots. By mapping environment information to parameters in this neural network, motion planning can be transformed to the state evolvement of SPCNN and the generated state represents information of the optimal path. The proposed method generates the best solution in static environments in real time. Extensive simulations about the above mentioned aspects demonstrate the effectiveness of the proposed approach. Keywords: CNN, SP-CNN, Mobile robot, Motion planning.

1 Introduction Motion planning is a crucial topic for mobile robot navigation. Mobile robots move from current position to the target position through motion planning. With the quick development of mobile robotics, it is required for some applications that mobile robots have the capability of planning their motions within dynamic environments or unknown environments in real time. This requirement is often made for exploration robots like planetary exploration rovers, field exploration robots, rescue robots, etc. In this context, a novel cellular neural network (CNN) entitled the shortest path CNN (SP-CNN) is proposed to solve the problem. The proposed method generates the optimal solution in real time not only in a static environment but also in an unknown environment according to its currently acquired map. Many studies have been performed in the motion planning field. Artificial potential field (APF) method [1], vector field histogram (VFH) method [2] and some other behavior based methods [3] are simple and have a low computational cost. In some simple environments, they may satisfy the requirement. However, APF, VFH and all the other behavior base motion planning approaches cannot generate the globally optimal solution. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 265–273. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

266

Y. Cao et al.

In a complex environment, the simple ‘attraction-repulsion’ mechanism of APF cannot result in satisfactory solution. It often drops into local minima or suffers from Oscillations in the presence of obstacles in such an environment [4].Various novel potential field methods, like Laplace potential field method [5], viscous fluid method [6], magnetic field method [7], electrostatic potential field method [8], dynamic field method [9], etc., are proposed to overcome the above mentioned shortcomings. These methods can generate a feasible path even in complex environments. However, they have a high computational cost and the generated path is not globally optimal. In recent twenty years, more and more researchers focus their attention on the study of neural network based motion planning approaches. Siemiatkowska B proposed a highly parallel cellular neural network based motion planning method. Experiments reveal that the proposed method can find a free path in unknown and varied environments [10]. R. Glasius and A. Komoda proposed a discrete neural network model and a continuous neural network model for motion planning [11] [12]. Yang and Meng proposed a new biological inspired neural network approach for motion planning [13][14]. It is more general and more powerful than the model of R. Glasius et al. However, all the previously mentioned neural networks based motion planning approaches cannot ensure the generated path is globally optimal if the related parameters are not properly chosen [15]. In this paper, a novel CNN entitled SP-CNN is proposed to solve the problem of real-time motion planning. Similar to other neural networks based motion planning, neurons in SP-CNN are also locally connected. Different from other model, SP-CNN has continuously changed neural state values. When SP-CNN becomes steady, the state value of each neuron is equal to the distance from the target. Thus, the state value of each neuron in SP-CNN has an excellent physical meaning. Heuristic information is applied to determine the initial state of SP-CNN to increase the efficiency. This paper is organized into four sections. In section 2, the model of SP-CNN is introduced and it is applied to motion planning problem. In section 3, the applications of SP-CNN for motion path in various environments are supported by computer simulations, which demonstrate the effectiveness of the proposed model. In section 4, several conclusions are drawn.

2 The SP-CNN Model 2.1 Neural Dynamics CNN was firstly presented in 1988 by Chua and Yang [16]. This kind of network is topologically organized with identical dynamical systems. The cells are only locally connected to its neighbors. Adjacent cells interact with each other directly while cells not in the immediate neighborhood interact indirectly through cells connected with them. This model has a simple nonlinear dynamics and is very well suitable for hardware implementations. The neurons in this network correspond to vertexes while connections between neighbor neurons correspond to edges in graph theory. Thus definitions in graph theory can be extended to SP-CNN. In the following part, some term coming from the graph theory will be used. In SP-CNN, there are three types of

A Novel Cellular Neural Network and Its Applications in Motion Planning

267

cellular neurons: target neurons, obstacle neurons and free neurons. Compared with original CNN, SP-CNN has different neural dynamics. This difference endows SPCNN with some new features. Each cellular neuron has a state x, an input u, and an output y. Here we always choose y=x, so y will not appear in the following part of this paper. The dynamics of the i-th neuron in SP-CNN is characterized as follows:

xi = − Axi + Af (ui ) Where

(1)

xi is the state of the i-th neuron. ui is the input of the i-th neuron. A is a

positive constant which determines the dynamic performance of the network. The larger the value of A, the more fast the response speed. However, if A is too large, high frequency noises in

ui results in severe consequence for the output of this

network system. The function

f (ui ) is defined as:

⎧G f (ui ) = ⎨ ⎩u i

ui ≥ G

(2)

otherwise

Where G is an enough large constant, which is larger than the distance between the target neurons and free neurons. The input of the i-th neuron the target neuron, else if it is the obstacle neuron,

ui is equal to zero if it is

ui is equal to G. Otherwise, it is

equal to the minimal value of the sum of the j-th neuron’s state and the distance between the i-th and the j-th neurons (here the j-th neuron is a neighbor of the i-th neuron).This can be expressed as follows: ⎧ ⎪⎪0 ui = ⎨G ⎪min{x + d } i− j ⎪⎩ j∈Ωi j

Where

target neurons obstacle neurons free neurons

(3)

d i − j is the weight of edge connecting the i-th and the j-th neurons. Ωi is the

neighbor set of the i-th neuron. Actually, the state of each neuron converges to a positive number finally. That is to say, when the time is enough large the expression of ui for free neurons can be expressed as follows:

ui = [xi1 + di−i1, xi 2 + di−i 2 ,..., xij + di−ij ,...]

−

−∞

=lim( ∑ (xij + di−ij )−n ) n→∞

ij∈Ωi

1 n

(4)

Where i1,i2,...,ij,…are all neighbor neurons of the i-th neuron. In equation (4), the input of a free neuron is expressed in the form of negative infinite-norm. According to the neural dynamic of SP-CNN, the input of target neurons is clamped to 0 and the input of obstacle neurons is clamped to G. The input of a free neuron is the output of a competitive layer. The input is the sum of the state of this

268

Y. Cao et al.

neighbor neuron and a bias. The bias is the weight of the corresponding edge in topology. The value of it can be chosen according to applications. For a free neuron, the first layer is a competitive layer while the second layer is a dynamic local dynamic layer. For SP-CNN the initial state of the i-th neuron is chosen as follows:

xi0 = random;

xi0 ∈ (-∞, +∞)

(5)

In a static environment, SP-CNN approaches steady gradually with the elapse of time. In practice, an index is required to determine whether the network is steady under certain accuracy level. Here we define an index entitled absolute rate of change (ARC) like this:

max { lim

for all i

dt → 0

xi (t + dt ) − xi (t ) } dt

(6)

ARC is an index revealing the difference between current states of the whole neural network and the stable states. It is not larger than the rate of change of any neuron in this network, so the change of the state value of any neuron is very small if ARC becomes very small and thus each neuron becomes steady. When ARC is enough small, we think the whole network has become steady. In practice, when the expression of (7) becomes true, SP-CNN is regarded to be steady.

ARC ≤ δ

(7)

Where δ is the accuracy level. In search algorithms, heuristic information is often adopted to improve the effectiveness. Generally the heuristic algorithm can find the best solution faster than Dijkstra’s algorithm if the heuristic information is properly defined. Heuristic information can also be used for a dynamic system. The system becomes steady in a short time if its initial state is close to the steady state. Thus, if we choose proper initial state according to the prior knowledge, the state which is very close to the steady state, time consumption is expected to reduce. In SP-CNN, heuristic information can be employed to accelerate the convergence. 2.2 Motion Planning through SP-CNN The purpose of motion planning is to find a free path connecting the target position and the start position and the generated path satisfy certain optimal condition. Motion planning is based on a navigation map. Different maps result in different paths. Here SP-CNN is the path planner, so we should transform the map to expressions which can be recognized by SP-CNN. The configuration space of the mobile robot is divided into grids. This way, we get a grid-based map. There are several types of grids in such a grid-based map as follows: Target grids: The grid identifies the positions of robots in configuration space. Obstacle grids: The grids in configuration space correspond to obstacles in workspace. This can be expressed in a set:

A Novel Cellular Neural Network and Its Applications in Motion Planning

269

Oc = {q ∈ C | R(q ) ∩ Ow ≠ ∅} Oc represents obstacle grids in configuration space; C represents all grids in configuration space; R ( q ) represents the positions occupied by the robot in workspace corresponding to the grid q in configuration space; Ow represents the Where

obstacle positions in workspace. Free grids: The grids in configuration space correspond to spaces except for obstacles in workspace. This can be expressed in a set:

Fc = C − Oc The grid based-map is also a kind a graph from the view of graph theory. The SPCNN used for motion planning is designed to have the same topological structure as the grid-based map. The weight of an edge (as mentioned in equation (3)) in SP-CNN is equal to that of its corresponding edge in the grid-based map. Thus, a vertex in the grid-based map identically corresponds to a neuron in SP-CNN. Two vertices in the grid-based map are adjacent if they are connected by an edge. Since the grid-based map is locally connected by edges, the corresponding neural network with the same topological structure is also locally connected. Until now, we get a SP-CNN with corresponds to the given grid-based map. That is to say, the transform from a gridbased map to a SP-CNN is realized. The output of SP-CNN is the state values of every neuron in the network. It carries information about the optimal path in the particular environment. In order to find the optimal path, it is necessary to carry out the inverse transformation from the state values of SP-CNN back to the path in the grid-based map. For the given i-th grid corresponding to the i-th neuron, the next grid (the j-th grid, which corresponds to the j-th neuron) is obtained by:

j = arg min{x j + di − j } j∈Ωi

Where

(8)

d i − j is the distance between the i-th grid and the j-th grid in the grid-based

map. It is equal to the distance between the i-th neuron and the j-th neuron in the corresponding SP-CNN. Ωi is the neighbor set of the i-th grid. Equation (8) means that the next grid is the current grid’s neighbor grid, the corresponding neuron of which is characterized by the minimal value of x j + d i − j . In steady state, the state values of neurons in SP-CNN indicate some important information. If the state value of certain neuron approaches 0, the corresponding grid must be a target grid. If the state value of certain neuron .is equal to G in steady state, there must be no feasible path from the corresponding grid to one of target grids. This grid may be an obstacle neuron. Otherwise it is a free neuron, all paths from which to target grids are obstructed by obstacle grids. This will be revealed in the following simulations. Compared with most of other neural network path planning methods [10]-[15], SPCNN has an important property: the steady states of SP-CNN are equal to the distance

270

Y. Cao et al.

to the target position. With this distinctive feature, the mobile robot can predict the distance from current grid to the target grid. Also SP-CNN is a type of parallel network. It has relatively low time consumption. In dynamic or unknown environments, the navigation map updates rapidly with the increase of environment information. Traditional off-line planning methods have bad performances in this situation. As to SP-CNN, the change of navigation map leads to the change of state values in SP-CNN and the states indicate the globally optimal motion based on the current map, so SP-CNN based motion planning performs successfully in unknown or dynamic environments, the map of which changes rapidly.

2

1

Fig. 1. The weight of edge in the2-D square Fig. 2. Motion planning in the U-shape grid-based map environment

3 Simulations To demonstrate the effectiveness of this proposed SP-CNN model in motion planning, simulations are conducted. Without losing generalization, the proposed approach is applied to a point mobile robot in a 2-D workspace. Of course, SP-CNN based motion planning is not confined to a point mobile robot in 2-D environments. For robots in higher dimensional environment with the consideration of the robot’s shape and its nonholomic constraints, only mapping from environment to neural networks is different from the simulations referred in this paper while the fundamental principle of the neural network dynamic is the same. In the following simulations, the environment is represented by 2-D grid-based maps. The grids are uniform cell squares. Eight grids adjacent to a grid is regarded as its neighbors and edges exist between them. The weight of each edge is equal to the Euclidean distance between the centers of the adjacent neighbor grids. As shown in Fig.1, the weight of edge is equal to 1 unit for grids in the same row or in the same column. For adjacent grids neither in the same row nor in the same column, the weight is Euclidean distance between the centers of the grids.

2 units. It is equal to the

3.1 U-Shape Environment Some methods encounter dead-lock in U-shape environments. The mobile robot drops into local minima and stops at that position instead of going to the target position. To

A Novel Cellular Neural Network and Its Applications in Motion Planning

271

demonstrate the effectiveness of SP-CNN, the proposed model is applied to solve the U-shape problem shown in Fig.1. The neural network is a 50x50 topologically ordered neuronal network. By choosing A = 10, G=2500, at time t=10s we obtain a globally optimal solution (blue solid points) from start (green solid point) to goal (red solid point) shown in Fig.2. After forming steady states, the distance from certain neuron to the target neuron is equal to its state value. The state value of the neuron corresponding to the start position is equal to 60.0416 finally. It is also equal to the distance from the start to the goal position. This can be calculated according to the path shown in Fig.2. The steady states of all neurons with the red solid point as the target neuron are shown in Fig.3. In this figure, we use pseudo color to display the state of each neuron. The deep the color is, the far the neuron is from the target neuron. For neurons with no free path to goal position, the state values are very large. In order to reflect the comparative state values of neurons, they are reduced in this figure without losing dimension difference between neurons. The state values of all neurons form a potential filed, the potential of which is equal to the distance to the target position. Some other neural networks based motion planning approaches also form a potential field when become steady. However, their potential has no precise physical meanings although they can reflect the distance qualitatively.

Fig. 3. The steady state values of all neurons in Fig. 4. Employing heuristic information V.S. the U-shape environment after value reduction no heuristic inforzmation

3.2 Employing Heuristic Information V.S. No Heuristic Information In this simulation, the environment map, the target position and neural network configuration are the same as those of the U-shape environment respectively as shown in Fig.3. In this simulation, we compared situation with prior knowledge with situation without prior knowledge. In concrete, for situation with prior knowledge, the initial state values are chosen as follows: for obstacle neurons, the initial state values are equal to G; for other neurons, the initial state values are equal to their Euclidean distance to the target. For situation without prior knowledge in this simulation, the initial state values are equal to zero. The y-axis in Fig.4 represents ARC as defined in expression (7).

272

Y. Cao et al.

In Fig.4, ARC of the situation with heuristic information drops more rapid than the situation without heuristic information and it takes less time for the former one to achieve steady states. Of course, heuristic information for SP-CNN is not confined to the Euclidean distance from certain neuron to the target neuron. Other heuristic information is also possible. To some extent, the evaluation of the adopted heuristic information depends on the application. The closer the heuristic information is to the final steady states, the lower the time consumption is. No matter what initial state values are, the final steady state for a particular environment are identical. This indicates that if the heuristic information is far from the final steady state values, it takes more time to reach the steady states indicating the shortest path for the network instead of achieving other state values. This is not the same as A* path planning approach for which inaccurate heuristic information may generate a false path.

Fig. 5. The generated path in a labyrinth environment

3.3 Labyrinth Environment The proposed SP-CNN model does not suffer from undesired local-minima even in complicated labyrinth environments as shown in Fig.5. The neural network is a 50 x 50 topologically ordered neuron network. By choosing A = 10, G=2500, at time t=20s we obtain a globally optimal solution (blue solid points) from start (green solid points) to goal (red solid point). The distance of 113.4975 between start and goal is derived according to the state value of the neuron corresponding to the start. From this figure we can see the proposed model find the shortest path in the labyrinth environment.

4 Conclusion In this paper, a novel CNN entitled SP-CNN is proposed. Each neuron in the neural network only has local connection with its neighbors and it has a competitive layer and dynamic layer composed two-layer architecture. Based on the topological structure of the grid-based map, the proposed model is applied to solve real-time motion planning problem in various environments. Theoretical analysis reveals that the model proposed by Willms and Yang [15] is the discrete form of SP-CNN by

A Novel Cellular Neural Network and Its Applications in Motion Planning

choosing A =

273

1 . With SP-CNN, motion planning in workspaces with different dt

traverse cost is able to be solved. Effectiveness of the proposed SP-CNN is illustrated by simulation results.

References 1. Li, S., Meng, M.Q.H.: SP-NN: a novel neural network approach for path planning. Inst. of Intell. Machines, Chinese Acad. of Sci., Hefei (2008) 2. Borenstein, J., Koren, Y.: The vector field histogram-fast obstacle avoidance for mobile robots. IEEE Transactions on Robotics and Automation 7, 278–288 (1991) 3. Murphy, R.: Introduction to AI robotics. MIT Press, Cambridge (2000) 4. Yoram, K., Johann, B.: Potential Field Methods and Their Inherent Limitations for Mobile Robot Navigation. In: Proceedings of the IEEE International Conference on Robot Automation, California, pp. 1398–1404 (1991) 5. Sato, K.: Deadlock-free motion planning using laplace potential field. Advanced Robotics 5, 449–461 (1993) 6. Louste, C., Liegeois, A.: Near optimal robust path planning for mobile robots: the viscous fluid method with friction. Journal of Intelligent and Robotic Systems 27, 99–112 (2000) 7. Singh, L., Wen, J., Stephanou, H.: Motion planning and dynamic control of a linked manipulator using modified magnetic field. In: Proc. IEEE International Conference on Robotics and Automation, Albuquerque, NM, pp. 1142–1147 (1997) 8. Tsourveloudis, N.C., Valavanis, K.P., Hebert, T.: Autonomous vehicle navigation utilizing electrostatic potential fields and fuzzy logic. IEEE Transactions on Robotics and Automation 17, 490–497 (2001) 9. Wong, B., Spetsakis, M.: Scene reconstruction and robot navigation using dynamic fields. Autonomous Robots 8, 71–86 (2000) 10. Siemiatkowska, B.: Cellular Neural Network for Mobile Robot Navigation. In: Proceedings of the Third IEEE International Workshop on Cellular Neural Networks and their Applications, pp. 285–290 (1994) 11. Glasius, R., Komoda, A., Gielen, S.C.A.M.: Neural network dynamics for path planning and obstacle avoidance. Neural Networks 8, 125–133 (1995) 12. Glasius, R., Komoda, A., Gielen, S.C.A.M.: A biologically inspired neural net for trajectory formation and obstacle avoidance. Biological Cybernetics 74, 511–520 (1996) 13. Yang, S.X., Meng, M.: Neural network approaches to dynamic collision-free trajectory generation. IEEE Trans. on Systems, Man, and Cybernetics, Part B 31, 302–318 (2001) 14. Yang, S.X., Meng, M.: An efficient neural network method for real-time motion planning with safety consideration. Robot. Auton. Syst. 32, 115–128 (2000) 15. Willms, R., Yang, S.X.: An efficient dynamic system for real-time robot-path planning. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 36, 755–766 (2006) 16. Chua, L.O., Yang, L.: Cellular neural networks: theory and applications. IEEE Trans. on Circuits and Systems 35, 1257–1290 (1988)

Evaluation of Enterprise ERP System Based on Neural Network Optimized by Ant Colony Hanmei Wang, Dongxiao Niu, and Chengkai Cai Economics and Management, North China Electric Power University Beijing 102206, China [email protected]

Abstract. Based on the basic theory of ERP (Enterprise Resource Planning), this paper summarized a series of indicators which reflected the effect of the ERP system. Then we used BP neural network and ant colony algorithm together to evaluate the effect of the ERP system. After we get the scores of the evaluation of the subsystems, we can find which subsystem is the weakest, so we can improve that subsystem and make the whole system better. The empirical results show that the ant colony network not only has the extensive mapping capability as the neural network, but also has the high-speed and global convergence features as the ant colony algorithm. When we use this model, we can get better accuracy and efficiency. Keywords: ERP system, Ant Colony Algorithm, BP neural network, evaluation.

1 Introduction The ERP system was from America and Europe countries which have the modern industrial manufacturing methods. After 50 years of continuous application, development and innovation, this system becomes a useful method to manage the company. We are in an information age, in order to beat competitors, companies must use advanced manage and technical methods to obtain and utilize information. The ERP system is a useful method which can provide and utilize information. The ERP system becomes more and more popular in modern companies. Many articles focus the methods of evaluating the ERP system. Comprehensive evaluation methods which we commonly use include composite index evaluation method, fuzzy comprehensive evaluation method, multivariate statistical analysis methods, DEA method and so on. With the rise of evolutionary computation study, more and more scholars use evolutionary computation and BP neural network together to solve the evaluation questions. Evolutionary computation has strong global convergence ability and robustness, when we use these two ways together, this new method not only has the extensive mapping capability, but also get more fast convergence speed and more strong learning ability. Ant colony algorithm is a kind of random search optimization algorithm which simulates the behavior of the ant colony. Ant colony algorithm has strong robustness, convergence and parallelism and so on. Therefore, ant colony algorithm can also optimize the neural network [1]. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 275–282. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

276

H. Wang, D. Niu, and C. Cai

Based on the basic theory of ERP, this paper establishes the evaluation system of the ERP system at first. In order to make the results more accurate, we use BP neural network and ant colony algorithm together to train the weights of network. After training, we use this neural network to evaluate an enterprise’s ERP system. The result indicates that this neural network has better accuracy and efficiency.

2 Evaluation Indicators System 2.1 The Concept of ERP In the concept of ERP, we define that the living environment of the enterprise is a supply chain which includes suppliers, enterprises, distribution network and customers which work together. And within the enterprise, the whole system was divided into several subsystems which work together, such as finance department, marketing department, manufacturing department, services and maintenance department, engineering and technical department. In all links, in order to manage and utilize logistics, information flows, capital flows, workflows, the ERP system makes the full use of modern information technology to build information network system. Through continuous improvement of process optimization and reorganization, the ERP can get more accurately date and more availably decision-making behaviors. These results can improve the level of management and final improve economic efficiency and competitiveness of enterprises [2]. 2.2 Determine the Evaluation Indicators System of the ERP System The ERP system in every enterprise has many subsystems; we can get the operational information and application information through the evaluation of each subsystem, then we will know which subsystem is the worst and which part is needed to be improved. We analyze the characteristics of ERP systems, building twelve indicators of decision evaluation to each subsystem. The indicators are as follows: (1) reliability of the system; (2) scalability of the system; (3) maintainability of the System; (4) security confidentiality of the system; (5) advanced level of the system; (6) economy level of the system; (7) sharing capacity of the system; (8) availability of the system; (9) system-friendly; (10) system user satisfaction; (11) friendship degree of the system; (12) effectiveness of the system. During the evaluation process, these twelve indicators are not independent; they have some inner-relationships, we must consider them in whole [3].

3 Ant Colony Neural Network 3.1 Ant Colony Algorithm Ant colony algorithm (ACA) is a bionic optimization algorithm which can simulate the foraging behavior of ant communities. This algorithm uses a parallel self-catalytic mechanism of positive feedback, and has strong robustness, good distributed computer system, and is easy to combine with other algorithms. It has demonstrated its excellence in the aspects of complex optimization and intelligent search.

Evaluation of Enterprise ERP System Based on Neural Network Optimized

277

The basic idea of the algorithm is: Assume that the number of parameter “θ” in the neural network is n. First of all, sort the parameters, set them as “θ1, θ2, ..., θn”. As for the parameter “θi” (1 ≤ i ≤ n) , set it as some random values whose number is “N”, and then the collection “Iiθ” is established. Each ant starts from the first collection, according to the state of the pheromone of each element, randomly selects an element from each collection “Iiθ”, and accordingly adjusts the pheromone of the selected elements. When the ants complete all the choices of the elements of all collections, it reaches the food source. That is, they have selected a set of network parameters in all the collections. Next the ants will return to the nest along the original path. At the same time, the ants will adjust the pheromone of the selected elements of the selected elements. Repeat the above steps. When all the ants converge to the same path, or achieve a given iteration number, the search will end, which means that the optimal solution of the network parameters has been found. The main steps of the ant gray forecasting model are as follows: Step 1: Initialization: consider the pheromone of the element “j” in collection “Iiθ”(1 ≤ i ≤ n-1) as “τj (Iiθ)(t)”. Make τj (Iiθ)(t) =C (1 ≤ j ≤ N) and the largest cycles is NCmax, and the number of ants is m. All the ants are in the nest. Step 2: Start all the ants. Starting from the first Collection, each ant chooses an element in turn from each Collection in accordance with the following rules. In the end, the selected elements can be considered as a set of network parameters. Path selection rules: as for the collection “Iiθ”, consider each ant as “k” (k =1, 2, …, m). Each ant selects its element “j” randomly according to the random probability in following formula. This process will be end until all the ants reach food source.

Pr ob(τ k j ( Iθ i )) = [τ j ( Iθ i )] /

N ∑

τ u ( Iθ i )

u =1

(1)

Step 3: When all the ants have selected an element in each collection and returned to the nest along the original path, the pheromone of the selected elements will be adjusted accordingly according to the following formula. Consider the number of time units that the process have cost as “n”.

τ j ( Iθ i )(t + n) = ρτ j ( Iθ i )(t ) + Δτ j ( Iθ i )

(2)

In the above formula, the parameter “ ρ ” (0 ≤ ρ Tij [ n] ⎧1, Yij [ n] = ⎨ otherwise ⎩0, Tij [ n] = exp( −α T )Tij [ n − 1] + γ T ∑ Ykl ( n − 1)

(1)

For image processing, the the original PCNN model can be simplified. Usually we let Fij [ n ] = I ij [ n ] Lij [ n ] = V L ∑ Wij kl Ykl ( n − 1) Lij

(2)

Each pixel in input image is treated as a neuron, and its intensity is an external stimulation input Iij. Suppose that the initial state of all neurons is zero, the internal activity Uij is equal to the external stimulation input Iij at first iteration. The threshold Tij of all neuron begin to attenuate from initial state, and one neuron will fire (actively fire), i.e. output a pulse, while its threshold decreases to less than or equal to Uij. Meanwhile, the threshold Tij of this neuron rapidly increases, and the pulse will stop output. Then Tij begin to attenuate and the pulse generates again, while Tij decrease to less than or equate to Uij again. Neurons output a serial of signals by repeating the above steps. In pulse generation, the fired neuron stimulate neighboring neurons to be fired (passively fire) by linking action (Lij). So if a neurons is activated, those adjacent pixels having similar intensity will be activated too. This is an important property of PCNN which we can use to segment image. 2.2 DLPFPCNN Model Structure At iteration of the conventional PCNN model, if a pulse of some neuron fire naturally, the other neurons captured by this neuron fire only at next iteration. In this way, the pulses of the nature fire neuron and fired by capturing can’t burst at the same time. This will result in appearing discontinuous areas or isolated noises in segmented image, which it is not anticipant for us. In order to solve the problem of segmenting image with low contrast, low SNR and uniform gently changing background, a DLPFPCNN model, which is presented in literature [8], is used in this paper. The first level is used to enhance image by improved PCNN model, and the second level uses the novel parallelized firing PCNN model to segment the enhanced image. Finally, certain criterion is used to determine the optimal segmentation result. The system structure diagram is shown in fig. 2.

314

Z. Peng et al.

Fig. 2. The flow chart of the proposed algorithm based on DLPFPCNN model

3 Water Area Segmentation Using DLPFPCNN The DLPFPCNN model is proposed for image segmentation under the condition of low contrast, low SNR and uniform slowly varying background or object gray. Based on the water area characteristics of remote sensing image: In remote sensing images, there is a large easily identifiable water area, the gray value of which is low, uniform distributed (MSE is small), and usually vary slowly, we can see that the new model is suitable for water area segmentation of remote sensing image. In fig.3 (a) we can see a primary SAR remote sensing image, which contains a bridge object. Fig. 3 (b) and (c), respectively, after the first stage to enhance the results of and parallel ignition PCNN segmentation results. Various parameters in experiments are defined as:Δ 0.01; N 100;β 0.2. The results of remote sensing image water area segmentation in another scene are shown in fig. 4(a)-(c). Various parameters in experiments are defined as:Δ 0.3; N 10;β 0.2. From the fig.3, we can see that in the image enhanced by improved PCNN model, the contrast has been greatly enhanced, the visual effects of the image has been significantly improved, the contrast has been enhanced, so that the image is more layered, and the problem of remote sensing image with low-contrast image is solved, which will help the follow-up division. In the result after the iteration of new PFPCNN model, the pixels within the scope of the different gray-scale are distributed in different iterations results; the pixels with similar gray and spatial location tends to fire synchronously, and the automatic wave transmit fully, so that area with slow varying gray can fires synchronously. In this way, the adjacent water areas with similar gray will fire synchronously in some iterative process.

＝

＝

＝

＝

＝

＝

Bridge Detection and Recognition in Remote Sensing SAR Images

(a)

(b)

315

(c)

Fig. 3. The results of remote sensing image water area segmentation

(a)

(b)

(c)

Fig. 4. The results remote sensing image water area segmentation in another scene

4 The Sequential Processing of Bridge Recognition 4.1 Determining the Optimal Results Using Minimum Class Variance Criterion For the problem that the best results are difficult to automatically determined, common solutions are the maximum Shannon entropy criterions, the minimum crossentropy criterion, the maximum number of edges criterion, and the maximum between-class variance criterion and so on. However, these criterions have a certain limitations, and they are not suitable for the water area segmentation of remote sensing image. In this case, except the gray uniform distribution of water area, the distributions of other parts are complicated. At this time, the largest between-class variance criterion is not suitable because the distributions of water area parts are complicated and its variance is not necessarily large, so the between-class variance is not necessarily large; The entropy-based criterion is not suitable because the distributions of water area are uniform and the entropy of the result is not necessarily large, and the cross-entropy is not necessarily small; the criterion based on the number of edges is also not suitable because the distributions of the background are complicated, and there will certainly be a lot of edge details, while the edge details of water area are less, which is contrary to our expectation. We presents a method based on the minimum within-class variance criterion to determine the best iteration results for the characteristics of the remote sensing image water area, the gray value of which is low, uniform distributed (MSE is small). This method will calculate and compare the variance of each class (background and the water area) of every iteration result. Then the iteration result with the minimum

316

Z. Peng et al.

within-class variance is the optimal segmentation result, while the result with the minimum variance will be the water area after segmentation. The formula to calculate variance is as follows, σ2 =

1 ∑ (I − μ )2 M −1 M

(3)

where, σ2 is the variance, M is the number of pixels contained in the class, I is the gray value of the pixels contained in the class, μis the mean gray value of the class. It is able to get very good result to use the minimum within-class variance criterion. The experimental results are shown in fig. 5.

(a)

(b)

Fig. 5. The optimal iteration results determined by the class variance, (a) The results of fig. 3, (b) the results of fig.4

Seeing from the fig.5, it is obvious that the smallest within-class variance criterion can accurately determine the best segmentation result from a large number of iteration results. The water area segmentation determined by the best result from this criterion is complete with less clutter interference; and the bridge outline is clear. It keeps the linear character of the bridge edge and rules character of water area outline well, which makes good preparation for the follow-up bridges recognition and achieves satisfactory results. The validity of the new DLPCNN model proposed in this paper will be proved, and the model is also very suitable for remote sensing image segmentation. 4.2 Morphological Clustering The purpose of morphological clustering is to remove the small non-water area with interference, while to prepare for tracing the outline of the next step. Based on the most nearest criterion, it clusters the neighboring pixels into one category, and assigns different categories with different gray values (for example, the first category with 250, the second 230, and so on, to distinguish between different categories), and at the same time it will calculate the number of pixels in each category and then remove the category with a small number of pixels in order to achieve the purpose of removing regions with interference. After the corrosion of the previous step, the area of the region with interference will become smaller and distributed discretely. However the water area is continuous with a large area. So the category got at last is category with water area region. The results are shown in fig. 6.

Bridge Detection and Recognition in Remote Sensing SAR Images

(a)

(b)

(c)

317

(d)

Fig. 6. The results of morphological clustering

4.3 Rough Detection of Bridge Outline The purpose of this phase is to detect the suspected part with straight line outline of the bridge and to receive the general location of the bridge. The detection process is shown as fig.7.

Fig. 7. Rough detection process of bridge outline

⑴ Remove the pseudo-objects which are of irregular shape

In the case of relatively strong interference, after the processing, there might be some pseudo-objectives which are of irregular shape and have some certain area in the result of water segmentation. As shown in fig. 8. The priori knowledge: the outline of the water area is much regular, and has a higher perimeter area ratio. So we can remove the pseudo-objects according to perimeter area ratio. The parts which has lower perimeter area ratio will be eliminated. As shown in fig. 8(b).

False objects

(a)

(b)

Fig. 8. Remove the pseudo-object with irregular shape

⑵ Contour tracking

The priori knowledge: two sides of the bridge are water area. It is apparently the edge line of the bridge is a part of the water area. So if we detecting the edge of the water

318

Z. Peng et al.

area which had been segmented, we will get the edge lines consist of the bridge and water area. As the bridge is surely located in the range of the water area, the bridge detecting work on the water in this article will be based on it. First of all, we will detect the outline of the water area. Using the common contour tracking method based on chain code to track contour. Separately do the contour tracking for each category of the clustering result of the last step. Meanwhile prepare for the algorithm of Straight line detection in the next step. The processing of tracking: starting from the track point, then searching for the next boundary point step by step and putting it into the chain code list. At the same time, setting this point as the current boundary point, and tracking the next one until the end. The tracking results as shown in fig. 9.

(a)

(b)

Fig. 9. Detecting results of the water area outline

⑶ Straight line detecting

According to the priori knowledge, bridges over the water usually have liner edge. So we can preliminary identify the location of the bridge by detecting straight line in the water area. At the same time, we eliminate parts of the pseudo bridge edge to reduce amount of calculation in the bridge detecting and recognition algorithm. The most used and traditional straight-line detection algorithm is based on the Hough transform. But the main problems of it are the calculation is complex and huge, not to mention its long process time. Here, we will use a new kind of straight-line detection algorithm, which is simple and useful, based on chain code. As follows, 1) Determining whether the total length of the chain code in the current chain code string meet the minimum straight-line distance constraint or not. If the total length of the chain code is less than the minimum threshold of the pre-set length, there will not be any line meet the requirement in the chain code. And it will be discarded. 2) Testing the chain code string with straight-line segment similarity by paragraph. Selecting sub-chain code string which meeting the minimum line segment length from the starting point. Meanwhile, calculate the real length of the sub-chain code string and the ideal straight distance between two endpoints. Calculate the sub-chain string’s approximate degree of straight line according to equation (4), S=

Ps − Pe Len( Ps − Pe)

(4)

Bridge Detection and Recognition in Remote Sensing SAR Images

‖

319

‖

where, Ps-Pe is the ideal straight-line distance between Ps and Pe, Len (Ps,Pe) is the actual pixel length from Ps to Pe, S is the straight line approximation degree. Obviously, the more the value of S is, the more it can be said as a straight line. If the value of S is less than the preset threshold St, it will not be regarded as a straight line. When calculating the actual length of the chain code string, the actual length between two pixels which connected by the chain code is 1 if the direction code will be 0, 2, 4, 6; and the actual length between two pixels is 2 if the direction code will be 1, 3, 5, 7. This algorithm can quickly detect a straight line and perfectly meet the requirement of this article.

⑷

Bridge edge rough detection based on gradient. Contour retains some very important parts of the information of the original image, and reduces a lot of the total amount of data, which meet the requirements of feature extraction. In this article, the contour tracking and straight line detection reduce the amount of the calculation. To locate the bridge line, based on straight line detection results, our next job is to get rid of the bank edge from the water area edge and retain the non-bank edge which includes all the bridge edge.

Fig. 10. The final detecting results of the bridge

According to the priori knowledge, both sides of the bridge edge along gradient direction is the water, however one side of the bank edge along gradient direction is water, the other side is land. After the water area segmentation, the binary image has follow properties: there is white area on both sides of the bridge, however refer to the bank, one side is white, the other is black. Then we distinguish bridge from bank through using their vertical line, that is, the gradation on both sides of the bank's vertical line is different, however the gradation on both sides of the bridge's vertical line is the same. According to the priori knowledge of the width of the bridge, for the 10-meter resolution remote sensing images, considering that the average width of bridge is between 50-100 meters, which means the average width of bridge is 5-10 pixels, the length of the gradation should be more than 20 pixels so as to ensure the two ends of the gradation lie in the water area. So, along the gradient direction, select two judgments point's abscissa as follows,

x judge = xm ± 20 × cos θ y judge = kGred ( x judge − xm ) + ym

(5)

320

Z. Peng et al.

where, θ is the included angle between the gradient direction and x-axis positive direction, which can be calculated by slope kGred, then we can get point's ordinate yjudge. For a boundary point, if the pixel gray value of two endpoints of the gradation is the same, then we look it as a boundary point of the bridge. Otherwise, we look it as a boundary point of the bank, so we can eliminate it. The detection results are shown as follows in fig.10. The remains are basically the top edge of the bridge. Considering there might be some noise and pseudo-bridge edge remains, we will do the coarse detection for the bridge boundary points. If we want to get the accurate location of the bridge, we should get rid of these pseudo-boundaries.

5 Conclusions This paper presents a new methods based on DLPFPCNN model and combined with the priori knowledge of the bridge on the water, by analyzing the complexity of the bridge object detection and recognition in remote sensing SAR images. The paper described its principles and work flow in detail, and made a simulation experiment for two typical remote sensing SAR images of the bridge scene, which achieved good application results and demonstrate the effectiveness of the method. Acknowledgments. This work is supported by National Natural Science Foundation of China (40874066, 40839905), Research Fund for the Doctoral Program of Higher Education of China (20070614016), National Defense Pre-research Foundation of China (9140A01060108DZ02).

References 1. Hou, B., Li, Y., Jiao, L.: Segmentation and recognition of bridges in high resolution SAR images. IEEE Trans. on Neural Networks 12, 411–417 (2000) 2. Eckhorn, R., Reitboeck, H.J., Arndt, M., et al.: Feature linking via synchronization among distributed assemblies: simulation of results from cat cortex. Neural Computa. 2, 293–307 (1990) 3. John, L.J., Mary, L.P.: PCNN models and applications. IEEE Transactions on Neural Networks 10, 480–498 (1999) 4. Jcaufield, H., Kinser, J.M.: Finding shortest path in the shortest time using PCNN’s. IEEE Transactions on Neural Networks 10, 604–606 (1999) 5. Kuntimad, G., Ranganath, H.S.: Perfect image segmentation using pulse coupled neural networks. IEEE Transactions on Neural Networks 10, 591–597 (1999) 6. Liu, X., Wang, D.L.: Range image segmentation using a relaxation oscillator networks. IEEE Transactions on Neural Networks 10, 564–573 (1999) 7. Peng, Z., Jiang, B., Wang, H.: Aerial targets detection using improved ULPCNN combining with contour tracking. In: 2nd International Symposium on Photoeletronic Detection and Imaging: Image processing. Proc. SPIE, vol. 6623, Beijing, p. 66230D (2007) 8. Jiang, B., Peng, Z., Wang, H.: Image segmentation based on double-level and parallelized firing PCNN in complex environment. In: Electronic Imaging and Multimedia Technology. Proc. SPIE, vol. 6833, Beijing, p. 68331z (2007)

Approaches to Robotic Vision Control Using Image Pointing Recognition Techniques Tian-Ding Chen Institute of Information and Electronics Engineering, Zhejiang Gongshang University, Hangzhou 310018 [email protected]

Abstract. Intelligent robot human-machine interactive technology will be incorporated into our daily lives and industrial production. This paper presents an autonomous mobile robot control system for human-machine interaction, The use of computer vision, 3D reconstruction of the two-dimensional image information of a specific coordinate point to point system. The use of a known point to the appearance characteristics of objects, by vision recognition algorithm, the original color image data for target screening and recognition. Allows users to easily through simple body movements, issuing commands to the robot. And by support vector machine(SVG) to classify non-linear non-separable type of data, accept user input and recognition actions to improve the robot vision system for target identification accuracy, and thus to achieve the goal of human-computer interaction. Keywords: Human-machine interaction, Computer vision, SVG, Robot control.

1 Introduction In recent years, many studies to explore how to use a number of actions on the human body to communicate with the computer. These actions include facial expressions, body movements and pointing recognition (Gesture). One point recognition is a very natural and common way of communication, it has become an important research areas and hot spots. The study in the Human-Computer Interaction (Human-Computer Interaction) field occupies a very important position. Recently, many with the pointing recognition related research works, most of the medical achievements, such as sign language recognition, care systems, which aims to make it clear that the needs of patients. Some scholars combine pointing recognition in human and robot interaction, in order to sign the order to the robot, the robot can be performed to know what kind of gesture action [1], to achieve the purpose of interaction, but relatively simple interactive object . Literature [2] used Adaboost algorithm to find the hand characteristics as a basis for recognition, and then compared with the previous samples are taken right to complete the identification. Similarly, the study of use the sample compared also includes the literature [3], by the morphology of the ways to find the outline of the hand, and then the localized contour sequence compared samples of the methods to distinguish the different gestures, literature [2] [3] Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 321–328. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

322

T.-D. Chen

disadvantage is that large learning training load. The literature [4] used moment invariants calculation the characteristics of gestures as neural network input vector, training a large number of samples to improve the recognition rate, but the large amount of data processing, real-time poor. In addition, a single camera is different from general research to identify the literature [5] use of two cameras able to complete the 3D gesture recognition, so that the recognition process is no longer limited to flat. Literature [6] [7] are detected by laser scanning the finger number of hand gestures and determine its gestures, their difference is [6] can only be right to judge nonarm gestures, while [7] can be identified with the arm gestures, but being unable to remove the arm information, if the arm image too much, it will affect the recognition rate. Literature [8] [9] in chronological determine the relative position of hands and face to determine what kind of dynamic hand gestures, the disadvantage is susceptible to false of justice resulting from the shadowing effects.

2 Object Tracking and Feature Extraction This paper develops an efficient real-time pointing region tracking and extraction method to track the moving pointing object and extract the feature. In order to getting this goal, the system uses two processes. The first process extract the stationary pointing object shape, whereas, the second process proposed multi-threshold and threshold of self-ajust mechanisms in place used a single threshold value of color screening in the past. So, a smart tracking theorem on the single camera to reduce the processing steps by using epipolar geometry. The environment of work space is complex because there are many other objects with different colors. In the beginning, there are many colors in the image, and the computer is hard to understand the concepts of each color. Therefore, some filters are used to let the image be transferred from RGB to gray, and then binary images. In order to assist the computer analysis, system should transfer the color image to binary image. In order to get the binary value, this step is to mapping the RGB color space to YIQ color space to get gray image. In YIQ color model, the value Y is more important than I and Q in this gray step. The transformation from the RGB color space to the YIQ color space is:

Y = 0.299 R + 0.587G + 0.114 B

(1)

After this step, a gray image is obtained. In order to separate objects from the whole image, the information of the gray level should be obtained. Therefore, the histogram is quoted to calculate the number of each gray level. Then a threshold is chosen to filter to background and the shape of object is obtained. This system can get the binary image by this step. Through the histogram and binary step, the image preprocessing would be finish in order to use other image filter to analyze this system. Then, the color image would be transfer red to gray by (1). And finally, through the threshold value, the binary image can be obtained.

Approaches to Robotic Vision Control Using Image Pointing Recognition Techniques

323

2.1 Extraction of Hand Image Extracted image format are RGB color space, image size is 320 × 240 pixels. Each image pixel contains three bytes, each byte representing the red (R), green (G), blue (B), each color intensity ranges from 0 ~ 255, R , G, B values were 0 for black, R, G, B values are for the 255 is white. In gesture recognition, how to find out hand from the background is very important. The use of skin color to distinguish between hand and background, often because of light or skin color differences of individual users, resulting in the extracted image is not satisfactory, the results of the extracted hand image to judge good or bad a great impact on the system. The system as background subtraction to find the input image and background differences, with CamShift algorithm to remove the body or head, and other area to extract the hand image. 2.2 Image Processing and Wrist to Remove The input image and the background has been established compared to determine the output of a binary image. Background separation method by section 2.1, the threshold T set to 60. If the input image I(x, y) and the background image B(x, y) subtract the result is less than the threshold T, is that the background. The output image O(x, y) pixel set to 0 (dark point), otherwise the image will be the output O(x, y) pixel set to 255 (bright spot). Removal of the background, the images still have a lot of noise, and image of the main itself, the existence of broken or slit, through morphological closing operation to eliminate the image of the main break or slit, through the disconnect operation to eliminate the image noise in the smaller point and make the image more smooth. The input image after background subtraction and noise removal, the obtained binary image with CamShift hand tracking, may be that the image contains an oval-shaped hand tracking results, in accordance with the scope of follow-up results will be applied to an oval-shaped binary image , the labeling algorithm with the largest block out as a true hand shape. Camshift available from the tracking object tilt angle, use of tilt angle normalization object to identify the length of the image of the most wide w, and the most wide-length center coordinates (Cy, Cx), The length of the most wide-coordinates of the center coordinates as a reference to formula (2) to remove wrist, The main purpose is to remove the wrist to keep only part of the palm and fingers, wrist or arm to reduce the impact of location on the fingers.

I ( x, y ) = 0

if

y > Cy + w/ 2

(2)

I(x, y): palm image; y: palm pixel y-coordinate; w: palm most wide-length; Cy: palm most wide-length center y coordinates. 2.3 Finger Search The hand image to the border search cycle of the sequence obtained coordinates of a set of hand contour {P0 , P1 , … Pn } ,P0 is outline of the lower left hand pixel coordinates, Clockwise direction along the contours of the split into k segment, each segment component by x pixels. The contours of the new set of coordinates {P0 , Px , P2 x … Pn } . Calculate the angle of the horizontal direction of each segment.

324

T.-D. Chen

θ i = tan −1 (

P( i +1) x ( y ) − Pix ( y ) P(i +1) x ( x) − Pix ( x)

)

i = 0,

k −1

(3)

Availability of k segment of their point of view, the following steps to determine the fingers:

θ start = θ i ; (2). determine whether i is equal to k-1, yes the end, other= θ i . Comparison of the angle difference between θ end and θ start , if

(1). set i=0, wise

θ end

θ end − θ start

point of difference is greater than 180o, expressed through the fingers,

turning in a clockwise direction, judged as a finger; (3). reset θ start

= θ i , continue go

to (2). 2.4 Epipolar Constraint Model In stereo vision, in order to be able to solve the image correspond to the issues, one established a number of constraints to reduce errors in matching corresponding points, such as the epipolar constraint, consistency constraints and uniqueness constraints and so on. These obtain an accurate match for the final results provide a strong guarantee. Here, the epipolar constraint matching algorithm to achieve dimensionality reduction processing reduces the data search space, as shown in Fig. 1.

Fig. 1. Epipolar geometry model

3 Support Vector Machine Classifier Support vector machine (SVM) was proposed by Vapnik in 1995 [10], not until 1998 did SVM became famous for its outstanding accuracy and performance on pattern

Approaches to Robotic Vision Control Using Image Pointing Recognition Techniques

325

recognition. Henceforward, SVM has been acknowledged a potential and powerful tool for classification problem and applied to different kinds of domain. This paper will use the SVM binary image of the palm for classification, the six kinds of gestures (0 to 5) through different angles and size of the record over and use SVM to find the Optimal Separation Hyperplane(OSH), re-use of this OSH on the test data set classification. Definition: Primal Optimization Problem Set objective function as f (Z ) , inequation and equation constraint condition as

g i (ω ) , i =1,…,k and h j (ω ) , j =1,…,m, function is defined in Ω ∈ ℜ n , Its primal optimization problem can be written as (4)

f (ω )

Minimize subject

to

ω ∈Ω

g i (ω ) ≤ 0 h j (ω ) ≤ 0

i = 1, … k

(4)

j = 1, … m

Z formed by a set is called the feasible region, while the optimal ∗ solution ω is the global minimum, it meets f (ω ) ≤ f (ω ) .

All satisfy (2) ∗

In recognition field, if appropriate feature extraction of data to identify a good degree, you can through the characteristic value of the distance between the space as a basis for classification.Therefore, suppose that for each n characteristic values, the data description of the use of n-dimensional space coordinates, you can find an optimal separation plane of the data do distinguish, while the best definition lies in the separation plane and the training data point distance (D) the largest. Shown in Fig. 2, Fig. 2 (a) and (b) are for the data point and the separation plane (black solid line) of the diagram, but Figure (b), the separation distance from the plane and the data points as the greatest, so for unknown data, classification of two categories are to maintain the largest boundaries(margin), so have a higher success rate of classification.

(a)

(b)

Fig. 2. Data points and the separation of the plane diagram

4 Experiment and Valuation Experimental platform shown in Fig. 3, this paper design of human-machine interaction robot based on computer controlled. In the visual part of the robot using monocular

326

T.-D. Chen

vision of the CCD camera. Robot system camera capture user's gestures images carry to computer, after the visual recognition system to handle, got the category of gesture. A total of six kinds of gesture form, shown in Fig. 4, these gestures correspond to the robot movement pattern are: (1) in situ circular motion, (2) straight, (3) back, (4) stop, (5) turn left, (6) turn right. Table 1 is an interactive recognition accuracy, Fig. 5 is based on the results of gesture recognition guiding.

Fig. 3. Experimental robot platform

(1) circular motion in same place

(4) stop

(2) go straight

(5) turn left

(3) back up

(6) turn right

Fig. 4. Total kinds of gesture category Table 1. Interactive recognition result

gesture category (1) (2) (3) (4) (5) (6)

recognition accuracy 0.901 0.912 0.905 0.930 0.906 0.906

Approaches to Robotic Vision Control Using Image Pointing Recognition Techniques

327

Fig. 5. Based gestures guiding result

5 Conclusions In this paper, we develop and put into practice a set of real-time computer vision human-machine interactive system. Tracking point objects, we use to the object color, shape and motion from image analysis of such features to the object. Focus of the study is to use the concept of SVM allows users to easily communicate with the computer to reach a friendly human-machine interface program, and develop a pointbased object tracking and reconstruction of physical characteristics, it can be effective in detecting the user's behavior. Experimental design a set of actions based on replaceable module control components to complete the system scalability. In the robot's human-computer interaction mechanisms, the current system to identify scope of the directive is only limited to six kinds of gestures, hope that the future can increase the recognition type, the application of the visual recognition system more widely.

References 1. Tseng, K.T., Huang, W.F., Wu, C.H.: Vision-based Finger Guessing Game in Human Machine Interaction. In: IEEE International Conference on Robotics and Biomimetics, pp. 619–624. IEEE Press, Kunming (2006) 2. Wagner, S., Alefs, B., Picus, C.: Framework for a Portable Gesture Interface. In: 7th International Conference on Automatic Face and Gesture Recognition, pp. 275–280. IEEE Press, Southampton (2006) 3. Gupta, L., Suwei, M.: Gesture-based Interaction and Communication: Automated Classification of Hand Gesture Contours. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 31, 114–120 (2001) 4. Premaratne, P., Nguyen, Q.: Consumer Electronics Control System Based on Hand Gesture Moment Invariants. IET on Computer Vision 1, 35–41 (2007) 5. Abe, K., Saito, H., Ozawa, S.: Virtual 3-D Interface System via Hand Motion Recognition From Two Cameras. IEEE Transactions on Systems, Man and Cybernetics, Part A 32, 536–540 (2002) 6. Holden, E.J., Owens, R.: Recognizing Moving Hand Shapes. In: 12th International Conference on Image Analysis and Processing, pp. 14–19. IEEE Press, Halmstad (2003) 7. Yin, X., Zhu, X.: Hand Posture Recognition in Gesture-Based Human-Robot Interaction. In: 1st IEEE Conference on Industrial Electronics and Applications, pp. 1–6. IEEE Press, Singapore (2006)

328

T.-D. Chen

8. Kim, K.K., Kwak, K.C., Chi, S.Y.: Gesture Analysis for Human-Robot Interaction. In: 8th IEEE Conference on Advanced Communication Technology, pp. 1824–1827. IEEE Press, Korea (2006) 9. Wu, A., Shah, M., Da Vitoria Lobo, N.: A virtual 3D Blackboard: 3D Finger Tracking Using a Single Camera. In: 4th IEEE Conference on Automatic Face and Gesture Recognition, pp. 536–543. IEEE Press, Grenoble (2000) 10. Vapnik, V.: The Nature of Statistical Learning Theory. Springer, New York (1995)

A Hybrid Evolutionary Approach to Band Selection for Hyperspectral Image Classification Hao Wu1, Jiali Zhu1, Shijin Li1,*, Dingsheng Wan1, and Lin Lin2 1

School of Computer and Information Engineering, HoHai University, Nanjing, 210098 [email protected] 2 Network Information Center, Institute of Water Resources and Hydropower Research, Beijing, 100038, China

Abstract. With the development of the remote-sensing imaging technology, there are more and more applications of hyperspectral image classification tasks, in which to select a minimal and effective subset from a mass of bands is the key issue. This paper put forward a novel band selection strategy based on conditional mutual information between adjacent bands and branch and bound algorithm for the high correlation between the bands. In addition, genetic algorithm and support vector machine are employed to search for the best band combination. Experimental results on two benchmark data set have shown that this approach is competitive and robust. Keywords: Hyperspectral Remote Sensing, Band Selection, Conditional Mutual Information, Support Vector Machine, Genetic Algorithm, Branch and Bound Algorithm.

1 Introduction With the development of the remote-sensing imaging technology, the use of hyperspectral image is becoming more and more widespread. Due to the dense sampling of spectral signatures of land covers, hyperspectral images have a better discrimination among similar ground cover classes than traditional multispectral scanners [1]. At the same time, these images are usually composed of tens of or hundreds of spectral bands with high redundancy and great amount of computation in hyperspectral image classification. Therefore the most important and urgent issue is how to reduce the number of those bands largely with little loss of information or classification accuracy. The commonly-used dimension reduction methods fall into feature selection and feature extraction. Because every band of the hyperspectral data has its own corresponding image, the way of feature extraction could not keep the original physical interpretation of the image. Thus these approaches are not suitable for that purpose. With the close spectral distance between the two adjacent bands in the hyperspectral data [2], there is a considerable redundancy, which should be largely *

Corresponding author.

Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 329–336. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

330

H. Wu et al.

reduced by the feature selection or band selection methods so that the classification efficiency and accuracy can be greatly improved. In fact, the band selection of hyperspectral remote sensing images is a very complex problem in combinatorial optimization, as some bands with less information may play an important role in the classification. For such a problem, the more effective method to obtain optimal subset of bands is based on the search strategy and evaluation criteria. In this paper, the search strategy combining the support vector machine and genetic algorithm (GASVM) is adopted. In the literature of feature selection, there are many search algorithms. However, for the hyperspectral image, there are hundreds of bands and the efficiency of search algorithms will deteriorate greatly. Serpico and Bruzzone [3] proposed a new suboptimal search strategy suitable for hyper-dimensional feature selection. Serpico and Moser [4] proposed a method to extract spectral channels of variable bandwidths and spectral positions. Because selection based on retaining features with high associated mutual information (MI) values may be problematic when the features were highly correlated, Guo et al. [5] proposed a fast feature-selection scheme based on a ‘greedy’ optimization strategy. Huang and He [6] presented a feature weighting method for band selection based on the pairwise separability criterion and matrix coefficients analysis. Another method is proposed by Wang et al. [7], that all the bands are separated into subspace according to the correlation between two adjacent ones, and select the bands with the best sensitivity of the support vector machine (SVM) from every subspace. According to Peng et al. [8], it has been recognized that the combination of individually good features do not necessarily lead to good classification performance. Sotoca and Pla [9] presented a band selection method using correlation among bands based on MI. Our method differs from the previous works [5, 7, 9] in three aspects. First, conditional mutual information (CMI) is employed to partition the bands into subspace. Second, GA-SVM is adopted to search for the optimal band combination. Lastly, the branch and bound search algorithm (BB) is used to prune the irrelevant bands from the result of GA-SVM to obtain the minimal band set for the classification task. Our system can not only reduce the search space of the genetic algorithm (GA) and the computational complexity, but also avoid the error resulting from elimination based only on the information of a single band. In this way, we can further reduce the irrelevant and ineffective band groups by BB so that the accuracy of classification will be improved. The rest of this paper is organized as follows: Section 2 presents the band grouping method based on CMI. Then Section 3 provides a review of GA-SVM used in this paper and Section 4 introduces the BB algorithm. The experimental results and comparison with other state-of-the-art methods are presented in Section 5. Finally, conclusions are drawn in Section 6.

2 Initial Band Grouping Based on Conditional Mutual Information The entropy is a frequently-used evaluation criterion of feature selection [10]. The mutual information is usually used to measure the correlation between two random variables and it is defined as [11]:

A Hybrid Evolutionary Approach to Band Selection

I(X, Y) = H(X) + H(Y) − H(X, Y)

331

(1)

If we assume that two features are represented with two discrete variables X and Y and the class is represented with a discrete variables C, when Y is given the conditional mutual information of X and C is defined as:

I(C, X | Y) = H(X|Y) − H(X | C, Y)

(2)

For the task of hyperspectral image classification, the basic principles of the band grouping is that the adjacent bands which have high correlation should be merged into one group and the ones with little redundancy should be divided into different groups. Then we can select the representative band from each group. In this way, the loss of useful information can be minimized and the reduction of redundant information can be maximized. Here we take Washington DC Mall data set with 191 valid bands as an example to illustrate this band grouping method. There are 191 bands in this data set after discarding those water absorption bands. Fig. 1 shows the conditional MI between the adjacent bands. 0.4 0.35 0.3 0.25 0.2 0.15 0.1 0.05 0 1

11 21 31 41 51 61 71 81 91 101 111 121 131 141 151 161 171 181 191

Fig. 1. The Conditional MI of Washington DC Mall data set

According to the CMI shown in Fig.1, the bands are grouped into 8 groups, as Band 1 to 37, 38 to 56, 57 to 72, 73 to 86, 87 to 102, 103 to 133, 134 to 140 and 141 to 191.After that, we divide the first band group into two groups and the eighth group into three in order to avoid omitting the important information, as there are too many bands in these two groups. The additional band grouping result is Band 1 to 18, 19 to 37, 38 to 56, 57 to 72, 73 to 86, 87 to 102, 103 to 133, 134 to 140, 141 to 158, 158 to 175 and 176 to 191. Because of the pruning by BB in the following section, there is no redundant band group finally.

3 Search Algorithm Based on the Combination of SVM and GA The search space of traditional band selection algorithm based on GA is too large to search because it is the total band space. In this paper, the search space is much smaller than the traditional method because we choose only one band from each band

332

H. Wu et al.

group which is organized as before. As stated in [12], simple GA sometimes provides solutions inferior or just comparable to classical heuristic algorithms. A practical and effective way of overcoming this limitation is to hybridize the GA, by incorporating domain-specific knowledge. In this paper, the special characteristics of hyperspectral bands can be regarded as the domain knowledge, and we have divided the whole mass of bands into several groups by the analysis of the corresponding CMI. Assume there are N bands in the hyperspectral image, which should be divided into K groups. Here we do not choose the commonly-used binary coding method as the chromosome coding mode. If we adopt that method, we are not able to make sure that there is one and only one band from each band group in the band combination after the mutation and crossing. Accordingly, we use the integer coding mode in our algorithm and define the corresponding mutation operator and crossover operator to satisfy the requirement of band selection in the context of band grouping beforehand. In addition, another key of GA is the design of fitness function. The traditional band selection algorithm based on GA requires the designers to pay attention to not only the accuracy of classification, but also the size of band subset. In our method, the group number K is the final band number. Hence, we use only the classification accuracy as the fitness function. In this paper, the adopted classifier is SVM, as it is one of the most competitive classifiers for small samples problem. Since it is very common nowadays, we omit the principles in the text and use the LibSVM [13] implementation.

4 Band Pruning with the Branch and Bound Algorithm In regard of feature redundancy, band grouping can handle the redundancy among adjacent bands, and GA-SVM is used to search for the relatively best combination of those irredundant bands from each group. We don’t know whether they are the fewest bands in the obtained band combination or not. So we propose to use the branch and bound algorithm to further search for the band combination with fewest bands. The only optimal feature selection algorithms are exhaustive search, the branch and bound algorithm and its variants. An exhaustive search finds the best subset of features by searching all possible feature combinations. The number of all possible feature sets becomes excessively large while the dimensionality of the original feature space increases; it is thus only applicable for low-dimensional problems. The BB algorithm is more efficient than an exhaustive search, because it avoids exhaustively exploring the entire search space. It is achieved by organizing the search space properly so that many subsets that are guaranteed to be sub-optimal are rejected. Since the CMI only considers the correlation between bands, and GA-SVM searches for the best combination of irredundant bands, instead of minimal set of bands. Hence we use the adaptive branch and bound (ABB) algorithm proposed in [14] as the pruning method to find out the minimal band combination with maximal relevance. Here we take Washington DC Mall data set as an example. From the pruning results, we find that the band combination with 7 bands has the largest fitness. In this combination, not only the 2nd, the 9th and the 10th group which are added for avoiding the loss of important information, but also the 8th group are cut out. As is shown in Fig. 1, there are fewer bands in the 8th group than others and the CMI between these

A Hybrid Evolutionary Approach to Band Selection

333

bands is not so consistent. Accordingly, this group should be merged into the 9th group. After pruning, the final band grouping result is Band 1 to 37, 38 to 56, 57 to 72, 73 to 86, 87 to 102, 103 to 133 and 134 to 191.

5 Experimental Results and Analysis In this paper, our experiment analysis is conducted on two well-known benchmark data sets, i.e., the Washington DC Mall data set and the Indian Pine data set. In the former data set with 191 valid bands contains 1280 scan lines with 307 pixels in each scan line, which has been studied extensively by many research groups [1, 16, 17, 18]. The second data set with 202 valid bands is made up of a 145 × 145 pixel portion of the AVIRIS image acquired over NW Indian Pine in June 1992 [1, 3, 4]. For the Washington DC Mall data set, we use 4428 training samples and 3651 test samples. And for the Indian Pine data set, we adopt the same partition of training/test sets as Serpico and Moser in [4]. In the following, a group of experimental results will be presented on these two benchmark data sets. The parameters of the GA algorithm are configured as follows. The chromosome population size is 30, the probability of mutation is 0.2, the probability of crossover is 0.8, and the maximum number of iterations is 100. For the SVM, the kernel function is radial basis function (RBF), whose best parameters are determined by grid searching heuristics accompanying the LibSVM software [13]. 5.1 Experimental Results on Washington DC Data Set In order to prove the necessity and efficiency of the pruning by BB for the initial band grouping, we search the best band combination with GA-SVM algorithm based on 8band group (As shown in Section 2) and 11-band group (As shown in Section 2) without pruning and 7-band group after pruning (As shown in Section 4). The 10-run averaging results of this experiment is given in Table 1. Table 1. Classification accuracy of before/after pruning on Washington DC Mall data set

Algorithms

Band Number

Average Accuracy(%)

Before Pruning 1

8

97.26

Before Pruning 2

11

96.02

After Pruning

7

98.25

As shown in Table 1, the band group after pruning can make the final band combination achieve higher classification accuracy by cutting out the irrelevant groups. The reduction of groups can also reduce the heavy computation burdens of GA-SVM so that the algorithm becomes more efficient. Ref. [16] used the same experiment data set. In that paper, the authors made a principal component analysis for hyperspectral image first, and then conducted

334

H. Wu et al.

mathematical morphological analysis for the first two main image component, after which NWFE and neural network classifier were utilized to train and classify. The authors of [16] achieved the accuracy of 98.90% for 26 bands and 98.5% for 14 bands. We can achieve an average accuracy of 98.25% and the maximum accuracy of 98.79%, for only 7 bands. 5.2 Experimental Results on Indian Pine Data Set As stated at the beginning of this section, there are 202 bands in the Indian Pine data set for our experiments, which are initially divided into 20 groups: Band 1 to 4, 5 to 18, 19 to 33, 34 to 44, 45 to 57, 58 to 63, 64 to 77, 78 to 85, 86 to 94, 95 to 102, 103 to 105, 106 to 125, 126 to 131, 132 to 143, 144 to 147, 148 to 152, 153 to 157, 158 to 170, 171 to 198 and 199 to 202. And the final band grouping result after BB pruning is a 12-band group, i.e., 1 to 18, 19 to 33, 34 to 44, 45 to 57, 58 to 77, 78 to 105, 106 to 125, 126 to 131, 132 to 147, 148 to 157, 158 to 170 and 171 to 202. In order to prove the necessity and efficiency of the pruning by BB for the original band grouping, we search the best band combination with GA-SVM algorithm based on 20-band group without pruning and 12-band group after pruning. Table 2 gives the results. Table 2. Classification accuracy of before/after pruning on Indian Pine data set

Algorithms

Band Number

Average Accuracy(%)

Before pruning

20

82.30

After pruning

12

83.95

As shown in Table 2, with the Indian Pine data set, the band group after pruning can also make the final band combination have a higher classification accuracy by cutting out the irrelevant groups. Serpico and Moser [4] proposed three search algorithms for band selection and conducted experiments on the same Indian Pine data set. Their best result in [4] were obtained by SABE with 28 bands, which was 81.57% and by FCBE with 26 bands, which was 81.46%, while ours is 83.95% with 12 bands. As shown in these experiment results, it can be observed that our method is very competitive with higher accuracy but fewer bands.

6 Concluding Remarks An innovative band selection algorithm with band grouping based on CMI, pruning by BB and GA-SVM evaluation for hyperspectral image has been proposed in this paper. With band grouping based on CMI and pruning by BB, we could find the irredundant band combination with minimal number of bands and rule out the

A Hybrid Evolutionary Approach to Band Selection

335

irrelevant groups and reduce the number of band groups so that the classification accuracy can become higher. Comparison results with other state-of-the-art algorithms on two benchmark data sets show that the proposed method is superior in classification accuracy and with fewer selected bands. The CMI can represent the correlation between the bands and the classes. It is an important topic to study in future that how to mine more information for band grouping of hyperspectral image from CMI and make the classification accuracy higher. Another concerned problem is how to further reduce the computation complexity of GA-SVM and make the searching procedure faster. Further work in this direction is undertaken currently. Acknowledgments. The Washington DC Mall data set are obtained from the student CD-ROM which accompanies Prof. Landgrebe’s book [1]. The authors are also grateful to Prof. Serpico and Dr. Moser for providing us the training/test samples of Indian Pine data set used in their paper [4].

References 1. Landgrebe, D.A.: Signal theory methods in multispectral remote sensing. Wiley, Hoboken (2003) 2. Liu, C., Zhao, C., Zhang, Y.: A new method of hyperspectral remote sensing image dimensional reduction. Journal of Image and Graphics 10, 218–224 (2005) 3. Serpico, S.B., Bruzzone, L.: A new search algorithm for feature selection in hyperspectral remote sensing images. IEEE Trans. on Geoscience and Remote Sensing 39, 1360–1367 (2001) 4. Serpico, S.B., Moser, G.: Extraction of spectral channels from hyperspectral images for classification purposes. IEEE Trans. on Geoscience and Remote Sensing 45, 484–495 (2007) 5. Guo, B., Damper, R.I., Gunn, S.R., et al.: A fast separability-based feature-selection method for high-dimensional remotely sensed image classification. Pattern Recognition 41, 1653–1662 (2008) 6. Huang, R., He, M.: Band selection based on feature weighting for classification of hyperspectral data. IEEE Trans. on Geoscience and Remote Sensing Letters 2, 156–159 (2005) 7. Wang, L., Gu, Y., Zhang, Y.: Band selection method based on combination of support vector machines and subspatial partition. Systems Engineering and Electronics 27, 974– 977 (2005) 8. Peng, H., Long, F., Ding, C.: Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans. on Pattern Analysis and Machine Intelligence 27, 1226–1238 (2005) 9. Sotoca, J.M., Pla, F.: Hyperspectral data selection from mutual information between image bands. In: Yeung, D.-Y., Kwok, J.T., Fred, A., Roli, F., de Ridder, D. (eds.) SSPR 2006 and SPR 2006. LNCS, vol. 4109, pp. 853–861. Springer, Heidelberg (2006) 10. Kwak, N., Choi, C.-H.: Input feature selection for classification problems. IEEE Trans. on Neural Networks 13, 143–159 (2002) 11. Novovicova, J., Somol, P., Haindl, M., Pudil, P.: Conditional mutual information based feature selection for classification task. In: Rueda, L., Mery, D., Kittler, J. (eds.) CIARP 2007. LNCS, vol. 4756, pp. 417–426. Springer, Heidelberg (2007)

336

H. Wu et al.

12. Oh, I.-S., Lee, J.-S., Moon, B.-R.: Hybrid genetic algorithms for feature selection. IEEE Trans. on Pattern Analysis and Machine Intelligence 26, 1424–1437 (2004) 13. Chang, C., Lin, C.: LIBSVM: a library for support vector machines (2009), Software available at: http://www.csie.ntu.edu.tw/~cjlin/libsvm 14. Nakariyakul, S., Casasent, D.P.: Adaptive branch and bound algorithm for selecting optimal features. Pattern Recognition Letters 28, 1415–1427 (2007) 15. Zuo, L., Zheng, J., Wang, F., et al.: A genetic algorithm based wrapper feature selection method for classification of hyper spectral data using support vector machine. Geographical Research 27, 493–501 (2008) 16. Benediktsson, J.A., Palmason, J.A., Sveinsson, J.R.: Classification of hyperspectral data from urban areas based on extended morphological profiles. IEEE Trans. on Geoscience and Remote Sensing 43, 480–491 (2005) 17. Dundar, M.M., Landgrebe, D.A.: Toward an optimal supervised classifier for the analysis of hyperspectral data. IEEE Trans. on Geoscience and Remote Sensing 42, 271–277 (2004) 18. Kuo, B.C., Chang, K.Y.: Feature extractions for small sample size classification problem. IEEE Trans. on Geoscience and Remote Sensing 45, 756–764 (2007)

Impulsive Environment Sound Detection by Neural Classification of Spectrogram and Mel-Frequency Coefficient Images Peerapol Khunarsa1, Chidchanok Lursinsap1 , and Thanapant Raicharoen2 1 Advanced Virtual and Intelligent Computing Center (AVIC), Department of Mathematics Chulalongkorn University, Bangkok 10330, Thailand [email protected], [email protected] 2 Defence Information and Space Technology Department The Office of Permanent Secretary of Defense Ministry of Defense, Bangkok 10200, Thailand [email protected]

Abstract. The problem of automatic detecting impulsive sounds such as human sound (screams, shout), gun shots, machine gun, thunder, fire alarm, and car horn are useful for hearing impairment person. In this paper, instead of filtering the frequency of each sound for identifying types of sound, the frequency of sound is transformed into a recognizable image. The transformation is based on audio spectrogram (Power Spectrum) and Mel-frequency cepstral coefficients (MFCC). The images of both power spectrum and Mel-frequency spectrum are used as the inputs for an artificial neural network to recognize the corresponding sound. The proposed technique is tested with six different types of sound, i.e. machine gun, human scream, gun shot, thunder, fire alarm, and car horn from a sound database containing more than one hour of six different impulsive sounds. The experimental results on impulsive sounds detection using a spectrogram with feed-forward neuron network can effectively detect the segments of impulsive sound region in audio signal with more than 94% accuracy. Keywords: Impulsive Environment Sound Detection, Impulsive Sounds, Spectrogram, Mel-frequency cepstral coefficient (MFCC), Feed-Forward Neuron Network.

1

Introduction

The use of the human sense hearing as a part of examination and diagnosing is used in many fields. The piano teacher listens to the students chords to evaluate the correctness of them. A doctor listens to a patient’s heartbeat to detect abnormalities. Similarly, a car engineer listens to the sounds of the car engine to analyses possible damaged parts. In the aim of letting technology perform human tasks, we have to let a system imitate the human conception of normal behavior and teach the system to react when there is an unexpected deviation the same way the senses informs humans. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 337–346. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com 

338

P. Khunarsa, C. Lursinsap, and T. Raicharoen

The use of a sound detection system can offer concrete potentialities for surveillance and security applications by contributing to alarm triggering or validation. Furthermore, these functionalities can also be used in portable devices to inform elderly persons or hearing impairment person that affected in their hearing capabilities about relevant environmental sounds (warning signals, etc.) [1] By using different analyzing methods, many researchs are developed in many applications. James Walker [2] described a method to detect abnormal heart rhythm by using Wavelets Transforms (WT). Kotani [3] described the use of frequency and amplitude to extract a gas leakage sound from a noisy environment. Another system based on Artificial Neural Networks (ANN) and sound recording for aircraft identification was evaluated by Claesson[4]. FU [5] developed a matching pursuit approach to predict small drill bit breakage in the drilling process. By applying ANN, Abu-Mahfouz [6] used vibration features for detecting drilling wear. The use of a sound detection system is useful for audio detection such as detecting speech and singing voice in polyphonic music. The issue of locating singing voice regions in musical songs has already been addressed following traditional statistical approaches applied on widely-used speech features. For example, Gaussian Mixture Models [7], Neural networks and SVM [8], [9] or Hidden Markov Model [10] were used. However, these studies are based on a limited number of features, traditionally used in speech recognition systems (MFCC and PLP). This may not be appropriate to capture the characteristics of the singing voice in background music as noted by [11] and [12]. Different methods are also difficult to compare since all published results are obtained on different corpora with different evaluation protocols. Here, the automatic impulsive sound detection problem is studied. We define this problem as classifying a segment of an audio file into fragment containing impulsive sounds happened. Based on it audio contents, there are two major issues in dealing with this problem: the feature extracted from a given an audio signal and classification to be used. In this paper, we used spectrogram (Power Spectrum) as well as Mel-frequency cepstral coefficients for feature extraction and neural networks for classification. The rest of this paper is organized as follow. In section 2, we briefly review the deteil of feature extraction. Section 3 and 4 describes the data collection and information regarding the clssification method and parameter used. Section 5 presents experimental results. Finally, section 6 draws a conclusion.

2

Feature Extraction

Feature extraction is the process of computing a compact numerical representation that can be used to characterize segments of audio signal. The design of descriptive features for a specific application is the main challenge in building a pattern classification system. The present work uses two different feature extraction algorithms, i.e. spectrogram analysis based on Fast Fourier transform

Impulsive Environment Sound Detection by Neural Classification

339

(FFT) and Mel-frequency cepstral coefficients. Figure 1 shows the block diagram of spectrograms generated by Fourier transformation (left) and Mel-frequency cepstral coefficients (right).

Fig. 1. Comparison of different spectrograms from Fourier and Mel-frequency processes. The left one is from Fourier transformation. The right one is from Mel-frequency ceptral coefficients.

2.1

Spectrogram (log Power Spectrum)

First the audio signal is divided into short time windows or segments. Fast Fourier transform (FFT) is applied to each time window for the discrete-time signal x(n) with length N, given by X(k) =

N −1  n=0

w(n)x(n)exp(−

j2πkn ) N

(1)

s for k = 0, 1, . . . , N − 1, where k corresponds to the frequency f (k) =( kf N ) , fs is the sampling frequency in Hertz and w(n) is a time-window. Here, we chose Hamming window as a time window, given by  πn  w(n) = 0.54 − 0.46 cos (2) N

In this paper, each segment is transformed with FFT in (1). After that the magnitude frequency vectors are stacked and plotted with the vertical axis representing the frequency and the horizontal axis representing the time. The magnitude

340

P. Khunarsa, C. Lursinsap, and T. Raicharoen

values are color codes. In this paper, we used each column of the spectrogram as a feature vector for recognizing the signal. The spectrogram displays just the energy and not the phase of the short-term Fourier transform, we compute the energy as

Fig. 2. Spectrograms of impulsive sound examples. The top row is gun sound. The second row is fire alarm sound. The third row is thunder sound. The last row is machine gun sound.

2.2

Mel Frequency Cepstral Coefficients

Mel-frequency cepstral coefficient(MFCC) differs from the real cepstrum in that it uses a non-linear frequency scale, which approximates the behavior of the auditory system. The audio signal is divided into short time windows or segments. Discrete Fourier transform (DFT) is applied to each time window for the discrete-time signal x(n) with length N , given by (1) and (2). The magnitude spectrum |X(k)| is now scaled in both frequency and magnitude. First, the magnitude spectrum is scaled logarithmically using the so-called Mel filter bank H(k, m) as follows N −1  X  (m) = ln( |X(k)|.H(k, m)) (3) k=0

for m = 1, 2, . . . , M , where M is the number of filter banks and M L). In this case, the size of Suppose that L is overestimated as L   k = 1, · · · m. h is [m(L + 1)] × 1 and we have hk (τ ) = 0, τ = L + 1, · · · , L;   − L) − 1 ≤ Correspondingly, if it is noise-free, rank[X(L)] = m(L + 1) − (L T  + 1) − 2, i.e., at least two eigenvalues of matrix X(L) X(L) are zeros. m(L Hence we can not obtain the right estimation by (10) or (11).  of h should be Furthermore, a desirable estimation h ⎧ T

T ⎪ = h  ,··· ,h T ⎪ h , 1 m ⎪ ⎪ ⎪ ⎪  L−L ⎨    T (12)  hk = c[0, · · · , 0, hk ]T = c [0, · · · , 0, hk (L) , · · · , hk (0)] , ⎪ ⎪    ⎪ ⎪ ⎪  ⎪ L+1 ⎩ k = 1, · · · , m.  we propose the following optimization To force  hk (l) = 0, l = L + 1, · · · , L, problem to estimate the channels: ⎧  p  m L ⎪  ⎪ min   w ·  ⎪ hk (L) l ⎪ ⎪  ⎨ h k=1 l=0 (13) ⎪ T  ⎪ st : X(L) X(L) · h = 0 ⎪ ⎪ ⎪ ⎩ =1 cT h 

+ where the weight sequence {wl }L is a monotonically decreasing. Typl=0 ∈ R l ically, we can choose wl = μ , where 0 < μ < 1. In (13), the sparsity is  by p measure imposed on the overestimated entries  hk (l), l = L + 1, · · · , L p | · | [8, 9, 10, 11, 12], where usually 0 < p < 2 (e.g., p = 1).  = Denote μ = (1, μ, · · · , μL )T , w = (μT , · · · , μT )T = (w1 , · · · , wm(L+1) )T , h      T T T 0m(L+1) X(L) X(L)  T T   and A = . [h1 , · · · , hm ] = (h1 , · · · , hm(L+1) ) ,y =  cT 1 Then problem (13) can be simplified as ⎧   p m(L+1) ⎪ ⎪  ⎪ min J(h)  = min  wi ·  ⎨ hi    h h i=1 (14) ⎪ ⎪ ⎪ ⎩ =y st : A · h

Robust Channel Identification Using FOCUSS Method

475

Similar to the FOCUSS algorithm [8, 9, 10, 11, 12], for (14), we have the following iterative formula  t+1 = Π −1 (h  t ) · AT · [A · Π −1 (h  t ) · AT ]−1 · y, h

(15)

 2−p |2−p | hm(L+1)   = diag( |h1 | ,··· , ). The proof is as follows: where Π −1 (h) w1 wm(L+1)   λ) for the problem (14): Construct the following Lagrange function L(h,  − y).  λ) = J(h)  + λT · (Ah L(h, From (16), we can obtain the following equation set: ⎧  λ) ⎪ ∂L(h, ⎪  ·h  + AT λ = 0 ⎨ = p · Π(h)  ∂h  ⎪ ⎪  −y =0 ⎩ ∂L(h, λ) = Ah ∂λ

(16)

(17)

 on both sides of the first equation Pre-multiplying the diagonal matrix Π −1 (h) of (17), we can derive  = − 1 · Π −1 (h)  · AT λ. h (18) p  we can derive the following expression by simultaneously Thus, using y = Ah, pre-multiplying A on the both sides of (18)  · AT ]−1 · y. λ = −p · [A · Π −1 (h)

(19)

Combining (18) and (19), we get  = Π −1 (h)  · AT · [A · Π −1 (h)  · AT ]−1 · y. h

5

Numerical Experiments and Result Analysis

Computer simulations are conducted in this section to test the performance of the proposed algorithm and compare it with Prediction Error Method (PEM) [3]. In the following two experiments, the parameters of the proposed algorithm are set as μ = 0.1 and p = 1. To evaluate the performance of the estimations, we define an average Signalto-Interference-Ratio (SIR) as follows: 2   ˆ Nt h − h    (i) (i) 1 SIR = −20 log [dB], (20)  2 Nt i=1 h(i)  ˆ (i) is the estimation of where Nt denotes the number of Monte Carlo trials, h ˆ the ith trial. Since the estimation h can be up to a nonzero scaling factor of h,

476

Z. He and A. Cichocki

ˆ = ch (c = 0), we normalize them to be the equal scaling (e.g. h = 1) i.e., h and then compute their SIR (20). Experiment 1. Consider a noise-free example with order overestimation. The input signal is independent and follow an identical Gaussian distributed. Both input signal s(t), t = 1, · · · , T and channels h(τ ), τ = 1, · · · , L are generated randomly using Matlab function randn(·). Then the measured signals x(t), t = 1, · · · , T were obtained from (1). 50 Monte Carlo tests were performed. The operating parameters were taken as follows: m = 3, L = 4 and T = 5000. The results are shown in Table 1. Table 1. SIRs of 50 Monte Carlo trials in Experiment 1 (in dB)  L 4 5 6 7 8 9 10 PEM 23.97 19.37 20.42 20.09 27.34 23.87 19.29 Our method 260.81 266.19 265.82 265.84 266.87 265.13 265.47

From Table 1, we can see that in the noise-free case, both the PEM method and the proposed method well identified the channels although the order of channels is overwhelmingly overestimated. The results of the proposed method are much more precise. Experiment 2. SIMO channel identification with order overestimation in the noisy case is tested in this experiment. All signals and channels were generated in the same manner as Experiment 1 except that the additive white Gaussian noise is added to the received signals x(t), t = 1, · · · , T . The system parameters were as follows: m = 3, L = 4 and T = 5000. Table 2. SIRs of 50 Monte Carlo trials in the noise case (in dB)  L 4 5 6 7 8 9 10 PEM 16.98 5.58 7.21 7.28 8.51 6.72 6.64 Our method 34.53 36.80 34.35 28.03 26.95 22.76 17.05

Table 2 shows that SIMO channel identification with order overestimation is much more challenging in the noise case than in the noise-free one. We can confirm this by comparing Table 1 and Table 2. For both methods, their SIRs are much lower in Table 2 than in Table 1. Relatively, the proposed method also outperformed the PEM method in this experiment.

6

Conclusions

Based on sparse representation and the FOCUSS algorithm, an efficient blind multichannel identification algorithm is proposed in this paper. It is easy to implement and works when the order of channels is overestimated. Differing from

Robust Channel Identification Using FOCUSS Method

477

the PEM [3], the new algorithm does not require the input signal to be independent and identical distribution, and even the input signal can be non-stationary. Compared with PEM, the new algorithm is more robust to the overestimation of channel order and much faster because it does not involves eigenvalue decomposition. The experiments have verified the good performance of the proposed algorithm.

References 1. Xu, G.H., Liu, H., Tong, L., Kailath, T.: A Least-squares Approach to Blind Channel Identification. IEEE Trans. Signal Processing 43, 2982–2993 (1995) 2. Moulines, E., Duhamel, P., Cardoso, J.F., Mayrargue, S.: Subspace Methods for the Blind Identification of Multichannel FIR Filters. IEEE Trans. Signal Processing 43, 516–525 (1995) 3. Abed-Meraim, K., Moulines, E., Loubaton, P.: Prediction Error Methods for Second-order Blind Identification. IEEE Trans. Signal Processing 45, 694–705 (1997) 4. Papadias, C.B., Slock, D.T.M.: Fractionally Spaced Equalization of Linear Polyphase Channels and Related Blind Techniques Based on Multichannel Linear Prediction. IEEE Trans. Signal Processing 47, 641–654 (1999) 5. Li, X., Fan, H.: Linear Prediction Methods for Blind Fractionally Spaced Equalization. IEEE Trans. Signal Processing 48, 1667–1675 (2000) 6. Zhou, Y., Leung, H., Yip, P.: Blind Identification of Multichannel FIR Systems Based on Linear Prediction. IEEE Trans. Signal Processing 48, 2674–2678 (2000) 7. Tong, L., Xu, G.H., Kailath, T.: Blind Identification and Equalization of Multipath Channels: A Time Domain Approach. IEEE Trans. Information Theory 40, 340– 349 (1994) 8. Gorodnitsky, I.F., George, J., Rao, B.D.: Neuromagnetic Source Imaging with FOCUSS: A Recursive Weighted Minimum Norm Algorithm. Electroencephalography and Clinical Neurophysiology 95, 231–251 (1995) 9. Gorodnitsky, I.F., Rao, B.D.: Sparse Signal Reconstruction from Limited Data Using FOCUSS: A Recursive Weighted Minimum Norm Algorithm. IEEE Trans. Signal Processing 45, 600–616 (1997) 10. Rao, B.D., Kreutz-Delgado, K.: An Affine Scaling Methodology for Best Basis Selection. IEEE Trans. Signal Processing 47, 187–200 (1999) 11. Cichocki, A., Amari, S.: Adaptive Blind Signal and Image Processing: Learning Algorithms and Applications. John Wiley & Sons, New York (2003) 12. Rao, B.D., Engan, K., Cotter, S.F., Palmer, J., Kreutz-Delgado, K.: Subset Selection in Noise Based on Diversity Measure Minimization. IEEE Trans. Signal Processing 51, 760–770 (2003)

Human Head Modeling Using NURBS Method Songyun Xie, Ningfei Li, and Zhuo Lv School of Electronics and Information, Northwestern Polytechnical University, Xi’an 710129, China [email protected]

Abstract. Human head modeling is the foremost research topic of the multimodal fusion for medical information in brain function research based on EEG/fMRI. Finding an effective way to construct the geometric model of human head is one of the focal point of research. Here a new method based on NURBS surface reconstruction is proposed. In the experiment, surface interpolation is employed, and a fairly good result is got with comparatively less points. Thus NURBS is proved a rapid and effective method for head modeling. Keywords: Realistic head model, NURBS, Interpolation, EEG, fMRI.

1 Introduction Human head modeling is the first step of forward and inverse magnet encephalographic (MEG) or electroencephalogram (EEG) calculations. And also it is the first problem of the multi-modal fusion of medical information in brain function research based on EEG/fMRI (function Magnetic Resonance Imaging). In practice spherical models have been widely used to simplify the EEG forward and inverse problems. Most commonly used is three-shell spherical head model which consists of scalp, skull and brain layer, other spherical models include five-shell spherical model and overlapping spherical model, and so on. However, when the dipoles method is adopted, there will be big errors of calculated value and localization caused by the spherical model due to the nonrealistic shape [1]. This limits the usage of the spherical model. To improve the accuracy new model which uses human brain MRI data with much more accuracy should be put into use. There are some numerical methods set so called realistic head modeling, such as Finite Element Method (FEM) based on volume elements and Boundary Element Method (BEM) based on surface elements [2], they mainly employ triangular elements to construct the model. To ensure the accuracy the elements should be small and so a large number of coordinates of nodes is needed, thus bring large computation load and limit the use of realistic head model [3]. Here a new method for realistic human head modeling based on Non-Uniform Rational B-Splines (NURBS) is proposed. NURBS surface has many applications in industry. And there is a lot of research concerned with the modeling method based on NURBS. Interpolation and approximation are two kinds of most basic methods [4]. In this paper we use interpolation to reconstruct the realistic head model from MRI image. Fewer points are adopted in the method, and it is proved a rapid method and gets a good result. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 479–484. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

480

S. Xie, N. Li, and Z. Lv

2 Method In a three-shell realistic head model, typically they are scalp, skull and brain layer. Here brain layer is taken as an example for the NURBS based method. 2.1 Image Segmentation The first step to construct a realistic head model is MRI image segmentation [5]. The data is a 181 217 181 scan. The result is showed as following. On the left is the original image (the 114th slice); on the right is the segmentation result.

×

×

(a)

(b) Fig. 1. Image segmentation

2.2 Boundary Extraction The second step is boundary extraction, Boundary extraction is based on 4connectivity contour tracing algorithm. The final boundary is showed as following. From this step we get rows of ordered data points.

Fig. 2. Boundary of the brain layer

Human Head Modeling Using NURBS Method

481

The total number of the points is very large. If we directly use these points to construct the model, it will turn out to be a large computation load. Meanwhile the generated model will lack of smoothness. So we must do some data preprocessing first before surface reconstruction. This will be discussed in the following section. 2.3 NURBS Based Surface Reconstruction 2.3.1 Data Preprocessing Since each slice contains different numbers of boundary points, first we must make the points rectangular arranged for surface reconstruction, and attempt to reduce the number of points in this process. In this experiment there are totally 146 slices with boundary points, i.e., 146 rows of points. And the numbers of columns vary from tens to hundreds. Suppose an m n matrix is needed for surface reconstruction, then to those rows which have more than n points just use the sampling, and to those rows which have less than n points insert some points by order. Thus an m n matrix of boundary points is generated, denoted by Qi , j (i = 1, ", m; j = 1,", n} . Because n is smaller than numbers of the points in most slices, the total number of the points has be reduced after preprocessing. Here the choosing of n number is based on the quality of the reconstructed curve in each slice.

×

×

2.3.2 Surface Reconstruction Now the problem comes to using NURBS to reconstruct the brain surface with its boundary points. Since the data points have been reshaped to a matrix, this problem can be solved by using regular surface interpolation. A B-spline surface of degree ( p, q ) has the following form [6]: m

n

S (u, v) = ∑∑ N i , p (u ) N j , q (v) Pi , j i =0 j = 0

(1)

Where Pi , j are control points, N i , p (u ) and N j , q (v ) are the B-spline base functions of degree p and q defining over knot vector u and v : U = {0, " , 0, " ,1} N  

u p +1 , "um ,1,

(2)

V = {0, ", 0, ",1} N  

vq +1 , "vn ,1,

(3)

p +1

q +1

p +1

q +1

482

S. Xie, N. Li, and Z. Lv

respectively. The surface interpolation problem can be expressed by n

m

j =0

i =0

Qi , j = ∑ (∑ Pi , j Ni , p (u )) N j , q (v )

(4)

Here bicubic spline surface are adopted, so p = 3 , q = 3 . The interpolation process consists of the following steps. (1) Data points parameterization, use chord length method; (2) Compute the knot vector U ; (3) Data points parameterization, use chord length method; (4) Employ NURBS curve interpolation in U direction, get a set of “intermediate” control points; (5) Employ NURBS curve interpolation in V direction, get control points Pi , j ; (6) Use U ,V and Pi , j to compute the NURBS surface.

3 Result The experiment is completed under MATLAB environment. The final reconstructed brain layer of the realistic head model is show as following.

Fig. 3. Result

Human Head Modeling Using NURBS Method

483

Fig. 3. (continued)

4 Discussion and Conclusion As one can see in Fig. 3, a good result is generated. Meanwhile during data preprocessing, the number of points have be reduced from more than 70,000 to 10,220. When triangular elements are used to construct the model, the number will increase greatly. The above modeling process takes only seconds on a standard PC with Intel Core(TM) 2 Duo Processor and 2G memory. Further evaluation of the effectiveness of the method can be done in the forward and inverse EEG calculation by quantitative analysis. This paper introduces the NURBS based method and is just a start. More sophisticated NURBS method can be employed in future study. However, we can see from here the high potential of NURBS method in realistic head modeling, which needs more investigation. Acknowledgments. This work was supported in part by the National Natural Science Foundation of China under Grant 30470459, the Basic Research Foundation of NPU under Grant W018102 and the Key Undergraduate Graduation Project 2009 of NPU. We appreciate Dr. Petra Ritter at Charité Universitätsmedizin Berlin for data providing. Thanks Prof. Jiadong Xu and Dr. Jianzhou Li for their technological support.

References [1] Uitert, R.V., Johnson, C.: Can a spherical model substitute for a realistic head model in forward and inverse MEG simulations? In: Proc. Conf. Biomagnetism, Jena, Germany (2002) [2] Akahn, Z., Acar, C.E., Gencer, N.G.: Development of realistic head models for electromagnetic source imaging of the human brain Engineering. In: Proceedings of the 23rd Annual International Conference of the IEEE Medicine and Biology Society, vol. 1, pp. 899–902 (2001)

484

S. Xie, N. Li, and Z. Lv

[3] He, J.J., Shen, H., Hu, D.W.: A Survey of the EEG/fMRI Fusion Analysis: Head Models, Methods and Applications. Computer Engineering & Science 29(12), 74–81 (2007) [4] Piegl, L., Tiller, W.: The NURBS Book. Springer, New York (1997) [5] Smith, S.M.: Fast Robust Automated Brain Extraction. Human Brain Mapping 17(3), 143– 155 (2002) [6] Piegl, L., Tiller, W.: Reducing Control Points in Surface Interpolation. IEEE Computer Graphics and Applications 20(5), 70–74 (2000)

Risk Sensitive Unscented Particle Filter for Bearing and Frequency Tracking Peng Li, Shenmin Song, and Xinglin Chen School of Astronautic, Harbin Institute of Technology 150001, Harbin, China

Abstract. Robust filter based on risk sensitive estimator is derived to estimate the state of the uncertain models, while the estimation error involves two terms, the first term coincides with the minimum value of the risk sensitive cost function, the second one is the distance between the true and design probability models. The proposed algorithm, which introduces risk sensitive estimator into the unscented particle filter, could automatically change the state noise covariance according to the magnitude of the risk function. As a result, sample impoverishment could be mitigated. In the simulation of submarine bearing and frequency tracking, the performance of the new algorithm is compared with the unscented kalman filter and the unscented particle filter. Simulation results show that the new algorithm performs better than the two others. Keywords: unscented particle filter, risk sensitive estimator, error bound.

1 Introduction Estimating the signal state, which is the information available in certain noisy observations of the signal, is the important function of filter. When the least squares criterion is used and when we assume the plant is completely and accurately described by an assumed model, then the optimal estimator is the conditional mean, computed using the assumed model. In view of the problems which can be encountered when the plant deviates from the assumed model, we consider the use of other error criteria. The risk-sensitive criterion is one such alternative, and in the context of robust control theory, it is known to enjoy robustness characteristics by virtue of its close connection with the H ∞ criterion[1]. That is actually a manifestation of the filters’ ability to increase the state noise covariance in an implicit and automatic way, which is the general characteristic of some solutions proposed to solve the sampling impoverishment problem in particle filters. This common point suggests that the application of a risk sensitive estimator based framework in particle filters can be a successful alternative in mitigating the problem of sample impoverishment[2-4]. The organization of the paper is as follows. In section 2, we firstly make a definition to the risk sensitive function and give it’s upper-bounded estimation error, then make a derivation of the risk sensitive filter. The main results are given in Section 3 where the unscented particle filter with risk sensitive function is completed. The performance of the proposed algorithm is illustrated on a simulation about the Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 485–491. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

486

P. Li, S. Song, and X. Chen

submarine bearing and frequency tracking in Section 4. The paper is finalized with conclusions in Section 5.

2 Risk Sensitive Filters In this section we define the risk sensitive filter and explain robustness in our context. The optimal signal processing schemes can suffer dramatically when reality has even small deviations from nominal assumptions about reality. This is the basic motivation for robust estimation. 2.1 Risk Sensitive Estimation One approach to robust estimation is the so called risk sensitive estimation. We use this approach to derive our basic result, namely a robust filter (robust to model uncertainties). First we give a general definition of a risk sensitive estimate. We assume that there always exists a true and fixed probabilistic model for the system, and denote the distribution of this true model by P0 . Since we declare that the model being used may not necessarily correspond to the true model, we denote the design model by Pd . Suppose ϕ = ϕ ( x ) is a real-valued function of x to be estimated by a random variable Let

ϕˆ ( ϕˆ ∈

ρ a ∈ C ( R) be

), which is

-measurable function[5].

a strictly convex function satisfying

∇ 2 ρ a ≥ ca > 0 and is

bounded from below. If we set

f (ϕˆ ) = Ed [ ρ a (ϕ − ϕˆ )] For any

-measurable

(1)

ϕˆ , we define a minimum risk sensitive estimator by

ϕˆ * = arg min f (ϕˆ ) ϕˆ∈

(2)

From the control theory we know that the controller which is designed minimizes the risk sensitive criterion, this controller is implying certain robustness properties. So we defined

ϕˆrs* by selecting

ρ a (ϕ − ϕˆ ) = exp( μρ (ϕ − ϕˆ ))

(3)

Where μ > 0 is a parameter determining the degree of “risk”. The minimum risk-sensitive estimator cost function is

f rs (ϕˆ ) = Ed [exp( μρ (ϕ − ϕˆ ))]

(4)

Risk sensitive estimators defined by (2) enjoy an upper bounded estimation error[6].

E0 ⎡⎣ρ(ϕ −ϕˆrs* )⎤⎦ ≤

log frs (ϕˆrs* ) + R(Pd P0 )

μ

(5)

Risk Sensitive Unscented Particle Filter for Bearing and Frequency Tracking

Where

487

R( Pd P0 ) denotes the relative entropy between the two probability models

P0 and Pd . This result establishes a precise meaning for risk sensitive estimation, it tells us that the error in risk sensitive estimation has an upper bound given by the sum of two terms. The first term coincides with a cost if the model were known precisely, while the second term is the measure of distance between the true probability models and the design probability models. 2.2 Risk Sensitive Filter We denote the discrete time non-linear system as follows.

⎧ xk = f ( xk −1 ) + ωk −1 ⎨ ⎩ yk = h( xk ) + υ k Where

xk ∈

ωk ∈

n

n

is the signal state at time step k , yk

∈

(6) m

is the observation,

υk ∈ are independent identical distribution random processes with densities pω (ω ) and pυ (υ ) . Let ρ1 , ρ 2 be continuous functions which are strictly convex, and μ1 ≥ 0 , μ2 ≥ 0 are two risk parameters. Define the risk sensitive cost function for ϕˆ ∈ [5]. and

m

k −1

f rs ,k (ϕˆk ) = Ed [exp( μ1 ∑ ρ1 (ϕ ( xl ) − ϕˆl* ) + μ2 ρ 2 (ϕ ( xk ) − ϕˆk ))]

(7)

l =0

The minimum risk sensitive estimator is defined by

ϕˆrs* ,k = arg min f rs ,k (ϕˆk )

(8)

ϕˆk ∈

It is easy to see that the error bound[7.8] k−1

Ed [exp(μ1∑ρ1(ϕ(xl ) −ϕˆl*) + μ2ρ2 (ϕ(xk ) −ϕˆk ))] ≤ log f rs ,k (ϕˆrs* ,k ) + Rk ( Pd P0 ) (9) l =0

Where

Rk ( Pd P0 ) is the relative entropy between the two probability models P0 and

Pd on the time 0,

k.

In filter, the aim is to calculate the state estimate xˆk , so if we adapt

xk instead of

ϕ k , the formula (7) becomes k −1

f rs ,k ( xk ) = Ed [exp( μ1 ∑ ρ1 ( xl − xˆl ) + μ 2 ρ 2 ( xk − xk* ))] l =0

(10)

488

P. Li, S. Song, and X. Chen

For the reasons of computational simplicity, the way we proceed is as follows. Let μ1 = μ 2 . ρ1 , ρ 2 are the same function, and

ρ1 ( xl − xˆl ) = ( xl − xˆl )T ( xl − xˆl ) . As formula (7), the formula (8) becomes

xˆk = arg min f rs ,k ( xk )

(11)

x*k ∈

Note that on the time k , all the previous estimates

xˆ0:k −1 are already determined and

fixed. The recursive calculation of the expected value in (10) can be done with the unnormalized density function α k k −1 ( x ) . k −1

α k k −1 ( x)dx = E[ ∏

l =0

The density function

α k k −1 ( x)

expectation in (10), that is

pv ( yk − h( xk )) k −1 i ∏ exp( μρ ( xl − xˆl ))] l =0 pv ( yk )

(12)

is actually an information state to calculate the

xˆk

xˆk = arg min ∫ p( yk xk ) exp( μρ ( xk − xk* )) β k k −1 ( xk )dxk

(13)

xk* ∈

Where

β k k −1 ( xk ) =

∫

α k ( xk ) α k (ς )d ς

is the normalized density. The normalized density function

β k k −1 ( xk ) =

1 ck

∫

β k k −1 ( xk )

satisfies[5]

p ( xk xk −1 ) p ( yk −1 xk −1 ) exp( μρ ( xk −1 − xˆk −1 ))

(14)

i β k −1 k − 2 ( xk −1 )dxk −1 Where

ck is a normalization factor.

3 Risk Sensitive Unscented Particle Filter The recursion (15) can be implemented using a particle filter as follows. At each time step k , the information state

β k k −1 ( xk )

(i )

is approximated with N samples xk N

β k k −1 ( xk ) ≈ ∑ wki δ ( xk − xki ) i =1

(15)

Risk Sensitive Unscented Particle Filter for Bearing and Frequency Tracking

489

Where δ denotes the Dirac delta function, the variable wk is known as the importance i

weights. Substituting this approximation into formula (14), we get N

xˆk ≈ arg ∑wki p( yk xki )exp(μρ(xk − xk* ))

(16)

x*k ∈ i =1

When the formula (17) is adapted in unscented particle filter to calculate the output estimate xˆk at each time step k . There are two parts different from the classic UPF in the new algorithm. First, the importance weights comprise another part, the risk sensitive function, which aims to generate more samples for that the risk function assigns more risk. Second, in order to achieve the recursions of the algorithm, one has to calculate the state estimate xˆk at each time step k . The differences between RSUPF and UPF is as follows[5]. 1. Computing the importance weights up to a normalizing constant.

w =w (i ) k

(i ) k −1

p( yk xˆk(i ) ) p( xˆk(i ) xk(i−)1 ) q( xˆk(i ) x0:(ik) −1 , y1:t )

⋅ μρ ( xki −1 − xˆk −1 )

(17)

2. Using formula (17) to get the estimation of xˆk .

4 Simulations Here we verify the new algorithm in submarine bearing and frequency tracking. Following state space model is used. Where the state vector is defined as the 2D relative position vector, relative velocity vector and the pure tone frequency emitted by the target (very stable). And the observations at time k are the bearing angle (in radians) and the Doppler-shifted frequency tone. c=1500 m/s (sound speed in water). The measures are given by the sonar every 4 seconds, At mid-course, submarine perform a maneuver to improve observability[9]. ⎡1 ⎢0 ⎢ x(k ) = ⎢ 0 ⎢ ⎢0 ⎢⎣ 0

4 0 0 0⎤ ⎡8 ⎥ ⎢4 1 0 0 0⎥ ⎢ 0 1 4 0 ⎥ x(k − 1) + ⎢ 0 ⎥ ⎢ 0 0 1 0⎥ ⎢0 ⎥ ⎢⎣ 0 0 0 0 1⎦

0 0⎤ 0 0 ⎥⎥ 8 0 ⎥ v(k − 1) ⎥ 4 0⎥ 0 1 ⎥⎦

y1 (k ) = arctan( x3 (k ) / x1 (k )) + υ1 (k ) y2 (k ) = x5 (k ) ⋅ (1 + 1/ c( x2 (k ) cos( y1 (k )) + x4 (k )sin( y1 ( k )))) + υ 2 (k )

Process noise and (10

v is zero mean white Gaussian noise, cov = (10−3 )2 (dynamics)

) (tone frequency). Observation noise υ is zero mean white Gaussian

−4 2

490

P. Li, S. Song, and X. Chen

noise, cov = (0.0175) (bearings) and follows. 2

(0.06)2 (frequencies). Results are as

1000

2

ukf upf rsupf

800

1 speed error(m/s)

600 range error(m)

ukf upf rsupf

1.5

400

200

0.5 0 -0.5

0 -1

-200

-400

-1.5

0

10

20

30

40

50 time(s)

60

70

80

90

-2

100

Fig. 1. Range error(m)

0

10

20

30

40

50 time(s)

60

70

80

90

100

Fig. 2. Speed error(m/s)

0.15 ukf upf rsupf

0.1

frequency error(Hz)

0.05

0

-0.05

-0.1

-0.15

-0.2

0

20

40

60

80

100

time(s)

Fig. 3. Frequency error(Hz)

In the experiments, the performance of the RSUPF is compared with the unscented kalman filter (UKF) and the UPF. From the Fig.1-3, we can see that the performance of UKF is very poor, significant divergence occurs on range estimation, and bearing estimation, significant uncertain on frequency estimation. The UPF performs much better, but sometimes very uncertain, and fairly useless. The RSUPF results show a further substantial improvement with significant divergence on the two others. Although markedly superior to the unscented kalman filter and the unscented particle filter, the performance of risk-sensitive unscented particle filter is not ideal in frequency tracking. This is almost certainly due to the submarine performs a maneuver at mid-course to improve observability. During this rapid change in line of sight, the measurements are most informative, but also the problem is most nonlinear. It is during this period that there is most chance of the filter misinterpreting the data and so diverging.

Risk Sensitive Unscented Particle Filter for Bearing and Frequency Tracking

491

5 Conclusion The risk sensitive function is applied in the unscented particle filter. When the new algorithm is applied to nonlinear bearing and frequency tracking, Simulation results have shown significant performance improvement over UKF and UPF. The great strength of the RSUPF is that the risk sensitive function penalizes higher order moments of the estimation error which implicitly makes the resulting algorithms cautious on the effects of the uncertain model and noise properties. If the risk sensitivity is moderate, the designed filter appears to have robust properties correcting sample impoverishment in especially uncertain parameter scenarios. The proposed algorithm is very simple and easy to program, although with little computationally expensive.

References 1. Whittle, P.: Risk Sensitive Optimal Control. Wiley, New York (1990) 2. James, M.R., Elliott, R.J.: Risk-Sensitive and Risk Neutral Control for Continuous-Time Hidden Markov Models. Journal of Applied Mathematics and Optimization 34(1), 37–50 (1996) 3. Dey, S., Moore, J.B.: Risk-sensitive filtering and smoothing via reference probability methods. In: Proceedings of the American control conference, Seattle, Washington, USA (1995) 4. Dey, S., Moore, J.B.: Risk sensitive filtering and smoothing via reference probability methods. IEEE Transactions on Automatic Control 42(11), 1587–1591 (1997) 5. Thrun, S., Langford, J., Verma, V.: Risk sensitive particle filters. In: Advances in Neural Information Processing Systems 14. MIT Press, Cambridge (2002) 6. Bhaumik, S., Ghoshal, T., Sadhu, S.: Alternative formulation of risk-sensitive particle filter (posterior). In: Proc. IEEE Indicon 2006, pp. 1–4 (2006) 7. Speyer, J.L., Fan, C., Banavar, R.N.: Optimal stochastic estimation with exponential cost criteria. In: Proc. 31st Conf. Decision Control, vol. 2, pp. 2293–2298 (1992) 8. Moore, J.B., Elliott, R.J., Dey, S.: Risk-sensitive generalizations of minimum variance estimation and control. J. Math. Syst. Estimat. Control 7(1), 1–15 (1997) 9. Gordon, N., Salmond, D., Ewing, C.: Bayesian State Estimation for Tracking and Guidance Using the Bootstrap Filter. Journal of Guidance, Control and Dynamics 18(6) (1995)

Fully Complex Multiplicative Neural Network Model and Its Application to Channel Equalization Kavita Burse, R.N. Yadav, and Sushil Chandra Shrivastava Department of Electronics and Communication, Maulana Azad National Institute of Technology, Bhopal, India, 462051 [email protected], [email protected]

Abstract. In this paper a novel fully complex multiplicative neural network (MNN) algorithm is proposed to extract Quadrature Amplitude Modulation (QAM) signals when passed through a non linear channel in the presence of noise. The inputs, weights, activation functions and the output of the proposed MNN are complex valued. The training algorithm for the multilayer feed forward fully complex MNN is derived. The equalizer is tested on 4, 16 and 32 QAM signals and compared with split complex feed forward MNN equalizer. The proposed equalizer is implemented on nonlinear and nonminimum phase stationary channel. The fast converging algorithm gives lower bit error rate performance even in the presence of substantial noise. Keywords: Multiplicative neural network, channel equalization, QAM signals, fully complex back propagation.

1 Introduction The potential of QAM schemes for higher data rate transmissions in communication channels is due to its high spectral efficiency [1]. When significant channel distortion along with noise is added to the transmitted signal linear boundaries are not optimal. In the absence of the noise the output signal occupies well defined M states of the MQAM signal constellation. As the signal to noise ratio decreases with the addition of channel effects and noise, a greater portion of the symbol values fall outside the detection threshold. Effectively the overall channel response becomes a nonlinear dynamic mapping and the problem can be tackled using neural network classification techniques. Traditional real valued neural network models such as multilayer perceptron (MLP), functional link artificial neural networks (FLANN), radial basis function (RBF) networks and recurrent neural networks (RNN) have been successfully used for solving equalization problems [2]-[3]. Since the nature of input signal to an equalizer is complex, complex valued neural networks find wider applications in signal processing [4]-[5]. Cha and Kassam [6] have proposed a complex-valued radial basis function (CRBF) network. A complex-valued minimal resource allocation network (CMRAN) equalizer is proposed in [7]. The performance of complex non minimum phase channel model of Cha and Kassam is evaluated in [8] using fully complex extreme learning machine algorithm. This new learning algorithm outperforms CRBF, CMRAN and complex back propagation equalizers. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 493–501. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

494

K. Burse, R.N. Yadav, and S.C. Shrivastava

All the above networks using multiple layers of summation units have been successfully used to equalize signals over linear as well non linear channels. Summation at nodes can be replaced by product units and its variants to form higher order networks which have drawn great attention from researchers due to their superior performance in nonlinear input-output mapping, function approximation and memory storage capacity. Some examples are product unit neural network (PUNN), sigma-pi network (SPN), pi-sigma network (PSN) etc. They allow neural networks to learn multiplicative interactions of arbitrary degree [9]-[10]. Replacing summation at each node by multiplication may present better approximation of higher order information from training data [11]-[12]. The real valued MNN is extended to the complex domain in this paper. The inputs, weights, activation functions and the outputs of the fully complex valued neural network (CVNN) are complex numbers. The back propagation algorithm is extended to complex domain to train the CVNN [13].The activation functions at the nodes can either be fully complex or split complex [14]. The split complex activation function processes the real and imaginary part of the input signal separately. The split complex approach is generally used to avoid singular points. Kim and Adali have proposed a subset of complex elementary transcendental functions as complex nonlinear activation functions [15]. These bounded and analytic complex nonlinear activation functions are used to further simplify the fully complex back propagation by applying Cauchy-Riemann equations. The remaining of the paper is organized as follows: Section 2 gives the introduction to the channel equalization process. Section 3 describes the learning rule for the fully complex MNN (FCMNN). In section 4 the experiments and results of transmitting 16 and 32 QAM signals over AWGN channel and 4 QAM signals over nonlinear channel is discussed. Section 5 concludes the paper.

2 Channel Equalization The process of adaptive equalization is explained as follows. The transmitter sends a known training sequence to the receiver. The learning algorithm compensates the influence of channel and noise on the received signal by minimizing the mean square error (MSE) between the adaptive filter output and the delayed test signal. To extract the phase characteristics of the channel from the received data, it is necessary to use higher order statistics of the received signal. The nonlinear function of the output of the equalizer gives rise to higher order statistics of the received signal. The system models a nonlinear FIR filter with memory at most equal to the number of delayed neural network inputs. The MSE is computed and the weights of the NN are updated. The process is repeated so as to minimize the MSE.

3 Learning Rule for MNN The complex valued output of the nth output neuron before applying nonlinear activation function is given by: (1) z n = xn + j yn

Fully Complex Multiplicative Neural Network Model N

(

= ∏ W nk + jW nk k =1

R

I

)(X

R nk

) (B

+ j X nk + I

R nk

+ j B nk I

495

)

(2)

Let the fully complex activation function be

f ( z ) = u ( x, y ) + jv ( x, y )

(3)

The sum of squared errors produced by the network defines the cost function

E=

1 * ∑ e n en 2 n

(4)

2 1 ∑ e n 2 n

(5)

=

en = d n − on o n = f ( z n)

(6) (7)

on is the output of the n th neuron, d n is the n th desired symbol and the subscripts R and I denote the real and imaginary parts. The complex back propagation minimizes the cost function en by recursively adjusting the complex weights of the MNN, using an approximation to the gradient descent technique. The weight update equation is

W (n + 1) = W (n) + ΔW (n)

(8)

The update term is defined as

ΔW ( n) = −η Where η

∂E ∂W

is the learning parameter rate and

(9)

∂E ∂W

is the gradient of cost

function E with respect to the complex weight which is defined by:

Wnk = WnkR + jWnkI ∇Wnk E =

∂E ∂E +j R ∂Wnk ∂WnkI

(10) (11)

Using the chain rule of calculus we can write

∂E ∂E = R ∂Wnk ∂u n

⎛ ∂u n ∂x n ∂u ∂y n ⎜⎜ + n R R ⎝ ∂x n ∂Wnk ∂y n ∂Wnk

⎞ ∂E ⎟⎟ + ⎠ ∂v n

⎛ ∂v n ∂xn ∂v ∂y n ⎜⎜ + n R R ⎝ ∂x n ∂Wnk ∂y n ∂Wnk

⎞ ⎟⎟ (12) ⎠

496

K. Burse, R.N. Yadav, and S.C. Shrivastava

∂E ∂E = I ∂Wnk ∂u n

⎛ ∂u n ∂xn ∂u ∂yn ⎜⎜ + n I I ⎝ ∂xn ∂Wnk ∂yn ∂Wnk

⎞ ∂E ⎟⎟ + ⎠ ∂ vn

⎛ ∂vn ∂xn ∂v ∂y n ⎜⎜ + n I I ⎝ ∂xn ∂Wnk ∂yn ∂Wnk

⎞ ⎟⎟ (13) ⎠

Evaluating the partial derivatives from equation 2 as

∂xn = X nkR ∂WnkR ∂xn = − X nkI I ∂Wnk

and and

∂yn = X nkI R ∂Wnk ∂y n = X nkR I ∂Wnk

(14)

Equations 12 and13 can be written as

⎛ ∂u n R ∂u n I ⎞ ∂E ⎜⎜ X nk + X nk ⎟⎟ + ∂ ∂ x y n ⎝ n ⎠ ∂vn

⎛ ∂vn R ∂vn I ⎞ ⎜⎜ X nk + X nk ⎟⎟ ∂ ∂ x y n ⎝ n ⎠

(15)

⎞ ∂E ⎛ ∂vn ⎞ ∂E ∂E ⎛ ∂u n ∂u ∂v ⎜⎜ ⎜⎜ ( − X nkI ) + n X nkR ⎟⎟ + (− X nkI ) + n X nkR ⎟⎟ = I ∂Wnk ∂u n ⎝ ∂xn ∂y n ∂y n ⎠ ∂v n ⎝ ∂xn ⎠

(16)

∂E ∂E = R ∂Wnk ∂u n

Since the complex weight belongs to an output neuron

en = d n − on = δ n

(17)

δ n = δ nR + jδ nI

(18)

∂E ∂vn R I Substituting the values of δ n and δ n in equations 15 and 16, we can write

Where,

δ nR = −

∂E ∂un

and

δ nI = −

⎛ ∂u ⎞ ⎛ ∂v ⎞ ∂u ∂v ∂E = −δ nR ⎜⎜ n X nkR + n X nkI ⎟⎟ − δ nI ⎜⎜ n X nkR + n X nkI ⎟⎟ R ∂Wnk ∂y n ∂y n ⎝ ∂xn ⎠ ⎝ ∂x n ⎠ ⎞ ⎞ ⎛ ∂u ⎛ ∂v ∂E ∂v ∂u = −δ nR ⎜⎜ n ( − X nkI ) + n X nkR ⎟⎟ − δ nI ⎜⎜ n ( − X nkI ) + n X nkR ⎟⎟ I ∂Wnk ∂y n ∂y n ⎠ ⎠ ⎝ ∂ xn ⎝ ∂ xn

(19)

(20)

Fully Complex Multiplicative Neural Network Model

497

Combining equations 19 and 20

⎞ ⎞ ⎛ ∂v ⎛ ∂u ∂v ∂u ∇Wnk = −δ nR ⎜⎜ n X nkR + n X nkI ⎟⎟ − δ nI ⎜⎜ n X nkR + n X nkI ⎟⎟ ∂y n ∂y n ⎠ ⎠ ⎝ ∂x n ⎝ ∂x n ⎛ ∂u ⎞ ⎛ ∂v ⎞ ∂u ∂v − jδ nR ⎜⎜ n ( − X nkI ) + n X nkR ⎟⎟ − jδ nI ⎜⎜ n ( − X nkI ) + n X nkR ⎟⎟ ∂y n ∂y n ⎝ ∂x n ⎠ ⎝ ∂x n ⎠

(21)

The equations can further be simplified applying Cauchy-Riemann equations according to which

∂u n ∂v =− n ∂y n ∂xn

or

∂v n ∂u =− n ∂x n ∂y n

⎛ ⎛ ∂u ⎛ ∂u ∂v ⎞ ∂v ⎞ ⎞ ∇Wnk E = − X nk ⎜⎜ ⎜⎜ n − j n ⎟⎟δ nR + ⎜⎜ − n + j n ⎟⎟δ nI ⎟⎟ ∂xn ⎠ ∂yn ⎠ ⎠ ⎝ ∂yn ⎝ ⎝ ∂xn ∇Wnk E = − X nk f ' ( z n )δ n

(22)

(23)

(24)

The complex weight update is defined by

ΔWnk = η X nk f ' ( z n )δ n

(25)

The complex bias update is defined by:

ΔBnk = η f ' ( z n )δ n

(26)

We have used a single layer MNN network with 3 input nodes, 3 hidden layer nodes and 1 output node. The activation function at each node is fully complex tanh function. The output y is calculated as:

sinh z e z − e − z y = f ( z ) = tanh z = = cosh z e z + e − z

(27)

n

z = ∏ ( wi xi + bi ) i =1

f ′( z ) =

d tanh z = sec h 2 z = 1 (1 + y )(1 − y ) dz 2

(28)

498

K. Burse, R.N. Yadav, and S.C. Shrivastava

The weight update for the output layer is given by:

ΔW = −η ΔW = −η

∂E = −ηf ′( z )δ n z ∂W

1 z ( y − d )(1 + y )(1 − y ) xi 2 ( wi xi + bi )

where, η is the learning rate and

(29)

d is the desired signal. The bias is updated as

1 z ΔB = −η ( y − d )(1 + y )(1 − y ) 2 ( wi xi + bi )

(30)

The weights are updated after each epoch. Similarly the error is back propagated to find the weight and bias update for the hidden layer.

4 Experiments and Results First a split complex tanh function is used to process the real and imaginary part of the signal separately [16]-[17]. A sequence of 1000 symbols is generated where each one is equiprobable. The incoming signals xi are multiplied by the corresponding weights wi of the links and a bias term bi is added. These terms are then multiplied to form the net input, to the neuron, which is subjected to a nonlinear bipolar sigmoidal activation function to create the final output. The convergence curves for split complex MNN network for 16 QAM and 32 QAM signals at 8 dB SNR is plotted in Fig. 1.

Fig. 1. Convergence curves for split complex MNN

The convergence curves for fully complex MNN network for 16 QAM and 32 QAM signals is plotted in Fig. 2. As compared to split complex equalizer the fully complex MNN equalizer converges faster. To study the BER performances the equalizer structure was trained with 100 iterations and tested over 10000 samples. 4

Fully Complex Multiplicative Neural Network Model

499

QAM signals were passed over a nonminimum phase stationary channel with the following transfer function: −1

CH: 0.447 + 0.894 z The nonlinearity introduced is NL: b(k ) = a(k ) + 0.2a 2 (k ) − 0.1a3 (k )

The data set was pre-processed by normalizing them between 0.1 and 1. In all simulations, the results reported are the average of several runs in each case. The architecture of the split complex MNN is 6 × 6 × 2 where as the architecture of the fully complex MNN is 3 × 3 × 1 . The BER performance for various values of SNR is plotted in Fig. 3.

Fig. 2. Convergence curves for fully complex MNN

Fig. 3. BER vs SNR for nonminimum phase stationary channel

500

K. Burse, R.N. Yadav, and S.C. Shrivastava

5 Conclusions A fully complex learning rule for feed forward MNN equalizer is proposed in this paper. The architecture of the fully complex neural network equalizer is compact as compared to the split complex neural network. The algorithm is tested on 4 QAM, 16 QAM and 32 QAM signals and shows faster convergence and improved performance in terms of BER as compared to split complex MNN equalizer.

References 1. Patra, J.C., Beng, P.W., Chaudhari, N.S., Das, A.: Nonlinear Channel Equalization with QAM Signal using Chebyshev Artificial Neural Network. In: Proc. International Joint Conference on Neural Networks, Montreal, Canada, July 31 - August 4, pp. 3214–3219 (2005) 2. Chen, S., Mulgrew, B., Grant, P.M.: A Clustering Technique for Digital Communications Channel Equalization using Radial Basis Function Networks. IEEE Trans. Neural Netw. 4(4), 570–579 (1993) 3. Kechriotis, G., Zervas, E., Manolakos, E.S.: Using Recurrent Neural Networks for Adaptive Communication Channel Equalization. IEEE Trans. Neural Netw. 5(2), 267–278 (1994) 4. You, C., Hong, D.: Adaptive Equalization using the Complex Backpropagation Algorithm. In: IEEE International Conference on Neural Networks, Washington, DC, USA, June 3-6, vol. 4, pp. 2136–2141 (1996) 5. Chen, S., Mclaughlin, S., Mulgrew, B.: Complex-valued Radial Basis Function Network, Part II: Application to Digital Communications Channel Equalization. Signal Processing 36, 165–188 (1994) 6. Cha, I., Kassam, S.A.: Channel Equalization using Adaptive Complex Radial Basis Function Networks. IEEE J. Sel. Area Communication 13(1), 122–131 (1995) 7. Deng, J., Sundararajan, N., Saratchandran, P.: Communication Channel Equalization using Complex-valued Minimal Radial Basis Function Neural Networks. IEEE Transactions on Neural Networks 13(3), 687–697 (2002) 8. Ming-Bin, L., Guang-Bin, H., Saratchandran, P., Sundararajan, N.: Fully Complex Extreme Learning Machine. Neurocomputing 68, 306–314 (2005) 9. Giles, C.L., Maxwell, T.: Learning, Invariance and Generalization in High-order Neural Networks. Applied Optics 26(23), 4972–4978 (1987) 10. Schmitt, M.: On the Complexity of Computing and Learning with Multiplicative Neurons. Neural Computing 14(2), 241–301 (2002) 11. Yadav, R.N., Kalra, P.K., John, J.: Time Series Prediction with Single Multiplicative Neuron Model. Applied Soft Computing 7, 1157–1163 (2007) 12. Yadav, R.N., Singh, V., Kalra, P.K.: Classification using single neuron. In: IEEE Int. Conf. on Industrial Informatics, Banff, Alberta, Canada, August 21-24, pp. 124–129 (2003) 13. Georgiou, G., Koutsougeras, C.: Complex Domain Backpropagation. IEEE Trans. Circuits Systems II: Analog Digital Signal Process. 39(5), 330–334 (1992) 14. Kim, T., Adali, T.: Fully Complex Backpropagation for Constant Envelope Signal Processing. In: IEEE Signal Processing Society Workshop, Sydney, Australia, December 11-13, pp. 231–240 (2000)

Fully Complex Multiplicative Neural Network Model

501

15. Kim, T., Adali, T.: Complex Backpropagation Neural Network using Elementary Transcendental Activation Functions. In: IEEE International Conference on Acoustics, Speech and Signal Processing, Salt Lake City, UT, USA, May 7-11, vol. 2, pp. 1281–1284 (2001) 16. Kavita, B., Yadav, R.N., Shrivastava, S.C.: Complex Channel Equalization using Polynomial Neuron Model. In: Proc. IEEE 3rd Int. Symposium on Information Technology, Kuala Lumpur, Malaysia, August 26-29, pp. 771–775 (2008) 17. Kavita, B., Yadav, R.N., Shrivastava, S.C., Singh, K.V.P.: A Compact Pi Network for Reducing Bit Error Rate in Dispersive FIR Channel Noise Model. International J. of Electronics, Circuits and Systems 3(3), 150–153 (2009)

Visual Navigation of a Novel Economical Embedded Multi-mode Intelligent Control System for Powered Wheelchair Jizhong Liu1,3,*, Xuepei Wu1,2, Jiating Xia1, Guanghui Wang3, and Hua Zhang1 1

School of Mechanical and Electrical Engineering, Nanchang University, Nanchang, 330029 China Fax: 86-791-3969630 [email protected] 2 School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore, 639798 Singapore [email protected] 3 Department of Electrical and Computer Engineering, University of Windsor, Windsor, N9B3P4 Canada [email protected]

Abstract. In order to assist different kinds of disabled people and senior citizens, and to improve the effectiveness and convenience of human-machineenvironment interaction, a novel economical embedded multi-mode control system for intelligent wheelchair is developed which is based on a highpowered 16-bit single chip system. In this paper the real-time image processing and control strategies for the visual navigation mode of the embedded system are studied. A less data image array, an adaptive recognition algorithm, a fuzzy control strategy and fusion control strategies with other modes are addressed. Some verifying experiments are carried out and results are given. Keywords: Visual navigation, Embedded control system, Intelligent wheelchair, Multi-mode interaction, Image processing.

1 Introduction With the increase in proportion of senior citizens in the population and the increase in number of disabled persons due to diseases and accidents, providing safe and economical vehicles with high performance and convenient communications is becoming one of the mostly concerned social problems[1, 2]. That the intelligent wheelchair is economical is very important for its wide use, especially in developing countries. Usually the price of an electric wheelchair ranges from $1,000 to $2,000. So it should not be increased too much for its intelligent control systems. Many different approaches including voice communication[3, 4], visual navigation[5-7], and *

Corresponding author.

Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 503–511. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

504

J. Liu et al.

automatic obstacle avoidance[8-10] etc have been proposed to improve the intelligence of human-machine-environment interaction for powered wheelchairs. Most of them are aimed at special applications and often based on laptop or PC systems. In order to improve the effectiveness, convenience and portability for different users and situations, we developed an economical embedded multi-mode intelligent control system for electric wheelchairs which is based on a high-powered 16-bit single chip system ------ SPCE061A. The system has four interactive modes, i.e. Manual Operation Mode (MM), Speech Communication Mode (SM), Visual Navigation Mode (VM), and Autonomous Running Mode (AM). Compared with the laptop or personal PC based systems, the proposed system has lower cost and more compact size. The total increased cost of our intelligent system is about $80 in contrast with other systems which are usually over several hundreds. As we all know, visual interactive communication is one of the most important human-machine-environment interaction mode. So does it in intelligent wheelchairs. Based on a personal computer, Ivanchenko[5] and Oda Naoki [6, 7] et al proposed two intelligent control systems for visual navigation of wheelchairs. However it is difficult to implement visual navigation in single chip system, especially multiinteractive modes in one system. In this study, we try to build the visual navigation techniques and module of the embedded system. Real-time image processing based on subsampled less data image array, adaptive recognition algorithm for marked line following, fuzzy control strategy of visual navigation and fusion control strategies with other modes will be discussed in the paper.

2 Structure of Visual Navigation Module The hardware structure of visual navigation module is shown in Figure 1, which includes visual sensor, image processing module, control and decision module, SPCE061A, D/A module and FS electric wheelchair platform. The visual sensor is a CMOS camera which is fixed on the crutch underneath the cushion according to the experimental result of vision field. There are two lights beside the sensor to assist the visual navigation running at night or in dark environment. Image processing module performs line and frame scanning of the vision field and the related computation. Decision and control are made by the control and decision module in SPCE061A to control the motion of the wheelchair. The control is based on the deviation between the center-axis of the marked line and the center-axis of the wheelchair. The control principle (software structure) of visual navigation mode is shown in Figure 2. The detailed information will be depicted in the following sections.

Enviro -nment

Visual Sensor

Image Processing Module

SPCE061A

D/A Module

Fig. 1. Hardware structure of visual navigation module

FS Electric Wheelchair Platform

Visual Navigation

505

Amplifying and Executive Element Object Arithmetic Element Deviation Control Variable DC Output Electric Controller Motor Wheelchair Position Velocity Voltage Signal Deviation

Comparing Element Input Reference + Central Position _ C of Wheelchair

Feedback Signal Central Position of Marked-line

Vision Sensor

Fig. 2. Control principle (software structure) of visual navigation mode

3 Image Data Acquisition Image data acquisition is based on the single chip system. Confined to its RAM memory and data handling capacity, an effective and subsampled less data image array is adopted for the navigation. According to the size of wheelchair, the position and vision range of camera and the analysis of tracking experiments, only 6 out of 380 lines and 24 pixels on each of those are actually captured and applied. In other words, the subsampled image resolution for microprocessor is 6×24 so as to improve the processing speed (See Figure 3: The two lines in the figure stand for one image line and the underlined data are the black tracking line image pixel data.). The waveform of the image analog signal is shown in Figure 4. To acquire the required signal, it is very important to catch the signal of vertical sync (VSYNC), horizontal ref (HREF) and pixel clock (PCLK), as shown in Figure 5, from the CMOS sensor promptly and accurately. In our implementation, pins of IOA8-IOA15 and IOB2IOB4 from SPCE061A are used in the process.

Fig. 3. Actual image and data array of vision field

Fig. 4. Viedo signal from visual sensor

Fig. 5. Timing signal from visual sensor

506

J. Liu et al.

4 Data Processing 4.1 Adaptive Threshold Algorithm for Marked Line Recognition To simplify computing process and therefore improve the capability of real-time, a simple gray-level threshold method is employed in recognizing the visual guidance line. The color of the pixel depends on its gray-level. When the gray-level is lower than the pre-defined threshold, the pixel is classified as the guidance line. Otherwise, it is assigned to the background, i.e., DP < DT ⇒ PP ∈ PL

(1)

DP ≥ DT ⇒ PP ∈ PB

Where DP denotes the gray-level of a single pixel PP collected from image and DT denotes the gray-level threshold that is defined beforehand. PL and PB indicate the pixel of the tracking line and the pixel of the background respectively. The gray-level of PL and PB is not fixed when light intensity is varying. Consequently, DT should be automatically regulated according to the environment so as to the wheelchair can perform properly in different light conditions, i.e., DT = DP min + λ ( DP max − DP min )

(2)

30 20 10 0

5

10

15

20

25

100

50 0 0

5

10

15

20

25

Gray-level value (decimal)

40

Gray-level value (decimal)

Gray-level value (decimal)

Where λ is the parameter for adaptive threshold. Normally λ is set to be greater than 0.05 and smaller than 0.1. Figure 6 shows the gray-level data of the image (including the background and the marked guidance line) under different light intensities. 250 200 150 100 50 0 0

5

10

15

20

25

Fig. 6. Gray-level data under different light conditions

(Left: Gray-level data from a line of image data array in weak light condition; Middle: Gray-level data from a line of image data array in medium light condition; Right: Gray-level data from a line of image data array in intense light condition. The data which the arrow indicates in each figure are the gray-level data from the marked line and others higher in each figure are from the background.) 4.2 Abstraction of Marked-Line Center and Removing Noise AS depicted above, the control strategy of visual navigation mode is based on the position deviation between the center of the wheelchair and the center of the tracking line. It is important to get the central-axis data of the marked line firstly. The following procedure explains how the data are obtained.

Visual Navigation

507

1) Get the value of DP and DT. If DP is not less than DT, the PP (Pixel Point) is of the background. 2) If DP is less than DT, it needs a further judgment. If the gray-level of the continuous N pixels (N =5 according to the experiments) are all less than DT, we think the pixel is belongs to the marked line. We take the first pixel which meets the requirement as the beginning pixel point of the marked line noted as Ledge_data. If the pixel data does not meet the requirement, we think it is a noise data. The last continuous qualified pixel of the same line is noted as Redge_data. From the Ledge_data and Redge_data, we can get the width of tracking line ------ Lwidth_data. 3) We may get more than one Ledge_data and Redge_data in the same line because of the bad tracking line (It means that there may have noise data in the marked line data.). In this situation, we take the beginning Ledge_data and the final Redge_data as the source data of Lwidth_data. 4) In order to further reduce the noise influence, we take the average width of the 6 lines in the image data array as the final Lwidth_data. It is defined as: Lwidth_data = (Lwidth_data1 + Lwidth_data2 + Lwidth_data3+ Lwidth_data4+ (3) Lwidth_data5+ Lwidth_data6)/6 4.3 Feature Parameters Abstraction of Marked-Line The objective of image processing is to obtain the parameters of the tracking line the position error d and the angle error θ between wheelchair and marked-line, as shown in Figure 7.

Fig. 7. Control principle of wheelchair turning

d = Lcenter _ data 6 − C

(4)

θ = arctan(H x /( H y − D))

(5)

Line 1 – 6 (corresponding to line 96, line 120, … , line 216 respectively) are the selected lines of the data array equispaced. Line 96 is the first valid acquired line

508

J. Liu et al.

because of the spent control time. Line 216 is the acquired line in the image corresponding with the pixel distance of the wheelchair moving in one control period. Lcenter_data6 indicates the central position of the sixth line which is the nearest one to the wheelchair in image data array. C is the value in the image which is in accordance with the central position of wheelchair after the camera having been fixed on the system. D denotes the pixel distance of the wheelchair to move forward in one control period. Hx is the far pixel distance of the first fine in data array to the center line of the wheelchair. Hy is the valid pixel distance of the whole image.

5 Control Strategy 5.1 Visual Navigation Control Strategy Based on Fuzzy Logic The FS electric wheelchair is a wheeled mobile robot and its kinematic model is two universal followers in the front and two drive wheels in the back with differential drive. The visual navigation mode of the intelligent wheelchair aims to coordinate motion direction with appropriate velocity so that the wheelchair can follow the route accurately and stably. A fuzzy logic controller is adopted to set up a mapping from position/angle error to control signal. The basic principle is that, the larger the error, the greater the control adjustment, and vice versa. 1) Input and output variables The first step of control is to get the inputs variables, angle error θ and position error d, and determine what is the appropriate degree of these inputs belong to in the fuzzy sets. The outputs are four analog voltage signals which correspond to the running velocity and turning angle of the wheelchair. The velocity is related with the system (user) initial setup. Here we only discuss the adjustable angle φ. 2) Fuzzification of variables Suppose the marked-line is a kind of continuous and smooth curve, the angle between the moving direction of wheelchair and marked-line θ is normally within the range of ± 60° . The universe of discourse of input variables θ and d are [− 60°,60°] and [-12, 11] respectively. The values applicable to angle φ are − 60° , − 40° , − 20° , 0° , 20° , 40° and 60° corresponding to 7 levels of control information as shown in Table 2. As illustrated in Table 1, 7 fuzzy subsets are defined for all variables, i.e., Negative Big (NB), Negative Medium (NM), Negative Small (NS), Zero (ZE), Positive Small (PS), Positive Medium (PM), Positive Big (PB), with the value [-3, -2, -1, 0, 1, 2, 3]. Triangle-shape grade of membership functions are adopted as shown in Figure 8. The control rules are presented in Table 2. The first row is the degree of membership of the fuzzy variable d. The first column is the degree of membership of the fuzzy variable θ. The information beneath the first row and to the left of the first column is the control information of the adjustable output variable φ. Figure 9 are the experiments carried out under different light conditions to show the robustness of proposed control strategy.

Visual Navigation

509

Table 1. Degree of membership for elements of fuzzy subset

Fuzzy variable Descrip tion

d θ

NB

NM

Far Left (-3) [-11,-8]

Second Left (-2) [-11,-4]

[-60,-40]

[-20,60]

NS

ZE

PS

Left (1) Middle(0) Right(1) [-8,0]

[-4,4]

[0,8]

[-40,0]

[-20,20]

[0,40]

PM

PB

Second Far Right Right (2) (3) [4,12] [8,12] [20,60]

[40,60]

Table 2. Rules table

NB NM NS ZE PS PM PB

NB PB PB PB PB PM PS PS

NM PB PB PB PM PS ZE ZE

NS PM PM PM PS ZE ZE NS

ZE PM PM PS ZE NS NM NM

PS PS ZE ZE NS NM NM NM

PM ZE ZE NS NM NB NB NB

PB NS NS NM NB NB NB NB Fig. 8. Membership function for variables

Fig. 9. Experiments of wheelchair running under different light conditions (Left: bright; Middle: medium; Right: dark)

5.2 Multi-mode Fusion and Security Measures Related with VM To make sure that the wheelchair works under visual navigation mode reliably and safely and to improve the effectiveness and fluency of human-machine-environment interaction, multi-mode fusion control strategy with navigation mode and security measures are adopted. 1) Manual operation mode has the first priority to control. It is in accordance with the reaction of normal users when facing emergency situations according to the experiments. The wheelchair will switch to MM as soon as the user manipulates joystick when running under vision-guided mode. 2) The visual navigation mode integrates with environment detection function of AM and voice hints function of SM. In this way, the wheelchair is able to be aware of the obstacle or pit ahead, stop automatic and inform user verbally while following the route.

510

J. Liu et al.

3) When the wheelchair finds no marked-line or meets the stop-sign in the route, the wheelchair will stop automatic and give corresponding voice hints to the user. 4) An emergency treadle-switch is designed to cut off energy supply when an emergency situation happens, so that the wheelchair can be immediately stopped by brake. Figure 10 shows the effect of employing multi-mode fusion and protective safeguards in two scenarios, confronting an object or approaching the termination of route (There is no stop-sign. It is a case that can not find the tracking line.), where the wheelchair will stop and give voice hints.

Fig. 10. Experiments of multi-mode fusion and security measures (Left: obstacle in front; Right: end of route)

6 Summary This paper briefly introduced SPCE061A-based visual navigation interactive mode of an intelligent wheelchair and the corresponding module of the multi-mode embedded control system. Hardware structure, image acquisition and data processing, adaptive recognition algorithm, fuzzy control strategy, multi-mode fusion and other safeguard measures are discussed in detail and verified experimentally. Because of the lower cost of use and compact size, and with the excellent mobility, convenience interaction, and the increasing demand of creating a harmonious society which is paying more attention to senior citizens and disabled folks, the economical intelligent wheelchair will be widely applied in future. Acknowledgments. Thanks to the reviewers for the valuable comments. The work is supported by the Foundation of Jiangxi Provincial Department of Science and Technology under the grant no. 2008BA00400.

References 1. Service robot key project information on “863” program in advanced manufacturing technologies, http://www.most.gov.cn/ 2. Ko, E., Ju, J.S., Kim, E.Y., Goo, N.S.: An Intelligent Wheelchair to Enable Mobility of Severely Disabled and Elder People. In: 2009 Digest of Technical Papers International Conference on Consumer Electronics (ICCE 2009), pp. 1–2. Institute of Electrical and Electronics Engineers Inc., Las Vegas (2009)

Visual Navigation

511

3. Qadri, M.T., Ahmed, S.A.: Voice Controlled Wheelchair Using DSK TMS320C6711. In: 2009 International Conference on Signal Acquisition and Processing (ICSAP 2009), pp. 217–220. Listed in IEEE Xplore, Kuala Lumpur (2009) 4. Simpson, R.C., Levine, S.P.: Adaptive Shared Control of a Smart Wheelchair Operated by Voice Control. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 1997), pp. 622–626. Listed in IEEE Xplore, Grenoble (1997) 5. Ivanchenko, V., Coughlan, J., Gerrey, W., Shen, H.Y.: Computer Vision-based Clear Path Guidance for Blind Wheelchair Users. In: The 10th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2008), pp. 291–292. Association for Computing Machinery, Halifax (2008) 6. Oda, N., Shimizu, H.: Vision Based Control for Power Assist Motion of Wheelchair Robots. In: 2006 SICE-ICASE International Joint Conference, pp. 5323–5328. Inst. of Elec. and Elec. Eng. Computer Society, Busan (2007) 7. Shimizu, H., Oda, N.: Vision Based Power Assist Control for Functional Behavior of Robotic Wheelchair. In: Proceedings of the 4th IEEE International Conference on Mechatronics (ICM 2007), pp. 1–6. Inst. of Elec. and Elec. Eng. Computer Society, Kumamoto (2007) 8. Fattouh, A., Nader, S.: Preliminary Results on Dynamic Obstacle Avoidance for Powered Wheelchair. In: International Conference on Information & Communication Technologies: from Theory to Applications Information and Communication Technologies, pp. 849–853. IEEE Press, Damascus (2006) 9. Trieu, H.T., Nguyen, H.T., Willey, K.: Shared Control Strategies for Obstacle Avoidance Tasks in an Intelligent Wheelchair. In: Proceedings of the 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS 2008 Personalized Healthcare through Technology, pp. 4254–4257. Inst. of Elec. and Elec. Eng. Computer Society, Vancouver (2008) 10. Tian, Z.H., Xu, W.H., Liu, X.H.: Obstacle Avoidance Method Study of Electric Wheelchair. In: 2009 International Workshop on Intelligent Systems and Applications, pp. 1–4. IEEE Computer Society, Wuhan (2009)

Neural Networks L2-Gain Control for Robot System Zhi-gang Yu1,2, Yong-liang Shen1, Shen-min Song2, and Da-wei Zhang2 1 2

Key lab of Electronic Engineering, College of Heilongjiang Province Harbin Inistituate of Technology, Harbin 150001, Heilongjiang, China [email protected]

Abstract. A new L2-gain disturbance rejection controller and adaptive adjustment are combined into a hybrid robust control scheme, which is proposed for robot tracking control systems. The proposed controller deals mainly with external disturbances and nonlinear uncertainty in motion control. A neural network (NN) is used to approximate the uncertainties in a robotic system. Meanwhile, the approximating error of the NN is attenuated to a prescribed level by the adaptive robust controller. The adaptive techniques of NN will improve robustness with respect to uncertainty of system, as a result, improving the dynamic performance of robot system. A simulation example demonstrates the effectiveness of the proposed control strategy. Keywords: neural network, L2-gain control, robot system.

1 Introduction In the presence of uncertainties of the robot system, several robust control strategies [1],[2],[3] provide asymptotic motion tracking and an ultimate bounded force error. However, an accurate mathematical model for a complex nonlinear system is very difficult to be achieved and costly to be developed under realistic situations. In order to account for the system uncertainties and disturbance, many adaptive neural network controllers are proposed. Such as adaptive controllers for robots have been employed by Kung and Hwang[4]. Some robust adaptive neural networks controller have been applied to both of the problems indicated above [4],[5],[6],[7]. However, the dynamic performance does not be improved with time, since the controller lacks the capacity to compensate for the nonlinearity, such as, friction is a ubiquitous phenomenon in mechanical systems that is difficult to model and often requires to be compensated. In this work, Lyapunov theory is used to provide a framework for stability. In conjunction with the Lyapunov theory, an adaptive learning scheme is introduced in order to improve the dynamic performances of robot system. Considering uncertainty in both environment and robot dynamics, the proposed robust controller with adaptive adjustment technique deals mainly with external disturbances and nonlinear uncertainty in motion control for the tracking control of the robot system. The simulations to a two-link robotic manipulator subject to uncertainties are performed to demonstrate the properties of the developed controller. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 513–520. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

514

Z.-g. Yu et al.

2 The Dynamics of the Robot System and Problem Formulation Consider the model describing the dynamics of an n-degrees-of-freedom rigid robot manipulator

τ = M ( q ) q + C ( q, q ) q + G ( q ) + Δ ( q, q ) + d .

(1)

where q ∈ R n , q ∈ R n and q ∈ R n represent the vectors of joint position, velocity and acceleration for robot system, respectively, τ ∈ R n are the vector of joint input torques (control input), M ( q ) ∈ R n×n is the symmetric and positive-definite inertia

matrix, C ( q, q ) ∈ R n×n is the vector of gravitational torques. Δ ( q, q ) is the model error, d represents the uncertain torque caused by friction forces, vibration, backlash and elasticity. In terms of the modeling uncertainties in a robotic system, firstly, we compensate the nonlinear component Δ ( q, q ) + d . Let τ = u + M ( q ) q + C ( q, q ) q + G ( q ) , where u is the output of the controller. Then, the tracking error dynamics is founded by u = M ( q ) e + C ( q, q ) e + Δ ( q, q ) + d .

(2)

where e ( t ) = q ( t ) − qd ( t ) , and qd ( t ) is the desired trajectory of the robot system. In order to represent the ability of anti-disturbance, we define the following reference signal z R . Thus, we consider an adaptive tracking performance criterion and formulate the robust tracking problem as follows: Given any smooth bounded desired motion trajectory, the robot system (1) uses a feedback controller u = ϕ ( qd , qd , e, e ) , for given positive constant γ > 0 , p1 ≥ 0 and p2 ≥ 0 , so that the L2 gain of the closed loop system with uncertainty Δ ( q, q ) + d is as small as possible

or less than a given constant γ , namely, satisfies zR

2

< γ Δ ( q, q ) + d 2 .

Based on the learning control strategy and the approximation capability of neural networks, a novel control scheme with learning strategy will be proposed. The adaptive law of the adjustable parameter vector in the neural networks adaptively compensates for the approximation error of neural network. As result in improving the transition performance of robot system described by (1). So that the ultimate result under the robust controller is that the tracking error e ( t ) = ( e ( t ) , e ( t ) ) will be attenuated to an arbitrarily small residual tracking error set. Lemma 1 (HJI inequation): Consider a nonlinear closed-loop system x = f ( x ) + g ( x ) ω , where x ∈ R n denotes state variable, f ( x ) , g ( x ) denotes a

vector function, z = h ( x ) is a reference signal. For a given positive constant γ > 0 , if

there exists a positive function L ( x ) ≥ 0 that satisfy the following HJI inequation: ∂L ∂L ∂L 1 1 2 L = x = f ( x) + g ( x ) ω ≤ γ 2 ω − zR ∂x ∂x ∂x 2 2

system is less than γ .

2

, such that the L2 gain of the closed-loop

Neural Networks L2-Gain Control for Robot System

515

3 Function Approximation by Neural Networks The NN approximation error ε f is a critical quantity, representing the minimum possible deviation of the ideal approximator W f *φ f ( x ) from the unknown smooth

function f ( x ) , where φ f ( x ) denotes the vector of the radial basis function (RBF) of

RBF neural networks, and W f * denotes the weight vector. Universal approximation results for radial basis function NN indicate that, if NN node number is sufficiently large, then ε f can be made arbitrarily small on a compact region [8],[9]. The ideal weight vector W f * is an “artificial” quantity required for analytical purposes. It is assumption that W f * is defined as the value of W f that minimizes ε f

for all

x ∈ VB ⊂ R in a compact region, i.e., m

⎧ ⎫ W f * := arg minN ⎨sup f ( x ) − W f φ f ( x ) ⎬ . W ∈R ⎩ x∈Ω x ⎭ In general, the ideal NN weight W f * is unknown and needs to be estimated. In this paper, we shall consider Wˆ f being the estimate of the ideal NN weight W f * .

4 Robust Tracking Control and Neural Networks Online Adaptive Adjustment Technique The control problem is to find a control law such that the state can track the desired trajectories in the presence of the uncertainties. Let Δf ( q, q ) = Δ ( q, q ) + d in robot system (1), then Δf ( q, q ) can be approximated by a neural network. So we have Δf ( q, q ) = Wˆ f φ f + ε f . The input of neural network is joint positions and their derivative; the output of neural network is the uncertainty Δf ( q, q ) . If the parameter adjustment mechanism of the neural network reduces the approximation error, then the tracking error can go to zero. Substituting Δf ( q, q ) = Wˆ f φ f + ε f into (2), then the dynamics equation of the closed-loop system becomes u = M ( q ) e + C ( q, q ) e + Wˆ f φ f + ε f .

(3)

where the approximation ε f is regard as the exterior disturbance, and we define the reference signal z R = pe , where p ≥ 0 are the given coefficient. So that L2 -gain is J R = sup z R ε f ≠0

ε f . We define two state vectors x1 = e , x2 = e + α e , where α is a

given positive constant. So we rewrite (3) as

516

Z.-g. Yu et al.

⎧⎪ x1 = x2 − α x1 . ⎨ ˆ ⎪⎩ Mx2 = u − Cx2 + ω + W f φ f + ε f

(4)

where ω = M α e + Cα e . Consider the positive-define Lyapunov candidate L=

{

}

T 1 T 1 1 x1 x1 + x2T Mx2 + tr (W − W * ) (W − W * ) . 2 2 2

We obtain the derivative of L 1  + tr (W − W * )T W L = x1T x1 + x2T Mx2 + x2T Mx 2 2

{

}

(

)

{

}

T 1 = x1T ( x2 − α x1 ) + x2T u − Cx2 + ω + Wˆ f ϕ f + ε f + x2T ( M − 2C ) x2 + tr (W − W * ) W . 2

According to the property of the symmetry of robot system, namely, for an arbitrary vector x2 , satisfied x2T ( M − 2C ) x2 = 0 . So we have

(

{

)

}

T L = − x1T α x1 + x2T x1 + u − Cx2 + ω + Wˆ f φ f + ε f + tr (W − W * ) W .

Regarding the approximation error 1 H = L − γ 2 ε f 2

2

+

εf

as the external disturbance, and let

1 2 z R . Thus, 2

(

{

)

}

T 2 1 1 2 2 H = −α x1 + x2T x1 + ω − Wˆ f φ f + u − x2T ε f − tr (W − W * ) W − γ 2 ε f + p 2 x1 2 2 T 2 1 2⎞ 1 2 ⎛ * T T = − ⎜ α − p ⎟ x1 + x2 x1 + ω − Wˆ f φ f + u − tr (W − W ) W − x2 ε f − γ 2 ε f . 2 ⎠ 2 ⎝

(

)

{

}

If we choose the inequation as follows: 1 − x2T ε f − γ 2 ε f 2 =−

1 1 x2 + γε f 2 γ

2

1⎛ 1 = − ⎜ 2 x2 2⎝γ

2

+

1 x2 2γ 2

2

≤

2

+ 2 x2T ε f + γ 2 ε f

1 x2 2γ 2

2

−

1

γ2

2⎞ x2 ⎟ ⎠ .

2

Then we have

(

)

{

}

T 1 ⎞ 1 2 ⎛ H ≤ − ⎜ α − p 2 ⎟ x1 + x2T x1 + ω − Wˆ f φ f + u − tr (W − W * ) W + 2 x2 2 ⎠ 2γ ⎝

{

T ⎛ ⎞ 1 ⎞ 1 2 ⎛ = − ⎜ α − p 2 ⎟ x1 + x2T ⎜ x1 + ω − Wˆ f φ f + u + 2 x2 ⎟ − tr (W − W * ) W 2 ⎠ 2γ ⎝ ⎝ ⎠

}

2

.

(5)

Neural Networks L2-Gain Control for Robot System

517

Let u ( t ) = − x1 − ω + Wˆ f ( t ) φ f − ( ε 2 + 1 2γ 2 ) x2 .

(6)

Substituting (6) and ε1 = α − p 2 2 into (5), we obtain H ≤ −ε1 x1 − ε 2 x2 2

2

{

}

T − tr (W − W * ) W ≤ 0 .

(7)

Let the neural networks adaptive adjustment law as follow

W = −φ ( q, q ) x2T .

{

}

(8)

Because of tr − (W − W * ) φ ( q, q ) x2T = − x2T (W − W * ) φ ( q, q ) , than T

T

H ≤ − (α − (1 2 ) p 2 ) x1 Substituting (8) and

ε1 = α − p 2 2 ≥ 0

Therefore L ≤ 1 2 γ 2 ε f

2

− 1 2 zR

2

2

.

into (7), so that

H ≤ −ε1 x1 ≤ 0 , 2

is achieved. According to the Lemma 1, the L2

gain J R of the closed loop system (4) is less than γ . To assure that the weights of neural network W ( t ) are bounded, we must define

{

}

the set Ωb , so that the weights W ( t ) are satisfied Ωb = W ( t ) ≤ M b . According to the projection algorithm[7], we will perfect (8) as follow ⎧−φ ( q, q ) x2T , i f W < M b , or ( W = M b , and W Tφ ( q, q ) x2T ≥ 0 ) ⎪⎪ . W = ⎨ WW T T T T T    − φ ( q, q ) x2 , i f W = M b , and W φ ( q, q ) x2 < 0 ⎪−φ ( q, q ) x2 − 2 W ⎪⎩

(9)

According to the property of the projection algorithm, the performance of the original close-loop system is satisfied still. Theorem 1. Consider the robot system (1) using the following controller (6) and the robust adaptive law (9), where ε 2 > 0 is the given constant. The reference signal

z R = px1 satisfies ε1 = α − p 2 2 ( ε1 > 0 ), such that L2 -gain J R of the closed-loop system (4) and (5) is less than γ . See the design process for proof.

5 Simulation Study We demonstrate the proposed control scheme by the tracking control of a two-link robotic manipulator with 2 degrees of freedom in the rotational angles described by angles q1 and q2, The dynamic equations describing the motion of the robotic system are of the following form [10]

518

Z.-g. Yu et al.

⎡τ 1 ⎤ ⎡( m1 + m2 ) r12 + m2 r22 + 2m2 r1r2 c2 ⎢ ⎥=⎢ m2 r22 + m2 r1r2 c2 ⎣τ 2 ⎦ ⎣

m2 r22 + m2 r1r2 c2 ⎤ ⎡ q1 ⎤ ⎥ ⎢  ⎥ m2 r22 ⎦ ⎣ q2 ⎦

⎡ −m r r s q ( q + q ) ⎤ ⎡( m + m2 ) l1 gc2 + m2 l2 gc1 ⎤ ⎡ d1 ⎤ + ⎢ 2 1 2 2 1 12 2 ⎥ + ⎢ 1 ⎥+⎢ ⎥ m2 r1r2 s2 q1 m2 l2 gc12 ⎣ ⎦ ⎣ ⎦ ⎣d2 ⎦

.

where m1 , m2 , r1 = 0.5l1 and r2 = 0.5l2 , are the mass, the moment of inertia, the half-

length of link 1 and 2, g = 9.8 m / s , and shorthand notations c1 = cos ( q1 ) , s1 = sin ( q1 ) , c12 = cos ( q1 + q2 ) etc. In the control experiments described below, the

kinematics and inertial parameters of the arm are chosen as l1 = 1.2m , l2 = 0.8m , m1 = 4kg , m2 = 2kg , respectively. The initiation conditions are q ( 0 ) = [ 0.7 0.5] rad , q ( 0 ) = [ 0 0] rad/s , q ( 0 ) = [ 0 0] rad/s 2 . In (5), (6) and T

T

T

(7), the design parameters are given by α = 50 , ε1 = 0.1 , ε 2 = 0.1 , γ = 0.06 , η = 0.2 , p = 20 . The combined effects of friction and the uncertainty of the robot system (The model error) are defined by

⎡0.5sign ( q1 ) ⎡0.1 + exp ( − q1 ) ⎤ ⎤ ⎣ ⎦⎥ N×m . Δ=⎢ ⎢ sign ( q ) ⎡ 0.2 + exp ( − q ) ⎤ ⎥ 2 2 ⎣ ⎦ ⎦ ⎣ The external torque disturbance d = [ d1

d 2 ] is random disturbance. The sampling T

time of robot system is 0.005s . The results of simulations are showed in fig.1 and fig.2 respectively by the different the coefficient of l2-gain γ = 0.05 , γ = 0.1 . 0.1

1.5 Real position (γ=0.05)

Error of joint2 (γ=0.05)

1

0.05 1

q (rad)

Tracking error(rad)

Error of joint1 (γ=0.05)

0

Desired position

0.5 0 -0.5 -1

-0.05 0

1

2 Tim e(s )

3

4

0

1

2

3

4

Tim e(s )

(a) The curve of tracking position error

(b) the tracking curve of position for joint-1

1.5

2

q (rad)

Velocity of joint1(rad/s)

Real position (γ=0.05)

1

Desired position

0.5 0 -0.5 -1

0

1

2 Time(s )

3

4

6

Real velocity (γ=0.05) Desired velocity

4 2 0 -2 -4

0

1

2 Time(s )

3

4

(c) The tracking curve of position for joint-2 (d) The tracking curve of velocity for joint-1 Fig. 1. The simulation curve of robot system for γ = 0.05

Neural Networks L2-Gain Control for Robot System 400 τ1 (γ=0.05)

Real velocity (γ=0.05)

4

Desired velocity

200 τ (N•m)

velocity of joint2(rad/s)

6

2 0

τ2 (γ=0.05)

0 -200

-2 -4

519

-400 0

1

2 Tim e(s)

3

4

0

(e) The tracking curve of velocity for joint-2

1

2 Time(s)

3

4

(f) The joint control torque of robot

Fig. 1. (continued)

0.2

2

Position error(rad)

Error of joint1(γ=0.1)

Real position (γ=0.1) Desired poistion

Error of jiont2(γ=0.1)

1

q (rad)

0.1

1 0

0 0

1

2

3

-1

4

0

1

2

Tim e(s)

(a) The curve of tracking position error

Veolcity of joint1(rad/s)

Real position (γ=0.1)

1

2

q (rad)

Desired position

0

0

1

2

4

(b) the tracking curve of position for joint-1

2

-1

3

Time(s )

3

4

6

Real velocity (γ=0.1)

4

Desired velocity

2 0 -2 -4

0

1

2

3

4

TIm e(s)

Tim e(s )

6

400

Real velocity (γ=0.1)

4

τ1 (γ=0.1)

Desired velocity

2

τ (N•m)

Velocity of joint2(rad/s)

(c) The tracking curve of position for joint-2 (d) The tracking curve of velocity for joint-1

0

τ2 (γ=0.1)

200 0

-2 -4

0

1

2

3

4

Tim e(s )

(e) The tracking curve of velocity for joint-2

-200

0

1

2

3

4

Time(s )

(f) The joint control torque of robot

Fig. 2. The simulation curve of robot system for γ = 0.1

The simulation results have shown the expected performance. Thus the proposed control strategy can control the robotic system to follow the desired trajectory without using any prior information about the uncertainty.

520

Z.-g. Yu et al.

6 Conclusion The robust adaptive neural network control integrates with the learning control of neural network to solve the tracking control problem of robot systems. The Lyapunov synthesis approach is used to develop robust adaptive control schemes based on neural network. A main feature of the adaptive neural network control law and parameter update algorithms is the stability properties of the algorithm being independent of the specific mechanism used to achieve the tracking performance with a prescribed attenuation.

Acknowledgements This work is sponsored by the Science and Technology Project of Education Department of Heilongjiang Province (Grant No 11544036) and is supported by Science Foundation of Heilongjiang University (Grant No QL200706).

References 1. Hwang, M.C., Hu, X.: A Robust Position / Force Linearing Controller of Manipulators via Nonlinear H Control and Neural Networks. IEEE Trans. on System, Man, Cybernetics – Part B 30, 310–321 (2000) 2. Dixon, W.E., Zergeroglu, E., Dawson, D.M.: Global Robust Output Feedback Tracking Control of Robot Manipulators. Robotica 22, 351–357 (2004) 3. Wang, D., Soh, Y.C., Chean, C.C.: Robust Motion and Force Control of Constrained Manipulators by Learning. Automatica 31, 257–262 (1995) 4. Kung, S.Y., Hwang, J.N.: Neural Network Architectures for Robotic Applications. IEEE Transactions on Robotics and Automation 5, 641–657 (1989) 5. Bai, P., Fang, T.J., Ge, Y.J.: Robust Neural-Network Compensating Control for Robot Manipulator Based on Computed Torque Control. Control Theory and Applications 18, 895–903 (2001) 6. Nasisi, O., Carelli, R.: Adaptive Servo Visual Robot Control. Robotics and Autonomous Systems 43, 51–78 (2003) 7. Wai, R.J.: Robust Control for Nonlinear Motor-Mechanism Coupling System Using Wavelet Neural Network. IEEE Transactions on Systems Man and Cybernetics-Part B: Cybernetics 33, 489–497 (2003) 8. Park, J., Sandberg, I.W.: Universal Approximation Using Radial-Basis-Function Networks. Neural Computation 3, 246–257 (1991) 9. Liao, Y., Fang, S.C., Nuttle Henry, L.W.: Relaxed Conditions for Radial-Basis Function Networks to Be Universal Approximators. Neural Networks 16, 1019–1028 (2003) 10. Leung, T.P., Zhou, Q.J., Su, C.Y.: An Adaptive Variable Structure Model Following Control Design for Robot Manipulators. IEEE Trans. Automatic Control 36, 347–353 (1991)

∞

Neural Network Control of Spacecraft Formation Using RISE Feedback Shicheng Wang1 , Haibo Min1,2 , Fuchun Sun2 , and Jinsheng Zhang1 1

High-Tech Institute of Xi’an, Xi’an, Shaanxi, P.R. China [email protected] 2 State Key Laboratory of Intelligence Technology, Department of Computer Science and Technology, Tsinghua University, Beijing, P.R. China

Abstract. We address the problem of tracking relative translation in a leader-follower spacecraft formation architecture using Robust Integral of the Sign Error (RISE) based Neural Network (NN) technique. Based on the relative translational dynamic model of the spacecraft formation, RISE is introduced to approximate the dynamics of the follower as well as various practical disturbances. It is shown that the errors of the entire formation closed-loop are asymptotical stability (AS), which takes significant advantage over the typical Uniformly Upper Bounded (UUB) property of most NN controllers in high-precision formation tasks. Finally, numerical simulation is provided to verify the effectiveness of the proposed algorithm.

1

Introduction

The past few years have witnessed the burgeoning interest in the coordinated control of multi-agent. Indeed, there are many potential advantages of such systems over a single body, including greater flexibility, adaptability and performance. In the space industry, this concept makes the way for new and better applications. Potential applications arising in current studies include resolution enhancement through multiple-spacecraft SAR [2], interferometry and autonomous in-orbit assembly of large real structures. Various control strategies for spacecraft formations have been reported in the literature, including behavior based methods [3], virtual structure techniques [4] and leader-follower schemes [5]. It has been well recognized that the leaderfollower architecture is notably simple to implement and is especially suitable for high-precision synchronization tasks which entails tight formation [6]. In practical applications, complete knowledge of the spacecraft masses and other disturbances are hard to achieve, since they often change during operation. Obviously, the consideration of these undesirable elements during high-precision attitude synchronization is nontrivial. By virtue of the universal approximators property of NN [9], NN based controller is well suited to deal with various of unknown uncertainties within system dynamics. However, typical NN-based controllers yield global uniformly ultimately bounded (UUB) stability results Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 521–528. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com 

522

S. Wang et al.

(e.g., see [10, 11] for examples and reviews of literature) which may not be sufficient for high-precision spacecraft formation task. Therefore, in this paper, we propose a new asymptotically stable (AS) NN torque control law using an NN combined with the recently developed robust integral of sign of the error (RISE) feedback method originating in [12] and referred to as RISE feedback in [13]. Notation: The notation used throughout the paper is fairly standard. We denote x˙ the time derivative of a vector x, i.e. x˙ = dx dt , and · the Euclidian norm of a vector and the induced L2 norm of a matrix. When the context is sufficiently explicit, we may omit to write arguments of functions or matrices.

2

Relative Translational Model

From the fundamental equation of the two-body problem with an assumption of small spacecraft masses relative to the Earth, the nonlinear position dynamics can be represented in the Fl frame on the form [1] p˙ = v mf v˙ = F a + F d − C t (υ)v ˙ − D t (υ, ˙ υ¨, rf )p − nt (rl , rf )

(1)

where ml and mf are the masses of the leader and flower spacecrafts, the vectors pointing from the center of the Earth to the center of the follower and the leader orbit frame are denoted rf and rl , respectively. υ is the true anomaly of the leader spacecraft. C t (υ) ˙ is a skew-symmetric Coriolis-like matrix, Dt (υ, ˙ υ¨, rf )p may be viewed as a time varying potential force, and nt (rl , rf ) is a nonlinear term. The composite perturbation force F d and the relative control force F a m m are given by F d = f df − mfl f dl and F a = f af − mfl f al , where f d. ∈ R3 is the perturbation term due to external effects and is the actuator force. Readers are referred to [1] and [6] for more details on this model.

3 3.1

Rise Based NN Dynamic Controller Multilayer NN Estimation

A two-layer NN is considered in this paper, where one layer takes randomly assigned constant weight V ∈ R(a+1)×L with a inputs and L being the number of neurons in the hidden layers, the other layer takes tunable weights W ∈ R(L+1)×b with b outputs. The main feature of the NN lies in its general approximation property [16]. For any smooth function f (x), there always exists an NN such that f (x) = W T σ(V T x) + ε(x) for some ideal weights W , V , where σ is the activation function and the sigmoid is considered here. Since V is stochastically chosen, the activation function σ(V T x) forms a stochastic basis, and therefore the approximation property holds for all inputs, x ∈ Ra , in the compact set S [14]. Also, the functional approximation error is bounded such that ε ≤ εN , where εN is a known bound constant and is dependent on S. Note that other activation functions (e.g., hyperbolic tangent or radial basis) could also be used.

Neural Network Control of Spacecraft Formation Using RISE Feedback

523

Before proceeding, some important assumptions and properties are in order. Assumption 1. The nonlinear disturbance term and its first two time derivatives are bounded, i.e., F d (t), F˙ d (t), F¨ d (t) ∈ L∞ . Assumption 2. The desired relative position pd as well as its first, second and (3) third derivatives exist and are all bounded, i.e., pd (t), p˙ d (t), p¨d (t), pd (t) ∈ L∞ . √ Property 1. Bounds on NN quantities exist as W  ≤ WM , σ ≤ Nh , √ σ(σ − 1) ≤ Nh + Nh , where Nh ,WM are known bounded constants [15]. 3.2

RISE Based Controller Development

The control object is to ensure that the follower spacecraft to track a desired trajectory pd designated by the leader one, under the assumption that the leader Δ is ideally controlled. Similar to [13], we define two related errors as: e1 = ep := Δ

p − pd , e2 = e˙ 1 + α1 e1 and a filter tracking error r = e˙ 2 + α2 e2 , where α1 , α2 ∈ R+ . Multiplying r by mf , we obtain mf r = F a + F d − C t (υ)v ˙ − Dt (υ, ˙ υ¨, rf )p − nt (rl , rf ) + mf (α1 e˙ 1 + α2 e2 ) − mf v˙ d

(2)

After adding and subtracting terms Ct (υ)v ˙ d and Dt (υ, ˙ υ¨, rf )pd , (2) reduced to mf r = f d + T + F a + F d

(3)

where f d (pd , v d , v˙ d ) ∈ R takes the form f d = C t (υ)v ˙ d + Dt (υ, ˙ υ¨, rf )pd − mf v˙ d

(4)

and T = −C t (υ)v ˙ − Dt (υ, ˙ υ¨, rf )p − nt (rl , rf ) ˙ d − D t (υ, ˙ υ¨, rf )pd + mf (α1 e˙ 1 + α2 e2 ) − C t (υ)v

(5)

To facilitate the stability analysis, the time derivative of (3) is determined as mf r˙ = −m ˙ f r + f˙d + T˙ + F˙ a + F˙ d

(6)

By construction a three layer NN, the time derivative of (4) can be expressed as f˙d = W T σ(V T xd ) + ε(xd )

(7)

By general approximation property, ε(xd ) is the bounded NN construction error. Based on Assumption 2, the following inequalities hold: ε(xd ) ≤ εN and ε(x ˙ d ) ≤ εN , where εN , εN are known bounded constants, and the input is defined as xd = [1, pd , v d , v˙ d , v ¨d ] (8)

524

S. Wang et al.

Then, we introduce the RISE feed back into the previous open-loop system (2). Similar to [14], we define the RISE term as t µ = (ks + 1)e2 (t) − (ks + 1)e2 (0) +

[(ks + 1)α2 e2 (s) + βsgn(e2 (s))]ds (9) 0

and its time derivative is thereby µ˙ = (ks + 1)r + βsgn(e2 (t))

(10)

where ks , β ∈ R+ . To approximate the dynamics and cancel out the uncertainties in the system, we further introduce a new term in our control force, denoted fˆd , which is composed of a three layer NN and is generated as ˆ T σ(V T xd ) fˆ˙d = W

(11)

ˆ T ∈ R(L+1)×b is the approximation weight of W T and is generated by where W ˆ˙ = Kσ(V T xd )eT2 W

(12)

where K is a positive definite symmetric matrix and is bounded by K ≤ ˆ˙ is upper bounded FN ∈ R+ . By Property 1, it is not difficult to show that W √  ˆ˙  by W  ≤ FN Nh e2 . Then, the overall force imposed on the system can be defined as F a = −(fˆd + µ)

(13)

ˆ˙ by (10) and (11) By taking time derivative of (13) and replacing µ˙ and W respectively, we get ˆ T σ(V T xd ) − (ks + 1)r − βsgn(e2 (t)) F˙ a = −W

(14)

Using (14), the dynamics of the closed-loop system (6) can be formulated as 1 ˜ + N B1 + N B2 − e2 − (ks + 1)r − β1 sgn(e2 (t)) ˙ fr + N mf r˙ = − m 2

(15)

T ˜ = −1m ˙ ˙ ˜T ˜T where N 2 ˙ f r + T + e2 , N B1 = ε + F d , and N B2 = W σ(V xd ). W ˜ T = WT − W ˆ T . Following the same procedure of [13], the is determined by W ˜ can be derived by mean value theorem as upper bound of N      1  ˜  ˙ + e2  ≤ ρ(z) z − m ˙ (16) r + T N  =  f  2 

where z = [e1 e2 r]T and ρ(z) is a positive nondecreasing function. To facilitate our following stability analysis, some important bounds are established in the following lemma.

Neural Network Control of Spacecraft Formation Using RISE Feedback

525

Lemma 1. [15] N B1 , N B2 and their first derivatives are all bounded, and inequalities (17)-(20) hold for their upper bounds.     N B1  ≤ εN + max F˙ d  ≡ ςB1 (17)      ˙     N B1  ≤ εN + max F¨ d  ≡ ςB1 N B2  ≤ (WM + FN

(18)

  Nh e2 ) Nh ≡ ςB2

(19)

       ˙  ˆ N B2  ≤ FM Nh e2  + ( Nh + Nh )(WM + W  )c(t)

(20)

F

≡ ς3 e2  + ς4  , ςB2 , ς3 , ς4 are positive known constants. where ςB1 , ςB1

3.3

Stability Analysis

To facilitate our stability analysis, a auxiliary function L(t) ∈ R is defined similar [13] to be Δ

L(t) = rT (NB1 − β1 sgn(e2 )) + e˙ 2 (t)T NB2 (t) − β2 e2 (t)

2

(21)

then the following lemma is immediate in order. Lemma 2. Given γ = β1 e2 (0) − eT2 (0)N B3 (0) ∈ R, then provided inequalities β1 ≥ ςB1 + ςB2 + N B1 + N B2 .

ς4 α2

t

L(s)ds ≤ γ,

0

and β2 ≥ ς3 hold, where N B3 =

Proof. t

t L(s)ds =

0

t 2

e˙ 2 (s) (N B3 (t) − β1 sgn(e2 ))ds −

β2 e2 (s) ds

T

0

0

(22)

t α2 e2 (s)T (N B1 (t) − β1 sgn(e2 ))ds

+ 0

Employing integration by part to the first part of RH in (22) and ret placing it back again, then using inequalities (17)-(20) reveals L(s)ds ≤ β1 e2 (0) − e2 (0)T NB3 (0) − e2  β1 + t

t



0

α2 e2  ςB1 + ςB2 − β1 +

0

ς4 α2



ds +

e2 2 (ς3 − β2 )ds + e2  (ςB1 + ςB2 ). It is not difficult to show that

0

t 0

 ς4 ds ≤ 0 α2 e2  ςB1 + ςB2 − β1 + α2

(23)

526

S. Wang et al.

t 2

e2  (ς3 − β2 )ds ≤ 0

(24)

0

e2  (ςB1 + ςB2 ) − e2  β1 ≤ 0

(25)

provided the above two inequalities are satisfied. Then by using inequalities t (23)-(25), it is easy to verify that L(s)ds ≤ γ. This completes the proof. 0

Up to this point, we are ready to give the main results of this paper. Theorem 1. Given the relative translation dynamics for spacecraft formation model (1), and the force imposed determined by (13), let the NN weight updating law be (12), then there exist some constants β1 , β2 , α1 , α2 which satisfy the condition (23)-(25), such that the position tracking error ep as well as the velocity tracking error ev of the follower spacecraft are AS, and the estimation error of the weight matrix is bounded. Proof. See the similar proof of [13].

4

Numerical Simulation

In this section, numerical simulation is conducted in Mathematica 7.0 platform and the simulation results for a spacecraft formation are presented. In the simulation, the leader spacecraft is assumed to be following an elliptic orbit with eccentricity e = 0.6. Both spacecraft have mass m = 100 kg. The follower is assumed to have available continuous thrust in all directions, limited to 27 N.The follower has initial values p = [20, 10, −20]T , and is further commanded to track sinusoidal trajectories around the leader, given as

   T 3π 4π 5π t , 10 sin t , 5 cos t p∗ (t) = −10 cos T0 T0 T0 and the disturbance force is set to

   T c0 π 1 7c0 π 3c0 π − cos t , sin t , cos t F d (t) = 50 T0 T0 T0 where T0 is the leader orbital period and c0 = 10 approximately denoting the disturbance effect regarding the natural frequency. A possible scenario for this motion is in-orbit inspection, where the follower moves in orbit around the leader. The follower spacecraft mass is assumed to be known with a 30% accuracy. The simulation results are listed in Figure 1 and Figure 2. Δer , Δeθ and Δeh are the tracking errors along corresponding axis. The position tracking error after 200 steps is shown in Figure 1, and the velocity tracking error is illustrated in Figure 2. It can be seen that the follower settles at the desired trajectory and proceeds to track the trajectory. The UAS property of the closed-loop system is seen in the figure as persistent oscillations around the origin.

Neural Network Control of Spacecraft Formation Using RISE Feedback

527

−3

4

x 10

2

Pos.Error[m]

0

−2

−4 Δ e

r

−6

Δ eθ Δ eh

−8 200

400

600

800

1000

1200

1400

1600

1800

2000

Fig. 1. Position tracking error

−3

8

x 10

Δ e

r

Δ e

6

θ

Δ e

h

Vel.Error[m/s]

4 2 0 −2 −4 −6 −8 200

400

600

800

1000

1200

1400

1600

1800

2000

Fig. 2. Velocity tracking error

5

Conclusion

We present a solution to the spacecraft formation within the leader-follower architecture. Based on the newly developed relative translational dynamics between the leader and the follower, a RISE based NN controller is developed. By virtue of the powerful approximation property of NN and the elaborately designed RISE feedback term, UAS of the closed-loop can still be guaranteed even with the parameter uncertainties of the dynamics as well as various external disturbances being accounted for. Numerical simulation is provided to verify the effectiveness of the proposed algorithm. Our future research involves the extension of the simple two-spacecraft formation to multi-spacecraft and the extensive analysis on the propagation error induced thereby. Acknowledgments. The authors would like to thank Dr. Raymond Kristiansen for his insightful instruction on system modeling and simulation. The first and forth authors were supported by the National Natural Science Foundation of China (Grants No: 60874093, 60904083). And the second and third author were jointly supported by the National Science Fund for Distinguished Young Scholars (Grant No: 60625304) and the National Natural Science Foundation of China (Grants No: 90716021, 60621062).

528

S. Wang et al.

References 1. Kristiansen, R., Lor´ıa, A., Chaillet, A., Nicklasson, P.J.: Adaptive output feedback control of spacecraft relative translation. In: Proc. of the 45th IEEE Conf. on Decision and Control (2006) 2. Sarlette, A., Sepulchre, R., Leonard, N.E.: Cooperative attitude synchronization in satellite swarms: a consensus approach. In: 17th IFAC symp. automatic control in aerospace (2007) 3. Beard, R.W., Lawton, J., Hadaegh, F.Y.: A coordination architecture for spacecraft formation control. IEEE Transactions on Control Systems Technology 9, 777–790 (2001) 4. Ren, W., Beard, R.W.: Virtual structure based spacecraft formation control with formation feedback. In: Proc. of AIAA Guidance, Navigation, and Control Conference (2002) 5. Dimarogonas, D.V., Tsiotras, P., Kyriakopoulos, K.J.: Leader-follower cooperative attitude control of multiple rigid bodies. Automatica 58, 429–435 (2009) 6. Kristiansen, R.: Dynamic synchronization of spacecraft. Ph.D. dissertation, Norwegian University of Science and Technology (2008) 7. Queirox, M.S.D., Yan, Q., Yang, G., Kapila, V.: Global output feedback tracking control of spacecraft formationflying with parametric uncertainty. In: Proc. of the 38th IEEE Conf. on Decision and Control (1999) 8. Queiroz, M.S.D., Kapila, V., Yan, Q.: Adaptive nonlinear control of multiple spacecraft formation flying. Journal of Guidance, Control, and Dynamics 23, 385–390 (2000) 9. Hornik, K., Stinchcombe, M., White, H.: Multilayer feedforward networks are universal approximators. Neural Networks 2, 359–366 (1989) 10. Sun, F., Sun, Z., Woo, P.Y.: Neural network-based adaptive controller design of robotic manipulators with an observer. IEEE Transactions on Neural Networks 12, 54–67 (2001) 11. Sun, F., Li, L., Li, H.X., Liu, H.: Neuro-fuzzy dynamic-inversion-based adaptive control for robotic manipulators discrete time case. IEEE Transactions on Industrial Electronics 54, 1342–1351 (2007) 12. Xian, B., Dawson, D.M., Queiroz, M.S.D., Chen, J.: A continuous asymptotic tracking control strategy for uncertain nonlinear systems. IEEE Transactions on Automatic Control 49, 1206–1211 (2004) 13. Patre, P.M., MacKunis, W., Kaiser, K., Dixon, W.E.: Asymptotic tracking for uncertain dynamic systems via a multilayer neural network feedforward and rise feedback control structure. IEEE Transactions on Automatic Control 53, 2180– 2185 (2008) 14. Lewis, F.L., Jagannathan, J., Yesildirak, A.: Neural Network Control of Robot Manipulators and Nonlinear Systems. Taylor and Francis, London (1999) 15. Dierks, T., Jagannathan, S.: Neural network control of mobile robot formations using RISE feedback. IEEE Trans. Syst. Man Cybern. B Cybern. 39, 332–347 (2008) 16. Tanner, H.G., Pappas, G.J., Kumar, V.: Leader-to-formation stability. IEEE Transactions on Robotics and Automation 20, 443–455 (2004)

A Simplified Modular Petri Net for the Walking Assistant Robot* Zhen Zhang, Qixin Cao, Chuntao Leng, and Peihua Chen State key laboratory of Vibration, Shanghai Jiao Tong University, 200240 Shanghai, China {zzh2000,qxcao,ctleng,cph}@sjtu.edu.cn

Abstract. The walking assistant robot (WAR) provides powder assist and navigation operation for the elderly and injured. The WAR need integrate different kinds of sensors to recognize people’s intents and execute different functions to help people, such as assist seating/standing up, assist walking, human tracking, location, obstacle detection, avoidance and so on. What’s more, WAR need to be flexible enough to integrate new equipments for different requests of the elderly and injured. In this paper, a Simplified Modular Petri Net (SMPN) is proposed to describe and control the distributed module behaviors of the walking assistant robot system. The simplified MPN has been applied to control SJTU’s WAR-Walkmate. The experiment results show that SMPN can control the Walkmate to integrate different modules and perform various tasks successfully. Keywords: Petri Net, Modular Petri Net, walking assistant robot.

1 Introduction Robotic system aims the provision of a grade of autonomy or a service to user that permanently or temporally lost part of their physical or mental ability. Many research institutes and universities are researching about assistant robot for the elderly. Nursing home robot “Flo” and “Pearl”[1][2]. Each robot is equipped differential drive system and on-board PC, laser scanner, ultra-sound range sensor, microphone, touch sensitive graphical display, actuated head unit, and stereo camera system. These systems have the advantage of various functions and emotional design. But these only focus for indoor environment. Our WAR is usable at indoor and outdoor environment. What’s more, it need to be flexible enough to integrate new equipments to form different solutions, such as seating/standing up assistance, walking assistance, human tracking, location, obstacle detection, avoidance and so on. People can order the robot *

This work was supported in part by the National High Technology Research and Development Program of China under grant 2006AA040203 and 2007AA041602.

Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 529–536. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

530

Z. Zhang et al.

system according to their requests. It is necessary to develop a unified flexible framework to organize different modules to work together. Petri nets (PN) is a graphical and mathematical modeling tool that is used to describe and study discrete event systems that are concurrent, distributed, or parallel. Various characteristics of such systems, such as liveness and boundedness can be investigated by determining the net’s structural properties. It has been widely used in robotics to model, analyze and evaluating performances[3-7]. In [3,4], generalized stochastic Petri nets are used to design a selection robotic framework of multiple navigation primitives. However, WARs work at indoor and outdoor environment, and need integrate different robotic units to recognize people’s intents and execute different functions to help people, such as assist seating/standing up, assist walking, human tracking, location, obstacle detection, avoidance and so on. With the system becoming more and more complex, the size of models is increasing. It is difficult to handle the complexity of the modeling as well as the analysis of the total model. Some researchers focus on modular methodologies in PN to decrease the complexity. Christensen et. al propose a MPN in which the modules are connected via synchronized transitions (ST) and synchronized places (SP) [5]. Given the actual need of WAR system, it only uses the ST to indicate the synchronized events of different robot modules, the MPN model can be simplified. Thus a SMPN is proposed to describe the distributed robot system, and it has been applied in Walkmate successfully. The rest of this paper is organized as follows. Section 2 introduces the SMPN. After describing the Walkmate in section 3, the experiments are shown in section 4. Finally, section 5 concludes the paper.

2 Simplified Modular Petri Net (SMPN) Before giving the definition of SMPN, we introduce the formal definition of PN. Definition 1: A PN is a tuple PN=(P,T,F,W,M0), satisfying: P={p1,p2,…,pm}is a finite set of places; in robot system, a place p stands for a unit’s state or movement, such as a sensor receiving its data, robot moving forward; T={t1,t2,…,tm} is a finite set of transitions; in robot system, a transition t indicates a event which can change the robot unit from one state to another, such as sensor finding an obstacle, the force on the handler is zero and so on. F⊆( P

×T)∪(T×P).

W is a function set which denotes the weight of the arcs. M0 is the initial marking. A marking of a Petri net (graph) is a multiset of token number in its places. A SMPN is used to check the properties of the modules separately and infer those of the whole system. It is composed of several local graphs (LG) and a synchronization graph (SG), each module has its own local graph which reflects only

A Simplified Modular Petri Net for the Walking Assistant Robot

531

local behavior: states are restricted to the places within the module, while the arcs captures the firing of local transitions only. Except for the LG, a SG describes the interactions between the modules. The arcs in the SG provide information on the firing of synchronized transitions (ST) only. The definition of SMPN is: Definition 2: A SMPN is a tuple MN=(S, TF), where (1) S is a finite set of modules such that:

∈

(i) each module, s S, is a PN-net:s=(Ps,Ts, Fs ,Ws,Mos)

∈:

(ii) the sets of nodes in different modules are pair-wise disjoint: ∀ s1,s2 S

[s ≠s →P ∪T ∪P ∩T ]; 1

2

s1

s1

s2

s2

(iii)TF is a finite set of synchronized transition sets where

∈

∀ tf TF:tf is synchronized event which happened in at least 2 modules at the same time.

3 The Walkmate Robot System The Walkmate is shown in Fig. 1, it is equipped with a pair of casters as front wheels, a pair of rear wheels driven by DC servo motors, a handle with 2 FSRs (Force Sensing Resister) embedded into the handlebars. The FSR is fixed with the handgrip directly to detect both pull and push pressure of the handlebar. Then these pressure signals are transformed into the PC to calculate the output to the motor. There are

Fig. 1. Walkmate robot

532

Z. Zhang et al.

Fig. 2. The system architecture

some candidate modules for different users to order: a panoramic camera for human tracking, a Sick laser sensor for obstacles detection and avoidance, a Leadtek 9553D GPS module for location, a CPC V818 GPRS module for sending the Walkmate’s data to the surveillance center by internet, a Proface touch panel which is used for users to set the synchronized transitions and make different modules work together and a microphone for voice prompt.

4 Experiments In this section, we did 3 experiments to illustrate that it’s easy to use the SMPN to control walking assistant robot when adding some new unit and functions easily. Mode 1: Walkmate only includes FSR module shown in fig.3. The meaning of places and transitions are also provided in fig.3. If people only want pull or push pressure of the handlebar to control the Walkmate, they can choose the mode 1.

Fig. 3. Mode 1: FSR and its places and transitions’ meanings

A Simplified Modular Petri Net for the Walking Assistant Robot

533

If users need the robot be able to detect and avoid the obstacles, they can choose the mode 2 in fig.4, that is FSR and laser work together. The meanings of places and transitions are shown in fig.5. In the FSR, synchronized transitions FT1 and FT2 replace the local transition Ta3. If place Pb3 and Pa2 have tokens at the same time, FT1 can be fired, then tokens move from Pb3, Pa2 to Pb1,Pa3(as shown in synchronization graph of fig.4). It indicates if the laser detects an obstacle and FSR generates command “move forward”, the robot will start executing fusion force control, and move to avoid the obstacle.

Fig. 4. Mode 2: FSR and laser

Fig. 5. Places and transitions’ meanings in mode 2

Fig. 6. Fusion force control

We use the artificial potential field (APF) method to implement the fusion force control[8]. As the fig.6 shows, we uses a repulsive potential field force to force a robot away from obstacles or forbidden regions. It is inversely proportional to the distance between robot and obstacle. Thus the robot moves under the action of a

534

Z. Zhang et al.

force, that is equal to the negative gradient of that potential, and it is driven towards the positions with the lower potential. The scenes of experiment are shown in fig.7, and the Walkmate avoided the obstacle successfully.

Fig. 7. The scenes of an obstacle avoidance

If users need the robot has the ability of human tracking, they can add a camera module (fig.8), that is FSR, laser and camera work together. The meanings of places and transitions are shown in fig.8. In the FSR, synchronized transition FT3 replaces the local transition Ta2. Iff place Pa1 and Pc1 have a token at the same time, FT3 can be fired, then tokens move from Pa1, Pc1 to Pa6, Pc2(which is shown in synchronization graph of fig.8). It indicates if the FSR’s force is zero, and the camera detects the user’s body, then the robot will execute human tracking behavior. The scenes of human tracking experiment are shown in fig.10, the result shows that the Walkmate followed its user successfully.

Fig. 8. Mode 3: FSR, laser and camera

A Simplified Modular Petri Net for the Walking Assistant Robot

Fig. 9. Places and transitions in mode 3

535

Fig. 10. The scenes of human tracking

5 Conclusions In this paper, a Simplified Modular Petri Net (SMPN) is proposed to describe and control the distributed module behaviors of the walking assistant robot system. Using the SMPN, it is easy to make robot integrate different function modules, such as laser sensor, camera sensor and so on, to finish some special work. The SMPN has been applied to control SJTU’s WAR-Walkmate by far. The experiment results show that SMPN can control the functional units to form different typies of robots and perform various tasks successfully. It is expected that SMPN will significantly aid in the development of assistant robots in the human environment.

References 1. Pineau, J., Montemerlo, M., Pollak, M., Roy, N., Thrun, S.: Towards robotic assistants in nursing homes:challenge and results. In: Robotics and Autonomous Systems, pp. 271–281 (2003) 2. Montemerlo, M., Pineau, J., Roy, N., Thrun, S., Varma, V.: Experiences With a Mobile Robotic Guide for the Elderly. In: Proceedings of the AAAI National Conference on Artificial Intelligence, Edmonton, Canada, pp. 582–592 (2002) 3. Kim, G., Chung, W.: Navigation Behavior Selection Using Generalized Stochastic Petri Nets (GSPN) for a Service Robot. IEEE Transactions on Systems, Man and Cybernetics Part C 37(4) (2007) 4. Kim, G., Chung, W., Park, S.-K., Kim, M.: Experimental Research of Navigation Primitive Selection Using Generalized Stochastic Petri Nets (GSPNs) for a Tour-Guide Robot. In: Proceedings of the 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2005), Alberta, Canada, pp. 1392–1398 (2005) 5. Christensen, S., Petrucci, L.: Modular Analysis of Petri Nets. The Computer Journal 43(3), 224–242 (2000)

536

Z. Zhang et al.

6. Huang, X., Xu, H., Jia, P.: Fuzzy Timed Agent Based Petri Nets for Modeling Cooperative Multi-Robot Systems. Int’l J. of Communications, Network and System Sciences 2, 827– 835 (2009) 7. Yasuda, G.: Design and Implementation of Petrinet Based Distributed Control Architecture for Robotic Manufacturing Systems. In: Gelbukh, A., Kuri Morales, Á.F. (eds.) MICAI 2007. LNCS (LNAI), vol. 4827, pp. 1151–1161. Springer, Heidelberg (2007) 8. Ge, S.S., Cui, Y.J.: Dynamic motion planning for mobile robots using potential field method. Autonomous Robots 13(3), 207–222 (2002)

Omni-directional Vision Based Tracking and Guiding System for Walking Assistant Robot Xiaoxiao Zhu, Qixin Cao, Hongbing Tan, and Aolin Tang Research Institute of Robotics, Shanghai Jiao Tong University, Shanghai, China {ttl,qxcao,bluesun}@sjtu.edu.cn, [email protected]

Abstract. This paper provides a novel WAR(Walking Assistant Robot) which only uses ODVS (Omni-Directional Vision System) as its main sensor to implement the task of Tracking and Guiding. We use Blob Detection and Camshift Algorithm to track, and use APF(Artificial Potential Fields) approach to guide. The experiments show that those methods have high precision and good robustness, and also show the advantages of the wide view field and low cost of the ODVS. Keywords: ODVS, tracking and guiding, walking assistant robot, blob detect, APF.

1 Introduction Recently, the severe situation of population aging makes the demand of assistant equipments for the elderly increasing day by day. Many researchers start to design some kind of Intelligent Walking Assistant Robot to serve the elderly, which can recognize the walking intention of the user, assist walking, self-localize and navigate, avoid obstacle, avoid slipping, slope alarming and so on. The Yasuhisa Hirata et al.[1] [2] designed a walking helper to recognize the walking intention of the user and assist walking. Aaron Morris et al. [3], designed robot can implement navigation and global orientation besides working assistant. Hyeon-Min Shim et al.[4] designed a robot to do walking assist and path guiding. We can find that those WARs usually have many sensors to perceptive the environment such as GPS, laser ranger finder or ultrasonic transducers, which cause high costing. We are inspired by the work of TOON GOEDEME et al. [5]. They present a novel autonomous mobile robot navigates with only an omni-directional camera as sensor, which builds accurate topologically organized environment maps of a complex, natural environment and then implement self location. The omni-directional camera TOON GOEDEME used is a kind of ODVS (also know as catadioptric system), whose most advantage was its wide field of view (360o in horizontal direction) and thus rich information content of the images acquired with, and also it’s very cheap. This paper provides a novel WAR based on only ODVS to track and guide (two basic intelligential function of the WAR). This paper consists of 5 chapters. In chapter 2, we introduce the hardware system of our WAR. In chapter 3, we describe the tracking Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 537–543. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

538

X. Zhu et al.

algorithm. In chapter 4, we describe the guiding approach. In chapter 5, we describe experimental results, and in chapter 6, we conclude.

2 ODVS Our WAR system shows as Fig.1-(a), it equipped with two Force sensors to interaction with user, one differential mobile base and one ODVS. As show in Fig.1-(a), our ODVS consisted with mirror, cover glass, anti-reflect needle and CCD Camera. We use hyperboloidal mirror to meet the single view point restraint [6], the presentation of the mirror as S ( p) =

x2 + y2 z2 − 2 = −1 . a2 b

(1)

Where p = ( x, y, z ) is one point on the hyperboloid.

Om Force Sensor

ODVS

z

2c

P2

Mirror

N P1

Image Plane

Oc x

Differential mobile base

l

h Measure Plane

d (a)

P3

(b)

Fig. 1. (a)Hardware system;(b) Imaging principle of ODVS

To introduce the imaging principle of ODVS, we only consider the cross section for the symmetry as show in Fig.1-(b). We assume that one ray l start from one point on the Measure Plane noted with P3, cross with the Mirror at point P2, and reflect by mirror, then cross with the Image Plane at point P1, finally end in camera center Oc. Om N is the normal at the point P2, h is the vertical distance between the camera center and the Measure Plane, d is the distance from P3 to the optical axis, which is what we try to know.So here we have Oc P2 = λ Oc P1

(λ > 0) .

(2)

Omni-directional Vision Based Tracking and Guiding System

539

Substitute equation (2) in (1) we can decide the λ . Since P2 P3 and Oc P2 symmetrical to the normal vector Om N , then we get

P2 P3 = Oc P2 + 2(Oc P2 • Om N )Om N . Oc P3 = Oc P2 + k P2 P3

( k > 0) .

(3) (4)

And we know the Z-Coordinate component is –h, then in equation (4) we can solve

Oc P3 and X-Coordinate component of it is d which one we try to know.

3 Tracking The object tracking function is one of the basic functions of WAR, which should have the ability to follow the user who tried to walk without the help of the WAR. Generally, there are three types of tracking algorithm, feature-base point tracking, kernel tracking, silhouette tracking. Because of the major distort of the ODI (omni-directional image), we use mean-kernel tracking algorithm, in which moving objects are characterized by their color-histograms. 3.1 Marker and Marker Recognize Algorithm

In the nature environment, there is much interference similar to the target, especial when the user is in crowd, so we design a special marker to identify the user. As show in Fig.2, the marker consisted with three color block, and the marker in the ODI. We use blob detection approach to find the marker in the ODI, it follows three steps. Step one, find all the blobs of the color in the marker; Step two, filter the blobs with the area threshold, divided into three categories by their color; Step three, traverse the blobs and find the marker according to the position relationship between those colors, such as the blue blob should near the red blob and far from the image center than the red one.

Fig. 2. Marker

540

X. Zhu et al.

3.2 Tracking Based on CA (Camshift Algorithm)

The blob detection approach is time-consuming, so we just use it to find the original position of the marker and then use the CA to track. CA uses color probability distribution to identify the target and every pixel in the image has a probability as their quality. The basic idea of CA is to set a rectangular window as the DW(Detection Window), then try to find a position where of center of gravity of all the pixels in the detection window and the center of the detection window are coincided. The algorithm follows steps. Preparing, we change the image from RGB color space conversion to HSV color space, thus reducing the impact of light intensity. Step two, use the histogram of H color components of all pixel of the marker as the probability model. Tracking, by querying the color histogram model, we can get the probability of pixel in the detection window, the following searching formula as p( w) = C ( w) / Max(C (h)), h ∈ (0,1) .

(5)

Where C (h) is the number of the pixel in histogram with h as H color components. Then we can calculate the position of center of gravity of all the pixels in the detection window. The formula is as follow. ( xc , yc ) = (

⎛ ∑∑ xI ( x, y) ∑∑ yI ( x, y ) ⎞ Z10 Z 01 ⎟. , )=⎜ , ⎜ ∑∑ I ( x, y ) ∑∑ I ( x, y ) ⎟ Z 00 Z 00 ⎝ ⎠

(6)

Where ( xc , yc ) is the center of gravity, I ( x, y ) was the probability of pixel at ( x, y ) , Z 00 is the Zero-order moment, Z 01 and Z10 was first-order moment. If ( xc , yc ) doesn’t coincide with the center of the DW, we move the DW to ( xc , yc ) , and do the work again, otherwise the ( xc , yc ) is the center of the target [7]. 3.3 Occlusion and the Relative Position of Target

When occlusion happen CA will get incorrect results, so as to the follow-up tracking which base on the last frame target position. When this happen, we stop using camshaft algorithm, but use Blob Detection again until we find the target. After using above algorithms we get the position of the target in the image plane, according to section 2.1 we if determine the value of h, we can get the relative position of the target. Because the mark server distorted, so we use the bottom of marker as the measure plane to calculate the relative position of the target.

4 Guiding Guiding is another basic function of WAR. When the user walking with WAR’s assistant, WAR should recognize the walking user’s intention and consider of the environment such as obstacle information, and then guiding the user walking safely.

Omni-directional Vision Based Tracking and Guiding System

541

4.1 Identification of Obstacles

In the analysis of the experimental pictures we found that, the color of the ground is usually uniform and single. Therefore, we identify obstacles using the color segmentation method. We begin to learn the color of the ground, calculate the range of every HIS component, then determine the image point if non-ground, will be recognized as obstacles. After we get the image position of the obstacles, we also need to translate it to the measure plane, here we choose the Ground as the measure plane. 4.2 APF Approach

Khatib first developed the application of APF(Artificial Potential Fields) to path planning [8]. The basic principle of APF is to assume the robot get force F in a potential fields U .U consists of a attractive potential fields U att provide attractive force Fatt by the goal, and repulsive potential fields U rep provide attractive force Frep by obstacles, As shown in Fig.3-(a). In our case, the Fatt produced by the user equal to pushing force, which can be obtained from the force sensor. So we get follow equations [8].

1 1 1 2 ⎧ ⎪U rep ( X ) = 2 k r ( ρ ( X , X ) − ρ ) ρ ≤ ρ 0 . 0 obs ⎨ ⎪U ( X ) = 0 ρ ρ > 0 ⎩ rep Frep = −∇U rep , F = Fatt + Frep , V = λF . Where kr is a constant repulsive potential field, and

(7)

(8)

ρ 0 represents the limited distance

of the potential field influence, and ρ ( X , X obs ) = X − X obs is the distance to the obstacle, and V is the goal velocity vector of the robot, and

λ

is a constant.

WAR2 WAR

(a)

(b)

Fig. 3. (a)Principle of APF;(b) Experiment environment

542

X. Zhu et al.

5 Experiments We conduct experiments on the WAR which was introduced in section 2. The mirrors parameter is a=23.4125mm, b=28.0950mm. The camera (Logitech pro 9000 USB camera) is pro-calibrated. And in the tracking experiments we use another WAR to take the marker moving at some certain speed, as show in Fig.3-(b). 5.1 Tracking

In this chapter, we mainly evaluate the Tracking algorithm’s accuracy, continuity, and the ability to deal with blocking situations. To test the accuracy, we put the WAR2 at 5 different distance front of the WAR, and hold on the WAR, and then we track the marker 20 times in every distance, the result is Table.1. To test the continuity, we set the WAR2 moving in some certain speed, let WAR follow it and keep in 1.5 meter away, and we record the number of the interrupting of tracking. We tracked 20 times and 3 minute each time for every speed, the result is Table 2. To test the ability to deal with Occlusion situations, we set WAR2 moving in 1m/s, and cover the marker in some certain time. We examine 20 times for each covering time. The result is Table 3. Table 1. Accuracy of tracking. 20 times for each distance Distance(m) 0.5 1 1.5 2 2.5

Maximum error(m)

Average error(m)

0.025 0.045 0.06 0.15 0.36

0.022 0.031 0.049 0.122 0.30

Table 2. Continuity of tracking Velocity (m/s) 0.5 1 1.5 2 2.5

Number of interrupting 0 0 0 1 2

Table 3. Ability to deal with Occlusion situations Time of Occlusion(s) 0.5 1 1.5 2 2.5

Number of interrupting 0 0 0 0 0

5.2 Guiding

In this chapter, we mainly evaluate guiding function of WAR. We design 100 times experiments, for each time, we set up Random Obstacle around the WAR, and try to moving in different direction. The success rate of the recognition of the barrier reach 99%, and The success rate of obstacle avoidance reach 99%.

Omni-directional Vision Based Tracking and Guiding System

543

6 Conclusion This paper provides the algorithms of WAR’s tracking and guiding functions only base on the ODVS. We design a marker with three color, and use Blob Detection and CA to track the marker, we also consider the Occlusion situations. For guiding function, we use color segmentation method to recognize the obstacles, and use ARF approach to guide. We conducted experiments on real WAR to test the algorithms, and the result show that, the Omni-directional Vision Based Tracking and Guiding System has high accuracy and robustness, it also demonstrated that the use of Omni-directional vision system will be lower costing.

References 1. Yasuhisa, H.: Motion Control of Omni-directional type Walking Support System - Walking Helper. In: IWRHIC 2003, pp. 85–91. IEEE Press, Millbrae (2003) 2. Yasuhisa, H.: Motion Control of Passive-type Walking Support System based on environment Information. In: International Conference on Robotics and Automation, pp. 2921–2927. IEEE Press, Barcelona (2005) 3. Aaron, M.: A Robotic Walker That Provides Guidance. In: ICRA 2003, pp. 25–30. IEEE Press, Taipei (2003) 4. Hyeon, M.S.: Implementation of an Intelligent Walking Assistant Robot for the Elderly in Outdoor Environment. In: Proceedings of the 2005 IEEE 9th International Conference on Rehabilitation Robotics, pp. 452–456. IEEE Press, Chicago (2005) 5. Mulder, E.F., Kothare, M.V.: Omnidirectional Vision Based Topological Navigation. International Journal of Computer Vision 74(3), 219–236 (2007) 6. Simon, B., Shree, K.N.: A Theory of Single-Viewpoint Catadioptric Image Formation. International Journal of Computer Vision 35(2), 175–196 (1999) 7. Liangliang, W.: A Face Tracking System Based on Face Detection and CAMSHIFT Algorithm. Microcomputer Applications 29(2), 14–18 (2008) 8. Peiyan, Z.: Soccer robot path planning based on the artificial potential field approach with simulated annealing. Robotica 22, 563–566 (2004)

Dynamic Eye-in-Hand Visual Servoing with Unknown Target Positions Hesheng Wang1, Weidong Chen1, and Yun-hui Liu2 1

Department of Automation, Shanghai Jiao Tong University, No. 800, Dong Chuan Rd. Min Hang, Shanghai, China [email protected] 2 Department of Mechanical and Automation, The Chinese University of Hong Kong, Shatin, NT, Hong Kong Abstract. This paper presents a new adaptive controller for a robot manipulator by using the visual feedback from an eye-in-hand camera. The controller is designed to cope with the case when the target 3-D positions are unknown. The controller employs the depth-independent interaction matrix to map the image errors onto the joint inputs of the manipulator. A new algorithm is developed to estimate the unknown parameters on-line. The Lyapunov theory is used to prove asymptotic stability of the proposed controller based on the nonlinear dynamics of the robot manipulator. Experiments have been conducted to demonstrate the performance of the proposed controller. Keywords: Visual servoing, Eye-in-hand, Uncalibrated.

1 Introduction Visual servoing is an approach to control motion of a robot manipulator using visual feedback signals from a vision system [1]. In eye-in-hand visual servoing, various methods have been developed for estimation of the camera parameters to avoid the tedious and difficult calibration work in visual servoing system. Hosoda and Asada [2] used the Broyden updating formula to estimate the image Jacobian. Papanikolopoulos et al. [3] developed an algorithm for estimating on-line the relative distance of the target with respect to the camera. Yoshimi and Allen [4] proposed an estimator of the image Jacobian for a peg-in-hole alignment task. Ruf et al. [5] proposed an on-line calibration algorithm for position-based visual servoing. However, the methods mentioned above are based on kinematics only and the nonlinear forces in robot dynamics are neglected. Neglecting the nonlinear forces affect not only control errors but also the stability. To compensate for the nonlinear forces, Carelli et al. [6] proposed an adaptive controller for the eye-in-hand setup. The controllers proposed in [7] can cope with the unknown parameters but apply to planar manipulators only. In our early work [8][9], we proposed an adaptive controller for dynamic and uncalibrated visual servoing in eye-to-hand configurations. In [10][11], we extend our earlier results to an eye-in-hand configuration. This paper presents a new adaptive controller for dynamic image-based visual servoing of a robot manipulator when the 3-D coordinates of the target points are unknown. To cope with nonlinear dependence of the image Jacobian on the unknown Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 545–550. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

546

H. Wang, W. Chen, and Y.-h. Liu

parameters, this controller employs a matrix called depth-independent image Jacobian which does not depend on the scale factors determined by the depths of target. Adaptive algorithm has been developed for estimating the unknown coordinates of the targets online. In the adaptive rule, we fixed one particular parameter with constant value, and adaptive other parameters. By applying this, we can avoid the singularity. A new Lyapunov function is employed to prove asymptotic convergence of the position errors on the image plane with a full consideration of robot dynamics. Experiments have been conducted on 3 degrees of freedom robot manipulator to verify the performance.

2 Camera and Robot Model Consider an eye-in-hand set-up (Fig. 1), in which a vision system is mounted on the end-effector to monitor a set of target points. Assume that the target points are fixed ones but their coordinates are not known. Suppose that the camera is a pin-hole camera with perspective projection. Problem 1: Given desired positions of the target points on the image plane, design a proper joint input for the manipulator such that the projections of the target points on the image plane is asymptotically convergent to the desired positions. Assuming that only the target 3-D position are unknown. Denote the joint angle of the manipulator by q(t), and the homogenous coordinates of the target w.r.t. the robot base frames by x . Note that the x is a constant vector for the fixed target point. Denote the homogenous transform matrix of the end-effector with respect to the base frame by Te (q) . M is the perspective projection matrix including camera intrinsic and extrinsic parameters. Let y = (u , v,1)T express the homogenous coordinates of the projection of the target point on the image plane. Under the perspective projection model,

Robot

The camera frame

The end-effector frame Point Feature

The robot base frame

Fig. 1. An eye-in-hand setup for visual servoing

1

MTe−1 (q)x

(1)

z = m T3 Te−1 (q)x

(2)

y=

c

z

The depth of the feature point is given by c

Dynamic Eye-in-Hand Visual Servoing with Unknown Target Positions

547

where m T3 denotes the third row vector of the perspective projection matrix M. c

z =

∂ (m T3 Te−1 (q)x) q = n T (q)q ∂q

(3)

It is important to note: Property 1: For any vector ρ , the product A(q)ρ can be written as a linear form of the unknown parameters ,i.e. A(q)ρ = Q(ρ, q, y)x (4) where Q(ρ, q, y) does not depend on the parameters representing the products of the camera parameters and the world coordinates of the target point. The dynamic equation of a robot manipulator has the form: 1   + ( H H(q)q (q) + C(q, q ))q + G(q) = τ (5) 2 where H(q) is the positive-definite and symmetric inertia matrix. C(q, q ) is a skewsymmetric matrix. The term G(q) represents the gravitational force, and τ is the joint input of the robot manipulator. The first term on the left side of (5) is the inertial force, and the second term represents the Colioris and centrifugal forces.

3 Adaptive Image-Based Visual Servoing with Unknown Target Positions Denote the desired position of the target point on the image plane by y d . The image error is given by: Δy = y − y d (6) Denote an estimation of the unknown parameters x by xˆ . Using the estimation, we propose the following controller: ˆ T (q) + 1 nˆ (q)Δy T )BΔy τ = G(q) − K 1q − ( A (7) 2 ˆ (q) is the estimated Depth-independent interaction matrix calculated by the A estimated parameters xˆ . nˆ (q) is an estimation of vector n(q), K 1 and B are positive

definite gain matrices. By substituting (7) into (5), we obtain:

1   + ( H H(q)q (q) + C(q, q ))q = −K 1q 2 (8) 1 ˆ T (q) − A T (q)) + 1 (nˆ (q) − n)Δy T ]BΔy − ( A T (q) + n(q)Δy T )BΔy − [( A 2 2 From the Property 1, the last term can be represented as a linear form of the estimation errors of the parameters as follows: ˆ T (q) − A T (q)) + 1 (nˆ (q) − n)Δy T ]BΔy = Y(q, y)Δx − [( A (9) 2

548

H. Wang, W. Chen, and Y.-h. Liu

where Δx = xˆ − x is the estimation error and the regressor Y(q, y) does not depend on the unknown parameters. Based on the discussion above, we propose the following adaptive rule: d (10) xˆ = − Γ −1Y T (q, y)q dt where Γ are positive-definite and diagonal gain matrices. Following will analysis the stability of the proposed controller. Theorem 1: Under the control of the controller (7) and the adaptive algorithm (10) for parameters estimation, the image error of the target point is convergent to zero, i.e. lim t → ∞ Δy = 0 Proof: Introduce the following positive function: 1 V (t ) = {q T H(q)q + c zΔy T BΔy + Δx T ΓΔx} 2

(11)

(12)

Notice that c z > 0 . Multiplying the q T from the left to (8) results in 1  (q)q = −q T K q − q T A T (q)BΔy − 1 q Tn(q)( Δy T BΔy ) + q T Y(q, y)Δx  + q T H q T H(q)q 1 2 2

(13) From equation (3), we have q T A T (q) = c zy T = c zΔy T

(14)

Multiplying the Δx from the left to (10), we obtain T

Δx T ΓΔx = − Δx T Y T (q, y)q

(15)

Differentiating the function V(t) results in 1  1  + H V (t ) = q T (H(q)q (q))q + Δx T ΓΔx+ c zΔy T BΔy + c zΔy TBΔy 2 2

(16)

Note that c

z = n Tq

By combining the equations (13)-(17), we have V (t ) = −q TK 1q

(17) (18)

Obviously we have V (t ) ≤ 0 , and hence q , Δy and Δx are all bounded. From (8)  and xˆ are bounded respectively. Then, we can conclude that and (10), we know q V(t ) is bounded. Consequently, from barbalat’s lemma, we have

limt →∞ q = 0 lim xˆ = 0

(19)

t →∞

In order to prove the convergence of the image error, consider the invariant set of the system when V (t ) = 0 . From the Barbalat’s lemma, we can conclude the convergence of the position error of the target point projections on the image plane to zero.

Dynamic Eye-in-Hand Visual Servoing with Unknown Target Positions

549

4 Experiments We have implemented the controller in a 3 DOF robot manipulator. Fig. 2 shows the experiment setup system. A frame processor MATROX PULSAR installed in a PC with Intel Pentium II CPU acquires the video signal. This PC processes the image and extracts the image features. The control gains are K 1 = 25 , B = 0.00007 , Γ = 5000 . the initial estimated target

position is xˆ (0) = [0.85 − 0.15 0.05]T . The real camera intrinsic parameters are au = 871 , av = 882 , u0 = 381 , v0 = 278 . As shown in Fig. 3, the image feature points asymptotically converge to the desired ones. The results confirmed the convergence of the image error to zero under control of the proposed method. The sampling time in the experiment is 40ms.

Fig. 2. The experiment setup system

Fig. 3. Experiments results: Position errors of the image feature

550

H. Wang, W. Chen, and Y.-h. Liu

5 Conclusions This paper presents a new adaptive controller for dynamic image-based visual servoing of a robot manipulator uncalibrated eye-in-hand visual feedback. To cope with nonlinear dependence of the image Jacobian on the unknown parameters, this controller employs a matrix called depth-independent interaction matrix which does not depend on the scale factors determined by the depths of target points. A adaptive algorithm has been developed to estimate the unknown 3-D coordinates of the target points. With a full consideration of dynamic responses of the robot manipulator, we employed the Lyapunov method to prove the convergence of the postion errors to zero. Experimental results illustrate the performance of the proposed methods. Acknowledgments. This work is supported in part by the National High Technology Research and Development Program of China under grant 2006AA040203, in part by the Natural Science Foundation of China under grant 60775062, grant 60934006, grant 60334010 and grant 60475029, in part by the Program for New Century Excellent Talents in University under grant NCET-07-0538, in part by Hong Kong RGC under the grant 414406 and 414707.

References 1. Hutchinson, S., Hager, G.D., Corke, P.I.: A Tutorial on Visual Servo Control. IEEE Tran. Robotics and Automation 12, 651–670 (1996) 2. Hosada, K., Asada, M.: Versatile Visual Servoing without Knowledge of True Jacobain. In: IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pp. 186–191 (1994) 3. Papanikolopoulos, N.P., Nelson, B.J., Khosla, P.K.: Six degree-of-freedom hand/eye visual tracking with uncertain parameters. IEEE Trans. Robotics and Automation 11, 725– 732 (1995) 4. Yoshimi, B.H., Allen, P.K.: Alignment Using An Uncalibrated Camera System. IEEE Trans. Robotics and Automation 11, 516–521 (1995) 5. Ruf, A., Tonko, M., Horaud, R., Nagel, H.H.: Visual Tracking of An End Effector by Adaptive Kinematic Prediction. In: IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pp. 893–898 (1997) 6. Carelli, R., Nasisi, O., Kuchen, B.: Adaptive Robot Control with Visual Feedback. In: Proc. of the American Control Conf., pp. 1757–1760 (1994) 7. Kelly, R., Carelli, R., Nasisi, O., Kuchen, B., Reyes, F.: Stable Visual Servoing of Camera-in-Hand Robotic Systems. IEEE/ASME Trans. Mechatronics 5, 39–48 (2000) 8. Liu, Y.H., Wang, H., Wang, C., Lam, K.: Uncalibrated Visual Servoing of Robots Using a Depth-Independent Image Jacobian Matrix. IEEE Trans. Robotics 22, 804–817 (2006) 9. Wang, H., Liu, Y.H., Zhou, D.: Dynamic Visual Tracking for Manipulators Using An Uncalibrated Fixed Camera. IEEE Trans. Robotics 23, 610–617 (2007) 10. Wang, H., Liu, Y.H., Zhou, D.: Adaptive Visual Servoing Using Point and Line Features with An Uncalibrated Eye-in-hand Camera. IEEE Tran. Robotics 24, 843–857 (2008) 11. Wang, H., Liu, Y.H.: A New Approach to Dynamic Eye-in-hand Visual Tracking Using Nonlinear Observers. IEEE/ASME Trans. Mechatronics 15 (2010)

Optimum Motion Control for Stacking Robot Xiaoming Zhang, Nan Luan, Zhong Dong, and Liming Chen School of Mechanical Engineering, Shanghai Jiaotong University Shanghai 200240, China

Abstract. Stacking speed is an important parameter to estimate the capacities of robotic system. In this paper, an innovative method based on ILC (Iterative Learning Control) could be used to optimize the speed of the industrial robots. By learning following error, the robotic system could reach the optimum speed within the permissible error range and achieve the optimization of the stacking robotic system. Furthermore, requiring little to model the robotic system, ILC is easy to be realized in practice. The results of the system simulation indicate, under the circumstances of computing robotic system parameter and comparing the optimization with ILC to the original results, that ILC could increase the stacking speed a lot through improving local speed. Keywords: ILC, Stacking robot, Motion control.

1 Introduction The applications of stacking robot could not only reduce the labor intensity, but also improve productivity greatly. Efficiency is the primary performance indicators of stacking robot. Under the certain conditions of the structure and performance of the stacking system, as well as stability of the robot, the article focuses on giving full play to the robot’s own ability, which is fully using the ability of existing controllers, drives and motors, and the coupling between the various joints of robot to maximize the efficiency of stacking robot speed. Based on the application of Iterative Learning Control method, the proposed approach could achieve a faster speed by learning following error. As early as 1984, Arimoto [1] and others have proposed the concept of Iterative Learning Control. This method, using “repeated learning” strategy, is suitable for the controlled object with the character of repetitive motion. It amends unsatisfactory control signal by using the deviation between actual output and expected output of the system and generates new control signals, thus improves tracking performance of the system. After several iterations, the system's output will be closed to the expectations of the ideal trajectory. Learning control method is very useful for robotic system which is too difficult to model accurately. This method that amends controller according to the effect of control – “quality” and does not need to identify the parameters of controlled object is based on the quality of self-learning control, which Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 551–558. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

552

X. Zhang et al.

is very useful for system modeling. This paper utilizes open-loop P-type Iterative Learning Control method.

2

Stacking Robot System Description

The architecture of robot system mainly consists of four parts: host computer, motion controller, servo motor and driver, and manipulator. Fig.1 shows the relationship between various parts. Host computer utilize IPC. Controller, using British TRIO-205 Controller, can control free movement of up to five axes individually and run seven threads simultaneously.

Fig. 1. Structural diagram of robot system

As is shown by Fig. 2, manipulator is a parallelogram linkage with four degrees of freedom. And Fig. 3 illustrates the specific distribution of all degrees of freedom. A drives grasp to rotate around the axis of robot body. B drives grasp to move up and down along the axis of robot body. C drives the grasp to move back and forth along the horizontal direction. D drives grasp to rotate around its own axis.

Fig. 2. Manipulator

Fig. 3. Schematic of manipulator structure

Optimum Motion Control for Stacking Robot

553

3 Application of Iterative Learning Control 3.1 Open-Loop Iterative Learning Control Theory ILC is a very useful method for the kind of system with non linearity, strong coupling, difficulties of modeling and high demand of trajectory like robot system. This paper, using P-type ILC law, achieves real-time monitoring robot position. The composition of control law is as follows: the first k+1-control is equal to the sum of the first k-control and the first k-correction item of output error, which is shown by (1)

uk +1 ( t ) = uk ( t ) + P ⋅ ek ( t )

(1)

When the first k-based system operates, the output error is (2)

ek = yd − yk

(2)

P is gain coefficient. Fig.4 shows the basic structure. The figure will not be instructed specifically because of the limit of this paper’s character. The mathematical analysis and proof of the convergence of the control law, as shown in (1), can be found in [2, 3, 4, 5].

uk +1 (t ) uk (t )

y k (t )

yd (t )

ek (t )

Fig. 4. Open-loop P-type ILC structure

3.2 Application of Iterative Learning The accuracy of the position of stacking robot is required because its motion is repetitive. But users are more concerned with the efficiency of stacking, namely velocity. The problem that we confronted is how to improve running speed as much as possible while the accuracy of position is guaranteed. In response to this problem, the characteristics of the applicable field of iterative learning and simplicity of algorithm determine that it is the best solution to such problem.

554

X. Zhang et al.

The use of ILC law is distinct at the beginning because of the unique of stacking robot, the pursuit of speed. The optimum index is tmin

= t ( e ) . Variable scope

is e ( s, t ) 0 . Denote Δuk = uk − uk −1 . This generally leads to a numerical solution. Theorem 1: If the parameter satisfies

⎛ ∂J (uk ) ⎞ 0 < λ ≤ J (uk ) ⎜ ⎟ ⎝ ∂uk ⎠

2

, the control algorithm given by

(15) will be convergent. Proof: (omitted)

4 An Illustration Example In order to show the applicability of the proposed control algorithm, let us consider the following feedback control system, which consists of the plant represented as follows 0.8 y1 ( k − 1) + u1 (k − 2) + 0.2u2 ( k − 3) ⎧ + ω1 (k ) ⎪ y1 ( k ) = (1 + y1 (k − 1)) 2 ⎪ ⎨ ⎪ y ( k ) = 0.9 y2 ( k − 1) + 0.3u1 ( k − 3) + u2 (k − 2) + ω (k ) 2 ⎪⎩ 2 (1 + y2 ( k − 1)) 2

(16)

The PDFs of the noise ω and induced delays τ 1 and τ sc2 are not needed, however, they are given in this simulation so as to illustrate the production of stochastic tracking errors clearly. sc

⎧375 2 ⎪ (0.8x − x ), x ∈[0,0.8]

γω (x) = ⎨ 32 1

⎪⎩ 0,

⎧125 (0.6 − x) 2 , x ∈ [0, 0.6] ⎪

γ ω ( x) = ⎨ 9 2

⎪⎩ 0,

⎧ ⎪

4

⎪⎩ 0,

x 3 (5 − x ) 5 , x ∈ [0, 5]

⎧ 4 2 x (3 − x), x ∈ [0, 3] ⎪ ⎪⎩0,

(19)

otherwise

γ τ ( x) = ⎨ 27 2 sc

(18)

otherwise

γ τ ( x ) = ⎨ 15625 1 sc

(17)

otherwise

(20)

otherwise

In this application, the set-points are set to r1 = r2 =1, the sampling period T = 1 . For the simulation, 2-5-1 and two1-4-1 MLPs are employed to estimate the pdf of tracking error γ (e1 , e2 ) , γ (e1 ) and γ (e2 ) for obtaining the corresponding entropies, the learning factor η = 0.7 , the samples for MLP are trained with a segment of K = 200 (the width of sliding window) at each instant k . The deceasing factor in this simulation is λ = 0.001 . The simulation results are shown in Figs. 2-4. The entropies of tracking error are illustrated in Fig. 2. The PDF of tracking error are given in Fig. 3 and Fig. 4, it is clear that the shape of PDF of two tracking errors

600

J. Zhang and H. Wang

become narrower along with the increasing time. Figs. 3 and 4 indicate that the control system has a small uncertainty in its closed loop operation. 0.65 He1 He2 He12

0.6 0.55

0.4 0.35 0.3 0.25 0.2

0

2

4

6

8

10 time(s)

12

14

16

18

20

Fig. 2. Entropies of tracking error

40

PDF1

30

20

10

0 20 15

0.3 0.2

10

0.1 0

5 0

Simulation time

-0.1 -0.2

Error1 range

Fig. 3. PDF of the tracking error1

50 40 30 PDF2

Entropy of error

0.5 0.45

20 10 0 20 15

0.4 0.3

10

0.2 0.1

5 Simulation time

0 0

-0.1

Error2 range

Fig. 4. PDF of the tracking error2

An Information Theoretic Approach for Design MIMO Networked Control Systems

601

5 Conclusions An information theoretic approach is designed for multi-input and multi-output (MIMO) networked control systems with random delays. The performance index of the control systems is constructed by entropies of tracking errors via multi-layer perceptron (MLP). A simulation example shows the efficiency and feasibility of the proposed method. Acknowledgments. This work is supported by National Natural Science Foundation of China under grants (No. 60674051, 60974029). These are gratefully acknowledged.

References 1. Tipsuwan, Y., Chow, M.Y.: Control methodologies in networked control systems. Control Engineering Practice 11, 1099–1111 (2003) 2. Antsaklis, P., Baillieul, J.: Guest editorial special issue on technology of networked control systems. Proceedings of the IEEE 95, 5–8 (2007) 3. Hespanha, J.P., Xu, Y.: A survey of recent results in networked control systems. Proceedings of the IEEE 95, 138–162 (2007) 4. Zhang, H., Yang, D., Chai, T.: Guaranteed cost networked control for T-S fuzzy systems with time delays. IEEE Trans. Syst. Man. Cybern. C 37, 160–172 (2007) 5. Wang, H.: Bounded Dynamic Stochastic Systems: Modeling and Control. Springer, London (2000) 6. Wang, H.: Minimum entropy control of non-Gaussian dynamic stochastic systems. IEEE Transactions on Automatic Control 47, 398–403 (2002) 7. Wang, H., Zhang, J.H.: Bounded stochastic distribution control for pseudo ARMAX systems. IEEE Transactions on Automatic Control 46, 486–490 (2001) 8. Guo, L., Wang, H.: Generalized discrete-time PI control of output PDFs using square root B-spline expansion. Automatica 41, 159–162 (2005a) 9. Guo, L., Wang, H.: PID controller design for output PDFs of stochastic systems using linear matrix inequalities. IEEE Trans. Sys. Man Cyb. 35, 65–71 (2005b) 10. Yue, H., Wang, H.: Minimum entropy control of closed-loop tracking errors for dynamic stochastic systems. IEEE Trans. on Automatic Control 48, 118–122 (2003) 11. Erdogmus, D., Pricipe, J.C.: An error-entropy minimization algorithm for supervised training of nonlinear adaptive systems. IEEE Transactions on Signal Processing 50, 1780– 1786 (2002a) 12. Erdogmus, D., Pricipe, J.C.: Generalized information potential criterion for adaptive system training. IEEE Transaction on Neural Networks 13, 1035–1043 (2002b) 13. Modha, D.S., Fainman, Y.: A learning law for density estimation. IEEE Transactions on Neural Networks 5, 519–523 (1994)

An Engineering Solution for Decoupling Control of Aircraft Motion Using Affine Neural Network Tsurng-Jehng Shen Hsing Wu College, No. 101, Sec.1, Fenliao Rd., LinKou, Taipei County 244, Taiwan China [email protected]

Abstract. In this paper, the traditional linearized model of the flight dynamics of a rigid airplane, in which aerodynamic forces, moments and thrust coefficient tables play an important role, are transformed into an analytical affine form using a newly proposed neural networks structure. This turns feasible the on-line inversion of the nonlinear flight dynamics. A neural decoupling control law based on nonlinear inverse control techniques (Input-Output Linearization) can then be used to perform flight trajectory tracking, Very good tracking performance is shown compared to a theoretical (tables based) autopilot. Keywords: flight dynamics, input-output linearization, neural networks, decoupling control, inverse control.

1 Introduction For more than a decade the problem of aircraft trajectory tracking [1][2] and the use of input-output linearization techniques [3], which provides decoupling control solutions for multi-input multi-output non-linear flight dynamics, have been widely studied. However, realizations of this kind of control technique needs not only that an accurate analytical model for the controlled system, but also that the system model satisfies strict inversibility conditions which are hardly verified in practical applications. So the applicability of input-output linearization techniques remain rather limited. In this paper, by applying a newly proposed neural networks structure [4], the dynamic motion model of rigid airplane which bases on aerodynamic and thrust coefficient tables is transformed into an analytical affine form. This turns feasible online inversion of the nonlinear flight dynamics model. A neural decoupling control law based on nonlinear inverse control techniques (input-output linearization) can then be used to perform trajectory tracking. And the effectiveness of the proposed approach is displayed by simulation results effectuated in a turbulent flight environment.

2 Analysis of Longitudinal Flight Dynamics of a Light Airplane Considering the equations of longitudinal motion of a specific airplane (TB20) supposed rigid and symmetric: Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 603–613. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

604

T.-J. Shen

− D + T cos α − mg sin γ ⎧ ⎪v= m ⎪ ⎪ γ = ( L + T sin α − mg cos γ ) ⎪ mv ⎪ ( L − mg cos γ + T sin α ) ⎪ ⎨α = q − mv ⎪ ⎪ M ⎪q = Iy ⎪ ⎪ z = −v sin γ ⎪ ⎩

(1)

where v : ground speed

m : mass

γ : flight angle α : angle of attack

D : drag force L : lift force

q : pitching angle speed z : altitude

T : thrust g : gravity

I y : y − axis moment of inertial M : pitching moment

A study of stability characteristics of this airplane shows us that when the input (control) elements of the system are chosen as the thrust force T and the elevator angle δ q , and respectively the ( v, q ) is chosen as outputs, the external dynamics (input-output relation) of the flight dynamics is globally inversible while the inner dynamics ( α , γ , z ) are asymptotically stable for admissible output trajectories. In this case the applicability of the nonlinear inverse control technique [1][2] depends directly on the expression of the thrust (T) and the aerodynamic effects (D and L) in the representation (1). In our study we suppose that the thrust can be represented by a known inversible function of throttle position and flight configuration, so the thrust being normally non-measurable in flight can be taken as an input element here. In practical case this function exist in rough precision and can induce a tracking delay at the velocity channel, while inverse control law is used for the tracking of desired outputs ( vd , qd ). But if the desired trajectories are chosen to keep the airplane flying (related to its available engine power) at a save airspeed range, this kind of error is normally less critical than airplane pitching attitude. However if higher tracking precision of flight speed is needed, the transfer function between thrust and system states should be modeled into the relation(1). Since aerodynamic coefficients cannot be measured in flight and normally only available in numerical tables (constructed from wind tunnel test), to transform these tables into a globally precise and continuous affine representation will be a possible solution of directly applying the input-output linearization techniques to the control of flight dynamics. To obtain a input-output linearizable normal form [3], D and M are modeled to take the form that the elevator angle δ q appears linearly (an affine form):

An Engineering Solution for Decoupling Control of Aircraft Motion M = M 0 + M δq δ q

605

(2)

D = D0 + Dδ q δ q

with M 0 ≡ 0.5 ρ v 2 SlCM 0 ; M δ q ≡ 0.5 ρ v 2 SlCM δ D0 ≡ 0.5ρ v 2 S CD0 ; Dδ ≡ 0.5 ρ v 2 S CDδ q

q

q

where ρ : the air density l : the airplane reference length S : the airplane reference area

and the C(⋅) ’s which don't exactly have the same mathematical meaning as conventional linearized aerodynamic coefficient are physically considered as the dimensionless derivative due to the variation of the sub-indexed factor, and should be approximated by our proposed neural networks later. Then by ignoring the stable inner dynamics, the reduced system states representation takes the following affine form: ⎛δq ⎞ ⎛v ⎞ ⎜ ⎟ = B + A⎜ ⎟ ⎝q⎠ ⎝T ⎠

⎡ Vq ⎛ V0 ⎞ ⎟ , A=⎢ ⎝ Q0 ⎠ ⎣Qq

where B = ⎜

(3)

VT ⎤ QT ⎥⎦

and V0 = ( − mg sin γ + D0 ) / m Q0 = M 0 / I y

, Qp = M δ / I y

V p = Dδ q / m

q

VT = cos α

QT = 0

A decoupling tracking control law basing on input-output linearization technique can then developed as follow: ⎛ ⎡vd + k1 ( vd − v ) ⎤⎞ ⎛δq ⎞ −1 ⎥⎟ ⎜ ⎟ = A ⎜⎜ − B + ⎢ ⎟ ⎝T ⎠ ⎣⎢ qd + k2 ( qd − q ) + k3 (θ d − θ ) ⎦⎥ ⎠ ⎝

(4)

here the sub-index d indicates the desired output (trajectory), and the parameters k1 , k2 , k3 can be designed referring to desired outputs response. In order that this decoupling control law provides bounded commands for the tracking of bounded desired flight trajectories, the determinant of matrix A should be non-zero. That means: CD

δq

CM δ

q

≠

ml cos α Iy

This (inversibility) condition will be forced to meet and verified later in this paper.

(5)

606

T.-J. Shen

3 Affine Neural Approximator and the Associated Training Algorithm In this study the Affine Neural Approximator [4] is applied to cope with the trajectory tracking mission for the mentioned airplane. The Affine Neural Approximator being proposed to approximate continuous input-output mapping by an affine model has been firstly presented by Shen T. J. et al [4]. For the convenience of reading, the basic ideas and some important results related to this original neural approximation algorithm is included as following. 3.1 Input-Output Representation Let Γ be a mapping that represents a continuous SISO system with output value y (k + d ) being defined by 2(ν − 1) variables y(k),...,y(k-v+1),u(k),..., u(k-v+1): y (k + d ) = Γ[ y (k ), y (k − 1),..., y ( k − v + 1), u (k ), u (k − 1),..., u ( k − v + 1)]

(6)

where u (k ) is the proposed control input. Remark: Γ can be taken as either a multidimensional numerical table (a set of inputoutput observation of some continuous mapping) or an unknown input-output relation for a SISO nonlinear dynamic system. In the first case (k+j) is simply considered as a notation, so, for example, y(k+i) and y(k+j) are two different (independent) variables. In the second case, (k+j) is the time-sequence index, the expression (6) is then a general input-output representation of a kth order nonlinear dynamic system where v is the state observation index and d is its relative degree, while the affine neural network is suitable for the approximation of these two kinds of input-output relation. 3.2 Affine Neural Approximator This neural network approximates the input-output mapping, Γ , by the affine form: y (k + d ) = F (W , Y (k )) + G (W , Y (k )) ⋅ u (k ) ∀Y (k ) ∈ U

(7)

where ( F (W , Y (k )), G (W , Y (k ))T is the output vector of the proposed neural network having W as its parameter weight matrices and bias vectors. In this relation Y (k ) = ( y (k ),..., y (k − ν + 1), u (k ),..., u ( k − ν + 1)), Y (k ) ∈ ℜ2(ν −1)

and U is the approximation domain (system operation domain), supposed compact. In our study we suppose that no analytical model of this system (mapping) is available and only input-output observations of the system (mapping) can be obtained. In this case the target functions for F and G are unknown. So that the training for affine neural network needs a suitable distal teacher which let the training process, in stead of minimizing the error between ( F , G ) and their unknown target functions ( F , G ) globally, minimize an auxiliary measurable performance index. The main result of [4] which concerns the use of distal teachers to globally train affine neural networks is briefly introduced below as well.

An Engineering Solution for Decoupling Control of Aircraft Motion

607

RESULT: Let U be an admissible operational domain for a smooth nonlinear function f, which has Γ as its input-output pairs training set (or its unknown input-output representation), let Vi, i=1,2,...,p be compact neighborhoods of p input-output pairs defined over U with V1 ∪ V2 ∪ … ∪ Vp ⊂ U , let N global be a multilayered feedforward neural network of type N 2(3 v −1),i ,i ,2 [5] which has P = [Y1 , Y2 ,..., Yp ] as batch input vectors i

2

and Q = [( F1 , G1 ) ,...,( Fp , G p ) ] as batch output vectors, T

T

let

(

p

) (

)

2 2 EG = ∑ 0.5 ⎡ y ( k j + d ) − y ( k j + d ) + u ( k j ) − u ( k j ) ⎤ ⎢ ⎥⎦ ⎣ j =1

(8)

be a global performance index for training, with u ( k j ) ≡ ⎡⎣ y ( k j + d ) − F ⎤⎦ G , then N global can be trained to be a global affine neural approximator of f (as well as an i , approximator of Γ ) with arbitrary precision, through p parallel distal teachers N DT i i=1,2,...,p, while N DT is chosen as the following matrix:

⎡ 1 i N DT =⎢ ⎣ −1 Gi

u ( ki ) ⎤ ⎥ y ( ki + d ) Gi2 ⎦

(9)

To train N global by using p distal teachers, a Modified Error Backpropagation architecture is developed, and very good performance were already displayed in [4], and is not to re-demonstrate in this paper.

4 Affine Neural Approximation of Aerodynamic Coefficient Tables The proposed neural network is used to transform the CM and CD of the studied airplane into continuous affine functions having the form (2). The coefficients CM and CD are approximated by CM and CD with: CM = CM 0 + CM δ δ q q

(10)

CD = CD0 + CDδ δ q q

where CM , CM , CD and CD 0

N CM

glocal

δq

and N C

Dglobal

δq

0

:

(C (C where WC

M

are the outputs of two affine neural approximators

and WC

D

) =N ) =N T

M0

CM δ

D0

CDδ

p

CM global

T

p

CDglobal

(WCM ,α ,α , q, v, δ p , z )

(11)

(WCD , α ,α , v, δ p , z )

represent respectively the weight matrix and (α ,α , q, v, δ q , z ) ;

(α ,α , v, δ q , z ) are input vectors of the neural net work N CM

global

and N C

Dglobal

.

608

T.-J. Shen

The input vectors elements chosen for each neural network are flight conditions which affect most significantly the evolution of the corresponding aerodynamic effects. The performance index for training is defined as: E(⋅) = (1 / 2)[(C(⋅) − C(⋅) )2 + (δ q − δ q ) 2 ]

(12)

with δ q = (C(⋅) − C(⋅) ) / C(⋅) . During training phase, the target values of the CM and CD δp

0

are checked up from existing aerodynamic coefficient tables constructed on the tables axis (α ,α , q, v, δ p , z ) and (α ,α , v, δ p , z ) . The structure of the affine neural approximator N C

Dglobal

and its distal teacher are

illustrated in Fig. 1.

N DTC

N CD

D

global

α

CD

CD0

α

1

δq

v

δq

−1/ CDδ

z

CDδ

CD / CC2D

δq

q

δq

q

Fig. 1. Input-output structure of the affine neural approximator for CD

4.1 Affine Neural Approximator for Drag and Pitching Moment Coefficients 3 For approximating the coefficients CD and CM , two neural networks of type N 5,25,15,2 3 and N 6,25,15,2 were chosen. These neural networks have hyperbolic tangent activation

function at input as well as hidden layers and sigmoid function with range parameters at output layers. The range parameters at the output layers of these neural networks are chosen by referring to qualitative knowledge of the airplane's aerodynamic characteristic, such that: −2 ≤ CM 0 ≤ −0.1

(13)

−4.8 ≤ CM δ ≤ −2.8 q

with

(

)

(

)

CM = CM 0 WCM ,α , α , v, q,δ q , z + CM δ WCM ,α ,α , v, q,δ q , z ⋅ δ q q

(14)

An Engineering Solution for Decoupling Control of Aircraft Motion

609

and −1 ≤ CD0 ≤ 1

(15)

0.05 ≤ CDδ ≤ 0.15 q

with

(

)

(

)

CD = CD0 WCM ,α ,α , v,δ q , z + CDδ WCM , α , α , v,δ q , z ⋅ δ q

With these choices, CD

δq

( ml cosα ) verified. Noting

CMδ

q

q

(16)

is a negative value, while in the case 0 ≤ α < π / 2 ,

I y is a positive value, then the inversibility condition (5) can be easily

that,

in

general,

the

first

order

linearization

coefficient

⎛ ΔD ⎞ −1 0.5 ρ v 2 S ) of an airplane may change its sign. But we limit here the CDδ ≡ ⎜ q ⎜ Δδ q ⎟⎟ ( ⎝ ⎠ corresponding neural approximation CDδ to a positive value, otherwise the training q

process would diverge when CD takes a near zero value. Anyway, it was shown in δq

[4] that parameter convergence of the training of affine neural approximation is not a necessary condition for obtaining a stable decoupling control law basing on it. However it should also be taken in mind that in our neural model the C(⋅) doesn't have the same mathematical meaning as conventional aerodynamics coefficients. 4.2 Training of N C

Dglobal

and N C

M local

In longitudinal flight the estimation precision of in-flight aerodynamic drag affects only flight speed, besides the elevator angle has less direct contribution to drag force. So, normally the resulted total speed tracking error (error due to drag and thrust estimation) can be compensated by a rapid thrust response. Basing on this observation, to reduce the neural networks training processing time, the affine neural and N C are trained differently. N C , whose precision has approximator N C M local

Dglobal

Dglobal

been judged little critical, is globally optimized by intensive off-line training over , being locally off-line trained for typical typical flight envelope, while the N C M local

initial flight conditions, is continuously optimized by on-line training. Remark: The N C

M local

is firstly off-line trained basing on the initial 5 seconds' flight.

Then the on-line training of the pitching moment coefficient approximator is effectuated for a moving windows which uses the last 5 in-flight observations ( τ observ = 0.5 sec ) of ( a,α , v, q,δ q , z ) ~ ( CM , δ q ) . The local neural approximation error is check every τ observ , in the case where the error is larger than a pre-defined threshold, on-line training takes place for maximum 20 iterations.

610

T.-J. Shen

The training data set for N C

Dglobal

is prepared from 20 simulations (under the control

of a theoretical autopilot) of standard maneuver flight (initial altitude 500ft) in a turbulent atmosphere, Totally 500 input-output vector pairs ( a,α , v, δ q , z ) ~ ( CD , δ q ) were recorded (by checking known coefficient tables and simulated flight configurations) in the training data set. After a successful training, the global performance index value is reduced to a acceptable level. The obtained neural networks’ outputs are shown in Fig. 2.~Fig.5. 0.16 0.12 0.08

App.

0.04

Table

0.00 0.50

10.50 20.50 30.50 40.50 50.50 60.50 70.50 80.50 Flight time (sec)

Fig. 2. CD0 (App-Approximated by N CD

global

) compared to CD0 (Table)

0.12 0.08 0.04

App.

0.00

Table

-0.04 0.50

20.50

40.50 60.50 Flight time (sec)

Fig. 3. CD (App-approximated by N CD δq

global

80.50

) compared to CDδ (Table) q

0.16 0.12 0.08

App.

0.04

Table

0.00 0.50

20.50

40.50 60.50 Flight time (sec)

80.50

Fig. 4. CD (App - Calculated by distal teacher N DTC ) compared to CD (Table -checked from D

coefficient Tables)

rad

An Engineering Solution for Decoupling Control of Aircraft Motion

0.00 -0.04 -0.08

611

App. Measured

-0.12 -0.16 -0.20 0.50

20.50

40.50 60.50 Flight time (sec)

80.50

Fig. 5. δ q (App - from distal teacher N DTC ) compared to δ q (Measured) D

5 Simulation Results for Trajectory Tracking The proposed neural decoupling control law takes the expression (4) while the matrix and N C A and B are calculated in-flight basing on the outputs of N C . The Dglobal

M local

desired flight speed is taken as: vd = 0.14 mach vd = 0

The desired trajectory qd and qd are taken equal to "0" while the desired pitching angle ( θ d ) is given by a guidance controller that takes altitude command as input. The Altitude command for this guidance system is given by: zd = z0 + 120sin ( ( t − t0 ) / 32 ) + 0.5 ( t − t0 )

with z0 = 152.4 m (500 ft ) . The control parameters were given as: k1 = 1, k2 = 14, k3 = 100 , which correspond a stable system represented by the following transfer function (Laplace Transformation form): v (s)

vd ( s )

=

1 S +1

(17)

θ (s) ωn 2 = 2 θ d ( s ) S + 2ηωn S + ωn 2

where ωn = 10 and the damping ration η = 0.7 . The simulation is initialized at stable level flight conditions with z =152.4 m, Mach = 0.14, δ q = −3.98 and θ = α = 3.94 . Simulation results show that, even with the presence of wind disturbances, the on-line trained N C

M local

can provide satisfactory

affine parameters ( Cm , Cm ) which guarantee the effectiveness of the decoupling 0

δq

control law. Several simulation results are illustrated in Fig.6.~Fig.8.

Altitude (m)

612

T.-J. Shen

400.00 Aircraft trajectory Desired trajectory

300.00 200.00 100.00 0.00 0.00

50.00

100.00

150.00

200.00

Flight time (sec)

On-line error

1E-06 1E-04 1E-02

Fig. 6. Evolution of aircraft altitude

0.00

50.00

100.00

150.00

200.00

Flight time (sec)

Fig. 7. The on-line neural approximation error (12) for local pitching moment coefficient

Iteration

20 15 10 5 0 0.00

50.00

100.00

150.00

200.00

Flight time (sec)

Fig. 8. On-line training iterations during the flight

6 Conclusion In this paper, the exist model of longitudinal motion of a rigid airplane, in which aerodynamic and thrust coefficient tables play an important role, are transformed into an analytical affine form using a newly developed neural network structure. This turns feasible on-line inversion of the flight dynamic model. The nonlinear inverse control law was used to perform trajectory tracking missions. Several interesting points of this new neural control structure are listed below: z

The neural controller shows a tracking performance as good as the conventional autopilot in the common flight envelope (Fig. 6.), but the neural controller can be more robust and adaptive than the conventional autopilot, which is designed to work with coefficient tables being established using linearized model and wind tunnel test.

An Engineering Solution for Decoupling Control of Aircraft Motion

z z z z

613

Only the non-analytical parts (numerical tables) of the flight dynamics model are approximated by the proposed neural networks, so the nonlinear characteristics of the controlled system are kept. The nonlinear inverse control law (input-output linearization) is based on affine neural representation. This directly provides a decoupling control, and stable response can be guaranteed by a priori chosen control parameters. Once the mission requirements concerning the system output response is changed, only the control parameters should be modified while the off-line trained neural networks need no change or extra training. The proposed neural networks can be trained on-line (in-flight), so can turn the controlled system more robust and adaptive to environment perturbation and uncertain dynamics.

In this study, the in flight target pitching moment coefficient value for on-line training is calculated from existing coefficient tables. This has limited the adaptive capability of the proposed decoupling control law. For turning the controller more adaptive for model uncertainties, instead of transforming numerical coefficient tables into neural affine functions, it needs only to apply the proposed approach to on-line measurable (precisely estimable) values such as the total external forces and moments of the aircraft motion.

References 1. Singh, S.N., Schy, A.A.: Output Feedback Non-linear Decoupled Control Synthesis and Observer Design for Maneuvering Aircraft. International Journal of Control 31(4), 781–806 (1980) 2. Zhiqiang, Z.: Nonlinear Decoupling Control of Aircraft Motion. Journal of Guidance, Control, and Dynamics 18(4) (July-August 1995) 3. Isidori, A.: Nonlinear Control Systems: An Introduction, 2nd edn. Springer, Heidelberg (1989) 4. Shen, T.J., Jou, C.S., Youh, M.J., Chen, C.T.: Specialized Affine Approximation for Nonlinear Systems Output Tracking Using Neural Networks. In: Yu, W., He, H., Zhang, N. (eds.) ISNN 2009. LNCS, vol. 5551, pp. 679–688. Springer, Heidelberg (2009) 5. Narendra, K.S., Mukhopadhyay, S.: Adaptive Control of Nonlinear Multivariable Systems Using Neural Networks. Neural Networks 7(5), 737–752 (1994)

Black-Box Input-Output Identification of a Class of Nonlinear Systems Using a Discrete-Time Recurrent Neurofuzzy Network Marcos A. Gonz´alez-Olvera1 and Yu Tang2 1

Universidad Aut´ onoma de la Ciudad de M´exico, San Lorenzo Tezonco, DF, M´exico 09790 Tel.: (52)-55-58501901 ext 14200 [email protected] 2 Facultad de Ingenier´ıa-UNAM Coyoacan DF, Mexico 04510, M´exico Tel.: (52)-55-56234142, (52)-55-56223013 [email protected]

Abstract. From theory based on adaptive observers, this paper presents a structure for black-box identification based on state-space recurrent neural networks for a class of dynamic nonlinear systems in discrete-time. The network catches the dynamics of the unknown plant and jointly identifies its parameters using only output measurements. The stability and the convergence of the training algorithm and the ultimate bound on the identification error as well as the parameter error are established in the Lyapunov sense. Numerical examples using simulated and experimental systems are included to demonstrate the effectiveness of the proposed method. Keywords: Neural-nework models, Fuzzy modelling, System Identification, Discrete-time systems.

1

Introduction

In the literature and research papers, neural networks and fuzzy systems have been intensively used as nonlinear static identifiers, based on their approximation capabilities and reported training algorithms for control, classification, clustering and identification problems [1,2,3,4]. Moreover, their combination into neurofuzzy structures has proved to be a valuable structure where the resultant structure can be trained as as a neural network, and the obtained parameters can be interpreted [5]. When feedback connections occur within the structure, so called dynamical or recurrent neurofuzzy networks (RNN) provide a powerful tool for identifying dynamic systems. One of the very first approaches was given by [6] for learning trajectories. Since then, recurrent networks have been reported with a wide variety of complexity. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 615–622. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com 

616

M.A. Gonz´ alez-Olvera and Y. Tang

In [4] the authors proposed several networks with feedback within the internal layers for identification and control of nonlinear systems, [7] used time-delay networks and [8] used fully recurrent networks for identification, where all layers are considered to be inputs. Networks such as globally static-locally recurrent structures have been proposed by [9,10] for identification, using neurons with linear filters in the synapsis. Algorithms such as Backpropagation-through-time [11] and Real-time recurrent learning [8] have been used for network training. However, these algorithms and structures tend to be slow and lack stability analysis [12]. Training based on linear approximations such as recursive-least-squares [13,14] and Kalman filters [15] have been reported and successfully applied. However, the Kalman filter is known to have a difficult stability analysis, so ellipsoidal methods have been reported instead [16]. Relying on the specific structure of the networks, [17,18,12] guarantee stable training with a state-space structure, but need the measurements of the actual states of the system, which may not be available in some situations. In this work, we propose a discrete-time recurrent neurofuzzy network in statespace representation derived from the work [19] where a continuous-time recurrent neurofuzzy network was considered. Motivated from research in adaptive observer design [20,21,22], the proposed network is useful to identify a class of nonlinear systems using only output measurements. The structure takes the advantage of the approximation capabilities of neurofuzzy networks, and the training algorithm is based on nonlinear adaptive observers. The resulting network and the training algorithm can estimate equivalent states of the system, jointly with the parameter estomation. In the rest of the paper, we give first in Section 2 the basic analysis of the problem. In Section 3 we present an identifier structure that trains and identifies the input-output response of a class of nonlinear systems. In this same section the boundness and convergence of the training algorithm are discussed. Examples of the performance of the neural identifier are shown in Section 4 by identifying a nonlinear system and an experimental system. Finally, conclusions are drawn in Section 5.

2

Problem Statement

In the first place, consider a single-input-single-otuput system in continuous time to be identified x˙ = f (x(t), u(t)), y(t) = h(x(t)),

(1)

where f : n ×  → n and h : n →  are smooth and bounded functions, x ∈ n is the state and y, u ∈  are the input and output signals respectively. The main problem to be addressed in this work is to identify system dynamics by reconstructing a dynamic mapping u → y when a few information on functions f and h is available and only input-output measurements are available. These conditions imply that the class of systems must be observable and controllable

Black-Box Input-Output Identification of a Class of Nonlinear Systems

617

in the region of interest, that is, that the entire dynamics of the system can be reconstructed from input-output measurements. The signals from the system (1), measured in an experimental framework, are generally obtained using analog-to-digital converters. Therefore, the signals that can be measured and stored are discontinuous in time rather than continuous. Having this in mind, let consider a discrete-time representation of the system (1) in the form (with a little abuse of notation): x[k + 1] = f (x[k], u[k]), y[k] = h(x[k]),

(2)

In the development of the identifier using only the input-output measurements, we need to make some assumptions about this system: Assumption 1. i) The system is controllable, that is, the input affects all the states and no uncontrollable spaces exist in the system, ii) The system is observable, that is, the states can be reconstructed from measurements of the input and output signals, Now, if (2) complies with the previous assumptions, it belongs to the class of nonlinear systems transformable to an output feedback form [21] z[k + 1] = Az[k] + f¯(y[k], u[k]), y[k] = Cz[k], with f¯(y, u) smooth and (A, C) in observer canonical form     I(n−1)×(n−1) A = 0n×1 , C = 1 01×(n−1) . 01×(n−1)

(3)

(4)

where it is assured that the pair (A, C) is observable. If an output injection Ly, with y = Cz, is considered in the state equation of (3), z[k + 1] = Az[k] + f¯(y[k], u[k]) + Ly[k] − Ly[k] ¯ = Az[k] + f¯(y[k], u[k]) + Ly[k] ¯ = Az[k] + f¯∗ (y[k], u[k])

(5)

Δ then some designed stable dynamics can be set into matrix A¯ = A − LC, and ∗ ¯ the problem is now to identify the term f (y[k], u[k]). By the universal approximation theorem [23], as f¯∗ is assumed smooth and bounded, then a neurofuzzy structure ϕ(y[k], u[k], θ) can identify it with arbitrary precision. In this particular case, we consider that the network is linearly parametrizable, so ϕ(y[k], u[k], θ) = ϕ(y[k], u[k])θ. In this form, the function f¯∗ can be approximated by the network

f¯∗ (y, u) = ϕ(y[k], u[k], θ) + ε where ε is the approximation error of the network. In this sense, the objective is now to find a discrete-time recurrent neural network described by ¯ η[k + 1] = Aη[k] + ϕ(y[k], u[k])θη , yˆ[k] = Cη[k],

(6)

618

M.A. Gonz´ alez-Olvera and Y. Tang

such that, by using only the input signal u[k] and output measurement y[k], it can generate a dynamic mapping u[k] → yˆ[k], such that supt≥0 |ˆ y[k] − y[k]| is minimized by tuning the network parameter θη . Note that the matrix A¯ is the Δ same one as the defined previously as A¯ = A − LC, The main problem is now to obtain an algorithm and a neurofuzzy structure to identify the parameters θη and estimate the states ηˆ[k] using only input-output measurements. This will be discussed in the next section.

3

Proposed Network and Training Algorithm

The neurofuzzy network proposed in this work is defined by the following fuzzy rules: ⎡ ⎡ ⎤ ⎤ θu,i,1 θy,i,1 ⎢ θu,i,2 ⎥ ⎢ θy,i,2 ⎥ ⎢ ⎢ ⎥ ⎥ i Ri : If yk is Fi and uk is Hi then ηk+1 = Aηk + ⎢ . ⎥ uk + ⎢ . ⎥ ρ(yk ). . . ⎣ . ⎦ ⎣ . ⎦ θu,i,n θy,i,n i = η i [k + 1],ρ :  → , ρ = ρ(y) a smooth with yk = y[k], uk = u[k], ηk+1 sigmoid function, Fi , Hi are fuzzy sets. The membership functions are defined 2 2 by gaussian shapes Fi (y, u) = exp(−σy,i (y − μy,i )2 − σu,i (u − μi,u )2 ). The main objective of the fuzzy part is to obtain a partition of the space y − u where each zone can be approximated and identified by the proposed local system in (7). Now, as the states η and parameters θ are unknown, the problem is to find them by input-output measurements. According to [24], this can be solved by the discrete-time adaptive observer

Υ [k + 1] = (A¯ − KC)Υ [k] + ϕ[k] ˆ + 1] = θ[k] ˆ + μ[k]Υ [k]T C T (y[k] − C ηˆ[k]) θ[k

(8)

ˆ + Ky[k] ηˆ[k + 1] = (A¯ − KC)ˆ η [k] + ϕ[k]θ[k] ˆ +Υ [k + 1](Δθ[k])

(9)

(7)

where ηˆ is the estimation of η, A − KC is a Hurwitz and filter matrix for ϕ, Γ is a positive definite gain matrix (possibly time-varying). The parameter and estimation error are exponentially convergent given that the regressor ϕ[k] is persistently exciting, as stated in the next theorem: Theorem 1. Consider the system (2) and the network (6). Assume that the approximation error is sufficiently small, then the training algorithm results in an exponential convergence of the parameters as well as the states to their true values (that is, η˜ = ηˆ − η → 0 and θ˜ = θˆ − θ → 0), given that the persistence of excitation condition k−L−1 1 T Υi [i]C T CΥ [i] ≥ αI (10) L i=k

is fulfilled.

Black-Box Input-Output Identification of a Class of Nonlinear Systems

619

Proof. Design Ao = A¯ − KC to be Hurwitz. In the same sense as [25], define ˜ ˆ − θ[k]. If the approximation the error signals η˜[k] = ηˆ[k] − η[k], θ[k] = θ[k] error ε is sufficiently small, then the error dynamics can be found as η˜[k + 1] = ˜ ˜ ˜ ˜ Ao η˜[k]+ϕ[k]θ[k], Υ [k+1](θ[k+1]− θ[k]). Defining the error ζ[k] = η˜[k]−Υ [k]θ[k], ˜ its dynamics is ζ[k + 1] = A0 ζ[k] + (Ao Υ [k] + ϕ[k] − Υ [k + 1])θ[k]. So using (9), one obtains the exponentially stable equation ζ[k + 1] = A0 ζ[k]. ˜ ˜ + 1] = θ[k] ˜ − Now, taking the parametric error θ[k], its dynamics is θ[k T T T T T T ˜ Γ Υ [k]C C η˜[k] = (I − Γ Υ [k]C CΥ [k])θ[k] − Γ Υ [k]C Cζ[k]; in which can be proven that the origin of the homogeneous part of the previous equation is exponentially stable, given that (10) is fulfilled. Remark 1. The assumption of the sufficient small approximation error (that is, that the approximation error is much lesser than the amplitude of the signals considered) was made in the proof of the above theorem. However, as neural networks involve an inherent approximation error, the result of the theorem can be considered to have a practical stability of the estimation and parameter error. Also note that the eigenvalues of A¯ − KC and Γ play an important role in the convergence and filtering of the identification algorithm. Remark 2. The persistence of excitation condition implies that all the rules are activated in a certain period of time, and then filtering Υ contains all the original frequencies in ϕ.

4

Examples

In this section two examples are given to demonstrate the effectiveness of the proposed method. First, a typical benchmark model reported by [4] is identified. Then, a DC motor with associated nonlinearities such as nonlinear inertia and dry friction is identified using experimental data. 4.1

Nonlinear Benchmark System

For this example, the system to be identified is described by the following equations [4]: x1 x2 x3 x5 (x3 − 1) + x4 . 1 + x23 + x22 (11) The system is excited using 500 samples of a a pseudo-aleatory signal u such that u ∈ [−1, 1] and changes every 20 seconds. Then, the recurrent neural network is designed with 5 rules and 3 states, resulting in 30 parameters, and the membership functions are obtained by running a standard Fuzzy C-Means algorithm [26,27] and are depicted in Fig. 1.a. The training algorithm is set using ¯ = {−0.1, −0.2}, λ{A¯ − KC} = {−0.3, −0.5}. In this case, Γ is chosen λ{A} as a diagonal matrix such that begins in the first epoch with a value of 0.1, and decays linearly to 0.01 in the 50th and last epoch. Then, the model is run using validation data, obtaining a RMS error of 0.0323. The comparison between the original system and the neural identifier is shown in Fig. 1.b. yk = f (yk−1 , yk−2 , yk−3 , uk , uk−1 ), f (x1 , x2 , x3 , x4 , x5 ) =

620

M.A. Gonz´ alez-Olvera and Y. Tang a)

b)

Fig. 1. Nonlinear System: a) Membership functions, b)Identification results

4.2

Experiment on a DC Motor

For this example it is considered a DC motor with associated nonlinearities such as dry friction and nonlinear inertia in an experimental framework. The sampling time is of 100ms and it is excited with an voltage input pseudo-random signal a)

b)

Fig. 2. DC Motor: a) Training Signals, b) Training result

Black-Box Input-Output Identification of a Class of Nonlinear Systems

621

that changes every second. The input and output signals of this system are shown in Fig. 2.a. In order to obtain the antecedent parts of the fuzzy rules, a Fuzzy C-Means algorithm is run with five rules (r = 5). The training algorithm is run ¯ = {−0.1, −0.2}, λ{A−KC} ¯ using two states (n = 2), λ{A} = {−0.3, −0.5} and Γ = InT ×nT . In Fig. 2.b it is shown the parameter evolution during training, and the input-output response of both systems (the plant and the obtained RNFN). It can be seen that the parameters remain bounded, and that the network catches the dynamics of the system accurately.

5

Conclusions

In this paper we have presented a new recurrent neurofuzzy network for modeling and identification a class of nonlinear systems using only input and output measurements in discrete-time. It gives both the state and parameter estimates of the plant. It was shown that the algorithm, based on adaptive observers theory, results in bounded identification error as well as bounded parametric error, that will depend on the approximation capabilities of the network. A benchmark and an experimental example were presented in order to show the identification effectiveness of this network. Also, for future work we consider to train also the antecedent parameters to improve the quality of the identification. Acknowledgements. This work was supported by project PAPIIT IN-120009. The work of Marcos Gonz´alez-Olvera is supported by CONACyT-Mexico.

References 1. Cybenko, G.: Approximation by superpositions of a sigmoidal function. Math. Con. Signal Syst. 2, 303–314 (1989) 2. Nelles, O.: Nonlinear System Identification. Springer, Berlin (2001) 3. Werbos, P.: Beyond regression: New tools for prediction and analysis in the behavioral sciences. Harvard University (1974) 4. Narendra, K., Parthasarathy, K.: Identification and control of dynamical systems using neural networks. IEEE Tras. Neural Networks 1, 4–27 (1990) 5. Jang, J.S.R.: Anfis: Adaptive-network-based fuzzy inference system. IEEE Transactions on Systems, Man and Cybernetics 23, 665–685 (1993) 6. Hopfield, J.: Neural Networks and Physical Systems with Emergent Collective Computational Abilities. Proceedings of the National Academy of Sciences 79(8), 2554–2558 (1982) 7. Weibel, A., Hanazawa, T., Hinton, G., Shikano, K., Lang, K.J.: Phenomene recognition using time-delay neural networks. IEEE Transactions on Acoustics, Speech and Signal Processing 37, 328–339 (1989) 8. Williams, R.J., Zipser, D.: A learning algorithm for continually running fully recurrent neural networks. Neural Computation 2(1), 270–280 (1989) 9. Wan, E.A.: Temporal back-propagation for fir neural networks. In: Proc. of the International Joint Conference on Neural Networks, pp. 575–580 (1990) 10. Back, A.D., Tsoi, A.C.: A time series modeling methodology using fir and iir synapses. In: Proc. Workshop on Neural Networks for Statistical and Economic Data, pp. 187–194 (1990)

622

M.A. Gonz´ alez-Olvera and Y. Tang

11. Rumelhart, D., Hinton, G., Williams, R.: Learning Internal Representations by Error Propagation, 8. MIT Press, Cambridge (1986) 12. Mastorocostas, P.A., Theocharis, J.B.: A recurrent fuzzy-neural model for dynamic system identification. IEEE Trans. onf Systems, Man and Cybernetics - Part B: Cybernetics 32(2), 176–190 (2002) 13. Johansson, R.: System Modeling and Identification, 1st edn. Prentice Hall Information and System Sciences. Prentice Hall, Englewood Cliffs (1993) 14. Passino, K.M.: Fuzzy Control. Addison Wesley Longman, Menlo (1998) 15. Haykin, S.: Kalman Filtering and Neural Networks. Wiley, Chichester (2001) 16. de Jes´ us Rubio, J., Yu, W.: Nonlinear system identification with recurrent neural networks and dead-zone Kalman filter algorithm. Neurocomputing 70(13-15), 2460–2466 (2007) 17. Poznyak, A., Poznyak, T., Chairez, I.: Dynamic neural observers and their application for identification and purification of water by ozone. Automation and Remote Control 67(6), 887–899 (2006) 18. Yu, W., Li, X.: Fuzzy identification using fuzzy neural network. IEEE Trans. on Fuzzy Systems 12(3) (June 2004) 19. Gonzalez-Olvera, M., Tang, Y.: Identification of a class of nonlinear systems by a continuous-time recurrent neurofuzzy network. In: American Control Conference, ACC 2009 (2009) 20. Bastin, G., Gevers, M.: Stable adaptive observers for nonlinear time-varying systems. IEEE Transactions on Automatic Control 33(7), 650–658 (1988) 21. Marino, R.: Adaptive observers for single output nonlinear systems. IEEE Transactions on Automatic Control 35(9), 1054–1058 (1990) 22. Zhang, Q.: Adaptive observer for multiple-input-multiple-output (MIMO) lineartime-varying systems. IEEE Transactions on Automatic Control 47(3), 525– 529 (2002) 23. Wang, L.X.: Adaptive Fuzzy Systems and Control. Prentice-Hall, Englewood Cliffs (1994) 24. Combastel, C., Zhang, Q., Lalami, A.: Fault Diagnosis based on the Enclosure of Parameters Estimated with an Adaptive Observer. In: Sixth IFAC Symposium on Fault Detection, Supervision and Safety of Technical Processes (2006) 25. Guyader, A., Zhang, Q.: Adaptive observer for discrete time linear time varying systems. In: Proc. of 3rd IFAC/IFORS Symposium on Identification and System Parameter Estimation, SYSID 2003 (2003) 26. Dunn, J.: A fuzzy relative of the ISODATA process and its use in detecting compact well-separated clusters. Journal of Cybernetics 3(3), 32–57 (1973) 27. Bezdek, J.: Pattern Recognition with Fuzzy Objective Function. Plenum Press, NY (1981)

Passivity Analysis of Stochastic Neural Networks with Mixed Time-Varying Delays Qinggao He1 and Qiankun Song2 1

Department of Mathematics, Yibin University, Yibin 644000, China 2 Department of Mathematics, Chongqing Jiaotong University, Chongqing 400074, China [email protected]

Abstract. In this paper, the passivity problem is investigated for a class of stochastic neural networks with discrete time-varying delay and distributed time-varying delay as well as generalized activation functions. By constructing appropriate Lyapunov-Krasovskii functionals, and employing the free-weighting matrix method and stochastic analysis technique, a delay-dependent criterion for checking the passivity of the addressed neural networks is established in terms of linear matrix inequalities (LMIs), which can be checked numerically using the effective LMI toolbox in MATLAB. An example is given to show the effectiveness and less conservatism of the proposed criterion. It is noteworthy that the traditional assumptions on the differentiability of the time-varying delays and the boundedness of its derivative are removed. Keywords: Passivity, Stochastic neural networks, Time-varying delays, Distributed delay.

1

Introduction

It is known that the passivity theory is an effective tool to the stability of nonlinear system [1]. The main idea of passivity theory is that the passive properties of system can keep the system internal stability [2]. Thus, the passivity theory has received a lot of attention from the control community since 1970s [3]-[5]. Recently, the passivity theory for delayed neural networks was investigated, some criteria checking the passivity were provided for certain or uncertain neural networks with time-varying delays, see [6]-[12] and references therein. It is worth pointing out that, the given criteria in [6]-[12] have been based on the following assumptions: 1) the time-varying delays are continuously differentiable; 2) the derivative of time-varying delay is bounded and is smaller than one; and 3) the activation functions are bounded and monotonically nondecreasing. However, time delays can occur in an irregular fashion, and sometimes the time-varying delays are not differentiable. In such a case, the methods developed in [6]-[12] may be difficult to be applied, and it is therefore necessary to further investigate the passivity problem of neural networks with time-varying delays under milder Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 623–630. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com 

624

Q. He and Q. Song

assumptions. To the best of our knowledge, few authors have considered the passivity problem for stochastic neural networks with discrete time-varying delay and distributed time-varying delay as well as generalized activation functions. Motivated by the above discussions, the objective of this paper is to study the passivity of stochastic neural networks with discrete time-varying delay and distributed time-varying delay as well as generalized activation functions by constructing appropriate Lyapunov-Krasovskii functionals, and employing the free-weighting matrix method and stochastic analysis technique. The obtained sufficient condition requires neither the differentiability of time-varying delays nor the monotony of the activation functions, and are expressed in terms of linear matrix inequalities (LMIs), which can be checked numerically using the effective LMI toolbox in MATLAB. An example is given to show the effectiveness and less conservatism of the proposed criterion.

2

Problem Formulation and Preliminaries

In this paper, we consider the following neural network model  t f (x(s))ds + u(t)]dt dx(t) = [−Dx(t) + Af (x(t)) + Bf (x(t − τ (t))) + C t−h(t)

+σ(t, x(t), x(t − τ (t)))dω(t)

(1)

for t ≥ 0, where x(t) = (x1 (t), x2 (t), · · · , xn (t))T ∈ Rn is the state vector of the network at time t, n corresponds to the number of neurons; τ (t) and h(t) denote the discrete time-varying delay and the distributed time-varying delay, respectively; D is a positive diagonal matrix, A = (aij )n×n , B = (bij )n×n and C = (cij )n×n represent the connection weight matrix, the discretely delayed connection weight matrix and the distributively delayed connection weight matrix, respectively; f (x(t)) = (f1 (x1 (t)), f2 (x2 (t)), · · · , fn (xn (t)))T denotes the neuron activation at time t; u(t) = (u1 (t), u2 (t), · · · , un (t))T ∈ Rn is a varying external input vector; σ ∈ Rn×q is the diffusion coefficient vector and ω(t) = (ω1 (t), ω2 (t), · · · , ωq (t))T is an q-dimensional Brownian motion defined on a complete probability space (Ω, F, {Ft }t≥0 , P ) with a filtration {Ft }t≥0 satisfying the usual conditions (i.e., it is right continuous and F0 contains all P -null sets); Throughout this paper, we make the following assumptions: (H1). The time-varying delays τ (t) and h(t) satisfy 0 ≤ τ (t) ≤ τ,

0 ≤ h(t) ≤ h,

where τ and h are constants. (H2). For any j ∈ {1, 2, · · · , n}, fj (0) = 0 and there exist constants Fj− and Fj+ such that fj (α1 ) − fj (α2 ) ≤ Fj+ Fj− ≤ α1 − α2 for all α1 = α2 .

Passivity Analysis of Stochastic Neural Networks

625

(H3). There exist two scalar ρ1 > 0, ρ2 > 0 such that the following inequality trace[σ T (t, u, v)σ(t, u, v)] ≤ ρ1 uT u + ρ2 v T v holds for all (t, u, v) ∈ R × Rn × Rn . Definition 1. ([6]) System (1) is called globally passive in the sense of expectation if there exists a scalar γ > 0 such that   tp    tp  2E f T (x(s))u(s)ds ≥ −E γ uT (s)u(s)ds 0

0

for all tp ≥ 0 and for all x(t, 0). To prove our results, the following lemmas that can be found in [8] are necessary. Lemma 1. ([8]) For any constant matrix W ∈ Rm×m , W > 0, scalar 0 < h(t) < h, vector function ω : [0, h] → Rm such that the integrations concerned are well defined, then  h(t)   h(t) T   h(t)  ω(s)ds W ω(s)ds ≤ h(t) ω T (s)W ω(s)ds. 0

0

0

Lemma 2. ([8]) For given matrices H, E and F with F T F ≤ I and a scalar ε > 0, the following holds: HF E + (HF E)T ≤ εHH T + ε−1 E T E. Lemma 3. ([8]) Given constant matrices P , Q and R, where P T = P , QT = Q, then   P R 0

3

and

P + RQ−1 RT < 0.

Main Results

For presentation convenience, in the following, we denote F1 = diag(F1− F1+ , · · · , Fn− Fn+ ),

F2 = diag(

F − + Fn+ F1− + F1+ ,··· , n ). 2 2

Theorem 1. Under assumptions (H1)-(H3), model (1) is passive in the sense of expectation if there exist two scalar γ > 0, λ > 0, four symmetric positive definite matrices Pi (i = 1, 2, 3, 4), two positive diagonal matrices L and S, and four matrices Qi (i = 1, 2, 3, 4) such that the following two LMIs hold: P1 < λI,

(2)

626

Q. He and Q. Song

⎡

Ω11 ⎢ ∗ ⎢ ⎢ ∗ ⎢ ⎢ ∗ ⎢ ⎢ ∗ ⎢ ⎢ ∗ Ω=⎢ ⎢ ∗ ⎢ ⎢ ∗ ⎢ ⎢ ∗ ⎢ ⎢ ∗ ⎢ ⎣ ∗ ∗

⎤ Ω12 Ω13 Q2 B Q3 0 Q2 C Q2 Q3 Q3 Q4 0 Ω22 Q1 A Q1 B 0 0 Q1 C Q1 0 0 0 0 ⎥ ⎥ ∗ Ω33 0 0 0 0 −I 0 0 0 0 ⎥ ⎥ ∗ ∗ −S F2 S 0 0 0 0 0 0 0 ⎥ ⎥ ∗ ∗ ∗ Ω55 Q4 0 0 0 0 0 Q4 ⎥ ⎥ ∗ ∗ ∗ ∗ −P2 0 0 0 0 0 0 ⎥ ⎥ < 0 (3) ∗ ∗ ∗ ∗ ∗ −P4 0 0 0 0 0 ⎥ ⎥ ∗ ∗ ∗ ∗ ∗ ∗ −γI 0 0 0 0 ⎥ ⎥ ∗ ∗ ∗ ∗ ∗ ∗ ∗ − τ1 P3 0 0 0 ⎥ ⎥ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ −P1 0 0 ⎥ ⎥ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ −P1 0 ⎦ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ − τ1 P3

where Ω11 = P2 − Q2 D − DQT2 − Q3 − QT3 − F1 L + (1 + τ )λρ1 I, Ω12 = P1 − DQT1 − Q2 , Ω13 = Q2 A + F2 L, Ω22 = −Q1 − QT1 + τ P3 , Ω33 = h2 P4 − L, Ω55 = −Q4 − QT4 − F1 S + (1 + τ )λρ2 I. t Proof. Let y(t) = −Dx(t)+Af (x(t))+Bf (x(t−τ (t)))+C t−h(t) f (x(s))ds+u(t), α(t) = σ(t, x(t), x(t − τ (t))), then model (1) is rewritten as dx(t) = y(t)dt + α(t)dω(t).

(4)

Consider the following Lyapunov-Krasovskii functional as  V (t, x(t)) = x (t)P1 x(t) + T





0



 x (s)P2 x(s)ds +

0

T

t−τ t

−τ



t

y T (s)P3 y(s)ds

t+θ

trace[αT (s)P1 α(s)]dsdθ

+ −τ

t

t+θ t



t

+h t−h

f T (x(s))P4 f (x(s))dsdξ.

(5)

ξ

By Itˆ o differential rule and Lemma 1, the stochastic derivative of V (t) along the trajectory of model (4) can be obtained as  dV (t, x(t)) ≤ 2xT (t)P1 y(t) + trace[αT (t)P1 α(t)] + xT (t)P2 x(t)  t T T y T (s)P3 y(s)ds −x (t − τ )P2 x(t − τ ) + τ y (t)P3 y(t) −  +τ trace[αT (t)P1 α(t)] −

t−τ t

trace[αT (s)P1 α(s)]ds

t−τ

+h2 f T (x(t))P4 f (x(t))  t   − f T (x(s))ds P4 t−h(t)

t

f (x(s))ds

t−h(t)

+[x (t)P1 α(t) + α (t)P1 x(t)]dω(t). T

 dt

T

(6)

Passivity Analysis of Stochastic Neural Networks

627

From the definition of y(t), we have   0 = 2 y T (t)Q1 + xT (t)Q2 − y(t) − Dx(t) + Af (x(t))  t  f (x(s))ds + u(t) . +Bf (x(t − τ (t))) + C

(7)

t−h(t)

Integrating both sides of (4) from t − τ (t) to t, we get  t  t x(t) − x(t − τ (t)) − y(s)ds − α(s)dω(s) = 0. t−τ (t)

t−τ (t)

Hence,   −2xT (t)Q3 x(t) − x(t − τ (t)) −



t

y(s)ds −

t−τ (t)

 α(s)dω(s) = 0.

t t−τ (t)

By Lemma 2 and assumption (H1), we get 0 ≤ −2xT (t)Q3 x(t) + 2xT (t)Q3 x(t − τ (t)) + τ xT (t)Q3 P3−1 QT3 x(t)  t + y T (s)P3 y(s)ds + xT (t)Q3 P1−1 QT3 x(t) t−τ (t) t

 +

T   α(s)dω(s) P1

t−τ (t)

 α(s)dω(s) .

t

(8)

t−τ (t)

Similarly, integrating both sides of (4) from t − τ to t − τ (t), we have 0 ≤ −2xT (t − τ (t))Q4 x(t − τ (t)) + 2xT (t − τ (t))Q4 x(t − τ )  t−τ (t) −1 T T y T (s)P3 y(s)ds +τ x (t − τ (t))Q4 P3 Q4 x(t − τ (t)) + +xT (t)Q4 P1−1 QT4 x(t) +



t−τ

t−τ (t)

T   α(s)dω(s) P1

t−τ

t−τ (t)

 α(s)dω(s) (. 9)

t−τ

For two positive diagonal matrices L and S, we can get from assumption (H2) that [13] 

x(t) f (x(t))

T 

F1 L −F2 L −F2 L L



 x(t) ≤ 0. f (x(t))

(10)

 x(t − τ (t)) ≤ 0. f (x(t − τ (t)))

(11)

and 

x(t − τ (t)) f (x(t − τ (t)))

T 

F1 S −F2 S −F2 S S



By assumption (H3) and inequality (2), we get trace[αT (t)P1 α(t)] ≤ λ[ρ1 xT (t)x(t) + ρ2 xT (t − τ (t))x(t − τ (t))].

(12)

628

Q. He and Q. Song

From the proof of [14], we have E =E E =E

  

t−τ (t)

t−τ t−τ (t)

t−τ   t



T   α(s)dω(s) P1

 α(s)dω(s)

t−τ

 trace[αT (s)P1 α(s)]ds , T   α(s)dω(s) P1

t−τ (t) t

t−τ (t)

(13)  α(s)dω(s)

t

t−τ (t)

 trace[αT (s)P1 α(s)]ds .

(14)

t−τ (t)

Taking the mathematical expectation on both sides of (6), and noting (7) to (14), we get     E dV (t, x(t)) − 2f T (x(t))u(t)dt − γuT (t)u(t)dt ≤ E η T (t)Πη(t)dt , (15)   where η(t) = (ζ(t), ς(t))T with ζ(t) = xT (t), y T (t), f T (x(t)), f T (x(t − τ (t))) ,   t ς(t) = xT (t − τ (t)), xT (t − τ ), t−h(t) f T (x(s))ds, uT (t) , and ⎡

Π11 ⎢ ∗ ⎢ ⎢ ∗ ⎢ ⎢ ∗ Π=⎢ ⎢ ∗ ⎢ ⎢ ∗ ⎢ ⎣ ∗ ∗

⎤ Ω12 Ω13 Q2 B Q3 0 Q2 C Q2 Ω22 Q1 A Q1 B 0 0 Q1 C Q1 ⎥ ⎥ ∗ Ω33 0 0 0 0 −I ⎥ ⎥ ∗ ∗ −S F2 S 0 0 0 ⎥ ⎥ ∗ ∗ ∗ Π55 Q4 0 0 ⎥ ⎥ ∗ ∗ ∗ ∗ −P2 0 0 ⎥ ⎥ ∗ ∗ ∗ ∗ ∗ −P4 0 ⎦ ∗ ∗ ∗ ∗ ∗ ∗ −γI

with Π11 = P2 −Q2 D−DQT2 −Q3 −QT3 +τ Q3 P3−1 QT3 +Q3 P1−1 QT3 +Q4 P1−1 QT4 − F1 L + (1 + τ )λρ1 I, Π55 = −Q4 − QT4 + τ Q4 P3−1 QT4 − F1 S + (1 + τ )λρ2 I. It is easy to verify the equivalence of Π < 0 and Ω < 0 by using Lemma 3. Thus, one can derive from (3) and (15) that E{dV (t, x(t))} − E{2f T (x(t))u(t) + γuT (t)u(t)} ≤ 0. dt

(16)

From (16) and the definition of V (t, x(t)), we can get  2E{ 0

tp

 f T (x(s))u(s)ds} ≥ −γE{

tp

uT (s)u(s)ds}

0

From Definition 1, we know that the stochastic neural networks (1) is globally passive in the sense of expectation, and the proof of Theorem 1 is then completed.

Passivity Analysis of Stochastic Neural Networks

4

629

An Example

Consider a two-neuron neural network (1), where       0.2 0.3 −0.1 0.2 2.1 0 , , B= , A= D= 0.6 −0.8 −0.3 0.2 0 1.9

 C=

 −0.4 −0.5 , 0.1 −0.2

f1 (z) = tanh(−0.2z), f2 (z) = tanh(0.4z), τ (t) = 0.5| sin t|, h(t) = 0.1| cos t|, and σ satisfies

  trace σ T (t, x(t), x(t − τ (t)))σ(t, x(t), x(t − τ (t)))     ≤ 0.1 x21 (t) + x22 (t) + 0.4 x21 (t − τ (t)) + x22 (t − τ (t)) .

It can be verified that assumptions (H1) to (H3) are satisfied, and F1 = 0, F2 = diag{−0.1, 0.2}, ρ1 = 0.1, ρ2 = 0.4. By the Matlab LMI Control Toolbox, we find a solution to the LMIs in (2) and (3) as follows:     331.7740 0.7435 524.5060 −7.6839 P1 = , P2 = , 0.7435 337.1634 −7.6839 504.0683     258.4630 7.2578 502.8959 29.8210 P3 = , P4 = , 7.2578 284.5552 29.8210 531.6793     119.5541 −1.6105 266.8932 10.7373 , Q2 = , Q1 = 5.4725 132.6758 −4.2467 295.2921     23.7433 −3.9124 213.4065 1.4686 , Q4 = , Q3 = 10.1240 45.7818 1.4667 222.5017     543.0720 0 474.0971 0 , , S= L= 0 445.8546 0 407.7937 γ = 699.6234,

λ = 365.9399.

Therefore, by Theorem 1, we know that the considered model is passive in the sense of Definition 1. It should be pointed out that the conditions in [6]-[12] cannot be applied to this example since it requires the differentiability of the time-varying delay.

5

Conclusions

In this paper, the passivity has been investigated for a class of stochastic neural networks with discrete time-varying delay and distributed time-varying delay as well as generalized activation functions. A new delay-dependent criterion for checking the passivity of the addressed neural networks has been established. The obtained results generalize and improve the earlier publications, and remove the traditional assumptions on the differentiability of the time-varying delay and the boundedness of its derivative. An example has been provided to demonstrate the effectiveness and less conservatism of the proposed criterion.

630

Q. He and Q. Song

Acknowledgments This work was supported by the National Natural Science Foundation of China under Grants 60974132 and 10772152.

References 1. Yu, W.: Passivity Analysis for Dynamic Multilayer Neuro Identifier. IEEE Transactions on Circuits and Systems I 50, 173–178 (2003) 2. Niculescu, S.I., Lozano, R.: On The Passivity of Linear Delay Systems. IEEE Transactions on Automatic Control 46, 460–464 (2001) 3. Yaesh, I., Shaked, U.: Stochastic Passivity and Its Application in Adaptive Control. IEEE Transactions on Automatic Control 54, 136–142 (2009) 4. Mahmoud, M.S., Ismail, A.: Passivity and Passification of Time-Delay Systems. Journal of Mathematical Analysis and Applications 292, 247–258 (2004) 5. Gao, H.J., Chen, T.W., Chai, T.Y.: Passivity and Passification for Networked Control Systems. SIAM Journal on Control and Optimization 46, 1299–1322 (2007) 6. Li, C.G., Zhang, H.B., Liao, X.F.: Passivity and Passification of Fuzzy Systems with Time Delays. Computers and Mathematics with Applications 52, 1067–1078 (2006) 7. Li, C.G., Liao, X.F.: Passivity Analysis of Neural Networks with Time Delay. IEEE Transactions on Circuits and Systems II 52, 471–475 (2005) 8. Park, J.H.: Further Results on Passivity Analysis of Delayed Cellular Neural Networks. Chaos, Solitons & Fractals 34, 1546–1551 (2007) 9. Lou, X.Y., Cui, B.T.: Passivity Analysis of Integro-Differential Neural Networks with Time-varying Delays. Neurocomputing 70, 1071–1078 (2007) 10. Chen, B., Li, H.Y., Lin, C., Zhou, Q.: Passivity Analysis for Uncertain Neural Networks with Discrete and Distributed Time-Varying Delays. Physics Letters A 373, 1242–1248 (2009) 11. Zhang, Z.X., Mou, S.S., Lam, J., Gao, H.J.: New Passivity Criteria for Neural Networks with Time-Varying Delay. Neural Networks 22, 864–868 (2009) 12. Xu, S.Y., Zheng, W.X., Zou, Y.: Passivity Analysis of Neural Networks with TimeVarying Delays. IEEE Transactions on Circuits and Systems II 56, 325–329 (2009) 13. Liu, Y.R., Wang, Z.D., Liu, X.H.: Global Exponential Stability of Generalized Recurrent Neural Networks with Discrete and Distributed Delays. Neural Networks 19, 667–675 (2006) 14. Yu, J.J., Zhang, K.J., Fei, S.M.: Further Results on Mean Square Exponential Sstability of Uncertain Stochastic Delayed Neural Networks. Communications in Nonlinear Science and Numerical Simulation 14, 1582–1589 (2009)

A Novel Recovering Initial Conditions Method from Spatiotemporal Complex Dynamical System Longquan Dai1 , Xiaoyun Kang1 , and Minfen Shen1,2 1

Electronic Engineering Department of Shantou University 2 Institute of Light Equipment of Shantou [email protected]

Abstract. In this contribution, the inverse problem of spatiotemporal complex dynamical system is investigated on the basis of a typical diffusively coupled map lattices (CML) model. A novel method of recovering initial conditions for CML system is proposed by utilizing symbolic dynamics. It is shown that convergent property goes hand in hand with coupling strength. Both theoretical and experimental results show that the proposed algorithm performs well in noiseless case when the coupling strength is not very strong. The presented results also provide a theoretical and factual basis for better analysis and description of the spatiotemporal complex dynamical behaviors of the actual system. Keywords: Symbolic dynamics, Initial conditions recovering, Complex system.

1

Introduction

Phenomena with spatiotemporal complexity are common in nature as are observed in the fluid, chemical, optical, and solid-state turbulence, pattern formation, neural networks, parallel computation problems and so on. Here, we use CML as dynamical system model for the complexity[1]. The CML is a dynamical system with discrete time, discrete space, and continuous state[2]. This model connects low-dimensional chaotic units by the global coupling, diffusively coupling and so on to construct special high-dimensional chaotic system, namely, space-time chaotic system[3]. As an effective methodology for modeling spatiotemporal chaos phenomena, the CML model has been revealed by Wang et.al that it is competent to reproduce the essential features of complex natural systems[4][5]. Fortunately, the complexity of understanding and studying the spatiotemporal chaotic phenomena has been greatly reduced at the mercy of the powerful CML model. A further simplification can be achieved by considering iterates of the resulting map through a partition which reduces the chaotic motion to a purely symbolic signal with associated transition rules. The discipline to handle such signals is the so-called symbolic dynamics[6]. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 631–638. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com 

632

L. Dai, X. Kang, and M. Shen

Symbolic dynamics theory as a possible way to comprehend space-time chaos has attracted a significant interest from variety research community, especially in the society of using symbolic sequence to solve inverse problem of complex dynamical system[7][8][9]. Once the initial condition is available from purely symbolic sequences, we can obtain more information embedded in the complex systems,such as the global orbits.Various symbolic dynamical methods have been proposed for the recovery of initial values from CML system. Zeng et.al[7] recovered the statistical characteristics of the initial conditions under the condition of overlooking the site interaction. Following the way of Zeng, Lin et.al[8] further exploited time-varying coefficients to recover the statistical properties of initial values from weak coupling system. Unfortunately, these methods can only restore the statistical properties of the initial value in the entire CML system. In order to obtain the initial value in every site, Wang et.al[9] roughly investigated the inverse problem of local CML system. But the convergence and divergence (CD), in backward iteration, influenced by the coupling coefficients has not been taken into account seriously. In this contribution, we will exhibit, from theoretical point, the mathematical quantity relation between the CD and the coupling strength in the backward iteration of diffusively CML system. As a typical example, we will elaborate the performance of recovering the initial condition of Logistic coupling lattices with random system size. In fact, both theoretical and factual results indicate that any point in CML do not necessarily converge to its initial condition. The reason is the coupling strength is critical to the CD in the inverse iteration process. Therefore, we introduce an efficient and effective method in this paper for analyzing physical process in practical complex dynamics system.

2

Model and Method

Let us consider a typical diffusively coupled map lattices with L sites labeled. Each site is described by a state xin in the interval I = [a, b]. The CML are introduced in (1) with open boundary conditions expressed as (2). ε i−1 xin+1 = (1 − ε)f (xin ) + [f (xi+1 n ) + f (xn )] 2

(1)

L−1 ) + (1 − ε)f (xL x1n+1 = (1 − ε)f (x1n ) + εf (x2n ), xL n+1 = εf (xn n)

(2)

where n is a discrete time step and i (i = 1, 2, ..., L) is a lattice point; ε represents coupling coefficient and f is constructed by either unimodal maps or piecewise linear/nonlinear maps. Here, Logistic Map which is discribed in (3) is chosen for comprehensible consideration. f (x) = 4x(1 − x)

0≤x≤1

(3)

Let P = {P0 , P1 , ..., Pm−1 } be a finite disjoint partition of the phase space I, i.e. m−1 m−1 j=0 Pj = φ and j=0 Pj = I. the time series S = {0, 1, ..., m − 1} is defined as a symbol set of the given system. Consequently, if data point x(n) at time

A Novel Recovering Initial Conditions Method

633

n of the system trajectory is the jth element of the partition, we can assign a symbol s(n) = j, j ∈ S by following partition rules. Then (1) produces the T symbolic vector sn = [s1n , s2n , ..., sL n ] at nth iteration and the symbolic vector sequence {s0 , s1 , ..., sn , ...} from 0th iteration, which describes the orbits of the total complex system at different times. For a CML system with L(L ≥ 2) lattice points, (1) could be rewrite at the T 1 2 L T term of matrix. Put xn = [x1n , x2n , ..., xL n ] , f (xn ) = [f (xn ), f (xn ), ..., f (xn )] , then xn+1 = A × f (xn ) denote (1) concisely, where A represents the coupling matrix of the CML system, mainly determined by the system size L and coupling coefficient. ⎤ ⎡ 1−ε ε 0 ... 0 0 ⎢ ε/2 1 − ε ε/2 ... 0 0 ⎥ ⎥ ⎢ ⎢ 0 ε/2 1 − ε ... 0 0 ⎥ ⎥ ⎢ A=⎢ . .. .. ⎥ .. .. .. ⎢ .. . . . . . ⎥ ⎥ ⎢ ⎣ 0 0 0 ε/2 1 − ε ε/2 ⎦ 0 0 0 ... ε 1 − ε L×L Given A is a nonsingular matrix, (1) is reformed as f (xn ) = A−1 xn+1 . For symbol sin is known, xn = fs−1 (A−1 xn+1 ) = fs−1 ◦ A−1 (xn+1 ) = Hs−1 (xn+1 ) n n n

(4)

with Hs−1 = fs−1 ◦ A−1 : I L → I L . n n Theoretically speaking, we could recover the initial condition in every site of coupled map lattices by continually iterating (4). However, the error caused by the quantization of AD transmitter and the floating point accuracy of computer (i.e finite precision effects) should be taken serious account as well as the inevitably noisy affection in the backward iterations. If the computation is a contracting mapping, we could ignore the error confidently and safely and then use appropriate algorithm to recover initial conditions successfully with respect to sufficient backward iterations. In practice, when the divergence dominates the overall iterations, thanks to (4) is extreme sensitivity to various deviations, in this case (4) could not be used to estimate the initial conditions as a candidate method.

3

The CD Research

In order to dive into the CD of Hs−1 , it is necessary to investigate the CD of n coupling coefficient matrix and Logistic Mapping function respectively. 3.1

The CD of Inverse Matrix of Coupling Coefficient Matrix

T 1 2 L T For any xn , yn ∈ I L , where xn = [x1n , x2n , ..., xL n ] , yn = [yn , yn , ..., yn ] , define



d(x, y) = xn − yn ∞ = max xin − yni

(5) 1≤i≤L

634

L. Dai, X. Kang, and M. Shen

L   when |A| = 0, let A−1 = (aij )L×L , define that A−1 ∞ = max ( |aij |), let 1≤i≤L j=1

T  −1 yn = [βn1 , βn2 , ..., βnL ]T , then x n = A−1 xn = [α1n , α2n , ..., αL n ] ,y n = A   −1  d(x n , y n ) = A−1 xn − A

yn ∞



L





L

= max

aij (xjn − ynj ) ≤ ( max |aij |). max xjn − ynj

1≤i≤L j=1 1≤i≤L j=1 1≤j≤L   =A−1 ∞ d(xn , yn )

(6)

meanwhile, according to the definition of matrix norm and the construction of coupling matrix, then A∞ = 1, d(xn , yn ) = d(Ax n , Ay n ) = Ax n − Ay n ∞ ≤ A∞ x n − y n ∞ = d(x n , y n ) It implies that

d(A−1 xn , A−1 yn ) ≥ d(xn , yn )

(7) 



According to (6) and (7), we obtain that d(xn , yn ) ≤ d(x    n , y n ) ≤ A−1  d(xn , yn ) What’s more, because of A = 1, 1 = AA−1  ≤ ∞ ∞ ∞   −1  A  ≥ 1. This can be confirmed by Figure 1. It A∞ A−1 ∞ ,  therefore ∞  also shows that A−1 ∞ slowly increases as the site coupling becomes stronger between 0 and 0.3.

  Fig. 1. The relationship between A−1 ∞ and ε under different system size L

A Novel Recovering Initial Conditions Method

635

According to the contraction mapping theorem, we know that A−1 is not a contraction map, but a divergent map. 3.2

The CD of Inverse Mapping of Logistic Mapping

The unimode chaotic Logistic Map function is expressed as (3), whose inverse map is given[10] by √ fs−1 (xn ) = [1 + (2sn − 1) 1 − xn ]/2 (8) n

0 xn ≤ 0.5 where sn = 1 xn > 0.5. (xn ) = xn , so two fixed Guarantied by definition, a fixed point must satisfy fs−1 n points of (7) are x = 0 and x = 0.75 respectively. According to the definition of Lyapunov exponent, we deduce that the Lyapunov exponents of these two orbits of fs−1 are −0.6933 and −1.3857 which are negative. This indicates that the n inverse system is in the stable periodic state, not chaotic. Therefore, the inverse map of Logistic Map is a convergent map in the interval [0, 1]. Figure 2 shows the result we discussed above.

Fig. 2. The inverse map of Logistic Map and the backward iteration from different initial point

3.3

CD of the Inverse System of CML System

as a Logistic lattices is In what follows, The CD of the inverse system Hs−1 n investigated.

636

L. Dai, X. Kang, and M. Shen

d(Hs−1 (x n ), Hs−1 (yn )) = d(fs−1 (x n ), fs−1 (y n )) n n n

n 



= max( 1 − αin − 1 − βni /2) (9) 1≤i≤L

   −1 



≤ max [1/(2 1 − αin + 1 − βni )] × A ∞ d(xn , yn ) 1≤i≤L

 



Let γ = max [1/(2 1 − αin − 1 − βni )] , which changes as the site states 1≤i≤L

 



vary at different time step. Assume that when i = p, (1/(2 1 − αin − 1 − βni ))

√





achieves its maximum value, i.e. γ = 1/(2 1 − αpn − 1 − βnp ). The analysis discussed above implies that the inversemap of Logistic Map is a convergent A−1  ≥ 1. For 0 ≤ αpn < 1 and map, and A−1 is divergent. Moreover,  ∞  0 ≤ βnp < 1, then γ ≥ 1/4. Thence γ A−1 ∞ ≥ (1/4). When the coupling is weak interaction, the value of ||A−1 ||∞ is relatively small and gradually raise up with the increasing of coupling Figure 1). Therefore, the   coefficient(seen from  probability P γ||A−1 ||∞ ≥ 1 is far less than P γ||A−1 ||∞ < 1 for the same state value in the phase space. In this case, convergence occupies the majority part of the total inverse iteration processing (8).  xn = Hs−1 (xn+1 ) = fs−1 ◦ A−1 (xn+1 )= [1 + (2sn − 1) 1 − A−1 (xn+1 )]/2 (10) n n Using (8), we could recover the initial conditions of the complex system by iterating   it infinitely. As the coupling coefficients swell constantly, the value of A−1  increases more quickly. Thereby the strength of divergence will sup∞ press the power of convergence finally and break the balance point between them, eventually characterize the behavior of the system alone. Even our novel algorithm presented in this contribution will fail to recover the initial conditions under strong coupling condition. The problem of strong coupling is still puzzling and open to discuss.

4

Simulation and Result Analysis

Assumed that the estimated values x ˆn = [ˆ x1n , x ˆ2n , . . . , x ˆL n ] 0 ≤ n ≤ N , where N denotes the number of total iterates at nth time have been given by iterating (10).The variance of the estimator can be defined reasonably as: L 2 log10 (M SE(ˆ xn )) = log10 (1/L × (ˆ xin − xin ) ) (11) i=1

Typical simulation results of the convergent rate of the new method are depicted in Figure 3 for the Logistic Map based on CML. The coupling coefficient changes continuously from 0 to 0.25, the total number of lattice point is L = 20 and the initial value is selected randomly in the interval [0, 1] to improve the universality. The number of total iterations is 20 (i.e.N = 20).As shown in Figure 3, the presented method successfully recovers the initial conditions of the complex dynamics system as the iterating length becomes larger in noise free environment. What’s more, as the coupling coefficient increases, the convergent rate of recovering the initial conditions becomes smaller.

A Novel Recovering Initial Conditions Method

637

Fig. 3. Convergence rate of the initial condition recovering at different coupling coefficient

5

Conclusion

In this contribution, we exploit the symbolic vector dynamics method to recover the initial condition of spatiotemporal complex dynamics system in noiseless environment. It has been well proved that the points in the state space do not converge to their initial condition all the time with respect to sufficient backward iterations, which is directly relevant to lattice point interaction strength. According to our theoretical analysis and experimental result, we found that coupling strength plays an important role in convergence rate of the inverse process. When it is very strong, this algorithm cannot be expected to recover initial condition. However, if it becomes weaker, the introduced method in this paper is able to perform better. The final results prove that this algorithm enables us to recover initial condition of complex dynamics system in noise free case as iteration length becomes larger, when the coupling strength is not too strong. It should be noticed that there is noise inevitably in practical physical system, therefore further search on this algorithm in this aspect will be made in progress in future.

References 1. Kunihiko, K.: Lyapunov analysis and information flow in coupled map lattices. Phys. D 23, 436–447 (1986) 2. Yang, W.M.: Spatiotemporal Chaos and Coupled Map Lattice, Shanghai, China (1994) 3. Zhang, J.S., Dang, J.L., Li, H.C.: Local support vector machine prediction of spatiotemporal chaotic time series. Acta Phys. Sin. 56, 67–77 (2007)

638

L. Dai, X. Kang, and M. Shen

4. Wang, Z.B., Hu, G.: Spatiotemporal Periodic Patterns of A Two-Dimensional Symmetrically Coupled Map Lattices. Acta Phys. Sin. 50, 1666–1669 (2001) 5. Antˆ onio, M.B., Ricardo, L.V.: Lyapunov exponents of a lattice of chaotic maps with a power-law coupling. Phys. Lett. A 286, 134–140 (2001) 6. Zheng, W.M., Hao, B.L.: Applied Symbolic Dynamics, Shanghai, China (1994) 7. Zeng, Y.C., Tong, Q.Y.: A statistical property to recover coarsely initial conditions from coupled map lattices. Acta Phys. Sin. 52, 285–291 (2003) 8. Lin, Y., Shen, M.F., Chen, H.Y.: Recovery of statistical property of initial conditions based on time-varying parameter from coupled map lattices. Acta Phys. Sin. 55, 564–571 (2006) 9. Wang, K., Pei, W.J., Xia, H.S., He, Z.Y.: Initial condition estimate from coupled map lattices based on symbolic vector dynamics. Acta Phys. Sin. 56, 130–134 (2007) 10. Ling, C., Wu, X.F., Sun, S.G.: A general efficient method for chaotic signal estimation. IEEE Trans. Signal Processing 47, 1424–1428 (1999)

An Intelligent Control Scheme for Nonlinear Time-Varying Systems with Time Delay Sun Zhou1, Guoli Ji1,*, Wei Lin1, and Zijiang Yang2 1

Department of Automation, Xiamen University, 361005 Xiamen, China [email protected] 2 School of Information Technology, York University, Toronto, Canada M3J 1P3 [email protected]

Abstract. A control scheme was designed for nonlinear time-varying systems with time delay, based on time-delay compensating theory and fuzzy logic. It consists of a Smith neural-network predictor and a modified parameter-selftuning fuzzy controller, given ACR (Acrylate Copolymer Resin) polymerizingkettle as the control plant. In the experiment, we verified the performance of the control system in two scenarios: one with invariant parameters and the other with time-varying parameters. Experimental results illustrate the effectiveness of the proposed scheme. Moreover, the comparison to other three typical control methods is also presented, which demonstrates that the proposed control scheme has satisfying dynamic performances, and when the system parameters varied with time it can still control stably with good robustness. Keywords: Neural network, Nonlinear, Polymerizing-kettle, Fuzzy control.

1 Introduction In many industrial productions, the controlled plant often has pure time delay. It is with very complex uncertainty or the parameters vary with time. Therefore, it is very difficult to establish the accurate analytical model to design the controller. In order to find a control approach for nonlinear time-varying systems with time delay researchers presented some typical methods including Smith-PID control [1], Smithfuzzy control [2] and PID control with neural-network based Smith compensator[3], just mention a few. However, some of these methods cannot adapt to situations when the system parameters vary with time, as often happen in actual production process and some are hard to be realized physically. We design an effective control scheme that has an intelligent predictor, taking advantage of both artificial neural network and fuzzy control. Fuzzy control does not need the accurate mathematical model. Moreover, it is insensitive to the variation of parameters and has good robustness. It is also easy to realize a fuzzy controller [4]. As for the artificial neural network, it can approximate any nonlinear systems and is good at fault tolerance. Based on these advantages, a control scheme based on Smith neural-network predictor and fuzzy logic controller is designed. We name it “Smith *

Corresponding author.

Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 639–646. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

640

S. Zhou et al.

intelligent control”. On the one hand, a Smith Neural-Network Predictor (SNNP) is designed to identify the controlled plant offline and online. On the other hand, the plant was controlled by a parameter-self-tuning fuzzy controller. Through this approach, the influence brought by the uncertainty of the plant will be conquered and the system robustness is improved. Furthermore, the dynamic performance will be enhanced. The structure of Smith intelligent control system we design is shown as Figure 1, which consists of Smith neural-network predictor and a fuzzy controller. The predictor is designed to predict and compensate the time-delay, and the fuzzy controller is used to control the large-inertia plant. The two parts will be discussed in Section 2 and 3.

Fig. 1. Structure of Smith intelligent control system

In our study, an Acrylate Copolymer Resin (ACR) polymerizing-kettle system is taken as the controlled plant, which is a nonlinear, time-varying system with time delay. The temperature of the emulsion in the kettle should be controlled by adjusting the flow of the condensate water. We linearized the nonlinear model of polymerizingkettles and got a second-order linear differential equation. The corresponding transfer function is

G(s) =

− C 4 e −τs Δθ ( s ) = 2 ΔWCWS ( s) C1C 3 s + (C1 − C 2 C 3 ) s − (C 2 + 1)

(1)

where the controlled variable Δθ represents how much the current temperature in the kettle deviates from the target value, the control variable ΔWCWS represents deviation of the flow of the condensate water, and C1, C2, C3, C4 are all functions about parameters contained in the original nonlinear model, which can be regarded as timevarying parameters. (1) is a second-order time-varying system with time delay.

2 Smith Neural-Network Predictor (SNNP) Smith prediction control is regarded as the most effective approach to solve the problem of how to control a process with large time-delay [5]. However, the conventional Smith predictor is not suitable for time-varying systems like polymerizing-kettles. Sometimes

An Intelligent Control Scheme for Nonlinear Time-Varying Systems with Time Delay

641

it even makes the system oscillating. In this paper, a Smith neural-network predictor is proposed, combining Smith prediction theory and the learning function of neural network. It uses neural network to estimate the parameters of Smith predictor online and modify them at real-time. Since multilayer back propagation (BP) neural network can approximate any function [6], we design a predictor made up of multilayer BP neural network SNNP1 and SNNP2 to approximate the controlled plant. By offline and online learning it will adapt to the change of the controlled plant continuously so that a good identification and control effect can be achieved. As shown in Figure 1, the BP network SNNP1 identifies the compensation part without time-delay: G (s ) , and SNNP2 identifies the compensation part with timedelay: G ( s ) ⋅ e −τ ⋅s . According to system identification theory [7], their mapping relationships are as below respectively:

y1 (t + 1) = a0 ⋅ u (t ) + " am ⋅ u (t − m) + b0 ⋅ y1 (t ) + " bn ⋅ y1 (t − n) y2 (t + 1) = a0 ⋅ u (t − τ ) + " am ⋅ u (t − τ − m) + b0 ⋅ y2 (t ) + " bn ⋅ y2 (t − n)

(2) (3)

Formula (2) and (3) are the relationships of the input and output of SNNP1 model that is without time-delay and SNNP2 model that is with time-delay respectively. Since the difference between the two SNNPs is just that the output of SNNP2 has a compared with that of SNNP1, we use SNNP1 as an example. As shown in lag Figure 2, it is implemented by 3-layer BP feedforward network. The nonlinear transmission function of the hidden layer is Tag-Sigmoid function. For the output layer, the function is a linear one. The numbers of nodes of each layer are N1=3, N2=11, N3=1.

τ

Fig. 2. Structure of SNNP neural network

When the network is forward propagating, the outputs of each layer are as follows: net1 j − 1 ; the output layer; the input layer: o1i = xi ; the hidden layer: o2 j = e net e 1 j +1

642

S. Zhou et al. M

M

o3o = y1 (t + 1) = net2 j ; the error index: E = ∑ E p = 1 ⋅ ∑ [ y (t + 1) − y1 (t + 1)]2 , p

2

i =1

n

where netij is the network value of the j neuron of the i layer, net1 j = ∑ w1 ⋅o1i , ij th

th

i =1

n

th net2 j = ∑ wij2 ⋅o2i ; oi is the output of the i neuron; Ep is the error of the network’s i =1

output for the pth sample; E is the sum of errors of the network’s outputs for all samples; M is the number of training samples. Regarding the learning algorithm of the network, for the control of polymerizingkettles’ temperature, first SNNP network trains its parameters offline using Levenberg-Marquardt method [8]. Thus, a proper neural network could be quickly created to simulate the controlled plant. Then during its online learning, according to the extended Delta rule which is an BP algorithm with varying learning rate, SNNP modifies the network’s weights at real-time based on the principle that the error index should decrease according to the gradient of variation of the network parameters. Therefore, the network can adapt to the change of the controlled plant and accelerate the learning process.

3 Parameter-Self-tuning Fuzzy Controller As mentioned above, we have obtained the model of polymerizing-kettles. However, general control methods need very accurate model to achieve good control effect. Nevertheless, as our knowledge about polyreaction process still remain inadequate, the accurate analytical model and the laws of how parameters of polymerizing-kettles vary are hard to acquire. While fuzzy control does not demand an accurate analytical model, good control effect may be achieved just depending on experiences of experts [9]. Thus, we adopt fuzzy controller in our control scheme. 3.1 Design of Fuzzy Controller For a polymerizing-kettle which is a second order plant, we had the deviation of temperature in the kettle (i.e. Δθ in formula (1)) – E and the deviation’s change rate – C as the input linguistic variables of the fuzzy controller, and the deviation of the flow of the condensate water (i.e. ΔWCWS in formula (1))– U as the output linguistic variable. Consequently a two-input, single-output fuzzy controller can be designed. The basic domains of the three fuzzy variables, temperature deviation E, change rate of temperature deviation C and output variable U are [-7, +7], [-1,+1], and [35,+35], respectively. The corresponding domains of fuzzy subsets are E = {-11, -10, …, -1, 0, 1, …, 10, 11}, C = {-11, -10, …, -1, 0, 1, …, 10, 11} and U = {-10, …, -1, 0, 1, …, 10}. The fuzzy subsets of the three variables are {NB, NM, NS, NO, PO, PS, PM, PB}. The fuzzy control rules are based on Mamdani control rules and operators’

An Intelligent Control Scheme for Nonlinear Time-Varying Systems with Time Delay

643

experience. After our simulation test and optimization, 8*8=64 rules were finally determined. 3.2 Self-tuning of the Scale Factor The scale factor Ku of the output plays an important role in fuzzy control. As the overall gain of a fuzzy controller, it affects the system performance directly. To meet the demand of quick system response, Ku should be big; while to get a good precision of steady state, a small Ku is necessary. However, for a typical fuzzy controller, this parameter is a constant. When the controlled plant is nonlinear with large time delay and time-varying parameters, and sometimes with heavy random disturbance as well, this kind of controller is hard to satisfy both dynamic and static property. In order to solve this contradiction, we make the scale factor Ku self-tuning. The ⎧k u1 E > e1 , C > c1 ⎪ ⎪k u 2 e1 ≥ E > e2 , c1 ≥ C > c 2 where ku1, ku2, self-tuning rule we designed is Ku = ⎨ ⎪k u 3 E ≤ e2 , C ≤ c 2 ⎪k u 4 y − yˆ > d ⎩ ku3, and ku4 are different values of Ku under different system states; e1 and e2 are the threshold of error E and e1>e2; c1 and c2 are threshold of error’s change rate C and c1>c2. Besides, y − yˆ is the difference between the output of the controlled plant and the output of SNNP and d is a constant. A set of parameters adopted in our simulation are (ku1, ku2, ku3, ku4) = (10, 60, 4.28, 3.28); (e1, e2) = (0.8, 0.1); (c1, c2) = (0.005, 0.002), and d = 0.001.

4 Simulation We examined the performance of the system in two scenarios: one with invariant parameters and the other with time-varying parameters. Moreover, not only Smith intelligent control scheme proposed here but also other three typical control methods, including Smith-PID control, Smith-fuzzy control and PID control with neuralnetwork based Smith compensator (called “SNN-PID control”), are tested in the simulation experiment in order to compare the effect. A set of parameter values that came from the real ACR polymerization production of Xiamen Electrochemical Factory were used to substitute into formula (1), and then − e−200 s , which we got a concrete system transfer function: G ( s) = 29403s 2 + 144.4s − 0.0375 is used to represent the controlled plant in our simulation. The control system was designed based on Figure 1. The stability and dynamic performance of the system was verified by the system response to a step input whose amplitude was 1. 4.1 Simulation with Invariant System Parameters Assuming that the parameters of polymerizing-kettle system are invariant, we took simulation tests to examine the dynamic performance of each control method. The

644

S. Zhou et al.

result simulation curves of the four control methods are shown in Figure 3. It is obvious that all the outputs of these control methods can finally get stable at the set value. However, their dynamic performances are different. The overshoot of Smith intelligent control scheme is just 1.25%, far smaller than those of the other three control methods. Its rise time and transition time are also short, which are just next to that of Smith-PID control.

Fig. 3. Simulation curves of four control methods when the system parameters are invariant

4.2 Simulation with Time-Varying System Parameters In real ACR production process, the parameters of polymerizing-kettle system vary with time. Hence, in the simulation experiment we make an assumption that the system parameters change every certain time. The experiment with time-varying parameters aimed to examine the robustness of each control scheme. During the simulation, the coefficient matrix A and C of the controlled-plant’s state equation varied as sine function every 500 seconds. The result simulation curves produced by four control methods are shown in Figure 4. Obviously, compared with the other control methods, when the plant’s parameters vary, the Smith intelligent control system will still have a stable control. Moreover, the effect of Smith intelligent control does not depend on the initial values of the control parameters of the system. As a result, it has a better adaptability than SNN-PID control and SmithPID control to some extent.

An Intelligent Control Scheme for Nonlinear Time-Varying Systems with Time Delay

645

Fig. 4. Simulation curves of four control methods when the system parameters varied with time

5 Conclusion An intelligent control scheme for time-varying system with time delay was proposed in this paper. According to the compensation theory of Smith predictor, we designed a Smith neural-network predictor that was made up of two BP neural networks to identify the controlled plant, and continuously identified the time-varying parameters by offline and online learning. In addition, an improved parameter-self-tuning fuzzy controller was adopted. The results of simulation experiments were presented to demonstrate that the control scheme proposed in this paper had obvious advantages in solving control problem of time-varying system with large time delay. Compared with the other three typical control methods, Smith intelligent control scheme achieved satisfying dynamic performance. Especially, due to the unique online learning function, the system showed good robustness when the plant’s property changed. Moreover, it is fairly easy for Smith intelligent control to be realized physically, and the advantages give good application prospect.

Acknowledgements Thanks are due to the support by the Fujian Natural Science Foundation (2009J01303), the Key Science & Technology Planning Project of Fujian Province (2009H0044) and Xiamen University national 211 3rd period project (0630-E62000). The project is also partially supported by Natural Sciences and Engineering Research Council of Canada.

646

S. Zhou et al.

References 1. Shi, D.N., Peng, G., Li, T.F.: Gray predictive adaptive Smith-PID control and its application. In: Proceedings of 2008 International Conference on Machine Learning and Cybernetics, vol. 1-7, pp. 1980–1984 (2008) 2. Zhong, L., Yuan, F., Ding, Z.L.: High precision temperature regulation with the intelligent control. In: Proceedings of the 3rd International Symposium on Instrumentation Science and Technology, vol. 3, pp. 1179–1183 (2004) 3. Wang, Y.Q., Sun, F., Liu, J., Sun, M.H., Xie, Y.H.: Application of Smith Predictor Based on Single Neural Network in Cold Rolling Shape Control. Chinese Journal of Mechanical Engineering 22(2), 282–286 (2009) 4. Kickert, W.J.M., Van Lautc Lemke, H.R.: Application of a fuzzy controller in warm water plant. Automatica 12, 301–308 (1976) 5. Smith, O.H.: Closed control of loops with dead times. Chem. Eng. Prog. 53(6), 217–219 (1957) 6. Hassan-Beygi, S.R., Ghobadian, B., Chayjan, R.A., Kianmehr, M.H.: Prediction of Power Tiller Noise Levels Using a Back Propagation Algorithm. Journal of Agricultural Science and Technology 11(2), 147–160 (2009) 7. Forssell, U., Ljung, L.: Closed-loop identification revisited. Automatica 35(7), 1215–1241 (1999) 8. Cao, J., Novstrup, K.A., Goyal, A., Midkiff, S.R., Caruthers, J.M.: A Parallel LevenbergMarquardt Algorithm. In: Proceedings of the 2009 ACM SIGARCH International Conference on Supercomputing, pp. 450–459 (2009) 9. King, P.J., Mamdani, E.H.: The application of fuzzy control system to industrial processes. Automatica 13, 235–242 (1977)

Master-Slave Chaos Synchronization of Uncertain Nonlinear Gyros Using Wavelet Neural Network Chien-Jung Chiu1, Chun-Fei Hsu2, Tsu-Tian Lee3, and Jang-Zern Tsai1 1

Department of Electrical Engineering, National Central University, Jung-Li 320, Taiwan, Republic of China [email protected], [email protected] 2 Department of Electrical Engineering, Chung Hua University, Hsinchu, 300, Taiwan, Republic of China [email protected] 3 Department of Electrical Engineering, National Taipei University of Technology, Taipei 106, Taiwan, Republic of China [email protected]

Abstract. In this paper, an adaptive wavelet neural network controller (AWNNC) is proposed to synchronize two nonlinear identical chaotic gyros. The proposed AWNNC system is composed of a neural controller and a compensation controller. The neural controller uses a wavelet neural network to online approximate an ideal controller, and the compensation controller is used to guarantee system stable based on Lyapunov function candidate. Some simulation results verify the chaotic behavior of two nonlinear identical chaotic gyros can be synchronized by the proposed AWNNC scheme. Keywords: Adaptive control, Sliding-mode control, Wavelet neural network, Chaos synchronization, Lyapunov function.

1 Introduction Gyro dynamics is considered to be one of the most important problems and has been studied by many authors. The gyro has attributes of great utility to navigational, aeronautical and space engineering [1]. In the past years, the gyros have been found with rich phenomena and give benefit for understanding of gyro systems. Different types of gyros with linear/nonlinear damping are investigated for predicting the dynamic response such as periodic and chaotic motions [1]. Chaos synchronization is an important problem in the nonlinear science. Up to now, many types of chaos synchronization have been proposed [2-4]. Some of them can not achieve favorable control performance; some of them require overly complex design procedures; and some of them require heavy computation loading. If the exact model of the controlled system is well known, there exists an ideal controller to achieve favorable control performance. Since the system parameters and the external load disturbance may be unknown or perturbed, the ideal controller can not be implemented [5]. To tackle this problem, many intelligent control schemes Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 647–654. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

648

C.-J. Chiu et al.

such as adaptive neural network control [6-8] have been developed. The most useful property is their ability of neural networks can approximate arbitrary linear or nonlinear mapping through learning [6]. To achieve better learning performance, some researchers have developed the structure of neural network based on the wavelet functions to construct the wavelet neural network (WNN) [9, 10]. Unlike the sigmoidal functions used in conventional neural networks, wavelet functions are spatially localized, so that the learning capability of WNN is more efficient than the conventional sigmoidal function neural network for system identification. The training algorithms for WNN typically converge in a smaller number of iterations than for the conventional neural networks [9]. There has been considerable interest in exploring the applications of WNN to deal with nonlinearity and uncertainties of realtime servo control system [11-15]. In this paper, an adaptive wavelet neural network controller (AWNNC) is proposed to synchronize two nonlinear identical chaotic gyros. The proposed AWNNC system is composed of a neural controller and a compensation controller. The neural controller uses a wavelet neural network to online approximate an ideal controller, and the compensation controller is used to guarantee system stable. Finally, some simulations are provided to verify the proposed AWNNC scheme drives the slave gyro system to synchronize the master gyro system.

2 Problem Statement In this paper, a symmetric gyro with linear-plus-cubic damping is considered. A symmetric gyro with linear-plus-cubic damping is given as [1]

T  D 2

(1  cosT ) 2  E sin T  c1T  c2T 3 sin 3 T

f sin Zt sin T

(1)

where θ is the angle; f sin ωt is the parametric excitation; c1θ and c2θ 3 are the

(1 − cosθ ) 2 − β sin θ is a nonlinear sin 3 θ resilience force. The open-loop system behavior was simulated with α 2 = 100 , β = 1 , linear and nonlinear damping, respectively; and α 2

c1 = 0.5 , c2 = 0.05 and ω = 2 for observing the chaotic unpredictable behavior. Consider two coupled master-slave chaotic gyros systems in a nominal form as Master system: x = g ( x, x) where g ( x, x )

f x sin Zt sin x  D 2

(2)

(1  cos x) 2  E sin x  c1 x  c2 x 3 . sin 3 x

Slave system:

y = g ( y, y) + u + F ( x, y )

(3)

Master-Slave Chaos Synchronization of Uncertain Nonlinear Gyros

649

(1  cos y) 2  E sin y  c1 y  c2 y 3 , u is the control 3 sin y input and F ( x, y ) is the coupling term. The control objective of the two coupled chaotic gyros system is that for different initial conditions of master and slaver systems, and different coupling term in slave system, the two coupled system, i.e. the master system (2) and the slave system (3), are synchronized by designing an appropriate signal control input u which is attached to the second equation of the slave system (3) such that

where g ( y, y )

f y sin Zt sin y  D 2

lim x− y =0 t →∞

(4)

To achieve this control goal, the tracking error can be defined as

e= x− y

(5)

.

From equation (2), (3) and (5), it can be obtained as

e = g ( x, x ) − g ( y , y ) − u − F ( x, y )

.

(6)

If the system dynamics are well known, there exits an ideal controller as [5] u * = g ( x, x) − g ( y, y ) − F ( x, y ) + k1e + k 2 e

(7)

where the k1 and k 2 are the non-zero constants. Apply the ideal controller (7) into error dynamic equation (6), it obtains that e + k1e + k 2 e = 0

(8)

.

If k1 and k 2 are chosen to correspond to the coefficients of a Hurwitz polynomial, that is a polynomial whose roots lie strictly in the open left half of the complex plane, then it implies that lim e = 0 [5]. t →∞

3 Description of WNN The WNN output with m wavelet basis functions as shown in Fig. 1 can perform the mapping according to [11] Ξ=

m

∑α Θ (σ , (z − c )) j

j

j

j

(9)

j =1

where z = [ z1 z 2 … z n ]T is the input vector, Θ j (σ j , (z − c j )) , j = 1,2,…, m are the wavelet functions, σ j = [σ 1 j σ 2 j … σ nj ]T and c j = [c1 j c2 j … cnj ]T are the dilation and

translation parameters, respectively, α j is the output layer weight. Each wavelet network’s neuron in the translation layer can be represented by Θ j = h j (z ) exp(−

n

∑σ k =1

2 kj

( z k − ckj ) 2 / 2)

(10)

650

C.-J. Chiu et al.

where the “Mexican hat” mother wavelet function is defined as h j (z ) =

n

∏ (1 − w z ) . 2 k

2 k

k =1

For ease of notation, (9) can be expressed in a compact vector form as

Ξ = α T Θ( z , σ , c )

(11)

where α = [α 1 α 2 ..α m ]T , Θ = [Θ1 Θ 2 ..Θ m ]T , σ = [σ1 σ 2 ..σ m ]T and c = [c1 c 2 ..c m ]T . By the universal approximation theorem, there exists an ideal WNN identifier such that [11]

Ω = α *T Θ* (z, σ * , c * ) + Δ

(12)

where α * and Θ* are optimal parameter vectors of α and Θ , respectively; σ * and c* are optimal parameter vectors of σ and c , respectively; and Δ is the approximation error. However, the optimal parameter vectors are unknown, so it is necessary to estimate the values. Define an estimative function

ˆ (z, σˆ , cˆ ) Ξˆ = αˆ T Θ

(13)

ˆ are optimal parameter vectors of α and Θ , respectively; and σˆ and where αˆ and Θ cˆ are optimal parameter vectors of σ and c , respectively. Define the estimation error as ~ ~ ~T ˆ ~ ~T Θ ˆ +Δ =α Ξ = Ω − Ξˆ = α*T Θ* − αˆ T Θ + αˆ T Θ + α Θ+Δ (14) ~ ~ = α * − αˆ and Θ ˆ . The Taylor expansion linearization technique is where α = Θ* − Θ employed such that [11] ~ ~ + Θ' | ~c + h(z, σ ~, ~c ) Θ = Θ' | σ (15) σ σ =σˆ

c c=cˆ

~ ) denotes the high-order term. Substitute ~ = σ * − σˆ ; ~c = c* − cˆ and h(x, ~c , σ where σ (15) into (14), it can obtain that ~ ~ ~T ˆ ~T ' T T ~T Θ Ξ =α Θ + σ Θ αˆ + ~c T Θ ' αˆ + αˆ T h + α +Δ σ

c

~T Θ ~ T Θ ' T αˆ + ~c T Θ ' T αˆ + ε ˆ +σ =α (16) σ c ~ ~T Θ + Δ which the uncertain term ε is bounded. where the uncertain ε = αˆ T h + α Θ1

z1

… zn

α1 Θ2

α2 ∑

Ξ

αm

…

z n −1

translation processing

z2

Θm

Fig. 1. Network structure of a wavelet neural network

Master-Slave Chaos Synchronization of Uncertain Nonlinear Gyros

651

4 AWNNC System Design In order to control the synchronization exactly, the adaptive wavelet neural network controller (AWNNC) system is proposed, where the controller output is defined as u = uˆnc + ucc

(17)

in which the tracking index is defined as

∫

t

s = e + k1e + k 2 e(τ )dτ

(18)

0

where the k1 and k 2 are the non-zero constants. The neural controller uˆ nc uses a WNN to approximate the ideal controller u * , and the compensation controller ucc is utilized to compensate the approximation error between the neural controller and the ideal controller. Substituting (17) into (6) and using (5) and (18), yields e + k1e + k 2 e = u * − uˆ nc − ucc = s

(19)

By substituting (16) into (19), equation (19) can be obtained as follows ~T Θ ~ T Θ ' T αˆ + ~c T Θ ' T αˆ + ε − u . ˆ +σ s=α σ c cc

(20)

Since the number of neurons in WNN is finite, the approximation errors introduced by WNN cannot be inevitable. In order to ensure the WNN-based adaptive control system stable, a compensator should be designed liking a sliding-mode control, which will cause chattering phenomena to wear the bearing mechanism [6]. In this paper, the compensation controller is designed as ucc = Eˆ + κ s

(21)

where Eˆ is the estimated value of ε , and κ is a positive constant. To proof the stability of the AWNNC system, define a Lyapunov function candidate in the following form V=

1 2 1 ~T ~ 1 ~T ~ 1 ~T ~ 1 ~2 s + E α α+ σ σ+ c c+ 2 2η1 2η 2 2η 3 2η 4

(22)

~ where η1 , η 2 , η 3 and η 4 are positive constants and E = ε − Eˆ . Taking the derivative of Lyapunov function in (22) and using (20), yields V = ss +

1 ~T ~ 1 ~T ~ 1 ~T ~ 1 ~ ~ α α+ σ σ+ c c+ EE

η1

η2

η3

η4

1 ~~ ~T Θ ~T α ~+ 1 σ ~ T Θ αˆ + ~c T Θ αˆ + ε − u ) + 1 α ~T σ ~+ 1 ~ ˆ +σ = s (α c T ~c + EE c cc 'T σ

~ T ( sΘ ˆ + =α

~ α

η1

'T

~ T ( sΘ ' T αˆ + )+σ σ

η1

~ σ

η2

T ) + ~c T ( sΘ 'c αˆ +

η2

~c

η3

η3

) + s (ε − u cc ) +

η4

1 ~~ EE

η4

(23)

652

C.-J. Chiu et al.

If the adaptation laws are selected as ~ = sη Θ ˆ αˆ = −α 1

(24)

~ = η sΘ ' T αˆ σˆ = − σ σ 2

(25)

T cˆ = −~c = η 3 sΘ 'c αˆ

(26)

Eˆ = η 4 s

(27)

then equation (23) can be rewritten as V = −κ s 2 ≤ 0

(28) ~ ~ ~ ~ Since V is negative semidefinite, it implies that s , α , σ , c and E are bounded. Let function Ω(τ ) ≡ −κs 2 ≤ −V , and integrate Ω(t ) with respect to time, then it is obtained that [5] t ~ ~, σ ~, σ ~, ~c , E~,0) − V ( s, α ~, ~ Ω(τ )dτ ≤ V ( s, α c, E, t) (29)

∫

0

~, σ ~, σ ~, ~c , E~,0) is bounded, and V ( s, α ~, ~c , E~, t ) is nonincreasing and Because V ( s, α bounded, the following result can be obtained

∫

t

lim Ω(τ )dτ < ∞ t →∞

(30)

0

Moreover, since Ω(t ) is bounded, by Barbalat’s Lemma, it implies that lim Ω(t ) = 0 . t →∞

That is, s → 0 as t → ∞ [5].

5 Simulation Results The proposed AWNNC is applied to synchronize two identical chaotic gyros with nonlinear damping globally. It should be emphasized that the development of the proposed AWNNC method does not need to know the system dynamics of the control system. In simulation, the control parameters of AWNNC are selected as k1 = 4 , k2 = 4 , η1 = 20 , η 2 = η 3 = η 4 = 2 , and κ = 1 . The simulation result of the AWNNC system for initial condition ( x, x) = (1,1) and ( y, y ) = (−1,−1) is shown in Fig. 2. The tracking response of states ( x, y ) is shown in Fig. 2(a), the tracking response of states ( x, y ) is shown in Fig. 2(b) and the associated control effort is shown in Fig. 2(c), respectively. And, the simulation result of the AWNNC system for f x = 33 and f y = 36 is shown in Fig. 3. The tracking response of states ( x, y ) is shown in Fig. 3(a), the tracking response of states ( x, y ) is shown in Fig. 3(b), and the associated control effort is shown in Fig. 3(c), respectively. These simulation results show the favorable tracking performance can be achieved without any knowledge of system dynamic functions by the AWNNC system.

Master-Slave Chaos Synchronization of Uncertain Nonlinear Gyros

State, (x,y)

uncontrolled

M aster system

controlled

Slave system

Time (sec) (a) uncontrolled

State, ( x, y)

controlled

M aster system

Slave system

Control effort, u

Time (sec) (b) uncontrolled

controlled

Tim e (sec) (c)

Fig. 2. Simulation results of AWNNC for different initial condition

State, (x,y)

uncontrolled

Master system

controlled

Slave system

Time (sec) (a)

State, ( x, y)

uncontrolled

controlled

Master system

Slave system

Control effort, u

Time (sec) (b) uncontrolled

controlled

Time (sec) (c)

Fig. 3. Simulation results of AWNNC for different system parameter

653

654

C.-J. Chiu et al.

6 Conclusions Chaotic system is a nonlinear deterministic system that displays complex, noisy-like and unpredictable behavior, so how to synchronize chaotic system become a great deal in engineering community. The dynamics of a gyro is a very interesting nonlinear problem in classical mechanics. This paper proposes adaptive wavelet neural network controller (AWNNC) system composed of a neural controller and a compensation controller. The stability of the AWNNC is proven by Lyapunov function with the online parameter tuning laws are given to adjust the interconnection weights, dilation and translation parameters of wavelet functions. Finally, some simulation results verify the chaotic behavior of two nonlinear gyros can be synchronized by the proposed AWNNC scheme. Acknowledgments. The authors appreciate the partial financial support from the National Science Council of Republic of China under grant NSC 98-2221-E-216-040.

References 1. Chen, H.K.: Chaos and chaos synchronization of a symmetric gyro with linear-plus-cubic

damping. Journal of Sound and Vibration 255, 719–740 (2002) 2. Park, J.H.: Synchronization of genesio chaotic system via backstepping approach. Chaos,

Solitons & Fractals 27, 1369–1375 (2006) 3. Yau, H.T.: Nonlinear rule-based controller for chaos synchronization of two gyros with

linear-plus-cubic damping. Chaos, Solitons & Fractals 34, 1357–1365 (2007) 4. Coelho, L.S., Bernert, D.L.A.: PID control design for chaotic synchronization using a

tribes optimization approach. Chaos, Solitons & Fractals 42, 634–640 (2009) 5. Slotine, J.J.E., Li, W.P.: Applied Nonlinear Control. Prentice Hall, Englewood Cliffs (1991) 6. Lin, C.M., Hsu, C.F.: Neural network hybrid control for antilock braking systems. IEEE

Trans. Neural Network 14, 351–359 (2003) 7. Lin, C.M., Hsu, C.F.: Supervisory recurrent fuzzy neural network control of wing rock for

slender delta wings. IEEE Trans. Fuzzy Systems 12, 733–742 (2004) 8. Hsu, C.F.: Adaptive recurrent neural network control using a structure adaptation

algorithm. Neural Computing and Applications 18, 115–125 (2009) 9. Zhang, Q.: Using wavelet network in nonparametric estimation. IEEE Trans. Neural

Network 8, 227–236 (1997) 10. Billings, S.A., Wei, H.L.: A new class of wavelet networks for nonlinear system

identification. IEEE Trans. Neural Networks 16, 862–874 (2005) 11. Lin, C.K.: Adaptive tracking controller design for robotic systems using gaussian wavelet

networks. IEE Proc. Control Theory Applications 149, 316–322 (2002) 12. Wai, R.J.: Development of new training algorithms for neuro-wavelet systems on the

robust control of induction servo motor drive. IEEE Trans. Industrial Electronics 49, 1323–1341 (2002) 13. Hsu, C.F., Lin, C.M., Lee, T.T.: Wavelet adaptive backstepping control for a class of nonlinear systems. IEEE Trans. Neural Networks 17, 1175–1183 (2006) 14. Lin, F.J., Shieh, H.J., Huang, P.K.: Adaptive wavelet neural network control with hysteresis estimation for piezo-positioning mechanism. IEEE Trans. Neural Networks 17, 432–444 (2006) 15. Hsu, C.F., Cheng, K.H., Lee, T.T.: Robust wavelet-based adaptive neural controller design with a fuzzy compensator. Neurocomputing 73, 423–431 (2009)

WNN-Based Intelligent Transportation Control System Chiu-Hsiung Chen1, Ya-Fu Peng2, and Tsai-Sheng Kao3 1

Department of Computer Science and Information Engineering, China University of Technology [email protected] 2 Department of Electrical Engineering, Ching-Yun University [email protected] 3 Department of Electronic Engineering, Hwa-Hsia Institute of Technology [email protected]

Abstract. In this paper, an intelligent transportation control system (ITCS) using wavelet neural network (WNN) is developed to increase the safety and efficiency in transportation process. The proposed control system is composed of a neural controller and compensation controller. The neural controller acts as the main tracking controller, which is designed via a WNN to mimic the merits of an ideal total sliding-mode control (TSMC) law. The learning algorithms are derived from the Lyapunov stability theorem, which are utilized to adjust the parameters of WNN on-line for further assuring system stability. Moreover, based on robust control technique, the compensation controller is developed to attenuate the effect of the approximation error, so that the desired attenuation level can be achieved. Finally, it is applied to control a marine transportation system. The simulation results demonstrate that the proposed control system can achieve favorable control performance than other control methods. Keywords: Intelligent control, Transportation system, Wavelet neural network (WNN).

1 Introduction Recently, a number of researches have been done on the applications of wavelet neural networks (WNNs) which combine the learning ability of NNs and the capability of wavelet decomposition [1-5]. Unlike the sigmoid functions used in conventional NNs, wavelet functions are spatially localized, so that the learning capability of WNN is more efficient than the conventional sigmoid function NN for system identification and control. Since the ship dynamic models are affected by unpredictable environment and external disturbance such as currents and waves; to design a ship autopilot system becomes a challenging problem. Thus, over the last few years, there has been a growing interest in developing the intelligent control systems for ship motion problem [6-8]. However, few papers concerned about WNN with its fast learning property, which is more suitable for dealing with ship autopilot. In this paper, a WNN-based ITCS is developed to increase the safety and efficiency in transportation process. This control system is composed of two parts: Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 655–661. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

656

C.-H. Chen, Y.-F. Peng, and T.-S. Kao

one is a neural controller that is utilized as the main tracking controller to mimic an ideal total sliding-mode control (TSMC) law, and the other is a compensation controller that is employed to suppress the influence of approximation error. The simulation results demonstrate that the proposed WNN-based control system can achieve favorable control performance than TSMC method.

2 Problem Formulation Consider a transportation system that can be modeled by the following equation:

z(t ) = f 0 ( z (t )) + g 0u (t ) + l ( z (t ), t )

(1)

where u (t ) and z (t ) denote the control input and output of the system, respectively. Define

z (t ) = [ z (t ), z(t )]T

as

the

system

state

vector.

In

addition,

f 0 ( z (t )) and g 0 represent nominal smooth nonlinear function and nominal constant of control input gain, respectively. l ( z (t ), t ) is called the lumped uncertainty. It is assumed l ( z (t ), t ) < β , where β is a given positive constant. The control objective is

to find suitable control law so that the state trajectories of transportation system z (t ) can follow the desired smooth trajectories z d (t ) = [ z d (t ), zd (t )]T .

3 Total Sliding-Mode Control (TSMC) System Design A TSMC system is comprised a baseline model controller and a curbing controller that designed to control the transportation system. In the TSMC system the controlled system has a total sliding motion without a reaching phase [9]. Define the tracking error as e (t )Δ z d − z and then the tracking error vector is defined as e (t )Δ [e, e, ]T . The baseline model controller is given as u BM = g 0−1 [− f 0 ( z (t )) + zd + k T e ]

(2)

where k = [k 2 , k1 ]T is the feedback gain. Applying (2) to (1), the controlled nominal system dynamics are e + k T e = 0 , if k is chosen to correspond to the coefficients of a Hurwitz polynomial, it implies lim e(t ) = 0 . Moreover, e + k T e = 0 can be written in t →∞

0 state variable form as e = AH e, where AH = ⎡⎢

⎣− k 2

1 ⎤ . Therefore, define a total − k1 ⎥⎦

sliding surface as s (t )Δ c(e ) − c(e (0)) −

∫

t ∂c (e ) 0

∂e T

AH e ( τ )dτ

(3)

WNN-Based Intelligent Transportation Control System

657

in which c(e ) satifies ∂c (e ) / ∂ e T = [0 1] , and e (0) is the initial state of e (t ). It is obvious that s (0) = 0 and s(t ) = ∂c(e ) e − ∂c(e ) AH e = 0. T T ∂e

∂e

Since s (0) = 0, there is no reaching phase in the conventional sliding-mode control (SMC). However, the lumped uncertainty is always unavoidable, (2) cannot guarantee the desired performance, and a curbing controller should be added to curb the lumped uncertainty. The curbing controller uC , is given as follow: uC = g 0−1 [ β sgn( s (t ))]

(4)

where β is a curbing control gain, the TSMC law is given as uTSMC = u BM + uC

(5)

Substituting (5) into (1), the state variable form in can be rewritten as follow: e = AH e + b[− β sgn( s (t )) − l ( z (t ), t )]

(6)

where b = [0 1]T . The derivative of total sliding surface s (t ) is s(t ) =

∂c(e ) ∂e

T

e −

∂c(e ) ∂eT

AH e = − β sgn( s (t )) − l ( z (t ), t )

(7)

Since s (0) = 0, to maintain the state on the surface s (t ) = 0, for all time, that only needs to show that s (t ) s(t ) < 0, if s (t ) ≠ 0 . To obtain this objective, a Lyapunov candidate is chosen as va (t ) = 0.5s 2 (t ) and va (t ) = s(t ) s(t ) = − s (t ) [ β − l ( z (t ), t ) ] < 0.

4 Intelligent Transportation Control System (ITCS) Design If the parameter of the system (1) are well known, there exits an ideal TSMC law as u ITSMC = g 0−1 [− f 0 ( z (t )) + zd + k T e − l ( z (t ), t ) + β sgn( s (t ))]

(8)

Applying the control law (8) into (1), the state variable form can be yield e = AH e + b[− β sgn( s (t ))]

(9)

The derivative of total sliding surface s(t ) = − β sgn( s (t )) . Therefore, (8) satisfies the condition s (t ) s(t ) = − β s (t ) < 0 and guarantees the sliding motion. However, in practical applications, the ideal TSMC law shown (8) is unobtainable. Thus, a WNN is proposed to construct the ITCS in this section. The proposed ITCS, which consists of a neural controller uˆ NC and a compensation controller uCC . The control law is assumed to take the following form: u ITCS = uˆ NC + uCC

(10)

658

C.-H. Chen, Y.-F. Peng, and T.-S. Kao

A WNN is proposed from [5]. The overall representation of the i-th input of xi(1) and output node y o is yo =

ni

nP

∑ ∏ wj

j =1

i =1

(

⎛ − x (1) − m ⎜ i ij ⎜ vij ⎜ ⎝

) ⎞⎟exp⎛⎜ − (x ⎟ ⎟ ⎠

⎜ ⎜ ⎝

(1) i

− mij

) ⎞⎟ 2

2vij2

⎟ ⎟ ⎠

(11)

where ni is the number of the input variables and n p is the total number of the product node. mij is the translation factor and vij is the dilation factor of the mother wavelet node, respectively; w j is the connection weight between j-th product node and output node, The output of WNN can be represented as yo =

nP

∑w y j

(3) (1) j ( xi , mij , vij )

(12)

j =1

Define the vectors w, m and v to collect all parameters of the connection weights and mother wavelets of WNN. Thus, the output of WNN is expressed as yo = w T σ where x = [ x1(1) , ... , xn(1) ]T , and σ = [ y1(3) , ... , yn(3) ]T . From (1), (8) and p

i

using (10), the state variable form can be yield e = AH e + b[ g 0 (u ITSMC − u ITCS ) − β sgn( s (t ))]

(13)

The derivative of total sliding surface s (t ) is s(t ) = g 0 [u ITSMC − u ITCS ] − β sgn( s (t ))

(14)

Assume there exists an optimal parameters to approach the ideal TSMC law such that T

u ITSMC ≡ w * σ * + ε1 , where ε1 is a reconstructed error; w * , σ * , m * and v * are the optimal parameter vectors of w, σ , m and v respectively. Thus, an estimating

neural controller uˆ NC is used to estimate the optimal neural controller. The control law (10) can be rewritten as u ITCS ≡ wˆ T σˆ + uCC , where wˆ , σˆ , mˆ and vˆ are the estimated vectors of w * , σ * , m * and v * respectively. Thus, (14) can be expressed ~ T σ~ + wˆ T σ~ + w ~ T σˆ + ε − u ] − β sgn( s (t )) s(t ) = g 0 [ w 1 CC

(15)

~ = w * − wˆ and σ~ = σ * − σˆ . The expansion of σ~ can be obtained as [4] where w ~ + σ v~ + ξ σ~ = σ m m v

(16)

~ = m * − mˆ ; v~ = v * − vˆ ; and ξ is a vector of higher-order terms. Substituting where m (16) into (15) yields ~ T σˆ + m ~ T σ T wˆ + v~ T σ T wˆ + ε − u ] − β sgn( s (t )) s(t ) = g [ w (17) 0

m

v

2

CC

~ T σ~ + ε . Moreover, w * is decomposed into three parts as where ε 2 = wˆ ξ + w 1 T

w * = w *P + w *I + w *D , where w *P ,

w *I and w *D are the proportional, integral and

WNN-Based Intelligent Transportation Control System

659

derivative terms of w * , respectively; the estimation vector wˆ is given as wˆ = wˆ P + wˆ I + wˆ D , where wˆ P , wˆ I and wˆ D are the proportional, integral and ~ can be expressed derivative terms of wˆ , respectively. Thus, w ~ = w * − wˆ and ε = w * + w * . It is obtained as ~=w ~ − wˆ − wˆ + ε , where w as w 3 I I P D 3 P D I I ~ T σˆ − wˆ T σˆ − wˆ T σˆ + m ~ T σ T wˆ + v~T σ T wˆ + ε − u ] − β sgn( s (t )) s(t ) = g 0 [ w I P D m v t CC

(18)

where the uncertain term εt = ε3T σˆ + ε 2 denotes the lump of approximation error. The Lyapunov function candidate is given by vb (t ) =

g ~T ~ g0 T g ~ T ~ g 0 ~T ~ 1 2 s (t ) + 0 w wˆ P wˆ P + 0 m m+ v v I wI + 2 2κ I 2κ P 2κ m 2κ v

(19)

where κ D , κ P , κ I , κ m and κ v are the learning-rates. Taking the derivative of the Lyapunov function (19) and using (18), yields ~ T ⎡ s (t )σˆ − 1 wˆ ⎤ + g 0 wˆ T wˆ + g m ~ T ⎡ s(t )σ T wˆ − 1 mˆ ⎤ vb (t ) = g 0 w ⎢ ⎥ I ⎢ P P m I⎥ 0 κ κ κm ⎦ I P ⎣ ⎦ ⎣ ⎡ 1 ⎤ vˆ ⎥ − β s(t ) + g 0 s (t )[− wˆ TP σˆ − wˆ TD σˆ + εt − uCC ] + g 0 v~ T ⎢ s(t )σ vT wˆ − κ v ⎦ ⎣ t Using wˆ P = κ P s σˆ , wˆ I = κ I s σˆ dt , wˆ D = κ D d [ s σˆ ], mˆ = κ m s σ mT wˆ , ∫ 0

dt

(20)

vˆ = κ v s ( t ) σ vT wˆ and

chosen κ P = κ D , (20) can be rewritten as vb (t ) ≤ g 0 s(t )[εt − uCC ]. Substituting uCC = (2r 2 ) −1 (r 2 + 1) g 0 s

into (20), it is obtained vb (t ) ≤ −0.5 g 02 s 2 (t ) + 0.5r 2 εt2 , where r is a prescribed attenuation constant. Thus, the robust tracking performance can be achieved [10].

5 Illustrative Examples The marine transportation system is described as z(t ) = f 0 ( z (t )) + g 0u (t ) + l (t ), where

f 0 ( z (t )) = −(k s / t m )(n3 z 3 (t ) + n1 z (t )) g 0 = k s / t m , and z (t ) is ship heading angle, u (t ) is rudder angle [8]. In order to confirm the robustness of TSMC system and the proposed ITCS, a lumped uncertainty is considered as ⎧0.05sin (2π ⋅ 0.1 t ), l (t ) = ⎨ ⎩0.5sin (2π ⋅ 0.1 t ),

if if

t < 67.5 sec ⎫ ⎬ t ≥ 67.5 sec ⎭

(21)

The feedback gains are chosen as k1 = 2 and k 2 = 1. For TSMC system, the curbing control gain is chosen β = 0.1. For the proposed ITCS, The learning-rates of WNN are selected as κ p = κ D = 30, κ I = 2 and κ m = κ v = 0.05, and r = 0.8 . The TSMC system is applied to control the ship heading angle, the simulation results are shown in Fig. 1. The tracking response heading angles are plotted in Fig. 1(a). The heading

660

C.-H. Chen, Y.-F. Peng, and T.-S. Kao

angular velocities are shown in Fig. 1(b). The control effort and the total sliding surface are depicted in Figs. 1(c) and 1(d), respectively. From the simulated results, degenerate tracking responses are induced due to the larger disturbance. Furthermore, the simulation results of the proposed ITCS are depicted in Fig. 2. From the simulated results, favorable tracking responses can be obtained and robust control characteristics under the larger disturbance are obvious. Control effort / rudder angle (rad)

Heading angle (rad)

(c)

(a) z

u

zd

add larger disturbance add larger disturbance

Time (sec)

Time (sec) Heading angular velocity (rad)/sec

Total sliding surface (d)

(b)

z d

s

add larger disturbance

z

add larger disturbance

Time (sec)

Time (sec)

Fig. 1. Numerical simulations of TSMC for marine transportation system Control effort / rudder angle (rad)

Heading angle (rad)

(c)

(a) z

u

zd

add larger disturbance

add larger disturbance

Time (sec)

Time (sec) Heading angular velocity (rad)/sec

Total sliding surface (b)

z d

(d) s

add larger disturbance

add larger disturbance

z

Time (sec)

Time (sec)

Fig. 2. Numerical simulations of ITCS for marine transportation system

6 Conclusions This study has successfully investigated an ITCS via WNN to increase the safety and efficiency in transportation process. The major contributions of this study are recited as follows. (1) the successful development of a model-free ITCS via WNN (2) the convergence of the tracking error and network parameter is accelerated by the learning algorithms, and (3) the successful application of the proposed control scheme to control a marine transportation systems.

Acknowledgment The author appreciates the partial financial support from the National Science Council of Republic of China under grant NSC 98-2221-E-163-001.

WNN-Based Intelligent Transportation Control System

661

References [1] Delyon, B., Juditsky, A., Benveniste, A.: Accuracy Analysis for Wavelet Approximations. IEEE Trans. Neural Networks 6, 332–348 (1995) [2] Hanbay, D., Turkoglu, I., Demir, Y.: An Expert System Based on Wavelet Decomposition and Neural Network for Modeling Chua’s Circuit. Expert Systems with Applications 34, 2278–2283 (2008) [3] Ho, D., Zhang, P.A., Xu, J.: Fuzzy Wavelet Networks for Function Learning. IEEE Trans. Fuzzy Systems 9, 20–211 (2001) [4] Hsu, C.F., Lin, C.M., Lee, T.T.: Wavelet Adaptive Backstepping Control for a Class of Nonlinear Systems. IEEE Trans. Neural Networks 17, 1175–1183 (2006) [5] Wai, R.J.: Development of New Training Algorithms for Neuro-wavelet Systems on the Robust Control of Induction Servo Motor Drive. IEEE Trans. Ind. Electron. 49, 1323– 1341 (2002) [6] Burns, R.S.: The Use of Artificial Neural Networks for the Intelligent Optimal Control of Surface Ships. IEEE J. Oceanic Engineering 20, 65–72 (1995) [7] Layne, J.R., Passino, K.M.: Fuzzy Model Reference Learning Control for Cargo Ship Steering. IEEE Control Systems Magazine 13, 23–34 (1993) [8] Rigatos, G., Tzafestas, S.: Adaptive Fuzzy Control for the Ship Steering Problem. Mechatronics 16, 479–489 (2006) [9] Wai, R.J.: Adaptive Sliding-mode Control for Induction Servomotor Drive. IEE Proc. Electric. Power Appl. 147, 553–562 (2000) [10] Chen, B.S., Lee, C.H., Chang, Y.C.: H∞ Tracking Design of Uncertain Nonlinear SISO systems: Adaptive Fuzzy Approach. IEEE Trans. Fuzzy Systems 4, 32–43 (1996)

Incident Detection in Urban Road Wu Bing-Fei, Kao Chih-Chung, Chen Chao-Jung, Li Yen-Feng, Chen Ying-Han, and Yang Cheng-Yen National Chiao Tung University, 1001 University Road, Hsinchu, 300, Taiwan, R.O.C. {bwu,kevinkao,cjchen,takolyf,winand,mitice}@cssp.cn.nctu.edu.tw

Abstract. A video-based incident detection system for monitoring the urban road is investigated in this paper. The developed surveillance system can monitor not only vehicles and motorcycles on the road surface but also pedestrians on the walkway and the prohibitive zones in the image. Several different kinds of incident can be detected in the presented system, such as the congestion, the illegally parking, the lane-changing vehicle, the falling object, the pedestrian across the road, and the pedestrian appearing in the prohibitive zone. The proposed method is based on the background subtraction. Therefore, the background image and lane markings are estimated in the beginning. Then the foreground image is obtained from the difference between the current image and the background image. In order to have isolated objects, the objects in the foreground image are separated by lane marking. The occlusion caused by the vehicles and the pedestrians can also be handled by the temprol-spacial analysis for the exact detection. After that, the tracking is applied to track the targets. Finally the proposed system can detect the incident by the integrated information from the pre-defined information and the tracking. Several challenge video are performed to verify and the experimental results demonstrate that our system is satisfying and effective. Keywords: Vehicle detection, Vehicle tracking, Pedestrian detection, Incident detection.

1 Introduction The past decade has seen the rapid development of surveillance system in many fields. Traffic management is a serious issue in the intelligent transportation systems (ITS), and one of the most significant current discussions is the intelligent incident detection (IID). When the incident happens, the traffic causes congestion, even calamity with death. Therefore, the automatic incident detection system is so important to detect the abnormal event quickly and inform the traffic manager to take proper action. A considerable amount of literature has been published to address the problems of the video-based vehicle detection, pedestrian detection, and incident detection. Zhang et al. [1] used multi-level framework to track vehicle and handler vehicle occlusion. Through the analysis of compactness and interior distance of occluded object in the intra-level, and the motion vectors, the partial occluded object can be Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 663–670. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

664

B.-F. Wu et al.

separated. However, the computation cost is heavily in this method. Ki et al. [2] presented a traffic accident recording system at intersections. In order to get over the effect of luminance change, the vehicle contour is extracted by the frame difference. However, the contour is not obvious, when the speeds of vehicles are slow. Kanhere et al. [3] track the local features for a stable tracking. Nevertheless, the 3-D perspective projection costs lot of time for transforming the image. In [4], the contour is growing when the vehicle is passing the detection line. The growing contour would be affected by the occluded vehicles and motorcycle. Shehata et al. [5] developed a video-based incident detector that can detect stop car, reversing vehicle, falling objects. Shehata also considered the effects of weather, and integrate the information to reduce the false alarm rate. HMM-based incident detection method was proposed by Zou et al. [6] to detect accident in intersection. However, the detection results are affected by the turn-left vehicles, turn-right vehicles, and traffic flow easily. Embedded-based system was developed by Glasl et al. [7] to predict the traffic congestion. Since information of average speed, timestamp, and lane change time was considered, the error from these information may cause prediction wrong. Zhao et al. [8] proposed one model-based approach, integrating the information of the appearance of humans, the visibility of the body obtained by occlusion reasoning and foreground image. The methods presented by Tao can still perform well even when the occlusion occurs between humans in crowded environment. However, the conclusions can be more convincing if the author considered the different walk directions of pedestrians. Viola et al. [9] developed a pedestrian detection system that trained the detector by the image intensity and motion information. Because the computation complexity is too high, the system can not work in real-time. Zhu et al. [10] used several online sample features that always similar in sequential frames. The system can track well even when two objects occluded together. This study would have been much more persuasive if she included more complicated cases. This study presents an incident detection system, based on Sobel edge, background subtraction, and the tracking process, to detect several kinds of incident, such as congestion, stop car, reversing car, falling object, and pedestrian passing through the road. This developed system focus on the camera setting up on the road side that this manner has smallest cost and is the most popular in the urban road. Section 2 introduces the presented architecture and methods. The experimental results are shown in the section 3. Finally, the conclusions are given in section 4.

2 The System Architecture The architecture of the developed system is depicted in Fig. 1. The image sequence captured from the camera inputs into the developed system. The foreground image is extracted as the basic information by the background model. The edge information is obtained based on Sobel operator. After that, the morphological operator, such as dilation and erosion, are operated to remove the noise in the information image. Connect-Component labeling then is performed in the object detection module to obtain the rectangle address of objects. Connect-Component labeling process labels the existing object, as shown in Fig. 2(a).

Incident Detection in Urban Road

665

Fig. 1. The working flow indicates the process of the presented system

The process labels the neighboring foreground pixels the same mark and these pixels with the same label mark are considered as one moving object (MO). With the label information, the system scans and gets the rectangle address of objects. The extracted foreground object with green rectangle is regarded as the motorcycle or pedestrian by the ratio of object height and with and the object with purple rectangle is the vehicle. The results are illustrated in Fig. 2(b). In order to confirm if the vehicles, motorcycles, and pedestrians appear in the successive images belong to the same objects or not object tracking is operated in the following. The tracings contain the address of the existing objects and inform the background update process to update the background image except the region where the object occupies. The proposed system checks the identified tracings by the following incident judgment criterion to make sure if the incident happens.

666

B.-F. Wu et al.

1

1 1 1

1

1

1

1

1

1

1 1 1 1

1

8

8 8

8 8

3

3

3 3

3

8 8 8 8

3

3

3

3 3

8 8 8 8

8 8 8

8 8 8

3

3 3 3 3 3 3

3

Fig. 2. The working flow indicates the process of the presented system

The tracking process working flow is shown in Fig. 3. In the tracking beginning, the system computes the distance between the detected MO and the tracking MO by (1) and (2). Ax and Ay represent the center x and y position of detected MO in the current frame respectively, and Bx and By are the tracking MO in the previous frame.

Fig. 3. The tracking work flow compares the detected MO with tracking MO

Incident Detection in Urban Road

667

Diffx = |Ax-Bx|.

(1)

Diffy = |Ay-By|.

(2)

If the computed distance is small and the cover degree of these two MO is high, these two closed MO can be considered as the same object in space domain. Although the detected MO position is closed to the tracking MO, we still have to verify these two MO that are identical. The verify process are in (3)-(6). R, G, and B represent the red, green, and blue color of detected MO and tracking MO respectively. The column and row position are denoted as x and y, and m and n indicates the width and height of MO respectively. δR, δG, and δB are the difference of R, G, and B color channel between the detected MO and the tracking MO respectively, and the total difference summation is denoted as δT in (6). If the total difference summation is large than the predefined threshold, these two MO are not the same object. Otherwise, the detected MO is updated to tracking MO in the current frame. If the comparison is fail, the tracking MO using prediction to predict the new position in the current frame. Finally, each tracing is considered as leaving ROI and can be deleting, if it is position is in the boundary of ROI. m

n

δ R = ∑∑ AR ( x, y ) − BR ( x, y )

(3)

x =1 y =1 m

n

δ G = ∑∑ AG ( x, y ) − BG ( x, y )

(4)

x =1 y =1 m

n

δ B = ∑∑ AB ( x, y ) − BB ( x, y )

(5)

δT = δ R + δG + δ B

(6)

x =1 y =1

With the tracking information, we can identify the event by the type of MO and the tracings. Firstly, the size of object is used to distinguish the type of the object. Since the size of the falling object is always smaller than the size of the vehicle, motorcycle, and pedestrian, the small size object detection is the basic in our falling object detection. If size of the object is small and the time that the object occupies the space is more than the pre-defined threshold, the object is identified as the falling object and warning message is sent. Because the experimental environment is in the urban and the pedestrians sometimes occur in on the walkway of detection zone (DZ), the pedestrian detection is integrated into our system to find the pedestrians. However, the motorcyclists also contain the characteristic of head. The speed of moving is utilized to help the system to distinguish the motorcyclist and the pedestrian. If the objects are identified as pedestrians and they appear in the prohibitive zone, the

668

B.-F. Wu et al.

system sends alarm message. The rest objects identified as the vehicle are judged whether the vehicles are in the same lane or not. If the driver changes the lane in DZ, lane change warning is transmitted. The system collects and computes the flow, occupation, and average speed information every five minutes. With the information mentioned above and the definition from the government, the system can determine if the congestion happens or not. If the vehicle stops in DZ for a long time and other vehicles pass smoothly, our system regards the situation as stop vehicle and give an alarm. Finally, the reversing car is identified, if the tracked vehicle moves backward.

3 Experimental Results The developed system has been implemented to one personal computer on the Pentium 3 GHz CPU, 1G RAM platform with Windows XP SP3. In our experiments, the size of each image is 320×240 and the frame rate of the image sequence is 30 fps. The average processing time of vehicle detection and incident detection is 20 ms. The detection results of reversing car are illustrated in Fig. 4. The reversing car is detected while the system determines the vehicle is moving backward. In Fig. 4(a), the region below the green lines is the detection zone and the black car is regarded as the reversing vehicle, while the tracking process checks the positions in several frames. The red line is the tracking line, and the tracking results indicate that the current position in Fig. 4(b) is lower than the position in Fig. 4(a).

Fig. 4. (a) The reversing car is detected on the road. (b) The start node of the tracking line indicates that the current position is backward to the detect position in the beginning.

The stop vehicle is detected and the results are shown in Fig. 5. The falling object detection results are displayed in Fig. 6. If the object is not separated from the vehicle, the system can not locate it. However, the system can detect the falling object, while the object is isolated on the road surface for a period of time. The pedestrian detection results are shown in Fig. 7. If the detected pedestrians are not on the walkway, they are determined as an incident. Therefore, the rectangles of pedestrians on the road surface and on the grass are drawn in red. Moreover, the proposed system can track the pedestrian when the occlusion occurs.

Incident Detection in Urban Road

669

Fig. 5. (a) The stop vehicle is detected in the detection zone. (b) The stop vehicle stops for a long time, and the developed system can track it.

Fig. 6. (a) The falling object is going to leave from the vehicle. (b) The falling object is detected in the detection zone.

Fig. 7. (a) The pedestrians are detected in prohibitive zone. (b) The tracking process can still handle well even when the occlusion between the pedestrian and the vehicle happens.

670

B.-F. Wu et al.

4 Conclusions In this work, the automatic vehicle detection and incident detection approach based on background model and edge is presented. The traffic information such as flow and average speed are computed. The developed system also can identify several different kinds of incident in the detection zone. The falling object is located if the small object occupies in the specific region for a long period. The pedestrian is distinguished with the vehicle and motorcyclist by the characteristic of head and moving speed. If the pedestrian runs into the road surface or other prohibitive zone, the warning is transmitted. The stop car is defined as the tracked vehicle stands in one region too long. The congestion condition happens when the numbers of passing vehicles in every interval are relatively small. With the lane information and the position of the pass vehicles, the lane change vehicle can be determined. The reversing vehicle also can be located if the track vehicle moves backward. Experimental results indicate that the proposed method can operate in real-time with high accuracy. Acknowledgments. This work was supported by National Science Council under Grand no. NSC 98-2752-E-009-003-PAE.

References 1. Zhang, W., Wu, Q.M.J., Yang, X., Fang, X.: Multilevel Framework to Detect and Handle Vehicle Occlusion. IEEE Trans. Intelligent Transportation Systems 9, 161–174 (2008) 2. Ki, Y.K., Lee, D.Y.: A Traffic Accident Recording and Reporting Model at Intersections. IEEE Trans. Intelligent Transportation Systems 8, 188–194 (2007) 3. Kanhere, N.K., Birchfield, S.T.: Real-Time Incremental Segmentation and Tracking of Vehicles at Low Camera Angles Using Stable Features. IEEE Trans. Intelligent Transportation Systems 9, 148–160 (2008) 4. Tai, J.C., Tseng, S.T., Lin, C.P., Song, K.T.: Real-Time image tracking for automatic traffic monitoring and enforcement applications. J. Image and Vision Computing 22, 485– 501 (2004) 5. Shehata, M.S., Cai, J., Badawy, W.M., Burr, T.W., Pervez, M.S., Johannesson, R.J., Radmanesh, A.: Video-Based Automatic Incident Detection for Smart Roads: The Outdoor Environmental Challenges Regarding False Alarms. IEEE Trans. Intelligent Transportation Systems 9, 349–360 (2008) 6. Zou, Y., Shi, G., Shi, H., Wang, Y.: Image Sequences Based Traffic Incident Detection for Signaled Intersections Using HMM. In: 9th IEEE International conference on Hybrid Intelligent Systems, pp. 257–261. IEEE Press, Shenyang (2009) 7. Glasl, H., Schreiber, D., Viertl, N., Veigl, S., Fernandez, G.: Video Based Traffic Congestion Prediction on an Embedded System. In: 11th IEEE International conference on Intelligent Transportation Systems, pp. 950–955. IEEE Press, Beijing (2009) 8. Zhao, T., Nevatia, R., Wu, B.: Segmentation and Tracking of Multiple Humans in Crowded Environments. IEEE Trans. Pattern Analysis and Machine Intelligence 30, 1198– 1211 (2008) 9. Viola, P., Jones, M.J., Snow, D.: Detecting Pedestrians Using Patterns of Motion and Appearance. International Journal of Computer Vision 63, 153–161 (2005) 10. Zhu, L., Zhou, J., Song, J.: Tracking Multiple Objects through Occlusion with Online Sampling and Position Estimation. J. Pattern Recognition 41, 2447–2460 (2008)

An Efficient Web-Based Tracking System through Reduction of Redundant Connections Bing-Fei Wu, Ying-Han Chen, Chao-Jung Chen, Chih-Chung Kao, and Po-Chia Huang Department of Electrical Engineering, National Chiao Tung University, Hsinchu, Taiwan, R.O.C. {bwu, winand, cjchen, kevinkao, slipper1220}@cssp.cn.nctu.edu.tw

Abstract. With the convergence of the Internet and the Global Positioning System (GPS), a common approach called GPS tracking system has been developed nowadays. A GPS tracking system is a system that uses GPS to determine the precise location of a vehicle, person, or other assets and the information can also be shared with remote clients through the Internet. For the device used in the outdoor, it usually connects to the Internet through mobile network like GPRS or 3G. However, the weaknesses of mobile network are low bandwidth and high charging strategies compared to Ethernet or even Wi-Fi. In this paper, an efficient web-based tracking system through reduction of redundant connections is proposed. By integrating several technologies nowadays we design a new connection scheme to improve the efficiency for the location-aware application. The proposed architecture is focus on two problems. One is the redundant connection, causing the waste of bandwidth and fees during the period when connecting to the mobile network. The other is the data asynchronization during the period of transmission. This paper not only provides a new connection scheme to solve above issues, but also has verified this architecture on an embedded platform with a prototype implementation for a web-based tracking system. Keywords: GPS, Google Maps, Tracking.

1 Introduction Recently, Location-based services (LBS) have become one of the fastest-growing areas with enormous modern devices capable of Global Positioning System (GPS). By providing information and services depending on the location of the user or mobile object, various applications have been developed to benefit people in many situations. Numerous studies and experiments have investigated the potential of LBS in fleet management, vehicle monitoring, individual life-log, tourism information, navigation system and other areas [1-6]. In addition, tracking or monitoring system in LBS is the most common application used for locating any interested people or items. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 671–677. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

672

B.-F. Wu et al.

An intelligent tracking system [7] have been developed which can provide extremely accurate location information for some specific persons. In [8], the user with a mobile phone can upload its current position to a server and obtain the specific information from the server with a query text. Goo-Tracking [9] uses an embedded device with a GPS/GPRS module as a client to periodically transmit its location to a server and display the location by using Google Earth software. To assist people to locate their family and friends, a client server system designed for a mobile application has been proposed [10]. However, the connection architecture based on traditional client server model in most applications mentioned above exist some shortcomings. Under this model, a client has to periodically send its location to a server because the server needs to maintain a latest location of the client to provide real-time information. Therefore, if the server desires to provide location of the client more close to the real position, the client has to increase its update rate of sending location. In a practical application, the client is usually a mobile target in the outdoor with a device equipped GPS to provide its location. Using a battery to power the mobile device and get access to the Internet via the mobile network such as GPRS and 3G is quite reasonable. Thus, frequent transmission of unnecessary information not only quickly drains energy form the mobile device’s battery, but also occupies the limited mobile network resource. Furthermore, communication costs on the mobile network to the user as the client could decrease the wiliness of using this service. To solve the problems mentioned above, several methods for reducing location updates to a server while maintaining the real-time responses have been proposed. To reduce the number of GPS fix transmissions to a server and the position recalculation on a mobile device, the Critical Point (CP) algorithm and the locationaware state machine is designed in [11]. By the dynamic management with these two designs, the mobile phone can reduce the amount of transmission costs and extend the usage life of a battery. Different from the improvement on the client site, [12] have proposed a solution based on Ajax push pattern using on the server site. By utilizing this technology, the connection between the server and the client will be kept alive with this persistent connection feature in HTTP/1.1. Therefore, once the latest location is uploaded by the client, the server can do the response actively to the browser without polling from the viewer as traditional methods. Although both methods provide solutions on the client site and the server site, they still use the traditional connection scheme, that is to say, the client has to send the location to the server first and the delay of the real position is decided by the update period. Using the traditional approach to develop a web-based GPS tracking system has two following drawbacks. Redundant connection: each client has to upload its location periodically to a server for viewers to monitor the latest location. However, it is difficult to define an ideal interval to avoid redundant connection because the client even does not know when viewers will connect to the server to request the client’s position. Data asynchronization: there are two intervals in the traditional data connection scheme. One is the interval that the client uploads its location to the server, and the other is the one that the viewer requests the location from the server. Both intervals have to be set and either will restrict the location update rate for the viewer.

An Efficient Web-Based Tracking System through Reduction

673

This paper proposes a new connection scheme for a web-based tracking system to solve the above two drawbacks. Three major roles designed for our scheme has to be defined first. One is a tracking device, a mobile device providing a target’s location. Another is a tracking server, a web server providing the location received from the tracking device. Last is a track viewer, a user monitoring the target’s location through general browsers. The concept of our scheme is simple. The tracking device only sends the location to the tracking server when the viewer requests. In other words, the tracking device will not actively and periodically upload the location to the server, but passively waits for the request instead. Hence, the existence of the redundant connection is eliminated because every data becomes meaningful. The data asynchronization will not occur because the process only relates to the delay time of transmitting. The rest of the paper is organized as follows. Section 2 describes the detail of the proposed system architecture. Section 3 shows the system prototype and the implementation results. Finally, the conclusion is given in Section 4.

2 System Architecture The proposed connection scheme aims to create a direct channel between the viewer and the tracking device. Although existing technologies may have some limitations, flexibly integrating these technologies can achieve our purpose. For a web-based tracking system, a server displaying target’s location on a map for viewers is necessary. Considering the efficiency and flexibility, Google Maps service is used on the tracking server to rapidly show the current target’s location for viewers. To build a direct channel, the tracking device has to be a server with a public IP for the tracking server to connect and forward the request from the viewer. The IP address, which is assigned from the Internet Service Provider (ISP) when connecting to the mobile network, is a dynamic IP. Therefore, the target has to set a fixed ID and send it with the obtained IP to the tracking server as a registration. To offer dynamic interfaces on web pages, Ajax is utilized to process the viewer’s action. However, Ajax is not allowed to make a cross domain request due to the security issue. This restriction fails the connection when the tracking server connects to the tracking device. Hence, a special proxy is designed to conquer this limitation. The detail for the tracking device and the tracking server is described in the following. 2.1 Tracking Device The tracking device can provide the location and access the Internet. Three modules are designed to process and transmit the information. HTTP Processing Module (HPM) analyzes the HTTP request from the viewer’s browser and provides a response message and Data Packaging Module (DPM) plays a role of packaging the data into Extensible Markup Language (XML) format. By Data Aggregation Module (DAM), all gathered data from sensors are parsed to acquire information. With these three modules, the tracking device can have flexibility in dealing with the providing information. Once receiving the GPS data, DAM parses the data to obtain the information, such as longitude, latitude, altitude, speed, bearing and other interested

674

B.-F. Wu et al.

values. Next DPM has to package these values in XML format as a response message and store it in a buffer for HPM to process. HPM is actually a server implemented the part function of HTTP/1.1 protocol. It analyses GET and POST methods from the tracking server and send back the response message with a standard HTTP header. 2.2 Tracking Server The tracking server transfers the data between the viewer and the tracking device, consisting of three components, a web server, a database and a proxy. The web server is used for receiving the request from the user and transferring the response web pages back. The database exists for storing the information of the tracking device, such as ID and IP address. The proxy is used to forward the request and the response message between the tracking server and the tracking device. It is implemented by a PHP script to conquer the cross domain limitation by socket communication. On receiving the response message, the tracking server parses it to get the location data and then cooperated with the Google Map API to produce the map for the viewer. The connection scheme is divided into two parts, a registration phase and a request phase. The registration phase includes steps1-2 and only executes when the tracking device gets access to the mobile network for the first time. On the other hand, the request phase includes steps 3-10 and executes whenever the viewer requests a new location update. The message flow including two phases of the proposed connection scheme is shown in Fig. 1. Tracking Device

Track Viewer

Tracking Server Proxy

Server

1.Register IP and ID

Database

2.Store IP and ID 3.Request Data 4.Query IP 5.Return IP 6.Forward Request

7.Forward Request 8.Response Data 9.Forward Response 10.Forward Response

Fig. 1. The message flow of proposed connection scheme

3 Implementation This section describes the implementation details and the experimental results. The tracking server is built with an Apache web server and programmed with PHP scripts and JavaScript. The tracking device is implemented on an ARM platform with Linux kernel 2.6.18, and programmed with Qt/Embedded 4.4.3. The peripherals include a

An Efficient Web-Based Tracking System through Reduction

Fig. 2. The tracking device Google Maps Server 1 Initial Time for Map

Tracking Server

Fig. 3. The web page for viewers Tracking Device

2 Timer interval 4 Transmit via Ethernet 3 Transmit via 3G 5 Loading Time of Map

Delay time (ms)

Viewer’s Browser

2000 1800 1600 1400 1200 1000 800 600 400 200 0

Ethernet

1

(a)

675

2

Wi-Fi

3

3G

4

5

6

Stages

(b)

Fig. 4. (a) The flow diagram of delay time. (b) The delay time in every stage.

GPS module and a general mobile phone as a 3G modem. Fig. 2 shows the prototype of our tracking device, and Fig. 3 demonstrates the web page of the tracking server for the viewer. On the web page the viewer can not only arbitrarily choose the target by selecting the registered target’s ID, but also decide the location update rate by selecting the timer period. The web page will repeatedly retrieve the new location data according to the period setting. To evaluate the update efficiency of the system, the process of update is divided into five stages including the time for processing and communicating as shown in Fig. 4(a). Stage 1 represents the initial setting time to request a map from the Google Maps Server. Stage 2 is the fixed timer interval set by the track viewer. Stage 4 means the transmitting time between the viewer’s browser and the tracking server via Ethernet. In contrast, stage 3 is the transmitting time between the tracking server and the tracking device via the mobile network, such as 3G in our experiment. Stage 5 means the time to show an icon on the center of map to represent the tracking target’s current location after the browser obtains the response message from the tracking device. By default configuration, the viewer connects to the tracking server via Ethernet. However, the viewer may use other methods to connect to the Internet. Therefore, an evaluation for the delay time in each stage through different communication medium, Ethernet, Wi-Fi and 3G, between the track viewer and the tracking server is shown in Fig. 4(b). An extra stage 6 shown in the figure means the total time spent from the stage 1 to stage 5. It is expected that the delay time in the stage 4 is dissimilar due to

676

B.-F. Wu et al.

the different communication medium. In our evaluation, the most considerable part is the worst delay time via the different medium, especially the low-bandwidth medium. Because the delay time in the stage 2 is a fixed time interval set by the viewer, it has to be ignored for evaluating the delay time of processing and communicating. Thus, the delay time subtracting the time in the stage 2 from the one in the stage 6 indicates that every request sent from the viewer can retrieve the response within 1 second. This result also guarantees that the user can monitor the status of the tracking target in real time even in the worst case.

4 Conclusion This paper has proposed a new connection scheme for the practical tracking system with the real-time monitoring performance. For the viewer, a friendly web-based interface is provided and there is unnecessary to install any software. The results also indicate that it keeps the real-time performance for transmitting data through different connection methods. In the future, the security, the rich content and the flexible management for sharing information will be interesting extensions. Acknowledgments. This work was supported by National Science Council under Grand no. NSC 98-2752-E-009-003-PAE.

References 1. Al-Taee, M.A., Khader, O.B., Al-Saber, N.A.: Remote Monitoring of Vehicle Diagnostics and Location Using a Smart Box with Global Positioning System and General Packet Radio Service. In: IEEE International Conference on Computer Systems and Applications, pp. 385–388. IEEE Press, Amman (2007) 2. Ryoo, D.W., Bae, C.: Design of The Wearable Gadgets for Life-Log Services based on UTC. IEEE Trans. Consumer Electronics 53, 1477–1482 (2007) 3. Lim, H., Yang, J.H., Lee, Y.S.: The Development of an Ubiquitous Learning System Based on Audio Augmented Reality. In: IEEE International Conference on Control, Automation and Systems, pp. 1072–1077. IEEE Press, Seoul (2007) 4. Modsching, M., Kramer, R., Riebeck, M., Stark, A., Ten Hagen, K., Kawalek, J.: Field Trial on the Efficiency and User Experience of GPS based State of the Art navigational Systems for Pedestrians. In: 4th Workshop on Positioning, Navigation and Communication, pp. 129–134. IEEE Press, Hannover (2007) 5. Philippopoulos, P.I., Soulos, G., Krukowski, A., Argyreas, N., Gorgas, V., Kyriazanos, D., Axiotis, D.I., Thomopoulos, S.C.A.: ToD: An Intelligent Location-Based Transport Management System. In: IEEE International Conference on Intelligent Vehicles, pp. 793– 798. IEEE Press, Istanbul (2007) 6. Thong, S.T.S., Han, C.T., Rahman, T.A.: Intelligent Fleet Management System with Concurrent GPS & GSM Real-Time Positioning Technology. In: IEEE International Conference on Telecommunications, pp. 1–6. IEEE Press, Sophia Antipolis (2007) 7. Mazinan, A.H.: Specific Persons Surveillance Using Satellite Technique. In: IEEE International Conference on Intelligent and Advanced Systems, pp. 474–478. IEEE Press, Kuala Lumpur (2007)

An Efficient Web-Based Tracking System through Reduction

677

8. Titica, D., Fratu, O., Stanescu, E., Halunga-Fratu, S.: Simple Location-based Application Development for Mobile Phones. In: 8th IEEE International Conference on Telecommunications in Modern Satellite, Cable and Broadcasting Services, pp. 15–18. IEEE Press, Serbia (2007) 9. Chadil, N., Russameesawang, A., Keeratiwintakorn, P.: Real-time Tracking Management System Using GPS, GPRS and Google Earth. In: 5th IEEE International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology, pp. 393–396. IEEE Press, Krabi (2008) 10. Al-Suwaidi, G.B., Zemerly, M.J.: Locating Friends and Family Using Mobile Phones with Global Positioning System (GPS). In: IEEE International Conference on Computer Systems and Applications, pp. 555–558. IEEE Press, Rabat (2009) 11. Barbeau, S., Labrador, M.A., Perez, A., Winters, P., Georggi, N., Aguilar, D., Perez, R.: Dynamic Management of Real-Time Location Data on GPS-Enabled Mobile Phones. In: IEEE International Conference on Mobile Ubiquitous Computing, Systems, Services and Technologies, pp. 343–348. IEEE Press, Valencia (2008) 12. Shen, L., Yang, C., Zhang, F., Ren, Y., Wu, L., Zhu, L.: A Scalable Web GPS Monitoring System Based on AJAX Push Pattern. In: 17th IEEE International Conference on Geoinformatics, pp. 1–5. IEEE Press, Fairfax (2009)

An Embedded All-Time Blind Spot Warning System Bing-Fei Wu, Chao-Jung Chen, Yen-Feng Li, Cheng-Yen Yang, Hai-Chang Chien, and Chia-Wei Chang Department of Electrical Engineering, National Chaio Tung University, Taiwan {bwu, cjchen, takolyf, mitice, as, chiawei}@cssp.cn.nctu.edu.tw

Abstract. Blind spot warning (BSW) systems play an important role in advanced driving assistance systems (ADAS). Crashes are frequently happened when drivers change their host lanes without taking notice of the vehicles in the blind spot area. In this paper, a BSW system is developed and has been real road verified with CCD/CMOS cameras. The proposed algorithm is robust to adapt different weather conditions in day and night, including sunny, cloudy and rainy. It has been implemented on our self-designed DSP system with a 600M Hz core processor. It deserves to be mentioned that the bilateral BSW function is completed on one DSP system and it processes more than 20 frames per second in CIF image format. The average detection ratio achieves 95.09%. In the future, this system will be integrated with lane departure warning systems, previous vehicle warning systems and parking assistance systems to be the omni-directional ADAS. Keywords: Driving assistance, Blind spot, Warning system.

1 Introduction From the statistical results, the terrible disasters have been increased dramatically. The misfortunes can be avoided if the drivers are notified with the real-time warning messages by the driving assistance systems (DAS) in a jiffy, includes lane departure warning, collision avoidance and blind spot warning (BSW) systems. Thus, many researchers focus on the developments of DAS to solve these huge problems. In BSW systems, ultra sonic sensors have been utilized to implement a blind spot accident prevention system [1]. In vision-based BSW systems, optical-flow is an efficient detection algorithm for moving vehicles, especially for the approaching vehicles [2-3]. However, the computation consumption will limit its application. An adaptive template matching algorithm is addressed in [4], where the shadow-based tracking algorithm achieves the blind spot detection in daytime. Not only the BSW systems for sedans, the BSW system for trucks and buses has also been presented. In [5], four cameras were mounted on a truck–trailer combination, and the images are combined such that obstacles are visible in an image presented to the driver. In that study, a clear combined bird-view image is provided to the drivers rather than the automatic detection and warning. A morphology-based saliency map model and modified Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 679–685. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

680

B.-F. Wu et al.

mean-shift tracking algorithms are proposed in [6] to detect the vehicles in the blind spot area and its detection accuracy achieves 82%. It is observed that the vision-based BSW systems always focus on the detection in daytime; however, fewer articles discuss the detection at nighttime. Therefore, an all-time blind spot detection system has been proposed in this paper, which includes a day-night transmission decision to ensure the detection is in the corresponding detection mode. In this paper, the proposed system overview has been presented in Section 2. Sections 3 to 5 introduce the detection algorithms in detail. The experimental results are given in Section 6. Finally, the conclusion is stated in Section 7.

2 System Overview In this paper, an embedded blind spot warning (BSW) system is proposed for lanechanging assistance, which is the one for all-time detection, i.e. it works well in day and night. This system has been implemented on an embedded system with a 600 MHz DSP, designed by the authors, as shown in Fig. 1(a), which achieve more than 20 frames per second. The developed algorithms are robust for CCD and CMOS cameras. The cameras are mounted on the side mirrors. The left and right sides ones are shown in Fig. 1 (b) and (c), respectively.

(a)

(b)

(c)

Fig. 1. (a) The self-designed DSP system. (b) The left side camera. (c) The right side camera.

Fig. 2. The flowchart of the proposed algorithm

In this BSW system, five modules have been schemed out for whole day detection in the algorithm, as shown in Fig. 2. They are day-night transmission decision, daytime detection, nighttime detection, distance estimation and warning strategy

An Embedded All-Time Blind Spot Warning System

681

decision modules. The day-night transmission module will define the detection mode so that the BSW system can follow the corresponding detection mode. The detected vehicles will be tracked in the blind spot area to estimate its distance from it toward the host vehicle. When the position of the vehicle is dangerous to the host vehicle, a warning signal will be generated to notify the driver.

3 Day-Night Transmission Decision Module The luminance intensity is the feature for the detection to decide the proper detection mode. As shown in Fig. 3 (a), the four blocks, I, II, III and VI, are the statistical area for luminance intensity calculation. The mean, G DN , and the standard deviation,

σ DN are calculated as (1) and (2), where N means the pixel numbers of a block and Ik is the kth pixel intensity of the block. Through G DN and σ DN , GradeDN is calculated as (3). G DN =

σ DN =

1 N

N

∑I k =1

N

1 N

∑I k =1

2 k

(1)

k

2

− G DN

(2)

4

Grade DN = ∑ ( G DNl - σ DNl )

(3)

l =1

(a)

(b)

Fig. 3. (a) The statistical area for day-night decision. (b) The decision thresholds and rules.

GradeDN is utilized to define the conditions, day and night. The thresholds of dusking and dawning are THDNH and THDNL, respectively, and the detection mode will be switched to the corresponding one with the decision, as shown in Fig. 3 (b). When GradeDNTHDNH.

682

B.-F. Wu et al.

4 Daytime Detection Module In this mode, side lane detection will be processed first to arrange the detection area. And then, the approaching vehicle detection zone (AVDZ) and the closing vehicle detection zone (CVDZ) are N1N2F2F1 and N2N3F3F2, respectively, in Fig. 4(a). In daytime detection algorithm, the shadow underneath the vehicle is the major feather to identify the existing vehicle. To recognize the shadow region in the image, the intensity of the road surface is a critical issue for shadow hunting. Therefore, the minimum ground statistic method is utilized to adaptive define the threshold. In consequence, there exists only one minimum row per column, and all minimum rows in those columns enclose a region of the ground, which is exactly the minimum ground. As shown in Fig. 4(b), the statistic area of the minimum ground is IJKL and the minimum ground area is illustrated in Fig. 4(c). Utilizing the men value, G MG , in the minimum ground, as shown in (4), the dynamic threshold for the shadow position detection is adopted, where G MG is defined in (4). Once the shadow position is stated, the vertical Sobel edge, as shown in Fig. 5(a), is applied to verify the confidence level of the shadow. If the shadow is reliable, the horizontal edge information will be employed to re-identify the vehicle position. As shown in Fig 5(b), the white dots mean the dark pixels those own the lower intensity than G MG and the gray dots are the horizontal edges from Sobel edge detector. Conforming to these rules, the location of the vehicle is determined as soon as LVS > 1/4 LRD, where LVS and LRD are the widths of the shadow and the lane, respectively, as shown in Fig. 5 (c).

G MG =

(a)

1 N

N

∑I k =1

(b)

(4)

k

(c)

Fig. 4. (a) The detection zone definitions. (b) The statistics area of the minimum ground. (c) The minimum ground area.

(a) (b) (c) Fig. 5. (a) The vertical edge of the vehicle. (b) The compound plane of the horizontal Sobel edge and the shadow information. (c) The shadow underneath the vehicle.

An Embedded All-Time Blind Spot Warning System

683

5 Nighttime Detection Module In this mode, the bright objects caused by the head lamps of the car are the key feathers for vehicle detection at night. However, the bright objects in the image are not only resulted from the lamps, but also the streetlamps, neon lamps, etc. In order to distinguish the desired objects, the connecting component labeling method and the couple characteristic are adopted. As shown in Fig. 6, the bright objects of Fig. 6(a) are illustrated in Fig.6 (b). The couple lamps are described in Fig. 6(c) with their contours, the similarity of the area and the light-dark-light identity.

(a)

(b)

(c)

Fig. 6. Nighttime vehicle detection. (a) The original image in urban roadway. (b) The bright objects of Fig. 6(a). (c) The detected vehicle with couple lamps characteristic.

6 Distance Estimation Module and the Conclusions In order to estimate the distance from the host vehicle toward the detected vehicles in the blind spot area on image, the coordinate transferring from 2D image plane to the 3D world space is applied. The projection relationship from global coordinates (x, y, z) to the image plane (u, v) is shown in (5), where eu= f/du and ev= f/dv with the physical width and height of an image pixel, du and dv, respectively, f is the focal length and H is the height of camera setting. Figure 7 shows the blind spot vehicle detection in daytime. In these cases, the joints of the bridges, the marks and texts painted on road surface and the shadow resulted from the trees and streetlamps will not cause the detection accuracy ratio. Even in the rainy day, the proposed system still works. u=eu(x/y), v=ev((z-H)/y)

(a)

(b)

(5)

(c)

Fig. 7. Daytime detection results. (a) Joint of bridges. (b) Shadows. (c) Rainy day.

684

B.-F. Wu et al.

(a)

(b)

(c)

Fig. 8. Nighttime detection results. (a)In highways. (b)(c)In urban roadway.

The detection results in nighttime are shown in Fig. 8. This system not only be applied in highway systems, but also in urban roadway. No matter the detection in daytime or nighttime, the distance estimation is included in the proposed system. In the distance estimation, camera parameters and the settle condition are considered to calibrate the distance accuracy. The detection ratio is collected in Table 1, where the average detection ratio is 95.09%. Table 1. Detection ratios under different conditions Weather Missed/Detected Frames Detection Ratio

Daytime (sunny & cloudy) 38/2462 98.48%

Rainy

Nighttime

Total

126/1874 93.7%

123/2377 95.08%

287/6713 95.90%

7 Conclusions The proposed embedded blind spot warning system has been successfully implemented on the DSP platform. According to the algorithms, the daytime and nighttime vehicle detection algorithms are addressed in this paper and verified on real road. Furthermore, a day-night mode decision rule is determined to enable the ability of whole day detection. In the future, this system will be integrated with lane departure warning system, previous vehicle warning system and parking assistance system to be an omni-directional driving assist system. Acknowledgments. This work was supported by National Science Council under Grand no. NSC 98-2752-E-009-003-PAE. The authors would like to give deeply thanks for the supports from CSSP laboratory members.

References 1. Mahapatra, R.P., Vimal Kumar, K., Khurana, G., Mahajan, R.: Ultra Sonic Sensor Based Blind Spot Accident Prevention System. In: 2008 International Conference on Advanced Computer Theory and Engineering, Phuket, pp. 992–995 (2008)

An Embedded All-Time Blind Spot Warning System

685

2. Díaz, J., Ros, E., Mota, S., Botella, G., Canas, A., Sabatini, S.: Optical Flow For Cars Overtaking Monitor: The Rear Mirror Blind Spot Problem. In: IEEE Intelligent Vehicles Symposium, Las Vegas, pp. 50–57 (2005) 3. Batavia, P.H., Pomerleau, D.E., Thorpe, C.E.: Overtaking Vehicle Detection Using Implicit Optical Flow. In: IEEE Intelligent Transportation System, Boston, pp. 729–734 (1997) 4. Krips, M., Velten, J., Kummert, A., Teuner, A.: AdTM Tracking For Blind Spot Collision Avoidance. In: IEEE Intelligent Vehicles Symposium, pp. 704–709 (2004) 5. Ehlgen, T., Pajdla, T., Ammon, D.: Eliminating Blind Spots for Assisted Driving. IEEE Transactions on Intelligent Transportation Systems 9, 657–665 (2008) 6. Jeong, S., Ban, S.W., Lee, M.: Autonomous Detector Using Saliency Map Model and Modified Mean-Shift Tracking for a Blind Spot Monitor in a Car. In: 2008 7th International Conference on Machine Learning and Applications, San Diego, pp. 253–258 (2008)

Design of Autonomous Parallel Parking Using Fuzzy Logic Controller with Feed-Forward Compensation Bo-Chiuan Chen1, Yi-Wen Huang1, Shiuh-Jer Huang2, and Bo-Jhao Liu1 2

1 Vehicle Engineering, National Taipei University of Technology, Taiwan Mechanical Engineering, National Taiwan University of Science and Technology, Taiwan [email protected]

Abstract. A fuzzy logic controller (FLC) with feed-forward Compensation is proposed for autonomous parallel parking in this paper. Two ultrasonic radars are used to detect the parking space. The law of cosines is used to increase the accuracy of the corner detection of the parking space. Triangular function is applied to obtain the desired trajectory, orientation angle, and feed-forward steering angle. Trajectory estimation is used to obtain the lateral position error and orientation error with respect to the reference trajectory. These two errors are inputs to the FLC and the control output is the steering angle correction. Preliminary experimental results show that the proposed FLC with feed-forward compensation can achieve better trajectory following and reduce the length requirement of parking space. Keywords: Autonomous, Parallel parking, Ultrasonic radar, Trajectory following, Feed-forward compensation, Fuzzy logic controller.

1 Introduction Parking is not an easy task for inexperienced drivers, especially in crowded cities. Due to the limited view angles of the driver’s vision, various sensors such as ultrasonic sensor, laser scanner, and video camera have been employed to develop parking aid system (PAS) to assist the driver in the parking maneuver. For parking guidance, the PAS consists of parking space detection, trajectory planning, and trajectory estimation for steering guidance. For autonomous parking, the electric power steering (EPS) is required to control the steering wheel for trajectory following. Degerman el al. [1] proposed a parking space estimation algorithm using a single ultrasonic sensor. A weighted version of Hough transform is employed to detect the line of outer edge of two parked vehicles and the line at the sonar horizon or the curb. The other two lines perpendicular to the parked vehicles are found using the statistical change detection. Jung et al. [2] developed a corner detection method using laser scanner. Their method can deal with the rectangular and round corner detection. Xu et al. [3] developed the color segmentation method based on the RCE (restricted coulomb energy) neural network. Their method can find the segmented markings of Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 687–694. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

688

B.-C. Chen et al.

parking slots. Inverse perspective mapping is then applied to find the position and orientation of the empty parking slot. Laumond et al. [4] proposed a motion planner for nonholonomic car-like robot. A collision-free path is first planed for the geometric system. Subdivision is then performed to satisfy the nonholonomic and curvature constraints. Optimization is employed to remove extra maneuvers and reduce the length of the path. Lyon [5] proposed a static curvilinear path generation for parallel parking with curvature and nonholonomic constrains. The path is a fifth-order polynomial which can result in zero steering and orientation angle at the end point. Inoue et al. [6] employed the circle with the minimum rotating radius and the associated tangent line to plan the desired trajectories for both garage and parallel parking maneuvers. Guldner and Utkin [7] proposed a Lyapunov function to prescribe a set of desired trajectories. They employed sliding model control to achieve ideal tracking via controlling the steering and vehicle speed at the same time. Ohata and Mio [8] proposed a nonlinear trajectory control combined with changing coordinates. Their approach is independent of vehicle speed and can guide the car to an appointed parking position from any position and posture angle. Zhu and Rajamani [9] developed the desired trajectory by considering the constraints of configuration, actuation, and collision. They assumed constant vehicle speed and employed inputstate feedback linearization to track the desired lateral position and vehicle orientation which are functions of the longitudinal displacement. A differential global position system (GPS) is used to obtain the position and orientation of the vehicle for the experimental vehicle. Their experimental results show that the vehicle is able to achieve the spacing accuracy around 40 cm. Sanchez et al. [10] proposed an adaptive fuzzy control system for automatic path tracking. A multivariable closed loop fuzzy controller controls the steering angle and vehicle speed to reach a goal point in the path. The fuzzy goal point selector determines the point in the path to track according to the current position, orientation, and path characteristics. Hu and Yang [11] proposed a fuzzy neural dynamics based tracking controller with the consideration of the nonholonomic constraints. Their approach can round off sharp terns with a smooth curve and generate the tracking velocities according to the path errors. Li and Chang [12] proposed an autonomous fuzzy logic controller for both garage and parallel parking. Image processing is employed to obtain the position and orientation of the vehicle for the controller to generate the appropriate steering commands. In reality, the driver is responsible for the safety during parking and should control the braking pedal to avoid moving objects in the parking space, i.e. the steering angle is the only available input for trajectory following. In order to solve this problem, a fuzzy logic controller (FLC) with feed-forward compensation is proposed for autonomous parallel parking in this paper. Two ultrasonic radars are used to detect the parking space. The law of cosines is used to enhance the accuracy of the corner detection of the parking space. Triangular function is applied to obtain the desired trajectory, orientation angle, and feed-forward steering angle. Trajectory estimation is used to obtain the lateral position error and orientation error with respect to the reference trajectory. These two errors are inputs to the FLC and the control output is the steering angle correction. The remainder of this paper is organized as follows: parking space detection is described in Section 2, followed by the proposed control in Section 3. Experimental results are presented in Section 4. Finally, conclusions are made in Section 5.

Design of Autonomous Parallel Parking Using FLC with Feed-Forward Compensation

689

2 Parking Space Detection For ultrasonic parking aid system, multiple transmitters emit the sonic pulses at the same time. If one of the multiple receivers detects the echo and the calculated distance is less than the threshold value, an audio alarm is sent to the driver. However, it is impossible to know which radar detects the object unless the ultrasonic beam is very narrow. Kuo and Chen [13] proposed a sensor array by activating the ultrasonic radars sequentially. The sample rate is reduced to half of the original rate to achieve the purpose of sequential scanning. Their approach can identify which sensor detects the object. However, if the object is detected by multiple sensors, they cannot determine the position of the object relative to these sensors. parking space

parked vehicle

A

D1 y

D2

Į x

R2

D0

R1

Fig. 1. Corner detection of parking space

In order to solve this problem, Huang et al. [14] employed the law of cosines to take into account of the distance between the radars of R1 and R2, i.e. D0, as shown in Fig. 1. D1 and D2 denote for the distance measured by the radars of R1 and R2, respectively. If the object is detected in the overlapped range of two radars, i.e. point A, we can obtain α as shown below.

α = cos −1

D12 + D0 2 − D2 2 2 D1 D0

(1)

The position of the point A relative to R1 can then be expressed as follows.

x A = D1 cos α and y A = D1 sin α

(2)

If D0, D1, and D2 do not satisfy the triangle inequality, the shorter one of D1 and D2 is used as the relative position in the y direction, i.e. y A = min( D1 , D2 ) at x A = 0.5D0 . If the space is large enough for parking, the system will indicate the driver that autonomous parallel parking can be executed.

3 Trajectory Following Control The block diagram of the proposed FLC with feed-forward compensation is shown in Fig. 2. After detecting the parking space, the desired trajectory, orientation angle, and feed-forward steering angle can be obtained from the trajectory planner. Trajectory estimation is used to obtain the lateral position error and orientation error with respect

690

B.-C. Chen et al.

to the reference trajectory. These two errors are inputs to the FLC and the control output is the steering angle correction.

Gd yd , T d + А

geT

Knowledge base

gey

Decision logic

eT

ey

yˆ r ,Tˆ xˆr

Trajectory Planner

Feed-forward compensation Defuzzification

+ +

' G FLC

G

'sr , 'sd

Trajectory Estimation

Vehicle

Fig. 2. Block diagram of the proposed trajectory following control hx

y

( X f ,Y f )

h

hy

c

(Bx , By ) ( X i ,Yi )

( Ax , Ay )

Y

x X

Fig. 3. Desired trajectory for parallel parking

3.1 Trajectory Planning

After obtaining the corner positions, width, and length of the parking space relative the vehicle, we can start to plan the desired trajectory to be followed (see Fig. 3). The configuration constraint is to have zero orientation angles at both initial and final points of the desired trajectory as follows.

θi = 0 at ( X i , Yi ) and θ f = 0 at ( X f , Y f )

(3)

where θ is the orientation angle of the vehicle; X and Y are the longitudinal and lateral positions of the center of the rear axle; the subscripts i and f denote for the initial and final points, respectively. If the initial point is selected as the origin, i.e. ( X i , Yi ) = (0, 0) , we can then obtain the final point as ( Ax − hx , Ay + hy ) . ( Ax , Ay ) is the coordinate of the corner position; hx is the desired clearance between the final point and the rear parked vehicle; hy is one half of the track width of the vehicle. Since the vehicle speed is controlled by the driver, the only actuation constraint is the steering angle δ , i.e. δ min ≤ δ ≤ δ max . The triangular function [9] used to design the desired trajectory can be expressed as follows.

yd = a cos(ω x + ψ ) + b

(4)

where a = b = 0.5Y f ; ω = π X f ; and ψ = π . We can then obtain the orientation angle of the desired trajectory as follows.

Design of Autonomous Parallel Parking Using FLC with Feed-Forward Compensation

691

θ d = tan −1 ( y ' ) = tan −1[−aω sin(ω x +ψ )]

(5)

3.2 Trajectory Estimation

For closed-loop trajectory following control, it is required to estimate the vehicle trajectory as shown below. n

n

i =0

i=0

xˆr = ∑ cos θˆi Δsi and yˆ r = ∑ sin θˆi Δsi

(6)

where ( xr , yr ) is the coordinate of the center of the rear axle; the hat denotes for estimated variable; L is the wheelbase; the subscript i denote for the ith sample data; n is the number of samples during the parking maneuver; Δs is the displacement of the vehicle during on sample interval. The estimated orientation angle is n

ΔsR ,i − ΔsL ,i

i =0

lr

θˆ = ∑

(7)

where lr is the track width; ΔsR and ΔsL are the displacements of the right and left rear wheels, respectively. The feed-forward compensation can be obtained as follows. 3

δ d = tan −1[ L y '' (1 + y ' 2 ) 2 ]

(8)

Table 1. Fuzzy control rules Δδ FLC ey eθ

NB

NM

NS

ZE

PS

PM

PB

NB NM NS ZE PS PM PB

NB NB NB NM NM NS ZE

NB NB NM NM NS ZE PS

NB NM NM NS ZE PS PM

NM NS NS ZE PS PM PM

NM NS ZE PS PM PM PB

NS ZE PS PM PM PB PB

ZE PS PM PM PB PB PB

3.3 Fuzzy Logic Controller

The lateral position error ey = yd − yˆ r and orientation error eθ = θ d − θˆ are inputs and the steering angle correction Δδ FLC is the control output of the FLC. 49 fuzzy rules are shown in Table 1. Since the lateral position error and orientation error stimulate two fuzzy subsets of the universe of e y and eθ , respectively, only four fuzzy control rules are fired in each sampling interval. The height method is employed for defuzzification to obtain the steering angle correction as follows.

692

B.-C. Chen et al.

Δδ FLC = ∑ ωi ui

∑ω

i

, ωi = ∏ μ Aij ( x oj ) j

(9)

where μ Aij ( x oj ) is the linguistic value of the fuzzy set variable and ui is the resulting fuzzy control value of the ith fuzzy rule.

4 Experimental Results Suzuki Swift is used as the prototype vehicle for autonomous parallel parking in this paper. The overall system configuration is shown in Fig. 4. Hall sensors and wheel drums modified with teeth are used to measure the angular displacement of the right and left rear wheels. Host PC is used to design the trajectory following controller to be downloaded to the target PC. MC240 control card is used to control the EPS unit equipped on the prototype vehicle. Engine SWIFT Electric power steering in AUTOPARKING USING 1.position cotrol by PI control 2.current control by adaptive digital PI control Bojhao.Liou 2009/7

0

10

Co n st a n t2

M a n u al S wit ch 3

x 4

x

y

Xf

0

8

c o m p e nsa t or

Prototype Vehicle

[p w m _ r]

[ du t y_ r] [ re f_ cu r re n t]

d ut y i_r

referenc e of Sine W ave 1 I n1

kp [ ia ]

an le_p osit ion

de g /s to ra d /s

[ fe d _ a ng l e ]

[ du t y_ l]

PI

9

phi_d

[d u t y]

M an u a l Swi tc h1 [ re f_ a n g le ]

y_d

phi _d

20

[a n g le _ e rr or ]

M a n u a l S wi tch

Ey 7 E phi

y_d

?? ?

0

ref erence of Si ne Wave 2

6

phi y_error phi_error Yf

Yf

xP C-M C240 Init ial izat ion

M a n ua l Sw itc h 2

Co n sta n t3

5 y aw angle

y Xf

3 .1

steeri ng angl e

3

delta

6 .4

[u ]

[wn ]

motor rad/s

?? ?

[ki ]

V_ ba t u

15

[p w m _ l]

| u| ki

[v _ ba t ] -K an g le sen so r

[kp ]

ia

PI co n tr o l po si tio n

s pee d

traje ctory tracki ng & estima ti on

0 [a ]

a

[b ]

b

va

[ va ]

x PC-MC240_PWM v _d riv er [ia ]

ia

[v a]

a

Va

ve m f c_s en sor

v _ bat

[c _se n so r] [ wn ]

Wm

[i a ]

ia

[v_ d ri ve r]

[a ]

b

[b ]

[c]

c

[ c]

[wn ]

[v em f]

c cd ivid eb

[ cd iv id e b ]

[ v_ b a t]

S te p 1 [ do t _r ]

wn ie rr or

me a_ bat

[ m e a _b a t ]

e rr or t enh a_ia

[ er ro r ]

?? ? n u m ( z) Out 1

d e n (z) e stim at ion _c urr en t

1 digit al input

a da p ti ve di g ita l PI c on t ro l I n1

[ te n h z_ ia ] [cu r re n t_ e rr o r]

d igit al in put

[e sti m a ti o n_ c ur re n t]

[d o t_ l ]

xPC -MC 240_DO

1 HZ Fil te r 1 m s lo se e ff ic ac y l o g ic u ni t 2

?? ? Fr o m 3 RL S Esti m a to r

co unt er

[ cu rr e nt _ er ro r ]

2

x PC-MC240_DO1

c ounter c ur re n t a ns b at te r y

Trajectory following

Host PC

Steering Angle Sensor

TCP/IP

Suzuki Swift Ultrasonic Radar

Target PC CAN-PCI Wheel Speed Sensor

EPS Unit

MC240

Fig. 4. System configuration of the prototype vehicle

The parking space detection result is shown in Fig. 5 (a). ‘x’ and ‘o’ denote for the range data when the law of cosines is applicable and non-applicable, respectively. The detected space is 16 cm shorter than the actual available space. GPS is used to verify the estimated trajectory as shown in Fig. 5 (b). The position error of the final point is 2 cm and 6 cm in the longitudinal and lateral directions, respectively. The trajectory following results of the proposed control are shown in Fig. 6. The proposed control with feed-forward compensation (FF+Fuzzy) can track the desired trajectory closely. On the contrary, the open-loop control, i.e. feed-forward (FF) compensation only, deviates from the desired trajectory gradually. The proposed control can achieve smaller tracking error for the orientation angle. If we continue the experiment until the orientation angle is zero, the distances between the rear bumper of the prototype vehicle and the front bumper of the rear parked vehicle are 0.13 m and 0.40 m for the open-loop control and the proposed control, respectively. Large trajectory error and orientation error for the open-loop control might be resulted from the following reasons: (1) the kinematic equations are not valid with the existence of tire compliance; (2) the desired trajectory is designed assuming constant vehicle speed which is not true during the real parking maneuver; (3) the steering angle is controlled by the EPS system which might introduce additional uncertainty due to the traction error of the angular position.

Design of Autonomous Parallel Parking Using FLC with Feed-Forward Compensation

(a) Parking Space Detection

(b) Trajectory Estimation Estimated GPS End point

0.5 True Boundary 1 Boundary 2 0 Data-cos Data-n/a -0.5

5

Y (m)

4

Y (m)

3

5.6 m

2

693

-1 -1.5

1

-2

0 0

2

4

X (m)

6

8

10

-2.50

2

4 X (m)

6

8

Fig. 5. (a) Parking space detection; (b) Trajectory estimation (a) Trajectory

(b) Orientation Angle 30

3

25

Angle (deg)

Y (m)

35 4

2 1 Desired FF FF+Fuzzy

0

2

3

4 X (m)

5

6

15 10 5 0

-1 0

20

7

-5 0

1

2

3 X (m)

4

5

6

Fig. 6. Trajectory following results

5 Conclusion A FLC with feed-forward Compensation is proposed for autonomous parallel parking in this paper. Two ultrasonic radars are used to detect the parking space. The law of cosines is used to increase the accuracy of the corner detection of the parking space. Triangular function is applied to obtain the desired trajectory, orientation angle, and feed-forward steering angle. Hall sensors with wheel drums modified with teeth are used to obtain the estimated trajectory. Estimated lateral position error and orientation error are inputs to the FLC and the control output is the steering angle correction. Preliminary experimental results show that the proposed FLC with feed-forward compensation can mitigate the problems of tire compliance, variable vehicle speed, and EPS tracking errors. Thus achieve better trajectory following and smaller orientation error, and reduce the length requirement of parking space. Acknowledgements. This project was supported by the National Science Council in Taiwan under the contract No. of NSC 98-2752-E-027-001-PAE.

References 1. Degerman, P., Pohl, J., Sethson, M.: Hough Transform for Parking Space Estimation Using Long Range Ultrasonic Sensors. SAE Paper No. 2006-01-0810 (2006) 2. Jung, H.G., Cho, Y.H., Yoon, P.J., Kim, J.: Scanning Laser Radar-based Target Position Designation for Parking Aid System. IEEE Transactions on Intelligent Transportation Systems 9(3), 406–424 (2008)

694

B.-C. Chen et al.

3. Xu, J., Chen, G., Xie, M.: Vision-guided Automatic Parking for Smart Car. In: Proceedings of the IEEE Intelligent Vehicles Symposium, pp. 725–730 (2000) 4. Laumand, J.-P., Jacobs, P.E., Taix, M., Murray, R.M.: A Motion Planner for Nonholonomic Mobile Robots. IEEE Transactions on Intelligent Transportation Systems 10(5), 577–593 (1994) 5. Lyon, D.: Parallel Parking with Curvature and Nonholonomic Constraints. In: Proceedings of the IEEE Intelligent Vehicles Symposium, pp. 341–346 (1992) 6. Inoue, T., Dao, M.Q., Liu, K.-Z.: Development of an Auto-parking System with Physical Limitations. In: Proceedings of SICE Annual Conference, pp. 2049–2054 (2004) 7. Guldner, J., Utkin, V.I.: Stabilization of Non-holonomic Mobile Robots using Lyapunov Functions for Navigation and Sliding Mode Control. In: Proceedings of IEEE Conference on Decision and Control, pp. 2967–2972 (1994) 8. Ohata, A., Mio, M.: Parking Control Based on Nonlinear Trajectory Control for Low Speed Vehicles. In: IECON Proceedings, pp. 107–112 (1991) 9. Zhu, C., Rajamani, R.: Global Positioning System-based Vehicle Control for Automated Parking. Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering 220(1), 37–52 (2006) 10. Sanchez, O., Ollero, A., Heredia, G.: Adaptive Fuzzy Control for Automatic Path Tracking of Outdoor Mobile Robots. In: Proceedings of IEEE International Conference on Fuzzy Systems, pp. 593–599 (1997) 11. Hu, Y., Yang, S.X.: A Fuzzy Neural Dynamics Based Tracking Controller for a Nonholonomic Mobile Robot. In: Proceedings of IEEE/ASME International Conference on Advanced Intelligent Mechatronics, pp. 205–210 (2003) 12. Li, T.-H.S., Chang, S.-J.: Autonomous Fuzzy Parking Control of a Car-like Mobile Robot. IEEE Transactions on Systems, Man, and Cybernetics Part A: Systems and Humans 33(4), 451–465 (2003) 13. Kuo, Y.-T., Chen, B.-C.: Design of Side Collision Warning System. In: Proceedings of the 9th National Conference on Vehicle Engineering, Taoyuan, Taiwan (2004) 14. Chao, S.-T., Huang, S.-J.: Application of Ultrasonic Sensors for Vehicle Lateral Impact Warning System. Chinese Society of Automotive Engineering. Journal of Vehicle Engineering 5, 1–14 (2008)

Telematics Services through Mobile Agents Jonathan Lee, Shin-Jie Lee, Hsi-Min Chen, and Wen-Tin Lee Department of Computer Science and Information Engineering, National Central University, Taiwan {yjlee, jielee, seeme, wtlee}@selab.csie.ncu.edu.tw

Abstract. OSGi is considered as one of the fundamental technologies underpinning telematics services, and has attracted both academic and commercial interest. Because of a desire and mandates to deliver telematics services on OSGi environments, professionals have identified a key issue that need to be addressed: the service delivery would suffer from the situation of low bandwidth and unstable links. In this work, we benefit from the inherent characteristics of mobile agents and develop OSGi-Based mobile agents to address this challenge. Furthermore, the mobile agents are enhanced with a Java bytecode extractor which makes a mobile agent partially reside in a contactless smart card for service invocations, and with a risk-enabled reputation model which supports a mobile agent for information filtering to reduce the size of data that need to be carried. Keywords: OSGi, Mobile agent, Reputation model, Telematics service.

1

Introduction

OSGi (Open Services Gateway initiative) as a standardization of advanced service-oriented and component-based environment for managing the software lifecycle has spanned applications in numerous domains, such as telematics [1], enterprise software infrastructure [2], Eclipse plugin development [3], home gateways [4,5] and mobile devices [6]. In telematics domain, recently, OSGi has being tailored and extended in order to meet vehicle-specific requirements by OSGi Vehicle Expert Group. However, there exists two challenges that need to be addressed by the professionals: – To deliver services under low bandwidth: Under a telematics environment, services may be delivered from infrastructures or Internet to a mobile device through wireless networks with low bandwidth. However, extant methods of accessing remote services consumes fairly high network bandwidth as the number of accessed remote services increases. – To deliver services through unstable links: It is recognized that network links would become more unstable under a telematics environment, especially over a vehicle in moving. Through the traditional Web services approach, however, it would become inefficient in re-sending/receiving data again from scratch after reconnections. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 695–702. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com 

696

J. Lee et al.

Because OSGi enables a flexible framework to integrate software components based on object’s method invocation mechanism, it impedes the fostering of the integrations of services located over different platforms. Although several efforts, such as R-OSGi [7], D-OSGi [8], SOAP-based OSGi [9], RMI-based OSGi [10] and P2P-based OSGi [11], have been made to address this problem, they still suffer from the aforementioned challenges. In this paper, we present our recent work on delivering telematics services through mobile agents [12,13,14,15,16,17]. we benefit from the inherent characteristics of mobile agents and develop OSGi-Based mobile agents to address the two challenges. Furthermore, the mobile agents are enhanced with a Java bytecode extractor which makes a mobile agent partially reside in a contactless smart card for service invocations, and with a risk-enabled reputation model which supports a mobile agent for information filtering to reduce the size of data that need to be carried. This paper is organized as follows. We first describe the OSGi-based mobile agents in section 2. A Java Bytecode extractor for making a mobile agent partially reside in a contactless smart card is introduced in section 3. A risk-enabled reputation model (REAL) that can be used by the mobile agent for information filtering is elaborated in section 4. Finally, we summarize this work in section 5.

2

Telematics Services Delivering through OSGi-Based Mobile Agents

In OSGi framework, a software component is implemented as an OSGi bundle that can be downloaded, installed, started, stopped or uninstalled on an OSGi platform on the fly. Although the communications among bundles can be well managed through registry and method invocation mechanisms, service invocation among bundles over different platforms is still not enabled. In this work, we propose an OSGi-based mobile agent approach to delivering services across distributed OSGi platforms to overcome this limitation [13]. A mobile agent is designed as an OSGi bundle by enhancing the bundle with the capability of mobilization. The capability is implemented as a Java package that can be used by developers to program mobile agents. An OSGi-based mobile agent can move over distributed OSGi platforms, access remote services and resources, perform task executions, and then carry the results back to the user. Figure 1 shows the lifecycle of an OSGi-based mobile agent. As a bundle is installed on an OSGi platform, the bundle will be in Installed state and then can be resolved in Resolved state through checking its manifest description. After that, the bundle can be trigger to start and be in Starting state. After the bundle is started, it will be in Active state. During the activation of a bundle, the bundle can mobilize to another OSGi platform through the mobilization APIs. The instance of the bundle will be serialized together with its bytecode and transmitted to the target platform. After the transmission, the bundle will be stopped and uninstalled. As the target platform receives the serialization, the bytecode part will be used to install the bundle, and then the instance part will be de-serialized to start the bundle.

Telematics Services through Mobile Agents

697

update refresh

install

Starting g

Installed start resolve uninstall

update refresh

Resolved

mobilize

Active

on-mobilized

Seriali ing Serializing De-serializing

uninstall

Transmitting stop

Uninstalled Stopping

Fig. 1. Lifecycle of an OSGi-based mobile agent

In contrast to object-level serialization exploited by other mobile agent systems for mobile agent transmission, our approach serializes and de-serializes mobile agents at bundle level so as to comply with the OSGi specification. To serialize a mobile agent instance before transmission is first constructed. Secondly, the output data obtained from service invocation are attached to the constructed bundle. Finally, the bundle is transmitted to the next OSGi platform. After the mobile agent bundle arrives at the next OSGi platform, it is installed and started in the same way as a general OSGi bundle, and in turn de-serializes the carried data for service invocation. Since a mobile agent is implemented as a bundle, it can be seamlessly integrated into OSGi without any modification of underlying framework.

3

Mobile Agents in Contactless Smart Cards through a Java Bytecode Extractor

In the extant applications, contactless smart cards are widely used to preserve private personal data, such as users’ ID, credit, or password. But what if we need to preserve programs in a contactless smart card? This problem arise naturally in mobile agents. As agent technologies emerge nowadays, a mobile agent can act on behalf of a user autonomously. However, a user’s proprietary information could be documented in the programs of a mobile agent instead of being kept in the instance of the mobile agent, which exposes the issue of how to protect the pieces of the programs, especially on a mobile device or an OBU, before a mobile agent is instantiated. In this work, we develop a tool (jExtractor) to extract the Code part of Java bytecode, which may contain some private information, from a Java class in a configurable way [14]. The tool also has been applied on the OSGi-based mobile

698

J. Lee et al.

Agent.class

jExtractor (extract)

Agent.Stub

Code

serialize

Code jExtractor ((merge) g )

deserialize

Contactless S Smartt Card C d

Agent.class

Fig. 2. Moving Java bytecode to contactless smart card

agent for building an emergency rescue application. Through jExtractor, the extracted part can be serialized and moved to a contactless smart card. When this class needs to be instantiated, jExtractor can merge the remaining part of the Java class with the extracted part to be the original one. Moreover, we use 86808 Java classes to validate jExtractor. In the experimental result, 81.7% of samples can successfully be extracted and moved to Mifare Standard S50 (1K) smart card with the highest configuration level. Figure 3 shows the concept of using jExtractor to move Java bytecode to a contactless smart card. jExtractor can extract a Code part, the body of a method that could contain proprietary algorithms for a user, from a Java class and then store the part in a contactless smart card through serialization. In addition, the part in the class will be erased, and the modified class (Agent.Stub) will not be able to be instantiated. Once the Java class needs to be instantiated, jExtractor can merge the modified class with the Code part from the contactless smart card to be the original Java class.

4

Information Filtering through a Risk-Enabled Reputation Model

Recently, a trend to use reputation model for selecting service providers becomes evident [18,19,20]. Most of the effort have been made on providing a reliability value associated with the predicted reputation to interpret how reliable the reputation is. In REGRET [21], a reliability value is provided together with a reputation to interpret how reliable the reputation is. However, the predicted reputation will become ineffective with a low reliability value. Rather than providing a reliability value, we propose a reputation model, called Risk-Enabled reputAtion modeL (REAL) [12], by introducing the notion

Telematics Services through Mobile Agents

699

of risks to better reflect the recent performance of a service provider. REAL predicts a service provider’s reputation together with a risk value. The predicted reputation has been validated as a better model to reflect a service provider’s recent performance than given by the extant on-line auctions sites or the models using the technique of adopting mean values. Moreover, the risk value can further identify how risky it is to transact with the service provider. REAL can be used by a mobile agent in an information center to filter out service providers with high risks. After the filtering, the information carried by the mobile agent will be more concise for a telematics environment with limited resources. For example, a mobile agent can move from a mobile device to an information center and then search for the sellers with low risks for its user through REAL. REAL works as follows: 1. To record ratings. Rating, as the basic ingredient of the reputation model, records the service requester’s impression of a service provider of a subject at a particular point of time. A rating is defined as a tuple r = a, b, s, t, v, where a is a service requester; b is a service provider judged by a with respect to subject s; t is the time when the judgment takes place; and v is the impression as a result of the judgment. This step is to record all the ratings for the analysis in the next step. 2. To discover active ratings. This step is to discover active ratings that are the ratings of the service provider’s recent performance, which can be further broken down into the following activities. Firstly, all the recorded ratings are clustered by the frequencies of the ratings in various durations based on a density-based clustering algorithm [22]. Secondly, the clusters that represent the recent performance are selected as active clusters. Finally, a time point will be identified and used to specify which ratings in the selected clusters can be used to calculate the reputation and its associated risk. 3. To calculate reputation and risk. In this step, the reputation is identified as the greatest rating value of the selected ratings, and a risk value is calculated as the sum of the products of two values for each rating value less than the calculated reputation: (1) the probability of the rating value; and (2) the difference between the calculated reputation and the rating value. Figure 3 shows an experimental result of using REAL to analyze sellers in Amazon Auctions. Three sellers (represented as A, B and C) are with 213, 423 and 252 ratings, respectively. The left three plots show the distributions of the ratings of sellers A, B and C. By visualizing these plots, we can observe that bad performance (characterized by 1 star and 2 stars) of seller A were more active in the recent 50 days than before; bad performance of seller B were inactive in the recent 70 days; and, the performance of seller C were excellent (5 stars) in the recent 50 days. In Amazon Auctions, a seller’s reputation is calculated as the average of his past rating values. In the experiment, the reputations of the three sellers given by Amazon Auctions are equal to 4.7 stars. By considering the weight of rating time into account, the reputations calculated by REGRET are 4.6, 4.7 and 4.7 stars,

700

J. Lee et al. taligned=707704

Seller A ˈ

Amazon.com REGRET

ˇ

REAL

ˆ

Rep: 4.7

˅

Rep: 4.6

˄ ˊ˃ˊˇ˅˃

ˊ˃ˊˇˊ˃

ˊ˃ˊˈ˅˃

ˊ˃ˊˈˊ˃

ˊ˃ˊˉ˅˃

ˊ˃ˊˉˊ˃

Seller B

ˊ˃ˊˊ˅˃

taligned=707637

ˈ ˇ ˆ ˅

Amazon.com REGRET

REAL

Rep: 4.7

Rep: 5 Risk5: 0.47

Rep: 4.7

˄ ˊ˃ˉˉ˃˃

ˊ˃ˉˋ˃˃

ˊ˃ˊ˃˃˃

ˊ˃ˊ˅˃˃

Rep: 5 Risk5: 0.85

ˊ˃ˊˇ˃˃

ˊ˃ˊˉ˃˃

Seller C

ˊ˃ˊˋ˃˃

taligned=707698

Amazon.com REGRET

ˈ

REAL

ˇ

Rep: 4.7

ˆ ˅ ˄ ˊ˃ˊ˃˃˃

ˊ˃ˊ˄˃˃

ˊ˃ˊ˅˃˃

ˊ˃ˊˆ˃˃

ˊ˃ˊˇ˃˃

ˊ˃ˊˈ˃˃

ˊ˃ˊˉ˃˃

ˊ˃ˊˊ˃˃

Rep: 4.7

Rep: 5 Risk5: 0

ˊ˃ˊˋ˃˃

Fig. 3. Comparison of REAL, REGRET and Amazon Auctions approaches

respectively. By applying REAL, the reputations of A, B and C are calculated as 5, which represents the three sellers did have 5 stars performance recently. In addition, each reputation is associated with a risk value. The risks of the reputations of A, B and C are calculated as 0.85, 0.47 and 0, which means a buyer will take more risk to transact with A than B or C. Therefore, the information provided by REAL will help a buyer distinguish among sellers with respect to their recent performance.

5

Conclusion

In this paper, we present three key approaches in our recent work to dealing with telematics service delivery. In OSGi-based mobile agents, an agent is implemented as an OSGi bundle, which enables a mobile agent to be seamlessly integrated into OSGi without any modification of underlying framework. The experimental results show that the performance of the approach is superior to that of centralized client-server approaches in terms of turnaround time, memory usage and network traffic. In addition, the Code parts of a mobile agent proprietary to a user can be extracted through the Java bytecode extractor, which makes the mobile agent be able to partially reside in a contactless smart card of the user. The experimental results also show that 81.7% of samples can successfully be extracted and moved to Mifare Standard S50 (1K) smart card. Moreover, the risk-enabled reputation model (REAL) can be used by a mobile agent to filter out service providers with high risks, which reduces the size of the information that need be carried by the mobile agent. REAL has been validated as a better model to reflect a service provider’s recent performance than given by the extant on-line auctions sites or the models using the technique of mean values.

Telematics Services through Mobile Agents

701

Acknowledgments. This research was sponsored by National Science Council (Taiwan) under the grant 98-2752-E-008-001-PAE.

References 1. Kim, M., Choi, Y., Moon, Y., Kim, S., Kwon, O.C.: Design and Implementation of Status Based Application Manager for Telematics. In: 8th International Conference Advanced Communication Technology, pp. 1364–1366. IEEE Press, Korea (2006) 2. Herault, C., Thomas, G., Lalanda, P.: A Distributed Service-Oriented Mediation Tool. In: IEEE International Conference on Services Computing, pp. 403–409. IEEE Press, Utah (2007) 3. Gruber, O., Hargrave, B., McAffer, J., Rapicault, P., Watson, T.: The Eclipse 3.0 Platform: Adopting OSGi Technology. IBM Systems Journal 44, 289–299 (2005) 4. Dobrev, P., Famolari, D., Kurzke, C., Miller, B.: Device and Service Discovery in Home Networks with OSGi. IEEE Communications Magazine 40(8), 86–92 (2002) 5. Wu, C.-L., Liao, C.-F., Fu, L.-C.: Service-oriented Smart-Home Architecture based on OSGi and Mobile-agent Technology. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 37(2), 193–205 (2007) 6. Brown, A., Kolberg, M., Bushmitch, D., Lomako, G., Ma, M.: A SIP-based OSGi Device Communication Service for Mobile Personal Area Networks. In: 3rd IEEE Consumer Communications and Networking Conference, pp. 502–508. IEEE Press, Las Vegas (2006) 7. Rellermeyer, J., Alonso, G., Roscoe, T.: R-OSGi: Distributed Applications through Software Modularization. In: Cerqueira, R., Campbell, R.H. (eds.) Middleware 2007. LNCS, vol. 4834, pp. 1–20. Springer, Heidelberg (2007) 8. Ibrahim, A., Zhao, L.: Supporting the OSGi Service Platform with Mobility and Service Distribution in Ubiquitous Home Environments. The Computer Journal 52(2), 210–239 (2009) 9. Knopflerfish 3, OSGi R4, http://www.knopflerfish.org/ 10. Yamasaki, I., Yata, K., Maeomichi, H., Tsutsui, A., Kawamura, R.: Implementation of A Distributed Network Middleware “CSC” on OSGi Frameworks. In: Second IEEE Consumer Communications and Networking Conference, pp. 150–155. IEEE Press, Las Vegas (2005) 11. Ferscha, A., Hechinger, M., Mayrhofer, R., Oberhauser, R.: A Lightweight Component Model for Peer-to-peer Applications. In: 24th International Conference on Distributed Computing Systems Workshops, pp. 520–527. IEEE Press, Japan (2004) 12. Lee, J., Lee, S.-J.: A Risk-Enabled Reputation Model for E-Commmerce. In: IEEE International Conference on Systems, Man, and Cybernetics, pp. 619–624. IEEE Press, Singapore (2008) 13. Lee, J., Lee, S.-J., Chen, Y.-L., Chen, H.-M., Wang, P.-F.: Design and Implementation of OSGi-Based Mobile Agent. In: Fifth Taiwan Conference on Software Engineering, SEAT, Taiwan (2009) 14. Lee, J., Lee, S.-J., Liang, J., Chen, Y.-L.: Moving Java Bytecode to Contactless Smart Card: A Tool and its Application to Mobile Agents. In: 2009 National Computer Symposium, Taiwan (2009) 15. Lee, J., Lee, S.-J., Chen, H.-M.: Dynamic Role Binding with Agent-centric Contract Net Protocol in Agent Organizations. In: IEEE International Conference on Systems, Man, and Cybernetics, pp. 636–643. IEEE Press, Singapore (2008)

702

J. Lee et al.

16. Lee, J., Lee, S.-J., Wu, C.-L., Deng, W.-Y.: Extending Object-Oriented Programming with Roles for Open Multi-Agent Systems and Its Application to ATIS. In: IEEE International Conference on Systems, Man, and Cybernetics, pp. 446–451. IEEE Press, Taiwan (2006) 17. Lee, J., Wang, Y.-C., Wu, C.-L., Lee, S.-J., Ma, S.-P., Deng, W.-Y.: A Possibilistic Petri-Nets Based Service Matchmaker for Multi-Agent Systems. International Journal of Fuzzy Systems 7(4), 199–213 (2005) 18. Sabater, J., Sierra, C.: Review on Computational Trust and Reputation Models. Artificial Intelligence Review 24(1), 33–60 (2005) 19. Ramchurn, S.D., Huynh, D., Jennings, N.R.: Trust in Multi-agent Systems. The Knowledge Engineering Review 19(1), 1–25 (2004) 20. Zacharia, G., Moukas, A., Maes, P.: Collaborative Reputation Mechanisms in Electronic Marketplaces. In: 32nd Annual Hawaii International Conference on Systems Sciences. IEEE Press, Los Alamitos (1999) 21. Sabater, J., Sierra, C.: REGRET: A Reputation Model for Gregarious Societies. In: Fifth International Conference on Autonomous Agents, pp. 194–195. ACM Press, Canada (2002) 22. Ester, M., Kriegel, H.-P., Sander, J., Xu, X.: A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise. In: 2nd International Conference on Knowledge Discovery and Data Mining, pp. 226–231. AAAI Press, Portland (1996)

Multi-agent System Model for Urban Traffic Simulation and Optimizing Based on Random Walk Yu Cheng1, Tao Zhang1, and Jianfei Wang2 1

Navigation and Control Research Lab, Department of Automation, Tsinghua University, 2 Department of Computer Science, Tsinghua University, Beijing 100084, China {chengy05, taozhang}@mails.tsinghua.edu.cn, [email protected]

Abstract. Intelligent control and guiding of urban traffic syste-m become so important nowadays especially under the current traffic status, which is more and more congested and complicated. In this paper, we proposed a method of modeling the urban traffic flow system combining the global and local model information for the whole city net. We consider that the traffic digraph is composed from several nodes and those nodes are linked with routes lines. The proposed system is inspired by the random walk theory: for each traffic flow in the urban network, we simulated it with a random walk processing, with vehicle flow density and driver strategy independent. These flows only shared traffic lights and affected each other in the congestion situation. Finally we get simulator solution only by seeking the stable solutions of random walk. This intelligent system is very powerful and only the topology structure of city, the start and destination and numbers of each vehicles flow are known, it can return all of the optimized control strategy for each traffic light and driver in the traffic net. For evaluation, different road situations with various system parameters are simulated on the proposed system. The experiments results are satisfied and show the feasibility and robustness our system. Keywords: Multi-agent System, Urban Traffic Net, Simulation and Optimizing, Random walk.

1 Introduction Urban traffic net, whose traffic flow has stochastic character, is a large complex system affected by many factors [1]. The modeling and optimizing of urban traffic system is the key of the deep understanding of the whole traffic and better intelligent guiding and controlling of urban traffic system, especially under the current traffic status, which is more and more congested and complicated In previous literature, the modeling of the urban traffic flow system mainly focuses on the macro-model [2, 3]. and the micro-model [4] respectively. However, the macro traffic flow model can only describe the dense and uniform flow, but can not explain the phenomenon of Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 703–711. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

704

Y. Cheng, T. Zhang, and J. Wang

congestion, stat-stop and jam. The micro traffic flow model can only describe the high-density traffic flow about the local parts of the system, but can not give the whole understanding of the traffic flow [5]. E.Lopez.Neri etc [6] has prosed a multiagent model, descriping road components and behavior hierarchically, is based on the microscopic UTS system. This paper presents a multi-agent modeling methodology for describing traffic network, control signals and road network users. A digraphconsisting of several nodes is introduced and the traffic flow is simulated as a random walk processing. Comparing these previous methods, our approach is more comprehensive, which can model both local and global situation with a complex traffic network. Optimization and simulation are also convenient and feasiblet for it. In addition, multi-agent models can be easily adapted to include all UTS characteristics [7]. Our system can combine all the road elements to simulate any topology structure in daily life. Figure 1 presents the structure of crossing modeling process in out system.

Fig. 1. The overall structure of our proposed system

This following of our paper is organized as below: in section 2, the multi-level for the model is introduced with the proposed system in details. It also presents the optimized problem. Section 3 shows how we implement this optimization by simulated annealing algorithm. Then we present experimental results as well as discussion in Section 4 before come conclusion in Section 5.

2 Modeling of System In this section, this proposed model is introduced combined with UTS structure and road user behaviour. Structural information regards the network topology, geographical information, and transit signs (traffic lights, speed limit, etc.). The behavioral part concerns the road network users, the traffic lights, and the traffic flow. Both the structure and behaviors are represented in this model after some assumptions. 2.1 The Proposed System The urban traffic road net can be discrete and represented as a digraph G (V , E ) . For the straight road, we can be depicted as a directed chain vi1 → vi 2 → ... → vim and the

Multi-agent System Model for Urban Traffic Simulation and Optimizing

705

length parameter of it: m is proportional with the actual road length. For the intersections with several streets, we link to each two roads with traffic light node v , while the links between them represents the communication channels between those agents. Each node v has their own status and can communicate with adjacent nodes based on rules. These rules are: for each absolute node, representing a traffic light state Li (these lights include two statuses: green-pass or red-stop), if a vehicle in street A can pass this node (green state) to enter road B , we draw a directed edge from A to Li and from another edge from Li to B . For an intersection, if there is no traffic light limitation between roads A and B , the directed edge is drawn from A to B . For each node vi in the network graph, it has a road width parameter bi to present the capability of traffic flow in this node. In this system the bi is an important and independent parameter, affecting the traffic flow and congestion or jam mechanism greatly. Figure 2 and figure 3 show the straight road and intersection model respectively based on (1) and (2). Based on network G (V , E ) , it can establish all kinds of vehicles flow network on G (V , E ) with several nodes vi : the crossing structure, T-junction (shown in figure 4), one way street, two-way street, etc. Figure 4 illustrates the structure of T-junction in our system.

Fig. 2. The overall structure of the straight road representation

Fig. 3. The overall structure of the intersection structure in our model

In our system, the road users are mainly concerned with vehicles (drivers) and traffic lights, while pedestrians and cyclists are all neglected. It is reasonable as the driver and traffic light strategy affect the traffic status most. In anytime, there exist t vehicles flows with its start point Si to destination Di . We describe traffic density with

706

Y. Cheng, T. Zhang, and J. Wang

Fig. 4. The overall structure of the T-junction structure model

t vector group (W1 , W2 ,...,Wt ) and Wi is a vector with length | V | .The j-th element of Wi is (Wi ) j , which stands for probability of the i-th traffic flow that appears in node V j . If the i-th flow has N i vehicles in the system, (Wi ) j N i is the exact number of vehicles in node V j . It is obviously that |V |

∑ (W ) i

j =1

j

=1.

(1)

The driver own factor must be considered (in this paper, it is called: driving strategy). In our system, for each car flow from Si to Di , a | V | × | V | decision matrix Ri is the diver strategy. ( Ri ) jk is the element of Ri . Also, it has |V |

∑ (R ) k =1

i

jk

=1.

(2)

According to actual situation, we consider the road transfer rate between two agents Γ jk . Γ jk is defined as (3) and bk is road with at node vk , N i is the number of vehicle and (Wi ) k is traffic density. From (3) we can see the overall: when the traffic t

t

i =1

i =1

flow is low ( ∑ (Wi )k N i < bk ) the cars can transfer successfully but when ∑ (Wi )k N i ≥ bk , the transfer becomes more difficult.

Γ jk = min{1,

bk

}.

t

∑ (W ) i =1

i k

Ni

(3)

The optimized driving behaviour assumption: the driver will follow the optimized route in the system and make the best control decision in order to minimize the time spend. The traffic lights behaviour assumption: for traffic lights nodes, we need not to consider the exactly time of a certain state (pass or stop) last. Instead, the time proportional gi of this state is considered. Figure 5 shows this as an example.

Multi-agent System Model for Urban Traffic Simulation and Optimizing

707

Fig. 5. The time proportion

The transfer mechanism: the actual successful transfer rate from a agent to another is determined by the diver’s transfer strategy and the actual traffic situation. It is defined as (4). transfer rate=decision matrix × road transfer rate

(Ti ) jk

⎧ g j Γ jk ( Ri ) jk ( j ≠ k ) ⎪ . j −1 V =⎨ 1 − ( T ) − ( T ) ( j = k ) ⎪ ∑ i jl ∑ i jl l =1 l = j +1 ⎩

(4)

2.2 The Optimized Problem

Our system simulates the road net and finally it will get a stable state: in each unit time the number of traffic cars entering this network system is equal to the cars leaving the city. Then traffic density of each node in the network keeps stable. Using the multi-agent viewpoint, for a certain road agent node and the i-th traffic flow, Wi ( k ) ,Wi ( k +1) are corresponding the traffic status in the system of i-th traffic flow when time = t , time = t + 1 . It is proved that (5)

Wi ( k +1) = Wi ( k )Ti ( k ) .

(5)

When the system gets a stable state, the sequence series Wi ( k ) gets converged and then there must exits Wi ( ∞ ) which satisfies the equation (6)

Wi ( ∞ ) = lim Wi ( k ) . k →∞

(6)

Then Wi ( k ) is called the stable solution of the traffic flow and it has (7) when all the stable solutions exit Wi ( ∞ ) = Wi ( ∞ )Ti ( ∞ ) (i = 1, 2,..., t ) .

(7)

When the system gets stable as (7), the time spend for passing through this city optimization problem for urban traffic control can be formulated as: minimize “target function”. Target function is t arg et ( R, g ) , which is determined by both the driver strategy R and traffic light strategy g

708

Y. Cheng, T. Zhang, and J. Wang

t

t arg et ( R , g ) =

Ni

∑ρ i =1

i

t

∑ρT i =1

t

⋅ ρ iT =

i

∑N i =1 t

∑ρ i =1

i

.

(8)

i

The “target function” we called in our system is the average passing time of the whole city for all traffic flow. In equation 8, ρi is traffic flow of the i-th flow. In a given and long T time, the total traffic flow enter this system is ρi T , the total consumed time is N i T , so the average passing time of each car is

Ni

ρi

. Once the G (V , E ) is given, the

driver strategy Ri , the traffic lights strategy g , start Si and destination Di can be calculated by 8. Our main aim is to find the most optimized traffic light strategy g * to minimize target total time function t arg et ( R, g ) .

g * = arg min{min{t arg et ( R, g )}} . g

(9)

R

3 Solution of the Optimized Problem The optimization problem in our system is to find an optimized control strategy both for traffic lights and the drivers. It is difficult to find the optimum solution but we can give a strategy which is close to the optimum. Finally the simulated annealing algorithm is chosen. Our algorithms mainly consist of two parts: (1) Find the stable state of the random work processing; (2) Adjust both the driver and traffic lights strategy by iteration to get an optimization solution. 3.1 The Stable Solution of Random Walk

The task of this function is to find the stable state of the random walk by iteration. Algorithm 1: find stable state ( R, g )

1: Initialization Wi 0 : (Wi (0) ) Si = 1 ,the other is 0; 2: 3: 4: 6: 7: 8: 8:

repeat for all i = 1, 2,..., t , do find the transfer matrix: Ti ( k )

Wi ( k +1) = Wi ( k )Ti ( k ) until for all i , the Wi ( k ) get stable end repeat out put

9: end algorithm 1

∑

∑ t i =1

[ N i (Ti

t i =1

(∞)

Ni

) Di → Si (Wi ( ∞ ) ) Di ]

Multi-agent System Model for Urban Traffic Simulation and Optimizing

709

3.2 The Proposed System

The main algorithm depicts a processing in which the driver strategy and the traffic lights strategy are adjusted through iteration for a combined optimum. Algorithm 2: the main optimization algorithm Input: the initial traffic strategy g , the initial diver’s strategy Ri for each traffic

flow, presenting the shortest route from Si to Di Output:

the optimized traffic light strategy g *

1: for all Ri and g 2: adjust_strategy( g , R ) = R 3: repeat adjust_strategy( g , R ) = G 4: adjust_strategy( g , R ) = R until the target function is stable 6: end repeat 7: 8: out put g * 9: end algorithm 2

4 Experiments and Results We apply our simulation on 2 × 2 traffic network with 8 entrances (export) Port1 ,..., Port8 and 4 crossings. Each crossing has its own traffic light labeled from 1 to 4 and each traffic light has two states: “|” and “—“: “|” allows pass of vertical and “—“allows pass of level.

Fig. 6. The 2 × 2 city traffic net

Four group car traffic flows with different stat point, destination and flow density are chosen for experiments evaluation. For each group, the traffic light strategy and time spend also are shown in the table. In our paper, the traffic light strategy is the time setting of “|” and “—“for each traffic light node in the intersection. After the

710

Y. Cheng, T. Zhang, and J. Wang Table 1. Initial traffic flow of group 1 ID: Traffic Flow

Si

Di

Ni

1

Port0

Port3

5

2

Port1

Port2

10

3

Port4

Port7

10

4

Port5

Port6

5

Table 2. Experiments results of group 1

Initial Optimization 1

Average time

Light 1

Light 2

Light 3

Light 4

14.02

| 0.50, —0.50

| 0.50, —0.50

| 0.50, —0.50

| 0.50, —0.50

14.02

| 0.48, —0.52

| 0.53, —0.47

| 0.45, —0.55

| 0.67, —0.33

Optimization 2

13.73

| 0.48, —0.52

| 0.53, —0.47

| 0.44, —0.56

| 0.67, —0.33

Optimization 3

13.65

| 0.50, —0.50

| 0.50, —0.50

| 0.50, —0.50

| 0.50, —0.50

Optimization 4

13.64

| 0.50, —0.50

| 0.50, —0.50

| 0.50, —0.50

| 0.50, —0.50

Fig. 7. The transfer graph of group 1

Multi-agent System Model for Urban Traffic Simulation and Optimizing

711

optimizations, we draw a transfer graph of every traffic flow. The color degree of depth of each node represents flow density at that point. The high color degree shows the high traffic density. The Table 1 shows and table 2 demonstrates the experiments condition and the results for group 1. N i is number of vehicles.

5 Conclusion The random-walk based urban traffic system is proposed in this paper. In this model, every node in the traffic system diagraph has its property and communication with other adjacent nodes by route lines. It can model systems with dynamic, stochastic and fluid properties. The system modeling with random walk has not only the character of easy looking, but also good mathematic analysis ability, which makes us have a deeper understanding of the whole complex flow system. The results show a combined optimized solution for both traffic lights and the driver sand can give us an intelligent guide and control of the city traffic. Experiments show this system is stable and robust.

References 1. Lu, H.: Intelligent Traffic System. Chin Communications Press, Beijing (2003) 2. Aw, A., Rascle, M.: Resurrection of “second order” models of traffic flow. SIAM J. Appl. Math. 60, 916–938 (2000) 3. Helbing, D., Hennecke, A., Shvetsov, V., Treiber, M.: Microandmacro-simulation of freeway traffic. Mathematical and Computer Modeling 35, 517–547 (2002) 4. Wang, X., Juan, Z., Ja, H.: Micro-simulation models of traffic flow of developing andevaluating ITS. Journal of Traffic and Transportation Engineering 2, 64–66 (2002) 5. Gong, X., Tong, S., Wang, Z., Chen, D.: Survey on freeway traffic flow modeling. Journal of Traffic and Transportation Engineering 2, 74–79 (2002) 6. Jan, Z.: Contiounous-time modeling of urban traffic. Journal of Traffic and Transportation Engineering 2, 75–76 (2005)

Vehicle Detection Using Bayesian Enhanced CoBE Classification Zhong Zheng, Shen Xu, and Yi L. Murphey Department of Electrical and Computer Engineering University of Michigan-Dearborn MI 48128, USA [email protected]

Abstract. This paper presents a noval computational framework, BN+CoBE, Bayesian enhaced Cascades of Boosted Ensemble, for on-road vehicle detection. The objective of this research is to reduce false alarms while keeping the detection rate high. In the proposed system, BN+CoBE, the CoBE is trained on image texture features and Bayesian conditional probability function is trained on vehicle features of location, size and confidence values generated by all the stages in CoBE. Experiment results on real world data show that the proposed BN+CoBE system is effective in reducing false alarms significantly while keeping the detection rate high. Keywords: Cascades of Boosted Ensemble, Vehicle Detection, Bayesian conditional probability.

1 Introduction Vehicle detection for driver assistance has received considerable attention over the last 15 years. It is a vision-based system where the camera is mounted on the vehicle [1]. Since scenes are dynamically changing and illumination conditions can vary significantly, real time on-road vehicle detection is a very challenging problem [1], [2], [3], [4]. Much research has been conducted to solve this problem. Schneiderman and Kanade [3] proposed to build a model for both object and non-object based on appearance represented by the product of two histograms, each of which represents the joint statistics of a subset of Haar wavelet features and their position on the object. Bootstrapping was used to gather the statistics of the non-vehicle class. Most recently, a comparative study has been conducted by Negri et al. [5] under the application context of on-road vehicle detection. Three feature sets have been compared: Haar-like feature, histograms of orient gradient (HoG), and the combination of the two features. And Wang et al. introduced a statistical approach in [4] that used principle component analysis (PCA) and independent component analysis (ICA) to characterize local features. In spite of these efforts, on-road vehicle detection remains a challenging research problem. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 713–719. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

714

Z. Zheng, S. Xu, and Y.L. Murphey

This paper presents a Bayesian enhaced Cascades of Boosted Ensemble (BN+CoBE) system that learns about vehicle detection based on texture as well as geometric features of vehicles. It is organized as follows: Section 2 presents the BN+CoBE for on-road vehicle detection. Section 3 presents experiments conducted on real world images and Section 4 concludes the paper.

2 Bayesian Enhanced Cascades of Boosted Ensemble (BN+CoBE) Classification Fig. 1 illustrates the structure of the proposed BN+CoBE classification system for vehicle detection. The system consists of two major components, CoBE and the Bayesian decision function (BN). For an input image, subwindows of images are cropped from the input image using a sliding window scheme (see Fig 1. (a)). The subwindows have various sizes so each vehicle has better chances to be contained entirely within a subwindow image. Every subwindow image is sent to the CoBE for classification. The CoBE consists of multiple stages of classifications (see Fig. 1. (b)). At each stage a subwindow image is either rejected or passed on to the next stage. If a subwindow image is passed through the last stage, it is considered as a vehicle image by the CoBE classification system. In vehicle detection, it is often that most subwindows of an input image contain no vehicles. By adopting such a sequential classification process, most of the negative sub-windows (non-vehicle) are rejected at very first few stages, which radically improves the computational cost of the detection process. The output at the CoBE are vehicle candidate images with various sizes and located at various locations inside the image frame. The CoBE for vehicle detection is developed using the similar technique proposed by Viola and Jones [6] for face detection. Haar-like features are used in CoBE and the vehicle detection problem is modeled in a binary classification problem, “vehicle” and “non-vehicle” classes. Since the CoBE is trained using only texture features, the non-vehicle images that have texture features similar to vehicles can be falsely detected as vehicle. In many applications, false alarms, i.e. false positives, are considered as a serious problem such as the driver assistant systems, therefore false alarms should be minimized while the detection rate is still kept high. To improve the performance of CoBE classifier, we incorporate a Bayesian decision function into the classification system. A Bayesian decision is a computational model that represents the probabilistic relationships among a set of variables. It has been implemented successfully in classification problem with large amount of features that have complex relationships [7]. In this paper, the Bayesian decision function is designed to enhance the CoBE by incorporating additional vehicle features into the vehicle classification process. Other useful features to describe on-road vehicles are the size and the location. Let X and Y be the variable to represent the coordinates of the center of a subwindow vehicle image, and W and H represent the width and height of the subwindow image. We conducted a statistic study on the dependency of these variables. We applied Pearson product-moment correlation to 1000 vehicle images and the results are shown in Table 1.

Vehicle Detection Using Bayesian Enhanced CoBE Classification

715

(a)

(b) Fig. 1. A Bayesian enhanced CoBE for vehicle detection Table 1. Correlation coeffients of vehicle position and size X

Y

W

X

1

Y

0.0134

1

W

-0.4375

-0.5788

1

H

-0.3948

-0.645

0.9561

H

1

The range of the correlation coefficients is [-1, 1], where 1 represents a strong positive correlation between two variables, and -1 represents a strong negative correlation. The closer the coefficient is to either −1 or 1, the stronger the correlation is between the variables. If two variables are independent, then the correlation coefficient is 0. Table 1 shows there is a strong positive correlation between Width and Height. There is a weak positive correlation between X and Y. And there is medium correlation exists in pairs (X, Width), (Y, Width), (X, Height) and (Y, Height). So, the position and size features are relevant features to characterize vehicles. Another feature used by the Bayesian decision function is the average stage confidence value (ASCV) in the CoBE classifier. The classification at each stage in

716

Z. Zheng, S. Xu, and Y.L. Murphey

the CoBE classifier is made by comparing a weighted linear combination of all weak classifier outputs with a threshold, as shown in Equation (1). 1

T

T

∑ α h ( x) ≥ ∑ α t

t

2

t =1

t

.

(1)

t =1

where the left side is called Stage Sum (SSi) and right side is called Stage Threshold (STi). The Stage Confidence value at stage i, denoted as SCi, is defined as the difference between SSi and STi, namely 1

T

SCi ≡ SS i − STi =

T

∑ α h ( x) − ∑ α t

t =1

t

2

t

.

(2)

t =1

Stage Confidence Value

In order to have a good understanding of SCs, a statistical analysis has been conducted by applying a CoBE classifier of 29 stages to 1000 vehicles images. A box plot of the stage confidence values for the 1000 vehicle images is shown in Fig. 2. The plot shows that most of the confidence values are in range (0, 8.1). The median of most stages are in range [2, 3], and IQRs (Inter Quartile Ranges) usually lie in [1,4] with whiskers between [0, 7]. The plot also shows that as stage number increases, more vehicle images have higher stage confidence values. 8 6 4 2 0 1 2 3 4 5 6 7 8 9 101112 1314 151617 181920 2122 232425 262728 29 Stage Number

Fig. 2. Box plot of stage confidence values

In order to reduce computational complexity and avoid sparseness in the probability table, we use average stage confidence value in the Bayesian function instead of all individual stage confidence values. The average stage confidence value can be obtained at stage i as follows:

ASCVi =

1 i ∑ SC j . i j =1

(3)

In summary, the Bayesian decision function uses five random variables, location variables, X and Y; size variables W and H; and a average confidence variable ASCV such that ASCV = ASCVN, where N is the number of stages in CoBE. The Bayesian decision function is represented by a probability table as shown in Fig. 1 (b). The Bayesian learning algorithm has the following steps.

Vehicle Detection Using Bayesian Enhanced CoBE Classification

717

Step 1. Applying CoBE classifier to a given training set. Step 2. For every possible tuple of <X, Y, W, H, ASCV>, calculating CP =

# TP # TP + # FP

where #TP and #FP are the number of TP and FP generated by CoBE on the training data samples that satisfy the tuple, where CP is the probability of the image represented by the tuple being a vehicle.

3 Experiments The raw vehicle data set used in the experiments consists of 84 on-road video clips. The video images were taken by cameras mounted on the windshield of a moving vehicle. Video images are in grayscale with a resolution of 750 pixels by 250 pixels. The number of frames contained in one clip varies from 707 to 3599. 78 of these video clips were used to generate the training data and the other 5 were used to generate the test data. We generated 110496 cropped vehicle images for training from frames randomly through the 78 video clips. We randomly cropped 15,393 background images from 2000 background frames randomly selected from the 78 video clips. The CoBE classifier trained on this data set contains 29 classification stages with 620 weak classifiers in total. The random variables used in the Bayesian decision function are quantified as follows. X and Y are bounded by image dimensions and are divided into intervals of equal length of 25 pixels. W and H are within the bound of [20, 100] and are divided to intervals of 10 pixels. During the classification, an input image is processed by the following sliding window scheme. For a predefined minimum window size, (Widthmin, Heightmin), subwindows of images are generated by sliding this window over the input image. Then subsequent N

N

windows with increased sizes, (Width * scale _ factor , Height * scale _ factor ) are slided over the input image to generate the subwindows of images, where scale_factor >1. In our experiments, (Widthmin, Heightmin) = (20, 25) and scale_factor = 1.1. The test set consists of 1000 images randomly sampled from 6 video clips. The 1000 images contain 1110 vehicle instances. In order to compare the effectiveness of the Bayesian function and its various random variables, we illustrate in Fig. 3 the performances of the three different classification systems stage-by-stage on the test data set. The three classification systems are, CoBE, BN+CoBE, which is the enhanced CoBE with the Bayesian function that uses four random variables, (X, Y, W, H), BN(ASCV)+CoBE, which is the enhanced CoBE with the Bayesian function that uses five random variables (X, Y, W, H, ASCV). In the figure, the detection rates and false alarms, also denoted as FA, of all three systems are illustrated. The curves represent the detection rates, also known as True Positive (TP). The bar graphs represent the number of False Alarms (FA) generated by all three systems. min

min

6000

1

5000

0.9

4000

0.8

3000

0.7

2000

TP CoBE TP BN(ASCV)+CoBE TP BN+CoBE

0.6

1000

FA CoBE FA BN(ASCV)+CoBE FA BN+CoBE

0.5

Detection Rate

Z. Zheng, S. Xu, and Y.L. Murphey

False Alarms

718

0 0.4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Stage Number

Fig. 3. Stage-by-stage performance Analysis of three classification systems

The following observations can be made from Fig. 3. • •

•

BN(ASCV)+CoBE has better performances than BN+CoBE. At every stage, BN(ASCV)+CoBE gave higher detection rate and lower false alarms than the BN+CoBE. Both systems, BN(ASCV)+CoBE and BN+CoBE gave huge reduction of false alarms in early stages. The detection rate dropped some but not large. For example at the 6th stage the FA dropped 96.45%. The detection rate dropped about 8% by BN(ASCV)+CoBE and more than 21% by the BN+CoBE BN(ASCV)+ CoBE is the most effective vehicle classification system. At earlier stages(between stage 6 and stage 12), it reduced the false alarms generated by CoBE between 50% ~ 96% while its detection rate decreased some, but not significant. For example, at stage 12, the BN(ASCV)+ CoBE classification system reduced more than 50% of false alarms, and its detection rate dropped less than 0.01% in comparison to CoBE. As the stage number increases, the detection rates of BN(ASCV)+ CoBE converges to the detection rates of CoBE while false alarms continue to be reduced. At the last 5 stages, BN(ASCV)+CoBE gave the same detection rates as the CoBE but lower false alarms. For example, at stage 28, while both systems gave detection rates at 96%, BN(ASCV)+CoBE further reduced 1270 false alarms.

4 Conclusion We presented a vehicle detection system that uses a Bayesian enhanced CoBE technique. The Bayesian enhanced CoBE system uses vehicle texture features as well as vehicle size and location to accurately detect on-road vehicles. The CoBE is trained using vehicle texture features and the Bayesian function is trained using vehicle location and size, and the average confidence value generated by CoBE. The experiment results show that the Bayesian enhanced CoBE classifier gave a decisive

Vehicle Detection Using Bayesian Enhanced CoBE Classification

719

performance improvement: significant false alarm reduction while still keeping the detection rates high.

References [1] Sun, Z., Bebis, G.: On-Road Vehicle Detection: A Review. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(5) (May 2006) [2] Bertozzi, M., Broggi, A., Cellario, M., Fascioli, A., Lombardi, P., Porta, M.: Artificial Vision in Road Vehicles. Proc. IEEE 90(7), 1258–1271 (2002) [3] Schneiderman, H., Kanade, T.: A Statistical Method for 3D Object Detection Applied to Faces and Cars. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, p. 1746 (2000) [4] Wang, C.C., Lien, J.J.J.: Automatic Vehicle Detection Using Local Features and a statistical Approach. IEEE Transactions on Intelligent Transportation Systems 9, 83–96 (2008) [5] Negri, P., Clady, X., Hanif, S., Prevost, L.: A cascade of boosted generative and discriminative classifiers for vehicle detection. EURASIP J. Adv. Signal Process. 2008, 1–12 (2008) [6] Viola, P., Jones, M.: Robust Real-time Object Detection. Presented at 2nd Intl. Workshop on Statistical and Computational Theories (2001) [7] Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, Heidelberg (2006)

Vibration Analysis of a Submarine Model Based on an Improved ICA Approach Wei Cheng, Zhousuo Zhang, and Zhengjia He State Key Laboratory for Manufacturing Systems Engineering, Xi’an Jiaotong University, Xi’an 710049, China [email protected], zzs, hzj @mail.xjtu.edu.cn

｛

}

Abstract. Vibration and noise reduction and control have obvious significance for submarines. A novel vibration analysis method based on an improved ICA algorithm is proposed in this paper. By using the clustering evaluation method, the stability and separating performance of the algorithm are significantly enhanced. The improved ICA algorithm is applied to feature extraction of the vibration signals and quantitative calculation of the source contributions of a scaled submarine model. The result shows that the proposed method is effective, and this research provides a primary basis for the vibration and noise control of submarines. Keywords: Independent component analysis, Clustering evaluation, Feature extraction, Quantitative calculation, Source contributions.

1 Introduction Submarines navigate underwater, so it is difficult for satellites, anti-submarine aircrafts and warships to detect them. However, the vibration and noise radiates into the water from the submarines, and poses a serious threat to the concealment. Besides, it also can affect the operational performance. Vibration and noise reduction and prediction can be improved by means of identifying the sources, and evaluating their contributions effectively [1]. Therefore, it has great significance to analyze the vibration signals generated by the equipments. Independent component analysis (ICA) is a typical method of blind source separation. Under the assumption of statistic independence, multi-channel measured signals can be blindly separated despite the source signals and transmission path characteristics are unknown, and the potential sources hidden in the mixed signals can be extracted [2]. Thus it provides a novel idea of processing the mixed signals whose components are unknown. ICA is firstly proposed by French scholars C. Jutten and J. Herault [3], and is firstly applied in blind source separation by P. Comon [4]. A well known ICA algorithm called fixed point algorithm is proposed by Finnish scholars A. Havarinen [5], and improved in the monographs [6]. A new stable algorithm of strong separating performance is proposed by J. Himeberg and Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 721–728. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

722

W. Cheng, Z. Zhang, and Z. He

A. Hyvarinen [7, 8]. Now ICA is widely used in fetal electrocardiogram extraction [9], gearbox vibration signal extraction and condition monitoring [10], batch process monitoring [11], and so on. In this paper, a novel quantitatively calculating the source contributions method based on an improved ICA algorithm is proposed. The separating performance and stability of the improved ICA algorithm are validated in the simulations. And then the proposed method is applied to extracting the mechanical signal feature and quantitative calculating the source contributions of a scaled submarine model.

2 Independent Component Analysis 2.1 Basic Theory of ICA Assume that n source signals S = {s1 , s2 ,..., sn } exist at the same time, and m mixed signals X = {x1 , x2 ,..., xm } which are composed of these source signals are obtained in the different places. Thus each mixed signal can be described as:

xi =

n

∑a

ij

s j + ni

i = 1,2 ,... n , j = 1,2 ,..., m

(1)

i =1

xi is the ith mixed signal measured at the place i, s j is the jth independent component, aij is the mixing coefficient, and ni is the noise of ith mixed signal. Where

X = AS + N

(2)

Where A is the mixing matrix, N is the noise matrix. Independent component analysis can be described as follow forms

Y = WX = WAS = GS

(3)

Where W is a separating matrix, and G is a global (or system) matrix. 2.2 Separation Criterion Based on Negentropy

Ng ( s ) = H ( sGauss ) − H ( s )

(4)

H ( s ) = − ∫ p( s ) lg p( s )ds

(5)

Ng ( s ) ∝ [ E{G ( s )} − E{G ( sgauss )}]2

(6)

Where Ng (•) is a negentropy function, mean function.

H (•) is an entropy function, and E (•) is a

Vibration Analysis of a Submarine Model Based on an Improved ICA Approach

723

2.3 Improving the Separation Performance by Clustering Evaluation In essential, most algorithms based on ICA are statistic. Therefore, their results may be different in repeated executing of the algorithm, thus the outputs of a single run of an ICA algorithm can not be trusted [7, 8]. In this paper, the stability and effectiveness of the ICA are improved by means of clustering evaluation.

d ij = 1 − ρij

IR =

1 L Slin ∑ L l =1 Slex 1

S lin =

S lex = min l ′≠ l

(7)

CL l

2

(8)

∑d

(9)

ij

i , j ∈CL l

1 CL l CL l ′

∑ ∑d

ij

(10)

i ∈CL l i∈CL l ′

dij is a non correlation coefficient, I R is a cluster validity index, CL is a set of all the components, and l is serial numbers of the components. Where

2.4 Waveform Correlation Coefficient

ρ sy

n

ρ sy =

∑ s( k ) y ( k ) k =1

n

(11)

n

∑ s ( k )∑ y ( k ) 2

k =1

2

k =1

3 Simulation Experiment Fast ICA algorithm has strong separating performance for linear mixed signals, however, the separated components may be different in repeated calculations because it is a statistical signal processing method. The improved ICA algorithm can solve this problem by clustering evaluation. The effectiveness of the improved ICA algorithm is validated by typical signals in the simulation experiment. The source signals: s1 is a white noise signal;

s2 is a sinusoidal signal; s3 is a triangle wave signal; and s4 is a

square wave signal. The data length of source signals is 1000, and the step is 1.0. The mixing matrix A is generated randomly, and the generating function of source signals and the mixed matrix A are as follows:

724

W. Cheng, Z. Zhang, and Z. He

⎡s1 (t ) ⎤ ⎡ N (t ) ⎤ ⎢s (t ) ⎥ ⎢sin(0.4 × π × t ) ⎥ ⎥ s(t)= ⎢ 2 ⎥ = ⎢ ⎢s3 (t ) ⎥ ⎢ sawtooth(0.5 × π × t ,0.5) ⎥ ⎢ ⎥ ⎢ ⎥ ⎣s 4 (t ) ⎦ ⎣ square(0.6 × π × t ,50) ⎦

⎡0.65 0.75 0.65 0.60⎤ ⎢0.95 0.70 0.75 0.85⎥ ⎥ A=⎢ ⎢0.88 0.90 0.80 0.32⎥ ⎢ ⎥ ⎣0.90 0.40 0.42 0.95⎦

The waveforms of source signals are shown in Fig.1, and the waveforms of the mixed signals are shown in Fig.2. From Fig.1, it can be seen that source signals have obvious and typical waveform features. The mixed signals are composed of these source signals by means of superposition. Fig.2 shows that the waveforms of the mixed signals are complicate, and the waveforms of source signals hidden in the mixed signals are not clear. 10

10

0

0

-10 0

200

400

600

800

1000

Amplitude

1 0 -1 0

200

400

600

800

1000

1 0 -1

10

Amplitude

-10

10 0 -10

0

200

400

600

800

1000

0

200

400

600

800

1000

10

1

0

0 -1

0 -10

-10

0

200

Fig. 1. Waveforms of source signals

200

400

600

800

1000

0

200

400

600

800

1000

0 -2 5

0

200

400

600

800

1000

5

0

200

400

600

800

1000

0

200

400

600

800

1000

0

200

400

600

800

1000

0

200

400

600

800

1000

0 -5 2 0 -2 2 0

0 -5

-2

Amplitude

Amplitude

2

1000

0 0

0 -2

800

2

0

2

600

Fig. 2. Waveforms of mixed signals

2

-2

400

Data sequence

Data sequence

0

200

400

600

800

1000

-2

Data sequence

Data sequence

Fig. 3. Waveforms by Fast ICA

Fig. 4. Waveforms by the improved ICA

The mixed signals are respectively separated by the fast fixed point algorithm and the improved ICA algorithm, and the waveforms of separated signals are respectively shown in Fig. 3 and Fig. 4. Fig. 3 shows one of the possible results of repeat calculations by the fast fixed point algorithm. From Fig. 3, it can be seen that the separated components do not have the same waveforms with that of source signals, which indicates the waveforms of source signals are not well separated in this calculation.

Vibration Analysis of a Submarine Model Based on an Improved ICA Approach

725

However, by using clustering evaluation method, the separated components by the improved ICA algorithm have obviously stable results, and the separated components have the same waveforms with that of source signals. From Fig. 4, it can be seen that each source signal is well separated, and the correlation coefficients between the separated components and the source signals are respectively 1.00, 0.99, 0.99 and 0.98, which indicates that the waveforms of source signals are extracted from the mixed signals effectively. Therefore, the improved ICA algorithm has strong separating performance, and the separated components are stable and effective as a result of clustering evaluation.

4 Application to Quantitatively Calculate the Source Contributions of a Submarine Model The submarine test bench is set up based on a 1:5 scaled submarine module model. Three motors are installed in the submarine model to simulate the rotating machinery equipments; blocks are added in each output shaft of the motors to generate eccentric vibration; rubber springs are applied to support the entire model, and they can also eliminate the impacts of the environment. The structural diagram of the module is shown in Fig. 5, and the experimental physical photo is shown in Fig. 6. Three acceleration sensors are installed on each motor bases to acquire the source vibration signals, and nine sensors are installed on the bulkhead of the model. In the testing, the sampling frequency is 16384 Hz; the data length is 16384; and the unit of signal amplitude is g (1.0g = 9.8ms-2). The rotation speed of motor 1, motor 2, and motor 3 are respectively 1350 n/min, 1470 n/min, and 1230 n/min.

Fig. 5. Structural diagram

Fig. 6. The photo of the model

Under the above condition, the vibration signals on the bulkhead and on the bases are measured. To improve the separating performance, the priori information is applied, which means that source signals on the bases are applied to enhance the separating performance. One of the vibration signal and three source signals are selected as the mixed signals. And three independent components are extracted by the improved ICA algorithm. The waveforms of mixed signals and independent components are respectively shown in Fig. 7 and Fig. 8.

726

W. Cheng, Z. Zhang, and Z. He

5

0.5 0

IC1

0 0.42

0.44

0.46

0.48

0.5

-5 0.4

a

0 -1 0.4 2

0.42

0.44

0.46

0.48

0.5

b

0 -2 0.4 0.5

0.42

0.44

0.46

0.48

0.5

c

0 -0.5 0.4

0.42

0.44

0.46

0.48

Amplitude / g

Amplitude /g

-0.5 0.4 1

0.44

0.46

0.48

0.5

IC2

0

-5 0.4

0.42

0.44

0.46

0.48

0.5

5

IC3

0

-5 0.4

0.5

0.42

5

0.42

Time / s

0.44

0.46

0.48

0.5

Time / s

Fig. 7. Waveforms of the mixed signals Fig. 8. Waveforms of the independent components

The waveform correlation coefficients are applied to evaluate the separating performance, and the correlation coefficients between independent components and source signals are shown in follow matrix ρ sy . According to the mathematical model of ICA, the contributions of each source are obtained by the mixing matrix Aˆ .

⎡0.2907 0.8903 0.0795 ⎤ ρ sy = ⎢0.8397 0.1041 0.3083 ⎥ ⎢ ⎥ ⎢⎣0.2717 0.1094 0.9539 ⎥⎦ It can be seen in the correlation matrix

ρ sy

⎡ -0.1299 ⎢-0.1971 Aˆ = ⎢ ⎢ 0.1424 ⎢ ⎣-0.0313

-0.0097 0.0440 ⎤ 0.0358 0.1656 ⎥ ⎥ 0.0318 0.0708 ⎥ ⎥ -0.1375 0.0275 ⎦

that the coefficients between IC1, IC2, IC3

and source signal b, a, c are respectively 0.84, 0.89 and 0.95, which means the waveforms of source signals are well extracted by the improved ICA algorithm. Thus the vibration signals on the bulkhead composed of these source signals can be decomposed as these independent components, and the contribution ratio of each source can be calculated as follows. 0.1299/(0.1299+0.0097+0.0440)×100%= 70.75%. 0.0440/(0.1299+0.0097+0.0440) ×100%= 23.97%. 0.0097/(0.1299+0.0097+0.0440) ×100%= 5.28%. The real contribution ratios are calculated by the measurement. Under the condition that only one motor is running with above speed, the vibration signal on the bulkhead is measured, and the energy of the vibration signal is calculated as the real contribution. Thus real contribution ratios of each motor can be obtained. The contribution ratios are shown in Table 1. Table 1. Contribution ratios Contribution ratio

Motor 1

Motor 2

Motor 3

Real value

77.55%

16.79%

5.66%

Calculation value

70.75%

23.97%

5.28%

Vibration Analysis of a Submarine Model Based on an Improved ICA Approach

727

Table 1 shows that the calculation values calculated by the improved ICA algorithm are very close to the real values by the measurement, and the errors are respectively 6.8%, 7.18% and 0.38%, which means the contribution ratios calculated are effective. Thus it provides a novel approach to quantitatively calculate the source contributions by the improved ICA algorithm and priori information. Besides, the vibration on the bulkhead can be actively online controlled, and it provides a primary basis for vibration and noise reduction and control of submarines.

5 Conclusions In this paper, a novel improved ICA method and the framework of the algorithms are introduced, and a quantitative calculating the source contributions method is proposed. The separating performance and stability of the improved ICA algorithm are enhanced by clustering evaluation, and the simulation results show that the fast ICA algorithm may produce unreliable independent components. However, the improved ICA algorithm can separate the mixed signals effectively and stably, and the waveform coefficients are more than 0.98. The improved ICA algorithm is applied to quantitatively calculate the source contributions of a scaled submarine model, and the result indicates that it has high accuracy, and the errors are less than 7.18%. Therefore, the proposed method not only solves the problem of unreliable separation results, but also provides a reliable reference to active control the vibration and noise online, which has great significance for the stealth of submarines. Acknowledgements. This paper is supported by the National Natural Science Foundation of China“Research on Hybrid Intelligent Technique and Its Application in Fault Diagnosis Based on Granular Computing” (Grant No. 50875197) and the Project Sponsored by the Scientific Research Foundation for the Returned Overseas Chinese Scholars, State Education Ministry.

References 1. Zhang, L.K., He, L., Zhu, S.J.: Summary of the main noise source identification and contribution calculation methods of submarines. In: The tenth symposium on the ship underwater noise (2005) 2. Ma, J.C., Niu, Y.L., Chen, H.Y.: Blind signal processing. National Defense Industry Press (2006) 3. Jutten, C., Herault, J.: Blind separation of sources. Part I: An adaptive algorithm based on neuromimatic architecture. Signal Processing 24, 1–10 (1991) 4. Comon, P.: Independent component analysis, a new concept? Signal Processing 36, 287–314 (1994) 5. Hyvarinen, A.: Fast and robust fixed point algorithm for independent component analysis. IEEE Trans. on Neural Network 10, 626–634 (1999) 6. Hyvarinen, A.: Independent component analysis. John Wiley and Sons, Chichester (2001) 7. Himberg, J., Hyvarinen, A.: Icasso: Software for investigating the reliability of ICA estimates by clustering and visualization. In: Neuro Networks for Signal Processing, Toulouse, France, pp. 259–268 (2003)

728

W. Cheng, Z. Zhang, and Z. He

8. Himberg, J., Hyvarinen, A., Esposito, F.: Validating the independent components of neuroimaging time-series via clustering and visualization. Neuro Image 22, 1214–1222 (2004) 9. Ye, Y.L., Zhang, Z.L., Zeng, J.Z.: A fast and adaptive ICA algorithm with its application to fetal electrocardiogram extraction. Applied Mathematics and Computation 205, 799–806 (2008) 10. He, Q.B., Feng, Z.H., Kong, F.R.: Detection of signal transients using independent components analysis and its application in gearbox condition monitoring. Mechanical Systems and Signal Processing 21, 2056–2071 (2007) 11. Tian, X.M., Zhang, X.L., Deng, X.G.: Multiway kernel independent component analysis based on feature samples for batch process monitoring. Neurocomputing 72, 1584–1596 (2009)

A Hierarchical Salient-Region Based Algorithm for Ship Detection in Remote Sensing Images Fukun Bi1, Feng Liu1, and Lining Gao2 1

School of Information and Electronics, Beijing Institute of Technology, Beijing 100081, China {bifukun, bit_liufeng}@bit.edu.cn 2 Department of Electronic Engineering, Tsinghua University, Beijing 100084, China

Abstract. In this paper, we present a hierarchical salient-region based algorithm and apply it for automatic ship detection in remote sensing images. The novel framework breaks down the complex problem of scene analysis by hierarchical attention, in a computationally efficient manner, such that only the salientregions which contain potential targets can be analyzed in detail. Firstly, a parallel method is adopted for crudely selecting saliency tiles from entire scene by using low-level feature extraction mechanisms, and then the Region-ofInterest (ROI) around each saliency object is taken out from the saliency tiles to pass to the further processing. Shape and texture features are extracted from the multiresource ROIs to describe more details for candidate targets respectively. Finally, Support Vector Machine (SVM) is applied for target validation. Experiments show the proposed algorithm achieves high probabilities of recall and correct detection, as well as the false alarms can be greatly diminished, with a reasonable time-consumption. Keywords: Target detection, Ship detection, Region of Interest (ROI), Remote sensing image, Visual attention, Salient region.

1 Introduction With the increasing volume of satellite image data which are collected from air- and space-borne sensors, automatic ship detection from remote sensing images is a crucial application for both military and civilian fields [1][2]. However, the detection systems are faced with, on the one hand, the need to process massive amounts of incoming data, and on the other hand, the requirement for nearly real-time capacity of reaction. One of the solutions to this bottleneck is to follow the mechanism used by the natural vision systems in which only significant portions are selected from the scene perceived through the eyes for thorough processing, and it can be followed to achieve more efficiency, intelligence, and robustness in artificial vision systems [3][4][5]. Many valuable studies are carried out in this field [6][7], however, these typical Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 729–738. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

730

F. Bi, F. Liu, and L. Gao

algorithms are usually effective only for common image analysis, not for the task of ship detection in remote sensing images, it is due to that they consume many computation-resources in processing, while the remote sensing image often contains vast data. Unlike the traditional studies, in terms of the sparseness of ship distribution on sea, and the region of ship target is a small part of remote sensing image, we propose a novel salient-region based algorithm to investigate the hierarchical attention mechanism for ship detection in remote sensing images, in order to increase the effectiveness. The proposed algorithm appears to employ a serial and hierarchical computational strategy, and break down the complexity of scene understanding into a sequence of low-cost steps, merely the salient regions could be analyzed in detail by more complex and time consuming processes. It also provides a massively parallel method for the fast extraction of salient regions of entire scene. At the first stage, a statistical strategy for finding saliency tile is controlled by lowlevel feature extraction mechanisms, which crudely select conspicuous tiles from entire visual scene, in a parallel process, and compete for attention. In order to apply more fine techniques to analyze the potential targets of saliency tiles, the ROI around each saliency object is taken out to pass to further processing. Unlike the traditional ROI extraction, we carry out the investigation at both binary and original ROIs, the corresponding features extracted from different sources of ROIs which can provide credible and rich information for candidate targets. Finally, attention focuses on the extracted ROIs to be analyzed in more detail by pattern recognition method.

2 Detection Algorithm The overall framework of the detection algorithm is summarized in Fig.1, and the following sections describe in detail about the major stages of the proposed algorithm.

Fig. 1. Process flow of the entire algorithm for automatic ship detection

A Hierarchical Salient-Region Based Algorithm for Ship Detection

731

2.1 Preprocessing The aim of preprocessing is to make the subsequent detection stages easier and hence more accurate by first masking the land regions out from the image, and then divide the entire image into some nonoverlap rectangular-regions (tiles). 2.1.1 Land Masking Land masking is important not only for the obvious reason that only ships in the ocean are interest but also because ship detectors can produce high numbers of false alarms when applied to land [8]. We propose an adaptive histogram-based method for masking, in terms of the uncertain contents about remote sensing images. (a) The high-resolution remote sensing image is firstly downsampled into indeximage to reduce the computational load, and the index-image also can provide sufficient and discriminative information for land and water as seen in Fig.2(b). (b) We note that the histogram shows unimodal property with a low gray-range in land images and high in water ones, in addition, the bimodal property of histogram appears in the images with both water and land parts. Binary segmentation is applied to divide the index-image, as considering it contains only water, only land, or both of them according to (1). The binary index-image is shown in Fig.2(c). ⎧1 ⎪0 ⎪ fth(i, j ) = ⎨ ⎪0 ⎪⎩1

if uL ≤ uS ≤ uH , x(i, j ) ≥ TOTSU ; if uL ≤ uS ≤ uH , x(i, j ) < TOTSU ; if uS < uL; if uS > uH ;

(1) .

Where TOTSU is the bimodal histogram threshold determined by OTSU method. We evaluate UL , UH denote the means of water and land from training data respectively, and US is the mean of input index-image. (c) To eliminate the isolated regions which always include ship targets out from the index-image, median filter is applied as in Fig. 2(d). (d) To eliminate the false alarms in land and near shoreline, the binary dilation operation is applied to fill the holes of land template and extend the shoreline as in Fig. 2(e). (e) Using the index-image to masking out the land from original image by assigning the value of 0 to all pixels that are denoted as land as in Fig. 2(f).

Fig. 2. Example of land masking

732

F. Bi, F. Liu, and L. Gao

2.1.2 Image Tiling The land-masked image is divided into some nonoverlap tiles of 512 × 512 pixels. This operation results in not only a faster processing time during parallelization but also in a more effective handling of local information especially in remote sensing images with heterogeneous sea clutter background. 2.2 Saliency Tile Extraction Considering the sparseness of ship distribution on sea, an effective histogram based method is proposed to select tiles which contain potential target and the further flows be only utilized to these saliency tiles. It avoids further algorithm chain is performed on each tile and most likely to reduce the complexity of whole scene analysis. 2.2.1 Spatial Saliency Enhancement Compared with the nonuniform background of sea surface and vast cloud disturbs, ship target represents a compact space characteristic. We adopt the morphology Top_Hat operator to suppress these dispersed space disturbs, however, it is believe that the key of Top_Hat operator is choosing structure element [9] [10]. Since ships have similar rectangle structure generally, the shape of structure element adopts n × n square, and an adaptive Clutter Rate Coefficient (CRC) is proposed to control the size of structural elements, in terms of the rough background is a local property of sea surface and the nonuniform background is a global one, we divide a tile into M × N grids and then calculate CRC for the tile which is defined as following equation:

CRC = α 1σ LocMean + α 2σ Whole ,

σ LocMean =

N

M

i =1

j =1

∑∑ σ

Local

N iM

(2)

(i , j )

.

(3)

Where σ Local represents the standard deviation of a grid, σ Whole represents the standard deviation of current tile, α 1 and α 2 are constants. The left term of (2) is influenced mostly by the nonuniform property of background, and the right one is influenced mostly by rough property. The size of structural element is decided by CRC, The higher the CRC value is, the greater the size for structural element. The proposed method achieves satisfying performance rather than the fixed size strategy which always fails in rough or nonuniform background, Fig. 3 shows an example.

Fig. 3. Comparison of background suppression by Top_Hat operator, (a) original tile, (b) result of fixed size method, (c) result of our proposed method

A Hierarchical Salient-Region Based Algorithm for Ship Detection

733

The targets are enhanced with their spatial compactness and the dispersed disturbs of background are suppressed obviously as can be seen in Fig. 4.

Fig. 4. Typical results of spatial saliency enhancement for tiles

2.2.2 Estimation of Saliency Threshold There is no obvious discrimination between the histograms of tiles which contain ship targets and nontarget ones, but we note that the tail of histogram should be longer in tiles which include potential ships, after the stage of spatial saliency enhancement, since the high gray pixels always represent compact objects in those saliency enhanced tiles. Therefore, we suggest the Ratio of Histogram Tail (RHT) for measuring this property and it could indicates the potential target tiles (saliency tiles), it is defined as: Ef = El / Eh , where El is the span of low gray which include the most pixels of the enhanced tile (e.g. 90%), and Eh is the span of high gray which remains except the El pixels, Fig. 5 is an example.

Fig. 5. Example of the RHT feature for an enhanced tile

2.2.3 Determination of Saliency Tile Estimating the RHT for each candidate tile, then a given tile is considered to be saliency tile when the RHT is greater than the saliency threshold. Note that this stage needs guarantee low probability of miss, and the false alarms cloud be controlled in the following stages.

734

F. Bi, F. Liu, and L. Gao

2.3 Multiresource ROIs Extraction The saliency tiles indicate the rude and extend regions for potential targets, we then extract Region-of-Interest (ROI) which contains potential targets with a smaller region for further analyzing, in terms of our hierarchical interpretation process. Unlike the traditional ROI extraction methods as in [11], we carry out the investigation at both binary and original ROIs, different sources of a ROI can provide credible and rich information for candidate targets. The extraction flow as follows: (1) A binary version of the saliency tile is created by thresholding the image with OTSU operator. Fig. 6(a) shows a locally original patch of the saliency tile, and Fig. 6(b) is the enhanced one, then the binary result is shown in Fig. 6(c). (2) The binary dilation operation is applied to connect the fragmental parts of a candidate target, but there are always some parts can’t be connected since there is no priori knowledge about the targets, as can be seen in Fig. 6(d). (3) We suggest an effective method to deal with the unconnected parts of a target, and combine such a set of parts into one representative target. Firstly, we take two connected regions A and B with the size of areas Sa and Sb out from the binary tile image at one time, and then evaluate the Euclidean Distance Dab between the centroids of A and B, if Dab ≤ dmax , A and B could be combined as one part, where, dmax = α ⋅ max( Sa, Sb ) and α is a constant. The new part could be selected in the flowing round, and this is an iterative process. Finally, these unconnected parts of a target can be deemed as an integrated object. The closer and bigger the parts are, the higher the probability for combining the parts as one target. (4) The centroids of candidate targets are evaluated as Focus of Attention (FOA) in the saliency tiles. (5) The relative object size is known as an important factor for detection results [12]. An adaptive method is applied to extract ROIs, in view of the size of candidate target. Guided by the FOAs, we extract ROIs from the saliency tiles, for a certain FOA, binary and original ROIs are obtained, the ROIs are L × L squares with the FOAs as their centroids, L is defined as L = σ ST , where ST is the area (connected region) of candidate target, and σ is a constant. Fig. 6(e) is an example to show the adaptive ROIs extraction.

Fig. 6. Example of multiresource ROIs extraction, the process of FOAs extraction shows in Fig. (a)-(d), (e) the result of adaptive multiresource ROIs extraction

A Hierarchical Salient-Region Based Algorithm for Ship Detection

735

2.4 Target Validation The ROIs searches for targets mainly by the features of contrast or brightness. However, when the remote sensing image contains clutter objects which have similar characteristics with ship, such as cloudlet, islet or strong waves, the contrast or brightness from them will compete with the true ship target. So the false alarm ROIs must be further reduced. The shape and texture features are extracted from binary and original ROIs respectively to eliminate the false alarm ROIs by integrating these features to make validation based on Support Vector Machine (SVM). 2.4.1 Feature Extraction of Multiresource ROIs After obtaining the multiresource ROIs from saliency tiles, we extract the shape and texture features from binary and original ROIs respectively, in terms of that the binary ROI suitable for shape analysis, and original ROI contains sufficient information about texture. (1) Shape features. The ship targets always present approximately rectangular shape in remote sensing images, we extract ratio of long and wide axis (RLW), area of target region (S) and form factor (FF) from binary ROIs to describe the shape feature. z z

z

RLW: The length and width of a target are obtained by calculating the minimum enclosing rectangle of binary target region. S: It simply counting the total amounts of the pixels in the connected target region, the feature could eliminate the oversize and small false alarms, since the size of ship targets has a certain range in remote sensing images during a given task. FF: FF = S / P2 , where P is the perimeter. The lower the shape complexity of candidate target is, the higher the corresponding value is and vice versa.

(2) Texture features. Optical remote sensing image contains sufficient information about texture, so the distribution-based and frequency -based descriptors are extracted from original ROIs. z z

Distribution-based descriptor. We estimate the following statistical features from gray level co-occurrence matrix: contrast, correlation, energy and homogeneity. Frequency-based descriptor. 2-D Fourier Transform is taken of each candidate ROI, and then the mean and standard deviation are calculated from it.

Finally, the feature vector for each ROI consists of 9 features, among which 3 from shape, and 6 from texture. 2.4.2 Elimination of Nontarget ROIs by SVM We choose Support Vector Machine (SVM) with radial basis function (RBF) kernel to validate the ship targets. SVM usually shows outstanding performance in other pattern recognition tasks as in [13]. The target validation is regarded as a binarypattern issue in our task, since we are not interest in what the false alarms are, and there are not enough samples for training various false alarms. The training data (ROIs) were collected from Google Earth manually, which contains 393 typical false

736

F. Bi, F. Liu, and L. Gao

alarms and 358 ship ROIs. After the training stage, the hyperplane is obtained to discriminate two classes (target and nontarget). The validation step (classification) consists of taking the extracted ROIs, computing the feature vectors and then applying the pre-computed SVM (hyperplane) to decision the ship targets.

3 Experimental Results and Discussion To verify the strategy of saliency tile extraction, we select 119 tiles of typical sea background in remote sensing images without ship, and 203 target-tiles from Google Earth, the size of these tiles is 512 × 512 pixels. The statistical results can be seen in Fig. 7, it shows that the RHT of target tiles are greater than those of nontarget ones obviously. When the saliency tiles are extracted with saliency threshold 1.7, the detection rate achieved 97.56% while the false alarm rate is 17.5%. The results show that this strategy ensures low probability of miss with a simple method, as well as most target information can be saved for future process.

Fig. 7. The statistical features of RHT, nontarget tiles on left, target tiles on right

The performance of our proposed hierarchical salient-region based algorithm is evaluated for remote sensing images captured at typical sea background conditions from Google Earth, and some of them contain both land and sea regions. The database consisted of 42 images with the size of 2048 × 2048 pixels for each one. In order to evaluate the performance, 223 ship targets were identified by manual method from all images, and the statistic results of time consumption are mean of all images results. The performance and time consumption are showed in Table 1. The Recall and Precision are defined to evaluate the performance as following: Recall =

Number of correct targets by Algorithm , Total number of targets in data set

Pricision =

Number of correct targets by Algorithm . Total number of extracted objects

(4)

(5)

A Hierarchical Salient-Region Based Algorithm for Ship Detection

737

Table 1. The performance and time consumption of the proposed algorithm Preprocessing Performance

Total tiles 672 Time consumption 5.53s Total performance:

Saliency tile extraction

ROIs extraction

Saliency tiles 217 13.31s

Recall: 91.03%

Precision: 82.52%

ROIs 763 4.98s

Target validation Extracted object Ship target 246 203 15.74s

Total time consumed: 39.57s

As can be seen in Table 1, the strategy of saliency tile extraction reduces the computational complexity of whole algorithm by removing out most of the nonsaliency tiles data (probably 67.71%) with a simple strategy. Lots of ROIs are extracted since there are many objects have high contrast and characteristics similar to the ship target in the real remote sensing images, and this stage also eliminates a lot of nontarget regions as the ROI only a small region-chip. The stage of target validation eliminates most tough false alarms by using the pattern recognition method. Although SVM is a complex operator, it shows inconspicuous time consumption in the whole framework, since the stage of target validation only be applied at the extracted ROIs. Note that the time-consumption do not increase obviously, even as the complexity of analysis strategies are increase gradually in our proposed algorithm. It is due to that our algorithm is a mechanism which filters out redundant nontargets information and detects the more relevant regions at each stage hierarchically, so the data needs to be processed is reduced rapidly for each following stage. The detection results show our algorithm achieves high probabilities of recall and correct detection with a low rate of false alarm, and distributes the computational resource reasonably.

4 Conclusions This paper presents innovative and computationally economical methods for automatic ship detection in remote sensing images. The hierarchical attention strategies allow fast extracting saliency regions to be analyzed by more complex and time consuming processes. firstly, an statistical strategy for finding saliency tile is controlled by low-level feature extraction mechanisms, which crudely select saliency tiles from entire visual scene, in a parallel process, and then both binary and original ROIs of each saliency object is taken out from the saliency tiles, which can provide enough information to describe the candidate targets. Then, shape and texture features are extracted from the multiresource ROIs respectively. Finally, SVM is applied for target validation. The experimental results show that the proposed algorithm achieves satisfy detection performance and maintains low false alarm rates in the face of varying and complex backgrounds with a reasonable time consumption.

References 1. Crisp, D.J.: The State-of-the-Art in Ship Detection in Synthetic Aperture Radar Imagery. Technical Report. Australian Government, Department of Defense Press, Austria (2004) 2. Buck, H., et al.: Enhanced Ship Detection from Overhead Imagery. In: Proceedings of SPIE. International Society for Optical Engineering, vol. 6945, pp. 1–12. SPIE Press, Orlando (2008)

738

F. Bi, F. Liu, and L. Gao

3. Scholl, B.J.: Objects and Attention: The State of The Art. J. COGNITION 80, 1–46 (2001) 4. Sun, Y., Fisher, R.: Object-Based Visual Attention for Computer Vision. J. Artificial Intelligence 146, 77–123 (2003) 5. Deco, G., Schurmann, B.: A Hierarchical Neural System with Antinational Top–Down Enhancement of the Spatial Resolution for Object Recognition. J. Vision Research 40, 2845–2859 (2000) 6. Itti, L., Koch, C.: Computational Modelling of Visual Attention. J. Nature Reviews Neuroscience 2, 194–203 (2001) 7. Meur, O.L., Callet, P.L., Barba, D.: A Coherent Computational Approach to Model Bottom-Up Visual Attention. J. IEEE Transactions on Pattern Analysis and Machine Intelligence 28, 802–817 (2006) 8. Corbane, C., Pecoul, E., Demagistri, L., et al.: Fully Automated Procedure for Ship Detection Using Optical Satellite Imagery. In: Proceedings of the SPIE, vol. 7150, pp. 1– 13. SPIE Press, New Caledonia (2008) 9. Manish, C., John, G.: Automatic Target Detection and Tracking in Forward–Looking Infrared Image Sequences Using Morphological Connected Operators. J. Electron. Imaging 13, 1–22 (2004) 10. Vincent, L.: Morphological Grayscale Reconstruction in Image Analysis: Applications and Efficient Algorithms. J. IEEE Transactions on Image Processing 2, 176–201 (1993) 11. Lee, S., Kim, K., Kim, J.Y., et al.: Familiarity Based Unified Visual Attention Model for Fast and Robust Object Recognition. J. Pattern Recognition 43, 1116–1128 (2009) 12. Walther, D., Rutishauser, U., Koch, C., et al.: Selective Visual Attention Enables Learning and Recognition of Multiple Objects in Cluttered Scenes. J. Computer Vision and Image Understanding 100, 41–63 (2005) 13. Inglada, J.: Automatic Recognition of Man-Made Objects in High Resolution Optical Remote Sensing Images by SVM Classification of Geometric Image Features. J. ISPRS Journal of Photogrammetry & Remote Sensing 62, 236–248 (2007)

Turning Tool Wear Monitoring Based on Fuzzy Cluster Analysis Hongtao Chen, Sui Huang, Dengwan Li, and Pan Fu Institute of Mechanical Engineering of Southwest Jiaotong University, Chengdu 610031, China [email protected]

Abstract. There are several stages of tool wear in turning process. We collect of the force signals and vibration signals at each stage. Using wavelet filtering and power spectrum methods, typical parameters changes are detected. We extract the signal feature for fuzzy clustering. Experimental results show that the tool wear monitoring is achieved in turning by using this pattern recognition method. Keywords: Tool wear, Power spectrum, Feature extraction, Fuzzy cluster analysis.

1 Introduction Cutting tool wear directly affects the quality, efficiency and production safety. How to estimate the tool wear state is the key problem to be solved urgently in the cutting process. We usually use wear-related signals for an indirect measurement of tool wear, because it is difficult to achieve it with a direct measurement. And among many signals, cutting force signals and vibration signals are very direct and real. They are free from chips and coolant effect, their signal amplitude is larger than others and they have strong anti-interference ability. Dealing with these two kinds of signals has great practical significance for tool wear monitoring [1]. A large number of experiments have shown that the cutting force increases with the increase of tool wear. We can monitor the power of the signals to understand the changes in tool wear. However, a change in force and vibration signal is due to a change in cutting conditions, such as the hardness of workpiece material, cutting parameters and even the length of the workpiece, so the actual cutting signal may be changed by many factors. In that case, we can do a series experiments and analysis to collect dates before cutting, then do tool wear pattern recognition [2]. Fuzzy cluster analysis is an advanced method of multivariate analysis. It can be formed from a large number of data and do not rely on objective information, expert experience and other subjective evaluation of the actual general (inductive) system. In this respect, it is better than artificial neural network. And by the clustering distance, we know clearly which tool wear state is the closest to the target under test. It is a good monitoring method for subsequent turning process. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 739–745. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

740

H. Chen et al.

2 Fuzzy Clustering Cluster analysis is a multivariate analysis method which studies the classification of things. In daily life, such things (samples) which come into contact with us are classified according to their nature and use. And we call this classification process is the cluster analysis. Here is the analysis of fuzzy clustering steps after the data sample collected, clustering as the following method [3] [4]. (1) Based on the domain U. (2) Set fuzzy relationship matrix R, seeking similarity relation matrix, which should be consistent with reflexivity and symmetry.

1 ⇔i= j ⎧⎪ m = r ij ⎨1 − c∑ xik − x jk ⇔ i ≠ j ⎪⎩ k =1

(1)

≦≦

Where c is the appropriate selection number, it makes 0 rij 1. (3) Find the fuzzy equivalence relation matrix R*. That R cube was R°R=R2, cube R2°R2=R4, until R2K=RK. Then RK is a fuzzy equivalence relation matrix. (4) Seeking Rλ and clustering 0 λ 1 . (5) Drawing dynamic clustering diagram.

（ ＜≦ ）

3 Test-Bed Structures and Data Collection 3.1 Turning Data Acquisition Schematic Fig. 1 shows the Schematic diagram of data acquisition.

Fig. 1. Schematic diagram of data acquisition

Turning Tool Wear Monitoring Based on Fuzzy Cluster Analysis

741

3.2 Test Conditions The test conditions are: z z z z z

Test machine: CK6143/100 CNC lathe; Workpiece blank: TiAl 6 V4, Φ 65mm×800mm; Test tool: M42+4(HSS); cutting edge angle 90°, rake angle 12°, clearance angle 12°, Angle of Inclination 0°; Cutting parameters: cutting velocity 25rpm, feed rate 0.20mm/rev, cutting depth 1mm; Cooling conditions: -20°C cold-air jet cutting.

3.3 Sensor and Installation 3.3.1 Force Sensor Kistler9257B Swiss Kistler Dynamometer provides dynamics and quasi-static measurement of three orthogonal force (Fx, Fy and Fz) acting from any direction onto the tool. The dynamometer has high rigidity and hence high natural frequency. The high resolution enables very small dynamic changes to be measured in large forces. Unloading the original lathe square turret tool block, we re-designed the tool seat which matched the dynamometer, as shown in Fig. 1. 3.3.2 Vibration Acceleration Sensor Kistler 8702B50M1 K-Shear accelerometers are shear mode shock and vibration measuring instruments. In accordance with the test, the three sensors comprise vibration measuring means mounted on the tool holder, as shown in Fig. 2. The vibrations of tool are measured during turning process to produce vibration signals corresponding to actual vibrations in mutually perpendicular directions.

Workpiece

Vibration sensor Tool holder

Dynamometer

Fig. 2. Data acquisition device

742

H. Chen et al.

3.3.3 IPC DEWE-3021 series PC instrument (portable data acquisition system) is used in our test. This system with up to 16 x MDAQ input channels can collect sensor data in time, single-channel sampling frequency of 100 KHz.

4 Tool Wear States Recognition 4.1 Signal Analysis For most cutting experiments, the flank wear developed in three stages: rapid initial wear, gradual uniform wear and accelerating wear. When initial tool surface and workpiece surface run-in, glitch signals occur within a short time and power spectrum will also be short-lived high-frequency pulse. Stage of uniform wear is a continuous process of gradual change, reflecting the trend of changes in tool wear signal. Four stages tool wear off-line are measured by SP 3020 two-dimensional optical measuring machine, it shows in Fig. 3. Each feed cuts titanium column 150mm length, that is to say, turning tool trips 149600mm to do a record. When the flank wear criterion VB reaches 0.25mm, the tool is failed.

Fig. 3. Off-line detection of tool wear

As an example Fig.4 and Fig.5 show Z-force signal and vibration signal corresponding to the direction of the power spectrum. We can clearly see the power of the spectral signals and vibration signals more visible trend move to higher frequency.

Turning Tool Wear Monitoring Based on Fuzzy Cluster Analysis

743

Fig. 4. Z-forces signal corresponding to the direction of the power spectrum

Fig. 5. Z-vibration signals corresponding to the direction of the power spectrum

4.2 Feature Extraction Typical features are extracted as a class from the above four kinds of states. Then we cut workpiece any three times and collected the data, extracting the features for clustering analysis. Involving cylindrical turning only in our experiment, therefore, the Y-signal does not focus. Force signals increaseing as the tool wear, it is necessary

744

H. Chen et al.

to research their mean time-domain signals. The RMS value of vibration signals may vary with the tool wear increases, so it is also as a feature of their characteristic values filled into the table 1. One of the difference between fuzzy cluster and artificial neural network is that fuzzy cluster does not need to volume normalize the characteristic values. This method can reduce man-made errors. Table 1. Characteristics value table of fuzzy clustering

0

234.5

Fx Maximum amplitude frequency (Hz) 900

0.1586

Vx Maximum amplitude frequency (Hz) 3800

413.1

Fz Maximum amplitude frequency (Hz) 200

0.2016

Vz Maximum amplitude frequency (Hz) 8400

0.08 to 0.1

240.8

1700

0.1652

4000

427.1

600

0.2128

8800

0.15 to 0.17

247.5

1800

0.1677

4300

439

1500

0.2155

9400

Tool Wear (mm)

Fx Mean (N)

Vx RMS

Fz Mean (N)

Vz RMS

0.25 to 0.27

259.5

1800

0.1738

4800

457.7

1800

0.2231

10300

UUT 1

243.7

1776

0.1643

4100

431.4

1100

0.2126

9100

UUT 2

248.6

1800

0.1684

4500

440.3

1300

0.2148

9300

UUT 3

253.2

1800

0.1729

4700

444.8

1700

0.2196

9700

(*UUT means Unit Under Test).

4.3 Fuzzy Clustering Feature values are built into a reflexive, symmetric relationship matrix R, U={u1,u2,u3……,u7}. According to fuzzy cluster analysis theorem [3], the similarity relation matrix is found. Then through the R2K=RK get fuzzy equivalence relation matrix R*, λ get 0.4 through the debugging clustering. The results of this experiment are programmed with JMP software, and drawn fuzzy two-way clustering diagram shown as Fig. 6.

Fig. 6. Fuzzy two-way clustering for experimental seven turning

Turning Tool Wear Monitoring Based on Fuzzy Cluster Analysis

745

From Fig. 6 we can see clearly that UUT1 tool is a class with 0.08 to 0.1mm tool wear; UUT2 tool is a class with 0.15 to 0.17mm tool wear; UUT 3 tool is a class with 0.25 to 0.27mm tool wear. At the same time, through the distance of the cluster leader in terms, we understand that the UUT2 tool is the closest with 0.15 to 0.17mm tool wear, and the tool wear of UUT3 is the lowest level close with 0.25 to 0.27mm tool wear. As the result of testing, the wear of UUT1, UUT2 and UUT3 measured by twodimensional optical measuring machine off-line are about 0.13mm, 0.17mm and 0.22mm. It is fully consistent with the results of cluster analysis. Then reading from the vertical clustering, Fx Mean and Fz Mean of the force signal are the closest. Vx RMS and Vz RMS of vibration signal are very close too. This can make definite conclusions for what we selected the RMS above.

5 Conclusions Applying fuzzy clustering method in the process of turning, we can find similar one group or several groups of characteristics from the vertical clustering of view. Clustering analysis of the conditions, it can extract information from a large number of objective characteristics for effective feature without expertise. Through the horizontal distance from the leading clustering, we also can determine which pattern is the closest to tool wear. So that we can clear the processing tool wear state recognition, and the process of turning tool condition monitoring applications achieve.

References 1. Wang, H.L., Ma, C.X., Shao, H.: The tool wear and breakage monitoring in turning using neural network. J. Shanghai Jiaotong Univ. 40, 2057–2062 (2006) 2. Choi, D., Kwon, W.T., Chu, C.N.: Real-Time Monitoring of Tool Fracture in Turning Using Sensor Fusion. Int. J. Advanced Manufacturing Tech. 15, 305–310 (1988) 3. Zadeh, L.A.: Fuzzy Sets. Inform. Contr. 8, 338–353 (1965) 4. Dimla, S., Dimla, E.: Sensor signals for tool-wear monitoring in metal cutting operations-A review of methods. Int. J. Mach. Tools Manuf. 40, 1073–1098 (2000)

Part-Machine Clustering: The Comparison between Adaptive Resonance Theory Neural Network and Ant Colony System Bo Xing1,*, Wen-Jing Gao1, Fulufhelo V. Nelwamondo2, Kimberly Battle1, and Tshilidzi Marwala1 1

Faculty of Engineering and the Built Environment, University of Johannesburg, P.O. Box 524 Auckland Park, Johannesburg 2006, South Africa [email protected] 2 Biometrics research group, Modelling and Digital Science Unit, Council for Science and Industrial Research (CSIR), P.O. Box 395, Pretoria 0001, South Africa

Abstract. The aim of part-machine clustering (PMC) in cellular manufacturing systems is to cluster parts that have similar processing requirements into part-families; and machines that meet these requirements into machine-groups. Although PMC problems are known as NP-complete in the literature, extensive research is still conducted in this field because of the considerable practical value of PMC for industries. In this paper, conventional adaptive resonance theory (ART1) neural network method and a novel meta-heuristic approach called ant colony system (ACS) are proposed for solving PMC problems. The experimental results show that ACS performs better than ART1 neural network on the same selected benchmark test problems. A PMC performance measure called grouping efficiency (GE) is also employed to evaluate the clustering result. Keywords: Group technology, Part-machine clustering, Grouping efficiency, Adaptive resonance theory neural network, Ant colony system.

1 Introduction In response to the competitive market’s need for product diversification, increased industrial automation and the trend towards shorter product life cycles, many manufacturing firms have adopted new manufacturing philosophies during the last three decades. Among those new manufacturing philosophies, group technology (GT) has been used to [1]: reduce throughput and material handling times, decrease work-in-progress and finished goods inventories, and increase ability to handle forecast errors. GT is not necessarily a substitute for advanced manufacturing technology, such as flexible manufacturing systems, but it has its own unique advantages and can serve as a building block to smooth the implementation of advanced manufacturing technology *

Corresponding author.

Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 747–755. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

748

B. Xing et al.

[2]. As an application of GT, cellular manufacturing (CM) has been recognized and utilized as an effective solution approach to low-variety, high-volume production as well as high-variety, low-volume production. The key step towards the successful CM system is to establish part-families and their associated machine-groups, which is known as the part-machine clustering (PMC) problems according to literature. Normally the part-family formation is to group parts with similar geometric characteristics or processing requirements to take advantage of their similarities for the manufacturing purpose; while the machine-group formation is to bring dissimilar machines together and dedicate them to the manufacture of one or more part-families [3]. The fundamental objective of PMC is to find mutually independent machine-groups with minimum interaction between groups so that a set of part family can be completely produced in a machine-group. Under the manufacturing environment like this, each group acts as an autonomous island with flow shop efficiency within a larger job shop because of physical adjacency among machines and similarities in manufacturing processes within groups [4]. Unfortunately, the problem of PMC, which clusters multivariate data into mutually exclusive groups, is known to be NP-complete [3]. That means it is difficult to find global optimal solution in polynomial computation time for this kind of combinatorial problem. So the development of efficient algorithms for PMC, which can find acceptable near-optimal solutions for CM system designer, is thus always desirable. Currently, there are many different approaches available in the literature such as mathematical programming, graph theory, hierarchical/non-hierarchical clustering, neural networks, and heuristics approaches [5]. In this paper, we will focus on the discussion of two approaches in solving PMC problems, namely adaptive resonance theory (ART1) neural network and ant colony system (ACS). The remainder of the paper is organized as follows: z z z z z

Section 2 states the binary PMC problems in cellular manufacturing; Section 3 introduces the clustering performance measure; Section 4 describes the working principles of ART1 neural network in solving binary PMC problems; Section 5 presents a detailed description of solving binary PMC problems by using ACS; Section 6 draws conclusions and gives suggestions for the future work.

2 Binary PMC Problems The first class of PMC problems are binary problems in which the production data such as the processing times and the machine capacities are not considered [6]. In this class, the basic input is a binary part-machine incidence matrix (PMIM) A ( = [aij]), where the element aij is “1” or “0” (see Fig. 1a), representing the operational relationships between parts and machines. In a binary-PMIM, “1” means machine i is required to process part j while “0” means part j will not be manufactured on machine i. As show in Fig. 1b, the final clustering result demonstrates that two cells can be formed: parts P02, P05, and machines M02, M04 are in one cell; while parts P01, P03, P04, and machines M01, M03 are in another cell.

PMC: The Comparison between Adaptive Resonance Theory Neural Network and ACS

(a)

749

(b)

Fig. 1. Binary-PMIM (size 4×5) and final clustering result without exceptional parts and bottleneck machines

In this case, there is no “1” outside the diagonal block and no “0” inside the diagonal block so that we call it a perfect result. That means the two cells are completely independent where each part-family will be processed only within one machine-group. Unfortunately, this perfect result for PMC is rarely seen in real situations. As you can see from Fig 2.b, there is a “1” outside the diagonal block. Normally we call part P03 is an exceptional part; and machine M01 is a bottleneck machine. Meanwhile, there is also a “0” inside the diagonal block which we always call it a void. Consequently, an optimal clustering result based on a binary-PMIM input is desired to satisfy two conditions: (i) to minimize the number of “0” inside the diagonal blocks; (ii) to minimize the number of “1” outside the diagonal blocks.

(a)

(b)

Fig. 2. Binary-PMIM (size 4×5) and final clustering result with exceptional parts and bottleneck machines

3 Clustering Performance Measure As shown in the preceding section, a perfect clustering result can be interpreted as 100% grouping. However, due to the existence of voids, exceptional parts and bottleneck machines, the evaluation of clustering performance becomes a vague issue. Therefore, a certain method is required to measure the performance of incomplete clustering. In this paper, we will use grouping efficiency (GE) [7] as a clustering performance measure: the higher value of GE, the better performance of the clustering results. Due to its simplicity of calculation, GE is the most widely used method. The GE is defined as follows:

η =ωη1 + (1 − ω )η 2 .

(1)

750

B. Xing et al.

o−e m⋅ p −o − v total diagonal entries ; η2 = ;ω= dimension of the PMIM input o−e+v m⋅ p −o − v + e m: number of machines; p: number of parts; o: number of "1" in PMIM

where η1 =

e: number of "1" outside the diagonal block; v: number of "0" in the diagonal block. So for the clustering result shown in Fig. 2b, we have m = 4, p = 5, o = 10, e = 1, ω = 9/20, and v = 1. Then we find η1 = 0.9 and η2 = 0.9. Thus, we finally have the GE of this case: η = 90%.

4 ART1 Neural Network for PMC Problems Over the last three decades, neural networks have been widely applied to PMC problems. In general, neural network models can be categorized into three types: feedforward networks, feedback networks, and competitive learning networks [8]. Since the competitive learning network is unsupervised approach, it is very suitable for solving PMC problems because, in practice, no information about correct clustering result is known a priori [9]. In addition, due to the intrinsic characteristic of handling binary input, adaptive resonance theory (ART1) as one of the competitive learning networks is found to be the best suited for solving binary PMC problems. 4.1 How Does ART1 Work in Solving PMC Problems?

Briefly, ART1 for PMC problems works as follows [10]: the columns and rows of a PMIM are binary vectors and they can be applied as inputs to a neural network classifier. Column vectors can first be grouped to obtain an intermediate matrix and then, row vectors of this matrix can be grouped to obtain the final matrix. The ART1 at first classifies the column vectors, based on their similarity. In the intermediate matrix, the similar columns are placed next to each other. This achieves a degree of clustering of the “1” elements in the PMIM. In the next step, the row vectors are applied as inputs to the ART1 which likewise classify them. Reordering the rows, such that similar rows are adjacent lead to the further clustering of the “1” elements resulting in the final matrix. In the example of Fig. 3a (for clarity, “0” elements were omitted in all of the examples presented next), for the classification of the column vectors, column vectors 1 through 7 are presented sequentially at the input layer of the ART1 neural network. Suppose W1 and W2 is initialized so that when I1 = [0 1 1 0 1] is presented, neuron 1 wins, that is I1·W1 > I2·W2. In the first learning iteration there is no exemplar pattern stored at the output neurons, so I1 is stored as the exemplar pattern (T1) at neuron 1, and by using the weight change equations we have W1 = [0 0.5 0.5 0 0.5]. Now when next column vector I2 = [1 0 1 1 0] is presented, there are the following two cases depending on the initial value of W2: z

Case 1: If we assume that I2·W1 > I2·W2, then (I2) AND (T1) is compared against the stored vector at the output neuron 1. In this example, (I2) AND (T1) = [1 0 1 1 0] AND [0 1 1 0 1] = [0 0 1 0 0] and the similarity between I2 and T1 is 0.333 which can be obtained from (2).

PMC: The Comparison between Adaptive Resonance Theory Neural Network and ACS

similarity=

z

number of "1" in (I 2 ) AND (T1 ) . number of "1" in I 2

751

(2)

If the prescribed vigilance parameter ρ is less than 0.333, then the pattern is associated with the first neuron, and T1 changes to [0 0 1 0 0] and W1 is also changed. In the opposite case, T1 and W1 are not changed, and the vector I2 is used to create the initial exemplar at neuron 2. Case 2: If we assume that I2·W1 < I2·W2, then the vector I2 is used to create an initial exemplar at neuron 2.

These iterations are repeated by presenting the input vectors one at a time, until the weight vectors show no change with further training iterations. The part groups can be identified by observation of the output activations, that is, the winning neurons, and all part vectors cause a particular output unit to respond or win are classified as a part-family. Next, the row vectors are also classified based on the same procedure and the machine-groups can be obtained. Fig. 3b shows the final clustering result with ρ = 0.4.

(a)

(b)

Fig. 3. Binary-PMIM (size 5×7) and final clustering result by using ART1 neural network

Based on (1), we have m = 5, p = 7, o = 14, e = 3, ω = 11/35, and v = 3. Then we find η1 = 11/14 and η2 = 6/7. Thus, we finally have the GE of this case: η ≈ 83.5%. 4.2 Numerical Example

In this example, the PMIM shown in Fig. 4a with 9 machines and 9 parts is adopted from [8]. The final clustering result obtained by using ART1 is shown in Fig. 4b. According to (1), we have m = 9, p = 9, o = 32, e = 3, ω = 29/81, and v =12. Then we find η1 = 29/41 and η2 = 37/40. Thus, we finally have the GE of this case: η ≈ 84.7%.

(a)

(b)

Fig. 4. Binary-PMIM (size 9×9) and final clustering result by using ART1 neural network

752

B. Xing et al.

5 ACS for PMC Problems Ant colony optimization (ACO) is a new population-based meta-heuristic approach. It was originally inspired from real ants’ foraging behavior in which real ants can always find the shortest path between their nest and a food source. The first successfully application of ACO was done by using a traveling salesman problem (TSP) as a test problem. Currently, there are several different variants of ACO such as ant colony system (ACS) and ANT-Q [11]. In this paper, we will focus on ACS which was developed by Dorigo and Gambardella [12] and it has three main characteristics: z z z

The state transition rule is applied to provide a direct way to balance between exploration of new edges and exploitation of a priori and accumulated knowledge about the problem; The local pheromone updating rule is applied while ants construct a solution; The global pheromone updating rule is applied only to edges which belong to the best ant tour.

5.1 How Does ACS Work in Solving PMC Problems?

Informally, ACS for machine-groups clustering problems works as follows: k artificial ants are initially positioned on m machines chosen according to some initialization rule (e.g., randomly). Each artificial ant builds a tour by repeatedly applying a stochastic greedy rule (the state transition rule). While constructing its tour, an ant also modifies the amount of pheromone on the visited edges by applying the local pheromone updating rule. Once all ants have terminated their tours, the amount of pheromone on edges is modified again (by applying the global pheromone updating rule). The ACS global pheromone updating rule causes the search to be more directed, by encouraging ants to search in the vicinity of the best solution found so far. In the procedure of building their tours, all ants are guided by both heuristic information (short edges are preferable), and by pheromone information (an edge with a high amount of pheromone is a very desirable choice). These iterations are repeated until the best machine sequence is found among m! different permutations. For part-family (n parts) clustering problems, the same procedure will be carried out again until the best part sequence is found among n! different permutations. 5.2 ACS State Transition Rule

For machine-group clustering problems, the state transition rule is as follows: an ant which is randomly positioned on r-th machine chooses the s-th machine to move to by applying the following rule: ⎧argmax k {τ ru (t )ηruβ (t )} if q ≤ q0 ⎪ u∈N ( t ) i s= ⎨ . ⎪ S q q if > 0 ⎩

(3)

Where q is a random number uniformly distributed in [0~1], q0 is a parameter (0 ≤ q0 ≤ 1), and S is a random variable selected according to the probability distribution given in (4).

PMC: The Comparison between Adaptive Resonance Theory Neural Network and ACS

prsk (t ) =

∑

τ rs (t )η rsβ (t ) . β k τ ru (t )η ru (t )

753

(4)

u ∈N ( t ) i

Where τ is pheromone, Nik(t) is the set of machines that remain to be visited, and β is a parameter which determines the relative importance of pheromone vs. distance (β > 0). The state transition rule resulting from (3) and (4) is called pseudo-random-proportional rule. This state transition rule favours transitions towards machines connected by short edges and with a large amount of pheromone. The parameter q0 is used to balance exploration and exploitation: If q ≤ q0 then the best edge is chosen (exploitation, according to (3)); otherwise algorithm explores. In order to satisfy the constraints that an ant visits all the m different machines, we associate with each artificial ant a data structure called the forbidden list, that saves the machines already visited up to time t and forbids the ant to visit them again before a tour has been completed. When a tour is completed, the forbidden list is used to compute the ant’s current solution (i.e., the length of the path followed by the ant). The forbidden list is then emptied and the ant is free again to choose. 5.3 ACS Local Pheromone Updating Rule

While building the best machine sequence, artificial ants visit edges which connect different machines and change the pheromone level of edges based on (5):

τ (r , s) ← (1 − ρ )iτ (r , s) + ρ iΔτ (r , s ) .

(5)

Where 0 < ρ pm . Then the state components of the interrupted schedule,

( k +1)

, is determined by the following updated formula:

Sch(n, t )

( k +1)

⎧ SchGbest (n, t )( k ) if rand () ≤ pg ⎪ = ⎨ SchPbest (n, t )( k ) if pg < rand () ≤ pg + p p ⎪ 1 − Sch(n, t )( k ) if rand () > p + p g p ⎩

(4)

788

P. Huang et al.

In the equation,

SchGbest (n, t )( k ) is the state component of the global optimal

k th iteration, and SchPbest (n, t )( k ) is the state component of the individual optimal interruption schedule in the k th iteration. rand () is the uniform random number in the interval [0,1].

interruption schedule in the

Construct the following MNLBPSO Algorithm: Step 1. Initialize, and set the evolutionary parameters, such as acceptance probability p p , pg , pm , population size, maximum number of iterations M , external file size

NPSize , and so forth. Randomly generate initial population P0 (the

initial position of the particles); Step 2. Establish the initial external file, and determine all the particles’

PPbest .

PGbest and add it to the external file NP. (The initial external file is the Pareto optimal solutions with regard to the population P0 .) Later determine the initial group’s

Step 3. Update particle states, and generate the next generation of particle swarm. Later apply the particle state update formula (4) to determine the new position of each particle, thus bringing a new generation of particle swarm. ( PGbest is randomly selected from the external file). Step 4. Update external file, and determine the

PPbest of each particle (When

comparing the partial orders of two particles, if they're not comparable, then the order can be determined by the dictionary sequence and the one with smaller quantity ranks first). Later determine PGbest and update external file NP (Actually that is to

Pk +1 ∪ NP ). If the size of the external file is larger than NPSize , then take out the first NPSize according to the dictionary order.

determine the Pareto optimal set of

Step 5. Output the optimal solution set, if the algorithm reaches the maximum number of iterations M or other termination conditions, then stops searching and outputs the external file NP as the optimal solution; otherwise, turn to Step3.

4 Experiment Results In order to compare different experiment results, the paper introduces the example of the literature [13] which includes 19 interruptible loads, 2 kinds of load demand reduction whose reduction of demand and load characteristics data can be found in literature [13]. Experiment 1. For the load demand reduction A, set up the relative parameters: the iteration number 300, the size of particle swarm 200, external file size 20, learning pg = 0.6 , mutation probability pm = 0.1 , running time probability p p = 0.3

、

200. And the experiment results are given in Table 1.

A New BPSO Algorithm and Applications in Interruptible Load Management

789

Experiment 2. For the load demand reduction B, set up the relative parameters: the iteration number 300, the size of particle swarm 200, external file size 20, learning pg = 0.7 , mutation probability pm = 0.05 , running probability p p = 0.25

、

time 200. As the iteration goes on, the evolutionary change of the objective function is displayed in Figure 1, and the experiment results are given in Table 2.

Fig. 1. The improvement of the objective function with the iterations going Table 1. The comparison of the results between MNLBPSO and BPSO (experiment 1) Hour

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

BPSO

（ ）

Curtailment Demand KW 0 0 0 0 444 444 440 0 340 820 460 80 0 0 0 0

MNLBPSO

（ ）

Curtailment Demand KW 0 0 0 0 444 444 444 0 340 812 456 72 0 0 0 0

total payment cost BPSO($)

MNLBPSO($)

63753

60241

790

P. Huang et al. Table 2. The comparison of the results between MNLBPSO and BPSO (experiment 2)

MNL-BSPO Payment cost

Mean Standard deviation maximum minimum BSPO $ MNLBSPO($)

（）

total payment cost

190652 2031 203530 186500 187852 186500

The experiment results show that, MNLBSPO is able to obtain better optimal schedules with lower total payment cost than BSPO, and these two algorithms can save 3512$ and 1352$ respectively. MNLBSPO doesn’t need to set up several penalty parameters, thus avoiding the difficulty of parameter selection stemmed from the sensitivity of the evolutionary process to penalty parameters. In MNLBSPO, one iteration can obtain multiple (20) optimal solutions, offering more choices for the actual ILS. And MNLBSPO (iterate 300 times) has better convergence than BSPO (iterate 500 times).

5 Conclusions This paper proposes a new natural learning mode based on BPSO, which is named MNLBPSO. Using a relaxing restriction method, we establish an unconstrained multiobjective optimization model for the ILS problem, and we apply the new algorithm, MNLBPSO, to solve that model. The two experiment results show that MNLBSPO is able to obtain better schemes with much less iteration time than BSPO. Also NLBSPO and MNLBPSO (put forward in multi-objective programming model of ILM problem) reasonably avoid the difficulty of selecting the penalty parameters, and can obtain multiple optimal solutions in one iteration, offering more available choices for the actual ILS. This new algorithm, MNLBPSO, is applicable to those complicated binary integer programming problems in other science and engineering fields. Acknowledgments. This study was partly supported by National Natural Science Foundation of China 19902005 and Major State Basic Research Development Program of China (20091072).

（

）

References 1. Xue, Y., Luo, Y.H., Li, B.J., Luo, J.Y.: An overview of the involvement of interruptible load in the system backup. Automation of Electric Power Systems 31, 1–6 (2007) 2. Zhang, S.H., Liu, G., Li, Y.Z.: Interruptible load management model with early notification. Proceedings of the Chinese Society of Universities for Electric Power System and its Automation 20, 63–66 (2008) 3. Kirschen, D.S.: Demand-side view of electricity markets. IEEE Trans. on Power Systems 18, 520–527 (2003)

A New BPSO Algorithm and Applications in Interruptible Load Management

791

4. Boisvert, R.N., Cappers, P.A., Neenan, B.: The benefits of customer participation in wholesale electricity markets. The Electricity Journal 15, 41–51 (2006) 5. Li, H.Y., Li, Y.Z., Zhang, S.H.: Model and algorithm of transmission congestion management based on interruptible load. Automation of Electric Power Systems 30, 18–21 (2002) 6. Zhou, C.M., Jiang, H., He, Y.Q., Wang, Y., Peng, J.C.: A multi-objective fuzzy optimization of congestion management with participation of interruptible loads. Power System Technology 30, 27–32 (2008) 7. Jazayeri, P., Schellenberg, A., Rosehart, W.D., Doudna, J., Widergren, S., Lawrence, D., Mickey, J., Jones, S.: A Survey of load control programs for price and system stability. IEEE Transactions on Power Systems 20, 1504–1509 (2005) 8. Majumdar, S., Chattopadhyay, D., Parikh, J.: Interruptible load management using optimal power flow analysis. IEEE Transactions on Power Systems 11, 715–720 (1996) 9. Bai, J., Gooi, H.B., Xia, L.M., Strbac, G., Venkatesh, B.: A probabilistic reserve market incorporating interruptible load. IEEE Transactions on Power Systems 21, 1079–1087 (2006) 10. Huang, K.Y.: Demand subscription services—an iterative dynamic programming for the substation suffering from capacity shortage. IEEE Transactions on Power Systems 18, 947–953 (2003) 11. Huang, K.Y., Chin, H.C., Huang, Y.C.: A model reference adaptive control strategy for interruptible load management. IEEE Transactions on Power Systems 19, 683–689 (2004) 12. Valle, Y.D., Venayagamoorthy, G.K., Mohagheghi, S., Hernandez, J.C., Harley, R.G.: Particle swarm optimization: basic concepts, variants and applications in power systems. IEEE Transactions on Evolutionary Computation 12, 171–195 (2008) 13. Pedrasa, M.A., Spooner, T.D., MacGill, I.F.: Scheduling of demand side resources using binary particle swarm optimization. IEEE Transactions on Power Systems 24, 1173–1181 (2009) 14. Kennedy, J., Eberhart, R.C.: Particle swarm optimization. In: IEEE Int. Conf. Neural Networks, vol. 4, pp. 1942–1948 (1995) 15. Kennedy, J., Eberhart, R.C.: A discrete binary version of the particle swarm algorithm. In: IEEE Int. Conf. Systems, Man and Cybernetics, vol. 5, pp. 4104–4108 (1997) 16. Gao, H.B., Zhou, C., Gao, L.: General particle swarm optimization model. Chinese Journal of Computers 28, 1980–1987 (2005) 17. Xu, Y.C., Xiao, R.B.: An improved binary particle swarm optimizer. Pattern Recognition and Artificial Intelligence 20, 788–793 (2007)

Force Identification by Using Support Vector Machine and Differential Evolution Optimization Zhichao Fu and Wei Cheng School of Aeronautic Science and Engineering, Beijing University of Aeronautics and Astronautics, Beijing, 100191, P.R. China [email protected]

Abstract. A novel method is presented to determine the external dynamic forces applied on structures from measured structural responses in this paper. The method utilizes a new SVM-DE model that hybridized the differential evolution (DE) technique and support vector machines (SVM) to resolve the problem of force identification. Both numerical simulations and experimental study are conducted to demonstrate the effectiveness, robustness and applicability of the proposed method. It is promising that the proposed method is practical to the real-life application. Keywords: force identification, support vector machine, differential evolution.

1 Introduction The knowledge of the dynamic forces is an essential requirement during the design and optimization stages of mechanical systems. However, it is difficult or impossible to directly measure the dynamic forces acting on a structure in many practical cases. This makes alternative methods of force estimation necessary and very valuable. Force estimation using vibration data has attracted a lot of attention from researchers. The classical approach to force reconstruction is to use a frequency domain technique [1].The forces acting on the structure is estimated from the multipliers of the FRF matrix and the frequency domain of responses. It is preferable in some applications to have a time domain algorithm capable of estimating the forces acting on a structure in real time. There are two main time domain techniques: the Sum of Weighted Accelerations Technique (SWAT) [2] and the Inverse Structural Filter (ISF) [2]. Allen and Carne [2] presented the comparison between the ISF and SWAT in time domain force identification. The support vector machine (SVM) is an emerging technique for learning relationships in data within the framework of statistical learning theory [3]. It is superior in robustness because it adopts the ε − insensitive loss function and the 'Max-Margin' idea. The SVM technique has been applied successfully in many fields. Recently, the application of the SVM was employed for structural dynamics. Mita et al Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 793–801. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

794

Z. Fu and W. Cheng

[4] employed SVM to detect structural damages based on observed modal frequency changes. Jian Zhang et al [5] wrote several papers on SVM for structural health monitoring and structural identification. Lute et al [6] applied SVM to analyze the aerodynamic characteristics of cable stayed bridges. Bornn [7] utilized SVM and nonlinear time series model for the structural health monitoring. In this present study, SVM is applied to the dynamic force estimation. And the SVM hyper-parameters are selected based on differential evolution (DE) technique. As a novel evolutionary computation technique, DE has gained much attention and wide applications since Storn and Price introduced the algorithm in 1995 [8]. It has demonstrated its power and robustness in a variety of applications, such as neural network learning [9], system identification [10]. The purpose of the present study is to employ support vector regression (SVR) technique combined with DE algorithm to resolve the problem of inverse force determination. This paper is organized as follows: In Section 2, the SVM and DE are described briefly, respectively. In Section 3, hyper-parameter selection based on DE is explained in detail. In Section 4, the present scheme is applied to the force reconstruction problem with both numerical cases and an experiment study. Section 5 gives summary and conclusions.

2 Method The quality of SVM models strongly depends on a proper setting of parameters. In this research, the DE technique is employed to select the optimal hyper-parameters of the SVM model including penalty parameter and the kernel function parameters. 2.1 Support Vector Machine Support vector regression (SVR) aims at producing an optimum regression function to predict future data accurately [3]. Learning systems for regression estimation can be described

as

follows

[6].

Given

a

set

of

input/output

training

data

D = {( x1 , y1 ), ( x 2 , y 2 ), " , ( x n , y n )} ∈ R × R , we need to estimate a function n

f : R n → R that will correctly predict unseen examples generated from the same

underlying probability distribution as the training data. The generic SVR regression estimation function is expressed

f ( x) = (w ⋅ Φ( x)) + b

(1)

where w ∈ R n ,b ∈ R , Φ denotes a nonlinear transformation from R n to a higher dimensional feature space. The goal is to find the value of w and b that minimize the regression risk. The optimal function is given by the minimum of the functional

Force Identification by Using SVM and Differential Evolution Optimization

Ψ(w,ξ ) =

795

n 1 2 w + C∑ (ξi− + ξ i+ ) 2 i =1

(2)

where the constant C controls the trade-off between complexity and losses. ξ i− ,ξ i+ are non-negative slack variables representing upper and lower constraints. The ε − insensitive loss function is given by Eq. (3a) [5]

⎧⎪ f ( x) − y − ε Γ( f ( x) − y ) = ⎨ ⎪⎩0

for f ( x) − y ≥ ε for f ( x) − y < ε

(3a)

The optimization problem is solved while minimizing the regression function and loss functions simultaneously by Eq. (3b) l

L(α ∗ ,α ) = ∑ (α i ( y i + ε ) − α i∗ ( y i − ε )) + i =1

1 l ∑ (α i∗ − α i )(α ∗j − α j ) K ( xi , x j ) 2 i , j =1

(3b)

where the dual variables α ,α ∗ are Lagrange multipliers and kernel K ( xi , x j ) is a symmetric function and ε is the e-insensitive. In practice, a low degree polynomial kernel or RBF kernel with a reasonable width is a good initial trial. In this work, RBF kernel is considered. It is defined as follows:

K ( x, y ) = exp(−

x− y 2σ 2

)

(4)

Thus, the approximation function is expressed as p

f ( x) = ∑ (α i∗ − α i ) K ( xi , x j ) + b

(5)

i =1

where p is the number of support vectors. 2.2 Differential Evolution (DE) [10] The DE algorithm is a population based algorithm like genetic algorithms using the similar operators: mutation, crossover and selection. An optimization task consisting of n parameters can be represented by an n- dimensional vector. Let S ∈ R n be the search space of the problem under consideration. Then, the DE algorithm utilizes NP, T n-dimensional vectors x i = (xi1 , xi 2 ," , xin ) ∈ S , i = 1,2," , NP as a population for each iteration, called a generation of the algorithm. In this paper, we use the DE scheme classified using notation as DE/rand/1/bin strategy [8].

796

Z. Fu and W. Cheng

According to the mutation operator, for each individual x ig ,i = 1," , NP , at the

(

generation g, a mutation vector v ig +1 = vig1+1 , vig2+1 ," , v ing +1

)

T

is determined using one of

the following equation

(

v ig +1 = x rg1 + F x rg2 − x rg3

)

(6)

where r1, r2, and r3 are mutually different integers, randomly selected from the set {1,2 ," .NP} . F is a real parameter, called mutation constant. After the mutation phase, it’s crossover operator. For each mutant vector v ig +1 , a trial

(

vector u ig +1 = u ig1+1 ,u ig2+1 ," ,u ing +1

)

T

is generated by the following scheme,

⎧⎪ v g +1 if (rand(j ) ≤ CR) or ( j = rand(i)) uijg +1 = ⎨ gij ⎪⎩xij if (rand(j) > CR) or ( j ≠ rand(i))

(7)

where j=1, 2… n; rand(j) is the jth independent random number uniformly distributed in the range of [0,1]. rand(i) is a randomly chosen index from the set {1,2,...,n}, and CR is user defined constant ∈ [0,1] that controls the diversity of the population. DE employs a greedy selection process that the better one of new offspring and its parent wins the competition. In this paper, the minimization optimization is considered. Thus, if f denotes the objective function under consideration, then ⎧u g +1 if ( f (u ig +1 ) < f (x ig )) x ig +1 = ⎨ ig g +1 g ⎩ x i if ( f (u i ) ≥ f (x i ))

(8)

Thus, each individual of the trial vector is compared with its parent vector and the better one is passed to the next generation. These steps are repeated until specified termination criterion is reached. DE has three key parameters: scaling factor (F), crossover control parameter (CR) and population size (NP). Generally, with a population size of 10n, F=0.8, CR within 0.8-1.0 appear to be reasonably good value to generate satisfactory results.

3 SVM-DE Hybrid Model In this work, there are two parameters to be optimized, which are regularization parameter C and kernel parameter σ . The fitness function of each individual is evaluated by the following equation:

Force Identification by Using SVM and Differential Evolution Optimization

RMSE =

1 n

797

n

∑ ( y pre (i) − y ori (i)) 2

(9)

i =1

where n is the number of sample points in the test data set. y pre , y ori are predicted and original values, respectively. RMSE is the fitness function. In summary, the SVM-DE hybrid method for input estimation is implemented as follows: (1) structural dynamic responses are preprocessed to zero mean and unit standard deviation; (2) the SVR-DE hybrid method are utilized to obtain optimal hyper-parameters; (3) force is estimated from a new data set using the trained SVM model. The principle of the proposed method is demonstrated in Fig.1.

Fig. 1. The principle of proposed method

4 Numerical Cases and Experiment Study The aim of this section is to illustrate the properties of the proposed method to solve the input estimation problem in structural dynamics. In this section, both numerical simulation cases and experimental tests are investigated. The results demonstrate that the proposed method is efficient to solve input estimation problem. 4.1 Numerical Simulations

The model, depicted in Fig.2, represents a 3 DOF mass-spring system and it is used in the following numerical examples. Stiffness of all springs is k = 100 N / m ; and all masses are equal m = 100kg . A mass-proportional damping with a proportionality constant of 1% is added to the model. The system is excited by

798

Z. Fu and W. Cheng

transient force applied to mass m1 . In this paper, the transient excitation is given as follows. u

(1  cos 2Sft ) sin 6Sft , 0  t  1 / f

(10)

where f=0.5Hz. It is sampled 30 seconds with a sampling frequency f s = 200Hz .

Fig. 2. Discrete structure with 3 DOF

To simulate the field measurement, a set of contaminated structural dynamic responses are constructed by artificially adding different noise levels of SNR (signal-to-noise ratio)=50, 40, 30, 20 and 10. The comparison will be made quantitatively by the way of the relative estimation error:

y exact − y estimated ~ × 100% f = y exact

(11)

For transient force estimation, both noise-free and noise cases are considered herein. The comparison of exact input and estimated force with noise-free accelerations are provided in Fig.3. It is easily observed that excellent accuracy is attained. Figure.4 illustrates the case when input estimation is carried out with the accelerations contaminated by noise. From Fig.4, it is clear that there is fairly good agreement between the exact and identified forces at the noise level of SNR=10. To conduct further investigation, input estimation is also carried out under different noise levels of SNR=50, 40, 30, 20. The quantitative results are listed in Table.1.

Table 1. Relative error under different conditions

SNR Noise-free

Relative estimation error (%) 50 40 30 20 10

1.61

1.92

2.25

2.81

3.54

9.88

Force Identification by Using SVM and Differential Evolution Optimization

799

Fig. 3. Transient force recovered from noise-free responses

Fig. 4. Transient force recovered from responses at the noise level of SNR=10.

4.2 Experiment Study The novel proposed method is robust in force estimation in the above numerical cases. An experiment on force determination of an test rig is conducted to validate the effectiveness and applicability of the proposed novel scheme in real practice. The test sample is made of aluminum, as shown in Fig.5. Eight piezoelectrical sensors are mounted to measure the responses. The impulse load was applied using a PCB instrumented hammer. A LMS data acquisition system was set up to measure signals from the sensors. The sampling frequency was set to 2048Hz and total 8 seconds were sampled for each channel per impact from each measurement. A pre-trigger was set to ensure that all of the appropriate waveforms were recorded. In this experiment, only 9 channels were adopted for data acquisition.

800

Z. Fu and W. Cheng

Fig. 5. Photo of experiment test bed rig

The measured forces and corresponding responses are required to estimate input. The experimental force determined using the proposed method is produced in Fig.6. It is clear that the identified impact force history agrees with the experimental one very well. Thus, the effectiveness of the new method of force estimation is validated.

Fig. 6. Experimental force recovered from measured responses

5 Conclusions This paper presented a novel method to determine the external force applied to a structure. This identification strategy is based on SVM hybrid DE technique. The approach is tested on simulations and real-life measurements. The results demonstrate that the proposed method can produce the identified force history accurately. It extends the SVM to the inverse problem successfully. The robustness of the proposed method is validated from the structure dynamic responses under different levels of noise. Further

Force Identification by Using SVM and Differential Evolution Optimization

801

studies will focus on the case of multi-point excitation and some open problems such as the number of sensors, types of structural responses.

References 1. Doyle, J.F.: Experimental determining the contact force during the transverse impact of an orthotropic plate. Journal of Sound and Vibration 118(3), 441–448 (1987) 2. Allen, M.S., Carne, T.G.: Comparison of Inverse Structural Filter (ISF) and Sum of Weighted Accelerations Technique (SWAT) Time Domain Force Identification Methods. In: 47th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, Newport, Rhode Island (2006) 3. Vapnik, V.N.: Statistical learning theory. Wiley, New York (1998) 4. Mita, A., Fujimoto, A.: Active detection of loosened bolts using ultrasonic waves and support vector machines. In: Proceeding of the 5th international workshop on structural health monitoring, pp. 1017–1024 (2005) 5. Zhang, J., Sato, T., Iai, S., Hutchinson, T.: A pattern recognition technique for structural identification using observed vibration signals: Linear case studies. Engineering Structures 30, 1439–1446 (2008) 6. Lute, V., Upadhyay, A., Singh, K.K.: Support vector machine based aerodynamic analysis of cable stayed bridges. Advances in Engineering Software 40, 830–835 (2009) 7. Bornn, L., Farrar, C.R., Park, G., Farinholt, K.: Structural Health Monitoring With Autoregressive Support Vector Machines. ASME Journal of Vibration and Acoustics 131(4), 021004-1– 021004-9 (2009) 8. Storn, R., Proce, K.: Differential evolution-a simple and efficient adaptive scheme for global optimization over continuous spaces. Technical REP. No. TR-95-012, International Computer Science Institute, Berkley (CA) (1995) 9. Ilonen, J., Kamarainen, J.K., Lampien, J.: Differential evolution training algorithm for feed-forward neural networks. Neural Processing Letters 7(1), 93–105 (2003) 10. Tang, H., Xue, S., Fan, C.: Differential evolution strategy for structural system identification. Computers and Structures 86, 2004–2012 (2008)

Soft Sensor Modeling of Ball Mill Load via Principal Component Analysis and Support Vector Machines Jian Tang1, Lijie Zhao1,2, Wen Yu3, Heng Yue1, and Tianyou Chai1 1

Key Laboratory of Integrated Automation of Process Industry, Ministry of Education, Northeastern University, Shenyang 110819, China 2 College of Information Engineering, Shenyang Institute of Chemical Technology, Shenyang 110142, China 3 Departamento de Control Automatico, CINVESTAV-IPN, Av.IPN 2508, México D.F. 07360, México [email protected], [email protected], [email protected], [email protected], [email protected]

Abstract. In wet ball mill, measurement accuracy of mill load (ML) is very important. It affects production capacity and energy efficiency. A soft sensor method is proposed to estimate the mill load in this paper. Vibration signal of mill shell in time domain is first transformed into power spectral density (PSD) using fast Fourier transform (FFT), such that the relative amplitudes of different frequencies contain mill load information directly. Feature variables at low, medium and high frequency bands are extracted through principal component analysis (PCA), which selects input as a preprocessing procedure to improve the modeling performance. Three support vector machine (SVM) models are built to predict the mill operating parameters. A case study shows that proposed soft sensor method has higher accuracy and better predictive performance than the other normal approaches. Keywords: Mill load, Soft sensor, Spectral data, Principal component analysis, Support vector machine.

1 Introduction Ball mills have been used widely in the grinding process, which implement the comminuting mainly by the impaction of the balls. Due to the lack of reliable on-line sensors to measure mill load (ML), the mills have to always operate at the low grinding production rate [1]. Real time ML measurement has not been accomplished completely [2]. Recently, the instrumentation of ML based on the signals of mill shell was developed for dry ball mill [3,4]. The wet ball mill load in grinding process is mainly detected by the experience of the experts to estimate the state of the ML [5]. Some key parameters of the wet ball mill can also be accomplished by measuring the Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 803–810. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

804

J. Tang et al.

mechanical vibration of axis and the acoustic pressure near the mill, which shows that some feature frequency bands are related to the operating parameters such as pulp density, particle size etc [6]. Therefore, it is essential to develop a method to monitor the operating parameters inside the ball mill which present the ball load, material load and water load of the wet ball mill [7]. In this paper, a soft sensor method based on FFT, PCA and SVM is proposed to estimate the mill operating parameters by measuring the vibration signals of the mill shell. First, the power spectral density (PSD) of the vibration signal is obtained using FFT. Since the spectral variables are collinear and the PSD can be divided into three partitions evidently, PCA is used to extract the feature variable of the spectral data at low, medium, high frequency bands respectively. SVM employs the criterion that minimizes the structure risk, and which is more prominent in the small-sample learning [8,9]. So, SVM models are developed to monitor the parameters of ML.

2 Scheme of the Grinding Process and Mill Load The two-stage closed-loop grinding circuit is generally adopted in the iron ore processing plants in China [1]. The grinding production rate is always decided by the ball mill in the first stage.

Fig. 1. Schematic diagram of the grinding circuits system

Mill load is defined as the total load of the grinding charge and material in the mill. Three operating parameters (pulp density, medium charge ratio and material to ball volume ratio) are usually used to represent the ML for the wet ball mill [8]. In this paper, a new operating parameter charge volume ratio which is defined as the total load to mill volume ratio, replaces the ‘medium charge ratio’. The relations between ML and mill operating parameters can be formulated as follows:

L = F ( Lb , Lm , Lw ),

(1)

C = Lm ( Lm + Lw ) ,

(2)

Soft Sensor Modeling of Ball Mill Load via PCA and Support Vector Machines

ϕ mb = Vm Vμ = φbmw = where

μ

Lm

ρm

1− μ

Vball ,

805

(3)

Vm + Vw + Vball Lm ρ m + Lw ρ w + Lb ρ b , = Vmill Vmill

(4)

L is the total load of the mill; Lb , Lw and Lm are ball load, water load and

material load respectively; C ,

ϕ mb

and

φbmw

are pulp density, material to ball

volume ratio and charge volume ratio respectively; of ball, material and water respectively;

ρb , ρ m

and

ρw

are the density

Vm , Vball , Vμ , Vw and Vmill are the

volume of material, balls, interspaces of balls, water and inside of mill; and interspaces ratio of the balls.

μ

is the

3 Soft Sensor Modeling of Mill Load 3.1 Soft Sensor Strategy The architecture of the proposed soft sensor method consists of data filter module, data transform module, feature extraction module and SVM soft sensor models.

Fig. 2. Architecture of soft sensor modeling for mill load

Mechanical grinding of ball mill produces strong vibration and acoustic signals which are stable and periodic over a given time interval, and the vibration signal of the mill shell is more sensitive. The relative amplitudes at different frequencies of these signals contain information directly related to the operating state of grinding [6]. There are more frequency elements and the energies of the spectral variables are

806

J. Tang et al.

linearly correlated. The PSD of the vibration signal of the wet mill shell can be divided into three portions: the natural frequency band, main impact frequency band and secondary impact frequency band. PCA is then performed to extract features of different bands separately in order to get different information from different bands. The reduced dimension data is used for SVM training, which can overcome the disadvantage of very large data and slow processing speed when building the SVM model [9]. 3.2 Principal Component Analysis PCA is used to solve multi-collinearity which exists in the spectral data of the vibration. This is achieved by transforming the spectral data to a new set of variables, which are uncorrelated, and the first few retain most of the variation present in all of the original variables [10]. PCA decomposes the data matrix X with n rows and p columns as the sum of the outer products of vectors

ti and pi plus a residual matrix E ,

X = t1 p1T + t2 p2T + " + tk pkT + E ,

(5)

ti and pi are known as principal component scores and loadings vectors respectively, k must be less than or equal to the smaller dimension of X . The ti where,

vectors are known as principal component scores and contain information on how the samples relate to each other. The pi vectors are known as loadings and contain information on how the variables relate to each other. The rule for choosing the number of principal components (PCs) is to select a cumulative percentage of total variation to which it is desired that the selected principal components should contribute. The definition of percentage of variation accounted for by the first h PCs is as follows: h

CPVh = 100∑ li i =1

where

k

∑l , i

(6)

i =1

li is a measure of the amount of variance described by the pi .

The model structure of the PCA for the vibration at different frequency bands are

X VL = TVL PVLT + EVL ,

(7)

T X VM = TVM PVM + EVM ,

(8)

T X VH = TVH PVH + EVH ,

(9)

where V represents vibration signal; and and high frequency bands respectively.

L , M and H represent the low, medium

Soft Sensor Modeling of Ball Mill Load via PCA and Support Vector Machines

807

Calculate the input of the models as follows:

X sel = [TVL TVM where

TVH ],

(10)

X sel represents the feature spectral variable.

3.3 Support Vector Machines SVM is a training algorithm for learning classification and regression rules from data, whose regression theory is to find a nonlinear map from input space to output space and map the data to a higher dimensional feature space through the map [9]. Thus, SVM is a linear classifier in the parameter space, but it becomes a nonlinear classifier as a result of the nonlinear mapping of the space of the input patterns into the high dimensional feature space. Here, we use SVM to build the soft sensor model between the feature spectral variable and operating parameters of ball mill. We use the input/output data [ x ( k ), y ( k )], k ∈ [1, l ] to approximate a nonlinear function. Consider the regression in a set of nonlinear functions

f ( x) = wT ϕ ( x) + b, where the Kernel trick

(11)

K ( x, xk ) = ϕ ( x)T ϕ ( x) . The cost function (empirical risk)

is defined as

Remp (θ ) = where

ε

1 l yk − ( wT ϕ ( x) + b) , ∑ ε l k =1

(12)

is the Vapnik’s insensitive loss function, it is defined as

0 ⎧ yk − f ( x ) ε = ⎨ ⎩ yk − f ( x ) − ε

yk − f ( x ) ≤ ε , otherwise,

(13)

and ε can be regarded as the accuracy of approximation. The optimization problem is

1 T w w, 2 yk − ( wT ϕ ( x) + b) ≤ ε . Jp =

min s.t : If

ε

is too small, then certain points will be outside of this

additional slack variables modified into

ξk

and

ξk

*

(14)

ε

tube. Therefore

are introduced. The optimization problem is

808

J. Tang et al. l 1 T w w + c∑ (ξ k + ξ k* ), 2 k =1

min

Jp =

s.t :

− (ε + +ξ ) ≤ y k − ( w ϕ ( x) + b) ≤ ε + ξ . T

* k

(15)

* k

The Lagrangian form is l

L = J p − ∑ α k [( wT xk + b) − yk + (ε + ξ k )] − k =1

l

∑ α [( y k =1

* k

l

− w xk + b) + (ε + ξ k )] − ∑ (ηk ξ k + η ξ ). T

k

k =1

(16)

* * k k

Optimization problem (14) can be changed into the problem

max* α ,α

J D (α , α * ) = −

1 l ∑ (α k − α k* )(α l − α l* ) xkT xl − 2 k =1

l

l

k =1

k =1

ε ∑ (α k − α k* ) +∑ yk (α k − α k* ), l

s.t :

∑ (α

k

(17)

− α k* ) = 0, α k ≥ 0.

k =1

The solution of

α k*

can be obtained by standard QP software package, and the sum

should be taken only over the non-zero

α k*

(support vector), so the result is

sv

f ( x) = ∑ (α k − α k* ) K ( xk , x) + b,

(18)

k =1

Where

SV is the number of the support vectors.

3.4 Steps of Soft Sensor Algorithms Main calculating steps of the soft sensor algorithms can be summarized as follows: • • • •

Filter the vibration signal of the mill shell using the band-pass filter. Calculate the PSD of the vibration signal using the Welch’s method. Decompose the PSD of vibration signal at different frequency bands using PCA. Select the PCs and train the SVM model.

4 Application Study Laboratory size ball mill of 460 mm in diameter and 460 mm in length was used to evaluate the soft sensor method. The test was performed by gradually changing the

Soft Sensor Modeling of Ball Mill Load via PCA and Support Vector Machines

809

mass of material and water with fixed mass of balls. Estimation of ML was carried out using 26 samples, and the PSD plot shows that the range of different frequency bands are 100~1800Hz, 1800~4000 Hz and 4000~7500 Hz. Then, the matrix XV was divided to XVL, XVM and XVH, and PCA was used to extracting feature spectral variable at XVL, XVM and XVH respectively. The PSD plot and the contribution ratio of the principle components are shown in Fig. 3.

Fig. 3. PSD plot and contribution ratio of principle components

We take 5, 3 and 1 PCs for low, medium and high frequency bands. After combining the feature spectral variables of three different bands, three SVM models of ML operating parameters were built. The results are shown in Fig. 4.

Fig. 4. Prediction results of the mill operate parameters

The result shows that the soft sensor model via FFT-PCA-SVM is effective. But the prediction performance of the soft sensor model may depends on the different grinding processes, therefore it is important to collect the industry mill data to test the model in industrial environment in further research.

810

J. Tang et al.

5 Conclusions This paper presents an effective approach for the modeling of the operating parameters of ball mill based on the vibration signal of the mill shell. First, the PSD of the vibration signal of the mill shell is obtained by FFT. Then, PCA is used to extract the feature variable of the spectral data at low, medium, high frequency bands respectively. Finally, support vector machines are used to produce support vectors of the combined scores, and three SVM models are developed to monitor the mill operate parameters. A case study shows that the proposed soft-sensor method produces higher fitting precision and better prediction performance. Acknowledgments. The work was supported by the Chinese National Hi-Tech Development Program (No.2006AA060202), Research Foundation of Ministry of Education of China (No.308007), National Science Foundations of China under Grant (No.60534010).

References 1. Zhou, P., Chai, T.Y., Wang, H.: Intelligent Optimal-Setting Control for Grinding Circuits of Mineral Processing. IEEE Transactions on Automation Science and Engineering 6, 730–743 (2009) 2. Behera, B., Mishra, B.K., Murty, C.V.R.: Experimental Analysis of Charge Dynamics in Tumbling Mills by Vibration Signature Technique. Minerals Engineering 20, 84–91 (2007) 3. Kolacz, J.: Measurement System of the Mill Charge in Grinding Ball Mill Circuits. Minerals Engineering 10, 1329–1338 (1997) 4. Huang, P., Jia, M.P., Zhong, B.L.: Investigation on Measuring the Fill Level of an Industrial Ball Mill Based on the Vibration Characteristics of the Mill Shell. Minerals Engineering 22, 1200–1208 (2009) 5. Zhou, P., Chai, T.Y.: Intelligent Monitoring and Control of Mill Load for Grinding Processes. Chinese Control Theory & Applications 25, 1095–1099 (2008) 6. Zeng, Y.G., Forsberg, E.: Monitoring Grinding Parameters by Vibration Signal Measurement—a Primary Application. Minerals Engineering 7, 495–501 (1994) 7. Wang, Z.H., Chen, B.C.: Present State and Development Trend for Ball Mill Load Measurement. Chinese Powder Science and Technology 1, 19–23 (2001) 8. Niu, D.X., Wang, Y.L., Wu, D.D.: Power Load Forecasting using Support Vector Machine and Ant Colony Optimization. Expert Systems with Applications 37, 2531–2539 (2010) 9. Yu, W., Li, X.O.: On-line Fuzzy Modeling via Clustering and Support Vector Machines. Information Sciences 178, 4264–4279 (2008) 10. Kadlec, P., Gabrys, B., Strandt, S.: Data-driven Soft Sensors in the Process Industry. Computers and Chemical Engineering 33, 795–814 (2009)

An Approach Based on Hilbert-Huang Transform and Support Vector Machine for Intelligent Fault Diagnosis Chang Che and Dan Hu School of Mechanical Engineering and Automation, Xihua University, Chengdu 610039, China [email protected]

Abstract. This work discuss the fault diagnosis of tool wear based on HHT (Hilbert-Huang Transform) and SVM (Support Vector Machine). Firstly, we introduce a novel approach to extract features of sampling data based on HHT. Our emphasis is to analyze the Hilbert amplitude spectrum and the Hilbert marginal spectrum of each IMF (Intrinsic Mode Function), which contains the physically meaningful characteristics of the tool states. Secondly, the Hilbert spectrum and marginal spectrum are used as the features, which would be as the inputs of LS-SVM (Least Square-SVM) model for pattern recognition to monitor cutting processing. Based on SVM theory, this dissertation develops a research on machine fault pattern classification by using the extracted features for tool wear applications. The simulation results show that the HHT has good performance in extracting the features, and the proposed method can increase the correct rate of classifier. Keywords: HHT, Empirical mode decomposition, data mining, Support vector machine, Feature extraction.

1

Introduction

Generally, worn tools adversely affect the surface finish of the workpiece and therefore there is a need to develop tool wear condition monitoring systems. Recent attempts have been concentrated on the development of the method which analyzes AE signals for the cutting processes. Feature selection plays very important role in machine fault diagnosis to improve the classification performance,time-frequency analysis techniques such as Wavelet Transform (WT) [1],[2],[3] and Wigner-Vile distribution (WVD)[4] are widely used in signal processing and faults diagnosis. An important limitation of the wavelet analysis is its non-adaptive nature, once the basic wavelet is selected, one will have to use it to analyze all the data. The difficulty of the WVD is the severe cross terms as indicated by the existence of negative power for some frequency ranges. Many artificial intelligence methods have been widely used, such as neural networks, wavelet analysis[4]. These methods are based on an empirical, risk Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 811–817. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com 

812

C. Che and D. Hu

minimization principle and have some disadvantages such as local optimal solution, low convergence rate, obvious ”over-fitting” and especially poor generalization when the number of fault samples is limited. Because of the excellent performance on classification, recently, support vector machines (SVMs) have received considerable attention because of their superior performance in pattern recognition[5],[6],[7]. In this paper, we first analyze the frequency characteristics of each IMF based on EMD (Empirical mode decomposition) for help in understanding the state during turning, and extract effective information in light of the Hilbert amplitude spectrum and marginal spectrum from the signal data. Then we discuss the general model of SVM and the application in fault diagnosis and possibility of fault forecast.

2 2.1

Features Extraction Based on HHT The Hilbert Spectrum and Marginal Spectrum

As discussed by Huang et al (1996, 1998a), the empirical mode decomposition (EMD) method is necessary to deal with both non-stationary and nonlinear data. This new method is intuitive, direct, a posteriori, and adaptive, with the basis of the decomposition being derived from the data[9],[10],[11]. In this section, we mainly discuss the feature extraction of AE signals, when these involve variations in both time and frequency, two quantifiable characteristics of AE can be displayed as follows: (1) Hilbert spectrum values of each IMF of the AE-signal, each IMF level represents one component of the decomposed AE signal within a certain frequency range. (2) Hilbert marginal spectrum can reveal the existence of energy at the frequency. The essence of this method is to empirically identify the intrinsic oscillatory modes by their characteristic time scale imbedded in a signal, and then decompose the signal into a collection of a finite and often small number of IMFs through sifting process. Each IMF component then represents only one mode of both amplitude and frequency modulated oscillation of the signal at a certain time scale or frequency band, and the sum of all the IMF components as well as a residual produces a perfect reconstruction of the original signal. By virtue of the IMF definition, the decomposition method can simply use the envelopes defined by the local maximum and minimum separately [10],[11]. For an arbitrary time series x(t), we can express the data in following form: x(t) =

n 

 αj (t) exp(i

ωj (t) dt

(1)

j=1

Here αj is the amplitude of Hilbert, and ωj is the instantaneous frequency. (1) gives both the amplitude and the frequency of each component as functions of time. The variable amplitude and the instantaneous frequency have not only greatly improved the efficiency of the expansion but also enabled the expansion to accommodate non-stationary data.

An Approach Based on HHT and SVM for Intelligent Fault Diagnosis

813

According to (1), the frequency-time distribution of the amplitude is designated as the Hilbert amplitude spectrum, h(ω, t).   n αj (t) exp(i ωj (t) dt), ωj (t) = ω j=1 H(ω, t) = (2) 0 others Then the squared values of amplitude can be substituted to produce the Hilbert energy spectrum. The Hilbert spectrum appears only in the skeleton form with emphasis on the frequency variations of each IMF. With the Hilbert spectrum defined, we can also define the marginal spectrum h(ω), as 

T

h(ω) =

h(ω, t) dt

(3)

0

The marginal spectrum offers a measure of total amplitude (or energy) contribution from each frequency value. In the marginal spectrum representation, the existence of energy at the frequency ω , means only that, in the whole time span of the data, there is a higher likelihood for such a wave to have appeared locally. 2.2

Features Extraction

In the light of HHT theory, we can make use of the Hilbert amplitude and marginal spectrum to obtain the features of AE signals in turning. The typical procedure of AE signal processing in metal cutting, the piezoelectric AE sensor is usually placed as close as possible to the cutting zone, which was attached to the side of a tool shank. Tool wear versus magnitude and energy of AE signal relationship is very complex. Here we consider the applicability of some basic measures of AE for tool condition monitoring while turning steel. Workpiece material was selected as AISI1018. Cutting parameters were as follows: cutting speed νc = 1m/s , feed f = 0.013mm/rev , depth of cut ap = 4.3mm .The AE signal was initially 40 dB pre-amplified, the sampling frequency is 4MHz, and the number of the sampling points is 2048. Here we investigate both the Hilbert amplitude spectrum and marginal spectral distribution of AE-signals based on HHT. The training experiments were conducted on a small data set from the above described experimental measurements. The data set from the experiment is divided into two sets, one is the normal cutting force, the other is the worn cutting force. A set of normal data and fault data are shown in figure 1 respectively. Using the empirical mode decomposition, then the Hilbert transform is applied to each intrinsic mode function. Firstly, The Hilbert spectrum values of each IMF of the AE-signal is shown in figure 2 by using the EMD algorithm, the normal and fault cutting data are distinguished into 9 IMFs in total, respectively. Each IMF level represents one component of the decomposed AE signal within a certain frequency range. Secondly, figure 4 shows that the Hilbert spectrum of normal data and fault data. The Hilbert amplitude spectra are a weighted non-normalized joint amplitudefrequency-time distribution. Finally, using the empirical mode decomposition,

814

C. Che and D. Hu

Fig. 1. This shows a figure consisting of normal and fault cutting data

Fig. 2. This shows a EMD figure of normal and fault cutting data

then the Hilbert transform is applied to each IMF, calculating the marginal spectrum hi (ω). These results are shown in figure 3. Compare the two figures in figure 3, the mode h1 with marginal spectrum centered from 200Hz to 2500Hz in figure 8, it is can be concluded that mode h1 is the high frequency noise. According to Mode h1 − h9 , the amplitude of the marginal spectrum of h2 and h3 are larger than that of the others. Therefore, it can be concluded that the worn fault is occurred in the tool cutting processing in this frequency range. Therefore, it seems that the Hilbert marginal spectrum represents the time scale of the transient caused by tool wear. In other words, the Hilbert marginal spectrum of each IMF mode is related to tool cutting state. The total marginal spectrum is shown in figure 5, which show the Hilbert marginal spectrum of the normal data and fault data, respectively. Thence, the Hilbert spectrum denotes that the HHT provides an effective way to analyze AE signals by decomposing the signal into IMFs, each IMF level represents one component of the decomposed AE signal within a certain frequency range. Energy carried by each IMF of the AE signal was analyzed by considering the full waveform, which reveals the relationship between the AE signal and tool wear. Especially, the Hilbert marginal spectrum can reveal the existence of energy at the frequency ω , that is to say, in the whole time span of the data, there is a higher likelihood for such a frequency wave to have appeared locally.

An Approach Based on HHT and SVM for Intelligent Fault Diagnosis

815

Fig. 3. This shows a figure consisting of Marginal spectrum of each IMF

Fig. 4. This shows a figure consisting of the Hilbert spectrum of normal and fault cutting data

Fig. 5. This shows a figure consisting of the Hilbert marginal spectrum of normal and fault cutting data

3

Application of LS-SVM in Fault Diagnosis of Tool Cutting Based on HHT

In the section, the effectiveness of the proposed features is verified by LS-SVM model [6], [8]. The training experiments were conducted on a small data set from the above described experimental measurements. This example attempts to demonstrate an entirely different view of the mechanisms of nonlinear tool cutting data based on the Hilbert-Huang transform. According to the fault features extracted through HHT method, the features are used as the inputs of the SVM classifier, the classification is divided into fault-free and fault.

816

C. Che and D. Hu Table 1. Features of sample data number F1 1 0.3712 2 0.4220 3 0.3546 ··· ··· 29 0.6919 30 0.7121

F2 0.3125 0.3228 0.3419 ··· 0.1986 0.2017

F3 0.0669 0.0624 0.0707 ··· 0.0385 0.0423

F4 0.1674 0.1123 0.1574 ··· 0.0453 0.0248

F5 0.0110 0.0096 0.0101 ··· 0.0090 0.0098

F6 0.0307 0.0190 0.0290 ··· 0.0095 0.0055

Table 2. Results of simulation Methods The number of SVMs Correct rate MSE Wavelet ananlysis 13 89.28% 0.12 HHT 18 94.23% 0.02

To verify the performance of the proposed method, in these simulations, radial basis function (RBF) kernel with σ 2 = 0.3 , γ = 7 is employed for the LS-SVM classifier. Features selected from the experimental data is shown in table 1, we collect 30 sets of data as training in 40 sets of sample data from measurement system, the other 10 sets of data are used as testing data. At the same time, we also extracted the features by using the 5-level wavelet analysis with Daubechies1 wavelet function. According to the theory of pre-described, the performances of HHT and wavelet analysis have been shown in table 2. We can conclude that the correct rate is increased otherwise the number of support vector machine is increased. Therefore the method of features extract is useful. It can increase the precision of fault diagnosis.

4

Conclusions

The purpose of the experiments is to determine whether the proposed method of features extracted could successful recognition the state of tool wear. The results show that the HHT has good performance in extracting the characteristics of the Hilbert spectrum and marginal for SVM classifier, it is a good reference method for solving the pattern recognition, system identification and simulation model problem. Nevertheless, we should further study on the influence in the results of the number of SVMs. Acknowledgments. This work is supported by the important project fund of the department of education of Sichuan province, China (No.07ZA118) and the doctor fund of Xihua university of Sichuan province, China (No. R0720208).

An Approach Based on HHT and SVM for Intelligent Fault Diagnosis

817

References 1. Farge, M.: Wavelet Transforms and Their Applications to Turbulence. Annu. Rev. Fluid Mech. 24, 395–457 (1992) 2. Claasen, T.A.C.M., Mecklenbrauker, W.F.G.: The Wigner Distribution a Tool for Time-frequency Signal Analysis. Part I:Continuous time signals 35, 217–50; Part II: Discrete time signals 35, 276–300; Part III: Relations with other time-frequency signal transformations. Philips J. Res. 35, 372–89 (1980) 3. Uchiyama, Y., Kawahara, T.: A Possible Mechanism for Frequency Down-shift in Nonlinear Wave Modulation. Wave Motion 20, 99–110 (1994) 4. Grossmann, A., Morlet, J.: Decomposition of Hardy Function into Square Integrable Wavelets of Constant shape. SIAM J. Math. Anal. 15, 723–727 (1984) 5. Zhang, L., Zhou, W.D., Jiao, L.C.: Scaling Kernel Function Support Vector Machines. Acta Electronica Sinica 4(4), 527–529 (2002) 6. Liu, G., Liu, X., Jing, Q.: Fault Diagnosis Approach Based on Hidden Marko Model and Support Vector Machine. Chinese Journal of Mechanical Engineering 20(5), 92–95 (2007) 7. Gao, G., Zhang, Y., Zhu, Y., Duan, G.: Hybrid Support Vector Machines Based Multi-fault Classification. China Univ. Mining and Technol. 17(2), 246–250 (2007) 8. Suykens, J.A.K., Vandewalle, J.: Least Squares Support Vector Machine Classifier. Neural Process. Lett. 9, 293–300 (1999) 9. Huang, N.E., Shen, Z., Long, S.: A New View of Non-linear Water Waves: the Hilbert spectrum. Annu. Rev. Fluid Mech. 31(1), 417–457 (1996) 10. Huang, N.E.: The Empirical Mode Decomposition and the Hilbert Spectrum for Nonlinear and Non-stationary Time Series Analysis. Pro. R. Soc. Lond. A 454(1971), 903–995 (1998) 11. Huang, N.E., Wu, M., Long, S., Shen, S., Qu, W., Gloerson, P., Fan, K.: A Confidence Limit for the Empirical Mode Decomposition and Hilbert Spectral Analysis. Proc. R. Soc. Lond. A 459(2037), 2317–2345 (2003)

Study on Factors of Floating Women’s Income in Jiangsu Province Based on Bayesian Networks Yingyu Ge1, Chunping Li2, and Qin Yin2 1 School of Economics and Management, Jiangsu Teachers Univ. of Technology, 213001 Changzhou, China 2 Department of Economics and Demography, Nanjing College for Population Programme Management, 210042 Nanjing, China

Abstract. Due to the uncertainty of the factors that influence the income and other characters of floating women in Jiangsu province, we propose using Bayesian Networks to model this kind of system. We use different algorithms for learning Bayesian Networks in order to compare several models. This study of a real problem includes preliminary data processing, the comparison of different algorithms, and the role of using Bayesian Networks for social problems. We suggest that researchers can use Bayesian Networks to explore the potential relationship between variables of complex social problems. Keywords: Bayesian networks, Learning algorithms, Network Scores, Floating women.

1 Introduction With the rapid economic and social development of Jiangsu Province and the steady decline in birth rate, as well as imbalance of economic development in province and between provinces , the number of floating population in Jiangsu Province are growing fast. In November 2005 China conducted a sample survey of 1% of the population, it indicated that from 2000 to 2005, inflow of population in Jiangsu Province from 9.1 million people in 2000 rose to 10.78 million, the proportion of the entire province resident population also from 12.5% in 2000 grows to 15.2% in 2005, increased 18.5% during 5 years. The average annual increase of floating population increased 3.45%. According to the data of Bureau of Statistics of Jiangsu Province, to the end of 2008, including peasant laborer hiring out for working in Jiangsu from other provinces, floating population of 16 years and over in Jiangsu whose household register in the countryside, and have left the household registration more than six months and worked in the city were 98.8 million. The floating population becomes an important force for economic development in Jiangsu Province. However, due to the differences in employment, security and other policy between China's household registration and non-residence of the population, the income of floating population is lower than the household registration. The living condition of floating population, especially floating women is increasingly a concern. This paper uses survey data in 2007 of the floating woman in Nanjing, Wuxi and Yancheng, and uses Bayesian networks to analyze the causal relationships of the behavior and effect of floating women. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 819–827. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

820

Y. Ge, C. Li, and Q. Yin

2 The Data Set The sample is randomly selected out of the floating women aged from15 to 49 who have migrated here at least 3 months in Jianye district of Nanjing city, Huting district of Yancheng city, and Xishan district of Wuxi city. 1757 samples are selected after eliminating the invalid questionnaire. 8 variables are considered which you can find in Table 1.Among the sample we discussed, 46% of the floating women are from Nanjing city, 26.4% and 27.6% of them are from Yancheng city and Wuxi city. 47% of the floating women are from developed areas(GDP per capita is large than 25000 yuan), 16.6% of them are from medium-developed areas(GDP per capita is between 15000 and 20000 yuan), and 36.4% of them are from underdeveloped areas(GDP per capita is less than 15000 yuan). The rate of illiterate population is 4.2%, and the rate of people with the education of primary school, junior secondary school and above senior secondary school are 20.9%, 54.7% and 20.2%. The percentage of the floating women who had taken at least a train is 49.3%. The average income of the floating women is 845 yuan in year 2007, it is significant lower than average wage of the residents, which is about 1800 yuan. Table 1. 8 variables of the data Variable province

Possible values 3

age edu city train job

6 4 3 2 5

time income

4 5

description 3 levels of the development of a province, measured by GDP per capita grouped by age:15-19,20-24,25-29,30-34,35-40,40+ 4 levels of education of floating women 3 cities: Nanjing, Wuxi and Yancheng whether or not had taken a training 5 kinds of job: public institution, factory, retailing, catering and other the frequency of backing home: 5 levels of income:”0-500”,”500-800”,”8001000”,”1000-1200”,”1200+”

3 Bayesian Network 3.1 Introduction A Bayesian network, belief network or directed acyclic graphical model is a probabilistic graphical model that represents a set of random variables and their conditional independencies via a directed acyclic graph (DAG). It is a graphical model for probabilistic relationships among a set of variables. A general Bayesian network can be defined as follows: Let G=(V,E) be a directed acyclic graph (DAG), and let X be a set of random variables indexed by V. Then X is a Bayesian network with respect to G if its joint probability density function can be written as a product of the individual density functions, conditional on their parent variables:

Study on Factors of Floating Women’s Income in Jiangsu Province

p ( x) = ∏ p ( xv x pa ( v ) ) v∈V

821

(1)

where pa(v) is the set of parents of v. Graphical models include Bayesian network have several advantages for data analysis. Firstly, because the model encodes dependencies among all variables, it readily handles situations where some data entries are missing that it is very common in a survey. Secondly, a Bayesian network can be used to learn causal relationships, and hence can be used to gain understanding about a problem domain and to predict the consequences of intervention. Thirdly, because the model has both a causal and probabilistic semantics, it is an ideal representation for combining prior knowledge and data. Finally, Bayesian statistical methods in conjunction with Bayesian networks offer an efficient and principled approach for avoiding the overfitting of data. 3.2 Structure Learning Algorithms Although there are a great many algorithms for learning Bayesian networks from data, they can be subdivided into two general approaches: The algorithms learn the network structure by analyzing the probabilistic relations entailed by the Markov property with conditional independence tests and then constructing a graph which satisfies the corresponding d-separation statements. The algorithms assign a score to each candidate Bayesian network and try to maximize it with some heuristic search algorithm. Greedy search algorithms (such as hill-climbing or tabu search) are a common choice. We have used the following algorithms, which are a representative sample of the different approaches for learning Bayesian networks: The GS algorithm was first proposed by Margaritis(2003). It consists of two phases, a grow phase and a shrink phase. The grow phase of X proceeds by attempting to add each variable Y to the current set of hypothesized neighbors of X, contained in BX, which is initially empty. BX grows by some variable Y during each iteration of the grow loop of X if and only if Y is found dependent with X given the current set of hypothesized neighbors BX. Due to the (unspecified) ordering that the variables are examined, at the end of the grow phase some of the variables in BX might not be true neighbors of X in the underlying MN—these are called false positives. This justifies the shrink phase of the algorithm, which removes each false positive Y in BX by testing for independence with X conditioned on BX −{Y}. If Y is found independent of X during the shrink phase, it cannot be a true neighbor (i.e., there cannot be an edge X−Y), and GSMN removes it from BX. Assuming faithfulness and correctness of the independence query results, by the end of the shrink phase BX contains exactly the neighbors of X in the underlying Markov network. Incremental Association (IAMB): based on the Markov blanket detection algorithm of the same name (Tsamardinos et al. 2003), which is based on a two-phase selection scheme (a forward selection followed by an attempt to remove false positives).IAMB follows the same two-phase structure with GS algorithm and adopts one dynamic heuristic in the growing phase to improve the static and potentially inefficient heuristic of GS. In detail, IAMB iteratively reorders the variables after a new variable enters the blanket, and the reordering operation is implemented using mutual information heuristic.

822

Y. Ge, C. Li, and Q. Yin

Fast Incremental Association: a variant of IAMB which uses speculative stepwise forward selection to reduce the number of conditional independence tests (Yaramakala and Margaritis 2005). Interleaved Incremental Association: another variant of IAMB which uses forward stepwise selection (Tsamardinos et al. 2003) to avoid false positives in the Markov blanket detection phase. Max-Min Parents and Children: a forward selection technique for neighborhood detection based on the maximization of the minimum association measure observed with any subset of the nodes selected in the previous iterations (Tsamardinos et al. 2006). It learns the underlying structure of the Bayesian network. Hill-Climbing greedy search is the common score-based learning algorithm on the space of directed graphs. The optimized implementation (again used by default) uses score caching, score decomposability and score equivalence to reduce the number of duplicated tests (Daly and Shen 2007).

4 Results After running the learning algorithms by using the bnlearn package(R Team 2009) and Rgraphviz package(Gentry et al. 2009) in R, we obtained six different networks. These networks are displayed in Fig.1. We interpret the arcs as direct dependence relationships between the linked variables, and the absence of arcs means the Hill-Climbing

Grow-Shrink

province

province

job

edu

Incremental Association

city

tim e

city

income

age

train

edu

job

age

income

train

city

edu

province

age

train

time

time

Fast Incremental Association

job

Interleaved Incremental Association

Max-Min Parents and Children province

city

city

income

city province

edu

edu

edu age

train

job

incom e

train

age

age

tim e tim e job

province

income

time

job

income

Fig. 1. The networks of 6 stucture learning algorithms

train

Study on Factors of Floating Women’s Income in Jiangsu Province

823

existence of conditional independence relationships. In some cases if one can make sure the logic of two variables, these arcs can be interpreted as causal interpretation. In order to summarize the differences and resemblances between models, Table 2 shows the two numbers edges/arcs for each pair of algorithms, where “edges” is the number of common edges (in either direction), and “arcs” the number of common arcs between the networks learned by these algorithms. The main diagonal in this table represents the number of arcs contained in each network. We can make a conclusion from table 2 that most of the edges constructed by these algorithms are same, but only a few arcs are same. The edges that are common to all models denote that there is strong relationship between the nods linked with these edges. These edges are between province and city, province and train, edu and age, edu and city, job and income, time and age. Other edges can be found in 5 networks denote that there is a less strong relationship between the nods linked with these edges. These edges are between city and train, job and time. These 8 edges are defined as common edges. Some of the directions of these edges can be recognized because of the consecution of these variables. For instance, the province you come from determines the city you come to, and the kind of province you come from determines you whether need a training or not, etc. Using the prior knowledge, we can build the structure of the relationships between these variables. Fig. 2 displays the edges that are common to most of the networks. Then running these algorithms again, and set the common edges in Fig.2 as the whitelist of each network, 6 new networks are displayed in Fig.3. The boldfaced Table 2. Number of common links and arcs, edges/arcs, between pairs of learned networks

gs hc iamb fast.iamb inter.iamb mmpc

gs 10/8 -

hc 7/4 8/8 -

iamb 10/4 7/3 10/8 -

fast.iamb 9/2 6/3 9/5 10/9 -

age

province

edu

inter.iamb 10/4 7/3 10/8 8/4 10/8 -

job

time

income

city

train

Fig. 2. Common edges of these algorithms which are defined above

mmpc 8/4 7/7 7/4 7/3 8/4 8/8

824

Y. Ge, C. Li, and Q. Yin

Hill-Climbing

Grow-Shrink

age

province

province

Incremental Association

age

age

job

edu

edu

province

city

income

job

time

income

income

age

job

train

Fast Incremental Association

Interleaved Incremental Association

Max-Min Parents and Children

age

age

job

job

train

time

edu prov ince

edu

edu

city

city train

time

time

province

income

time

job

time

income

edu

province city

income

city

city train

train

train

Fig. 3. 6 new networks containing the common edges above

edges are common edges discussed above, and the thin edges are supplementary edges in each algorithm. The additional edges are few and quite similar. In order to pick out which one fitted the data best, we should with the help of Network scores.

5 Network Scores In general, a scoring function Score(G,D) for learning a Bayesian network structure is called decomposable, if it can be expressed as a sum of local scores m

Score(G , D ) = ∑ S ( Di , DGi )

(2)

i =1

where G represents a directed acyclic graph (DAG) and D represents a certain data set. Many popular scoring functions avoid overfitting by balancing the fit to the data with the complexity of the model. A common form of this idea can be expressed as

Score(G, D) = log Pˆ ( D G ) − Δ ( D, G ) where Δ ( D , G ) is a complexity penalty.

(3)

Study on Factors of Floating Women’s Income in Jiangsu Province

825

Several score functions are available for estimating the fitting of these algorithms. The score to be used is specified with the score and type arguments, respectively. In the discrete case the following options are implemented: The likelihood and log-likelihood scores, which are equivalent to the entropy measure used by Weka (Witten and Frank 2005). The maximized likelihood

Pˆ ( D G ) decomposes by the network structure, and for the decomposable scores handled in this paper, the complexity penalty decomposes too. Hence, we can write the penalized scores in the decomposed form (2), with the local scores given by

S ( Di , DGi ) = log Pˆ ( Di DGi ) + Δ i ( Di , DGi )

(4)

Different scores differ in how the local penalty Δ i ( Di , DGi ) is determined. Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC) are two popular decomposable scores for learning Bayesian network structures. The penalty terms for these scores are

ΔAIC = qi ( ri − 1) , and ΔBIC = i i

q i ( ri −1) 2

ln N

(5)

Both of these complexities are independent of the data, and only depend on the arities ri of random variables and the structure of the Bayesian network. The Akaike and Bayesian Information Criterion scores, defined as BIC , and BIC = log L ( X 1 ," X v ) − Δ i AIC = log L ( X 1 ," X v ) − ΔAIC i

(6)

The Bayesian Dirichlet Equivalent (BDE) score also uses Bayesian analysis to evaluate a network given a dataset. The BDE scoring method was developed by Heckerman(Heckerman et al. 1995) and is based on the BD metric developed by Cooper and Herskovits. The Dirichlet distribution is a multinomial distribution that describes the conditional probability of each variable in the network, and has many properties that are useful for learning. The logarithm of the K2 score, another Dirichlet posterior density (Cooper and Herskovits 1992) defined as v

Li

i =1

j =1

K 2 = ∏ K 2( X i ) , K 2( X i ) = ∏

(ri − 1)!

ri

∏n

(∑ ki =1 nijk + ri − 1)! k =1 r

ijk

!

(7)

Unlike the Bayesian Dirichlet Equivalent score it's not score equivalent. The values of the different scores for all the networks considered are showed in Table 3. The boldfaced and italic numbers represented the highest scores of these algorithms. In the light of the resulting values, we can conclude that the Hill-Climbing algorithm performs quite well with respect to all the scores except the log-likelihood score, which is just close to the maxima score of Grow-Shrink algorithm. So the final network is represented as Fig 4. The thicker the edge is, the stronger the relationship is. So one can make a short conclusion that, which city the floating women chose to go was mainly determined by the value of province and education. Different cities

826

Y. Ge, C. Li, and Q. Yin Table 3. The scores of each algorithms

Grow-Shrink Hill-Climbing Incremental Association Fast Incremental Association Interleaved Incremental Association Max-Min Parents and Children

loglik -15662.74 -15691.10 -15811.71

k2 -16198.27 -16130.95 -16392.57

bde -16637.36 -16424.01 -16860.5

aic -15959.74 -15907.10 -16139.71

bic -16772.23 -16498.00 -17037.01

-15818.92

-16384.50

-16835.19

-16137.92

-17010.60

-15811.71

-16392.57

-16860.5

-16139.71

-17037.01

-15961.98

-16327.08

-16578.61

-16139.98

-16626.93

age

province

edu city

train

job

income

time

Fig. 4. The final network

mainly offered different kind of jobs, which determined the income one gained. Floating women had been trained or not was influenced by the province and city.

6 Conclusion Due to the complexity of social systems like the behavior and choices of floating women, they should be represented, studied and forecasted with the appropriate tools. Bayesian Networks offer a very attractive formalism for researchers understanding the behavior of people. We believe that our work illustrates the usefulness of Bayesian Networks and their technologies for social problems. Our study of several algorithms for learning Bayesian Networks using the survey dataset has revealed some interesting facts: (1)Income is influenced directly by the kind of job, which is different between cities, but not the education and train experiences as we think. (2)Train experiences and job have same parents on the Markova Blanket, but they are conditional independent with each other. (3)Age is not a strong reason of any variables.

Study on Factors of Floating Women’s Income in Jiangsu Province

827

References 1. Cooper, G.F., Herskovits, E.: A bayesian method for the induction of probabilistic networks from data. Machine Learning 9, 309–347 (1992) 2. Lam, W., Bacchus, F.: Learning bayesian belief networks: an approach based on the MDL principle. Computational Intelligence 10, 269–293 (1994) 3. Scutari, M.: Learning bayesian networks with the bnlearn package. Journal of Statistical Software (2009) (submitted for publication) 4. Tsamardinos, I., Brown, L.E., Aliferis, C.F.: The max-min hill-climbing bayesian network structure learning algorithm. Machine Learning 65, 31–78 (2006) 5. Daly, R., Shen, Q.: Methods to accelerate the learning of bayesian network structures. In: Proceedings of the 2007 UK Workshop on Computational Intelligence. Imperial College, London (2007) 6. Gentry, J., Long, L., Gentleman, R., Falcon, S., Hahne, F., Sarkar, D.R.: Provides Plotting Capabilities for R Graph Objects. R package version 1.22.1 (2009) 7. Heckerman, D., Geiger, D., Chickering, D.M.: Learning bayesian networks: the combination of knowledge and statistical data. Machine Learning 20, 197–243 (1995) 8. Margaritis, D.: Learning Bayesian Network Model Structure from Data. Ph.D. thesis, School of Computer Science, Carnegie-Mellon University, Pittsburgh, PA. Available as Technical Report CMU-CS-03-153 (2003) 9. Tsamardinos, I., Aliferis, C.F., Statnikov, A.: Algorithms for large scale Markov blanket discovery. In: Proceedings of the Sixteenth International Florida Artificial Intelligence Research Society Conference, pp. 376–381. AAAI Press, Menlo Park (2003) 10. Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques, 2nd edn. Morgan Kaufmann, San Francisco (2005) 11. Yaramakala, S., Margaritis, D.: Speculative Markov blanket discovery for optimal feature selection. In: ICDM 2005: Proceedings of the Fifth IEEE International Conference on Data Mining, pp. 809–812. IEEE Computer Society, Washington (2005) 12. R Development Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria (2009) ISBN 3-900051-07-0, http://www.R-project.org

Variation Trend Analysis of Groundwater Depth in Area of Well Irrigation in Sanjiang Plain Based on Wavelet Neural Network Hong Ding1, Dong Liu1,2,*, and Fei-fei Zhao1 1

School of Water Conservancy & Civil Engineering, Northeast Agricultural University, Harbin Heilongjiang 150030, China 2 Postdoctoral Scientific Research Mobile Station of Agriculture and Forestry Economic Management, Northeast Agricultural University, Harbin Heilongjiang 150030, China

Abstract. The groundwater level has continuously decreased due to the rapid increase of paddy field acreage in area of well irrigation paddy in Sanjiang Plain in recent years, some time happen such things as more and more “hanging pump” and local overdraft. The authors took 853 Farm as an example and established the dynamic prediction model of groundwater depth by using the multi-resolution function of wavelet analysis and nonlinear approximation ability of artificial neural network in order to solve above problems. The results of dynamic variation regularities analysis and precision inspection and comparison showed that the model had high accuracy in fitting and prediction. The prediction results also showed the groundwater level will descend continually in the future years and has an average annual downrange of about 0.66m. Therefore, the local government should reinforce the scientific groundwater management. This model revealed the dynamic variation regularities of regional groundwater and provided the scientific basis for sustainable utilization of groundwater resource in area of well irrigation paddy in 853 Farm and even entire Sanjiang Plain. Keywords: Sanjiang Plain, well irrigation paddy, groundwater, Wavelet Neural Network.

1 Introduction Sanjiang Plain is the important commodity grain base of the country, located in the northeast of Heilongjiang Province, and is an alluvial plain by three major rivers of Heilong, Songhua and Wusuli [1]. The low and flat terrain, soil glue serious in most areas and poor drainage capacity of Sanjiang Plain are easy to appear water logging disaster that lead to yield reduction. This area realized the flood treatment by planting rice since 1980s and obtained significant benefits. The rice planting area increased year by year and had a rapid development after 1990s, in which the area of well Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 829–836. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

830

H. Ding, D. Liu, and F.-f. Zhao

irrigation was 65.9 million hm2[2]. The rapid increase of paddy field acreage has led to rapidly increasing of groundwater exploitation. The groundwater level has continuously decreased due to above reason and the human waste and bad management, some time happen things such as more and more “hanging pump” and local overdraft and serious destroy of dynamic equilibrium. Therefore, it’s important to research the dynamic variation regularities and forecast the future tendency of groundwater for determine the paddy scale of well irrigation and sustainable utilize and manage the groundwater resources in Sanjiang Plain. By means of various methods, such as regression analysis, time series analysis, grey theory, artificial neural network theory, chaos theory and wavelet theory, etc, the groundwater level was predicted. These methods belong to single prediction method, so it’s difficult to obtain ideal prediction results. In order to improve prediction accuracy, the authors took 853 Farm as an example and established the wavelet neural network model to simulate and forecast the groundwater depth and reveal the dynamic variation regularities in this region.

2 Wavelet Analysis 2.1 Wavelet Transform Wavelet analysis had a rapid development since the wavelet transform was first proposed by Morlet in the research of seismic wave in 1982. It has been proved to be very efficient in nonlinear signal analysis, image processing, model identification, automatic control and various engineering community [3]. The wavelet transform was called “mathematical microscope” that overcame the deficiency of Fourier transform and reflected the general characteristics of hydrological time series and local information in time-frequency domain [4~6]. 2.2 Fast Wavelet Transform Algorithm Wavelet coefficient information redundancy and large calculation are the most obvious demerits when using continuous or discrete wavelet transforms to analyze the hydrological time series [7]. Therefore, the fast wavelet transform algorithm that including Mallat algorithm and A Trous algorithm is adopted to calculate wavelet transform coefficient in practical applications. The A Trous algorithm adopted has simple, rapid and low computational complexity in this paper [6, 8~10]. Firstly, the hydrological time series f (t ) ( t = 1 2 … N ) is decomposed based on wavelet analysis. When suppose the C (t ) = f (t ) , the decomposition and reconstruction processes of A Trous algorithm are defined as follows:

，， ，

0

+∞ ⎧ j j −1 j ⎪C (t ) = ∑ h(k )C (t + 2 k ) ⎨ k = −∞ ⎪⎩W j (t ) = C j −1 (t ) − C j (t )

(1)

P

C 0 (t ) = C P (t ) + ∑ W j (t ) j =1

(2)

Variation Trend Analysis of Groundwater Depth in Area of Well Irrigation

831

Where C (t ) are scale coefficients (background signal) and W (t ) are wavelet coefficients (detail signal) at scale j ; P is the scale parameter; h(k ) is the discrete low-pass filter; {W (t ) W (t ) " W (t ) C (t )} is called wavelet transform series at scale P . j

j

，

1

2

，，

，

P

P

3 Wavelet Neural Network 3.1 Basic Principle

The complex hydrological time series f (t ) is decomposed to wavelet transform series in different frequency bands based on wavelet analysis, in which the detail signal series W (t ) is high frequency component and the background signal series C (t ) is low frequency component. The first m values of series W (t ) and C (t ) as input and the (m+1)-th value as output are used to structure the Artificial Neural Network (ANN) mode respectively. The predictive values of original hydrological series can be obtained by superimposing output results. The combined model is called Wavelet Artificial Neural Network (WANN) based on above methods [5, 8]. j

j

j

j

3.2 Model Structure

The WANN model needs to structure p+1 BP neural networks that the inputs of these networks are W (k ) W (k + 1) … W (k + m − 1) W (k ) W (k + 1) … W ( k + m − 1) … W (k ) W (k + 1) … W (k + m − 1) C (k ) C (k + 1) … C (k + m − 1) and the number of nodes are m, the outputs are W (k + m) W ( k + m) … W (k + m) C ( k + m) and the node is 1 and the nodes of hidden layer can be determined by trial and error, in which k 1~ N − m . The topological structure of the WANN employed in this study is shown in Fig.1.

；

2

P

； ；

P

，

1

P

；

1

，

P

， ， ， ，

1

； ， ； ， 2

P

P

1

P

2

P

；

， ， ， ， 2

＝

Fig. 1. Structure of Wavelet Neural Network

3.3 Main Steps of Modeling

Step 1: Decomposing the original hydrological time series to wavelet transform series at scale P based on formula (1).

832

H. Ding, D. Liu, and F.-f. Zhao

Step 2: Establishing BP neural networks and obtaining the predictive values of each wavelet transform series respectively. Step 3: Reconstructing predictive values of each wavelet transform series and obtaining predictive values of original hydrological time series by using formula (2).

4 Case Study 853 Farm is located in the eastern of Heilongjiang Province that belongs to Hongxinglong Branch Bureau of General Reclamation Bureau as the research object. According to investigation, the average annual groundwater exploitation of 853 Farm was 0.4186 billion m3/a but 0.8046 billion m3 /a in 2000. The serious groundwater overdraft and “hanging pump” have already appeared conspicuously. It must be threaten industry and agriculture and household water-consumption in accordance with this trend. In order to provide the references for groundwater scientific management, the WANN model is established by using the measured monthly groundwater depth series of 1991 2004 as the basic data and the data of 2005 as reservation test to simulate and forecast the dynamic variation regularities.

～

4.1 Wavelet Decomposition and Reconstruction of Measured Series

Wavelet decomposition series can be obtained by decomposing the measured monthly groundwater depth series H (t ) during 1991 2004 based on A Trous algorithm that is shown in Fig.2, in here P equal to 2. The C (t ) = H (t ) represents original series (Fig.2 (a)); W (t ) and W (t ) represent random component and periodic term respectively (Fig.2 (b)); C (t ) represents trend item (Fig.2(c)). The reconstruction series can be obtained by superimposing each wavelet decomposition series is shown in Fig.2 (d). The Fig.2 demonstrates that the reconfiguration process is consistent with measured process completely. Therefore, it’s feasible to decompose the measured monthly groundwater depth series by using A Trous algorithm.

～

0

1

2

2

4.2 Determination of Input and Output Paired Samples

We need to structure p+1 equal to 3 BP neural networks for P=2. The obvious periodic character (the cycle is 1 year) of the dynamic variation regularities of groundwater in 853 Farm can be observed clearly from Fig.2. Therefore, the (i+1)-th point is calculated by using the first i point values in order to reflect this periodic character in network. In this research, the first 12 months data of wavelet decomposition series W 1 (t ) , W 2 (t ) and C 2 (t ) are used for input and 1 data of present month for output to generate 156 paired samples of each wavelet decomposition (all contain 168 data) in 14 years.

Variation Trend Analysis of Groundwater Depth in Area of Well Irrigation

Original seriesC0(t) (m)

Groundwater depth H(t) (m)

4

833

Detail series at resolution lev el 1 Detail series at resolution lev el 2

2 0 -2 -4

0

12

24

36

48

60

72

84

96

Time series t

108 120 132 144 156 168

Time series t

(b)

Background seriesC2(t) (m)

Reconstruction series (m)

(a)

Time series t

Time series t

(c)

(d)

～

Fig. 2. Wavelet decomposition and reconstruction of measured series of monthly groundwater depth in 853 Farm (1991 2004)

4.3 Determination of WANN Model Structure

The node numbers of input layer are 12 and output layer is 1 of the 3 BP neural networks as mentioned above. It is widely agreed that node number of hidden layer is max(nh nj ) ≤ ni ≤ 2nh + 1 [12], in which nh and nj are the node number of input layer and output layer respectively, and the ni is the node number of hidden layer, here ni = 12 , determined by trial and error in this research. So the structure of each BP neural networks in WANN model is 12:12:1.

，

4.4 WANN Model Fitting

By means of wavelet reconstruction algorithm, fitted values of WANN model can be obtained by superposing fitting results of each BP neural networks based on the number of training is 3000, which is shown in Fig.3. Groundwater depth H(t) (m)

12 Measured curve

10

Fitting curve

8 6 4 2

0

12

24

36

48

60

72

84

96

108 120 132 144 156 168 Time series

Fig. 3. Fitting curve of WANN model of monthly groundwater depth in 853 Farm

834

H. Ding, D. Liu, and F.-f. Zhao

4.5 Precision Inspection of WANN Model

The fitted values of WANN model are used for testing fitting effect, and the measured monthly groundwater depth of 2005 that can not part in modeling are used for testing forecast effect. The specific results are listed in table 1. According to reference [13], indexes reached primary standard. Thus, the WANN model can be applied to forecast the future groundwater depth of 853 Farm because of well fitting effect and high prediction accuracy. Table 1. Precision test results of WANN model

C 0.0020

Fitting effect index p E1 (%) 1

0.12

E2

Try forecasting effect index E3 (%)

1

100

4.6 Prediction of Groundwater Depth

We forecast the groundwater depth from 2006 to 2009 based on the WANN model that has been well trained and tested, the predictive values and curve are shown in table 2 and Fig.4. It can be seen that the groundwater level will decrease continuously and seriously if we still use the past mining mode. The annual decreasing ranges of groundwater depth from 2006 to 2009 are 0.42m, 0.52m, 0.79m and 0.90m respectively, and the average annual decreasing range is about 0.66m next 4 years. Table 2. Prediction value of monthly groundwater depth in 853 Farm(2006 2006 7.0743 7.3655 7.7166 8.6952 10.5423 11.5616

2007 7.6935 7.9901 8.2661 9.3276 10.8266 11.9046

2008 8.2543 8.7309 9.2958 9.9793 11.6437 12.8137

2009 8.8813 9.5253 9.8549 10.8742 12.2377 13.7848

7 8 9 10 11 12

2006 11.0223 10.6332 8.6077 7.8716 7.3795 7.2722

2007 11.5693 10.8843 9.1945 8.5005 7.9756 7.8583

2008 12.2258 11.3255 10.3875 9.3729 8.7666 8.6434

2009 13.1495 12.5178 11.3926 10.2111 9.8982 9.8878

Groundwater depth H(t) (m)

1 2 3 4 5 6

～2009) unit：m

Time series t

～

Fig. 4. Predictive curve of WANN model of monthly groundwater depth in 853 Farm (2006 2009)

Variation Trend Analysis of Groundwater Depth in Area of Well Irrigation

835

5 Discussions Now, we still use the measured series of monthly groundwater depth from 1991 to 2004 of 853 Farm to establish the groundwater depth forecast model by using Times series analysis and BP neural network(neglected the process of modeling) to fit the monthly groundwater depth and forecast the monthly groundwater depth of 2005 respectively. Through statistical analysis, we compared fitting and forecast results of Times series (TS) and BP neural network model (BP) with WANN model (WANN), the results are given in Table 3 and Table 4. Table 3. Fitting results comparison of monthly groundwater depth Model TS

Absolute error(m) Maximum Average 0.9625 0.2422

Relative error(%) Maximum Average 33.42 5.59

BP

0.1322

0.0120

3.79

0.31

WANN

0.0234

0.0041

0.97

0.11

From Table 3 and Table 4, it can be seen that BP neural network and WANN model have higher fitting and forecast precision than Times series obviously. But the fitting and forecast precision of WANN model is a bit higher than BP neural network. Table 4. Prediction results comparison of monthly groundwater depth in 853 Farm in 2005 Month 1 2 3 4 5 6 7 8 9 10 11 12 Average

Measured value (m) 6.63 6.91 7.67 7.78 10.51 11.38 10.83 9.71 8.02 7.57 6.85 6.84 8.39

Predictive value(m) TS BP WANN 6.7542 6.6968 6.6327 7.2052 6.8323 6.8140 7.6094 7.1660 7.4877 7.7932 7.6319 7.8106 10.0553 10.3528 10.4838 10.3988 11.6731 10.9983 10.3804 9.8147 10.6246 9.6530 9.3670 9.8734 8.6482 8.5464 8.4636 8.0146 7.7705 7.5652 7.5733 7.3950 7.1727 7.4821 6.9539 7.0568 8.4640 8.3417 8.4153

Relative error (%) TS BP WANN 1.87 1.08 0.04 4.27 1.12 1.39 0.79 6.57 2.38 0.17 1.90 0.39 4.33 1.50 0.25 8.62 2.58 3.35 4.15 9.37 1.90 0.59 3.53 1.68 7.83 6.56 5.53 5.87 2.65 0.06 10.56 7.96 4.71 9.39 1.67 3.17 4.87 3.74 2.07

6 Conclusions (1) WANN model is an effective method to research nonlinear and uncertainty problem by binding the multi-resolution function of wavelet analysis and nonlinear approximation function of artificial neural network. In this research, we established the WANN forecast model of monthly groundwater depth of 853 Farm based on the

836

H. Ding, D. Liu, and F.-f. Zhao

hydrological wavelet analysis and BP neural network and comprehensive revealed the dynamic variation regularities of groundwater, and provided a reliable gist for sustainable utilization of groundwater resource. (2) The WANN model has certain advancement and practicality because of simple principle and rapid calculation capability. The comparison results revealed that WANN model has higher fitting and prediction precision than traditional Times series and BP neural network. (3) The prediction results showed that the groundwater level of 853 Farm will decrease continuously in future years. Therefore, the local government should reinforce the scientific management and control groundwater mining strictly in order to prevent the deterioration of logical environment because of continuing decreased groundwater. Acknowledgements. This study has been funded by Natural Science Foundation of China (No.30400275), Dr Start-up Fund Research of Northeast Agricultural University (No.2009RC37), Postdoctoral Science Foundation of China (No.20080440832), Scientific and Technology Foundation of Education Department of Heilongjiang Province (No.11541024).

References 1. Liang, C.Y., Wang, X., Yang, T.W.: Precision Agriculture’s Application Research and Discussion in Sanjiang Plain. Journal of Agricultural Mechanization Research 3, 193–194 (2004) 2. Wang, S.H., Tian, Y.: Preliminary Research on Groundwater Table Change and Cause in Sanjiang Plain. Journal of Irrigation and Drainage 22(2), 61–64 (2003) 3. Wu, D.J., Wang, J.S., Teng, Y.G.: Application of Wavelet Decomposition and Wavelet transform Method to Forecasting of Groundwater Regime. Journal of Hydraulic Engineering 5, 39–45 (2004) 4. Li, H.B., Chen, B., Zhao, J.W., et al.: Application of Wavelet Transform Technology to the Improvement of Analyzing Accuracy of Vinegar Near-Infrared Spectrum. Transactions of the CSAE 16(6), 114–117 (2000) 5. Subasi, A., Kiymik, M.K., Akin, M., et al.: Automatic Recognition of Vigilance State by Using a Wavelet-based Artificial Neural Network. Neural Compute & Apples 14(1), 45–55 (2005) 6. Wang, W.S., Ding, J., Li, Y.Q.: Hydrological wavelet analysis. Chemical Industry Publishing House, Beijing 7. Zhang, W.J., Su, Z.Q.: Exploration of Wavelet Analysis in Fault Diagnosis of Vehicle Gearbox. Transactions of the CSAE 17(2), 101–103 (2001) 8. Liang, Y.F., Miao, R., Yang, X.C.: Application of Wavelet Network in Forecast of Annual Runoff. Northeast Water Conservancy and Hydropower 24(261), 9–11, 71 (2006) 9. Zhong, P.A., Zhao, X.: Cycle Forecasting Model of the Reservoir’s Stream Process based on the Wavelet Transform. Hydropower 32(2), 17–19 (2006) 10. Wegner, F.V., Both, M., Fink, R.H.A.: Automated Detection of Elementary Calcium Release Events Using the À Trous Wavelet Transform. Biophysical Journal 90(6), 2151–2163 (2006) 11. Jiao, H.M., Yu, Q.M.: Surplus Capacity Predication of MH-Ni Battery for Electric Vehicle. Computer Simulation 23(3), 211–214 (2006) 12. Fu, Q.: System Analysis and Comprehensive Evaluation of Agriculture Water and Land Resources. China Press of Water Resources and Hydropower Research, Beijing 13. Chen, N.X.: Dynamic Prediction Model Precision Evaluation of Groundwater. Engineering Investigation 3, 35–38 (1999)

A Petri-Net Modeling Method of Agent’s Belief-Desire-Intention and Its Application in Logistics Weifeng Zhu and Qi Fei Institute of Systems Engineering, Huazhong University of Science and Technology, Wuhan 430074, P.R. China [email protected]

Abstract. A Petri-Net modeling method of agent’s belief-desire-intention called BDIPN is proposed to describe managers’ belief. It is combined into Batch Deterministic and Stochastic Petri Nets to describe inventory systems, where the model is called BDI-BDSPN. Two types of logistics, Complex Logistics and Agile Logistics, are both modeled using BDI-BDSPN. Keywords: Petri-Net, Agent, Belief-Desire-Intention, Complex Logistics, Agile Logistics.

1 Introduction Supply chains or logistics are typical discrete event systems. The complexities of such systems include structural complexity, uncertainty, information distortion and enlargement, organization dynamic and so on. In addition to objective factors, there are human factors resulting in such complexities. Manager’s subjective idea sometimes will have influence on such logistics decisions as what goods should be ordered, when order, and order how much. In other words, logistics management does not always follow a fixed mode, where manager’s intention is sometimes a very important factor. Petri net is a powerful tool for modeling and analysis of discrete event systems. Colored Petri Nets are used to model supply chains [1], where every entity in a chain is modeled by a block with action, resource, and control, and each block is modeled by a colored Petri net. Supply chains are also modeled as Generalized Stochastic Petri Nets [2]. Recently, First-Order Hybrid Petri Nets [3] that combine fluid and discrete event dynamics are used to model inventory systems with independent demand [4]. Batch Deterministic and Stochastic Petri Nets [5] are developed as a tool for modeling and performance evaluation of supply chain. It is derived by enhancing deterministic and stochastic Petri nets with batch places and batch tokens. Batch tokens, which reside in batch places, have sizes and are viewed as different individuals. They are used to describe the information flow in a supply chain. By incorporating stochastic Petri nets with the batch features, inhibitor arcs, and marking dependent weights, operational policies of a supply chain can be easily described in the model. However, the common shortage of these models is that they do not reflect managers’ belief when describing supply chains or logistics. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 837–844. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

838

W. Zhu and Q. Fei

In this paper, a Petri-net modeling method of agent’s belief-desire-intention is proposed to describe managers’ belief. It is combined into Batch Deterministic and Stochastic Petri Nets to describe inventory systems and two types of logistics, which can more completely reflect the intelligence of such systems. The remainder of this paper is organized as follows: The Petri-net modeling method of agent’s belief-desire-intention called BDIPN is proposed in Section 2. The method is then combined into Batch Deterministic and Stochastic Petri Nets to model inventory systems in Section 3, which is called BDI-BDSPN. Two types of logistics, Complex Logistics and Agile Logistics, are modeled using BDI-BDSPN in Section 4 and Section 5 respectively. Conclusions are given in Section 6.

2 Petri-Net Modeling Method of Agent’s Belief-Desire-Intention As we know, agent’s action is driven by its mind. Belief, desire and intention are the three main mind statuses. Agent’s belief is the information it has about the world, which may be incomplete or even incorrect. The information can be divided into objective factors and subjective attitudes. Agent’s desire is the goal it hopes to get. Agent’s intention is the action it takes to achieve the goal. Thus, agent’s belief and desire decide its intention. The relationship of belief, desire and intention can be described by colored Petri Nets shown in Figure 1. Because there are commonly multi-agents and multi-decisions to be described, different colors must be used to color these types of agents and decisions. Then it forms colored Petri Nets consisting of three parts: net structure, declaration and inscription [6]. The net structure includes places, transitions and arcs. In Figure 1, place B denotes agent’s belief, place D denotes agent’s desire, place I denotes agent’s intention, transition tB denotes creating or updating agent’s belief, transition tD denotes creating or updating agent’s desire, transition tI denotes creating or updating agent’s intention according to agent’s belief and desire. The three transitions can be denoted by instantaneous, deterministic delay or stochastic delay transition, which lying on the type of time period the agent takes in the decision. tB and tD are source transitions which do not have input places. In order to chart Petri nets easily, tB and tD are omitted in the following discussion. In the declaration, 8 color sets and 4 variables are defined, where A is the set of agents, V is the set of decision variables, BeliefList is the set of believes, DesireList is the set of desires, IntentionList is the set of intentions, B, D and I are compound color sets, x and y are agent variables, m and n are decision variables. The inscription includes place names, color sets, initial expressions, transition names and guards, and arc expressions. Transition guards are logical expressions comprised in square brackets which must be verified before the transition acts. Only when the logical expressions are all true can the transition act. In Figure 1, [x=y] and [m=n] are two guards, as denotes that only when the agent variable and the decision variable are both equal can tI act. f(< x, m, Belief>,< y, n, Desire>) is an arc expression, which denotes that the agent’s intention is decided by agent’s belief and desire, and there are functional relationships between them. This Petri net modeling method of agent’s belief, desire and intention is called BDIPN by us. The corresponding Petri nets are called BDIPNs.

A Petri-Net Modeling Method of Agent’s Belief-Desire-Intention

839

Color A = Agent List: Ai, i = 1…I; Color V = Decision Variable List: Vj, j = 1…J; Color BeliefList = List Belief; Color DesireList = List Desire; Color IntentionList = List Intention; Color B = product A*V*BeliefList; Color D = product A*V*DesireList; Color I = product A*V* IntentionList; var x : A; var y: A; var m:V; var n: V;

›< Ai, Vj, Belief>

< x, m, Belief> B tB

< x, m, Belief>

[x=y] [m=n]

tI < y, n, Desire> D tD

›< Ai, Vj, Intention> I

f(< x, m, Belief>,< y, n, Desire>)

< y, n, Desire> ›< Ai, Vj, Desire> Fig. 1. Colored Petri Nets of an agent

Figure 1 is the basic colored Petri nets of agent’s BDI which should be extended during application. When an agent forms the intention of a decision, it considers not only its belief and desire but also other agents’ intentions. Thus agent types can be divided into autonomous agent, self-autonomous agent and dependent agent according to agent’s different management and decision privileges. The decision of an autonomous agent is completely autonomous where the intention is completely formed by itself belief and desire. But it does not mean that the autonomous agent will not be affected by global agents or up-level agents, sometimes it has competition and cooperation relationships with them. The decision of a self-autonomous agent is self-autonomous where the intention is not only formed by itself belief and desire but also constrained by its global agents or up-level agents. A dependent agent has not decision privileges where the intention is obtained from its global agents or up-level agents. Figure 2(a) (b) (c) shows respectively the BDIPNs of these three types of agents. LB, LD and LI denotes respectively the belief, desire and intention of a local agent. GI denotes the intention of a global agent. Transition t denotes the formation of intention. The definition of color, variable declaration and inscription can reference Figure 1.

LB t LI

t

LD

LI

GI

t

LI

(c) BDIPNs of a dependent agent

LD

GI

(a)BDIPNs of an autonomous agent (b) BDIPNs of a self-autonomous agent

Fig. 2. BDIPNs of three types of agent

840

W. Zhu and Q. Fei

3 Modeling Inventory Systems with BDIPNs and Batch Deterministic and Stochastic Petri Nets Inventory policies play a very important role in logistics management. They are designed to address the fundamental issues: when should a stock be replenished and how much should be ordered with a replenishment order. Batch Deterministic and Stochastic Petri Nets (BDSPNs) have been developed to model and performance evaluation of inventory systems. However, manager’s subjective belief was not considered when describing inventory decision in BDSPNs. In this section, we discuss how BDIPNs can be combined into BDSPNs to model inventory systems with independent demand. Inventory systems with independent demand may use fixed or variable order quantity policies based on either continuous review or periodic review of inventory position. Fixed order quantity policies place an order of fixed size whenever stock falls to a certain position. They usually need continuous monitoring of stock position. Periodic review policies place orders of variable size at regular intervals to raise the inventory position to a specified value. The (t, S) inventory policy is widely used in logistics. In each period, the inventory position that is the inventory on-hand plus outstanding orders minus backorders is periodically inspected and a quantity is ordered to bring the position to a given orderup-to level S [7]. Thus S is the main decision variable in an inventory system which depends on inventory managers’ subjective belief besides market factors. Thus, decision variable S can be described by BDIPN. LBS, LDS and LIS denotes respectively the belief, desire and intention of the local agent for S. GBS, GDS and GIS denotes respectively the belief, desire and intention of the global agent for S. The belief of S expresses the possible scope of S that the manager believes. The desire of S can be cost minimization, profit maximization and highest fill rate. The intention of S is a subset of its belief, which is the action the manager takes to achieve the goal above. The BDSPNs of an inventory system built by H.Chen [5] shows the batch orders and stochastic demands features. We combine BDIPNs and BDSPNs to describe inventory systems. This method is called BDI-BDSPN by us. The corresponding Petri nets are called BDI-BDSPNs. Figure 3 shows the BDI-BDSPNs of a self-autonomous inventory system using (t, S) inventory policy whose S is affected by the core enterprise. So its LIS is formed not only by LBS, LDS but also by GIS. In figure 3, transition t8 denotes creating or updating local agent’s intention of S, transition t9 denotes creating or updating its intention of S influenced by the core enterprise, both of which are stochastic delay transitions. Place I represents the on-hand inventory of the stock considered (the stock is called stock I below), represents the on-hand inventory of the stock from which stock I is replenished, batch place O represents outstanding order-the orders that are placed by stock I but not filled, discrete place B represents backorders of the stock. Place IL represents the on-hand inventory of the stock plus its outstanding orders. The marking of I, the marking of B and the marking of IL are denoted by MI, MB and MIL, the inventory position of stock I is equal to

A Petri-Net Modeling Method of Agent’s Belief-Desire-Intention

841

MIL-MB. Customer demand is a stochastic Poisson process, which is specified by transition t7 whose firing time is subject to an exponential distribution.

Color E=Entity List:I|J|…; Color S = Order-upto level; Color ES = product E*S; Color BeliefList = List Belief of Orderup-to level; Color DesireList = List Desire of Orderup-to leve; Color EB = product E* BeliefList; Color ED = product E* DesireList; Var e: E; Var s: S;

t3

J

t2

I

t1

O

t7

B

t4 s-MIL+MB IL

s+MB s-MIL+MB

t5

LIS

<e,belief>

LBS

[e=I]

P

LDS

<e,desire>

GBS P t6

t8

<e,s>

GIS

t9

GDS

Fig. 3. The BDI-BDSPNs of a self-autonomous inventory system using (t, S) inventory policy

Customer demand will be fulfilled if there is available on-hand inventory, otherwise it will be backordered. The fulfillment of customer demand will decrease on-hand inventory as well as the inventory level. This is described by the arcs from place I, B and IL to transition t1. When the inventory position MIL-MB of stock I is below the order-up-to level S, i.e. MIL<S+MB, an order with size S-(MIL-MB) will be placed to bring inventory position to S. The placement of the order will increase the outstanding orders by S-MIL+MB. This is described by immediate transition t4, its associated arcs to places O and IL, and the weights of the arcs. When the marking MIL of place IL is less than S+MB, transition t4 will be fired, which will generate a batch token (order) with size S-MIL+MB in place O and S-MIL+MB tokens in place IL,. If there is a batch token (order) in place O and the numbers of tokens in place J is equal to or larger than the size of the batch of the token, transition t2 will be fired, which delivers the order to stock I. As the (t, S) inventory policy is time period, transition t4 is deterministic delay. The replenishment of stock J is described by transition t3. The subset with place P , P and transition t4, t5, t6 is designed to describe the periodic review feature of the policy where transition t6 is a deterministic transition with its firing time being each review period. Transition t4 and t5 are two immediate transitions with the priority of t4 higher than that of t5.

842

W. Zhu and Q. Fei

Similarly, we can use BDI-BDSPN to model autonomous inventory systems and dependent inventory systems using the (t, S) inventory policy. And other inventory policies can also be modeled with this method.

4 Application of BDI-BDSPN in a Complex Logistics A complex logistics (CL) is a netlike supply chain (SC) logistic system consisting of manufacturers, wholesalers and retailers which is always in an uncertain environment. CLs have different logistics net forms including chainlike, treelike, bilateral treelike and starlike. In this section, we build the BDI-BDSPNs of a CL with this structure considering material flow and information flow only. In material flow, suppliers provide raw materials to manufacturers. Then manufacturers convert raw materials into intermediate products or end-products which are transported to wholesalers through distribution channels. The wholesalers also will ship products to retailers, and retailers will eventually sell products to consumers. The whole logistics contains raw material flow, intermediate product flow and final product flow. In information flow, a manufacturer receives orders from wholesalers. When the stock level of end-products in the manufacture is below a pre-specified level, an assembly order with a given batch size will be sent to the assembly lines of the manufacturer. At the same time, when the stock level of raw materials in the manufacture is below a pre-specified level, a purchase order with a given batch size will also be placed to their corresponding suppliers. Each purchase or assembly order contains the information such as order time and order quantity, which are determined by the inventory policy adopted by the manufacture. Customer demand is fulfilled by retailers. When the stock level of end-products in a retailer is below a pre-specified level, a purchase order with a given batch size will also be placed to their corresponding wholesalers. In addition, the core enterprise of CL will send the decisions of some key parameters, such as the order-up-to level we mentioned above, product price and so on, to manufactures, wholesalers and retailers which are dependent or self-autonomous entities. That’s to say, different CL management modes have different types of information flow. Suppose a CL composed of two suppliers providing different raw materials, a manufacturer, a wholesaler and two retailers where the suppliers are autonomous entities, the wholesaler is a dependent entity, the retailers are self-autonomous entities and the manufacturer is a dependent entity. They all adopt the (t, S) inventory policy. Figure 4 shows the BDI-BDSPNs of this CL. In this model, the suppliers only consider stock replenishment. Both manufacturer’s intention and wholesaler’s intention of order-up-to levels are get from the core enterprise (the arcs from place GIS to transition tMO1, tMO2, tWO1). Retailer’s intention of order-up-to level is determined by both itself and the core enterprise (the arcs from place GIS to transition tSR1 and tSR2). Because the core enterprise will manage the order-up-to levels of all entities in CL, different colors must be used to color GBS, GDS and GIS. In figure 4, we only describe the logic relationship about belief, desire and intention between the core enterprise and other entities. The coloring method can reference figure 1 and figure 3.

A Petri-Net Modeling Method of Agent’s Belief-Desire-Intention

tS1

PS1

tSM1

LBSR1

PMM1

tRO1

PRIL1

LISR1

LDSR1

PB1

tMO1 PMILM1

tRS1

PMO1

tSR1

PR1

tWR1

GBS tGS

tD1 t

GIS GDS tS2

843

tWR2 PR2 PMP

PMO2

tWO1 PS2

tMW1 PW1

tSM PMM1

PB2 tRO2

PWIL1

tMO2 PMILM2

tRS2

PRO2 PWO LBSR2

PRIL2

tSR2

tD2

LISR2

LDSR2 Fig. 4. The BDI-BDSPN model of a CL

5 Application of BDI-BDSPN in an Agile Logistics An agile logistics (AL) includes a number of manufacturers, military warehouses and troop inventories. The structure of an AL is similar to a CL. However, the management mode of an AL is different from that of a CL. The goal of an AL is to meet the goods demands of troops unconditionally. Thus the entities in an AL are always self-autonomous even dependent. The purpose of the Petri nets description is to correctly distinguish the distributed and commanding characteristics of an AL, preparing for the simulation modeling. The key point to build the BDI-BDSPNs of an AL is depicting the organization relationship of the AL correctly, combining with material flow and information flow. In an AL, there usually exists a specialized material supply management department. In force logistics management, there are command centers at various levels to manage and dispatch materials. Therefore, the command centers in an AL are similar to the core enterprise in a CL. All of entities in an AL also face the decision problem of order-up-to level S. The troop inventories are usually selfautonomous entities whose order-up-to level is determined by both their command centers and themselves. Military warehouses are dependent entities where the orderup-to level is only determined by their command centers. The manufacturers or suppliers are autonomous entities where the order-up-to level is determined by themselves. Then we can get BDI-BDSPNs of an AL shown in figure 5. In this model, GBSN, GDSN and GISN denote the belief, desire and intention of S for the Nth level command center respectively. N-1 denotes the upper level command center. N+1 denotes the lower level command center. BDSPNN-1, BDSPNN and BDSPNN+1 denotes the BDSPNs of each level. As an AL often uses continuous review policy, the time periodic transitions should be changed into immediate ones.

844

W. Zhu and Q. Fei

GBSN-1 tGSN-1

BDSPNN

GIS

GDSN+1

GBSN

GDSN-1 BDSPNN-1 upper BDI-BDSPN model

GDSN

GBS

tGS

GIS

local BDI-BDSPN model

tGSN+1 GISN+1

BDSPNN+1 lower BDI-BDSPN model

Fig. 5. The BDI-BDSPN model of AL

6 Conclusion Manager’s subjective idea sometimes will have influence on such logistics decisions. Before simulation modeling, we should analyze and model the intelligent characteristics of logistics. A Petri-Net Modeling Method of Agent’s Belief-DesireIntention is proposed in this paper to describe managers’ belief. It is combined into Batch Deterministic and Stochastic Petri Nets to describe inventory policies and two types of logistics, which can more completely reflect the intelligence of such systems. Acknowledgments. This research was supported by National Natural Science Foundation of China (Grant No. 60674081 and 60804038), and key scientific and technical research project of Ministry of Education of China (Grant No. 109104). This support is acknowledged gratefully.

References 1. Van der Aalst, W.M.P.: Timed Colored Petri Nets and Their Application to Logistics. Technical University of Eindhoven (1992) 2. Viswanadham, N., et al.: Performance Analysis and Design of Supply Chains: a Petri Net Approach. Journal of the Operational Research Society 51, 1158–1169 (2000) 3. Balduzzi, F., et al.: First-order Hybrid Petri Net: a Model for Optimization and Control. IEEE Trans. on Robotics and Automation 16, 382–399 (2000) 4. Furcas, R., et al.: Hybrid Petri Net Modeling of Inventory Management Systems. APIIIJESA 35, 417–434 (2001) 5. Chen, H., Amodeo, L., Chu, F.: Batch Deterministic and Stochastic Petri Nets: a Tool for Modeling and Performance Evaluation of Supply Chain. In: Proceedings of the 2000 IEEE International Conference on Robotics & Automation, Washington, DC (May 2002) 6. Jensen, K.: Coloured Petri Nets: a High Level Language for System Design and Analysis. In: Rozenberg, G. (ed.) Advances in Petri Nets. LNCS, vol. 383, pp. 342–416. Springer, Heidelberg (1990) 7. Chopra, S., Meindl, P.: Supply Chain Management, Strategy, Planning, and Operation. Tsinghua University Press, Beijing (2001)

Supply Chain Flexibility Assessment by Multivariate Regression and Neural Networks Ananda S. Jeeva1 and William W. Guo2 1

School of Information Systems, Curtin Business School, Curtin University of Technology, P.O. Box U1987 Perth, Western Australia [email protected] 2 Faculty of Arts, Business, Informatics and Education, Central Queensland University, North Rockhampton QLD 4702, Australia [email protected]

Abstract. This paper compares two vastly different methods of analysis – multiple regression and neural networks, in supply chain flexibility assessment. Data of manufacturing firms evaluating their prominent suppliers were analysed by multiple regression and simulated using three-layer multilayer perceptron (MLP) neural networks. Our study shows that NN can accurately determine a supplier’s flexibility capability within an error of 1% The incorporation of these two methods can lead to better understanding and dynamic prediction of supply chain flexibility for buyers. Keywords: Neural network, supply chain, flexibility, multivariate regression.

1 Introduction The current economic climate pushes businesses to be more competitive and thus leads to re-evaluating their supply chain and supplier strategies. This competitiveness requires organisations to be more effective, efficient and productive with lower margin for error. It is also pushing companies to work more strategically with their supply chains and partners [1]. Hence strategic supply partnership is becoming more critical and will determine the sustainability and competitiveness of companies in the future [2]. Flexible supply policies also help companies adapt quickly to changing market conditions and become more competitive [3]. Competitiveness takes many forms, and this paper focuses on the parts of the supply chain activities that inter-relate with suppliers and manufacturers in their operational aspects. Supply chain operational aspects have many dimensions of functions and activities. Specific dimensions are selected and considered in this paper. These selected dimensions are related to the activities of the responding manufacturing firms with their most prominent suppliers. These dimensions are information exchange, level of supplier integration, product and delivery variations, logistics, and the organisation’s capability. These dimensions are selected as they represent some of the principal components of published supply chain literature as proposed by Croom et al [4]. These selected dimensions are hypothesised to have the highest ability and agility to be flexible. This flexibility is related to changes in the external and internal Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 845–852. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

846

A.S. Jeeva and W.W. Guo

environment. As the factors in the external environment change, these supply chain dimensions are also expected to change accordingly to maintain the same level of efficiency and effectiveness. Flexibility and agility have been promulgated as strategic responses to the stochastic economy [5,6]. Aprile et al [7] agree that flexibility is a complex and multi-dimensional concept and propose that it reflects the capacity to adapt to changing internal and external conditions. Therefore, flexibility is necessary to maintain the competitive edge. From a practitioner’s point of view flexibility and adaptability are very necessary and useful in practical applications. Flexibility is necessary in many dimensions of the supply chain. Flexibility in delivery dates, quantity and inventory replenishments buffer the impact of uncertainties. Practitioners desire a flexibility guideline with practical applications. The 2009 AMR Research [8] of the top 25 leading supply chain leader companies shows that agility, responsiveness, reliability and predictability were rated very highly. Efficacy of supply chain flexibility is further promulgated by Khan et al [9] who provide empirical justification that the flexible distribution practices promote greater organisational performance. This paper focuses on the determinate factors of supply chain flexibility to predict performance by flexibility measures. We describe two systems that use statistical and artificial neural network techniques respectively for determining the flexibility of suppliers based on their selected elements. The data set used in this study was collected from 241 manufacturers commenting about their suppliers’ flexibility based on five dimensions of supply activities. A comparison is also made between these two approaches in terms of assessing supply chain flexibility.

2 Selected Supply Chain Flexibility Dimensions The five dimensions that this study focuses on are information exchange, supplier integration, product and delivery, logistics, and organisational strategy. These five dimensions were found to be the common set of supply chain flexibility dimensions in most research literature [10]. The information exchange (IE) investigates the elements of suppliers’ information, such as sufficiency of information, reliability of information, timeliness, accuracy, easy of sending, amount of human intervention required, and connectivity and compatibility of information systems. Agarwal [11] concluded that flexibility in IT systems is a strategic feature that offers unique advantages for decision making, competitiveness and systems design. The supplier integration (SI) activity takes into consideration of the operational elements of suppliers’ inventory levels, such as delivery schedule changes, service level standard, costs, ease and time to switch suppliers. The supplier integration activities represent the amount of suppliers’ ability to integrate into the manufacturing process. Palsson and Johansson [12] conclude that the greater the integration the greater the opportunities for improvements in the supply chain activities.

Supply Chain Flexibility Assessment by Multivariate Regression and Neural Networks

847

Material compatibility is an issue in manufacturing. Ball et al [13] state that these issues arise when there are a number of different component variants. Product and delivery (PD) elements include the suppliers’ ability to design and deliver new components, to modify the product mix, to implement product design, and to modify components variability. This element relates to the suppliers’ capability and capacity to adapt to consumer’s changing demands. Supplier logistics (SL) elements are the physical movement and handling of the procured products. It acts as the physical link between supplier and buyer enabling the flow of products, components and materials. The delivery speed and reduction of lead time is critical in the production process and customer satisfaction [14]. The degree of flexibility may be constrained by the logistics carrier’s environment and operational framework. For example, the cost of fuel and transportation may decide the best choice for logistics operations and how flexible it is for the supplier to modify logistics arrangements [15]. Therefore, logistics plays a critical role in supply chain flexibility. They include the modification of routes, transportation, performance and product variability, and materials handling in terms of product attributes. Organisational strategy (OS) indicates how well the internal structure of the manufacturing organisation can adapt to changing external environment in terms of the supply chain functions. They include business strategy, cost of implementation, and responsive actions. Fantasy et al [16] in their empirical study found that firms need to build flexibility dimensions to fit into their organisational strategies to deal with direct effects of flexibility on performance. Manufacturing organisations’ suppliers that have the most flexibility in the above mentioned activities and elements will be able to respond to their own internal stochastic environment as well as their suppliers’ external environment. The flexibility attributes must be built into all the activities and elements of supply chain dimensions discussed above.

3 Multiple Regression Analysis This study was based on finding out the extent of manufacturers’ suppliers flexibility capability in five key supply chain dimensions. Scale items were developed using Churchill [17] paradigm. Items were developed from literature review. Q-sort technique was used to refine the survey items. Pilot surveys were conducted and subjected to validity and reliability tests. Data was collected on a seven point Likert scale. The higher the number; the higher the flexibility. The final survey instrument used a self administered postal survey which received a response rate of 21.1%. The survey was administered to senior managers in the roles of supply chain/procurement/logistics/manufacturing/purchasing managers in large manufacturing companies. The sampling frame was 1300 companies from different industries ranging from fabricated metal products to food and kindred products. The main respondent criteria were manufacturers with more than 70 employees and annual procurement value of more than AUD$10 million. We know that the higher the five selected dimensions; the higher the flexibility of a supplier, but the correlation between the flexibility and five selected dimensions is

848

A.S. Jeeva and W.W. Guo

still an unknown function. This is because the quantification of the five dimensions is based on human judgment that varies from person to person; hence correspondingly the flexibility is somehow an estimation based on the five dimensions. Traditional model for measuring and analysing such problem is based on multiple regression, which is expressed as Y = A0 + A1x1 + A2x2 + … + Amxm + e,

(1)

where Y is the dependent variable as a function of m independent variables x1, x2, …, xm; A0, A1, …, Am are the coefficients that determine the contribution of each independent variable to the outcome; e is the error component and supposed to be a small constant [18]. This expression can be rewritten as Y = y + e,

(2)

where y = A0 + A1x1 + A2x2 + … + Amxm is the deterministic portion of the model and can be used to estimate the output of this model if all m+1 coefficients can be determined using sufficient known points. In fact, the set of coefficients determined by known points can only be regarded as the best estimate of individual coefficients due to the existence of random errors contained in the observed points. Therefore, the deterministic portion of the model, y, is approximated by y ≈ a0 + a1x1 + a2x2 + … + amxm,

(3)

where a0, a1, …, am are the estimated coefficients for A0, A1, …, Am, respectively. One way to determine those estimated coefficients is to use the method of least squares through minimising the sum of squares of the errors (SSE) of n known points as n

n

i =1

i =1

SSE = ∑ (Yi − yi ) 2 =∑[Yi − (a0 + a1 xi1 + a2 xi 2 + ... + a m xim )]2 .

(4)

Minimising SSE will be achieved through solving the following system of simultaneous linear equations. ∂SSE =0. ∂a j

(j = 0, 1, …, m).

(5)

For n known points, this should lead to the (m + 1) least-squares linear equations in a0, a1, …, am as na0 + (∑ xi1 )a1 + (∑ xi 2 )a 2 + ... +(∑ xim )am = ∑ yi (∑ xi1 )a0 + (∑ xi21 )a1 + (∑ xi1 xi 2 )a2 + ... +(∑ xi1 xim )am = ∑ xi1 yi (∑ xi 2 )a0 + (∑ xi1 xi 2 )a1 + (∑ xi22 ) a2 + ... +(∑ xi 2 xim ) am = ∑ xi 2 yi

.

(6)

# 2 (∑ xim )a0 + (∑ xi1 xim )a1 + (∑ xi 2 xim )a2 + ... +(∑ xim ) am = ∑ xim yi

Solving this system of linear equations should determine the (m + 1) coefficients that hence define a multiple regression model.

Supply Chain Flexibility Assessment by Multivariate Regression and Neural Networks

849

Using statistical tools provided in MATLAB, we apply multiple regression to the 241 entries mapped between the five factors as inputs and the flexibility as output (Flex), which defines a multiple regression model shown below Flex = 0.0643 - 0.8593IE + 0.6009SI - 0.9827PD - 0.8693SL - 0.9538OS.

(7)

This regression is significant with a coefficient of 0.9852 and a maximum error of just about 10%. The accuracy of determining the flexibility of a supplier using this regression will be discussed later incorporating the results from the neural network simulation.

4 Neural Network Simulation Neural networks have been widely used in investment prediction and financial analysis [19,20]. This paper attempts to use it in a supply chain management environment. The use of neural networks offers several advantages over traditional statistical methods. There are no restrictive assumptions as imposed by traditional methodologies. Neural networks use all available information while incorporating new information through a learning process. It updates the old output and learns from its experience. Therefore it is flexible and adaptable and can be used in a changing environment like stochastic economy. Supply chain management activities need to be flexible to respond to external environmental uncertainties. The core of a neural network is actually an adaptive mathematical model that is capable of approximating any arbitrary unknown function constrained by training datasets. It has been proven that a three-layer multilayer perceptron (MLP) neural network can approximate any continuous function mapped from one finite-dimensional space to another by adjusting the number of nodes in the hidden layer [21]. Numerous cases of three-layer MLP applications have been successful in different fields [22-25]. Figure 1 illustrates the structure of a three-layer MLP with one hidden layer of L nodes, a p-dimensional input vector X and a q-dimensional output vector Y. The relationship between the input and output components for this MLP can be generally expressed as. L

y k = ϕ ( ∑ vkjψ (∑ u ji xi )) .

(8)

j =1

where ϕ and ψ are the transfer functions; uji denotes the input-to-hidden layer weights at the hidden neuron j; and vkj is the hidden-to-output layer weights at the output unit k. The neural network process involves two phases – training the network with known datasets and testing the trained network using different known datasets for model generalisation. This is different from statistical approaches, in which the same data can be used for both model generation and evaluation. In this study, the MLP model is constructed and trained using neural network tools in MATLAB® [26]. The Levenberg-Marquardt algorithm [27] is chosen to train the selected MLPs because this algorithm has been reported to be the fastest method for training moderate-sized feedforward neural networks [28].

850

A.S. Jeeva and W.W. Guo

Fig. 1. Three layer multilayer perceptron (MLP) network

The 241 entries are compiled into two sets. The first set contains 207 (86%) entries for training and the second set includes 34 (14%) entries for testing. The log-sigmoidlinear combination is chosen as the transfer functions for our MLPs. The running of a number of nodes in the hidden layer shows that a 25-node hidden layer is good enough to achieve satisfactory training. The test results for this 25-neuron MLP are tabulated in Table 1, along with the results for the regression model using the same test set. Both models show good quality for forecasting, but the MLP model is more accurate than the regression model, which can be clearly seen in Fig. 3. Table 1. Testing result of MLP and regression models MLP Regression

Test size 34 34

Correlation 0.9999 0.9829

MAE (%) 0.15 3.12

Fig. 2. Plots of relative errors

SD (%) 0.24 2.53

MAX (%) 0.88 9.43

Supply Chain Flexibility Assessment by Multivariate Regression and Neural Networks

851

5 Discussion and Conclusion Although the multivariate regression defined in Relation (7) gives an analytical expression that is easier to understand, its performance for predicting a supplier’s flexibility through its five attributes may result in an average error of 3% with a maximum error up to 10%. This is still useful if the manufacturing scale is relatively small. However, if the scale of manufacturing is large, an error of 1% would mean a loss of millions of dollar in revenue. The MLP model predicts the flexibility of a supplier with an average error of less than 0.2% and a maximum error within 1%. The contrast of these two approaches is clearly illustrated in Fig. 3. Therefore, the neural network model outperforms the statistical model in quantitative simulation. This MLP model creates a dynamic discrete system that is able to approximate the nonlinear relation existing internally among the known entries. It is discrete because by feeding a new entry that falls in the range of the known domain to this neural system, it returns the closest approximation to the most relevant known entries in the domain. It is dynamic because when some new entries are added to the training data, this neural system is able to retrain the model in order to absorb these new entries into its coverage. This either enlarges the known applicable domain if the additional training data are beyond the previous range, or refines the local approximating mechanism if the additional training data are within the known domain. In conclusion, the three-layered MLP is able to predict the supplier’s flexibility with respect to its five attributes with a higher accuracy, compared with the multivariate regression analysis. However, multivariate regression is still useful for manufacturers with lesser number of suppliers and can give an understanding on the importance of individual attributes for the flexibility assessment through its analytical expression. This cannot be achieved by neural networks working like a black-box translator.

References 1. Wadhwa, S., Saxena, A., Chan, F.T.S.: Framework for flexibility in dynamic supply chain management. International Journal of Production Research 46, 1373–1404 (2008) 2. Min, S., Roath, A.S., Daugherty, P.J., Genchev, S.E., Chen, H., Arndt, A.D., Richey, R.G.: Supply Chain Collaboration: what’s happening. The International Journal of Logistics Management 16, 237–256 (2005) 3. Xu, N.: Flexible supply policy with options and capacity constraints. Operations Research Letters 34, 508–516 (2006) 4. Croom, C., Romano, P., Giannakis, M.: Supply chain management: An analytical framework for critical review. European Journal of Purchasing and Supply Management 6, 67– 83 (2000) 5. Swafford, P.M., Ghosh, S., Murthy, N.: The antecedents of supply chain agility of a firm: scale development and model testing. Journal of Operations Management 24, 170–188 (2006) 6. Duclos, L.K., Vokurka, R.J., Lummus, R.R.: A concept model of supply chain flexibility. Industrial Management & Data Systems 103, 446–456 (2003) 7. Aprile, D., Garavelli, A.C., Giannoccaro, I.: Operations planning and flexibility in a supply chain. Production Planning & Control 16, 21–31 (2005)

852

A.S. Jeeva and W.W. Guo

8. O’Marah, K., Hofman, D.: Top 25 supply chains. Supply Chain Management Review, 12– 19 (October 2009) 9. Khan, A.K., Bakkappa, B., Metri, B.A., Sahay, B.S.: Impact of agile supply chains’ delivery practices on firms’ performance: cluster analysis and validation. Supply Chain Management: An International Journal 14, 41–48 (2009) 10. Croom, C., Romano, P., Giannakis, M.: Supply chain management: an analytical framework for critical review. European Journal of Purchasing and Supply Management 6, 67– 83 (2000) 11. Agarwal, A.: Flexibility in information technology systems and organizational competitiveness. Global Journal of Flexible Systems Management 5, 35–38 (2005) 12. Palsson, H., Johansson, O.: Supply chain integration obtained through uniquely labeled goods: a survey of Swedish manufacturing industries. International Journal of Physical Distribution & Logistics Management 39, 28–46 (2009) 13. Ball, M.O., Chen, C.Y., Zhao, Z.Y.: Material compatibility constraints for make-to-order production planning. Operations Research Letters 31, 420–428 (2003) 14. Zhang, Q., Vonderembse, M.A., Lim, J.S.: Logistics flexibility and its impact on customer satisfaction. The International Journal of Logistics Management 16, 71–95 (2005) 15. Naim, M.M., Potter, A.T., Mason, R.J., Bateman, N.: The role of transport flexibility in logistics provision. The International Journal of Logistics Management 17, 297–311 (2006) 16. Fantazy, K.A., Kumar, V., Kumar, U.: An empirical study of the relationships among strategy, flexibility and performance in the supply chain context. Supply Chain Management: An International Journal 14, 177–188 (2009) 17. Churchill, G.A.: A paradigm for developing better measures of marketing constructs. Journal of Marketing Research 16, 64–73 (1979) 18. Mendenhall, W., Sincich, T.: Statistics for engineering and the sciences. Pearson Prentice Hall, Upper Saddle River (2007) 19. Yang, B., Li, L.X., Xu, J.: An early warning system for load risk assessment using artificial neural networks. Knowledge Based Systems 14, 303–306 (2001) 20. Kryzanowski, L., Galler, M., Wright, D.W.: Using artificial neural networks to pick stocks. Financial Analysts Journal 49, 21 (1993) 21. Hornik, K., Stinchcomb, M., White, H.: Multilayer feed forward networks are universal approximators. Neural Networks 2, 359–366 (1989) 22. Zhang, G.P., Keil, M., Rai, A., Mann, J.: Predicting information technology project escalation: A neural network approach. European Journal of Operational Research 146, 115–129 (2003) 23. Yan, H., Jiang, Y., Zheng, J., Peng, C., Li, Q.: A multilayer perceptron based medical decision support system for heart disease diagnosis. Expert Systems with Applications 30, 272–281 (2006) 24. Guo, W.W., Li, M.M., Whymark, G., Li, Z.X.: Mutual complement between statistical and neural network approaches for rock magnetism data analysis. Expert Systems with Applications 36, 9678–9682 (2009) 25. Guo, W.W.: Incorporating statistical and neural network approaches for student course satisfaction analysis and prediction. Expert Systems with Applications 37, 3358–3365 (2010) 26. Demuth, H., Beale, M., Hagan, M.: Neural network toolbox 5. The MathWorks, Natick (2007) 27. Marquardt, D.: An algorithm for least-squares estimation of nonlinear parameters. SIAM Journal on Applied Mathematics 11, 431–441 (1963) 28. Hagan, M.T., Demuth, H.B., Beale, M.H.: Neural network design. PWS Publishing, Boston (1996)

An Intelligent Mobile Location-Aware Book Recommendation System with Map-Based Guidance That Enhances Problem-Based Learning in Libraries Chin-Ming Chen1 and Yu-Chieh Yang2 1

Graduate Institute of Library, Information and Archival Studies, National Chengchi University, Taipei Taiwan, R.O.C. 2 Graduate Institute of Industrial Education, National Taiwan Normal University, Taipei Taiwan, R.O.C. [email protected], [email protected]

Abstract. By integrating the PBL model with book resources in libraries, this study identifies the advantages of libraries in supporting e-Learning. This study supports problem-based learning (PBL) in library using a novel intelligent mobile location-aware book recommendation system (IMLBRS) with map-based guidance. Experimental results reveal the learning performance during PBL supported by the proposed IMLBRS for book searches is superior to Online Public Access Catalogue (OPAC) system. Furthermore, this study confirm that the proposed system facilitates better learning performance for learners with the field-dependent learning style than for learners with the field-independent learning style. Keywords: Learning supported by a library, Location-aware book recommendation, Mobile learning, Problem-based learning, Learning style.

1 Introduction Mobile devices provide an opportunity for developing new library services, such as in-library exploration, social engagement, and outreach, to traditionally underserved populations [1]. Many libraries, such as MobiLIB, a mobile library at North Carolina State University (NCSU) (www.lib.ncsu.edu/m/about.html), are reformatting their presence on mobile devices to provide access to library services from mobile devices. MobiLIB provides users with quick and convenient access to many NCSU library services from web-enabled mobile devices. Lippincott [2] provided examples of innovative ways via which a limited number of academic libraries are designing services around mobile technologies and mobile users. A study by Cambridge University and the Open University [3] found that students are more interested in receiving information about library hours, location maps, contact info, and access to a complete catalog than they are in reading e-books and e-journals on mobile devices. As a result, libraries providing particular services in a mobile-friendly way is becoming a widespread trend. Nevertheless, despite the rapid adoption of mobile devices and Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 853–860. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

854

C.-M. Chen and Y.-C. Yang

importance of learning supported by book resources in libraries, few learning approaches have integrated mobile learning technologies and library environments as an innovative learning model emphasizing the key roles of libraries in learning. The situational learning approach [4] argues that context is an important factor affecting learning performance and can enhance learning interest and efficiency. Moreover, meaningful knowledge is constructed only when learning processes are integrated with cultural and life contexts [5]. Dey [6] identified four main contexts—identity, time, activity, and location—for building context-aware applications. E-learning supported in a location-based context opens up new opportunities and challenges, such that learning activities become mobile and location-aware [7]. Many location-aware mobile services have been proposed for public spaces such as museums and libraries [8]. In museums, location-based information is typically used to aid navigation and provide users with information related to nearby exhibits. In libraries, user needs are different. Library users are typically searching for a particular books associated with a given topic. The SmartLibrary at the University of Oulu, Finland, allows library users to browse collections through their mobile devices (personal digital assistant, PDA and high-end cell phones) based on their location within the library [9]. Problem-based learning (PBL) is a learning model in which students frequently solve problems associated with learning using online resources [10]. Libraries, which contain authoritative, organized and effective resources, have potential benefits that support PBL. Therefore, this study presents a novel intelligent mobile location-aware book recommendation system (IMLBRS) with map-based guidance that facilitates book searches on handheld mobile devices such as Eee PCs and Tablet PCs. The proposed book recommendation system integrates PBL and real library environments with rich book resources to enhance PBL. A real teaching experiment provides evidence demonstrating that the proposed system promotes PBL performance in libraries, and particularly provides better learning effect to learners with the field-dependent learning style than learners with the field-independent learning style.

2 System Design 2.1 System Architecture Figure 1 shows the architecture of the proposed IMLBRS with map-based guidance. The proposed system is composed of three subsystems—the library book recommendation system, PBL system, and Chinese word-segmentation system. The left side of Fig. 1 shows the components of the library book recommendation system, which includes a wireless positioning agent, location-based book recommendation agent, map presentation agent, and library book database. The wireless positioning agent determines a learner’s location and assists the location-based book recommendation agent in recommending appropriate books based on individual learner location. The location-based book recommendation agent recommends books close to an individual’s location and associated with user search queries to individual learners to support PBL in library environments. The map presentation agent immediately displays individual learner locations on an electronic map to guide learners in obtaining books recommended by the location-based book recommendation agent. The library book database comprises book

An Intelligent Mobile Location-Aware Book Recommendation System

855

information, including book title, authors, publisher, publication date, and location, to support the location-based book recommendation agent, which recommends appropriate books to individual learners based on search queries and learner locations. The right side of Fig. 1 shows the components of the PBL system, which has a PBL agent, book search and display interface, login account database, learning task database, and user portfolio database. The problem-based learning agent provides PBL learning procedures with the “cognition-action-reflection” (i.e., knowing, doing, and thinking) mental process to guide individual learners performing PBL in a library environment. The book search and display interface is a user-friendly environment that conveniently searches for books that individual learners need during PBL and displays book recommendations. The login account database records learner account information. The learning task database stores learning scaffolds that, in a timely manner, offer learning support for solving a given problem in PBL. To construct this learning task database, the PBL agent provides a user-friendly interface that assists course instructors in designing a learning scaffold for solving a given problem. The user portfolio database records personal information, including individual learning processes, and the PBL outcomes of individual learners. Moreover, the Chinese word-segmentation system segments Chinese user search queries and book titles into separate Chinese words for similarity computations when recommending location-aware books for supporting PBL in libraries.

Fig. 1. The system architecture of the proposed IMLBRS with map-based guidance for supporting problem-based learning in library

2.2 The Employed Average-KNN Classifier for Identifying Learner Location Detecting the location of a learner is important in the proposed IMLBRS with map-based guidance. Therefore, this study proposes the average k-NN classifier, which

856

C.-M. Chen and Y.-C. Yang

considers the distance of each of k nearest neighbors with the test pattern, to reason user location based on the gathered data pairs comprising the signal strengths of the installed access points (APs) in a library and corresponding location label. In contrast to the original k-NN classifier, the proposed average k-NN classifier addresses the average distances between the test pattern with the k nearest neighbors belonging to different classes, rather than classifying by a majority vote of the k nearest neighbors with the test pattern. Hence, the test pattern is recognized as the class with the shortest average distance while applying the proposed average k-NN classifier. 2.3 Recommending Books Associated with Learner Search Query and Location The word segmentation is a necessary step in processing Chinese texts, such as machine translation, Chinese text mining and information retrieval. To perform Chinese word segmentation for learner search queries and book titles, our previous study proposed a Chinese word segmentation system ECScanner with automatically expanding the lexicon of the Chinese word segmentation based on new word extraction from Google news in order to speed up the word identification capability [11]. The study adopts ECScanner to perform Chinese word segmentation process for the proposed system. Moreover, the study employed cosine similarity measure to assess the similarity between learner search query and book title for recommending books which is close to learner location and is relating to PBL requirement. The proposed system displays this book information for learner based on the ranking of proposed similarity degree between a user search query and book title to solve the assigned PBL problem under considering the learner location. 2.4 The Implemented Intelligent Mobile Location-Aware Book Recommendation System with Map-Based Guidance for Supporting PBL The proposed IMLBRS with map-based guidance for supporting PBL is composed of a wireless positioning agent, map presentation agent, location-based book recommendation agent, and library book database. The book recommendation services provided by the location-based book recommendation agent for supporting PBL is based on the sensed location of an individual learner via the wireless positioning agent in the Muzha Vocational High School library. The wireless positioning agent can immediately identify the location of individual learners as two-dimensional coordinates when the wireless card embedded in a learner’s mobile device receives a wireless signal from access points (APs) installed in the library. The wireless positioning agent then sends the computing coordinates to the map presentation agent, which displays the position of individual learners in the library on an electronic map. This functionality is very helpful in that learners do not get lost in large libraries and learners are rapidly guided to the books recommended by the location-based book recommendation agent. Moreover, this functionality also simultaneously shortens book search time and enhances book search convenience. Figure 2 shows the user interface of the implemented library book recommendation system based on the adopted WLAN positioning methodology for supporting PBL. Via this user friendly interface, learners can input search queries to search for books associated with PBL missions assigned by

An Intelligent Mobile Location-Aware Book Recommendation System

857

course instructors. The implemented library book recommendation system can correctly determine whether books are on the bookshelf, which books are closest to a learner, and which books are closely related to learner search queries. The proposed system then displays this book information for learner based on a similarity ranking to solve the assigned PBL problem.

Fig. 2. The user interface of the implemented IMLBRS with map-based guidance based on the adopted WLAN positioning methodology for supporting problem-based learning (The symbol “z”denotes the detected learner location)

3 Experiment Design This study randomly invited students in two classes at Muzha Vocational High School to participate in the PBL activity in a history course. The Muzha Vocational High School library is the learning field. Each class has 40 students. One class was randomly assigned as the experimental group and the other class as the control group. The experimental group was assigned a four-week PBL activity in a history course using the PBL system supported by the proposed IMLBRS with map-based guidance. The experimental group and the control group were assigned the same PBL activity. The control group uses the PBL system supported by the OPAC. Moreover, this study designed a cooperative PBL activity for the PBL mission assigned by the instructor. Five students were randomly distributed as a cooperative learning group for the PBL mission assigned by instructor, regardless of whether they were in the experimental or control group. In other words, each of the 16 learning groups had 5 students. To evaluate learning performance, both groups completed a pre-test and post-test, and test results were compared.

4 Experimental Results 4.1 WLAN Positioning Experiment To collect the signal strength features of the installed access points for the employed average-KNN classifier is the most important work affecting positioning learner

858

C.-M. Chen and Y.-C. Yang

location in the proposed system. However, for some reasons such as multi-path, and refraction, the signal therefore generates irregular fluctuation and becomes as noisy data. Therefore, it is essential to apply noisy data filtering mechanism to eliminate noisy training data. The noisy data filtering mechanism employed in this study is based on the concepts of mean and bias in the statistics analysis. Once the better training data is generated after filtering out noisy data, the inference effect of the average-KNN classifier can be obviously promoted. The experimental results demonstrate that the proposed average-KNN classifier has the accuracy rate of positioning learner location is up to 76% within the area of 2 meters square. It is enough to aid the location detection of learner for the implemented mobile location-aware book information recommendation system with map-based guidance. 4.2 Learning Performance Assessment 4.2.1 Comparison on Learners’ Learning Performance of Both Groups Learners in the experimental and control groups differed only in that they used different tools for searching books in the library during the “action” stage. This study viewed the average score in the “cognition” stage as the pre-test score. The average scores in the “action” and “reflection” stages were the post-test scores. Table 1 shows the experimental and control groups’ pre-test and post-test scores. To assess the learning performance for both groups, one-way analysis of covariance (ANCOVA) was applied in this study. The pre-test score was a covariance, the different book searching tools for PBL were independent variables, and the post-test score was a dependent variable in comparing student learning performance before and after the experiment using one-way ANCOVA. In one-way ANCOVA, the first step is to analyze the homogeneity of regression coefficients. The F test result (F=0.223, sig of F=0.645) does not reach the significant level, thus it means the regression slope of two groups is equivalent. This result confirms the assumption of homogeneity of coefficients, so this study further preceded the analysis of covariance (ANCOVA). The ANCOVA result reaches the significant level (F=5.911, sig of F=0.030) after adjusting the dependent effect with respect to the covariance (pre-test). This result indicates that the post-test of two groups has significantly different. Moreover, this study found that the post-test score of the experimental group is higher than that of the control group. Therefore, this experimental result proves that the proposed IMLBRS integration and support with library book improved PBL performance. Table 1. The experimental and control groups’ pre-test and post-test scores

Number of learning groups Experimental Group Control Group

Mean Pre-Test Post-Test

Std. Deviation Pre-Test Post-Test

8

80.875

82.362

.501

3.317

8

80.958

78.953

.843

2.835

An Intelligent Mobile Location-Aware Book Recommendation System

859

4.2.2 Analysis on Learning Performance of Learners with Different Learning Styles in the Action Learning Stage Next, this study compares learners in both two groups who have different learning styles to indentify how learning style influences learning performance in the action learning stage. This study applied the Group Embedded Figures Test (GEFT) [12] to classify learners into two learning style groups—field-independent and field-dependent groups. Analytical results show that of the 40 students in the experimental group, 21 learners had the field-independent learning style and 19 learners had the field-dependent learning style. Of the 40 students in the control group, 28 had the field-independent learning style and 12 had the field-dependent learning style. Table 2 shows the average score of learners with different learning style on the action learning stage. Table 2. The average score of learners with different learning styles on the action learning stage (FI: Field Independence, FD: Field Dependence)

Experimental Group Control Group

Number of Learners FI 21 FD 19 FI 28 FD 12

Mean

Std. Deviation

82.200 87.160 80.230 79.000

6.887 7.507 6.801 7.627

First, the independent sample t test was applied to examine whether learners with the field-dependent and field-independent learning styles in the experimental group during the action learning stage had different learning performances. The result of independent sample t test reaches the significant level (t=2.151, sig. of t=0.038). This means that learners with different learning styles in the experimental group have significantly different learning performances in the action learning stage. Learners with the field-dependent learning style had better learning performance than learners with the field-independent learning style because the mean score of the field-dependent style learners is higher than the mean score of the field-independent style learners. Next, the independent sample t test was further applied to examine whether learners with the field-dependent learning style in the experimental and control groups during the action learning stage had different learning performances. The result of independent sample t test also reaches the significant level (t=2.929, sig. of t=0.007). This means that learners with the field-dependent learning style in the experimental group, who were assisted by the proposed IMLBRS during PBL, had significantly better learning performance than learners with the field-dependent learning style in the control group, who were assisted by the traditional OPAC in the library during the action learning stage.

5 Conclusion This study investigated the difference in using the proposed IMLBRS to support PBL in a library and OPAC, the existing library book search system, and investigated whether the IMLBRS can effectively support learner PBL in a library. Experimental results reveal that PBL performance of the experimental group supported by the proposed IMLBRS was superior to that of the control group supported the OPAC.

860

C.-M. Chen and Y.-C. Yang

Moreover, by the independent sample t test, this study confirmed that learning performance of the experimental group with the field-dependent learning style was better than that of learners with the field-independent learning style during the “design method” task in the action learning stage. The learning performance of field-dependent learners in the experimental group was much better than that of field-dependent learners in the control group. Thus, the proposed IMLBRS has a positive effect on learning performance for field-dependent learners.

Acknowledgment The authors would like to thank the National Science Council of the Republic of China, Taiwan for financially supporting this research under Contract No. NSC97-2628-S-004-001-MY3.

References 1. Hahn, J.: Mobile learning for the twenty-first century librarian. Reference Services Review 36(3), 272–288 (2008) 2. Lippincott, J.K.: Mobile technologies, mobile users: implications for academic libraries. ARL Bimonthly Report, no. 261 (2009), http://www.arl.org/bm~doc/arl-br-261-mobile.pdf 3. Mills, K.: M-Libraries: information use on the move. Technical report, Cambridge University and the Open University (2009), http://arcadiaproject.lib.cam.ac.uk/docs/ M-Libraries_report.pdf 4. Hornby, A.S.: The situational approach in language teaching. A series of three articles in English Language Teaching 4, 98–104, 121–8, 150–6 (1950) 5. Chen, C.M., Li, Y.L.: Personalized context-aware ubiquitous learning system for supporting effective English vocabulary learning. Interactive Learning Environments (2009) (Article in press) 6. Dey, A.K.: Providing architectural support for building context-aware applications. PhD Thesis, Georgia Institute of Technology, Georgia Institute of Technolgy, USA (2000) 7. Zhou, R., Rechert, K.: Personalization for location-based e-learning. In: The Second International Conference on Next Generation Mobile Applications, Services, and Technologies, pp. 247–253 (2008) 8. Hall, T., Bannonw, L.: Designing ubiquitous computing to enhance children’s learning in museums. Journal of Computer Assisted Learning 22, 231–243 (2006) 9. Aittola, M., Ryhänen, T., Ojala, T.: SmartLibrary – location-aware mobile library service. In: Proceedings of the Fifth International Symposium on Human Computer Interaction with Mobile Devices and Services, pp. 411–416 (2003) 10. Chen, C.M., Chen, C.C.: Problem-based Learning Supported by Digital Archives-Case Study of Taiwan Libraries’ History Digital Library. The Electronic Library 28(1), 1–24 (2010) 11. Hong, C.M., Chen, C.M., Chiu, C.Y.: Automatic extraction of new words based on Google news corpora for supporting lexicon-based Chinese word segmentation systems. Expert Systems with Applications 36(2), 3641–3651 (2009) 12. Witkin, H.A., Goodenough, D.R.: Field-dependent and field-independent cognitive styles and their educational implications. Review of Educational Research 47(1), 1–64 (1977)

Applying Least Squares Support Vector Regression with Genetic Algorithms for Radio-Wave Path-Loss Prediction in Suburban Environment Kuo-Ping Lin1, Kuo-Chen Hung2, Jen-Chang Lin3, Chi-Kai Wang4, and Ping-Feng Pai5 1

Department of Information Management, Lunghwa University of Science and Technology, Taoyuan 333, Taiwan [email protected] 2 Department of Logistics Management, National Defense University, Beitou Taipei 112, Taiwan [email protected] 3 Department of Mechanical Engineering, Minghsin University of Science and Technology, Hsinchu 304, Taiwan [email protected] 4 Chung-Shan Institute Science and Technology, P.O. Box 90008-15-18 Lung-Tan, Taoyuan 325, Taiwan [email protected] 5 Department of Information Management, National Chi Nan University, University Rd., Puli, Nantou 545, Taiwan [email protected]

Abstract. This paper presents least squares support vector regression with genetic algorithms (LS-SVRGA) models for the prediction of radio-wave pathloss in suburban environment. The least squares support vector regression (LSSVR) model is a novel forecasting approach and has been successfully used to solve time series problems. However, the application of LS-SVR models in a radio-wave path-loss forecasting has not been widely investigated. This study aims at developing a LS-SVRGA model to forecast radio-wave path-loss data. Furthermore, in the LS-SVRGA model genetic algorithms is applied in order to select two parameters of LS-SVR models. In this study, four forecasting models, Egli, Walfisch and Bertoni (W&B), generalized regression neural networks (GRNN), and support vector regression with genetic algorithms (SVRGA) models are employed for forecasting the same data sets. Empirical results indicate that the LS-SVRGA outperforms others models in terms of forecasting accuracy. Thus, the LS-SVRGA model is an effective method for radio-wave path-loss forecasting in suburban environment. Keywords: Least squares support vector regression, genetic algorithms, radiowave path-loss. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 861–868. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

862

K.-P. Lin et al.

1 Introduction An Efficient path-loss prediction model is very important for both modeling of up-todate wireless communication system and RF spectrum planning and managing. A lot of global and local parameters (relief, objects at the propagation path, climate zones, foliage characteristics, reflecting index in atmosphere, propagation along many paths, etc.) affect path-loss prediction model. Previous models mainly focus on experimental models. Traditional path-loss prediction model mainly used experimental models such as Egli [1], W&B [13] models, which briefly be introduced in following sections, [7], and [3]. The problem of these models is that expressions are based on the qualitative propagation environments such as urban, suburban and open areas. Recent years the artificial neural networks (ANNs) has been adopted to forecast path-loss data. Milovanović et al. [5] hybrid Okumura-Hata with neural networks to estimate pathloss measured result. However, the ANNs model has not been widely investigated in path-loss prediction. In this study, a LS-SVRGA model is presented to forecast radiowave path-loss data in suburban environment. The remainder of this paper is organized as follows. Introduction of the theory of LS-SVRGA is given in Section 2. Numerical examples of radio-wave path-loss predictions and empirical results are presented in Section 3. Some concluding remarks are provided in Section 4. 1.1 Egli Model Egli prediction model is a empirical model which has been proposed by Egli [1]. The Egli model is a simplistic model to approach radio-wave path-loss of irregular topography. Based on real data the path-loss approaching can be formulated as following:

L = 117 + 40 log d + 20log f − 20 log(hb hm )

(1)

where d (Km) is total distance in meter, f (MHz) is frequency of transmit, hb and hm are the transmitting and receiving antenna heights, respectively. Thus the free spaces path-loss model is better considered. The free spaces model ( L f s ), which includes total distance in meter and is independent of frequency, is formulated as following

L f s = 32.27 + 20 log d + 20 log f

(2)

The Egli model considers the free spaces path-loss, hence the Egli prediction model ( LE ) can be formulated as following: LE = L − L f s = 84.73 + 20 log(d ) − 20 log(hb hm )

(3)

1.2 Walfisch-Bertoni Model

Walfisch and Bertoni [13] proposed the half empirical prediction model. The model considers the impact of rooftops and building height by using diffraction to predict average signal strength at street level. The method describes urban propagation loss as

Applying LS-SVRGA for Radio-Wave Path-Loss Prediction in Suburban Environment

863

a sum of three terms: (1) free space losses (2) rooftop-to-street losses and (3) multiple diffraction losses. Hence, the Walfisch and Bertoni prediction model can be formulated as following: LWB = L0 + Lmsd + Lrts

(4)

where L0 is free spaces path-loss, Lmsd means multiple diffraction losses and Lrts means rooftop-to-street losses. Furthermore, the Walfisch and Bertoni prediction model can be derived as following: LWB = 89.55 + 21log f + 38log d − 38log H + (5log((b / 2) 2 + ( hb − hm ) 2 )

(5)

−9 log b + 20log(tan −1 (2(hb − hm ) / b))) − 18log(1 − ( d 2 /17 H ))

where H is height of transmit, b is distance between buildings. Fig. 1 use geometric displays the transmitting diagram of Walfisch and Bertoni model.

H

hm

hb

b d Fig. 1. Illustration of transmitting of Walfisch and Bertoni model

2 Least Square Support Vector Regression with Genetic Algorithm A successful method for regression problem is the least squares support vector regression ([6] and [9]), which attempts to minimize the sum square errors (SSEs) of training data set while simultaneously minimizing the margin error. The LS-SVR approach is to approximate an unknown function by a training data set {(xi, yi), i=1, …, N}. The regression function can be formulated as follows: y = f ( x) = wT ϕ ( xi ) + b

(6)

where ϕ(xi) denotes the feature of the inputs, and w and b indicate coefficients. LSSVR introduces a least squares version to SVR by formulating the regression problem as Min J1 ( w, b, e) =

1 1 N 2 w + C ∑ ei2 2 2 i =1

subjective to yi =wT ϕ ( xi ) + b + ei , i = 1, 2,

,N

(7)

864

K.-P. Lin et al.

Introduce the Lagrangian as N

L1 ( w, b, e, α ) = J1 ( w, b, e) + ∑ α i ( yi − wT ϕ ( xi ) − b − ei ).

(8)

i =1

where α is the Lagrangian multiplier vector. The conditions for optimality are N ∂L1 = 0 ⇒ w = ∑ α iϕ ( xi ), ∂w i =1 N ∂L1 = 0 ⇒ ∑ α i = 0, ∂b i =1 ∂L1 1 = 0 ⇒ ei = α i , i = 1,… , N , C ∂ei

(9)

∂L1 = 0 ⇒ yi = wT ϕ ( xi ) + b + ei , i = 1,… , N . ∂α i

Herein, k(xi, xj) denotes a Kernel function whose value equals the inner product of two vectors, xi and xj , in the feature space ϕ(xi) and ϕ (xj), meaning that k(xi, xj)=ϕ T(xi) ϕ (xj). Any function that satisfies Mercer’s condition [4] can act as the Kernel function. This work uses the Gaussian function. Hence, we can denote the K=(kij)N×N, kij= k(xi, xj), and V=diag{1/C,1/C,…, 1/C}, LS-SVR can be described as ⎡ A 1 ⎤ ⎡α ⎤ ⎡ y ⎤ ⎢1T 0 ⎥ ⎢ b ⎥ = ⎢ 0 ⎥ , ⎣ ⎦⎣ ⎦ ⎣ ⎦

(10)

where A=K+V. Hence, the regression model (see Eq. (6)) is found by solving Eq. (10). The resulting LS-SVR becomes then N

f ( x) = ∑ α i k ( x, xi ) + b.

(11)

i =1

Note that in the case of Gaussian kernels, one has only two additional tuning parameters (C, σ), which is less than for standard SVR. The experiment uses genetic algorithms (GA) [2] to search optimal parameter of LS-SVR. In this study, binary encoding is specified in GA search. The size of the population is set fifty. The roulette wheel selection principle is used to select chromosomes for reproduction. In the crossover operation, chromosomes are paired randomly. The single-point-crossover principle is employed in the GA search. In this study, the probabilities of crossover and mutation are set to 0.5 and 0.1, respectively. By the GA search the optimal parameters C and σ can be obtained. Fig. 2 shows the flowchart of the proposed LS-SVRGA model. The LS-SVR model was then employed to forecast wave-radio path-loss data; and the GA was performed to determine parameters of LS-SVR models. To minimize forecasting error of the SVR model, the negative values of mean absolute percentage error (MAPE, see Eq. (12)) were treated as the objective values of the GA; thus, each iteration generates a smaller MAPE value. The parameter search procedure is conducted until the stop criterion is reached. The two parameters resulting in the smallest validation error are

Applying LS-SVRGA for Radio-Wave Path-Loss Prediction in Suburban Environment

865

then used to develop a appropriate LS-SVR model. Finalized forecasting vales are obtained by LS-SVR. Finally, after comparison with the testing data, the testing errors are obtained. Genetic algorithms

Path-loss data

Coding C and σ in parameters

Training data

Validation data

Testing data

Randomize initial parameters Obtain validation error by tentative Training LS-SVR model

LS-SVR model

Evaluating fitness

Satisfy stopping

Use LS-SVR model to forecast

Yes

data

No Selection

Calculate test error

Crossover and mutation

Form new parameters

Fig. 2. A flowchart of the LS-SVRGA

3 Numerical Examples and Experimental Results In this research numerical example, which is actual wave-radio path-loss data in suburban environment, is examined. Table1 shows the actual wave-radio path-loss data with measurement distance between the 0.5 and 5 Km. Moreover, the experimental parameters value should be known, which frequency of transmit (f) is 937 MHz, the transmitting antenna height (hb) is 30 m, and receiving antenna height (hm) is 1.5 m, respectively. In addition, MAPE and the root mean square error (RMSE) are used to measure the forecasting accuracy of three models. Eqs. (12) and (13) show the expression of MAPE and RMSE respectively: MAPE(%) = −

⎧1 RMSE = ⎨ ⎩M

100 M gl − zl ∑ M l =1 gl M

∑ ( g l − zl ) l =1

(12)

0.5

2

⎫ ⎬ ⎭

(13)

where M is the number of forecasting periods; gl is the actual value at period l; and zl is the forecasting value at period l. Table 1 also shows the estimated results of Egli and W&B models, which have been introduced in previous sections, for comparison of performance. Moreover, this research compares popular ANN models, which are GRNN [8] and SVRGA models,

866

K.-P. Lin et al.

with LS-SVRGA. The comparison models used in this research are briefly introduced here. In this study the traditional SVR ([10], [11] and [12]) model also adopts genetic algorithm (SVRGA) to improve the forecasting accuracy of radio-wave path-loss data. The procedures of GA are as LS-SVRGA which can see section 2. For GRNN, SVRGA, and LS-SVRGA models, experimental data are divided into training, validation and testing data sets. Table 1 depicts the actual path-loss data employed in this work for ANN models. The training data set was used to determine the forecasting model; the validation data set was for the purpose of preventing overfitting of forecasting models; and the testing data set is employed to investigate the performance of different forecasting models. For both examples, the training data set and the validation set are used to design LS-SVRGA models. For comparing the forecasting accuracy, the same testing data set is examined for ANN forecasting models. In the research we test the path-loss data between 2.5 and 6 Km. Table 2 show the forecasting performances and preferred parameters of three ANN models for path-loss data. LS-SVR models outperformed the GRNN models and SVRGA models in terms of forecasting accuracy. In Table 2 we also list the optimal parameters of SVRGA and LS-SVRGA models. Fig. 3 (a) and (b) make point-to–point comparisons of actual values and predicted values of path-loss data. As shown in Fig. 3 (a), the traditional Egli and W&B models cannot efficiently capture the trend of data. In Fig. 3 (b) the ANN models can find that efficiently captures the trend of data, and the LSSVRGA is able to follow the data trend and can make more accurate forecast than others models. Table 3 shows and ranks the performances of five models. Again, the LS-SVRGA the outperforms others models in terms of forecasting accuracy. Table 1. Comparison of the forecasting results from traditional path-loss prediction model Distance(×10 Km) 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.55 0.60

Actual path-loss 89 85 86 93 99 100 113 118 120 125 130 132

Egli model 89.835 96.966 100.83 103.49 105.53 107.18 108.56 109.76 110.81 111.74 112.59 113.36

W&B model 74.06 87.611 94.953 100.02 103.9 107.04 109.68 111.96 113.97 115.76 117.39 118.87

Table 2. Comparison of the forecasting results and parameters from ANN model Distance(×10 Km) 0.25 0.30 0.35 0.40 0.45 0.50 0.55 0.60

GRNN 92.19 98.29 99.86 111.51 117.38 119.75 124.42 129.41

SVRGA 92.99 98.96 99.80 112.94 116.48 119.67 121.77 126.51

C/ε/σ 99.8/0.137/0.99 99.7/0.037/1.099 99.8/0.18/0.18 86.93/0.41/0.63 96.32/0.028/0.63 97.67/0.32/0.73 99.51/0.90/0.69 99.90/ 0.04/0.88

LS-SVRGA 92.98 101.18 103.19 115.82 123.27 123.81 127.02 132.64

C/σ 83.93/0.88 99.03/0.81 76.29/0.44 96.80/ 0.29 61.38/0.29 81.70/6.84 67.09/0.19 68.16/ 0.39

Applying LS-SVRGA for Radio-Wave Path-Loss Prediction in Suburban Environment Actual values

Egli

W&B

GRNN

SVRGA

LS-SVRGA Testing data

Path-loss (dB)

Path-loss (dB)

Training data

867

Distance(u10 Km) (b) ANN models

Distance(u10 Km) (a) Traditional models

Fig. 3. Illustration of actual values and forecasting values of five models Table 3. Forecasting performances of five models Forecasting models MAPE(%) RMSE Rank

Egli model 9.5 11.42 (5)

W&B model 7.4 8.82 (4)

GRNN 4.7 6.48 (2)

SVRGA 5.0 6.85 (3)

LS-SVRGA 3.0 4.47 (1)

Finally, the Wilcoxon signed rank test was implemented to test the forecasting result for ANN models. The test results, presented in Table 4, show significant differences among the LS-SVRGA, the GRNN and SVRGA models, indicating the forecasting result of LS-SVRGA is better than those of the existing neural network approaches based on MAPE and RMSE. Table 4. Wilcoxon signed rank test for the forecasting results from ANNs model

LS-SVRGA SVRGA Cell: standardize value(p value). *significant different at α= 0.05.

GRNN -2.52(0.012) * -0.7(0.484)

SVRGA -2.38(0.017) *

4 Conclusions This study proposed a hybrid LS-SVRGA model for forecasting path-loss, with experimental results being valid and satisfied. The superior performance of the LSSVRGA model can be ascribed to two causes. First, the LS-SVRGA can efficiently capital trend of nonlinear data and estimate precisely. Second, based on GA searching, the LS-SVRGA model can provide optimal parameters for radio-wave path-loss predicting. For future work, forecasting other environments of path-loss data by a SVR-related model is a challenging issue for study. Another future research

868

K.-P. Lin et al.

direction could consider using data preprocessing techniques for improvement in the forecasting accuracy of the LS-SVR model in path-loss data. Acknowledgments. This research was conducted with the supports of National Science Council of Taiwan (NSC 98-2410-H-262 -002) and the supports of Ministry of Economic Affairs of Taiwan (98-EC-17-A-05-S2-0012).

References 1. Egli, J.J.: Radio Propagation above 40 Mc over Irregular Terrain. In: IRE, pp. 1383–1391 (1957) 2. Holland, J.H.: Adaptation in Natural and Artificial System. University of Michigan Press, Ann Arbor (1975) 3. Joshi, G.G., Dietrich Jr., C.B., Anderson, C.R., Newhall, W.G., Davis, W.A., Isaacs, J., Barnett, G.: Near-ground Channel Measurements over Line-of-sight and Forested Paths. IEE Proc.-Microw. Antennas Propag. 152, 589–596 (2005) 4. Mercer, J.: Function of Positive and Negative Type and Their Connection with the Theory of Integral Equations. Philosophical Transactions of the Royal Society A 209, 415–446 (1909) 5. Milovanović, B., Stanković, Z., Milijić, M., Sarevska, M.: Near-Earth Propagation Loss Prediction in Open Rural Environment using Hybrid Empirical Neural Model. In: TELSIKS 2007, pp. 423–426. IEEE Press, Serbia (2007) 6. Peng, X., Wang, Y.: A Normal Least Squares Support Vector Machine (NLS-SVM) and Its Learning Algorithm. Neurocomputing 72, 3734–3741 (2009) 7. Rama Rao, T., Vijaya Bhaskara Rao, S., Prasad, M.V.S.N., Sain, M., Iqbal, A., Lakshmi, D.R.: Mobile Radio Propagation Path Loss Studies at VHF/UHF Bands in Southern India. IEEE Transactions on Broadcasting 46, 158–164 (2000) 8. Specht, D.F.: A General Regression Neural Network. IEEE Transactions on Neural Networks 2, 568–576 (1991) 9. Suykens, J.A.K., Van Gestel, T., De Brabanter, J., De Moor, B., Vandewalle, J.: Least Squares Support Vector Machines. World Scientific, Singapore (2002) 10. Vapnic, V.: The Nature of Statistical Learning Theory. Springer, New York (1995) 11. Vapnik, V., Golowich, S., Smola, A.: Support Vector Machine for Function Approximation Regression Estimation, and Signal Processing. In: Advances in Neural Information Processing Systems, vol. 9, pp. 281–287 (1996) 12. Vapnic, V.: Statistical Learning Theory. Wiley, New York (1998) 13. Walfisch, J., Bertoni, H.L.: A Theoretical Model of UHF Propagation in Urban environments. IEEE Transactions on Antennas and Propagation 36, 1788–1796 (1988)

A Probabilistic Neural Network Approach to Modeling the Impact of Tobacco Control Policies by Gender Xiaojiang Ding, Chung-Hsing Yeh, and Susan Bedingfield Clayton School of Information Technology, Monash University, Victoria, Australia {Xiaojiang.Ding,Chunghsing.Yeh, Sue.Bedingfield}@infotech.monash.edu.au

Abstract. This paper evaluates the impact of tobacco control policies on female and male smokers. The data used in this study are from the 2002-2006 International Tobacco Control Four Country Survey. Based on eleven smokers’ motivational attributes used in the survey, principle component analysis is used to identify tobacco control policy drivers which are labeled as personal concerns, cigarette price, environmental restrictions and social encouragement. To examine the relative impact degrees of these four policy drivers on the groups of female and male smokers for their quit attempts, probabilistic neural network models are developed using hypothetical policy impacted populations. The experimental result shows that the most significant motivator for female smokers to make a quit attempt is their personal concerns. For male smokers, social encouragement plays a dominant role for them to make a quit attempt. The result indicates that smoking restrictions in public places or at workplace can somewhat encourage them to make a quit attempt. However, increasing the cigarette price is less likely to affect the MQA rate of both female and male smokers. Keywords: Probabilistic neural network, Quit attempt, Tobacco control policy driver.

1 Introduction Scientific evidence has shown that tobacco consumption leads to serious consequences for public health [1]. Recent studies have revealed that quitting related motivations have been identified as strong predictors of smokers’ cessation behaviours [2][3]. Based on that, examining smokers’ quitting motivations and then incorporating them into the setting of relevant tobacco control policies can effectively encourage smokers to reduce the tobacco use. Smokers’ responses to various tobacco control policies vary from one to the other. However, smokers of the same gender may have similar motivations for making a quit attempt. Thus, evaluating the impact of tobacco control policies on female and male smokers is significantly important. In this study, we develop probabilistic neural network models using the International Tobacco Control Four Country (ITC-4C) survey data to examine the relative impact degrees of different tobacco control policies on female and male smokers. Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 869–876. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

870

X. Ding, C.-H. Yeh, and S. Bedingfield

In subsequent sections, we first describe the data from ITC-4C and explain how we use the data in this study. Next, we group motivational attributes into four policy drivers which can be used to represent specific tobacco control policies. We divide the survey data into two groups by smokers’ gender. For each group, we build a probabilistic neural network (PNN) model to formulate the relationship between the four policy drivers and smokers’ quit attempts. By using these models, we conduct experiments to examine the impact of these policy drivers on smokers’ quit attempts. Finally, we discuss the implication of our research findings.

2 The International Tobacco Control Survey Data on Smokers’ Quitting Motivations The data for this study are from the 2002-2006 International Tobacco Control Four Country Survey [4][5]. Eleven survey questions are selected from the ITC-4C survey to assess Australian smokers’ motivations for thinking about quitting. Answers to these eleven questions are all on a three-point scale: 1 = not at all, 2 = somewhat, and 3 = very much, which are the possible responses for each question. Table 1 shows the survey questions. Table 1. Questions to assess smokers’ motivations for thinking about quitting Even though you mentioned that you are not currently planning to quit, in the past 6 months, have each of the following things led you to think about quitting, not at all, somewhat, very much? 1 – Not at all 2 – Somewhat 3 – very much Concern for your personal health? QM01 Concern about the effect of your cigarette smoke on non-smokers? QM02 Society disapproves of smoking? QM03 Price of cigarettes? QM04 Smoking restrictions at work? QM05 Smoking restrictions in public places like [restrictions OR bars/cafes OR pubs]? QM06 Advice from doctor, dentist, or other health professional to quit? QM07 Free or lower cost, stop-smoking medication? QM08 Availability of telephone helpline/Quitline/information line? QM09 Advertisements or information about the health risks of smoking? QM10 Warning labels on cigarette packages? QM11 Setting an example for children? QM12 Have you made any attempts to stop smoking? QA01 1 – Yes 2 – No

Of 4319 respondents collected, 1752 (40.56%) smokers made a quit attempt (MQA) and 2567 (59.44%) smokers did not make a quit attempt (NMQA). In this study, we aim to examine how specific tobacco control policies represented by policy drivers will affect smokers’ quitting motivations and resultant quit attempts. In analysing ITC-4C survey data, we find that there are always a relatively resistant

A Probabilistic Neural Network Approach

871

‘hard-core’ group of smokers whose quitting motivations or beliefs are impervious to their quit attempts. Devising a policy that changes such a group of smokers’ beliefs is not likely to lead to any significant influence on their quit attempts. Therefore, in this study we focus on that group of smokers whose quit attempts are impacted by their beliefs which in turn can be impacted by tobacco control policies. For example, in the survey data, a small proportion of smokers answered all or most of the questions “very much”, but they did not make a quit attempt; while some smokers answered all or most of the questions “not at all”, but they still made a quit attempt. As the average value of smokers’ responses to these survey questions is 1.78, we remove the cases which satisfy either of the following conditions: (a) average response value > 2 and NMQA; (b) average response value = 1 and MQA. Thus, the data set used in this study consists of 3879 cases (89.81% of the original cases). MQA and NMQA numbers are 1723 (44.42%) and 2156 (55.58%), respectively.

3 Identifying Tobacco Control Policy Drivers Tobacco control policies are designed to drive social, environmental and system changes in order to reduce the health burden of tobacco use [6]. A tobacco control policy is often developed using one or more policy drivers for encouraging smokers to make a quit attempt. As such, identifying tobacco control policy drivers from these motivational attributes shown in Table 1 will help develop effective tobacco control policies. Principle component analysis (PCA) is widely used to identify new meaningful underlying factors based on given attributes [7]. In this study, we use the PCA to group motivational attributes into factors to identify the underlying tobacco control policy drivers. The statistical package SPSS16.0 is used for conducting the PCA experiments. We use the default values in parameter settings except for selecting ‘varimax’ in the ‘Rotation’ method option. We also set the ‘suppress absolute value’ to less than 0.4. Table 2 shows the result of rotated component matrix using the PCA when the eleven attributes in Table 1 are loaded. According to the derived matrix, we examine the common theme for each factor and provide a label representing a policy driver. As shown in Table 2, three factors are clearly identified. QM03 and QM09 remain ambiguous. Both of these attributes are assigned two loadings which are close. When examining QM03, the question is about cigarette price which has no clear relationship with other attributes. As a result, we identify this attribute as a new factor to represent a policy driver. QM09 is about the advertisement of health risks of smoking which can be classified as one type of social encouragement for quitting, so it can be categorised in the same group as QM06, QM07, QM08 and QM10. Based on the PCA, eleven attributes are grouped into four factors as policy drivers. These factors are labeled as personal concerns (P1), cigarette price (P2), environmental restrictions (P3) and social encouragement (P4), as shown in Table 2. To examine the effect of these four policy drivers for representing tobacco control policies, we develop PNN models using these four policy drivers as the input variables.

872

X. Ding, C.-H. Yeh, and S. Bedingfield Table 2. Tobacco control policy drivers and associated attributes Motivational Attribute

Policy Driver P1

Personal concerns

P2 P3

Cigarette price Environmental restrictions

P4

Social encouragement

QM01 QM02 QM11 QM03 QM04 QM05 QM06 QM07 QM08 QM09 QM10

Factor Loading 1

2

3

0.440

0.403 0.848 0.801

0.678 0.740 0.647

0.424 0.672 0.815 0.585 0.570

0.429

4 Building Probabilistic Neural Network Models Using Policy Driver Attributes 4.1 Probabilistic Neural Network Neural network models are widely applied to model complex data and examine the relationship between input and output variables [8]. In this study, we use probabilistic neural network models to predict smokers’ quit attempts. The main advantage of PNN over backpropagation neural network is its simplified architecture which overcomes the difficulty of specifying various parameters in the middle layer [9][10]. A PNN is a four-layer architecture which consists of input, pattern, summation and output layers. The inputs of the network are fully connected with the M nodes of the pattern layer. Each node of the pattern layer corresponds to one training example. The input vector X is processed by a corresponding pattern node which forms a product of the weight vector W and the given object for classification, where the weights entering a node are from a particular example. The product is then passed through the activation function Oi given by (1): (1) ⎡ X TW i − 1 ⎤ O

i

= exp ⎢ ⎣

σ

2

⎥ ⎦

where X and W are the input vector and weight vector, respectively. The parameter controls the width of the activation function. The outputs of the pattern nodes are passed to the summation layer that consists of k competitive nodes each corresponding to one class. Each summation node k is connected to the pattern nodes that involve training examples that belong to class k. For an input vector X, the summation node k simply takes the outputs of the pattern nodes to which it is connected with to produce an output fk(X) as shown in (2):

A Probabilistic Neural Network Approach

fk ( X ) =

1 M

∑ k

⎡ X TW i − 1 ⎤ exp ⎢ ⎥ σ 2 ⎣ ⎦

873

(2)

where Mk is the number of training objects that belong to class k. 4.2 Developing Probabilistic Neural Network Models for Groups of Female and Male Smokers Eleven attributes listed in Table 1 are used as input variables and quit attempt is the output variable. To evaluate the effect of tobacco control policies on different groups of smokers, we split the data into two groups by smokers’ gender. Each data set is divided into a training set (70%) and a test set (30%). For each group of smokers, we first load the training set into the PNN model, then we test the accuracy of the test set. For female smokers, the prediction accuracies on the training set and the test set are 68.49% and 68.90%, respectively. For male smokers, the corresponding accuracies are 66.75% and 67.48%. Both the training set and the test set accuracies in each data set are close, indicating the generalisation of the each model built.

5 Evaluating the Impact of Tobacco Control Policies on Female and Male Smokers Tobacco control policymakers are concerned about the significance and effectiveness of various policies on smokers’ quit attempts. In this study, we have identified four policy drivers which can be used to develop tobacco control policies. For each group of smokers, to examine their relative impacts of these four policy drivers, we can first create hypothetical policy impacted populations and use the PNN model to investigate how the smokers’ MQA rate changes if we increase their responses to each policy driver respectively. New populations can be created as follows: for each policy driver Pi, we assume that a policy has been devised which increases the average value of Pi by an increment ∆Pi. When we replace the value of Pi in the original population by the value Pi +∆Pi, this effectively creates a new hypothetical population. We then use the PNN model to determine the relative change in the MQA rate from the original to the new hypothetical population. We do this for all values of i = 1, 2, 3, 4 and for values of ∆i from 0 to 1 in steps of 0.2 point. Table 3 shows the change of predicted MQA rate on the group of female smokers when we increase the value of each individual policy driver. Among these four policy drivers, the most influential one for the female smokers is their personal concerns (P1). With a steady increase of smokers’ response values to their personal concerns, a notably increase of the MQA rate occurs. This indicates that the female smokers’ response values regarding their personal concerns may significantly influence their motivations for making a quit attempt. This result suggests that if a policymaker can devise a policy that increases the average value of the female smokers’ response to their personal concerns by one point (e.g. from 1 to 2 in a three-point scale), then the MQA rate will increase by 149.30%. Another effective policy driver is environmental restrictions (P3), showing that the predicted MQA rate improves quickly by varying

874

X. Ding, C.-H. Yeh, and S. Bedingfield

the increments. When the value of the increment is one point, the predicted MQA rate increase is 73.71%. Social encouragement (P4) ranks the third followed by cigarette price (P2) which appears to have no impact on the female smokers. Table 3. Predicted MQA rate increase on the group of female smokers Policy Driver P1 P2 P3 P4

Personal Concerns Cigarette Price Environmental Restrictions Social Encouragement

Estimated increase in the MQA rate (%) +0.2 +0.4 +0.6 +0.8 +1.0 35.21 65.26 100.47 125.35 149.30 0 0 0 0 0 11.27 25.82 41.78 55.40 73.71 8.45 20.19 34.74 49.30 58.22

For the group of male smokers, Table 4 shows that the most significant policy driver is social encouragement (P4). When the value of the increment on P4 is one point, the predicted MQA rate increase is 395.00%. The second effective policy driver is environmental restrictions (P3). The result from Table 4 shows that both personal concerns (P1) and cigarette price (P2) have very weak impact on the male smokers. Table 4. Predicted MQA rate increase on the group of male smokers Policy Driver P1 P2 P3 P4

Personal Concerns Cigarette Price Environmental Restrictions Social Encouragement

Estimated increase in the MQA rate (%) +0.2 +0.4 +0.6 +0.8 +1.0 12.50 16.25 23.75 27.50 33.75 2.50 2.50 7.50 11.25 12.50 15.00 23.75 42.50 55.00 92.50 42.50 127.50 211.25 303.75 395.00

6 Discussion This study demonstrates that female and male smokers have notably different motivations for making a quit attempt. The experimental result indicates that the most significant motivator for female smokers to MQA is their personal concerns which include their personal health, effect of smoking on non-smokers and setting an example for children. In comparison with their personal concerns, social encouragement has a much weaker impact on them for quit attempts. For male smokers, social encouragement plays a significant role to encourage them to MQA. Social encouragement reported in this study consists of advice from health professionals, cheap stop-smoking medication, quitline, advertisements about the health risks of smoking and warning labels on cigarette packs. This indicates that the male smokers are more like to use one of these social encouragement methods to MQA and the result shows that these intervention methods are remarkably effective for male smokers to MQA. The result highlights that the policy driver regarding environmental restrictions has a moderate impact on both female and male smokers. It suggests that smoking restrictions in public places or at workplace can force smokers to stop smoking so as

A Probabilistic Neural Network Approach

875

to encourage them to MQA. In contrast, increasing the cigarette price is less likely to affect the MQA rate on both female and male smokers. The findings in this study provide useful insights in designing and implementing tobacco control policies which can effectively boost the MQA rate of smokers. For instance, if policymakers can devise a policy which can guide female smokers to focus on their personal concerns about the negative effects of smoking and develop a policy which can encourage male smokers to pay more attention to social encouragement for quitting to a certain degree, it will result in a significant increase in the MQA rate on each group of smokers.

7 Conclusion In this paper, we have presented a PNN approach to modeling the impact of tobacco control policies on the groups of female and male smokers. We have identified four tobacco control policy drivers for developing policies to help increase the MQA rate of smokers. Based on the groups of female and male smokers, we have also identified how the impact degrees of a specific policy driver change in terms of the MQA rate if we gradually increase smokers’ response values to that policy driver. The result of this study suggests that the most influential policy driver for female smokers is their personal concerns while the dominant motivator for male smokers to make a quit attempt is social encouragement for quitting. The result indicates that social restrictions may have a certain impact on both groups of smokers. It appears that increasing cigarette price has little effect on both female and male smokers.

References 1. David, R.M.: 50 Years of Reporting on Tobacco and Health. British Medical Journal 320, 74 (2000) 2. Hyland, A., Borland, R., Li, Q., Yong, H.-H., McNeill, A., Fong, G.T., O’Connor, R.J., Cummings, K.M.: Individual-level Predictors of Cessation Behaviours among Participants in the International Tobacco Control (ITC) Four Country Survey. Tobacco Control 15(3), 83–94 (2006) 3. Ding, X., Bedingfield, S., Yeh, C.-H., Borland, R., Young, D., Zhang, J.Y., PetrovicLazarevic, S., Coghill, K.: A Decision Tree Approach for Predicting Smokers’ Quit Intentions. Journal of Electronic Science and Technology 6(3), 220–224 (2008) 4. Fong, G.T., Cummings, K.M., Borland, R., Hastings, G.B., Hyland, P., Giovino, G.A., Hammond, D., Thompson, M.E.: The Conceptual Framework of the International Tobacco Control (ITC) Policy Evaluation Project. Tobacco Control 15(3), 3–11 (2005) 5. Thompson, M.E., Fong, G.T., Hammond, D., Boudreau, C., Driezen, P., Hyland, P., Borland, R., Cummings, K.M., Hastings, G.B., Siahpush, M., Machintosh, A.M., Laux, F.L.: Methods of the International Tobacco Control (ITC) Four Country Survey. Tobacco Control 15(3), 12–18 (2006) 6. Wisotzky, M., Albuquerque, M., Pechacek, T.F., Park, B.Z.: The National Tobacco Control Program: Focusing On Policy To Broaden Impact. Public Health Reports 19, 303– 310 (2004) 7. Jolliffe, I.T.: Principal Component Analysis. Springer, Heidelberg (1986)

876

X. Ding, C.-H. Yeh, and S. Bedingfield

8. Yeh, C.-H., Lin, Y.-C.: Neural Network Models for Transforming Consumer Perception into Product Form Design. In: Wang, J., Yi, Z., Żurada, J.M., Lu, B.-L., Yin, H. (eds.) ISNN 2006. LNCS, vol. 3973, pp. 799–804. Springer, Heidelberg (2006) 9. Specht, D.: Probabilistic Neural Networks. Neural Networks 3, 109–118 (1990) 10. Gaganis, C., Pasiouras, F., Doumpos, M.: Probabilistic Neural Networks for the Identification of Qualified Audit Opinions. Expert Systems with Applications 32, 114–124 (2007)

The BPNN-Fuzzy Logic Pre-control of an Underground Project in City Center of Shanghai Zhenzhen Yin1, Youliang Chen1,2, and Peng Wang1 1

Department of Civil Engineering, University of Shanghai for Science and Technology, 516 Jungong Road, 200093 Shanghai, China [email protected] 2 Department of Engineering Geology and Hydrogeology, RWTH Aachen University, Lochnerstr. 4-20, 52064 Aachen, Germany [email protected]

Abstract. This study is an attempt to provide a methodology for predicting the stability of building foundation pits basing on the employment of intelligent tools: neural network and fuzzy logic. For this purpose, a back-propagation network (BP network) model was established by training the prophase horizontal displacement data of a building foundation pit, which was situated in center of Shanghai city. The obtained BP network model was then adopted to predict the later stage displacements of the above-mentioned foundation pit. Following it, a fuzzy logic model was set up by employing Mamdani fuzzy algorithm and applied to the pre-control of foundation pit excavation. In the fuzzy logic model, the displacement values, which were predicted by BP neural network, were employed as input data. The obtained output result, namely building foundation pit pre-control measures, is proven to be very ideal through practical verification. The developed theory and methodology offers a new way for the pre-control of building foundation pit excavation and other underground construction. Keywords: Building foundation pit, Displacement prediction, Construction precontrol, Neural networks, Fuzzy logic.

1 Introduction Due to the increasing demands upon superhigh buildings in Shanghai, the stability of foundation pits has already turned into one of the key problems in building construction. The prediction of stability of foundation pits and its corresponding engineering measures is paid great attention by pepole. However, most of the former predictive means are based on empirical methods, since complete quantitative theories have not been developed. In order to improving the controlling effect of foundation pit design and construction, some theoretical studies are conducted by geotechnical researchers. For examples: Li Shou-Ju employed ANNs to predict the parameters of water-bearing stratum[1]. Chua CG and Goh ATC adopted the hybrid Bayesian backpropagation neural network methodology to conduct multivariate modeling[2]. Deng Jian, Yue Zhong-Qi, Tham LG and Zhu He-Hua combined FEM method, neural Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 877–884. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

878

Z. Yin, Y. Chen, and P. Wang

networks and reliability to conduct the pillar design approach of Feng Huangshan copper mine[3]. Basma Adnan A. and Kallas Nabil employed BP neural networks to model soil collapse successfully[4]. Panigrahi D. C. and Sahu H. B. applied adaptive resonance theory of artificial neural networks (ANN) to classify the coal seams with respect to their proneness to spontaneous heating[5]. Goh ATC and Kulhawy FH employed artificial neural networks to study the reliability assessment of serviceability performance of braced retaining walls[6]. Artificial neural networks (ANNs) consist of numerous, simple processing units or “neurons” that can be globally programmed for computation. They can be programmed or trained to store, recognize and associated retrieve patterns or database entries, to solve combinative optimization problems, to reject noise from measured data and to control ill-defined problems. In summary, to estimate sampled functions when the form of the functions is not known. ANNs do not use a mathematical model of how a system’s output depends on its input. They are especially applicable to simulating some nonlinear and multivariate systems. Fuzzy set theory was introduced by Zadeh (1965) as a general approach to express different types of uncertainty inherent in human systems. Fuzzy sets come from the generalization of traditional ones. An element belongs to a fuzzy set with a degree of possibility or membership. The membership grade takes value within the interval [0, 1], and represents the degree of how an element is similar to or compatible with the behavior described by fuzzy set. Fuzzy set boundary is vague, and the transition from member to nonmember is gradual rather than abrupt. Fuzzy sets can also overlap and therefore, an element can belong to a number of fuzzy sets with different membership degrees. Fuzzy decision making is one of the most powerful functions of fuzzy theory. It has been applied successfully to many scientific and industrial problems. The most important step for fuzzy decision making is to sum up the most rational fuzzy rules, which can be expressed through the form Ri: if x1 is Ai1, x2 is Ai2, …, xm is Aim then y1 is Bi1, y2 is Bi2, …, yn is Bin (for i= 1,2,…,k) where k is the number of rules, x1, …, xm are input variables, and y1, …, yn output variables. In current paper, artificial neural networks (ANNs) and fuzzy systems are naturally combined. Firstly, the foundation pit of building No.4 in plot No.211 of Shanghai and `its supporting structure are considered as a system. For this system, BP neural networks are employed to establish the relation between time and the corresponding displacements. Time is input vector and displacements are output vector. Giving the BP neural network model parameters tentatively and substituting the prophase excavation data into the tentative BP neural network model, a high-accuracy neural network model could be obtained by training. Secondly, the obtained BP neural network model is employed to predict the later stage foundation pit displacements. Thirdly, a fuzzy logic model is set up for predicting the excavation control plan of the foundation pit. In this model, the ANNs-predicted total horizontal displacements and incremental horizontal displacements are adopted as input variables. The output variable is the excavation control plan. The purpose of fuzzy logic prediction is to limit the deformation of the foundation pit and to ensure the safety of the foundation

The BPNN-Fuzzy Logic Pre-control of an Underground Project

879

pit excavation. Finally, the accuray of the ANN-Fuzzy systems is verified compared with engineering practice.

2 Analysis of the Foundation Pit Displacement Data during Prophase Excavation The project block covers an area of 25,000 m2. The buildings occupy an area of 6,700 m2. In line with the design, this project targets the residential area, consisting of four main high-rise buildings of 21~30 stories and other subsidiary installations of 2~ 3 stories. In addition, there is a basement of two stories. Figure 1 give the governing point and illustration of the shaft of building No.4. In the figure the letter “Q”denotes a governing point of shaft. In this paper, building No.4 is selected as the key analyzing object. The BP neural networks and fuzzy logic model are combined and applied to the displacement prediction and construction control. It could be found from figure 2 that the foundation pit displacements of building No.4 are comparatively big. Although it has not reached the warning value, it has a great influence to the construction of buildings and neighboring constructions.

Settlement(mm)

Fig. 1. Governing point and illustration of the shaft of building No.4 15 Q20

10

Q22 5

Q23 Q24

0 1

2

3

4

5

6

7

Q25

Tim e(day)

Fig. 2. The time-settlement curves of foundation pit walls of building No.4 during prophase excavation

880

Z. Yin, Y. Chen, and P. Wang

3 Analysis and Prediction on the Horizontal Displacements of Project Foundation Pit by BP Neural Networks In this paper, a BP network model is established by training the prophase horizontal displacements of the foundation pit of building No.4, and the obtained BP network model is applied to the prediction of the later stage horizontal displacements of the same foundation pit. The objective function is the summed squared error(SSE) between the calculated displacements and the observed ones. In the current paper, the given maximum SSE is 0.001. The training data consist of 16 sets of horizontal displacements (1-16 day), and the displacements of other 5 days (17-21 day) are predicted by the established BPNN model. The predicted results are listed in Table 1. It could be concluded that the predicted results are very accurate. Table 1. The predicted values of horizontal displacements for the foundation pit walls of building No.4 (mm)

Q20

Q22

Q23

Q24

Q25

Mea- Pred- Mea- Pred- Mea- Pred- Mea- Pred- Mea- PredTime sured icted sured icted sured icted sured icted sured icted (day) Values Values Values Values Values Values Values Values Values Values 1 1 1 0.5 7 16.5 2 6 1.5 0.5 9.5 19.5 3 9.5 2.5 1.5 15.5 22 4 12 7 3 19 24 5 15.5 17 6 24 26 6 17 20 7 26 27 7 18 22 7.5 27 28 8 19 23 8.5 27.5 28.5 9 20 25.5 9.5 28 28.5 10 20.5 27.5 10 28 28.5 11 21.5 28.5 10.5 28.5 28.5 12 22 29.5 11 28.5 29 13 22.5 30 11.5 28.5 29 14 23.5 30.5 11.5 28.5 29.5 15 24 31 12 29 29.5 16 25 31.5 12 29 30 17 25.5 26.94 32 33.88 12.5 12.24 29 29.69 30 30.53 18 25.5 26.07 32.5 33.27 12.5 12.40 29.5 28.15 30.5 30.03 19 26 26.25 33 33.21 12.5 12.41 29.5 28.36 30.5 30.33 20 27 28.21 33.5 33.27 12.5 12.41 29.5 28.57 31 31.03 21 27.5 28.23 33.5 33.47 12.5 12.41 29.5 28.75 31 31.23

In following part, a fuzzy logic model will be established between the horizontal displacement and the corresponding construction control measure based on the expert experience. And the construction control measures of 17-21 day will be predicted by input the predicted displacements into the established fuzzy logic model.

The BPNN-Fuzzy Logic Pre-control of an Underground Project

881

4 The Fuzzy Logic Prediction of the Foundation Pit Excavation Control Measures of Building No.4 Fuzzy theory is more and more widely adopted in geotechnical engineering. In this paper, a fuzzy logic model is established for predicting the excavation control measures of the foundation pit. In this model, the total horizontal displacements and incremental horizontal displacements are adopted as input variables. The output variable is the excavation control measure. The purpose of fuzzy logic prediction is to limit the deformation of the foundation pit and to ensure the safety of the foundation pit excavation. The definition of the membership function is an important step for fuzzy logic modeling. The following membership function has been employed. u(x)=1/[1+k(x-a)b], (k>1,a>0, b is a positive and even number) .

(1)

In this paper, the if-then rules are obtained based on careful studies to the construction data of building No.3 and other adjacent buildings. The concluded nine if-then rules are listed in Table 2. For the if-then rules, the grades of horizontal displacements are defined as Table 3 Table 2. “if-then” rules adopted by the fuzzy logic model

Rule No. 1 2 3 4 5 6 7 8 9

Definition of if-then rules If the total horizontal displacement is small and the incremental Displacement is small then output is normal construction If the total horizontal displacement is small and the incremental Displacement is medium then output is slowdown construction If the total horizontal displacement is medium and the incremental Displacement is small then output is slowdown construction If the total horizontal displacement is medium and the incremental Displacement is medium then output is slowdown construction If the total horizontal displacement is small and the incremental Displacement is large then output is suspend construction If the total horizontal displacement is medium and the incremental Displacement is large then output is suspend construction If the total horizontal displacement is large and the incremental Displacement is small then output is suspend construction If the total horizontal displacement is large and the incremental Displacement is medium then output is suspend construction If the total horizontal displacement is large and the incremental Displacement is large then output is suspend construction

The total and incremental horizontal displacements obtained from BP neural networks prediction for foundation pit of building No.4 are shown in Table 4. The excavation control measures can be generated by substituting the predicted displacement data in Table 4 into the fuzzy logic model. The obtained excavation control measures are shown in Table 5.

882

Z. Yin, Y. Chen, and P. Wang

Table 3. The grade of total horizontal displacements and incremental horizontal displacements used by “if-then” rules

Small Total horizontal displacement Below 30 mm Incremental horizontal Below 1 mm/d Displacement (per day)

Medium 30-40 mm

Large Greater than 40 mm

1-2.5 mm/d

Greater than 2.5 mm/d

Table 4. The total horizontal displacements and incremental horizontal displacements of foundation pit of building No.4 obtained by BPNN prediction (mm)

Q20 DisplaTime cement (day) Accumulation

17 18 19 20 21

Displacement Increment

26.94 1.94 26.7 -0.87 26.25 0.18 28.21 1.96 28.23 0.02

Q22 Displacement Accumulation

Displacement Increment

Q23 Displacement Accumulation

33.88 2.38 12.24 33.27 -0.61 12.40 33.21 -0.06 12.41 33.27 0.06 12.41 33.47 0.20 12.41

Displacement Increment

Q24 Displacement Accumulation

0.24 0.16 0.01 0 0

Displacement Increment

Q25 Displacement Accumulation

Displacement Increment

29.69 0.69 30.53 0.53 28.15 -1.54 30.03 -0.50 28.36 0.21 30.33 0.30 28.57 0.21 31.03 0.70 28.75 0.18 31.23 0.20

Table 5. The construction control measures obtained from the BPNN-Fuzzy logic model Time(day) 17 18 19 20 21

Q20 slowdown construction normal construction normal construction slowdown construction normal construction

Q22 slowdown construction slowdown construction lowdown construction slowdown construction slowdown construction

Q23 normal construction normal construction normal construction normal construction normal construction

Q24 normal construction normal construction normal construction normal construction normal construction

Q25 slowdown construction slowdown construction slowdown construction slowdown construction slowdown construction

The measured horizontal displacements are listed in Table 6.By substituting the site measured displacements into the fuzzy logic model, the construction control measures as shown in Table 7 can be obtained. It could be found from the comparison between Table 5 and 7 that the excavation control measures obtained from BPNN-predicted displacements and the measures obtained from site measured displacements are almost identical, except for the check (17, Q20). For this check, the excavation control measure of Table 5 is more secure and conservative than that of Table 7. The scheme of Table 5 was completely adapted in the excavation process and the whole construction course went smoothly and successfully. So the application of the coupling BPNN-fuzzy logic model in foundation pit mentioned above is successful.

The BPNN-Fuzzy Logic Pre-control of an Underground Project

883

Table 6. The total horizontal displacements and incremental horizontal displacements for foundation pit of building No.4 obtained from site measurement (mm) Q20 Displa- DisplaTime cement cement (day) Accum- Increulation ment 17 25.5 0.5 18 25.5 0 19 26 0.5 20 27 1 21 27.5 0.5

Q22 Displa- Displacement cement Accum- Increulation ment 32 0.5 32.5 0.5 33 0.5 33.5 0.5 33.5 0

Q23 Displa- Displacement cement Accum- Increulation ment 12.5 0.5 12.5 0 12.5 0 12.5 0 12.5 0

Q24 Displa- Displacement cement Accum- Increulation ment 29 0 29.5 0.5 29.5 0 29.5 0 29.5 0

Q25 Displa- Displacement cement Accum- Increulation ment 30 0 30.5 0.5 30.5 0 31 0.5 31 0

Table 7. The construction control measures obtained from site-measured horizontal displacement and BPNN-Fuzzy logic model Time(day) 17 18 19 20 21

Q20 normal construction normal construction normal construction slowdown construction normal construction

Q22 slowdown construction slowdown construction slowdown construction slowdown construction slowdown construction

Q23 normal construction normal construction normal construction normal construction normal construction

Q24 normal construction normal construction normal construction normal construction normal construction

Q25 slowdown construction slowdown construction slowdown construction slowdown construction slowdown construction

5 Conclusions In this paper, the BP neural networks and fuzzy logic are combined and successfully applied to the excavation pre-control of the foundation pit project located in plot No. 211 of Shanghai city center. The research results are highly beneficial and instructive for the future foundation pit construction pre-control. The established BP neural network-fuzzy logic coupling model shows a good potential in the respect of modeling complex, non-linear and multivariable geotechnical engineering systems. A noted advantage of this coupling model is that one does not need to have a very well defined formulation of the problem in terms of first principles to construct a reasonably accurate prediction model from measured data. Another remarkable advantage of this coupling model is that it has both efficient prediction and effective control abilities. The BP neural network-fuzzy logic coupling model can also easily be extended to other engineering domains such as modeling the dynamic interaction between the rock mass and the excavating machine, and controlling the functioning of the machine. Furthermore, the BP neural network-fuzzy logic coupling model can also be used for risk analysis of slopes, rock mass classification, hydrogeology and so forth. The BP neural network-fuzzy logic coupling model is much more efficient in

884

Z. Yin, Y. Chen, and P. Wang

predicting and controlling a geotechnical engineering problem than either BP neural network or fuzzy logic alone. It is worthwhile to emphasize that a thorough investigation and study for huge amount of data and the possession of expertise are very paramount prerequisite for establishing the most efficient BP neural network-fuzzy logic coupling model. Different model establishers may come up with different models because of different levels in understanding the same process. Acknowledgements. The financial support from the USST Manheng Education Development Foundation, the National Natural Science Foundation of China (10872133) and Shanghai Pujiang Talent Program (09PJ1407700) for this study is gratefully acknowledged.

References 1. Li, S.J.: The inversion method of rock mass osmotic coefficient and its engineering application. Chinese Journal of Rock Mechanics and Rock Engineering 21, 479–483 (2002) (in Chinese) 2. Chua, C.G., Goh, A.T.C.: A hybrid bayesian back-propagation neural network approach to multivariate modeling. International Journal for Numerical and Analytical Methods in Geomechanics 27, 651–667 (2003) 3. Deng, J., Yue, Z.Q., Tham, L.G., Zhu, H.H.: Pillar design by combining finite element methods-neural networks and reliability: a case study of the feng huangshan copper mine. International Journal of Rock Mechanics and Mining Science 40, 259–268 (2003) 4. Basma, A.A., Kallas, N.: Modeling soil collapse by artificial neural networks. Geotechnical and Geological Engineering 22, 427–438 (2004) 5. Panigrahi, D.C., Sahu, H.B.: Classification of coal seams with respect to their spontaneous heating susceptibility—a neural network approach. Geotechnical and Geological Engineering 22, 457–476 (2004) 6. Goh, A.T.C., Kulhawy, F.H.: Reliability assessment of serviceability performance of braced retaining walls using a neural network approach. International Journal for Numerical and Analytical Methods in Geomechanics 29, 627–642 (2005)

Optimal Parameter Inversion of Marine Water Quality Model Using a BPNN Data-Driven Model –– A Case Study on DIN Mingchang Li1, Bin Zhou1, Shuxiu Liang2, and Zhaochen Sun2 1

Laboratory of Environmental Protection in Water Transport Engineering, Tianjin Research Institute of Water Transport Engineering Tianjin 300456, China 2 State Key Laboratory of Coastal and Offshore Engineering, DUT, Dalian 116024, China [email protected]

Abstract. The accurate inversion of model parameters is a major difficulty for marine water quality model. In this paper, a data-driven model (DDM) method based on back-propagation neural network (BPNN) is developed to inverse the value of model parameters and to find out the relationship between model parameters and the pollution concentration values of interior stations. All training data are calculated by numerical water quality model from results of multi-parameter matching design cases, so the physical properties are not disturbed. Filed data is imported into the relationship for inversing optimal parameter. Finally optimal prediction method is applied to validate the long term stability of inversion results. Case tests are carried out in Bohai Sea, China. Dissolved inorganic nitrogen (DIN) and its sensitive parameters are considered for validating the present method. Case studies show that the present method can make a more satisfactory inversion for a practical problem. Keywords: water quality model, parameter inversion, data-driven model, BPNN, multi-parameter matching.

1 Introduction Substantial uncertainties exist in marine ecosystem (Wan, 1999), which mainly originated from scarcity for understanding of biochemical process, the values of pollution source, pollution background field, the field data and model parameter (Ganoulis, 1994; Takyi, 1999). One of the most significant difficulties is the estimation of model parameters for model calibration. Since Shastry (1973) estimated the parameter of BOD-DO model, many estimated methods have been developed (Malve, 2007; Arhonditsis, 2007; Yang, 2000; Fasham, 1995; Zhang, 2005; Zhang, 2005; Liu, 2006). In these methods, Trial-and –error (Gerristen, 1995) is a widely used technique for model calibration (Seo, Canale, 1996). How much the model come close to real conditions depends on the abundance of field data. However, model calibration process might spend much time. Data assimilation methods have been employed for model calibration with the abundance of satellite data. Adjoint Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 885–892. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

886

M. Li et al.

technique is the most widely used one of them (Lawson, 1995, 1996; Vallino, 2000; Xu, 2005, 2006). Optimal estimation for parameters, initial and boundary conditions can be obtained by this technique. However, both water quality model equations and adjoint equations need to be calculated. Adjoint equations are as complicated as water quality model equations, so much time is needed. When adjoint technique is chosen for practical engineering, many uncertainties originating from the absence of field data might exist in the calibrated model owing to the limitation of investment. The application of data-assimilation method is limited (Ma, 2005). The present paper aims to develop a more practical technique for optimal estimation of model parameters. In the technique, optimal parameters are estimated by data-driven model (Solomatine, 2002) which is based on artificial neural network. Osaka Daigaku Estuary Model (ODEM, Nakatuji, 2001) is employed to simulate the marine water quality in interested area. The structure of this paper is as follows. In section 2, the basic idea and theory of data-driven model, Back-Propagation Neural Network (BPNN) and water quality model (ODEM) are introduced briefly. The detailed steps about how to inverse model parameters are described in section 3. In section 4, the method in section 3 is verified with an entitative ocean. In section 5, conclusions are made.

2 Numerical Model 2.1 BPNN Data-Driven Model The so-called data-driven models are different from knowledge-driven models (physically-based model). These kinds of models are based on a limited knowledge of the modelling process and rely purely on the data describing input and output characteristics. They make abstractions and generalizations of the process, so play often a complementary role to physically-based model. Data-driven model can use results from artificial neural networks (ANN), expert systems, fuzzy logic concepts, rule-induction and machine learning systems. The fundamental expression is as follows:

( y1 , ", yi , " ym ) = F ( x1 , " , xi , " xn )

(1)

( x1 ," , xi , " xn ) and ( y1 , ", yi , " ym ) are the input and output variables respectively F is the objective function which need to be dug by model. In this paper, the results of ANN are used for fitting of F . The merit of the BPN is that it can approach any nonlinear continuous function after trained (Hornik, 1991). The BPNN proposed by Rumelhart et al. (1986) is the most commonly used among the entire artificial neural network. The key point is the error back-propagation technique. In learning process of the BPN, the interconnection weights are adjusted from back layers to front layers to minimize the output error. The detailed information can be referred in Li and Liang et al (2007). There are only two differences in this study. One is the leaning rate and it is set to be 0.05. The other one is the introduction of appended momentum and its value is 0.5.

；

Optimal Parameter Inversion of Marine Water Quality Model

887

2.2 Water Quality Model Water quality model is fundamental since it tries to explain the underlying physical processes. There are many water quality models which can retrieve entitative ocean conditions with great precision. ODEM developed by Nakatuji is one of them. ODEM has been used not only for water quality modeling but for costal waters studies.

3 Optimal Parameters Inversion Model In this paper, a new technique is developed which combines data-driven model with water quality model automatically. The detailed technique is as follows: Step 1: Choice of control variables There are many parameters in water quality model and they act with each other. If all of them are included, the computation cost is excessive and uncertainty increases (Friedrichs, 2007). So sensitive model parameters have been analyzed (Hakanson, 2000), which aim to select control variables. Step 2: Cases computation by water quality model In this paper, initial guess values for all the control variables are assumed and their corresponding ranges are set. If the number of control variables is m, and n values are taken for one control variable, there are as many as

∏ ⎡⎣Cn1 ⎤⎦ = n m designed cases.

i =m

i

All the designed cases are computed by ODEM one by one. The results of pollution concentration are output and stored for data-driven model. Step 3: model parameters’ estimation by data-driven model Input the results of pollution concentration at interior stations and their corresponding parameters’ values into data-driven model. After training, the relationship of interior stations and model parameters is generalized. Input the field data of interior stations into the above relationship and obtain the optimal solution. Step 4: verification of optimal solution Input the optimal solution into the water quality model and repeat the computation. The relative error between measurement and results of numerical computation are calculated. In figure 1, the process of model parameter estimation is described. 1-8 is the sequence of it. In whole process, there are two modules—water quality and optimal estimation. The computation of designed cases and the final verification are finished by water quality module. Optimal estimation module is responsible for the analysis of water quality model results and generalization of relationship between model parameters and interior stations. A and B are the connection of the two modules.

4 Case Studies Case tests are carried out with Bohai Sea, China, which is a semi-closed sea with a 2 mean depth of 18.7m and an area of more than 80 000km . The sea is mainly divided

888

M. Li et al.

Water Quality Simulation Module for Regional Water Quality Model

（

1

Water Quality Model ODEM

7 Designed Cases with Multi Control Variables Combination

A

Field data of gauge station 4

3

Simulated results of ORMP 8 Compare

）

2

6

ORMP 5

Data-Driven Model

3

Simulated Results of Designed Cases

B

Optimal Estimation Module with Data-Driven Model

Note: ORMP: Optimal Resolution of Model Parameter Fig. 1. Diagram of Optimal Estimation of Model Parameter Using Data-driven Model Combined with Water Quality Model

Fig. 2. Diagram of research region

into four parts: Laizhou bay, Bohai bay, Liaodong bay and the central part. Its bottom is very flat and average slope is 28”. Figure 2 shows the location of computation domain.

Optimal Parameter Inversion of Marine Water Quality Model

889

4.1 Choices of Control Variable In this paper, DIN as state variables is considered to verify the present optimal inversion method. Coefficient of variation is computed by Monte-Carlo method to validate the sensitivity of model parameters. State variables and its sensitive parameters are listed in table 1. Table 1. State variables and sensitive parameters State variable

Sensitive parameters Fastest growth rate of phytoplankton VMMAX Most suitable growth temperature of phytoplankton TEMPS Decomposed rate of DIN DRDN

（

DIN

） （

（

）

）

Unit 1/d °C 1/d

In table 1, DIN has three sensitive parameters. Control variables corresponding ranges are set among initial guess values in table 2. Table 2. Values of sensitive parameters State variable DIN

Sensitive parameter VMMAX TEMPS DRDN

1.44 15 0.012

Values 2.5 25 0.022

3.36 35 0.028

In table 2, the values of control variables are listed. For each control variable, three values are taken in its range, 27 designed cases are obtained for DIN. 4.2 Optimal Estimation 27 cases pollution concentration data are acquired by 27 times model computation. Input the 27 data and relative data in table 2 into data-driven model, the relationship is generalized. In this paper, the so-called “twin experiment” method is used to verify the efficiency of optimal inversion method. Table 3. Design of twin experiment State variable DIN

Sensitive parameter VMMAX TEMPS DRDN

Values 2.4 26 0.02

The parameters in table 3 are as real values and input them into ODEM for the pollution concentration data as pseudo-field data. The optimal model parameters in table 4 are inversed by inputting the pseudo-field data into the above relationship.

890

M. Li et al. Table 4. Optimal solution of control variables State variable

Sensitive parameter VMMAX TEMPS DRDN

DIN

Values 2.533114 26.56910 0.02002

The optimal solutions are listed in table 4.

Fig. 3. The validation and forecasted results of DIN optimal parameter 0.5

CC=0.9988 0.4

Field data

0.3

0.2

0.1

0 0

0.1

0.2

0.3

0.4

0.5

Forecasted data

Fig. 4. The validation and forecasted Correlation coefficient results of DIN optimal parameter

Optimal Parameter Inversion of Marine Water Quality Model

891

4.3 Verification of Optimal Solution Pseudo-field data in the validation part are transported into the relationship in step 3 for model parameters and used to verify the optimal solution as step 4. Pseudo-field data in the forecasted part are used to validate the long-term stability of the optimal solution for water quality model. In figure 3, the forecasted data is closed to field data in validation and forecasted sections, the correlation coefficient is 0.9999 (figure 4), which indicates the present optimal inversion method can inverse realistic model parameters in better accuracy.

5 Conclusion In this paper, a new method is developed to estimate model parameters’ values. In this method, data–driven model and water quality model are coupled automatically. Water quality model repeat a number of computation for designed cases, the results of pollution concentration data are output and stored for data-driven model. Data-driven model generalizes the relationship between model parameters and interior stations. After field data are imported, optimal solution are obtained. Compared with adjoint method, present method is simple, flexible and less time-consuming.

Acknowledgments This work was supported by National Nonprofit Institute Research Grant of TIWTE (TKS090204).

References 1. Xiantao, W.: Research and application on water quality monitor in rivers. Jiangsu Environment Science and Technology 2, 1–4 (1999) (in Chinese) 2. Ganoulis, J.G.: Engineering risk analysis of water pollution. VCH, New York (1994) 3. Takyi, A.K., Lence, B.J.: Surface water quality management using a multiple realization chance constrained method. Water Resources Research 35(5), 1657–1669 (1999) 4. Shastry, J.S., Fan, L.T., Erickson, L.E.: Nonlinear parameter estimation in water quality modeling. J. Envir. Engrg. Div., ASCE 99(3), 315–331 (1973) 5. Malve, O., Laine, M., Haario, H., et al.: Bayesian modeling of algal mass occurrences using adaptive MCMC methods with a lake water quality model. Environmental Modeling & Software 22, 966–977 (2007) 6. Arhonditsis, G.B., Paerl, H.W., Valdes-Weaver, L.M., et al.: Application of Bayesian structural equation modeling for examining phytoplankton dynamics in the Neuse River Estuary (North Carolina, USA). Estuarine Coastal and Shelf Science 72, 63–80 (2007) 7. Yang, M.D., Sykes, R.M., Merry, C.J.: Estimation of algal biological parameters using water quality modeling and SPOT satellite data. Ecological Modeling 125, 1–33 (2000)

892

M. Li et al.

8. Fasham, M.J.R., Evans, G.T.: The use of optimization techniques to model marine ecosystem dynamics at the JGOFS station at 47° N 20° W. Philosophical Transaction of the Royal Society of London B 348, 203–209 (1995) 9. Zhangmeiyi: Estimation method of oxygen consumption coefficient K1 in the water quality model. Water Resources & Hydropower of Northeast 22(250), 42–44 (2005) (in Chinese) 10. Fangwei, Z., Dexin, C., Wang, E.: On new method and its signicance of BOD-DO model parametric estimation. Journal of Liaocheng University 8(1), 27–29 (2005) (in Chinese) 11. Jianguo, L., Jianhua, L., Hongguang, G.: Study on parametric recognition & comparison of improved S-P model. Journal of Changchun normal university 25(6), 16–21 (2006) (in Chinese) 12. Gerritsen, H., de Vries, H., Philippart, M.: The Dutch Continental Shelf Model. Quantitative Skill Assessment for Coastal Ocean Models. Coastal Estuarine Studies 47, 425–468 (1995) 13. Dong II, S., Canale, R.P.: Performance, reliability and uncertainty of total phosphorus models for lakes—I: deterministic analyses. Water Res. 30, 83–94 (1996) 14. Lawson, L.M., Spitz, Y.H., Hofmann, E.E., Long, R.B.: A data assimilation technique applied to a predator-prey model. Bulletin of Mathematical Biology 57, 593–617 (1995) 15. Lawson, L.M., Hofmann, E.E., Spitz, Y.H.: Time series sampling and data assimilation in a simple marine ecosystem model. Deep Sea Research II 43(2-3), 625–651 (1996) 16. Vallino, J.J.: Improving marine ecosystem models: use of data assimilation and mesocosm experiments. J. Mar. Res. 58, 117–164 (1995) 17. Qing, X., Yuguang, L., Yongcun, C., et al.: Adjoint method in marine ecosystem models. Marine Science Bulletin 24(6), 58–64 (2005) (in Chinese) 18. Qing, X., Yuguang, L., Yongcun, C., et al.: Adjoint assimilation technique in a marine ecosystem model in the Bohai Sea and the Yelloe Sea: control variables and twin experiment. High Technology Letters 16(1), 78–83 (2006) (in Chinese) 19. Ma, Z., Jing, a.: Data assimilation method applied in marine science –its significance, system configuration and development situation. Coastal Engineering 24(4), 83–99 (2005) (in Chinese) 20. Solomatine, D.P.: Data-driven modelling: paradigm, methods, experiences. In: Proc. 5th international conference on hydroinformatics (2002) 21. Murota, A., Nakatsuji, K., Huh, J.Y.: A numerical study of three-dimensional buoyant surface jet. In: Proc. 6th APD-IAHR Congr., vol. 3, pp. 57–64 (1988) 22. Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back propagating errors. Nature 323, 359–366 (1989) 23. Hornik, K., Stinchcombe, M., White, H.: Multilayer feedforward networks are universal approximators. Neural Networks 2(5), 359–475 (1988) 24. Li, M.-c., Liang, S.-x., Sun, Z.-c.: Application of artificial neural networks to tide forecasting. Journal of Dalian University of Technology 47(1), 101–105 (2007) (in Chinese) 25. Friedrichs, M.A.M.: A data assimilation marine ecosystem model for the central equatorial Pacific: numerical twin experiments. Journal of Marine Research (59), 859–894 (2001) 26. Hakanson, L.: The role of characteristic coefficients of variation in uncertainty and sensitivity analyses, with examples related to the structuring of lake eutrophication models. Ecological Modelling 131, 1–20 (2000)

Determination of Sugar Content of Instant Milk-Tea Using Effective Wavelengths and Least Squares-Support Vector Machine Fei Liu and Yong He* College of Biosystems Engineering and Food Science, Zhejiang University, Hangzhou 310029, China Tel./Fax: +86-571-86971143 [email protected]

Abstract. Visible and near infrared (Vis/NIR) spectroscopy combined with least squares-support vector machine (LS-SVM) was investigated to determine the sugar content of instant milk-teas. The Savitzky-Golay smoothing, standard normal variate and 1st derivative were applied as preprocessing methods. The PLS model was developed and the optimal latent variables (LVs) and effective wavelengths (EWs) were also selected. The LV-LS-SVM model with LVs outperformed PLS models. Wavelengths at 484, 515 and 957 nm were confirmed to be EWs and the EW-SL-SVM model achieved the best performance in all developed models. The correlation coefficient (r), root mean square error of prediction (RMSEP) and bias for validation set were 0.964, 0.087 and 0.005, respectively. The results indicated that the EWs combined with LS-SVM method was successfully implemented for the prediction of sugar content of instant milk-teas, and the confirmed EWs were helpful to develop commercial instrument to progress the quality evaluation of instant milk-teas. Keywords: Visible/near infrared spectroscopy, Instant milk-tea, Sugar content, Variable selection, Least squares-support vector machine.

1 Introduction The instant milk-tea powder is a kind of solid beverages which are nourishing and very popular in recent food market. There are many different varieties of milk-teas with different qualities and prices in the market, but the basic constitutions of instant milk-tea powder are sugar, instant tea powder, milk powder, non-dairy creamer and other ingredients such as flavouring and stabilizer. In order to meet different people, instant milk-teas with different tastes are developed even in the same brand. Ever since food become to be an exchange object, many faults and willful manipulation of a product have happened for financial profits without considering the potential risks to consumers’ health [1]. Hence, it is well known that the brand stands for the quality of the products in modern industrial market. Sugar content is one of main quality parameters of instant milk-tea powders. Sugar content influences the tastes of instant *

Corresponding author.

Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 893–900. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

894

F. Liu and Y. He

milk-teas. However, there is no simple and rapid method for the determination of sugar content of instant milk-teas until now. Nowadays, near infrared spectroscopy is widely applied for quantitative and qualitative analysis in industries, such as agriculture, pharmaceuticals, food, and other industry [2]. The instant milk-tea powder is a complex food matrix with complex chemical constitutes. In the literature, some researchers had applied spectroscopic techniques for the quality evaluation and discrimination of milk and tea separately. These reports dealt with the common adulterants in powdered milk [3], the nutritional parameters in infant powdered milk [4] and the energetic values of fruit and milk-based beverages [5]. Some other researches reported the qualitative identification of tea categories [6], the determination of soluble solids content of tea soft drinks [7] and the antioxidant capacity of green tea [8]. But only a few literatures reported the detection of instant milk-tea powder. Ferruzzi and Green (2006) studied the catechins form milk-tea by enzyme assisted extraction followed by high performance liquid chromatography [9]. Bai et al. (2004) used FAAS for the determination of iron in milk-tea powder [10]. Liu et al. (2009) applied Vis/NIR for the discrimination of instant milk teas [11]. However, there were few reports about the sugar content determination of instant milk-teas using visible and near infrared spectroscopy. Hence, it was quite necessary to develop a rapid detection method for the determination of sugar content of instant milk-teas using visible and near infrared (Vis/NIR) spectral techniques. The objectives of this study were (1) to investigate the feasibility of using Vis/NIR spectroscopy to determine the sugar content of instant milk-teas, (2) to search the optimal latent variables (LVs) for the prediction of sugar content by LV-LS-SVM models, (3) to confirm the effective wavelengths (EWs) within the visible and near infrared spectral region for the determination of sugar content of instant milk-teas by EW-LS-SVM models.

2 Materials and Methods 2.1 Instant Milk-Tea Samples Four brands of original taste instant milk-teas were obtained in local market including Cici milk-tea (Strong Group Co. Ltd.), Mocca royal milk-tea (Mocca Food Co. Ltd.), Nestlé milk-tea (Société des Produits Nestlé S.A.), and Xiangpiaoo milk-tea (Xiangpiaopiao Food Co. Ltd.). The samples were firstly dissolved by distilled water by 10g milk-tea powder per 100 ml distilled water. Then the samples were mixed round till the milk-tea powders were totally dissolved. The samples were stored in the laboratory until they reached a constant temperature of 25±1 ºC. A total of 240 samples (60 for each variety) were prepared for spectral analysis. 180 instant milk-tea samples (45 samples for each variety) were randomly selected for the calibration set, and the remaining 60 samples for the validation set. The well-prepared dissolved milk-tea was churned up every 15 minutes to avoid any possible precipitation. 2.2 Spectral Acquisition and Pretreatment The dissolved milk-tea was churned up to make sure that there was no sedimentation at the bottom. For each sample, two reflectance spectra were scanned by a handheld

Determination of Sugar Content of Instant Milk-Tea

895

FieldSpec Pro FR (325–1075 nm)/A110070, Trademarks of Analytical Spectral Devices, Inc. (Analytical Spectral Devices, Boulder, USA). The field-of-view (FOV) of the spectroradiometer is 25°. The light source consists of a Lowell pro-lam interior light source assemble/128930 with Lowell pro-lam 14.5V Bulb/128690 tungsten halogen bulb that could be used both in visible and near infrared region. Instant milktea sample was placed in a cuvette. The spectroradiometer was placed at a height of approximately 100 mm and 45° angle away from the center of sample. The light source was placed at a height of approximately 250 mm 45° angle away from the sample. The reflectance spectra from 325 to 1075 nm were measured at 1.5 nm intervals with an average reading of 30 scans for each spectrum. Two spectra were collected for each sample and the average spectrum of these two measurements was used as the sample spectrum in later analysis. All spectral data were stored in a computer and processed using the RS3 software for Windows (Analytical Spectral Devices, Boulder, USA) designed with a Graphical User Interface. After the spectra acquisition, the sugar content was measured by an Abbebenchtop refractometer (Model: WAY-2S, Shanghai Precision & Scientific Instrument Co. Ltd., Shanghai, China). The refractive index accuracy is ±0.0002 and the °Brix (%) range is 0-95% with temperature correction. The reflectance spectra were firstly transformed into ASCII format by using the ASD ViewSpecPro software (ASD, Boulder, USA). Then two reflectance spectra for each sample were averaged into one spectrum and transformed by log(1/R) into absorbance spectrum. The pretreatments were implemented by “The Unscrambler 9.5” (CAMO AS, Oslo, Norway). After some trial computations, the optimal smoothing way of Savitzky-Golay with 3 segments and zero polynomials was applied to decrease the noise. Standard normal variate (SNV) was applied for light scatter correction and reducing the changes of light path length. The 1st derivative with 3 segments was applied to reduce the base line shift. To avoid low signal-to-noise ratio, only the region of wavelengths (400-1000 nm) were employed for the calculations. 2.3 Partial Least Squares Analysis Partial least squares (PLS) analysis is a commonly used regression method for the prediction of chemical components in the present near infrared spectroscopic analysis. The regression coefficients by PLS analysis could be applied as a way for the selection of effective wavelengths (EWs). The EWs described the features of spectra for the determination of sugar content of instant milk-teas. The regression coefficients were primarily used to validate the effects of different wavelengths (X-variables) in predicting the sugar content (Y). Wavelengths with large absolute regression coefficient values were more significant for prediction of sugar content (Y). According to the regression coefficients, certain EWs were selected as the inputs of LS-SVM models. The predictive performance of the model was evaluated by the following standards: correlation coefficient (r), root mean square error of calibration (RMSEC) and prediction (RMSEP), and bias. Some other standards such as slope and bias should be taken into consideration for distinguishing systematic errors and studying the correlation between the reference and Vis/NIR models. A good model should have high correlation coefficients, low RMSEC, RMSEP and absolute bias values.

896

F. Liu and Y. He

2.4 Least Squares-Support Vector Machine Least squares-support vector machine (LS-SVM) is a state-of-the-art learning algorithm. LS-SVM has a good theoretical foundation and has the capability of dealing with linear and nonlinear multivariate calibration and resolving these problems in a relatively fast way [12]. The details of LS-SVM algorithm could be found in the literature [6,13]. The LS-SVM regression model can be expressed as: n

y ( x ) = ∑ α i K ( x , xi ) + b i =1

where K(x, xi) is the kernel function, αi is the Lagrange multipliers called support value, b is the bias term. Before applying LS-SVM, three crucial problems are required to be solved to obtain a good validation performance. They are the selection of the optimal inputs, proper kernel function and the optimal model parameters. The optimal inputs are the selected LVs and EWs. Radial basis function (RBF) as a nonlinear function is a more compacted supported kernel. RBF is also able to reduce the computational complexity of the training procedure and give a good performance under general smoothness assumptions [14]. Thus, RBF kernel was recommended as the kernel function of LS-SVM in this paper. The parameters of LS-SVM including the regularization parameter gam (γ) and parameter sig2 (σ2) were obtained by a grid search technique with leave-one-out cross validation. The ranges of γ and σ2 within (10-3-103) were set based on experience and previous researches [6,14]. All the calculations were performed using MATLAB 7.0 (The Math Works, Natick, USA). The free LS-SVM toolbox (LS-SVM v 1.5, Suykens, Leuven, Belgium) was applied with MATLAB 7.0 to develop the calibration models.

3 Results and Discussion 3.1 Spectral Features and Statistics of Sugar Content The absorbance spectra of four varieties of instant milk-teas are shown in Fig. 1a. The preprocessed 1st derivative absorbance spectra are shown in Fig. 1b. In Fig. 1a, the trends of the spectral curves were quite similar, but there were some crossovers and overlapping near 500 and 900 nm among different varieties. There were small absorbance peaks near 960 nm for these for varieties. The preprocessed spectra showed the enhanced features of the spectral data such as the peaks and valleys near 500, 650 and 950 nm in the visible and near infrared spectral region. The statistic values of sugar content are shown in Table 1. Table 1. The statistic values of sugar content (°Brix) in calibration and validation sets Data set Calibration Validation All samples

Samples 180 60 240

Range 9.4-10.3 9.4-10.3 9.4-10.3

Mean 9.900 9.882 9.888

Standard deviation 0.303 0.319 0.307

Determination of Sugar Content of Instant Milk-Tea

897

3.2 PLS Models and Analysis of LVs and EWs After the preprocessing of Savitzky-Golay smoothing, SNV and 1st derivative, the PLS models were developed for the determination of sugar content of instant milkteas. Twenty latent variables were calculated and three LVs were optimal to develop the PLS model. The correlation coefficient (r), root mean square error of prediction (RMSEP) and bias for validation set were 0.954, 0.097 and 0.010 for sugar content, respectively. An acceptable prediction precision were achieved by PLS model.

(a)

(b)

Fig. 1. The spectral curves of (a) original absorbance spectra and (b) 1st derivative spectra

After PLS analysis, the total explained variance of sugar content by the first to the top four LVs were 73.2%, 85.9%, 90.8% and 91.6%, respectively. The top three and top four LVs could explain more than 90% variance, so the first fours LVs could represent most of the useful information and the features of the spectral data. In order to obtain the optimal number of LVs, different LVs (2, 3, and 4 LVs) were applied as the inputs of LS-SVM models to predict the sugar content of instant milk-teas seperately. RBF kernel and a two step grid search technique were used for the development of LV-LS-SVM models. According to the regression coefficients, certain wavelengths were selected as effective wavelengths to represent the features of spectral data. There were two principles were used for the selection: (1) the absolute regression coefficient value should be more than 0.1 and (2) the selected wavelengths should be at the peaks or valleys of the regression coefficient curve. According to these principles and regression coefficient curve, five wavelengths were selected as the preliminary effective wavelengths (EWs) including 484, 515, 716, 734 and 957 nm. In the visible region 400-780 nm, the selected effective wavelengths might represent the colour differences and variance which might be caused by the pigment of instant tea powder, flavouring and other ingredients [15]. Wavelengths around 957 nm might be related to the chemical components in instant milk-tea, such as sugar and other chemical constitutions of tea or milk [16]. 3.3 LS-SVM Models According to the analysis of the selected LVs and EWs, the LV-LS-SVM and EWLS-SVM models were developed separately for the determination of sugar content of

898

F. Liu and Y. He

instant milk-teas. Different LVs were used to develop the LS-SVM models. Through the grid search technique, different optimal combinations of (γ, σ2) were obtained. The prediction results of calibration and validation sets are shown in Table 2. The best prediction results were achieved with the first two latent variables. The correlation coefficient (r), root mean square error of prediction (RMSEP) and bias for validation set were 0.957, 0.094 and -6.3×10-4 for sugar content, respectively. In Table 2, the performance of LS-SVM models with 3 and 4 LVs was almost identical in the validation set. The reason was that the first 4 LVs contributed no more useful information than the top 3 LVs for the prediction of sugar content. This was consistent with PLS models which achieved the optimal performance with the first 3 LVs. Comparing these developed models, the performance of LV-LS-SVM model outperformed that of PLS model. Table 2. The prediction results of sugar content by LV-LS-SVM models rb RMSEC/RMSEP LVs a (γ, σ2) 2 (1.2, 1.7) 0.962 0.083 3 (1.9, 2.8) 0.962 0.082 4 (3.3, 8.0) 0.964 0.081 Validation 2 (1.2, 1.7) 0.957 0.094 3 (1.9, 2.8) 0.945 0.106 4 (3.3, 8.0) 0.945 0.106 a LVs: Latent variables; b r: Correlation coefficients. Parameters Calibration

Bias 4.8×10-9 2.7×10-10 -6.8×10-9 -6.3×10-4 -0.009 -0.009

Slope 0.911 0.911 0.916 0.860 0.825 0.825

Offset 0.877 0.881 0.833 1.379 1.723 1.723

Wavelengths at 484, 515, 716, 734, and 957 nm were used as the inputs to develop different EW-LS-SVM models. Wavelengths at 484, 515, 716 and 734 nm were in the visible region and 957 nm was in the short-wave near infrared spectral region. Different EW-LS-SVM models were developed in order to confirm which wavelengths were optimal for the prediction of sugar content. The prediction results for calibration and validation sets are shown in Table 3. In Table 3, the results indicated that wavelengths at 716 and 734 nm were not necessary for the prediction of sugar content. Because the performance of using three EWs (484, 515, and 957 nm) was as good as that of using all five EWs. The combination of 484 and 957 nm or the combination of 515 and 957 nm could achieve similar performance. The performance was also similar with that of using the inputs of 484, 515, and 957 nm. In Fig. 2, wavelength at 484 nm showed a positive regression coefficients with the value larger than 0.1, and 515 nm showed a negative regression coefficients with the absolute value larger than 0.1. Wavelengths at 484 and 515 nm were in the visible region and represent opposite directions of the regression coefficients. Hence, wavelengths at 484 and 515 nm were selected together as the confirmed effective wavelengths. Since the wavelength at 957 nm was in the short-wave near infrared spectral region and it had a direct correlation with the sugar content, wavelength at 957 nm was also selected as the confirmed effective wavelength. Considering the prediction performance for calibration and validation sets, wavelengths at 484, 515, and 957 nm were confirmed as the effective wavelengths for the prediction of sugar content of instant milk-teas. The correlation coefficient (r), RMSEP and bias for validation set were 0.964, 0.087 and

Determination of Sugar Content of Instant Milk-Tea

899

Table 3. The prediction results of sugar content by EW-LS-SVM models Wavelengths (nm)

(γ, σ2)

Calibration 484, 515, 716, 734, 957 (3.2, 7.5) 484, 515, 734, 957 (2.1, 1.4) 484, 515, 716, 957 (17.6, 3.3) 484, 515, 957 (1.6, 0.8) 484, 957 (1.1, 0.6) 515, 957 (1.0, 2.6) Validation 484, 515, 716, 734, 957 (3.2, 7.5) 484, 515, 734, 957 (2.1, 1.4) 484, 515, 716, 957 (17.6, 3.3) 484, 515, 957 (1.6, 0.8) 484, 957 (1.1, 0.6) 515, 957 (1.0, 2.6) a r: Correlation coefficients.

ra

RMSEC/ RMSEP

Bias

Slope

Offset

0.962 0.965 0.967 0.964 0.963 0.960

0.083 0.080 0.077 0.081 0.081 0.085

-6.6×10-9 8.7×10-11 1.1×10-7 -8.9×10-10 -4.1×10-10 6.8×10-12

0.917 0.918 0.930 0.914 0.908 0.907

0.821 0.816 0.691 0.850 0.906 0.924

0.965 0.964 0.963 0.964 0.965 0.965

0.085 0.086 0.086 0.087 0.086 0.085

0.005 0.004 0.005 0.005 0.006 0.007

0.885 0.872 0.903 0.870 0.869 0.873

1.137 1.270 0.966 1.290 1.303 1.266

0.005, respectively. The performance of EW-LS-SVM model outperformed that of LV-LS-SVM and PLS models. Therefore, the selected EWs were correct and they were very helpful for the development of portable instrument for the quality evaluation of instant milk-teas. This was the preliminary step for the development of such quality evaluation instruments of instant milk-teas. In conclusion, the results indicated that Vis/NIR spectroscopy combined with EW-LS-SVM models could be utilized as a fast approach for the determination of sugar content of instant milk-teas.

4 Conclusion Effective wavelengths combined with LS-SVM method was successfully implemented for the determination of sugar content of instant milk-teas. The PLS regression model were developed. The latent variables were selected through the explained variance and effective wavelengths were selected by the regression coefficients. The latent variables were used to develop the LV-LS-SVM models and the first two latent variables achieve the best performance which was better than that of PLS model. Wavelengths at 484, 515, and 957 nm were confirmed to be the effective wavelengths and EW-LS-SVM model showed the best prediction performance in all developed models. The results indicated that Vis/NIR spectroscopy combined with LS-SVM models had the capability to determine the sugar content of instant milk-teas. Furthermore, the selected effective wavelengths would be important for the development of portable instrument for commercial applications of quality evaluation of instant milk-teas.

Acknowledgments This study was supported by 863 National High-Tech Research and Development Plan (2007AA10Z210).

900

F. Liu and Y. He

References 1. Anklam, E., Battaglia, R.: Food analysis and consumer protection. Trends Food Sci. Technol. 12, 197–202 (2001) 2. Yan, Y.L., Zhao, L.L., Han, D.H., Yang, S.M.: The foundation and application of near infrared spectroscopy analysis. China Light Industry Press, Beijing (2005) 3. Borin, A., Ferrão, M.F., Mello, C., Maretto, D.A., Poppi, R.J.: Least-squares support vector machines and near infrared spectroscopy for quantification of common adulterants in powdered milk. Anal. Chim. Acta 579, 25–32 (2006) 4. Moros, J., Garrigues, S., De la Guardia, M.: Evaluation of nutritional parameters in infant formulas and powdered milk by Raman spectroscopy. Anal. Chim. Acta 593, 30–38 (2007) 5. Moros, J., Iñón, F.A., Garrigues, S., de la Guardia, M.: Determination of the energetic value of fruit and milk-based beverages through partial-least-squares attenuated total reflectance- Fourier transform infrared spectrometry. Anal. Chim. Acta 538, 181–193 (2005) 6. Chen, Q.S., Zhao, J.W., Fang, C.H., Wang, D.M.: Feasibility study on identification of Green, Black and Oolong teas using near-infrared reflectance spectroscopy based on support vector machine (SVM). Spectrochim. Acta Part A 66, 568–574 (2007) 7. Li, X.L., He, Y., Wu, C.Q., Sun, D.W.: Nondestructive measurement and fingerprint analysis of soluble solid content of tea soft drink based on Vis/NIR spectroscopy. J. Food Eng. 82, 316–323 (2007) 8. Zhang, M.H., Luypaert, J., Fernández Pierna, J.A., Xu, Q.S., Massart, D.L.: Determination of total antioxidant capacity in green tea by near-infrared spectroscopy and multivariate calibration. Talanta 62, 25–35 (2004) 9. Ferruzzi, M.G., Green, R.J.: Analysis of catechins from milk-tea beverages by enzyme assisted extraction followed by high performance liquid chromatography. Food Chem. 99, 484–491 (2006) 10. Bai, S.Z., Chen, B.G., Zhang, L., Zhang, S.Y.: FAAS determination of iron in milk tea powder. PTCA Part B: Chemical Analysis 40, 648–649 (2004) 11. Liu, F., Ye, X.J., He, Y., Wang, L.: Application of visible/near infrared spectroscopy and chemometric calibrations for variety discrimination of instant milk teas. J. Food Eng. 93, 127–133 (2009) 12. Suykens, J.A.K., Vanderwalle, J.: Least squares support vector machine classifiers. Neural Process. Lett. 9, 293–300 (1999) 13. Wang, W.J., Xu, Z.B., Lu, W.Z., Zhang, X.Y.: Determination of the spread parameter in the Gaussian kernel for classification and regression. Neurocomputing 55, 643–663 (2003) 14. Belousov, A.I., Verzakov, S.A., von Frese, J.: Applicational aspects of support vector machines. J. Chemometr. 16, 482–489 (2002) 15. Somers, C.: The wine Spectrum: An Approach towards Objective Definition of Wine Quality. Winetitles, Adelaide (1998) 16. Šašić, S., Ozaki, Y.: Short-wave near-infrared spectroscopy of biological fluids. 1. quantitative analysis of fat, protein, and lactose in raw milk by partial least-squares regression and band assignment. Anal. Chem. 73, 64–71 (2001)

Sports Video Summarization Based on Salient Motion Entropy and Information Analysis Bo-Wei Chen1, Jhing-Fa Wang1, Jia-Ching Wang2, and Chen-Yu Chen3 1 Department of Electrical Engineering, National Cheng Kung University, Taiwan, R.O.C. [email protected] 2 Department of Computer Science and Information Engineering, National Central University, Taiwan, R.O.C. 3 Innovative Digitech-Enabled Applications & Services Institute, Institute for Information Industry, Taiwan, R.O.C.

Abstract. In this study, we presented a novel summarization method for generating sports video abstracts, which utilized motion entropy analysis and mutual information. Both of them are based on an attentive model. In order to capture and detect significant segments among a video, we exploited saliency maps by calculating color contrast, intensity contrast, and orientation contrast of frames. In the next step, motion vectors between maps were computed and converted into salient motion entropy. Meanwhile, a new algorithm based on mutual information was proposed to improve the smoothness problem when we selected boundaries of segments. The experiments showed that our proposed algorithm could not only detect highlights effectively but also generate smooth playable clips. Compared with the traditional approaches, our system improved the precision by 7.6% and enhanced smoothness by 1.2, which also verified feasibility of our system. Keywords: Video summarization, motion entropy analysis, salient motion entropy, information analysis.

1 Introduction Traditional video summarization approaches [1]–[11] can be roughly classified into two categories: One is the static storyboard [2], [8], [11], which is composed of still images extracted from the original video; the other is the dynamic skimming [1], [3]– [7], [9], [10], which concatenates several shorter clips. Both of them aim to offer users a compact view of a video. In this work, we mainly focus on the study of dynamic skimming approaches because generating playable clips is suitable for users to navigate sports videos. To date, a large amount of research has been proposed to analyze sports videos, including colors, motion vectors [1], [6], [10], [12], saliency maps [13], [14], and so forth. However, although these systems are capable of capturing conspicuous shots among videos, it is still difficult to detect genuine events and remove false alarms. For example, Walther et al. [13], [14] attempted to highlight objects in an image by using Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 901–908. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

902

B.-W. Chen et al.

low-level visual features and generating saliency maps. Like many other color-based approaches, they usually need heuristic rules for filtering unnecessary frames. On the other hand, Yeo and Liu [1] focused on another features and balanced this problem via estimating motion vectors. Nevertheless, it is susceptible to global motion changes, such as camera panning and zooming, which may cause false positive results. In order to solve such problems, we employ a compromised way to combine two merits of the above algorithms for summarizing sports video events. Therefore, in this work, we endeavor to provide: A novel approach based on salient motion entropy analysis, which joins the merits of saliency models and motion descriptors; a new mutual information estimator, that exploits salient motion vectors, for detecting similar frames. With the use of the proposed techniques, we can decrease false positives caused by conventional motion analyses and create a better navigation processing for users. The remainders of this study are organized as follows: Section 2 reviews the related techniques. Section 3 elaborates the details of our proposed methods. We exhibit the performance of our system and the experimental results in Section 4. Section 5 summarizes the conclusions of this study.

2 Related Work To date, a great amount of research has devoted to generating dynamic video abstracts, using various kinds of approaches. In [10] You et al. employed two types of models, “global motion model” and “local motion model,” to distinguish camera events from sports events. Ma et al. [6] made use of different features, such as motions, color, and other media descriptors, fusing them into a curve. Such curves are often addressed as attentive curves; they can offer developers numeric conversion of images. In 2004, Cernekova et al. [7] adopted another viewpoint, modeling the shot and scene change by detecting mutual information between frames. There were also some approaches [5], [6] that exploited a more sophisticated way to extract highlights. For example, the authors [6] also utilized content-based parsing techniques to detect face regions. However, the major drawback is that it requires a recognition system and involves a training process. In addition to the aforementioned methods, heuristic rules are often used as a kind of criteria for identifying event patterns. In [5], the authors presented several feature extraction methods, including wavelet-based motion trend analyses, hybrid field-color models, and prior knowledge-driven line detection. Liu et al. [3] has devised a perceived motion model for key-frame extraction. They came up with a triangular modeling rule to reshape the generated attentive curves. Generally speaking, finding an effective way to transform video data into numeric curves is the major challenge in dynamic skimming. Accordingly, in this study, we aim to provide another feasible solution to this issue.

3 Proposed Method The proposed system consists of three stages, of which the first one is the saliency map extraction, the second computes motion entropy between frames, and the last

Sports Video Summarization Based on Salient Motion Entropy and Information Analysis

903

stage calculates mutual information based on salient motions. The output is the concatenated playable clips. 3.1 Saliency Map Extraction We begin our description of the saliency map extraction with a short introduction. A saliency map is a kind of image that can point out the visual regions, which have the most perceptual impact on human brains. For these years, this technique has been intensively and broadly studied, and it has a wide range of applications [6], [13], [14]. The idea behind the algorithm is to provide visual perceptron analysis by using color contrast, intensity contrast, and orientation contrast in an image. In this study, the main reason we adopted such techniques in our system is that by employing saliency map extraction, unnecessary global motions could be removed, and this can avoid detecting false highlights in videos. The following are the key-points of the saliency map according to the authors’ research [13]: After we extract each frame, the image is subsampled into a Gaussian pyramid and then decomposed into several channels for red(R), green(G), blue(B), yellow(Y), intensity(I), and local orientation( Oθ ). From these channels, two feature maps, “center c” and “surround s” are constructed and normalized:

(

FI , c , s = N I ( c )

)

I (s) .

(1)

FRG ,c , s = N

( ( R ( c ) − G ( c )) ( R ( s ) − G ( s )) ) .

(2)

FBY ,c , s = N

( ( B ( c ) − Y ( c )) ( B ( s ) − Y ( s )) ) .

(3)

(

Fθ , c , s = N Oθ ( c )

)

Oθ ( s ) .

(4)

Here, the operator means the across-scale difference between two feature maps; N ( ⋅) is an iterative, nonlinear normalization operator. A saliency map could be obtained by averaging these functions at this stage.

Fig. 1. Example of a saliency map. The left figure is the original image, whereas the right one is its saliency map.

3.2 Salient Motion Entropy After extracting saliency maps of video frames, in the next step, we utilize these saliency maps to calculate motion entropy. As mentioned in the previous section, despite

904

B.-W. Chen et al.

dominant motion features were widely used for key-frame identification in sport videos, we observed that such techniques were subject to global camera motion effects. Therefore, motion entropy based on the saliency maps is proposed to alleviate those problems in this system. Motion entropy has been proved to be effective in rejecting false alarms and efficient in computation (see our previous work [9]). The major difference between motion entropy and salient motion entropy is that the latter is operated in the saliency-map domain. Salient motion entropy is given as follows:

H Saliency = ∑ wk × hk

(5)

k

where k is the index of the sector after we equally partition the polar axis into segments; hk (motion directivity entropy) is the total entropy that belongs to the corresponding sector, and wk is the total weights in the sector. Let pk represent the proportion of motion vectors, whose angles fall into the kth sector, and we could define hk as

hk = − pk × log pk .

(6)

It describes the activity rate of a frame when it is converted to a saliency map; the strength of directivity entropy is calculated by

wk = ∑ ri , k i

where

∑r

i i ,k

∑r

(7)

i

i

is the sum of the lengths of the motion vectors in the kth region, and

the denominator means the sum of the total lengths.

(a)

(b)

Fig. 2. Curves of the proposed method and the other ones. The horizontal axis is the time index, and the vertical is the score. We use the solid line to represent proposed curve, whereas the dotted line means the other method. The marked place (dashed line) in the figure represents a motional event.

3.3 Mutual Information Based on Salient Motions

Once a highlighted frame is detected in a video, we have to determine the boundaries of an event. Thus, playback would become smooth when users watch it. A simple way to estimate the boundaries of an event is by using color information between frames,

Sports Video Summarization Based on Salient Motion Entropy and Information Analysis

905

such as [7]. The authors presented a novel concept of applying mutual information for detecting similar images. Although it may work in movies, some problems still arise in analyzing sports videos. This is because color features in different scenes among sports videos do not change a lot. Therefore, we turn to employ motion data by modifying the original equation into:

MI tSaliency = MI tActiveSaliency − MI tInactiveSaliency , t +1 , t +1 , t +1 = ^ t ,t +1 × log

^ t ,t +1 ^ Active ^ Active t t +1

− ^′t ,t +1 × log

^′t ,t +1

(8)

^ Inactive ^ Inactive t t +1

where ^ t ,t +1 denotes the proportion of the motion vectors, which remain as active (directional) states from one frame to the next, whereas ^′t ,t +1 represents inactive ones; ^ Active or ^ Inactive calculates the percentage of the directional or undirectional vectors t t in a single frame respectively. Both of the two terms in the equation measure the coherent status between consecutive frames. The higher mutual information is, the more coherent the frames are. With the use of this equation, we may find the start and the end of a motional event efficiently because a sports event usually comes along with significant motion activity changes.

Fig. 3. Mutual information based on salient motions. It mainly computes the coherent status between consecutive frames. The higher mutual information is, the more coherent the frames are. In this figure, we detect two segments that have relatively higher coherence in their time duration respectively. Meanwhile, two significant changing points are also detected, based on the same method.

4 Experimental Results In order to assess the performance of our system, we collected six videos, which were listed in Table 1. They contained several highlights, including corner kicks, goalkeeper shots, block tackles, goal-and-cheer shots, hits, homeruns, double plays, etc, which were labeled by humans. Besides, unnecessary segments, such as commercial parts, anchor shots, and captions, were removed from the videos. We employed the average precision as the criterion for evaluation.

906

B.-W. Chen et al. Table 1. Testing video dataset Video number I II III IV V VI

Genre Soccer Soccer Soccer Soccer Baseball Baseball

Length 110 minutes 120 minutes 98 minutes 80 minutes 100 minutes 95 minutes

Number of highlights 25 16 20 28 15 16

Table 2 lists the experimental results of the precision and recall. As shown in this table, our results outperform the other methods. The precision score is increased to 85.1% on average, and it has been improved by up to 7.6%. The recall score also reflects the same phenomenon. In contrast to the other methods, the proposed one has demonstrated its efficacy of detecting highlights. Table 2. Evaluation results Method Saliency map (Walther et al. [14]) Motion activity (Yeo and Liu [1]) Motion entropy (Chen et al. [9]) Proposed

Precision 77.5% 80.3% 81.0% 85.1%

Recall 73.2% 75.9% 79.5% 84.8%

In order to benchmark the smoothness of selected segments against the results of the baseline, we also invited ten persons, and each of them was asked to give a score (integer) ranging from 1 (worse) to 3 (better). The evaluation results are shown in Table 3. We observed that the average scores of our system are above 2.0, whereas those of the baseline are less than 1.5. Such results imply that our algorithm has better ability to generate smooth segments than the baseline does. Table 3. Smoothness test. After watching video summaries, each user can give a score (integer), which ranges from 1 (worse) to 3 (better), to each system. Video number I II III IV V VI Average

Yeo and Liu [1] 1.2 1.4 1.4 1.0 1.4 1.4 1.3

Proposed 2.2 2.6 2.2 2.8 2.6 2.6 2.5

5 Conclusion In this work, we introduced a new approach for summarizing sports videos. In order to detect the highlights in a video, salient motion entropy was employed by

Sports Video Summarization Based on Salient Motion Entropy and Information Analysis

907

combining saliency maps with motion entropy. Another event-boundary detection algorithm was also proposed to search similar frames in a video, which was based on mutual information. With the above-mentioned methods, our system is capable of generating video abstracts efficiently. Besides, one of the advantages is that it can decrease the searching range for event recognition systems. This is because using this highlight detecting scheme can mark out candidate segments first without inputting a whole video. Compared with the traditional methods, our proposed system managed to increase the precision rate by 7.6% and smoothness by 1.2 respectively. The experiments also verified the performance of our system and proved the feasibility of the algorithm.

Acknowledgment This work was supported in part by the National Science Council of the Republic of China under Grant NSC97-2221-E-006-249-MY3. We would like to thank C.-Y. Chen for his system; it was conducted under the “Applied Information Services Development & Integration Project — Phase II” of the Institute for Information Industry which was subsidized by the Ministry of Economy Affairs of the Republic of China.

References 1. Yeo, B.-L., Liu, B.: Rapid Scene Analysis on Compressed Video. IEEE Trans. Circuits and Systems for Video Technology 5, 533–544 (1995) 2. Bagga, A., Hu, J., Zhong, J., Ramesh, G.: Multi-Source Combined-Media Video Tracking for Summarization. In: Proc. 16th IEEE Int. Conf. Pattern Recognition, pp. 818–821 (2002) 3. Liu, T., Zhang, H.-J., Qi, F.: A Novel Video Key-Frame-Extraction Algorithm Based on Perceived Motion Energy Model. IEEE Trans. Circuits and Systems for Video Technology 13, 1006–1013 (2003) 4. Li, Z., Schuster, G.M., Katsaggelos, A.K.: Minmax Optimal Video Summarization. IEEE Trans. Circuits and Systems for Video Technology 15, 1245–1256 (2005) 5. Liu, T.-Y., Ma, W.-Y., Zhang, H.-J.: Effective Feature Extraction for Play Detection in American Football Video. In: Proc. 11th Int. Multimedia Modelling Conference, pp. 164– 171 (2005) 6. Ma, Y.-F., Hua, X.-S., Lu, L., Zhang, H.-J.: A Generic Framework of User Attention Model and Its Application in Video Summarization. IEEE Trans. Multimedia 7, 907–919 (2005) 7. Cernekova, Z., Pitas, I., Nikou, C.: Information Theory-Based Shot Cut/Fade Detection and Video Summarization. IEEE Trans. Circuits and Systems for Video Technology 16, 82–91 (2006) 8. Taskiran, C.M., Pizlo, Z., Amir, A., Ponceleon, D., Delp, E.J.: Automated Video Program Summarization Using Speech Transcripts. IEEE Trans. Multimedia 8, 775–791 (2006) 9. Chen, C.-Y., Wang, J.-C., Wang, J.-F., Hu, Y.-H.: Event-Based Segmentation of Sports Video Using Motion Entropy. In: Proc. 9th IEEE Int. Symp. Multimedia, pp. 107–111 (2007)

908

B.-W. Chen et al.

10. You, J., Liu, G., Sun, L., Li, H.: A Multiple Visual Models Based Perceptive Analysis Framework for Multilevel Video Summarization. IEEE Trans. Circuits and Systems for Video Technology 17, 273–285 (2007) 11. Chen, B.-W., Wang, J.-C., Wang, J.-F.: A Novel Video Summarization Based on Mining the Story-Structure and Semantic Relations among Concept Entities. IEEE Trans. Multimedia 11, 295–312 (2009) 12. Black, M.J.: The Robust Estimation of Multiple Motions: Parametric and PiecewiseSmooth Flow Fields. Computer Vision and Image Understanding 63, 75–104 (1996) 13. Walther, D., Rutishauser, U., Koch, C., Perona, P.: Selective Visual Attention Enables Learning and Recognition of Multiple Objects in Cluttered Scenes. Computer Vision and Image Understanding 100, 41–63 (2005) 14. Walther, D., Koch, C.: Modeling Attention to Salient Proto-Objects. Neural Networks 19, 1395–1407 (2006)

A Neural Network Based Algorithm for the Retrieval of Precipitable Water Vapor from MODIS Data Shenglan Zhang, Lisheng Xu, Jilie Ding, Hailei Liu, and Xiaobo Deng Atmospheric Radiation & Satellite Remote Sensing Lab, College of Atmospheric Sounding, Chengdu University of Information Technology, Chengdu, China 610225 [email protected]

Abstract. A neural network (NN) based algorithm for retrieval of precipitable water vapor (PWV) from the Moderate Resolution Imaging Spectroradiometer (MODIS) radiance is proposed. A multilayer feedforward neural network (MFNN) is selected, in which the at-sensor brightness temperature, the surface emissivity of MODIS chs. 31 and 32, and the land surface temperature (LST) are input variables, and PWV is the output variable. The input parameters for the MFNN are mainly based on the radiative transfer simulation with MODTRAN 4.0 code and the latest global assimilation data. The algorithm is applied to retrieval of the PWV over northeast area in china using MODIS data. Compared with the MODIS PWV products, the RMSE of the PWV retrieved by our algorithm is 0.45g/cm2. Furthermore, a comparison of our retrieval PWVs with radiosonde data is carried out. The results show that the MFNN-based retrieval algorithm for PWV is robust and efficient. Keywords: Precipitable water vapor, Retrieval, Neural network, MODTRAN, MODIS.

1 Introduction Water vapor is widely recognized to be a key climate variable, and has an important influence on global climate change [1-2], which makes it necessary to know the distribution and transport of water vapor. Knowledge of water vapor is needed to further understanding of the energy balance and water cycle at continental or global scales. PWV is the total atmospheric water vapor contained in a vertical column of unit area from the earth`s surface to the top of atmosphere [3], which is highly variable in spatio-temporal in the atmosphere. PWV is also an important input variable to determine surface parameters from satellite observations. Many efforts have been invested in the study of water vapor in the atmosphere, especially from the science communities such as meteorology, hydrology and climatology [4]. However, conventional radiosonde observation could not satisfy the needs for water vapor data due to its poor coverage and limited spatial representative. To overcome this shortcoming, numerous methods have been proposed to estimate the PWV from satellite sensor data [5-6]. According to the wavelength used, those approaches may be grouped into three categories: near-infrared techniques, passive microwave Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 909–916. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

910

S. Zhang et al.

techniques, and thermal infrared techniques [7]. Dalu [8] computed the PWV related to brightness temperature differences at 11 μm and 12 μm using radiative transfer model simulations. Kleespies and McMillin [9] proposed a split window technique establishing an inverse relationship between the transmittances of two different infrared channels and PWV, by assuming that the atmosphere and surface emissivities in these channels are invariant. Many extensions of this technique [10] were proposed with many doubts. In this paper, a multilayer feedforward neural network (MFNN) based algorithm is proposed to derive PWV from thermal infrared radiance measured by MODIS.

2 Algorithm Generally speaking, the ground is not a blackbody, thus ground emissivity has to be considered for computing the thermal radiance emitted by the surface. Also atmosphere has important effects on the received radiance at remote sensor level. Considering all these impacts, for a cloud-free atmosphere under local thermodynamic equilibrium, the radiative transfer equation [11] describing the radiation intensity observed in channel i at zenith angle θ can be formulated as: Ii(θ) = Bi(Ti) = τi(θ) εi(θ) Bi(Ts) + Ii↑(θ) + τi(θ) (1 - εi(θ)) Ii↓(θ) .

(1)

Where TS is the land surface temperature (LST). Ti(θ), τi(θ) and εi(θ) are the at-sensor brightness temperature, the atmospheric transmittance and the ground emissivity in channel i and at zenith angle θ, respectively. Bi(Ts) is the ground radiance in channel i, Ii↑(θ) and Ii↓(θ) are the upwelling and downwelling atmospheric radiances in channel i at zenith angle θ, respectively. In equation (1), τi↑(θ), Ii↑(θ) and Ii↓(θ) are generally called atmospheric effect parameters [12]. These atmospheric parameters have a close and complicated relationship with PWV, so the estimation of PWV from the measured radiance is an inverse problem relating PWV to a set of Ti(θ), TS and εi(θ) through a function F: PWV = F( Ti(θ), Ts, εi(θ) ) .

(2)

Highly nonlinear dependency of PWV on Ti(θ), TS and εi(θ) in radiative transfer limits the accuracy of those previous methods. However, it should be addressed that the neural network is one of the best methods to solve this problem, which can account for the nonlinear dependency of PWV on Ti(θ), TS and εi(θ) and inherent nonlinear changes of surface reflectance with wavelength. Many studies have proved the capabilities of neural networks based algorithm. Thus, a MFNN based algorithm is proposed in this study.

3 Methodology In this section, we use the radiative transfer model (MODTRAN 4.0) to simulate data and the MFNN algorithm to retrieve PWV.

A Neural Network Based Algorithm for the Retrieval of Precipitable Water Vapor

911

3.1 Neural Network Neural network, with their remarkable ability to derive meanings from complicated or imprecise data, can be used to extract patterns and detect trends [13]. A neural network is an interconnection of simple computational elements, with activation functions that are usually nonlinear, monotonically increasing, and differentiable. Neural networks are able to deduce input-output relationships directly from the training ensemble without requiring underlying assumptions about the distribution of the data. Furthermore, a neural network with only a single hidden layer of a sufficient number of nodes with nonlinear activation functions is capable of approximating any realvalued continuous scalar function to a given precision over a finite domain [14]. 3.2 Training and Testing of the MFNN A MFNN based algorithm is presented to derive transfer functions F relating Ti(θ), TS and εi(θ) (inputs) to PWV (output). To investigate the configuration which yielded a minimum root mean square error (RMSE), a MFNN with five neurons in input layer, two hidden layers each containing fifteen neurons and one output neuron for PWV is considered. The architecture of the MFNN is shown in Fig. 1.

Fig. 1. The architecture of the MFNN

In this study, we use MODTRAN 4.0 to simulate the radiance transfer model for MODIS channels 31 and 32. And 6757 atmospheric profile sets including geopotential height, air temperature, and humidity are carefully selected from monthly mean products in January and July from 2000 to 2007 provided by the NCEP climate data assimilation system (CDAS) reanalysis project. Finally, the variables for training and testing the MFNN are obtained, and 18 000 pairs of input-output patterns for training and testing are selected from 851 382 pairs of the data, which are split into both the training data with 15000 patterns and the testing data with 3000 patterns. As shown in Fig. 2, the retrieval result is very good with the RMSE of 0.36g/cm2. The PWV retrieved by the MFNN is close to the PWV from the NCEP climate data.

912

S. Zhang et al.

4.5

RMSE=0.36 Water vapor content retrieved by NN

4 3.5 3 2.5 2 1.5 1 0.5 0

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

Water vapor content (g/cm2)

Fig. 2. PWV from the NCEP climate data versus the estimation using a MFNN based algorithm

4 Validation and Discussion In this section, we validated the efficiency of the MFNN based algorithm by using both the MODIS data and the radiosonde data, and discussed the influence of the LST and the ground emissivity over the accuracy of the PWV retrieval algorithm. 4.1 Validation of the MFNN Algorithm Using MODIS Data In order to validate the MFNN algorithm, which has been trained to retrieve the PWV from the simulated data, a set of independent data from MODIS products is employed. One MODIS image in northeast of China on 4 October 2007 from 03:00:00 to 03:05:00 GMT is selected for the study. The inputs are Ti(θ), εi(θ) (i=31, 32) and TS, and the output is the PWV. A comparison between the retrieved PWV and MOD05_L2 is worked out, as shown in Figs. 3 and 4. Fig. 3 (a) is the retrieved results by MLP-NN, and (b) is the MOD05_L2 PWV products. Fig. 4 depicts the differences between the PWV retrieved by the MFNN and the MOD05_L2 PWV products. Obviously, most retrieved results by the MFNN are close to the MODIS PWV products, and the RMSE of this algorithm is 0.45g/cm2. 4.2 Validation of the MFNN Algorithm Using Radiosonde Data The radiosonde data are gotten from the University of Wyming. Here only one station data (54161 ZYCC Changchun Observations at 00Z 04 Oct 2007) are selected. The PWV results retrieved by our MFNN algorithm, from the corresponding MODIS products and the radiosonde data are 2.50g/cm2, 2.20g/cm2 and 2.31g/cm2, respectively.

A Neural Network Based Algorithm for the Retrieval of Precipitable Water Vapor 44

3

44

3 *ZYCC(Satation ID 54161)

2.5

2.5

2

2

1.5

1

Latitude , degrees

Latitude , degrees

*ZYCC(Satation ID 54161)

43

43

1.5

1

0.5

0.5

42 124

125 126 Longitude, degrees

127

913

0

42 124

125 126 Longitude, degrees

(a)

127

0

(b)

Fig. 3. (a) PWV retrieved by MFNN. (b) MODIS PWV product.

44

1.5 * ZYCC(Satation number 54161) 1

Latitude , degrees

0.5

43

0

-0.5

-1

42 124

125 126 Longitude, degrees

127

-1.5

Fig. 4. The differences between the PWV retrieved by the MFNN and the MODIS PWV product

Generally, despite some little non-coincidence, the retrieved results accord well with both the corresponding MODIS products and the radiosonde data, indicating that the MFNN based algorithm for retrieving PWV is a valid method. 4.3 Discussion Some analysis has been done in order to make a comprehensive evaluation of the sensitivity of the algorithm to LST. The brightness temperatures of MODIS channels 31 and 32 are simulated using MODTRAN 4.0 assuming the constant emissiivty of 0.98

914

S. Zhang et al.

and the LST of 300K. As shown in Fig. 5 the accuracy of the algorithm is affected by LST, and the PWV retrieval error decreases obviously with the decrease of the LST retrieved error. The RMSE of PWV retrieved by the MFNN is of 0.4~0.5g/cm2 when the RMSE of the LST achieves 1K. However, when the RMSE of the LST increases to 2K, the algorithm trends to unacceptable with the RMSE of 0.6~0.9g/cm2. Thus, the exact retrieval of the PWV requires the LST retrieval error within a RMSE of 1K. Also another analysis has been done in order to verify the necessary of the ground emissivity εi(θ) for the MFNN based algorithm. In this section, we use the same method as in 3.2, but we only select three variables (Ti(θ)(i=31,32), TS) as inputs without εi(θ). Fig. 6 shows the performance of the three-input MFNN with the RMSE 1 0.9 0.8

PWV error (g/cm2)

0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 -2

-1.5

-1

-0.5

0 0.5 LST error (in K)

1

1.5

2

Fig. 5. The change of PWV error with the change of LST error simulated by MODTRAN 4.0 under the mid-latitude summer atmosphere

Water vapor content retrived by NN (g/cm2)

5

RMSE = 0.47 linear polynomial

4.5 4 3.5 3 2.5 2 1.5 1 0.5 0 0

1

2

3

4

5

Water vapor content (g/cm2)

Fig. 6. Scatter plot of PWV from the NCEP climate data versus the estimation using a three-input MFNN based algorithm

A Neural Network Based Algorithm for the Retrieval of Precipitable Water Vapor

915

of 0.47g/cm2. As we see, the accuracy of three-input MFNN without ground emissivity as inpust is much worse than the upper five-input MFNN. So the ground emissivity should be taken into great consideration in the retrieval of PWV.

5 Conclusion It is known that the relationship between PWV and the atmospheric parameters, and surface properties in radiative transfer is complex with highly nonlinear dependency, which limits the accuracy of those commonly used methods. In this paper we utilize neural network technique combined with radiative transfer calculations to study this problem. A new algorithm based on MFNN is proposed to derive the transfer functions F relating Ti(θ), TS and εi(θ) (inputs) to PWV (output). The radiance transfer model MODTRAN 4.0 is employed to MODIS channels 31 and 32, with the NCEP climate data assimilation system (CDAS) reanalysis products being used. The simulation retrieval result is good with the RMSE of 0.36g/cm2. The validation of the MFNN based algorithm is performed. The retrieved results by the MFNN based algorithm are compared with both the MOD05_L2 PWV products and the radiosonde observation data, respectively. For the former the RMSE is 0.45g/cm2; for the latter the result also accords well, indicating that the proposed PWV retrieval algorithm is promising.

Acknowledgements The data used in this study are acquired from National Centers for Environment Prediction reanalysis project, the ASTER Spectral Library, U.S. Geological Survey, and the University of Wyoming.

References 1. Anthony, D., Genio, D.: The dust settle on water vapor feedback. Science 296, 665–666 (2002) 2. Raval, A., Ramanathan, V.: Observational determination of the greenhouse effect. Nature 342, 758–761 (1989) 3. Akatsuka, S., Takeuchi, W., Sawada, H.: Estimation of Precipitable Water distribution over northeast Asia using NOAA AVHRR. In: Asia Conference on Remote Sensing 2008, Colombo, Sri Lanka (2008) 4. Wang, W., Sun, X., Zhang, R., Li, Z., Zhu, Z., Su, H.: Multi-layer perception neural network based algorithm for estimating precipitable water vapor from MODIS NIR data. Int. J. Remote Sensing 27, 617–621 (2006) 5. Sobrino, J.A., El Kharraz, J., Li, Z.L.: Surface temperature and water vapour retrieval from MODIS data. Int. J. Remote Sensing 24, 5161–5182 (2003) 6. Chrysoulakis, N., Kamarianakis, Y., Xu, L., Mitraka, Z., Ding, J.: Combined use of MODIS, AVHRR and radiosonde data for the estimation of spatio-temporal distribution of precipitable water. J. Geophys. Res. 113, D05, 101 (2008)

916

S. Zhang et al.

7. Li, Z.L., Li, J., Su, Z.B., Wan, Z.M., Zhang, R.H.: A new approach for retrieving precipitable water from ATSR2 split-window channel data over land area. Int. J. Remote Sensing 24, 5095–5117 (2003) 8. Dalu, G.: Satellite remote sensing of atmospheric water vapour. Int. J. Remote Sensing 7, 1089–1097 (1986) 9. Kleespies, T.J., McMillin, L.M.: Retrieval of Precipitable Water from Observations in the Split-Window over Varying Surface Temperatures. J. Appl. Meteorol. 29, 851–862 (1990) 10. Sobrino, J.A., Li, Z.L., Stoll, M.P., Becker, F.: Improvements in the Split-Window Technique for Land Surface Temperature Determination. IEEE Trans. Geosci. Remote Sens. 32, 243–253 (1994) 11. Otlle, C., Stoll, M.: Effect of atmospheric absorption and surface emissivity on the determination of land temperature from infrared satellite data. Int. J. Remote Sensing 14, 2025– 2037 (1993) 12. Tonooka, H.: Accurate Atmospheric Correction of ASTER Thermal Infrared Imagery Using the WVS Method. IEEE Trans. Geosci. Remote Sens. 43, 2778–2792 (2005) 13. Hsu, S., Masters, T., Olson, M., Tenorio, M., Grogan, T.: Comparavtive analysis of five neural networks models. Remote Sens. Rev. 6, 319–329 (1992) 14. Hornik, K.M., Stinchcombe, M., White, H.: Multilayer feedforward networks are universal approximators. Neural Netw. 4, 359–366 (1989)

A Neural Network Based Approach to Wind Energy Yield Forecasting Piers R.J. Campbell, Faheem Ahmed, Haydar Fathulla, and Ahmad D. Jaffar College of Information Technology, UAE University, P.O. Box 17551, Al Ain, UAE {P.Campbell,F.Ahmed,HFathulla,AJaffar}@uaeu.ac.ae

Abstract. It is commonly acknowledged that wind energy is the leading renewable energy generation method; currently producing a power yield equivalent to 35 GW, with an estimated projection of 40-60 GW by 2012. In order to successfully integrate wind energy with traditional generation supplies it is necessary to have the ability to accurately forecast the available yield of a wind installation during a period of time. In this paper we present a neural network based estimation tool which produces wind speed estimates for a given wind installation. These predications are subsequently used in industry standard calculations to produce an energy yield estimate for the wind installation over a given time period. The proposed approach produces forecasts that can be used for two main purposes; firstly, delivery of wind (energy) yield estimations and secondly to assess the suitability of a given location for development into a wind park site. The tool makes use of a Multi-layered Perceptron which has been trained with historical data to produce a set of predicted wind speed data for a given period. This data is then processed in conjunction with independent variables, including Turbine Generator type and altitude to give an estimated power yield and expected uncertainty of the forecast (in terms of percentage capacity factor). Our results indicate that by using such a neural network approach the accuracy of the tool is sufficiently accurate to for short to medium estimations and could prove a valuable tool for wind energy producers and utility operators. Keywords: Neural Network, Renewable Energy, Back Propagation.

1 Introduction Wind energy is increasing in popularity throughout the world and particularly in the UK and Ireland due to increased assistance from government. In Europe a target of 22% of power generation coming from renewable energy technology has been set, and currently wind energy is seen as the most viable method for generating the necessary power. In order to successfully integrate wind energy with traditional generation supplies it is necessary to have the ability to accurately forecast the available yield of a wind park over a given period. In order to achieve this goal it is essential to have a method of estimating firstly, the amount of wind which can be expected over the short to medium term. Secondly, a reliable process for approximating the level of power Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 917–924. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

918

P.R.J. Campbell et al.

available in the wind and finally, a means to derive the energy which can be produced by Wind turbine Generators, given the expected wind regime. This paper presents a wind power and subsequently an energy yield forecast tool which is based on a multilayered perceptron to satisfy these requirements.

2 Neural Network Design A 25 WTG (Wind Turbine Generator) park was selected as the test site for the modelling and testing process. This site was selected because it is one of the longest operational sites in Ireland and so provides a large dataset. The dataset is collected centrally at the site and holds information on every aspect of site operation including performance and mechanical operation. This study focuses solely on the wind speed data collected by each Wind Turbine Generator on a 10 minute interval. These recorded wind speeds are then computed to produce an average monthly wind speed per WTG and across the site. The study focuses on predicting the average monthly wind speed per WTG. From the dataset a subsection of data was selected for the years 1996 to 2006. Data between the years 1996-2005 was used as the training and testing set for the neural network. While the data for 2006 was used as the verification set for the validation of the network model. Data is collected at ten minute intervals by each individual WTG, however, the study focused at a higher level of granularity, by modelling monthly wind speed values. To predict the wind speed a 4 layer model (Figure 1) was used, which consisted of an Input layer, 2 hidden layers and an output layer. A number of different network architectures were tested including a common engineering approach previously employed by Kalogirou et al [1] however, the 4 layer design proved to be the most efficient and accurate for this application. As with the overall network design a range of activation functions were tested, these included; Sigmoid, Gaussian, Hyperbolic Tangent and Hyperbolic Secant. After analysis it was discovered that the network operated most efficiently using the activation functions shown in Table 1. Table 1. Network Activation Functions Layer

Layer Type

Nodes

1

Input

3

Activation Function Linear

2

Hidden

2

Sigmoid

3

Hidden

2

Gaussian

4

Output

1

Sigmoid

The input data, comprised wind data values collected at each WTG. These are, the month of the year, and the mean monthly values of wind speed, which has been calculated from the ten minute values recorded by each of the three test turbines. The output of the network is the mean monthly values of wind speed of a third WTG called “target WTG”. The inputs for the network consisted of; month of the year and monthly wind speeds for each of the two test WTG’s.

A Neural Network Based Approach to Wind Energy Yield Forecasting

919

As the model makes use of the backpropagation technique it is necessary to have all training targets normalised between 0 and 1, as the output neuron is restricted to a signal of only 0 or 1 values. As a result all inputs have also been normalised to the 0 to 1 range. It is accepted that by conducting this normalising process the characteristics of the training process are enhanced [2,3]. A random inclusion method was also selected for the training process. In addition the normalisation process attempted to spread the values throughout the range (0-1), as it is possible that all normalised values could fall between a range of 0.1 - 0.2. Spreading these values across a range will present a much more accurate output.

Fig. 1. The Proposed 4-Layer Network Architecture

The learning rate for the network was set at 0.01, obviously this decision slowed the speed of learning by the network, however, when tested with a higher learn rate (closer to 1) the network began to experience instabilities. The momentum was also set at a generally accepted standard of 0.8, changes in the momentum failed to yield any significant differences in network performance.

3 Neural Network Output Nine hundred randomly selected monthly wind speeds were used for training and 300 wind speeds were used for validation purposes. The root mean square error (RMS error) for the validation set was 0.11078. The network yielded a correlation of 0.983, and further demonstrated its ability by yielding a maximum percentage error within the validation set of 8%, a maximum error of 0.79 [4]. This represents a difference in real terms of 0.7 m/s, a value generally deemed insignificant in terms of wind speed and its potential for energy generation. As can be clearly seen the ANN model has a obvious role to play in assessing a locations suitability for development into a wind park, by allowing the prediction of long term wind speed data from a reduced dataset and therefore truncated monitoring period. However in this case the ANN model is used to provide wind speed data for inclusion into a modelling application which will not only assess a sites suitability for development in terms of wind speed but also provide an indication of the energy likely to be available in the wind spectra and also an estimate of firstly how much of that energy can he harnessed and secondly the likely production based on the configuration of the park.

920

P.R.J. Campbell et al.

4 Wind Spectra Computation The following section explains the calculations generally used to assess the potential energy yield for a given location (based on the estimated wind speeds generated by the ANN model) and also discusses how and why some of these common calculations have been adjusted to produce a more accurate estimate, within the application. The energy available in the wind spectra is given by generally by:

Watts = 1 * ρ *V 3 2

(1)

where p is the air density and V is the recorded wind speed in metres per second. However this method is greatly over simplified and makes the assumption that the wind speed V is constant. Clearly this is not the case and consequently while (1) is generally stated as the method for assessing power available in wind it is not used in this application, its functionality is replaced by the calculation shown below. As wind speed V and air density ρ (to a lesser extent) will change with each data reading, clearly the most accurate result will be obtained from a calculation which is inclusive of each data reading. A more appropriate calculation would there for be: n

WPD = 1 * 1 * 2 n

∑ (ρ

j

*V j3 )

(2)

j =1

where n is the number of wind speed readings ρ j and V j are the j (1st, 2nd, 3rd…) readings of air density and wind speed. This method produces greater accuracy when considering a locations potential or assessing a parks output, when compared to the common approach of simply basing estimates on the mean wind speed of a given location. The practice of assessing the wind conditions of a site based on the mean wind speed is in fact highly precarious, as the example below highlights. Five wind speeds have be recorded for a given location (5, 7, 15,10,4 m/s). In this example the air density is a constant, 1.225 Kg/m3 (normal atmospheric pressure). If we first assess the locations wind density using the mean wind speed and (1) we have the following: th

P = 1 *1.225*8.23 2

(3)

then power density = 337.7 W/m2. However, if we assess the wind power density of the site using the methodology shown in (2), then we have the following: 5

P= 1 *1 * 2 5

∑ (ρ

j

*V j3 )

j =1

P = 0.1*[(1.225*5 ) + (1.225*73 ) 3

+(1.225*153 ) + (1.225*103 ) + (1.225* 43 )] then power density = 601.1 W/m2.

(4)

A Neural Network Based Approach to Wind Energy Yield Forecasting

921

As this example shows, while the application of (1) seems perfectly reasonable, and is frequently used, in reality it can significantly under estimate the wind density. The application enables user to access a set of wind speed figures (either recorded or generated by the ANN) and assess the potential power available in the free wind using (2). This is the first step is assessing a locations potential for wind park development, allowing poor sites to be eliminated from consideration at an early stage.

5 Turbine Output Calculations As shown in the previous section we are able to produce an accurate calculation of the energy which is freely available in wind spectra. However, it is not possible for a Wind Turbine Generator (WTG) to extract 100% of the power contained in free wind. A WTG obtains produces power by transferring the power of the wind into torque which turns the blades. Obviously as a result the amount of energy which can be produced relies on the density of the air, the diameter of the WTG blades and the wind speed itself. It is generally accepted that a wind turbine can harness a maximum of 59% of the wind’s power, as proved by Betz (referred to as the power coefficient) [5]. In practice however this value is also acknowledged as being unattainable, with a capture figure of between 35-49% being recognised as more accurate. Resultantly the calculation shown below is accepted as a substitution for (1) when the potential output of a WTG is required [6].

P = 1 * ρ * A * Cp *V 3 * GenE * GbE 2

(5)

where A is the swept area of the WTG, Cp is the power coefficient of the WTG, GenE is the generator efficiency and GbE is the efficiency of the gearbox and bearings. 3

As can be seen from (5) V is once again used in the calculation. This method of assessing only a single (or mean) wind speed figure can lead to the same computation errors as encountered in the assessment of (1) in comparison to (2). Therefore we propose (6) as a more accurate method for calculating the energy yield of a WTG when assessing more than a single data reading.

P = 1 * A * Cp * cV 3 * GenE * GbE 2

(6)

or in its complete form n

P = 1 * A * Cp *( 1 * ∑ ( ρ j *V j )) * GenE * GbE n j =1 2 If we consider the example used in the previous section and assume a swept are of 1735m2 , a power coefficient of 0.40(Betz would suggest 0.59), GenE of 0.80 and GbE 0.80 also, then using (3) we have a Power output of 149,980kW. However if we apply the same data to (4) we produce a figure of 266,984 kW. This example

922

P.R.J. Campbell et al.

highlights that once again the simplified approach (which is in general use) vastly underestimates the potential power generation. It has now been demonstrated how the application makes use of firstly, the wind speed figures generated by the Neural Network, and secondly the calculations to provide; the amount of power available in the wind spectra and the potential power yield of a turbine. Obviously in order to apply (4) the application allows the user to specify the type or swept area of a turbine and also enables a site generalisation to be applied though the specification of the number of WTGs at the location.

6 Turbine Output Calculations The calculations which have been described in the earlier sections are applicable to any location. However both (2) and (6) assume that the wind speed data has been recorded at a height corresponding to the hub height of the WTG unit which is to be (or is already) installed at the location. Frequently during the process of selecting a site measurements are taken at a lower level, 10m for example. A typical WTG will have a hub height of greater than 40m. In addition it is frequently necessary to adjust the recorded wind speeds to take into account the surface roughness (different geographic surroundings). The application currently allows the user to enter and adjust low level wind data recordings to the appropriate hub height and surface roughness [7]. This adjustment is made using the following calculation:

V 2 = V 1*(h 2 / h1) * n

(7)

where V2 is the adjusted wind speed to be found, V1 is the recorded wind speed, at height h1, h2 is the target hub height and n is the exponent determining the wind change (surface roughness). Table 2 shows the range of possible values for n and their corresponding surface conditions. Table 2. Surface roughness adjustment values Surface Conditions Sea or Sand (smooth surface) Low grass High grass or low crops Tall crops or low woodland High woodland or small town

n value 0.10 0.16 0.18 0.20 0.30

Currently this functionality is only applicable to single data records however it is intended that the operation will be extended to encompass a full dataset enabling data collected through a lower height (and cost) measuring station to be adjusted for site selection processing.

7 Errors Rates in Estimations Obviously when dealing with predicted and even measured values there is a considerable opportunity for errors to occur. Indeed it is common place for functioning sites to

A Neural Network Based Approach to Wind Energy Yield Forecasting

923

have a number of errors in their records and calculations. There a number of reasons why errors can occur in the wind speed, and energy production figures. Obviously on measured sites these errors are mainly caused by equipment breakage or malfunction. However as this paper deals mainly with the case in which wind speed figures are estimated through the use of a neural network these causes are not an issue. As stated in section 3, the neural network itself is the first area in which error can and does occur. The network used to estimate both monthly and hourly wind speed values has a maximum percentage error of 8%, equating to 0.79m/s. Whilst this is an insignificant error when dealing purely with wind speed, it can have a significant affect when used to calculate power output. In practice the error value of an application such as this is stated in terms of the capacity factor for the target location. The capacity factor is simply the amount of energy (in kWh) which is produced by the wind park (or turbine, in the case of standalone units). If we take the example of a single WTG with a rated power output of 660kW, then the capacity factor for this unit over a 24 hour period would be 15.84MW. However, this figure is based on the assumption that the unit was operating at its maximum power outage for the full time period. Obviously in practice this is highly unlikely. As a result it is typical for a unit or park which operates at 25-35% of its capacity factor to be classed as performing well. This application will assess the performance of the WTG (or site) and reports to the user the current capacity factor and potential error in that figure (also in terms of a percentage of the capacity factor). Currently the maximum errors reported by the system based on estimated wind speed data supplied by the neural network have been 4% of the capacity factor. However it has been found that whilst the total production has mirrored closely (within the stated error range) the recorded production of both single units and full parks (25 WTG units) the final output to grid from the park has been lower than estimated. This anomaly can be explained by the losses which occur when the energy is transferred and switched to interface with the grid.

8 Conclusion We have shown that wind speed data can be estimated by use of an artificial neural network, created using a backpropagation technique, with four layers and three input nodes. The model has proven to be accurate at predicting monthly and hourly wind speed data, with a maximum error of 0.79 m/s, a value classed trivial when assessing wind speed. Within the paper the wind speeds derived from the model have been used to demonstrate the ability of a software application, to provide users with a range of wind assessment tools. A number of calculations have been described and demonstrated which enable the power available in the free wind to be calculated and the evaluation of the energy which can be harnessed from the free wind by use of a Wind Turbine Generator. A number of short comings have been identified in the calculations commonly applied to both free and harnessed wind and modified expressions have been demonstrated to provide a more accurate estimate of these resources. By use of these modified calculations it has been demonstrated that the application can be used to generate a range of wind speed data, assess the power it will provide and ultimately allow a

924

P.R.J. Campbell et al.

user to experiment with different variables to consider a locations suitability for wind park development and the likely energy yield. Significant opportunities still exist to extent and refine this application to deliver shorter term and more accurate estimates, improving on the current percentage errors of 8% in terms of wind speed estimation and 4% in terms of energy yield. The study has also highlighted the applicability of neural computing techniques to new and developing domains.

References 1. Pham, D.T., Liu, X.: Neural Networks for Identification. Prediction and Control. Springer, London (1995) 2. Mohandes, A.M., Rehman, S., Halawani, T.O.: A neural network approach for wind speed prediction. Renewable Energy 13(3), 345–354 (1998) 3. Kalogirou, S., Neocleous, C., Paschiardis, S., Schizas, C.: Wind Speed Prediction using Artificial Neural Networks. In: European Symposium on Intelligent Techniques ESIT on CDROM (1999) 4. Campbell, P.R.J., Adamson, K.: A Multi Layered Perceptron Network for Wind Speed Generation. In: 12th IASTED Conference on Applied Simulation and Modelling, pp. 490– 494. ACTA Press (2003) 5. Betz, A.: Wind-Energie und ihre Ausnutzung durch Windmühlen (Bandenhoeck & Ruprect, Göttingen, 1926). Facsimile edition by Ökobuch Verlag, Staufen (1994) 6. Burton, T.: Wind Energy Handbook. Wiley and Sons, London (2001) 7. Troen, I., Lundtang Petersen, E.: Wind Energy Resources & Computing Wind Turbine Energy Output. Risoe National Laboratory, Denmark (1991)

Research on New Intelligent Business-Oriented Decision-Making Model Based on MA and GA Weijin Jiang1 and Qing Jiang2 1

School of Computer and Electronic Engineering, Hunan University of Commerce, Changsha 410205, China [email protected] 2 School of Computer, South University of Technology, Guangzhou 510092, PRC

Abstract. Distribution system optimal planning has vital significance, but there isn't efficient and practical algorithm at Traditional genetic algorithm has a poor expressive power for complicated problem because of the restriction of its norm mode, which limits the application fields of genetic algorithm. This paper adapts the idea of “Ethogenetics” reference, and presents a new type of genetic algorithm based on Agent behavior and paradigm learning. Unlike the based creating mode of feasible solution in traditional genetic algorithm, a feasible solution is created by ~ series o! ye behaviors of Agent based on knowledge in the new genetic algorithm. To adapt the new creating mode of feasible the traditional mechanism of evolution optimization based on Darwinism is abandoned and the mechanism of learning' is adopted to realize the evolution optimization. At last, an example distribution network is optimized by Ilene tic algorithm and traditional genetic algorithm respectively. The comparative result proves the new genetic I has higher expressive power, computing efficiency, convergent stability and extendable capability. Keywords: Agent intellective behavior, Paradigm learning, Evolution algorithm, Distribution computation, Optimal planning.

1 Introduction At present, many optimized plans (for example city net plan) question lacks the effective practical algorithm, Its main reason is: The city net plan is a typical NP difficulties to combine the optimized question; Because of the distribution network must satisfy the radiation shape movement restraint, This restraint is unable with the number form to express. Usually thought, the heredity algorithm (GA) is one kind of solution combination optimization question adoptions algorithm[1,2]. However, when solute this kind of complex combination optimization question like city net, the heredity algorithm often appears lacks the ability to do what one would like. The main reason as stated in the text [3] pointed out that:” The heredity algorithm is only suitable for the parameter, the variable optimized question which solving a series of explicitly assigns, but lacks regarding the complex question indicates the ability”. Specifically mentions[4-7], is precisely because the heredity algorithm uses "the determination form code described Z. Zeng & J. Wang (Eds.): Adv. in Neural Network Research & Appli., LNEE 67, pp. 925–932. springerlink.com © Springer-Verlag Berlin Heidelberg 2010

926

W. Jiang and Q. Jiang

waits the optimized question to seek the superior space" this fundamental mode to limit its ability to indicate the complex question. Regarding basic complex questions, must seek some encoding method to cause its encoded combination space and the superior space of this question to have some kind of corresponding relations, very is often difficult, even is unable to realize[8,9]. Regarding city net plan question, although also h radiation movement the restraint after passed through overlapping, heredity operation and so on variation[10-13]. For many years, the heredity algorithm’s success application in each domain proved "the evolution optimizes" thought is correctness. However, the imitation biology evolution (namely imitation heredity and variation of gene) but not means that the only way to realizes the evolution optimizes. In recent years emerged MAS (multi agent system) the research in cognitive science social collective cognition model and the research as well as two unified, all could transform the traditional heredity algorithm for us to provide the new mentality[14]. The thought the article [10] proposed "Orthogenetic (behavior heredity)", broke through the correspond since long ago about a code string to the combination optimizes always an all policy-making variable combination way tradition, the static understanding of the people. This article has absorbed the thought of article [10], further proposed one kind new heredity algorithm which based on the Agent behavior and the model study. In this algorithm, the Agent had certain intelligent to substitute for "the individual" in the traditional heredity algorithm; a feasible solution non- encoding method produces by Agent based on the knowledge a series of policy-making behavior treats the optimized question, has substituted for the traditional heredity algorithm based on the encoded feasible solution production way[15]; In order to unify feasible solution production way realization evolution seeks superiorly based on the Agent behavior proposed in this article, the evolution sought the superior mechanism based on "model study" substitute for the traditional heredity algorithm based on the imitation gene heredity algorithm and the traditional heredity algorithm to the identical city net gauge to carry on the contrast to optimize[16], the contrast result proved, this article method not only could avoid optimizing the process Central Africa feasible solution the production, had the very strong ability to indicate question, moreover had the better counting yield and restrains the stability.

2 The Solution Method Based on Cognition In the new heredity algorithm introduced in this article, each Agent all based on its own ability with analysis, synthesis, computation and inference produce to treat the plan question the feasible solution, regarding the combination optimization question, the existing heredity algorithm all will be carries on the operation to the code string. But in this article algorithm, all Agent will face question itself, directly will carry on to the policy-making variable takes the shed operation, if some policy-making variable is selected, will be allowed to record "1", otherwise will record "0". Thus, finally forms feasible solution although or a 0-1 yard string form, but its nature and the traditional heredity algorithm has essential the difference, here 0-1 yard string only is a record, its process is produces the code string by the feasible solution, but is not like

Research on New Intelligent Business-Oriented Decision-Making Model

927

the traditional heredity algorithm by the code string production feasible solution. In this article, each Agent, through imitates the person (to plan expert) to realize its inference in the distribution network plan process cognition behavior, the decisionmaking process, thus proposed a feasible plan for city net plans. In the chart, the biggest strut tree is refers to including all pitch points but not to form the link to connect the chart. The distribution network plans a feasible plan inevitably is by all prepares chooses a plan network analysis sites structure drawing biggest strut tree which the line is composed. Regarding the plan network chart which composed by nearby n pitch point, then essential condition to form the biggest strut tree is must have nearby the l-n+1 strip is separation, regarding a plan network chart , n pitch point (with k transformer which composed by k transformer substation, l substation pitch point), constitutes a biggest production tree to have to the line spattered number is: l-(n-k+1)+1=l-n+k. After the plan network architecture determined still had to restore. After all radiations sides is simplified removes, the plan network will turn the network which will constitute only by the non- radiation side. In fact the city net plan is to determine which “sides” should "separation" (separation side not need to restore). When separates l-n+k sides suitably in the chart, the side may divide into following two kinds: (1) radiation side, the characteristic was its beginnings and ends only one has 1 vertex (the vertex reflected the sides’ number of pitch point connected), (2) non- radiation side, is the one constitutes the return route with other sides. Obviously, the radiation side is cannot separate, otherwise is unable to supply the power to its connected load spot. However, just because this these sides cannot separate. Each kind of movement network structure drawing all must contain these sides. Therefore said it with change of the network architecture, or said that has nothing to do with the network basic structure’ determination, may temporarily simplify from the initial planning drawing. These temporarily simplify radiation side on the one hand when the network basic 2nd kind. All has not been carried out "the separation" operation will to constitute the basic structure of the plan network, in the basic structure foundation, restore the side which removes all simplification again, has formed a biggest production tree in plan network chart, also is a feasible plan which the distribution network plans. First defines following 3 arrays:

馈线

(1) Switch variable array Ki, constitutes by 0-1 binary number, its each element serial number correspondence one side marking of plan network, its function is to judge whether each side in the plan network can carry out the separation operation, 1 represents this side may separate, 0 represents cannot separate. (2) Feasible plan recording array K. In the form is completely consistent with the switch variable array, its function is to record the finally feasible plan, 1 represents this side is selected in the final plan, 0 represents has not been selected. (3) Pitch point " vertex" recording array Kd, constitutes by n decimal digit’ number, the element serial number correspondence n pitch point marking in plan network, its function is records various pitch points “vertex” as necessary in the feasible plan’s generating process.

928

W. Jiang and Q. Jiang

Agent concrete inference process step as follows: Step 1: Reads the person distribution network plan network chart master data, including all pitch points’ "vertex", the source pitch point’ "vertex" records for the negative value, the non- source pitch point’ "vertx" records for positive. Switch variable array Ki, the element of the feasible plan recording array K completely evaluation is 1. Step 2: Simplifies all radiations side, the corresponding elements in Ki’ evaluation is 0; Judges these two vertices "vertex" whether is 0,if is, then the “vertex” of the two vertices subtracts 1, otherwise is bigger than 0, If is, then the "vertex" of the two vertices all subtracts 1, otherwise the “vertex” lie in 0 vertices reduce 1, smaller than 0 vertices Canada 1. Step 3: Select 1 element randomly in which element still was "1" from k (uses even distribution random number production) , let Its serial number correspondence side “separation" (is K in this element evaluation is 0). Step 4: Verifies whether all the complete pitch point can guarantee the power supply, if can guarantee the power supply, then the separation operation in Step 3 is effective, then let the element which corresponds opens part is 0 in k, judge the two vertices “vertex” whether all is bigger than 0. if is ,all the two vertices subtracts 1, otherwise the vertices’ “vertex” which is bigger than 0 subtracts 1, the vertices’ “vertex” which is smaller than 0 add 1, and enter step e, If cannot guarantee the power supply, this separation operation is invalid, returns to Step 3. Step 5: Transfers Step 1, after simplifies all radiations side once more to enter Step 6. Step 6: Choice next 2nd kinds carries on the separation operation, repeatedly carries out Step 3, 4, 5, until the Kj in element value all is 0. After above 6 steps inference, the side element in Kr "the value will be 1" correspondence will constitute the plan network which will be able to guarantee the radiation shape movement the feasible plan. The array (or code string) K, is this feasible plan record. N Agent will be able to have n independent feasible plan, thus will constitute some generation of Agent community. Because behavior way of the Agent is based on imitates person's cognition process, therefore may widely fuse the existing each kind of artificial intelligence and the cognitive science research achievement in the Agent design process (for example each kind of expert system may be provides the very good reference for the Agent design. Thus enable the method of this article have the very good application flexibility and may the extension when faces each kind of complex question.

3 Model and Algorithm In order to realize the evolution to seek superiorly, union the non- encoding method feasible solution production process introduced front, this article proposed seeks the superior mechanism which is based on the model study evolution. First produces a simple principle of the model establishment and refresh. The optimized advancement carried on the Goth generation already to find m different partial {1,2... M} may seek superior optimal solution correspondence optimized plan Ci, i

∈

Research on New Intelligent Business-Oriented Decision-Making Model

929

m for the G+1 generation "the model", Here, the model also is the feasible plan record, has the same form with Ki, is only for the convenience narration, therefore has introduced mark C. Describes the above principle with the mathematical linguistics, let M model Ci is {1,2... M} the value from slightly (city net plan usually objective function f (Ci), i is asks objective function to the big arrangement to be smallest), supposes community the partial optimal solution objective function in the Goth generation is f (CG), Its used in the model CG, if this model comparatively had some model objective function is small, than increases progressively m Fan Pinching appropriate position model CG insertion goal function value f, after the insertion certain model orders moves, original model Cm is eliminated, in the entire optimized advancement, this m model throughout is in one kind of dynamic renewal. Wrote the plan form is:

∈

⎧⎪if : f (C j−1 ) < f (C G ) < f (C J ) ⎨ ⎪⎩then : C j = C G ; C j−1 = C j ;" C m = C m −1

(1)

Agents in a new generation of a reconciliation process, the reference to study the existing paradigm. Agent ,reference to the so-called learning is a decision variable in determining the choice can choose one example, The choice of variables which reference implementation. To solve the problem of urban network planning, In this paper, the agent in the birth of a viable plan to step reasoning for the decision-making process whenever (1,2,…, `}. Ci said the serial 3:00. should first select a model example of Ci m, i numbers corresponding to the elements of the edge it in the example of a viable plan represents a starting point, one said to be selected, 0 expressed not selected. At this point in the array of Cassation said the serial numbers corresponding to the elements while it continued to operate for one said it would break apart. 0 preclude calls, agent for the study of concrete examples of Ci : Elements from the example of the value of 0, and serial number of array elements of Cassation, with the value of all the elements of one. Set posed by its serial number (in mind that this is a mark of Cassation, in accordance with the above rules, said Ci match instead of focusing strictly on) randomly choose one or several (choose one of this type), corresponding to the edge calls. For example choice, which can be used with reference to "roulette", the probability that a particular model selected:

∈

Pi =

f i−1 m

∑f i −1

−1 i

(2)

Where : Pi was chosen for the i th paradigm : the probability of th East fi fitness function for the objective function value. If all m exceptional array ki match with the paradigm for the empty pool, China remains one ki directly from the one chosen randomly from all the elements, while its counterpart serial numbers apart.

930

W. Jiang and Q. Jiang

New genetic algorithm to strengthen the capacity of local optimums. can "best current example of" (m because of "paradigm" in accordance with the objective function value increased over the line but in a dynamic update process, So it is the current optimal Fan Ci) has been studying the probability value of Pi "by substituting attenuation." Following specific method: If the G-generation optimization process have a new "best current example of Ci," then G+1 populations in the process of formation, were artificially designated succes(0, 1). Other examples are Pi, the probability sor Ci example, the probability Pi, Pi of succession, i (2,3,…, m) :

∈

∈

Pi =

f i−1 m

∑f i−2

−1 i

(1 − p i )

(3)

In its resolution G+t G+2 generation to generation (assuming +1 after the "best current examples Ci" will again be updated) generation groups in the process of formation, the probability model Ci Pi followed by the succession:

PIG −1

∈

⎧ p1G (100 − μ (i −1) ) p1G (100 − μ (i −1) ) 1 > ⎪⎪ 100 100 n =⎨ G ( i −1) ⎪ 1 p1 (100 − μ ) ≤ 1 ⎪⎩ n 100 n

(4)

(2,3,…, t); To control the decay rate, the smaller the value, the slower Where : i attenuation, in general, (1, 3) is appropriate. Other examples are Pi, the probability of succession, i (2,3,…, m), upon the type (4) computers. With the traditional GA the "variability" similar. This paper introduces a new type of genetic algorithm Agent can also have some breakthrough in the current "paradigm" in the "create mechanisms "As a global optimization with the ability to complete. To achieve the following steps:

∈

∈

Step 1: Agent sets the probability threshold r, a value judgment whether the rebellious character Agent; Variations δ the probability threshold is a value judgment Agent rebellious character in the entire course of conduct which one or What specific act of treason. Step 2: Months Agent began its reasoning, the decision-making process, Firstly, with a uniform distribution of random numbers generated a random number If the probability is less than the threshold value of r agent, the agent is not rebellious character. its act in strict accordance with section 3.2 of the existing paradigm for the study on the manner; Otherwise, The agent is willing to be rebellious character, to step c. Step 3: Step 2 is judged as rebellious character of the agent, the agent in every step of reasoning, decision-making, First evenly distributed random number generator to produce a random number, if not more than the variance threshold δ The acts were not

Research on New Intelligent Business-Oriented Decision-Making Model

931

acts of treason, in accordance with section 2.2 of the existing paradigm on the way; Otherwise, The act of treason, according to section 3.2 of direct examples, the paper's content.

4 Conclusions By absorbing the literature [10] "Ethogenetic" put forward a new idea Genetic Algorithm. The algorithm is intelligent agent that replaced the traditional genetic algorithm individual; Agent acts with a series of decisions based on knowledge generated a feasible solution to the problem of non-optimized code, Based on genetic algorithm code to replace the traditional ways of generating feasible solution; Use based on the "learning paradigm" evolutionary optimization mechanisms, replaced the traditional genetic algorithm based on the genetic variation to imitate the evolutionary optimization mechanisms. Network Planning Board adopted a specific numerical example shows that compared with traditional genetic algorithm. New Genetic Algorithm for a better applicability of this issue, On one hand, to shake off the shackles of the traditional forms of genetic coding algorithm, the problem is more complex expression, plan to avoid a lot of radiation distribution network operating in violation of the constraints; the other hand, Based on the "learning paradigm and update the" evolutionary optimization mechanisms have higher efficiency and better convergence and stability.

Acknowledgments This work is supported by the Society Science Foundation of Hunan Province of China No. 07YBB239.

References 1. Bredin, J., Kotz, D., Rus, D., Maheswaran, R.T., Imer, C., Basar, T.: sComputational markets to regulate mobile-agent systems. Autonomous Agents and Multi-Agent Systems 6(3), 235–263 (2003) 2. Archer, A., Tardos, E.: Truthful mechanisms for one- parameter agents. In: Proceedings of the 42nd IEEE Symposium on Foundations of Computer Science, pp. 482–491. IEEE Computer Society Press, Lasvegas (2001) 3. Rosado, I.J., Bernal-Agustin, J.L.: Genetic Algorithms in Multistage Distribution Network Planning. IEEE Trans Power Systems 9(4), 1927–1933 (1994) 4. Scherrer, B., Charpillet, F.: Cooperative Co-learning: A Model-based Approach for solving MultiAgent Reinforcement Problems. In: Proceedings 14th IEEE International Conference on Tools with Artificial Intelligence, pp. 463–468 (2002) 5. Dillenbourg, P., Self, J.A.: A Computational Approach to Distributed Cognition. European Journal of Psychology Education 7(4), 252–373 (1992) 6. Jiang, W.J., Wang, P.: Research on Distributed Solution and Correspond Consequence of Complex System Based on MAS. Journal of Computer Research and Development 43(9), 1615–1623 (2006)

932

W. Jiang and Q. Jiang

7. jiming, L., Han, J., Tang, Y.Y.: Multi-agent oriented constraint satisfaction. Artificial Intelligence 136(1), 101–144 (2002) 8. ke, X., yuoxuan, W., chen, W.: Test Technology and application of model of chain of services of Grid. Science in China E 37(4), 467–485 (2007) 9. Weicai, Z., Jin, L., Mingzhi, X., et al.: A multi agent genetic algorithm for global numerical optimization. IEEE Transactions on System, Man, and Cybernetics 34(2), 1128–1141 (2004) 10. Bredin, J., Kotz, D., Rus, D., Maheswaran, R.T., Imer, C., Basar, T.: Computational markets to regulate mobile-agent systems. Autonomous Agents and Multi-Agent Systems 6(3), 235–263 (2003) 11. Busoniu, L., Babuska, R., De Schutter, B.: A Comprehensive Survey of Multiagent ReinforcementLearning. IEEE Transactions On System, Man, And Cybernetics-Part C: Application And Review 38(2), 156–172 (2008) 12. Gou, Y., Huan, J.Z., Rong, H.: Adaptive grid job allocation with genetic algorithm. Future Genre Comp. Sys. 21, 151–161 (2005) 13. Matsui, M.: A generalized model of convey-serviced production station (CSPS). Journal of Japan IndustrialManagement Association 44(1), 25–32 (1993) 14. Yamada, T., Satomi, K., Matsui, M.: Strategic selection of assembly system under viable demands. Assembly Automation 26(4), 335–342 (2006) 15. Nakase, N., Yamada, T., Matsui, M.: A management design approach to a simple flexible assembly system. International Journal of Production Economics 76, 281–292 (2002) 16. Jiang, W.J., Pu, W., Lianmei, Z.: Research on Grid Resource Scheduling Algorithm Based on MAS Cooperative Bidding Game. Chinese Science F 52(8), 1302–1320 (2009)

Author Index

Agogino, Alice M. 211 Ahmed, Faheem 917 Auephanwiriyakul, Sansanee Bai, Xue 423 Battle, Kimberly 747 Bayro-Corrochano, Eduardo Bedingfield, Susan 869 Bi, Fukun 729 Bing-Fei, Wu 663 Burse, Kavita 493 Cai, Chengkai 275 Cai, Qiao 347 Campbell, Piers R.J. 917 Cao, Qixin 529, 537 Cao, Yan 265 Chai, Tianyou 803 Chang, Chia-Ting 91 Chang, Chia-Wei 679 Chang, Hung-Teng 201 Chang, Yu-Teng 1, 161, 201 Chao-Jung, Chen 663 Che, Chang 811 Chen, Bo-Chiuan 687 Chen, Bo-Wei 901 Chen, Chao-Jung 671, 679 Chen, Chen-Yu 901 Chen, Chin-Ming 853 Chen, Chiu-Hsiung 655 Chen, Guojin 445 Chen, Hongtao 739 Chen, Hsi-Min 695 Chen, Jing 235 Chen, Liming 551 Chen, Peihua 529 Chen, Pin-Chang 161 Chen, Sheng 347 Chen, Tian-Ding 321 Chen, Weidong 545 Chen, Xiaoling 355 Chen, Xinglin 485 Chen, Ying-Han 671 Chen, Youliang 877 Chen, Zhang 311

363

577

Cheng, Cheng 227 Cheng, Wei 721, 793 Cheng, Yu 703 Cheng-Yen, Yang 663 Chien, Hai-Chang 679 Chih-Chung, Kao 663 Chiu, Chien-Jung 647 Cichocki, Andrzej 471 Cui, Wei 355 Dai, Longquan 631 Dai, Mingrui 413 Deng, Jiahao 129 Deng, Xiaobo 909 Ding, Hong 829 Ding, Jilie 909 Ding, Xiaojiang 869 Ding, Zhenghu 283 Dong, Zhong 551 Eiamkanitchat, Narissara

363

Fang, Chen 453 Fathulla, Haydar 917 Fei, Qi 837 Fu, Pan 739, 757 Fu, Zhichao 793 Gao, Feng 373 Gao, Lining 729 Gao, Wen-Jing 747 Garcia-Hernandez, Ramon 577 Ge, Yingyu 819 Gonz´ alez-Olvera, Marcos A. 615 Gu, Xueqiang 235 Gu, Ying-Kui 145 Guo, William W. 845 Han, Seung-Soo 253 Han, Yi-Seul 253 Hattori, Kiyohiko 567 He, Haibo 347 He, Ping 29 He, Qinggao 623 He, Yong 765, 893

934

Author Index

He, Zhaoshui 471 He, Zhengjia 721 Hope, A.D. 757 Hou, Guolian 47 Hsu, Chun-Fei 647, 773 Hu, Dan 811 Hu, Hsiu-Yun 201 Hu, Liping 301 Hu, Nansai 347 Hu, Zhifeng 101 Huang, Kai-Qi 145 Huang, Kai-Siang 439 Huang, Ping 783 Huang, Po-Chia 671 Huang, Shiuh-Jer 687 Huang, Sui 739 Huang, Yi-Wen 687 Hung, Kuo-Chen 861 Israel, Cruz Vega

117

Jaffar, Ahmad D. 917 Jeeva, Ananda S. 845 Jeon, Se-Youn 253 Jiang, Li 757 Jiang, Lulu 765 Jiang, Qing 185, 925 Jiang, Weijin 185, 925 Ji, Guoli 639 Jin, Long 55 Jin, Yanbo 65 Kang, Xiaoyun 631 Kao, Chih-Chung 671 Kao, Tsai-Sheng 655 Khunarsa, Peerapol 337 Kim, Min-Woo 253 Lee, Jonathan 695 Lee, Shin-Jie 695 Lee, Tsu-Tian 647 Lee, Wen-Tin 695 Lee, Yun-Keun 253 Leng, Chuntao 529 Leu, Min-Shyang 91 Li, Chunping 819 Li, Dengwan 739 Li, Gang 431 Li, Guang 453 Li, Hongjun 413 Li, Mingchang 885

Li, Ningfei 479 Li, Peng 485 Li, Pengcheng 783 Li, Rong-li 405 Li, Shijin 329 Li, Shuai 265 Li, Weiling 757 Li, Xiao-li 587 Li, Xiaochen 347 Li, Yangmin 153 Li, Yanling 431 Li, Yanmei 109 Li, Yen-Feng 679 Li, Yi 265 Li, Yongning 445 Li, Zhan 355 Liang, Shuxiu 885 Liang, Yue 559 Lin, Wei 639 Lin, Jen-Chang 861 Lin, Ji-Yuan 439 Lin, Kuo-Ping 861 Lin, Lin 329 Lin, Qiu-Hua 381 Liu, Bo-Jhao 687 Liu, Cuiling 21 Liu, Dong 829 Liu, Fei 765, 893 Liu, Feng 729 Liu, Hailei 909 Liu, Hongwei 301 Liu, Jizhong 503 Liu, Mingzhe 55 Liu, Pei 439 Liu, Shijun 311 Liu, Tao 101 Liu, Xiao-yong 405 Liu, Yun-hui 545 Liu, Zhong 559 Lo, Chih-Yao 1, 201 Lu, Chi-Jie 73 Lu, Sheng 265 Lu, Wei 83 Luan, Nan 551 Luo, Siwei 423 Lursinsap, Chidchanok Lv, Jun 373 Lv, Jun-Wei 293 Lv, Zhuo 479

337

Author Index Tao, Tao 311 Teng, Hongqiu 397 Theera-Umpon, Nipon Tian, Guiyou 311 Tsai, Jang-Zern 647

Ma, Qing 461 Ma, Wei-min 83 Marwala, Tshilidzi 747 Meng, Qian 355 Min, Haibo 521 Mitola, Joseph 347 Murphey, Yi L. 713 Nelwamondo, Fulufhelo V. Niu, Dongxiao 275 Niu, Huifen 21 Otani, Masayuki

747

567

Pai, Ping-Feng 861 Pan, Li-fang 219 Peng, Ya-Fu 655 Peng, Zhenming 311 Raicharoen, Thanapant Ruan, Weizheng 397

337

Saad, Maarouf 577 Sanchez, Edgar N. 577 Saqi, Rina 37 Sato, Hiroyuki 567 Shang, Li 101 Shen, Lincheng 235 Shen, Minfen 631 Shen, Tsurng-Jehng 603 Shen, Yong-liang 513 Shrivastava, Sushil Chandra Song, Qiankun 623 Song, Shen-min 513 Song, Shenmin 485 Sun, Fuchun 521 Sun, Fuming 391 Sun, Gao-Ji 245 Sun, Gongxian 373 Sun, Lei 265 Sun, Lingfang 37 Sun, Shifeng 137 Sun, Xiaowen 21 Sun, Zhaochen 885 Takadama, Keiki 567 Tan, Hongbing 537 Tan, Lihong 765 Tan, Qing-chang 405 Tang, Aolin 537 Tang, Jian 803 Tang, Yu 615

493

Wan, Dingsheng 329 Wang, Bin 283 Wang, Bo 293 Wang, Chi-Kai 861 Wang, Chih-Hu 773 Wang, Dong-Mei 293 Wang, Dongxia 391 Wang, Guanghui 503 Wang, Hanmei 275 Wang, Hesheng 545 Wang, Hong 595 Wang, Jhing-Fa 901 Wang, Jia-Ching 901 Wang, Jianfei 703 Wang, Jiayin 173 Wang, Jiesheng 137 Wang, Jinfeng 227 Wang, Jingyue 47 Wang, Jinqi 21 Wang, Li-Dan 381 Wang, Lin 235 Wang, Nan 235 Wang, Peng 877 Wang, Rui 47 Wang, Rulong 453 Wang, Sheng 355 Wang, Shicheng 521 Wang, Wanqiang 445 Wang, Zongrun 65 Wu, Bing-Fei 671, 679 Wu, Caipeng 129 Wu, Chen-Feng 1 Wu, Hao 329, 355 Wu, Jiansheng 55 Wu, Jui-Yu 73 Wu, Xuepei 503 Wu, Xuewu 265 Xi, Wujun 29 Xia, Jiating 503 Xiao, Ming 373 Xie, Honggang 37 Xie, Shu-tong 219 Xie, Songyun 479

363

935

936

Author Index

Xing, Bo 747 Xu, Dongqiang 11 Xu, Lisheng 909 Xu, Qingsong 153 Xu, Shen 713 Xu, Yunyun 11 Yadav, R.N. 493 Yang, Cheng-Yen 679 Yang, Ming-Der 439 Yang, Yanli 129 Yang, Yu-Chieh 853 Yang, Zijiang 639 Yao, Yu-Dong 347 Yeh, Chung-Hsing 869 Yeh, Ming-Feng 91 Yen-Feng, Li 663 Yin, Fuliang 391 Yin, Qin 819 Yin, Zhenzhen 877 Ying-Han, Chen 663 Yu, Jinyang 783 Yu, Wen 117, 803 Yu, Xiang 587 Yu, Ying 283 Yu, Zhi-gang 513 Yuan, Yongquan 783 Yue, Heng 803 Zeng, Wei 193 Zeng, Wenyi 173 Zhang, Da-wei 513 Zhang, Feng 265 Zhang, Guodong 559

Zhang, Hua 503 Zhang, Jianhua 47, 595 Zhang, Jin 453 Zhang, Jinsheng 521 Zhang, Liming 283 Zhang, Shenglan 909 Zhang, Tao 703 Zhang, Xiaoming 551 Zhang, Xiaopeng 413 Zhang, Yan 587 Zhang, Yao 783 Zhang, Ying 211 Zhang, Yong 137 Zhang, Yu 765 Zhang, Zhen 529 Zhang, Zhousuo 721 Zhao, Fei-fei 829 Zhao, Lijie 803 Zhao, Qian 193 Zhao, Shu-Kun 253 Zheng, Zhong 713 Zhou, Bin 885 Zhou, Ping 301 Zhou, Sun 639 Zhou, Yanju 65 Zhu, Jiali 329 Zhu, Ke-Jun 161 Zhu, Miaofen 445 Zhu, Shangwu 453 Zhu, Weifeng 837 Zhu, Xiaoxiao 537 Zhuang, Han-Sheng 1