Lecture Notes in Computer Science Edited by G. Goos, J. Hartmanis, and J. van Leeuwen
2869
3
Berlin Heidelberg New York Hong Kong London Milan Paris Tokyo
Adnan Yazici Cevat S¸ener (Eds.)
Computer and Information Sciences – ISCIS 2003 18th International Symposium Antalya, Turkey, November 3-5, 2003 Proceedings
13
Series Editors Gerhard Goos, Karlsruhe University, Germany Juris Hartmanis, Cornell University, NY, USA Jan van Leeuwen, Utrecht University, The Netherlands Volume Editors Adnan Yazici Cevat S¸ener Middle East Technical University Department of Computer Engineering 06531 Ankara, Turkey E-mail: {yazici/sener}@ceng.metu.edu.tr Cataloging-in-Publication Data applied for A catalog record for this book is available from the Library of Congress. Bibliographic information published by Die Deutsche Bibliothek Die Deutsche Bibliothek lists this publication in the Deutsche Nationalbibliografie; detailed bibliographic data is available in the Internet at .
CR Subject Classification (1998): H, C, B, D, F, I ISSN 0302-9743 ISBN 3-540-20409-1 Springer-Verlag Berlin Heidelberg New York This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer-Verlag. Violations are liable for prosecution under the German Copyright Law. Springer-Verlag Berlin Heidelberg New York a member of BertelsmannSpringer Science+Business Media GmbH http://www.springeronline.com © Springer-Verlag Berlin Heidelberg 2003 Printed in Germany Typesetting: Camera-ready by author, data conversion by PTP-Berlin GmbH Printed on acid-free paper SPIN: 10963555 06/3142 543210
Preface
The International Symposium on Computer and Information Sciences (ISCIS) brought together researchers and practitioners from all over the world to exchange ideas and experiences on their recent research in all areas of the computer and information sciences. This year, the 18th ISCIS was organized by the Department of Computer Engineering at the Middle East Technical University, and was held in Antalya, Turkey. This year was the first in which the proceedings of ISCIS were published in the LNCS series by Springer-Verlag. The proceedings include 135 papers that were selected by the program committee from over 360 contributions submitted from 25 different countries. The symposium this year covered the following topics in the computer and information sciences: architectures and systems, computer science theory, databases and information retrieval, e-business, graphics and computer vision, intelligent systems and robotics, multimedia, networks and security, parallel and distributed computing, soft computing, and software engineering. The support from the following institutions is deeply appreciated: – – – –
METU (Middle East Technical University) ¨ ITAK ˙ TUB (Scientific and Technical Research Council of Turkey) IEEE, Turkey Section IFIP (International Federation for Information Processing)
I must also express my sincere thanks to all the people who contributed their time and great effort to make the symposium possible. My very special thanks ˙ go to the organizing committee, namely Semra Do˘ ganda˘ g, Sertan Girgin, Ismail ¨ urk, Ahmet Sa¸can and Cevat S¸ener, who did very active and endless work Ozt¨ and put tremendous effort into the symposium. I also thank many other colleagues for their special contributions to the symposium. Some of them are Ne¸se ¨ coluk, Ay¸se Kiper, Halit O˘ Yalabık, G¨ okt¨ urk U¸ guzt¨ uz¨ un, Panagiotis Chountas, ¨ or, Alfred Hofmann and Anna Kramer. Lee Dong Chun, Yo-Sung Ho, Alper Ung¨ In addition, I would also like to thank all of this year’s invited speakers and the members of the program committee of ISCIS 2003, whose reviews were very important for the quality of the symposium.
November 2003
Adnan Yazıcı
Organization
ISCIS 2003 was organized by the Department of Computer Engineering, Middle East Technical University.
Executive Committee Honorary Chair
Erol Gelenbe (University of Central Florida, USA) Adnan Yazıcı (Middle East Technical University, Turkey)
Conference and PC Chair
Program Committee Sibel Adalı Levent Akın Ergun Akleman Varol Akman Demet Aksoy Reda Alhajj Ferda Nur Alpaslan Ethem Alpaydın Mehmet Altınel Makoto Amamiya Troels Andreasen Peter Arbenz Volkan Atalay I¸sık Aybay Cevdet Aykanat Fevzi Belli Semih Bilgen Arndt Bode M¨ uslim Bozyi˘git Bob Brazile Sel¸cuk Candan Panagiotis Chountas Maurice Clint ˙ Ilyas C ¸ i¸cekli Nihan C ¸ i¸cekli Hasan Davulcu Kudret Demirli Asuman Do˘ga¸c
Jack Dongarra Iain Duff Bob Duin Kemal Ebcio˘ glu Jean-Michel Fourneau Roy George Muhittin G¨ okmen Ratan Guha U˘gur G¨ ud¨ ukbay Marifi G¨ uler Atilla G¨ ursoy Dilek Hakkani-Tur Emre Harmancı Pete Harrison Sven Helmer Alain Jean-Marie Janusz Kacprzyk Laszlo T. Koczy C ¸ etin Kaya Ko¸c Uzay Kaymak Ay¸se Kiper Ta¸skın Ko¸cak Selahattin Kuru Jonathan Lee Dan Marinescu Sinan Neft¸ci Halit O˘ guzt¨ uz¨ un Sema Oktu˘ g
¨ G¨ ultekin Ozsoyoglu ¨ Fatma Ozcan ¨ Tamer Ozsu Ilias Petrounias Fred Petry Faruk Polat Ramon Puigjaner Guy Pujolle Y¨ ucel Saygın Roberto Scopigno Marek Sergot Pavlos Spirakis Andreas Stafylopatis George Stamon Erol S ¸ ahin Abdullah Uz Tansel Hakkı Toroslu ¨ ur Ulusoy Ozg¨ ¨ or Alper Ung¨ Fatos (Yarman) Vural Roman Wyrzykowsky Tatyana Yakhno Berrin Yanıko˘ glu Kokou Yetongnon Slawomir Zadroznz Emilio Zapata
VIII
Organization
Organizing Committee Semra Do˘ ganda˘ g Sertan Girgin ˙ ¨ urk Ismail Ozt¨ Ahmet Sa¸can Cevat S ¸ ener
Sponsoring Institutions Middle East Technical University Scientific and Technical Research Council of Turkey IEEE, Turkey Section International Federation for Information Processing
Table of Contents
Invited Talk Review of Experiments in Self-Aware Networks . . . . . . . . . . . . . . . . . . . . . . Erol Gelenbe
1
Web Information Resource Discovery: Past, Present, and Future . . . . . . . . Gultekin Ozsoyoglu, Abdullah Al-Hamdani
9
Architectures and Systems Courses Modeling in E-learning Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Vincenza Carchiolo, Alessandro Longheu, Michele Malgeri, Giuseppe Mangioni Fast Hardware of Booth-Barrett’s Modular Multiplication for Efficient Cryptosystems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Nadia Nedjah, Luiza de Macedo Mourelle Classification of a Large Web Page Collection Applying a GRNN Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ioannis Anagnostopoulos, Christos Anagnostopoulos, Vergados Dimitrios, Vassili Loumos, Eleftherios Kayafas
19
27
35
Fast Less Recursive Hardware for Large Number Multiplication Using Karatsuba-Ofman’s Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Nadia Nedjah, Luiza de Macedo Mourelle
43
Topological and Communication Aspects of Hyper-Star Graphs . . . . . . . . Jong-Seok Kim, Eunseuk Oh, Hyeong-Ok Lee, Yeong-Nam Heo
51
An E-tutoring Service Architecture Based on Overlay Networks . . . . . . . . . Nikolaos Minogiannis, Charalampos Patrikakis, Andreas Rompotis, Frankiskos Ninos
59
A Simple Scheme for Local Failure Recovery of Multi-directional Multicast Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Vladim´ır Dynda
67
Modelling Multi-disciplinary Scientific Experiments and Information . . . . . Ersin Cem Kaletas, Hamideh Afsarmanesh, L.O. (Bob) Hertzberger
75
KinCA: An InfiniBand Host Channel Adapter Based on Dual Processor Cores . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sangman Moh, Kyoung Park, Sungnam Kim
83
X
Table of Contents
Ontological Cognitive Map for Sharing Knowledge between Heterogeneous Businesses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jason J. Jung, Kyung-Yong Jung, Geun-Sik Jo
91
Template-Based E-mail Summarization for Wireless Devices . . . . . . . . . . . . Jason J. Jung, Geun-Sik Jo
99
Some Intrinsic Properties of Interacting Deterministic Finite Automata . . 107 H¨ urevren Kılı¸c Design and Evaluation of a Source Routed Ad Hoc Network . . . . . . . . . . . . 115 Faysal Basci, Hakan Terzioglu, Taskin Kocak Temporal Modelling in Flexible Workflows . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 Panagiotis Chountas, Ilias Petrounias, Vassilis Kodogiannis Low Cost and Trusted Electronic Purse System Design . . . . . . . . . . . . . . . . 131 Mehmet Ercan Kuruoglu, Ibrahim Sogukpinar Uncorrelating PageRank and In-Degree in a Synthetic Web Model . . . . . . 139 Mieczyslaw A. Klopotek, Marcin Sydow Integration of Static and Active Data Sources . . . . . . . . . . . . . . . . . . . . . . . . 147 Gilles Nachouki Conditional Access Module Systems for Digital Contents Protection Based on Hybrid/Fiber/Coax CATV Networks . . . . . . . . . . . . . . . . . . . . . . . 155 Won Jay Song, Won Hee Kim, Bo Gwan Kim, Byung Ha Ahn, Munkee Choi, Minho Kang
Computer Science Theory Approximation Algorithms for Degree-Constrained Bipartite Network Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 ¨ or Elif Ak¸calı, Alper Ung¨ Demonic I/O of Compound Diagrams Monotype/Residual Style . . . . . . . . 171 Fairouz Tchier Fuzzy Logic and Neural Network Applications on the Gas Sensor Data: Concentration Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 Fevzullah Temurtas, Cihat Tasaltin, Hasan Temurtas, Nejat Yumusak, Zafer Ziya Ozturk A Security Embedded Text Compression Algorithm . . . . . . . . . . . . . . . . . . . 187 Ebru Celikel, Mehmet Emin Dalkılı¸c An Alternative Compressed Storage Format for Sparse Matrices . . . . . . . . 196 Anand Ekambaram, Eur´ıpides Montagne
Table of Contents
XI
Database and Information Retrieval Ranking the Possible Alternatives in Flexible Querying: An Extended Possibilistic Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204 Guy de Tr´e, Tom Matth´e, Koen Tourn´e, Bert Callens Global Index for Multi Channel Data Dissemination in Mobile Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212 Agustinus Borgy Waluyo, Bala Srinivasan, David Taniar A Robust Scheme for Multilevel Extendible Hashing . . . . . . . . . . . . . . . . . . . 220 Sven Helmer, Thomas Neumann, Guido Moerkotte A Cooperative Paradigm for Fighting Information Overload . . . . . . . . . . . . 228 ´ Daniel Gayo-Avello, Dar´ıo Alvarez-Guti´ errez, Jos´e Gayo-Avello Comparison of New Simple Weighting Functions for Web Documents against Existing Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236 Byurhan Hyusein, Ahmed Patel, Ferad Zyulkyarov Stemming in Agglutinative Languages: A Probabilistic Stemmer for Turkish . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244 B. Taner Din¸cer, Bahar Karaoˇglan A Multi-relational Rule Discovery System . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252 Mahmut Uludaˇg, Mehmet R. Tolun, Thure Etzold A Flexible Querying Framework (FQF): Some Implementation Issues . . . 260 Bert Callens, Guy de Tr´e, J¨ org Verstraete, Axel Hallez Similarity for Conceptual Querying . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268 Troels Andreasen, Henrik Bulskov, Rasmus Knappe Virtual Interval Caching Scheme for Interactive Multimedia Streaming Workload . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276 Kyungwoon Cho, Yeonseung Ryu, Youjip Won, Kern Koh RUBDES: A Rule Based Distributed Event System . . . . . . . . . . . . . . . . . . . . 284 Ozgur Koray Sahingoz, Nadia Erdogan A Statistical µ-Partitioning Method for Clustering Data Streams . . . . . . . . 292 Nam Hun Park, Won Suk Lee Text Categorization with ILA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300 Hayri Sever, Abdulkadir Gorur, Mehmet R. Tolun Online Mining of Weighted Fuzzy Association Rules . . . . . . . . . . . . . . . . . . . 308 Mehmet Kaya, Reda Alhajj
XII
Table of Contents
Application of Data Mining Techniques to Protein-Protein Interaction Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316 A. Kocatas, A. Gursoy, Reng¨ ul C ¸ etin Atalay
E-Business A Heuristic Lotting Method for Electronic Reverse Auctions . . . . . . . . . . . 324 Uzay Kaymak, Jean Paul Verkade, Hubert A.B. te Braake A Poisson Model for User Accesses to Web Pages . . . . . . . . . . . . . . . . . . . . . 332 ¨ S ¸ ule G¨ und¨ uz, M. Tamer Ozsu An Activity Planning and Progress Following Tool for Self-Directed Distance Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340 Nigar S ¸ en, Ne¸se Yalabik MAPSEC: Mobile-Agent Based Publish/Subscribe Platform for Electronic Commerce . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348 Ozgur Koray Sahingoz, Nadia Erdogan CEVS – A Corporative E-voting System Based on EML . . . . . . . . . . . . . . . 356 Dessislava Vassileva, Boyan Bontchev Smart Card Terminal Systems Using ISO/IEC 7816-3 Interface and 8051 Microprocessor Based on the System-on-Chip . . . . . . . . . . . . . . . . . . . . 364 Won Jay Song, Won Hee Kim, Bo Gwan Kim, Byung Ha Ahn, Munkee Choi, Minho Kang
Graphics & Computer Vision Practical Gaze Detection by Auto Pan/Tilt Vision System . . . . . . . . . . . . . 372 Kang Ryoung Park License Plate Character Segmentation Based on the Gabor Transform and Vector Quantization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381 Fatih Kahraman, Binnur Kurt, Muhittin G¨ okmen Segmentation of Protein Spots in 2D Gel Electrophoresis Images with Watersheds Using Hierarchical Threshold . . . . . . . . . . . . . . . . . . . . . . . . 389 Youngho Kim, JungJa Kim, Yonggwan Won, Yongho In Multi-resolution Modeling in Collaborative Design . . . . . . . . . . . . . . . . . . . . 397 JungHyun Han, Taeseong Kim, Christopher D. Cera, William C. Regli Model-Based Human Motion Capture from Monocular Video Sequences . . 405 Jihun Park, Sangho Park, J.K. Aggarwal Robust Skin Color Segmentation Using a 2D Plane of RGB Color Space . 413 Juneho Yi, Jiyoung Park, Jongsun Kim, Jongmoo Choi
Table of Contents
XIII
Background Estimation Based People Detection and Tracking for Video Surveillance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 421 Murat Ekinci, Ey¨ up Gedikli Quaternion-Based Tracking of Multiple Objects in Synchronized Videos . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 430 Quming Zhou, Jihun Park, J.K. Aggarwal License Plate Segmentation for Intelligent Transportation Systems . . . . . . 439 Muhammed Cinsdikici, Turhan Tunalı A Turkish Handprint Character Recognition System . . . . . . . . . . . . . . . . . . . 447 ¨ Abdulkerim C ¸ apar, Kadim Ta¸sdemir, Ozlem Kılıc, Muhittin G¨ okmen Image-Space-Parallel Direct Volume Rendering on a Cluster of PCs . . . . . 457 B. Barla Cambazoglu, Cevdet Aykanat Generalization and Localization Based Style Imitation for Grayscale Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465 Fatih Nar, Atılım C ¸ etin Robot Mimicking: A Visual Approach for Human Machine Interaction . . . 474 Algan Uskarcı, A. Aydın Alatan, M. Serdar Dindaroˇglu, Aydın Ersak Wavelet Packet Based Digital Watermarking for Remote Sensing Image Compression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 482 Seong-Yun Cho, Su-Young Han Facial Expression Recognition Based upon Gabor-Wavelets Based Enhanced Fisher Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 490 Sung-Oh Lee, Yong-Guk Kim, Gwi-Tae Park Image Sequence Stabilization Using Membership Selective Fuzzy Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 497 M. Kemal G¨ ull¨ u, Sarp Ert¨ urk Texture Segmentation Using the Mixtures of Principal Component Analyzers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 505 Mohamed E.M. Musa, Robert P.W. Duin, Dick de Ridder, Volkan Atalay Comparison of Feature Sets Using Multimedia Translation . . . . . . . . . . . . . 513 ¨ ¨ Pınar Duygulu, Ozge Can Ozcanlı, Norman Papernick
Intelligent Systems and Robotics Estimating Distributions in Genetic Algorithms . . . . . . . . . . . . . . . . . . . . . . 521 Onur Dikmen, H. Levent Akın, Ethem Alpaydın
XIV
Table of Contents
All Bids for One and One Does for All: Market-Driven Multi-agent Collaboration in Robot Soccer Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 529 Hatice K¨ ose, C ¸ etin Meri¸cli, Kemal Kaplan, H. Levent Akın Fuzzy Variance Analysis Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 537 Mahdi Jalili-Kharaajoo, Farhad Besharati Effects of the Trajectory Planning on the Model Based Predictive Robotic Manipulator Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 545 Fevzullah Temurtas, Hasan Temurtas, Nejat Yumusak, Cemil Oz Financial Time Series Prediction Using Mixture of Experts . . . . . . . . . . . . . 553 M. Serdar Yumlu, Fikret S. Gurgen, Nesrin Okay Design and Usage of a New Benchmark Problem for Genetic Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 561 ¨ coluk Emin Erkan Korkmaz, G¨ okt¨ urk U¸ A New Approach Based on Recurrent Neural Networks for System Identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 568 Adem Kalinli, Seref Sagiroglu The Real-Time Development and Deployment of a Cooperative Multi-UAV System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 576 Ali Haydar G¨ oktoˇgan, Salah Sukkarieh, G¨ ur¸ce I¸sikyildiz, Eric Nettleton, Matthew Ridley, Jong-Hyuk Kim, Jeremy Randle, Stuart Wishart The Modular Genetic Algorithm: Exploiting Regularities in the Problem Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 584 Ozlem O. Garibay, Ivan I. Garibay, Annie S. Wu A Realistic Success Criterion for Discourse Segmentation . . . . . . . . . . . . . . 592 ¨ coluk Meltem Turhan Y¨ ondem, G¨ okt¨ urk U¸ Nonlinear Filtering Design Using Dynamic Neural Networks with Fast Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 601 Yasar Becerikli Prediction of Protein Subcellular Localization Based on Primary Sequence Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 611 ¨ Mert Ozarar, Volkan Atalay, Reng¨ ul C ¸ etin Atalay Implementing Agent Communication for a Multi-agent Simulation Infrastructure on HLA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 619 Erek G¨ okt¨ urk, Faruk Polat
Table of Contents
XV
Multimedia Improved POCS-Based De-blocking Algorithm for Block-Transform Compressed Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 627 Yoon Kim, Chun-Su Park, Kyunghun Jang, Sung-Jea Ko Lossy Network Performance of a Rate-Control Algorithm for Video Streaming Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 635 Aylin Kantarcı, Turhan Tunalı A Hierarchical Architecture for a Scalable Multicast . . . . . . . . . . . . . . . . . . . 643 Abderrahim Benslimane, Omar Moussaoui Effect of the Generation of MPEG-Frames within a GOP on Queueing Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 651 Jos´e-Carlos L´ opez-Ardao, Manuel Fern´ andez-Veiga, Ra´ ul-Fernando Rodr´ıguez-Rubio,, C´ andido L´ opez-Garc´ıa, Andr´es Su´ arez-Gonzalez, Diego Teijeiro-Ruiz Multiple Description Coding for Image Data Hiding in the Spatial Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 659 Mohsen Ashourian, Yo-Sung Ho POCS-Based Enhancement of De-interlaced Video . . . . . . . . . . . . . . . . . . . . 667 Kang-Sun Choi, Jun-Ki Cho, Min-Cheol Hwang, Sung-Jea Ko A New Construction Algorithm for Symmetrical Reversible Variable-Length Codes from the Huffman Code . . . . . . . . . . . . . . . . . . . . . . . 675 Wook-Hyun Jeong, Yo-Sung Ho A Solution to the Composition Problem in Object-Based Video Coding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 683 Jeong-Woo Lee, Yo-Sung Ho Real-Time Advanced Contrast Enhancement Algorithm . . . . . . . . . . . . . . . . 691 Tae-Chan Kim, Chang-Won Huh, Meejoung Kim, Bong-Young Chung, Soo-Won Kim A Video Watermarking Algorithm Based on the Human Visual System Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 699 Ji-Young Moon, Yo-Sung Ho Network-Aware Video Redundancy Coding with Scene-Adaptation for H.263+ Video . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 707 Jae-Young Pyun, Jae-Hwan Jeong, Kwang-Il Ji, Kyunghun Jang, Sung-Jea Ko Scheduling Mixed Traffic under Earliest-Deadline-First Algorithm . . . . . . 715 Yeonseung Ryu
XVI
Table of Contents
Fast Mode Decision for H.264 with Variable Motion Block Sizes . . . . . . . . . 723 Jeyun Lee, Byeungwoo Jeon An Optimal Scheduling Algorithm for Stream Based Parallel Video Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 731 D. Turgay Altılar, Yakup Paker
Networks and Security A Practical Approach for Constructing a Parallel Network Simulator . . . . 739 Yue Li, Depei Qian, Wenjie Zhang Distributed Multicast Routing for Efficient Group Key Management . . . . . 747 John Felix C, Valli S Multi-threshold Guard Channel Policy for Next Generation Wireless Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 755 Hamid Beigy, M.R. Meybodi Application of Fiat-Shamir Identification Protocol to Design of a Secure Mobile Agent System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 763 Seongyeol Kim, Okbin Lee, Yeijin Lee, Yongeun Bae, Ilyong Chung Neural Network Based Optical Network Restoration with Multiple Classes of Traffic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 771 Demeter G¨ okı¸sık, Semih Bilgen Network Level Congestion Control in Mobile Wireless Networks: 3G and Beyond . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 779 Seungcheon Kim Network Dependability: An Availability Measure in N-Tier Client/Server Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 786 Fl´ avia Est´elia Silva Coelho, Jacques Philippe Sauv´e, Cl´ audia Jacy Barenco Abbas, Luis Javier Garc´ıa Villalba One-Time Passwords: Security Analysis Using BAN Logic and Integrating with Smartcard Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . 794 Kemal Bicakci, Nazife Baykal Design and Implementation of a Secure Group Communication Protocol on a Fault Tolerant Ring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 802 ¨ ur Saˇglam, Mehmet Emin Dalkılı¸c, Kayhan Erciye¸s Ozg¨ A New Role-Based Delegation Model Using Sub-role Hierarchies . . . . . . . . 811 HyungHyo Lee, YoungRok Lee, BongHam Noh POLICE: A Novel Policy Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 819 ¨ Taner Dursun, B¨ ulent Orencik
Table of Contents
XVII
Covert Channel Detection in the ICMP Payload Using Support Vector Machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 828 Taeshik Sohn, Jongsub Moon, Sangjin Lee, Dong Hoon Lee, Jongin Lim An Efficient Location Area Design Scheme to Minimize Registration Signalling Traffic in Wireless Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 836 ¨ Umit Aslıhak, Feza Buzluca A Simple Pipelined Scheduling for Input-Queued Switches . . . . . . . . . . . . . 844 Sang-Ho Lee, Dong-Ryeol Shin Transport Protocol Mechanisms for Wireless Networking: A Review and Comparative Simulation Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 852 ¨ ¨ Alper Kanak, Oznur Ozkasap Performance Analysis of Packet Schedulers in High-Speed Serial Switches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 860 Oleg Gusak, Neal Oliver, Khosrow Sohraby Practical Security Improvement of PKCS#5 . . . . . . . . . . . . . . . . . . . . . . . . . . 869 Sanghoon Song, Taekyoung Kwon, Ki Song Yoon Access Network Mobility Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 877 Sang-Hwan Jung, Do-Hyeon Kim, You-Ze Cho Design of a Log Server for Distributed and Large-Scale Server Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 885 ¨ ˙ Attila Ozgit, Burak Dayıoˇglu, Erhan Anuk, Inan Kanbur, Ozan Alptekin, Umut Ermi¸s On Fair Bandwidth Sharing with RED . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 892 Diego Teijeiro-Ruiz, Jos´e-Carlos L´ opez-Ardao, Manuel Fern´ andez-Veiga, C´ andido L´ opez-Garc´ıa, Andr´es Su´ arez-Gonzalez, Ra´ ul-Fernando Rodr´ıguez-Rubio, Pablo Argibay-Losada
Parallel and Distributed Computing PES: A System for Parallelized Fitness Evaluation of Evolutionary Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 900 Onur Soysal, Erkin Bah¸ceci, Erol S ¸ ahin Design and Evaluation of a Cache Coherence Adapter for the SMP Nodes Interconnected via Xcent-Net . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 908 Sangman Moh, Jae-Hong Shim, Yang-Dong Lee, Jeong-A Lee, Beom-Joon Cho
XVIII Table of Contents
Low Cost Coherence Protocol for DSM Systems with Processor Consistency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 916 Jerzy Brzezi´ nski, Michal Szychowiak Minimizing Communication Cost in Fine-Grain Partitioning of Sparse Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 926 Bora U¸car, Cevdet Aykanat Scalability and Robustness of Pull-Based Anti-entropy Distribution Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 934 ¨ ¨ Oznur Ozkasap Extended Partial Path Heuristic for Real-Time Staging in Oversubscribed Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 942 ¨ uner Mohammed Eltayeb, Atakan Doˇgan, F¨ usun Ozg¨
Soft Computing Signal Compression Using Growing Cell Structures: A Transformational Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 952 Borahan T¨ umer, Bet¨ ul Demir¨ oz A New Continuous Action-Set Learning Automaton for Function Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 960 Hamid Beigy, M.R. Meybodi A Selectionless Two-Society Multiple-Deme Approach for Parallel Genetic Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 968 Adnan Acan Gene Level Concurrency in Genetic Algorithms . . . . . . . . . . . . . . . . . . . . . . . 976 ¨ coluk Onur Tolga S ¸ ehito˘glu, G¨ okt¨ urk U¸ Fuzzy Cluster Analysis of Spatio-Temporal Data . . . . . . . . . . . . . . . . . . . . . . 984 Zhijian Liu, Roy George
Software Engineering Multi-agent Based Integrated Framework for Intra-class Testing of Object-Oriented Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 992 P. Dhavachelvan, G.V. Uma Test Case Generation According to the Binary Search Strategy . . . . . . . . . 1000 Sami Beydeda, Volker Gruhn Describing Web Service Architectures through Design-by-Contract . . . . . . 1008 Sea Ling, Iman Poernomo, Heinz Schmidt Modeling Web Systems Using SDL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1019 Joe Abboud Syriani, Nashat Mansour
Table of Contents
XIX
Software Quality Improvement Model for Small Organizations . . . . . . . . . . 1027 Rabih Zeineddine, Nashat Mansour Representing Variability Issues in Web Applications: A Pattern Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1035 Rafael Capilla, N. Yasemin Topaloglu Modeling and Analysis of Service Interactions in Service-Oriented Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1043 Woo Jin Lee Designing Reusable Web-Applications by Employing Enterprise Frameworks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1051 Marius Dragomiroiu, Robert Gyorodi, Ioan Salomie, Cornelia Gyorodi
Multimedia Modeling and the Security in the Next Generation Network Information Systems Multimedia Synchronization Model for Two Level Buffer Policy in Mobile Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1060 Keun-Wang Lee, Jong-Hee Lee, Kwang-Hyoung Lee SSE-CMM BPs to Meet the Requirements of ALC DVS.1 Component in CC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1069 Sang-ho Kim, Eun-ser Kim, Choon-seong Leem, Ho-jun Shin, Tai-hoon Kim Improved Structure Management of Gateway Firewall Systems for Effective Networks Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1076 Si Choon Noh, Dong Chun Lee, Kuinam J. Kim Supplement of Security-Related Parts of ISO/IEC TR 15504 . . . . . . . . . . . 1084 Sang-ho Kim, Choon-seong Leem, Tai-hoon Kim, Jae-sung Kim New CX Searching Algorithm for Handoff Control in Wireless ATM Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1090 Dong Chun Lee, Gi-sung Lee, Chiwon Kang, Joong Kyu Park Performance Improvement Scheme of NIDS through Optimizing Intrusion Pattern Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1098 Jae-Myung Kim, Kyu-Ho Lee, Jong-Seob Kim, Kuinam J. Kim
Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1107
Review of Experiments in Self-Aware Networks Erol Gelenbe Dennis Gabor Chair Department of Electrical and Electronic Engineering Imperial College London SW7 1BT
[email protected] Abstract. We show how “self-awareness”, through on-line selfmonitoring and measurement, coupled with intelligent adaptive behaviour in response to observed data, can be used to offer quality of service to network users based on the “Cognitive Packet Network” (CPN) design.
1
Introduction
At the periphery of the Internet novel networked systems are emerging to offer user oriented flexible services, using the Internet and LANs to reach different parts of the same systems, and to access other networks, users and services. Examples include Enterprise Networks, Home Networks (Domotics), Networks for Military Units and for Emergency Services. The example of home networks is significant in that a family may be interconnected as a unit with PDAs for the parents and children, health monitoring devices for the grand-parents, video cameras connected to the network in the infants’ bedroom, connections to smart home appliances, the home education server, the entertainment center, the security system, etc.. As an example, the home network will simultaneously use different wired and wireless communication modalities including WLAN, 3G, wired Ethernet and will tunnel packets through IP in the Internet. Such systems must allow for diverse Quality of Service (QoS) requirements, and they raise interesting issues of intelligence and adaptation to user needs and to the networking environment, including routing, self healing, security and robustness. the “Cognitive Packet Network (CPN)” [7,8,10,6]. CPN has the ability to address “QoS” in a significantly broader sense than typically employed in networking. Examples of QoS Goals that a user can request of a CPN include: “Get me the data object Ob via the path(s) of highest bandwidth which you can find”, where Ob is the handle of some data object [2], or “Find the paths with least power consumption to the mobile user M n”, “Get the video output V i to my PDA as quickly as possible”. The CPN Accepts Direction, by inputing Goals prescribed by users. It exploits Self-Observation with the help of smart
This research is supported by U.S. Army PEOSTRI via NAWC under Contract No. N61339-02-C0117, and NSF under Grants No. EIA0086251 and EIA0203446.
A. Yazici and C. S ¸ ener (Eds.): ISCIS 2003, LNCS 2869, pp. 1–8, 2003. c Springer-Verlag Berlin Heidelberg 2003
2
E. Gelenbe
packets (SPs) so as to be aware of its state including connectivity of fixed or mobile nodes, power levels at mobile nodes, topology, paths and path QoS. It performs Self-Improvement, and Learns from the experience of smart packets using neural networks and genetic algorithms to determine routing schemes with better QoS. It will Deduce hitherto unknown routes by combining or modifying paths which have been previously learned so as to improve QoS and robustness. CPN makes use of four types of packets: smart packets (SP) for discovery, source routed dumb packets (DP) to carry payload, source routed probe packets (P) to test paths, acknowledgments (ACK) to bring back information that has been discovered by SPs and Ps. Conventional IP packets tunnel through CPN to seamlessly operate mixed IP and CPN networks. SPs are be generated by a user (1) requesting that a path having some QoS value be created to some CPN node, or (2) requesting to discover parts of network state, including location of certain fixed or mobile nodes, power levels at nodes, topology, paths and their QoS. SPs exploit the experience of other packets using random neural network (RNN) based Reinforcement Learning (RL) [3,7]. RL will be carried out using a Goal which is specified by the user who generated a request for a connection. The decisional weights of a RNN will be increased or decreased based on the observed success or failure of subsequent SPs to achieve the Goal. Thus RL will tend to prefer better routing schemes, more reliable access paths to data objects, and better QoS. (2) Secondly, the system can deduce new paths to users, nodes and data objects by combining previously discovered paths, and using the estimated or measured QoS values of new paths select the best new paths. This is similar conceptually to a genetic algorithm which generates new entities by combination or mutation of existing entities, and then selects the best among them using a fitness function. These new paths will then be tested by forwarding Ps so that the actual QoS or success can be evaluated. When a SP or P arrives to its destination, an ACK is generated and heads back to the source of the request. It updates mailboxes (MBs) in the CPN nodes it visits with information which has been discovered, and provides the source node with the successful path to the node. All packets have a life-time constraint based on the number of nodes visited, to avoid overburdening the system with unsuccessful requests. A node in the CPN acts as a storage area for packets and mailboxes (MBs). It also stores and executes the code used to route smart packets. It has an input buffer for packets arriving from the input links, a set of mailboxes, and a set of output buffers which are associated with output links. CPN software is integrated into the Linux kernel 2.2.x, providing a single application program interface (API) for the programmer to access CPN. CPN routing algorithms also run seamlessly on ad-hoc wireless and wired connections [9], without specific dependence on the nature (wired or wireless) of the links, using QoS awareness to optimize behavior across different connection technologies and wireless protocols. Smart packet routing outlined above is carried out by code stored in each router whose parameters are updated at the router. For each successive smart packet, the router computes the appropriate outgoing link based on the outcome of this computation. A recurrent RNN with as many “neurons” as there are
Review of Experiments in Self-Aware Networks
3
possible outgoing links, is used in the computation. The weights of the RNN are updated so that decision outcomes are reinforced or weakened depending on how they have contributed to the success of the QoS goal. In the RNN [1] the state qi of the i − th neuron in the network is the probability that the i − th neuron is excited. Each neuron i is associated with a distinct outgoing link at a node. The qi satisfy the system of non-linear equations: qi = λ+ (i)/[r(i) + λ− (i)], where λ+ (i) =
j
+ qj wji + Λi ,
λ− (i) =
j
(1) − qj wji + λi ,
(2)
+ wji is the rate at which neuron j sends “excitation spikes” to neuron i when j − is the rate at which neuron j sends “inhibition spikes” to neuron is excited, wji i when j is excited, and r(i) is the total firing rate from the neuron i. For an n neuron network, the network parameters are these n by n “weight matrices” W+ = {w+ (i, j)} and W− = {w− (i, j)} which need to be “learned” from input data. RL is used in CPN as follows. Each node stores a specific RNN for each active source-destination pair, and each QoS class. The number of nodes of the RNN are specific to the router, since (as indicated earlier) each RNN node will represent the decision to choose a given output link for a smart packet. Decisions are taken by selecting the output link j for which the corresponding neuron is the most excited, i.e. qi ≤ qj for all i = 1, .. , n. Each QoS class for each sourcedestination pair has a QoS Goal G, which expresses a function to be minimized, e.g., Transit Delay or Probability of Loss, or Jitter, or a weighted combination, and so on. The reward R which is used in the RL algorithm is simply the inverse of the goal: R = G−1 . Successive measured values of R are denoted by Rl , l = 1, 2, ..; These are first used to compute the current value of the decision threshold: (3) Tl = aTl−1 + (1 − a)Rl ,
where 0 < a < 1, typically close to 1. Suppose we have now taken the l − th decision which corresponds to neuron j, and that we have measured the l − th reward Rl . We first determine whether the most recent value of the reward is larger than the previous value of the threshold Tl−1 . If that is the case, then we increase very significantly the excitatory weights going into the neuron that was the previous winner (in order to reward it for its new success), and make a small increase of the inhibitory weights leading to other neurons. If the new reward is not greater than the previous threshold), then we simply increase moderately all excitatory weights leading to all neurons, except for the previous winner, and increase significantly the inhibitory weights leading to the previous winning neuron (in order to punish it for not being very successful this time). Let us denote by ri the firing rates of the neurons before the update takes place: ri =
n 1
[w+ (i, m) + w− (i, m)],
(4)
4
E. Gelenbe
We first compute Tl−1 and then update the network weights as follows for all neurons i = j: – If Tl−1 ≤ Rl • w+ (i, j) ← w+ (i, j) + Rl , Rl • w− (i, k) ← w− (i, k) + n−2 , if k = j. – Else Rl ,k = j, • w+ (i, k) ← w+ (i, k) + n−2 − − • w (i, j) ← w (i, j) + Rl . Since the relative size of the weights of the RNN, rather than the actual values, determine the state of the neural network, we then re-normalize all the weights by carrying out the following operations. First for each i we compute: ri∗ =
n
[w+ (i, m) + w− (i, m)],
(5)
1
and then re-normalize the weights with: w+ (i, j) ← w+ (i, j) ∗ rr∗i , i w− (i, j) ← w− (i, j) ∗ rr∗i . i
Finally, the probabilities qi are computed using the non-linear iterations (1), (2). The largest of the qi ’s is again chosen to select the new output link used to send the smart packet forward. This procedure is repeated for each smart packet for each QoS class and each source-destination pair.
2
Review of Some Experiments
Cold Start Set-Up Time Measurements. One of the major requirements of a CPN is that it should be able to start itself with no initial information, by first randomly searching, and then progressively improving its behaviour through experience. Since the major function of a network is to transfer packets from some source S to some destination D, it is vital that the CPN be able to establish a path from S to D even when there is no prior information available in the network. The network topology we have used in these experiments is shown at the top of Figure 1, with the source and destinations nodes marked at the left and right ends of the diagram. The network contains 24 nodes, and each node is connected to 4 neighbours. Because of the possibility of repeatedly visiting the same node on a path, the network contains an unlimited number of paths from S to D. However, the fact that SPs are destroyed after they visit 30 nodes, does limit this number though it still leaves a huge number of possible paths. In this set of experiments, the network is always started with empty mailboxes, i.e. with no prior information about which output link is to be used from a node, and with neural network weights set at identical values, so that the neural network descison algorithm at nodes initially will produce a random choice. Each point
Review of Experiments in Self-Aware Networks
5
Fig. 1. CPN Network Topology for Cold Start Experiments Set−Up Measurements from Cold Start (Empty Network)
Set−Up Time Measurements from Cold Start (Empty Network) 2
0
10
Average Set−Up Time
Probability that the Requested Path is Set Up (ACK Received)
10
−1
1
10 0 10
1
2
3
10 10 10 Number of Smart Packets Successively Sent into the Network
10
4
0
10
10
1
2
10
10
3
4
10
10
Number of Smart Packets Succesively Sent into the Network
Fig. 2. Average Network Set-Up Time (Left) and Probability of Successful Connection (Right) from Cold Start, as a Function of the Initial Number of Smart Packets 2
Set−Up Measurements from Cold Start (Empty Network)
Average Number of Smart Packets Needed to get a Valid Path
10
1
10
0
10 0 10
1
2
3
10 10 10 Number of Smart Packets Succesively Sent into the Network
4
10
Fig. 3.
shown on the curves of Figure 1 is a result of 100 repetitions of the experiment under identical starting conditions. Let us first comment on the left-hand topmost curve. An absissa value of 10 indicates that the number of SPs used was 10, and – assuming that the experiment resulted in an ACK packet coming back to the source – the ordinate gives the average time (over the 100 experiments) that it elapse between the instant that the first SP was sent out, and the first ACK comes back. Note that the first ACK will be coming back from the correct destination node, and that it will br bringing back a valid forward path that can be used by the subsequent useful traffic. We notice that the average set-up time decreases significantly when we go from a few SPs to about 10, and after that, the average set-up time does not improve appreciably. Its value somewhere between 10 and 20 milliseconds actually corresponds to the round-trip transit time through the hops. This does not mean that it suffices to have a small number
6
E. Gelenbe eth0:10.0.10.10 eth1:10.0.12.10 eth2:10.0.14.10 eth3:10.0.15.10
eth3
eth0:10.0.10.30 eth1:10.0.13.30 eth2:192.168.2.30
10.0.10.0
eth0
CPN-NODE a10
eth0
.0
2.0 .0.1 10
.0 .1 5
ISP1 Router_1
eth 1
1 eth
10
eth2
IP CPN-NODE a30
eth1
10.0.14.0
IP eth0
INTERNET
CPN-NODE a100 eth2
.0.1
eth1:10.0.17.7
1
10
1 eth
10.0.17.0
3.0
.0 6 .1 .0 0 1
eth0:10.0.17.100 eth1:10.0.15.100 eth2:10.0.16.100
eth
eth3
10.0.11.0 CPN-NODE a20
IP
eth0
eth0
eth0:X.X.X.X
ISP2
eth2
CPN-NODE a40
eth0:10.0.11.20 eth1:10.0.13.20 eth2:10.0.14.20 eth3:10.0.16.20
HTTP - SERVER
Router_2
eth0:10.0.11.40 eth1:10.0.12.40 eth2:192.168.1.40
INTERNET HTTP Client
Fig. 4. Experimental Set-Up for Dynamic QoS Control Load Balancing − a40 100
80
90
70
80
Percentage of the total BW used by a40
Percentage of the total BW used by a30
Load Balancing − a30 90
60
50
40
30
20
70
60
50
40
30
10
20
0 −200
−150
−100
−50
0 50 delay a30 − a40 (ms)
100
150
10 −200
200
−150
−100
−50
0 50 delay a30 − a40 (ms)
100
150
200
Fig. 5. Percentage of traffic flow through ports A30 (L) and A40 (R) function of difference in delay Traffic − a30 [0,0]
Traffic − a40 [0,0]
900
800
800
700
700 600 600
Kbytes/s
Kbytes/s
500 500
400
400
300 300 200 200
100
100
0
0
50
100
150 seconds
200
250
0
300
0
50
100
150 seconds
200
250
300
Fig. 6. Instantaneous traffic flow through ports A30 (L) and A40 (R) when delays are identical Traffic − a30 [160,0]
Traffic − a40 [160,0]
600
1000
900 500 800
700 400
Kbytes/s
Kbytes/s
600
300
500
400 200 300
200 100 100
0
0
50
100
150 seconds
200
250
0
300
0
50
100
150 seconds
200
250
300
Fig. 7. Instantaneous traffic flow through ports A30 (L) and A40 (R) when DA30-DA40 = 160ms Traffic − a40 [0,160] 500
900
450
800
400
700
350
600
300
Kbytes/s
Kbytes/s
Traffic − a30 [0,160] 1000
500
250
400
200
300
150
200
100
100
0
50
0
50
100
150 seconds
200
250
300
0
0
50
100
150 seconds
200
250
300
Fig. 8. Instantaneous traffic flow through ports A30 (L) and A40 (R) when DA30-DA40 = -160ms
Review of Experiments in Self-Aware Networks
7
of SPs at the beginning, simply because the average set-up time is only being measured for the SPs which are successful; unsuccessful SPs are destroyed after 30 hops. Thus the top-most curve on the righ-hand-side of Figure 1 is needed to obtain a more complete understanding of what is happening. Again for an x-axis value of over 10 packets, we see that the probability of successfully setting up a path is 1, while with a very small number of packets this figure drops down to about 0.65. These probabilities must of course be understood as the empirically observed fraction of the 100 tests which result in a successful connection. The conclusion from these two sets of data is that to be safe, starting with an empty system, a fairly small number of SPs, in the range of 20 to 100, will provide almost guaranteed set-up of the connection, and the minimum average set-up time. The third curve that we show as a result of these experiments provides some insight into the dynamics of the path set-up. Inserting SPs into the network is not instantaneous, and they are fed into the network sequentially by the source. The rate at which they are fed in is determined by the processing time per packet at the source, and also by the link speeds. Since the link speed is 100M b/s and because SPs are only some 200Bytes long at most, we think that the limiting factor here is the source node’s processing time. Since, on the other hand, the previous curves show that connections are almost always established with as few as 10 SPs, and because the average round-trip connection establishment time is quite short, we would expect to see that the connection is generally established before all the SPs are sent out by the source. This is exactly what we observe on this third curve. The x axis shows the number of SPs sent into the network, while the y axis shows the average number sent in (over the 100 experiments) before the first ACK is received. For small numbers of SPs, until the value 10 or so, the relationship is linear. However as the number of SPs being inserted into the network increases, we see that after (on the average) 13 packets or so have been sent out, the connection is already established (i.e. the first ACK has returned to the source). This again indicates that a fairly small number of SPs suffice to establish a connection. Dynamic QoS Control over the Internet. The Internet uses the Internet Protocol (IP) is a de facto standard. Thus any new system will have to inter-operate seamlessly with this existing world-wide system. We demonstrate experimentally the seamless operation of a CPN together with the Internet, and show how a CPN can be used to control QoS in the Internet according to requests formulate by a user. In order to provide this practical demonstration, we set up the experimental environment described at the top of Figure 1. The Workstation (W) shown at the bottom of the figure is used to generate requests to the web server (WS) shown at the left-hand side of the figure. The WS responds to these requests by generating standard Internet IP packets which enter into the CPN CPN set-up at the top of the figure, so that the WS creates connections back to W which tunnel through the CPN. These packets then are dynamically directed back into the Internet via two distinct Internet Service Providers (ISP) ports A30 and A40 shown at the right-hand-side of the figure. From there they merge into the Internet, and return to the W as shown
8
E. Gelenbe
on the figure. Thus the experimental set-up demonstrates that we are able to seamlessly transmit IP packets into a CPN, and that the CPN can forward them back into the Internet again seamlessly. In the experiments we have run using this set-up, user QoS requested by the W as it makes requests to the WS has been selected to be to minimize delay. Thus we have artificially introduced different delay values at A430 and A40 so that the difference in delay between the two can be varied. The second top-most set of curves show the fraction of traffic taking A30 (L) and A40 (R) as this delay is varied, and demonstrates that the CPN is indeed dynamically directing traffic quite sharply in response to the user’s QoS goal. Figure 6 shows that when the delay through either port is identical, the instantaneous traffic is very similar. On the other hand, Figures 7 and 8 clearly show that the instantaneous traffic strongly differs depending on which port has a higher measured delay.
Acknowledgements. Special thanks go to my former and current PhD students who have contributed to and participated to this research, including Dr. Zhiguan Xu, Dr. Ricardo Lent, and Juan Arturo Nunez, Peixiang Liu, Pu Su, Michael Gellman.
References 1. E. Gelenbe. “Learning in the recurrent random neural network”, Neural Comp. 5(1), 154–164, 1993. 2. R. E. Kahn, R. Wilensky “A framework for digital object services”, c.nri.dlib/tn9501. 3. U. Halici, “Reinforcement learning with internal expectation for the random neural network” Eur. J. Opns. Res., 126 (2) 2, 288–307, 2000. 4. E. Gelenbe, E. S ¸ eref, Z. Xu. “Simulation with learning agents”, Proc. IEEE, Vol. 89 (2), 148–157, 2001. 5. E. Gelenbe, R. Lent, Z. Xu, “Towards networks with cognitive packets”, Opening Invited Paper, International Conference on Performance and QoS of Next Generation Networking, Nagoya, Japan, November 2000, in K. Goto, T. Hasegawa, H. Takagi and Y. Takahashi (eds), “Performance and QoS of next Generation Networking”, Springer Verlag, London, 2001. 6. E. Gelenbe, R. Lent, Z. Xu, “Design and performance of cognitive packet networks”, Performance Evaluation, 46, pp. 155–176, 2001. 7. E. Gelenbe, R. Lent, Z. Xu “Measurement and performance of Cognitive Packet Networks”, J. Comp. Nets., 37, 691–701, 2001. 8. E. Gelenbe, R. Lent, Z. Xu “Networking with Cognitive Packets”, Proc. ICANN., Madrid, August 2002. 9. E. Gelenbe, R. Lent “Mobile Ad-Hoc Cognitive Packet Networks”, Proc. IEEE ASWN, Paris, July 2–4, 2002. 10. E. Gelenbe et al. “Cognitive packet petworks: QoS and performance”, Keynote Paper, IEEE MASCOTS Conference, San Antonio, TX, Oct. 14–16, 2002. 11. E. Gelenbe, K. Hussain “Learning in the multiple class random neural network”, IEEE Trans. on Neural Networks 13 (6), 1257–1267, 2002.
Web Information Resource Discovery: Past, Present, and Future Gultekin Ozsoyoglu and Abdullah Al-Hamdani Dept of Electrical Engineering and Computer Science Case Western Reserve University, Cleveland, Ohio 44106 WHNLQDEG #HHFVFZUXHGX
1 Introduction ,QDWLPHVSDQRIWZHOYH\HDUVWKH:RUOG:LGH:HERQO\DFRPSXWHUDQGDQLQWHUQHW FRQQHFWLRQDZD\IURPDQ\ERG\DQ\ZKHUHDQGZLWKDEXQGDQWGLYHUVHDQGVRPHWLPHV LQFRUUHFWUHGXQGDQW VSDP DQG EDG LQIRUPDWLRQKDV EHFRPH WKH PDMRU LQIRUPDWLRQ UHSRVLWRU\IRUWKHPDVVHVDQGWKHZRUOG7KHZHELVEHFRPLQJDOOWKLQJVWRDOOSHRSOH WRWDOO\ REOLYLRXV WR QDWLRQFRXQWU\FRQWLQHQW ERXQGDULHV SURPLVLQJ PRVWO\ IUHH LQIRUPDWLRQ WR DOO DQG TXLFNO\ JURZLQJ LQWR D UHSRVLWRU\ LQ DOO ODQJXDJHV DQG DOO FXOWXUHV :LWK ODUJH GLJLWDO OLEUDULHV DQG LQFUHDVLQJO\ VLJQLILFDQW HGXFDWLRQDO UHVRXUFHVWKHZHELVEHFRPLQJDQHTXDOL]HUDEDODQFLQJIRUFHDQGDQRSSRUWXQLW\IRU DOOHVSHFLDOO\IRUXQGHUGHYHORSHGGHYHORSLQJFRXQWULHV7KHZHELVERWKH[FLWLQJDQG RYHUZKHOPLQJFKDQJLQJWKHZD\WKHZRUOGFRPPXQLFDWHVIURPWKHZD\EXVLQHVVHV DUHFRQGXFWHGWRWKHZD\PDVVHVDUHHGXFDWHGIURPWKHZD\UHVHDUFKLVSHUIRUPHGWR WKHZD\UHVHDUFKUHVXOWVDUHGLVVHPLQDWHG,WLVIDLUWRVD\WKDWWKHZHEZLOORQO\JHW PRUHGLYHUVHODUJHUDQGPRUHFKDRWLFLQWKHQHDUIXWXUH $V LW LV WKH ZHE LV D UHSRVLWRU\ RI WH[W PXOWLPHGLD DQG K\SHUWH[W GRFXPHQWV ZKLFKDUHPRVWO\GLVSOD\RQO\+70/ GRFXPHQWVRULQIRUPDWLRQH[FKDQJH;0/ GRFXPHQWVFUHDWHGIRUWKHFRQVXPSWLRQRIZHEEDVHGDSSOLFDWLRQV 7KHZHEFRQWLQXDOO\JURZVDQGFKDQJHVZLWKDQHVWLPDWHGVL]HRI%LOOLRQ WR% SDJHV ODUJHVW ZHE LQGH[ 2SHQ)LQG ZZZRSHQILQGFRPWZ LV % SDJHV QRW LQFOXGLQJ KLGGHQ ZHE LQWUDQHWV DQG GDWDEDVHHQDEOHG SDJHV 7ZR VWUHQJWKV RI WKH ZHE DUH LW JURZV LQFUHPHQWDOO\ DQG WKXV VFDODEOH DQG HDFK LQGLYLGXDO LQ HDFK QDWLRQZLWKDFRQQHFWLRQWRWKHLQWHUQHWFDQFRQWULEXWHWRFRQWHQWJHQHUDWLRQRQWKH ZHE OHDGLQJ WR WKH GLVVHPLQDWLRQ RI IDFWV DQG SURSDJDQGD DQG VRPHWLPHV LQFRUUHFW LGHDVDQGRSLQLRQVLQDQLQGHSHQGHQWDQGYHU\GHPRFUDWLFPDQQHU $V YDOXDEOH DQG ULFK DV WKH ZHE LV SUHVHQWO\ WKHUH DUH IHZ ZD\V WR VHDUFK DQG ORFDWHLQIRUPDWLRQRQWKHZHERQHFDQXVHL WKHH[LVWLQJVHDUFKHQJLQHVWRUHDFKWRD VHOHFWVHWRIUDQNHGVLWHVLL PHWDVHDUFKHQJLQHVWKDWLQWXUQHPSOR\PXOWLSOHVHDUFK HQJLQHVDQGDJJUHJDWHDQGUDQNVHDUFKUHVXOWVLLL TXHVWLRQDQVZHULQJV\VWHPVHJ $VN-HHYHV ZZZDVNFRP WKDW DOORZ XVHUV WR SRVH TXHVWLRQV DQG UHWXUQ WKHLU DQVZHUVRURQHFDQLY IROORZOLQNVDQGEURZVHZHESDJHV,QWKLVSDSHUZHUHYLHZ WKHXQGHUO\LQJWHFKQRORJLHVIRUFROOHFWLQJLQIRUPDWLRQPHWDGDWD DERXWWKHZHEDQG HPSOR\LQJ WKHVH WHFKQRORJLHV IRU VHDUFKLQJ DQG TXHU\LQJ WKH ZHE )XOO SDSHU ZLWK ODUJHUUHIHUHQFHVLVDWDUWFZUXHGX:HE6HDUFK4XHU\LQJSGI ,Q VHFWLRQ ZH VXPPDUL]H WKH KLVWRU\ DQG FDSDELOLWLHV RI ZHE VHDUFK HQJLQHV 6HFWLRQVDQGDUHGHYRWHGWRWKHDXWRPDWHGDQGPDQXDOZD\VRIDGGLQJVHPDQWLFV WRWKHZHEWRKHOSXQGHUVWDQGDQGWKXVVHDUFKWKHZHEEHWWHU6HFWLRQRIIHUVRXU SUHGLFWLRQVRQZKDWWKHQHDUIXWXUHKROGVIRULPSURYLQJZHEVHDUFKDQGTXHU\LQJ $@ 0RUH DGYDQFHG PRGHOV LQFOXGH SUREDELOLVWLF UHOHYDQFH IHHGEDFNPRGHOVRUED\HVLDQLQIHUHQFHQHWZRUNV$VLWWXUQVRXWXVHUVUDUHO\SURYLGH IHHGEDFNDQGPRVWVHDUFKHQJLQHVGRQRWLQFRUSRUDWHUHOHYDQFHIHHGEDFNIHDWXUHV +\SHUOLQN$QDO\VLV ,Q ZHE UHWULHYDO WKH OLQNDJH LQIRUPDWLRQ DPRQJ ZHE SDJHV LV IRXQG WR EH PRUH LQIRUPDWLYHWKDQWKH,5EDVHGGRFXPHQWVLPLODULWLHV7RHPSOR\OLQNDJHLQIRUPDWLRQ FRQFHSWV IURP %LEOLRPHWULFV D ILHOG WKDW VWXGLHV WKH FLWDWLRQ SDWWHUQV RI VFLHQWLILF
:HE,QIRUPDWLRQ5HVRXUFH'LVFRYHU\3DVW3UHVHQWDQG)XWXUH
SDSHUV DUH HPSOR\HG L SUHVWLJH RI DQ DUWLFOH D UHDO QRUPDOL]HG QXPEHU UHSUHVHQWLQJWKHQXPEHURIFLWDWLRQVWRLWDQGLWHUDWLYHO\WRWKRVHWKDWUHIHUWRLWDQG VRRQDQGLL FRFLWDWLRQEHWZHHQWZRGRFXPHQWVWKHQXPEHURIGRFXPHQWVWKDWFR FLWHWKHWZRGRFXPHQWVQRUPDOL]HGE\WKHWRWDOQXPEHURIGRFXPHQWV 3DJH5DQN DQG +,76 K\SHUOLQN LQGXFHG WRSLF VHDUFK DUH WZR YHU\ LQIOXHQWLDO DOJRULWKPVIRUUDQNLQJSDJHVGHYHORSHGLQHDUO\UHVSHFWLYHO\E\ 6HUJH\ %ULQ DQG /DUU\ 3DJH DW 6WDQIRUG > @ DQG -RQ .OHLQEHUJ DW ,%0 $OPDGHQ >@ 3DJH5DQN RI D SDJH LV D QRUPDOL]HG VXP RI WKH SUHVWLJHV RI SDJHV OLQNLQJ WR LW ,Q HVVHQFH3DJH5DQNGHILQLWLRQLVEDVHGRQWKHLQWXLWLRQWKDWDSDJHKDVKLJKUDQNLH LPSRUWDQW LI WKH VXP RI WKH UDQNV RI LWV LQFRPLQJ OLQNV UHFXUVLYHO\ LV KLJK >@ 3DJH5DQNFRPSXWDWLRQFRQYHUJHVUDSLGO\IRUWKHZHE ,Q +,76 ILUVW D VHW RI GRFXPHQWV DUH VHOHFWHG IURP WKH ZHE E\ XVLQJ WKH XVHU¶V TXHU\DQGSHUKDSV,5WHFKQLTXHV2QHFDQYLHZWKHVHOHFWHGGRFXPHQWVDVQRGHVDQG WKH K\SHUOLQNV DPRQJ WKHP DV HGJHV UHVXOWLQJ LQ D ³VXEJUDSK´ RI WKH ZHE ,Q WKLV VXEJUDSK DXWKRULWDWLYH SDJHV DUH QRGHVSDJHV WR ZKLFK PDQ\ SDJHV KDYH OLQNV GLUHFWHG HGJHV DQG KXE SDJHV DUH QRGHV ZLWK PDQ\ OLQNV RXWJRLQJ HGJHV WR DXWKRULWDWLYHSDJHVDQGWKHV\VWHPUHSRUWVWKHKLJKHVWUDQNLQJDXWKRULWLHVDQGKXEVLQ WKH VHOHFWHG VHW RI GRFXPHQWV ,QWXLWLYHO\ LQ %LEOLRPHWULFV DXWKRULWLHV DUH DUWLFOHV ZLWKGHILQLWLYHKLJKTXDOLW\LQIRUPDWLRQDQGKXEVDUHKLJKTXDOLW\VXUYH\DUWLFOHV 3DJH5DQN DQG +,76 DUH VLPLODU LQ WKDW WKH\ DUH ERWK UHFXUVLYH DQG LQYROYH FRPSXWLQJ WKH HLJHQYDOXHVRI WKH DGMDFHQF\ PDWUL[ RI D VXEJUDSK RIWKHZHEXVLQJ SRZHU LWHUDWLRQV ,W LV NQRZQ WKDW WKH XWLOL]DWLRQ RI VXFK OLQNDJH LQIRUPDWLRQ LQ UHWULHYDO OHDGV WR KLJKHU UHWULHYDO HIIHFWLYHQHVV DV H[HPSOLILHG E\ *RRJOH >@ 1RWH KRZHYHU WKDW *RRJOH LV PRUH WKDQ WKH 3DJH5DQN PHFKDQLVP LW HPSOR\V SKUDVH PDWFKHV DQFKRU WH[W LQIRUPDWLRQ DQG PDWFK SUR[LPLW\ 7KH EDVLF FULWLFLVP RI 3DJH5DQN >@ LV WKDW WKH SUHVWLJH VFRUHV DUH TXHU\LQGHSHQGHQW OHDGLQJ WR D GLVFRQQHFWEHWZHHQWKHUDQNLQJDQGDJLYHQTXHU\ 6HYHUDOVWRFKDVWLFYDULDQWVRI3DJH5DQNDQG+,76DUHUHVHDUFKHGLQWKHOLWHUDWXUH HJ VWRFKDVWLF OLQN VWUXFWXUH DQDO\VLV 6$/6$ >@ PRUH VWDEOH +,76 DOJRULWKP >@ HOLPLQDWLQJ WKH HIIHFWV RI WZRZD\ VHOI UHIHUHQFLQJ QHSRWLVWLF OLQNV >@ HOLPLQDWLQJWKHHIIHFWVRIVHOIFUHDWHGFOLTXHV>@RXWOLHUHOLPLQDWLRQ>@PL[HGKXEV >@HOLPLQDWLQJWKHHIIHFWVRIWRSLFFRQWDPLQDWLRQDQGWRSLFGULIW>@DQGH[SORLWLQJ DQFKRUWH[W>@ )RFXVHG&UDZOHUV *HQHUDOSXUSRVH FUDZOHUV GR QRW PDNH DQ DWWHPSW WR FUDZO RQO\ SDJHV VDWLVI\LQJ D FHUWDLQFULWHULD,WGRHVPDNHVHQVHKRZHYHUWRFUDZOSDJHVWKDWDUHOLNHO\WRKDYHKLJK ³LPSRUWDQFH´ZKHUHLPSRUWDQFHPD\EHGHILQHGLQWHUPVRI3DJH5DQNYDOXHVRUKLJK LQGHJUHHRXWGHJUHHOLQNYDOXHVRISDJHVRUDQ\RWKHUXVHUGHILQHGIXQFWLRQ&KRHWDO >@VWXGLHVYDULRXV85/SULRULWL]DWLRQDQGNH\ZRUGVHQVLWLYHZHEVHDUFKVWUDWHJLHV ,W LV DOVR IRXQG WKDW >@ EUHDWKILUVW FUDZO TXLFNO\ ORFDWHV SDJHV ZLWK KLJKHU 3DJH5DQN 2WKHUV SURSRVHG ,5EDVHG KHXULVWLFV HJ SDJHV FRQWDLQLQJ VSHFLILF ZRUGV RU FRQWDLQLQJ SKUDVHV WKDW PDWFK D JLYHQ UHJXODU H[SUHVVLRQ VXFK DV WKH )LVK6HDUFK>@DQGLWVIROORZXS6KDUN6HDUFK>@ )RFXVHG FUDZOHUV DUH V\VWHPV ZLWK D FUDZOHU DQG D FODVVLILHU V\VWHP ZKHUH WKH FUDZOHU EDVHV LWV FUDZOLQJ VWUDWHJLHV RQ WKH MXGJPHQWV RI WKH FODVVLILHU V\VWHP SRVVLEO\DVXSHUYLVHGFODVVLILHUWUDLQHGDSULRULDQGXSGDWHGDVGRFXPHQWVDUHIRXQG
*2]VR\RJOXDQG$$O+DPGDQL
7KXV IRFXVHG FUDZOHUV VHDUFK RQO\ D VXEVHW RI WKH ZHE QRW QHFHVVDULO\ RQO\ D VSHFLILFZHEUHVRXUFH WKDWSHUWDLQVWRDVSHFLILFUHOHYDQWWRSLF$QXPEHURIIRFXVHG FUDZOHUVDUHSURSRVHGLQWKHOLWHUDWXUH>@ZKLFKGLIIHUE\WKHSURSHUWLHV RIWKHHPSOR\HGFODVVLILHUVWKHKHXULVWLFVXVHGWRMXGJHWKHLPSRUWDQFHRISDJHVWREH IHWFKHGRUWRLGHQWLI\DQGH[SORLWWKHKXEVDQGOHDUQLQJWKHVWUXFWXUHRISDWKVUHOHYDQW IRUDQGOHDGLQJWRLPSRUWDQWSDJHV>@
3 Automated Metadata Extraction from Web ,I LW ZHUH SRVVLEOH WR H[WUDFW HQWLWLHV DQG UHODWLRQVKLSV DERXW HQWLWLHV IURP ZHE GRFXPHQWV VXFK PHWDGDWD FRXOG WKHQ EH XVHG WR GHILQH PRUH SRZHUIXO TXHULHV RYHU WKH VHW RI ZHE GRFXPHQWV 7KXV WKH ILHOG RI GDWD H[WUDFWLRQ LV LPSRUWDQW WR ZHE TXHU\LQJ,QWKLVVHFWLRQZHEULHIO\OLVWDQXPEHURIUHFHQWZRUNVLQWKHILHOGRIGDWD H[WUDFWLRQZLWKDQLQWHUHVWWRXVLQJWKHPHWDGDWDIRUZHEVHDUFKDQGTXHU\LQJ ',35( >@ HPSOR\V D KDQGIXO RI WUDLQLQJ WXSOHV RI D VWUXFWXUHG UHODWLRQ 5 WKDW UHSUHVHQWV D VSHFLILF PHWDUHODWLRQVKLS DPRQJ HQWLWLHV LQ WKH GDWD WR H[WUDFW DOO WKH WXSOHV RI 5 IURP D VHW RI +70/ GRFXPHQWV &RQVLGHU WKH UHODWLRQ 52UJDQL]DWLRQ /RFDWLRQ ZLWKWKHWXSOH0LFURVRIW5HGPRQG!$VVXPHWKDW',35(HQFRXQWHUVWKH WH[W ³0LFURVRIW¶V KHDGTXDUWHUV LQ 5HGPRQG´ ZKLFK LW FKDQJHV LQWR WKH SDWWHUQ S ³675,1*!¶V KHDGTXDUWHUV LQ 675,1*!´ ',35( WKHQ VHDUFKHV WKH KWPO GRFXPHQWV IRU SKUDVHV PDWFKLQJ S $VVXPH WKDW LW HQFRXQWHUV WKH VWULQJ ³%RHLQJ¶V KHDGTXDUWHUVLQ6HDWWOH´ZKLFKUHVXOWVLQWKHQHZWXSOH%RHLQJ6HDWWOH!EHLQJDGGHG LQWR 5 7KDW LV ',35( XVHV WKH QHZ WXSOHV WR JHQHUDWH PRUH SDWWHUQV DQG XVHV WKH QHZO\JHQHUDWHGSDWWHUQVWRH[WUDFWPRUHWXSOHVDQGVRRQ 6QRZEDOO>@DQH[WHQVLRQWR',35(LPSURYHVWKHTXDOLW\RIWKHH[WUDFWHGGDWD E\ LQFOXGLQJ DXWRPDWLF SDWWHUQV DQG WXSOH HYDOXDWLRQ 4;WUDFW >@ XVHV DXWRPDWHG TXHU\EDVHG WHFKQLTXHV WR UHWULHYH GRFXPHQWV WKDW DUH XVHIXO IRU H[WUDFWLQJ D WDUJHW UHODWLRQ IURP ODUJH FROOHFWLRQ WH[W GRFXPHQWV 7KH 3URWHXV LQIRUPDWLRQ H[WUDFWLRQ V\VWHP>@XVHVILQLWHVWDWHSDWWHUQVWRUHFRJQL]H^QDPHVQRXQVYHUEVDQGRWKHU VSHFLDO IRUPV` VFHQDULR SDWWHUQ PDWFKLQJ WR H[WUDFW HYHQWV DQG UHODWLRQVKLSV IRU D JLYHQUHODWLRQDQGDQLQIHUHQFHSURFHVVWRORFDWHLPSOLFLWLQIRUPDWLRQDQGWRPDNHLW H[SOLFLW7KHQ3URWHXVFRPELQHVDOOWKHLQIRUPDWLRQDERXWDVLQJOHHYHQWXVLQJHYHQW HPHUJLQJUXOHV 7KHILHOGRIPHWD GDWDH[WUDFWLRQIURPWKHZHEKDVDORQJZD\WRJRDWWKLVVWDJH +RZHYHU ZH EHOLHYH WKDW LW SURYLGHV DQ DOWHUQDWLYH WR PDQXDO FRQWHQWJHQHUDWRU GHSHQGHQWZD\VRIDGGLQJVHPDQWLFVWRWKHZHEZKLFKLVGLVFXVVHGQH[W
4 Adding Manually-Supplied Semantics to Web 1H[W ZH EULHIO\ VXPPDUL]H WKH OHDGLQJ ZHE LQIRUPDWLRQ UHSUHVHQWDWLRQ PRGHOV ZLWK H[WHQVLYH UHVHDUFK DQG VWDQGDUGL]DWLRQ HIIRUWV QDPHO\ WKH 5HVRXUFH 'HVFULSWLRQ )UDPHZRUN5') WKHVHPDQWLFZHEDQGRQWRORJLHV 5HVRXUFH'HVFULSWLRQ)UDPHZRUN 5HVRXUFH'HVFULSWLRQ)UDPHZRUN5') >@LVGHVLJQHGWRGHVFULEHZHELQIRUPDWLRQ VRXUFHV E\ DWWDFKLQJ PHWDGDWD VSHFLILHG LQ ;0/ 5') LGHQWLILHV UHVRXUFHV XVLQJ
:HE,QIRUPDWLRQ5HVRXUFH'LVFRYHU\3DVW3UHVHQWDQG)XWXUH
8QLIRUP 5HVRXUFH ,GHQWLILHUV 85, DQG GHVFULEHV WKHP LQ WHUPV RI SURSHUWLHV DQG WKHLU YDOXHV >@ 5') LV D JUDSKEDVHG LQIRUPDWLRQ PRGHO DQG FRQVLVWV RI D VHW RI VWDWHPHQWV UHSUHVHQWHG DV WULSOHV $ WULSOH GHQRWHV DQ HGJH EHWZHHQ WZR QRGHV DQG KDVDSURSHUW\QDPHHGJH DUHVRXUFHQRGH DQGDYDOXHDQRGH $UHVRXUFHFDQEH DQ\WKLQJIURPDZHEGRFXPHQWWRDQDEVWUDFWQRWLRQ$YDOXHFDQEHDUHVRXUFHRUD OLWHUDO DQ DWRPLF W\SH 5') 6FKHPD >@GHILQHV D W\SH V\VWHP IRU 5') VLPLODU WR WKHW\SHV\VWHPVRIREMHFWRULHQWHGSURJUDPPLQJODQJXDJHVVXFKDV-DYD 5')6FKHPDDOORZVWKHGHILQLWLRQRIFODVVHVIRUUHVRXUFHVDQGSURSHUW\W\SHV7KH UHVRXUFH &ODVV LV XVHG WR W\SH UHVRXUFHV DQG WKH UHVRXUFH 7\SH LV XVHG WR W\SH SURSHUWLHV 9DULRXV SURSHUWLHV VXFK DV 6XE&ODVV2I 6XE3URSHUW\2I LV'HILQHG%\ VHH$OVR W\SH DUH DYDLODEOH 9DULRXV FRQVWUDLQWV RQ UHVRXUFHV DQG RQ SURSHUWLHV DUH GHILQHG ,WUHPDLQVWREHVHHQKRZZLGHO\5')ZLOOEHDGDSWHGE\ZHEFRQWHQWJHQHUDWRUV ,QLWLDOUHVXOWVDUHQRWHQFRXUDJLQJ(EHUKDUW>@LQYHVWLJDWHGWKHDPRXQWDQGWKHW\SH RI5')GDWDRQWKHZHE7KH5')GDWDZHUHJDWKHUHGIURPWKHZHELQ 7KHUHVXOWVLQGLFDWHWKDWL 5')LVQRWZLGHO\XVHGRQWKHZHELL 5')GDWDRQWKH ZHELVQRWHDVLO\UHDFKDEOHDQGLLL LWLVQRWYHU\VRSKLVWLFDWHG 6HPDQWLF:HE 6HPDQWLF:HE>@LVDQ5')VFKHPDEDVHGHIIRUWWRGHILQHDQDUFKLWHFWXUHIRUWKH ZHEZLWKDVFKHPDOD\HUORJLFDOOD\HUDQGDTXHU\ODQJXDJH7KHRYHUDOOJRDORIWKH VHPDQWLF ZHE LV WR VXSSRUW ULFKHU LQIRUPDWLRQ GLVFRYHU\ GDWD LQWHJUDWLRQ WDVN DXWRPDWLRQ DQG QDYLJDWLRQE\ SURYLGLQJ VWDQGDUGV DQG WHFKQLTXHV IRU ZHE FRQWHQW JHQHUDWRUVWRDGGPRUHVHPDQWLFVWRWKHLUZHEGDWD>@ $ FRPSOH[ VHW RI OD\HUV RI ZHE WHFKQRORJLHV DQG VWDQGDUGV DUH GHILQHG WR LPSOHPHQWWKHVHPDQWLFZHE>@ 7KH8QLFRGHDQG85, OD\HUV DUHXVHG WR LGHQWLI\ REMHFWV LQ WKH VHPDQWLF ZHE DQG WR PDNH VXUH WKDW LQWHUQDWLRQDO FKDUDFWHU VHWV DUH XVHG 7KH ;0/ OD\HU LV XVHG WR LQWHJUDWH WKH VHPDQWLF ZHE GHILQLWLRQV ZLWK RWKHU ;0/EDVHGVWDQGDUGV7KH5')DQG5')6FKHPDOD\HUDUHXVHGWRGHILQHVWDWHPHQWV DERXWREMHFWVDQGYRFDEXODULHVWKDWFDQEHUHIHUUHGXVLQJ85,V7KH2QWRORJ\OD\HULV XVHGWRHYDOXDWHYRFDEXODULHVDQGWRGHILQHUHODWLRQVKLSVEHWZHHQGLIIHUHQWFRQFHSWV 7KH'LJLWDO6LJQDWXUHOD\HUGHWHFWVDOWHUDWLRQVWRGRFXPHQWV7KH/RJLFOD\HULVXVHG WR ZULWH UXOHV WKDW DUH H[HFXWHG E\ WKH 3URRI OD\HU ZKLOH WKH 7UXVW OD\HU LV XVHG WR GHFLGHZKHWKHUWRWUXVWDJLYHQSURRIRUQRW ,WUHPDLQVWREHVHHQZKHWKHUWKHFRQFHSWVDQGVWDQGDUGVGHILQHGE\WKHVHPDQWLF ZHEHIIRUWZLOOEHDGDSWHG2QHPDMRUSUREOHPLVWKHFRPSOH[LW\RIWKHVHPDQWLFZHE DVGHILQHGQRZ7KH6HPDQWLF:HELVDQDFWLYHLQGXVWU\OHGUHVHDUFKDUHD 2QWRORJLHV $QRQWRORJ\LVDVSHFLILFDWLRQRIDFRQFHSWXDOL]DWLRQLHPHWDLQIRUPDWLRQ >@,WLV XVHGWRGHVFULEHWKHVHPDQWLFVRIWKHGDWDZLWKDUROHVLPLODUWRWKHGDWDEDVHVFKHPD >@ 2QWRORJLHV HVWDEOLVK D MRLQW WHUPLQRORJ\ EHWZHHQ PHPEHUV RI D FRPPXQLW\ RI LQWHUHVW $Q H[DPSOH LV WKH *HQH 2QWRORJ\ ZZZJHQHRQWRORJ\RUJ IRU JHQHWLFLVWV DQG ELRORJLVWV 7R UHSUHVHQW D FRQFHSWXDOL]DWLRQ D UHSUHVHQWDWLRQ ODQJXDJH FDQ EH
*2]VR\RJOXDQG$$O+DPGDQL
DQGXVXDOO\DUHQRW XVHGDQGWKHUHDUHVHYHUDOUHSUHVHQWDWLRQODQJXDJHV>@5') DQG5')6FKHPDEHLQJDPRQJWKHP +RUURFNV HW DO >@ SURSRVHG WKH 2QWRORJ\ ,QIHUHQFH /D\HU 2,/ ZKLFK LV D VWDQGDUG IRU D ZHEEDVHG UHSUHVHQWDWLRQ DQG LQIHUHQFH OD\HU WR H[SUHVV RQWRORJLHV EDVHG RQ 5') DQG ;0/ VFKHPDV 2,/ SURYLGHV ULFK PRGHOLQJ SULPLWLYHV IURP IUDPHEDVHG ODQJXDJHV D ZHOOGHILQHG VHPDQWLFV EDVHG RQ 'HVFULSWLRQ /RJLF DQG DXWRPDWHG UHDVRQLQJ VXSSRUW 2QWRORJLHV DUH GHVFULEHV LQ 2,/ XVLQJ WKUHH GLIIHUHQW OD\HUVWKHREMHFWOHYHOWKHILUVWPHWDOHYHORQWRORJ\GHILQLWLRQ DQGWKHVHFRQGPHWD OHYHORQWRORJ\FRQWDLQHU 7KHREMHFWOHYHOXVHGWRGHVFULEHFRQFUHWHLQVWDQFHVIRUD JLYHQRQWRORJ\7KHILUVWPHWDOOHYHOSURYLGHVVWUXFWXUHGYRFDEXODU\DQGZHOOGHILQHG VHPDQWLFV E\ GHILQLQJ WHUPLQRORJ\ WKDW FDQ EH XVHG LQ WKH REMHFW OHYHO 7KH VHFRQG PHWDOHYHO GHVFULEHV WKHIHDWXUHVRI D JLYHQ RQWRORJ\ VXFK DV DXWKRU QDPH VXEMHFW HWF2,/LVFRPSDWLEOHZLWK5')VFKHPD>@XVHV5')PRGHOLQJSULPLWLYHVWRPDS 2,/VSHFLILFDWLRQVWRWKHLUFRUUHVSRQGLQJ5')VHULDOL]DWLRQV 2QWRORJLHV IRUP DQRWKHU HIIRUW WR DGG FRPPXQLW\VXSSRUWHG DQG PDQXDOO\ JHQHUDWHG VHPDQWLFV WR WKH ZHE LW UHPDLQV WR EH VHHQ KRZ EURDGO\ WKH\ ZLOO EH DGDSWHG
5 What Next? 0DMRUVHDUFKHQJLQHVKDYHFRPHDORQJZD\LQUHFHQW\HDUVLQFUDZOHUFRYHUDJHRIWKH ZHE IDVW VHDUFK RYHU YHU\ ODUJH LQGH[HG GDWD DQG SURYLGLQJ XVHUV ZLWK YHU\ JRRG UHVSRQVHVIRURQHRUWZRZRUGTXHULHV7KHUHVHDUFKRQJHQHUDOSXUSRVHZHEVHDUFK WHFKQRORJ\KDVDOVRVWDUWHGWRPDWXUHZLWKZHOOGHYHORSHGWHFKQLTXHV VXUHO\ LQ WKH QHDU IXWXUH HIIHFWLYH NH\ZRUGEDVHG ZHE VHDUFK XVLQJ PRVW ODQJXDJHV ZLOO EH SURYLGHGE\WKHPDMRUVHDUFKHQJLQHV +RZHYHU WKH QH[W QDWXUDO VWHS RI SURYLGLQJ PRUH LQIRUPDWLYH DFFHVVHV WR ZHE LQIRUPDWLRQ UHVRXUFHV QRW WR WKH ZKROH ZHE LV \HW WR FRPH &RQVLGHU WKH TXHU\ ³)LQGIURP$&06,*02'$QWKRORJ\ILYHPRVWLPSRUWDQWSUHUHTXLVLWHSDSHUVRIWKH SDSHU ³3UHGLFDWH 0LJUDWLRQ´ E\ +HOOHUVWHLQ DQG 6WRQHEUDNHU´ 3UHVHQWO\ QR WRROV H[LVWWRDQVZHUVXFKDTXHU\ 7KH QH[W HQDEOLQJ VWHS IRU HIIHFWLYH ZHE VHDUFK DQG TXHU\LQJ ZLOO FRPH ZKHQ PHWDGDWDDERXWWKHZHEEHFRPHVZLGHO\DYDLODEOH,WLVQRWFOHDUWKDWWKH5')DQGWKH VHPDQWLFZHEHIIRUWVZLOOVXFFHHGLQDGGLQJVHPDQWLFVWRDVLJQLILFDQWSRUWLRQRIWKH ZHE GXH WR D WKH FRPSOH[LW\ RI WKH VHPDQWLF ZHE DUFKLWHFWXUH ZLWK LWV QXPHURXV OD\HUVDQGE WKHDGGLWLRQDOPDQXDOHIIRUWQHHGHGWRGHILQHDQGDGGVHPDQWLFVWRZHE GDWD7KHDOWHUQDWLYHGLUHFWLRQRIDXWRPDWHGPHWDGDWDH[WUDFWLRQIURPWKHZHELV\HW SUHPDWXUH :H WKLQN WKDW ZKHQ LW PDWXUHV DXWRPDWHG PHWDGDWD H[WUDFWLRQ ZLOO FR H[LVWZLWKLIQRWWDNHRYHUWKHPDQXDOO\JHQHUDWHGPHWDGDWD 5HJDUGOHVVLQWKHIXWXUHZHELQIRUPDWLRQUHVRXUFHVEXWQRWWKHZKROHZHEZLOO KDYH PHWDGDWD DYDLODEOH DOORZLQJ XVHUV WR VHDUFK DQG TXHU\ ZHE LQIRUPDWLRQ UHVRXUFHV XVLQJ KLJKO\ SRZHUIXO VW RU KLJKHU RUGHU ORJLFEDVHG TXHU\ ODQJXDJHV 6XFKODQJXDJHVZLOOEHXQLTXHDQGGLIIHUHQWWKDQGDWDEDVHTXHU\ODQJXDJHV
:HE,QIRUPDWLRQ5HVRXUFH'LVFRYHU\3DVW3UHVHQWDQG)XWXUH
References 1. E. Agichtein, E. Eskin, L. Gravano, “Combining Strategies for Extracting Relations from Text Collections”, ACM SIGMOD, 2000.
2. Agichtein, E., Gravano, L., “Snowball: Extracting relations from large plain-text collections”, The 5th ACM International Conference on Digital Libraries, June 2000.
3. Agichtein, E., Gravano, L., “Querying Text Databases for Efficient Information Extraction”, Proce. of the 19th IEEE Intl Conference on Data Engineering (ICDE), 2003.
4. Brickley, D., Guha, R.V., “Resource Description Framework Schema (RDFS)”, W3C Proposed Recommendation, 1999, available at http://www.w3.org/TR/PR-rdf-schema.
5. K. Bharat, M.R. Henzinger, “Improved algorithms for topic distillation in a hyperlinked environment”, ACM SIGIR Conf., 1998.
6. J. Broekstra, M. Klein, D. Fensel, and I. Horrocks, “Adding formal semantics to the Web: building on top of RDF Schema”, In Proc. of the ECDL, 2000.
7. Berners-Lee, T., “Semantic Web Roadmap”, W3C draft, Jan 2000, available at http://www.w3.org/DesignIssues/Semantic.html
8. P.M.E. De Bra, R.D.J. Post, “Searching for arbitrary information in the WWW: Making Client-based searching feasible”, WWW Conf., 1994.
9. Sergey Brin, Lawrence Page, “The anatomy of a large-scale hypertextual Web search engine”, Computer Networks and ISDN Systems, Brisbane, Australia, 1998.
10. Sergey Brin, “Extracting patterns and relations from the world wide web”, In WebDB Workshop at EDBT, 1998. http://citeseer.nj.nec.com/brin98extracting.html.
11. S. Chakrabarti et al, “Mining the web’s link structure”, IEEE Computer, Aug. 1999. 12. S. Chakrabarti, M. van den Berg, B. Dom, “Focused crawling: A new approach to topicspecific web resource Discovery”, In Proceedings of WWW 8 Conf., 1999.
13. J. Cho, H. Garcia-Molina, L. Page, “Efficient crawling through URL ordering”, In Proceedings of the Seventh International World-Wide Web Conference, 1998.
14. Mining the Web: Discovering knowledge from hypertext data, Chakrabarti, S., MorganKaufmann Publishers, 2003.
15. M. Diligenti, F. Coetzee, S. Lawrence, C. L. Giles, M. Gori, “Focused Crawling using Context Graphs”, VLDB 2000.
16. Eberhart, A., “Survey of RDF data on the web”, In Proc. of the 6th World Multiconference on Systemics, Cybernetics and Informatics (SCI), 2002.
17. Google History, at http://www.google.com/corporate/history.html. 18. Gruber, T., “A translation approach to portable ontologies”, Knowledge Acquisition, 1993. 19. Guarino, N., “Formal Ontology and Information Systems”, In N. Guarino (ed.), Formal Ontology in Information Systems, Proc. of the 1st International Conference, 1998.
20. R. Grishman, S. Huttunen, R. Yangarber, “Real-Time Event Extraction for Infectious Disease Outbreaks”, In Proceedings of Human Language Technology Conference, 2002.
21. Ralph Grishman, “Information extraction: Techniques and challenges”, In Maria Teresa Pazienza, editor, Information Extraction, Springer-Verlag, LNAI, 1997.
22. M. Hersovici et al, “The sharksearch algorithm—an application: Tailored web site mapping”, WWW 7 Conf., 1998.
23. Horrocks et al, “The Ontology Inference Layer OIL”, Technical report, Free University of Amsterdam, 2000. http://www.ontoknowledge.org/oil/.
24. Kleinberg, J., “Authoritative Sources in hyperlinked environments”, In the 9th ACMSIAM Symposium on Discrete Mathematics, 1998.
25. M. Koivunen and E. Miller, “W3C Semantic Web Activity”, In the proceedings of the Semantic Web Kick-off Seminar in Finland Nov 2, 2001.
26. R. Lempel, S. Moran, “SALSA: The stochastic approach for link-structure analysis”, ACM TOIS, April 2001.
*2]VR\RJOXDQG$$O+DPGDQL
27. Lassila, O., Swick, R., “Resource Description Framework (RDF) Model and Syntax Specification”, W3C Recommendation 22 February 1999.
28. Frank Manola, Eric Miller, “RDF Primer”, W3C Working Draft, 23 January 2003 29. F. Menczer, G. Pant, M. Ruiz, P. Srinivasan, “Evaluating topic-driven Web crawlers”, In
Proc. 24th Intl. ACM SIGIR Conf., 2001 30. M. Najork, J. Weiner,“Breadth-First search crawling yields high-quality pages”, WWW’98. 31. Ng, A. Zheng, M. Jordan, “Stable algorithms for link analysis”, ACM SIGIR, 2001. 32. L. Page, S. Brin, R. Motwani, T. Winograd, “The PageRank citation ranking: Bringing order to the web”, Stanford Digital Libraries Working Paper, 1998. 33. G. Salton, Automatic Text Processing, Addison-Wesley, 1989. 34. International Directory of Search Engines, Search Engine Colossus, 2003, available at http://www.searchenginecolossus.com. 35. The Major Search Engines and Directories, Search Engine Watch Report, Danny Sullivan, 2003, available at searchenginewatch.com/links/article.php/2156221. 36. The Semantic Web Community Portal, at http://www.semanticweb.org 37. Search Links, available at http://searchenginewatch.com/links/index.php.
Courses Modeling in E-learning Context Vincenza Carchiolo, Alessandro Longheu, Michele Malgeri, and Giuseppe Mangioni Dipartimento di Ingegneria Informatica e delle Telecomunicazioni Universita’ di Catania - Viale Andrea Doria, 6 - I95125 Catania (Italy) {car,alongheu,mm,gmangioni}@diit.unict.it
Abstract. The computer-based sharing and dissemination of knowledge and learning activities are known as ”E-learning”. In this paper we propose an E-learning model, focusing in particular on courses modeling, which aims at promoting the sharing and reuse of courses contents and teaching materials, allowing the construction of personalized learning paths.
1
Introduction
Many services today benefit significantly from the stressed use of computer-based technologies, thus leading to the ”E-anything” paradigma. Among these, the set of services ranging from the sharing and dissemination of knowledge to learning activities, is nowadays known as ”E-learning” ([1,2,3]). Whenever computer-based technologies are adopted in an existing context, it is important to re-consider features and purposes of the desired service in order to assure its real enhancement, avoiding to simply automatize its functions. Considering the learning process, its aims can be summarized as follows: – to share courses contents among teachers, in order to provide students with a single and (possibly) uniform set of concepts to be learned; – to share teaching materials (lessons), in order to exploit as far as possible existing material developed by teachers; to provide this modularity, material is actually separated from contents (set of concepts); – to promote active learning, allowing the construction of courses which are personalized in terms of both contents and teaching materials, selected according to each student’s needs and capabilities, also taking teacher guidelines into account; – to provide students with an adaptive environment which dynamically adjusts the personalized course during the learning process; – to provide both teachers and students with a simple, flexible, open, modular learning environment. To satisfy such goals, an E-learning system must be characterized with a complete management (storing, creation and retrieval) of courses and teaching materials (as well as of student capabilities and teachers guidelines), and a proper A. Yazici and C. S ¸ ener (Eds.): ISCIS 2003, LNCS 2869, pp. 19–26, 2003. c Springer-Verlag Berlin Heidelberg 2003
20
V. Carchiolo et al.
model for course contents and teaching materials, in terms of both data they should include and the structure such data should be organized into, in order to guarantee concepts/lessons sharing and reuse. Moreover the E-learning system must provide an open and modular interface (e.g., web-based) for end-users (both students and teachers), and a generator of on-demand customized courses, to create and propose to the student all the (customized) paths starting from his previous knowledge towards the desired knowledge (topic of interest) This paper deals with these aspects, introducing a model for an E-learning environment. We first show the architecture of the model, then we consider in detail how courses contents and teaching materials are modeled.
2
Learning System Architecture
In order to build a complete learning system, various tasks must be performed [4,5,6,7]. Figure 1 shows our model for a learning system consisting of Courses, Engine, Profiles and E-Learning Interface module.
Fig. 1. Learning system architecture
Courses module consists of two sub-modules, called Domain Database (DDB) and Teaching Materials Database (TMDB), each with its manager sub-module. DDB contains all lessons, each intended as an atomic set of concepts to be learned separated from its teaching materials, and all courses, each intended as a set of lessons properly grouped together. Lessons (and courses) are connected
Courses Modeling in E-learning Context
21
together through precedence-succession relations. The DDB manager interacts with the Engine module by providing all lessons and courses. For each lesson, the DDB also contains a link to the related teaching materials (e.g. slides), placed in the TMDB, separated from its related lesson(s). Teaching materials are extracted by the TMDB manager and properly arranged by the Engine module. Engine module contains the course generation and course presentation submodules, together with their management sub-module. The course generation sub-module is devoted to building courses and learning paths, based on lessons (courses) contained in the domain database, as well as on information in the profile databases. Once courses/learning paths are built by the course generation sub-module, the course presentation sub-module retrieves all the related teaching materials and organizes the way courses are made available, e.g. depending on the media actually available or desired, for instance a PC or a palmtop, or based on the available bandwidth, always considering profile information, so that the proper teaching material can be provided. After preparation, courses are finally attended by students and/or consulted by teachers through the E-learning Interface module as shown in Figure 1; the virtual learning environment can be simply a web site, or it may also integrate different media (e.g. a PC to attend lessons or cell phone text messages for news about courses). Profiles module contains all relevant information about teachers and students, stored in proper databases, together with their management sub-module. Profile Databases are used by the course generation and course presentation sub-modules when courses/learning paths are generated to tailor their contents (and the related teaching materials). Access to the Profile databases is actually granted through the Profile manager, similarly to DDB/TMDB managers. The last module is the E-learning Interface, through which students attend courses (perform exercises and tests, view/modify their profile), teachers/course creators view and manage their courses, and administrators manage the whole system.
3
Courses Module
The databases in the Course module have been designed on the basis of two important requisites: modularity and reusability. Meeting the first requirement made it possible to design easily extensible databases (by adding new topics to an existing domain or inserting new domains). The aim of the second requisite was to make it possible to reuse or share part or all of the teaching material. For these reasons databases have been developed with a clear separation between the contents and descriptions of the various topics. From the logical viewpoint, the information contained in the databases is structured on three levels: course level, unit level and teaching level (see Figure 2). From the physical point of view, Courses Module contains the DDB and the TMDB. The DDB contains implementation of the course and unit level, while the TMDB contains all the information for the teaching material level.
22
V. Carchiolo et al. CN2
CN4 CN3 CN6
CN1
COURSE LEVEL
CN5
CU4 CU2
CU6
CU1
CU5
UNIT LEVEL
CU3
CM2
11 00 00 11 CM1
TEACHING MATERIAL LEVEL
Fig. 2. Databases logical view
Typical database reading operations are performed during the generation and presentation of courses, while writing operations are performed when new courses, lessons or teaching material are inserted. New information can be inserted into the database while creating a new course/lesson or while creating a new learning path meeting the specific requirements of a student. Those involved in the first case are teachers (or course creators) who can use various authoring tools to insert a new course (course level ), lesson (unit level ) or teaching material (teaching material level ). The operations are supervised by the engine, which ensures correct and efficient use of the Course Database (for more details see [8]). The second case is when a student expresses the need to learn a given topic (not included in the courses at the course level ) and the engine generates a new learning path for the student’s specifications (adaptive), combining the lessons and teaching material already in the database. The general learning path will become part of the information in the course level and can be reused in future on request. 3.1
Domain Database
The DDB contains information about the courses and lessons available to students, according to their particular domain of interests. It has two levels: course level and unit level. The course level contains a representation of the relations linking the various courses, using an oriented graph of the and/or type. The nodes of the graph, called Course Nodes (CN ), contain information about the structure of each course. The arcs between the various nodes indicate whether one course has to be completed before another can be taken. The sequence of lessons making up a course is stored in each CN as a graph of Course Units (CU ) contained in the unit level. In reality a CN does not directly
Courses Modeling in E-learning Context
23
contain the single CU but a pointer to them, as they can be used in different courses. For this reason, all the CU , irrespective of the courses they belong to, are in turn organized in an and/or graph with the various nodes connected by arcs representing whether one is preliminary to another. In this way the CU have an independent existence from the courses using them, and this increases the possibility of reusing them in different courses. The orientation of the arcs between both courses (CN ) and units (CU ) is defined as follows: an arc oriented from X to Y means that node X depends on node Y ; in the case of CN (or CU ), that is, if X depends on Y , it means that all concepts included in CN (or CU ) X must be learned before concepts of CN (or CU ) Y can be understood. This precedence-succession is the only type of relation considered since it appears to be the most unbiased, i.e. when X is connected to Y this should depend on the concepts of both X and Y , even different arcs may connect X and Y (e.g. a direct one and a second X → Z → Y ). Moreover, the criteria according to which courses are connected can be refined or even re-defined by teachers - course creators. Two or more arcs involving the same node, for example the two arcs X → Z and Y → Z can represent alternative paths (or arcs, whereby either X or Y must be known in order to understand the concepts in Z), or they may represent paths that are both necessary to understand the concepts in the node Z to which they refer. The two situations are modelled as or and and arcs. Arcs are not weighted in the context of the DDB, but weights are evaluated when considering personal profile information, in order to provide learning paths in the best order for a given student (in other words, weights for arcs are not absolute but depend on each student profile). Precedence-succession relationships are provided when courses are created and stored in the DDB in a semi-automatic fashion, i.e. based on course concepts and mediated by teachers-course creators. Description of CN and CU . Courses can be built by the system if simple precedence-succession relationships are considered, or a teacher may build his own course based on personal educational guidelines, or again a group of teachers may provide predefined courses within a specific study context, or finally a student may require the system to build a course for his specific needs. In all these cases, courses are built considering the personal profile information stored in the Profile Database as shown in Figure 1. Moreover, not all lessons belonging to the same course need to be explicitly related by a precedence-succession relation; this occurs, for instance, when a course includes several different topics. Moreover, teachers can also explicitly indicate lesson relationships, as when suggesting specific paths to students; such relationships, however, should always be subject to precedence-succession relationships. The DDB contains both predefined courses as well as all personalized courses created on students requests; in this way, even personalized courses can be shared and reused.
24
V. Carchiolo et al.
Each CN in the graph in Figure 2 features a series of information which allow the dynamic construction of adaptive learning paths: More specifically: 1. 2. 3. 4. 5. 6.
Prerequisites. The previous knowledge required. Objectives. The knowledge that will be acquired. Title. A title describing the course. Timing (e.g. duration of the course, total time required for learning). Level of abstraction of the course (e.g. highly theoretical, mostly practical) Level at which topics are dealt with (e.g. introductory course, in-depth treatment, for specialists). 7. Level of detail with which contents are dealt with (general overview of problem, details of specific problems). It should be noted that the level of detail and the general level at which topics are dealt with may overlap, although there may be courses on specialist topics with a low level of detail (e.g. a survey of a specialist problem). 8. Graph of lessons making up the course. The prerequisites indicate the knowledge the student needs to have to be able to learn the course, while the objectives indicate the knowledge that will have been acquired by the end of the course and will be included in the profile of students passing the end-of-course test. Both the prerequisites and the objectives comprise a list of keywords from the vocabulary of the domain of interest (to which the Domain Database refers). The descriptive information associated with each CU is essentially the same as that used for the CN . This time, instead of the graph of lessons in CN , at the CU level we find information about the teaching material (called course material or CM ) used in CU . General remarks. It is important to establish the size (i.e. the extent in terms of contents) of each CU . From an exclusively qualitative point of view, it can be stated that too large a size (i.e. too many concepts) may make a difficult to combine CU with others when constructing a course, while too small a size may boil down to a single statement, thus becoming too fragmentary and making the CU graph very complex, with negative repercussions on the course creation algorithms as well. However, as neither of these two extreme situations is a technically insurmountable problem, it is only teaching requirements (and therefore teachers and course creators) that can establish the optimal size for each lesson. In the case of CN size is not a problem, as the minimum is a coinciding with a CU , whereas the maximum size can only be established on the basis of teaching requirements. Each teacher-course creator can insert new CN and/or CU into the Domain Database. To ensure the maximum teaching flexibility, it was decided not to have any control in terms of either size or overlapping of contents. In accordance with this principle, the Domain Database may contain courses that partially share the same contents but differ in the level of overall detail, for example, or the time required, or may simply reflect the different teaching methods used by several teachers.
Courses Modeling in E-learning Context
3.2
25
Teaching Material Database
The TMDB contains all the teaching material (CM ) used in the various stages of a course/lesson (e.g. presentation, testing ), and generally comprises multimedia and/or hypertext material (e.g. HTML pages, animated graphics). It is possible to associate each CU with the flow of associated CM , where each CM is a selfcontained unit of teaching material. This flow models the sequence in which the CM have to be followed. It is also possible to provide different templates for paths in which the CU can be arranged when they are associated with a given CU . For example, a typical template (as illustrated in Figure 3) could have a presentation I, followed by one or more sequences of pairs (detailed study Di and relative test Ti ), ending with an overall test T ; other criteria for the creation of templates could consider the type of media, the length of the CM , the type of contents (theoretical, practical) and so on.
Fig. 3. A typical template for a lesson presentation path
Some remarks should be made about the relation between CU and CM . First of all, the separation between a lesson and the corresponding teaching material makes it possible to create CU without corresponding CM and vice versa, although a CU can clearly only be used if there is at least one CM connected with it, and vice versa although a inserted CM into the TMDB increases the set of available, it will only be used if there exists at least one CU in which it is inserted. Again, besides the fact that a CU can be associated with various CM , a CM can be used at the same time in different CU . This situation is not only possible but advisable to promote the reuse of teaching material. The principle of separating lessons and teaching material also requires appropriate choice of the granularity of a CM (size in terms of contents) and its relation with the granularity of the CU , so as to: – have an acceptable size for the CM , on the basis of considerations similar to those made for CU , that is, excessively small sizes lead to fragmentation, while excessively large sizes make them difficult to combine with other CM ; – avoid situations in which a CU cannot be associated with the teaching material because there are no CM small enough to contain only the concepts of that CU . It is beyond the scope of this work to consider all aspects relating to the insertion of CM in the TMDB, e.g. classification criteria according to the media involved, choice of storage format, media adaptation.
26
4
V. Carchiolo et al.
Conclusions and Future Work
In this paper we introduced a model for an e-learning system aiming at promoting active learning, allowing the construction of courses which are personalized in terms of both contents and teaching materials, thus providing students with an adaptive environment. In particular, we focused on lessons and courses modeling, introducing a representation based on a hierarchical graph. It permits to express precedence-succession relationships among concepts as well as courses. We validated the model above presented by a prototype, it allows a student to attend on-line courses: each user logs into the system, possibly specifying his membership in a given user class (with specific characteristics), then he chooses a course (or searches for a specific topic), and consequently all the related learning paths are built, also based on that student’s needs; finally, the student selects one of these paths and attends the course, which also includes exercises and tests (for details see [8]).
References 1. Soine, R.: Instructional design in a technological world: Fitting learning activities into the larger picture. (In Proc. of ICALT 2001) 2. Luchini et al., K.: An engineering process for constructing scaffolded work environments to support student inquiry: A case study in history. (In Proc. of ICALT 2001) 3. Heinrich et al., E.: Teaching cognitively complex concepts: Content representation for audiograph lectures. (In Proc. of Ed-Media 2001) 4. Vassileva, J.: Dynamic courseware generation: at the cross point of cal, its and authoring. (In Proc. of ICCE’95) 5. Anido et al., L.: A standards-driven open architecture for learning systems. (In Proc. of ICALT 2001) 6. Hampel, T., Keil-Slawik, R.: steam - designing an integrative infrastructure for web-based computer-supported cooperative learning. in Proc. of WWW10 (2001) 7. Gehne et al., R.: Technology integrated learning environment - a web-based distance learning system. (In Proc. of IASTED 2001) 8. Carchiolo, V., Longheu, A., Malgeri, M.: A model for a web-based learning system. Technical report, Dip. di Ingegneria Informatica e delle Telecomunicazioni (2002)
Fast Hardware of Booth-Barrett’s Modular Multiplication for Efficient Cryptosystems Nadia Nedjah and Luiza de Macedo Mourelle Department of Systems Engineering and Computation, Faculty of Engineering, State University of Rio de Janeiro, Brazil {nadia, ldmm}@eng.uerj.br
Abstract. Modular multiplication is fundamental to several public-key cryptography systems such as the RSA encryption system. It is also the most dominant part of the computation performed in such systems. The operation is time consuming for large operands. This paper examines the characteristics of yet another architecture to implement modular multiplication. An experimental modular multiplier prototype is described in VHDL and simulated. The simulation results are presented.
1
Introduction
The modular exponentiation is a common operation for scrambling and is used by several public-key cryptosystems, such as the RSA encryption scheme [1]. It consists of a repetition of modular multiplications: C = T e mod M , where T is the plain text such that 0 ≤ T < M and C is the cipher text or vice-versa, e is either the public or the private key depending on whether T is the plain or the cipher text, and M is called the modulus. The decryption and encryption operations are performed using the same procedure, i.e. using the modular exponentiation [2], [3], [4]. The performance of such cryptosystems is primarily determined by the implementation efficiency of the modular multiplication and exponentiation. As the operands (the plain text of a message or the cipher or possibly a partially ciphered) text are usually large (i.e. 1024 bits or more), and in order to improve time requirements of the encryption/decryption operations, it is essential to attempt to minimise the number of modular multiplications performed and to reduce the time requirement of a single modular multiplication. In the rest of this paper, we start off by describing the algorithms used to implement the modular operation. Then we present the architecture of the hardware modular multiplier and explain in details how it executes a single multiplication. Then we comment the simulation results obtained for such architecture.
2
Multiplication Algorithm
Algorithms that formalise the operation of multiplication generally consist of two steps: one generates a partial product and the other accumulates it with the previous partial products. The most basic algorithm for multiplication is based A. Yazici and C. S ¸ ener (Eds.): ISCIS 2003, LNCS 2869, pp. 27–34, 2003. c Springer-Verlag Berlin Heidelberg 2003
28
N. Nedjah and L. de Macedo Mourelle
on the add-and-shift method: the shift operation generates the partial products while the add step sums them up [5]. The straightforward way to implement a multiplication is based on an iterative adder-accumulator for the generated partial products. However, this solution is quite slow as the final result is only available after n clock cycles, n is the size of the operands. A faster version of the iterative multiplier should add several partial product at once. This could be achieved by unfolding the iterative multiplier and yielding a combinatorial circuit that consists of several partial product generators together with several adders that operate in parallel. In this paper, we use such a parallel multiplier as described in Fig. 1. Now, we detail the algorithms used to compute the partial products and to sum them up.
Fig. 1. Parallel multiplier architecture
Now, we concentrate on the algorithm used to compute partial products as well as reducing the corresponding number without deteriorating the space and time requirement of the multiplier. Let X and Y be the multiplicand and multiplicator respectively and let n be their size. So, we can denote X, Y and the product X × Y as follows: X=
n−1 i=0
xi × 2i , Y =
n−1 i=0
bi × 2i , X × Y =
n−1
xi × Y × 2 i
(1)
i=0
Inspired by the above notation of X, Y and that of X × Y , the add-and-shift method [5] generates n partial products: xi × Y, 0 ≤ i < n. Each partial product obtained is shifted left or right depending on whether the starting bit was the less or the most significant and added up. The number of partial products generated is bound above by the size (i.e. number of bits) of the multiplier operand. In cryptosystems, operands are quite large as they represent blocks of text (i.e. ≥ 1024 bits). Another notation of X and Y allows to halve the number of partial
Fast Hardware of Booth-Barrett’s Modular Multiplication
29
products without much increase in space requirements. Consider the following notation of X and X×Y: (n+1)/2+1
X=
x ˜i × 22×i ,
(2)
i=0
x ˜i = x2×i−1 + x2×i − 2 × x2×i+1 , x ˜−1 = x ˜n = x ˜n+1 = 0
(3)
(n+1)/2+1
X=
x ˜i × Y × 22×i ,
(4)
i=0
The possible values of with the respective values of x ˜2×i+1 , x2×i , and x2×i−1 are -2 (100), -1 (101, 110), 0 (000, 111), 1 (001, 010) and 2(011). Using this recoding will generates (n + 1)/2 partial products. Inspired by the above notation, the modified Booth algorithm [6], [7] generates the partial products x ˜ × Y . These partial products can be computed very efficiently due to the digits of the new representation x ˜i . The hardware implementation will be detailed in Section 3. In Algorithm 1, the terms 4 × 2n+1 and 3 × 2n+1 are supplied to avoid working with negative numbers. The sum of these additional terms is congruent to zero modulo 2n+(n+1)/2−1 . So, once the sum of the partial products is obtained, the rest of this sum in the division by 2n+(n+1)/2−1 is finally the result of the multiplication X × Y . Algorithm 1. ModMulti(x2×i−1 , x2×i , x2×i+1 , Y ) int product = 0; int[] pp[(n + 1)/2 − 1];pp[0]= (˜ xi × Y + 4 × 2n+1 ) × 22×i ; for i=0 to (n + 1)/2 − 1 pp[i] = (˜ xi × Y + 3 × 2n+1 ) × 22×i ;product = product + pp[i]; return product mod 2n+(n+1)/2−1 ; end.
3
Reduction Algorithm
A modular reduction is simply the computation of the remainder of an integer division. It can be denoted by: X XmodM = X − ×M (5) M However, a division is very expensive even compared with a multiplication. Using Barrett’s method [8], we can estimate the remainder using two simple multiplications. The approximation of the quotient is calculated as follows: X × 2n−1 ×2n+1 X × 22×n 2n−1 2n−1 M M X ≈ (6) = n+1 n+1 M 2 2
30
N. Nedjah and L. de Macedo Mourelle
The equation above can be calculated very efficiently as division by a power of two 2x are simply a truncation of x-least significant digits of the operand. The term 22×n /M depends only on the modulus M and is constant for a given modulus. Hence, can be pre-computed and saved in an extra register. Hence the approximation of the remainder using Barrett’s method [8] is a positive integer smaller than 2 × (M − 1). So, one or two subtractions of M might be required to yield the exact remainder.
4
Modular Multiplier Architecture
In this section, we outline the architecture of the multiplier, which is depicted in Fig. 2. Later on in this section and for each of the main parts of this architecture, we give the detailed circuitry, i.e. that of the partial product generator, adder and reducer. The multiplier of Fig. 2 performs the modular multiplication X ×Y mod M in four main steps: 1. 2. 3. 4.
Computing Computing Computing Computing
the the the the
product P = X × Y ; estimate quotient Q = P/M ⇒ Q ≈ P/2n−1 × 22×n /M ; product Q × M ; final result P − Q × M .
During the first step, the modular multiplier first loads register1 and register2 with X and Y respectively; then waits for PPG to yield the partial products and finally waits for the ADDER to sum all of them. During the second step, the modular multiplier loads register1 , register2 and register3 with the obtained product P , the pre-computed constant 22×n /M and P respectively; then waits for PPG to yield the partial products and finally waits for the ADDER to sum all of them. During the third step, the modular multiplier first loads register1 and register2 with the obtained product Q and the modulus M respectively; then awaits for PPG to generate the partial products, then waits for the ADDER to provide the sum of these partial products and finally waits for the REDUCER to calculate the final result P −Q×M , which is subsequently loaded in the accumulator ACC. 4.1
The Multiplier
The multiplier interface is composed of the multiplicand and multiplicator as input signals and the partial products P Pi , 0 ≤ i < k, each one of n + 3 bits, as output signals. It is composed of a partial product generator and an adder. The partial product generator is in turn composed of k Booth recoders [6], [7] that communicate directly with k partial product selectors. The interface of the Booth decoder is composed of three bits, lsb, midle and msb, as input signals and it outputs the signals SelectY, Select2Y and Sign. The Booth selection logic circuitry used is very simple. The inputs are the three bits forming the Booth digit and outputs are three bits: the first one
Fast Hardware of Booth-Barrett’s Modular Multiplication
31
Fig. 2. Architecture of the modular multiplier
SelectY is set when the partial product to be generated is Y or −Y , the second one Select2Y is set when the partial product to be generated is 2 × Y or −2 × Y and the last bit is simply the last bit of the Booth digit given as input. It allows us to complement the bits of the partial products when a negative multiple is needed. The output signals are yielded from the input ones as follows: SelectY = lsb ⊕ midle; Sign = msb
(7)
Select2Y = ¬(¬(middle ⊕ msb) + (lsb ⊕ midle))
(8)
The required partial products, i.e. x ˜ × Y are easy multiples. They can be obtained by a simple shift. The negative multiples in 2’s complement form, can be obtained form the positive corresponding number using a bit by bit complement with a 1 added at the least significant bit of the partial product. The additional terms introduced in the previous section can be included into the partial product generated as three/two/one most significant bits computed as follows, whereby, ++ is the bits concatenation operation, A is the binary notation of integer A, 0i is a run of i zeros and B[n:0] is the selection of the n less significant bits of the binary representation B.
pp0 = s0 s0 s0 + + |x˜0 × Y ⊕ s0 + s0
(9)
pp2×j = (1s2×j + + |˜ x2×j | × Y ⊕ s2×j + s2×j ) + +0
2×j
(10)
For 1 ≤ j < k − 1 and j = k − 1 = k , we have: x2×k | × Y ⊕ s2×k + s2×k ) + +02×k
pp2×k = (1s2×k + + |˜
(11)
32
N. Nedjah and L. de Macedo Mourelle
pp2×k = |˜ x2×k × Y [n:0] + +02×k
(12)
The interface of the partial product generator is consists of the Booth’s digit, which is constituted of 3 bits and the multiplicand, which has n bits, as input signals and the partial product, which is a signal of n + 1 bits, and an extra bit which represents the sign of the partial product generated, as output signals. The n + 1 bits of partial products P Pi are yield using the logic below:
4.2
P Pi0 = (SelectY × y0 ) ⊕ Sign
(13)
P Pii = (Select2Y × yi−1 ) + SelectY × yi ⊕ Sign, 1 ≤ i ≤ n
(14)
P Pin+1 = (Select2Y × yn ) ⊕ Sign
(15)
The Adder
In order to implement the adder of the generated partial products, we use a hybrid new kind of adder. It consists of an initial stage of carry save adders followed by a cascade of stages of delayed carry adders [7] and a final stage of full adder. The carry save adder CSA is simply a parallel ensemble of f full adders without any horizontal connection. Its function is to add three f -bit integers a, b and c to yield two new integers carry and sum such that carry +sum = a+b+c. The pair (carry, sum) will be called a delayed carry integer. The delayed carry adder DCA is a parallel ensemble of f half adders. Its function is to add two delayed carry integers (a1 , b1 ) and (a2 , b2 ) together with an integer c to produce a delayed carry integer (sum, carry) such that sum+carry = a1 +b1 +a2 +b2 +c. The general architecture of the proposed adder is depicted in Fig. 3, where the partial products P Pi , 0 ≤ i ≤ 15 are the input operands.
Fig. 3. The main cell of the proposed adder
Fast Hardware of Booth-Barrett’s Modular Multiplication
33
Using the carry save adder, the ith bit of carry and sum are defined as sumi = ai ⊕ bi ⊕ ci and carryi = ai × bi + ai × ci + bi × ci respectively. The architecture of the delayed carry adder uses 5 × n half adders as in Fig. 4.
Fig. 4. The structure of the delayed carry adder
4.3
The Reducer
The main task of the reducer consists of subtracting Q × M , i.e. the product obtained in the third step of the modular multiplier from P , i.e. the product computed in the first step of the modular multiplier. A subtraction of an p-bit integer K is equivalent to the addition of 2p − x. Hence the reducer simply performs the addition P + (2n+m − Q × M ). The latter value is simply the two’s complement of Q × M . The addition is performed using a carry look-ahead adder. It is based on computing the carry bits Ci prior to the actual summation. The adder takes advantage of a relationship between carry bits Ci and input bits Ai and Bi , wherein Gi = Ai × Bi and Pi = Ai + Bi . Ci = Gi−1 + (Gi−2 + . . . + (G1 + (G0 + C0 P0 ) × P1 . . .) × Pi−1 ) 4.4
(16)
The Controller
In order to synchronise the work of the MULTIPLIER, ADDER and REDUCER, we designed a module called the CONTROLLER that consists of a simple state
34
N. Nedjah and L. de Macedo Mourelle
machine, that has 13 states defined as follows: where next(Si ) = Si+1 and next(S12 ) = S0 . S0: Initialise state machine; S1: Load multiplicator into register1; Load multiplicand into register2; S5: Load product P into register1; Load constant into register2; Load P into register3; S6: Wait for MULTIPLIER; S7: Wait for ADDER;
5
S2: S3: S4: S8:
Wait Wait Wait Load Load S9: Wait S10: Wait S11: Wait S12: Load
for MULTIPLIER; for ADDER; for the SHIFTER; Q into register1; M into register2; for MULTIPLIER; for ADDER; for REDUCER; ACC with result.
Conclusion
In this paper, an alternative architecture for computing modular multiplication based on Booth’s algorithm and on Barrett’s relaxed residium method is described. The Booth’s algorithm is used to compute the product while Barrett’s method is used to calculate the remainder. The architecture was validated through behavioural simulation results using the 0.6mm CMOS-AMS standard cell library. The total execution time is 3570 nanoseconds for 1024-bit operands. One of the advantages of this modular multiplication implementation resides in the fact that it is easily scalable with respect to the multiplier and modulus lengths.
References 1. R. Rivest, A. Shamir and L. Adleman, A method for obtaining digital signature and public-key cryptosystems, Communications of the ACM 21, (1978) 120–126 2. E. F. Brickell, A survey of hardware implementation of RSA, Proc. of CRYPTO’98, Lecture Notes in Computer Science 435, Springer-Verlag, (1989) 368–370 3. C. D. Walter, Systolic modular multiplication, IEEE Transactions on Computers 42(3), (1993) 376–378 4. S. E. Eldridge and C. D. Walter, Hardware implementation of Montgomery’s Modular Multiplication Algorithm, IEEE Transactions on Computers 42(6), (1993) 619– 624 5. J. Rabaey, Digital integrated circuits: A design perspective, Prentice-Hall, (1995) 6. A. Booth, A signed binary multiplication technique, Journal of Mechanics and Applied Mathematics, (1951) 236–240 7. O. MacSorley, High-speed arithmetic in binary computers, Proc. of the IRE, (1961) 67–91 8. P. Barrett, Implementating the Rivest, Shamir and Aldham public-key encryption algorithm on standard digital signal processor, Proc. of CRYPTO’86, Lecture Notes in Computer Science 263, Springer-Verlag, (1986) 311–323
Classification of a Large Web Page Collection Applying a GRNN Architecture Ioannis Anagnostopoulos, Christos Anagnostopoulos, Vergados Dimitrios, Vassili Loumos, and Eleftherios Kayafas School of Electrical and Computer Engineering, Heroon Polytechneiou 9, Zographou, 15773, Athens, *5((&( MDQDJ#WHOHFRPHFHQWXDJU
Abstract. This paper proposes an information system that classifies web pages according a taxonomy, which is mainly used from seven search engines/ directories. The proposed classifier is a four-layer Generalised Regression Neural Network (GRNN) that aims to perform the information segmentation according to web page features. Many types of web pages were used in order to evaluate the robustness of the method, since no restrictions were imposed except for the language of the content, which is English. The system can be used as an assistant and consultative tool in order to help the work of human editors.
1 Introduction 7KHWHFKQLTXHVPRVWXVXDOO\HPSOR\HGLQWKHFODVVLILFDWLRQRIZHESDJHV XVH FRQ FHSWVIURPWKHILHOG RI LQIRUPDWLRQ ILOWHULQJ DQG UHWULHYDO >@ >@ 6XFK WHFKQLTXHV XVXDOO\DQDO\VHDFRUSXVRIDOUHDG\FODVVLILHGZHESDJHFRQWHQWVH[WUDFWIURPWKHP ZRUGV DQG SKUDVHV ZLWK WKH XVH RI VSHFLILF DOJRULWKPV SURFHVV WKH WHUPV DQG WKHQ IRUPWKHVDXULDQGLQGLFHV$VHFRQGODUJHJURXSRIWHFKQLTXHVDUHQHXUDOQHWZRUNV 1HXUDO QHWZRUNV DUH FKRVHQ PDLQO\ IRU FRPSXWDWLRQDO UHDVRQV VLQFH RQFH WUDLQHG WKH\ RSHUDWH YHU\ IDVW DQG WKH FUHDWLRQ RI WKHVDXUL DQG LQGLFHV LV DYRLGHG >@ >@ 1HYHUWKHOHVVEDVLFFRQFHSWVIURPLQIRUPDWLRQILOWHULQJDQGUHWULHYDODUHVWLOOXVHGLQ WKHFRPSXWDWLRQVLQRUGHUWRDFKLHYHWKHEHVWSRVVLEOHUHVXOWV,QDGGLWLRQWKHXVHRI HYROXWLRQEDVHG JHQHWLF DOJRULWKPV DQG WKH XWLOL]DWLRQ RI IX]]\ IXQFWLRQ DSSUR[L PDWLRQKDYHDOVREHHQSUHVHQWHGDVSRVVLEOHVROXWLRQVIRUWKHFODVVLILFDWLRQSUREOHP >@>@)LQDOO\PDQ\H[SHULPHQWDOLQYHVWLJDWLRQVRQWKHXVHRIQHXUDOQHWZRUNVIRU LPSOHPHQWLQJUHOHYDQFHIHHGEDFNLQDQLQWHUDFWLYHLQIRUPDWLRQUHWULHYDOV\VWHPKDYH EHHQSURSRVHG>@
2
Collection of Web Pages
%DVHG RQ ERWK LQIRUPDWLRQ ILOWHULQJ WHFKQLTXHV DQG DUWLILFLDO QHXUDO QHWZRUNV WKLV SDSHU GHVFULEHV D *HQHUDOLVHG 5HJUHVVLRQ 1HXUDO 1HWZRUN WKDW GLVWLQJXLVKHV ZHE $@7KLVODUJHVHWZDVXVHGLQRUGHUWRILQGWKHWHUPV WKDWDUHFDSDEOHRIGHVFULELQJHDFKRQHRIWKHHLJKWLQIRUPDWLRQFOXVWHUV$QLQIRU PDWLRQFOXVWHULVUHSUHVHQWHGDVDVHSDUDWHFODVVLQWKHSURSRVHGFODVVLILHUDQGLWLV HTXLYDOHQWWRWKHFODVVLILHGZHESDJHFODVV)XUWKHUPRUHWKHVDPHODUJHDPRXQWRI ZHE SDJHV ZDV XVHG IRU WUDLQLQJ SXUSRVHV ,W PXVW EH QRWHG WKDW RQO\ ZHE SDJHV ZLWK(QJOLVKFRQWHQWZHUHILQDOO\XVHGLQWKLVZRUN7KHFRQWHQWRIHDFKZHESDJH ZDVGLVWLQJXLVKHGWRWKHPHWDWDJVWRVRPHVSHFLDOWDJVDQGILQDOO\WRWKHGLVVHPL QDWHGSODLQWH[W7DEOHSUHVHQWVWKHVHDUFKHQJLQHVXVHGWKHHLJKWGLVWLQFWFODVVHV DV ZHOO DV WKH FRQWULEXWHG LQIRUPDWLRQ FOXVWHUV SHU VHDUFK HQJLQHGLUHFWRU\ UHVSHF WLYHO\
= ¥
¥
¥
([FLWH
¥
¥
¥
/\FRV
¥
¥
¥
0616HDUFK
¥
'02=
¥
]
]
¥
¥ ¥
]
%XVLQHVV (FRQRP\
6FLHQFH
]
5HIHUHQFH /LEUDULHV
]
¥
$OWD9LVWD
5HFUHDWLRQ 6SRUWV
1HZV0HGLD
]
+HDOWK
'LUHFWRU\?&ODVV
&RPSXWHUV ,QWHUQHW
$UWV+X PDQLWLHV
Table 1. Distinct classes and contributed information categories per search engine/directory
] ¥
¥
¥ ¥ ¥
¥
¥
1HWVFDSH6HDUFK
¥
¥
¥
¥
¥
¥
¥
¥
4
Fig. 2. Architecture of KaratsubaOf man8
Fig. 3. (a) Operation performed by Shif tnAdder2n and (b) Last addition performed by the Shif tSubnAdder2
48
N. Nedjah and L. de Macedo Mourelle
Fig. 4. Architecture of Shif tSubnAdder2n
5
Efficient Architecture of 4-Bit Multiplier
The modifications of Karatsuba-Ofman’s algorithm we introduced in the previous sections will improve the space and time requirements of the implementation if and only if the the 4-bit multiplier, i.e. F ourBitsM ultiplier in Fig. 2 is implemented efficiently. For this purpose, we specialise once again the Karatsuba-Ofman’s algorithm. Let A = A3 A2 A1 A0 , B = B3 B2 B1 B0 and P be the product AB, with P = P7 P6 P5 P4 P3 P2 P1 P0 . Applying Equation (5) we can compute P as in Equation (13) and Equation (14), wherein AL = A2 A1 A0 and BL = B2 B1 B0 and 0i is a run of i zeros. Assuming that A3 A2 A0 B2 B1 B0 = C5 C4 C3 C2 C1 C0 then, we can compute the bits forming product P as indicated in Equation (15) and Equation (16), wherein CH = C5 C4 C3 and Cy is the most significant bit of the sum, i.e. the carry out signal of summation process. Based on these eqautions, we designed the hardware architecture of an efficient 4-bit multiplier. (13) P = 26 02 A3 02 B3 + 23 02 A3 BL + 02 B3 AL + AL BL P = A3 .B3 06 + A3 .B2 A3 .B1 A3 .B0 03 + B3 .A2 B3 .A1 B3 .A0 03 + AL BL (14) P2 P1 P0 = C2 C1 C0 , P7 = Cy .A3 .B3 , Cy P6 P5 P4 P3 = CH + A3 .B2 A3 .B1 A3 .B0 + B3 .A2 B3 .A1 B3 .A0
(15) (16)
The multiplier of Fig. 5 uses a 3-bit multiplier to yield the product of the lower 3 bits of X and Y and a 3:1-adder to perform the necessary two 3-bit additions. It also employs 8 and-gates. The 3-bit multiplier, denoted by T hreeBitM ultiplier, is implemented using the most compact and most efficient 3-bit multiplier that was obtained through an evolutionary process [6]. It uses 24 gates and is the 8-levels circuit shown in Fig. 6.
Fast Less Recursive Hardware for Large Number Multiplication
49
Fig. 5. Architecture of F ourBitM ultiplier using the evolved T hreeBitM ultiplier
6
Area and Time Requirements
The design of the fully recursive as well as the less recursive were engineered using the Xilinx Project Manager (version Build 6.00.09) [7]. The design was elaborated using VHDL and implemented into logic blocks using SPARTAN S05PC84-4.
Fig. 6. Architecture of F ourBitM ultiplier using the evolved T hreeBitM ultiplier
Table 2 shows the time and area in terms of required CLBs obtained from the Xilinx project manager for the fully recursive (KO) versus the adapted and less recursive (ALKO) Karatsuba-Ofman’s multiplication algorithm. Table 3 shows the delay and area required by the hardware implementation of the modified Booth multiplier which uses a Wallace tree for adding up the partial products (BW ) and another hardware implementation of Booth’s algorithm that uses a redundant binary Booth encoding (P RB) [8]. The engineered KaratsubaOfman multiplier works faster than the other three multipliers. However, it consumes more hardware area. Nevertheless, it also improves also the the area× time factor. Moreover, we strongly think that for larger operands, the proposed Karatsuba-Ofman multipliers will yield much better characteristics.
50
N. Nedjah and L. de Macedo Mourelle Table 2. Performance figures for different operand size operand size KOarea ALKOarea KOdelay ALKOdelay 8 16 32 64
108 525 2645 12348
101 428 2351 10056
12.6 22.8 29.1 41.0
10.7 16.9 22.4 30.4
Table 3. Delays and areas for different multipliers operand size BWdelay BWarea P RBdelay P RBarea 8 16 32
7
44.6 93.9 121.5
1092 5093 20097
31.8 46.6 64.9
862 3955 17151
Conclusion
In this paper, we designed a hardware for big number multiplication using Karatsuba-Ofman’s algorithm. This hardware is efficient and less recursive. It provides better response time than those of Booth-Wallace and redundant binary encoding Booth multiplier. On the other hand, the hardware consumes less area than the fully recursive one. Furthermore, both proposed multipliers improve the area× time product as well as time requirement while the other three improve area at the expense of both time requirement and the factor area× time.
References 1. T. ElGamal, A public-key cryptosystems and signature scheme based on discrete logarithms, IEEE Transactions on Information Theory 31,(1985) 469–472 2. C ¸ . K. Ko¸c, High speed RSA implementation, Technical report, RSA Laboratories, RSA Data Security Inc. CA, version 2, (1994) 3. D.E. Knuth, The art of computer programming: seminumerical algorithms, vol 2, 2nd Edition, Addison-Wesley, (1981) 4. J.F. Dhem, Design of an efficient public-key cryptographic library for RISC-based smart cards, Ph.D. Thesis, Catholic University of Louvain, (1998) 5. N. Nedjah,L. Mourelle, A reconfigurable recursive and efficient hardware for Karatsuba-Ofman’s multiplication algorithm, Proc. of IEEE Conference on Control Applications, (2003) 6. J.F. Miller, D. Job and V.K. Vassilev, Principles in the evolutionary design of digital circuits, Journal of Genetic Programming and Evolvable Machines 1, (2000) 259–288 7. Xilinx, Inc. Foundation Series Software, http://www.xilinx.com. 8. J.H. Kim, J. H. Ryu, A high speed and low power VLSI multiplier using a redundant binary Booth encoding, Proc. of Korean Semiconductor Conference, PA-30, (1999)
Topological and Communication Aspects of Hyper-Star Graphs Jong-Seok Kim1 , Eunseuk Oh2 , Hyeong-Ok Lee3 , and Yeong-Nam Heo1 1
3
Department of Computer Science, Sunchon National University, Sunchon, Chonnam, 540-742, KOREA, {rockhee,hyn}@sunchon.ac.kr 2 Department of Computer Science, Texas A&M University, College Station, TX 77843-3112, USA,
[email protected] Department of Computer Education, Sunchon National University, 315, Maegok-dong, Sunchon, Chonnam, 540-742, KOREA,
[email protected] Abstract. A hyper-star graph HS(m, k) has been introduced as a class of lower cost interconnection networks. Hyper-star graph has more merit than hypercube when degree × diameter is used as a cost measure. In other words, they have smaller degree and diameter than hypercubes. In this paper, we consider some of the important properties of hyperstar graphs such as symmetry, w-diameter, and fault diameter. We show that HS(2n, n) is node-symmetric. We also show that the w-diameter of HS(2n, n) is bounded by the shortest path length plus 4, and fault diameter of HS(2n, n) is bounded by its diameter plus 2. In addition, we introduce an efficient broadcasting scheme in hyper-star graphs based on a spanning tree with minimum height.
1
Introduction
Graph theory has provided powerful mathematical tools for designing interconnection networks, where vertices represent processing nodes and edges correspond to communication links. The performance of a particular parallel computer is heavily dependent on the graph topology chosen for it. Many graph topologies have been proposed in literature, ranging from simple graphs such as trees to more sophisticated graphs such as hypercubes and de Bruijn graphs. They possess various degrees and diameters. These graphical parameters not only have their importance in graph theory and combinatorics but also have relevance to applications in commercial networks. The graph model having a smaller degree and diameter is considered more desirable because this implies a lower hardware implementation cost and shorter transmission time of messages[8]. Several new topologies have been proposed as alternatives to hypercube, which is one of the most popular graph topologies, to further improve the degree and diameter of the hypercube[2,6,9]. A hyper-star graph is one of such new topologies. A hyper-star graph HS(m, k) where 1 ≤ k ≤ m 2 , possesses many desirable properties as an interconnection network topology. It has been shown that it A. Yazici and C. S ¸ ener (Eds.): ISCIS 2003, LNCS 2869, pp. 51–58, 2003. c Springer-Verlag Berlin Heidelberg 2003
52
J.-S. Kim et al.
has better scalability, a simple routing algorithm, maximum fault-tolerance, and lower cost of degree × diameter than hypercube and its variations[7]. In this paper, we further investigate its desirable properties as an interconnection network topology and its communication aspects based on fault diameter and broadcasting. Specifically, a hyper-star graph HS(m, k) is a regular graph only when m = 2k. Thus, we focus on a regular hyper-star graph HS(2n, n). The first graphical parameter we consider is symmetry. Symmetry is an important feature for most graph models for interconnection networks. In a symmetric interconnection network, the load can be evenly distributed through all nodes, reducing congestion problems. Moreover, symmetry makes the design of routing algorithms easier because it allows routing between any two nodes to be mapped to routing between an arbitrary node and a specific node. With regard to the reliability and communication efficiency of HS(2n, n) in the presence of failures, we study w-diameter and fault diameter. The w-diameter of a graph is a natural generalization of the diameter of the graph. The concept of w-diameter is also closely related to the concept of fault diameter. A hyper-star graph HS(2n, n) is n-connected [7]. For any copy of HSf of HS(2n, n) with at most n − 1 faults, the fault diameter of HS(2n, n) is the maximum diameter of HSf . If the fault diameter of HS(2n, n) is bounded by its diameter plus a small additional constant, then the communication delay of HS(2n, n) would not increase dramatically. In this paper, we show that the fault diameter of HS(2n, n) possesses the feature of having a small constant increase in the diameter. Finally, we address a one-to-all broadcasting scheme in HS(2n, n). One-toall broadcasting is a mechanism for disseminating information from a designated node in a graph to all other nodes in the graph. It is not difficult to see that broadcasting is central to many applications on interconnection networks. These applications include a variety of linear algebra algorithms such as matrix-vector multiplication, LU-factorization, and Householder-transformations [4]. For single source broadcasting, we consider the spanning tree as a one-to-all communication graph. The graph of minimum height is desirable because it provides a minimum propagation time of messages. Our scheme takes time equal to the diameter of HS(2n, n), which is optimal.
2
Preliminaries
A hyper-star graph HS(m, k) is an undirected graph consisting of m k nodes, where a node is represented by the string of m bits b1 b2 . . . bi . . . bm such that the cardinality of the set {i|1 ≤ i ≤ m, |bi = “1”|} = k. Let σi be an operation that exchanges b1 and bi , 2 ≤ i ≤ m, where bi is a complement of b1 . Then, two nodes u = b1 b2 . . . bi . . . bm and v = bi b2 . . . b1 . . . bm are connected when v is obtained from the operation σi (u). The edge (u, v) is called an i-edge. For a node u, we denote by [k1 , k2 , . . . , kt ] a path obtained by applying operations σk1 , σk2 , . . . , σkt to u. For example, there is a path [3, 2, 4] or [4, 2, 3] from 0011 to 1100. In this paper, we will concentrate on a hyper-star graph HS(2n, n), which is regular and node-symmetric. Its node-symmetric properties
Topological and Communication Aspects of Hyper-Star Graphs n
53
n
will be discussed below. We write a node 0 . . . 0 1 . . . 1 in HS(2n, n) as 0n 1n . Fig. 1 shows a hyper-star graph HS(6, 3). Let dist(u, v) be the distance from 000111
100011
010011
110001
100101
100110
001011
010101
001101
010110
001110
101001
110100
101100
110010
101010
011001
011100
011010
111000
Fig. 1. HS(6, 3) graph
u = u1 u2 . . . u2n to v = v1 v2 . . . v2n . If a bit string R is obtained by applying the bitwise Exclusive-OR 2n operation to them such as r1 r2 . . . r2n where ri = ui ⊕ vi , then dist(u, v) = i=2 ri . Let R− be the set of bit positions i such that ri = 1. Property 1. For a node w in a path P from u to v, consider an i-edge connecting w and its neighbor w . If i ∈ R− and an edge (w, w ) is on P, then the edge (w, w ) leads to a shortest path from u to v. Definition 1. For two nodes u and v in HS(2n, n), a node w on a path from u to v is said to be in the level Lm if dist(u, w) is m. Property 2. For a node u in HS(2n, n), any two paths P = [k1 , . . . , kt ] and Q = [h1 , . . . , ht ] from u lead to the same node if the sets of numbers with even indices in P and Q are the same, and the sets of numbers with odd indices in P and Q are also the same.
3
Symmetry Properties
A graph G = (V, E) is said to be node-symmetric if, for any two nodes u and v, there exists an automorphism of the graph G that maps u into v. In other words, G has the same shape from any node. For two given nodes u = u1 . . . u2n and v = v1 . . . v2n , we first provide a mapping scheme. Definition 2. For a node u in H(2n, n), a mapping tree Tu rooted at u consists of n children of u, c1 , . . . , cn and n − 1 children of c1 , g1 , . . . , gn−1 .
54
J.-S. Kim et al.
Let Iu = (i1 , . . . , i2n−1 ) be a sequence of i-edges on a mapping tree Tu such that u and cj are connected by an ij -edge, and c1 and gk are connected by an in+k -edge. Similarly, consider a sequence Iv = (i1 , . . . , i2n−1 ) for a node v. A mapping node u to node v can be constructed using the following rules: Mapping Rules(briefly m-rules) Rule 1. If u1 = v1 , then u1 is mapped to v1 , and uij is mapped to vij for ij ∈ Iu and ij ∈ Iv . Rule 2. If u1 = v1 , then u1 is mapped to v1 , and uij is mapped to vij for ij ∈ Iu and ij ∈ Iv . Let two nodes be u = 010110 and v = 000111 in HS(6, 3), then we have mapping trees, Tu and Tv as shown in Fig.2(a). From Tu , we have a sequence Iu = (2, 4, 5, 3, 6), and from Tv , we have a sequence Iv = (4, 5, 6, 2, 3). Since u1 = v1 , u can be mapped to v by Rule 1. Let two nodes be u = 110001 and v = 001011, then we have mapping trees, Tu and Tv as shown in Fig.2(b). Similarly, we have a sequence Iu = (3, 4, 5, 2, 6) and a sequence Iv = (3, 5, 6, 2, 4). Since u1 = v1 , u can be mapped to v by Rule 2. 010110 2 3
4
000111 5
6
4 2
5
6
3
1 2 3 4 5 6
: bit position
0 1 0 1 1 0
:u
1 2 3 4 5 6
: bit position
0 0 0 1 1 1
:v
1 2 3 4 5 6
: bit position
1 1 0 0 0 1
:u
1 2 3 4 5 6
: bit position
0 0 1 0 1 1
:v
(a) 110001 3 2
4 6
001011 3
5 2
5 4
6
(b)
Fig. 2. Mapping from a node u into a node v in HS(6, 3) by corresponding mapping trees: (a) Mapping from u = 010110 into v = 000111 by Rule 1 (b) Mapping from u = 110001 into v = 001011 by Rule 2
It is easy to verify that there is an automorphism of the graph that maps u into v by the m-rules. Thus: Theorem 1. A hyper-star graph HS(2n, n) is node-symmetric. Similarly, we can reduce routing between two arbitrary nodes u and v to routing from an arbitrary node u to a special node v, say 0n 1n . If we find a path P = [p1 , p2 , . . . , pt ] from u to v , then we can reduce the path P to a path Q = [q1 , q2 , . . . , qt ] from u to v such that pi is mapped to qi by m-rules applied to two nodes v and v. Consider a path P = [6, 4, 3, 2] between 001011 and 010110 in HS(6, 3) whose mapping trees for 010110 and 000111 are as shown in
Topological and Communication Aspects of Hyper-Star Graphs
55
Fig.2(a). Then the path P is reduced to a path Q = [3, 5, 2, 4] between 011001 and 000111. Lemma 1. From a node u = 0n 1n in HS(2n, n), a subgraph consisting of nodes in Li , 0 ≤ i ≤ n − 1, and a subgraph consisting of nodes in Lj , n ≤ j ≤ 2n − 1, are symmetric. Proof. The number of nodes of HS(2n, n) is 2n n . To construct a shortest path, we apply operations σi , n + 1 ≤ i ≤ 2n, and σj , 2 ≤ j ≤ n, alternately. Thus, there is no edge between nodes in the same level or between nodes in levels Li and Lj such that |Li − Lj | ≥ 2. A node v = 1n 0n is in L2n−1 and u can be connected to it by a path P = [n + 1, 2, n + 2, 3, . . . , n, 2n]. Since a shortest path from u can be constructed by applying unique operations σi , n + 1 ≤ i ≤ 2n, and σj , 2 ≤ j ≤ n, alternately, any shortest path [k1 , . . . , kt ] from u to a node has the same sets of numbers with even and odd indices as P. Thus, by Property 2, the node v is a unique node in L2n−1 . Nodes in L1 are connected to u by operations σi , n + 1 ≤ i ≤ 2n, and nodes in L2n−2 are connected to v by the same operations. For a node u in L1 , some nodes u , . . . , u in L2 are connected to u by operations σj , 2 ≤ j ≤ n, and some nodes v (= u ), . . . , v (= u ) in L2n−3 are connected to a node v (= u ) by the same operations, and so on. In other words, nodes in Li and nodes in L2n−i−1 , 0 ≤ i ≤ n − 1, are complements, and the number of nodes and corresponding edges in these levels are also the same.
4
w-Diameter and Fault Diameter
Following the conventions introduced in [3], let C(u, v) be a container, which is a set of node-disjoint paths between u and v in a graph G. The width of C(u, v) is the number of paths in C(u, v), and the length of C(u, v) is the length of the longest path in C(u, v). The w-distance is the minimum length over all containers C(u, v) of width w. The w-diameter Dw (G) of G is the maximum w-distance between any pair of nodes in G. If G is k-connected, then for any copy Gf of f (G) of G is the maximum G with at most k − 1 faults, the fault-diameter Dk−1 diameter of Gf . The concept of w-diameter is closely related to the concept of fault diameter. f (G) ≤ Dk (G), where D(G) is the diameter It is well-known that D(G) ≤ Dk−1 of G [1]. If the fault diameter is bounded by the diameter of G plus a small additional constant, then the communication delay of G will not increase dramatically. HS(2n, n) is n-connected and its diameter is 2n−1 [7]. In this section, we discuss w-diameter and fault diameter of HS(2n, n). Consider the cyclic permutation of two sequences S1 = (a1 , a2 , . . . , ap ) and S2 = (b1 , b2 , . . . , bq ) denoted by S1 S2 . S1 S2 is the set of sequences obtained by merging symbols in S1 and S2 alternately. If only one sequence is permuted, say S2 , then we write S1 S2− . For example, if S1 = (5, 6, 7) and S2 = (2, 3, 4), then S1 S2 = {(5, 2, 6, 3, 7, 4),(6, 3, 7, 4, 5, 2),(7, 4, 5, 2, 6, 3)}. Also, S1 = (5, 6, 7) and S2 = (2, 3), then S1 S2− = {(5, 2, 6, 3, 7), (6, 2, 7, 3, 5), (7, 2, 5, 3, 6)}. Paths
56
J.-S. Kim et al.
obtained by applying operations corresponding to sequences of S1 S2 are nodedisjoint because the number of symbols in the set sharing an internal node is p+q. Similarly, paths obtained by applying operations corresponding to sequences of S1 S2− are also node-disjoint. For simplicity, we regard a symbol i in a sequence as an operation σi . other. In addition, |S1 S2 | = max{p, q} and |S1 S2− | = p. Let φ = dist(u, v). Lemma 2. For two nodes u = 0n 1n and v in HS(2n, n), there is a length φ container of width φ2 . Proof. Suppose φ = dist(u, v) is even. Consider the set of paths constructed by S1 S2 , where S1 = (n + 1, n + 2, . . . , n + φ2 ) and S2 = (2, 3, . . . , φ2 + 1). They are node-disjoint, and the number of paths constructed by S1 S2 is φ2 . Let us assume that u is in L0 . Then, for any pair of adjacent nodes (pi , pj ) in P, dist(u, pj ) = dist(u, pi ) + 1. In other words, each node in P is in a unique level. Suppose φ = dist(u, v) is odd. Consider the set of paths constructed by S1 S2− , where S1 = (n + 1, n + 2, . . . , n + φ2 ) and S2− = (2, 3, . . . , φ2 ). Paths constructed by S1 S2− are node-disjoint, and the number of such paths is φ2 . Therefore, the lemma holds. Theorem 2. For any two nodes u and v in HS(2n, n), Dn (HS(2n, n)) ≤ dist(u, v) + 4. Proof. Since HS(2n, n) is node-symmetric, we assume that a node u is 0n 1n . Suppose there is a shortest path P=[n + 1, 2, n + 2, 3, . . . , n + φ2 , φ2 + 1] from u to v. From lemma 2, we know that φ is even, and there are φ2 node-disjoint paths of length dist(u, v) from u to v. They are constructed by S1 S2 where S1 = (n + 1, n + 2, . . . , n + φ2 ) and S2 = (2, 3, . . . , φ2 + 1). Also, there are n − φ2 symbols between n + φ2 + 1 and 2n which are unused in paths constructed by S1 S2 . We construct n − φ2 paths of the form [j,P ,j] for all j, n + φ2 ≤ j ≤ 2n from u to v where P is a sequence that transposes adjacent symbols (pi , pj ) in P. That is, P =(2, n + 1, 3, n + 2, . . . , φ2 + 1, n + φ2 ). Paths constructed by S1 S2 and paths of the form [j,P ,j] are node-disjoint because j is a unique symbol. Thus, if φ is even, then we can construct n node-disjoint paths that consist of φ2 paths of length dist(u, v) and n − φ2 of length dist(u, v) + 2. Suppose there is a shortest path P=[n + 1, 2, n + 2, 3, . . . , φ2 , n + φ2 ] from u to v. Similarly, from lemma 2, φ is odd, and there are φ2 node-disjoint paths of length dist(u, v) from u to v. They are constructed by S1 S2− where S1 = (n + 1, n + 2, . . . , n + φ2 ) and S2− = (2, 3, . . . , φ2 ). Also, there are n − φ2 pairs of symbols (j, k) where n + φ2 + 1 ≤ j ≤ 2n and φ2 + 1 ≤ k ≤ n. These are unused in paths constructed by S1 S2− . We construct n − φ2 paths of the form [j, k,P,j, k], for all j, n + φ2 ≤ j ≤ 2n, and for all k, φ2 + 1 ≤ k ≤ n, from u to v where j = k + n. Paths constructed by S1 S2− and paths of the form [j, k,P,j, k] are node-disjoint because a pair (j, k) is unique. Let v be a node
Topological and Communication Aspects of Hyper-Star Graphs
57
obtained by σk , σj from v, and Q be a path of the form [j, k,P,j, k]. Thus, if φ is odd, then we can construct n node-disjoint paths that consist of φ2 paths of length dist(u, v) and n − φ2 of length dist(u, v) + 4. f (HS(2n, n)) = D(HS(2n, n)) + 2 = 2n + 1. Theorem 3. Dn−1
5
Broadcasting Scheme
For broadcasting from a source node, we define a spanning tree rooted at the source node of HS(2n, n). We assume that a source node u is 0n 1n , but our construction can be easily generalized for trees rooted at arbitrary nodes. Let Parent(v) be a function that represents the parent of v, and Children(v) be a function that represents the children of v. Then for a node v and its grandparent g, Parent(Parent(v)), let I = {i|ri = gi ⊕ vi = 1}, and define i0 ∈ I and i1 ∈ I with 1 ≤ i0 ≤ n and n + 1 ≤ i1 ≤ 2n when v is in an even level and 1 ≤ i1 ≤ n and n + 1 ≤ i0 ≤ 2n when v is in an odd level. Also, let rh = 0 for all h ∈ Ψ where Ψ = {i1 + 1, i1 + 2, . . . , 2n} or Ψ = {i1 + 1, i1 + 2, . . . , n}, in the first instance and second instance, respectively. That is, Ψ is the set of trailing zeros in R from the i1 th position. Definition 3. Let a source node u be 0n 1n . Then a spanning tree ST(u) rooted at u is defined by the functions Parent(v) and Children(v) as follows: Children(v) = σh (v), for all h in Ψ , Parent(v) = σi0 (v). Specifically, for the source node u = v, let i1 = n and Parent(v) = φ, and for children of u, let i1 = 1 and I = {i|ri = ui ⊕ vi = 1}. It is easy to see that children of v are nodes connected by h-edges for all h in Ψ , and the parent of v is the node connected by an i0 -edge. Theorem 4. For the source node u = 0n 1n , the spanning tree ST (u) has optimum height equal to 2n − 1. Proof. For any node w in ST (u), consider a bit string R = r1 . . . r2n obtained by applying the bitwise Exclusive-OR operation to w and u, and the set of bit position i, R− such that ri = vi ⊕ ui . Then the function Parent(w) finds a parent σi0 (w) such that i0 ∈ R− . Thus, by Property 1, the edge (w, Parent(w)) leads to a shortest path to u. Since the hyper-star graph is node-symmetric, the height of ST (u) is equal to the diameter of the hyper-star graph. Specifically, if w = 1n 0n , then edges connecting w and u in ST (u) construct a shortest path between w and u of length 2n − 1. In our broadcasting scheme, we restrict communication to a single-port at a time. We briefly mention our broadcasting scheme: First, the source node u sends a message M to the left-most child u1 , with the result that u and u1 hold M . Then u sends M to the next left-most child u2 , and u1 sends M to its left-most child. We continue this operation until all nodes in ST (u) receive the message M . This scheme takes 2n − 1 time, which is optimal.
58
6
J.-S. Kim et al.
Concluding Remarks
We have shown that HS(2n, n) is node-symmetric. Further study on other properties of HS(2n, n) such as Hamiltonicity would be interesting. We also showed that the fault diameter of HS(2n, n) is its diameter plus 2. Our result is optimal because some node-disjoint path must lead to that bound. Latifi[5] has suggested investigating whether the whole family of Cayley graphs have the feature that their fault diameters are one greater than their diameters. We suspect that HS(2n, n) is a member of the Cayley graph family. Further investigation would be interesting. Finally, we developed a broadcasting algorithm of HS(2n, n) based on a spanning tree with optimal height. We restricted communication to a single-port at a time. Developing a communication graph with other capabilities such as multi-port or all-port for concurrent communication would also be interesting.
References 1. C.-P. Chang, T.-Y. Sung, and L.-H. Hsu, “Edge Congestion and Topological Properties of Crossed Cubes,” IEEE Trans. Parallel and Distributed Systems, Vol. 11, No. 1, pp. 64–80, 2000. 2. K. Efe, “A Variation on the Hypercube with Lower Diameter,” IEEE Trans. Computers, vol. 40, no. 11, pp. 1312–1316, 1991. 3. D. F. Hsu, “On Container Width and Length in Graphs, Groups, and Networks,” IEICE Trans. Fundamentals E77-A, pp. 668–680, 1994. 4. S. L. Johnsson, “Communication Efficient Basic Linear Algebra Computations on Hypercube Architectures,” J. Parallel Distrib. Comput., vol. 4, pp. 133–172, 1987. 5. S. Latifi, “Combinatorial Analysis of the Fault-Diameter of the n-cube,” IEEE Trans. Computers, vol. 42, no.1, pp. 27–33, 1993. 6. C.-N. Lai, G.-H. Chen, and D.-R. Duh, “Constructing One-to-Many Disjoint Paths in Folded Hypercubes,” IEEE Trans. Computers, vol. 51, no. 1, pp. 33–34, 2002. 7. H.-O. Lee, J.-S. Kim, E. Oh, and H.-S. Lim, “Hyper-Star Graph: A New Interconnection Network Improving the Network Cost of Hypercube,” Proc. EurAsia ICT: Information and Communication Technology, LNCS 2510, pp. 858–865, 2002. 8. B. Parhami and D.-M. Kwai, “Unified Formulation of Honeycomb and Diamond Networks,” IEEE Trans. Parallel and Distributed Systems, vol.12, no.1, pp. 74–80, 2001. 9. S.-K. Yun and K.-H.Park, “Comments on Hierarchical Cubic Networks,” IEEE Trans. Parallel and Distributed Systems, vol.9, no.4, pp. 410–414, 1998.
An E-tutoring Service Architecture Based on Overlay Networks Nikolaos Minogiannis, Charalampos Patrikakis, Andreas Rompotis, and Frankiskos Ninos Telecommunications Laboratory National Technical University of Athens, Heroon Politechniou 9, Zographou Campus, Greece 15773, tel: +30 210 7721513, fax: +30 210 7722534 {minogian,bpatr,arobot,ninosf}@telecom.ntua.gr Abstract. In this paper, a comprehensive e-tutoring service framework is presented. It compromises a set of individual applications implemented based on the use of existing open source tools. The significant point here is the flexibility of the implementation proposed, that allows it to use a combination of unicast and multicast without posing any requirements for the supporting network infrastructure. This is achieved through the use of an overlay network architecture that allows for dynamic configuration of the media transmission and relay points.
1
Introduction
Recently, there is a remarkable increase in research around online tutoring and learning and their respective use in educational technology. There are many academic journals in research community study this area, while IEEE has appointed a relative committee, Computer Society Learning Technology Task Force (LTTF) to work on the topic. There is a focus on collaborative discourse [1], [2] and the individual development of meaning through construction and sharing of texts, image, video streaming and other social artifacts. From these perspectives, learners are apprenticed into ”communities of practice” which embody certain interests and educational behaviors. An e-tutoring service imposes a great burden to the underlying network in terms of bandwidth resources and packet traffic administration. In order to fulfill such requirements, many technologies have been utilized, such as Multi-agent systems [3], multicasting [4] and adaptive network mechanisms for controlling and adjusting the generated traffic. This paper proposes an implementation of an e-tutoring service based on standard IP and using overlay networks. The rest of the paper is organized as follows: First a definition of the service framework for an e-tutoring application is provided in which the requirements for such a service, together with a presentation of the supporting technology for implementing such a service are presented. In the next section, and after having set the framework for the e-tutoring service to be presented, the actual implementation scheme is provided in details covering both architectural issues and specific implementation details. Closing the paper, a section about future (and in fact ongoing work) for providing a more dynamic model of the presented idea is provided. A. Yazici and C. S ¸ ener (Eds.): ISCIS 2003, LNCS 2869, pp. 59–66, 2003. c Springer-Verlag Berlin Heidelberg 2003
60
2
N. Minogiannis et al.
Definition of the Service Framework for E-tutoring
The methodology for the definition of the required application components for an e-tutoring service arises by answering the following questions: what is the functionality provided by a real classroom, which applications can support it and what are the features of these applications. The criterion for the success is to efficiently provide all the necessary tools available in a real school environment. 2.1
The E-classroom Framework
This service framework is presented in Figure 1, in which the outside circle entities represent the actual needs of a real life classroom, covering the whole process of lectures, while the inner “grey” circle gives the equivalent applications offered by this platform to cover these needs, respectively [6].
Fig. 1. E-classroom framework in relation to real classroom needs
Lecture in class. This is the lesson itself delivered through the presentation provided by the tutor. It is covered by a Live Video or Video on Demand application that is used to send the video of the class presentation to the students. Questions in class and group discussion. This covers the process of interaction between the teacher and the students during the presentation. To cover this need in the most efficient and cost effective way, a chat application can be used. Blackboard. This represents the means for communicating with the students through written or image projected information. In the blackboard, not only the teacher, but also the students (having the consent of the teacher) may have access. The related application is a whiteboard, access to which is administered by the teacher. Taking into account the above presentation and trying to balance the need for a comprehensive e-tutoring service with the need for supporting it through a cost effective, easy to install service platform, we will proceed in the following
An E-tutoring Service Architecture Based on Overlay Networks
61
sections of the paper by presenting such a platform for covering the lecturing phase of the lesson. Before we proceed in the design and implementation details of the e-tutoring service, we have to stress the requirements that drove us to the selection of overlay networks as the supporting infrastructure. For this, a presentation of the requirements of an e-learning (and especially e-tutoring) service follows, together with the existing state of the art for supporting them in real implementations. 2.2
Requirements for Collaborative Applications
There is a diverse range of applications that inherently require group communication and collaboration: video conferencing, distance tutoring, distance learning, distributed databases, data replication, multi-party games and distributed simulation, network broadcast services and many others. The diversity of these applications demands versatile support from the underlying system in many dimensions. Examples of these dimensions include bandwidth, latency, reliability, multi-source, scalability and dynamics requirements. The main interest of this paper is focused on distance learning, and specifically in e-tutoring. Supporting these applications has imposed a serious challenge to the current communication systems. Due to the prevalence of underlying point-to-point connections, communication systems are quickly reaching their limit. A typical example is that requests to a popular web server usually experience long response time due to server overload since it has to establish individual connections for each incoming data request, even for requests for the same objects. The inadequacy of unicast-only systems is more significant for these forward-looking applications, especially in distributed systems where data needs to be constantly updated and synchronized. 2.3
The Native IP Multicast Deployement and Torsion to Overlay Architectures
The IP Multicast service was proposed as an extension to the Internet architecture to support efficient multi-point packet delivery at the network level. In a multicast network, the connections of the routers that form a multicast tree are maintained by a multicast routing protocol. Many protocols have been proposed and are in use today on the Internet. In spite of the rigorous efforts of a generation of researchers, there remains many unresolved issues in the IP multicast model that hinder its development and deployment of multicast applications. To overcome the fundamental problems related to IP multicasting “global” deployment, research effort has been turned to other solutions based on application layer data forwarding. As a consequence overlay network architectures have been proposed for supporting data distribution and in some cases used for serving streaming applications. The primary advantage of an overlay network architecture is that it does not require universal network support (which has become increasingly hard to achieve) to be useful. This enables faster deployment of desired network functions and adds flexibility to the service infrastructure, as
62
N. Minogiannis et al.
it allows the co-existence of multiple overlay networks each supporting a different set of service functions. An Overlay Multicast Network is one type of overlay network that provides multicast services to end users on top of the general Internet unicast infrastructure [7]. For this, the present model focuses on the deployment of an overlay network incorporating and exploiting the application-layer multicasting capabilities.
3
Presentation of the Implementation
In the following paragraphs, an implementation of an e-tutoring service based on the above ideas will be presented with specific implementation details on the modules developed and the open source tools used. 3.1
Description of the Architecture
The proposed platform provides a solution for media distribution through peripheral reflector points. The whole architecture incorporates multicasting techniques deployed at a local level while the further communication with users don’t reside at the same network neighbourhood can be accomplished through unicast streams. The group of students participate in the e-tutoring framework accessing the lecture information through different serving points. A dedicated Video Server -VS is used in order to transmit and manage the streams produced by the camera located near the teacher’s site. These streams before entering the VS should be encoded in appropriate formats. In our architecture the MPEG-4 encoding technique is selected. The main components of the distribution chain are the Video Streaming Server used for sending the video and the Video Relay Nodes - VRNs (both based on a QTSS implementation), used for retransmitting the video to neighbouring nodes in the overlay architecture. The VRN is also responsible for transforming a unicast stream coming from the VS to multicast for supporting the clients within its vicinity (e.g. a LAN). Another significant part of the architecture is the Web Server. Web Server is used as staring point to which students are addressing their requests for participating in an e-tutoring session. The Web-Server (Content Management Element) is the key component for the provisioning of a user-friendly interface for Chat and Whiteboard applications and also for the redirection of user requests for Streaming Video access to the VS or the closest VRN. The students terminals, make use of the Chat and Whiteboard applications through separate software modules. Live Video Application/Streaming Service. In Live Video Application, digitally compressed video and audio streams are sent to the end-user upon demand and can be retrieved on an individual basis. The Live Video System supports the MPEG family of protocols, namely MPEG-2 and MPEG-4. For the implementation, MPEG4 standard was adopted due to its inherent ability to support video transmission even to very low rates. Figure 2 depicts the stages
An E-tutoring Service Architecture Based on Overlay Networks
63
that the overlay architecture encompasses in order to provide real-time services (live video) as well as e-tutoring applications.
Fig. 2. Stages followed in live video distribution procedure
The top row represents the content production procedure, while the bottom row provides the description of the distribution mechanism in terms of network elements. Apart from the camera located at the teacher’s site, the main components of the content production stage are the VS and the encoder. Content Preparation: The MPEG4IP encoder [10] used for this purpose, provides a tool that incorporates the necessary functionality for MPEG-4 real-time encoding. The package includes many existing open source packages and the ”glue” to integrate them together. This is a tool for streaming video and audio that is standards-oriented and free from proprietary protocols and extensions. Input to the MPEG4IP was provided through a camera connected to a PC running LINUX. Content Management: The Web Server initiates the communication between the teacher and the students group. Once the e-tutoring session starts, the students follow a URL in order to open the corresponding Web page hosted by the Server (in our case an Apache web server). Through the HTTP protocol, the students have intermediate access to the video content. The Web Server is responsible for the user’s redirection to a Video Streaming Server (this being the VS itself or a VRN according the process described later in the paper) which is streaming the teacher’s lecture video to the students. Content Distribution: The QTSS server [9] is the key component in the distribution chain. It is responsible for the manipulation and transmission of the video content to the rest network infrastructure, with ulterior scope the video delivery to student’s group. The distribution of the video content may be accomplished through two different methods: Unicast transmission used in cases where the student does not reside in the same cluster as the origin VS is located and Unicast/Multicast transmission in cases the overlay procedure is activated. In this approach QTSSs are placed at the edges of the network or other cooperating
64
N. Minogiannis et al.
networks exploiting application data forwarding capabilities. This architecture appeals to peer-to-peer collaboration activities. Content Relaying: A QTSS is also used for this purpose. Each client instead of using a direct connection to the VS, as it would in normal data distribution; it receives the distributed data through the use of the nearest relay server. This is supported by the overlay network management mechanism. In this way, we avoid multiple unicast connections to the VS, and instead, we serve these requests through peripheral reflectors. Redirection of the client’s requests to the peripheral reflectors is provided through the Web Server, which is kept aware of the overlay architecture, through reports from the overlay network management mechanism. Content Reproduction: At the clients, a video player that is capable of supporting MPEG4 playback is used for content reproduction. In our implementation the QuickTime player was used. Overlay Network Implementation and Management. Proceeding to our analysis, we will introduce the basic notions of the applied overlay structure implemented for supporting the e-tutoring service. As it is depicted in Figure 3, the overlay architecture is based on four modules: the Overlay Architecture Server (OAS), the Overlay Architecture Relay (OAR), the Overlay Architecture Client (OAC) and the Overlay Architecture Manager (OAM). These three modules are responsible for the manipulation of the video access requests of the clients, the connection of each client to the most suitable point (this being either a VRN or a VS) for video access.
Fig. 3. Overlay network architecture
The way this architecture works is the following: Prior to the connection of a student’s terminal to the VS serving the live transmission of the e-lecture stream, the student connects to the web server in order to request access to the
An E-tutoring Service Architecture Based on Overlay Networks
65
e-tutoring service. Once access is granted, the OAC (located at the student’s terminal) is activated and by communicating with the OAM (located on the Web server), it receives the address of the OAS, (located on the same node as the VS), and of all the available OARs for the specific e-lecture class (located at the VRNs). Following, the OAC initiates an inquiry mechanism for determining the closest node for video transmission (or video relaying). This mechanism is based on the sending ping like messages to all the nodes reported by the OAM as possible video serving nodes. Upon receipt of all the answers to this ping, the OCM selects the closest one and reports this selection to the OAM. This in turn, provides the address of the selected video serving node to the web server, which presents it to the student. Now the student can be directed to this node for accessing the video of the lecture. The OAM keeps a list of all connections to a VS or VRN during an e-lecture session for monitoring and administrative purposes (i.e indication of a problematic relay node as we will present later). The above procedure covers the discovery of the closest video serving point. But what happens when we need to deploy multicasting for a number of clients in the same LAN, and how is the student’s player notified about this possibility of receiving video over multicast? The answer is the ping like message used for closest server discovery. This message, instead of transporting only information about the distance between a server/relay node and the requesting client, it also transfers information about the availability of multicast transmission, the local domain of the video serving node and the clients it can serve. The last is needed in cases where a relay node is serving clients through unicast, where a limited number of clients is supported in order to avoid degradation in the QoS of relayed video due to insufficient processing power. Based on this information, the client may perform a comprehensive selection over the available solutions, and report this selection to the web server through OAC to OAM communication. The overlay modules are independent from these responsible for video distribution. In this way, they introduce a dynamic video distribution configuration through the use of an administrating web server acting as the intermediate link.
Completing the Service Framework with Chat and Whiteboard Applications. As it has been presented earlier, to support the e-tutoring process the service framework needs to be completed by some additional applications that assist the tutor - students’ communication. These are the chat and whiteboard applications that may operate individual and autonomously in relation to the live video application. Chat Application. The most significant part is the creation of a multi-channel environment, where students participating in a working group could be able to talk with each other in virtual rooms managed by their teacher. The teacher’s role in this case is to create and manage the channels according to the existing working groups. In order to achieve this, the e-lecturing chat application consists of three IRC sub-components: a typical IRC server application and two versions of the chat client, each providing student and teacher functionality respectively.
66
N. Minogiannis et al.
Whiteboard Application. The Whiteboard application allows the user (a teacher or student) to draw an arbitrary number of elements on a single surface and share that surface with other participants of an e-learning session. The application also allows a surface to be persisted between sessions by use of the ‘Save’ command. All these commands are easily accessible through a typical Windows GUI. At last, a custom protocol was designed to guarantee that all participants share the same Whiteboard surface.
4
Future Work
The implementation presented here, though it makes use of an overlay architecture for providing dynamic configuration of the e-tutoring service in terms of media distribution, is still depending on some decisions taken from the student regarding the connection to the video serving node. This is due to the fact that the web server is treated as the link point between the open source components used for serving video transport and the developed modules for overlay network architecture management. An alternative implementation in which these modules are integrated within a compact application deployed at the relay points and the users’ terminals would offer the ability for providing a dynamically selfconfigured e-tutoring service, transparently to the user.
References 1. D. Keegan: Theoretical Principles of Distance Education. Routledge (1993) 2. C. J. Bonk, & K. S. King (eds.): Electronic collaborators: Learner-centered technologies for literacy, apprenticeship, and discourse. Erlbaum (1998) 3. A. Garro: An XML Multi-Agent System for e-Learning and Skill Management. Third International Symposium on Multi-Agent Systems, Large Complex Systems, and E-Businesses (MALCEB’2002), Erfurt, Thuringia Germany, (2002) 4. Sirgio Deusdado, Paulo Carvalho: An Adaptive e-learning System based on IP Multicast Technology. International Conference on Information and Communication Technologies in Education (ICTE2002), Badajoz, Espanha, Nov 20-23 (2002) 5. Ch. Patrikakis, K. Karapetsas, N. Minogiannis, S. Vrontis, N. Igoumenidis, G. Diakonikolaou: A QoS aware e-learning service framework: The MOICANE case. International Conference on Cross-Media Service Delivery, Santorini, May (2003) 6. Y. S. Shi: Design of Overlay Networks for Internet Multicast. Washington University Severe Institute of Technology, Dept of Computer Science, Phd Thesis, August 2002 7. QuickTime Streaming Server: http://developer.apple.com/darwin/projects/streaming 8. MPEG4IP Project: http://www.mpeg4ip.net
A Simple Scheme for Local Failure Recovery of Multi-directional Multicast Trees Vladimír Dynda Department of Computer Science and Engineering Faculty of Electrical Engineering, Czech Technical University in Prague Fh¼y¸½¸ûÿ· ²³ü"!"$ Q¼htÃr!8ªrpuSr¦Ãiyvp `q'·qh5sryp½Ã³pª
Abstract. When a node in a multicast tree fails, the tree is divided into several fragments. To achieve a fault-tolerant communication, failure recovery schemes are necessary to restore the tree. We present a simple recovery scheme for overlay multicast trees that involves only failure-neighboring nodes into the restoration and keeps the original structure of the rest of the tree. The scheme is based on virtual bypass rings providing alternative paths to eliminate the faulty node and reroute the traffic. Our scheme is scalable, independent of message source and traffic direction in the tree, restores the multicast tree in real time without a significant delay penalty and our experiments show that it is efficient even under a heavy traffic in the tree.
1
Introduction
Srprû³y'·hû'qv²³¼viórqh¦¦yvph³v¸û²¦h¼³vpÃyh¼y'¦rr¼³¸¦rr¼ vû³r¼ûr³ih²rq ²'² ³r·² uh½r irp¸·r vûp¼rh²vûty' ¦¸¦Ãyh¼ 6² ³ur²r ²'²³r·² ûrrq ³¸ ¦¼¸½vqr ¼ryvhiyr ·Ãy³v¦yr¦¸vû³ p¸··Ãûvph³vût ²r¼½vpr hû rssvpvrû³ hûq shÃy³³¸yr¼hû³ ·Ãy³vph²³ v² p¼v³vphys¸¼³urv¼²Ãppr²² 6² ²¸·r shvyür² ¸ppüh·Ãy³vph²³ ³¼rrZuvpuv² òÃhyy'òrq ³¸p¸ûûrp³hyy ¦h¼ ³vpv¦h³vût û¸qr² ·h' ir ¦h¼³v³v¸ûrq vû³¸ ³Z¸ ¸¼ ·¸¼r ²Ãi³¼rr² û¸³ p¸ûûrp³rq Zv³u ¸ûr hû¸³ur¼ Zuvpu h¼r phyyrq s¼ht·rû³² ³ur¼rhs³r¼ Uur¼r h¼r ³Z¸ h¦¦¼¸hpur² s¸¼ h qv²³¼viórq³¼rr¼r²³¸¼h³v¸û 9'ûh·vp¼r²³¸¼h³v¸ûq¸r²û¸³ph¼r hi¸Ã³ ¦h¼³v³v¸ûvûtir s¸¼r v³¸ppü² hûq¸ûy'Zurû¦h¼³v³v¸ûvûtv²qr³rp³rq q¸r²³ur²'²³r· h³³r·¦³³¸¼rw¸vû ³ur s¼ht·rû³² vû³¸ h p¸ûûrp³rq t¼h¦u òvût h ph¦hpv³' ²rh¼pu hyt¸¼v³u· Uuv² ¸û qr·hûq¼rp¸½r¼'òÃhyy's¸¼pr²hssrp³rq t¼¸Ã¦ ·r·ir¼²³¸hihûq¸û ³urr`v²³vût p¸û ûrp³v¸û² hûq ¼rw¸vû³ur³¼rr bdb&dþUur²u¸¼³p¸·vûtv² ³ury¸ût ¼r²³¸¼h³v¸û yh³rûp' Zuvpup¸ÃyqirÃûqr²v¼hiyrs¸¼·hû'¼rhy³v·rh¦¦yvph³v¸û²hûqv³v²û¸³hppr¦³hiyr s¸¼ ²¸·r ¸³ur¼ ³v·rp¼v³vphy h¦¦yvph³v¸û² Zuvpu ¼r¹Ãv¼r tÃh¼hû³rrq û¸ûvû³r¼¼Ã¦³rq p¸··Ãûvph³v¸û²Uurhy³r¼ûh³v½rv² ³ur¦¼r¦yhûûrqh¦¦¼¸hpu Zur¼rihpx漸ór²h¼r ²r³irs¸¼ruhûqvûhqqv³v¸û ³¸³ur¦¼v·h¼'p¸¼rþ¼¸Ã³r²hûq ³ur'h¼rhp³v½h³rq Zurûh shvyür ¸û ³ur ¦¼v·h¼' ¼¸Ã³r v² qr³rp³rq b#d b$dþ 6y³u¸Ãtu q'ûh·vp ¼r²³¸¼h³v¸û ²pur·r² h¼r ·¸¼r ph¦hpv³' rssvpvrû³ v³ v² òÃhyy' ¦¼¸sv³hiyr ³¸ qr¦y¸' h ²v·¦yr ¦¼r ¦yhûûrq ·r³u¸qsv¼²³irs¸¼r h ·¸¼r³v·r p¸û²Ã·vûtq'ûh·vp¼r²³¸¼h³v¸û v²¦¸²²viy' òrqr²¦rpvhyy'vû³ur ph²r¸s²vûtyr¸¼²·hyy²phyrshvyür² 6Áhªvpvhûq8ùrûr¼@q²þ)DT8DT!"GI8T!©%(¦¦%&ý!" T¦¼vûtr¼Wr¼yht7r¼yvûCrvqryir¼t!"
68
V. Dynda
Xr ¦¼r²rû³ h ¦¼r¦yhûûrq shÃy³ ¼rp¸½r¼' ²pur·r ih²rq ¸û ½v¼³Ãhy p'pyvp ihpxæ ¦h³u² phyyrqE\SDVVULQJVòrq ³¸i'¦h²²³urshÃy³'û¸qrhûq¼r²³¸¼r ³ur³¼rr Zv³u¸Ã³ ³¼hssvp vû³r¼¼Ã¦³v¸û xrr¦vût ³ur ¸¼vtvûhy ³¼rr ²³¼Ãp³Ã¼r vû²³rhq ¸s ²³h¼³vût s¼¸· ²p¼h³puTvûpr¸Ã¼ ¦¼¸³¸p¸yrû²Ã¼r²³uh³hyys¼ht·rû³²h¼r¼rp¸ûûrp³rq hûq ³uh³û¸p' pyrv²s¸¼·rqqüvût ¼r²³¸¼h³v¸ûû¸ ·h³³r¼h³u¸Z·hû'û¸qr² ³ur shÃy³v²qr³rp³rqv³ phûiròrq h²h ¦yh³s¸¼· s¸¼½h¼v¸Ã²·r³u¸q²¸s³¼rr ¼rp¸ûûrp³v¸û³uh³vûsyÃrûpr³ur ¼r²Ãy³vût³¼rr ³¸¦¸y¸t'vû ³ury¸phy h¼rh¸s ³urshvyürUur ²pur·r phûiròrq Ãûv½r¼ ²hyy' vû h¦¦yvph³v¸û² òvût ¸½r¼yh' ³¼rr³¸¦¸y¸t' ûr³Z¸¼x² s¸¼ ·r²²htr ¦¼¸¦hth³v¸û Ãûvph²³vût¼¸Ã³vût¸iwrp³y¸ph³v¸û¼r¦yvph·hûhtr·rû³r³pþ Uur¼r²³¸s¦h¦r¼v²²³¼Ãp³Ã¼rq h²s¸yy¸Z²Dû²rp³v¸û ! ³ur¼ryh³rqZ¸¼xv²²Ã··h ¼vªrqTrp³v¸û" ¦¼r²rû³²³ur·¸qryhûq ³urû¸³h³v¸û²Ã²rqsü³ur¼vû ³ur ³r`³Dû²rp ³v¸û# Zrvû³¼¸qÃpr³ur shÃy³¼rp¸½r¼' ²pur·r qr²p¼vir ³ur shvyür¼rp¸½r¼' ·rpuh ûv²· hûqqrhyZv³u·r³u¸q²s¸¼ ³¼rr¼rp¸ûûrp³v¸û Trp³v¸û$qv²pò²r²³ur¦¼¸¦r¼³vr² ¸s¸Ã¼¦¼¸³¸p¸y hûq²rp³v¸û % p¸û³hvû²p¸ûpyòv¸û hûq ²r³²²¸·rsóürqv¼rp³v¸û²
2
Related Work
Tr½r¼hy shÃy³ ¼rp¸½r¼' ²pur·r² s¸¼ ·Ãy³vph²³ ³¼rr² uh½r irrû ¦¼r½v¸Ã²y' ¼r¦¸¼³rq ih²rq ¸û ¦¼r¦yhûûrq shvyür ¼rp¸½r¼' Qh³u ¼r²³¸¼h³v¸û ¸¼ yvûx ¼r²³¸¼h³v¸û ²pur·r² ·hvû³hvû ¦¼rp¸·¦Ã³rq ihpxæ ½v¼³Ãhy ¦h³u² ³uh³ rv³ur¼ ¦¼¸³rp³ hû rû³v¼r vûqv½vqÃhy rûq³¸rûq ½v¼³Ãhy¦h³u¸¼h¼r òrq³¸ ¼r¼¸Ã³r³ur³¼hssvp¸¼vtvûhyy'ph¼¼vrqi'h²vûtyr shvyrqyvûxDû²pur·r²b%d hûqb(d h ½r¼³r`qv²w¸vû³ihpxæ¦h³u s¼¸· ³ur ·Ãy³vph²³ ²¸Ã¼pr³¸ rhpuqr²³vûh³v¸ûv²²r³Ã¦Uur¼r²³¸¼h³v¸û¦¼¸pr²² v²sh²³ió³ur²pur·r²h¼r ²Ãv³hiyr¸ûy's¸¼²vûtyr²¸Ã¼pr·Ãy³vph²³³¼rr²Gvûx ¼r²³¸¼h³v¸û²pur·rbd¦¼¸³rp³² hthvû²³h ²vûtyryvûxshvyür0h ihpxæ ¦h³u s¸¼rhpuyvûxv²·hvû³hvûrqUuv² ²pur·r v² ²phyhiyr hûq ¦¼r²r¼½r² y¸phyv³' ¸s ³ur ¼r²³¸¼h³v¸û u¸Zr½r¼ v³ phû û¸³ ir rh²vy' r` ³rûqrq³¸¼rp¸½r¼³ur³¼rrs¼¸·h·Ãy³v¦yrshvyür¸shqwhprû³û¸qr² Dû ³ur9ÃhyU¼rr Tpur·rb#dh ²rp¸ûqh¼'³¼rr ¦¼¸½vqvût hy³r¼ûh³v½rqryv½r¼'¦h³u² v² iÃvy³ hûq v³ v² hp³v½h³rq Zurû û¸qr ¸¼ yvûx shvyür v² qr³rp³rq vû ³ur ¦¼v·h¼' ³¼rr Uur ²pur·r h²²Ã·r² ³uh³ ³ur Ãûqr¼y'vût ûr³Z¸¼x v² h ivp¸ûûrp³rq t¼h¦u Zur¼r h ²rp¸ûqh¼' ³¼rr qv²w¸vû³ s¼¸· ³ur ¦¼v·h¼' ³¼rr phû ir p¸û²³¼Ãp³rq Dû ³ur @ssvpvrû³ AhÃy³U¸yr¼hû³ HÃy³vph²³ S¸Ã³vût Q¼¸³¸p¸y b$di'¦h²²¦h³u²p¸ûûrp³ û¸qr² hûq ³urv¼ t¼hûq¦h¼rû³² vû³ur³¼rr hûqh¼ròrq ³¸¼rp¸ûûrp³s¼ht·rû³²Zurû ³ur¦h¼rû³û¸qr ¸s ³ur²rû¸qr²shvy² Uuv²²¸yóv¸ûv²û¸³ qr²vtûrqs¸¼ ·Ãy³v²¸Ã¼pr·Ãy³vph²³³¼rr² Zur¼r ³ur·r²²htrqv¼rp³v¸û¸s³rûhy³r¼² 6qvssr¼rû³h¦¦¼¸hpuv²Ã²rqvû 7h'rÃ`h¼puv³rp³Ã¼rb!dUur·r²²htr²h¼r¼¸Ã³rq vûh ³¼rrqr³r¼·vûrqi'hp¸··¸û ¦¼rsv`¸s hqq¼r²²r²Zv³u ²·hyy²hy³½hyÃr²rûhiyvût qryv½r¼' r½rû vû ³ur ph²r ³uh³ h ¦h¼³vpÃyh¼ vû³r¼·rqvh³r hqq¼r²² v² û¸³ h½hvyhiyr 7h'rÃ`h¼puv³rp³Ã¼r¼r¹Ãv¼r²Uh¦r²³¼'bdh²v³²Ãûqr¼y'vûtvûs¼h²³¼Ãp³Ã¼r
3
Model and Notations
UurÃûqr¼y'vûtûr³Z¸¼x²Ãi²³¼h³r³uh³Zrp¸û²vqr¼vû ³uv²¦h¦r¼ v²h ûr³Z¸¼x¦¼¸½vq vûth ¼¸Ã³vût ²r¼½vprrtDQûr³Z¸¼xUh¦r²³¼'bdQh²³¼' b©d¸¼ wò³h²r³¸s ½v¼³Ãhy
A Simple Scheme for Local Failure Recovery of Multi-directional Multicast Trees
69
vû³r¼¦¼¸pr²² p¸ûûrp³v¸û² Zv³u ¼¸Ã³vût ph¦hivyv³'þ ·¸qryrq h² h t¼h¦u 6129(þ Zur¼r9 v²hsvûv³r²r³¸s ½r¼³vpr²¼r¦¼r²rû³vûtû¸qr²0( v²h svûv³r²r³¸srqtr² ¼r¦¼r ²rû³vûtyvûx²ir³Zrrûû¸qr²vû³urûr³Z¸¼x 6 ·Ãy³vph²³t¼¸Ã¦0* vrh ²r³¸sû¸qr² ¼rprv½vût ·Ãy³vph²³·r²²htr²þv²hû h¼iv ³¼h¼'²Ãi²r³¸sû¸qr²s¼¸· 9 0*2ºQL0L 2 «N0QL ∈ 9±Zur¼rQL h¼rû¸qr²s¼¸· 61 ³uh³h¼r³¸ r`puhûtrvûs¸¼·h³v¸ûhûq N2_0*_v² ³ur ·Ãy³vph²³t¼¸Ã¦²vªrHr· ir¼² ¸sh tv½rû·Ãy³vph²³t¼¸Ã¦ h¼r p¸ûûrp³rq i'hû ¸½r¼yh'·Ãy³vph²³³¼rrZuvpuv² òrq h² h ²¸Ã¼prvûqr¦rûqrû³ ²³¼Ãp³Ã¼r s¸¼ ·r²²htr ¦¼¸¦hth³v¸û ³¸ ³ur t¼¸Ã¦ ·r· ir¼²Uur·Ãy³vph²³³¼rrv²·¸qryrqh²h t¼h¦u0720*&(þZur¼r &( v²h²r³¸s FRUHWUHHHGJHV½v¼³Ãhy yvûx² iÃvy³¸û ³¸¦ ¸s 61þ p¸ûûrp³vûtû¸qr²vû 0*6ûh¼iv ³¼h¼' ²r³¸s û¸qr² s¼¸· 0* v² ³ur ²r³ ¸s ·Ãy³vph²³ ²¸Ã¼pr û¸qr² qv²²r·vûh³vût ·r² ²htr²³¸¸³ur¼û¸qr²vû³ur³¼rrUuò·r²²htr²h¼r³¼hû²·v³³rqhy¸ût rhpu½v¼³Ãhyyvûx vû i¸³uqv¼rp³v¸û²h²¸¦¦¸²rq³¸h²vûtyr²¸Ã¼pr·Ãy³vph²³³¼rrþ Xr h²²Ã·r³uh³û¸qr² vû07 ·h'shvyhûq ³uh³³urv¼ shÃy³'²³h³rphû ir qr³rp³rq i' ûrvtui¸¼vûtû¸qr²I¸³r³uh³Zrq¸ û¸³ uh½r³¸ qrhyZv³u ¦u'²vphyyvûxshvyür²²vûpr ³ur·Ãy³vph²³³¼rrv²hû¸½r¼yh'²³¼Ãp³Ã¼rUuò ·r²²htrqryv½r¼'hp¼¸²²h½v¼³Ãhyyvûx p¸ûûrp³vût³Z¸07ûrvtui¸¼² Q hûqQ! qr¦rûq²¸û¼¸Ã³vûtvû ³ur Ãûqr¼y'vût ûr³Z¸¼x shi¼vp hûq Zrr`¦rp³v³ ³¸qryv½r¼³ur·r²²htrvs ³ur¼r v² h¦h³u s¼¸· û¸qr Q³¸ û¸qr Q! vû 61 Aü³ur¼ Zr p¸û²vqr¼ rhpu û¸qr Q ∈ 9 ³¸ ir h²²vtûrq Zv³u hû 61Ãûv¹Ãr vqrû³vsvr¼,'Qs¼¸· h³¸³hyy'¸¼qr¼rq²r³¸svqrû³vsvr¼² rtDQhqq¼r²² ¸¼v³²uh²uþ
4
Fault Recovery Scheme
HÃy³v¦yr¦¸vû³ p¸··Ãûvph³v¸û òvût h ·Ãy³vph²³ ³¼rr v² ½Ãyûr¼hiyr ³¸ r½rû h ²vûtyr û¸qr shvyür Xr vû³¼¸qÃpr h ¦¼¸³¸p¸y ih²rq ¸û ½v¼³Ãhy i'¦h²² ¼vût² p¸û²³¼Ãp³rq h¼¸Ãûq rhpuû¸qrvû h ·Ãy³vph²³t¼¸Ã¦hûq òrq³¸ i'¦h²²³urû¸qrvû ³urph²r¸sshvy ür hûq³¸¼rp¸ûûrp³³ur³¼rrs¼ht·rû³²h½¸vqvûtp'pyr² 4.1
Basic Definitions
Gr³ 0720*&(þ ir ³ur ³¼rr³¸¦¸y¸t' p¸··Ãûvph³v¸û ûr³Z¸¼x 7'¦h²² ¼vût sü ³ur¼ qrû¸³rq %5Fþprû³r¼rq h³û¸qrF ∈ 0* v²h pv¼pÃv³p¸û²v²³vût ¸shû ¸¼qr¼rq ²r ¹Ãrûpr¸sû¸qr²QQ!« QW ∈ 0* Q2 QW²Ãpu ³uh³) !
A¸¼hyy QLL 2«Wqv²³hûprvû³ur³¼rrGLVW07QLFþ2hûq ,'QL 1 ,'QL + s¸¼ L 2«W
Uuri'¦h²² ¼vût %5Fþ²u¸Zûvû Avtp¸û²v²³² ¸s³ur¸¼qr¼rq ²r¹Ãrûpr¸s û¸qr² ©!©768"© uh½vût ¦¼¸¦r¼³' !Uurqv²³hûpr¸srhpu¸s³ur²rû¸qr²s¼¸· ³urprû³r¼û¸qrFv²u¸¦¦¼¸¦r¼³'þ 6qv¼rp³rq E\SDVVHGJH EHL2QLQLþ s¸¼hyyL2«Wþ v² h ½v¼³Ãhyyvûxs¼¸· û¸qr QL³¸ û¸qrQLZur¼rQLv²³urvûv³vhyû¸qrhûqQL³ur³r¼·vûhyû¸qr¸sEHL Dû³rt¼h³v¸û¸s hyyi'¦h²²¼vût²hûq hû 07t¼h¦up¼rh³r²hû H[WHQGHGPXOWLFDVWWUHH (0720*&( ∪ %(þZur¼r&( v²³ur²r³¸s p¸¼r³¼rrrqtr² ¸s¸¼vtvûhy07 hûq%( v²³ur²r³¸s hyyi'¦h²²rqtr²07v²³urûh²¦hûûvût ³¼rr¸s (07
70
V. Dynda
6 8" © HÃy³vph²³ ³¼rr 7'¦h ²² ¼vQt %5ÿÿ 8rQ³r¼ Q¸qr F
F !
©7
6
I¸qr Zv³uD926 hQq v³² ²Ãi³¼rr ¸s ·Ãy³vph²³ ³¼rr
)LJ@`h·¦yr¸s h i'¦h²² ¼vût
4.2
Failure Recovery
Uurt¸hy ¸s ³ur shvyür¼rp¸½r¼' v² ·Ãy³vph²³ ³¼rr ¼r²³¸¼h³v¸û hs³r¼ shvyür¸s h ²vûtyr ³¼rrû¸qrUur ¼r²³¸¼h³v¸û¦¼¸pr²² p¼rh³r²ûrZ p¸¼r ³¼rrrqtr²Ã²vûth i'¦h²²¼vût ³¸ p¸ûûrp³³¼rr s¼ht·rû³²hûq¼r²³¸¼r³ur·Ãy³vph²³³¼rr07³¸ ³ur p¸ûûrp³rqhûqp¸û²v² ³rû³²³h³r D³ uh²³¸ irh²²Ã¼rq³uh³³ur¼r²³¸¼h³v¸û¦¼¸pr²²¸¦r¼h³r²¦¼¸¦r¼y'û¸·h³³r¼ Zur¼rhûqh³u¸Z·hû'û¸qr²³ur ³¼rr¼r²³¸¼h³v¸û v²²v·Ãy³hûr¸Ã²y'vûv³vh³rq Gr³ û¸qr F ir³urshÃy³'û¸qrvû(0720*&( ∪ %(þhûq %5Fþv³² i'¦h²² ¼vût p¸û²v²³vût¸sû¸qr²Q«QW 9rsvûrsÃûp³v¸ûSQLþ s¸¼hyyQLL2 « W h²s¸yy¸Z²) ! !!
SQLþ2 ,'Q vsû¸qr QLuh² irrûsv¼²³û¸³vsvrqhi¸Ã³û¸qr Fshvyüri' û¸qr QI ∈ %5FþDsQLqr³rp³²³urshvyürv³²rys ³urû QI2 QL SQLþv²ÃûqrsvûrqvsQLuh²û¸³'r³irrûvû½¸y½rqvûû¸qrFshvyür¼rp¸½r¼' ÿ
9rsvûr¼ryh³v¸û " "! ""
)QL
QMvshûq¸ûy'vs)
QLQMþ v²³uri'¦h²² rqtr¸s %5Fþ hûq SQLþv²qrsvûrqhûq SQLþ1SQMþ¸¼SQMþv²û¸³ qrsvûrq
The basic idea behind the restoration process is that each neighbor nf = ni of faulty node c detecting the failure consecutively iterates along the ring BRc(1) in the direction of the bypass edges through the nodes QLÿ , QLþ , … ∈ BRc(1) ( QLÿ is the ringneighbor of ni = QLý , QL þ is the ring-neighbor of QLÿ , etc.) until it reaches a node QL S that has already been notified about failure (i.e., R( QL S ) is defined). At each hop QLT , QLT hûq QLT v² û¸³vsvrq hi¸Ã³ shvyür ¸s û¸qr F T S v³ v² qr³r¼·vûrq vs QLT − ²Ãpu³uh³S QLT þ2 ,'QL Ur¼·vûh³v¸û¸s³uv²¦¼¸pr²²v²tÃh¼hû³rrq²vûpr %5þp¸û ²v²³²¸shsvûv³rû÷ir¼¸sû¸qr²rhpuû¸qruh²hsvûv³r²r³¸sûrvtui¸¼²vû07þ 6s³r¼ û¸qr QL T uh² irrû û¸³vsvrq hûq vs QLT − QLT ³urû h ûrZ p¸¼r ³¼rr rqtr FHL T 2 QLT QL þv²p¸û²³¼Ãp³rq Zv³u³urs¸yy¸Zvût ¦¼¸¦r¼³vr²) ÿ
4.1 #! 4.3
r < q and QLT − QLT hûq There is a path QLT , QL , …, ni in the restored MT. ÿ
Several methods can be deployed for selection of node QL (see Sect. 4.3). ÿ
A Simple Scheme for Local Failure Recovery of Multi-directional Multicast Trees "
!
S@TUPS@p ©ÿ
S@TUPS@p ©ÿ
6
7'¦h²² ¼vQt%5ÿÿ @qtr² ¸s %5ÿÿ³¸ir ²Ãi²³v³Ã³rq Zv³uQrZ p¸¼r³¼rrrqtr²
!
# S@T UPS@ p ©7ÿ
S@E@8U
AhÃy³' Q¸qrF
S@TUPS@ p ©ÿ
F
T³r¦Q÷ir¼ !
©7
·ÿ
P¼vtvQhy p¸¼r rqtr²
©
8"
"
·þ
71
S@TUPS@p ©7ÿ
·ÿ Hr²²htrP ÿ ÿþýüûÿþ Hr²²htr¦h²²vQt 6
#
I¸qr Zv³uD926 hQqv³² s¼ht·rQ³ ¸s·Ãy³vph²³³¼rr
Fig. 2. Example of single failure recovery using a two-way restoration algorithm
After all nodes n1, …, nt ∈ BRc(1) have been notified, ³ur¼ryh³v¸û ir³Zrrû the incident nodes of all bypass edges bei is known. With the definition of the bypass ring ¼ryh³v¸û, it can be proven ³uh³¼ryh³v¸û v² not cyclic on the bypass ring and hûq that there is only one edge bei = (nu, nv) ∈ BRc(1) such ³uh³ QX QY is not true. Together with properties 4.1 – 4.3 of newly constructed core tree edges, we can get the following: 5.1 5.2
The restored MT consists of all fragments induced by a node failure. The restored MT is a tree graph.
Moreover, the described technique is independent of the restoration initiating node and also prevents collisions in cases when multiple nodes initiate restoration simultaneously (which can easily happen in a distributed system). A practical tree restoration algorithm may incorporate several performance improvements: • The iteration along the ring from the failure detecting node ni is performed in both directions (further referred to as a two-way algorithm). • The iteration is not performed by ni, it is rather delegated by the ring members. • The new core tree edges are constructed on the fly, immediately after node QLT is QLT determined. notified and relation QLT − The two-way algorithm works as follows: Each node ni that during message routing through the multicast tree realizes that one of its neighbors, say, node c, is down, excludes node c from its bypass ring %5 Q (1) and sends S@TUPS@(IDc, ,'Q ) messages to the both its ring-neighbors on the ring BRc(1). Upon receiving a S@TUPS@ message from its ring-neighbor, each node QLT determines relation QLT − QLT and forwards the S@TUPS@message along the ring. If there is relation QLT − QLT û¸qr QLT ²ryrp³² QL according to the chosen reconnection method and creates a new core tree edge FHL T = ( QLT , QL ). An example of the distributed construction of ¼ryh³v¸û on BR(1) is shown in Fig. 2. In the first step, upon routing of multicast message m1, node © finds that node c is not available so it sends S@TUPS@c©ÿ messages to its ring-neighbors (step 2) to determine ³ur relation. In step 4, another node (©7) initiates the tree restoration ÿ
ÿ
ÿ
ÿ
72
V. Dynda
and similarly sends a S@TUPS@c©7ÿ to its ring-neighbors. Since the first S@TUPS@message received by node 6 originated at node © (R(6) = ©) and the second one was received from node ©7, node 6 rejects the second message (there is no relation ©7 6, since R(©7)
Abstract. In this study, a fuzzy logic based algorithm is presented for the concentration estimation of the CCl4 and CHCl3 gases by using the steady state sensor response and an artificial neural network (ANN) structure is proposed for the concentration estimation of the same gases inside the sensor response time by using the transient sensor response. The Quartz Crystal Microbalance (QCM) type sensors were used as gas sensors. A computer controlled measurement and automation system with IEEE 488 card was used to control the gas concentration values and to collect the sensor responses. Acceptable performance was obtained for the concentration estimation with fuzzy inference. The appropriateness of the artificial neural network for the gas concentration determination inside the sensor response time is observed.
1 Introduction The volatile organic vapours in ambient air are known to be reactive photochemically, and can have harmful effects upon long-term exposure at moderate levels. These type organic compounds are widely used as a solvent in a large number of the chemical industry and in the printing plants [1]. Developing and designing sensors for the specific detection of hazardous components in the mixture of many others is important [2]. In recent years, a variety of sensitive and selective coating materials have been investigated for chemical sensors. The molecules to be detected interact with the sensitive coating. They may be identified quantitatively by changes of physical or chemical parameters of adsorbate/analyte systems such as the refracting index, capacitance, conductivity, total mass, etc. The total mass changes can be monitored by means of quartz crystal microbalance (QCM) sensors, which are widely used as thickness monitors [3,4]. One of the most important problems which is related to the sensor parameters encountered in the high sensitive sensor research is slow response times [5]. In this study, fuzzy inference is employed as a concentration estimation method with a QCM gas sensor, which shows good performance regardless of the ambient temperature- humidity variations as well as the concentration changes and the performance for the CCl4 and CHCl3 gases and the suitability of this method are discussed based on the experimental results. In this method, the steady state responses
$2 values and the other is to implement the algorithm on different source languages. A preliminary work is already being carried out for Turkish texts.
References ! " # $ 6. &
Xv³³rÿDH¸ssh³ 6 7ryyU8)Hhÿhtvÿt Bvthi'³r² Thÿ A¼hÿ²v²p¸ (((þ Iry²¸ÿH)Uur 9h³h 8¸·¦¼r²²v¸ÿ7¸¸x HýUQÃiyv²uvÿt IrZÁ¸¼x ((%þ Iry²¸ÿH)6¼v³u 8¸qvÿtT³h³H¸qryvÿt29h³h 8¸·¦ 9¼ 9¸ii¶² E¸Ã¼ÿhy((þ Urhuhÿ XE)H¸qryyvÿt @ÿtyv²uUr`³ Qu9Uur²v² Vÿv½ ¸sXhvxh³¸ Ia((©þ 7ü¼¸Z² HXurryr¼ 9E)67y¸pxT¸¼³vÿtG¸²²yr²² 8¸·¦ 6yt TS8 VT6((#þ Stallings W.: Cryptography and Network Security, Prentice Hall, NJ (1999) 9hyxvyvp B) T³h³v²³vphy Q¼¸¦r¼³vr² ¸s8¸ÿ³r·¦¸¼h¼'Uüxv²uhÿqh Ur`³8¸·¦¼r²²v¸ÿ6¦ ¦yvph³v¸ÿ MSc.Thesis. (In Turkish), Ege Univ., Turkey (2001) © 8hyth¼' 8¸¼¦Ã²VSG)s³¦p¦p²Ãphyth¼'ph¦Ãi¦¼¸wrp³²³r`³p¸·¦¼r²²v¸ÿp¸¼¦Ã² ( Hrÿrªr²6 ½hÿP¸¼²pu¸³ Q Whÿ²³¸ÿrT)Chÿqi¸¸x¸s 8¼'¦³¸ 8S8 Q¼r²²((&þ B6QHhÿÃhy u³³¦)ZZZth¦²'²³r·¸¼t
An Alternative Compressed Storage Format for Sparse Matrices Anand Ekambaram and Eur´ıpides Montagne School of Electrical Engineering and Computer Science University of Central Florida Orlando, FL 32816 {ekambara,eurip}@cs.ucf.edu
Abstract. The handling of the sparse matrix vector product(SMVP) is a common kernel in many scientific applications. This kernel is an irregular problem, which has led to the development of several compressed storage formats such as CRS, CCS, and JDS among others. We propose an alternative storage format, the Transpose Jagged Diagonal Storage(TJDS), which is inspired from the Jagged Diagonal Storage format and makes no assumptions about the sparsity pattern of the matrix. We present a selection of sparse matrices and compare the storage requirements needed using JDS and TJDS formats, and we show that the TDJS format needs less storage space than the JDS format because the permutation array is not required. Another advantage of the proposed format is that although TJDS also suffers the drawback of indirect addressing, it does not need the permutation step after the computation of the SMVP.
1
Introduction
The irregular nature of sparse matrix-vector multiplication, Ax = y , has led to the development of a variety of compressed storage formats, which are widely used because although these formats suffer the drawback of indirect addressing, they have the advantage that they do not store any unnecessary elements [1], [4], [5]. The main idea considered in compressed storage formats is to avoid the handling and storage of zero values. This is accomplished by means of the storage of the non-zero elements of the sparse matrix in a contiguous way using a linear array. However, some additional arrays are needed for knowing where the non-zero elements fit into the sparse matrix. The number of subsidiary arrays varies depending on the storage format used. Research into sparse matrix reorganization has dealt with developing various static storage reduction schemes such as Compressed Row Storage(CRS), Compressed Column Storage(CCS), and Jagged Diagonal Storage(JDS). One of these methods, the Jagged Diagonal Storage format(JDS) is, in addition, considered very convenient for the implementation of iterative methods on parallel and vector processors [3], [5]. In this work we present the Transpose Jagged Diagonal Storage format(TJDS), which is inspired from the Jagged Diagonal Storage format. The performance of both A. Yazici and C. S ¸ ener (Eds.): ISCIS 2003, LNCS 2869, pp. 196–203, 2003. c Springer-Verlag Berlin Heidelberg 2003
An Alternative Compressed Storage Format for Sparse Matrices
197
algorithms is similar in terms of the memory access pattern, four load operations and one store operation to compute each partial result. However, TJDS does not need a permutation array nor does it need a permutation step to compute the matrix vector product, Ax = y. In Sect. 2 we describe how departing from the CRS and CCS formats we obtain the compressed matrices, Ajds and Atjds , using the JDS format and the TJDS format respectively. In Sect. 3 we show a comparison of the memory space required for a set of matrices using each storage format. Also we give the execution times for the matrix-vector product using the JDS and TJDS algorithms. Finally, in Sect. 4 we present our conclusions.
2
Storage Requirements for the JDS and TJDS Formats
When a sparse matrix A is compressed using the Jagged Diagonal Storage format, all the non-zero elements(nze) in each row are shifted to the left and this way we obtain the matrix Acrs , a compressed row version of matrix A. Then we reorder the rows of the matrix Acrs in decreasing number of non-zero elements from top to bottom, in order to create a Ajds matrix, which is the jagged diagonal version of Acrs . As follows, we present the three steps required to transform a matrix A into is JDS version Ajds . Step 1: The matrix vector product Ax = y is expressed in matrix form as follows:
a11 0 a31 0 0 0
a12 a22 0 a42 0 0
0 a23 a33 0 0 0
a14 0 a34 a44 0 0
0 a25 0 a45 a55 a65
x1 0 y1 x2 y 2 0 0 x3 = y 3 a46 x4 y 4 a56 x5 y 5 a66 x6 y6
Step 2: By applying the CRS scheme to the sparse matrix A we obtain the matrix Acrs , while the vectors x and y remain the same.
Acrs
a11 a22 a31 = a42 a55 a65
a12 a23 a33 a44 a56 a66
a14 a25 a34 a45 0 0
0 0 0 a46 0 0
x1 00 y1 x2 y2 0 0 0 0 x = x3 y = y3 x4 y4 0 0 x5 y5 0 0 00 x6 y6
Step 3: We obtain Ajds a Jagged Diagonal Storage version from Acrs by reordering the rows of Acrs in decreasing order from top to bottom according to the number of nonzero elements per row. It can be seen that the rows one and four of the matrix Acrs were permuted as shown below.
198
A. Ekambaram and E. Montagne
Ajds
a42 a11 a22 = a31 a55 a65
a44 a12 a23 a33 a56 a66
a45 a14 a25 a34 0 0
a46 0 0 0 0 0
00 y1 x1 y2 x2 0 0 0 0 x = x3 y = y3 y4 0 0 x4 y5 x5 0 0 00 x6 y6
We now store the Ajds as a single dimension array. The nonzero elements of the Ajds matrix are stored in a floating-point linear array value list, one column after another. Each one of these columns is called a jagged diagonal and we use a semi-colon(;) in the array value list to separate them. The length of the array value list is equal to the number of non-zeros elements in Ajds . We also need another array column indexes to store the column indices of the non-zero elements. Since the rows have been permuted to obtain a different matrix, Ajds , from the original matrix A, we need an array perm vector to permute the resulting vector back to the original ordering. Obviously the size of this array is n for an n-dimensional matrix. A fourth array is also needed, start positions, which stores the starting position of the jagged diagonals in the array value list. The length of this array is the number of elements in the row of A having the maximum number of non-zero elements plus one. The last element of this array is used to control the inner f or loop of the algorithm shown below. The data structures required to compute Ax = y using JDS are shown below: value list a42 a11 a22 a31 a55 a65; a44 a12 a23 a33 a56 a66; a45 a14 a25 a34; a46 column indexes 2 1 2 1 5 5; 4 2 3 3 6 6; 5 4 5 4; 6 start positions 1 7 13 17 18 perm vector 4 2 3 1 5 6 X x1 x2 x3 x4 x5 x6 Y y1 y2 y3 y4 y5 y6 The sequential algorithm to perform the matrix-vector product Ax = y using the JDS format is shown below. In this algorithm, num jdiag stands for the number of jagged diagonals: disp := 0 for i ← 1 to num jdiag for j ← 1 to (start position[i+1] - start position[i] - 1) Y [j] := Y [j] + value list[disp] × X[column indexes[disp]] disp := disp + 1 endfor endfor
An Alternative Compressed Storage Format for Sparse Matrices
199
The performance of this algorithm is clearly determined by the number of memory accesses which are four load operations and one store operation to compute each partial result. In addition, we need to perform the permutation of the resulting vector back to the original ordering using an algorithm similar to the one shown below: for i ← 1 to N T emp[perm vector[i]] := Y [i] for i ← 1 to N Y [i] := T emp[i] Another popular format for storing sparse matrices is the Compressed Column Storage(CCS) format or Harwell-Boeing sparse matrix format. In this storage format a matrix A is compressed along the columns by shifting all the non-zero elements(nze) upwards. By applying the CCS to the sparse matrix A we obtain the Accs matrix, while the vectors x and y remain the same, as shown below: y1 a11 a12 a23 a14 a25 a46 x1 y2 a31 a22 a33 a34 a45 a56 x2 y3 0 a42 0 a44 a55 a66 x3 Accs = 0 0 0 0 a65 0 x = x4 y = y4 y5 0 0 0 0 0 0 x5 x6 y6 0 0 0 0 0 0 The Transpose Jagged Diagonal Storage(TJDS) scheme is obtained from the CCS format by reordering the columns of Accs , from left to right, in decreasing order of the number of non-zero elements per column and reordering the elements of the vector x accordingly as if it were an additional row of A. This new arrangement, Atjds , is presented as follows: y1 a25 a12 a14 a46 a11 a23 x5 y2 a45 a22 a34 a56 a31 a33 x2 y3 a55 a42 a44 a66 0 0 x4 Atjds = a65 0 0 0 0 0 x = x6 y = y4 y5 0 0 0 0 0 0 x1 x3 y6 0 0 0 0 0 0 To obtain the corresponding arrays, value list, row indexes, and start position we proceed as follows. The nonzero elements of the compressed ordered matrix Atjds are stored in a floating point linear array value list, one row after another. Each of these rows is called a transpose jagged diagonal(tjd). Another array of the same length of value list is used to store the row indexes of the non-zero elements in the original matrix A. This new array that we denote, row indexes, is of type integer. Finally, a third array called start position stores the starting position of each tjd stored in the array value list. These arrays are shown below:
200
A. Ekambaram and E. Montagne
value list a25 a12 a14 a46 a11 a23; a45 a22 a34 a56 a31 a33; a55 a42 a44 a66; a65 row indexes 2 1 1 4 1 2; 4 2 3 5 3 3; 5 4 4 6; 6 start position 1 7 13 17 18 X x5 x2 x4 x6 x1 x3 Y y1 y2 y3 y4 y5 y6 In the above data structures, each transpose jagged diagonal is separated by a semi-colon(;) in the array value list. The sequential algorithm to perform the matrix-vector product Ax = y using the TJDS format is shown below. In this algorithm, num tjdiag stands for the number of transpose jagged diagonals: for i ← 1 to num tjdiag k := 1 for j ← start position[i] to start position[i + 1] − 1 p := row indexes[k] Y [p] := Y [p] + value list[j] × X[k] k := k + 1 endfor endfor
3
Evaluation of Memory Requirements and Execution Times
We present a selection of sparse matrices from the Matrix-Market collection [2] to evaluate the storage requirements of TJDS compared to JDS. For each matrix we present its name, dimension, Nnze , the longest Nnze per column and the longest Nnze per row as shown in Table. 1 and Table. 2. We see that, to store the non-zero elements that constitutes the actual data, we also need subsidiary data, which varies for each storage scheme. This subsidiary data includes storing of row/column indices(JDS, TJDS), the permutation vector(JDS), and the number of jagged diagonals(JDS, TJDS). As you can see in the case of JDS, we need an array of size N to store the original permutation. However, TJDS eliminates the need for this array of size N . The subsidiary storage requirements for the JDS, and TJDS formats are calculated using the following equations respectively: ST ORAGEjds = (Nnze × 1) + N + Njd . ST ORAGEtjds = (Nnze × 1) + Ntjd . Where, Nnze denotes the number of non-zero elements of matrix A.
(1) (2)
An Alternative Compressed Storage Format for Sparse Matrices
201
Table 1. Selection of small sparse matrices from Matrix Market.
1 2 3 4 5 6 7 8 9
Matrix CRY2500 GEMAT11 LNS3937 SHERMAN5 BCSPWR10 DW8192 DWT 2680 LSHP3466 BCSSTM25
Dimension 2500 x 2500 4929 x 4929 3937 x 3937 3312 x 3312 5300 x 5300 8192 x 8192 2680 x 2680 3466 x 3466 15439 x 15439
Nnze Longest Nnzec Longest Nnzer 12349 6 5 33185 28 27 25407 13 11 20793 17 21 13571 14 14 41746 8 8 13853 19 19 13681 7 7 15439 1 1
Table 2. Selection of large sparse matrices from Matrix Market.
1 2 3 4
Matrix CRY10000 BCSSTK18 BCSSTK25 MEMPLUS
Dimension 10000 x 10000 11948 x 11948 15439 x 15439 17758 x 17758
Nnze Longest Nnzec Longest Nnzer 49699 6 5 80519 49 49 133840 59 59 126150 353 353
(Nnze × 1) gives the storage required to store the array indices of the non-zero elements. N denotes the number of rows and M denotes the number of columns. Since we are considering square matrices, N = M . In the case of JDS, N stands for the length of the perm vector. Njd denotes the number of jagged diagonals. Ntjd denotes the number of transpose jagged diagonals. In Fig. 1 we show the subsidiary storage required for the selected matrices using the two storage formats. On the other hand, it is worthwhile mentioning that the number of jagged diagonals in any sparse matrix is equal to the maximum number of non-zero elements per row. Likewise, in the case of the TJDS format the number of transpose jagged diagonals is equal to the maximum number of non-zero elements per column. Hence, for symmetric matrices we have Njd = Ntjd = longest Nnzpr ≤ N . When we compare TJDS with other widely used storage formats such as Compressed Row Storage(CRS) and Compressed Column Storage(CCS)[6], we still have a substantial subsidiary storage that is saved. These two storage formats need to store the starting positions of either the rows or the columns which is of size N or M. For square matrices, we will have that N equals M and Ntjd is generally much smaller than N or M . For example, in the case of the matrix MEMPLUS in Table. 2, N = M = 17, 758 and Ntjd = 353.
202
A. Ekambaram and E. Montagne
Fig. 1. Subsidiary storage requirements using the JDS and TJDS formats.
The matrix-vector product algorithm for JDS and TJDS are similar in the sense that they both have four load operations and one store operation to compute each partial result. But we see that TJDS outperforms JDS because the permutation step needed in the JDS algorithm is not required for the TJDS algorithm. Fig. 2 gives the execution time for the matrix-vector product using the JDS and TJDS algorithms. The JDS algorithm and TJDS algorithm have been executed sequentially on a AMD T-Bird 900MHz processor with 256MB RAM. The matrix-vector product was executed 1000 times to obtain the timing results.
Fig. 2. Matrix-Vector product execution time using the JDS and TJDS algorithms.
4
Conclusions
We have presented a solution to the sparse matrix vector product problem using TJDS, an alternative storage format that requires less storage space than the JDS format. We have also shown that TJDS does not need the permutation vector required by JDS to permute the resulting vector back to the original ordering nor does it need the permutation step to compute the sparse matrix vector product. This new format is suitable for parallel and distributed processing because the data partition scheme inherent to the data structures
An Alternative Compressed Storage Format for Sparse Matrices
203
keeps the locality of reference on the non-zero values of the matrix and the elements of the x array. Currently a parallel implementation of the SMVP using the TJDS format is under way.
Acknowledgements. We would like to thank the anonymous referees for their valuable comments and suggestions to improve the presentation of this paper.
References 1. J. Dongarra, Sparse Matrix Storage Formats, In Zhaojun Bai et al, Eds., Templates for the Solution of Algebraic Eigenvalue Problems: A Practical Guide, SIAM, Philadelphia, 2000. Electronic version available at: http://www.cs.utk.edu/˜dongarra/etemplates/node372.html. 2. Matrix Market, the electronic version is available at the following URL: http://math.nist.gov/MatrixMarket/. 3. Y. Saad,Krylov Subspace methods on Supercomputers, Siam J. Sci. Stat. Comp., vol 10(6), pp. 1200–1232, 1989. 4. Y. Saad, SPARSKIT : A basic tool kit for sparse matrix computations Report RIACS-90-20, Research Institute for Advanced Computer Science, NASA Ames Research Center, Moffet Field, CA, 1990. 5. Y. Saad, Iterative Methods for Sparse Linear Systems, PWS publishing company, ITP, Boston, MA, 1996. 6. E. Montagne and A. Ekambaram, An Optimal Storage Format for Sparse Matrices, Dec. 2002(submitted for publication).
Ranking the Possible Alternatives in Flexible Querying: An Extended Possibilistic Approach Guy de Tr´e, Tom Matth´e, Koen Tourn´e, and Bert Callens Computer Science Laboratory, Department of Telecommunications and Information Processing, Ghent University, Sint-Pietersnieuwstraat 41, B-9000 Gent, Belgium
[email protected] Abstract. An important facet of flexible querying and information retrieval is the ranking of the possible alternatives in the result. Adequate ranking information simplifies decision making and helps the user in finding faster the requested information. This paper deals with the construction of ranking methods, which are based on the use of extended possibilistic truth values. Extended possibilistic truth values are a flexible means to model query satisfaction, which additionally allow to deal with cases where some of the imposed query criteria are not applicable. Three alternative ranking approaches are presented and compared with each other based on their applicability in flexible database querying. Keywords: Flexible database querying, extended possibilistic truth values, ranking functions.
1
Introduction
With the incorporation of more flexibility in database querying and information retrieval, query satisfaction is no longer a matter of bivalence, but becomes a matter of degree. The ‘degrees’ to which a record in the database satisfies the criteria imposed by a given query, determine to which extent the instance belongs to the result of the query. Providing an adequate ranking method for the possible alternatives in the result of a query is important, since it simplifies and fastens the retrieval by associating an indication of relative importance. An adequate ranking method is also necessary for a satisfactory representation of the results of a flexible database query. The current state of the art in the modelling of flexible query satisfaction can be summarized in three approaches. The first approach is to associate a fuzzy grade of membership with each record in the result. This grade may be interpreted as a degree of satisfaction or as a degree of confidence [1]. Such an approach is taken, for instance, by Baldwin and Zhou [2]. The second approach is to associate a so-called possibilistic truth value with each record. This approach is taken, for instance, by de Tr´e et al. [3] and has been extended by using so-called extended possibilistic truth values, which additionally allow to deal with cases where some of the imposed query criteria are not applicable [4]. The third approach is based on the use of a [0, 1]-valued possibility measure (and an A. Yazici and C. S ¸ ener (Eds.): ISCIS 2003, LNCS 2869, pp. 204–211, 2003. c Springer-Verlag Berlin Heidelberg 2003
Ranking the Possible Alternatives in Flexible Querying
205
associated necessity measure) [5,6] in order to represent (un)certainty about the satisfaction of the querying criteria for a given record. This approach is taken, e.g., by Prade and Testemale [7], by Bosc et al. [8], by Yazici et al. [9,10], and by Bordogna et al. [11]. In this paper the ranking problem in the approach with extended possibilistic truth values is discussed and handled. In Section 2, the basic definitions and properties of extended possibilistic truth values (EPTV’s) are summarized. In general an EPTV has three components, which respectively denote the possibility that an instance belongs to the result of a query, the possibility that an instance does not belong to the result of a query and the possibility that (some of) the query criteria do not apply for the instance. It is the third component which makes the ranking problem of EPTV’s non-trivial. This problem and the impossibility to define a natural ordering on the universe of EPTV’s are described in Section 3. In Section 4, three approximate ranking alternatives for EPTV’s are introduced: in the first alternative the ranking function is approximated by a ‘weighted sum’ of Eucledian distances. In the second alternative a more simple analytical ranking function is used, whereas the idea behind the third alternative is to minimize the impact of the third component of an EPTV on the ranking. The presented ranking approaches are summarized and compared with each other on the basis of their practical applicability in flexible database querying in Section 5. Finally, some conclusions are given in Section 6.
2
Extended Possibilistic Truth Values
The concept ‘extended possibilistic truth value’ (EPTV) is defined as an extension of the concept ‘possibilistic truth value’ (PTV) which was originally introduced by Prade in [12]. Possibilistic truth values provide an epistemological representation of the truth of a proposition, which allows to reflect our knowledge about the actual truth. Their semantics is defined in terms of a possibility distribution. Definition 1 (Possibilistic Truth Value). With the understanding that P represents the universe of all propositions and ℘(I) ˜ denotes the set of all ordinary fuzzy sets that can be defined over the Boolean set I = {T, F } of truth values (where T represents ‘True’ and F represents ‘False’), the so-called possibilistic truth value t˜(p) of a proposition p ∈ P is formally defined by means of a mapping t˜ : P → ℘(I) ˜ which associates with each p ∈ P a fuzzy set t˜(p). The semantics of the associated fuzzy set t˜(p) is defined in terms of a possibility distribution. With the understanding that t:P →I is the mapping function which associates the value T with p if p is true and associates the value F with p otherwise, this means that (∀ x ∈ I)(Πt(p) (x) = µt˜(p) (x)), i.e. (∀ p ∈ P )(Πt(p) = t˜(p))
206
G. de Tr´e et al.
where Πt(p) (x) denotes the possibility that the value of t(p) conforms to x and µt˜(p) (x) is the membership grade of x within the fuzzy set t˜(p). The truth value of a proposition can be unknown. This is the case if some data in the proposition exist but are not available. For example, the truth value of the proposition “the price of car A is 20.000” is unknown if car A is for sale, but no information about its price is given. An unknown truth value is modelled by the possibility distribution {(T, 1), (F, 1)}, which denotes that it is completely possible that the proposition is true (T ), but it is also completely possible that the proposition is false (F ). For the definition of an extended possibilistic truth value, an extra element ⊥, which represents an undefined truth value, is added to the Boolean set I [13]. The addition of the element ⊥ is inspired by the assumption that the truth value of a proposition can be undefined. This is for example the case if the proposition can not be evaluated due to the non-applicability of (some of) its elements. For example, the truth value of the same proposition “the price of car B is 20.000” is considered to be undefined if it is known for sure that car B is not for sale, in which case it does not make sense to ask for its price (in the supposition that price information is not applicable to cars that are not for sale). Definition 2 (Extended Possibilistic Truth Value). With the understanding that P represents the universe of all propositions and ℘(I ˜ ∗ ) denotes the set of all ordinary fuzzy sets that can be defined over the universal set I ∗ = {T, F, ⊥}, the so-called extended possibilistic truth value t˜∗ (p) of a proposition p ∈ P is formally defined by means of a mapping t˜∗ : P → ℘(I ˜ ∗) which associates with each p ∈ P a fuzzy set t˜∗ (p). The semantics of the associated fuzzy set t˜∗ (p) is defined in terms of a possibility distribution. With the understanding that t∗ : P → I ∗ is the mapping function which associates the value T with p if p is true, associates the value F with p if p is false and associates the value ⊥ with p if (some of ) the elements of p are not applicable, undefined or not supplied, this means that (∀ x ∈ I ∗ )(Πt∗ (p) (x) = µt˜∗ (p) (x)), i.e. (∀ p ∈ P )(Πt∗ (p) = t˜∗ (p)) where Πt∗ (p) (x) denotes the possibility that the value of t∗ (p) conforms to x and µt˜∗ (p) (x) is the membership grade of x within the fuzzy set t˜∗ (p). EPTV’s are not necessarily assumed to be normalized. For practical applications putting a normalization restriction on EPTV’s makes sense. This however excludes the existence of other truth values than T , F and ⊥, which is rather a philosophical matter.
Ranking the Possible Alternatives in Flexible Querying
207
Special cases are: t˜∗ (p) {(T, 1)} {(F, 1)} {(T, 1), (F, 1)} {(⊥, 1)} {(T, 1), (F, 1), (⊥, 1)}
interpretation p is true p is false p is unknown p is undefined p is unknown or undefined
New propositions can be constructed from existing propositions, using socalled logical operators. An unary operator ‘˜ ¬’ is provided for the negation of a ˜ ’, ‘∨ ˜ ’, ‘⇒’ proposition and binary operators ‘∧ ˜ and ‘⇔’ ˜ are respectively provided for the conjunction, disjunction, implication and equivalence of propositions. The arithmetic rules to calculate the EPTV of a composite proposition and the algebraic properties of extended possibilistic truth values are presented in [13]. As illustrated in [4], EPTV’s can be used to express query satisfaction in flexible database querying: the EPTV representing the extent to which a given record satisfies a flexible query can be obtained by aggregating the calculated EPTV’s that denote the extents to which the instance satisfies the different criteria imposed by the query [14].
3
The Problem of Ranking EPTV’s
˜, ∨ ˜, ¬ It turns out that the algebraic structure (℘(I ˜ ∗ ), ∧ ˜ ) is not a lattice [13]. This is due to the semantics of the extra truth value ⊥, which has been interpreted as representing the “undefined” cases. As a consequence, there does not exist a natural ordering relation for this structure. Nevertheless, an adequate ordering function is needed, in order to obtain an efficient representation of the results of a flexible query. With the understanding that t˜1 = {(T, µt˜1 (T )), (F, µt˜1 (F )), (⊥, µt˜1 (⊥))} and t˜2 = {(T, µt˜2 (T )), (F, µt˜2 (F )), (⊥, µt˜2 (⊥))} are two EPTV’s, a partial ordering ‘≥’ can e.g. be defined by t˜1 ≥ t˜2 ⇔ (µt˜1 (T ) ≥ µt˜2 (T )) ∧ (µt˜1 (F ) ≤ µt˜2 (F )) ∧ (µt˜1 (⊥) ≤ µt˜2 (⊥)) However, such a partial ordering function is less adequate for practical purposes since only a limited number of EPTV-pairs can be compared.
4
Three Alternative Ranking Methods for EPTV’s
In this section three approximate ranking alternatives for EPTV’s are presented. In the first alternative, the ranking function is approximated by a ‘weighted sum’ of Eucledian distances (Subsection 4.1). In the second alternative, an extension of the analytical ranking function for PTV’s [15] is used (Subsection 4.2). The third alternative is based on the minimization of the impact of the ⊥-component of an EPTV on the ranking (Subsection 4.3).
208
4.1
G. de Tr´e et al.
‘Weighted Sum’ Approach
In order to have an idea of the relative importance of EPTV’s, the ranking function ˜ ∗ ) → [0, 1] rw : ℘(I has been introduced and obtained by considering the 3D-universe of EPTV’s and by associating a ‘weight’ wi , i = 1, 2, . . . , 7 with the relevant angular points t˜1 = {(T, 1)}, t˜2 = {(T, 1), (⊥, 1)}, t˜3 = {(T, 1), (F, 1)}, t˜4 = {(T, 1), (F, 1), (⊥, 1)}, t˜5 = {(⊥, 1)}, t˜6 = {(F, 1), (⊥, 1)} and t˜7 = {(F, 1)} of the cube (the origin is excluded since it does not represent a valid truth value) [14]. For a given EPTV t˜ = {(T, µT ), (F, µF ), (⊥, µ⊥ )}, the ranking value rw (t˜) is defined as the ‘weighted’ sum of the Euclidian distances d(t˜, t˜i ), i = 1, 2, . . . , 7, i.e. rw (t˜) =
7
wi d(t˜, t˜i )
i=1
In order to calculate the ‘weights’ wi , i = 1, 2, . . . , 7 the following ordering is considered: {(T, 1)} > {(T, 1), (⊥, 1)} > {(T, 1), (F, 1)} > {(T, 1), (F, 1), (⊥, 1)} > {(⊥, 1)} > {(F, 1), (⊥, 1)} > {(F, 1)} i.e. t˜1 > t˜2 > t˜3 > t˜4 > t˜5 > t˜6 > t˜7 This ordering can be justified with the initial ordering {(T, 1)} > {(⊥, 1)} > {(F, 1)} The respective weights are then obtained by solving the linear system of equations r(t1 ) = 1 (the ‘best’ truth value), r(t2 ) = 5/6, r(t3 ) = 4/6, r(t4 ) = 3/6, r(t5 ) = 2/6, r(t6 ) = 1/6 and r(t7 ) = 0 (the ‘worst’ truth value), i.e. √ √ √ √ w2 + w3 + 2w4 + 2w5 + 3w6 + 2w7 = 1 √ √ √ w1 + 2w3 + w4 + w5 + 2w6 + 3w7 = 5/6 √ √ √ + 2w2 + w4 + 3w5 + 2w6 + w7 = 4/6 w √ √ √1 2w1 + w2 + w3 + 2w5 + w6 + 2w7 = 3/6 √ √ √ √ 2w1 + w2 + 3w3 + 2w4 + w6 + 2w7 = 2/6 √ √ √ 3w1 + 2w2 + 2w3 + w4 + w5 + w7 = 1/6 √ √ √ √ 2w1 + 3w2 + w3 + 2w4 + 2w5 + w6 = 0 which yields w1 = −0.199, w2 = −0.082, w3 = −0.088, w4 = 0.123, w5 = 0.195, w6 = 0.062 and w7 = 0.434.
Ranking the Possible Alternatives in Flexible Querying
4.2
209
Extended Analytical Approach
The ‘weighted sum’ approach involves a lot of calculations, that can introduce a lot of overhead when comparing a large number of EPTV’s. Therefore, a simpler analytical ranking function1 has been defined by ra : ℘(I ˜ ∗ ) → [0, 1] {(T, µT ), (F, µF ), (⊥, µ⊥ )} →
1 + (µT − µF )(1 −
µ⊥ ) 2
2
With this ranking function, the ordering of the relevant angular points of the cube of EPTV’s becomes {(T, 1)} > {(T, 1), (⊥, 1)} > {(T, 1), (F, 1)} = {(T, 1), (F, 1), (⊥, 1)} = {(⊥, 1)} > {(F, 1), (⊥, 1)} > {(F, 1)} Since no distinction can be made between the EPTV’s {(T, 1), (F, 1)}, {(⊥, 1)} and {(T, 1), (F, 1), (⊥, 1)}, this ranking is more crude than the one obtained by function rw . 4.3
Minimization of the Impact of the ⊥-Component
For both the ‘weighted sum’ and the extended analytical approaches, it has been assumed that the initial ordering of the ‘basic’ truth values is the following {(T, 1)} > {(⊥, 1)} > {(F, 1)} Furthermore, the impact (on the ranking process) of the (F)alse component is considered to be more negative than the impact of the ⊥-component. As an alternative, the impact of the ⊥-component on the ranking process could be minimized. This can be done by ranking the EPTV’s as if there were PTV’s —using ranking the function r (cf. footnote 1 )— and by subsequently using a threshold value α for the ⊥-component. The EPTV’s are then ranked with a decreasing possibility of being true (i.e. µT ) and the additional restriction that their respective possibilities of being inapplicable (i.e. µ⊥ ) must not be greater than α. The resulting ranking function is then defined by ˜ × [0, 1] → [0, 1] rα : ℘(I) µT + (1 − µF ) ({(T, µT ), (F, µF ), (⊥, µ⊥ )}, α) → 2 0 1
if µ⊥ ≤ α, if µ⊥ > α
The ranking function ra is in fact an extension of the ranking function r : ℘(I) ˜ → [0, 1] : {(T, µT ), (F, µF )} → for PTV’s as presented in [15].
µT + (1 − µF ) 2
210
5
G. de Tr´e et al.
Applicability in Flexible Database Querying
The presented ranking functions rw , ra and rα are in fact mappings from the three dimensional space [0, 1]3 onto the unit interval [0, 1]. For each of these functions, there exist subsets of EPTV’s which all have the same ranking value and therefore are indistinguishable. So the presented ranking functions are still not perfect. (As an illustration, the subsets corresponding with the ranking values 0.25, 0.5 and 0.75 of both functions rw (left cube) and ra (right cube) have been presented in Figure 1.)
Fig. 1. Subsets of indistinguishable EPTV’s w.r.t. the ranking functions rw and ra .
Each of the presented ranking functions can be used in flexible database querying. Which function is best suited, depends on the application and is mainly determined by performance requirements and the expectations of the user about the impact of non-applicability of (some of) the query criteria on the query results: – Ranking function rw involves most calculations, but the resulting ranking is less crude than with ranking functions ra and rα . – Ranking function ra is a useful alternative, which can more easily be calculated. – Ranking function rα is preferable in those cases where the existence of nonapplicability of (some of) the query criteria in only allowed to a given extent (α). All EPTV’s {(T, µT ), (F, µF ), (⊥, µ⊥ )} where µ⊥ > α will be ignored in the ranking. Database instances with such an associated EPTV will be omitted from the query result.
6
Conclusions
An adequate method to rank the possible alternatives in the result of a flexible database query is essential, since adequate ranking information simplifies decision making and helps the user to find quickly the requested information. This paper deals with the ranking of EPTV’s. EPTV’s can be used to model query
Ranking the Possible Alternatives in Flexible Querying
211
satisfaction and explicitly allow to model the impact of the non-applicability of some of the querying criteria. Three alternative ranking functions are presented and compared with each other based on their applicability in flexible database querying. The first function rw is obtained as a ‘weighted sum’ of Eucledian distances, the second function ra is an extension of the analytical ranking function for PTV’s, whereas the third function rα uses a threshold value α to minimize the impact of non-applicable querying criteria. Which function to use in flexible querying, depends on the performance requirements and the expectations of the user about the impact of non-applicability of query criteria on the query results.
References 1. Dubois, D., Prade, H.: Certainty and Uncertainty of (Vague) Knowledge and Generalised Dependencies in Fuzzy Databases. In: Proc. of Int. Fuzzy Engineering Symposium ’91, Yokahoma, Japan (1991) 239–249 2. Baldwin, J.F., Zhou, S.Q.: A Fuzzy Relational Inference Language. Fuzzy Sets and Systems 14 (1984) 155–174 3. De Tr´e, G., De Caluwe, R., Van der Cruyssen, B.: A Generalised Object-Oriented Database Model. In: Recent Issues on Fuzzy Databases, Bordogna, G., Pasi, G. (eds.) Physica-Verlag, Heidelberg, Germany (2000) 155–182 4. De Tr´e, G., De Caluwe, R.: Modelling Uncertainty in Multimedia Database Systems: An Extended Possibilistic Approach. Int. Journal of Uncertainty, Fuzziness and Knowledge-Based Systems 11 1 (2003) 5–22 5. Zadeh, L.A.: Fuzzy sets as a basis for a theory of possibility. Fuzzy Sets and Systems 1 1 (1978) 3–28 6. Dubois, D., Prade, H.: Possibility Theory. Plenum Press, New York, USA (1988) 7. Prade, H., Testemale, C.: Generalizing Database Relational Algebra for the Treatment of Incomplete or Uncertain Information and Vague Queries. Information Sciences 34 (1984) 115–143 8. Bosc, P., Pivert, O.: Some Approaches for Relational Databases Flexible Querying. Journal of Intelligent Information Systems (JIIS) 1 3–4 (1992) 323–354 9. Yazici, A., George, R., Aksoy, D.: Design and Implementation Issues in the Fuzzy Object-Oriented Data Model. Information Sciences 108 1–4 (1998) 241–260. 10. Yazici, A., Buckles, B.P., Petry, F.E.: Handling Complex and Uncertain Information in the ExIFO and NF2 Data Models. IEEE Trans. on Fuzzy Systems 7 6 (1999) 659–676. 11. Bordogna, G., Pasi,G., Lucarella, D.: A Fuzzy Object-Oriented Data Model for Managing Vague and Uncertain Information. Int. Journal of Intelligent Systems 14 7 (1999) 623–651 12. Prade, H.: Possibility sets, fuzzy sets and their relation to Lukasiewicz logic. In: Proc. of the 12th Int. Symp. on Multiple-Valued Logic, Paris, France (1982) 223– 227 13. De Tr´e, G.: Extended Possibilistic Truth Values. Int. Journal of Intelligent Systems 17 (2002) 427–446 14. De Tr´e, G., De Caluwe, R. , Verstraete, J., Hallez, A.: Conjunctive Aggregation of Extended Possibilistic Truth Values and Flexible Database Querying. Lecture Notes in Artificial Intelligence 2522 (2002) 344–355 15. Dubois, D., Prade, H.: Degree of Truth and Truth-Functionality. In: Proc. of the 2nd conf. on mathematics at the service of man, Las Palmas, Spain (1982) 262–265
*OREDO,QGH[IRU0XOWL&KDQQHO'DWD'LVVHPLQDWLRQLQ 0RELOH'DWDEDVHV $JXVWLQXV%RUJ\:DOX\R%DOD6ULQLYDVDQDQG'DYLG7DQLDU
6FKRRORI&RPSXWHU6FLHQFHDQG6RIWZDUH(QJLQHHULQJ0RQDVK8QLYHUVLW\$XVWUDOLD ^$JXVWLQXV%RUJ\:DOX\R %DOD6ULQLYDVDQ`#LQIRWHFKPRQDVKHGXDX
6FKRRORI%XVLQHVV6\VWHPV0RQDVK8QLYHUVLW\$XVWUDOLD 'DYLG7DQLDU#LQIRWHFKPRQDVKHGXDX
$EVWUDFW 'DWD EURDGFDVWLQJ VWUDWHJ\ LV NQRZQ DV D VFDODEOH ZD\ WR GLVVHPLQDWHLQIRUPDWLRQWRPRELOHXVHUV+RZHYHUZLWKDYHU\ODUJHQXPEHURI EURDGFDVWLWHPVWKHDFFHVVWLPHRIPRELOHFOLHQWVLQFUHDVHDFFRUGLQJO\GXHWR KLJKZDLWLQJWLPHIRUPRELOHFOLHQWVWRILQGWKHLUGDWDRILQWHUHVW2QHSRVVLEOH VROXWLRQLVWRVSOLWWKHGDWDEDVHLQIRUPDWLRQLQWRVHYHUDOEURDGFDVWFKDQQHOV,Q WKLV SDSHU ZH LQWURGXFH JOREDO LQGH[LQJ WHFKQLTXH IRU PXOWL EURDGFDVW FKDQQHOV$VLPXODWLRQPRGHOLVGHYHORSHGWRILQGRXWWKHSHUIRUPDQFHRIWKH WHFKQLTXH
,QWURGXFWLRQ
1RZDGD\V SHRSOH DUH QR ORQJHU DWWDFKHG WR D VWDWLRQDU\ PDFKLQH WR GR WKHLU ZRUN ZLWK ZLUHOHVV DSSOLFDWLRQ WKH\ DUH HQDEOHG WR FRQGXFW WKHLU EXVLQHVV DQ\ZKHUH DQG DQ\WLPH XVLQJ SRUWDEOH VL]H ZLUHOHVV FRPSXWHU SRZHUHG E\ EDWWHU\ 7KHVH SRUWDEOH FRPSXWHUV FRPPXQLFDWH ZLWK FHQWUDO VWDWLRQDU\ VHUYHU YLD ZLUHOHVV FKDQQHO 7KH WHFKQRORJ\ LV NQRZQ DV PRELOH FRPSXWLQJ > @ 0RELOH FRPSXWLQJ KDV LWV RZQ GDWDEDVH PDQDJHPHQW V\VWHP '%06 WKDW SURYLGHV WKH VDPH IXQFWLRQ DV LQ WUDGLWLRQDO GDWDEDVHV FDOOHG PRELOH GDWDEDVHV 0RELOH GDWDEDVHV IDFH D QXPEHU RI OLPLWDWLRQV SDUWLFXODUO\ SRZHU VWRUDJH DQG EDQGZLGWK FDSDFLW\ ,Q UHJDUG WR SRZHU FDSDFLW\LWKDVEHHQLQYHVWLJDWHGWKDWWKHOLIHH[SHFWDQF\RIDEDWWHU\LVDQWLFLSDWHGWR LQFUHDVH RQO\ LQ HYHU\ \HDUV >@ &RQVHTXHQWO\ WKH QHHG WR XVH SRZHU HIILFLHQWO\DQGHIIHFWLYHO\DUHFUXFLDOLVVXHV 'DWDGLVVHPLQDWLRQVWUDWHJ\RUNQRZQDVEURDGFDVWVWUDWHJ\UHIHUVWRSHULRGLFDOO\ EURDGFDVWGDWDEDVHLWHPVWRFOLHQWVWKURXJKRQHRUPRUHEURDGFDVWFKDQQHOV0RELOH FOLHQWV ILOWHU WKHLU GHVLUHG GDWD RQ WKH IO\ 7KLV VWUDWHJ\ LV DQ HIIHFWLYH ZD\ WR GLVVHPLQDWH GDWDEDVH LQIRUPDWLRQ WR D ODUJH VHW RI PRELOH FOLHQWV +RZHYHU WKH FKDOOHQJHLQEURDGFDVWVWUDWHJ\LVWRPDLQWDLQWKHTXHU\SHUIRUPDQFHRIWKHFOLHQWWR REWDLQLQIRUPDWLRQIURPWKHFKDQQHO ,QJHQHUDOHDFKPRELOHXVHUFRPPXQLFDWHVZLWKD0RELOH%DVH6WDWLRQ0%6 WR FDUU\ RXW DQ\ DFWLYLWLHV VXFK DV WUDQVDFWLRQ DQG LQIRUPDWLRQ UHWULHYDO 0%6 KDV D ZLUHOHVVLQWHUIDFHWRHVWDEOLVKFRPPXQLFDWLRQZLWKPRELOHFOLHQWDQGLWVHUYHVDODUJH QXPEHURIPRELOHXVHUVLQDVSHFLILFUHJLRQFDOOHGFHOO0RELOHXQLWVRUPRELOHFOLHQWV $@ KDYH SURSRVHG FOXVWHULQJ LQGH[ QRQFOXVWHULQJ LQGH[ DQG PXOWLSOH LQGH[ PHWKRGV,QWKLVVFKHPHVRPHIRUPRIGLUHFWRU\LVEURDGFDVWDORQJZLWKWKHGDWDWKH FOLHQWVREWDLQWKHLQGH[GLUHFWRU\IURPWKHEURDGFDVWDQGXVHLWLQVXEVHTXHQWUHDGLQJ 7KH LQIRUPDWLRQ DOVR FRQWDLQV WKH H[DFW WLPH RI WKH GDWD WR EH EURDGFDVW +RZHYHU WKHVH WHFKQLTXHV FRPELQHV LQGH[ DQG GDWD VHJPHQW LQ D VLQJOH FKDQQHO :KHQ WKH LQGH[ DQG GDWD VHJPHQWV VWD\ LQ WKH VDPH FKDQQHO WKH OHQJWK RI EURDGFDVW F\FOH LQFUHDVHV &RQVHTXHQWO\ WKH DYHUDJH DFFHVV WLPH LPSURYHV 7KLV VLWXDWLRQ LV HYHQ ZRUVH ZKHQ FOLHQW PLVVHV WKH DSSURSULDWH LQGH[ DQG KDV WR ZDLW IRU WKH ZKROH EURDGFDVW F\FOH WKDW LQFOXGHV GDWD DQG LQGH[ VHJPHQW EHIRUH UHFHLYLQJ WKH GHVLUHG GDWDLWHP(YHQWKRXJKFOLHQWVFDQVZLWFKLQWRDGR]HPRGHGXULQJWKHSURFHVVEXW WKHUHVSRQVHWLPHRUDFFHVVWLPHEHFRPHVYHU\ODUJH $QRWKHU LQGH[LQJ WHFKQLTXH LV XVHG E\ >@ 7KLV WHFKQLTXH LV EDVHG RQ %WUHH VWUXFWXUHDQGLWLQFRUSRUDWHVDVHSDUDWHLQGH[FKDQQHOWRORFDWHWKHGDWDVHJPHQWLQ PXOWLGDWDFKDQQHO7KHSK\VLFDOSRLQWHUQRWRQO\LQGLFDWHVWKHWLPHYDOXHEXWDOVRWKH FKDQQHO LQ ZKLFK WKH LQGH[HG LWHP ZLOO EH EURDGFDVW 8VLQJ WKLV PHFKDQLVP FOLHQW ILUVWWXQHVLQWRWKHLQGH[FKDQQHOILQGWKHLQGH[HGWLPHYDOXHVZLWFKWRGDWDFKDQQHO DQGZDLWIRUWKHGDWDLWHPWRDUULYH7KHDGYDQWDJHLVPRELOHFOLHQWRQO\QHHGVWRZDLW IRUWKHLQGH[EURDGFDVWF\FOHZKHQLWPLVVHVWKHGHVLUHGLQGH[ZKLFKLVFRQVLGHUDEO\ VKRUW DV FRPSDUHG WR WKH RQH WKDW LQWHUOHDYHV ZLWK GDWD LWHPV +RZHYHU VLQFH LW LQFRUSRUDWHV D VLQJOH LQGH[ FKDQQHO WR EURDGFDVW WKH HQWLUH LQGH[ VWUXFWXUH WKHUH LV VWLOO D FKDQFH WKDW PRELOH FOLHQW KDV WR ZDLW IRU VRPHWLPHV EHIRUH WKH ULJKW LQGH[ DUULYHV
*OREDO,QGH[IRU0XOWL%URDGFDVW&KDQQHOV3URSRVHG0HWKRG 7KLV LQGH[LQJ VWUDWHJ\ LV GHVLJQHG WR PLQLPL]H WKH LQGH[ DFFHVV WLPH RI PXOWL GDWD FKDQQHOV ,W LV DVVXPHG WKDW WKH QXPEHU RI FKDQQHO UHTXLUHG WR EURDGFDVW D FHUWDLQ DPRXQWRIGDWDLWHPVLVNQRZQ>@SURSRVHDVWUDWHJ\XVHGWRVSOLWWKHOHQJWKRIWKH EURDGFDVWF\FOHZKHQWKHQXPEHURIEURDGFDVWLWHPVUHDFKHVDQRSWLPXPSRLQW7KLV SURFHVVFRQWLQXHVXQWLOWKHDFFHVVWLPHLVDERYHWKHRSWLPDOSRLQW6LQFHWKHOHQJWKRI WKHEURDGFDVWF\FOHLVRSWLPDOWKHZDLWLQJWLPHZLOOEHFRQVLGHUDEO\VKRUW *OREDOLQGH[LVGHVLJQHGEDVHGRQ%WUHHVWUXFWXUH,WFRQVLVWVRIQRQOHDIQRGHV DQGOHDIQRGH/HDIQRGHLVWKHERWWRPPRVWLQGH[WKDWFRQVLVWVRIXSWRNNH\VZKHUH HDFKNH\SRLQWWRDFWXDOGDWDLWHPVDQGHDFKQRGHKDVRQHQRGHSRLQWHUWRDULJKWVLGH QHLJKERXULQJOHDIQRGH8QOLNHOHDIQRGHQRQOHDIQRGHPD\FRQVLVWRIXSWRNNH\V DQG N SRLQWHUV WR WKH QRGHV RQ WKH QH[W OHYHO RQ WKH WUHH KLHUDUFK\ LH FKLOG QRGHV $OOFKLOGQRGHVZKLFKDUHRQWKHOHIWKDQGVLGHRIWKHSDUHQWQRGHKDYHWKH
*OREDO,QGH[IRU0XOWL&KDQQHO'DWD'LVVHPLQDWLRQLQ0RELOH'DWDEDVHV
NH\YDOXHVOHVVWKDQRUHTXDOWRWKHNH\RIWKHLUSDUHQWQRGH2QWKHRWKHUKDQGNH\V RIFKLOGQRGHVRQWKHULJKWKDQGVLGHRIWKHSDUHQWQRGHDUHJUHDWHUWKDQWKHNH\RI WKHLUSDUHQWQRGH +DYLQJDOOGDWDSRLQWHUVVWRUHGRQWKHOHDIQRGHVLVFRQVLGHUHGEHWWHUWKDQVWRULQJ GDWDSRLQWHUVLQWKHQRQOHDIQRGHVOLNHWKHRULJLQDO%WUHHV>@0RUHRYHUE\KDYLQJ QRGHSRLQWHUVLQWKHOHDIOHYHOLWEHFRPHVSRVVLEOHWRWUDFHDOOOHDIQRGHVIURPWKHOHIW PRVWWRWKHULJKWPRVWQRGHVSURGXFLQJDVRUWHGOLVWRINH\V:KHQEHLQJEURDGFDVW HDFK SK\VLFDO SRLQWHU WR WKH QHLJKERXULQJ OHDI QRGH DV ZHOO DV DFWXDO GDWD LWHP DUH UHSODFHG E\ D WLPH YDOXH ZKLFK LQGLFDWHV ZKHQ WKH OHDI QRGH RU GDWD LWHP ZLOO EH EURDGFDVW 2XUVLPSOHVFHQDULRLVWREURDGFDVW ZHDWKHU FRQGLWLRQ IRU DOO FLWLHV LQ $XVWUDOLD 7KHUHDUHFLWLHVDOWRJHWKHUWREHEURDGFDVWDQGZHDVVXPHWKHRSWLPXPQXPEHURI FLWLHV LQ D GDWD FKDQQHO LV 6XEVHTXHQWO\ ZH HQG XS ZLWK GDWD FKDQQHOV HDFK FKDQQHOFRQWDLQVFLWLHV,QWKLVFDVHZHXVHDWDEOHFRQVLVWLQJRIUHFRUGVRI,'V FLW\ZHDWKHUFRQGLWLRQDQGWHPSHUDWXUH7KHLQGH[LVLQVHUWHGEDVHGRQWKHRUGHURI WKHGDWDLWHPLQWKHWDEOH6LPLODUO\WRFRQVWUXFWRXUGDWDFKDQQHOVWKHGDWDLWHPLV SODFHGEDVHGRQWKHRUGHURIWKHWDEOHDQGRQFHLWUHDFKHVRSWLPXPQXPEHURIFLWLHV DQHZGDWDFKDQQHOLVFUHDWHG$VVXPHWKDWLQWKHLQGH[WUHHWKHPD[LPXPQXPEHURI QRGHSRLQWHUVIURPDQ\QRQOHDIQRGHLVDQGWKHPD[LPXPQXPEHURIGDWDSRLQWHUV IURPDQ\OHDIQRGHLV *OREDOLQGH[H[KLELWHGLQ)LJXUHXVHVWKH,'DWWULEXWHDVWKHLQGH[SDUWLWLRQLQJ DWWULEXWH ZKLFK LV GLIIHUHQW IURP WKH WDEOH SDUWLWLRQLQJ :H DVVXPH WKH UDQJH SDUWLWLRQLQJUXOHVXVHGDUHWKDWLQGH[FKDQQHOKROGVGDWD,'VEHWZHHQWRLQGH[ FKDQQHOKROGVGDWD,'VEHWZHHQWRDQGWKHUHVWJRWRLQGH[FKDQQHO1RWLFH IURP )LJXUH WKDW WKH ILIWK OHDI QRGH LV UHSOLFDWHG WR FKDQQHO DQG EHFDXVH NH\ EHORQJV WR LQGH[ FKDQQHO ZKLOH NH\V DQG EHORQJ WR LQGH[ FKDQQHO $OVR QRWLFH WKDW VRPH QRQOHDI QRGHV DUH UHSOLFDWHG ZKHUHDV RWKHUV DUH QRW )RU H[DPSOH WKH QRQOHDI QRGH LV QRW UHSOLFDWHG DQG ORFDWHG RQO\ LQ LQGH[ FKDQQHOZKHUHDVQRQOHDIQRGHLVUHSOLFDWHGWRLQGH[FKDQQHODQG,WLVDOVR FOHDUWKDWWKHURRWQRGHLVIXOO\UHSOLFDWHG :HPXVWVWUHVVWKDW WKH ORFDWLRQ RI HDFK OHDI QRGH LV WKH QRW VDPH DV ZKHUH WKH DFWXDO GDWD LV EURDGFDVW 2XU LQGH[ LV EURDGFDVW VHSDUDWHO\ ZLWK WKH GDWD DQG HDFK LQGH[NH\SRLQWVWRWKHUHOHYDQWGDWDFKDQQHO7KXVRQFHWKHULJKWLQGH[LVIRXQGLQD VSHFLILFLQGH[FKDQQHOPRELOHFOLHQWVZLWFKWRWKHULJKWGDWDFKDQQHODQGZDLWIRUWKH GDWDRILQWHUHVWWRDUULYH 7KH GDWD VWUXFWXUH IRU RXU *OREDO LQGH[ HPSOR\V VLQJOH QRGH SRLQWHUV PRGHO ,Q WKHVLQJOHQRGHSRLQWHUPRGHOHDFKQRGHSRLQWHUKDVRQO\RQHRXWJRLQJQRGHSRLQWHU ,IDFKLOGQRGHH[LVWVORFDOO\WKHQRGHSRLQWHUSRLQWVWRWKLVORFDOQRGHRQO\HYHQ ZKHQWKLVFKLOGQRGHDOVRUHSOLFDWHGWRRWKHULQGH[FKDQQHOV)RUH[DPSOHIURPQRGH DWLQGH[FKDQQHOWKHUHLVRQO\RQHQRGHSRLQWHUWRWKHORFDOQRGH7KHFKLOG QRGHDWLQGH[FKDQQHOZLOOQRWUHFHLYHDQLQFRPLQJQRGHSRLQWHUIURPWKHURRW QRGH DW LQGH[ FKDQQHO LQVWHDG LW ZLOO UHFHLYH RQH QRGH SRLQWHU IURP WKH ORFDO URRWQRGHRQO\ ,I D FKLOG QRGH GRHV QRW H[LVW ORFDOO\ WKH QRGH SRLQWHU ZLOO FKRRVH RQH QRGH SRLQWHU SRLQWLQJ WR WKH QHDUHVW FKLOG QRGH LQ FDVH LI PXOWLSOH FKLOG QRGHV H[LVW VRPHZKHUH HOVH )RU H[DPSOH IURP WKH URRW QRGH DW LQGH[ FKDQQHO WKHUH LV
$%:DOX\R%6ULQLYDVDQDQG'7DQLDU
RQO\RQHRXWJRLQJULJKWQRGHSRLQWHUWRFKLOGQRGH DWLQGH[FKDQQHO,QWKLV FDVH ZH DVVXPH WKDW LQGH[ FKDQQHO LV WKH QHDUHVW QHLJKERXU RI LQGH[ FKDQQHO 7KHFKLOGQRGH ZKLFKDOVRH[LVWVDWLQGH[FKDQQHOZLOOQRWUHFHLYHDQRGH SRLQWHUIURPURRWQRGHDWLQGH[FKDQQHO 7DEOH,'&LW\:HDWKHU&RQGLWLRQ7HPSHUDWXUH
)LJ*OREDO,QGH[0RGHO
8VLQJWKLVVLQJOHQRGHSRLQWHUPRGHOLWLVDOZD\VSRVVLEOHWRWUDFHD QRGH IURP DQ\ SDUHQW QRGH )RU H[DPSOH LW LV SRVVLEOH WR WUDFH WR QRGH IURP WKH URRW QRGH DW LQGH[ FKDQQHO DOWKRXJK WKHUH LV QR GLUHFW OLQN IURP URRW QRGH DW LQGH[FKDQQHOWRLWVGLUHFWFKLOGQRGH DWLQGH[FKDQQHO7UDFLQJWRQRGH FDQ VWLOO EH GRQH WKURXJK QRGH DW LQGH[ FKDQQHO $ PRUH IRUPDO SURRI IRU WKH VLQJOH QRGH SRLQWHU PRGHO LV DV IROORZV )LUVW JLYHQ D SDUHQW QRGH LV UHSOLFDWHGZKHQLWVFKLOGQRGHVDUHVFDWWHUHGDWPXOWLSOHORFDWLRQVWKHUHLVDOZD\VD GLUHFWOLQNIURPZKLFKHYHUFRS\RIWKLVSDUHQWQRGHWRDQ\RILWVFKLOGQRGHV6HFRQG XVLQJ WKH VDPH PHWKRGRORJ\ DV WKH ILUVW VWDWHPHQW DERYH JLYHQ D UHSOLFDWHG JUDQGSDUHQW QRGH WKHUH LV DOZD\V D GLUHFW OLQN IURP ZKLFKHYHU FRS\ RI WKLV JUDQGSDUHQW QRGH WR DQ\ RI WKH SDUHQW QRGHV &RQVLGHULQJ WKH ILUVW DQG WKH VHFRQG VWDWHPHQWVDERYHZHFDQFRQFOXGHWKDWWKHUHLVDOZD\VDGLUHFWOLQNIURPZKLFKHYHU FRS\RIWKHJUDQGSDUHQWQRGHWRDQ\RILWVFKLOGQRGHV )LJXUHVKRZVDQH[DPSOHRIDVLQJOHQRGHSRLQWHUPRGHO,WRQO\VKRZVWKHWRS WKUHHOHYHOVRIWKHLQGH[WUHHH[KLELWHGSUHYLRXVO\LQ)LJXUH
*OREDO,QGH[IRU0XOWL&KDQQHO'DWD'LVVHPLQDWLRQLQ0RELOH'DWDEDVHV
,QGH[&KDQQHO
,QGH[&KDQQHO
,QGH[&KDQQHO
)LJ6LQJOH1RGH3RLQWHUV0RGHO
'DWDUHWULHYDOPHFKDQLVPLQWKLVVFKHPHFDQEHGHVFULEHGDVIROORZV 0RELOHFOLHQWWXQHVLQRQHRIWKHLQGH[FKDQQHOLHFDQEHDQ\LQGH[FKDQQHO 0RELOH FOLHQW IROORZ WKH LQGH[ SRLQWHU WR WKH ULJKW LQGH[ NH\ 7KH SRLQWHU PD\ OHDGWRDQRWKHULQGH[FKDQQHOWKDWFRQWDLQVWKHUHOHYDQWLQGH[:KLOHZDLWLQJIRU WKHLQGH[WRDUULYHPRELOHFOLHQWVFDQVZLWFKWRGR]HPRGH 0RELOH FOLHQW WXQHV EDFN RQ DW WKH ULJKW LQGH[ NH\ ZKLFK SRLQW WR WKH GDWD FKDQQHOWKDWFRQWDLQVWKHGHVLUHGGDWDLWHP,WLQGLFDWHVDWLPHYDOXHRIWKHGDWDWR DUULYHLQWKHGDWDFKDQQHO 0RELOHFOLHQWWXQHVLQWRWKHUHOHYDQWGDWDFKDQQHODQGVZLWFKEDFNWRGR]HPRGH ZKLOHZDLWLQJIRUWKHGDWDLWHPWRFRPH 0RELOH FOLHQW VZLWFKHV EDFN WR DFWLYH PRGH MXVW EHIRUH WKH GHVLUHG GDWD LWHP DUULYHVDQGUHWULHYHWKHLQIRUPDWLRQ
3HUIRUPDQFH(YDOXDWLRQ
,Q WKLV VHFWLRQ ZH DQDO\]H WKH SHUIRUPDQFH RI RXU *OREDO LQGH[ PRGHO 7KH VLPXODWLRQ LV FDUULHG RXW XVLQJ D VLPXODWLRQ SDFNDJH 3ODQLPDWH DQLPDWHG SODQQLQJ SODWIRUPV>@$VIRUWKHVLPXODWLRQFDVHZHXVHWKHZHDWKHUVFHQDULRDVLOOXVWUDWHGLQ )LJXUHXVLQJWKHVDPHVHWRIGDWDLWHPV:HLQFRUSRUDWHRXU*OREDOLQGH[LQWKUHH LQGH[FKDQQHOVDQGFRPSDUHWKHDFFHVVWLPHSHUIRUPDQFHZLWKVLQJOHLQGH[FKDQQHO WKDW FRQWDLQV WKH HQWLUH LQGH[ VWUXFWXUH 7KH SHUIRUPDQFH HYDOXDWLRQ LQ WKLV SDSHU UHODWHVWRVLQJOHLQGH[UHWULHYDO7DEOHVKRZVWKHSDUDPHWHUVRIFRQFHUQ 7DEOH3DUDPHWHUVRI&RQFHUQ
DUDPHWHUV 3 XPEHURI,QGH[QRGHLQ*OREDO,QGH[&KDQQHO 1 XPEHURI,QGH[QRGHLQ*OREDO,QGH[&KDQQHO 1 XPEHURI,QGH[QRGHLQ*OREDO,QGH[&KDQQHO 1 1XPEHURI,QGH[QRGHLQ1RQ*OREDO,QGH[ 1RGH3RLQWHU6L]H 'DWD3RLQWHU6L]H ,QGH[HG$WWULEXWH6L]H %DQGZLGWK ,QGH[$UULYDO5DWH
9DOXH E\WHV E\WHV E\WHV
E\WHV SHUVHF
$%:DOX\R%6ULQLYDVDQDQG'7DQLDU
:H UXQ WKH VLPXODWLRQ PRGHO IRU ILIW\ LWHUDWLRQ WLPHV DQG FDOFXODWH WKH DYHUDJH DFFHVVWLPHIRUJLYHQQXPEHURIUHTXHVWDQG *OREDO,QGH[ ,QGLYLGXDO&KDQQHO3HUIRUPDQFH
6LQJOH,QGH[YV*OREDO,QGH[
6LQJOH,QGH[
*OREDO,QGH[
,QGH[$FFHVV7LPHVHF
,QGH[$FFHVV7LPHVHF
1XPEHURI5HTXHVW
D $FFHVV7LPHRI6LQJOH,QGH[YV*OREDO,QGH[
5HWULHYH,QGH[ IURP&KDQQHO
5HWULHYH,QGH[ IURP&KDQQHO
5HWULHYH,QGH[ IURP&KDQQHO
1XPEHURI5HTXHVW
E *OREDO,QGH[±,QGLYLGXDO&KDQQHO3HUIRUPDQFH
)LJ6LQJOH,QGH[&KDQQHOYV*OREDO,QGH[$FFHVV7LPH
$V VKRZQ LQ )LJXUH D ZH FDQ VHH WKDW RXU *OREDO ,QGH[ RXWSHUIRUPV WKH 6LQJOH ,QGH[ ZLWK PRUH RU OHVV WZR WR WKUHH WLPHV ORZHU DYHUDJH DFFHVV WLPH )XUWKHUPRUHDPRQJWKUHHFKDQQHOZLWKLQ*OREDO,QGH[ZHIRXQGIURP)LJXUHE WKDW LQGH[ FKDQQHO SURYLGHV D EHWWHU DYHUDJH DFFHVV WLPH DV FRPSDUHG WR RWKHUV FKDQQHO,WLQGLFDWHVWKDWPRUHUHTXHVWVZLWKLQGH[NH\ORFDWHGLQFKDQQHOWKHEHWWHU DYHUDJH DFFHVV WLPH LV 7KLV LV GXH WR WKH VKRUW LQGH[ EURDGFDVW F\FOH WKDW H[LVWV LQ LQGH[ FKDQQHO VR WKDW PRELOH FOLHQWV GR QRW ZDLW WRR ORQJ WR ILQG WKH ULJKW LQGH[ NH\&RQVHTXHQWO\LWUHGXFHVVXEVWDQWLDODPRXQWRISRZHUFRQVXPSWLRQ
&RQFOXVLRQVDQG)XWXUH:RUN
'DWD GLVVHPLQDWLRQ RU NQRZQ DV GDWD EURDGFDVWLQJ VWUDWHJ\ LV DQ HIIHFWLYH ZD\ WR NHHSXSZLWKQXPEHURIFOLHQWVLQDFHOODQGWKHLUIUHTXHQF\RIUHTXHVWV7RPDLQWDLQ WKHSHUIRUPDQFHRIEURDGFDVWVWUDWHJ\RYHUDODUJHVHWRIGDWDLWHPVZHLQWURGXFHDQ LQGH[LQJ WHFKQLTXH FDOOHG *OREDO LQGH[ WR NHHS WKH DYHUDJH DFFHVV WLPH ORZ 2XU *OREDO LQGH[ PRGHO LQFRUSRUDWHV PXOWLSOH LQGH[ FKDQQHOV DQG WKH GDWD LWHPV DUH EURDGFDVW VHSDUDWHO\ LQ PXOWLSOH GDWD FKDQQHOV 0RELOH FOLHQWV RQO\ QHHG WR WXQH LQ RQHRIWKHLQGH[FKDQQHODQGWKH*OREDOLQGH[JXLGHVWKHFOLHQWWRWKHULJKWQRGHWKDW PD\EHORFDWHGLQGLIIHUHQWLQGH[FKDQQHO :HFRPSDUHWKHSHUIRUPDQFHRIRXU*OREDOLQGH[WR6LQJOHLQGH[FKDQQHO,WLV IRXQG WKDW *OREDO LQGH[ SURYLGHV DERXW WKUHH WLPHV EHWWHU DFFHVV WLPH WKDQ 6LQJOH LQGH[PRGHO&RQVHTXHQWO\LWXWLOL]HVSRZHUPRUHHIILFLHQWO\ )RU IXWXUH ZRUN ZH ZLOO FRQVLGHU PXOWL LQGH[ DWWULEXWHV LQ WKH JOREDO LQGH[ VWUXFWXUH7KXVLWZLOOVXSSRUWPRELOH FOLHQWV WR ILQG WKH GDWD LWHP HIILFLHQWO\ XVLQJ DOWHUQDWLYH VHDUFK NH\ RU FRPELQDWLRQ WKHUHRI 0RUHRYHU ZH SODQ WR H[WHQG WKH JOREDOLQGH[WRLQYROYHMRLQWDWWULEXWHVIURPWZRRUPRUHGLIIHUHQWWDEOHV
*OREDO,QGH[IRU0XOWL&KDQQHO'DWD'LVVHPLQDWLRQLQ0RELOH'DWDEDVHV
5HIHUHQFHV %DUEDUD ' 0RELOH &RPSXWLQJ DQG 'DWDEDVHV$ 6XUYH\ ,((( 7UDQV RQ .QRZOHGJH DQG'DWD(QJLQHHULQJ ± (OPDVUL 5 DQG 1DYDWKH 6 % )XQGDPHQWDOV RI 'DWDEDVH 6\VWHPV 7KLUG (GLWLRQ $GGLVRQ:HVOH\86$ ,PLHOLQVNL 7 DQG 9LVZDQDWKDQ 6 $GDSWLYH :LUHOHVV ,QIRUPDWLRQ 6\VWHPV 3URFRI 6,*'%66SHFLDO,QWHUHVW*URXSLQ'DWDEDVH6\VWHPV &RQIHUHQFH2FWREHU ,PLHOLQVNL 7 9LVZDQDWKDQ 6 DQG %DGULQDWK % 5 'DWD RQ $LU 2UJDQLVDWLRQ DQG $FFHVV,(((7UDQVRQ.QRZOHGJHDQG'DWD(QJLQHHULQJ ± /HRQJ+9DQG6L$'DWD%URDGFDVWLQJ6WUDWHJLHV2YHU0XOWLSOH8QUHOLDEOH:LUHOHVV &KDQQHOV 3URF RI WKH WK ,QWHUQDWLRQDO &RQIHUHQFH RQ ,QIRUPDWLRQ DQG .QRZOHGJH 0DQDJHPHQWSS±'HFHPEHU 6HHOH\'HWDO3ODQLPDWHWP$QLPDWHG3ODQQLQJ3ODWIRUPV,QWHU'\QDPLFV3W\/WG 6KHQJ6&KDQGUDVHNDUDQ$DQG%URGHUVRQ5:$3RUWDEOH0XOWLPHGLD7HUPLQDOIRU 3HUVRQDO&RPPXQLFDWLRQ,(((&RPPXQLFDWLRQVSS±'HFHPEHU 7DQLDU ' 5DKD\X -: ³$ 7D[RQRP\ RI ,QGH[LQJ 6FKHPHV IRU 3DUDOOHO 'DWDEDVH 6\VWHPV´'LVWULEXWHGDQG3DUDOOHO'DWDEDVHV± :DOX\R $% 6ULQLYDVDQ % 7DQLDU ' ³2SWLPDO %URDGFDVW &KDQQHO IRU 'DWD 'LVVHPLQDWLRQ LQ 0RELOH 'DWDEDVH (QYLURQPHQW´ $GYDQFHG 3DUDOOHO 3URFHVVLQJ 7HFKQRORJLHV/1&66SULQJHU9HUODJLQSUHVV
A Robust Scheme for Multilevel Extendible Hashing Sven Helmer, Thomas Neumann, and Guido Moerkotte University of Mannheim, Mannheim, Germany
Abstract. Dynamic hashing, while surpassing other access methods for uniformly distributed data, usually performs badly for non-uniformly distributed data. We propose a robust scheme for multi-level extendible hashing, allowing efficient processing of skewed data as well as uniformly distributed data. In order to test our access method, we implemented it and compared it to several existing hashing schemes. The results of the experimental evaluation demonstrate the superiority of our approach in both index size and performance.
1
Introduction
Skewed data is ubiquitous, and the ways in which it is skewed are typically unpredictable. Thus, while it is possible to choose a well-behaved hash function if the data is known beforehand, this is rarely helpful in practice. We propose a robust extendible hashing scheme that performs well on both uniformly distributed and skewed hash keys. The main idea is dividing the directory of our hash table hierarchically into several subdirectories, such that the subdirectories on lower levels share pages in an elegant way. This allows us to save space without introducing a large overhead or compromising retrieval performance. We show the effectiveness of our approach by presenting results from our extensive experiments. What are the reasons for skewed hash keys? The dominant opinion is that by using appropriate hash functions hashing skewed data will result in a reasonably uniform distribution of the hash keys [1,8]. We do not share this point of view. Obviously good hash functions can be chosen when the data that is to be indexed is known beforehand. In practice, this is very rarely the case. Here, one has to expect bursts of data that are heavily skewed in not always anticipated ways. Furthermore, multiple identical data values also induce skew in hash keys as do specialized hash functions, e.g. those allowing order preserving hashing. One of the unresolved issues with respect to the distribution of the hash keys is the lack of hashing schemes whose performance does not deteriorate when faced with non-uniformly distributed hash keys. This has prevented the large-scale use of indexes based on hashing in practice. Vendors of commercial databases are reluctant to integrate these access methods into their database management systems and rely on the more robust B-trees instead. We try to remedy this situation by proposing a robust hashing index. A. Yazici and C. S ¸ ener (Eds.): ISCIS 2003, LNCS 2869, pp. 220–227, 2003. c Springer-Verlag Berlin Heidelberg 2003
A Robust Scheme for Multilevel Extendible Hashing
221
The paper is organized as follows. In the next section we describe the problems associated with dynamic hashing. Section 3 covers our new approach. In Section 4 we describe the results of our experimental evaluation. Section 5 contains a brief comparison with other approaches. Section 6 concludes our paper.
2
Problems with Extendible Hashing
An extendible hashing index is divided into two parts, a directory and buckets (for details see also [4]). In the buckets we store the full hash keys of and pointers to the indexed data items. We determine the bucket into which a data item is inserted by looking at a prefix hd of d bits of the hash key h. For each possible bit combination of the prefix, we find an entry in the directory pointing to the corresponding bucket. The directory has 2d entries, where d is called global depth (see Figure 1). When a bucket overflows, it is split and all its entries are divided among the two resulting buckets. In order to determine the new home of a data item, the length of the inspected hash key prefix has to be increased until at least two data items have different hash key prefixes. The size of the current prefix d of a bucket is called local depth. If we notice after a split that the local depth d of a bucket is larger than the global depth d, we have to increase the size of the directory. This is done by doubling the directory as often as needed to have a new global depth d equal to the local depth d . For the bucket that was split, the new pointers are put into the directory. For the other buckets, the directory entries are copied. 000
001
010
011
100
101
110
111
d=3
d’=2 h 2=00
d’=3 h 3 =010
d’=3 h 3 =011
d’=1 h 1 =1
Fig. 1. Extendible hashing
Inserting skewed data into an extendible hash table makes the directory grow exponentially, as some buckets split rapidly while other parts of the table stay almost empty. The physical limit of the directory growth is reached quite fast. When the limit is reached, we have two alternatives. We can introduce overflow buckets or use a hierarchical directory. Overflow buckets are contrary to the basic idea of extendible hashing [9], because long chains of overflow buckets lead to severe performance losses. Simply organizing the extendible hash directory hierarchically as a hash tree (as suggested in [4] and [14]) does not get the job done
222
S. Helmer, T. Neumann, and G. Moerkotte
either. Although superior to an ordinary extendible hashing scheme for skewed data, extendible hash trees waste a lot of space for uniformly distributed data. When a bucket overflows, we have to allocate another hash table beneath the current one and insert the elements of the split bucket into this table. Inserting uniformly distributed data leads to “waves of expansions”: many buckets split at roughly the same time, so many new tables are created at once. However, most of these tables hold only a few overflow records, which results in a waste of space.
3
Our Approach
Due to space constraints, we can only give a brief description, for details see [6]. We propose a multi-level extendible hash tree in which hash tables share pages according to a buddy scheme. In this buddy scheme, z-buddies are hash tables that reside on the same page and their stored hash keys share a prefix of z bits. Consequently, buddy hash tables have the same global depth, namely z. Let us illustrate our index with an example. We assume that a page can hold 2n entries of a hash table directory. Furthermore, we assume that the top level hash table directory (also called the root) is already filled, contains 2n different entries at the moment, and that another overflow occurs (w.l.o.g. in the first bucket). In this case, we allocate a new hash table of global depth 1 (beneath the root) to distinguish the elements in the former bucket according to their n + 1th bit. However, we do this not only for the overflowing bucket, but also for all 1-buddies of this bucket. The hash tables for the buddies are created in anticipation of further splits. All of these hash tables can be allocated on a single page, resulting in the structure shown in Figure 2. (In a simple hash tree, we would have just one newly allocated hash table with two entries. The rest of the table directory would be empty.) If another overflow occurs in one of the hash tables on level 2, causing its growth, we increase the global depth of all hash tables on this page by 1, doubling their directory sizes. We now need two pages to store these tables, so we split the original page and copy the content to two new pages. Adjusting the pointers in the parent directory is our next task. The left half of the pointers referencing the original page now point to one of the new pages, the right half to the other new page (see Figure 3). The space utilization of our index can be improved by eliminating pages with unnecessary hash tables. The page on the right-hand side of the second level in Figure 3 is superfluous, as the entries in the directories of all hash tables point to a single bucket, i.e. all buckets have local depth 0. In this case, the page is discarded and all buckets are connected directly to the hash table on the next higher level. Due to our buddy scheme, we have a very regular structure we can exploit. Indeed, we can compute the global depths of all hash tables (except the root) by looking at the pointers in the corresponding parent table. Finding 2n−i identical pointers there means that the referenced page contains 2n−i i-buddies of global
A Robust Scheme for Multilevel Extendible Hashing 0000
0001
0010
0011
0100
0101
0110
0111
1000
1001
1010
1011
1100
1101
1110
1111
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
1101
223
Fig. 2. Overflow in our multi-level hash tree
00
01
10
11
00
01
10
11
00
0000
0001
0010
0011
01
11
01
11
10
00
10
0100
00
0101
0110
0111
1000
1001
1010
1011
1100
01
11
01
11
01
11
01
11
10
00
10
00
10
00
10
1110
1111
Fig. 3. Overflow on the second level
depth i. Consequently, we can utilize the whole page for storing pointers, as no additional information has to be kept. 3.1
Lookups
Lookups are easily implemented. We have to traverse inner nodes until we reach a bucket. On each level we determine the currently relevant part of the hash key. This gives us the correct slot in the current hash table. As more than one hash table can reside on a page, we may have to add an offset to access the right hash table. Due to the regular structure, this offset can be easily calculated. We just multiply the last n − i bits of the relevant pointer in the parent table by the size of a hash table on the shared page. If n − i = 0, we do not need an offset, as only one hash table resides on this page. If we reach a bucket, we search there for the data item. If the bucket does not exist (no data item is present there at
224
S. Helmer, T. Neumann, and G. Moerkotte
the moment), we hit a NULL-pointer and can abort the search. We use the most significant bit in pointers to tell apart pointers referencing buckets from those referencing directory pages. 3.2
Insertions
After finding the bucket where the new data item has to be inserted (using the lookup procedure), we have to distinguish several cases. We concentrate on the most difficult case, where an overflow of the bucket occurs and the global depth of the hash table on the current level increases. The other cases can be handled in a straightforward manner. If the hash table has already reached its maximal global depth (i.e. it resides alone on a page), we add a new level with 2n−1 hash tables of global depth 1 to the existing index structure (comparable to Figure 2). If we have not reached the maximal global depth yet (the hash table shares a page with its buddies), the global depth of all hash tables on this page is increased by 1. The hash tables on the first half of the page remain there. The hash tables on the second half of the page are moved to a newly allocated page. Then the pointers in the parent hash table are modified to reflect the changes. We optimize the space utilization at this point if we discover that the buckets of all hash tables in one of the former halves have a local depth of one (or are not present yet). In this case (compare the node in the lower right corner of Figure 3) we do not need this node yet and connect the buckets directly to the parent hash table.
4
Experimental Evaluation
We used different sets of data to compare our index structure to extendible hashing and a simple extendible hashing tree. First, we generated synthetic hash keys with varying degrees of skew by creating bit vectors with a length of 64 bits. The state of each bit (0 or 1) was determined randomly with different distributions (uniform, 40% probability and 30% probability that the bit is set to 1). Second, we indexed a list of URLs generated by a Web crawler. The strings of the URLs were hashed using a shift-XOR scheme. 4.1
Comparing the Size of the Directories
In Figure 4 we plotted the growing directory sizes depending on the number of inserted hash keys. (Note that we used double-logarithmic scales.) While the extendible hashing scheme (Figure 4(a)) performs poorly for skewed data (we limited the directory size to 4096 pages, reverting to overflow buckets), the simple hash tree (Figure 4(b)) has severe problems with uniformly distributed data. The smallest directories are found in our multi-level hashing scheme (Figure 4(c)). Neither skewed nor uniformly distributed data pose any problems. When comparing the results for real data in Figure 4(d) with those for synthetic data, we clearly see that real data is far from uniformly distributed. (Note
A Robust Scheme for Multilevel Extendible Hashing
225
Fig. 4. Size of directory in pages for varying data skew
that the depth of the hash tables is 11, as we increased the pointer size from 32 to 64 bits.) The directory of extendible hashing reaches the limit (set to 8192 pages this time) almost instantaneously. Indexing this kind of data with extendible hashing is out of the question. The extendible hash tree starts out strong, but deteriorates as more hash keys are inserted. 4.2
Comparing the Retrieval Costs
Table 1 shows the retrieval performance of the different hashing schemes. The left column for each index structure contains the average running time in milliseconds for a lookup in a completely filled index (20,000,000 inserted hash keys). In each right column, the average number of page accesses is displayed. We determined the numbers by averaging 20,000,000 random accesses to the hash tables. Several observations are worth mentioning. The increase in page accesses for extendible hashing for heavily skewed data is due to the increased use of overflow buckets in this case. As fewer pages of the index can be held in main memory with increasing skewedness of the hash keys, we have more page faults during access, which results in a higher running time. Although our hashing scheme has the highest number of page accesses, its performance measured in run time is, on average, still better than that of the other schemes. This emphasizes the importance of a small directory, as it directly influences the retrieval performance.
226
S. Helmer, T. Neumann, and G. Moerkotte Table 1. Average retrieval performance per hash key
skew uniform 40:60 30:70 URLs
ext. hash hash tree ml hash tree time page acc. time page acc. time page acc. (in msec) (in pages) (in msec) (in pages) (in msec) (in pages) 5.91 2.00 6.74 3.00 6.01 3.00 5.91 2.00 8.04 3.00 5.58 3.00 9.01 2.10 6.51 3.01 5.86 3.24 12.37 2.38 6.95 3.41 6.08 3.41
The performance of the access methods for real data is summarized in the last row of Table 1. Overall, the retrieval costs for real data are very similar to those for skewed synthetic hash keys. We also measured the performance of insertion operations (which are omitted due to space constraints). Overall, the insertion costs for our scheme were not worse than those for the other indexes.
5
Further Comparisons with Other Approaches
Since the initial work by Larson [7], there have been many proposals for improving dynamic hashing (for an overview see [3]). Most of these schemes, e.g. [8,12, 13], assume that the hash values are distributed in a reasonable way, i.e. close to uniform distribution, so that any skew in the original keys is eliminated for the most part by an adequate hash function. Theoretical analyses [5,10] also have not considered heavily skewed hash keys. The papers published on non-uniform distributions of hash keys are few and far between. Otoo proposes a balanced extendible hash tree in [11], which grows from the leaves to the root, similar to a B-tree. Although this solves the problem of wasted space, the lookup costs increase. In a balanced extendible hash tree all lookups will cost as much as the most expensive lookup. For example, if we can store tables up to a depth of 10 on a page and we need 30 bits to distinguish the most skewed data items, the tree will have (at least) three levels. Due to the balancing, all queries have to traverse them. In our scheme, only the most frequent hash keys are found on the lowest level of the tree. Finally, Du and Tong suggest a multi-level extendible hash tree in [2], which is also based on page sharing. They try to reduce the space expenditure further by compressing the entries. This, however, backfires as their scheme is very complicated, due to the fact that compression destroys the regular structure of extendible hashing. Consequently, more organizational data has to be stored, which cancels out the effect of compression. We give a more detailed comparison with the approaches by Otoo and Du/Tong in [6]
A Robust Scheme for Multilevel Extendible Hashing
6
227
Conclusion and Outlook
We developed an extendible hashing index able to efficiently handle skewed data. We implemented the hashing scheme and tested it thoroughly, comparing it to several other existing access methods. Our approach proved to be superior in view of directory size as well as retrieval performance, especially for real data. Moreover, our index also performs excellently for uniformly distributed data, making it a very robust scheme. The size of our directory does not grow exponentially. Consequently, our index is also faster, because a larger fraction of the directory can be held in main memory. The results obtained from our experiments encourage us to extend the scheme to allow processing of multi-attribute queries and partial-match retrieval. Another important subject is the creation time of the index. The current version of creating the index is not efficient enough. So a next step would be to devise efficient bulk-loading procedures to reduce the creation time significantly.
References 1. J.L. Carter and M. Wegman. Universal classes of hash functions. J. Comput. Syst. Sci., 18(2):143–154, 1979. 2. D.H.C. Du and S.-R. Tong. Multilevel extendible hashing: A file structure for very large databases. IEEE Trans. on Knowledge and Data Engineering, 3(3):357–370, 1991. 3. R.J. Enbody and H.C. Du. Dynamic hashing schemes. ACM Computing Surveys, 20(2):85–113, June 1988. 4. R. Fagin, J. Nievergelt, N. Pippenger, and H.R. Strong. Extendible hashing – a fast access method for dynamic files. ACM Transactions on Database Systems, 4(3):315–344, September 1979. 5. P. Flajolet. On the performance evaluation of extendible hashing and trie searching. Acta Informatica, 20(4):345–369, 1983. 6. S. Helmer, T. Neumann, and G. Moerkotte. A robust scheme for multilevel extendible hashing. Technical Report 19/01, Universit¨ at Mannheim, 2001. http://pi3.informatik.uni-mannheim.de. 7. P.A. Larson. Dynamic hashing. BIT, 18:184–201, 1978. 8. D.B. Lomet. Bounded index exponential hashing. ACM Transactions on Database Systems, 8(1):136–165, March 1983. 9. Y. Manolopoulos, Y.Theodoridis, and V.J. Tsotras. Advanced Database Indexing. Kluwer Academic Publishers, Dordrecht, 1999. 10. H. Mendelson. Analysis of extendible hashing. IEEE Trans. Software Eng., 8(6):611–619, November 1982. 11. E.J. Otoo. Linearizing the directory growth in order preserving extendible hashing. In Int. Conf. on Data Engineering, pages 580–588, 1988. 12. M.V. Ramakrishna and P.A. Larson. File organization using composite perfect hashing. ACM Transactions on Database Systems, 14(2):231–263, June 1989. 13. K. Ramamohanarao and J.W. Lloyd. Dynamic hashing schemes. The Computer Journal, 25(4):478–485, 1982. 14. M. Tamminen. Order preserving extendible hashing and bucket tries. BIT, 21:419– 435, 1981.
A Cooperative Paradigm for Fighting Information Overload 'DQLHO*D\R$YHOOR'DUtRÈOYDUH]*XWLpUUH]DQG-RVp*D\R$YHOOR Department of Informatics, University of Oviedo, Calvo Sotelo s/n 33007 Oviedo (SPAIN) ^GDQLGDULRD`#OVLXQLRYLHV
Abstract. The Web is mainly processed by humans. The role of the machines is just to transmit and display the contents of the documents, barely being able to do something else. Nowadays there are lots of initiatives trying to change this situation; many of them are related to fields like the Semantic Web [1] or Web Intelligence. In this paper we describe the Cooperative Web [2] that can be seen as a new proposal towards Web Intelligence. The Cooperative Web would allow us to extract semantics from the Web in an automatic way, without the need of ontological artifacts, with language independence and, besides of this, allowing the usage of browsing experience from individual users to serve the whole community of users.
1
Introduction
Although the Web provides access to a huge amount of information it is not a perfect information retrieval mechanism. Search engines perform a really useful task but we can say that they are toping out since they provide a view of the Web quite poor to get a more powerful use. This claim can seem exaggerated but if we take into account two latest initiatives from the main search engine –Google Answers1 and the First Annual Google Programming Contest2– it is clear that, implicitly, Google admits that state of the art techniques have reached their limit and “something more” is needed. However, and for the moment, users continue using search engines that provide only a lexical view of the Web and force them to browse hundreds of documents for an increasing time until they find the piece of information they are looking for. Besides of this, the current Web shows a problem as serious as its lack of semantics: each time a user browses the Web, he opens a path which could be useful for others and, in the same way, other users can have yet followed such path and have found its worth or its uselessness. However, all that experimental knowledge is lost. Such situation is flawed and something is required to provide intelligence and semantics to the Web. We think that this is possible in an automatic way, transparent to the user, and language independent by using software agents and computational biology algorithms. Through this paper we will show in which way we think this task could be accomplished. 1 2
http://answers.google.com/answers/faq.html#whatis http://www.google.com/programming-contest/
$ 1 are constants (belief coefficients). The belief coefficients ensure that documents containing more keywords from the query will be considered to be more relevant than documents containing less keywords even though these terms appear more frequently. The proposed new functions W1 and W2 are simple and do not depend on document requency. The purpose is that the weight (the importance) of a term in a given document should depend on its absolute frequency of occurrence but at the same time the terms with high frequency of occurrence should not drag the weights of other terms with less frequencies way down the scale. This is achieved by using the combination of ln function and belief coefficients that restricts the weights to a maximum values of (c1 , c1 + 1] for W1 and [c2 − 1, c2 ) for W2. The main advantages of the proposed weighting functions W1 and W2 are that they are suitable for use in dynamic document databases. Adding or deleting documents from a collection does not affect the terms’ weights because the functions do not involve idf. W1 is similar to the function used for the calculation of the term frequency factor in PIVOT [7]. However, W1 can be used without idf . The second proposed weighting function W2 is simpler than W1 and does not depend on the term frequency of other terms belonging to the document.
Comparison of New Simple Weighting Functions for Web Documents
239
From all our experiments for SMART, INQUERY, PIVOT, W1 and W2, we show in the next section how the proposed weighting functions achieve better results when querying documents.
3
Experiments and Results
In our experiments, we use the 10 gigabyte collection of documents (WT10g) [8] available for participants of Web track main task in the TREC-9 [9] experiments. This collection is a subset of the 100 gigabyte VLC2 collection created by Internet Archive in 1997 [10]. Queries are created using only the title part of Web track topics 451-500. The full text of all the documents was indexed. Stop word’s lists of 595 English, 172 German and 352 Spanish words were used. Porter’s stemming algorithm [11] was applied to both documents and queries. Average index size per document was 132.54 words. All terms in the query were equally weighted by one. Inner product is used as the similarity function in the experiments. Many variations of PIVOT, SMART and INQUERY weighting functions have been used with different document collections by other researchers. In our experiments we used the functions shown below to calculate the term frequency factor for a term t within a document D. From each of the three families of weighting functions, the following gave the best results for the WT10g collection of documents: (P IV OT )
0.4 + 0.6
1 + ln(tf ) 1 + ln(tfmax )
(SM ART )
0.5 + 0.5
tf tfmax
(IN QU ERY )
0.4 + 0.6
tf tfmax
Table 1. Summary statistics W1 W2 PIVOT SMART INQUERY Retrieved 50000 50000 50000 50000 50000 Relevant ret. 1260 1223 1268 1220 1137 Relevant 2617 Num. of queries 50
The results obtained for W1, W2, PIVOT, SMART and INQUERY weighting functions are given in Tables 1, 2 and 3. The explanation of the results in the tables are as follows:
240
B. Hyusein, A. Patel, and F. Zyulkyarov Table 2. Recall level precision averages Recall
Precision W1 W2 PIVOT SMART INQUERY 0.00 0.4260 0.4593 0.4260 0.4086 0.3696 0.10 0.3199 0.3068 0.3199 0.2893 0.2460 0.20 0.2593 0.2644 0.2592 0.2298 0.1897 0.30 0.2323 0.2234 0.2325 0.2004 0.1626 0.40 0.1837 0.1859 0.1830 0.1542 0.1289 0.50 0.1647 0.1705 0.1640 0.1363 0.1080 0.60 0.1236 0.1321 0.1228 0.0950 0.0846 0.70 0.1033 0.0970 0.1036 0.0830 0.0562 0.80 0.0664 0.0769 0.0675 0.0509 0.0288 0.90 0.0591 0.0689 0.0588 0.0413 0.0240 1.00 0.0362 0.0473 0.0360 0.0237 0.0175 Average precision over all relevant documents W1 W2 PIVOT SMART INQUERY non-interpolated 0.1592 0.1673 0.1591 0.1368 0.1126
– Summary statistics: • Retrieved - number of documents retrieved. • Relevant ret. - total number of relevant documents returned for all queries. • Relevant - total possible relevant documents within a given task. • Num. of queries - the number of queries used in the search runs. – Recall Level Precision Averages: • Precision at eleven standard recall levels. The precision averages at eleven standard recall levels are used to compare the performance of the weighting functions and as the input for plotting the recall-precision graph (see Figure 1). • Average precision over all relevant documents, non-interpolated. This is a single-valued measure that reflects the performance over all relevant documents. The measure is not an average of the precision at standard recall levels. Rather, it is the average of the precision value obtained after each relevant document is retrieved. – Document Level Averages: • Precision at nine document cut-off values. Each document precision average is computed by summing the precisions at the specified document cut-off value and dividing by the number of queries (50). • R-Precision - precision after R documents have been retrieved, where R is the number of relevant documents for the queries. The average RPrecision is computed by taking the mean of the R-Precisions of the individual queries. The results are evaluated using the trec eval package written by Chris Buckley of Sabir Research (available at ftp://ftp.cs.cornell.edu/pub/smart/).
Comparison of New Simple Weighting Functions for Web Documents
241
Table 3. Document level averages
At At At At At At At At At
Exact:
4
Precision W2 PIVOT SMART INQUERY 0.2480 0.2360 0.2000 0.1840 0.2000 0.2060 0.1800 0.1720 0.1640 0.1840 0.1587 0.1520 0.1490 0.1660 0.1480 0.1390 0.1387 0.1540 0.1367 0.1300 0.0912 0.1030 0.0928 0.0802 0.0707 0.0740 0.0691 0.0578 0.0400 0.0418 0.0398 0.0366 0.0245 0.0254 0.0244 0.0227 R-Precision W1 W2 PIVOT SMART INQUERY 0.1909 0.1895 0.1934 0.1678 0.1380
W1 5 0.2400 10 0.2060 15 0.1813 20 0.1660 30 0.1527 100 0.1034 200 0.0739 500 0.0419 1000 0.0252
Discussion of Results
To soften the influence of the terms with a high frequency of occurrence, the natural logarithm function was used in W1 and W2 weighting functions. Independence of document frequency ensures that adding or deleting documents from a collection does not affect the term’s weights. Varying the weights of the indexed terms in the range of (c1 , c1 +1] for W1 and in [c2 −1, c2 ) for W2 and using inner product similarity function, the documents containing more keywords from the query are considered to be more relevant than documents containing less keywords even if these terms appears more frequently. We also found that the W1 and W2 proposed weighting functions are not sensitive to the belief coefficients c1 and c2 . We achieved approximately the same precision and recall varying c1 in the interval of (0, 0.9] and c2 in the interval of (1, 2.5]. The best results are achieved using constant c1 = 0.9 and c2 = 2.5. We obtained 1260 relevant documents with W1 (48.14%), 1223 with W2 (46.73%), 1268 with PIVOT (48.45%), 1220 with SMART (46.61%) and 1137 with INQUERY (43.44%) weighting function of the total possible 2617 relevant documents (Table 1). The recall-precision graph in Figure 1 compares the results obtained by newly proposed weighting functions W1 and W2 against the results obtained for PIVOT, SMART and INQUERY weighting functions. This graph is created using the eleven recall levels from the recall level precision averages (Table 2). Figure 1 also shows that W1 and W2 give significantly better precision than SMART and INQUERY weighting functions in all of the eleven standard recall levels. For most of the recall levels W1 and W2 give better precision than PIVOT as well. In two recall levels W1 has the same precision as PIVOT and in six recall levels W1 is superior. W2 is superior in eight of possible eleven recall levels. W1 and W2 weighting functions also give better results than PIVOT, SMART and INQUERY with respect to average precision over all relevant doc-
242
B. Hyusein, A. Patel, and F. Zyulkyarov
Fig. 1. Recall-precision graphs
uments (Table 2), significantly better R-Precision and precision at the nine document cut-off values than SMART and INQUERY (Table 3). Because of the greater number of retrieved documents for PIVOT it gives better R-Precision and better precision in the most of the document cut-off values.
5
Conclusions and Future Work
In this paper we proposed two new simple and effective weighting functions for Web document retrieval. The weighting schemes are tested and compared with results obtained for the PIVOT, SMART and INQUERY methods on the WT10g collection of documents. Experiments showed that our weighting functions perform better than SMART, INQUERY and PIVOT weighting functions with respect to average precision and recall. The proposed weighting functions are computationally fast and suitable for use in SEs. Weighting function W2 is currently in use in the ADSA search engine because it gives better precision than W1 (which was used in the previous version of the ADSA search engine). From the experimental results, our current research work is concentrating on the development of a more complex indexer which will allow any attribute of Web documents to be indexed. This will permit us to investigate in detail how the different parts of the documents contribute to the average precision in the process of search in a much more comprehensive manner.
Acknowledgement. The research was funded by Enterprise Ireland as part of the Enterprise Ireland Informatics Research Initiative.
Comparison of New Simple Weighting Functions for Web Documents
243
References 1. G. Salton and C. Buckley. Term Weighting Approaches in Automatic Text Retrieval. Information Processing and Management, 24(5):513–523, 1988. 2. R. Khoussainov, T. O’Meara, and A. Patel. Independent Proprietorship and Competition in Distributed Web Search Architectures. In Proceeding of the Seventh IEEE International Conference on Engineering of Complex Computer Systems (ICECCS 2001), pages 191–199. IEEE Computer Society Press, 2001. 3. G. Salton and M. McGill. Introduction to Modern Information Retrieval. McGrawHill, New York, NY, 1983. 4. C. Buckley and J. Walz. SabIR Research at TREC 9. In Proceeding of the 9th Text REtrieval Conference (TREC-9), pages 475–477. The National Institute of Standards and Technology, 2000. 5. R. Larson. Term Weighting in Smart, October 1998 Available from: http://www.sims.berkeley.edu/courses/is202/f98/Lecture18/sld021.htm [Accessed July 14th, 2003]. 6. J. Broglio, J. P. Callan, W. B. Croft, and D. W. Nachbar. Document Retrieval and Routing Using the Inquery System. In Proceeding of the Third Text REtrieval Conference (TREC-3), pages 29–38. The National Institute of Standards and Technology, 1995. 7. A. Singhal, C. Buckley, and M. Mitra. Pivoted Document Length Normalization. In Hans-Peter Frei, Donna Harman, Peter Sch¨ auble, and Ross Wilkinson, editors, Proceedings of the Nineteenth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 21–29, New York, 1996. ACM Press. 8. P. Bailey, N. Craswell, and D. Hawking. Engineering a Multi-Purpose Test Collection for Web Retrieval Experiments. Information Processing and Management, 2002. 9. D. Hawking, CSIRO Mathematical, and Information Sciences. Overview of the TREC-9 Web Track. In Proceeding of the 9th Text REtrieval Conference (TREC9), pages 87–102. The National Institute of Standards and Technology, 2000. 10. Internet Archive: Building an Internet Library, http://www.archive.org. 11. M. F. Porter. An algorithm for suffix stripping. Program, 14(3):130–137, July 1980.
T³r··vÿtvÿ6ttyóvÿh³v½rGhÿtÃhtr²)6Q¼¸ihivyv²³vp T³r··r¼s¸¼Uüxv²u 7 Uhür¼9vüor¼ hüq7huh¼Fh¼h¸÷yhü @tr ÿüv½r¼²v³r²v Vyòyh¼h¼h²Õ7vytv²h'h¼ @ü²³v³² "$7¸¼ü¸½h øª·v¼ U¼xv'r ^GWDQHUEDKDU`#XEHHJHHGXWU
6i²³¼hp³ Dü ³uv² ¦h¦r¼ Zrvü³¼¸qÃprhürZ yr`vp¸üs¼rr¦¼¸ihivyv²³vp²³r··r¼ ³¸ ir òrq vü hqr½ry¸¦vüt Uüxv²u Düs¸¼·h³v¸ü Sr³¼vr½hy ²'²³r·D³ uh² h yvürh¼ p¸·¦Ã³h³v¸ühy p¸·¦yr`v³' hüq v³² ³r²³ ²Ãppr²² ¼h³v¸ v² ($© Uur ·hvü p¸ü³¼vióv¸ü ¸s ³uv² ¦h¦r¼ v² ³¸ tv½r h ³u¸¼¸Ãtu qr²p¼v¦³v¸ü ¸s h ¦¼¸ihivyv²³vp ¦r¼²¦rp³v½r s¸¼ ²³r··vüt Zuvpu phü hy²¸ ir trür¼hyvªrq ³¸ h¦¦y' ³¸ ¸³ur¼ httyóvüh³v½r yhütÃhtr² yvxrAvüüv²uCÃüth¼vhü@²³¸üvhühüq8ªrpu
Dÿ³¼¸qÃp³v¸ÿ
Dü hühy'³vp yhütÃhtr² yvxr @ütyv²u ²³r··vüt v² ¼ryh³v½ry' Ãü²¸¦uv²³vph³rq irphòr ·¸¼¦u¸y¸tvphy ½h¼vh³v¸ü² ¸s Z¸¼q² s¸¼·² h¼r yv·v³rq Pü ³ur ¸³ur¼ uhüq vü ³ur httyóvüh³v½r yhütÃhtr² yvxr Uüxv²u ²³r··vüt v² ²³vyy h uh¼q ¦¼¸iyr· ²vüpr ³ur' uh½r ³urph¦hpv³' ³¸trür¼h³r³ur¸¼r³vphyy' hüvüsvüv³rü÷ir¼ ¸s¦¸²²viyrZ¸¼qs¸¼·² b !d Uur ·¸¼¦u¸y¸t' hühy'ªr¼² h¼r ³ur ¸üy' ¦¼rpv²r ³¸¸y² h² ²³r··r¼² s¸¼ Düs¸¼·h³v¸üSr³¼vr½hyDSö²'²³r·²vühttyóvüh³v½ryhütÃhtr²b " hüq #d 7óvü ³uv² Zh'²³r··vüts¸¼ DST'²³r·²vühttyóvüh³v½ryhütÃhtr²¼r²Ãy³²vü h ¦h¼hq¸`³uh³ phü ir qrsvürq i' h p¸üsyvp³rq ¦hv¼ ¸s ¼r¹Ãv¼r·rü³²) y¸Z ²³¸¼htr hüq vüp¼rh²rq ¦r¼s¸¼·hüprT³r··vütv²qr·hüqrqs¼¸· DS²'²³r·²s¸¼ httyóvüh³v½ryhütÃhtr²³¸ ¸½r¼p¸·r ²³¸¼htr p¸·¦yr`v³' hüq ³¸ rüuhüpr ¦¼rpv²v¸ü¼rphyy ·r³¼vp² b$ %d Dü shp³ DS ²'²³r·²¶¸½r¼hyy¦r¼s¸¼·hüpr qrp¼rh²r irphòr ¸s h uvtu yr½ry¸s p¸·¦Ã³h³v¸ühy p¸·¦yr`v³' ³uh³ v² ¼hv²rq s¼¸· òvüt ·¸¼¦u¸y¸t' hühy'ªr¼² h² ²³r··r¼² b&d Uur ¦¼¸ihivyv²³vp²³r··vüt³uh³Zr¦¼¸¦¸²rqur¼rphüiròrq³¸¸½r¼p¸·r³uv²puhyyrütr rssrp³v½ry' Dü ³uv² ¦h¦r¼ Zr ¦¼r²rü³ h ¦¼¸ihivyv²³vp ²³r··vüt ·¸qry ih²rq ¸ü ³ur ¦¼¸¦¸²rq²³h³v²³vphys¼h·rZ¸¼x s¸¼ Uüxv²uDS²'²³r·² 6qqv³v¸ühyy' Zr hy²¸½hyvqh³r ³ur¦¼¸¦¸²rq ·¸qryi' ³urhühy'²r²¸s³ur¼r²Ãy³²¸s hür`¦r¼v·rü³Uur¼r²Ãy³²¼r½rhy ³uh³³urürZ²³r··r¼¸½r¼p¸·r²³ur·rü³v¸ürq ¸i²³hpyr²i' uh½vüt h p¸·¦Ã³h³v¸ühy p¸·¦yr`v³' ¸s Püö vr yvürh¼ö hüq h p¸·¦¼r²²v¸ü yr½ry ¸s %( vr ²³¸¼htr p¸·¦yr`v³'ö
Dü vüs¸¼·h³v¸ü Sr³¼vr½hy ¦r¼s¸¼·hüpr v² ·rh²Ã¼rq i' Q¼rpv²v¸ü hüq Srphyy b!d Dü shp³ ¼r²¦¸ü²r ³v·r ¸s ³ur ¸½r¼hyy ²'²³r· v² hy²¸ h ¦r¼s¸¼·hüpr ·rh²Ã¼r ió vü ³uv² ²³Ãq' v³ v² ¼rsr¼¼rq i'p¸·¦Ã³h³v¸ühy p¸·¦yr`v³'Ãüyr²² ¸³ur¼Zv²r ·rü³v¸ürq
6Áhªvpvhüq8ùrür¼@q²ö)DT8DT!"GI8T!©%(¦¦!##±!$!" T¦¼vütr¼Wr¼yht7r¼yvüCrvqryir¼t!"
T³r··vüt vü6ttyóvüh³v½rGhütÃhtr²)6 Q¼¸ihivyv²³vp T³r··r¼s¸¼Uüxv²u!#$
Uur¦h¦r¼v² ¸¼thüvªrqh²s¸yy¸Z²)Dü²rp³v¸ü!Zr tv½ri¼vrsqrsvüv³v¸ü²hüq pyh¼vs' ³urürrq s¸¼ ²³r··vütvü DS²'²³r·²s¸¼httyóvüh³v½ryhütÃhtr²Dü ²rp³v¸ü" Zr ¸½r¼½vrZ ¼ryh³rq ¦¼r½v¸Ã² Z¸¼x vü hqqv³v¸ü Zr ¦¼r²rü³ ih²vp ü¸³h³v¸ü² hüq Zr r`¦yhvü³ur²³h³v²³vphys¼h·rZ¸¼x hüq³ur ¦¼¸ihivyv²³vp²³r··vüt·¸qry DüTrp³v¸ü# Zr h²²r¼³ ³ur ¦¼¸ihivyv²³vp ²³r··vüt ·¸qry hüq ³ur ·r³u¸q Dü ²rp³v¸ü $ Zr ²Ã··h¼vªr³urhühy'²r²hüq ¼r²Ãy³²¸s¸Ã¼r`¦r¼v·rü³Uurp¸üpyòv¸ü² h¼r tv½rü vü ²rp³v¸ü%
!
T³r··vÿthÿqDÿs¸¼·h³v¸ÿSr³¼vr½hy
DS²'²³r·²h¼ròrq ³¸uhüqyr vüs¸¼·h³v¸ü th³ur¼rq s¼¸· yh¼trh·¸Ãü³²¸sryrp³¼¸üvp q¸p÷rü³² Düs¸¼·h³v¸ü vü h q¸p÷rü³ v² ih²vphyy' p¸·¦¸²rq ¸s Z¸¼q²¶ ²r·hü³vp² Crüpr DS ²'²³r·² hp³Ãhyy' qrhy Zv³u ³u¸²r Z¸¼q² Zuvpu h¼r ³ur ¼r¦¼r²rü³h³v½r² ¸s ²r·hü³vp² ³uh³ h¼r ³¼Ãy' ³ur iÃvyqvüt iy¸px² ¸s vü³rüqrq vüs¸¼·h³v¸ü 6 Z¸¼q vü h q¸p÷rü³ ·h' uh½r qvssr¼rü³ ·¸¼¦u¸y¸tvphy s¸¼·² hpp¸¼qvüt ³¸ v³² t¼h··h³vphy òhtr vü ³ur ³r`³ ²r¹Ãrüpr ió ³ur ²r·hü³vp ³uh³ v² ¼r¦¼r²rü³rq i' v³² ²³r· ¼r·hvü² Ãüpuhütrq Uur¼rs¸¼r DS ²'²³r·²trür¼hyy'òr³ur²³r·²vü²³rhq ¸s Z¸¼q s¸¼·²i¸³u ³¸¸½r¼p¸·r²³¸¼htr¦¼¸iyr·²hüq³¸vüp¼rh²r¦r¼s¸¼·hüprb©d Uurqrsvüv³v¸ü ¸s²³r··vütv²tv½rü i' G¸½vü²b(dh²þh¦¼¸prqür ³¸¼rqÃpr hyyZ¸¼q² Zv³u ³ur²h·r²³r· ³¸hp¸··¸üs¸¼· òÃhyy' i' ²³¼v¦¦vütrhpuZ¸¼q¸sv³² qr¼v½h³v¸ühy hüq vüsyrp³v¸ühy ²Ãssv`r²´ T³r··vüt s¸¼ DS vü hühy'³vp yhütÃhtr² yvxr @ütyv²u v² ¼h³ur¼ rss¸¼³yr²² ³uhü vü httyóvüh³v½r yhütÃhtr² 6² hü r`h·¦yr Q¸¼³r¼¶² hyt¸¼v³u· bd s¸¼ @ütyv²u v² ih²rq ¸ü h ²r¼vr² ¸s h yv·v³rq ü÷ir¼ ¸s ph²phqrq ¼rZ¼v³r ¼Ãyr² hüq phü hppr¦³ ürh¼y' hyy Z¸¼q s¸¼·² i' ¸üy' r`uhò³vüt hi¸Ã³ ! ·¸¼¦u¸y¸tvphy½h¼vh³v¸ü²bd7ó vüUüxv²uh²hü httyóvüh³v½r yhütÃhtrr½rü ³ur¼r h¼r h¦¦¼¸`v·h³ry' !" ²³r·²0 ³ur¸¼r³vphyy' hü vüsvüv³r ü÷ir¼ ¸s qvssr¼rü³ Z¸¼q s¸¼·² phü ir trür¼h³rq b !d Xv³u ³uv² yr½ry ¸s ·¸¼¦u¸y¸tvphy p¸·¦yr`v³' ³ur ·¸¼¦u¸y¸t' hühy'ªr¼² Zuvpu h¼r ³¼hü²qÃpr¼ ·¸qry² s¼¸· h yr`vp¸ü ¦yò r·irqqrq ¼Ãyr² h¼r ³ur ·r¼ry' hppr¦³rq ²³hüqh¼q s¸¼ ·¸¼¦u¸y¸tvphy ¦h¼²vüt bd 7ó ³ur²r ·¸¼¦u¸y¸t' hühy'ªr¼² h¼r ih²rq ¸ü ³Z¸yr½ry yhütÃhtr ·¸qry b" d Zuvpu v² ³ur¸¼r³vphyy' IQCh¼qb&d6² hp¸ü²r¹Ãrüpr³ur¼rh¼r³Z¸ puhyyrütr² s¸¼ DS ²'²³r·²) ö Xv³u ·¸¼¦u¸y¸t' hühy'ªr¼ h² h ²³r··r¼ p¸·¦Ã³h³v¸ühy p¸·¦yr`v³' h¼v²r² !ö Zv³u¸Ã³ h ²³r··r¼²³¸¼htrp¸·¦yr`v³'h¼v²r² s¼¸·vüqr`vütrhpu qvssr¼rü³ s¸¼·¸s h Z¸¼qZuvpu uh²³ur²h·r ²r·hü³vp ió h qvssr¼rü³·¸¼¦u¸y¸t'Uur¼rs¸¼r ²³r··vüt v² qr·hüqrq s¼¸· DS ²'²³r·² s¸¼ httyóvüh³v½r yhütÃhtr² ³¸ ¼rhpu h ·hühtrhiyr ²³¸¼htr ²vªr hüq h ²h³v²shp³¸¼' yr½ry ¸s ¦r¼s¸¼·hüpr b$ %d 7ó vü ³Ã¼ü yvürh¼ p¸·¦Ã³h³v¸ühyp¸·¦yr`v³' v²v·¦¸¼³hü³s¸¼ ²³r··vüthyt¸¼v³u·² ³¸irrssvpvrü³y' òrq s¸¼DS¦Ã¼¦¸²r²b©d U¸uvtuyvtu³³urürpr²²v³'¸s²³r··vütvühttyóvüh³v½ryhütÃhtr²³urqrt¼rr ¸s p¸·¦¼r²²v¸ü hpuvr½rq i' Q¸¼³r¼¶² bd ²³r··r¼ h¦¦yvrq ³¸ h ürZ²Zv¼r qh³hih²r p¸ü³hvüvüt $%&$%# @ütyv²u Z¸¼q² hüqPsyhªr¼¶² b#d ·¸¼¦u¸y¸t' hühy'ªr¼ òrq h² h ²³r··r¼ h¦¦yvrq³¸ ¦¸yv³vphyürZ²qh³hih²r ³uh³p¸ü³hvürq "&%©&Uüxv²uZ¸¼q²h¼r ³hiÃyh³rqvüUhiyr
!#%
7U9vüor¼hüq 7Fh¼h¸÷yhü Table 1. Level of compression for Turkish and English text corpora with stemming[5]. 8¸¼¦Ã² Uüxv²u @ÿtyv²u
X¸¼qU¸xrÿ² "&%©& $%&$
9v²³vÿp³³r¼·² #"& ©"©#
9v²³vÿp³²³r·² %"%" %&
8¸·¦¼r²²v¸ÿþ ©#% "%#
6² ²rrü vü Uhiyr qr²¦v³r ³ur shp³ ³uh³ @ütyv²u qh³hih²r uh² ·hü' ·¸¼r Z¸¼q ³¸xrü²³uhü Uüxv²u qh³hih²r0 ³ur ü÷ir¼¸s qv²³vüp³Uüxv²u ³r¼·²h¼rt¼rh³r¼ ³uhü ³ur ü÷ir¼ ¸s qv²³vüp³ @ütyv²u ³r¼·² Uuv² vüqvph³r² ³ur qv½r¼²v³' ¸s Uüxv²u Z¸¼q qr¼v½h³v¸ü² Zuvpu v² hy²¸ ½r¼vsvrq i' ³ur p¸·¦¼r²²v¸ü ¼h³v¸ ¼r²Ãy³² Uur p¸·¦¼r²²v¸ü yr½ry ¸s "%# v² sh¼ ²·hyyr¼ ³uhü ©#% Zuvpu pyrh¼y' vüqvph³r² ³ur y¸Zr¼qrt¼rr¸s·¸¼¦u¸y¸tvphy½h¼vh³v¸ü²¸s Z¸¼q² vü@ütyv²uUurp¸·¦¼r²²v¸ü ¼h³v¸ ¸s©#%¸i²r¼½rqs¸¼ Uüxv²uqh³hih²rr·¦uh²vªr²³urürrqs¸¼ ²³r··vüt
"
T³r··vÿthÿqSryh³rqX¸¼x
Uur ²³r··vüt hyt¸¼v³u·² òrq s¸¼ DS ¦Ã¼¦¸²r² phü ir t¼¸Ã¦rq vü s¸Ã¼) ö Uhiyr G¸¸xæ hyt¸¼v³u·² !ö TÃppr²²¸¼ Wh¼vr³' hyt¸¼v³u·² "ö üt¼h· hyt¸¼v³u·² #ö 6ssv` Sr·¸½hy hyt¸¼v³u·² 6yy ¸s ³ur²r s¸Ã¼ t¼¸Ã¦² ¸s ²³r··r¼² h¼r qr½ry¸¦rq s¸¼ rv³ur¼ hühy'³vp yhütÃhtr² yvxr @ütyv²u ¸¼ ²¸·r vüq¸@ü¸¦rhü yhütÃhtr² yvxr A¼rüpu hüq Br¼·hü7ó³ur²r hyt¸¼v³u·²phüü¸³p¸¦rZv³uuvtuyr½ry¸s·¸¼¦u¸y¸tvphy¼vpuür²² vü yhütÃhtr² uh½vüt vüsyrp³v¸ü ¦h¼hqvt· yvxr ²¸Ã³uZr²³r¼ü ¸¼ Ptuê t¼¸Ã¦ Uüxvp httyóvüh³v½ryhütÃhtr²Zuvpuhy²¸ vüpyÃqrUüxv²u Uüx·rü6ªr¼ihvwhüv¸¼ 6ªr¼v Buh²tuhvhüqBhthêb!"d Uur¼r h¼r hy²¸ h ü÷ir¼ ¸s ²³r··vüt hyt¸¼v³u·² qr½ry¸¦rq s¸¼ Uüxv²u Zuvpuh¼rih²rq ¸ü ³urrss¸¼³¸s sv³³vüt ³ur ·¸¼¦u¸y¸t' hühy'ªr¼vü³¸hü DS²'²³r· i' qrp¼rh²vüt ³ur ²³¼vp³ür²² ¸s ²¸·r ·r³u¸q² ²Ãpu h² ³ur ³httvüt hüq vqrü³vsvph³v¸ü ¸s ·¸¼¦ur·r²q¸Zü ³¸hü hppr¦³hiyryr½ry¸shppühp'@`h·¦yr²¸s ³ur²r²³r··r¼²h¼r ¸s 9ühü¶² b#d 6y¦x¸ohx r³ hy b$d Uur²r hyt¸¼v³u·² ²rh¼pu h ¦¼rp¸ü²³¼Ãp³rq ryrp³¼¸üvp yr`vp¸ü ³¸ svüq h ¦¼¸ihiyr ²³r· Uurü ³ur' ½r¼vs' ³uh³ v³ v² p¸¼¼rp³ i' ·h³puvüt³ur¼r²³¸s³ur Z¸¼q s¸¼· Zv³uh ¼rp¸tüvªhiyr²Ãssv`²r¹Ãrüpr i' y¸¸xvütæ ¦¸²²viyr p¸·ivüh³v¸ü² ¸s ²Ãssv`r² hpp¸¼qvüt ³¸ ·¸¼¦u¸³hp³vp² ¸s Uüxv²u 9rt¼rr ¸s p¸·¦h¼v²¸ü ¸s ³uv² yh³r¼ ½r¼vsvph³v¸ü ¦uh²r hssrp³² i¸³u ³ur hppühp' hüq ³ur p¸·¦Ã³h³v¸ühyp¸·¦yr`v³'0vüh uvtuyr½ry¸s½r¼vsvph³v¸ühppühp' Zvyyiruvtuió ³ur p¸·¦Ã³h³v¸ühy p¸·¦yr`v³' Zvyy ir py¸²rq ³¸ IQuh¼q hüq ½vpr ½r¼²h Uur¼rs¸¼r s¸¼³ur²r²³r··r¼²³uryr½ry¸sp¸·¦Ã³h³v¸ühyp¸·¦yr`v³' vüp¼rh²r² ¸½r¼hthvüs¼¸· h ¦¸vü³ py¸²r ³¸ yvürh¼ ³¸ h ¦¸vü³ py¸²r ³¸ IQCh¼q h² ·Ãpu h² ³ur yr½ry ¸s ·¸¼¦u¸y¸tvphyxü¸Zyrqtròrv² vüp¼rh²rq Pür ¸s ³ur sv¼²³ h³³hpx² ³¸ ³ur ²³r··vüt ¦¼¸iyr· vü Uüxv²u v² q¸ür i' F|x²hy b%d D³ v²p¸·¦yr³ry'hqvssr¼rü³rss¸¼³ s¼¸·³ur¸³ur¼² i' v³² ü¸üyvütÃv²³vp² ¦r¼²¦rp³v½r Cruh²qr½ry¸¦rq h ²³r··r¼³uh³²v·¦y' ³hxr²h sv`rq yrüt³u ¦h¼³¸s h Z¸¼q s¸¼· s¼¸·³ur²³h¼³h² ³ur ²³r·¸s ³uh³Z¸¼qs¸¼· P¼vtvühyy' F|x²hypyhv·rq ³uh³ h ²³r· yrüt³u ¸s $ tv½r² ³ur ir²³ ¼r²Ãy³ vü ³r²³² 7ó vü ¦¼hp³vpr ³ur¼r v² ü¸ p¸··¸üsv`rq yrüt³u s¸¼hyyq¸p÷rü³²r³² 6 ½hyÃr ³uh³ Z¸¼x² Zryy Zv³u h ¦h¼³vpÃyh¼ q¸p÷rü³ ²r³ Ãüqr¼ h ¦h¼³vpÃyh¼ ³¸¦vp ·h' qrp¼rh²r ³ur ¦r¼s¸¼·hüpr ¸s DS ²'²³r· qrhyvüt Zv³u h q¸p÷rü³ ²r³ Ãüqr¼ h qvssr¼rü³ ³¸¦vp 9r²¦v³r v³² vürssvpvrüpvr² ³uv²
T³r··vüt vü6ttyóvüh³v½rGhütÃhtr²)6 Q¼¸ihivyv²³vp T³r··r¼s¸¼Uüxv²u!#&
h¦¦¼¸hpu v²¸Ã¼¼rsr¼rüpr p¸üpr¦³²vüprv³¦¼¸½r² ³uh³ r½rüh³ Z¸¼²³ph²rvrZv³u¸Ã³ hü' ·¸¼¦u¸y¸tvphyxü¸Zyrqtrö³ur DS¦r¼s¸¼·hüprs¸¼Uüxv²u phü ir vüp¼rh²rq i' wò³³hxvütsv`rq yrüt³u²Ãi²³¼vüt²s¼¸· ³ur²³h¼³¸s ³urZ¸¼qs¸¼·²h²¸s³urv¼²³r·² Uur¼rs¸¼rv³v²¼rh²¸ühiyr³¸r`¦rp³³uh³ ³ur ¦r¼s¸¼·hüprp¸Ãyqsü³ur¼irvüp¼rh²rq i' rüuhüpvüt ³uv² ²v·¦yr p¸üpr¦³ Zv³u ¦¼ht·h³vp² ¸s ³ur ·¸¼¦u¸y¸tvphy xü¸Zyrqtr Dü¸Ã¼¦¼¸¦¸²rq·¸qry Zròr ³ur ²³h³v²³vp²¸strür¼hyvªrq·¸¼¦u¸y¸tvphyxü¸Zyrqtr th³ur¼rq s¼¸· h¦¼r½v¸Ã²y' hühy'ªrq³r`³qh³hih²rh² ³ur ¦¼ht·h³vp²³¸rüuhüpr³uv² ¦¼v¸¼¼rsr¼rüpr·¸qry7' ³ur òr¸s¦¼rp¸·¦Ã³rq ²³h³v²³vp²Zrhy²¸ uh½rtÃh¼hü³rrq ³¸²³h' h¼¸Ãüqhyvürh¼p¸·¦Ã³h³v¸ühyp¸·¦yr`v³' "
I¸³h³v¸ÿ
Xr qrü¸³rh Z¸¼qs¸¼· vühü' httyóvüh³v½ryhütÃhtri' h²³¼vüt ³uh³v² ¼r¦¼r²rü³rq i' V Q = KK! " KQ Zur¼r rhpu KL L = ! ! Q ö v² h ·r·ir¼ ¸s ³ur p¸¼¼r²¦¸üqvüt hy¦uhir³$hüq Q v²³urü÷ir¼ ¸syr³³r¼² vr yrüt³u¸s³ur²³¼vütöDü ¸Ã¼ ²³Ãq' Zr òrq ³ur Uüxv²u hy¦uhir³ Zuvpu uh² !( yr³³r¼² hüq ³ur Ãüqr¼²p¸¼r µf¶ s¸¼ iyhüx puh¼hp³r¼h²s¸yy¸Z²) $ = {D E F o G H I J ÷ K Õ L M N O P Q R | S U V ú W X Y \ ] õ fõ}
hüqZrhqh¦³rq ³urs¸yy¸Zvütü¸³h³v¸ü²³¸ qrü¸³r²Ãi²³¼vüt² ¸shü'²³¼vüt V Q s¸¼ ≤ L ≤ M ≤ Q V Q [L ) M ] = KL KL + " K M VQ [) M ] = KK! " K M hüq VQ [L )]= KL + " KQ
7h²rq ¸ü¸Ã¼ ü¸³h³v¸ü³ur²¦rpvhy²Ãi²³¼vüt V Q [L ) L + ] = KL KL + v²qrü¸³rqi' ³ur¸¼qr¼rq ¦hv¼¸s yr³³r¼² K K! ö L Zur¼r²Ãivüqr` L L = ! Q − övüqvph³r²³ur ²³h¼³vüt ¦¸²v³v¸ü ¸s ³ur ¸¼qr¼rq ¦hv¼ vü ³uh³ ²³¼vüt hüq K = KL K! = KL + ∈ $ A¸¼ L = Q ³ur¸¼qr¼rq ¦hv¼ v²s¸¼·rqi' hühqqrqiyhüxh² KQ õ fõ ö L = Q Uuòhü'²³¼vüt V Q = KK! " KQ uh² Q¸¼qr¼rq¦hv¼²vü ¸Ã¼²³Ãq' A¸¼ h tv½rü ¸¼qr¼rq ¦hv¼ ¸s yr³³r¼² K K! ö M ³uh³ phü h¦¦rh¼ h³ h ¦¸²v³v¸ü ≤ M ≤ Q ·h` vü hü' Uüxv²u Z¸¼q s¸¼· Zur¼r Q·h` v² ³ur ·h`v·Ã· ü÷ir¼ ¸s
yr³³r¼² ³uh³ h Uüxv²u Z¸¼q s¸¼· phü uh½r vü ¸Ã¼ qh³hih²r Q ·h` = !" ö hüq h tv½rü ¦h¼³vpÃyh¼ Z¸¼q s¸¼· qrü¸³rq i' V Q = KK! " KQ Zur¼r Q ≥ M ³ur ü¸³h³v¸ü K K! ö M ∈ V Q ¼rsr¼²³uh³³ur¼rr`v²³²hü ¸¼qr¼rq¦hv¼ K K! ö L h³¦¸²v³v¸üL ≤ L ≤ Q ö vü V Q ¦¼¸½vqrq ³uh³ K K! ö L = K K! ö M s¸¼ L = M Avühyy' Zr qrsvür ³Z¸ ·¸¼r
²'·i¸y²üh·ry' J P = V Q [) P] hüq H P = V Q [P )] vü¸¼qr¼ ³¸ ¼r¦¼r²rü³hü' Z¸¼qs¸¼· h²hü ¸¼qr¼rq¦hv¼ ¸s³Z¸²Ãi²³¼vüt²i' V QP = J P H P ö s¸¼hyy PZur¼r ≤ P ≤ Q
!#©
"!
7U9vüor¼hüq 7Fh¼h¸÷yhü
UurTh·¦yrT¦hprhÿq ³urP¼qr¼rq Qhv¼Q¼¸ihivyv³vr²
Ds Zryr³ G ir³ur²r³¸s hyy¦¸²²viyr¸¼qr¼rq¦hv¼² ¸syr³³r¼² K K! ö L ³uh³phüh¦¦rh¼ vü hü' Uüxv²u Z¸¼q s¸¼· h³ ¦¸²v³v¸ü² L = ! Q ·h` ³urü / Zvyy ir ³ur ²h·¦yr ²¦hprhüqphüirqrsvürqh²s¸yy¸Z²)
/ = {K K! ö L K K! ∈
hüq ≤ L ≤ Q·h` }
6üq sü³ur¼ ²Ã¦¦¸²r ³uh³ ³ur ²r³² *N ( N hüq 7N Zur¼r *N ( N 7N ⊂ / hüq ≤ N ≤ Q ·h` ¼r¦¼r²rü³³urr½rü³²qrsvürq iry¸Z)
*N = {K K! ö L
L = N hüq K K! ö L ∈ J P hüq ≤ · ≤ Q·h` }
(N = {K K! ö L L = N hüq K K! ö L ∈ HP hüq ≤ · ≤ Q·h` }
{
7N = K K! ö L L = N K = V Q bN ) N d K! = V Q bN + N + d ≤ L ≤ Q·h`
}
Uuò s¸¼ rhpu ¸¼qr¼rq ¦hv¼ K K! ö L h³ ¦¸²v³v¸ü² L = ! Q ¸s hü' tv½rü Z¸¼q s¸¼· qrü¸³rqi' V Q = KK! " KQ ¸ür phüqrsvür¦¼¸ihivyv³vr² ¸s irvüt vü ³ur hi¸½r³u¼rr ²r³²h² s¸yy¸Z² Q¼ (V Q [L ) L + ]∈ *L ) = Q¼ (K K! ö L ∈ *L )= QB (K K! ö L )
ö
Q¼ (V Q [L ) L + ]∈ ( L ) = Q¼ (K K! ö L ∈ ( L ) = Q@ (K K! ö L )
!ö
Q¼ (V Q [L ) L + ]∈ 7L ) = Q¼ (K K! ö L ∈ 7L ) = QU (K K! ö L )
"ö
Xur¼r r¹Ãh³v¸ü ö ¼rsr¼² ³¸ ³ur ¦¼¸ihivyv³' ¸s ³ur ¸¼qr¼rq ¦hv¼ K K! ö L irvüt vü³ur²³r· ¦h¼³ ¸s³urtv½rüZ¸¼q s¸¼·Tv·vyh¼y' r¹Ãh³v¸ü !ö ¼rsr¼² ³¸ ³ur ¦¼¸ihivyv³' ¸s ³ur ¸¼qr¼rq ¦hv¼ K K! ö L irvüt vü ³ur ²Ãssv` ¦h¼³ ¸s ³ur tv½rü Z¸¼q s¸¼· hüq svühyy' r¹Ãh³v¸ü "ö ¼rsr¼² ³¸ ³ur ¦¼¸ihivyv³' ¸s ³ur ¸¼qr¼rq ¦hv¼ K K! ö L irvüt vüir³Zrrü³ur²³r· ¦h¼³ hüq ³ur ²Ãssv`¦h¼³ ¸s³urtv½rü Z¸¼q s¸¼·vr Kv² ³uryh²³yr³³r¼ ¸s³ur²³r·¦h¼³hüqK v²³ursv¼²³yr³³r¼ ¸s²Ãssv`¦h¼³ö
#
Q¼¸ihivyv²³vpT³r··vÿt6²²r¼³v¸ÿ
Uur ¦¼¸ihivyv²³vp s¼h·rZ¸¼x vü ³uv² ²³Ãq' v² s¸¼·hyvªrq vü ³ur H¸qry hüq ³ur ¦¼¸ihivyv²³vp²³r··vüthyt¸¼v³u·v²ih²rq¸ü³urHr³u¸q Uur½hyvqv³' ¸s³ur·¸qry hüq ³ur·r³u¸qv²r`h·vürqi' hür`¦r¼v·rü³qr²p¼virqh²s¸yy¸Z² H¸qry)Ds³ur¼rr`v²³² hüvü³rtr¼·Zur¼r ≤ P ≤ Q ³uh³²h³v²svr² [3( (K K! ö P ) > 3* (K K! ö P )]hüq 37 (K K! ö P− ) ≥ α
T³r··vüt vü6ttyóvüh³v½rGhütÃhtr²)6 Q¼¸ihivyv²³vp T³r··r¼s¸¼Uüxv²u!#(
s¸¼ hZ¸¼qs¸¼· V QP = J P HP ö hüqhp¸ü²³hü³ ≤ α ≤ ³urü J P− v² ²hvq³¸ ir³ur ¦¼¸ihiyr²³r·¸s³uh³Z¸¼q s¸¼· Hr³u¸q) Uur¦¼¸ihivyv²³vp²³r··vüt¸shü' Z¸¼qs¸¼· qrü¸³rqi' V Q = KK! " KQ ≤ Q ≤ Q ·h` ö v²q¸üri' ²³¼v¦¦vüt¸ss²Ãi²³¼vüt H P− hüq ³hxvüt ²Ãi²³¼vüt J P− h² ³ur²³r·¸s ³uh³Z¸¼qs¸¼· h³³ur¦¸²v³v¸üP ³uh³²h³v²svr²³urp¸üqv³v¸ü²¸s³urH¸qry s¸¼h¦h¼³vpÃyh¼ ≤ α ≤ Dü shp³ ³ur ²³r··r¼ svüq² qvssr¼rü³ ²³r·² s¸¼ h ¦h¼³vpÃyh¼ Z¸¼q s¸¼· s¸¼ qvssr¼rü³ ½hyÃr² ¸s α Dü ¸Ã¼ ³r²³ Zr òrq qvssr¼rü³ ½hyÃr² ¸s α ³uh³ ²³r¦² ³ur vü³r¼½hyi' r¹Ãhy²³hüqh¼qvªrq ¦r¼prü³vyr²Uurü³ur¼r²Ãy³hü³²³r·² ¸srhpu ³¼vhy² h¼r ·r¼trq vü³¸ h ²vütyr p¸yyrp³v¸ü Uuò Zr uh½r ³ur p¸yyrp³v¸ü Zuvpu uh² ³ur ¦¸²²viyr ²³r·² ³uh³ phü ir ¦¼¸qÃprq i' ¸Ã¼ ¦¼¸¦¸²rq ·r³u¸q Uur shp³ ³uh³ ³ur ²ryrp³v¸ü¸sh¦h¼³vpÃyh¼½hyÃrs¸¼ α hssrp³²³ur¸½r¼hyyDS²'²³r· ¦r¼s¸¼·hüpr0v³Zvyy ir¸i²r¼½rq r·¦v¼vphyy' Zurü ³urvü³rüqrq DS²'²³r· v²p¸·¦yr³rq7rphòr³ur α ½hyÃr v² ²Ãiwrp³ ³¸ puhütr q'üh·vphyy' Zv³u ¼r²¦rp³ ³¸ DS ²'²³r·¶² ¦r¼s¸¼·hüpr ürrq² 6³ ³uv² ³v·r v³ v² ²Ãssvpvrü³ ³¸ ½hyvqh³r ³uh³ ¸ür phü qr³r¼·vür ³ur vü³rüqrq ²³r·²i' h ¦¼¸ihivyv²³vp·r³u¸q¸½r¼hyy¦¸²²viyr α ½hyÃr²h²Zr ¦¼¸¦¸²rq
$
@`¦r¼v·rÿ³hÿq³urSr²Ãy³²
7¸³u r`¦r¼v·rü³h³v¸ü hüq ³r²³ qh³hih²r² h¼r ²ryrp³rq s¼¸· h Uüxv²u ³r`³ qh³hih²r Zuvpu v² h p¸yyrp³v¸ü ¸s Uüxv²u ürZ² ³r`³² uh½vüt H Z¸¼q² s¸¼·² Zuvpu h¼r ·¸¼¦u¸y¸tvphyy' hühy'ªrq hüq qv²h·ivtÃh³rq i' ChxxhüvU¼ r³ hy b&d Q¼¸¦r¼³vr² ¸s³ur²ryrp³rqqh³hih²r²hüq³ur ¼r²Ãy³²h¼rtv½rüvüUhiyr! Table 2. Properties of Experimentation and Test databases 9h³hih²r² @`¦r¼v·rü³hy Ur²³
Ur¼· 8¸Ãü³ #(©( #©#©%
9v²³vp³ Ur¼·² "%(! "%$%"
9v²³vüp³ T³r·² $%© !$"
Vüxü¸Zü T³r·² #!
Q¼¸qÃprq 8¸¼¼rp³ö (©!©
Qhv¼ 8¸Ãü³ $%#©
TÃppr²²yr½ry ö ($©
Ur¼·²vüi¸³uqh³hih²r²uh½rirrü¦¼r¦¼¸pr²²rq³¸vqrü³vs' ²³r· hüq²Ãssv` ¦h¼³²hpp¸¼qvüt³¸ ³urv¼ ³httrq·¸¼¦u¸y¸tvphyhühy'²r² ¼r²Ãy³²Uurqh³hih²r²h¼r ü¸³ ¦¼¸pr²²rqhü' sü³ur¼²Ãpu h²ryv·vüh³v¸ü¸s ²³¸¦Z¸¼q²qr³r¼·vüh³v¸ü¸s·v²²¦ryyrq Z¸¼q²r³pD³uh²³¸ ir ü¸³rq³uh³vsh·¸¼¦u¸y¸t' hühy'ªr¼v²Ã²rq h²h ²³r··r¼s¸¼ ³ur ³r²³ qh³hih²r ³ur hi²¸yór æ¦r¼ i¸Ãüq ¸s ³ur p¸·¦¼r²²v¸ü ¼h³v¸ v² & vr Zurü hyy¸s³ur"%$%"qv²³vüp³³r¼·² h¼r ¼rqÃprq p¸·¦yr³ry' vü³¸³urv¼!$" hp³Ãhy ²³r·²öhüq³ur¦¼¸¦¸²rq ²³r··r¼phü¼rhpuhp¸·¦¼r²²v¸ü ¼h³v¸ ¸s %(vr(©! ²³r·² ¸s Zuvpu ³ur (©!©²³r·²h¼rp¸¼¼rp³y's¸Ãüqhüq $#³r¼·²h¼r·v²²rq¸Ã³¸s "%$%"qv²³vüp³³r¼·²ö Uur ¦¼¸ihivyv³vr² tv½rü vü r¹Ãh³v¸ü² ö !ö hüq "ö h¼r phypÃyh³rq i' ³ur s¸yy¸Zvüts¸¼·Ãyh²
!$
7U 9vüor¼hüq 7 Fh¼h¸÷yhü
3* (K K! ö L ) = I J L û Z J L 1
0 3( (K K! ö L ) = I HL û ZHL 1 0
37 (K K! ö L ) = I W L û ZW L 1
Zur¼r
I J L
I HL hüq
I W L h¼r ³ur s¼r¹Ãrüpvr² ¸i²r¼½rq s¼¸· ³ur
r`¦r¼v·rü³h³v¸üqh³hih²rs¸¼³ur r½rü³²³uh³³ur ¸¼qr¼rq¦hv¼²¸syr³³r¼²uh½rh¦¦rh¼rq vüh ²³r·¦h¼³ ²Ãssv`¦h¼³ hüq³¼hü²v³v¸üir³Zrrüh²³r· ¦h¼³hüq ²Ãssv`¦h¼³¸s h Z¸¼q s¸¼· ¼r²¦rp³v½ry' Uur p¸¼¼rp³v¸ü shp³¸¼² Z J L ZHL hüq ZW L h¼r ¦¼r½v¸Ã²y' phypÃyh³rq¦¸²v³v½r ¼rhy½hyÃr² s¸¼³urp¸¼¼r²¦¸üqvüt s¼r¹Ãrüpvr² I J L I HL hüq I W L 1 v²³ur³¸³hyü÷ir¼¸s ¸¼qr¼rq¦hv¼²vü³urr`¦r¼v·rü³h³v¸üqh³hih²r 6s³r¼³ur¦¼¸pr²²vüt ¸s³urr`¦r¼v·rü³h³v¸üqh³hih²r³ur³¸³hyü÷ir¼¸s $%#© Ãüv¹Ãr ¸¼qr¼rq ¦hv¼² ¸syr³³r¼²h¼r¸i²r¼½rq s¸¼³ur²h·¦yr²¦hpr / ¸sZuvpu !©#$ ¦hv¼² ¸üy' iry¸üt ³¸ ³ur r½rü³²r³ *N ©# ¦hv¼²¸üy' iry¸üt ³¸ ³urr½rü³²r³ ( N hüq "!!¦hv¼² ¸üy' iry¸üt ³¸³urr½rü³ 7N Dü³urvü³r¼²rp³v¸ü ¸s³urr½rü³²r³² *N hüq ( N ³ur¼r h¼r "(& Ãüv¹Ãr ¸¼qr¼rq ¦hv¼² Düqrrq ³ur²r ¸i²r¼½h³v¸ü² h¼r ½r¼' rüp¸Ã¼htvüt ²vüpr³ur' vüqvph³r³uh³$¸shyy¸¼qr¼rq¦hv¼²¸syr³³r¼²h¦¦rh¼¸üy' vü ³ur ²³r·¦h¼³(¸s ³ur·h¦¦rh¼¸üy' vü²Ãssv`¦h¼³hüq !# ¸s ³ur· h¼r²uh¼rq i' ²³r· hüq ²Ãssv` ¦h¼³² Uuv² ²³h³v²³vp² pyrh¼y' r`¦yhvü² Zu' ¸Ã¼ ¦¼¸¦¸²rq ·¸qry uh² ²Ãpprrqrq Pü³r²³qh³hih²r uh² !$" qv²³vüp³²³r·² ¸sZuvpu #! ²³r·² h¼rü¸³ vü³urr`¦r¼v·rü³hyqh³hih²rUuv²¼h³v¸v²shv¼y' yh¼tr hüqh·¸Ãü³²³¸ hi¸Ã³ # ¸s Ãüxü¸Zü²³r·²Uur²³r··r¼¦¼¸qÃprq&!(%& ¦¸²²viyr²³r·²s¸¼"%$%"qv²³vüp³³r²³ ³r¼·² s¸¼ ¦¼rqr³r¼·vürq α ½hyÃr² Uuò s¸¼ rhpu ³r¼· h¦¦¼¸`v·h³ry' ! ²³r·² uh½r irrü ¦¼¸qÃprq Dü shp³ ·¸²³ ¸s ³ur²r &!(%& ¦¼¸qÃprq ¦¸²²viyr ²³r·² h¼r ¼rp¸tüvªrq Uüxv²u Z¸¼q s¸¼·² ió rv³ur¼ ³ur' h¼r Ãüqr¼²³r··rq ³¸ h ¼¸¸³ Zuvpu uh² h qvssrü³ ²r·hü³vp ³uhü ³ur ³h¼tr³ ²³r· ¸¼ ¸½r¼²³r··rq ³¸ h qr¼v½h³v¸ühy ¸¼ vüsyrp³v¸ühy s¸¼· Zuvpu v² ·¸¼¦u¸y¸tvphyy' qvssr¼rü³ s¼¸· ³ur ³h¼tr³ ²³r· Dü phypÃyh³v¸ü²Zr hppr¦³rq³urr`hp³·h³puZv³u³ur ³h¼tr³²³r· h²p¸¼¼rp³6²h ¼r²Ãy³ ¸Ã¼ ¦¼¸¦¸²rq ²³r··r¼ hpuvr½rq ³¸ ¦¼¸qÃpr p¸¼¼rp³ ²³r·² s¸¼ ³ur ($© ¸s ³ur ³r²³ ³r¼·² vr (©!©!$"ö Pü ²³r··r¼ p¸Ãyqü¶³ ²³r· $# qv²³vüp³ ³r¼·² Zuvpu Zr¼r ³ur½h¼vh³v¸ü²¸s #!$qv²³vüp³²³r·² ¸Ã³ ¸s"%$% 7ó ·¸²³¸s³ur·v²²rq³r¼·² h¼rs¸Ãüq³¸irs¸¼rvtüZ¸¼q²hii¼r½vh³v¸ü²hp¼¸ü'·²¸¼¦¼¸ü¸Ãü² ³uh³ h¼r¦¸²²viy' ryv·vüh³rqi' hü' ³r`³¦¼r¦¼¸pr²²¸¼²Ã²rq vühü¸¼qvüh¼'DS²'²³r·
%
8¸ÿpyòv¸ÿ²
Xruh½r¦¼r²rü³rqur¼rhürZ ²³r··vüt·¸qryih²rq ¸üh ¦¼¸ihivyv²³vps¼h·rZ¸¼x s¸¼httyóvüh³v½ryhütÃhtr²Pü·¸qryuh² h yvürh¼ p¸·¦Ã³h³v¸ühyp¸·¦yr`v³' hüqv² ph¦hiyr ¸s hpuvr½vüt h ²Ãppr²² ¸s ($© Uuv² ¼r²Ãy³ ²Ãttr²³² ³uh³ ³ur ¦¼¸¦¸²rq ·¸qry¸s²³r··vütphüirtrür¼hyvªrq ³¸h¦¦y' ³¸¸³ur¼httyóvüh³v½ryhütÃhtr²yvxr CÃüth¼vhü Avüüv²u @²³¸üvhü hüq 8ªrpu Zuvpu uh½r ¦¼¸qÃp³v¸ü hüq vüsyrp³v¸ü ¦h¼hqvt·²
T³r··vüt vü6ttyóvüh³v½rGhütÃhtr²)6 Q¼¸ihivyv²³vp T³r··r¼s¸¼Uüxv²u!$
6pxÿ¸Zyrqtr·rÿ³² Xr ³uhüx F Psyhªr¼ s¸¼ ¦¼¸½vqvüt ò Zv³u ·¸¼¦u¸y¸tvphyy' hühy'ªrq Uüxv²u qh³hih²r hüq 6 gª³¼x s¸¼ ½r¼' ury¦sÃy vü²vtu³² hüq p¸··rü³² hi¸Ã³ ²³h³v²³vphy h²¦rp³² hüq g 7r¼x hüq F 9r·v¼ ¸s HÃ÷yh Vüv½r¼²v³' hüq 6 8 F¸¼¸÷yÃs¸¼ ½r¼'ury¦sÃyqv²pò²v¸ü²¸ü yvütÃv²³vp²ihpxt¼¸Ãüqvü³uv²²³Ãq'
Srsr¼rÿpr² ! " # $
% & © ( ! " # $ % &
Eühs²x' 9 Hh¼³vü E H) T¦rrpu hüq GhütÃhtr Q¼¸pr²²vüt Q¼rü³vprChyy IrZ Er¼²r' VT6!ö Chüxh·r¼ E) Uüxv²u trür¼h³v½r ·¸¼¦u¸y¸t' hüq ·¸¼¦u¸y¸tvphy ¦h¼²vüt Dü) Trp¸üq Dü³r¼üh³v¸ühy 8¸üsr¼rüpr ¸ü Uüxv²u GvütÃv²³vp² D²³hüiÃy Uüxr'(©#ö F¸²xrüüvr·v F) UZ¸yr½ry H¸¼¦u¸y¸t') 6 Brür¼hy 8¸·¦Ã³h³v¸ühy H¸qry s¸¼ X¸¼q A¸¼· Srp¸tüv³v¸ü hüq Q¼¸qÃp³v¸ü Dü) QÃiyvph³v¸ü² ¸s ³ur 9r¦h¼³·rü³ ¸s Brür¼hy GvütÃv²³vp² W¸yVüv½r¼²v³'¸sCry²vüxv Cry²vüxv (©"ö Psyhªr¼F)UZ¸ Gr½ry9r²p¼v¦³v¸ü ¸sUüxv²u H¸¼¦u¸y¸t' Dü) Q¼¸prrqvüt² ¸s@68G¶(© V³¼rpu³ ³urIr³ur¼yhüq² (("ö @x·rxov¸tyà A ýÃüh G Hvpuhry A Xvyyr³³ Q) T³r··vüt hüq It¼h· ·h³puvüt s¸¼ ³r¼· p¸üsyh³v¸ü vü Uüxv²u ³r`³²Dü) Düs¸¼·h³v¸ü Sr²rh¼pu ö 6½hvyhiyr h³) u³³¦)vüs¸¼·h³v¸ü¼ür³v¼!!¦h¦r¼"u³·y ((%ö T¸yhx 6 8hü A) @ssrp³² ¸s T³r··vüt ¸ü Uüxv²u Ur`³ Sr³¼vr½hy Urpuüvphy Sr¦¸¼³ 7V8@DT(#! 9r¦h¼³·rü³¸s 8¸·¦Ã³r¼@ütvürr¼vüt hüq Düs¸¼·h³v¸ü Tpvrüpr 7vyxrü³ Vüv½r¼²v³'6üxh¼h((#ö 7h¼³¸ü B @qZh¼q) 8¸·¦Ã³h³v¸ühy 8¸·¦yr`v³' vü UZ¸Gr½ry ·¸¼¦u¸y¸t' Dü) 68G Q¼¸prrqvüt²!#³u 6üüÃhyHrr³vüt(©%ö S7hrªhÁh³r²7 Svirv¼¸Ir³¸) H¸qr¼ü Düs¸¼·h³v¸ü Sr³¼vr½hy²³rq6qqv²¸üXr²yr' @ütyhüq (((ö G¸½vü² E7) 9r½ry¸¦vüt ¸s h T³r··vüt 6yt¸¼v³u· Dü) Hrpuhüvphy U¼hü²yh³v¸ü hüq 8¸·¦Ã³h³v¸ühyGvütÃv²³vp² W¸y (%©ö !!±" Q¸¼³r¼ H A) 6ü6yt¸¼v³u·s¸¼ TÃssv`T³¼v¦¦vüt Dü) Q¼¸t¼h·W¸y# I¸" (©ö "± "& gª³hür¼ TH)6X¸¼q B¼h··h¼ ¸sUüxv²uZv³uH¸¼¦u¸¦u¸ür·vpSÃyr² HTp Uur²v² 9r¦h¼³·rü³¸s 8¸·¦Ã³r¼ @ütvürr¼vütH@UV6üxh¼hUüxr'((%ö 8¼'²³hy 9) Uur 8h·i¼vqtr @üp'py¸¦rqvh ¸s GhütÃhtr 8h·i¼vqtr Vüv½r¼²v³' Q¼r²² 8h·i¼vqtrVF(©&ö GrZv²BG) Uüxv²uB¼h··h¼ P`s¸¼q Vüv½r¼²v³'Q¼r²² VF((ö 9ühü B) Uüxv²u T³r··vüt 6yt¸¼v³u· H Tp Uur²v² 9r¦h¼³·rü³ ¸s 8¸·¦Ã³r¼ @ütvürr¼vütChpr³³r¦r Vüv½r¼²v³'6üxh¼h ((&ö 6y¦x¸ohx 6 Fó 6 gªxh¼huhü @) 7vytv 7Ãy·h Tv²³r·yr¼v vovü P³¸·h³vx U¼xor 9vªvüyr·r Á|ü³r·v Dü) 7vyvúv· 7vyqv¼vyr¼v 9¸xê @'yy ÿüv½r¼²v³r²v øª·v¼ U¼xv'r (($ö !#&±!$" F|x²hy 6) 7vytv @¼vúv· T¸¼Ãüà ½r 7v¼ 7rytr 9vªvüyr·r ½r @¼vúv· 9vªtr²v Uh²h¼Õ· ½r Br¼orxyrú³v¼v·v 9¸orü³yvx Urªv Arü 7vyv·yr¼v @ü²³v³² 7vytv²h'h¼ 7vyv·yr¼v Hurüqv²yv÷v 6ühivyv·9hyÕChpr³³r¦rÿüv½r¼²v³r²v6üxh¼h(&(ö ChxxhüvU¼ 9a Psyhªr¼ F U¼ B) T³h³v²³vphy H¸¼¦u¸y¸tvphy 9v²h·ivtÃh³v¸ü s¸¼ 6ttyóvüh³v½rGhütÃhtr²Dü)8PGGDIB!ö
A Multi-relational Rule Discovery System 0DKPXW8OXGD÷0HKPHW57ROXQDQG7KXUH(W]ROG 1
LION Bioscience Ltd., Compass House, 80-82, Newmarket Road, Cambridge, CB5 8DZ, United Kingdom ^PDKPXWXOXGDJWKXUHHW]ROG`#XNOLRQELRVFLHQFHFRP 2 $WLOLP8QLYHUVLW\, Dept. of Computer Engineering, ,QFHN$QNDUD, 7XUNH\ WROXQ#DWLOLPHGXWU KWWSFPSHHPXHGXWUULOD/
Abstract. This paper describes a rule discovery system that has been developed as part of an ongoing research project. The system allows discovery of multirelational rules using data from relational databases. The basic assumption of the system is that objects to be analyzed are stored in a set of tables. Multirelational rules discovered would either be used in predicting an unknown object attribute value, or they can be used to see the hidden relationship between the objects’ attribute values. The rule discovery system, developed, was designed to use data available from any possible ‘connected’ schema where tables concerned are connected by foreign keys. In order to have a reasonable performance, the ‘hypotheses search’ algorithm was implemented to allow construction of new hypotheses by refining previously constructed hypotheses, thereby avoiding the work of re-computing.
1 Introduction Most of the current data mining algorithms are designed to use data from a single table. They require each object to be described by a fixed set of attributes. Compared to a single table of data, a relational database containing multiple tables makes it possible to represent more complex and structured data. In addition, today, a significant amount of scientific data is stored in relational databases. For these reasons, it is important to have discovery algorithms running for relational data in its natural form without requiring the data to be viewed in a single table. A relational data model consisting of multiple tables may represent several object classes, i.e. within a schema while one set of tables represents a class of object, a different set of tables may represent another class. Before starting discovery processes, users should analyze the schema and select the list of tables that represents the kind of objects they are interested in. One of the selected tables will be central for the objects and each row in the table should correspond to a single object in the database. In the previous multi-relational data mining publications, this central table is named as ‘target table’ in [1] and [2], ‘primary table’ in [3], ‘master relation’ in [4], and ‘hub table’ in [5]. $ 0 =⇒ µCi (Su ) = 1) =⇒ µC (Su ) = 1, where C is the weighted version of Ci .
3
Framework Description
This section gives an overview of the design and capabilities of the Flexible Querying Framework. 3.1
Implementation and Design Issues
FQF is a framework that allows flexible querying on traditional database systems. FQF was initially developed as an environment for evaluating new theories on flexible data-retrieval from existing database systems. However, due to the increasing performance of JDO implementations, FQF could as well be used as a query engine in practical applications. The main design goal of FQF is to keep it as general as possible, and therefore make it useful and usable for a wide range of users. The application programmer who makes use of FQF therefore has a lot of facilities at her disposal: – The underlying database model is free to choose, as long as there is a JDO implementation that supports it. If an application has to be written for an existing database system, FQF can be used in the majority of cases. The javax.jdo API and the API offered by the JDO vendor are used. – Different aggregation operators and weight interpretations are implemented and can be used, according to the data administrator’s preference. More than one set of aggregation operators and weight interpretations can be supported by the same application. – End-user interfaces for an FQF application are virtually without limitations. They can vary from command-line interfaces, to form-based and graphical interfaces. These interfaces must only specify the filter condition and candidate class, and deliver these to the framework. Additional functionalities that are to be implemented in FQF are described in section 4. 3.2
The FastObjects JDO Implementation
As stated before, JDO is a new technology and is therefore not widely spread to this day. Because JDO is just a specification, its success will largely depend on the implementations that are available. In FQF, the FastObjects j1 Community Edition is used. It implements the obliged functionalities of JDO. In the future, several optional features will be available too. Some JDO implementations have extra features, to aid in the ease of use of JDO, but these are better left aside, for portability’s sake, because other implementations probably don’t offer these.
A Flexible Querying Framework (FQF)
3.3
265
Querying Methods
To perform a query with FQF, conditions that are passed to an interface must be transformed in a JDOQL filter, and the class of the candidate instances must be declared in some way. The latter will be called resultclass form here on. When a simple proposition is constructed, a weight can be attached to it. As previously mentioned, different weight interpretations are available and can be chosen from. These propositions can be combined into composed propositions, using one of the aggregation operators AND, OR and NOT. Again, for these, several definitions are available. Following is an overview of the different kinds of single propositions that can be constructed with FQF. For clarity, the possible operators, the possible left operands and right operands are considered separately. For the left operand, the following kinds of propositions are possible, each of which will be clarified with a concrete example: – Any attribute of the resultclass can be in the left operand of the proposition, whether it’s a Java primitive type, a Java standard library type or a userdefined type, as long as the FastObjects implementation supports it. E. g. with resultclass ‘Person’ following queries are possible: ‘getName== “Bert Callens”’ for the name attribute, ‘getAddress== myAddress’ for the address attribute. – Queries aren’t limited to an attribute depth of one when going down the attribute hierarchy starting from the resultclass. E. g. with resultclass ‘Person’: ‘getAddress.getStreet== “streetx”’ is valid. For the operator facilities, there is a distinction between traditional operators and flexible operators. Traditional operators are used in this way: – When comparing primitive data types, operators like ==, , where vi = ρx (P i). (ii) P denotes the equivalence relation on U defined by the values of P, that is, P = − − {(x, y)|x, y ∈ U ∧ ρ x (P ) = ρ y (P )}. − − (iii) [x]P = {y | ρ x (P ) = ρ y (P )} denotes a block of P. (iv) U/P denotes the set of blocks of P. to that of U/DEC, A decision algorithm, induced from S, relates the elements of U/CON which is also main concern of a TC system. ILA considers the combinations of features whose distinct values clearly distinguishes a class from the others, under the assumption that training set is consistent. On other words, for P ⊆ CON, if [x]P ⊆ [y]DEC , where x, y ∈ U, we say that the de−
scription of corresponding feature(s), ρ x (P ), functionally determines the class label, ρy (DEC). ILA uses stepwise forward technique to search instance space. That is, ILA inspects the partitions of U with respect to individual features, |P | = 1. If description of a feature, say ρx (c ∈ P ), functionally determines a class label, say d ∈ DEC, then a rule is generated and all instances covered by that rule,i.e. [x]c˜, is marked. This process is continued for P = 2, 3, ... over unmarked instances and ILA stops when all instances of U are marked. It is evident from our discussion that ILA applies stepwise forward technique to select an equivalence block of feature(s) included in that of a class, which provides basis for generating certain rules. It is worth stating that ILA with version of ILA-2[3] operates on incomplete features, where documents are non-exclusively partitioned by class labels. On other words, ILA is able to generate almost true rules when a document is attributed to more than one class label. 2.2
Feature Selection
Feature selection is the problem of choosing a small subset of features that is necessary and sufficient to describe target concepts. We consider in this study two measurements: Information Gain (IG) and χ2 (CHI). In the following formulas, probabilities are interpreted as usual on an event space of documents (e.g., P (tk , ci ) thus means the probability that, for a random document d, k th feature tk , does not occur in d and d belongs to category ci ), and are estimated by counting occurrences in the training set[1]. Information Gain (IG). One of the most popular feature selection method that has been applied for TC. It takes into account the term goodness criterion, which is basically equivalent to the number of bits of information gained for a category prediction by knowing the presence or absence of a feature in a document [1]. Let {ci }m i=1 denote the set of categories in the target space. The information gain of feature t is defined as: IG(tk , ci ) = P (tk , ci ) · log
P (tk , ci ) P (tk , ci ) + P (tk , ci ) · log P (tk ) · P (ci ) P (tk ) · P (ci )
(3)
The above formula measures the goodness of a feature globally with respect to all categories on average.
304
H. Sever, A. Gorur, and M.R. Tolun
Simplified χ2 (CHI). χ2 (Chi-square) is a measure of association. It checks whether there is a certain degree of dependence between feature and category. By selecting the features with highest χ2 -values, one can determine the most distinctive features of given a category. For the feature selection in our experiments we have used equation 4 (see [1] for further discussions). sχ2 (tk , ci ) = P (tk , ci ) · P (tk , ci ) − P (tk , ci ) · P (tk , ci )
3
(4)
Effectiveness Measures
In order to measure the performance of a text classifier, we use text categorization effectiveness measures. There are a number of effectiveness measures employed in evaluating text categorization algorithms. Most of these measures are based on the contingency table model. Consider a system that is required to categorize n documents by a query, the result is an outcome of n binary (or multi-level) decisions from which a 2X2 dichotomous table is generated. In this table, the cell values show the number of documents being predicted as relevant and also truly relevant (say a), relevant and also truly nonrelevant (say b), non-relevant and also truly relevant (say c), and non-relevant and also truly non-relevant (say d). In our experiment, the performance measures are based on precision and recall whose values are computed as a/(a + b) and a/(a + c). Usually a single composite recall-precision graph is reported reflecting the average performance of all individual queries in the system. Two average effectiveness measures, widely used in the literature, are: Macro-average and Micro-average [6]. In information retrieval, Macro-average is preferred in evaluating query-driven retrieval, while in text categorization Micro-average is preferred. Consider a system with n documents and q queries. Then there are q dichotomous tables each of which represents the outcomes of two-level decisions (relevant or nonrelevant) by the filtering system (predicted label) and the user/expert (actual label) when a query is evaluated against all n documents. Macro-average computes precision and recall separately from the dichotomous tables for each query, and then computes the mean of these values. Micro-average, on the other hand, adds up the q dichotomous tables all together, and then precision and recall are computed. For the purpose of plotting a single summary figure for recall versus precision values, an adjustable parameter is used to control assignment of documents to profiles (or categories in text categorization). Furthermore recall and precision values at different parameter settings are computed to show trade-off between recall and precision. This single summary figure is then used to compute what is called breakeven point, which is the point at which recall is approximately equal to precision [6]. It is possible to use linear interpolation to compute the breakeven point between recall and precision points.
4 The Experiment In this section, we describe the experimental set up in detail. First, we describe how the Reuters-21578 dataset is parsed and the vocabulary for indexing is constructed. Upon discussion of our approach to training, the experimental results are presented.
Text Categorization with ILA
4.1
305
Reuters-21578 Data Set and Text Representation
To experimentally evaluate the proposed information filtering method, we have used the corpus of Distribution 1.0 of Reuters-21578 text categorization test collection 1 . This collection consists of 21,578 documents selected from Reuters newswire stories. The documents of this collection are divided into training and test sets. Each document has five category tags, namely, EXCHANGES, ORGS, PEOPLE, PLACES, and TOPICS. Each category consists of a number of topics that are used for document assignment. We restrict our study to only TOPICS category. To be more specific, we have used the Modified Apte split of Reuters-21578 corpus that has 9,603 training documents, 3,299 test documents, and 8,676 unused documents. The training set was reduced to 7,775 documents as a result of screening out training documents with empty value of TOPICS category. There are 135 topics in the TOPICS category, with 118 of these topics occurring at least once in the training and test documents2 . Each of the three topics out of 118 ones has been assigned to only one document in the test set. We have chosen to experiment with all of these 118 topics despite the fact that three topic categories with no occurrence of training set automatically degrades system performance. We have produced a dictionary of single words excluding numbers as a result of pre-processing the corpus including performing parsing and tokenizing the text portion of the title as well as the body of both training and unused documents. We have used a universal list of 343 stop words to eliminate functional words from the dictionary 3 . The Porter stemmer algorithm was employed to reduce each remaining words to word-stems form 4 . Since any word occurring only a few times is statistically unreliable, the words occurring less than five times were eliminated. The remaining words were then sorted in descending order of frequency. Our categorization framework is based on the inductive classifier model in which documents and queries are represented as vectors of binary features as described in previous section. A category is assigned to a test document, if distinctive features of the category is included in the descriptive features of that document with some degree of certainty. 4.2 Training In contrast to information retrieval systems, in text categorization systems, we have neither a retrieval output nor a user query. Instead, we have a number of topics and for each topic the document collection is partitioned into training and test cases. The training set contains only positive examples of a topic. In this sense, the training set is not a counterpart of the retrieval output due to the fact that we do not have any negative examples. We can, however, construct a training set for a topic that consists of positive 1
Reuters-21578 collection is available at: http://www.research.att.com/˜lewis. In the description of the Reuters-21578 read-me file it was stated that the number of topics with one or more occurrences in TOPICS category is 120, but we have found only 118. The missing two topics were assigned to unused documents. 3 The stop list is available at: http://www.iiasa.ac.at/docs/R Library/libsrchs.html. 4 The source code for the Porter Algorithm is found at: http://ils.unc.edu/keyes/java/porter/index.html. 2
306
H. Sever, A. Gorur, and M.R. Tolun
and negative examples, under the plausible assumption that any document considered as positive example for the other topics and not in the set of positive examples of the topic at hand is a candidate for being a negative example of this topic. The maximum number of positive examples per topic in the corpus is 2877 and the average is 84. The size and especially the quality of the training set is an important issue in generating an induction rule set. In an information routing study [7], the learning method was not applied to the full training set but rather to the set of documents in the local region for each query. The local region for a query was defined as the 2000 documents nearest to the query, where similarity was measured using the inner product score to the query expansion of the initial query. Also, in [2] the rules for text categorization were obtained by creating local dictionaries for each classification topic. Only single words found in documents on the given topic were entered in the local dictionary. In our experiment, the training set for each topic consists of all positive examples while the negative data is sampled from other topics. The reason for including the entire set of positive examples is based on the conjecture that larger the number of positive examples are more effective induction rules, if preventive steps are incorporated in the production of rules to avoid from the overfitting problem. Additionally, the result published by Dumais et al. [8] for the Reuters-21578 data shows that with respect to micro-averaged score of the SVM (Support Vector Machine) over multiple random samples of training sets for the top 10 categories with varying sample size, but keeping size of negative data the same, performance of the SVM was degraded from 92% to 72.6% while the size was reduced from whole training set down to 1%.Another important finding reported in that study shows that performance of the SVM becomes somewhat unstable when a category has fewer than 5 positive training examples. In our previous study[4], we have found that the quality of the induction is effectively enhanced best when proportion of negative data is in the range of 50%-80%. Therefore we fixed the size of negative sample to 50% of the positive set. In this study, we preferred IG over CHI measure for the following reason. We fixed the size of features to 100 and produced production rules with IG and CHI separately. In the result, we have found that precision (or accuracy) of the rules with IG is 3% better than that of rules with CHI, which is not considered as statistically important difference to pick up one over another. 4.3
Results
As a comparative study, Table 1 presents results of ILA and other six inductive algorithms that were experimented on Reuters-21578 dataset [8,4]. Findsim method is a variant of Rocchio’s method for relevance feedback. The weight of each term is the average (or centroid) of its weight in positive instances of the topic. The SDA is based on utilizing preference relation through Steepest Descent Algorithm [4]. The list of names such as NBayes, BayesNets, Trees, and SVM in Table 1 stand for Naive Bayes, Bayes Nets, Decision Trees, and Linear Support Vector Machines methods, respectively. For further details of these methods the reader is referred to [8]. ILA rules with 20 features and with penalty factor of 2 ,i.e., a/b, are produced. ILA yields comparable results when compared with others. It represents, however, the categories with the smallest number of features.
Text Categorization with ILA
307
Table 1. Comparing results with other five inductive algorithms. Breakeven is computed on top 10 topics and on overall 118 topics. Topic Findsim NBayes BayesNets Trees SDA SVM ILA earn 92.9% 95.9% 95.8% 97.8% 98.5% 98.0% 95.4% acq 64.7% 87.8% 88.3% 89.7% 95.9% 93.6% 79.2% money-fx 46.7% 56.6% 58.8% 66.2% 78.4% 74.5% 61.4% grain 67.5% 78.8% 81.4% 85.0% 90.6% 94.6% 80.1% crude 70.1% 79.5% 79.6% 85.0% 86.5% 88.9% 86.5% trade 65.1% 63.9% 69.0% 72.5% 76.06% 75.9% 74.0% interest 63.4% 64.9% 71.3% 67.1% 77.29% 77.7% 72.4% wheat 68.9% 69.7% 82.7% 92.5% 82.2% 91.9% 79.4% ship 49.2% 85.4% 84.4% 74.2% 88.1% 85.6% 69.7% corn 48.2% 65.3% 76.4% 91.8% 82.36% 90.3% 71.0% Avg. Top 10 64.6% 81.5% 85.0% 88.4% 87.99% 92.0% 76.9% Avg All Cat. 61.7% 75.2% 80.0% N/A 81.28% 87.0% 71.4%
Acknowledgments. This research is partly supported by TUBITAK-EEEAG (The Scientific and Technical Research Council of Turkey) with the grant of 199E003.
References 1. Fabrizio Sebastiani. Machine learning in automated text categorization. ACM Computing Surveys, 34(1):1–47, 2002. 2. Chidanand Apte, Fred Damerau, and Sholom M. Weiss. Automated learning of decision rules for text categorization. Information Systems, 12(3):233–251, 1994. 3. M.R. Tolun H. Sever M. Uludag and S.M. Abu-Soud. Ila-2: An inductive learning algorithm for knowledge discovery. Cybernetics and Systems: An International Journal, 30(7):609–628, Oct-Nov 1999. 4. A.H. Alsaffar, J.S. Deogun, and H. Sever. Optimal queries in information filtering. In Z.W. Ras, editor, 12th International Symposium on Methodologies for Intelligent Systems (ISMIS’00), volume 1932 of Lecture Notes in Artificial Intelligence, pages 435–443. Springer-Verlag, 2000. 5. Thorsten Joachims. Text categorization with support vector machines: learning with many relevant features. In Claire N´edellec and C´eline Rouveirol, editors, Proceedings of ECML98, 10th European Conference on Machine Learning, volume 1398, pages 137–142. Springer Verlag, 1998. 6. D. D. Lewis. Evaluating Text Categorization. In Proceedings of Speech and Natural Language Workshop, pages 312–318. Morgan Kaufmann, 1991. 7. Hinrich Schutze, David A. Hull, and Jan O. Pedersen. A comparison of classifiers and document representations for the routing problem. In Research and Development in Information Retrieval, pages 229–237, 1995. 8. Susan Dumais, John Platt, David Heckerman, and Mehran Sahami. Inductive learning algorithms and representations for text categorization. In Proceedings of the seventh international conference on Information and knowledge management, pages 148–155. ACM Press, 1998.
2QOLQH0LQLQJRI:HLJKWHG)X]]\$VVRFLDWLRQ5XOHV 0HKPHW.D\D1DQG5HGD$OKDMM2
'HSDUWPHQWRI&RPSXWHU(QJLQHHULQJ)ÕUDW8QLYHUVLW\(OD]Õ÷785.(< ND\D#ILUDWHGXWU 'HSDUWPHQWRI&RPSXWHU6FLHQFH8QLYHUVLW\RI&DOJDU\&DOJDU\$%&$1$'$ DOKDMM#FSVFXFDOJDU\FD
$EVWUDFW 0LQLQJ XVHIXO LQIRUPDWLRQ DQG KHOSIXO NQRZOHGJH IURP GDWD WUDQVDFWLRQV LV HYROYLQJ DV DQ LPSRUWDQW UHVHDUFK DUHD &XUUHQW RQOLQH WHFKQLTXHV IRU PLQLQJ DVVRFLDWLRQ UXOHV LGHQWLI\ WKH UHODWLRQVKLS DPRQJ WUDQVDFWLRQVXVLQJELQDU\YDOXHV+RZHYHUWUDQVDFWLRQVZLWKTXDQWLWDWLYHYDOXHV DUH FRPPRQO\ HQFRXQWHUHG LQ UHDOOLIH DSSOLFDWLRQV ,Q WKLV SDSHU ZH DGGUHVV WKLV SUREOHP E\ LQWURGXFLQJ D IX]]\ DGMDFHQF\ ODWWLFH DQG WKHQ LQWHJUDWH WKH ODWWLFH VWUXFWXUH ZLWK OLQJXLVWLF ZHLJKWV LQ D ZD\ WR UHIOHFW WKH LPSRUWDQFH RI LWHPV([SHULPHQWVFRQGXFWHGXVLQJV\QWKHWLFGDWDVKRZWKHHIIHFWLYHQHVVRIWKH SURSRVHGPHWKRGIRURQOLQHJHQHUDWLRQRIZHLJKWHGIX]]\DVVRFLDWLRQUXOHV
,QWURGXFWLRQ 'DWD PLQLQJ LV WKH SURFHVV RI H[WUDFWLQJ SUHYLRXVO\ XQNQRZQ DQG SRWHQWLDOO\ XVHIXO KLGGHQ SUHGLFWLYH LQIRUPDWLRQ IURP ODUJH DPRXQWV RI GDWD 'LVFRYHULQJ DVVRFLDWLRQ UXOHV LV RQH RI WKH VHYHUDO GDWD PLQLQJ WHFKQLTXHV GHVFULEHG LQ WKH OLWHUDWXUH $VVRFLDWLRQV DOORZ FDSWXULQJ DOO SRVVLEOH UHJXODULWLHV WKDW H[SODLQ WKH SUHVHQFH RI VRPHDWWULEXWHVDFFRUGLQJWRWKHSUHVHQFHRIRWKHUDWWULEXWHVLQWKHVDPHWUDQVDFWLRQ $Q DVVRFLDWLRQ UXOH LV GHILQHG DV DQ LPSOLFDWLRQ ;⇒@SURSRVHGDPHWKRGEDVHGRQWKH DGMDFHQF\ODWWLFHLWZRUNVIRUDVSHFLDOFDWHJRU\RITXDQWLWDWLYHDVVRFLDWLRQUXOHV ,QWKLVSDSHURXUDLPLVWRFRPELQHWKHDGYDQWDJHVRIIX]]LQHVVZHLJKWHGPLQLQJ DQG OLQJXLVWLF PLQLPXP VXSSRUW DQG PLQLPXP FRQILGHQFH FRQFHSWV LQ RUGHU WR JHQHUDWHRQOLQHDVVRFLDWLRQUXOHVDFFRUGLQJWRWKUHVKROGDQGZHLJKWFKDQJHV)RUWKLV SXUSRVHZHSURSRVHZKDWZHFDOOZHLJKWHGIX]]\ODWWLFHZKLFKLVFRQVLGHUHGDVRQH RIRXUPDLQFRQWULEXWLRQV 7KH UHVW RI WKLV SDSHU LV RUJDQL]HG DV IROORZV )X]]\ UHSUHVHQWDWLRQ RI LWHP LPSRUWDQFHPLQLPXPVXSSRUWDQGPLQLPXPFRQILGHQFHDUHJLYHQLQ6HFWLRQ7KH ZHLJKWHGIX]]\DGMDFHQF\ODWWLFHLVGHILQHGLQ6HFWLRQ2XUDSSURDFKRIH[WUDFWLQJ UXOHVIURPWKHZHLJKWHGIX]]\ODWWLFHLVGHVFULEHGLQ6HFWLRQ([SHULPHQWDOUHVXOWV DUHSUHVHQWHGLQ6HFWLRQ6HFWLRQLVWKHFRQFOXVLRQV
)X]]\5HSUHVHQWDWLRQRI,WHP,PSRUWDQFH0LQLPXP6XSSRUW DQG0LQLPXP&RQILGHQFH In general, for a rule to be interesting, it should have enough support and high confidence value, i.e., larger than user specified thresholds. However, as weighted mining is concerned, items are assigned weights to reflect their importance and
0.D\DDQG5$OKDMM
weighted support and confidence values are employed in deciding on interesting association rules. µPLQVXS 9HU\ 9HU\ 8QLPSRUWDQW 2UGLQDU\ ,PSRUWDQW ,PSRUWDQW 8QLPSRUWDQW
:HLJKWRI DQLWHP
Fig. 1. Membership functions that represent the weights of an item
Most data PLQLQJ algorithms set weights of items as numerical values, with the weight of an attribute varies between 0 and 1, according to user’s experience or intuition; “0” means the attribute will be neglected and “1” identifies the most interesting attribute for the user. Most conventional data mining algorithms set the weights of items at numerical values. However, based on our understanding, the measure showing the importance of an item should be expressed as linguistic term in order to be more flexible and more understandable for humans. For this purpose, we use fuzzy sets to represent the weights of items. The membership functions used to represent the weight of an item are given in Figure 1, which shows that the weight of an item can take 5 different linguistic terms and the membership functions have uniform structure. 7KHLPSRUWDQFHRIWKHFRQVLGHUHGDWWULEXWHVLVQRWRQO\DQLPSRUWDQWPHDVXUHRI LQWHUHVWLQJQHVVEXWDOVRDZD\WRSHUPLWXVHUVWRFRQWUROWKHPLQLQJUHVXOWVE\WDNLQJ VRPHVSHFLILFDFWLRQV µPLQVXS 9HU\/RZ
/RZ
0LGGOH
+LJK
9HU\+LJK
0LQLPXP 6XSSRUW
)LJ0HPEHUVKLSIXQFWLRQVWKDWUHSUHVHQWPLQLPXPVXSSRUWYDOXHV
,QWKLVVWXG\ZHDOVRXVHGOLQJXLVWLFPLQLPXPVXSSRUWDQGPLQLPXPFRQILGHQFH YDOXHV LQ D ZD\ VLPLODU WR LWHP LPSRUWDQFH 7KXV LQVWHDG RI D VKDUS ERXQGDU\ ZH DFKLHYHG D ERXQGDU\ ZLWK FRQWLQXRXV LQWHUYDO RI PLQLPXP VXSSRUW RU FRQILGHQFH )LJXUHVKRZVWKHPHPEHUVKLSIXQFWLRQVGHILQHGIRUPLQLPXPVXSSRUW
)X]]\$GMDFHQF\/DWWLFH $FFRUGLQJWR$JJDUZDODQG<X>@DQLWHPVHW;LVDGMDFHQWWRDQLWHPVHW@ :+ $X DQG .&& &KDQ ³$Q (IIHFWLYH $OJRULWKP IRU 'LVFRYHULQJ )X]]\ 5XOHV LQ 5HODWLRQDO'DWDEDVHV´3URFHHGLQJVRI,(((,QWHUQDWLRQDO&RQIHUHQFHRQ)X]]\6\VWHPV SS± >@ & + &DL : & )X & + &KHQJ DQG : : .ZRQJ ³0LQLQJ $VVRFLDWLRQ 5XOHV ZLWK :HLJKWHG,WHPV´3URFHHGLQJVRI,'($6SS±
2QOLQH0LQLQJRI:HLJKWHG)X]]\$VVRFLDWLRQ5XOHV >@ .&& &KDQ DQG :+ $X ³0LQLQJ )X]]\ $VVRFLDWLRQ 5XOHV´ 3URFHHGLQJV RI $&0 &,.0/DV9HJDVSS± >@ %& &KLHQ =/ /LQ DQG 73 +RQJ ³$Q (IILFLHQW &OXVWHULQJ $OJRULWKP IRU 0LQLQJ )X]]\ 4XDQWLWDWLYH $VVRFLDWLRQ 5XOHV´ 3URFHHGLQJV RI ,)6$ :RUOG &RQJUHVV DQG 1$),36,QWHUQDWLRQDO&RQIHUHQFH9ROSS± >@ $:&)XHWDO³)LQGLQJ)X]]\6HWVIRUWKH0LQLQJRI$VVRFLDWLRQ5XOHVIRU1XPHULFDO $WWULEXWHV´3URFHHGLQJVRIWKH,QWHUQDWLRQDO6\PSRVLXPRI,QWHOOLJHQW'DWD(QJLQHHULQJ DQG/HDUQLQJSS±2FW >@ $ *\HQHVHL ³$ )X]]\ $SSURDFK IRU 0LQLQJ 4XDQWLWDWLYH $VVRFLDWLRQ 5XOHV´ 78&6 7HFKQLFDO5HSRUW1R >@ .+LURWDDQG:3HGU\F]³/LQJXLVWLF'DWD0LQLQJDQG)X]]\0RGHOOLQJ´3URFHHGLQJV RI,(((,QWHUQDWLRQDO&RQIHUHQFHRQ)X]]\6\VWHPV9ROSS± >@ -+ +ROODQG $GDSWDWLRQ LQ 1DWXUDO DQG $UWLILFLDO 6\VWHPV 7KH 0,7 3UHVV &DPEULGJH 0$0,73UHVVHGLWLRQ)LUVWHGLWLRQ8QLYHUVLW\RI0LFKLJDQ3UHVV >@ 73 +RQJ &6 .XR DQG 6& &KL ³$ IX]]\ GDWD PLQLQJ DOJRULWKP IRU TXDQWLWDWLYH YDOXHV´ 3URFHHGLQJV RI WKH ,QWHUQDWLRQDO &RQIHUHQFH RQ .QRZOHGJH%DVHG ,QWHOOLJHQW ,QIRUPDWLRQ(QJLQHHULQJ6\VWHPVSS± >@ 73+RQJ&6.XRDQG6&&KL³0LQLQJ$VVRFLDWLRQ5XOHVIURP4XDQWLWDWLYH'DWD´ ,QWHOOLJHQW'DWD$QDO\VLV9ROSS± >@ 73+RQJ0-&KLDQJDQG6/:DQJ³0LQLQJIURP4XDQWLWDWLYH'DWDZLWK/LQJXLVWLF 0LQLPXP6XSSRUWVDQG&RQILGHQFHV´3URFHHGLQJVRI,(((,QWHUQDWLRQDO&RQIHUHQFHRQ )X]]\6\VWHPVSS± >@ + ,VKLEXFKL 7 1DNDVKLPD DQG 7 @ 0 .D\D 5 $OKDMM ) 3RODW DQG $ $UVODQ ³(IILFLHQW $XWRPDWHG 0LQLQJ RI )X]]\ $VVRFLDWLRQ5XOHV´3URFHHGLQJVRI'(;$ >@ &0 .XRN $: )X DQG 0+ :RQJ ³0LQLQJ IX]]\ DVVRFLDWLRQ UXOHV LQ GDWDEDVHV´ 6,*02'5HFRUG9RO1RSS± >@ 5- 0LOOHU DQG @ 5 6ULNDQW DQG 5 $JUDZDO ³0LQLQJ TXDQWLWDWLYH DVVRFLDWLRQ UXOHV LQ ODUJH UHODWLRQDO WDEOHV´3URFHHGLQJVRI$&06,*02'0RQWUHDO&DQDGDSS± >@ 55 N represents the attributes that test the existence of the domains that appear in RHS of the protein-protein interaction. We can categorize the types of rules that the ID3 algorithm can produce from such an example set into a few categories: 1. 2. 3. 4. 5.
∀ α : α ∈ LHS → + ∀ α : α ∈ RHS → + ∀α : (∃α : α ∈ LHS & α = 1) & (∃α : α ∈ RHS & α = 1) → + α ∈ (LHS or RHS) & α = 0 → + α ∈ (LHS or RHS) & α ∈ {0, 1} → −
Where α is the attributes included in the rule and α ∈ LHS means that α is an attribute seen on LHS of the interactions, + and − determines the presence of interaction. Among these rules, the most interesting type is the third one. First two types include information about only one side domains, namely left or right hand side. Fourth type has all its attributes set to zero, meaning that if a set of domains does not exists, then interaction occurs. This is very hard to interpret, because it is about rules that depend on the non-existence of some domains. Last type is the negative interaction rules, so they should not be considered as interaction rules. Negative examples should also be presented to the ID3 learning algorithm. Since there is no database of non-interacting proteins, we have used a method that shuffles the domains and creates artificial proteins while preserving the domain occurrence frequencies within the generated proteins. This provides a better way of creating a negative example set instead of taking the complement of the interaction data, because the resulting proteins are more like natural and a small error rate is expected on classification of these proteins. This is because we don’t expect a large number of proteins from a random set to interact with each
Application of Data Mining Techniques
321
other. However, the best approach would be to get the results of biological experiments that were carried out to find the protein-protein interactions. Extracting the negative samples from those experiments would give the best results because they are natural data, obtained from biological experiments. Nevertheless, ID3 learning algorithm is known to be robust to errors in the training set. So, we expect that this randomized artificial protein interactions will not contribute to the error too much.
4
Databases Used
DIP [1] database stores information about protein-protein interactions that are confirmed experimentally. Over 17000 interactions are listed in DIP database, which can be used freely for academic purposes. DIP is presented in both html and xml format. Information in the xml file is basically divided into two major sections: nodes and edges. Nodes are presented by their IDs and various other fields like cross-links and features. Edges (which represent the interactions) are listed with two nodes per entry. Number of nodes listed in the database is 6807 and the number of edges listed is 17693. Yeast Database was created by Uetz et al. [6] and presented in the Curagen’s web site in html format. All of the interactions presented here are identified experimentally by high throughput yeast two-hybrid screens on open reading frames of Saccharomyces Cerevisiae genome sequence. Every protein listed in the database is associated with its interacting pairs and also cross-links to other databases, also with a visualization that shows the role of the protein in the whole genetic network. Pfam is the protein family database [3]. It supports searching by keywords or sequences and retrieves the domains (families) of a given protein. We have used Pfam in order to get the domain decomposition of the proteins, by extracting data from a large text file. This is the swisspfam part of Pfam, which is keyed by the Swiss-Prot [7] names and accession numbers of the proteins.
5
Results and Discussion
23910 interactions were given to the association rule miner and various experiments were carried out with varying support and confidence values. As the support and confidence values get more strict, number of rules found by the algorithm decreases. However, it is important to decide on which support and confidence value pair gives the closest match to a set of rules which consists of logical, valuable rules. For example, number of rules generated given a minimum support of 0.1% and a minimum confidence of 10% is around 130000, which is very high. Indeed, when examined more closely, one can see that most of these rules are trivial, or does not mean much at all, because their dependability measures are very low. In order to find more meaningful rules, one should increase the minimum support and confidence variables. However, then, we face the risk of missing some valuable, but not so frequent rules. Below is a table which shows
322
A. Kocatas, A. Gursoy, and R. Atalay
the change in the number of rules with fixed support and varying confidence and varying support and fixed confidence. Support Confidence Number of Rules 0.5 10 416 0.5 50 398 0.5 90 376 0.1 50 111872 0.5 50 398 0.9 50 4 It is seen from the table above that the number of rules is gradually decreasing as the minimum confidence requirement is increased linearly. The same is true with the minimum support value, but the table proves that the support is a more constraining variable than the confidence value. As the support increases linearly, a rapid decrease in the number of rules is observed. Still in the search for optimum support and confidence pair for our data, we look for other conclusions. For example, looking at the number of composite rules may give us an idea about the dependability of the resulting rules. Composite rules, by their nature, contain both RHS and LHS domains in either side of the rule. Although we have observed composite rules, it is hard to associate a biologically meaningful explanation for such rules. A careful look at the data, however, leads to an explanation. Assume that the rules L1 → R1 and L1 → R1 R2 have been already produced. Then the algorithm will produce L1 R1 → R2 as well since the following is always true: support(L1 ) ≥ support(L1 R1 ) ≥ support(L1 R1 R2 ). It was observed that the number of composite rules also gradually decreases with increasing support and confidence requirements. In our experiments, it was observed that beyond 0.2% support and 20% confidence, these rules totally disappeared from the resulting rule sets. On the other hand, the number of useful rules detected by the ID3 algorithm were far fewer than that of the association rule mining. One of the rules that were detected by the ID3 algorithm was: P F 01423 L = 1, P F 01423 R = 1 → +. This rule briefly says that if both the right hand side and the left hand side proteins contain PF01423 domain, they will interact. Indeed, this domain is a domain that is specific to Sm proteins. In the literature, Sm proteins are known to be involved in mRNA splicing. Seven Sm Proteins form a complex around the Sm site to splice the mRNA. This information verifies that the rule is indeed, correct. Following are two of the rules that were detected by association rule mining method: P F 00227 L → P F 00227 R P F 00069 L → P F 00134 R PF00227 of rule 1 is the Proteosome A-type and B-type domain annotated in Pfam. It is also claimed that members of this domain form a large ring based complex, which verifies that proteins that contain this domain interact with each other. Rule 2, on the other hand, is related with two distinct domains: PF00134
Application of Data Mining Techniques
323
is the cyclin, N-terminal domain and PF00069 is the protein kinase domain. It is mentioned in Pfam that cyclins regulate the cell division cycle in eukaryotes and protein kinases form a complex with them.
6
Conclusion
In this paper, we have described and used two different methods to find rules about protein-protein interactions in domain decomposition level. It was observed that some of the rules found out by the techniques were indeed true and interactions among these domains were mentioned in the literature. It was also observed that not both techniques produce the same set of rules. In fact, association rule mining outperforms the ID3 method in the number of rules generated. However, it is difficult to find the correct support and confidence values for the association rule mining algorithm. As a future work, different features can be incorporated along with the domain decomposition of the proteins. We believe, for example, motifs and amino acid patterns, as well as expression profiles and micro-array data would yield to interesting rules about protein-protein interactions. From the biological side, rules that are generated frequently with reasonable support and confidence value pairs can be checked with laboratory experiments. By this way, we can understand if the method can discover novel protein-protein interactions.
References 1. Ioannis Xenarios, Lukasz Salwinski, Xiaoqun Joyce Duan, Patrick Higney, Sul-Min Kim and David Eisenberg: DIP, the Database of Interacting Proteins: a research tool for studying cellular networks of protein interactions. Nucleic Acids Research Vol. 30 No. 1 303–305, 2002 2. Rakesh Agrawal, Ramakrishnan Srikant: Fast Algorithms for Mining Association Rules. Proc. 20th Int. Conf. Very Large Data Bases, VLDB 1994 3. Bateman A, Birney E, Cerruti L, Durbin R, Etwiller L, Eddy SR, Griffiths-Jones S, Howe KL, Marshall M, Sonnhammer EL: The Pfam Protein Families Database. Nucleic Acids Research, Vol. 30 276–280, 2002 4. R. Agrawal, T. Irnielinski, and A. Swami: Mining Association Rules between Sets of Items in Large Databases. Proceedings of A CM SIGMOD, 207–216, May 1993 5. T. Oyama, K. Kitano, K. Satou and T. Ito: Extraction of knowledge on proteinprotein interaction by association rule discovery, Bioinformatics. Vol. 18, no. 5 2002 6. Uetz et al: A comprehensive analysis of protein-protein interactions in Saccharomyces cerevisiae. Nature, Vol. 403. Page 623–627, 2000 7. Boeckmann B., Bairoch A., Apweiler R., Blatter M.-C., Estreicher A., Gasteiger E., Martin M.J., Michoud K., O’Donovan C., Phan I., Pilbout S., Schneider M: The SWISS-PROT protein knowledgebase and its supplement TrEMBL in 2003. Nucleic Acids Res, Vol. 31. Page 365–370, 2003 8. Christian Borgelt’s Software Page: http://fuzzy.cs.uni-magdeburg.de/˜borgelt/
A Heuristic Lotting Method for Electronic Reverse Auctions Uzay Kaymak, Jean Paul Verkade, and Hubert A.B. te Braake Erasmus University Rotterdam, Faculty of Economics P. O. Box 1738, 3000 DR, Rotterdam, The Netherlands
[email protected] Abstract. An increasing number of commercial companies are using online reverse auctions for their sourcing activities. In reverse auctions, multiple suppliers bid for a contract from a buyer for selling goods and/or services. Usually, the buyer has to procure multiple items, which are typically divided into lots for auctioning purposes. By steering the composition of the lots, a buyer can increase the attractiveness of its lots for the suppliers, which can then make more competitive offers, leading to larger savings for the procuring party. In this paper, a clustering-based heuristic lotting method is proposed for reverse auctions. Agglomerative clustering is used for determining the items that will be put in the same lot. A suitable metric is defined, which allows the procurer to incorporate various approaches to lotting. The proposed lotting method has been tested for the procurement activities of a consumer packaged goods company. The results indicate that the proposed strategy leads to 2–3% savings, while the procurement experts confirm that the lots determined by the proposed method are acceptable given the procurement goals.
1
Introduction
Electronic auctions are one of the more promising applications of e-business. In addition to the selling of many items through online auction sites (e.g. www.ebay.com), many industries also consider the use of reverse auctions for industrial sourcing and procurement. Nowadays, virtually every major industry has begun to use electronic sourcing (e–sourcing) and adopt online reverse auctions [1]. More than 40 specialized solutions providers such as the FreeMarkets Inc. offer e–sourcing platforms, services and the technology for online reverse auctions [2]. It has been estimated that the annual throughput in online reverse auctions is over $40 billion [1]. Price negotiation is one of the most time–intensive activities in the purchasing process [3]. Online reverse auctions can significantly speed up the pricing process. Apart from the gains in the time spent on purchasing, the popularity of the online reverse auctions stems from the fact that they can help reduce the purchasing costs. It has been reported in the literature that online reverse auctions can produce cost savings from 5% to 40% [4]. Furthermore, online auctions have the potential to restructure the procurer’s relation to its suppliers, for example, by allowing contact with more suppliers. Multiple issues must be considered for a successful online reverse auction, such as the auditing of the suppliers, training of the users, implementation of the technology A. Yazici and C. S¸ener (Eds.): ISCIS 2003, LNCS 2869, pp. 324–331, 2003. c Springer-Verlag Berlin Heidelberg 2003
A Heuristic Lotting Method for Electronic Reverse Auctions
325
and the specification of the auctioned items. One of the important considerations for maximizing the cost savings is the so–called lotting, the grouping of items that will be auctioned as a single entity. On one hand, the lots must be as attractive as possible for the suppliers, so that they will have the incentive to make good offers that will save money for the buyer. On the other hand, the buyer wants to ensure that all the items are bid for, even if some of them are of less interest for the suppliers. Therefore, the optimal lotting should balance the interests of the suppliers and of the buyer. This suggests that the lotting problem should be handled through negotiations between the suppliers and the buyer. There are indeed auctioning systems in development, which allow negotiation between different parties during the auctions [5]. However, many existing online reverse auction systems do not allow negotiations. Furthermore, one of the main advantages of sourcing through reverse auctions is that it saves time through the elimination of negotiations, and hence negotiations should never become a substantial aspect of online auctions. In other words, attention must be paid to the lotting strategy that will lead to the maximal benefits. In this paper, we investigate automated approaches to lotting that can support the lottig decisions of the procurement experts. We propose a clustering-based approach for lotting in online reverse auctions. Agglomerative clustering is used for determining the items that are put in the same lot. A suitable metric is defined, which allows the procurer to incorporate various approaches to lotting. We have applied the proposed lotting algorithm to a procurement campaign of a consumer packaged goods company. The results are encouraging, indicating that the proposed algorithm leads to 2–3% savings, while the procurement experts confirm that the lots determined by the proposed strategy are acceptable, given the procurement goals. The outline of the paper is as follows. Reverse auctions and the significance of lotting for reverse auctions are discussed in Section 2. An overview of clustering methods is given in Section 3. A lotting algorithm based on clustering is proposed in Section 4. The algorithm uses hierarchical clustering and a distance metric that allows the procurer to incorporate various approaches to lotting. The application of the proposed method in a procurement campaign of a consumer packaged goods company is discussed in Section 5, where the results from the proposed lotting method are compared to the solution provided by experts. Finally, conclusions are given in Section 6.
2
Reverse Auctions and Lotting
An auction is a mechanism to re-allocate goods or services to a set of market participants on the basis of bids and asks [6]. In general, there are two types of participants in an auction: the auctioneer and the bidders. In reverse auctions, the auctioneer is the buyer of a good or service, while the bidders are the suppliers of the good or service. An integral part of every auction are the auction rules, which consist of two parts: the bidding rules and the market clearing rule. The bidding rules define what the bidders may bid for and when they may place their bids. The market clearing rule defines when and how the allocation of items to bidders is decided, and what the bidders have to pay. The bids in a reverse auction evolve from large amounts to smaller amounts, hence making the bids more and more attractive for the auctioneer (i.e. the buyer). It is possible to design different reverse auction mechanisms, such as descending reverse auctions or sealed-bid reverse auctions [7]. In all these auctions, the suppliers bid for entities called lots.
326
U. Kaymak, J.P. Verkade, and H.A.B. te Braake
A lot is an item or a combination of items that the suppliers can bid for in its entirety. Lotting is the process of dividing the items into lots. Lotting is needed, because it is not efficient to auction all the items separately. A company may need to procure thousands of different items. It is then simply not time and cost efficient to auction the items separately. Furthermore, if the quantity of an item to be procured is very large, it may be desirable to divide the total amount into multiple lots. It is important to realize that lotting gives the procurer the possibility to influence the attractiveness of the auctioned items for the suppliers, balancing two (possibly conflicting) goals. On one hand, the lots must be as attractive as possible for the suppliers, so that they will have the incentive to make good offers that will save costs for the buyer. On the other hand, the buyer wants to ensure that all the items are bid for, even if some of them are of less interest for the suppliers. Let the items to be procured be represented by K–dimensional vectors xn , n = 1, . . . , N . Hence, each item is described by a vector of K features. The goal of lotting is to divide the vectors xn into I lots, so that some criterion (e.g. total procurement costs) is minimized, while the constraints imposed by the auctioneer are satisfied. Mathematically, minimize
I
fi (wi1 , . . . , wiN , x1 , . . . , xN )
i=1
such that gp (wi1 , . . . , wiN , x1 , . . . , xN ) ≤ 0, I
win = 1,
p = 1, . . . P,
n = 1, . . . , N.
i=1
In the above formulation, fi denotes the price for lot i, and gp are the constraints that may be imposed by the auctioneer. These constraints can be a result of the procurement approach, or they can be the result of boundary conditions. The decision variables are the allocation weights win ∈ {0, 1}, which indicate whether the item n is included in lot i. Despite the mathematical formulation of the lotting problem, it can not be solved as a mathematical optimization problem, because the functions fi are in general unknown. Therefore, expertise and heuristic–based approaches are often used for determining the lots. Note that the lotting problem is related to combinatorial reverse auctions, but it is not the same. In combinatorial auctions, all items are auctioned and the bidders can bid for subsets of the items, thereby revealing their preference information during the auction. Lotting is done before the auction, and hence much less information is available regarding the valuation of the items by the bidders. The auctioneer can use different strategies for lotting. For example, one can group all items with similar characteristics together. In this case, the suppliers may be able to exploit economies of scale, which could be reflected in their bids. However, there may be lots, which consist of rather unattractive items, for which the bids are very high. In that case, the overall procurement costs are not minimized. Alternatively, one may consider putting different types of items in the same lot. However, this may increase the complexity of production for the supplier, which may be reflected in their bids as increased costs. In the following sections, we investigate the use of clustering algorithms as a method for supporting the lotting decisions of procurers.
A Heuristic Lotting Method for Electronic Reverse Auctions
3
327
Data Clustering
When the allocation weights win are determined, the items to be procured are distributed over multiple lots. In this sense, the lotting problem could be interpreted as a segmentation problem. One of the methods that can be used for segmentation is clustering [8]. In clustering, a set of vectors is partitioned into several groups based upon similarity within the group and dissimilarity amongst the groups. There are two general types of clustering algorithms, the hierarchical clustering algorithms, and the objective function based (nonhierarchical) clustering algorithms. Objective function based clustering algorithms solve an optimization problem to partition the data set into a pre-determined number of groups (see e.g. [9]). In contrast, the hierarchical clustering techniques proceed by a series of successive divisions or mergers of data to determine the partitions. Hence, the number of clusters is not pre-determined. This gives the user the possibility to analyze the clustering results at different resolutions, without additional computational burden of re-clustering. Within the group of hierarchical clustering techniques, the most popular are the linkage algorithms. In linkage algorithms, the distance between all clusters is computed, and at each step the most similar clusters are merged. The linkage algorithms can be summarized as follows [10]. Algorithm 1. Given N items, place each item in a separate cluster. do Compute the distance between all possible cluster pairs. Merge the most similar (minimum distance) clusters. until all clusters are merged One obtains different linkage algorithms by modifying the way the distance between the clusters is measured. In the single linkage algorithm, the distance between two clusters is defined as the distance between their nearest members. In the average linkage algorithm, the distance between two clusters is defined as the average distance between pairs of the members in the two clusters. In the complete linkage algorithm, the distance between two clusters is defined as the distance between their farthest members. The result of hierarchical clustering can be represented graphically in a dendogram. A dendrogram is a special kind of tree structure that visualizes clusters as the branches in a tree. It is usual in a dendogram to convert the distance into similarity, which is normalized between 0 and 1. In that case, one can obtain different clusters by thresholding with different values of λ ∈ [0, 1] (see Fig. 2 for an example of a dendogram).
4
Clustering-Based Lotting
In this section, we propose a clustering-based lotting algorithm that can be incorporated in a procedure for reverse auctions. The lotting algorithm is based on hierarchical clustering in order to analyze the clustering results at different resolutions. We have chosen to use a complete linkage algorithm. Complete linkage algorithm has the attractive property that for merging two clusters, all the items in those clusters must be within a certain level of similarity to one another. Consequently, the complete linkage algorithm has a tendency to find relatively compact clusters composed of highly similar items. Furthermore, long
328
U. Kaymak, J.P. Verkade, and H.A.B. te Braake
chains of clusters are avoided, a disadvantage associated with single linkage and average linkage algorithms. After having selected a clustering algorithm, the distance metric to be used in the algorithm must be determined. A suitable metric is defined in consultation with procurement experts of a consumer packaged goods company. Procurement experts find it important to have an algorithm that can be tailored in various ways according to the knowledge and the expertise they have developed in many years. The experts have confirmed that, in many cases, they try to put items that look alike in the same lots. This is thought to provide economies of scale for the suppliers, who are then expected to reflect it in their bids. However, in some other cases, the experts want to place the items that are dissimilar in the same lots, for example because of policy reasons. This decision is taken on a feature-by-feature basis. Similarity of items in the same lot is required for some features, but it is not required for other features. Furthermore, some features take nominal or categorical values, while some other features take real values. Keeping these considerations in mind, the following distance metric is proposed for lotting. k∈FS αk δijk + k∈FS αk (1 − δijk ) , (1) dij = K k=1 αk where dij is the distance between item xi and xj , FS is the set of features where items are judged on their similarity (the items are judged on their dissimilarity in the complementary set), αk ∈ {0, 1} indicates whether feature k is of importance for the lotting problem considered and δijk indicates the distance between item xi and xj measured along feature k. The way δijk are computed depends on the type of the feature. For nominal or categorical features, δijk is given by δijk =
0 1
if xik = xjk , otherwise
(2)
while for continuous valued features it is given by δijk =
|xik − xjk | Rk
(3)
with Rk representing the range of feature k. Note that all distances are normalized in [0, 1], so that their complements indicate the similarity between the items xi and xj . Complete linkage algorithm is now applied with distance metric (1) after the analyst decides which features should be clustered on similarity and which features on dissimilarity. Once the clustering results are obtained, one need to select a threshold λ ∈ [0, 1], from which the final lot compositions are obtained. Figure 1 shows the difference between clustering on similarity and clustering on dissimilarity for a data set consisting of three groups described by two features. Note that similarity based clustering finds the three natural groups in the data, while dissimilarity based clustering results ensures that all clusters have members from the three natural groups present in the data. This is the type of behavior expected by procurement experts when the items in a lot should be dissimilar.
A Heuristic Lotting Method for Electronic Reverse Auctions
(a)
329
(b)
Fig. 1. A clustering example based on (a) similarity, and (b) dissimilarity.
5 Application We have tested the performance of the clustering-based lotting by using the distance metric (1) applied to data from the online procurement campaigns of a consumer packaged goods company. The data set consists of 913 different items that the company had to procure for packaging purposes. Each item is described by 45 variables regarding the brand characteristics, the geographical region, the quality of the material, the size of the packaging material, type of print, etc. All variables except for the required volume are nominal or categorized. The data set contains missing values, which we have treated as separate categories for each of the features. Twelve suppliers have taken part in an online reverse auction that was set up by the company. The lots had been defined by the procurement experts of the company, based upon their expertise and expectations from the auction. After the auction, the suppliers have been asked to provide a cost breakdown for the lots that they have bidden for. Hence, the suppliers provided an estimate of their bids for each of the items that they have bidden for. For purposes of testing the proposed clustering-based lotting algorithm, two subsets of data have been selected. The first subset (case 1) consists of the items for which five suppliers made a bid. There are 90 items in this data set, divided into four lots by the experts. The second subset (case 2) consists of five lots for which four suppliers made a bid. There are 142 items in this data set. For the clustering-based lotting, the experts have indicated which features they consider to be relevant for the study, which ones should be clustered based on similarity and which ones should be clustered based on dissimilarity. Then the algorithm is applied on the data sets. Figure 2 shows the dendogram obtained for case 2. In order to compare the cluster-based lotting to the lotting of experts, the threshold for determining the lots is selected in such a way that the number of lots obtained equals the number of lots that the experts had used in their lotting. It may be the case, as in Fig. 2, that no threshold gives the same number of lots as the experts. In that case, the threshold has been selected so as to obtain a larger number of lots and then some of the obtained lots are combined manually such that the final number of lots equals the number of lots used by the experts. Then, the performance of the final lot composition has been determined and compared to the performance of the lots determined by the experts. Additionally, the lot composi-
330
U. Kaymak, J.P. Verkade, and H.A.B. te Braake
tion obtained from clustering is presented to the experts who have judged the solution qualitatively for its acceptability.
Fig. 2. A dendogram obtained as a result of cluster-based lotting.
The cost breakdown estimates by the suppliers have been used to compare the performance of the clustering-based lotting to the lotting determined by the experts. It is assumed that the cost breakdown estimates correspond to the true valuation of an item by the supplier. In reality, the valuation for an item depends on the lot composition, and hence the “true” valuation is unobserved for different lot compositions. However, a better estimate of an item’s valuation is not available, and so we have assumed that the cost breakdown from the suppliers is independent of the lot composition. The performance of different lotting solutions is depicted in Table 1. In case 1, the clustering-based lotting achieves a cost saving of 1.9%, while it achieves a saving of 2.5% in case 2. The column titled “optimal solution” indicates the optimal solution when one assumes that the cost breakdown estimates provide the true valuations independent of the lot composition. We conclude from these results that the clustering-based lotting leads to about 2–3% savings in costs, which is significant when one considers the total throughput in online reverse auctions. The solutions provided by the clustering-based lotting have also been presented to the procurement experts, who have confirmed that the solutions are acceptable, given the procurement goals. Table 1. Performance (in M€) of different lotting solutions Expert solution Clustering solution Optimal solution Case 1 Case 2
6
1.06 9.73
1.04 9.49
0.85 9.00
Conclusions
Lotting is an important component of electronic reverse auctions. Large savings can be achieved and significant value can be added to the procurement by carefully considering
A Heuristic Lotting Method for Electronic Reverse Auctions
331
the lotting strategy employed in a reverse auction. In this paper, we have considered a clustering-based heuristic lotting method. The method uses complete linkage hierarchical clustering algorithm. A special distance metric is defined for this problem, which allows the procurement experts to specify which features are relevant for the problem, for which features the items in a lot should resemble one another and for which feature they should not resemble one another. This metric corresponds to the way the procurement experts reason about the lotting problem. The proposed algorithm has been applied to the procurement activities of a consumer packaged goods company by using online reverse auctions. It has been found that compared to the expert-based lotting, the proposed lotting algorithm leads to 2–3% savings in the procurement costs, while the procurement experts have confirmed that the resulting lotting solution is acceptable, given the procurement goals.
References 1. Jap, S.D.: Online reverse auctions: issues, themes and prospects for the future. Journal of the Academy of Marketing Science 30 (2002) 506–525 2. Minahan, T., Howarth, F., Vigoroso, M.: Making e–sourcing strategic. Research report, Aberdeen Group, Boston (2002) 3. Emiliani, M.L.: Business-to-business online auctions: key issues for purchasing process improvement. Supply Chain Management 5 (2000) 176–186 4. Tully, S.: The B2B tool that really is changing the world. Fortune 141 (2000) 132–145 5. Teich, J.E., Wallenius, H., Wallenius, J., Zaitsev,A.: Designing electronic auctions: an internetbased hybrid procedure combining aspects of negotiations and auctions. Electronic Commerce Research 1 (2001) 301–314 6. M¨uller, R.: Auctions – the big winner among trading mechanisms for the internet economy. Merit–infonomics research memorandum series, University of Maastricht, MERIT – Maastricht Economic Research Insititute on Innovation and Technology (2001) 2001–016. 7. Klemperer, P.: Auction theory: a guide to the literature. Journal of Economic Surveys 13 (1999) 227–286 8. Wedel, M., Kamakura, W.A.: Market Segmentation: conceptual and methodological foundations. Kluwer, Boston (1998) 9. Bezdek, J.C.: Pattern Recognition with Fuzzy Objective Function. Plenum Press, New York (1981) 10. Johnson, R.A., Wichem, D.W.: Applied Multivariate Statistical Analysis. Prentice Hall, New Jersey (1982)
A Poisson Model for User Accesses to Web Pages 2 ¨ S ¸ ule G¨ und¨ uz1 and M. Tamer Ozsu 1
Department of Computer Engineering, Istanbul Technical University Istanbul, Turkey, 34469
[email protected] 2 School of Computer Science, University of Waterloo Waterloo, Ontario, Canada N2L 3G1
[email protected] Abstract. Predicting the next request of a user as she visits Web pages has gained importance as Web-based activity increases. There are a number of different approaches to prediction. This paper concentrates on the discovery and modelling of the user’s aggregate interest in a session. This approach relies on the premise that the visiting time of a page is an indicator of the user’s interest in that page. Even the same person may have different desires at different times. Although the approach does not use the sequential patterns of transactions, experimental evaluation shows that the approach is quite effective in capturing a Web user’s access pattern. The model has an advantage over previous proposals in terms of speed and memory usage.
1
Introduction
Web mining is defined as the use of data mining techniques to automatically discover and extract information from Web documents and services [5]. With the rapid growth of the World Wide Web, the study of modelling and predicting a user’s access on a Web site has become more important. There are three steps in this process [2]. Since the data source is Web server log data for Web usage mining, the first step is to clean the data and prepare for mining the usage patterns. The second step is to extract usage patterns, and the third step is to build a predictive model based on the extracted usage patterns. The prediction step is the real-time processing of the model, which considers the active user session and makes recommendations based on the discovered patterns. An important feature of the user’s navigation path in a server session 1 is the time that a user spends on different pages [12]. If we knew the desire of a user every time she visits the Web site, we could use this information for recommending pages. Unfortunately, experience shows that users are rarely willing 1
The term server session is defined as the click stream of page views for a single visit of a user to a Web site [2]. In this paper we will use this term interchangeably with “user session” and “user transaction”.
A. Yazici and C. S ¸ ener (Eds.): ISCIS 2003, LNCS 2869, pp. 332–339, 2003. c Springer-Verlag Berlin Heidelberg 2003
A Poisson Model for User Accesses to Web Pages
333
to give explicit feedback. Thus, the time spent on a page is a good measure of the user’s interest in that page, providing an implicit rating for that page. If a user is interested in the content of a page, she will likely spend more time there compared to the other pages in her session. However, the representation of page visit time is important. If the representation is not appropriate for the model, the prediction accuracy will decrease. In [3] we proposed a new model that uses only the visiting time and visiting frequencies of pages without considering the access order of page requests in user sessions. Our experiments showed that Poisson distribution can be used to model user behavior during a single visit to a Web site. In that paper we examine the effect of several representation methods of time that a user spent on each page during her visit. In our previous work we employed a model-based clustering approach and partitioned user sessions according to the similar amount of time spent on similar pages. In this paper, we present a key extension to the representation of user transactions that improves the resulting accuracy for predicting the next request of a Web user. To confirm our findings, the results are compared to the results of two other well known recommendation techniques. The rest of the paper is organized as follows. Section 2 briefly reviews the work related to model based clustering. Section 3 presents the proposed model. Section 4 provides detailed experimental results. In Section 5, we examine related work. Finally, in Section 6 we conclude our work.
2
Model-Based Cluster Analysis
In this section, we first describe the mixture model for clustering objects and then describe how the parameters of the clusters are derived in the context of the mixture model. Model-based clustering methods optimize the fit between the given data and some mathematical model. Such methods are often based on the assumption that the data are generated by a mixture of underlying probability distributions, defined by a set of parameters, denoted Θ [6]. An observation xi in a data set of K observations, D = {x1 , ..., xK }, is generated by a mixture of G components as follows: p(xi |Θ) =
G
p(cg |Θ)p(xi |cg , Θg ) =
g=1
G
τg p(xi |cg , Θg )
(1)
g=1
where Θg (g ∈ [1...G]) is a vector specifying the probability distribution function G G (pdf) of the g th component, cg , and g=1 p(cg |Θ) = g=1 τg = 1. Statisticians refer to such a model as mixture model with G components. The maximum likelihood (ML estimation) approach maximizes the log likelihood of the training data in order to learn the model parameters:
L(Θ1 , ..., ΘG ; τ1 , ..., τG |D) =
K i=1
ln
G g=1
τg p(xi |cg , Θg )
(2)
334
3
¨ S ¸ . G¨ und¨ uz and M.T. Ozsu
Web Page Recommendation Model
In this research, we use three sets of server logs. The first one is from the NASA Kennedy Space Center server over the months of July and August 1995 [8]. The second log is from ClarkNet (C.Net)Web server which is a full Internet access provider for the Metro Baltimore-Washington DC area [7]. This server log was collected over the months of August and September, 1995. The last server log is from the Web server at the University of Saskatchewan (UOS) from June to December, 1995 [11]. For each log data set we apply the same pre-processing steps. Since the cleaning procedure is beyond the scope of this paper, the details of this procedure are not given here. In this work, visiting page times 2 , which are extracted during pre-processing step, are represented by four different normalization values in order to evaluate the effect of time to the prediction accuracy. The visiting times are normalized across the visiting times of the pages in the same session, such that the minimum value of normalized time is 1. We try 4 different maximum values: 2, 3, 5 and 10. If a page is not in the user session, then the value of corresponding normalized time is set to 0. This normalization captures the relative importance of a page to a user in a transaction. The aggregate interest of a user in a transaction is then defined by a vector which consists of the normalized visiting times of that transaction. The details of this step is given in [3]. Our previous work has presented a new model that uses only the visiting time and visiting frequencies of pages. The resulting model has lower run-time computation and memory requirements, while providing predictions that are at least as precise as previous proposals [3]. The key idea behind this work is that user sessions can be clustered according to the similar amount of time that is spent on similar pages within a session without considering the access order of page requests. In particular, we model user sessions in log data as being generated in the following manner: (i) When a user arrives to the Web site, his or her current session is assigned to one of the clusters, (ii) the behavior of that user in this session, in terms of visiting time, is then generated from a Poisson model of visiting times of that cluster. Since we do not have the actual cluster assignments, we use a standard learning algorithm, the ExpectationMaximization (EM) [4], to learn the cluster assignments of transactions as well as the parameters of each Poisson distribution. The resulting clusters consist of transactions in which users have similar interests and each cluster has its own parameters representing these interests. Our objective in this paper is to assess the effectiveness of non-sequentially ordered pages and the representation methods of normalized time values in predicting navigation patterns. In order to obtain a set of pages for recommending and rank these pages in this set, recommendation scores are calculated for every page in each cluster using the Poisson parameters of that cluster. The cluster parameters of a cluster cg are then in the form: pcg = {τg ; (rsg1 , ..., rsgn )} 2
It is defined as the time difference between consecutive page requests.
A Poisson Model for User Accesses to Web Pages
335
where τg is the probability of selecting the cluster cg and rsgj , j = [1...n] is the recommendation score of cluster cg at dimension3 j. Those are the only parameters that the system needs in order to produce a set of pages for recommendation. We define the number of parameters stored in the memory as model size. It is clear that the smaller the model size the faster the online prediction. We use five different methods for calculating recommendation scores for every page. The recommendation scores are then normalized such that the maximum score has a value of 1. These methods can be briefly summarized as follows: For the first method, we only use the Poisson parameters of the active cluster as recommendation scores. In the second method we use only the popularity of each page, which we define as the ratio of the number of the requests of a page in a cluster to the total number of page requests in that cluster. The intuition behind this is to recommend pages that are most likely visited in a cluster. For the third method, we calculate recommendation scores by multiplying the popularity by the Poisson parameter. For the last two methods we take advantage of a technique used in decision theory called the entropy. We calculate the entropy for each page using the relative frequency of each of the ten possible values of normalized times. A low entropy value means that the visiting time of that page mostly has one of the normalized values. High entropy value, on the other hand, indicates wide divergence in page visiting times among transactions. We calculate the recommendation scores of the fourth method by multiplying the inverse of entropy by popularity and Poisson parameters. For the last calculation, the log of the popularity is taken in order to decrease the effect of the popularity in recommendation score and is multiplied by the inverse of entropy and Poisson parameters. The real-time component of the model calculates cluster posterior probability P (cg |w) for every cluster cg ∈ C = {c1 , ..., cG } where w is the portion of a transaction in test set that is used to find the most similar cluster. The active transaction is assigned to the cluster that has the highest probability. We define this cluster as the active cluster. A recommendation set, which is the set of predicted pages by the model, is then produced ranking the recommendation scores of active cluster in descending order.
4
Experimental Results
In this research we use three different transaction sets prepared for experiments as mentioned in Section 3. We measure the performance of our technique using the proposed methods for calculating recommendation scores. Approximately 30% of these cleaned transactions are randomly selected as the test set, and the remaining part as the training set. The experiments are repeated with different number of clusters and with different initial parameters for EM algorithm. We define the following metrics to evaluate our method: Hit-Ratio. Given the visiting time of a page in the current transaction, the model recommends three pages that have the highest recommendation score 3
Each page in the Web site corresponds a dimension in the model
336
¨ S ¸ . G¨ und¨ uz and M.T. Ozsu
in the active cluster. A hit is declared if any one of the three recommended pages is the next request of the user. The hit-ratio is the number of hits divided by the total number of recommendations made by the system. Precision. For each transaction t in the test set we select the first w requests in t. These w requests are used to calculate the active cluster and produce the recommendation set. The recommendation set contains all the pages that have a recommendation score greater than the threshold ξ and that are not in the first w requests. We denote this set as P S(w, ξ) and the number of pages in this set that match with the remaining part of active transaction as m. Then the precision for a transaction is defined as: precision(t) =
m |P S(w, ξ)|
(3)
In our experiments, we try different values for the threshold, ξ, of recommendation scores ranging from 0.1 to 0.9. If the threshold is high then fewer recommendation are produced. If it is small then irrelevant pages are recommended with a low recommendation score. Our experiments show that setting ξ to 0.5 and w to 2 produces few but highly relevant recommendations. We perform the experiments with different number of clusters changing from 4 to 30. These experiments show that normalizing time between 1 and 2 improves the prediction accuracy. Due to lack of space, we just present the results of the experiments in which the normalized time has a value between 1 and 2. We identify that the values for the number of clusters in Table 1 are best among the other values we consider if page time is normalized between 1 and 2. For these numbers we have a higher log likelihood for the training sets as well as a better prediction accuracy for the test sets. The increase of the log likelihood means that the model fit better to the data. Figure 1(a) presents the prediction accuracy of the model for different number of clusters where time is normalized between 1 and 2. Figure 1(b) presents the prediction accuracy for different normalization values of time. As can be seen from Figure 1(a), the model is insensitive to the number of clusters in a reasonable range around the best numbers of clusters. The remarkable changes in the number of clusters results in a decrease of the performance of the model. Table 1. Results (in %) of the model. Visiting time is normalized between 1 and 2. Data Set No.Of Clusters NASA 30 C.Net 10 UOS 30
Method 1 H-R Pre. 51.5 34.4 48.7 37.9 50.8 40.6
Method 2 H-R Pre. 51.3 34.7 49.2 37.6 50.6 40.7
Method 3 H-R Pre. 52 35 49.6 38.2 50.8 40.7
Method 4 H-R Pre. 51.1 33.8 48.2 35.4 50.5 39.3
Method 5 H-R Pre. 47.5 33.8 46.6 32.9 50.1 38.7
As mentioned in the previous section, we use 5 different methods for calculating recommendation scores. The application of methods that calculate the recommendation scores using popularity term results in marked improvement of
A Poisson Model for User Accesses to Web Pages
60
60
50
50
337
NASA(H-R) 40
NASA(Pre.)
NASA C.Net
30
Acc.
Acc.
40
C.Net(H-R)
30
C.Net(Pre.)
UOS 20
20
10
10
UOS(H-R) UOS(Pre.)
0
0 0
10
20
30
40
Number Of Cl.
(a) Number of Clusters-Accuracy
0 1
2
3 4
5 6 7 8
9 10 11
Norm.Time
(b) Normalization values-Accuracy
Fig. 1. Impacts of number of clusters and normalization values on prediction accuracy
the prediction accuracy. This is not surprising, because the popularity represents the common interest among transactions in each cluster. The results show that using entropy during calculation of recommendation score does not improve the accuracy. This is not surprising for the experiments where page time is normalized in a narrow range. However, even for a wide change in normalized time the entropy does not improve the prediction accuracy. This may be due to the fact that the popularity of some pages in most of the clusters are zero due to the sparse and scattered nature of the data. Thus, we can not calculate entropy values for most of the pages in a cluster. All of our experiments show that in general we can use method 3 for calculating recommendation scores discarding the metric we use for evaluation. Table 2. Comparison of recommendation models. Data Set Poisson Model Model 1 Model 2 NASA 52 4 47.84 C.Net 49.6 15 49.3 UOS 50.8 5 44.59
For evaluating the effect of the Poisson model, we repeated the experiments with the same training and test sets using two other recommendation methods[9, 10]. The recommendation model proposed in [10] (Model 1 in Table 2) is comparable to our model in terms of speed and memory usage. Since the hit-ratio metric has not performed well for the model in [10], we use the precision metric for evaluation. The C.Net data set has a precision of 15%, whereas the NASA data set has 4% and the UOS has 5%. Since the model in [9] is based on association rule discovery, it has obviously a greater model size than our model. We select this model in order to compare our results to the results of a model that uses a different approach. For the method in [9] (Model 2 in Table 2) we use a sliding window with a window size 2. The sliding window is the last portion
338
¨ S ¸ . G¨ und¨ uz and M.T. Ozsu
of the active user session to produce the recommendation set. Thus, the model is able to produce the recommendation set only after the first two pages of the active user session. We set the support for association rule generation to a low value such as 1 % discarding the model size in order to have a good prediction accuracy. The hit ratio for the NASA, C.Net and UOS data sets are 47.8%, 49.3%, 44.50% respectively. These results prove that modelling the user transaction with a mixture of Poisson distributions produces satisfactory prediction rates with an acceptable computational complexity in real-time and memory usage when page time is normalized between 1 and 2.
5
Related Work
The major classes of recommendation services are based on collaborative filtering techniques and the discovery of navigational patterns of users. The main techniques for pattern discovery are sequential patterns, association rules, Markov models, and clustering. Collaborative filtering techniques predict the utility of items of an active user by matching, in real-time, the active user’s preferences against similar records (nearest neighbors) obtained by the system over time from other users [1]. One shortcomings of these approaches is that it becomes hard to maintain the prediction accuracy in a reasonable range while handling the large number of items (dimensions) in order to decrease the on-line prediction cost. Some authors have used association rules, sequential patterns and Markov models in recommender systems. These techniques work well for Web sites that do not have a complex structure, but experiments on complex, highly interconnected sites show that the storage space and runtime requirements of these techniques increase due to the large number of patterns for sequential pattern and association rules, and the large number of states for Markov models. It may be possible to prune the rule space, enabling faster on-line prediction. Page recommendations in [10] are based on clusters of pages found from the server log for a site. The system recommends pages from clusters that most closely match the current session. Two crucial differences between our approach and the previous one are that we consider the user interest as a statistical model and we partition user sessions using a model-based approach. As the experiments demonstrate, our model’s precision and robustness is superior. Furthermore, our model has the flexibility to represent the user interest with a mixture of binomial distributions (or with different distributions) if one wishes to ignore the visiting time in determining the navigational pattern. We provide some intuitive arguments for why our model has an advantage in terms of speed and memory usage. The online prediction time correlates strongly with the model size. The smaller the model size the faster the online recommendation. Since we only store the cluster parameters for the prediction of the next page request, our model size is very small. The model size only increases with the number of clusters or the number of pages in the Web site when the Web site has a complex structure. However, it is clear that in that case the application of methods such as sequential pattern mining, association rules or Markov models generate more complex
A Poisson Model for User Accesses to Web Pages
339
models due to the increasing size of rules or states. Thus, all of these models require some pruning steps in order that they be effective. However, our model provides a high prediction accuracy with a simple model structure.
6
Conclusion
We have considered the problem of representing page time in a user session. In this article, the mixture of Poisson model is used for modelling the interest of a user in one transaction. The experiments show that the model can be used on Web sites with different structures. To confirm our finding, we compare our model to two previously proposed recommendation models. Results show that our model improves the efficiency significantly.
References 1. J. S. Breese, D. Heckerman, and C. Kadie. Empirical analysis of predictive algorithms for collaborative filtering. Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence, pages 43–52, 1998. 2. R. Cooley, B. Mobasher, and J. Srivastava. Data preparation for mining world wide web browsing patterns. Journal of Knowledge and Information Systems, 1(1), 1999. ¨ 3. S ¸ . G¨ und¨ uz and M. T. Ozsu. A user interest model for web page navigation. In Proc. of Int. Workshop on Data Mining for Actionable Knowledge, Seoul, Korea, April 2003. to appear. 4. A. P. Dempster, N.M. Laird, and D.B. Rubin. Maximum likelihood from incomplete data via the em algorithm. Journal of Royal Statistical Society, 39(1):1–38, 1977. 5. O. Etzioni. The world wide web: Quagmire or gold mine. Communications of the ACM, 39(11):65–68, 1996. 6. D. Hand, H. Mannila, and P. Smyth. Principles of Data Mining. The MIT Press, 2001. 7. ClarkNet WWW Server Log. http://ita.ee.lbl.gov/html/contrib/ClarkNet-HTTP.html. 8. NASA Kennedy Space Center Log. http://ita.ee.lbl.gov/html/contrib/NASA-HTTP.html. 9. B. Mobasher, H. Dai, T. Luo, and M. Nakagawa. Effective personalization based on association rule discovery from web usage data. In Proceedings of the 3rd ACM Workhop on Web Information and Data Management, pages 9–15, November 2001. Atlanta, USA. 10. B. Mobasher, H. Dai, T. Luo, and M. Nakagawa. Improving the effectiveness of collaborative filtering on anonymous web usage data. Proceedings of the IJCAI 2001 Workshop on Intelligent Techniques for Web Personalization (ITWP01), Aug. 2001. Seattle. 11. The University of Saskatchewan Log. http://ita.ee.lbl.gov/html/contrib/SaskHTTP.html. 12. C. Shahabi, A. Zarkesh, J. Adibi, and V. Shah. Knowledge discovery from users web-page navigation. Proceeding of the IEEE RIDE97 Workshop, pages 20-29, Birmingham, England, April 1997.
An Activity Planning and Progress Following Tool for Self-Directed Distance Learning 1LJDUùHQDQG1HúHΦ Φ Φ7 @
JG
(1)
where 'L ; L 0 . The covariance matrix, 67 , is computed as:
(2)
5REXVW6NLQ&RORU6HJPHQWDWLRQ8VLQJD'3ODQHRI5*%&RORU6SDFH 7 JJGJJG 7 67 = ∑ Φ L Φ L = ΦΦ7
.
(3)
L =
The eigenvalues and eigenvectors of 67 are obtained using equation (4). 67 Ψ = ΨΛ
(4)
JG JG JG
where Ψ = >ψ ψ ψ @ and Λ = >λ λ λ @ ( λ ≥ λ ≥ λ ) represent the eigenvectors and eigenvalues, respectively.
JG
JG
Two eigenvectors, ψ and ψ , that correspond to smallest eigenvalues, λ and λ , JG represent two directions with smallest spread of ; L ’s (i. e. RGB values of skin JG pixels). A 2D lookup table (i. e. color histogram) is built by projecting all ; L ’s onto JG JG the 2D plane defined by the two axes, ψ and ψ , as in equation (5). This lookup table is loaded for online detection of skin colors. :SFD is a linear projection matrix of JG JG which two column vectors are ψ and ψ . JG JJG 7 @&ORVHGIUDPHVHDUFKLVWKHPHWKRGRIILQGLQJUHF WDQJXODUIUDPHVRIFRQQHFWHGOLQHV7KLVIDLOVZKHQWKHOLQHVRISODWHDUHORVWGXHWR WKHODFNRIHGJHHQKDQFHPHQW>@&RORUFRGHLVDQRWKHUPHWKRGRIVHDUFKLQJVSHFLDO FRORUDUHDRYHULPDJH1DWLRQDOIODJFRORURQSODWHLVDJRRGH[DPSOHIRUWKLVPHWKRG 7KLV FDVH LV WRR UHVWULFWLYH VLQFH PRVW RI WKH OLFHQVH SODWHV DUH EODFN DQG ZKLWH >@ +LVWRJUDPDQDO\VLVLVWKHPHWKRGRIVHDUFKLQJVSHFLDOKLVWRJUDPRIWKHWHPSODWHFKDU DFWHUVRQLPDJH7KLVPHWKRGGRHVQRWZRUNRQLPDJHVWKDWKDYHKLJKEULJKWQHVVOLNH WKHRQHVWDNHQDWWZLOLJKW>@
License Plate Segmentation for Intelligent Transportation Systems
441
7KHPHWKRGVDQGWKHLUGUDZEDFNVIRU3(V\VWHPVFDQEHVXPPDUL]HGDVIROORZV &RUQHU ILQGLQJ PHWKRG LV WKH SURFHVV RI VHDUFKLQJ OLQH RYHUODSV GXULQJ HGJH HQ KDQFHPHQW7KLVFDQEHPLVVHGZKHQHGJHGHWHFWLRQORRVHVWKHSXUHFKDUDFWHULVWLFVRI OLQHV>@&KDUDFWHUZLGWKPHWKRGLVGHILQHGDVVHDUFKLQJWHPSODWHVXLWDEOHIRUSRV VLEOHFKDUDFWHUZLGWKVL]HVOLNHSL[HOV7KLVPHWKRGFDQQRWEHXVHGRQSODWHVFDU U\LQJFKDUDFWHUVRIYDULDEOHVL]H>@ 7KHPHWKRGVDQGWKHLUGUDZEDFNVIRU&6V\VWHPVFDQEHVXPPDUL]HGDVIROORZV %ORE&RORULQJPHWKRGFDQEHXVHGDVDUHJLRQJURZLQJSURFHGXUHZKLFKODEHOVSL[HOV WKDW IRUP FRQQHFWHG FRQWLJXRXV UHJLRQV ³EOREV´ WKDW UHFHLYH D XQLTXH ODEHO %XW WKLVPHWKRGKDVZHDNQHVVRIFRQQHFWHGFKDUDFWHUSUREOHP>@*DSVHDUFKLVDQRWKHU PHWKRG ZKLFK LV EDVHG RQ ILQGLQJ H[DFW JDS EHWZHHQ WKH FKDUDFWHU SDWWHUQV 7KLV LV QRWDSSOLFDEOHZKHQVRPHSRUWLRQVRIFKDUDFWHUVDUHRYHUODSSLQJRUQRLV\SDWWHUQVDUH FRQQHFWLQJWKHFKDUDFWHUV>@ $VFDQEHVHHQIURPWKHDERYHGLVFXVVLRQRQHVLQJOHPHWKRGFDQQRWIXOO\UHDOL]H RQHRIWKHIXQFWLRQVRI$5)3(RU&6IRUDFWXDOSLFWXUHV7KHUHIRUHGHYHORSLQJDVH TXHQFHRIRSHUDWLRQWKDWZRXOGHOLPLQDWHWKHVHGHILFLHQFLHVLVHVVHQWLDOIRUKLJKSHU IRUPDQFH,QWKLVVWXG\ZHKDYHGHYLVHGWKHVHRSHUDWLRQVHTXHQFHVIRUHDFKIXQFWLRQ
7KH6WUXFWXUHRIWKH'HYHORSHG6\VWHP
*HQHUDO VWUXFWXUHRI RXU VHJPHQWDWLRQ V\VWHP LV JLYHQ LQ )LJ D 7KHLQSXW WR WKH V\VWHP LV D WULJJHUHG FDU LPDJH DQG RXWSXW LV D VHW RI FKDUDFWHU LPDJHV )LJ E VKRZV WZR UHDO WULJJHUHG LPDJHV )ROORZLQJ VHFWLRQV JLYH GHWDLOV RI VHJPHQWDWLRQ VWHSVDQGFRPSDUHWKHPZLWKRWKHUDOWHUQDWLYHV
3UHSURFHVVLQJIRU/35
&RQYHUWLQJFRORULPDJHVWRJUD\VFDOHLVWKHILUVWVWHSRISUHSURFHVVLQJ7KLVLVHVVHQ WLDOIRUHOLPLQDWLQJFRORULQIRUPDWLRQWKDWLVUHGXQGDQWIRURXUSXUSRVHV7KHRULJLQDO LPDJHVWULJJHUHGDUHSUHVVHGLQWKHYHUWLFDOGLPHQVLRQGXHWRWKHW\SHRI&&'FDPHUD XVHG7KLVPDNHVWKHLPDJHGLIILFXOWWRDQDO\]H5HVL]LQJFDQEHGRQHE\XVLQJVHY HUDOPHWKRGV/LQHFORQLQJLVWKHVLPSOHVWPHWKRGRIUHVL]LQJLQZKLFKHDFKOLQHIURP LPDJHLVDGGHGDVVXFFHVVRURILW$SSO\LQJWKLVPHWKRGFDXVHVWKHLPDJHWRORRVHLWV YDOXDEOHLQIRUPDWLRQOLNHHGJHVDQGVPRRWKJUD\OHYHOYDULDWLRQ,WLVUHSRUWHGWREH XQVXLWDEOHIRUDQDO\]LQJLPDJH>@/LQHDULQWHUSRODWLRQLVDQRWKHUZD\RIPDNLQJUH VL]LQJZLWKSUHVHUYLQJDSSUR[LPDWHVPRRWKYDULDWLRQRQJUD\OHYHOV'LVDGYDQWDJHLV GLVFUHWH VWHS VL]H RI LQWHUSRODWLRQ LQWHUYDO WKDW PD\ QHJDWLYHO\ DIIHFW HGJH GHWHFWLRQ SHUIRUPDQFH )UHTXHQF\ GRPDLQ EDVHG WUDQVIRUPDWLRQV VXFK DV %LOLQHDU +DQQ DQG +DPPLQJ%ODFNPDQDQG/DQF]RVPHWKRGVFDQDOVREHXVHG>@:HWULHGDOORIWKHP DQGREVHUYHGWKDW/DQF]RVPHWKRG\LHOGHGEHWWHUUHVXOWVWKDQRWKHUV>@,WXVHV6LQF [ IXQFWLRQDVEDVHWUDQVIRUPDWLRQIXQFWLRQ 7KLVWUDQVIRUPDWLRQIXQFWLRQLVZLQ GRZHGDQGGLIIHUHQWLDWHGZLWKFRQYROXWLRQIXQFWLRQVRI
0&LQVGLNLFLDQG77XQDOÕ
Fig. 1. (a) Three lines are taken .(b) These lines are horizontally projected. Last signal shows high frequency object on this line. This is candidate plate location 7DEOH$OWHUQDWLYH0HWKRGVIRUWKHVHJPHQWDWLRQSURFHVV $SSUR[LPDWH5HJLRQ)LQGLQJ 7HPSODWHPDWFKLQJ
3ODWH([WUDFWLRQ
&RXQWHU .QRZOHGJHEDVHG VHDUFK &ORVHGIUDPHVHDUFK
/LQHSURMHFWLRQ 6SHFLDOSDWWHUQ &RORUFRGHV +LVWRJUDPDQDO\VLV
&RUQHU)LQGLQJ &ORVHGIUDPH VHDUFK /LQHSURMHFWLRQ &KDUDFWHUZK
&KDUDFWHU6HJ PHQWDWLRQ 7HPSODWHPDWFKLQJ &KDUDFWHU ZLGWKKHLJKW *DSRQSURMHFWLRQ &RXQWHU &RORUFRGHV
/LFHQVH3ODWH6HJPHQWDWLRQIRU,QWHOOLJHQW7UDQVSRUW6\VWHPV
0DLQ6WHSVRI 6HJPHQWDWLRQ
6XE6WHSVRI0DLQ6WHSV
7ULJJHUHG&DU ,PDJH 3UHSURFHVVLQJ
$SSUR[LPDWH 5HJLRQ)LQGHU
+LVWRJUDP (TXDOL]DWLRQ
*UD\/HYHO &RQYHUVLRQ
5HVL]LQJ
/DQFVR]
&RQWUDVW EULJKW QHVVEDODQFHG
9HUWLFDO(GJH 'HWHFWLRQ
3L[HO+LW5DWH
&RPSUHVVLRQ
6REHO
5RWDWLRQ
3ODWH([WUDFWRU
,PDJHLV URWDWHGWROHIW
&KDUDFWHU 6HJPHQWDWLRQ
0RUSKRORJLF)LOWHU
'LODWH
ZE GHQVLW\
%HUNOH\/DE $OSKD
0RWLRQ
%LQDUL]DWLRQ
SL[HOV ULJKW
WZRYDOXHG FURVVHQWURS\
0HGLDQ)LOWHU
&XWWLQJ &KDUDFNWHUV
3URMHFWLRQ0DUNLQJ
%97
/RFDO 0LQ 0D[
&KDUDFWHU,PDJHV
Fig. 2. (a) System flowchart. (b) Triggered image VLQ p
6LQF[ VLQ[ [
/ [ t =
p
p t
p t
[ @ ,W JLYHV JRRG EDODQFH EHWZHHQ SUHVHUYLQJ VKDUSQHVV DQGLWGRHVQRWLQWURGXFHULQJLQJHIIHFWV7KHGLVDGYDQWDJHLVLWVORQJFRPSXWDWLRQDO WLPH 2WKHU PHWKRGV OLNH +DQQ DQG %ODFNPDQ DUH QRW DV VXFFHVVIXO DV /DQF]RV PHWKRG>@ ,Q )LJ D NHUQHO RI /DQF]RV ILOWHU LV GHPRQVWUDWHG LQ VSHFWUDO IRUP ,Q RXUSURSRVHGV\VWHP/DQFVR]PHWKRGLVXVHGWRUHVL]HLPDJHYHUWLFDOO\ZLWKUDWH $IWHU/DQF]RVUHVL]LQJWKHLPDJHLVWDNHQWKURXJKWKHKLVWRJUDPHTXDOL]DWLRQSUR FHVVWREDODQFHJUD\OHYHOFRQWUDVWRYHULPDJH>@+LVWRJUDPHTXDOL]DWLRQJLYHVWKH DGYDQWDJH RI WDNLQJ LPDJHV XQGHU GLIIHUHQW VXQOLJKW FRQGLWLRQV ,PDJH WDNHQ LQ WKH DIWHUQRRQHYHQLQJRUDWQLJKWFDQEHLQWHUQDOO\EDODQFHGZLWKKLVWRJUDPHTXDOL]DWLRQ $IWHUKLVWRJUDPHTXDOL]DWLRQWKHLPDJHLVUHDG\IRUDSSUR[LPDWH UHJLRQ ILQGHU PRG XOH
)LJD /DQF]RVE :LQ6LQFF 6SHFWUXPG /RJDULWPLF]RRP
$SSUR[LPDWH5HJLRQ)LQGHU0RGXOH
3UHSURFHVVHG LPDJH LV UHDG\ IRU RYHUDOO VHJPHQWDWLRQ SURFHVV 3URSRVHG V\VWHP GL YLGHVVHJPHQWDWLRQLQWRWKUHHSKDVHV7KHILUVWSKDVHLVWKHDSSUR[LPDWHUHJLRQILQGHU PRGXOH 7KLV PRGXOH LQFOXGHV YHUWLFDO HGJH GHWHFWLRQ SL[HO KLW UDWH UHJLRQ ILQGLQJ DQGFRPSUHVVLRQVXEPRGXOHV 7KHSODWHVFRQWDLQFKDUDFWHUVWKDWKDYHGHQVHYHUWLFDODQGKRUL]RQWDOLQIRUPDWLRQ 7KHUHIRUH VHDUFK IRU YHUWLFDO DQG KRUL]RQWDO LQIRUPDWLRQ FDQ OHDG XV WR WKH SODWH +RZHYHULQDFDULPDJHKRUL]RQWDOLQIRUPDWLRQLVDOVRGHQVHLQPDQ\RWKHUUHJLRQV VXFK DV WKH UHDU ZLQGRZ DQG WKH WUXQN 7KHUHIRUH ZH RQO\ FRQFHQWUDWH RQ YHUWLFDO HGJHGHWHFWLRQ7\SLFDOPHWKRGVIRUH[WUDFWLQJYHUWLFDOHGJHVDUH3UHZLWW6REHOILOWHUV DQG&DQQ\HGJHGHWHFWRU:HSUHIHUUHG6REHOILOWHURYHU3UHZLWWILOWHUDQG&DQQ\HGJH GHWHFWRU >@ GXH WR LWV VLPSOLFLW\ DQG SHUIRUPDQFH )LJ D GHSLFWV WKH RXWSXW RI D 6REHOILOWHULQJSURFHVVZLWKWROHUDQFHYDOXHRI ,QSL[HOKLWUDWHUHJLRQILQGLQJVWHSZHVOLGHDUHFWDQJXODUZLQGRZRIVL]H· SL[HOVDQGFRPSXWHWKHPDWFKLQJZKLWHSL[HOGHQVLW\$OOFDQGLGDWHVDUHUHFRUGHGZLWK WKHLUFRRUGLQDWHV$VGHSLFWHGLQ)LJE DFDQGLGDWHH[WUDFWHGIURPWKHLPDJHPD\ QRWFRQWDLQSODWHLWVHOI\HW 7RH[WUDFWWKHSODWHLWVHOIZHQHHGWRLVRODWHWKHIRUHJURXQGWKDWFRQWDLQVFKDUDF WHUV IURPEDFNJURXQG:HZLOOXVHWKUHVKROGLQJWRSHUIRUPWKLVLVRODWLRQ+RZHYHU WKHFDQGLGDWHH[WUDFWHGSODWHFRQWDLQVJUD\OHYHOVWKDWLVWRRKLJKIRUDQLQSXWWR
0&LQVGLNLFLDQG77XQDOÕ WKUHVKROGLQJSURFHVV)RUWKLVSXUSRVHZHDSSO\FRPSUHVVLRQWRUHGXFHWKHQXPEHURI JUD\OHYHOVE\DSSUR[LPDWHO\%HUNOH\/DEV$OSKDFRPSUHVVLRQDOJRULWKPLVVH OHFWHGDVWKHWRRORIWKLVODWWHUSURFHVV
)LJD 6REHOILOWHUHGLPDJH)LJE SODWHUHJLRQREWDLQHGLQRXUFDVH
3ODWH([WUDFWRU0RGXOH
$IWHU DSSUR[LPDWH SODWH UHJLRQ LV H[WUDFWHG IURP WKH LPDJH VRPH H[WUD VWHSV DUH QHHGHG WR H[WUDFW WKH SODWH LWVHOI 7KLV LV SHUIRUPHG E\ SODWH H[WUDFWRU PRGXOH WKDW FRQVLVWVRIURWDWLRQELQDUL]DWLRQPRWLRQDQGPHGLDQILOWHULQJVWHSV7ZRRXWSXWVDUH REWDLQHGDVELQDU\DQGJUD\OHYHOIRUPRISODWH7KHELQDU\RXWSXWLVXVHGIRUFKDUDF WHU LVRODWLRQ ZKHUHDV WKH JUD\ OHYHO LPDJH ZLOO EH XVHG E\ WKH UHFRJQLWLRQ PRGXOH 5RWDWLRQVWHSFRUUHFWVWKHSODWHDQJOHE\URWDWLQJLWFORFNZLVHLQWKHUDQJHRI %HFDXVHRIVWDWLFGLVWRUWLRQRQURWDWLRQWKHURWDWLRQSURFHVVLVVLPSOHWRDSSO\7KHUH DUH VRPH RWKHU DOWHUQDWLYH PHWKRGV WR URWDWH WKH SODWH G\QDPLFDOO\ )RU LQVWDQFH +RXJKWUDQVIRUPLVDSRSXODUPHWKRGEXWVLQFHLWLVPRUHFRPSOH[DQGLWLVQRWXVHG RQ LPDJHV FDUU\LQJ GHWDLOHG LQIRUPDWLRQ OLNH FKDUDFWHU SDWWHUQV 1H[W LV ELQDUL]DWLRQ RIWKHSODWH7KHUHDUHVHYHUDOPHWKRGVIRUELQDUL]DWLRQOLNH1LEODFN>@WZRYDOXHG FURVV HQWURS\ WKUHVKROGLQJ >@ PHDQ DQG VWDQGDUG GHYLDWLRQ WKUHVKROGLQJ 1LEODFN PHWKRG LV EDVHG RQ ORFDO QHLJKERULQJ WKUHVKROGLQJ XVLQJ WKH HTXDWLRQ ,Q WKLV HTXDWLRQ7LVQHZYDOXHRISL[HODQGwLVORFDOZHLJKWsm DUHVWDQGDUGGHYLDWLRQ DQGPHDQYDOXHRI[ QHLJKERULQJSL[HOVUHVSHFWLYHO\1LEODFNPHWKRGLVYDOXDEOH IRUWKHZKROHLPDJHOLNHLQ)LJD %XWIRUVPDOOLPDJHVOLNHSODWHUHJLRQLWORRVHV RQHRUWZRFKDUDFWHUVRQSODWHZKLOHWKUHVKROGLQJ IL IL IL >W FLM>V 7cg w scg mcg m W = IL<W FLM@DVJLYHQLQ LVPRUH VXFFHVVIXORQVPDOOLPDJHV,W LV QRW VXLWDEOH IRUELJJHU LPDJHV OLNH LQ )LJ D EH FDXVHLWLVVORZ3URSRVHGV\VWHPXVHVWZRYDOXHGFURVVHQWURS\WKUHVKROGLQJ,Q DUHWZRWKUHVKROGLQJYDOXHVRYHUJUD\OHYHOGLVWULEXWLRQPDWUL[>@7KLUGVWHS
LVDUWLILFLDOPRWLRQDSSOLFDWLRQWRWKHSODWHUHJLRQ7KHSODWHLVVKLIWHGWRWKHULJKWE\ SL[HOV LQ SURSRVHG V\VWHP 7KH FKRLFH RI ULJKW VKLIW LV DUELWUDU\ :H FRXOG DV ZHOO KDYH VKLIWHG OHIW WR VHUYH WKH VDPH SXUSRVH 7KLV VKLIWHG SODWH UHJLRQ LPDJH LV VXE WUDFWHGIURPWKHRULJLQDOSODWHUHJLRQLPDJH7KHUHVXOWDQWLPDJHVKRZVXVWKHVKDG RZVRIFKDUDFWHUVRQWKHSODWH7KLVPRWLRQFOHDUVWKHXQZDQWHGKRUL]RQWDOOLQHV7R FOHDUWKHVDOWDQGSHSSHUQRLVHWKHSODWHELQDUL]HGIRUPLVSDVVHGWKURXJKWKHPHGLDQ KLJKSDVVILOWHUDQGWKHQGLODWLRQPRUSKRORJLFILOWHU>@DVGHSLFWHGLQ)LJD 3UR
License Plate Segmentation for Intelligent Transportation Systems
445
SRVHG V\VWHP XVHV WKH FRUQHUV RI VKDGRZHG LPDJH DQG H[WUDFWV RULJLQDO SODWH LWVHOI OLNHLQ)LJE )LJD ([WUDFWHGSODWHDQGE LWVJUD\OHYHOILQDOIRUP
&KDUDFWHU6HJPHQWDWLRQ0RGXOH
3ODWHZLWKLWVELQDUL]HGIRUPLVUHDG\IRUFKDUDFWHULVRODWLRQ&KDUDFWHUVHJPHQWDWLRQ PRGXOHWDNHVELQDU\H[WUDFWHGSODWHSDVVHVLWWKURXJKWKHGLODWHPRUSKRORJLFILOWHUWR RYHUFRPH RI WKH SUREOHP RI QRLV\ SDWWHUQ FRQQHFWLRQV EHWZHHQ FKDUDFWHUV ,W WKHQ VHDUFKHV ORFDO PLQLPD DQG PD[LPD 7KHQ WR ILQG FDQGLGDWH FXW SRLQWV D QHZ WHFK QLTXH FDOOHG %LGLUHFWLRQDO 9HUWLFDO 7KUHVKROGLQJ %97 LV DSSOLHG 7KH ILUVW VWHS LV WDNLQJYHUWLFDOSURMHFWLRQRIWKHELQDUL]HGH[WUDFWHGSODWHIRUPOLNHLQ)LJD 7KHQ DOOWKHORFDOPLQLPDDQGPD[LPDDUHVLJQHGE\XVLQJ 7KLVVLJQDORQ)LJLVOLQH DUO\WUDQVIRUPHGDORQJ³\´D[LV7KHQELGLUHFWLRQDOVWDQGDUGGHYLDWLRQEDVHGWKUHVK ROGLQJLVDSSOLHGWRPDUNFKDUDFWHUFXWSRLQWRQWKHVLJQDODVGHSLFWHGLQ)LJD )RU UHDOH[WUDFWHGSDWWHUQVDOOFXWSRLQWVDUHXVHGWRREWDLQ)LJE 7KHILQDOGHFLVLRQ DERXWWKHFXWSRLQWVLVWREHPDGHDWWKHUHFRJQLWLRQVWDJH
)LJ+RUL]RQWDOSURMHFWLRQRIELQDUL]HGIRUPRIWKHSODWHRQ)LJ
)LJD &KDUDFWHUFXWSRLQWVIRXQGE\%97E LVRODWHGFKDULPDJHVE\D VLJQ - 6LJQ VLJQDO = VLJQ VLJQ +
D [ L L - < D [ L L - =
D [ L L - <
3HUIRUPDQFHDQG&RQFOXVLRQ
:H KDYH WHVWHG RXU V\VWHP RQ LPDJHV FDSWXUHG DW WKH QRQVWRS WROO FROOHFWLRQ JDWHVRIWKH%RVSKRUXV%ULGJH7KHVHLPDJHVDUHWULJJHUHGLQELWPDSFRORUHGIRUPLQ WKHVL]HRI[SL[HOV$OOLPDJHVKDYHVRPHURWDWLRQGLVWRUWLRQDERXWFRXQWHU
0&LQVGLNLFLDQG77XQDOÕ FORFNZLVH7KHLPDJHVZHUHFDSWXUHGDWGLIIHUHQWKRXUVRIDGD\:HXVHG0DWODELQ DOORIRXUFRPSXWDWLRQV7KHSUHSURFHVVLQJPRGXOHSHUIRUPVFRQYHUWLQJFRORULPDJHV WRJUD\VFDOHDQGWKHQUHVL]HVLPDJHVLQWR·SL[HOIRUPDQGWKHQPDNHVKLVWR JUDPHTXDOL]DWLRQWREDODQFHWKHFRQWUDVWDQGEULJKWQHVVRIWKHLPDJH:HREVHUYHG WKDW ZKHQ DOO RWKHU SDUDPHWHUV DUH IL[HG XVLQJ /DQF]RV PHWKRG LQFUHDVHV WKH SHU IRUPDQFHIURPWRZLWKSHUFHQWDJHUHIHUULQJWROLFHQVHSODWHVLQZKLFKDOO FKDUDFWHUV DUH SURSHUO\ LVRODWHG )RU DSSUR[LPDWH UHJLRQ ILQGHU ZH KDYH FRPSDUHG SHUIRUPDQFHRIRXUV\VWHPZLWKWKDWRIOLQHSURMHFWLRQ>@:HKDYHREVHUYHGWKDWRXU V\VWHP/DQF]RVPHWKRGDQG9HUWLFDO(GJH'HWHFWLRQ IRXQGRIDOOSODWHUHJLRQV ZKHUHDVOLQHSURMHFWLRQZDVDW :HKDYHDVVHVVHGWKHRYHUDOOSHUIRUPDQFHRIRXUV\VWHPE\KXPDQH\H,QRXW RILPDJHV RXU V\VWHP ZDV DEOH WR LVRODWH DOO RI WKH FKDUDFWHUVSURSHUO\ 2Q WKH UHVW RI WKH SODWHV WKH LVRODWLRQ SURFHVV IDLOHG GXH WR PRVWO\ VLQJOH IDXOW RI HLWKHU PLVVLQJWKHFXWSRLQWRUDGGLQJH[WUDFXWSRLQW:HH[SHFWWKDWWKHUHFRJQLWLRQPRG XOHWKDWZHDUHLQWKHSURFHVVRIGHYHORSLQJZLOOEHDEOHWRUHMHFWWKRVHFDQGLGDWHVDQG RYHUDOOSHUIRUPDQFHRIWKHV\VWHPZLOOEHVLJQLILFDQWO\LPSURYHG
5HIHUHQFHV >@ 2LYLQG'7$QLO.-7RUILQQ7³)HDWXUH([WUDFWLRQ0HWKRGVIRU&KDUDFWHU5HFRJQLWLRQ ±$6XUYH\´3DWWHUQ5HFRJQLWLRQ9RO1RSS± >@ 7DNDVKL17VXNDGD7@ 0LQ @ $NVR\ 06 dD÷ÕO * 7UNHU $. ³1XPEHU 3ODWH 5HFRJQLWLRQ 8VLQJ ,QGXFWLYH /HDUQ LQJ´5RERWLFVDQG$XWRQRPRXV6\VWHPV9RO1RSS±1RYHPEHU >@ &RHW]L & %RWKD &:HEHU ' ³3& %DVHG 1XPEHU 3ODWH 5HFRJQLWLRQ 6\VWHP´ ,((( KWWSFSERWKDQHWHPDLOFSERWKD#LHHHRUJ >@ *HRUJH:³'LJLWDO,PDJH:DUSLQJ,PDJH0DQLSXODWLRQ´,(((&RPSXWHU6RFLHW\3UHVV /RV$ODPLWRV&$ >@ /L+/HH.³0LQLPXP&URVV(QWURS\7KUHVKROGLQJ´3DWWHUQ5HFRJQLWLRQ9RO1R
A Turkish Handprint Character Recognition System 6iqÃyxr¼v·þh¦h¼Fhqv·Uhúqr·v¼!gªyr·FÕyÕp!hýqHÃuv³³výB|x·rý
ø²³hýiÃy Urpuývphy Výv½r¼²v³'8¸·¦Ã³r¼ @ýtvýrr¼výt9r¦h¼³·rý³ Hh²yhx "##%( ø²³hýiÃy FDSDU#LWXHGXWU JRNPHQ#FVLWXHGXWU !
ø²³hýiÃyUrpuývphyVýv½r¼²v³'Dý²³v³Ã³r¸sDýs¸¼·h³vp² Hh²yhx "##%( ø²³hýiÃy ºWDVGHPLUNLOLFR]`#EHLWXHGXWU
Abstract. This paper presents a study for recognizing isolated Turkish handwritten uppercase letters. In the study, first of all, a Turkish Handprint Character Database has been created from the students in Istanbul Technical University (ITU). There are about 20000 uppercase and 7000 digit samples in this database. Several feature extraction and classification techniques are realized and combined to find the best recognition system for Turkish characters. Features, obtained from Karhunen-Loéve Transform, Zernike Moments, Angular Radial Transform and Geometric Features, are classified with Artificial Neural Networks, K-Nearest Neighbor, Nearest Mean, Bayes, Parzen and Size Dependent Negative Log-Likelihood methods. Geometric moments, which are suitable for Turkish characters, are formed. KLT features are fused with other features since KLT gives the best recognition rate but has no information about the shape of the character where other methods have. The fused features of KLT and ART classified by SDNLL gives the best result for Turkish characters in the experiments.
1 Introduction P¦³vphy8uh¼hp³r¼Srp¸týv³v¸ýP8S÷v²¸ýr ¸s³ur·¸²³Zvqry' ¼r²rh¼purq h¼rh²vý ¦h³³r¼ý ¼rp¸týv³v¸ý h¦¦yvph³v¸ý²UurZ¸¼x²¸ý ³uv² ³¸¦vp·hvýy' s¸pò ¸ý¼rp¸tývªvýt uvtu ¹Ãhyv³' ¦¼vý³rq ³r`³ q¸p÷rý³² Pý ³ur ¸³ur¼ uhýq p¸·¦Ã³r¼ ¼rp¸týv³v¸ý ¸s uhýqZ¼v³³rýpuh¼hp³r¼² uh²¼rprv½rq¼ryh³v½ry' yr²²h³³rý³v¸ýZurýp¸·¦h¼rq³¸P8S bdC¸Zr½r¼¼rp¸týv³v¸ý ¸suhýq¦¼vý³q¸p÷rý³²v²irp¸·výth½r¼' ¦¸¦Ãyh¼ ¦h³³r¼ý ¼rp¸týv³v¸ý ¦¼¸iyr·³uhýx²³¸³uræqh³výt¸su÷hýýrrq² Uur¦¼¸pr²²¸suhýq¦¼vý³ puh¼hp³r¼ ¼rp¸týv³v¸ý v²Ã²Ãhyy' ýh·rqD8S Dý³ryyvtrý³ 8uh¼hp³r¼ Srp¸týv³v¸ý÷ Uur ·hvý ¦¼¸iyr·² h³ D8S ²'²³r·² h¼r ³ur ivt ½h¼vh³v¸ý² ¸s puh¼hp³r¼ ²uh¦r² h·¸ýt¦r¸¦yrhýq³ur¼hýq¸·ýr²²vý³urZ¼v³³rýpuh¼hp³r¼²r½rýs¸¼³ur²h·r¦r¼²¸ý Uur·¸¼r'¸Ãtr³¼vq¸s³ur²r¦¼¸iyr·²³ur·¸¼r²Ãppr²²sÃy'¸Ãh¼r There are two goals to be achieved in order to maximize the performance of handwritten character recognition. The first one is designing a feature extractor which 6Áhªvpvhýq8ùrýr¼@q²÷)DT8DT!"GI8T!©%(¦¦##&±#$%!" T¦¼výtr¼Wr¼yht7r¼yvýCrvqryir¼t!"
448
A. Çapar et al.
does not miss meaningful aspects where the second one is designing a classifier, which has good generalization power and minimum substitution error [2]. Recognition of Turkish characters is a harder problem with respect to recognition of characters in English alphabet, because of some additive letters (Fig. 1). 6y³u¸Ãtu ³ur¼rh¼r ²¸·rp¸··r¼pvhy Z¸¼x² ³uv² ¦h¦r¼v² ³ur¸ýr vý³ursrZ hph qr·vp ²³Ãqvr² ¸ý Uüxv²u Chýq¦¼vý³ 8uh¼hp³r¼ Srp¸týv³v¸ý Zuvpu óvyvªr² ³ur sv¼²³ p¸·¦¼rurý²v½r Uüxv²u Chýq¦¼vý³ 9h³hih²r Dý ³uv² Z¸¼x Zr vý½r²³vth³rq hqqv³v½r ¼rp¸týv³v¸ý¼Ãyr²³uh³h¼r²¦rpvhys¸¼Uüxv²uyr³³r¼²
2
Handprint Character Recognition System
The history of handwriting recognition systems is not complete without mentioning the Optical Character Recognition (OCR) systems [3]. Optical Character Recognition (OCR) is a problem recognized as being as old as the computer itself. Nowadays, researchers focus on handprint numeral and character recognition. Unfortunately, the success of OCR could not be carried on to Intelligent Character Recognition (ICR), due to the variations in people’s handwriting. As for the recognition of isolated handwritten numerals, Suen [4], addresses many researches which have already obtained very promising results using various classification methods. Suen mentions that the key issue reaching high recognition rates is the feature extraction. Uur¸ssyvýr v²¸yh³rq uhýqZ¼v³³rýpuh¼hp³r¼ ¼rp¸týv³v¸ý ²'²³r·² výpyÃqr ³u¼rr ·hw¸¼ ¦h¼³²)Q¼r¦¼¸pr²²výtQh¼³Arh³Ã¼r@`³¼hp³v¸ýQh¼³hýq8yh²²vsvph³v¸ýQh¼³
Fig. 1. Sample characters from Turkish Handprint Database
2.1 Preprocessing Preprocessing algorithms provides the required data suitable for the further processing. In other words, it establishes the link between real world and recognition engine. Preprocessing steps consist of digitization, binarization, blob coloring, segmentation, size normalization and skeletonization (Fig. 2).
A Turkish Handprint Character Recognition System
a)
b)
c)
d)
e)
f)
449
Fig. 2. Output of preprocessing steps. a) Digitization, b)Binarization, c)Blob Coloring, d) Segmentation, e) Size Normalization, f) Skeletonization
2.2 Feature Extraction Feature extraction problem is that of extracting from the raw data the information which is most relevant for classification purposes, in the sense of minimizing the within-class pattern variance while enhancing the between-class pattern variance. Arh³Ã¼r r`³¼hp³v¸ý ·r³u¸q² òrq vý ³uv² ²³Ãq' h¼r Fh¼uÃýrýG¸ü½r ³¼hý²s¸¼· FGU÷ 6ýtÃyh¼ ShqvhyU¼hý²s¸¼· 6SU÷ar¼ývxr·¸·rý³²hýqtr¸·r³¼vpsrh³Ã¼r² FGUsrh³Ã¼r² h¼ryvýrh¼ srh³Ã¼r²ióq¸ ý¸³ uh½rhý' výs¸¼·h³v¸ýhi¸Ã³ ³ur²uh¦r¸s ³ur ¦h³³r¼ý Pý ³ur ¸³ur¼ uhýq 6SU ar¼ývxr hýq tr¸·r³¼vp srh³Ã¼r² h¼r s¸¼·rq hpp¸¼qvýt³¸ ³urtr¸·r³¼vp ¦¼¸¦r¼³vr²¸s³ur¦h³³r¼ýUur¼rs¸¼r ²¸·r tr¸·r³¼vp·¸ ·rý³² Zuvpu h¼r ²Ãv³hiyr s¸¼ Uüxv²u puh¼hp³r¼² h¼r s¸¼·rq Dý hqqv³v¸ý FGU srh ³Ã¼r²h¼r p¸·ivýrqZv³u ¸³ur¼srh³Ã¼r² ²výprFGUtv½r²³urir²³¼rp¸týv³v¸ý¼h³r²ió uh²ý¸výs¸¼·h³v¸ýhi¸Ã³³ur²uh¦rZur¼r¸³ur¼·r³u¸q²uh½r The first extractor method, KLT, is a linear transform and corresponds to the projection of images onto the eigenvectors of the covariance matrix, where the covariance matrix is formed by the character images of training set [5]. The covariance matrix is formed from training samples and diagonalized using standard linear algebra routines, producing eigenvalues 9λ and corresponding eigenvectors Φ , that is
SΦ = Φ 9 λ .
(1)
UurFGU¼hý²s¸¼·½¸s ³ur½rp³¸¼ `v² ³ur¦¼¸wrp³v¸ý ¸s³ur½rp³¸¼·výò³ur·rhý ½rp³¸¼¸ý³¸³urrvtrý½rp³¸¼ih²v² Φ
½ = ΦU ` − µ÷ .
(2)
The second method, Zernike moments, is introduced by Zernike [6]. He introduced a set of complex polynomials which form a complete set over the interior of the unit circle. If the set of these polynomials is denoted by {Wÿ· ` ' ÷}, the form of these polynomials is:
450
A. Çapar et al.
Wÿ· ` ' ÷ = Wÿ· ρ θ ÷ = Sÿ· ρ ÷ r`¦ w·θ ÷ . where n
Positive integer or zero
m
Positive and negative integers subject to constraints n- · even , · ý
(3)
Grýt³u¸s½rp³¸¼s¼¸·¸¼vtvý³¸`'÷¦v`ry 6ýtyrir³Zrrý½rp³¸¼ hýq`h`v²výp¸Ãý³r¼py¸pxZv²rqv¼rp³v¸ý hýq Sÿ· ρ ÷ ¼hqvhy¦¸y'ý¸·vhyv²qrsvýrqh²
Sÿ· ρ ÷ =
(ÿ − · ) !
∑ ² =
−÷ ²
ÿ − ² ÷ . ρ ÿ−! ² ÿ− · ÿ+ · ² − ² − ² ! !
(4)
ar¼ývxr·¸·rý³²h¼r³ur¦¼¸wrp³v¸ý¸s³urv·htrsÃýp³v¸ý¸ý³¸³ur ¸¼³u¸t¸ýhy ih²v² sÃýp³v¸ý² hýq ³urar¼ývxr·¸·rý³ ¸s¸¼qr¼ýZv³u ¼r¦r³v³v¸ý·s¸¼hp¸ý³výøò v· htrsÃýp³v¸ýs`'þ³uh³½hýv²u¸Ã³²vqr ³urÃýv³pv¼pyrv²
6ÿ· =
ÿ + û s ` ' ÷Wÿ· ρ θ ÷q`q' . ! ! ∫∫ + ≤ ` ' π
(5)
To compute the Zernike moments of a given image, first the image is interpolated with a predetermined p¸ý²³hý³ b7]. The third method, Angular Radial Transform, is a 2-D complex transform defined on a unit disk in polar coordinates, Aÿ· = Wÿ· (ρ θ ) s (ρ θ ) =
!π
∫ ∫ W (ρ θ ) s (ρ θ )ρqρqθ . ∗ ÿ·
(6)
Here, Aÿ· is an ART coefficient of order n and m, s ρ θ ÷ is an image function in polar coordinates, and W ÿ· ρ θ ÷ is the ART basis function [8]. The ART basis functions are separable along the angular and radial directions,
W ÿ· (ρ θ ) = 6· (θ )S ÿ (ρ ),
(7)
UurhýtÃyh¼ hýq¼hqvhyih²v²sÃýp³v¸ý² h¼r qrsvýrqh² s¸yy¸Z²)
6· (θ ) =
r`¦( w·θ ), !π
ÿ= , S ÿ (ρ ) = ! p¸²(πÿρ ) ÿ ≠
(8)
(9)
The imaginary parts have similar shape to the corresponding real parts but with different phases.
A Turkish Handprint Character Recognition System
451
The geometric features, the fourth extractor method, are formed since KLT assumes no model of the human perception mechanism, but more directly references statistical information on how handwritten characters are formed [9]. Therefore, some additional rule-based methods need to be used to find features, in order to recognize many variations of the same character.
Fig. 3. Special point features a) End/Connection point regions b) End point directions
@ýq ¦¸vý³² hýq p¸ýýrp³v¸ý ¦¸vý³² ¸s h puh¼hp³r¼ v·htr ph¼¼vr² ¼¸iò³ výs¸¼·h³v¸ý hi¸Ã³³urpuh¼hp³r¼r²¦rpvhyy' s¸¼Uüxv²uyr³³r¼²µþ¶hýq µù¶9v¼rp³v¸ý hýq y¸ph³v¸ý ¸s ³ur²r ²¦rpvhy ¦¸vý³² ¸ý ³ur puh¼hp³r¼ ¼rtv¸ý h¼r yhiryrq h² vý Avtür " ³¸ r`³¼hp³ srh³Ã¼r²
Fig. 4. Background region types and their labels
Xuvyr·¸²³ puh¼hp³r¼¼rp¸týv³v¸ý²'²³r·² h¼r p¸ýpr¼ýrqZv³u³ur¦v`ry²iry¸ýtvýt ³¸³urpuh¼hp³r¼²³ur·²ry½r²³ur¼rh¼rt¸¸q¦h¼h·r³r¼²Zuvpuphýirs¸Ãýqi' hýh y'ªvýt ³ur²vªr ²uh¦r hýq ³ur¦¸²v³v¸ý¸s ihpxt¼¸Ãýq ¼rtv¸ý ²Ã¼¼¸Ãýqvýt ³urpuh¼hp ³r¼ v·htrA¸¼r`h·¦yrµ7¶uh²³Z¸u¸yr²¸ýrvý³uræ¦r¼¦h¼³hýq³ur¸³ur¼vý³ur y¸Zr¼¦h¼³0µP¶uh²¸ýy' ¸ýr u¸yrvý ³ur·vqqyr0µa¶uh²hyrs³shpvýtp¸ýph½r¼rtv¸ý h³ ³ur³¸¦ uhyshýqh¼vtu³shpvýt p¸ýph½r¼rtv¸ýh³³uri¸³³¸·uhys@hpuihpxt¼¸Ãýq ¼rtv¸ý v² s¸Ãýq hýq yhiryrq Zv³u ³ur p¸qr ¸s ¼rtv¸ý ³'¦r h² vý Avtür # Uuhý Zr phypÃyh³rq Cö² ²rp¸ýq hýq ³uv¼q ¸¼qr¼ prý³¼hy ·¸·rý³ vý½h¼vhý³² b(d ¸½r¼ ³ur²r ihpxt¼¸Ãýq ¼rtv¸ý²I÷ir¼ ¸srhpu³'¦r¸sihpxt¼¸Ãýq ¼rtv¸ý²¸½r¼³ur puh¼hp³r¼ ¼rtv¸ýv²Ã²rqh²srh³Ã¼r²³¸¸ Uur ý÷ir¼ ¸sp¸ýýrp³rq p¸·¦¸ýrý³²iy¸i²÷ výhpuh¼hp³r¼v·htrhy²¸ tv½r² ¼ryv hiyrsrh³Ã¼r²s¸¼Uüxv²upuh¼hp³r¼²²Ãpuh²µø¶µÿ¶hýqµg¶ Dýhqqv³v¸ý³¸ òr ³ur s¸Ã¼ srh³Ã¼rr`³¼hp³v¸ý·r³u¸q²výqv½vqÃhyy'0Zruh½rp¸· ivýrqFGUsrh³Ã¼r² Zv³u³ur¸³ur¼ ³u¼rr·r³u¸q²³¸²³¼rýt³u ²³h³v²³vphyFGUsrh³Ã¼r² Zv³u³urtr¸·r³¼vp¦¼¸¦r¼³vr²¸s³ur¦h³³r¼ý²
452
A. Çapar et al.
2.3 Classification The task of pattern classification is to assign an input pattern Z represented a feature vector X to one of the pre-specified classes 8 8 ! 8 " 8 F . After the feature set is defined and the feature extraction algorithm is applied, a typical recognition process involves two phases: training and prediction. Once the mapping into the feature space has been established, the training phase may begin. The recognition engine is adjusted such that it maps feature vectors (derived from the training set) into the categories with a minimum number of misclassifications. In the second phase (prediction phase), the trained classifier assigns the unknown input patterns to one of the categories based on the extracted feature vector. Fýrh¼r²³ ýrvtui¸¼ ýrh¼r²³ ·rhý 7h'r² Qh¼ªrý ²vªr qr¦rýqrý³ ýrth³v½r y¸t yvxryvu¸¸qT9IGG÷ hýq 6¼³vsvpvhy Irühy Ir³Z¸¼x² 6II÷ h¼r òrq s¸¼ pyh²²vsvph ³v¸ý F ýrh¼r²³ýrvtui¸¼ xýý÷ hýqýrh¼r²³·rhýpyh²²vsvr¼²qr¦rýq¸ýsvýqvýt³ursrh ³Ã¼r ½rp³¸¼ ¸s·výv·Ã·qv²³hýpr ³¸ ³urÃýxý¸Zýsrh³Ã¼r½rp³¸¼Dý¸¼qr¼³¸svýq³ur ýrh¼r²³srh³Ã¼r½rp³¸¼²r½r¼hyqv²³hýpr·rh²Ã¼r²h¼ròrqDý³uv²¦h¦r¼ ³ur@Ãpyvqrhý qv²³hýpr ·r³¼vpv² òrqb!d Dýxýýxv² h ¦¼rqr³r¼·výrq½hyÃrUurx ýrh¼r²³ýrvtu i¸¼²³¸³urÃýxý¸Zýsrh³Ã¼r½rp³¸¼h¼rs¸Ãýqhýq³ur½¸³r¸srhpupyh²²v²h²²vtýrq ³¸³urý÷ir¼ ¸sýrvtui¸¼²³ur' uh½rUurpyh²²vsvph³v¸ý¼r²Ãy³v²³urpyh²² Zuvpuuh² ³ur·h`v·Ã· ½¸³r ¸s x ýrh¼r²³ýrvtui¸¼²b!d Dýýrh¼r²³·rhýpyh²²vsvr¼³ur¼r²Ãy³v² ³urpyh²²uh½výt³ur·rhýZv³u³ur·výv·Ã·qv²³hýpr³¸³urÃýxý¸Zýsrh³Ã¼rb!d Bayes quadratic classifier depends on finding the class having the maximum discriminant function for the given unknown feature [2]. In designing the Parzen classifier, the method of Hummels and Fukunaga is used [10]. This method uses the following Gaussian kernel
x `÷ =
(10)
−q ÿ `÷
!π ÷ ÿ Γp u ÿ
r
!uÿ
.
Zur¼r q v² ³ur Hhuhy¸ý¸iv² qv²³hýpr Uur pyh²² p¸ýqv³v¸ýhy ¦¼¸ihivyv³' qrý²v³' h³ h ¦¸vý³v²r²³v·h³rqi'
¦ `÷ = ¦ ` _ Z÷ =
Ip
Ip
∑ x` − ` Õ =
p÷ Õ
÷.
(11)
C÷·ry²hýqAÃxÃýhthòrh³u¼r²u¸yq³s¸¼ ivh²r²Uur¼r h¼rs¸Ã¼qvssr¼rý³¸¦³v¸ý² s¸¼ ³ur²rivh²r² Uur¸ýròrqvý³uv²¦h¦r¼ h²²Ã·r²³urqv²³¼vióv¸ý²irýrh¼y' Bhò ²vhýUurivh²s¸¼³uv²ph²rv²
³=
u! ÷ yýΓp ÷ + ! yý Qp , ! u! + u +
so the discriminant function becomes
(12)
A Turkish Handprint Character Recognition System
tp ` ÷ = − yý ¦ p ` ÷ − ³ .
453
(13)
Uurpyh²²Zv³u³ur·h`v·Ã·qv²p¼v·výhý³½hyÃrv²³urpyh²²vsvph³v¸ý¼r²Ãy³ The SDNLL classifier is proposed by Hwang and Weng [11]. The idea behind this classifier is to use the available information as efficient as possible. There are three measures in terms of belongingness: likelihood, Mahalonobis distance, and Euclidean distance. In order to estimate the correct covariance matrices, it requires many samples. In the case of Mahalonobis, fewer samples are sufficient and in the case of Euclidean the least number is required. Therefore, a size dependent scatter matrix is proposed [11]. The resulting matrix is
Xv = Zr ρ ! D + Z· T Z + Zp Γv .
(14)
Zur¼r v² ³ur ²³hýqh¼q qr½vh³v¸ý D v² ³ur vqrý³v³' ·h³¼v` TZ v² ³ur Zv³uvý ²ph³³r¼ ·h³¼v` v² ³ur p¸½h¼vhýpr ·h³¼v` hýq ZrZ·Zp h¼r Zrvtu³² ¸s qv²³hýpr ·h³¼vpr² Uuò³urqv²p¼v·výhý³sÃýp³v¸ý v²¸i³hvýrqh² s¸yy¸Z²)
t p `÷ =
¹ − yý!π ÷ + yý Xv ÷ . ` − p ÷ U Xv − ` − p ÷ + ! ! !
(15)
Zur¼r¹v²³urpyh²²²vªrUurpyh²²vsvph³v¸ý¼r²Ãy³v²³urpyh²²Zv³u³ur·h`v·Ã·qv² p¼v·výhý³sÃýp³v¸ý In this study, we worked on an important class of neural networks, namely, multilayer feed-forward networks. Typically, the networks consists of a set of sensory units ( `v ) (source node) that constitute the input layer, one or more hidden layers of computation nodes ( ª w ), and an output layer of computation nodes ( ' x ) [12]. The input signal propagates through the network in a forward direction, on a layer-bylayer basis. These neural networks are commonly referred as multilayer perceptrons (MLPs), which represent a generalization of the single-layer perceptron. We used error back-propagation algorithm as training method for multi-layer feed-forward neural networks, which uses gradient-descent methods. Back-propagation training algorithm involves an iterative procedure for minimizing the specific cost function.
3
Experimental Results
In this section, the results of combining the feature extraction techniques and the classifiers are discussed by using the ITU Handprint Character Database, which includes about 20000 uppercase letters. 12644 of them have been used as training patterns while 6322 of them have been used as test patterns. Uursrh³Ã¼rr`³¼hp³v¸ý·r³u¸q²r`h·výrqvý³ur¦h¦r¼h¼rs¸¼·rqvý³urs¸yy¸Zvýt Zh' ar¼ývxrsrh³Ã¼r²h¼rphypÃyh³rq 泸³ur!³u ¸¼qr¼Zuvpu¦¼¸½vqr²#&qv·rý ²v¸ýhy srh³Ã¼r² Zuvyr 6SU srh³Ã¼r² h¼r phypÃyh³rq Zv³u ¸¼qr¼ " hýq ¼r¦r³v³v¸ý ! Zuvpu ¦¼¸½vqr² "% qv·rý²v¸ýhy srh³Ã¼r² A¸¼ tr¸·r³¼vp srh³Ã¼r² Zr uh½r r`³¼hp³rq ³¸³hyy' "(srh³Ã¼r²s¼¸· ³ur tr¸·r³¼vp ¦¼¸¦r¼³vr² ¸spuh¼hp³r¼²FGUsrh³Ã¼r²h¼rr`
454
A. Çapar et al.
³¼hp³rqs¼¸·rv³ur¼%# ¸¼!©rvtrý½rp³¸¼²Avýhyy' ³ursòrqsrh³Ã¼r²h¼rs¸¼·rq i' FGUsrh³Ã¼r²Zv³u%#rvtrý½rp³¸¼² UhiyrDvýqvph³r²³ur¦r¼prý³ pyh²²vsvph³v¸ý ¼r²Ãy³²s¸¼²³h³v²³vphyhýq ³r·¦yh³r ih²rq pyh²²vsvr¼² ½r¼²Ã² výqv½vqÃhy srh³Ã¼r ²r³² D³ v² ¸i²r¼½rq ³uh³ T9IGG tv½r² ³ur ir²³ ¼rp¸týv³v¸ý¼h³rs¸¼hyysrh³Ã¼r² r`pr¦³6SU¸s ¸¼qr¼ " Zv³u ¼r¦r³v³v¸ý ! Dý³uh³ ph²r Qh¼ªrýtv½r²³urir²³¼r²Ãy³Uurir²³¼r²Ãy³s¸¼³ur²rsrh³Ã¼r²r³²v² ¸i³hvýrqh² (" Uhiyr DD ¦¼r²rý³² ¦r¼prý³ pyh²²vsvph³v¸ý ¼r²Ãy³² s¸¼ ²³h³v²³vphy hýq ³r·¦yh³r ih²rq pyh²²vsvr¼² ½r¼²Ã² ³ur²r sòrq srh³Ã¼r ²r³² 6y³u¸Ãtu ³ur ¼r²Ãy³² h¼r Z¸¼²r ³uhý ³ur ¼r²Ãy³²¸sFGUsrh³Ã¼r²Zv³u ³urpyh²²vsvr¼²xII IHI hýq7h'r²¹Ãhq¼h³vppyh²²v svr¼ ³ur¼r v² v·¦¼¸½r·rý³ s¸¼ FGU Zv³u T9IGG hýq Qh¼ªrý Uur ¸½r¼hyy ¦r¼s¸¼· hýpr výp¼rh²r² i' òvýt hqqv³v½r ar¼ývxr ¸¼ 6SU srh³Ã¼r² Uur pyh²²vsvph³v¸ý ¼h³r irp¸·r² ("% s¸¼ FGU Zv³u 6SU hýq ("# s¸¼ FGU Zv³u ar¼ývxr srh³Ã¼r² i' òvýtT9IGGpyh²²vsvr¼ Table 1. Percent classification results for statistical and template based classifiers versus individual feature sets )HDWXUH FGU%#÷ &ODVVLILHU xIIx2 (!
FGU!©÷ (&
a@SIDF@#&÷ &(©
6SU"%÷ &&$
6SU%#÷ ©
xIIx2"
((
(%
©&
&©$
©(
xIIx2$
((
($
©(
&(
©
IHI
&$"
&$#
$©&
$%
$&&
7h'r²
(!!
(%
©!
©!(
©"$
Qh¼ªrý
(!"
((
©$
©#
T9IGG
©"
Table 2. Percent classification results for statistical and template based classifiers versus fused feature sets )HDWXUH &ODVVLILHU
FGUa@SIDF@
FGU6SU
FGU6SU
xIIx2
©"©
(
(!
xIIx2"
©#"
(
(
xIIx2$
©#&
(
(
IHI
%"%
&$#
&$#
7h'r²
((
(!
(!
Qh¼ªrý T9IGG
(!$
(!©
(!&
Uur¼r²Ãy³² s¸¼³ur6¼³vsvpvhy Irühy Ir³Z¸¼x6II÷pyh²²vsvr¼s¸¼³ursrh³Ã¼r²r³² FGU6SUFGUZv³u6SUhýq FGUZv³u6SUsrh³Ã¼r² h¼r²u¸Zý vý UhiyrDDDUur ¦r¼s¸¼·hýpr¸s sòrq srh³Ã¼r²v²hy²¸ ¸½r¼¦r¼s¸¼·rqFGUhýq6SUvýqv½vqÃhyy'Uur ir²³¼r²Ãy³s¸¼6IIv²¸i³hvýrqs¸¼FGUZv³u³urtr¸·r³¼vpsrh³Ã¼r²
A Turkish Handprint Character Recognition System
455
UurxII pyh²²vsvr¼uh²³urir²³¦r¼s¸¼·hýprs¸¼³urFGUsrh³Ã¼r¸sqv·rý²v¸ý %# Zur¼rýrh¼r²³·rhýuh²³urir²³¼r²Ãy³s¸¼ FGUZv³u qv·rý²v¸ý ¸s!© Dýhqqv³v¸ý ³urir²³¼rp¸týv³v¸ý¼h³rs¸¼ 7h'r² ¹Ãhq¼h³vp pyh²²vsvr¼ v² ¸i³hvýrq i' FGU srh³Ã¼r² Pý³ur¸³ur¼uhýqvý³urph²r¸sQh¼ªrýpyh²²vsvr¼hýqT9IGG³ursòrqsrh³Ã¼r²¸s FGUhýq6SUZv³u¸¼qr¼ ³u¼rrhýq¼r¦r³v³v¸ý³Zry½rtv½r³urir²³¼r²Ãy³ Table 3. Percent classification results for artificial neural networks versus feature sets )HDWXUH &ODVVLILHU
FGU"!÷
6II
4
©$(
FGU%#÷ ©©"
6SU %(
FGU6SU
FGU6SU
©(!
(#
FGUBr¸·r³¼vp
Conclusion
Dý ³uv² ¦h¦r¼ Zr²³Ãq' ¸ý³ur¼rp¸týv³v¸ý¸sUüxv²uæ¦r¼ph²r uhýq¦¼vý³puh¼hp³r¼² Xruh½rs¸¼·rq³ursv¼²³p¸·¦¼rurý²v½rUüxv²uuhýq¦¼vý³puh¼hp³r¼ qh³hih²rZuvpu p¸ý²v²³²¸shi¸Ã³!puh¼hp³r¼² Xr uh½r òrq qvssr¼rý³srhóürr`³¼hp³v¸ý ·r³u¸q²²Ãpuh² FGUar¼ývxr 6SU Dý hqqv³v¸ý tr¸·r³¼vp srh³Ã¼r² h¼r r½hyÃh³rq r²¦rpvhyy' s¸¼ Uüxv²u puh¼hp³r¼² H¸¼r¸½r¼FGUsrh³Ã¼r²h¼rsòrqZv³u ar¼ývxr6SU ¸¼tr¸·r³¼vpsrh³Ã¼r²vý ¸¼qr¼ ³¸výpyÃqr³ur²uh¦rvýs¸¼·h³v¸ý¸spuh¼hp³r¼² Dý ³ur²³Ãq' xýrh¼r²³ ýrvtui¸¼ýrh¼r²³·rhýýrvtui¸¼7h'r²¹Ãhq¼h³vppyh²²vsvr¼ Qh¼ªrý pyh²²vsvr¼ T9IGG hýq 6II h¼r òrq Uur ir²³ pyh²²vsvph³v¸ý ¦r¼prý³htr v² ¦r¼s¸¼·rqi' T9IGGUursòrqsrh³Ã¼r²tv½rir³³r¼ ¼r²Ãy³²³uhý³urvýqv½vqÃhy¸ýr² s¸¼T9IGG Qh¼ªrý hýq6II Uur ir²³ pyh²²vsvph³v¸ý¼r²Ãy³ ¸i³hvýrqvý ³uv²¦h¦r¼v² ("%
References 1. 2. 3. 4. 5. 6.
J. Hu, “HMM Based On-Line Handwriting Recognition”, IEEE Trans. On Pattern Analysis and Machine Intelligence, vol. 18, pp. 1039–1045, 1996. K. Fukunaga, Introduction to Statistical Pattern Recognition, 2nd ed., Boston: Academic Press, 1990 B. Verma, M. Blumenstein, and S. Kulkarni, “Recent Achievements in Off-Line Handwriting Recognition Systems”, School of Information Technology Griffith University – Gold Coast Campus PMB 50,1997. C.Y. Suen, et al., “Building a New Generation of Handwriting Recognition System”, Pattern Recognition Letters, vol. 14, pp. 303–315, 1993. P. J. Grother, “Karhunen-Loéve Feature Extraction For Neural Handwritten Character Recognition”, Image Recognition Group, National Institute of Standards and Technology, 1709, pp. 155-166, 1992. F. Zernike, Physica, vol. 1. p. 689, 1934.
456 7.
A. Çapar et al.
A. Khonzad, Y. H. Hong, “Invariant Image Recognition by Zernike Moments”, vol 12, No. 5, May 1990. 8. B.D. Ripley, Pattern Recognition and Neural Networks, Cambridge University Press, 1996. 9. O. D. Trier, A. K. Jain and T. Taxt, “Feature Extraction Methods For Character Recognition: A survey”, Revised July 19, 1995. 10. K. Fukunaga, and R.R. Hummels, “Bayes error estimation using Parzen and k-NN procedures”, IEEE Trans. Pattern Analysis and Machine Intelligence, vol.9, no 5, pp. 634–643, Sept. 1987 11. W.S. Hwang, J. Weng, “Hierarchical Discriminant Regression”, IEEE Trans. Pattern Analysis and Machine Intelligence, vol.22, no 11, pp. 1–17, Nov. 2000 12. A. K. Jain, J. Mao and K. Mohiuddin, “Artificial Neural Networks: A Tutorial”, Accepted to appear in IEEE Computer Special Issues on Neural Computing, March 1996.
Image-Space-Parallel Direct Volume Rendering on a Cluster of PCs B. Barla Cambazoglu and Cevdet Aykanat Bilkent University, Department of Computer Engineering, 06800, Ankara, Turkey {berkant,aykanat}@cs.bilkent.edu.tr
Abstract. An image-space-parallel, ray-casting-based direct volume rendering algorithm is developed for rendering of unstructured data grids on distributed-memory parallel architectures. For efficiency in screen workload calculations, a graph-partitioning-based tetrahedral cell clustering technique is used. The main contribution of the work is at the proposed model, which formulates the screen partitioning problem as a hypergraph partitioning problem. It is experimentally verified on a PC cluster that, compared to the previously suggested jagged partitioning approach, the proposed approach results in both better load balancing in local rendering and less communication overhead in data migration phases.
1
Introduction
In many scientific simulations, data is located at the vertices (data points) of a 3D grid that represents the physical environment. Unstructured datasets are a special case of grid-based volumetric datasets in which the data points are irregularly distributed and there is no explicit connectivity information. Direct volume rendering (DVR) is a popular volume visualization technique [1] employed in exploration and analysis of such 3D grids used by scientific simulations. The main aim in DVR is to map a set of data values defined throughout a 3D volumetric grid to some color values over a 2D image on the screen. DVR algorithms are able to produce high-quality images, but due to the excessive amount of sampling and composition operations performed, they suffer from a considerable speed limitation. Furthermore, memory requirements for some recent datasets are beyond the capacities of today’s conventional computers. These facts bring parallelization of DVR algorithms into consideration [9]. Image-space-parallel (IS-parallel) [5,8] or object-space-parallel (OS-parallel) [6,7] methods can be used for distributed-memory parallelization. OS-parallel methods decompose the 3D data into subvolumes and distribute these subvolumes to processors. Each processor works only on its assigned subvolume and produces a partial image using its local data. IS-parallel methods try to decompose the screen and assign subscreens to processors. Each processor computes only
This work is partially supported by The Scientific and Technical Research Council ¨ ITAK) ˙ of Turkey (TUB under project EEEAG-199E013.
A. Yazici and C. S ¸ ener (Eds.): ISCIS 2003, LNCS 2869, pp. 457–464, 2003. c Springer-Verlag Berlin Heidelberg 2003
458
B.B. Cambazoglu and C. Aykanat
a small but complete portion of the whole image on the screen. In IS-parallel DVR, screen decomposition and subscreen-to-processor mapping is important for successful load balancing and minimization of the data migration overhead. This work investigates IS-parallel DVR of unstructured grids on distributedmemory architectures, and focuses on load balancing and minimization of data migration overhead. In the following section, we start with a brief description of the underlying sequential DVR algorithm. In Section 3, we present the details of our parallel algorithm together with a description of the proposed screen partitioning model. In Section 4, we present some experimental results that verify the validity of the proposed work. We give our conclusions in Section 5.
2
Sequential Algorithm
The underlying sequential algorithm used in our IS-parallel DVR algorithm is based on Koyamada’s ray casting algorithm [4]. The algorithm starts with scanconverting all front-facing external faces. From each pixel found, a ray is shot into the volume and is followed until it exits from a back-facing external face, forming a ray segment (Fig. 1). Along a ray segment, some sampling points are determined. The number and location of the sampling points depend on the sampling technique used. In this work, mid-point sampling technique is used. At each sampling point, new sampling values are computed by interpolating the actual data values on nearby data points. Passing the sampling values from appropriate transfer functions, RGB color triples and opacity values are obtained. The color and opacity values found along a ray segment are composited in visibility order, using a weighting formula that assigns higher weights to values on points closer to the screen. Consequently, for each ray segment, a final color is generated. The generated color is put in the related pixel buffer to which the ray segment contributes. Due to the concavity of the volume, there may be more than one ray segments generated for the same pixel, and hence more than one color values may be stored in the same pixel buffer (two in the example of Fig. 1). After all ray segments are traced, the colors in pixel buffers are composited in visibility order and the final colors over the screen are generated. front-facing external face
back-facing external face
internal face
ray segment tetrahedral cell
screen sampling points along the ray
data points
Fig. 1. Ray-casting-based rendering of unstructured grids with mid-point sampling
Image-Space-Parallel Direct Volume Rendering
3
459
IS-Parallel DVR Algorithm
3.1
View-Independent Preprocessing
Parallelization Parameters
View-Independent Preprocessing Dataset Retrieval
Visualization Parameters
Dataset
The view-independent preprocessing phase, which is performed once at the beginning of the whole visualization process (Fig. 2), performs the operations that are independent of the current visualization parameters. These include retrieval of the data from disk, cell clustering and the initial data distribution. Since the data to be visualized is usually produced by simulations performed on the same parallel machine, it is assumed that the data is already partitioned and stored in the local disks of the processors. Hence, processors can read a subset of cells in parallel. After the data is read, processors concurrently perform a top-down clustering on their local cells. For this purpose, a graph-partitioning-based clustering approach is followed, where the cells and faces respectively form the vertices and edges of the graph representing the unstructured grid. In the implementation, state-of-the-art graph partitioning tool MeTiS [3] is used to partition this graph and create the cell clusters (subvolumes). Since clustering is performed just once, at the beginning of the whole visualization process, independent of the changing visualization parameters, the cost of cell clustering is almost negligible. The main idea behind cell clustering is forming groups of close tetrahedral cells and obtaining some volumetric chunks within the data such that the total surface area to be scan-converted is smaller. This way, the computational cost of screen workload calculations during the view-dependent partitioning phase is reduced. Furthermore, since the parallel algorithm works on cell clusters instead of individual tetrahedral cells, the housekeeping work is simplified, number of iterations in some loops are reduced, and some data structures are shortened. After cell clustering, an initial cluster-to-processor mapping is found. All the following view-dependent preprocessing phases utilize this initial cell cluster mapping. Even if a cell cluster may be temporarily replicated in other processors during the local rendering, it is statically owned by only a single processor. This static owner keeps the cluster’s data structures throughout the whole visualization process, and is responsible from sending them to the other processors.
View-Dependent Preprocessing Transformations
Cell Clustering
Workload Calculations
Initial Cell Cluster Mapping
Screen Partitioning
Cell Cluster Migration Point-to-Point Communication
Rendering Local Rendering Local Pixel Merging Image Generation
Subscreen Mapping Start Next Visualization Instance
Fig. 2. The proposed IS-parallel DVR algorithm
An Image Frame
460
B.B. Cambazoglu and C. Aykanat
3.2
View-Dependent Preprocessing
In IS-parallel DVR, the changing visualization parameters at each visualization instance require repartitioning of the screen and assignment of subscreens to processors for a load-balanced rendering. In order to decompose the screen, the rendering-load distribution over the screen pixels should be known beforehand. For this purpose, approximate rendering load of a cell cluster is estimated by summing the areas of its front-facing internal faces. This estimated rendering load for the cell cluster is evenly distributed over the pixels under the projection area of the cell cluster. After this process is repeated for all cell clusters, it becomes possible to know approximately how many samples will be taken along a ray fired from a specific pixel. Once the workload distribution over the screen is computed, the screen is decomposed into subscreens such that the total load of the pixels in subscreens will be almost equal. Here, the number of subscreens is chosen to be equal to the number of processors, so that each processor is assigned the task of rendering one of the subscreens. In our implementation, for efficiency purposes, an n by n coarse mesh, which forms n2 screen cells, is imposed over the screen. In this scheme, an individual screen cell corresponds to an atomic task which is assigned to a processor and is completely rendered by that processor. The set of screen cells assigned to the same processor forms a subscreen for the processor. A subscreen is not necessarily composed of geometrically connected screen cells. We model the the screen partitioning problem as a hypergraph partitioning problem. A hypergraph can be considered as a generalization of a graph, where each hyperedge can connect more than two vertices. In this model, the vertices of the hypergraph correspond to screen cells, that is, a vertex represents the atomic task of rendering a respective screen cell. Each vertex is associated with a weight, which is equal to the total rendering load of the pixels contained in the respective screen cell. Hyperedges of the hypergraph represent the cell clusters. Each hyperedge connects the screen cells which intersect the projection area of the corresponding cell cluster. Each hyperedge is associated with a weight, which is equal to the number of bytes needed in order to store the corresponding cell cluster. By partitioning the hypergraph into equally-weighted parts (subscreens), the model tries to minimize the total replication amount while maintaining a load balance in rendering of subscreens. In the implementation, our hypergraph partitioning tool PaToH [2] is used for partitioning the hypergraph. Moreover, we formulate the subscreen-to-processor mapping problem as a maximal-weighted bipartite graph matching problem. The K processors in the system and the K subscreens produced by hypergraph partitioning form the partite nodes of a bipartite graph. An edge connects a processor vertex and a subscreen vertex if and only if the processor stores at least one cell cluster of which projection area intersects with the subscreen. Each edge is associated with a weight, which is equal to the sum of migration costs of such cell clusters. By applying maximal-weighted bipartite graph matching algorithm to this graph, the total volume of communication during cell cluster migration is minimized.
Image-Space-Parallel Direct Volume Rendering
3.3
461
Cell Cluster Migration
After partitioning the screen and mapping subscreens to processors, the processors at which the cell clusters will be replicated are determined. Then, the cell clusters are migrated from their home processors to the new processors through point-to-point communication, according to the replication topology found. 3.4
Rendering
Once the cell cluster migration is complete, processors are ready to render their assigned subscreens in parallel. Most parts of the local rendering is similar to that of the sequential algorithm. The rays are followed throughout the data utilizing the connectivity information, which keeps the neighbor cluster, cell and face ids for the four faces in each cell. Although it is possible to have non-convex cell clusters as a result of the clustering algorithm, this does not cause an increase in the number of ray segments created. Existence of such non-convexities is eliminated due to replication of the cell clusters. However, the non-convexities in the nature of the dataset still require the use of ray buffers. Since, after the local rendering, all processors have a subimage, an all-to-one communication operation is performed and the whole image frame is generated in one of the processors.
4
Experiments
As the parallel rendering platform, a 24-node PC cluster is used. The processing nodes of the cluster are equipped with 64 Mb of RAM, and are interconnected by a Fast Ethernet switch. Each node contains an Intel Pentium II 400 Mhz processor and runs Debian GNU Linux operating system. The DVR algorithm is developed in C, using MPI as the communication interface. Each experiment is repeated over three different datasets using five different viewing parameter sets and the average values are reported. In the experiments, each processor is assigned 10 cell clusters, which is an empirically found number. As mentioned before, to make the view-dependent preprocessing overhead affordable, coarse meshes of varying size are imposed over the screen. In all experiments, the proposed hypergraph-partitioning-based (HP-based) model is compared with the previously suggested jagged-partitioning-based (JP-based) model. The details of the JP-based model can be found in [5]. Experiments are conducted over Blunt Fin, Combustion Chamber and Oxygen Post datasets obtained from NASA Research Center. Fig. 3 displays our renderings and the subscreen-to-processor mapping for different models. In each figure, the first image represents the actual rendering obtained using the standard viewing directions. The second and third images show the screen partitioning (by associating each subscreen with a different color) produced by the JP-based and HP-based models, respectively. The experiments which verify the solution quality are conducted at high number of virtual processors by allocating more than one executable instance per node. Fig. 4 displays the total volume of communication with the varying number
462
B.B. Cambazoglu and C. Aykanat
a) The Blunt Fin dataset
b) The Combustion Chamber dataset
c) The Oxygen Post dataset Fig. 3. Example renderings and subscreen-to-processor assignments of the datasets S=1200x1200
Communication Volume (Mb.)
200 JP HP
M=30x30
M=60x60
150
100
50
0 16
32
48
64
80 96 16 32 Number of Processors
48
64
80
96
Fig. 4. Total communication volume with the changing number of processors
Image-Space-Parallel Direct Volume Rendering S=1200x1200
80
Actual Load Imbalance (%)
463
JP HP
M=60x60
M=30x30
60
40
20
0 16
32
48
64
80 96 16 32 Number of Processors
48
64
80
96
Fig. 5. Load imbalance in local rendering with the changing number of processors
of processors. With a coarse mesh resolution of 60 × 60, using 96 processors, HP-based model results in around %30 less communication overhead than the JP-based model. When coarse mesh resolution is decreased to 30 × 30, there occurs a slight decrease in communication volume. This can be explained with the decrease in lengths of subscreen boundaries, and hence the decrease in the amount of overlaps between cell clusters and subscreen boundaries. Fig. 5 gives the load imbalance rates in local rendering phase. With a coarse mesh resolution of 60 × 60, using 96 processors, JP-based model results in a load imbalance of 40.7%. With the same parameters, load imbalance values for the HP-based model is 16.3%. When coarse mesh resolution is decreased to 30 × 30, both models perform worse in load balancing. This is due to the decrease in the number of screen cells, and hence the reduction in solution space.
S=1500x1500
S=900x900
20 JP HP
M=30x30
M=50x50
Speedup
15
10
5
0 4
8
12
16
4 20 24 8 Number of Processors
12
16
20
Fig. 6. Speedups at different screen and mesh resolutions
24
464
B.B. Cambazoglu and C. Aykanat
At 900 × 900 screen resolution, with 24 processors, our parallel algorithm is capable of producing around 12 image frames per minute. Fig. 6 displays the speedups achieved by the two models with the varying number of processors. At 24 processors, with a screen resolution of 1500 × 1500 and a coarse mesh resolution of 50 × 50, speedups are 16.1 and 17.3 for the JP-based and HP-based models, respectively. It is observed that the increasing screen resolution and number of processors favor the proposed HP-based model.
5
Conclusions
Compared to the jagged partitioning (JP) model, the proposed hypergraph partitioning (HP) model results in both better load balancing in the rendering phase and less total volume of communication during the data migration phase. The experiments conducted at the available number of processors indicate that HPbased model yields superior speedup values than the JP-based model. We should note that, HP step, which is carried out sequentially on each processor, is the limiting factor on the speedup values. A parallel HP tool, which will probably be implemented in the future, can eliminate this current drawback of our implementation. A nice aspect is that as the new heuristics are developed for HP and the existing HP tools are improved, the solution quality of the proposed model will also improve.
References 1. T. T. Elvins, A survey of algorithms for volume visualization, Computer Graphics (ACM Siggraph Quarterly), 26(3) (1992) pp. 194–201. ¨ V. C 2. U. ¸ ataly¨ urek and C. Aykanat, PaToH: Partitioning tool for hypergraphs, Technical Report, Department of Computer Engineering, Bilkent University, 1999. 3. G. Karypis and V. Kumar, MeTiS: A software package for partitioning unstructured graphs, partitioning meshes and computing fill-reducing orderings of sparse matrices, Technical Report, University of Minnesota, 1998. 4. K. Koyamada, Fast traversal of irregular volumes, in: T. L. Kunii (Ed.), Visual Computing, Integrating Computer Graphics with Computer Vision, Springer-Verlag New York, 1992, pp. 295–312. 5. H. Kutluca, T. M. Kurc, and C. Aykanat, Image-space decomposition algorithms for sort-first parallel volume rendering of unstructured grids, Journal of Supercomputing, 15(1) (2000) pp. 51–93. 6. K.-L. Ma, Parallel volume ray-casting for unstructured-grid data on distributedmemory architectures, in: Proceedings of the IEEE/ACM Parallel Rendering Symposium ’95, 1995, pp. 23–30. 7. K.-L. Ma and T. W. Crockett, A scalable cell-projection volume rendering algorithm for unstructured data, in: Proceedings of the IEEE/ACM Parallel Rendering Symposium ’97, 1997, pp. 95–104. 8. M. E. Palmer and S. Taylor, Rotation invariant partitioning for concurrent scientific visualization, in: Parallel Computational Fluid Dynamics ’94, 1994. 9. C. M. Wittenbrink, Survey of parallel volume rendering algorithms, in: Proceedings of the PDPTA’98 Parallel and Distributed Processing Techniques and Applications, July 1998, pp. 1329–1336.
Generalization and Localization Based Style Imitation for Grayscale Images )DWLK1DUDQG$WÕOÕPdHWLQ
,QIRUPDWLFV,QVWLWXWH0LGGOH(DVW7HFKQLFDO8QLYHUVLW\ IDWLKQDU#LLPHWXHGXWU 'HSDUWPHQWRI&RPSXWHU(QJLQHHULQJ%LONHQW8QLYHUVLW\ DWLOLP#FVELONHQWHGXWU Abstract. An example based rendering (EBR) method based on generalization and localization that uses artificial neural networks (ANN) and k-Nearest Neighbor (k-NN) is proposed. The method involves learning phase and application phase, which means that once a transformation filter is learned, it can be applied to any other image. In learning phase, error back-propagation learning algorithm is used to learn general transformation filter using unfiltered source image and filtered output image. ANNs are usually unable to learn filtergenerated textures and brush strokes hence these localized features are stored in a feature instance table for using with k-NN during application phase. In application phase, for any given grayscale image, first ANN is applied then k-NN search is used to retrieve local features from feature instances considering texture continuity to produce desired image. Proposed method is applied up to 40 image filters that are collection of computer-generated and human-generated effects/styles. Good results are obtained when image is composed of localized texture/style features that are only dependent to intensity values of pixel itself and its neighbors.
1 Introduction ,QUHFHQW\HDUVUHVHDUFKHUVZLWKLQFRPSXWHUJUDSKLFVDQGFRPSXWHUYLVLRQFRPPXQL WLHV SXW D JUHDW HIIRUW RQ H[DPSOH EDVHG UHQGHULQJ QRQSKRWRUHDOLVWLF UHQGHULQJ 135 DQGWH[WXUHV\QWKHVLV7KH\XVHYDULRXVPHWKRGVIURPRWKHUGLVFLSOLQHVVXFK DVPDFKLQHOHDUQLQJDQGLPDJHSURFHVVLQJ(%5LVDQXVHIXOWRROIRUFUHDWLQJDQLPD WLRQVDQGLPDJHVIRUYDULRXVSXUSRVHVVXFKDVLPLWDWLRQRIVW\OHVFUHDWLQJHGXFDWLRQDO VRIWZDUHVIRUFKLOGUHQDQGFUHDWLQJYLGHRJDPHVLQFDUWRRQZRUOGIRUHQWHUWDLQPHQW ZLWKOHVVKXPDQHIIRUW>@7KHVHVW\OHVHIIHFWVWKDWDUHDLPHGWRLPLWDWHFDQEHKX PDQJHQHUDWHGRUFRPSXWHUJHQHUDWHG 8OWLPDWHDLPRI(%5LVFUHDWLQJDQDQDORJRXVLPDJHIRUDJLYHQXQILOWHUHGLPDJH XVLQJXQILOWHUHGDQGILOWHUHGLPDJHSDLUV,QVKRUWLWLVVW\OHLPLWDWLRQDVVHHQLQ)LJ XUH,GHDOO\WKHILOWHUOHDUQHGIURPDQLPDJHSDLUPXVWEHVXIILFLHQWIRUFUHDWLQJDQ DQDORJRXVLPDJHIRUDQ\LPDJHHYHQWKHILOWHULVDFRPSOH[RUQRQOLQHDURQHVXFKDV DUWLVWLFVW\OHVRUWUDGLWLRQDOLPDJHILOWHUV>@ Varying edge definitions and brush strokes, crossing edge boundaries with varying sizes, textures, randomness, smoothness, and directions are used in different
$@ Edge detection, coarse levels, binarisation and similar image-processing operators have been tried with FFANN as input features to enhance training. Since all these operators are explained in introductory chapters of image-processing books [5], so no further detail will be given in the rest of this paper. Results and example outputs will be given in section 4. Conclusions and future works will be given in section 5.
Generalization and Localization Based Style Imitation for Grayscale Images
467
2 Learning Phase In learning phase error-backpropagation algorithm is used for training FFANN to find optimum network weights. Input patterns are taken from unfiltered input image A and output patterns are taken from filtered image A’ where two images are pixel wise registered. There are 9 units (3x3 kernel) in input layer and a unit in output layer and there are 21, 17, 13, and 9 units in 4 hidden layers as seen in Figure 2. i1 i2
h1 ...
i1 i2
h1
i9 Unfiltered image: A
i9
h1
h2
h21
input layer
h17
h13
h1
ok
ok
h9 Filtered image: A’
hidden layers
output layer
)LJ))$11VWUXFWXUHVKRZLQJUHODWLRQVKLSRILQSXWDQGRXWSXWSDWWHUQVZLWKLPDJHV
Image intensity values lay between –1 and +1 where they correspond to black and white accordingly. Incremental (stochastic) training strategy is used with number of 100K patterns that are taken from input and output images in random order where each pixel is used only once unless all pixels are used. Error is calculated using least mean square (LMS) as given in formula 1 where tk is desired output and ok is actual output, which are calculated in feed-forward phase.
K ( G Z = W N − RN ∑ N∈RXWSXWV
(1)
Network weights are initially selected from floating random numbers in the range of –1 and +1. Bipolar ok = tanh(.) is chosen as activation function, and it produces activation values between –1 and +1 which is also in the same range of image’s intensity values [2]. Incremental training strategy and bipolar activation function are chosen because they show better convergence and speed properties [4]. Weight update rule for output and hidden layers are given in formula 2 and 3 where η is learning rate and chosen as 0.01 in this study. ∆Z MN = η W N − RN − RN L MN (2)
∆ZLM = η − R M [LM
∑δ
V∈RXWSXWV
N
Z MN
(3)
ANN is applied to several training image pairs that are filtered with 40 different filters for learning transient filter F. For some simple effects generalization ability of ANN is sufficient for learning the filter without localization ability of k-NN as seen in Figure 3. Common properties of these filters are that they are relatively simple filters (i.e. there is no texture information that must be learned by ANN). Results of ANN can be enhanced by using extra input features such as gradient flows or coarse
)1DUDQG$dHWLQ
level images but experiments shows that it causes ANN to converge harder and total learning and application time is also increased.
)LJ2ULJLQDOLPDJHDQGRXWSXWLPDJHVHPERVVLQYHUVHVRODUL]H SURGXFHGE\$11
ANN is good at generalization but provides very weak results at memorization especially when filter contains extra information such as textures and brush strokes, which cause information-gain in output image. Difference of original image A and filtered image A’ can give us texture information as seen in top right image (D’) in Figure 4. Despite ANN cannot help us to memorize that textural information, it is still valuable tool for extracting this texture information better than simply finding D’. In bottom right image (Figure 4) you can see image D” which is the subtraction of filtered image A’ and output image A” that is produced by ANN. This new difference map D”, which acts as texture map for us, is convolved with 5x5 kernel and whenever the value of middle point in kernel is different than zero (or very near to zero), this pattern is stored in indexes H1 and H2 (intensities that are taken from A”: H1 and intensities that are taken from D, D equals to D’ or D”, itself: H2) for later retrieval with k-NN in application phase. In this pattern, 25 intensity values and position of patterns are kept. As you can see in image D’, total numbers of patterns that will be stored is much more comparing to image D” since image D’ contains much more nonzero intensity values comparing to image D”. Hence memory consumption and query time in application phase is dramatically decreased if image D” is used instead of using image D’ for image pairs A and A’ in Figure 4. So the method proposed in this paper is based on the observation that ANN provides good texture extraction from filtered and unfiltered images for texture synthesis for some cases. In other
Generalization and Localization Based Style Imitation for Grayscale Images
469
cases texture map D’ contains more nonzero intensity values comparing to D” so D’ is used instead of D” in such cases.
8QILOWHUHGVRXUFHLPDJH$
8QILOWHUHGVRXUFHLPDJH$
)LOWHUHGRXWSXWLPDJH$¶
'LIIHUHQFHRILPDJH$¶DQG$'¶
)LOWHUHG RXWSXW LPDJH YLD $11 'LIIHUHQFH RI LPDJH $¶ DQG $´ $´ '´
)LJ&UHDWLRQRIWUDLQLQJLQVWDQFHVWH[WXUDOSDWWHUQV IRUN11XVLQJ$11
In this study, training instances are arranged in dynamic arrays using their mean and variance of intensity values (5x5 kernel: 25 intensity values). In k-NN search whenever a new pattern is introduced, its mean value and variance is calculated and neighbor instances are searched according to this mean and variance values within predefined radius (i.e. [calculated mean – radius, calculated mean + radius]) via indexes H1 and H2. This assumption is based on the fact that mean and variance of a new instance is not too much different than mean and variance of its nearest neighbors in training instances. Resultant quality is decreased in small amount but query time is increased considerably. After learning filter F, using ANN and extracting difference (texture) map (appropriate one is selected from D’ or D” texture maps and named as D) the problem becomes a texture synthesis problem. Wei-Levoy proposed casual neighborhood for texture synthesis problem [7] and Hertzmann extends this approach for EBR in his image analogies study [6]. Our study is inspired from image analogies study by Hertzmann and latest developments about texture synthesis and makes extension to these studies with generalization ability of ANN.
)1DUDQG$dHWLQ
3 Application Phase In application phase we are trying to gather filtered image B’ from unfiltered image B. Normally image B” (here B” resembles to A” that is seen in Figure 4) is produced using ANN and then texture map T is synthesized using D and then final image B’ is gathered by adding images B” and T as seen in Figure 5. This filtering operation may occur in three ways. In first way, ANN can be successful to produce target image as seen in Figure 3 so image produced by ANN (B”) is considered as B’ and no further process is necessary. In second way, ANN is unsuccessful to extract texture so D’ is taken as D and B” is just equals to B, further processes are just same with third way. In third way, image B” is produced using ANN and then texture T is synthesized using D (which equals to D” for third way) and then final image B’ is gathered by adding images B” and T as seen in Figure 5. Image B” is convolved with 5x5 kernel and for each pattern that are taken from B” its k nearest neighbor (k=16) in difference map D is found (using H1 index). T is a texture information which we try to synthesis to merge with B” (by image addition) for producing final image B’. Here we want to make T resembles to D. Since T must contain continuous texture, k neighbor of T itself is found (using H2 index and casual neighborhood [7] as seen in Figure 6) and then T is produced using nearest neighbor of these two k neighbor set. For finding two k neighbor set, similarity measure is intensity values whereas for finding nearest neighbor of these two sets similarity measure is pixels’ positions. B
H2 D’ or D” (D)
B”
T
B’
H1
)LJ3URFHVVVFKHPDRIFUHDWLQJLPDJH%¶IURP%XVLQJLQGH[HV++DQGWH[WXUH'
Pixel intensity value at P(x, y) in image B’ is B’(x, y) = B”(x, y) + T(x, y) where values less than –1 are set as –1 and values greater than +1 are set as +1. We know the value of B”(x, y) since B” = F(B) or B. Only the value we do not know is T(x, y) and it can be found using the nearest neighbor pattern Pij from set S1 and S2 where distance metric is position of patterns in S1 and S2. S1 is found using k nearest neighbor pattern of B”(x, y) that exits in D(xi, yi) where i +
!#þ
!$þ
In the above, + is presented by decision maker. Using expression of KLM > + as D E
0LQ ∑∑µ F + [ þ F L + [! þ F M + [ [! þLMF þ L= M =
VXFK WKDW) −
− γ L [ L + βL @
(7)
EL @ = I [ L = [ L
Because of the dynamics, initial conditions on the state variables xi(0) must be specified. Most work in neural networks allows for bias or threshold weights with processing units. They are not explicitly shown above, but equivalent to weights from a constant unity input signal. This model is similar to some in the existing literature including [4],[15]-[17]. Differences exist in the treatment of inputs, in the form and relative placement of the nonlinearity. Other researchers have also used similar models for network with feedback connections, but have simplified the continuous-time dynamical aspect. Tsung [18] adopts a first order finite difference approximation to Eq.(7), and Williams and Zipser [19] use non- dynamical units (equivalent to Ti=0). Given a set of parameters, initial conditions, and input trajectories, the set of equations (6) and (7) can be numerically integrated from t=0 to some desired final time tf. This will produce trajectories over time for the state variables xi and the internal outputs yi. From these, the trajectory for z can be obtained. Point and periodic attractors can be realized. In practice, stability problems appear to rarely arise. The Eqs. (6) and (7) are a set of first-order nonlinear differential equations. We have used the fifth order Runge-Kutta method [20]. The integration time step has to be commensurate with the temporal scale of dynamics, determined the time constants Ti. In our work, we have specified a lower bound on Ti and have used a fixed integration time step of some fraction (e.g.,1/10) of this bound. Fig. 3 shows example of state space trajectories for two-unit networks with one constant unity input. The network parameters were:
Z
Z
E
X
S
7
[
V
[
X ]
K
\
T
S
T
SQ
TP ZQ
TQ
S
O
Z
X O
E
7
[
V
[
]P
ZQQ
Z
]
TPQ
SQ
O
S
ZQ
EL
K
T \
7L
[ L V
[L K [ γ β L L L L
\ L
Z
Fig. 1. State diagram of a two-unit SISO DNN
Fig. 2. General DNN model structure
604
Y. Becerikli
[
[
Fig. 3. State space trajectory of a DNN (periodic attractor)
S = Z T 7 . Initial conditions in the case were x1(0)=1.0, −
x2(0)=1.0. The network converges to a periodic attractor or limit cycle. The parameters for the network were adapted from [18], although due to differences in models network behavior is not identical. The integration time step was 0.01. We consider a particular application of dynamic neural networks: trajectory tracking for process simulation That is, we desire to configure the network so that its output initial conditions and input trajectories have been specified, but all the parameters SZT DQG T are adjustable (in this paper we do not discuss the issue of determining model structure, such as the number of neural network units). A performance index can be associated with the network for this task. For example: W - = ∫ >] W − ]G W @7 >] W − ]G W @GW I
(8)
It is easiest to think of zd(t) and z(t) as actual and modeled process responses, but in fact this formulation can serve many other purposes. For example, we can conceive of a dynamical neural network parameter estimator, where we would like the output of the network at time tf to equal some relevant parameter (e.g.,a time constant or gain) associated with the process that generated the data. In this case the performance index would simply be [] L W I − ]G L W I ] . Applications of feedforward networks to the parameter estimation of dynamical systems are described in [21]-[23]. The formulation is also easily extended to the case where multiple target trajectories, e.g., corresponding to different input trajectories and initial process conditions, are given.
4
Performance Index Minimization in Dynamic Neural Networks
Training dynamic neural networks to mimic a given set of trajectories is equivalent to determining network parameters so that the performance index is minimized. This is a dynamic nonlinear optimization problem. Our focus in this paper has been on gradient-based algorithms for solving this problem. Abstractly, the algorithms consist of iterating the following three steps: 1.) Simulate the dynamic neural network from specified initial conditions, using specified input trajectories. This requires the
Nonlinear Filtering Design Using Dynamic Neural Networks with Fast Training
605
solution of the n differential equations (7) with current network parameter values. 2.) Compute the performance index gradients given the target trajectory. 3.) Compute new network parameters. These steps are repeated until the performance index is sufficiently small. Initially, all parameters are set to specified or randomly-generated values. In this section, we discuss two approaches each to Steps 2 and 3: gradient computation and parameter updating. We require the computation of gradients or sensitivities of the performance index with respect to the various parameters: ∂( ∂( ∂( etc.. An expression for the ∂SLM ∂Z LM ∂7L
output weight gradients is readily obtained by differentiating Eqs.(8) and (5): ∂] ∂= ∫ >] W − ]G W @ GW = ∫ H W \ GW ∂T ∂T WI
WI
L
L
LM
L
L
LM
(9)
M
The simulation of the dynamic neural network gives us the trajectories z(t) and yi(t), from which the above expression can be evaluated. The remaining gradients require the solution of additional sets of linear differential equations. Two methods which are forward and adjoint sensitivity analysis can be used. Here the second one used is described below. Adjoint Sensitivity Analysis: The forward sensitivity analysis approach results from viewing dynamic network training as the minimization of performance index (8) subject to the constraints imposed by Eq. (7). Efficient another approach to solving constrained optimization problems is based on the use of Lagrange variables. Under this formulation, an alternative “Adjoint’’ method for sensitivity computation can be derived [24]-[26]. A new dynamical system is defined with adjoint state variables λi: − λ L = − λ L + 7L 7L
∑ Z LM \ ′M λ M + H L W ∑ T LM \ ′M λ L W I = M
(10)
M
The size of adjoint vector is n and is independent of network parameters. There are n quadratures for computing the sensitivities. The integration of the differential equations must be performed backwards in time, from tf to 0. We have used 5th order Runga-Kutta-Butcher integration rule [20]. Let p be a vector containing all network parameters. Then, the cost gradients w.r.t. parameters are given by the following quadratures; WI
∂= ∂S ∫
( ) λGW ∂I 7 ∂S
(11)
Some of the cost gradients as in [24]-[26] are as bellows; λ L \ M , ∂∂λ GW = = ∫ GW ∂Z LM ∫ 7L 7 ∂E WI
WI
L
L
L
WI
Q O λ ∂= − ∫ L >− [ L + ∑ Z LM \ M + ∑ S LM X M + E L @GW ∂7L 7 M= M= L
WI
λ ∂= ∑ L Z LM [ M K M − K M GW ∂γ M ∫ L 7L
All other gradients can be easily derived.
(12) (13) (14)
606
Y. Becerikli
Equation (10) specifies final conditions. The integration must therefore be performed backwards in time, from tf to 0. A general backwards integration capability is likely to require a more sophisticated approach. The published literature on the adjoint method and dynamic networks is small. Pearlmutter [16],[24]-[27] describes an approach to trajectory learning that is similar to ours. Barhen, Toomarian and Gulati [17] give a theoretical treatment of the adjoint method for training dynamic neural networks to desired steady-state attractors. The complexities of backward integration are not addressed in either. The examples discussed in [16] employ constant inputs, a special case in which the first-order integration scheme used appears to suffice. Constrained Levenberg-Marquardt Algorithm: We would like to find values for S (this can be considered as a vector obtained by concatenating p, the m parameters a w and T, etc) for which the quadratic performance index between network trajectory z(t) and desired trajectory zd(t) is minimized. We assume that, at each iteration, gradients of the performance index J with respect to all network parameters have been computed. Here we describe an algorithm we have used for updating parameter values based on this gradient information. A simple gradient-descent approach can be applied to update parameters: ∆a S = −η∇ aS -
(15)
where η is a step size parameter. Gradient descent is used in the majority neural network applications, usually under the banner of “backpropagation’’. Instead of simple gradient method, we used the well-known Levenberg-Marquardt technique [20],[28]. The extension consists of the incorporation of a method for handling constraints on parameter values. This Levenberg Marquardt (LM) algorithm has been used extensively for parameter estimation in linear continuous-time systems (a nonlinear parameter estimation problem). The parameter update is computed as:
[
]
Ö + µ, − ∇ a ∆a S=−+ S
(16)
I is the mxm identity matrix, and + is an approximation to the Hessian matrix of S ). The exact the response (the matrix of second derivatives of z with respect to a Hessian is computationally prohibitive. The Levenberg-Marquardt scheme approximates it as; ∂] ∂] ∂ ] ≅ ∂a SL ∂a S M ∂a SL ∂a S M
7
(17)
µ in Eq. (16) is a parameter that controls the parameter step size and the extent to which second-order information influences the update. For large µ, Eq. (16) essentially implements a gradient-descent rule. µ is updated automatically by the algorithm based on the error history. Thus the Levenberg-Marquartd algorithm implements a continuously varying hybrid of gradient descent and an approximate second order optimization. This algorithm is recommended for small and medium-sized systems. For significantly larger systems, the matrix inversion in Eq. (16) requires careful
Nonlinear Filtering Design Using Dynamic Neural Networks with Fast Training
607
treatment. Singular value decomposition and iterative inversion techniques are available (and in fact have been employed in the neural networks community) and these should facilitate scale-up. It can be shown that the exact system Hessian can be computed economically using both the adjoint and response gradients [29]. Thus by performing both the forward and adjoint sensitivity analyses, an exact Newton method can be implemented at substantially lower cost than that involved in the “forward’’ computation of exact second derivatives.
5
Filter Design with DNN
In this study, Equ. (3) has been used as a nonlinear filter (NLF). For linear filter (LF), Equ. (4) will be used. DNN and desired filter are trained as in Fig. 4.
XW
1/)RU%3) '\QDPLFV
%:Z IRU%3)
]GW
S
S
3HUIRUPDQFH
]W HW
'11 '\QDPLFV
-S
0HDVXUHPHQW
'11
DQG 6HQVLWLYLW\
SN
1RQOLQHDU 2SWLPL]HU
$QDO\VLV
∂-∂S
7UDLQLQJ $OJRULWKP
SN
Fig. 4. Schematic diagram of DNN training as NLF and BPF
6
Simulation Results
In the simulation, for target NLF, a=0.6, b=3 and c=1/3 were selected, and the input function as in Fig. 5 is used for adjusting the DNN parameters. *LULú)RQN
X
W
Fig. 5. Input function for NLF training with DNN
DNN parameters were randomly selected, but its initial conditions were used the same as NLF. After the training, to test the DNN filter performance, another input as in Fig. 6 (a) is applied to both converged DNN and original NLF. DNN and NLF outputs are shown in Fig. 6 (b). As the last example, DNN was used as BPF using some values (w0,BW) as indicated in Table 1 and the input in Fig. 5.
608
Y. Becerikli
Outputs
Test Input
1
4
2 0.5
0 0
u
x -2 -0.5
1/) '11
-4
-1
-6
-1.50
1
2
3
4
5 t
6
7
8
9
-8 0
10
1
2
3
4
5 t
6
7
8
9
10
Fig. 6. NLF testing for DNN (a) Test input trajectory (b) DNN and NLF outputs Table 1. BPF parameters used for training DNN as BPF BW(rd/sec) 2.103 103 5.102 102
w0(rd/sec) 104 5.104 105 5.105
The values of center frequency w0 and band width (BW) in Table 1 were used to be mixed in time during training. To test, w0=2.104Hz, BW=750Hz and the input trajectory as in Fig. 7 (a) were applied to DNN. The result is excellent shown in Fig. 7 (b) Test Input
2
outputst
1.5
1
0.5 u
X
0
-0.5
-1
-1.5 -2 0
BPF
0.5
1
(a)1.5
2
2.5 t
3
3.5
4
5
4.5
-3
x 10
DNN
(b)
W
[
Fig. 7. Testing of DNN as BPF (a) Test input trajectory (b) DNN and BPF outputs
7
Conclusions
It I shown that analog linear and nonlinear filters can be implemented with DNN. A novel class of algorithm that combine the updating scheme with nonlinear approximate second order gradient descent class of algorithm which is LM has been proposed for training DNN as filter. Any nonlinear physical dynamic system such as
Nonlinear Filtering Design Using Dynamic Neural Networks with Fast Training
609
nonlinear filter used above can be captured by DNN. Simulation results show that the dynamic network structure can grow more accurate by dynamic neuron approximators. High order linear and nonlinear filters can be considered to be implemented.
References [1]. Pitas, I., "Nonlinear Digital Filters, Principles and Applications", Kluwer Academic Publishers, 1990. [2]. Khalil, H.K., “Nonlinear Systems”, Prentice-Hall, Inc., 1996. [3]. Hopfield, J.J., “Neural networks and physical systems with emergent collective computational abilities.’’ Proc. of the National Academy of Sciences, 79, pp. 2554– 2558, 1982. [4]. J.J., Hopfield, J.J., “Neurons with graded response have collective computational properties like those of two-state neurons.’’ Proc. of the National Academy of Sciences, 81, pp. 3088–3092, 1984. [5]. Konar, A.F., and Samad, T., “Dynamic Neural Networks”, Technical Report SSDC-92I 4152-2, Honeywell Technology Center, 3660 Technology Drive, Minneapolis, MN 55418, 1992. [6]. Konar, A.F., Becerikli, Y., and Samad, T., “Trajectory Tracking with Dynamic Neural Networks”, Proc. of the 1997 IEEE Int. Sym. On Intelligent Control (ISIC’97), Ýstanbul 1997. [7]. Lippmann, R.P., “An Introduction to Computing with Neural Nets”, IEEE, ASSP Magazine, pp.4–22, 1987. [8]. Haykin, S., “Neural Networks, A Comprehensive Foundation”, Prentice-Hall, Inc., 1999. [9]. Cichocki, A., and Unbehauen, R.,: “Neural Networks for Optimization and Signal Processing”, John Wiley & Sons. Ltd. & B.G. Teubner, 1993. [10]. Scales, L.E.,: “Introduction to Nonlinear Optimization”, Springer-Verlag New York Inc., 1985. [11]. Jordan, M.I., “Attractor dynamics and parallelism in connectionist sequential machine.’’ Proc. of the Eighth Annual Conference of Cognitive Science Society. Hillsdale, N.J.: Lawrence Erlbaum Associates, 1986. [12]. Elman, J.L., “Finding Structure in Time’’. Technical Report 8801, Center for Research in Language, University of California, San Diego, La Jolla, CA 92093, 1988. [13]. Servan-Schreiber, D., Cleeremans, A., and McClellans, J.L. “Encoding Sequential Structure in Simple Recurrent Networks.’’ Technical Report CMU-CS-88-183, School of Computer Science, Carnegie Mellon University, Pittsburg, PA 15213, 1988. [14]. Morris, A.J., “On artificial neural networks in process engineering.’’ IEE Proc. Pt. D. Control Theory and Applications, 1992 [15]. Pineda, F., “Generalization of backpropagation to recurrent and higher order neural networks.’’ Proc. of the IEEE International Conference on Neural Networks, 1987 [16]. Pearlmutter, B., “Learning state space trajectories in recurrent neural networks.’’ Neural Computation, 1, pp. 263–269, 1989. [17]. Barhen, J., Toomarian, N., and Gulati, S., “Adjoint operator algorithms for faster learning in dynamical neural networks’’. In Advances in Neural Information Processing Systems 2, D.S. Touretzky (Ed.) San Mateo, Ca.: Morgan Kaufmann, 1990. [18]. Tsung, F-S. “Learning in recurrent finite difference networks. In Connectionist Models:’’ Proceedings of the 1990 Summer School, D.S. Touretzky et al. (Eds.) San Mateo, Ca.: Morgan Kaufmann, 1990.
610
Y. Becerikli
[19]. Williams, R.J. and Zipser, D., “A Learning Algorithm for Continually Running Fully Recurrent Neural Networks.’’ ICS Report 8805, Institute for Cognitive Science, University of California, San Diego, La Jolla, Ca. 92093, 1988. [20]. Chapra, S.C., and Canale, R:P., “Numerical Methods for Engineers”, McGraw-Hill, Inc., 1989. [21]. Samad, T., and Mathur, A., “System Identification with neural Networks.’’ Technical Report CSDD-89-I4920-2, Honeywell SSDC, 3660 Technology Drive, Minneapolis, MN 55418, 1988. [22]. Samad, T., and Mathur, A., “Parameter estimation for process control with neural networks.’’ International Journal of Approximate Reasoning, 1992. [23]. Konar, A.F., and Samad, T., “Hybrid Neural Network/ Algorithmic System Identification.’’ Technical Report SSDC-91-I4051-1, Honeywell SSDC, 3660 Technology Drive, Minneapolis, MN 55418, 1991. [24]. Konar, A.F., and Samad, T., “Dynamic Neural Networks.’’ Technical report SSDC-92I 4142-2, Honeywell Technology Center, 3660 Technology Drive, Minneapolis, MN 55418, 1992 [25]. Becerikli, Y., “Neuro-Optimal Control”, Ph.D Thesis, Sakarya University, Sakarya, Turkey, 1998 [26]. Becerikli, Y., Konar, A.F. and Samad, T., “Intelligent optimal control with dynamic neural networks” Neural Networks, vol.16(2), pp.251–259, March, 2003 [27]. Leistritz, L, Galicki, M, Witte, H., and Kochs, E., “Training Trajectories by Continuous Recurrent Multilayer Networks”, IEEE Trans. on Neural Networks, Vol.13, no.2, March 2002 [28]. Marquardt, D.W:, “An algorithm for least-squares estimation of nonlinear parameters’’ J. Soc. Ind. Appl. Math., 11, pp. 431–441, 1963. [29]. Konar, A.F., “Gradient and curvature in nonlinear identification.’’ Proc. Honeywell Advanced Control Workshop, January 21–22, 1991.
Prediction of Protein Subcellular Localization Based on Primary Sequence Data 1 ¨ Mert Ozarar , Volkan Atalay1 , and Reng¨ ul C ¸ etin Atalay2 1 2
Dept. of Computer Eng., Middle East Technical University, Ankara, TURKEY {ozarar,volkan}@ceng.metu.edu.tr, Dept. of Molecular Biology and Genetics, Bilkent University, Ankara, TURKEY
[email protected],
Abstract. This paper describes a system called prediction of protein subcellular localization (P2SL) that predicts the subcellular localization of proteins in eukaryotic organisms based on the amino acid content of primary sequences using amino acid order. Our approach for prediction is to find the most frequent motifs for each protein (class) based on clustering and then to use these most frequent motifs as features for classification. This approach allows a classification independent of the length of the sequence. Another important property of the approach is to provide a means to perform reverse analysis and analysis to extract rules. In addition to these and more importantly, we describe the use of a new encoding scheme for the amino acids that conserves biological function based on point of accepted mutations (PAM) substitution matrix. We present preliminary results of our system on a two class (dichotomy) classifier. However, it can be extended to multiple classes with some modifications.
1
Introduction
Eukaryotic cells are subdivided into functionally separate membrane enclosed compartments. Each compartment and its vicinity contain functionally linked proteins related to the activity of that cell compartment [1]. In an eukaryotic cell, each protein is targeted to its specific cell localization where it is functionally active. Large scale genome analysis provides high number of putative genes to be characterized. Therefore, prediction of the subcellular localization of a newly identified protein is invaluable for the characterization of its function. Furthermore, studying subcellular localization is useful for understanding the disease mechanisms and developing novel drugs. If the rules for the prediction were biologically interpretable, this knowledge could help in designing artificial proteins with desired properties. Hence, a fully automatic and reliable prediction system for protein subcellular localization would be very useful.
This work was supported by the Turkish Academy of Sciences to R.C ¸ .A. (in the ¨ ˙ framework of the Young Scientist Award Program-RCA/TUBA-GEB IP/2001-2-3).
A. Yazici and C. S ¸ ener (Eds.): ISCIS 2003, LNCS 2869, pp. 611–618, 2003. c Springer-Verlag Berlin Heidelberg 2003
612
¨ M. Ozarar, V. Atalay, and R.C ¸ . Atalay
The aim of this work is to design and develop a system called prediction of protein subcellular localization (P2SL) that predicts the subcellular localization of proteins in eukaryotic organisms based on the amino acid content of primary sequences. The amino acid composition in the full or partial sequences can be taken as a global feature and the order may represent the local features such as the sequence order of amino acids that are found in protein sequence motifs [2]. In this paper, we are interested in the prediction using only local features. Our approach for prediction is to find the most frequent (hopefully the most significant) motifs for each protein (class) based on clustering and then to use these most frequent motifs as features for classification. This approach allows a classification independent of the length of the sequence. Another important property of the approach is to provide a means to perform reverse analysis and analysis to extract rules. In addition to these and more importantly, we describe the use of a new encoding scheme for the amino acids that conserves biological function based on point of accepted mutations (PAM) substitution matrix [3]. PAM is used to score aligned peptide sequences to determine the similarity of these sequences. The scores in PAM are derived by comparing aligned sequences of proteins with known homology and determining the observed. By using PAM substitution matrix, we believe that we are able to represent the chemical differences of each amino acid in protein sequences. In the literature, each amino acid is traditionally represented in binary form independent of their chemical properties. In this study, we present preliminary results of our system on a two class (dichotomy) classifier. However, it can be extended to multiple classes with some modifications. The manuscript is organized as follows. In Section 2, we indicate the related studies. The data and computational methods used in this study are presented in Section 3. Experiments and comments on the results are then explained in the subsequent section. Finally, the eventual improvements are indicated together with conclusions and future work.
2
Related Work
Several attempts have been made to predict protein subcellular localization. Most of these prediction methods can be classified into two categories: one is based on the recognition of protein N-terminal sorting signals and the other is based on amino acid composition. PSORT was developed based on rules for various sequence features of known protein sorting signals [4]. iPSORT was developed based on a decision tree with an amino acid index rule and an alphabet indexing and pattern rule [5]. TargetP was developed based on neural networks and achieved high prediction accuracy [6]. There are several studies to predict the specific localizations. MitoProt was developed for analyzing mitochondrial proteins and MitoProt II was for predicting the mitochondrial ones [7]. MTS prediction was based on hidden Markov models for mTPs [8]. SignalP was developed based on neural networks for SP prediction [9]. ChloroP was developed based on neural networks for chloroplast
Prediction of Protein Subcellular Localization
613
targeting peptide prediction [10]. SortPred was developed for using both neural networks and hidden Markov models for four class problem of subcellular localization [11]. The prediction accuracy of SortPred is 86.4% for plant and 91.3% for non-plant, while the accuracy of TargetP and iPSORT is 85.3% and 84.5%, respectively for plant and 90% and 88%, respectively for non-plant. Among them, PSORT and iPSORT use global features, on the other hand MitoProt and MTS use local features. TargetP, SignalP, ChloroP and SortPred use both global and local features. A combination of clustering followed by classification seems to be the most important aspect of this study. P2SL dichotomizer prediction results are very promising compared to the experimental results indicated in the literature. Our ultimate goal is to design a system that performs predictions only on human proteins yielding comparable results with TargetP. When compared to the studies in the literature, – we use windows (motifs) similar to the work by Emanuelsson et.al. [6], – we use self organizing maps (SOM) similar to the work by Cai et.al. [12], – we use a novel encoding scheme based on PAM [13].
3
Methods
The main idea for the prediction of protein subcellular localization using local features is based on finding the substrings which are common for a protein class and infrequent for the other classes. Such substrings are called as motifs. We use a self organizing map for this purpose. For an unknown input sequence, we determine which motifs exist and the sequence is classified according to this information. The flow diagram of P2SL is illustrated in Figure 1. The input to the system is amino acid sequences. The sequences are extracted from this data. The primary sequence is then decomposed into substrings. Each substring is encoded with PAM250 [13] substitution matrix. We apply clustering on the encoded substring via a self organizing map. During the training phase, motifs for each class are determined. Throughout the test phase, when the substrings of an unknown input sequence is given, according to the winning nodes in the self organizing map, we form a binary vector which indicates the existence of motifs of a particular class in the input. k-nearest neighborhood classifier is then applied to this binary vector to determine the label of the unknown protein. 3.1
Data Representation
Protein sequences are strings of arbitrary size and amino acids correspond to ˆ represent a protein sequence whose the letters in a protein sequence. Let X ˆ ˆ length is len(X). X can be decomposed into substrings of some fixed length, κ. ˆ there are exactly (len(X) ˆ − κ + 1) substrings in X. ˆ X(j ˆ : m + j) If κ < len(X), th ˆ then denotes j substring in a protein sequence X. In order to perform further computational analysis, we need to encode the amino acids. Although, the most
614
¨ M. Ozarar, V. Atalay, and R.C ¸ . Atalay
Fig. 1. Flow diagram of P2SL.
popular way of encoding reported in the literature is to represent each amino acid in binary form, in this study, we make use of substitution matrices. While aligning two protein sequences, certain methods are used to score the alignment of one residue against another. Substitution matrices indicate score values for this purpose. We employ PAM250 scoring matrix to encode an amino acid. In ˆ the rest of the manuscript, X denotes a PAM encoded protein sequence X. The frequencies of these mutations are in this table as a ”log odds-matrix” where: Mij = 10(log 10 Rij ), In this equation, Mij is the matrix element and Rij is the probability of that substitution as observed in the database, divided by the normalized frequency of occurence for amino acid i. All of the number are rounded to the nearest integer. The base 10 log is used so that the numbers can be added to determine the score of a compared set of sequences, rather than multiplied. 3.2
Clustering
We use a self organizing map (SOM) for clustering. SOM is an unsupervised artificial neural network model that relates similar input vectors to the same region of a map of nodes or neurons [14]. Topological organization of neurons in SOM is its essential feature, since it does a topological ordering of the input. SOM is often used to categorize and interpret data, by mapping a high-dimensional input data to a lower dimensional space which is usually 2. Each input data is composed of a vector of elements. The map is an array of nodes which are also called neurons and it is often laid out in a rectangular or hexagonal lattice. Each node has an associated reference vector of the same size as input feature vectors. Input vectors are compared with these reference vectors. There are two phases of a SOM: training and testing. For training, all the input vectors, one at a time are presented to the network. Each input vector is compared to weight vectors associated with every neuron. The neuron having the weight vector with
Prediction of Protein Subcellular Localization
615
the smallest difference to the current input vector becomes the winning neuron. The weight vector of this winning neuron and those of the neighboring neurons as well are then updated in the direction of the input vector. Note that in this training scheme similar input vectors are mapped to nodes that are close together on the map. This, in fact generates both the topological ordering and clustering properties of a SOM. In the training phase, there is no update in the weight vectors and therefore we are only interested to determine to which of the nodes (cluster) the input data is mapped. In our system, clustering occurs during training and in this phase substrings are topologically grouped. The problem of finding motifs of a protein class turns out to finding the nodes specific to a class. At the end of training, these nodes are found by simply looking at the difference of the number of substrings of two classes assigned to each node. “assigned” is used in the sense that for an input vector X, SOM cell i is the winning node. Y Let cX i and ci denote the cardinal of the set of substrings of class X and class Y, respectively assigned to the cell i. If the size of the two dimensional SOM is m by n, then there are m × n nodes. The difference of substrings of the two classes assigned to cell i is then Y Y Y X X X ∆cX i = (ci − ci ) and ∆ci = −∆ci = (ci − ci )
Let the sets of SOM nodes to which motifs of class X and Y are assisgned during the training phase be P X and P Y , respectively. P X and P Y are deterX then i ∈ P X , otherwise i is not in P X where τ X mined as follows. If ∆cX i >τ Y is a threshold value emprically determined. Similarly, If ∆cY then j ∈ P Y , j >τ otherwise j is not in P Y . Remark that we choose τ X and τ Y such that there are equal number s of elements in P X and P Y . 3.3
Classification
The k nearest neighbor (kNN) classification method is one of the most popular classification methods in pattern recognition. In kNN, the class label of an unknown sample is determined by the majority of class labels of the k nearest training samples to the unknown sample. Nearest neighbor method using only one nearest training sample is called 1-Nearest Neighbor (1NN) method. On the other hand, that using k > 1 nearest training samples is called k-nearest neighbor (kNN) method. Compared to linear or quadratic classifers, the nearest neighbor method is more effective for a complex distribution of samples. On the other hand, kNN method requires more computational cost than 1NN method. After clustering, the next step is classification. For this purpose, for each training input X a binary vector Z of 2s elements is formed as follows. The elements 0, . . . , s − 1 represent class X while those s, . . . , 2s − 1 represent class Y. If the winning node corresponding to input substring X(j : j+κ) is lth element of P X , then Z(l) = 1. Similarly, for class Y, if the winning node corresponding to input substring X(j : j + κ) is lth element of P Y , then Z(s + l) = 1. Note that final form of Z is obtained after processing all of the available substrings in a protein sequence X.
616
¨ M. Ozarar, V. Atalay, and R.C ¸ . Atalay
Assume that Z represents the binary vector for a protein sequence in the training set and Z for that in the test set. For each protein i in the test set, Hamming distance between corresponding Zi and Zj (for all of the elements in the training set) is calculated. Then, k proteins of the training set having the least Hamming distance to Zi is checked. Suppose that among those k protein sequences, there are q proteins belonging to class X and r proteins belonging to class Y. Hence, q+r=k. Then, a voting mechanism takes place: if q > r, then the ith element of the test set is classified as of class X . If r > q, then the ith element of the test set is classified as class of Y. Since, the k is chosen as an odd value, there is no chance of q = r.
4
Material, Results, and Discussions
In our experiments, we first employ our method to a previously published data set [6]. By using this data set, we have generated four data subsets (classes), signal peptide containing proteins (SP class), nuclear proteins (NP class), cytosolic proteins (CP class) and mitochondrial proteins (MP class), of which only two (SP and NP) are used in this study. Both classes are represented in Fasta format. From each class, 40 proteins are randomly chosen for training from the input data. Randomly chosen 20 proteins from each class exclusive of those in the training set are used for testing. Substrings of size κ = 30 are extracted from the protein sequences. Therefore, 18815 SP substrings and 23290 NP substrings are formed. For the self organizing map, SOM-PAK [15] is used as a computational tool. We have tried different map sizes, topologies and neighborhood functions. Experiments yield that a SOM with map size 25x25, randomly initialization, rectangular topology and Gaussian neighborhood function gives better results for our problem. Both ordering and fine-tuning training is used with 10000 and 50000 epochs, respectively. Figure 2 shows the difference of histograms of SOM nodes after the training phase. The peaks are obvious for both classes, but the important nodes are the ones in which a high number of samples exists for one class yet not a significant value exists for the other class. For example, the difference value for node (0,0) is approximately 40. This shows that unless there are cells with lower difference values, it shall not be treated as a particular node for both classes. However, for instance, the frequency difference between SP and NP is approximately 90 for the node (24,0) and it is one of the specific cells for the SP class. By keeping the thresholds (τ X = τ Y ) as 20, (16,7), (14,5), (16,8), (17,4), (17,8), (12,8), (17,5), (18,9), (19,9), (19,6), (15,8), (17,7), (13,8), (19,8), (18,7), (14,4), (17,11), (18,3), (15,7), (0,19) represent the particular set of cells for class NP and (24,0), (15,0), (0,3), (7,19), (6,21), (16,0), (9,4), (24,19), (24,21), (3,10), (6,22), (21,0), (5,2), (8,0), (5,3), (9,24), (4,9), (4,24), (6,24), (9,0) represent the specific set of nodes for class SP. The k value is taken as 5, for kNN and 100% accuracy is obtained from the test results. The test set ratio over training set ratio is 1/4 which is quite large for a pattern recognition experiment. This shows that our methodology is meaningful even though the sample size is small.
Prediction of Protein Subcellular Localization
617
Difference frequencies of the SOM cells
Difference between Histograms of two classes
300
200
100
0
−100 0 5 25
10 20
15 15
20
10 25
Ordinate of the SOM cell
5 30
Abcissa of the SOM cell
Fig. 2. Difference of histograms of SOM cells for SP and NP classes.
5
Conclusion
We describe P2SL for the prediction of protein subcellular localization sites using amino acid order. It achieves higher prediction accuracy than the previous two class classifiers. Our clustering strategy is training self organizing maps and using kNN for classification purposes. In order to extract features from amino acid sequences, PAM250 scoring matrix is used. This preserves the biological meaning of each independent amino acid found in protein subcellular targeting sequence motifs. Our classification strategy is based on choosing the dominant vectors which are extracted by clustering. As the next step, we intend to increase the number of classes to 4 rather than 2. Cytoplasmic and mitochondrial classes shall be added. More samples should be used for training the SOM. Futhermore, another classifier such as multilayer perceptrons can be employed together with kNN to obtain better results. In addition to these, SOM clustering can be useful for rule extraction and reverse analysis.
618
¨ M. Ozarar, V. Atalay, and R.C ¸ . Atalay
References 1. C. van Vliet, E.C. Thomas, A. Merino-Trigo, R.D. Teasdale, P.A. Gleeson: Intracellular sorting and transport of proteins. Prog Biophys Mol Biol., 83(1)(2003) 1–45. 2. F. Corpet, F. Servant, J. Gouzy and D. Kahn: ProDom and ProDom-CG: tools for protein domain analysis and whole genome comparisons. Nucleic Acids Res., 28(2000) 267–269. 3. M.O. Dayhoff, R.M. Schwartz and B.C. Orcutt: A model of evolutionary change in proteins. Atlas of protein sequence and structure. Vol. 5, Suppl. 3. National Biomedical Research Foundation, Washington, D.C., 1979 345–352. 4. K. Nakai and M. Kanehisa: A knowledge base for predicting protein localization sites in the eukaryotic cells, Genomics, 14(1992) 897–991. 5. iPSORT is available at: http://hypothesiscreator.net/iPSORT 6. O. Emanuelsson, H. Nielsen, S. Brunak and G. von Heijne: Predicting subcellular localization of proteins based on their N-terminal amino acid sequence, J.Mol.Biol., 300(2000) 1005–1016. 7. M.G. Claros: MitoProt: a Macintosh application for studying mitochondrial proteins, Computer Applications in the Biosciences 11(4)(1995) 441–447. 8. Y. Fujiwara, H. Asogawa and K. Nakai: Prediction of mitochondrial targeting signals using hidden Markov models, Genome Informatics, 8(1997) 53–60. 9. H. Nielsen, J. Engelbrecht, S. Brunak, G. von Heijne: A neural network method for identification of prokaryotic and eukaryotic signal peptides and prediction of their cleavage sites, International Journal of Neural Systems, 8(5–6)(1997) 581–599. 10. O. Emanuelsson, H. Nielsen and G. von Heijne: ChloroP, a neural network-based method for predicting chloroplast transit peptides and their cleavage sites, Protein Sci., 8(1999) 978–984. 11. Y. Fujiwara and M. Asogawa: Prediction of Subcellular Localization Using Amino Acid Composition and Order, Genome Informatics, 12(2001) 103–112. 12. Y. Cai, X. Liu, K. Chou: Artificial neural network model for predicting protein subcellular location, Computers and Chemistry, 26(2002) 179–182. 13. S.F. Altschul: Amino acid substitution matrices from an information theoretic perspective, J. Mol. Biol., 219(1991) 555–565. 14. T. Kohonen: The self-organizing map. Proceedings of the IEEE, 78(9)(1990) 1464– 1480. 15. The SOMPAK package is available at: http://www.cis.hut.fi/nnrc/papers/
Implementing Agent Communication for a Multi-agent Simulation Infrastructure on HLA Erek G¨ okt¨ urk and Faruk Polat Department of Computer Engineering, Middle East Technical University, ˙ on¨ In¨ u Bulvarı, 06531, Ankara, Turkey {erek, polat}@ceng.metu.edu.tr
Abstract. Multi-agent simulation is gaining popularity due to its intuitiveness and ability in coping with domain complexity. HLA, being a distributed simulation architecture standard, is a good candidate for implementing a multi-agent simulation infrastructure on, provided that agent communication can be implemented. In this paper, we present a layered approach to the agent communication, followed by a discussion of implementing the lowest two layers of this approach using HLA and the agent communication language KQML in the context of a multi-agent simulation.
1
Introduction
A multi-agent architecture is defined as one in which decisions and reasoning are distributed among the agents. This distribution of processes creates an intuitive style of modeling, when considered from a simulation perspective. Modeling using multi-agency helps tackling with the domain complexity by allowing one to separately study the structure of components that make up the system and the interactions between these components which create the domain’s dynamics [1]. Communication plays a key role in the multi-agent systems. The agents have to encode information and their attitudes regarding it using a language. The design of a multi-agent system is influenced by the structure and assumptions behind the agent communication language from two aspects: The social structure of the agents and the implications of language on the architecture of the resultant multi-agent system. High Level Architecture (HLA) is a distributed simulation standard (IEEE 1516) that aims at interoperable and component-based reusable simulations. HLA can be used as an infrastructure to build multi-agent simulations on, to foster its interoperability concept thereby allowing a multi-agent simulation to act itself as a component in a bigger simulation. In order to build a multi-agent simulation using HLA, an agent communication scheme must be developed. In this paper, we draw upon a layered approach to the agent communication, followed by a discussion of implementing the lowest two layers of this approach using the HLA and the agent communication language KQML in the context of a multi-agent simulation. A basic implementation demonstrating the concepts A. Yazici and C. S ¸ ener (Eds.): ISCIS 2003, LNCS 2869, pp. 619–626, 2003. c Springer-Verlag Berlin Heidelberg 2003
620
E. G¨ okt¨ urk and F. Polat
is realized using the Run-Time Infrastructure (RTI) implementation of Defense Modeling and Simulation Office of Department of Defense of USA. Implementing KQML in a multi-agent simulation seems most likely be considered when the agents are considerably deliberative, and the deliberative functions include reasoning that uses communication explicitly. There are alternatives to KQML as an agent communication language, but the choice of KQML is not arbitrary: It is due to the fact that KQML puts less restrictions on the policies and leaves more room for the designer’s preferences, when compared to its major alternative, the FIPA Abstract Architecture. This work is refined in some part from the experience gained in a 3D multiagent simulation of commando operations project, called SAVMOS, being carried on in the Modeling and Simulation Laboratory co-established by METU and the Turkish Armed Forces. The rest of the paper is organized as follows: Next section two chapters are on agent communication languages and their implications on the multi-agent system architecture. In the following section, we discuss how to realize a simulation consisting of agents that communicate using an agent communication language, KQML. The last section is conclusion and contains our final comments.
2
Agent Communication
By definition of the multi-agent systems, agent communication plays a crucial role as the only way of coordination between the problem solving entities, the agents. The communication between the agents take place either explicitly, that is by agents issuing communication actions and sending messages to other agents, or implicitly by agents having the results of their actions perceived by other agents. Without explicit communication, coordination becomes the very hard problem of opponent modeling, which is not a problem to be tackled for most of the practical multi-agent system implementations. For decades, the state of the art in modeling communication systems has been the use of layers whose functions and interfaces are more or less well-defined. In the literature, we see that agent communication is also structured in layers [2] [3]. We take an approach similar to the one in the FIPA Abstract Architecture, and propose a three layered model. The highest layer is the content layer. In the content layer, the information intended to be passed between the agents is expressed in a content language such as Knowledge Interchange Format (KIF) or Semantic Language (SL), using an ontology known by both participants of the communication. Below the content layer is the communication language layer, in which the content received from the upper layer is encapsulated using an agent communication language (ACL) such as KQML or FIPA-ACL [4] [5]. Usually, these agent communication languages contain performatives, which are said to stem from the speech-act theory [6]. The ACL message also contains information such as the names of the sender and receiver agents as well as interaction protocol related fields, ontology and the content language used.
Implementing Agent Communication
621
As the lowest layer, just above the actual information transfer such as a network connection, IPC or a middleware such as HLA, we observe the transport layer whose task is to handle the specific transfer related issues, such as converting the ACL message in a form transferable over the actual connection, converting agent names to transport addresses and encapsulating the ACL message with the sender and receiver transport addresses and the ACL type.
3
Implications of ACLs on MA System Architecture
The agent communication languages have an influence on the structure of social interactions in the multi-agent system they are used in. Moreover, the model assumed at ACL’s design for the social interactions that will take place using the ACL has influence on the design of it, so the assumed model necessarily have implications for the realizations. This resembles the fact that social interactions in a society both influences and is influenced by the natural language the members of the society speak. In the FIPA-ACL case, the architectural design assumptions supporting the agent communication are made explicit through the FIPA Abstract Architecture Specification. In their architecture, ACL is relieved from the task of finding other agents, their properties, and services provided by some entities which are not necessarily agents, by defining directory services separately from the agents. The designers of the architecture is also aware that these directories may be implemented by some special agents, yet they also put the note that discovering and communicating in such a case is a problem that’s outside the scope of the FIPA Abstract Architecture Specification document. By this way, FIPA abstracts and excludes some functions that’s being accomplished by certain performatives in KQML from its agent communication language FIPA-ACL. In KQML, the architectural assumptions are not made explicit, rather they’re implied in the specification. The KQML agents get to know of each other by exchanging advertisements. There are special agents in a KQML speaking agent society called the facilitators. These facilitators resemble the directories in FIPA and can act as mediators between the agents. There exists some performatives in KQML, which are said to not be speech acts in the pure sense, to assist this mediation [7]. KQML relies on the multi-agent system implementation to handle the details of agent and facilitator discovery mechanisms and requires only agents to have the ability to discover other agents. It should also be added that the transport details, e.g. addressing is said to be left outside KQML, yet KQML performatives register and transport-address requires the agent to record the transport address the KQML message come from, introducing the transport details into their semantics and blur the distinction between the transport and the communication layers. KQML-Lite should also be mentioned here, as an attempt to merge FIPA and KQML specifications [8]. Although KQML-Lite is meant to be light, it adds more performatives from the FIPA communicative acts library to the performative set of KQML. KQML-Lite also calls for standardization of the services
622
E. G¨ okt¨ urk and F. Polat
offered by the facilitator agents, thereby emphasizing the correlation between KQML implied architecture and FIPA Abstract Architecture. Furthermore, in an attempt to establish a better separation between the transport and communication layers, KQML-Lite defines a layer above the KQML expression and calls it the transmittal layer, whose output is again in the form of a KQML message but restricted to use only the networking and facilitation performatives, and performatives related to agent execution control.
4
Implementing KQML
The title of this section is deliberately chosen to remind the reader that an ACL is not an interpreted or a compiled language, and cannot have an implementation per se [9]. What we mean by implementing KQML is implementing an architecture in which agents may communicate respecting the semantics of KQML performatives as defined in [7], building on top of HLA architecture. The first architectural entity to be mapped onto HLA is the concept of agents. From the KQML perspective, agents are computational entities carrying unique names and causing some KQML messages be sent. So it’s pretty obvious that the agents will be mapped onto federates. The agents should be able to find out about at least the presence of other agents in the runtime environment, thus we need to design a way so that agents discover other agents. This can be accomplished by making agent executing federates publish agent objects to RTI, and subscribe to agent class so that the RTI will inform them of new agents or deleted ones. Discovering in this sense does not mean that federates can discover abilities of agents, to do that takes sending perhaps several KQML messages for querying agent abilities. Furthermore, RTI assigns names to the created objects, names which are unique in the scope of a federation, so using these names as the agent symbolic names relieves the designer of the problem of ensuring unique agent names. The other architectural entity is a KQML message. A KQML message is nothing but a piece of transient data to be transferred between the agents. Since the agents are executed by federates in our HLA based architecture, the KQML messages are to be perceived as data passed between the federates. It must be noted that semantically this data is sent from an agent to an agent, but architecturally the message is to be passed between federates. We assume that the exchanged KQML data is a somewhat encoded form of the KQML message, that all the federates know how to decode and parse it. KQML designers have made some assumptions about the transport layer that will carry the KQML messages. These assumptions are: 1. Agents are connected by unidirectional communication links that carry discrete messages. These links may have a non-zero message transport delay associated with them. 2. When an agent receives a message, it knows from which incoming link the message arrived. When an agent sends a message, it may direct the message to a particular outgoing link.
Implementing Agent Communication
623
3. Messages to a single destination arrive in the order they were sent. 4. Message delivery is reliable. Straightforward ones first: The ability of transferring data between agents which is mentioned in the first assumption is trivially satisfied by HLA’s ability to transfer data between the federates. Furthermore, the fourth assumption is about the delivery of messages and HLA RTI is defined to be configurable about using reliable transport for interactions and attribute updates. So these two assumptions create no problem. Two major difficulties is that the first assumption, combined with the second one, calls for point-to-point links between the agents, and the third assumption calls for send-order reception of messages. These two have no immediate counterpart solutions in HLA, so various alternative solutions have been studied on. The following two subsections cover the solutions we devised. 4.1
Point-to-Point Links Problem
Providing point-to-point links between the agents in the multi-agent architecture defined over the HLA is difficult due to HLA’s policy of preventing federates being aware of each other directly. This implies that the links we talk about are simulated links between the agent HLA objects, wherever their execution may be taking place. It turns out that one may use Data Distribution Management (DDM) quite effectively for the purpose of limiting the distribution of the message to only the relevant federates. The use of DDM is postponed to the end of this subsection. The first way of transporting KQML messages between the federates that execute the communicating parties is using the interactions. In this method, all federates publish and subscribe to a KQMLMessage interaction class. When a federate wants to send a message on behalf of an agent it simulates, the message along with the source and destination agent names is encoded into a KQMLMessage interaction and sent. Without DDM, this interaction is received by all federates and the federate simulating the destination agent may easily deduce that it should handle the message. The second way of transporting KQML messages is using the object attribute update mechanisms of HLA. One can either use this method in a push or pull communication scheme. Pull communication scheme is relatively more simple than the push scheme. In the pull scheme, a MessageUttered attribute is defined for the Agent HLA object class. All federates that simulate an agent publish an instance of this class for all agents they simulate, and subscribe to this class to get the messages other agents produce. Whenever a federate wants to send a message, it encodes the message and updates the MessageUttered attribute of the agent instance, and, having subscribed, all other federates receive this attribute update and decode the message to see if its destination is an agent they simulate. The push scheme uses a MessageReceived attribute defined in the Agent class, and HLA’s object attribute ownership management facilities. In this scheme, as
624
E. G¨ okt¨ urk and F. Polat
in the pull case, all federates publish an instance of the Agent class and subscribe to the Agent class to get the attribute updates for the MessageReceived attribute. When a federate wants to send a message on behalf of an agent it simulates, it first acquires the ownership of the MessageReceived attribute of the agent which is the destination of the message. When acquisition is completed, the message is sent by updating the value of the attribute, thereby causing all federates to receive the message and decode to see if it is for an agent they simulate. As it’s most easily seen in the last scheme, these methods of transporting the KQML message are not very efficient due to the fact that all federates receive all messages, whether it is relevant for them or not. This is surely not desirable, instead we conceive of point-to-point links as being between the corresponding parties, and not causing any load for others that may be present in the communication network. As said before, this inefficiency is due to HLA’s design. The solution proposed by the standard is use of DDM services. We applied the DDM services to the mentioned solutions. To use DDM, a routing space is created and every agent in the simulation is given a different region in this space. All the federates simulating agents publish, for every agent they simulate, the region of the agent through an instance of an Agent HLA object class. All federates also subscribe to the Agent HLA object class so that all federates can access the regions of any agent present. In the case of employing DDM when interactions are used for delivery, all federates subscribe to the interaction class KQMLMessage with a region that covers the union of all regions of the agents it is simulating. When a message is to be sent, the sending federate sends the interaction to the target agent’s region, and DDM routes the interaction to the only relevant federate. When the pull scheme is used with DDM, a federate subscribes to the MessageUttered attribute with a region that is the union of all regions of the agents it simulates, as in the interaction case. Whenever a federate, on behalf of an agent, wants to send a message, it first makes the source agent’s MessageUttered attribute visible to the federate simulating the destination agent by changing the publication region of the MessageUttered attribute to cover the destination agent’s region. Then the source agent’s object intance’s attribute is updated and the message gets transferred. The push scheme is similar with the extra step of acquiring the attribute ownership. 4.2
Send-Order Reception Problem
The third transport assumption of KQML calls for send-order reception of messages. This mechanism has no immediate equivalent in the HLA architecture: The HLA architecture supports receive-order or time-stamp-order message transport. We devised two solutions, both using time to make messages arrive in send-order, one defining the flow of time in terms of the messages created by all agents (or federates we may say) in a federation, and the other relying on the physical nature of message generation. The first solution defines the flow of simulation time in terms of messages generated throughout the federation. All federates work in optimistic time man-
Implementing Agent Communication
625
agement mode, thereby observing all the events in their simulated future by issuing FLUSH QUEUE REQUEST service of RTI. When a federate generates a message, it increments its simulation time by a fixed amount. When it receives messages, it tries to increment its time to the time of the latest message it received. This solution creates problems with DDM though, since it depends on all federates receiving all events in the future to update their time, and DDM effectively prevents this by providing true point-to-point links (Fig. 1 shows diagrammatically). It is possible to use a separate event for the coordination purpose, and using such a small event may be justified taking the size of messages to be delivered into consideration.
F1 F2
m1,1 m1,2 m1,3
m2,1
F3 F1 F2 F3
m1,1 m1,2 m1,3
m2,1
F1
F1
F2
F2
F3
F3
F1
F1
F2
F2
F3
F3
Fig. 1. The upper three simulation timeline diagrams shows how the federates advance their time when FLUSH QUEUE REQUEST service is used without DDM. Messages m1, 1 and m1, 3 are from federate F 1 to federate F 2, m1, 2 is from F 1 to F 3 and m2, 1 is from F 2 to F 1. F 1 and F 2 attempts advancing their time to the latest time stamp on their own messages, and F 3 attempts to advance its time to the latest message it receives. When F 2 is granted its time, it detects that there are other messages created by F 1, and re-increments its time. This way all three federates end up at the same simulation time. The lower three diagrams show the case when DDM is used. DDM effectively prevents federate F 3 gaining the necessary information about m1, 3, therefore federates F 1 and F 2 get stuck in time advancing state.
The other solution is to assume that in a simulation, time has a physical meaning and using time as merely a mechanism for synchronization would render the federation’s reuse and interoperability. In such a view of time, creating a message is a physical action that takes some amount of time to accomplish. This implies that either all the messages created by an agent have different time stamps, or the number of messages created with the same time stamp is limited, and that not receiving the messages until an agent’s local time, which may considerably differ from agent to agent in a distributed simulation, passes the message creation simulated time.
626
5
E. G¨ okt¨ urk and F. Polat
Conclusion
HLA, being a standard for distributed simulation architecture, is destined to provide better support for the use of agent technology since agent-based modeling is gaining popularity. Agent communication languages may be of importance as tools to build upon the data transport medium of HLA for agent based simulations. We believe that this paper presents one solution to the agent communication issue, using the language KQML. Of the solutions described in this paper, we believe that using interactions with DDM provides the most elegant solution. This belief is empirical, and further analysis need to be made in order to defend one method against other. Further studies include analysis of such design decisions and implementing the FIPA architecture using HLA as the transport medium.
References 1. Marcenac, P., Giroux, S.: Geamas: A generic architecture for agent-oriented simulations of complex processes. Applied Intelligence 8 (1998) 247–267 2. FIPA: FIPA abstract architecture specification. Foundation for Intelligent Physical Agents standard (2002) 3. Finin, T., Weber, J., Wiederhold, G., Genesereth, M., Fritzson, R., McGuire, J., Pelavin, R., Shapiro, S., Beck, C.: Draft specification of the KQML agentcommunication language plus example agent policies and architectures. ARPA Knowledge Sharing Initiative, External Interfaces Working Group working paper (1993) Available in web site http://www.cs.umbc.edu/kqml/. 4. FIPA: FIPA ACL message structure specification. Foundation for Intelligent Physical Agents standard (2002) 5. FIPA: FIPA communicative act library specification. Foundation for Intelligent Physical Agents standard (2002) 6. Searle, J.R.: Speech Acts. Cambridge University Press, Cambridge, UK (1969) 7. Labrou, Y., Finin, T.: A proposal for a new kqml specification. Technical Report TR CS-97-03, Computer Science and Electrical Engineering Department, UMBC (1997) 8. Kogut, P.: KQML lite specification, revision 0.3. Technical Report ALP-TR/03, Lockheed Martin Mission Systems (1998) 9. Labrou, Y., Finin, T.: Semantics and conversations for an agent communication language. In Huhns, M.N., Singh, M.P., eds.: Readings in Agents. Morgan Kaufmann, San Francisco, CA, USA (1997) 235–242
Improved POCS-Based De-blocking Algorithm for Block-Transform Compressed Images Yoon Kim1 , Chun-Su Park1 , Kyunghun Jang2 , and Sung-Jea Ko1 1
2
Department of Electronics Engineering, Korea University, Anam-Dong, Sungbuk-Ku, Seoul 136-701, Korea. {ykim, cspark, sjko}@dali.korea.ac.kr i-Networking Lab. Samsung Advanced Institute of Technology, Suwon, Korea
[email protected] Abstract. This paper presents a postprocessing technique based on the theory of projections onto convex sets (POCS) to reduce the blocking artifacts in low bit rate BDCT-coded images. In the block discrete cosine transform (BDCT), the image is divided into a grid of non-overlapped 8 × 8 blocks, and then each block is coded separately. Thus, a block, which is shifted one pixel diagonally from the original grid of BDCT, will include the boundary of the original 8 × 8 block. If blocking artifacts are introduced along the block boundary, the frequency characteristics of this shifted block will be different from those of the original block. Therefore, a comparison of frequency characteristics of these two overlapping blocks can detect the undesired high-frequency components, mainly caused by the blocking artifacts. By eliminating these undesired high-frequency components adaptively considering the local statistical properties of the image, robust smoothing projection operator can be obtained. Computer simulation results indicate that the proposed scheme provides better visual performance than conventional postprocessing algorithms.
1
Introduction
Image compression techniques based on BDCT have been popularly used for both still and moving image coding standards, such as JPEG [1], H.26x [2], and MPEG [3]. In BDCT, an image is partitioned into spatially adjoining blocks, each of which is processed independently without considering the inter-block correlation. At low bits rates, image compression based on BDCT inevitably produces the blocking artifacts. Various algorithms have been proposed to improve the quality of block-coded images in the decoder without increasing the bit rate. Among them, iterative approaches based upon the theory of POCS have been widely used for blocking artifacts reduction [4,5] and error concealment [6]. In this paper, based on the theory of POCS, we propose a new postprocessing technique to reduce the blocking artifacts in BDCT-coded images. In BDCT,
This work is supported by a grant from Korea University and Samsung Advanced Institute of Technology.
A. Yazici and C. S ¸ ener (Eds.): ISCIS 2003, LNCS 2869, pp. 627–634, 2003. c Springer-Verlag Berlin Heidelberg 2003
628
Y. Kim et al.
the image is divided into a grid of non-overlapped 8 × 8 blocks, and then each block is coded separately. Consider a block that is shifted one pixel diagonally from the 8 × 8 original grid. Since this shifted block includes boundaries of the original 8 × 8 block, the two overlapping blocks will have different frequency characteristics due to the blocking artifacts along the block boundaries. Based on these characteristics, we introduce a novel closed convex set and its projection operator in the DCT domain based on the theory of POCS to reduce blocking artifacts effectively. The rest of this paper is organized as follows. In the next section, conventional postprocessing algorithms based on the theory of POCS are briefly introduced. The proposed postprocessing scheme is presented in Sect. 3. Sect. 4 presents and discusses the experimental results. Finally, our conclusions are given in Sect. 5.
2
Conventional Postprocessing Techniques Based on the Theory of POCS
The theory of projections onto convex sets (POCS) is a special case of the composite mapping algorithms for image restoration. It is known that the postprocessing techniques based on the theory of POCS satisfy two requirements [4, 5,7]: First, the closed convex sets, which capture the properties of the image free from the blocking artifacts and close to the original image, should be defined. Second, the projection operators onto them are derived.
f0 k
k
1
1
Ps onto Cs
Pq onto Cq
fk
Converge? Yes No
Fig. 1. Block diagram of the postprocessing technique based on the theory of POCS
Zakhor [4] first introduced an iterative algorithm using the theory of POCS for the reduction of blocking artifacts of block DCT-coded images. In this approach, a decoded image is reconstructed by the iterative projections onto two constraint sets. Generally, these constraint sets can be classified into either smoothness constrain set (SCS, Cs ) and quantization constraint set (QCS, Cq ) (Fig. 1). The projection Ps onto SCS bandlimits the image in the horizontal and vertical directions in order to reduce directional blocking artifacts. On the other hand, in common the QCS is defined as the quantization interval for the original DCT coefficient. Since the quantization parameters are transmitted from the encoder, the quantization interval for each DCT coefficient is already known to the decoder. And its projection operator Pq moves the DCT coefficient into the quantization interval. In [5,7,8], the SCSs are proposed and the same quantization constraint utilized in Zakhor’s method is used.
Improved POCS-Based De-blocking Algorithm
3
629
Proposed Smoothness Constraint Set and Its Projection for the Reducing of Blocking Artifacts
Next, we introduce a method to reduce the blocking artifacts effectively by analyzing frequency characteristics of the original block and the shifted block which is one pixel diagonally apart from the original block. To do this, we propose the smoothness constraint set and its projection operator in the DCT domain. First, let us denote an mN × nN input image and its pixel by f and f (k, l), respectively, where (k, l) is the pixel coordinate. Also, let f Aij and f Bij be sets consisting of N 2 elements representing pixels in each N × N block, given by f Aij = {aij (0, 0), · · · , aij (N − 1, N − 1)} , f Bij = {bij (0, 0), · · · , bij (N − 1, N − 1)} ,
(1) (2)
where aij (x, y) = f (iN + x, jN + y), bij (x, y) = f (iN + x + 1, jN + y + 1), i = 0, · · · , (m − 2), and j = 0, · · · , (n − 2), respectively.
fA
ij
fB
ij
(a)
(b)
Fig. 2. Graphical explanation: (a) f Aij and f Bij . (b) Zig-zag scan order
As shown in Fig. 2 (a), f Bij , that is one pixel diagonally apart from f Aij , includes both right vertical and bottom horizontal block boundaries of f Aij . For the sake of simplicity in notation, let us denote aij and bij by a and b, respectively. And let the DCT of a(x, y) and b(x, y) be A(u, v) and B(u, v), respectively. Then, we have A(u, v) = α(u)α(v)
N −1 N −1
a(x, y) × cos[
π(2x + 1)u π(2y + 1)v ] cos[ ], 2N 2N
(3)
b(x, y) × cos[
π(2y + 1)v π(2x + 1)u ] cos[ ], 2N 2N
(4)
x=0 y=0
B(u, v) = α(u)α(v)
−1 N −1 N x=0 y=0
where
630
Y. Kim et al.
1, N α(k) = 2, N
k = 0,
(5)
otherwise.
To reduce the blocking artifacts in the decoded image, we use the frequency characteristics of B. If nonzero coefficients of B occur at higher order in the zigzag scan (See Fig. 2 (b)) than the highest nonzero valued location of A, those coefficients correspond to high frequencies caused by the blocking artifacts. In order to eliminate these high frequencies, we propose a POCS-based de-blocking algorithm using the smoothness constraint set and its projection operator as follows: Let us define closed convex set CB as CB = {f |f B ⊂ f , N Z(B) ≤ KB },
(6)
KB = N Z(A) + VB ,
(7)
where
and N Z(·) represents the order of the last nonzero DCT coefficient in the zig-zag scan. Note that VB is a factor representing how many DCT coefficients of B do contribute to the blocking artifacts. Before defining VB , we consider measures D and E, the former is a measure of the intensity variation caused by blocking artifacts in shifted block f B and the latter is the total intensity variation in original block f A . The mathematical form of D and E are given by N −1 12 2 D= |b(x, N − 1) − b(x, N − 2)| x=0
+
N −1
12 |b(N − 1, y) − b(N − 2, y)|
2
,
(8)
.
(9)
y=0
and E=
N −1
N −1
y=1
x=0
+
12 2
|a(x, y) − a(x, y − 1)|
N −1
N −1
x=1
y=0
12 2
|a(x, y) − a(x − 1, y)|
˜ = E for scaling. It is obvious that when D is smaller than E, ˜ blocking Define E N −1 2 artifacts do not exist and VB = N − N Z(A). As D increases, blocking artifacts increase, while VB decreases. A function representing the above property is given by
˜ 2 D−E 1 − max (10) VB = round ,0 · N − N Z (A) , ˜ D+E
Improved POCS-Based De-blocking Algorithm
631
where, max(·, ·) is larger one between two values, and round(c) is the integer value closest to c. By observing the smoothness constraint set defined above, the projection operator onto that set can be easily found as follows: By discarding the nonzero coefficients of B at higher orders than the predetermined value KB , we can ˆ obtain the projected element B. Given a decoded image f 0 , we determine mN ×nN closed convex set Cq ’s by using the quantization level for each DCT coefficient of the block. Then, we use the closed convex set CB in (6). Then, the projection operator onto smoothness constraint set is applied. Note that this computation is performed for all of the (m − 1) × (n − 1) smoothness constraint set CBij ’s. Next, the projection operator onto quantization constraint set is applied [4] to generate the postprocessed image f 1 . This procedure is iteratively performed to obtain the postprocessed image f k .
4
Experimental Results
In this section, the performance of four postprocessing techniques, namely, Zakhor’s [4], Yang’s [5], Paek’s [8], and the proposed technique is evaluated on still images including “LENA”, “ZELDA”, “CAMERA”, “BRIDGE”, and etc. “LENA” and “ZELDA” images contain low-frequency components and smooth area, while “CAMERA” and “BRIDGE” images contain high-frequency components and edge area. Thus, we will be able to examine how much the highfrequency components in the original image affect the performance, and the robustness of the content of the input images. Several decoded images, with visible blocking artifacts, are obtained by JPEG [1] with the same quantization table as shown in Table 1. Table 1. Quantization table 50 60 70 70 90 120 255 255
60 60 70 96 130 255 255 255
70 70 80 120 200 255 255 255
70 96 120 145 255 255 255 255
90 130 200 255 255 255 255 255
120 255 255 255 255 255 255 255
255 255 255 255 255 255 255 255
255 255 255 255 255 255 255 255
In Fig. 3, the PSNR performance of the proposed technique is compared with those of other techniques, with iteration increasing. One iteration of postprocessing is carried out by applying two projection operators first onto Cs and next onto Cq , as described in the previous section. As shown in Fig. 3, it is observed that the proposed technique converges, regardless of the input images, and can
632
Y. Kim et al.
Zelda
Lena 28.0
30.4
27.8
30.2
30.0
27.4
Zakhor Paek Yang Proposed
27.2
PSNR
PSNR
27.6
29.8
27.0
29.4
26.8
29.2
26.6
Zakhor Paek Yang Proposed
29.6
29.0 0
2
4
6
8
10
12
14
16
18
20
0
2
4
6
Iterations
10
12
14
16
18
20
Iterations
(a)
(b)
Camera
Bridge
26.8
25.2
26.6
25.0
26.4
24.8
26.2
24.6
PSNR
PSNR
8
26.0
25.8
25.6
24.4
24.2
Zakhor Paek Yang Proposed
Zakhor Paek Yang Proposed
24.0
25.4
23.8 0
2
4
6
8
10
Iterations
(c)
12
14
16
18
20
0
2
4
6
8
10
12
14
16
18
20
Iterations
(d)
Fig. 3. PSNR performance variations on four decoded images. (a) “LENA” image. (b) “ZELDA” image. (c) “CAMERA” image. (d) “BRIDGE” image
be easily implemented. From Fig. 3, we can see that the Zakhor’s technique fails to converge, in terms of the theory of POCS. Also, the performance of the proposed technique is better than those of Yang’s and Paek’s techniques in terms of the PSNR. Thus, it is believed that our technique is robust to the input images, in terms of convergence behavior and performance improvement. To make a comparison of the subjective quality more clear, we present enlarged photographs of the details of “LENA” in Fig. 4. As shown in Fig. 4, although Zakhor’s technique alleviated the blocking artifacts, it yields an excessively smoothed image due to the use of a global low-pass filter. Fig. 4 reveal that the blocking artifacts are alleviated more effectively by the proposed technique than the other techniques. It is also observed that edges are preserved very faithfully by the proposed technique. Similar results are also observed on the other three test images. Hence the proposed smoothness constraint set and its projection operator are properly defined, providing good performance in terms of both the PSNR and subjective quality.
Improved POCS-Based De-blocking Algorithm
(a)
(b)
(c)
(d)
633
Fig. 4. Comparison of subjective quality on “LENA”. Post-processed images: (a) Zakhor’s algorithm. (b) Yang’s algorithm.(c) Paek’s algorithm. (d) The proposed method
In Table 2, the PSNR performance of the proposed technique is compared with those of Zakhor’s, Yang’s, and Paek’s techniques. The simulation results show that the performance of the proposed algorithm is better than those of other algorithms.
5
Conclusions
In this paper, based on the theory of POCS, we proposed a postprocessing visual enhancement technique to reduce the blocking artifacts in BDCT-coded images. The novel smoothness constraint set and its projection operator were introduced. By using two overlapping blocks each of which is one pixel apart from the other, we can detect undesired high-frequency components mainly caused by the blocking artifacts. By eliminating these undesired high-frequency components adaptively, we obtained the robust smoothing projection operator. Computer simulation results indicate that the proposed scheme performs better than conventional postprocessing schemes.
634
Y. Kim et al. Table 2. PSNR for the different de-blocking methods
JPEG Zakhor’s algorithm Yang’s algorithm Paek’s algorithm Proposed algorithm
LENA ZELDA CAMERA BRIDGE PEPPER BARBARA CHURCH 27.68 29.81 26.69 24.79 28.84 28.43 28.44 27.85 30.33 26.56 24.98 29.27 29.13 28.77 27.78
29.97
26.56
24.77
28.71
28.96
28.35
27.80
29.99
26.78
24.85
29.01
28.64
28.22
27.94
30.18
26.81
24.96
29.23
28.98
28.74
References 1. ISO/IEC/JTC1/SC1/WG8 JPEG technical specification, vol. Revision 8, 1990. 2. ITU-T Video codec for audiovisual services at px64kbps, vol. Recommendation H.261, 1993. 3. I. JTC1/SC29 Coding of moving pictures and association audio, vol. Recommendation H.262: ISO/IEC 13818, 1993. 4. A. Zakhor, “Iterative procedures for reduction of blocking effects in transform image coding,” IEEE Trans. on Circuits Syst. Video Technol., vol. 2, pp. 91–95, Mar. 1992. 5. Y. Yang, N. P. Galatsanos, and A. K. Katsaggelos, “Regularized reconstruction to reduce blocking artifacts of block discrete cosine transform compressed images,” IEEE Trans. on Circuits Syst. Video Technol., vol. 3, pp. 421–432, Dec. 1993. 6. H. Sun and W. Kwok, “Concealment of damaged block transform coded images using projections onto convex sets,” IEEE Trans. Image Processing, vol. 4, pp. 470– 477, Apr. 1995. 7. Y. Yang, N. P. Galatsanos, and A. K. Katsaggelos, “Projection-based spatially adaptive reconstruction of block-transform compressed images,” IEEE Trans. on Circuits Syst. Video Technol., vol. 5, pp. 298–304, Aug. 1995. 8. H. Paek, R. C. Kim, and S. U. Lee, “On the pocs-based postprocessing technique to reduce the blocking artifacts in transform coded images,” IEEE Trans. on Circuits Syst. Video Technol., vol. 8, pp. 358–367, June 1998.
Lossy Network Performance of a Rate-Control Algorithm for Video Streaming Applications $\OLQ.DQWDUFÕDQG7XUKDQ7XQDOÕ 1
Computer Engineering Department, Ege University, Bornova, Izmir, Turkey NDQWDUFL#ERUQRYDHJHHGXWU 2
International Computer Institute, Ege University, Bornova, Izmir, Turkey WXQDOL#XEHHJHHGXWU
Abstract. In this paper, we present performance results of our previously developed rate control algorithm under various controlled packet loss rates. Since IP has no QoS support for real-time delivery of multimedia data, QoS control service has to be provided by the applications. Unlike traditional rate control approaches that take into account only the loss statistics, our algorithm integrated receiver buffer status into the feedback messages from the clients. Under various packet loss rates, we compared our approach to traditional two watermark approach and observed that our algorithm is more robust to varying network conditions.
1
Introduction
7UDGLWLRQDO ,QWHUQHW DSSOLFDWLRQV RZH WKHLU VWDELOLW\ WR WKH FRQJHVWLRQ FRQWURO DQG DYRLGDQFH DOJRULWKPV LPSOHPHQWHG LQ 7&3 WKH GRPLQDQW WUDQVSRUW SURWRFRO RI WKH ,QWHUQHW %DVHG RQ DGGLWLYH LQFUHDVH PXOWLSOLFDWLYH GHFUHDVH $,0' SULQFLSOH 7&3 HPSOR\V D ZLQGRZ EDVHG FRQJHVWLRQ FRQWURO DSSURDFK 7KH FRQJHVWLRQ FRQWURO PHFKDQLVP LQFRUSRUDWHG LQWR 7&3 SUREHV IRU DYDLODEOH EDQGZLGWK E\ VORZO\ LQFUHDVLQJ D FRQJHVWLRQ ZLQGRZ :KHQ RQH RU PRUH SDFNHWV JHW ORVW LW UHGXFHV WKH FRQJHVWLRQZLQGRZPXOWLSOLFDWLYHO\E\DIDFWRURIWZR +RZHYHU7&3LVQRWVXLWDEOHIRUUHDOWLPHDXGLRDQGYLGHRWUDQVPLVVLRQ2QHRIWKH UHDVRQVLVWKDWUHOLDELOLW\DQGRUGHULQJVHPDQWLFVRI7&3LQFUHDVHVHQGWRHQGGHOD\V DQG GHOD\ YDULDWLRQV $QRWKHU UHDVRQ LV WKH DJJUHVVLYHQHVV RI WKH ZLQGRZV EDVHG FRQJHVWLRQFRQWUROPHFKDQLVPRI7&3(YHU\ORVWSDFNHWUHVXOWVLQDGHFUHDVHLQWKH WUDQVPLVVLRQUDWHE\DIDFWRURIWZR7KLVVKDUSGHFUHDVHOHDGVWRXQSOHDVDQWGLVSOD\ TXDOLWLHV2QWKHRWKHUKDQGPXOWLPHGLDDSSOLFDWLRQVFDQWROHUDWHORVVWRVRPHH[WHQW 3HUSDFNHWDFNQRZOHGJHPHQWVWUDWHJ\LVDQRWKHUUHDVRQWKDWPDNHV7&3XQVXLWDEOHIRU PXOWLPHGLD VWUHDPLQJ $FNQRZOHGJHPHQW SDFNHWV LQWURGXFH D ODUJH EDQGZLGWK RYHUKHDGWRKLJKEDQGZLGWKPXOWLPHGLDDSSOLFDWLRQV>@ 'XHWRWKHDERYHUHDVRQV8'3LVSUHIHUUHGRYHU7&3DVWKHWUDQVSRUWSURWRFROIRU UHDOWLPH PXOWLPHGLD VWUHDPV +RZHYHU 8'3KDV QR VXSSRUW IRU 4XDOLW\ RI 6HUYLFH 4R6 ZKLFK H[SUHVVHV WKH UHTXLUHPHQWV RI PXOWLPHGLD DSSOLFDWLRQV LQ WHUPV RI SDUDPHWHUVVXFKDVWKURXJKSXWORVVGHOD\DQGMLWWHU$OVR8'3KDVQRSURYLVLRQIRU $@ (DUOLHU VWXGLHV RQ JXDUDQWHHLQJ D FHUWDLQ OHYHO RI 4R6 ZHUH EDVHG RQ PHFKDQLVPV VXFK DV DGPLVVLRQ FRQWURO DQG UHVRXUFH UHVHUYDWLRQ 7KHVH PHFKDQLVPV DVVXPH WKDW UHVRXUFH UHTXLUHPHQW RI PXOWLPHGLD DSSOLFDWLRQV NQRZQ D SULRUL DQG XVH ZRUVW FDVH UHVRXUFH XVDJH VFHQDULRV ZKLFK OHDG WR UHVRXUFH XQGHUXWLOL]DWLRQ 'LIIHUHQWLDWHG RU SULRULWL]HGVHUYLFHVDWQHWZRUNVZLWFKHVPD\DOVREHXVHGWRSURYLGHDFHUWDLQOHYHORI 4R6ZLWKWKHDSSOLFDWLRQV>@/DWHULWLVREVHUYHGWKDWDGMXVWLQJWKHUHVRXUFHXVDJH DFFRUGLQJ WR WKH H[LVWLQJ QHWZRUN FRQGLWLRQV LV PRUH DSSURSULDWH 7KLV DSSURDFK LV SUHIHUUHG WR WKH SUHYLRXVO\ SURSRVHG PHFKDQLVPV EDVHG RQ DGPLVVLRQ FRQWURO DQG UHVRXUFH UHVHUYDWLRQ EHFDXVH LW OHDGV WR EHWWHU XWLOL]DWLRQ RI DYDLODEOH QHWZRUN UHVRXUFHVZKLFKFKDQJHG\QDPLFDOO\$GGLWLRQDOO\WKLVDSSURDFKLVZHOOVXLWHGWRWKH QDWXUH RI PXOWLPHGLD DSSOLFDWLRQV ZKLFK DOORZ WKH PHGLD UDWH DQG TXDOLW\ FDQ EH FKDQJHGRYHUDZLGHUDQJH,QOLWHUDWXUHWKLVDSSURDFKLVFDOOHGUDWHFRQWURO>@ In this study, we implemented a rate control mechanism [6] for video streaming of MPEG-1 data on Solaris 2.8 platform and demonstrated the behavior of our rate control algorithm in lossy networks using a WAN simulation package, CLOUD. Most of the rate control mechanisms in literature [1] employ only loss statistics for adaptation. In our work, buffer status has been incorporated into the rate control algorithm in addition to loss statistics. This paper is organized as follows: In Section 2 our rate control algorithm is described in detail. Performance results are given in Section 3. Finally, Section 4 is the conclusion.
2
The Rate Control Algorithm
Our rate-control algorithm is developed in [5, 6]. In this study, we examine the robustness of our algorithm under varying network conditions. This section of the paper summarizes some of the design principles of our algorithm. Some features of our system are as follows:
Lossy Network Performance of a Rate-Control Algorithm
637
1. Videos are pre-encoded at multiple rates, namely 100Kbps, 200 Kbps, 500 Kbps and 1000 Kbps. Videos are transmitted in unicast manner. Each video file is associated with a meta data file that describes the structure of the video. The metafiles are useful to facilitate application layer framing and to adjust data rate. Meta files store information about the frame types, frame sizes, GOP statistics, sequence statistics and packet interval values for different adjustment levels. 2. We employed source based rate control with a probe based approach for bandwidth detection. Receivers send their feedback on congestion status via RTCP reports. The server then adjusts its rate properly. In non-congested state, the sender probes for extra bandwidth by switching to a better quality video and monitors RTCP loss statistics. As long as no congestion is detected, probing experiments continue. When congestion occurs the previous version of the video is switched back. 3. In addition to loss statistics, buffer status has been incorporated into the rate control algorithm. 4. A rate shaping mechanism with frame dropping filter approach [3] has been implemented. The frame dropping filter omits frames from the GOP pattern to be sent according to the importance of different frame types. Our adaptation policy is based on three observed variables, namely congestion level, receiver buffer level and the rate of change in receiver buffer level. Congestion level is monitored through loss rate statistics reported in RTCP feedback messages. Table 1 shows the Table 1. Congestion Classes congestion levels in the system Congestion Status Loss % according to the loss fraction field in NONCONGESTED 0-5 RTCP reports. In the uncongested MILDLY CONGESTED 5-15 state, the sender progressively CONGESTED 15-50 increases its transmission rate in an SEVERELY CONGESTED 50additive manner. In congested state, the sender progressively reduces its transmission rate in a more aggressively additive manner than in the mildly congested state. In the severely congested state, the rate is reduced multiplicatively to alleviate the congestion in the network. At the client side, two parameters have been taken into account to determine the buffer status and are sent regularly to the server over a UDP channel. The first one is ttp, the duration of video currently present in the buffer. The second on is dttp, the rate of change in ttp. We determine three dttp regions. First region includes the negative dttp values, which corresponds to decrease in buffer level. The second region corresponds to the case when the dttp is between 0 and 1. In the third region, dttp values are greater than 1, which we assume that dttp is above 1. We divided our buffer space into zones as follows: We set four points for our receiver buffer. Byte-overflow (byte-ovfl) is the top most level above which the receiver buffer enters a physical overflow region. Next comes time-to-play-overflow (ttp-ovfl) point set in terms of seconds. Note that byte-ovfl must be higher than highest quality video data rate multiplied by ttp-ovfl. Next what we call adjust threshold (adj-thr) is a point somewhere in the adjustable region. Next comes ttp-unfl, which determines the underflow region. We try to keep the buffer level around adj-thr level.
$.DQWDUFÕDQG77XQDOÕ
The rate control module at the server side reacts to the feedback messages by changing the video quality in accordance with the loss, ttp and dttp parameters. Video quality is changed in a seamless manner by switching to a video encoded at another rate, by dropping/adding frames from the current GOP pattern and by increasing/decreasing packet interval values between consecutive packets. Each video in the system is stored at multiple encoding rates. Currently, four encoding rates, namely 100 Kbps, 200 Kbps, 500 Kbps and 1000 Kbps, are supported. Frame dropping/adding is performed by taking into consideration the dependencies among different frame types and the smoothness of the video stream. Each encoding rate and GOP pattern adjustment level pair corresponds to a different video rate in the network. With these pairs, a grid may be formed where y axis and x axis correspond to encoding bit rate and GOP pattern adjustment levels respectively. Moving down along the y axis and right along the x axis reduces the video rate in the network. Various paths can be followed to change the rate of video in the network. Among all possible paths, we chose three of them to be used in different congestion and buffering conditions given in Fig. 1. In linear movement, effective data rate changes in an approximately linear fashion. Depending on the current state, any linear movement adaptation may involve frame discard rate or encoding date rate however, the latter change is much less infrequent. In cubical movement, depending on the current state, effective data rate changes either in additive or multiplicative manner. In diagonal movement, effective data rate changes in a multiplicative manner.
Fig. 1. Linear, cubical and diagonal movements
When the metadata files are being prepared, packetization module is executed for each encoding rate and adjustment level to determine the number of packets to be transmitted per video file. For each encoding rate and adjustment level, video duration is divided by the number of packets to calculate the transmission interval between successive packets during delivery. For flexibility, we used five packet interval values for each encoding rate and adjustment level pair. Packet intervals determined by using actual packet counts correspond to MEDIUM value of the packet interval. The other four values are obtained by dividing the interval between MEDIUM values of subsequent adjustment levels into four sub-intervals. The first and second values after the MEDIUM value of adjustment level k is called MEDIUM-HIGH and HIGH values of adjustment level k, respectively. The third and fourth values are LOW and MEDIUM-LOW values for adjustment level k+1, respectively. The proper packet
Lossy Network Performance of a Rate-Control Algorithm
639
interval is selected according to the severity of the congestion level and client buffer level. For example, in uncongested state and when the buffer level is low, LOW values of packet intervals are used to increase the buffer level in a shorter time period.
3
Performance Results
We tested our rate control algorithm by using CLOUD simulation software that simulates actual WAN environment by dropping packets received from the server in accordance with a given loss pattern. We collected performance results under regular loss patterns and with a congested network scenario. Fig. 2 illustrates the loss conditions that we used in our experiments. Fig. 2.a, 2.b and 2.c shows the cases where the average loss rate is at a constant level in uncongested state and Fig 2.d illustrates a congested network. When the loss rate is 1% and 3%, the system is in uncongested state. Therefore, rate control algorithm takes into account only the buffering conditions at the receiver side. Transmission starts at the encoding rate of 200 Kbps with a frame rate of 10 fps. Probing experiments show that the network is able to support higher quality. Therefore, the encoding rate and frame rate are increased to 1000 Kbps and 30 fps, respectively, which represents the highest video quality. Transmission of high quality video results in higher number of packets with larger end-to-end delay with respect to the case with lower video quality. After the prebuffering period, the client starts to consume the video. As the end-to-end delay increases due to transmission of large number of packets, it is observed that the consumption rate from the buffer becomes larger than incoming packet rate to the buffer. The goal of the adaptation algorithm is to keep the ttp values above adj-thr(=55 sec) as long as possible and react to congestion in the network. Shortly after the display begins, the ttp value falls below the adj-thr and packet loss is reported. The adaptation algorithm reacts by decreasing the video quality to 500 Kbps. When the loss rate is 1%, at time t=350, t=466 and t=510 sec., buffer level falls below the adj_thr (Fig. 4.a). At these points, encoding rate is reduced to increase the buffer level above adj_thr (Fig. 3.a). Reducing the encoding rate decreases network load. This is shown in the bps graph in Fig. 2.a. When the load decreases, delay also decreases and buffer level increases at a higher rate. Fig. 3 shows that, in all loss cases, frame level is changed more frequently than the encoding rate. When the average loss rate is 3%, more packets get lost in the network and consequently, buffer level drops more frequently below adh-thr(Fig. 4.b). Fig. 3.b shows that the encoding rate decreases to 200 Kbps to keep the buffer level at the desired level. In conjunction with the reduction in the encoding rate, network load decreases to 200 Kbps in average as given in Fig. 2.b. The case with the average loss rate of 5% corresponds to the behavior of algorithm at the boundary point between the uncongested and mildly congested states. With this loss rate, loss statistics start to contribute to the rate control decisions when the instantaneous loss rate exceeds 5%. A regular loss pattern of 5% causes quick decreases in buffer occupancy . Our algorithm uses a trick to prevent the buffer from draining. When the buffer level is under ttp_unfl and the system is in uncongested state, packet interval is reduced to the smallest value of the prevailing encoding rate
$.DQWDUFÕDQG77XQDOÕ
UI V V R O G H K W R R P V
V S E
IU VV OR G H K W R P VP
VPRRWKHGORVVUDWH
VPRRWKHGORVVUDWH
ESV
(a) loss= 1%
U I VV R O HG WK R R VP
U IV V OR G H WK R R P V
WV H F
VPRRWKHGORVVUDWH
ESV
(b) loss = 2%
WV H F
WVHF
V S E
V S E
V S E
WV HF
ESV
(c) loss= 5%
VPRRWKHGORVVUDWH
ESV
(d) congestion scenario
Fig. 2. Regular loss and congestion scenarios used in the experiments. Loss percentage is the integer part after dividing the loss fraction by 256.
until the buffer occupancy rises above adj_thr. High increases of ttp values given in Fig. 4.c reflects this behavior of our rate control algorithm. When the loss rate is 5%, an oscillating pattern is observed for the encoding rate. In conjunction with losses and decreases in buffer levels under adj_thr, encoding rate is decreased to 100 Kbps. The trick with packet interval values increases the ttp value above adj_thr and subsequently encoding rate is increased to its highest value for a period of time. This cycle is repeated during the playout of the video as shown in Fig. 3.c. We also conducted experiments with regular loss patterns at loss rates higher than 10%. In those cases, encoding rate and frame rate decreased to the lowest values. More packets got lost in the network and buffer quickly drained, resulting in interrupts during the playout. Fig. 2.d shows a variable loss pattern that we used in our experiments. The system is subjected to periods of congestion (t=90-200 sec, t=351-406 sec, t=550-626 sec).and periods of low loss values (Fig. 3.d). During congestion periods, ER values are quickly set to 100 Kbps and frame rates dropped to 1 fps. During uncongestion periods, video quality is increased to higher encoding rates and frame rates. When Figure 4.d is examined, the trick with packet interval values again prevents the buffer from draining when the loss rate is low. In conclusion, when Fig. 4 is examined, it is seen that at low values of loss, ttp values are almost constant. Packet interval values are also almost constant indicating a stable network traffic. As the loss values increase, buffer occupancy decreases
Lossy Network Performance of a Rate-Control Algorithm
641
sharply. As a result of the switching between various encoding rates and frame rates, packet interval values show variation.
(5
/ $
5 (
W
WV H F
(5
(5
$/
(a)
5 (
/ $
(c)
$/
(b)
/ $
5 (
WVH F
(5
/ $
WVHF
$/
(5
(c )
$/
(d)
Fig. 3. ER/AL diagrams corresponding to the loss scenarios given in Fig. 2.
F H V S W W
F H V P O D Y U H W Q L S
F H V S W W
WV HF
WWS
WV H F
SLQWHUYDO
WWS
(a)
F H V P O D Y U H W LQ S
F H V S W W
(c )
SLQWHUYDO
F H V P O D Y U H W Q L S
WV HF
WWS
SLQWHUYDO
(b)
F H V S W W
F H V P O D Y U H W Q L S
WV H F
WWS
SøQWHUYDO
(d)
Fig. 4. Buffer levels and packet interval values for the loss patterns given in Fig. 2
We also compared the performance of our adaptation module with a traditional rate control algorithm with no consideration of buffer occupancy. Fig. 5 shows the buffer
4
WWS
occupancy for the latter case. With the traditional rate control, buffer quickly drains leading to interruption in the display. In addition, employment of a simple two watermark approach [5] in buffer management resulted in the same situation. Introduction of adjthr and employment of a control mechanism related to adj-thr enabled to preserve the continuity of the display.
GWWS
$.DQWDUFÕDQG77XQDOÕ
W WWS
GWWS
Fig. 5. Buffer level when buffer status is not considered
Conclusion
In this paper, we reported performance results of a rate control algorithm under various packet loss scenarios. CLOUD simulation software is used to generate the simulated end-to-end lossy channel. Among the scenarios are three fixed loss rates. We have gradually increased the loss rate to find out the breaking point. We have observed that our system is able to provide uninterrupted display even if we experience a fixed continuous loss rate of 10%. The varying loss rate scenario we have reported also involves burst losses of relatively long intervals, yet we still observe that our system is able to maintain uninterrupted display. In our experiments, our bandwidth is approximately 500 Kbps. In future, we are planning to extend this performance study to variable bandwidth situations. Acknowledgement. 7KH DXWKRUV DUH LQGHEWHG WR 'U 5HKD &ÕYDQODU IRU KHOSIXO VXJJHVWLRQV
References 1. 2. 3. 4. 5.
6.
Wang X., Schulzrinne H.: Comparison of Adaptive Internet Multimedia Applications, IEICE Transactions on Communications , Vol. E82-B, No. 6, pp. 806–818, (1999). Sisalem D., Schulzrinne H.: The Loss-Delay Based Adjustment Algorithm: A TCP Friendly Adaptation Scheme, NOSSDAV, Cambridge, UK, (1998). Schulzrinne H., Cosner S., Frederic R., Jacobson V.: FC1889: RTP: A transport protocol for real-time applications, (1996). Wu D., Hou Y. T., Zhang Y.: Transporting Real-time Video over the Internet: Challenges and Approaches, Proceedings of the IEEE, Vol. 88, No.12 ,(2000). Kantarci A., Tunali T.; Design and Implementation of a Streaming System for MPEG-1 Videos, accepted to be published in Journal of Multimedia Tools and Applications, Kluwer Academics. Tunali T., Kantarci A., Özbek N.: , Robust Quality Adaptation for Internet Video Streaming in Heterogenous Networks, Journal of Multimedia Tools and Applications, in review.
A Hierarchical Architecture for a Scalable Multicast Abderrahim Benslimane and Omar Moussaoui Laboratoire d’Informatique d’Avignon LIA 339 chemin des Meinajaries BP 1228 – 84911 Avignon cedex 9 EHQVOLPDQH#OLDXQLYDYLJQRQIU
Abstract. Multimedia applications, such as videoconferences, require an efficient management of the Quality of Service (QoS) and consist of a great number of participants. In order to support such applications, we propose a hierarchical architecture allowing efficient communication between disjoint local groups. To connect these groups, we use a shared-based tree method. For an effective evaluation, we compare it to a source-based tree method. With NS-2 simulations, we show that the two methods are better than the direct architecture. Shared-based tree method performs well when using dynamically several rendezvous points instead of unique and static point. The performances are the reduction of used bandwidth and the communications delays.
1 Introduction The demand for multimedia which combines audio, video and data streams over a network is rapidly increasing. Some of the more popular real-time interactive applictions are teleconferencing, tele-teaching, games, white boards, etc. Multimedia applications require in general a considerable amount of bandwidth and a great number of participants. Multicast is regarded as a promising solution for group multimedia applications. It allows to reduce considerably the costs of communication (i.e., a single sender to several receivers or several senders to several receivers). Several multicast protocols were proposed in the literature. However, the majority of them are not scalable. Indeed, these protocols are badly adapted to the networks of great dimension like Internet. Techniques based on hierarchical trees represent very promising solutions. Nevertheless, studies are primarily related to only theoretical aspects without taking account implementation. In addition to the need of scalability, group based multimedia applications also demand stringent QoS requirements such as the minimum end-to-end delay, jitter delay and the efficient use of bandwidth. Several works was proposed to support QoS in networks. They concern the network functionalities of low levels (scheduling, traffic management, etc). The development of routing sensitive to QoS attracted less attention, although it is essential for the multimedia applications. In this work, we propose a scalable multicast protocol supporting QoS. Indeed, we use hierarchical trees, rendezvous points and criteria on QoS to connect multicast $@DOORZVWREXLOG&%7RU637WUHHV5HQGH]YRXV SRLQWV DUH IL[HG DGPLQLVWUDWLYHO\ DW WKH WLPH RI WKH FRQILJXUDWLRQ RI WKH URXWHUV 6SDUVHPRGHLVPRUHHFRQRPLFLQWHUPRIVLJQDOL]DWLRQH[FKDQJHVRUQXPEHURI VWRUHG VWDWHV ZKHQ WKH JURXS LV VOLJKWO\ GLVWULEXWHG LQ WKH QHWZRUN 2Q WKH FRQ WUDU\WKHGHQVHPRGHLVZHOODGDSWHGZKHQWKHJURXSPHPEHUVDUHVWURQJO\FRQ FHQWUDWHGLQWKHQHWZRUN
646
A. Benslimane and O. Moussaoui
- MOSPF Protocol: MOSPF [5] is an extension of the unicast protocol OSPF (Open Shortest Path First). It builds only SPT trees. Each router has a topological vision of the network and the location of each member of the multicast group allows to build a representation of SPT tree. 7KHPDMRUOLPLWDWLRQRIH[LVWLQJPXOWLFDVWSURWRFROVOLHVLQWKHIDFWWKDWWKH\DUHEDGO\ DGDSWHGWRODUJHVFDOHQHWZRUNVOLNHWKH,QWHUQHW0RUHRYHUDOOWKHVHDERYHSURWRFROV GRQRWWDNHLQWRDFFRXQWDOOWKHUHTXLUHPHQWVRIJRRGPXOWLFDVWURXWLQJSURWRFROVXFK DV HQGWRHQG GHOD\ DQG EDQGZLGWK 7R RYHUFRPH WKHVH OLPLWDWLRQV ZH SURSRVH D KLHUDUFKLFDO DUFKLWHFWXUH DOORZLQJ HIILFLHQW FRPPXQLFDWLRQ EHWZHHQ JURXS PXOWLFDVW PHPEHUV DQG VXSSRUWLQJ JXDUDQWLHV RI 4R6 7KH DUFKLWHFWXUH FRQVLVWV RQ JURXSLQJ PHPEHUV DFFRUGLQJ WR WKHLU ORFDOL]DWLRQ DQG WR WKHLU UHVSRQVH WR WKH 4R6 UHTXLUH PHQWVRIWKHDSSOLFDWLRQLQWRGLVMRLQWJURXSV:HXVHWZRGLIIHUHQWPHWKRGVWRFRQ QHFWWKHVHJURXSV7KHILUVWRQHSUHVHQWHGLQWKLVSDSHUFRQVLVWVRQXVLQJWKHFRQFHSW RI VRXUFHEDVHG WUHH :KHUHDV ZRUN LQ >@ XVH UHQGH]YRXV SRLQWV LQ VWDWLF ZD\ RXU SURWRFROGHWHUPLQDWHVUHQGH]YRXVSRLQWVLQDG\QDPLFZD\7KHVHFRQGRQHFRQVLG HUHG LQ WKH VLPXODWLRQ FRQVLVWV RQ XVLQJ D VRXUFHEDVHG WUHH PHWKRG WR FRQQHFW JURXSVEHWZHHQWKHP7KHVHDUFKLWHFWXUHVDOORZDVFDODEOHPXOWLFDVWWKDWLVVHQVLWLYH WR4R6
3 Hierarchical Architecture for Multicast Group Hierarchical trees are particularly interesting to support multicast in the wide-area networks. For this raison, we are interested in the shared-based tree. So, we consider a multicast group M constituted of N processes distributed geographically on inter-connected different networks. These processes have different identities and communicate between them toward the Internet. Moreover, they participate at the same multimedia application such as videoconferencing. Communication between two processes can take different path. 3.1 Local Group Construction Considering the high number of participants in multicast group M, it appears essential to divide them into local groups according to their concentration in the various areas of the Internet to ensure QoS. This decomposition allows to use efficiently the bandwidth, to reduce the consumption of resources, to optimize the delay and to ensure communications between processes. The construction of local groups was introduced by A. Benslimane and A. Abouaissa [1] in a logical way. Their algorithm supposes that processes communicate between them via virtual channels. In this work, we use real paths, i.e. physical links existing between the hosts and routers participating in the multicast group. To build these local groups, we construct a neighboring set and transit set for each process. A neighboring set is constructed according to a Time To Live value and a threshold value. A transit group is constructed according to the
A Hierarchical Architecture for a Scalable Multicast
647
neighboring set and the capabilities of processing and buffering of media units of each process. After having constructed the transit groups the processes proceed to the grouping in local groups. Each local group elects its local server. Thus, the multicast group M is divided into local sub-groups. Each element of M belongs to only one local group. The communication in the group may pass trough the server. In other term, a process communicates with the other participants of the multicast group only through the server of its local group. The complete description of these algorithms is given in [2]. 3.2 Hierarchical Tree Construction between Servers 1RZOHWXVFRQVLGHUWKDW.ORFDOJURXSVDUHEXLOWDQGHDFKRQHKDVLWVORFDOVHUYHU /HWΓ ^66.`WKHVHWRIWKHVHORFDOVHUYHUVZKLFKFDQEHGLVWULEXWHGLQGLIIHUHQW QHWZRUNV5HPDUNWKDWWKHQXPEHURIWKHVHVHUYHUVFDQEHYHU\KLJK0RUHRYHULQD YLGHRFRQIHUHQFH VHYHUDO VRXUFHV FDQ EHORQJ WR WKH VDPH PXOWLFDVW JURXS &RQVH TXHQWO\ LW LV QHFHVVDU\ WR EXLOG D PXOWLFDVW WUHH ZKLFK DOORZV WR OLQN WKHVH VHUYHUV ZKLOH UHGXFLQJ WKH SDUWLFLSDQWV RI WKH PXOWLFDVW JURXS 6R ZH SURSRVH WKH XVH RI FRUHEDVHG WUHH FDOOHG 5'9 5HQ'H]9RXV KLHUDUFKLFDO DUFKLWHFWXUH ,Q WKLV ZRUN ZHGHWHUPLQHWKHUHQGH]YRXVSRLQWVLQG\QDPLFZD\0RUHRYHUZHSURSRVHWKHXVH RIVHYHUDOUHQGH]YRXVSRLQWVWRDYRLGWKHFRQJHVWLRQSUREOHPRIWKHWUDIILFZKHQRQO\ RQHUHQGH]YRXVSRLQWLVXVHG Algorithm description: :HVXSSRVHWKDW6S∈ΓLVWKHORFDOVHUYHURIWKHJURXSZKHUHLVORFDWHGWKHILUVWVRXUFH ZKLFKZDQWWREURDGFDVWPHGLDXQLWVWRWKHRWKHUPHPEHUVRIWKHPXOWLFDVWJURXS0 7KLVVHUYHUUHSUHVHQWVWKHILUVWUHQGH]YRXVSRLQW53*LYHQDWKUHVKROGGHOD\'¶WKDW LVVHOHFWHGE\WDNLQJDFFRXQWRIWKHQHWZRUNH[WHQW $W WKH EHJLQQLQJ 53 GRHV QRW KDYH DQ\ LQIRUPDWLRQ RQ WKH DGGUHVVHV RI WKH RWKHU VHUYHUV&RQVHTXHQWO\LWEURDGFDVWVKRSE\KRSDQLQLWLDOL]DWLRQPHVVDJH,1,753 DGU*77/ LQFOXGLQJWKHDGGUHVVRIWKHUHQGH]YRXVSRLQW53LDWLPHWROLYH77/DQG JURXS PXOWLFDVW DGGUHVV DGU* $OO PHPEHUV RI 0 WKDW DUH QRW VHUYHUV GHOHWH WKH UH FHLYHG PHVVDJH +RZHYHU HDFK ORFDO VHUYHU 6L VHQGV DQ DFNQRZOHGJH PHVVDJH $&.6L53 WR53:KHQ53UHFHLYHVDFNQRZOHGJHPHQWPHVVDJHIURP6LLWFRP SXWHVWKHURXQGWULSGHOD\UG,IWKLVGHOD\LVVPDOOHUWKDQWKHWKUHVKROG'¶LWVHOHFWV WKHVHUYHU6LDQGSXWVLWLQLWVDXWRQRPRXVGRPDLQ'$ DVHWRISURFHVVHV 7KHQLW VWRUHVLQ 6 DOORWKHUVHUYHUVZKLFKDUHQRWLQLWVGRPDLQ'$ 6 =Γ − '$ $IWHUFRQ VWUXFWLRQRI'$53VWRUHVWKHFRPSXWHGGHOD\EHWZHHQLWDQGHDFKVHUYHU6LRI 6 LQ DPDWUL[0$7DOORZLQJWRVWRUHWKHURXQGWULSGHOD\VEHWZHHQWKHHOHPHQWVRI 6 DQG WKHH[LVWLQJUHQGH]YRXVSRLQWVLQWKHPXOWLFDVWWUHHLQRUGHUWRGHWHUPLQDWHWKHQH[W UHQGH]YRXV SRLQW 7DEOH 7KHQ LW VHQGV WKH PHVVDJH 68&&(66256L 53 6 0$7 WR WKH QHDUHVW VHUYHU 6L ZKLFK GRHV QRW EHORQJ WR LWV GRPDLQ '$ ,Q RWKHU
648
A. Benslimane and O. Moussaoui
ZRUGVWKDWZKLFKLVOLQNHGWR53E\DSDWKKDYLQJWKHVPDOOHVWYDOXHRIURXQGWULS GHOD\UGLVWULFWO\KLJKHUWKDQ'¶DQGZKLFKEHORQJVWR 6 Table 2. Round-trip delays between rendezvous points and servers of S 53? 6 53 53 «
6L UGL UGL
6M UGM UGL
6N UGN UGL
6O UGO UGL
:KHQ D ORFDO VHUYHU 6L UHFHLYHV PHVVDJH 68&&(66256L 53S 6 0$7 LW XQGHU VWDQGVWKDWLWFRQVWLWXWHVDQHZUHQGH]YRXVSRLQWZHQRWHG537KHQHZUHQGH]YRXV SRLQW53SOD\WKHUROHRILQLWLDWLQJVHUYHURIWKHVHW 6 WKHVHWRIVHUYHUVZKLFKDUH QRWLQWKHGRPDLQRI537KDQNVWRWKHPHVVDJH68&&(662553NQRZVWKHDG GUHVVHVRIDOOSURFHVVHVLQ 6 WKHQLWVHQGVWRWKHPPHVVDJH-2,1536M LQXQLFDVW ZD\:KHQVHUYHU6MUHFHLYHVWKLVPHVVDJHLWUHSOLHVZLWK$&.6M53 WR537KHQ 53 FRPSXWHV WKH PLQLPXP URXQGWULS GHOD\V EHWZHHQ LW DQG HOHPHQWV RI 6 DQG FDUULHVRXWWKHIROORZLQJRSHUDWLRQV ,WVHOHFWDOOVHUYHUVRI 6 KDYLQJDURXQGWULSGHOD\VPDOOHURUHTXDOWRWKHWKUHVK ROG'¶DQGSXWVWKHPLQLWVDXWRQRPRXVGRPDLQ'$ ,WUHPRYHVIURP 6 WKHVHOHFWHGVHUYHUV 6 = 6 − '$ ,WUHPRYHVWKHFROXPQVRIWKHPDWUL[ 0$7 FRUUHVSRQGLQJ WR WKH HOHPHQWV RI LWV GRPDLQ'$ ,WDGGVDOLQHLQ0$7WRVWRUHWKHPLQLPXPURXQGWULSGHOD\VEHWZHHQLWDQGDOO VHUYHUVRI 6 WKRVHZKLFKGRQRWEHORQJWRDQ\GRPDLQ ,W VHDUFKHV WKH VPDOOHVW YDOXH RI GHOD\V LQ 0$7 DQG WDNHV WKH FRXSOH 53S 6T FRUUHVSRQGLQJ WR WKLV YDOXH 7KXV ZH GHWHUPLQHG G\QDPLFDOO\ WKH QH[W UHQGH] YRXVSRLQW6TDPRQJWKHVHUYHUVRI 6 ZKLFKIDWKHULV53SWKDWLVDUHQGH]YRXV SRLQWDOUHDG\FUHDWHG,QRWKHUWHUP6TLVDVHUYHURIWKHVHW 6 DQGLWLVWKHQHDU HVWWRWKHH[LVWLQJUHQGH]YRXVSRLQW53SZKLOHWDNLQJWKHURXQGWULSGHOD\DVPHW ULF ,WVHQGVPHVVDJH68&&(66256T53S 6 0$7 WRWKHQHZUHQGH]YRXVSRLQW6T QRWHG53 )LQDOO\53MRLQVPHPEHUVRILWVGRPDLQ'$DQGLWVUHQGH]YRXVSRLQWIDWKHU53 WRZDUGWKH637WUHHZKLFKURRWLV53 $IWHUWKHQHZUHQGH]YRXVSRLQW53UHFHLYHVWKHPHVVDJH68&&(66256T53S 6 0$7 LWPDNHVWKHVDPHWKLQJDV53DQGVRRQXQWLOHPSWLQHVVRI 6 ,QRWKHUZRUGV XQWLODOOVHUYHUVDUHFRQQHFWHGWRWKHPXOWLFDVWWUHH
4 Simulations and Analysis In these simulations, we are interested with the scalability problem that arises when multimedia applications in multicast groups are to be executed. The way with which
A Hierarchical Architecture for a Scalable Multicast
649
the communication is done between participants influences the guarantee of QoS and the scalability. So, for these raisons we focus the analysis on the communication between local groups. So, to compare our methods with other, we implement a specific-source tree, called LDS (Link Direct Shortest path) hierarchical architecture. This method allows to connect a group where there is a source to all other groups directly. We, also, compare them to a DIRet architecture DIR. We consider a connected graph constituted of 2000 nodes which are set in a matrix of 5000x5000 and linked between them in a random way. We choose 8 sources of multimedia stream located randomly in 50 sub-groups. As a metrics, we use the shortest path with a given TTL=5 hops to consider the delay. Moreover, for each group a node with a smallest identity number is chosen as a local server. For RDV, we have selected five local servers as rendezvous points RP. Then all other local servers are connected to these RP. For LDS (Direct links between servers), we connected directly, each local server of the group where is located one source to all others. For DIR, we used the shortest paths between each source and the other nodes. Simulation parameters are: simulation time is 100(s), generation rate of each source is 200 (media units/s) and media unit size is 500 bytes. The scheduling of multimedia multicast for each source is as follows: source 1 from 0 s to 80 s, source 2 from 10 s to 80 s, source 3 from 20 s to 80 s, source 4 from 30 s to 80 s, source 5 from 50 s to 80 s, source 6 from 60 s to 80 s, source 7 from 70 s to 80 s, source 8 from 70 s to 100 s.
a. Charge of the network.
b. Average of the transmission delay.
Fig. 1. Simulation example on a network of 2000
Thanks to NS-2, we generate three files corresponding to the three architectures. To interpret obtained results, we counted, at each 0.1 s, the number of packets in the network (Figure 1.a). The number of packets in the network depends on the number of multicast sources. In curves corresponding to the network load, we observe that the number of packets is very high between 70(s) and 80 (s) because the 8 sources, broadcast streams at the same time.
650
A. Benslimane and O. Moussaoui
Also, we computed the average of transfer delay for each packet, from its emission by a source until its arrival to all destinations (Figure 1.b). The average of transfer delay of a packet is the time necessary so that this packet is delivered by all receivers of the multicast group (i.e., total transmission time of all copies of the same packet sent by a source in the network).
5 Conclusion We have described different types of multicast trees and existing multicast protocols dedicated to the Internet network. As we have shown, these protocols are limited when we consider scalability and QoS requirements for multimedia applications. To overcome these limitations, we have proposed a decomposition of the system to local groups according to the neighboring set, the capabilities of processing and buffering of media units of each process. In this paper, we are interested to the communication path between local groups. We show that when using shared-based tree with dynamic rendezvous points, the performances are better compared to source-based tree or direct connection. So, delays and resources use are reduced. Our purpose is confirmed by NS-2 simulations.
References 1. A. Benslimane et A. Abouaissa, "Dynamical Grouping Model for Distrubuted Real Time Causal Ordering", Computer Communications journal, Vol. 25, pp. 288–302, January 2002. 2. A. Benslimane, “Multicast Protocols: Scalability and QoS Studies”, Technical Recherche Repport, LIA, Université d’Avignon, April 2003. 3. S.Deering et al., "Protocol Independent Multicast-Dense Mode (PIM-DM): Protocol Specification", Internet draft, June 1999. 4. D. Estrin et al., "Protocol Independent Multicast-Sparse Mode (PIM-SM): Protocol Specification", RFC 2362, June 1998. 5. J. Moy, "Multicast Routing Extensions for OSPF", Commun. ACM, Vol. 37, pp. 61–66, August 1994. 6. B. Mukherjee L.H. Sahsrabuddhe, "Multicast Routing Algorithms and Protocols", IEEE Network, January/February 2000. 7. A. Striegel and G. Manimaran, “A Survey of QoS Multicasting Issues”, IEEE Communications Magazine, June 2002, pp. 82–87. 8. D. Waitzman and C. Partridge, "Distance Vector Multicast Routing Protocol", RFC 1075, November 1988.
Effect of the Generation of MPEG-Frames within a GOP on Queueing Performance J.C. Lopez-Ardao, M. Fernandez-Veiga, R.F. Rodriguez-Rubio, C. Lopez-Garcia, A. Suarez-Gonzalez, and D. Teijeiro-Ruiz Telematics Engineering Department, University of Vigo (Spain) ETSET Telecomunicacion, Campus Universitario 36200 Vigo (Spain) {jardao, mveiga, rrubio, candido, asuarez, diego}@det.uvigo.es
Abstract. MPEG video traffic is expected to represent most of the load in the future high-speed networks. Adequate traffic models for MPEG Variable Bit-Rate (VBR) video are important for network design, performance evaluation, admission control and resource allocation. It is well-known that VBR video exhibits long-range correlations over arbitrarily long time-scales, a phenomenon usually referred to as LongRange Dependence (LRD). Many models for VBR video traffic have been proposed in the literature. However, while the correlation between Groups of Pictures (GOPs) has been widely analyzed in literature, little effort has been devoted up to now to the generation of the different frame-types within a MPEG GOP. Due to the difficulty and complexity involved in this issue, this is often neglected even though it is a fundamental characteristic of MPEG traffic and it might have an important impact on queueing performance. In fact, many works avoid to descend to the frame level proposing models for the GOP process. In this paper, we analyze the impact on queueing performance of different solutions proposed in the literature for generating the different types of MPEG frames within a GOP (these solutions have never been compared and analyzed jointly). In this way, we can be in a good position to use the best model for MPEG in terms of simplicity, computational efficiency and queueing performance, depending on the performance metric to study (loss rate, mean delay and jitter).
1
Introduction
MPEG video traffic is expected to represent most of the load in the future highspeed networks. So, adequate traffic models for MPEG Variable Bit-Rate (VBR) video are important for network design, performance evaluation, admission control and resource allocation. The output of a MPEG encoder consists of a deterministic periodic sequence of encoded frames where the period is called GOP (Group of Pictures). A GOP contains three different types of frames: one I-frame (like JPEG, achieved after A. Yazici and C. S ¸ ener (Eds.): ISCIS 2003, LNCS 2869, pp. 651–658, 2003. c Springer-Verlag Berlin Heidelberg 2003
652
J.C. Lopez-Ardao et al.
removing the spatial redundancy within a still image) and several P- and Bframes, which are encoded adding motion compensation with respect to the previous I-frame. It is well-known that VBR video traffic is characterized by a high burstiness and a strong positive correlation. These properties were found to be induced by the presence of a characteristic inherent to VBR video traffic, called selfsimilarity. This characteristic involves long-range correlations over arbitrarily long time-scales, a phenomenon usually referred to as Long-Range Dependence (LRD) [1]. LRD is mainly characterized by an ACF with a hyperbolic decay. Many models for VBR video traffic have been proposed in the literature. In Section 2 we describe their main characteristics. However, while inter-GOP correlation has been widely analyzed in literature [2], [3], [4], little effort has been devoted up to now to the generation of the different frame-types within a MPEG GOP. This issue is often neglected even though it is a fundamental characteristic of MPEG traffic and it might have an important impact on queueing performance [4], [5]. In fact, many works avoid to descend to the frame level and propose models only for the GOP process [6], [7]. Different solutions have been proposed in the literature, but these have never been compared and analyzed jointly. So, this is the main goal of this work. The paper is organized as follows: Section 2 starts classifying the different video models proposed in the literature and discussing their advantages and drawbacks. Also the problem of generating frames within a MPEG GOP and the solutions proposed are dealt. In Section 3 we show the main results related to queueing performance. Finally, Section 4 summarizes the main conclusions of this study.
2
On VBR Video Modeling
Depending on how the relevant time-scales are captured, the VBR video models proposed in the literature (see [8] for a survey) can be classified into two types: hierarchical and non-hierarchical. Hierarchical models typically consider two time-scales: scene or activity level (time-scale of minutes) and GOP/frame (time-scale of milliseconds/seconds) [9], [10], [6]. Scene changes occur when the mean of the bit-rate process changes significantly as a result of a considerable change in picture content (camera cuts). Hierarchical models have several well-known drawbacks. On the one hand, the scene layer adds complexity to the model and, on the other hand, the model has an important subjective component in the scene detection process [11] Non-hierarchical models consider only the GOP/frame level. For this reason, only those models based on stochastic processes that incorporate inherently a wide range of time-scales are valid. In this field, the use of self-similar processes in video traffic modeling [2], [12], [3], [5], [11] has a clear advantage over the traditional use of Markovian or Autoregressive processes [13], [9], [7], since the former models ignore the correlations beyond a particular lag intending to keep
Effect of the Generation of MPEG-Frames
653
the analytical tractability, and so it is said that they exhibit Short-Range Dependence (SRD). But their validity is in doubt because modeling LRD through SRD processes requires many parameters. In other words, it is well-known that it is necessary a sum of infinite exponentials to match a hyperbolic function. The requirement to use few parameters is important because they must be estimated from the empirical data. Each estimate incurs a certain amount of error and, on the other hand, a video traffic model should be independent of any empirical data, and so it is very important that its parameters have physical meaningful. For this reason, the rise of self-similar processes for video traffic modeling purposes has been essential due to their capability to exhibit LRD over all timescales by making use of few parameters (parsimonious modeling). In any case, the impact of the LRD on queueing performance depends much on the length of the busy period (the number of the relevant correlations that interact at the queue) and so this effect can be less important when the utilization is low or with small buffers [14], [15], [16]. However, though this discussion is not within the scope of this paper, it seems clear that, if it is feasible, the use of a simple, parsimonious and efficient model based on self-similar processes (or, in general, exhibiting LRD) will permit to achieve better estimations of the system performance. Furthermore, the model will neither depend on particular and hypothetical assumptions about the load or the buffer sizes, nor on the subjectivity of the scene detection. In order to reinforce this statement, in [11] it is shown that the hyperbolic decay (k −β ) is the best match for the ACF of the different frame-types of many commonly used MPEG-encoded video sequences. As for the choice of the type of LRD processes, we opt to use Gaussian FARIMA(1,d,0) processes for simplicity reasons (only two parameters: the AR(1) coefficient for SRD and d for LRD, with H = d + 0.5 = 1 − β/2) and computational cost (using the fast generation method proposed in [17]). Queuing performance depends not only on the correlation structure, but also on the marginal distribution [14], [16]. In this respect, Huang et al. [3] demonstrated that Gaussian processes enjoy the attractive advantage that their marginal distribution (parameters included) can be changed by a sufficiently regular transformation without modifying their LRD structure. Nevertheless, this transformation induces a slightly decrease of the magnitude of the SRD. However, the use of Gaussian FARIMA(1,d,0) processes permit to easily overcome this inconvenience by adequately adjusting the AR component. An extensive study in [4] concludes that Lognormal or Gamma distributions are an useful approximation of the histograms of the I, P, and B frame sizes for a wide range of MPEG video sequences. Similar results are found for the GOP sizes. We use Lognormal distributions for efficiency reasons. 2.1
On Generating MPEG Frames within a GOP
An important work in this field is due to Lombardo et al. [5], that obtains the conditional distribution of P- and B-frames sizes, given an I-frame size. However,
654
J.C. Lopez-Ardao et al.
this model requires a huge computational cost and it is little flexible. We will focus on other more efficient and simpler solutions for generating MPEG frames within a GOP. In [3] and [12], three different distribution transformations for each frametype, hI (x), hP (x) and hB (x), are applied to a background LRD process that matches the empirical ACF of the I-frame sizes (inter-GOP correlation). We will refer to it as solution HUA. Krunz et al. [10] propose a hierarchical model where the inter-GOP correlation is only taken into account for the I-frames. P- and B-frames are modeled by two independent processes of i.i.d. random variables with Lognormal distribution. We will refer to it as solution KRU. Rose [18] uses a background process that matches the inter-GOP correlation to obtain a GOP-size process. The frames sizes are computed by multiplying the GOP size by a scaling factor. The scaling factor for each frame-type is the quotient between the mean frame-size for that type and the mean GOP-size. With this simple method, P-frames (and B-frames) have the same size in every GOP, and so the frame-by-frame correlation is lost. We will refer to this solution as solution ROS. Finally, Ansari et al. [11] propose to use three independent processes matching the ACFs RI (k), RP (k) and RB (k) of each frame-size process, I(n), P (n) and B(n). We will refer to it as solution ANS.
3
Queueing Performance
Although the validation of a source model requires distribution and autocorrelation fitting, the ultimate validity of a model must be determined by its performance predictions in comparison to empirical data. In the literature, VBR video traffic models are commonly used to predict the queueing performance at a switch. The system used is a single-server FIFO queue with finite buffer size. The capability of the server (bits per second), or its utilization factor, and the buffer size are system variables. In order to analyze the aforementioned solutions for generating the different types of MPEG frames within a GOP, we have made exhaustive trace-driven simulation experiments. The video frames, whose sizes are object of modeling, arrive at switch at a constant rate (25 frames per second in our case). When the buffer limit is reached, new arrivals are dropped. Given that ”goodness” of a traffic model may depend on the metric under study, we analyze three important performance measures (statistics) of the system: loss probability, mean delay (E [Ti ]) and jitter (E [Ti − Ti−1 ]). For the sake of brevity, in this paper we present only the results for the empirical MPEG encoded data The Simpsons, although the considerations and conclusions are valid for all the traces we have analyzed1 . The trace consists of 40.000 frames which is equivalent to approximately half an hour (at a rate of 25 frames per second). The GOP length is 12 (IBBPBBPBBPBB). 1
The traces were courtesy of Dr. O. Rose [4]
Effect of the Generation of MPEG-Frames Buffer size = 1 Mbits
655
Buffer size = 5 Mbits
0.12
0.07 Solution ROS Solution KRU Solution HUA Solution ANS Simpsons trace
0.1
Solution ROS Solution KRU Solution HUA Solution ANS Simpsons trace
0.06
0.05 Loss Probability
Loss Probability
0.08
0.06
0.04
0.03
0.04 0.02
0.02
0 0.4
0.01
0.5
0.6
0.7
0.8
0.9
0 0.4
0.5
Utilization
0.6
0.7
0.8
0.9
Utilization
Fig. 1. Loss probability results
3.1
Loss Probability
Regarding the estimated loss probabilities, the results are very interesting (see Figure 1). Curiously, the solution ROS achieves very good predictions, clearly better than the other solutions. The solution ANS always underestimates the loss rate, although it does not work bad for low and moderate utilization. Nevertheless, the other two solutions work really bad. The solution HUA gives rise to a very high loss rates, indicating that the method for inducing the intra-GOP correlation is not suitable. Finally, the solution KRU hardly generates losses, confirming that modeling just the inter-GOP correlation, in this case through the I-frame sizes, even capturing all the marginal distributions, leads to a very poor model for loss-rate predictions. These results indicate that the GOP-size process is dominant for loss performance. That is, loss rates will be better predicted by the solution that best matches the PDF and the ACF of the GOP-size process, and also takes into account the intra-GOP correlation, even by means of simple averaging factors. It does not seem necessary to devote efforts to capturing the correlation behaviors of the size of each frame-type. 3.2
Mean Delay
For mean delays, the results are slightly different (see Figure 2). Now both the solution ROS and the solution ANS work well enough, although the solution ROS goes on being slightly better. Specifically, it is the best option for small and moderate buffer sizes. The predictions of both solutions get worse when buffer size is increased, whereas the solution HUA obtains better and better predictions as buffer size and/or utilization is increased. In fact, for infinite buffer the predictions are very accurate. Results for the solution KRU are not shown in the graphics because of its extremely poor predictions (very low mean delays) due to the reasons cited in the previous section. In short, when the queue size is not very large, a good approximation consists of modeling only the GOP-size process and introducing the intra-GOP correla-
656
J.C. Lopez-Ardao et al. Buffer size = 1 Mbits
Buffer size = 5 Mbits
0.9
4.5 Solution ROS Solution HUA Solution ANS Simpsons trace
0.8
3.5
0.6
3
0.5
2.5
Delay
Delay
0.7
0.4 0.3
2 1.5
0.2
1
0.1
0.5
0 0.4
Solution ROS Solution HUA Solution ANS Simpsons trace
4
0.5
0.6
0.7
0.8
0.9
0 0.4
0.5
Utilization
0.6
0.7
0.8
0.9
Utilization
Fig. 2. Mean Delay results
tion by simple averaging. Alternatively, adequately capturing the correlation structure of each type of frame also provides acceptable performance results. However, when queue size is very large, and so the correlation structure is much more relevant to performance, the results suggest that the GOP-size process loses relevance in favor of the intra-GOP correlation, and so another method for characterizing this type of correlation is necessary. Perhaps, that is the reason why the interpendence introduced by using the same background process in the solution HUA provides the best results for the largest queue sizes.
3.3
Jitter
Jitter depends on the difference in size between consecutive frames. So, P-frames, and mainly B-frames, play a fundamental role owing to the relative number of these in comparison to I-frames. Properly characterizing the ACF and PDF of the B-frame sizes is still more important because they are sent consecutively. For this reason, the solution ANS is clearly the best. Instead, the solutions ROS or KRU can never be valid. The former loses the frame-by-frame correlation, introducing several consecutive frames of identical size in the same GOP, so leading to many zero-jitter samples. Simulation results (see Figure 3) demonstrate that jitter is grossly underestimated. On the other hand, the solution KRU removes both the intra-GOP correlation and the autocorrelation of each frame-type, and so it is expected that jitter predictions are highly overestimated as the simulation results demonstrate. The solution HUA obtains similar predictions to those of the solution ROS, also due to the fact that the empirical P- and B-frame correlations are not captured. However, as the utilization and the buffer size is increased, slightly better and better predictions are evidenced, perhaps for similar reasons to those cited in the mean-delay case.
Effect of the Generation of MPEG-Frames Buffer size = 1 Mbits
657
Buffer size = 5 Mbits
0.035
0.035 Solution ROS Solution KRU Solution HUA Solution ANS Simpsons trace
Solution ROS Solution KRU Solution HUA Solution ANS Simpsons trace
0.025
0.025 Jitter
0.03
Jitter
0.03
0.02
0.02
0.015
0.015
0.01 0.4
0.5
0.6
0.7
0.8
0.9
0.01 0.4
Utilization
0.5
0.6
0.7
0.8
0.9
Utilization
Fig. 3. Jitter results
4
Conclusions
Many models for VBR video traffic have been proposed in the literature. However, while inter-GOP correlation has been widely analyzed in literature, little effort has been devoted up to now to the generation of the different frame-types within a MPEG GOP. In this paper we have analyzed different solutions proposed in the literature GOP. The results showed us that the suitability of a model depends much on the performance metric to estimate. For loss rate, the results indicate that the GOP-size process is dominant, and so the best option is the one that best matches the PDF and the ACF of the GOP-size process, but also taking into account the intra-GOP correlation, even by means of simple averaging factors applied to the GOP sizes. It does not seem necessary to devote efforts to capturing the correlation behaviors of the size of each frame-type. For mean delay, when the queue size is not very large, the GOP-size process and obtaining the MPEG frame sizes by means of averaging factors are the best option again. However, when the queue size is very large, and so the correlation structure dominates performance, the results suggest that the GOP-size process loses relevance in favor of the intra-GOP correlation, and so another method for characterizing this type of correlation might be necessary. For jitter, adequately characterizing the ACF and PDF of the B-frames is very important because they are sent consecutively. Again, when the utilization and the buffer size are increased, intra-GOP correlation gains relevance. However, regardless of the target performance metric, adequately capturing the distribution and the correlation structure of each type of frame provides acceptable predictions and it is a reasonable compromise solution in those scenarios where the intra-GOP correlation is not important enough, that is, as utilization and buffer size increase, the predictions lose accuracy.
658
J.C. Lopez-Ardao et al.
References 1. Beran, J., Shreman, R., Taqqu M.S., Willinger W.: Long-Range Dependence in Variable-Bit-Rate Video Traffic. IEEE Transactions on Communications Vol. 43, Num. 2-4 (1995) 1566–1579. 2. Garrett, M.W., Willinger, W.: Analysis, Modeling and Generation of Self-Similar VBR Video Traffic. In Proc. of ACM SIGCOMM ’94, London, UK (1994) 269–280. 3. Huang, C., Devetsikiotis, M., Lambadaris, I., Kayevol, A.R: Modeling and Simulation of Self-Similar Variable Bit Rate Compressed Video: A Unified Approach. In Proc. of ACM SIGCOMM ’95, Cambridge, MA USA (1995) 114–125. 4. Rose, O.: Statistical Properties of MPEG Video Traffic and their Impact on Traffic Modeling in ATM Systems. In Proc. of the 20th Annual Conference on Local Computer Networks, Minneapolis, MN (1995) 397–406. 5. Lombardo A., Morabito G., Palazzo., Schembra, G.: MPEG Traffic Generation Matching Intra- and Inter-GoP Correlation. Simulation, Vol. 47, Num. 2 (2001) 6. Jelenkovic, P.R., Lazar, A.A, Semret, N.: The Effect of Multiple Time Scales and Subexponentiality of MPEG Video Streams on Queueing Behavior. IEEE Journal on Selected Areas in Communications, Vol. 43, Num. 2-4 (1995) 1566–1579 7. Frey, M., Nguyen-Quang, S.: A Gamma-Based Framework for Modeling VariableRate MPEG Video Sources: the GOP GBAR Model. IEEE/ACM Transactions on Networking, Vol. 8, Num. 6 (2000) 710–719. 8. Izquierdo, M.R., Reeves, D.S: A Survey of Statistical Source Models for VariableBit-Rate Compressed Video. Multimedia Systems, Vol. 7 (1999) 199–213. 9. Heyman D.P., Lakshman, T.V.: Source Models for VBR Broadcast-Video Traffic. IEEE/ACM Transactions on Networking, Vol. 4, Num. 1 (1996) 40–48 10. Krunz, M., Tripathi, S.K.: On the Characterization of VBR MPEG Streams. In Proc. of ACM SIGMETRICS ’97, Seattle, WA (1997) 192–202. 11. Ansari N., Liu H., Shi Y.Q.: On Modeling MPEG Video Traffics. IEEE Transactions on Broadcasting, Vol. 48, Num. 4 (2002) 337–347. 12. Enssle, J.: Modeling of Short- and Long-Term Properties of VBR MPEGCompressed Video in ATM Networks. In Proc. of the 1995 Silicon Valley Networking Conference & Exposition, San Jose, CA (1995) 95–107. 13. Melamed, B., Pendarakis D.: A TES-Based Model for compressed Star Wars Video. In Proc. of IEEE GLOBECOM ’94 (1994) 120–126 14. Grossglauser, M, Bolot, J.-C.: On the Relevance of Long-Range Dependence in Network Traffic. In Proc. of ACM SIGCOMM ’96, Stanford University, CA (1996) 15–24 15. Heyman, D.P., Lakshman, T.V.: What Are the Implications of Long-Range Dependence for VBR-Video Traffic Engineering?. IEEE/ACM Transactions on Networking, Vol. 4, Num. 3 (1996) 301–317. 16. Ryu B.K., Elwalid A.: The Importance of Long-Range Dependence of VBR Video Traffic in ATM Traffic Engineering: Myths and Realities. In Proc. of ACM SIGCOMM ’96, Stanford University, CA (1996) 3–14 17. L´ opez-Ardao, J.C., Su´ arez-Gonz´ alez A., L´ opez-Garc´ıa, C., Fern´ andez-Veiga, M., Rodr´ıguez-Rubio, R.: On the use of self-similar processes in network simulation. ACM Transactions on Modeling and Computer Simulation, Vol. 10 Num. 2 (2000) 125–151. 18. Rose, O.: Simple and Efficient Models for Variable Bit Rate MPEG Video Traffic. Performance Evaluation, Vol. 30 (1997) 69–85
Multiple Description Coding for Image Data Hiding in the Spatial Domain Mohsen Ashourian1 and Yo-Sung Ho2 1
Azad University of Iran, Majlesi Branch P.O. Box 86315-111, Isfahan, Iran PRKVHQD#LDXPDMOHVLDFLU 2 Kwangju Institute of Science and Technology (K-JIST) 1 Oryong-dong Puk-gu, Kwangju, 500-712, Korea KR\R#NMLVWDFNU
Abstract. In this paper, we develop a robust image data hiding scheme based on multiple description coding of the signature image. At the transmitter, the signature image is encoded by balanced two-description scalar quantizers in the wavelet transform domain. The information of the two descriptions are embedded in the host image in the spatial domain with a masking factor derived from the gradient of the image intensity values. At the receiver, the multiple description decoder combines the information of each description and reconstructs the original signature image. We experiment the proposed scheme for embedding a gray-scale signature image of 128×128 pixels size in the spatial domain of the gray-scale host image of 512×512 pixels. Simulation results show that data embedding based on multiple description coding has low visible distortions in the host image and robustness to various signal processing and geometrical attacks, such as addition of noise, quantization, cropping and down-sampling.
1 Introduction Various digital data hiding methods have been developed for multimedia services, where a significant amount of signature data is embedded in the host signal. The hidden data should be recoverable even after the host data has undergone some signal processing operations, such as image compression. It should also be retrieved only by those authorized [1]. The main problem of image hiding in another host image is a large amount of data that requires a special data embedding method with high capacity as well as transparency and robustness. Chae and Manjunath used the discrete wavelet transform (DWT) for embedding a signature image into another image, which has high visible distortion in the smooth area of the host image [2]. It is possible to improve their scheme by employing the human visual system (HVS) model in the process of information embedding [3,4]; however, exact adjustment of the HVS model is not easy in many applications. As another approach for improving the robustness of data embedding, Mukherjee et. al. [5] designed a joint source-channel coding scheme for hiding $ JCONF Update SHARED TREE –> JREQ 3. [Join operation of a Host] Send JREQ(IP) to all immediate access router If Router NOT in group Initiate Step 2 Else Send Host Add Request JADD(IP, HOST IP)
4. [Leave Operation of MAC] If SHARED TREE = empty Send PRUNE(MAC IP) to all neighbor MAC [On Receive of PRUNE] Delete MAC ROUT TAB –> PRUNE(MAC IP) 5. [Leave Operation from a Non-MAC Host] Send Leave Request MLVE(IP) Delete MAC SHARED TREE –> MLVE 6. [Defining Link as Active or Passive] Receive IP Packet >> RPACKET If RPACKET already RECEIVED Send Make Passive MPAS(MAC IP) [On the Sender MAC] Update MAC ROU TAB –> MPAS Delay t Increase t expotentially
Case 1. In this scenario a router joins the multicast group as a MAC. The joining router sends a join request JREQ to every other immediate MAC neighbors. All the immediate MACs respond with join confirmation JCONF which updates the routing database of the MAC. MAC maintains a routing table for storing every shortest path to all immediate routers in the multicast backbone graph. The MAC will adapt dynamically to its available resources by adding or eliminating some entries in the multicast routing database. Elimination in done only to passive links entries in the MAC routing database. Case 2. In the second case of join operation, the joining node is non-MAC and will have to be spanned to the shared tree managed by a MAC domain, which is second dimension routing of the model. The MAC responds to the join request JREQ from the Non-MAC by sending a join confirmation signal JCONF. Join confirmation will find a shortest sub-optimal path that exist in the shared tree form the MAC towards the joining Non-MAC. If the joining non-MAC is multicast enabled then the JREQ contains only the IP information
Distributed Multicast Routing
751
of the joining node, but in case the joining router is multicast disabled then the JREQ contains IP information of the router and all hosts under the Non-MAC. This information will update the routing database in the MAC, which maintain the IP information for all routers, and hosts in its domain. Case 3. If joining network entity is a host, then it is added to the second dimension of the routing scheme i.e. the shared tree managed by a MAC domain. The host will send a Unicast JREQ to the immediate access router in its network, which will initiate a non-MAC join operation. In case the immediate router is already a member of the shared tree and is multicast disabled, it will send a Unicast packet to the MAC defining the IP information of the joining host. The MAC on receiving the update packet will span the host to the existing shared tree of its domain.
4
Multicast Security
Existing multicast service model raises deployment issues in multicast security. The senders or receivers failed to authenticate before joining the group. Moreover, any host can join the group as a receiver or sender, which makes the multicast vulnerable to eaves dropping and attacks such as Denial of Service, theft of service. These threats can be well defended using group key management and access control mechanisms. 4.1
Design Objectives
In multicast group key management, encrypting the data with a shared key provides confidentiality. Encryption of the multicast traffic is done using a symmetric key and all the authorized users of the multicast group are given the decryption key. Issues regarding group key management get complicated as the group membership is dynamically changed due to newly joining and leaving hosts in the group. When a new member joins the group, the new group key can be sent to the new members using the old key. Matters’ get complicated only when a leave operation is taking place. The naive approach is to re-compute the keys on every leave operation and new key is distributed to all the receivers in the group. This is not optimal since each leave operation requires N different encryptions and transmissions. This model emphasize on reducing N by distributing the host and grouping them in domain controlled by MAC. Distributed two-dimensional routing, supports the security mechanisms such as group key management, access control schemes at an optimum level. Similar to the multicast routing, the group key management uses a traditional tree approach for distributing the keys. Group key management uses the MAC as a key distribution center (KDC) for the domain it controls. In this model, GMAC (Gateway multicast access control) is a MAC that is capable of supporting the multicast backbone graph at the transport layer. Therefore, GMAC controls all the Non-GMACs in the domain of the GMAC. Non-GMACs are MACs, which can support the multicast backbone graph at the network layer. The GMACs in
752
J.F. C and V. S
the multicast backbone graph form a secure backbone for the multicast model. Each GMAC is responsible for distributing the keys to its domain independent of every other GMAC. In the multicast backbone graph network, no security is required at this dimension since any hosts such as a sender or receiver cannot join directly to multicast backbone network. Therefore GMACs are independent and requires no centralized control for graph management.
5
Analysis
The objective of the model is to reduce N, the number of encryptions and transmissions done by a key distribution center by distributing the overhead independently. Rekeying in a conventional approach requires N encryptions and transmission, where N is the group size. This model divides N independently by use of m MAC modes. Therefore rekeying is done only to the domain of the MAC. A discrete event simulator was used for the simulation. The simulation dynamically changed the network topology. This dynamic property of the network topology is required to analyze the multicast graph backbone of the first dimension routing scheme. Every node existence in the topology is dependent upon a probability ranged over uniform distribution. The assumption based on which this property is performed is that network cannot be partitioned. Occurrence of join/leave operation of hosts is ranged over normal distribution. 5.1
Complexity
The simulation process probed the complexity incurred at the node of a MAC. Initial complexity at the MAC node was high compared to IP multicast. The complexity as shown in figure 2a increases proportionally with the group size until it reaches the threshold (T) of the MAC. At this point, the MAC splits its domain by initializing a new MAC, to share its overhead. MAC’s domain size (n) decreases by the new MAC’s domain size (m). IP multicast complexity of the centralized node increases linearly with-respect to the group size of the multicast tree. Analysis of complexity with help of delay characteristics of the model revealed more properties. 5.2
Delay
The latency of the packet forwarded is proportional to the current MAC’s domain size (n), which will not exceed the latency (Md) when T is the domain size and Md is the maximum delay for a MAC. Another important consideration for latency is the sub-optimality of paths in the shared tree maintained by the MAC for its second dimension routing domain. It is important to note that latency is not affected by the actual group size (N) where N is the summation of domain size n for every MAC node in the graph. The domain size of MAC (n) is dependent upon the maximum threshold or overhead the MAC can service without degrading service to higher layers. Average delay incurred in this model is proportional with the number of MAC
Distributed Multicast Routing 10000
10 MAC Threshold Distributed Model IP Multicast
4000
250 Domain Size of the MAC
6000
300
8 Processing time (msec)
Number of Encryptions
8000
6
4
2
2000
0
50 100 150 200 250 300 350 400 450 500 Join Operations for MAC
200 150 100 50
0
0
753
0 0
5
10 15 20 25 30 35 40 45 50 Number of MAC Nodes
0
50 100 150 200 250 300 350 400 450 500 Join Operations for MAC
Fig. 2. a. MAC Domain Split-up at Threshold b. Delay is proportional to No. of MACs c. Increased scalability until threshold is reached
nodes in the multicast network backbone as shown in figure 2b. Network locality of the MAC nodes affects the MACs routing database requirements. The dynamic property of the network topology requires regular updates to the routing database. Locality of node decides the number of immediate neighbor MACs to each other MAC in the topology. Network locality also affects the complexity of join/leave operations. MACs, which exist very remotely relative to the multicast group, will have more stress in link and the nodes of its neighbor MACs , whereas a Non-MAC, which is within the domain of another MAC, will join as MAC with very less complexity. 5.3
Scalability
Scalability is dependent on the maximum domain size of the MAC (T), which is predetermined by the model. Scalability is limited to the point at which no new MAC can be initialized and the current MAC’s domain size has reached the threshold (T) as shown in figure 2c. However, the model provides a fair scalability feature until (threshold) T of the domain is reached and the actual group size (N) is not the domain size of the MAC (n).
6
Conclusion
This routing model supports the deployment of the security and network management schemes such as group key management, access control and packet authentication schemes. Security in multicast content distribution has matured over the years, but there remain issues, which still needs solution. This model reduces latency but if the error-rate in the transmission medium is high then the Quality of Service will be poor compared to IP multicast since the model doesn’t support reliability mechanisms except for FEC techniques. Analysis of the model revealed that distribution of complexity requires more number of active participants as MACs so that overhead is shared. Though this model provides transparency in deployment but it still requires ISPs to enable routers to support multicast. It can be concluded that multicast requires a protocol stack similar to unicast, which obtains support from all the layers.
754
J.F. C and V. S
References 1. Matthew J. Moyer, Josyula R. Rao, and Pankaj Rohatgi, “A Survey of Security Issues in multicast transmission”, IEEE Network Magazine, Vol. 13 No. 6, November 1999 pp 12–23. 2. Jack Snowyink, “A Lower bound for Multicast key distribution”, IEEE INFOCOM 2001. 3. Y. H. Chu, S. G. Rao and H. Zhang, “A case for end system multicast”, in proc. ACM Sigmetrics, Santa Clara, CA, June 2000. 4. N.G.Duffield, J. Horowitz, F.Lo.Presti, “Adaptive multicast topology inference”, IEEE INFOCOM 2001 5. S. Deering, D. Estrin, D. Farinacci, V. Jacobson and L Lei, “The PIM Architecture for wide area multicast” IEEE/ACM Transactions on Networking, Vol. 4, No. 2 April 1996, pp 153–162. 6. T.Ballardie, P. Francis, and J. Crowcroft, “Core Based Trees (CBT): An Architecture for scalable multicast routing” ACM Sigcomm 1995 7. K. Almeroth, “The Evolution of Multicast: From the Mbone to Inter-domain multicast to internet2 deployment” IEEE Network Magazine, Vol. 14 No. 1, January 2000 pp 10–30. 8. Kin-Ching Chan and S.H Gary Chan, “Distributed server approach for large-scale secure multicast”,IEEE Journal of selected areas of communication. Vol 20. No. 8, October 2002, pp 1500–1510. 9. D. Maughan, M. Schertler, M. Schneider and J. Tuner, “ Internet security association and key management protocol” in RFC 2048, April 1998 10. Robert Beverly, “Wide Area IP Multicast Traffic Characterization”, IEEE Network, Vol. 17, No. 1, Jan 2003 pp 8–15. 11. Paul Judge and Mostafa Ammar,”Security Issues and solutions in Multicast content distribution: A Survey”, IEEE Network Magazine, Vol. 17, No. 1, January 2003 pp 30–36. 12. Chung Kei Wong, Mohamed Gouda and Simon S. Lam, ”Secure group communication using key graphs”, IEEE/ACM Transactions on Networking, Vol. 8 No. 1, Feburary 2000 pp 16–30.
Multi-threshold Guard Channel Policy for Next Generation Wireless Networks Hamid Beigy and M.R. Meybodi Soft Computing Laboratory Computer Engineering Department Amirkabir University of Technology Tehran, Iran {beigy, meybodi}@ce.aut.ac.ir
Abstract. In this paper, we consider the call admission problem in next generation wireless networks, which must handles multi-media traffics. We give an algorithm, which finds the optimal number of guard channels, which minimizes the overall blocking probability calls with lowest level of QoS in a multi-cell cellular network subject to the hard constraint on the blocking probabilities of other calls.
1
Introduction
The next generation wireless networks are expected to eventually carry multimedia traffics. In order to support such wide range of traffics on a network, the network must be capable of satisfying various quality of service (QoS) requirements. The satisfying QoS means that the various traffics should get predictable service from the available resources in the network. Since wireless spectrum remains as the prime limited resource in the next generation networks, hence, it is necessary to develop mechanisms that can provide effective bandwidth management while satisfying the QoS requirement for incoming calls. In order to maintain the QoS requirements, call admission control is needed. The call admission control policies determine whether a call should be either accepted or rejected at the base station and assign the required channel(s) to the admitted call. Several call admission control policies have been proposed to reduce the dropping of voice calls in wireless networks [1,2,3,4,5]. However, little attention is paid to wireless multi-media networks. In what follows, we review some of proposed call admission policy in the cellular networks. The simplest call admission control policy is called guard channel policy (GC), which reserves a subset of channels, called guard channels, allocated to the cell for sole use of handoff calls [1]. Whenever the channel occupancy exceeds a certain threshold, this policy rejects new calls until the channel occupancy goes below the threshold. This policy accepts handoff calls as long as channels are available. If only the dropping probability of handoff calls is considered, the
This work is partially supported by Iranian Telecommunication Research Center (ITRC), Tehran, Iran.
A. Yazici and C. S ¸ ener (Eds.): ISCIS 2003, LNCS 2869, pp. 755–762, 2003. c Springer-Verlag Berlin Heidelberg 2003
756
H. Beigy and M.R. Meybodi
guard channel policy gives very good performance, but the blocking probability of new calls is degraded to a great extent. In order to have more control on both the dropping probability of handoff calls and the blocking probability of new calls, limited fractional guard channel policy (LFG) is proposed [3]. The LFG policy reserves non-integral number of guard channels for handoff calls. In [5], uniform fractional guard channel policy (UFG) is introduced, which accepts new calls with a fixed probability. It is shown that, the UFG policy performs better than GC policy in low handoff traffic conditions. In [6,7], a multi-media cellular network with two traffic classes are considered and the call admission problem is formulated as a semi-Markov decision process problem and Q-learning [6] and neuro-dynamic programming [7] are used for finding the optimal policy. All of the above mentioned call admission control policies consider only one threshold to decide for accepting/rejecting of new calls. These policies fail when there is several classes of traffics with different level of QoS. In such cases, we need multi-threshold policy, which provides different thresholds for different classes. In [8], dual-threshold reservation (DTR) scheme is given for integrated voice/data wireless networks. In this scheme, three classes of calls in ascending order of level of QoS are considered which are data calls (both new and handoff calls), new voice calls and handoff voice calls. The basic idea behind the DTR scheme is to use two thresholds, one for reserving channels for handoff voice, while the other is used to block data traffic into the network in order to preserve the voice performance in terms of handoff dropping and call blocking probabilities. The equations for blocking probabilities of DTR are derived and the effect of different values for number of guard channels on dropping and blocking probabilities are plotted, but no algorithm for finding the optimal number of guard channels is given. In [9], a two-threshold guard channel (TTGC) policy, which uses two sets of guard channels, is introduced in which three classes of traffics in ascending order of level of QoS are considered. The limiting behavior of TTGC policy is analyzed under stationary traffic and an algorithm for finding the minimum number of channels in each cell in order to maintain the QoS for all classes of calls. In [10], a prioritized channel assignment algorithm is given which maintains the level of QoS for different classes of calls. This channel assignment algorithm uses the TTGC as call admission control policy. In this paper, we extend the idea given in [11] for single cell case to the multi-cell cellular networks and introduce a channel assignment algorithm for multi-cellular networks consisting of several clusters, where a typical cluster m contains Nm cells. We give an optimal algorithm that divides the channel set allocated to the network into Nm disjoint channel sets, where each channel set is allocated to one cell in the cluster. Then, the channel set of each cell is divided into N subsets. The objective is to find the optimal value for thresholds, which minimize the overall blocking probability of the lowest priority class of calls subject to the hard constraint other calls. By applying our algorithm to each cluster in the system, the channel assignment is obtained for the whole network. The rest of this paper is organized as follows: Section 2 presents the limiting behavior of multi-threshold guard channel policyunder stationary traffics. Section 3 gives an algorithm to find the optimal value of thresholds in multi-cell system and section 4 concludes the paper.
Multi-threshold Guard Channel Policy
2
757
Multi-threshold Guard Channel Policy
In [11], the multi-threshold guard channel policy (MTGC) is introduced. In this policy, we considered network cells with N classes of traffics W = {w1 , . . . , wN } and C full duplex channels. Each traffic class wi (for 1 ≤ i ≤ N ) consists of a stream of statistically identical calls with Poisson arrival at rate λi and independent identical exponentially distributed call holding times with mean 1/µi . Every call requests only one channel. Class k (for k = 1, 2 . . . , N ) has a level of QoS, qk , which must be satisfied. Without loss of generality, it is assumed that q1 ≥ q2 ≥ . . . ≥ qN . Thus, the priority of calls for class k (for k = 1, 2 . . . , N − 1) N k is less than the priority of calls for class (k + 1). Let Λk = j=k+1 λj , αk = Λ Λ0 , N Λ0 µ−1 = j=1 µ−1 j , and ρ = µ . We consider a cell in isolation. The state of a particular cell at time t is defined to be the number of busy channels in that cell, which is denoted by c(t). In order to provide the specified level of QoS for calls of each class, channels which are allocated to the given cell are partitioned into (N ) subsets. In order to partition the channel sets, (N − 1) thresholds, T1 , . . . , TN −1 (0 < T1 ≤ T2 ≤ . . . ≤ TN −1 ) are defined. For the sake of simplicity, two additional fixed thresholds T0 = −1 and TN = C are used. The procedure for accepting calls in MTGC policy, as shown in figure 1. In the MTGC policy, if (Call of Class k) then if (c(t) < Tk ) then accept call else reject call end if end if Fig. 1. Multi-threshold guard channel policy.
{c(t)|t ≥ 0} is a continuous-time Markov chain (birth-death process) with states 0, 1, . . . , C. The state transition diagram of a particular cell in the network, which has C full duplex channels and uses MTGC policy as shown in figure 2. From structure of the Markov chain, we can easily write down the solution to the steady-state balance equations. Define the steady state probability Pn = lim Prob[c(t) = n] t→∞
n = 0, 1, . . . , TN .
(1)
By writing down the equilibrium equations for the steady-state probabilities Pn (n = 0, 1, . . . , TN ), we obtain the following expression for Pn (Tk < n ≤ Tk+1 ). Pn = P0
T n k (ραk ) αj−1 j , n! j=1 αj
(2)
where P0 is the probability that all channels are free and obtained from equation C n=0 Pn = 1 and can be expressed by the following expression.
758
H. Beigy and M.R. Meybodi
- 0
0
-T T T 0
0
1
2
2
-Tk -k Tk k
1
1
( 1 + 1)
1
-TN
N
1
1
TN
Fig. 2. Markov chain model of cell for multi-threshold guard channel policy.
N −1 k
P0 =
k=0 j=1
αj−1 αj
Tj
Tk+1
n=Tk +1
n
−1
(ραk ) n!
.
(3)
Define T as [T0 , T1 , . . . , TN ], ej as the unit ((N + 1) × 1) vector with 1 as the j j th element and the rest zero and Eij as k=i ek . Thus, the blocking probability of class N is equal to
BN (T ) =
N −1 i=1
αi−1 αi
Ti
TN
(ραN −1 ) TN !
P0 .
(4)
Similarly , the blocking probability of class k (K < N ) is given by Bk (T ) =
j N −1 j=k i=1
αi−1 αi
Ti
Tj+1
n=Tj +1
n
(ραj ) P0 . n!
(5)
Bk (.) has interesting properties some of which are listed below. The proof of these properties are given in [12]. Property 1. For any given value of T , we have Bk (T ) ≥ Bk+1 (T ). Property 2. For any given value of T , Bk (.) is a monotonically decreasing function of Tk . Property 3. For any given value of T , Bk (.) is a monotonically increasing function of Tj (j < k). Property 4. For any given values of T , Bk (.) is a monotonically increasing function of Tj (j > k).
Multi-threshold Guard Channel Policy
3
759
Optimal Channel Assignment in Multi-cell Networks
In this section, we extend the idea given in [11] for single cell case to the multicell cellular networks and introduce a channel assignment algorithm for networks consisting of several clusters, where a typical cluster m contains Nm cells. Assume that a total of C full duplex channels are allocated to the whole network and hence to each cluster. Under our channel assignment scheme, the allocated channels will be divided into Nm disjoint channel sets, where each channel set is allocated to one cell in the cluster. Then, the channel set of each cell is divided into N subsets. By applying our algorithm to each cluster in the system, the channel assignment is obtained for the whole network. Assume the exponential channel holding time and Poisson arrivals for each call as in section 2. Let Nm i λ1 be the total arrival rate of class 1 traffics over all cells in cluster m Λ = i=1 and λi1 is the arrival rate of class 1 calls in cell i of cluster m. Define the overall blocking probability of class 1 traffics by B=
Nm λi1 i B (T ), Λn 1 i=1
(6)
where B1i (T i ) is the blocking probability of class 1 traffics in cell i when C i channels are allocated to that cell and T i is the set of thresholds for that cell. The objective is to find the optimal value for T i (i = 1, 2, . . . , Nm ), which minimizes B subject to the hard constraint other calls. This problem is formulated as the following non-linear optimization problem. Problem 1. Minimize the overall blocking probability of class 1 traffics, B, subject to the following hard constraints. Bki (T i ) ≤ qk Nm
C i = C,
for k = 2, . . . , N,
(7) (8)
i=1
for all cells i = 1, 2, . . . , Nm in cluster m. In what follows, we propose an algorithm for solving problem 1. This algorithm uses the convexity1 property of B1 with respect to TN (property 5). Property 5. If thresholds T are fixed, then B1 (T ) is convex in TN provided that ραN −1 TN +1 < 1. Using property 5, it is evident that by adding more channels to a cell while the level of QoS are fixed, the blocking probability of class 1 traffics is decreased. An optimal solution of the problem 1 is found by exploiting the convexity property of B1 . Initially for each cell i, the smallest number of channels required to satisfy the 1
A function f (z) defined on the set of integers Z = {z|z is an integer} is called convex if its first differences are increasing. That is, f (z) is convex if f (z)−f (z +1) ≤ f (z − 1) − f (z) for all z ∈ Z.
760
H. Beigy and M.R. Meybodi
given QoS is found. To do this, we use algorithm given in [12] to find the minimum channels for each cell. Then the remaining channels, if any, are allocated to cells one by one. Let γi denotes the potential amount of decrement in B1i brought by allocation of an additional channel to cell i. Note that the additional channel can be used in any of N subsets. In order to find the usage of the additional channel, the algorithm given in [11] is used. The potential amount of decrement in B1i are computed for all cell i (for i = 1, 2, . . . , Nm ) according to the following equation. λi
γi = 1 B1i (T ∗i ) − B1i T ∗i + eN . Λ Note that γi is always positive. Then a cell with the largest potential decrease in B1 is found among all cells in the cluster and an additional channel is assigned to it. This procedure is repeated until all available channels C in the cluster are used. Algorithm given in figure 3 summarizes this procedure. Algorithm MultiCell-MinBlock 1. Use algorithm in [12] for all cells. Nm i C 2. set S ← C − i=1 3. if S = 0 then terminate. T is optimal. 4. if S ¡ 0 then terminate. C channels cannot satisfy the specified QoS. 5. for i ← 1 to Nm do 6. Use algorithm given in [11] for cell i with C i and C i + 1 channels. 7. 8. 9. 10. 11.
λi
set γi ← Λ1 B1i (T i ) − B1i (T i + eN ) . end for for i ← 1 to S do set j ← argmaxi γi . set C j ← C j + 1. λ
j
12. set γj ← Λ1 B1j (T j ) − B1i (T j + eN ) . 13. end for 14. {T i |i = 1, 2, . . . , Nm } is the optimal solution. end Algorithm Fig. 3. Multi-cell channel assignment algorithm
Theorem 1. Algorithm given in figure 3 finds the optimal solution of problem 1. Proof. The initial assignment is an undominated solution, in the sense that it uses the minimum number of channels to satisfy constraints (7). This assignment results the maximum value of B subject to constraints (7). Then the algorithm assigns channels one by one to cells which results the largest decrement in blocking probability of new calls. This strategy results the optimal solution. Let ji be the index of the cell with the largest decrement in B at step i (for i = 1, 2, . . . , S). Assume that there is another strategy which is optimal and at step i chooses cell
Multi-threshold Guard Channel Policy
761
ki = ji . Thus, there is δi ≥ 0 for which we have γki = γji −δi . Then interchanging cell ji with cell ki results in assignment ki BN =
Nm l λ
1
Λ
l=1
B1l (T l ).
By subtracting B ki from B ji , we obtain B ji − B ki = δi . Repeating this procedure for S steps, we obtain S S
ji
B − B ki = δi , i=1
i=1
which is positive. Thus, no index other than the index with the largest value of γi would results in the optimal solution. Hence, the proposed cell selection mechanism minimizes the value of B subject to the hard constraints (7) and results in the optimal solution. 3.1
Numerical Example
Consider a cellular system with clusters having 4 cells. Assume that a total of 80 full duplex channels are available in this system. The level of QoS for different classes are 0.05, 0.035, 0.02, and 0.01, respectively. The call arrival rates, which are normalized to the call holding time, are given in table 1. The result of algorithm 3 is given in table 2. Table 1. The traffic parameters of cellular network Cell 1 2 3 4
4
Λ0 14 14 30 18
Λ1 9 10 28 12
Λ2 5 5 18 9
Λ3 2 3 5 5
Conclusions
In this paper, we studied the problem of call admission control in the next generation cellular mobile networks, which supports multi-classes of services. We introduced a multi-threshold guard channel policy and derived blocking probabilities of the network. Then we proposed an optimal prioritized channel assignment for multi-class services multi-cellular networks.
762
H. Beigy and M.R. Meybodi Table 2. The result of channel assignment for multi-cell system Cell 1 2 3 4
T1 12 14 21 22
T2 13 15 22 22
T3 13 15 22 22
T4 15 17 24 24
B1 0.180902 0.131336 0.119553 0.016902
B2 0.025208 0.023735 0.024871 0.016902
B3 0.025208 0.023735 0.024871 0.016902
B4 0.002966 0.00356 0.004288 0.002914
References 1. D. Hong and S. Rappaport, “Traffic Modelling and Performance Analysis for Cellular Mobile Radio Telephone Systems with Priotrized and Nonpriotorized Handoffs Procedure,” IEEE Transactions on Vehicular Technology, vol. 35, pp. 77–92, Aug. 1986. 2. S. Oh and D. Tcha, “Priotrized Channel Assignment in a Cellular Radio Network,” IEEE Transactions on Communications, vol. 40, pp. 1259–1269, July 1992. 3. R. Ramjee, D. Towsley, and R. Nagarajan, “On Optimal Call Admission Control in Cellular Networks,” Wireless Networks, vol. 3, pp. 29–41, 1997. 4. G. Haring, R. Marie, R. Puigjaner, and K. Trivedi, “Loss Formulas and Their Application to Optimization for Cellular Networks,” IEEE Transactions on Vehicular Technology, vol. 50, pp. 664–673, May 2001. 5. H. Beigy and M. R. Meybodi, “Uniform Fractional Guard Channel,” in Proceedings of Sixth World Multiconference on Systemmics, Cybernetics and Informatics, Orlando, USA, July 2002. 6. S.-M. Senouci, A.-L. Beylot, and G. Pujolle, “A Dynamic Q-Learning-Based Call Admission Control for Multimedia Cellular Networks,” in Proceedings of the 3rd IEEE International Conference in Mobile and Wireless Communication Networks, MWCN’2001, Recife, Brazil, pp. 37–43, Aug. 2001. 7. S.-M. Senouci, A.-L. Beylot, and G. Pujolle, “Call Admission Control for Multimedia Cellular Networks Using Neuro-Dynamic Programming,” in Proceedings of the IFIP Networking, NETWORKING’02, Pisa, Italy, May 2002. 8. L. Yin, B. Li, Z. Zhang, and Y. Lin, “Performance analysis of a dual-threshold reservation (DTR) scheme for voice/data integrated mobile wireless networks ,” in Proceedings of the IEEE Wireless Communications and Networking Confernce, WCNC. 2000, pp. 258–262, Sept. 2000. 9. H. Beigy and M. R. Meybodi, “An Optimal Prioritized Channel Assignment Scheme for Using in Mobile Transaction Environments,” in Proceedings of 8th Annual International Computer Society of Iran Computer Conference CSICC-2003, Mashhad, Iran, pp. 66–74, Feb. 2003. 10. H. Beigy and M. R. Meybodi, “An Optimal Channel Assignment Scheme,” in Proceedings of 3th Iranian Conference on Electrical Engineering, ICEE-03, Shiraz, Iran, vol. 2, pp. 634–641, 2003. 11. H. Beigy and M. R. Meybodi, “A General Call Admission Policy for Next Generation Wireless Networks,” in Accepted for Presentation and will be Appeared in Proceedings of the second International Symposium on Telecommunications (IST2003), Isfahan, Iran, Aug. 2003. 12. H. Beigy and M. R. Meybodi, “A General Call Admission Policy for Next Generation Wireless Networks,” Tech. Rep. TR-CE-2003-003, Computer Engineering Department, Amirkabir University of Technology, Tehran, Iran, 2003.
Application of Fiat-Shamir Identification Protocol to Design of a Secure Mobile Agent System Seongyeol Kim1 , Okbin Lee2 , Yeijin Lee3 , Yongeun Bae3 , and Ilyong Chung3 1 2
School of Computer and Information, Ulsan College, Ulsan, 682-090, Korea
[email protected] Dept. of Computer Science, Chungbuk University, Cheongju, 361-763, Korea
[email protected] 3 Dept. of Computer Science, Chosun University, Kwangju, 501-759, Korea
[email protected] Abstract. Even though an agent system contributes largely to mobile computing on distributed network environment, it has a number of significant security problems. In this paper, we analyze security attacks to this system presented by NIST[3]. In order to protect it from them, we suggest a security protocol for a mobile agent system by employing Identity-based key distribution and digital multi-signature scheme. To solve the problems described on NIST, securities of mobile agent and agent platform should be accomplished. Comparing with other protocols, our protocol performs both of these securities, while other protocols mention only one of them. Also, it is designed to guarantee the liveness of agent, and to detect message modification immediately by verifying the execution of agent correctly.
1
Introduction
Due to the progress of distributed technology[2] paradigms for drawing up network-based application have been gradually developed. Among these paradigms, the mobile agent paradigm extending the concept of Code-onDemand model has drawn considerable attentions since it has many advantages from the aspects of system level, middleware and user-level. In a mobile agent system, its executable code that is not depended upon a specific system performs tasks during traversal between systems. This system has characteristics of the mobile code executed in JAVA applet, and furthermore, has characteristics of travelling lots of systems and of being mobile in its own when needed. So it is more efficient than the previous models by providing low cost of communication, better asynchronous interaction, and improved flexibility. However, since it has to execute a code coming from the exterior system and this code is exposed thoroughly to the executing environment, this mobile agent system has been faced with serious security problems since the birth.
Corresponding Author: Ilyong Chung (
[email protected])
A. Yazici and C. S ¸ ener (Eds.): ISCIS 2003, LNCS 2869, pp. 763–770, 2003. c Springer-Verlag Berlin Heidelberg 2003
764
S. Kim et al.
In this paper, we analyze security attacks to this system presented by NIST[3]. To solve the problems described on NIST, securities of mobile agent and agent platforms should be accomplished. Vigna et al[4] presents a structure of protecting an agent by transmitting the execution status of agent to a verifier and this verifier would audit it. However, this method would be expensive since a significant amount of network resources would be utilized and execution codes would be expanded. Bennet et al[5] also proposes a Proof Verification, but it was evaluated to be inappropriate for application. In Baek et al[6], the problems seen in [4] and [5] can be solved. An agent is protected by performing a security system using routing information on a agent and information on execution status obtained from digital signatures and audit tools. However, a drawback of this method is that modification activities done by agents can be audited only at the final stage. Since immediate detection of these activities is not performed, a system can take unnecessary overheads and lengths of signature becomes longer owing to repeated usage of signature. A few access methods to protect a mobile agent have been suggested - the first method that does not allow a sufficient amount of time for attackers to analyze the code[7], the second that encrypta an agent where it is located on the perfect environment TPE(tamper-proof environment)[8], the third that enables the execution of the agent encrypted with the key generated by time and environment of server[9]. Although a security protocol for a server against unauthorized agents was suggested while ignoring the danger of server to an agent[10], these systems do not provide mutual authentication between an agent and a server. In this paper, we propose a new protocol to solve the security issues for mobile agent systems by employing the digital multi-signature and Identity-based key distribution scheme. In Identity(ID)-based cryptographic system, there are a public key ID and a secrete key that corresponds to this ID, where a secret key can be generated only at the key distribution center[11]. Since an ID is used as a public key, authentication of the public key is not needed. Furthermore, the electronic signature that plays an important role in an exchange of electronic document satisfies fixing signature length, allowing verification, detection of tampering, maintaining confidentiality and commonality[12], and has accomplished a multi-signature scheme. Also, our protocol can verify an interim signature and the number of communications for a signature is smaller.
2
Design of the Security Protocol for Mobile Agent Systems
The security protocol proposed in this paper obtains secure communications by using one-time password between hosts for each section through Identity-based key distribution rather than maintaining a public key directory. It generates multi-signature on the mobile code and verifies results of the previous step when migrating to next server. Then, the executable code and resulting data can be protected and unauthorized tampering can be detected in real time. Moreover, malicious disposal of agent and unauthorized copying can be detected by moni-
Application of Fiat-Shamir Identification Protocol
765
toring the migration condition of agent at the agent management center. Fig.1 illustrates an interaction of factors required for executing this protocol and Table 1 shows the notation used in this Chapter. 2.1
Registration (Agent-Server Registration and Key Distribution)
All agent platforms should be registered ahead in order to execute services in mobile agent systems. When a request of registration from an AP is called, the AMC generates a key according to the following method, distributes it to agent platforms and registers it. (1) (2) (3) (4) (5)
Select two large prime numbers, p and q, and calculate N = p × q. Compute ϕ(N ) = (p-1)(q-1) and e with gcd(e, ϕ(N )) = 1. Compute d with ed = 1 mod ϕ(N ). Select g that is a primitive root both in GF (p)andGF (q). Compute the values Si and Sij (1 ≤ j ≤ k), kix for APi . Si = APi d mod N Iij = f(APi , j) (j = 1, 2, ..., k) Iij −1 = Sij 2 mod N, (∃ Iij , Iij ∈ QRN )1 kix , where 1 ≤ kix ≤ k , Iij ∈ QRN
(6) Register APi at the agent-sever DB and store p, q securely (7) Distribute the key using smart card containing the following factors
(N, e, g, Si , f, h, Si1 , ...., Sik , ki1 , ..., kix , AP1 , ..., APm , AM C) ki1 , .., kix is a series of j that Iij −1 is a quadratic residue. When the key is renewed in the future, a new key is received using this session key between APi and the AMC and the smart card key is updated.
2.2
CreateAgent
When a agent is created at the homeplace, the APH let the AMC inform this creation and conduct security management. For these purposes, the procedure is designed as follows. (1) The migration path of agent is generated as follows.
A route = AP1 AP2 ... APn
(2) A sign value is setup with executable code of agent and multi-signature
which are yet null state. A sign = HH = f(A code) A code, A sign ∈ MA (3) kH,AM C is created by generating a session key shared with AMC according to registration process. 1
quadratic residue for modulus N
766
S. Kim et al. APH
AP1
AP n
AMC
registration registration createAgent()
A registration
TransferAgent move A createAgent()
exe_result
migrationRequestToNextServer
TransferAgent move A excuteAgent()
exe_result
migrationRequestToNextServer TransferAgent move A
auditAgent()
TransferAgent
exe_result
report()
Fig. 1. Interaction of factors required for execution
(4) kH,1 shared with AP1 in which an agent is executed for the first time, is
created. (5) Agent code and path information are sent to the AMC as follows.
APH → AMC : EkH,AM C (MA , t1H )
(6) A signature is generated as follows.
XH = RH 2 mod N (eH1 , ... , e Hk ) = h (A route, A exe results, HH , XH ) YH = RH SHj mod N , j = 1, 2, ..., k eHj =1
A sign = (HH , XH , kH , YH ) ,
A sign ∈ MA
(7) APH transmits the data to AP1 .
APH → AP1 : EkH,1 (MA , t1H )
Generation of a session key between APi and APj . The session key of APi and APj is generated by applying Diffie-Hellman key exchange method. (1) APi selects a random number Ri ∈ ZN . (2) APi calculates Ci and send it to APj .
Ci = gRi mod N
Application of Fiat-Shamir Identification Protocol
767
Table 1. Notatio used in this protocol Notations
Explanation
AMC
Agent Management Center
APi
ID of the i th
APH
Home Place of Agent
MA
Mobile Agent. This is a set of { A_name , A_code , A_route ,
A_route
Migration Path of Agent
A_name
Name of Agent
A_exe_results
Result of executing an agent
A_code
Executable code of Agent
A_sign
multi-signature for result of executing an agent
f , h
public unilateral function
Hm
Hash values of agent code created by
ki,j
Session key between APi
Ek(M)
Message M is encrypted with the key
Ri
The random number generated by AP i
Ri,j
The random number generated by
t1i , t2 i
Timestamp generated by APi
agent platform on migration path of agent
and
A_exe_results ,
A_sign }
APm
APj
APi
k
to transmit to APj
(3) After APj selects a random number Rj , calculates Cj and send it to APi .
Cj = gRj mod N
(4) APi computes the session key shared with APj
ki,j = (Cj )Ri mod N = gRi ·Rj mod N
(5) APj computes the session key shared with , as follows.
ki,j = (Ci )Rj mod N = gRi ·Rj mod N
2.3
ExecuteAgent
When an agent migrates to APi+1 , a host treats this agent as one thread. (1) APi executes a mobile agent and then it makes logi , and renews
A exe results logi = Eki,AM C (A exe resultsi , t2i ) A exe resultsi = A exe resultsi−1 logi The results of execution should be protected from other agent platforms and finally be verified by the AMC. (2) A signature is made as follows. Ri ∈ ZN Xi = Ri 2 Xi−1 mod N , (i = 1, 2, . . . , k) h(A route, A exe resultsi , Xi , HH ) (eil , . . . , eik ) = Yi = Yi−1 Ri Sij mod N , (j = 1, 2, . . . , k) eij =1
A sign = (HH ,XH ,kH ,X1 ,k1 ,X2 ,k2 ,...,Xi ,ki ,Yi )
768
2.4
S. Kim et al.
TransferAgent
When an agent is transmitted at APH , or a migration request made at APi and an agent migrates to APi+1 , TransferAgent procedure is accomplished between APi (or APH ) and APi+1 . It is designed to guarantee that no security problems have occurred in agent’s performance. (1) APi,i+1 , the session key of APi and APi+1 , is generated. (2) APi+1 to receive an agent generates the session key ki+1,AM C shared with
the AMC. (3) APi transmits the sign for results of execution and agent code to APi+1 .
APi → APi+1 : Eki,i+1 (MA,t1i )
(4) APi+1 can verify the signature written by previous AP as follows.
Ixj = f(APx ,j), (x = 1, 2, . . . , i , j = 1, 2, . . . , k) (ei1 , . . . , eik ) = h(A route, A exe resultsi , Xi , HH ) i Ixj mod N, (j = 1, 2, . . . , k) Xi = Yi 2 x=1 eij =1
If Xi obtained from the fourth step of this procedure is equal to Xi in A sign, the result would be effective. (5) When the verification is correct, APi+1 execute this agent, but if incorrect, APi+1 reports immediately to the AMC. 2.5
AuditAgent
The AMC knows the migration path according to APH → AMC : EkH,AM C (MA,t2H ) performed at the procedure of CreateAgent. Moreover, each time an agent migrates, a session key is shared with APi , the AMC can detect it until an agent stop. When an agent normally travels the planned path and arrives at the AMC, the entire processes of signing can be verified as follows. Iij = f(APi ,j) (ei1 , . . . , eik ) = h(A route, A exe resultsi , Xi , HH ) i Iij mod N, , (x = 1, 2, . . . , n , j = 1, 2, . . . , k) X i = Yi 2 x=1 eij =1
Since A exe results = log1 . . . logn and logi = Eki,AM C (A exe result,t1i ), the result of execution recorded by each server can be deciphered. And then this deciphered data is ciphered with kH,AM C and transmitted to home. Finally, the mobile agent reports to home and terminates execution.
3
Analysis of the Proposed Protocol
Table 2 shows analysis of our protocol compared with other schemes such as Hole method[7], Whlhelm-Stamann method[8], Riordan-Schneier method[9], Baek method[6], in terms of security features.
Application of Fiat-Shamir Identification Protocol
769
Table 2. Table of session key in AMC schemes matters protection method cryptography method authentication method
[7]
[8]
[9]
[6]
proposed protocol
time limited and code mixture
hardware
environmetal key
repeated simple signature
information hiding and muli-signature synchronous by ID key
asynchronous
asynchronous
asynchronous
asynchronous
unidirectional
unidirectional
unidirectional
unidirectional
confidentiality integrity preventing repudiation The result of execution is opened
bidirectional
partially supported not described but supported not described but supported
liveness
oepn
open
open
open
secret
Our protocol can satisfy security requirements - bi-directional authentication, confidentiality, integrity and nonrepudiation and solves the drawback of [6], which constructs a long signature. It is designed to take measures when an illegal tampering against an agent occurred. Employing this scheme, it is able to protect an agent and data efficiently and to guarantee the liveness of an agent.
4
Conclusions
In this paper, we propose the security protocol for a mobile agent system using Identity-based on key distribution and Fiat-Shamir digital signature in order to authenticate between agent and server, to protect the resulting data, and to guarantee liveness of an agent. It is able to do interim verification so that it would not have unnecessary overheads. Identity-based cryptography can be applied in key distribution and digital signature, and have advantages of simplification of key management and of fast signature compared with the public key mechanism. The following are the summary of the characteristics of the proposed protocol. The first characteristic is the structure of simplifying key management. This not only can overcome problems related to directory management for the public key mechanism but also has an advantage to setup a new onetime session key every time. Secondly, it provides security services - authentication, confidentiality, integrity, and nonrepudiation and prevention of replay attack. In this system, one’s own data is never exposed to any other APs by ciphering execution result. Each AP can confirm integrity of agent code, path information, and status information through a verification process before it executes the arrived agent. It immediately reports to the AMC in case that some problems occur. And the repudiation service can be offered by using digital signature. Thirdly, liveness of an agent can be guaranteed. The AMC receives A route generated by each AP at every hop of agent migration and then the AMC monitors unauthorized termination done by an arbitrary AP. Fourth, the result of execution should be protected. Problems related to agent behaviors by reading the results of execution obtained from previous steps or by modifying them illegeally are solved.
770
S. Kim et al.
This security model premises that an agent will migrate along the planned path. Later this method, if the mobile platform is determined at the time of migration, should consider management of locations for remote access and recording an active list of APs.
References 1. Dale, J. and Mamdani, E., ”Open Standards for Interoperating Agent-Based Systems,” In Software FOCUS, Wiley, 2001. 2. Poslad, S. and Calisti, M., ”Towards Improved Trust and Security in FIPA Agent Platforms,” Autonomous Agents 2000 Workshop on Deception, Fraud and Trust in Agent Societies, Spain, 2000. 3. Jansen, W. and Karygiannis, T., ”Mobile Agent Security,” NIST Special Publication 800–19, 1998. 4. Vigna, G., ”Protecting Mobile Agents through Tracing,” Mobile Object Systems ECOOP Workshop, 1997. 5. Bennet, S. Y., ”A Sanctuary for Mobile Agents,” DARPA Workshop on Foundations for Secure Mobile Code Workshop, pp.26–28, 1997. 6. Baek, J. and Lee, D., ”Security of Mobile Agent Using Digital Signature and Audit trail,” Proc. of KISS Fall Conference, Vol. 24, No.2, KISS, 1997. 7. Hohl, F., ”An approach to solve the problem of malicious hosts,” Universitat Stuttgart, Fakultat Informatik, Fakultatsbericht Nr., 1997.3. 8. Wilhelm, U. G. and Stamann, S., ”Protecting the Itinerary of Mobile Agents.” Proc. of the ECOOP Workshop on Distributed Object Security, pp.135–145, INRIA, France, 1998. 9. Riordan, J. and Schneier, B., ”Environmental Key Generation towards Clueless Agents,” Mobile Agents and Security, pp.15–24, Springer-Verlag, 1998. 10. Ordille, J., ”When agents roam, who can you trust?,” Proc. of the First Conference on Emerging Technologies and Applications in Communications, Porland, May 1996. 11. Shamir, A., Identity-based cryptosystem and signature scheme,” Advances in Cryptology, Springer-Verlag, pp.47–57, 1985. 12. Kim, S. and Won, D., ”A Study on the Special Digital Signature Systems,” KIISC, Vol.6, No.2, pp.21–32, 1996. 13. Gang, C., ”A study on the digital multisignature scheme and applications,” Ph.D. dissertation, Chungnam National Univ., Taejon, Korea, 1993.
Neural Network Based Optical Network Restoration with Multiple Classes of Traffic 'HPHWHU*|NÕúÕNDQG6HPLK%LOJHQ Electrical and Electronics Engineering Department, METU, Ankara, 06531 Turkey. Phone: +90 312 210 2319 Fax: +90 312 210 1261 GHPHWHUBJRNLVLN#\DKRRFRPELOJHQ#PHWXHGXWU Abstract. Neural-network-based optical network restoration is illustrated over an example in which multiple classes of traffic are considered. Over the preplanned primary and backup capacity, optimal routing and wavelength assignment is carried out. In case of a network failure, protection routes and optimum flow values on these protection routes are extracted from a previously trained feed-forward neural network which is distributed over the optical data communications network. Keywords: optical networks, wavelength division multiplexing, restoration, neural networks.
1 Introduction Optical networks provide a large communication capacity which needs to be managed effectively. Restoration of failed facilities after network failures is a significant problem of network management. The purpose of this work is to illustrate the operation of a neural-network-based restoration method [1, 2] over an example that involves traffic from two classes of security (secure – insecure) and three classes of reliability (path protected, link protected and unprotected). The restoration method combines a previously defined distributed routing algorithm [3] with needs introduced for different types of users in terms of security restrictions and survivability needs.
2 Spare Capacity Assignment User specific static optimal design of a %100 mesh survivable optical network is handled using Integer Linear Programming. The basic approach is similar to that proposed by [4] but here, traffic that belongs to all three classes of reliability and secure as well as insecure users is aggregated. 2.1 Input Parameters Peak Demand Values of Users. Maximum number of sessions that can be requested between each network node is input to the system.
$7 @ = (>τ @ + − λ(>τ @ ∞ (2) (>τ @ = ∫ [Iτ [ G[ = S + − S µ µ
∞ (3) (>τ @ = ∫ [ Iτ [ G[ = S + − S µ µ From the equations (1), (2) and (3), we can get the average response time of the mobile stations as shown in the Fig. 1. To get this result, the average response times of mobile stations in the subnet are assumed as 0.1 second for the migratory mobile stations and 0.01second for the resident mobile stations. As shown in Fig. 1, we can see that the average response time of the mobile station in the subnet is growing as the whole data arrival rate increases and the probability of the migratory mobile station in the subnet increases, which can be explained by the triangular route of the mobile station. [2] In the situation that the response time increases, it becomes very hard for the mobile station to regulate its data rate according to the TCP ACK/NAK and also the probability of the congestion will
Network Level Congestion Control in Mobile Wireless Networks
781
A v e rag e R e spo nse T im e C o m p arison p= 0.3 p= 0.5 p= 0.7 p= 0.9
1 0.9
Averag e T im e
0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0
1
2
3
4
5
6
7
8
9
10
Arrival R ate
Fig. 1. Average response time with variation of the migratory mobile stations
increase mostly in the connecting routers. It can also be interpreted that the average response time of the whole traffic in a subnet is mainly affected by the data rate and the probability of the migratory mobile stations in the subnet. As a result we need a subsidiary congestion control mechanism in the mobile access subnet in addition to TCP’s window control. That scheme must prevent the mobile subnet from going into congestion and help all the mobile stations to keep as much performance as the subnet can support.
3 ECN Broadcasting in Local Sub-networks The very good candidate for the congestion control in the mobile wireless networks can be the Explicit Congestion Notification (ECN) [7], which is now being considered to be used in Internet. The ECN provides a mechanism for intermediate routers to send early congestion feedback to the source before actual packet losses happen. The routers monitor their queue length. If the queue length exceeds a threshold, the router marks the Congestion Experienced bit in the IP header. Upon the reception of a marked packet, the receiver marks the ECN Echo bit in the TCP header of the acknowledgement to send the congestion feedback back to the source. Then the source can reduce the data rate on receiving the message. However, in the mobile wireless networks, where the mobility can affect the delay and the congestion as shown in the previous section, the return path of the mobile station can be varied. Mostly the return path of the migratory mobile stations will increase because the returning packets will take the detour to the source. And the easily congested part of the mobile wireless networks would be the sub domain router (SDR) or the foreign agent (FA) router. When it experiences congestion, all destinations that have received packets passing through it have to return a message to
782
S. Kim
,30 R ELOLW\ 0'
%76%6&
66 9/5
+/5
$$$
,QWHUQHW
6'5)$
6'5)$
,3 %DFNERQH QHWZRUN
'*
6'5)$
0'0R ELOH'HY LFH %76%DVHWUD QVFHLYHUV\ VWHP %6&%DVHVWDWLRQFRQWUROOHU )6)UD PHVHOHFWLRQ
6'5) $6 XEGR PDLQURXWHU)RUHLJQDJH QW '*+$'R PDLQJDWHZD\ +R PHD JHQW +/5+R PHORFDWLRQUHJLVWHU 9/59LVLWRUORFDWLRQUH JLVWHU $$$$XWKH QWLFDWLRQDXWKRUL ]DWLRQDFFRXQWLQJVHUYHU
Fig. 2. Future Mobile Wireless Network Architecture
the each source to inform the congestion. Consequently ECN alone cannot be effective in resolving congestion in the mobile wireless networks. Here, we propose the ECN broadcasting SDR or FA of the wireless access network. The future mobile wireless networks will be composed of mobile access networks as shown in Fig.2. And each access network is managed by SDR or FA. When the ECN broadcasting is adopted in SDR or FA, the congestion information can be broadcasted on the congestion of the corresponding local access network. For an example, if the congestion happened in the local access network, that information could be returned to the mobile station after the corresponding receiver of each mobile station sent the packet with congestion experienced bit set. That will definitely cause the performance degradation of the communication in the local wireless access network. However, if the ECN is broadcasted in the local wireless network using backward congestion radio channel, all the mobile stations could react on congestion promptly no matter how long the return path of each mobile station is. In addition, the broadcasting is an advantageous feature in wireless radio network and IP network.
4 Simulation Results Simulations are done with OPNET MIL3 v6.0. In the simulations, we assumed that all the data traffic of the wireless subnet goes to Internet through the SDR or FA that can deal with traffic amount to Internet with a bandwidth allowance and manage the mobility with the ability of forwarding the data to the corresponding mobile terminals.
Network Level Congestion Control in Mobile Wireless Networks
783
And the all mobile terminals are assumed to have the sliding window and slow start to regulate the traffic amount of the end terminal based on the time-out and NAK received. The response time of the migratory mobile station is assumed to be delayed in Internet due to the mobility support based on Mobile IP. All other simulation details are followings. Link Speed: 10Kbps SR/FA router Buffer Size: 10 packets One way packet delivery time of mobile station: 1sec Packet return time of a migratory mobile station: 3sec Max. window size in sliding window: 64 packets Source data rate: 0.1~10 packets/s Packet time-out in sliding window: 5sec ECN threshold in SR/FA router: 7packets Proportional variation of the new and existing mobile stations: 0.1~0.9 Figure (a) and (b) in Fig.3 show the throughput comparisons between when the ECN broadcasting is adopted and when only ECN is adopted. From each result, we can see that the throughput has been improved when ECN broadcasting is adopted. This improvement was found mostly after the congestion was encountered around 1 packet/sec of source data rate, which shows the proposed ECN broadcasting scheme has performed more properly after the congestion. The packet drop rate in SDR/FA with only ECN goes up sharply as source data rate increases as shown in Fig.3 (d). Instead when the ECN broadcasting is adopted, the packet drop rate in SDR/FA is relatively low as in Fig.3 (d) and this can be managed by handling the threshold of ECN in the SDR/FA router buffer. Therefore, we can expect that the mobile subnet that is linked through one SDR/FA to the Internet can be more stable when the ECN broadcasting is adopted in the SDR/FA.
5 Conclusions To provide the Internet service in the future mobile wireless networks, it’s necessary to consider the network level assistance to keep end-to-end throughput better and network stable. In accordance with these requirements, ECN could be the solution for resolving and avoiding the network congestion. However, since the mobility has an effect on delay and congestion together, ECN alone cannot be the best solution for the congestion in the mobile wireless networks. In this paper, we have proposed the ECN broadcasting to utilize the advantages of IP and radio access networks. The proposed ECN broadcasting can be the easy and simple way to resolve the congestion when it is adopted in subnet router or FA router. This paper also has shown the performance improvement with simulation results. Judging from these results, the proposed scheme, ECN broadcasting is expected to become a fairly good solution in reducing congestion in mobile wireless subnet and improve the throughput of the Internet service in wireless IP networks.
784
S. Kim
(a) Throughput comparison with p=0.4
(b)Throughput comparison with p=0.9
3DFNHW'URS5DWH SD FNHWVHF
3URSRUWLRQRIQHZFRPLQJ PRELOHWHUPLQDOV
'DWD5DWHSDFNHWVHF
(c) Packet drop rate with ECN broadcasting
3DFNHW'URS5DWH SDFNHWVHF
'DWD 5DWHSDFNHWVHF
3URSRUWLRQRIQHZFRPLQJ PRELOHWHUPLQDOV
(d) Packet drop rate with ECN without Broadcasting Fig. 3. Simulation Results
Network Level Congestion Control in Mobile Wireless Networks
785
Still more research is necessary in comparing the performances when the traffic types are various in the mobile wireless networks and the service priority has to be considered in the future research work. Acknowledgement. This Research was financially supported by Engineering Research Center of Hansung University in the year of 2003.
References >@ *LULVK 3DWHO 6WHYHQ 'HQQHWW ³7KH *33 DQG *33 0RYHPHQWV 7RZDUG DQG $OO,3 0RELOH1HWZRUN´,(((3HUVRQDO&RPPXQLFDWLRQV$XJXVWSS± >@ -DPHV'6RORPRQ³0RELOH,37KH,QWHUQHW8QSOXJJHG´3753UHQWLFH+DOO8SSHU 6DGGOH5LYHU1HZ-HUVH\ >@ %DODNULVKQD + .DWV 6HVKDQ 6 ³$ &RPSDULVRQ RI 0HFKDQLVP IRU ,PSURYLQJ 7&3 3HUIRUPDQFHRYHU:LUHOHVV/LQN´$&06,*&200$XJXVW >@ 5DPDFKDQGUDQ 5 HW DO ³,3%DVHG $FFHVV 1HWZRUN ,QIUDVWUXFWXUH IRU 1H[W*HQHUDWLRQ :LUHOHVV'DWD1HWZRUNV´,(((3HUVRQDO&RPPXQLFDWLRQV$XJXVWSS± >@ &DFHUHV5DQG,IWRGH/³,PSURYLQJWKH3HUIRUPDQFHRI5HOLDEOH7UDQVSRUW3URWRFROVLQ 0RELOH &RPSXWLQJ (QYLURQPHQWV´ ,((( -RXUQDO RQ 6HOHFWHG $UHD LQ &RPPXQLFDWLRQV -XQH >@ $GEL 5 0 6HVKDGUL 0 ³&RQWURO DQG 0DQDJHPHQW LQ 1H[W*HQHUDWLRQ 1HWZRUNV &KDOOHQJHVDQG2SSRUWXQLWLHV´,(((&RPPXQLFDWLRQ0DJD]LQH6HSWHPEHUSS± >@ . 5DPDNULVKQDQ DQG 6 )OR\G CC$ 3URSRVDO WR DGG ([SOLFLW &RQJHVWLRQ 1RWLILFDWLRQ (&1 WR,3
5)&-DQXDU\
Network Dependability: An Availability Measure in N-Tier Client/Server Architecture )OiYLD(VWpOLD6LOYD&RHOKR-DFTXHV3KLOLSSH6DXYp &OiXGLD-DF\%DUHQFR$EEDVDQG/XLV-DYLHU*DUFtD9LOODOED 1
Catholic University of Brasília, Dept. Computer Science, Brasília-DF, Brazil IFRHOKR#XFEEU 2 Fed. Univ. of Campina Grande, Dept. Systems and Computation, Campina G.-PB, Brazil MDFTXHV#GVFXIFJHGXEU 3 University of Brasilia, Dept. Electrical Engineering, LabRedes, Brasília-DF, Brazil EDUHQFR#UHGHVXQEEU 4 Complutense Univ. of Madrid, Dept. Computer Systems and Programming, Madrid, Spain MDYLHUJY#VLSXFPHV Abstract. Published work on computer network reliability frequently uses availability as a dependability measure. However, although several ways of defining availability have been proposed, none capture the overall level of service obtained by client hosts in a modern n-tier client/server architecture. We propose such a measure by calculating the fraction of client hosts receiving complete services from the network. We also extend a published efficient heuristic method for calculating availability to take into account our new proposed measure. The end result is a procedure of polynomial complexity O(nt4), where nt is the total number of components (hosts, links and interconnection equipment) in the network.
1
Introduction
Computer networks must provide infrastructure that meets requirements imposed by the applications necessities. One of the main performance measures of interest is availability [1]. This measure is especially important in applications such as ecommerce, banking transaction systems, etc., where the mission-critical role of applications is evident. Today, the main possibility of deploying applications is through the use of an n-tier client/server architecture [2]. Especially popular is the 3tier architecture composed of the user services tier (client hosts), the middle tier (web servers, application servers, directory services servers, etc.) and the data tier (corporate database servers, mail servers, etc.). The middle tier is also frequently called the business tier, since this is where business application logic mostly executes. Other than corporate servers accessed by all clients, other servers may exist: these are departmental servers, frequently used by workgroups for file storage and print services. It must be noted that whereas departmental servers are accessed by client hosts in a particular workgroup, corporate servers in the business and data tiers are accessed by all, or most, client hosts in the network. In the context of such corporate networks, a particular client host obtains services from many servers. In order for a particular client host to obtain full services from the network, the following $VFPVLM@ZKHUH ≤V ≤QI ≤LM ≤ QK 6&0VLVRIRUGHUQK[QKDQGLVVWUXFWXUHGVLPLODUO\WRWKH'HVLUHG&RQQHFWLYLW\ 0DWUL[ 7KH YDOXHV VFPVLM DUH REWDLQHGDV VKRZQ DV IROORZV VFPVLM LI KRVW L LV FRQQHFWHGWRKRVWMVFPVLM RWKHUZLVH:LWKWKHVHPDWULFHVLQKDQG2$06$0V '&0L6&0V ZHDUHQRZUHDG\WRGHILQHWKHQHZDYDLODELOLW\PHDVXUHIRUDQHWZRUN EDVHGRQDQQWLHUFOLHQWVHUYHUDUFKLWHFWXUH:HFRQVLGHUDOOSRVVLEOHIDLOXUHVWDWHV,Q D QHWZRUN ZLWK QW FRPSRQHQWV VXVFHSWLEOH WR IDLOXUH WKHUH DUH QW SRVVLEOH IDLOXUH VWDWHV7KHIDLOXUHVWDWHVDUHGHQRWHGE\6NZKHUH ≤N ≤QW6NUHSUHVHQWVWKHNWK IDLOXUHVWDWHZKHUH6 φ6 ^`6 ^`DQGVRRQ7KXV6FRUUHVSRQGVWRWKH VWDWH ZLWKRXW IDLOXUH DQG 6Q W FRUUHVSRQGV WR WKH VWDWH ZKHUH DOO FRPSRQHQWV KDYH QW
IDLOHG6WDWH6NRFFXUVZLWKSUREDELOLW\36N ∏ L SLTLSL 7L6N ZKHUHSL≤L≤QW LVWKHSUREDELOLW\RIFRPSRQHQWLEHLQJRSHUDWLRQDOTL SLDQGWKHYDOXHRI7L6N LV VKRZQ DV IROORZV 7L6N LI FRPSRQHQW L LVRSHUDWLRQDOLQVWDWH 6N7L6N RWKHUZLVH)LQDOO\WKHQHZQHWZRUNDYDLODELOLW\PHDVXUHWKDWLVWKHIUDFWLRQRIFOLHQW QF KRVWVUHFHLYLQJIXOOQHWZRUNVHUYLFHVLVJLYHQE\$ ∑ N 36N $6N ZKHUH$6N LV WKHIUDFWLRQRIFOLHQWVUHFHLYLQJIXOOVHUYLFHVLQIDLOXUHVWDWH6NDVVKRZQDVIROORZV$ QF >∑ L $L6N @QF ZKHUH $L6N LQGLFDWHV ZKHWKHU FOLHQW L LV UHFHLYLQJ IXOO QHWZRUN 6 VHUYLFHV LQ IDLOXUH VWDWH 6N $L6N LI '&0 L $1' 6&0 N = DCM i , Ai(Sk) = 0 otherwise. In this last equation, AND is a Boolean operation and must be realized cell by cell to compare the Desired Connectivity Matrix for client i (DCMi) and the Actual Connectivity Matrix for failure state Sk (SCMS k). Thus, DCM i AND SCMS k = DCM i, if and only if dcmijt AND VFP6N GFPLMW∀≤MW≤QK In this case, connectivities for client i are satisfied and it is receiving full services from the network in failure state Sk. Evaluating network availability by considering all 2nt possible failure states results in an algorithm with exponential complexity and thus imposes serious restrictions on the size of the network considered. [5] presents a detailed study concerning the difficulty of calculating traditional availability measures. In the next section, we present a heuristic method for evaluating network availability using the measure introduced above. Only the most probable failure states are considered and the algorithm has polynomial complexity. 3
Efficient Evaluation Method for the Availability Measure
The straightforward evaluation of the new availability measure, A, has computational complexity O(2nt), since all failure states must be considered. We seek a heuristic method which would allow its approximate calculation in polynomial time. Such a method can be found in [6] and is based on enumerating the most probable failure states. The mathematical model used is as follows: a computer network is represented by a non-directed graph G = (V, E) with nv = nh + ni vertices (one vertex for each host and interconnection equipment) and nl edges (one for each bi-directional communication link). V is a finite set of vertices V = {v1, v2, ..., vnv } and E is a finite VHWRIHGJHV( ^HHHQO`ZKHUHHDFKHGJHLVLGHQWLILHGE\DSDLURIYHUWLFHVHN
Network Dependability: An Availability Measure in N-Tier Client/Server Architecture
791
YDYE ZKHUH≤N≤QLDQG≤DE≤QYDQGYDYE∈93URSHUORRSVD YDYD DUH SHUPLWWHG6SHFLILFDOO\ZHKDYH9 ^&&QF6G6GQGV6PF6PFQPFV 6GF6GFQGFV(T(TQL`DQG( ^HHQO`9HUWLFHVDQGHGJHVPD\EHLQ RQH RI WZR VWDWHV RSHUDWLRQDO RU IDLOHG ,QWHUPHGLDWH VWDWHV VXFK DV RQH LQ ZKLFK FRPSRQHQW SHUIRUPDQFH LV GHJUDGHG DUH QRW FRQVLGHUHG. To each component (vertex or edge) i is associated a probability pi of being in the operational state. qi = 1 - pi is the probability of this same component being in the failed state. These states change at random and are independent of one another. Finally, the components are ordered such that R1 ≥ R2 ≥ ... ≥ Rnt, where Ri = qi/pi. To determine the most probable network failure states, the states are generated in decreasing order of probability according to a failure state generation algorithm given in [6]. For the nm most probable failure states, Sk, where ≤N≤QP we have P(S1) ≥ P(S2) ≥ ... ≥ P(Snm). The value of nm may be chosen in several ways: as a fixed value, as a fraction of all states, as a function of the total number of components, or in such a way as to obtain a desired level of precision while calculating network availability. Since only most probable states are considered, it would be useful to obtain upper (Asup) and lower (Ainf) bounds for the availability measure, A. [6] shows how to do this as follows. Let M be an arbitrary performance measure, and let M(Sk) be its value when the network is in failure state Sk. This measure must be such as to obey the following relation n given by: M(S2 t) ≤ M(Sk) ≤ M(S1), where 1 ≤ k ≤ 2nt. In other words, network performance is best in state S1, when the network is fully operational and is worse in state S2nt, when all components have failed. We can thus obtain an upper bound by using the measure M(S1) for all states not considered in the enumeration and we can obtain a lower bound by using the measure M(S2nt) for all states not considered in the enumeration. For the proposed availability measure, A, we have the next expressions: n n n Ainf = Σ m k=1 P(Sk)A(Sk) + (1 - Σ m k=1 P(Sk))A(S2nt) and Asup = Σ m k=1 P(Sk)A(Sk) + n
(1 - Σ m k=1 P(Sk))A(S1). Since in state S2nt all components have failed, we have A(S2nt) = 0. Further, in state S1 no component has failed and we have A(S1) = 1. Then, n
n
n
Ainf = Σ m k=1 P(Sk)A(Sk) and Asup = Σ m k=1 P(Sk)A(Sk) + (1 - Σ m k=1 P(Sk)). We are ready to give the algorithm for calculating the upper and lower bounds for network availability. Purpose: calculate upper and lower bounds for the availability of a network based on a n-tier client/server architecture, that is, the fraction of client hosts receiving full network services. Input: number of clients, number of departmental servers, number of middle tier corporate servers, number of data tier corporate servers, total number of components, type of each component and its probability of failure (in increasing order of probability), list of departmental servers and the clients it serves, list of adjacencies between hosts and interconnection equipment. Output: upper and lower bounds for the network availability by considering the nm most probable network failure states.
792
F.E. Silva Coelho et al.
(9$/8$7(1(7:25.$9$,/$%,/,7 100) since group operations are carried out in parallel within subrings. As a future work, we plan on improving the SSRP protocol by adding authentication service into the key management process.
References 1.
Akay, O., Erciyes, K.: A Dynamic Load Balancing Model For A Distributed System. Mathematical and Computational Applications,Vol. 8 No. 3 (2003) 353–363 2. Amir, Y., et. al.: Secure Group Communication in Asynchronous Networks with Failures: Integration and Experiments. Proceedings of the 20th IEEE International Conference on Distributed Computing Systems (2000) 330–343 3. Amir, Y., et al.: Exploring Robustness in Group Key Agreement. Proc. of the 21st Int’l. Conf. on Distributed Computing Systems --ICDCS’01 (2001) 399–408 4. Amir, Y., et al.: On the Performance of Group Key Agreement Protocols. Proc. of 22nd Int’l. Conf. on Distributed Computing Systems --ICDCS'02 (2002) 463–464 5. Burmester, M., Y. Desmedt, Y.: A Secure and Efficient Conference Key Distribution System. Advances in Cryptology – EUROCRYPT’94 (1994) 275–286 6. Erciye•, K.: Implementation of A Scalable Ring Protocol for Fault Tolerance in Distributed Real-Time Systems. Proc. of 6th Symp. On Comp. Networks –BAS 2001 (2001) 188– 197 7. Kihlstrom, K.P., Moser, L. E., Melliar-Smith, P.M.: The Secure Ring Protocols for Securing Group Communication. Proc. of the IEEE 31st Hawaii International Conference on System Sciences, Vol. 3 (1998) 317–326 8. Kim, Y., Perring, A., Tsudik G.: Simple and Fault-tolerant Key Agreement for Dynamic Collaborative Groups. 7th ACM Conf. on Comp. & Communication Security (2000) 235– 244 9. Mittra, S.: Iolus: A Framework for Scalable Secure Multicasting. Proceedings of the ACM SIGCOMM’97 (1997) 277–288 10. Rodeh, O., Birman, K., Hayden, M., Xiao, Z., Dolev, D.: Ensemble Security. Tech. Rep. TR98-1703, Cornell University, Dept. of Computer Science (1998). 11. Steer, D., Strawczynski, L., Diffie, W., Wiener, M.: A Secure Audio Teleconference System. Advances in Cryptology – Lecture Notes in Computer Science (1990) 520–528
g6D÷ODP0('DONÕOÕoDQG.(UFL\Hú 12. Steiner, M., Tsudik, G., Waidner, M.: Key Agreement in Dynamic Peer Groups. IEEE Trans. on Parallel and Distributed Systems. Vol. 11. No. 8 ( 2000) 769–781 13. Steiner, J.G., Neuman, C., Schiller, J.I.: Kerberos: An Authentication Service for Open Network Systems. Usenix Winter Conference (1988) 191–202 14. Tunal•, T., Erciye•, K., Soysert, Z.: A Hierarchical Fault-Tolerant Ring Protocol For A Distributed Real-Time System. Special issue of Parallel and Distributed Computing Practices on Parallel and Distributed Real-Time Systems, Vol. 2, No. 1 (2000) 33–44 15. Vitenberg, R., Keidar, I., Chockler, G.V., Dolev, D.: Group Communication Specifications: A Comprehensive Study. Tech. Rep. CS0964, Comp. Sci. Dept., Technion (1999)
A New Role-Based Delegation Model Using Sub-role Hierarchies† HyungHyo Lee1, YoungRok Lee2, and BongHam Noh2 1
Div. of Information and EC, Wonkwang University, Iksan, 570-749, Korea KOHH#ZRQNZDQJDFNU 2 Dept. of Computer Science, Chonnam National University, Gwangju, 500-757, Korea ^\UOHHERQJQDP`#FKRQQDPDFNU
Abstract. Delegation in computer systems plays an important role in relieving security officer’s management efforts, especially in a large-scale, highly decentralized environment. By distributing management authorities to a number of delegatees, scalable and manageable security management functionality can be achieved. Recently, a number of researches are proposed to incorporate delegation concept into Role-Based Access Control(RBAC) model, which is becoming a promising model for enterprise environment with various organization structures. In this paper, we propose a new role-based delegation model using sub-role hierarchies supporting restricted inheritance functionality, in which security administrator can easily control permission inheritance behavior using sub-roles. Also, we describe how role-based user-to-user, role-to-role delegations are accomplished in the proposed model and analyze our delegation model against various delegation characteristics.
1 Introduction The basic idea behind delegation is that some active entity in a system delegates authority to another active entity to carry out some functions on behalf of the former[4]. Delegation can be defined as a trust-based authority transfer among users, and it be used as a basis for distributed network and systems management. In the mean time, Role-Based Access Control(RBAC) model is becoming a promising model for enterprise environments with various organization structures. In RBAC model, permissions are associated with roles, and users are made members of appropriate roles based on their responsibilities and qualifications, acquiring the permissions of these roles[1]. Recently, a number of research activities are published to incorporate delegation concept into RBAC model[3,4,10]. RBDM0[3] and RDM2000[10] are precursors of delegation-enhanced RBAC models that focus on user-to-user, total delegation. But there would be a security breach of least privilege principle since they only support total delegation, and they entail an increased overhead for multiple delegation due to user to user delegation. †
This work was supported by University IT Research Center Project and partially by Wonkwang University in 2002
$0 for di = +1 (1) wT xi + b 2 i=1 j=1
(9)
Covert Channel Detection in the ICMP Payload
subject to 0 ≤ αi ≤ c, i = 1, ..., l and
l
αi yi = 0
831
(10)
i=1
The hyperplane decision function can thus be written as l yi αi · (x · xi ) + b f (x) = sgn
(11)
i=1
where b is computed using (8). To construct the SVM, the optimal hyperplane algorithm has to be argumented by a method for computing dot products in feature spaces nonlinearly related to input space. The basic idea is to map the data into some other dot product space (called the feature space) F via a nonlinear map φ, and to perform the above linear algorithm in F , i.e nonseparable data {(xi , di )}li=1 , where xi ∈ RN , di ∈ {+1, −1}, preprocess the data with, φ : RN → F , x → φ(x) where l dimension(F )
(12)
Here w and xi are not calculated. According to Mercer’s theorem, φ(xi ), φ(xj ) = K(xi , xj )
(13)
and K(x, y) can be computed easily on the input space. Finally the nonlinear SVM classifier becomes l αi di K(xi , x) + b (14) f (x) = sgn i=1
4 4.1
ICMP Covert Channel Detection An Overview of ICMP Covert Channel
ICMP type 0x0 specifies an ICMP echo reply and type 0x8 indicates an ICMP echo request. This is what the ping program does. This ping traffic is ubiquitous to almost every TCP/IP based network and subnetwork. As such, many networks consider ping traffic to be benign and will allow it to pass through, unmolested. ICMP echo packets also have the option to include a payload. This data section is used when the record route option is specified, or the more common case, usually the default to store timing information to determine round-trip times. Although the payload is often timing information, there is no check by any device as to the content of the data. So, as it turns out, this amount of data can also be arbitrary in content as well[8]. The arbitrary contents of the payload can have various data according to the message types of ICMP protocol and kinds of the operating system(OS). In case of the normal ICMP packet, it has insignificant values or null values and so on. Namely, therein can lie the covert channels.
832
4.2
T. Sohn et al.
Proposing the Detection Method of ICMP Covert Channel
In this paper, we propose a model to detect covert channel in the ICMP payload. The payload of ICMP packets have generally null values or several characteristics dependent on the OS such as Windows, Linux and Solaris as illustrated in Table 1. At this time, the characteristic of payload of each OS is normally the same or it has the successive change of one byte in the payload. The rest 4 bytes of ICMP header are dependent on the each ICMP message type. Thus, we propose the detection method of ICMP covert channel using SVM with the characteristic of ICMP packet payload and the 4 bytes of ICMP header described above. First, we collect normal ICMP packets using a packet capturing tool like tcpdump and abnormal ICMP packets generated by covert channel tool like Loki2[8]. Then we preprocess the collected raw packets such as ICMP payload(13 dimensions) and ICMP payload plus the rest 4 bytes of ICMP header(15 dimensions) as illustrated in Figure 1. One dimension of preprocessed data is comprised of Table 1. The Characteristic of ICMP payload
Null Packet Win Packet Solaris Packet Linux Packet
ICMP Payload 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0900 6162 6364 6566 6768 696a 6b6c 6d6e 6f70 7172 7374 7576 7761 50ec f53d 048f 0700 0809 0a0b 0c0d 0e0f 1011 1213 1415 1617 1819 9077 063e 2dbd 0400 0809 0a0b 0c0d 0e0f 1011 1213 1415 1617 1819 15 dimensions type
code
checksum
Rest Of Header
type
code
checksum
Rest Of Header
Payload
13 dimensions Payload
Fig. 1. The features of SVM
two bytes of raw packet. So, 26bytes ICMP payload is 13 dimension and the rest of header(4bytes) + 26bytes ICMP payload is 15 dimension. At this time, the preprocessed packets are classified as a training data set and a test data set. Each dimension, which means one input feature in SVM, is converted to decimal value, that is, the hexa values of 16bits(2bytes) are rearranged by the integer value of decimal in the raw dump values of packet. Finally, we learn the SVM using training data set and classified test data set with the SVM classifier[11].
5 5.1
Experiment Experiment Methods
The SVM data is first comprised of two data sets : a training data set and a test data set. Also these data sets consist of the training set1, training set2 and test set1, test set2 as followed in Table 2. We preprocessed the raw packet
Covert Channel Detection in the ICMP Payload
833
Table 2. SVM Training Data Set and Test Data Set Data Set Normal Packet Abnormal Packet
Training Set1(Total 4000) All typed ICMP packet(2000) Loki packet (2000)
Data Set Normal Packet
Test Set1 (Total 1000) All typed ICMP packet(250) + OS based ICMP packet(250) Abnormal Packet Loki packets(500)
Training Set2(Total 4000) OS based ICMP packet(2000) Loki packet (2000) Test Set2 (Total 1000) All typed ICMP packet(250) + OS based ICMP packet(250) Loki packets(500)
values in order to make training data and testing data in the SVM. Through the preprocessing, the feature is determined. The determined features are comprised of two cases : a 15 dimension including the rest 4bytes of packet header and packet payload and a 13 dimension including only the payload of packet. Each training data set is comprised of 4000 packets. Training set1 data has general ICMP packets collected by CIST server(Our Institute Web Server). Training set2 data has ICMP packets based on the characteristic of operating systems(Linux, Solaris, Windows). Also, abnormal packets are generated by Loki2 tool. Each training set contains 2000 abnormal packets. Next, the test set1 and the test set2 consist of 500 normal packets and 500 abnormal packets. Here, the normal packets of the test set have 250 general ICMP packets and 250 OS dependent packets. The abnormal packets of the test set have packets which are forged using Loki2 tool. SVM detection tool used here is the freeware package mySVM[11]. To compare the detection performance, we used the two SVM kernels : linear, polynomial type. Table 3. The experiment results of ICMP covert channel detection
Training Set Training Set1
Training Set2
Kernel
Features 13 Linear 15 13 Polynomial 15 13 Linear 15 13 Polynomial 15
Test Set1 FP FN TC 2.5 0.4 97.1 1.1 0.8 98.1 0.2 0.6 99.2 0.8 0.6 98.6 24.3 0.8 74.9 0 0.8 99.2 3.8 0.6 95.6 0.8 0.6 98.6
Test Set2 FP FN TC 0.7 0.7 98.6 0 0.6 99.4 0 0.8 99.2 0 0.8 99.2 12.1 1.6 86.3 0 0.6 99.4 2.5 1.0 96.5 0 0.2 99.8
*The degree of Polynomial Kernel = 3, FP = False Positive(%), FN = False Negative(%), TC = Total Correctness(%)
5.2
Experiment Results
In the detection experiments of the ICMP covert channel using an SVM, the learning data set is classified as training set1 and training set2 according to the
834
T. Sohn et al.
characteristic of the ICMP payload. We analyzed the detection results of each test set1 and test set2 as the two SVM kernel functions and the variation of the number of features. Table 3 shows overall experiment results. The resultant graph of covert channel detection using training set 1 is shown in Figure 2 and the resultant graph of covert channel detection using training set 2 is illustrated Figure 3. In case of the SVM training set, we can see that it is more efficient to classify the abnormal packets by assuming all the general types of ICMP packets are normal(The detection rate of Training set1 is 98.68%). Also, we can see in table 4 that detection performance is better in a 15 dimension and is best in polynomial kernel with degree of 3. In this paper, we proposed the SVM method with some features of ICMP. Such an SVM could detect ICMP covert channel with the correction rate of nearly 99%(As illustrated in Table 4). Table 4. The experiment results of each parameter(%) TR1 TR2 KR1 KR2 F13 F15 Detection(%) 98.68 93.79 94.13 98.34 93.43 99.04 *TR1 = Training Set1, TR2 = Training Set2, KR1 = Linear, KR2 = Polynomial, F13 = 13 Features, F15 = 15 Features
Correctness False Positive False Negative
100
80 70
40
98.9
98.75
50
99.2
60 97.85
Detection Rate (%)
90
30 20 10
1.65 0.5
Linear Kernel, 13 Features
0.7
0.55 0.7 0.1
0.7 0.4
Linear Kernel, Polynomial Kernel,Polynomial Kernel, 15 Features 15 Features 13 Features The Result of Training Set1
Fig. 2. The results of Training Set1
6
Conclusion and Future Work
Covert channel attacks are an increasingly potential threat to Internet environments. We did not yet have the best solution for covert channel detection. The goal of this paper was to propose the detection method for ICMP covert channels with SVM among the many covert channels. The preprocessing for SVM learning to detect a covert channel consisted of two cases: one case includes only
Covert Channel Detection in the ICMP Payload
835
an ICMP payload and the other case includes an ICMP payload and the remaining 4 bytes of the ICMP header. We classified training sets into training set 1 with generally used ICMP packets and training set 2 with ICMP packets based on the characteristic of the operating system. Also, the experiment environment has been subjected to informal tests in a laboratory test bed. The experiment results show that under these conditions, the detection provided by the SVM learning described the high correction rate as illustrated in Table 4. Future work will include the expansion of training sets, test sets and the experiments for various kernels which can be use for performance improvement and some of its constraint parameters.
Correctness False Positive False Negative
100
80 70
40
99.2
50
96.05
99.3
60
80.6
Detection Rate (%)
90
30 18.2 20 10
3.15 1.2
Linear Kernel, 13 Features
0
0.7
0.8
0.4 0.4
Linear Kernel, Polynomial Kernel, Polynomial Kernel, 15 Features 15 Features 13 Features The Result of Training Set 2
Fig. 3. The results of Training Set2
References 1. U.S. Department Of Defence, ”Trusted Computer System Criteria.”,1985 2. Vapnik V., ”The Nature of Statistical Learning Theory”, Springer-Verlag, 1995. 3. Bueges C J C., ”A Tutorial on Support Vector Machines for Patter Recognition.”, Data Mining and Knowledge Discovery, Boston, 1988. 4. S.M. Bellovin, ”Security Problems in the TCP/IP protocol suite”, Computer Communication Reviews,19(2):32–48, April 1989 5. John McHugh, ”Covert Channel Analysis”, Portland State University, 1995 6. Craig H. Rowland, ”Covert Channels in the TCP/IP protocol suite”, First Monday, 1996 7. John Giffin, ”Covert Messaging Through TCP Timestamps”, PET2002 8. Daemon9, ”Loki: ICMP Tunneling”, Pharack Magazine, Volume 6, Issue 49 9. Cristianini N., ”An Introduction to Support Vector Machines.”, Cambridge University press, 2000. 10. S. Mukkamala, G. Janowski, A. H. Sung, ”Intrusion Detection Using Neural Networks and Support Vector Machines”, IEEE IJCNN, May 2002, pp.1702–1707. 11. Joachmims T,”mySVM – a Support Vector Machine”, Univerity Dortmund
An Efficient Location Area Design Scheme to Minimize Registration Signalling Traffic in Wireless Systems ÿ·v³6²yÕuhxhýqArªh7êyÃph! Uüxv²uXh¼8¸yyrtr²Xh¼Bh·výt
hýq Tv·Ãyh³v¸ý8rý³¼rÁrývyr½rý³ D²³hýiÃy Uüxr' XDVOLKDN#KDUSDNWVNPLOWU !D²³hýiÃy UrpuývphyVýv½r¼²v³'@yrp³¼vphy @yrp³¼¸ývp² AhpÃy³' 9r¦³ ¸s8¸·¦Ã³r¼ @ýtvýrr¼výt Hh²yhx "##%(D²³hýiÃyUüxr' EX]OXFD#FVLWXHGXWU
6i²³¼hp³ Dý ³uv² ¦h¦r¼hýrZ ²³h³vp ³¼hssvpih²rq y¸ph³v¸ý h¼rhqr²vtý ²pur·r ýh·rq @U7G69rýuhýprq U7G69ü s¸¼Zv¼ryr²² ²'²³r·² v² ¦¼¸¦¸²rq Uuv² ²pur·rv² hý rýuhýpr·rý³ ¸ý ³ur¦¼r½v¸Ã²y' ¦Ãiyv²urq U7G69²pur·rDý ³ur ýrZ ·r³u¸q ³ur r`¦rp³rq vý³r¼pryy ·¸½r·rý³ ¦h³³r¼ý² ¸s ·¸ivyr² h¼r qr³r¼ ·výrq hýq ³urý ³ur pryy² h¼r ¦h¼³v³v¸ýrq vý³¸ y¸ph³v¸ý h¼rh² G6²ü i' h¦¦y'výt ³ur ýrZ ³¼hssvpih²rq pryy t¼¸Ã¦výt hyt¸¼v³u· Zuvpu uh² ³Z¸ t¸hy² Av¼²³ ³ur pryy ¦hv¼² Zv³u uvtur¼ vý³r¼pryy ·¸ivyr ³¼hssvp h¼r t¼¸Ã¦rq vý³¸ ³ur ²h·r G6 Trp¸ýq ³ur G6²vý Zuvpu ³ur ýrvtui¸Ã¼ pryy² uh½r uvtur¼ vý³r¼pryy³¼hssvphy y¸Zrq ³¸ výpyÃqr ·¸¼rpryy² ³uhý ³urG6² Zur¼r ³ur vý³r¼pryy³¼hssvpv² y¸ZUur hv·¸s³ur ²pur·rv² ³¸qrp¼r·rý³³ur vý³r¼G6·¸½r·rý³² Zuvpu p¼rh³r¼rtv² ³¼h³v¸ý ³¼hssvp @`¦r¼v·rý³hy ¼r²Ãy³²²u¸Z³uh³ ³urýrZ²pur·r ¦r¼s¸¼·² ir³³r¼ ³uhý ³ur U7G69hýq ¦¼¸`v·v³' ih²rq y¸ph³v¸ý h¼rh qr²vtý Q7G69ü²pur·r² vý³ur²rý²r³uh³v³phý¼rqÃpr ³ur y¸ph³v¸ý æqh³r²
Dÿ³¼¸qÃp³v¸ÿ
Pýr ¸s³ur·hvý ¦¼¸iyr·²vý ·¸ivyrp¸·¦Ã³výt v² ³ur³¼hpxvýt ¸s³urpü¼rý³y¸ph³v¸ý ¸s ³ur òr¼² ·¸ivyr²ü Xurý h p¸ýýrp³v¸ý ýrrq² ³¸ ir r²³hiyv²urq s¸¼ h ¦h¼³vpÃyh¼ òr¼³urýr³Z¸¼xuh²³¸qr³r¼·výr³uròr¼¶²r`hp³y¸ph³v¸ýZv³uvý ³urpryyt¼hýÃyh¼v³' G¸ph³v¸ý ·hýhtr·rý³ vý½¸y½r² ³Z¸ ¸¦r¼h³v¸ý²) Uur ¸¦r¼h³v¸ý ¸s výs¸¼·výt ³ur ýr³ Z¸¼x hi¸Ã³³urpü¼rý³ y¸ph³v¸ý ¸s ³ur ·¸ivyr òr¼v² xý¸Zý h² y¸ph³v¸Q¼rtv²³¼h³v¸Q hýq ³ur ¸¦r¼h³v¸ý ¸s qr³r¼·vývýt ³ur y¸ph³v¸ý ¸s ³ur ·¸ivyr òr¼ v² phyyrq ³r¼·vQhy ¦htvQtbd H¸ivyv³' ³¼hpxvýtr`¦rýq²³uryv·v³rq¼r²¸Ã¼pr²¸s³urZv¼ryr²²ýr³Z¸¼x 7r²vqr³ur ihýqZvq³u òrq s¸¼ ¼rtv²³¼h³v¸ý hýq ³r¼·výhy ¦htvýt ¦¸Zr¼ v² hy²¸ p¸ý²Ã·rq s¼¸· ¦¸¼³hiyr qr½vpr² A¼r¹Ãrý³ ²vtýhyyvýt ·h' hy²¸ ¼r²Ãy³ vý qrt¼hqh³v¸ý ¸s RÃhyv³' ¸s Tr¼½vprqÃr ³¸vý³r¼sr¼rýpr²Uur¼rs¸¼r³urt¸hy¸sy¸ph³v¸ý·hýhtr·rý³v²³¸·výv ·vªr³ur²vtýhyyvýt³¼hssvpb!d 8ü¼rý³ýr³Z¸¼x² òr y¸ph³v¸ý h¼rhG6üih²rq ·hýhtr·rý³³rpuýv¹Ãr²Uurp¸½ r¼htrh¼rh v²¦h¼³v³v¸ýrq vý³¸ h ý÷ir¼ ¸s G6² rhpu p¸ý³hvývýt h t¼¸Ã¦¸s pryy² Xuvyr h·¸ivyr v²·¸½výts¼¸· ¸ýr G6³¸ hý¸³ur¼ v³¼r¦¸¼³² v³² ýrZG6 i' h ¼rtv²³¼h³v¸ý ¦¼¸ pr²²6ý G6·h' ²³h³vp¸¼ q'Qh·vp6²³h³vpG6p¸ý²v²³²¸sht¼¸Ã¦ ¸spryy²³uh³h¼r ¦r¼·hýrý³y' h²²vtýrq ³¸³uh³G6hýq v²sv`rqs¸¼hyy·¸ivyr²Pý³ur¸³ur¼uhýqq' 6Áhªvpvhýq8ùrýr¼@q²ü)DT8DT!"GI8T!©%(¦¦©"%û©#"!" T¦¼výtr¼Wr¼yht7r¼yvýCrvqryir¼t!"
An Efficient Location Area Design Scheme
837
ýh·vpG6²b"db#dh¼r p¼rh³rq s¸¼rhpu·¸ivyrqüvýth¼rtv²³¼h³v¸ý ¦¼¸pr²²ih²rq ¸ý ·¸ivyv³' hýqphyy¦h³³r¼ý²¸s³uròr¼ 6y³u¸Ãtu²vtýhyyvýt³¼hssvp phýir¼rqÃprq i' òvýtq'ýh·vpG6²³ur' v·¦¸²ruvtur¼p¸·¦Ã³h³v¸ýhýq ²r¦h¼h³rqh³h²³¸¼htr ¸sG6² s¸¼rhpu·¸ivyr6²h¼r²Ãy³·¸²³¸s³urpü¼rý³pryyÃyh¼²'²³r·²Ã²r²³h³vpG6² UZ¸·hvýshp³¸¼²hssrp³výt³ur ²vtýhyyvýt³¼hssvph¼r³urý÷ir¼¸spryy²výhý G6 hýq ³urpryy¦h¼³v³v¸ývýt ³rpuýv¹Ãrb$dXurý ³urý÷ir¼¸spryy²výhý G6v²uvtu³ur ¼rtv²³¼h³v¸ý³¼hssvpqrp¼rh²r²ió³ur¦htvýt ³¼hssvpvýp¼rh²r²b%dPý ³ur¸³ur¼uhýq s¸¼ ²·hyyr¼ G6² ³ur¼rtv²³¼h³v¸ý ³¼hssvpvýp¼rh²r²ió³ur¦htvýt ³¼hssvpqrp¼rh²r²Uur pryy¦h¼³v³v¸ývýt ³rpuýv¹Ãr v² hy²¸ ½r¼' v·¦¸¼³hý³ Ds ³ur G6² h¼r qr²vtýrq ²Ãpu ³uh³ ³urvý³r¼G6 ·¸ivyr³¼hssvpv² ¼rqÃprq³ur¼rtv²³¼h³v¸ý³¼hssvp qrp¼rh²r²s¸¼³ur²h·r G6²vªr Dý ³uv² ¦h¦r¼ Zr ¦¼¸¦¸²r h ýrZ ²³h³vp y¸ph³v¸ý h¼rh qr²vtý ²pur·r ýh·rq @U7 G69 @ýuhýprq U7G69ü Zuvpu v² hý rýuhýpr·rý³ ¸ý¦¼r½v¸Ã²y' ¦Ãiyv²urq U¼hs svpih²rq y¸ph³v¸ý h¼rh qr²vtý U7G69ü ³rpuýv¹Ãr b&d Dý U7G69 ³rpuýv¹Ãr ³ur vý³r¼pryy³¼hssvp¦¼rqvp³v¸ýhýq ³¼hssvpih²rqpryyt¼¸Ã¦výt²pur·r²h¼ròrq p¸ý²rpà ³v½ry' ³¸ ¦h¼³v³v¸ý³urpryy²vý³¸G6² Uurýrvtui¸Ã¼ pryy²Zv³u uvtur¼ vý³r¼pryy³¼hssvp h¼rh²²vtýrq³¸ ³ur ²h·rG6²³¸qrp¼rh²r³urvý³r¼G6 ·¸½r·rý³²¸s ·¸ivyr²7óvý ³uv²³rpuýv¹Ãr³ur¦¼¸¦r¼ý÷ir¼¸spryy²vý hG6v²ý¸³qr³r¼·výrq Pü@U7G69²pur·rqvssr¼² ²vtývsvphý³y' s¼¸·³urU7G69²pur·ri' r`¦yvp v³y' ³hxvýtvý³¸ hpp¸Ãý³³ur ý÷ir¼¸spryy²výhýG6Uur vý³r¼pryy³¼hssvp¦¼rqvp³v¸ý² h¼ròrq i' ³ur³¼hssvpih²rqpryyt¼¸Ã¦výt ²pur·r³¸t¼¸Ã¦pryy²vý³¸G6²hýqhy²¸³¸ qr³r¼·výr³ur¦¼¸¦r¼ ý÷ir¼ ¸spryy²výhý G6UurG6² vý Zuvpu³urýrvtui¸Ã¼pryy² uh½r uvtur¼vý³r¼pryy³¼hssvp·h' výpyÃqr·¸¼rpryy²³uhý ³urG6²Zur¼r³urvý³r¼pryy ³¼hssvp v² y¸Z Xr ³¼' ³¸ výp¼r·rý³ ³ur vý³¼hG6 ·¸½r·rý³² ¸s ·¸ivyr² vý ¸¼qr¼ ³¸ qrp¼r·rý³³urvý³r¼G6·¸½r·rý³²Zuvpup¼rh³r¼rtv²³¼h³v¸ý³¼hssvp Dý ³ur ¼r·hvýqr¼ ¸s ³ur ¦h¦r¼ Zr hýhy'ªr ³ur ýrZ y¸ph³v¸ý h¼rh qr²vtý ²pur·r @U7G69 hýq ²u¸Z ³uh³ v³ ¼rqÃpr² ³ur ¼rtv²³¼h³v¸ý ³¼hssvp p¸·¦h¼rq ³¸ ¸³ur¼ ³Z¸ ²³h³vpy¸ph³v¸ýh¼rhqr²vtý ²pur·r²ýh·ry' ³ur¦¼¸`v·v³'ih²rq y¸ph³v¸ý h¼rhqr²vtý Q7G69ühýq ³ur³¼hssvpih²rqy¸ph³v¸ý h¼rhqr²vtýU7G69ü²pur·r
!
Q¼¸`v·v³'7h²rqhÿqU¼hssvp7h²rqG¸ph³v¸ÿ6¼rh9r²vtÿ
Dý ³ur¦¼¸`v·v³'ih²rq ²³h³vp G69 Q7G69ü pryy²h¼rt¼¸Ã¦rqvý³¸ ²³h³vpG6²ih²rq ¸ý ¦¼¸`v·v³'Uurpryy²py¸²r³¸rhpu ¸³ur¼h¼r t¼¸Ã¦rq vý³¸G6²²Ãpu ³uh³hyy¸s³ur pryy² iry¸ýt ³¸ ¸ýr hýq ¸ýy' ¸ýr G6 Dý Avt h sv½rpryy ²³h³vp G6² h¼r p¼rh³rq i' òvýt³urQ7G69³rpuýv¹Ãr@hpu ur`ht¸ý¼r¦¼r²rý³²hpryyhýq i¸yq yvýr²hy¸ýt ³ur pryy i¸¼qr¼² ²r¦h¼h³r³urG6² DýAvt h vsh·¸ivyr·¸½r² ir³Zrrý ¦¸vý³² h hýqi s¸yy¸Zvýt³uruvtuZh'v³p¼¸²²r²³urG6 i¸¼qr¼²sv½r³v·r²Crýpr³uv²·¸ivyrýrrq² ³¸¼rtv²³r¼sv½r³v·r²ZurýZròr³ur Q7G69²pur·rh²²u¸ZývýAvth Dý ³ur³¼hssvpih²rq ²³h³vpG69U7G69üb&dpryy² h¼rt¼¸Ã¦rq hpp¸¼qvýt³¸³ur r`¦rp³rqvý³r¼pryy·¸ivyv³' ¦h³³r¼ý² ²Ãpu³uh³³urpryy¦hv¼²Zv³uuvtur¼ vý³r¼pryy·¸ ivyr ³¼hssvph¼rt¼¸Ã¦rqvý³¸ ³ur²h·r G6 Uur t¸hy¸s ³ur ²pur·rv² ³¸¼rqÃpr ³urvý ³r¼G6·¸½r·rý³²¸s·¸ivyr²Uur ²h·r²prýh¼v¸výAvt i v²hy²¸Ã²rq ³¸ r`¦yhvý u¸ZU7G69 ³rpuýv¹Ãr¼rqÃpr² ³urý÷ir¼ ¸s vý³r¼G6 ·¸½r·rý³² Xurý Zr òr ³ur U7G69 ²pur·r ³ur ·¸ivyr ·¸½výt ir³Zrrý ¦¸vý³² h hýq i p¼¸²²r² hý G6 i¸Ãýqh¼'¸ýy' ¸ýpr
©"©
ÿ 6²yÕuhxhýqA 7êyÃph
highway i
i
h
h
hþQ¼¸`v·v³'ih²rqG69
i h
iþ U¼hssvpih²rqG69
pþ @ÿuhÿprqU7G69
Avt 8¸·¦h¼v²¸ý¸s³u¼rr ²³h³vpG69²pur·r²
"
Uur@ÿuhÿprqU¼hssvp7h²rqG69r²vtÿTpur·r@U7G69þ
Dý ³uv² ²rp³v¸ý Zr qr½ry¸¦ h ýrZ ²³h³vp y¸ph³v¸ý h¼rh qr²vtý ²pur·r ýh·rq @U7 G69³uh³v²hýrýuhýprq ½r¼²v¸ý¸s³ur ¦¼r½v¸Ã²y' ¦Ãiyv²urq U7G69²pur·rUur ·hvý qvssr¼rýpr ir³Zrrý @U7G69 hýq U7G69 v² ³uh³ vý U7G69 ³ur ¦¼¸¦r¼ ý÷ir¼ ¸s pryy²vý hý G6 ³ur²vªr ¸sG6üv² ý¸³ qr³r¼·výrq A¸¼ hyy G6²³ur²h·r ¦¼rqrsvýrq·h`v·Ã·½hyÃrv² òrq³¸ yv·v³³urý÷ir¼ ¸s³urv¼·r·ir¼ pryy² Pý ³ur ¸³ur¼ uhýq vý @U7G69 qvssr¼rý³ yv·v³ ½hyÃr² h¼r h¦¦yvrq ³¸ G6² hpp¸¼qvýt ³¸ ³ur ³¼hssvp ir³Zrrý ³urv¼ ýrvtui¸Ã¼ pryy² Uur G6² vý Zuvpu ³ur ýrvtui¸Ã¼ pryy² uh½r uvtur¼vý³r¼pryy³¼hssvp·h' výpyÃqr·¸¼rpryy²³uhý³ur G6²Zur¼r ³urvý³r¼pryy³¼hs svp v²y¸Z T¸Zr phýxrr¦ ·¸¼rpryy ¦hv¼² Zv³uuvtu vý³r¼pryy³¼hssvpvý ³ur²h·rG6 6²v³v²²u¸ZývýAvtp³urG6³uh³p¸½r¼²³uruvtuZh' uh²·¸¼rpryy²³uhý ³ur ¸³u r¼²vs³ur@U7G69v²Ã²rq D³ phòr²³uh³³ur·¸ivyr·¸½výtir³Zrrý¦¸vý³²h hýq i ²³h'²vý³ur²h·rG6hýqq¸r²ý¸³p¼rh³rh¼rtv²³¼h³v¸ý³¼hssvp 7rphòr ³ur ¦h¦r¼ ¼r¦¸¼³² hý rýuhýprq ½r¼²v¸ý ¸s ³ur U7G69 ²pur·r s¸¼ rh²' p¸·¦h¼v²¸ý Zr ²uhyy ¼r³hvý hýq òr¹Ã¸³r ·¸²³ ¸s ³ur ý¸³h³v¸ý² hq¸¦³rq i' 8h'v¼pv hýq 6x'vyqvªb&dvý ³urv¼ qr½ry¸¦·rý³hýqhýhy'²v² ¸s³urU7G69 ²pur·r hýq hq¸¦³ ³ur²h·rs¼h·rZ¸¼xh²Ã²rqi'³ur· "Uur A¸¼·Ãyh³v¸ÿ¸s³urTpur·r Av¼²³ ²³r¦ s¸¼ ³ur @U7G69 v² ³¸ qr³r¼·výr ³ur r`¦rp³rq h·¸Ãý³ ¸s ·¸ivyr ·¸½r ·rý³²ir³Zrrýýrvtui¸Ã¼pryy²Uurvý³r¼pryy³¼hssvp¸s·¸ivyr²v²¦¼rqvp³rq i' r`h· vývýt ³ur¼¸hq²uvtuZh'² ¼hvy¼¸hq² ²³¼rr³² s¸¸³¦h³u² r³pü ²výpr ·¸ivyr² trýr¼hyy' ·¸½r¸ý ¼¸hq²Uur r`¦rp³rq ³¼hssvp³xvw s¼¸·pryyv¸s G6 x ³¸ v³²ýrvtui¸Ã¼ pryywv² p¸·¦Ã³rqs¼¸· ¼
ÿþý
³xvw = ∑ q xvwª ª =
Zur¼r
ü
An Efficient Location Area Design Scheme
839
qxvwªv²³ur³¼hssvpqrý²v³' ¸s¼¸hq ªir³Zrrýpryyv¸s G6 x hýqpryywhýq ¼xvw v²³ur³¸³hyý÷ir¼¸s¼¸hq²ir³Zrrýpryy v¸s G6 x hýqpryyw Uur³¼hssvp qrý²v³' qxvwª ¸srhpu¼¸hq phýirqr³r¼·výrqi' ¸i²r¼½výt ³ur³¼hssvps¸¼ hy¸ýt ¦r¼v¸q¸s³v·rrth'rh¼Tvýpr³uv² ·r³u¸q·h' irv·¦¼hp³vphyh²¸phyyrq qrQ²v³'y¸¸x æ³hiyr phý ir iÃvyqi' òvýt³ur ³'¦rhýq ²Ã¼shpr puh¼hp³r¼v²³vp² ¸s h ¼¸hqUuv²³hiyrp¸ý³hvý²³urqxvwª ½hyÃr²Zuvpu h¼r ³urý÷ir¼¸s½ruvpyr²r`¦rp³rq ¦r¼Ãýv³¸s³v·r¸ýh¼¸hq ªir³Zrrýpryyv¸sG6 x hýqpryyw Uur ²rp¸ýq ²³r¦ ¸s ³ur @U7G69 ²pur·r v² qr³r¼·vývýt ³Z¸ ³u¼r²u¸yq ½hyÃr² ³G hýq ³Cýh·ry' ³ury¸Zhýq uvtu³u¼r²u¸yq²i' r`h·vývýt³urr`¦rp³rq vý³r¼pryy³¼hssvp ³xvwü½hyÃr² Uur²r³Z¸³u¼r²u¸yq² ¦h¼³v³v¸ý³ur²r³¸sr`¦rp³rq vý³r¼pryy³¼hssvp ½hyÃr² vý³¸³u¼rrph³rt¸¼vr²Dshý vý³r¼pryy³¼hssvp v²t¼rh³r¼³uhý³Cv³v²vý³r¼¦¼r³rqh² úuvtuú ²v·vyh¼y' ³¼hssvp½hyÃr² Ãýqr¼³ur³G h¼rvý³r¼¦¼r³rqh² úy¸Zúhýq h³yh²³³¼hssvp½hyÃr² ir³Zrrý ³ur ³u¼r²u¸yq² h¼r úý¸¼·hyú Uur²r ph³rt¸¼vr² h¼r òrq ³¸ qr³r¼·výr ³ur ·h`v·Ã·²vªr¸shýG6h²v³v²²u¸Zývý#ü Uur¸iwrp³v½r ¸s¸Ã¼@U7G69²pur·rv²³¸qr²vtý G6²²Ãpu ³uh³³urý÷ir¼ β· ¸svý³r¼G6·¸½r·rý³²¸s³ur·¸ivyr²v²·výv·v²rqUurr`¦rp³rq ½hyÃr ¸s β· phýir ¦¼rqvp³rqi'òvýt³ur¼¸hqqh³hh²s¸yy¸Z²) ηÿ
y
β ³ = ∑∑ ∑ ³xvw
!ü
x = v = w∈T ÿ′
Zur¼r β³v²³urr`¦rp³rq½hyÃr¸sβ· ³xvwv²³¼hssvpir³Zrrýpryyv¸sG6xhýqpryyw Txÿv² ³ur²r³¸spryy²Zuvpuh¼rý¸³·r·ir¼²¸s G6 x ηx v²³urý÷ir¼ ¸s pryy²výG6xhýq yv²³urý÷ir¼¸sG6² Dý ¸¼qr¼ ³¸·výv·vªr β³Zr·h`v·vªr³urr`¦rp³rq vý³¼hG6h¼rh³¼hssvp ϕ³ qr³r¼ ·výrqs¼¸·) y
ηÿ
ϕ³ = ∑∑ ∑ ³xvw
"ü
x = v = w∈T ÿ
Zur¼r Txv² ³ur²r³¸spryy² Zuvpuh¼r·r·ir¼²¸sG6 x Pü¸iwrp³v½r sÃýp³v¸ýv² ·h`v·v²r ϕ³ #ü ²Ãiwrp³³¸ η·h` + ! vs ³x·vQ ≥ ³ C ≤ ηx ≤ ηx·h` ηx·h` = η·h` vs ³x·vQ < ³ C hýq³x·vQ ≥ ³ G s¸¼ x2y η − ! vs ³ < ³ x·vQ G ·h` Zur¼r ·h` v²³ur·h`v·Ã· ý÷ir¼¸spryy²výhý G6výZuvpu³ur²·hyyr²³vý³r¼pryy³¼hssvp v²ir³Zrrý³u¼r²u¸yq²½hyÃr²³Ghýq ³C x·h`v² ³ur·h`v·Ã·ý÷ir¼¸spryy²výG6 x xv²³urý÷ir¼ ¸spryy²výG6x ³x·vQv²³ur²·hyyr²³vý³r¼pryy³¼hssvpvýG6xD³v²qrsvýrqh² ³x·vQ2·výº³xvw±s¸¼vw∈Tx 6²v³ v²²u¸Zývý #ühý h G6·h'výpyÃqr³Z¸·¸¼r pryy²³uhý hý¸³ur¼ G6 vsv³² ²·hyyr²³vý³r¼pryy³¼hssvpv²úuvtuú¸½r¼³C ü hýq ³ur ²·hyyr²³vý³r¼pryy³¼hssvpvý³ur¸³ur¼
©#
ÿ 6²yÕuhxhýqA 7êyÃph
G6 v² úý¸¼·hyú Dý ³uv² ¦h¦r¼ výp¼r·rý³qrp¼r·rý³ ½hyÃr ³uh³ v² òrq ³Z¸ qr³r¼·výr ³ur·h`v·Ã· G6²vªrv²!Uur ¸¦³v·hy½hyÃrqr¦rýq²¸ý ³urpuh¼hp³r¼v²³vp²¸s³ur ·r³¼¸¦¸yv³hýh¼rh9r³r¼·výh³v¸ý¸s³uv²½hyÃrv²yrs³h²hsü³ur¼ ²³Ãq' "!Uur 6yt¸¼v³u· s¸¼ @U7G69 Dýb&dv³uh²irrý¦¼¸½rý³uh³³ur¸¦³v·hy U7G69 Ãýqr¼ ³ur p¸ýqv³v¸ý¸s ³ur ·h`v·v ²h³v¸ý¸svý³¼hG6 ³¼hssvpv²IQp¸·¦yr³rUur¼rs¸¼rZrqr²vtýrq³urh¦¦¼¸`v·h³v¸ý hyt¸¼v³u·výAvt! 8¼rh³rhyv²³¸sýrvtui¸¼výtpryy²s¸¼rhpupryy Uuryv²³¸sýrvtui¸¼výtpryy² ·Ã²³²h³v²s' ³v· ³v·Zur¼r³v· v²³ur³¼hssvps¼¸· pryyvs¸¼Zuvpu³uryv²³v² ¦¼r¦h¼rq³¸ v³²ýrvtui¸¼ ·vý³uryv²³ ! 8¼rh³rhý¸¼qr¼rqyv²³¸spryy²výZuvpu³v³v Zur¼r ³vv²³ur³¼hssvpir ³Zrrý³urpryyv vý³uryv²³hýq³ursv¼²³pryyvýv³²yv²³¸s ýrvtui¸¼výtpryy² " Dýv³vhyvªrhyy³urpryy²h²µIPUDI8GV9@9¶ # 9r³r¼·výr³ury¸Z³Gühýq uvtu³Cü³u¼r²u¸yq½hyÃr²i' r`h·výt³urvý³r¼pryy ³¼hssvp½hyÃr² $ Dýv³vhyvªrpü¼rý³ G6ý÷ir¼v % Sr¦rh³Ãý³vy³ur¼rv²ý¸pryy³uh³v²µIPUDI8GV9@9¶ % 8¸ý²³¼Ãp³h ýrZG6hv %! Dýv³vhyvªr³urý÷ir¼ yhv ¸spryy²s¸¼ hv yhv %" DýpyÃqr³uruvtur²³¸¼qr¼rqµIPUDI8GV9@9¶pryypyhv s¼¸· ³urpryyyv²³ %# Hh¼xpyhvh² µDI8GV9@9¶ %$ Dýv³vhyvªrsyhtq¸Qr úA6GT@ú %% Sr¦rh³Ãý³vy³ur¼r v²ý¸ ·¸¼rpryyZuvpuv²vý³urýrvtui¸¼²yv²³¸s¸ýr¸s ³urpryy²výpyÃqrqs¸¼ hvhýq µIPUDI8GV9@9¶¸¼ q¸Qr2úUSV@ú %% AvýqhýrZµIPUDI8GV9@9¶pryypQrZZv³uuvtur²³³·pQrZs¼¸· ³urýrvtui¸¼yv²³²¸s³urpryy²výpyÃqrqs¸¼hv η·h` + ! vs ³· pQrZ ≥ ³ C vs ³· pQrZ < ³ C hýq ³· pQrZ ≥ ³ G %%! ηx·h` = η·h` η − ! vs ³ · pQrZ < ³ G ·h`
ÿ
Ds yhv x·h`³urýq¸Qr úUSV@ú ry²r %%# DýpyÃqr pQrZ vý³¸ hv Hh¼xpQrZ h²µDI8GV9@9¶ %%#! Dýp¼r·rý³yhv yhv yhv %%#" Ds yhv x·h`³urýq¸Qr úUSV@ú %& Dýp¼r·rý³v v v %%" %%#
Avt! Uurh¦¦¼¸`v·h³v¸ýhyt¸¼v³u·
V²výt³urhyt¸¼v³u·výAvt!Zr sv¼²³p¼rh³rhyv²³¸sýrvtui¸Ã¼²s¸¼rhpupryyUur ýrvtui¸Ã¼pryy²h¼r¸¼qr¼rq hpp¸¼qvýt ³¸³ur ¦¼rqvp³rq ³¼hssvpir³Zrrý³urýrvtui¸Ã¼ pryy²hýq ³uryv²³urhq² vr³urpryys¸¼Zuvpu³urýrvtui¸Ã¼ yv²³v²p¼rh³rq UurýZr ¸¼qr¼³ur²r yv²³²hpp¸¼qvýt ³¸³ur ³¼hssvpir³Zrrý³uryv²³urhq² hýq³ursv¼²³ýrvtui¸Ã¼²
An Efficient Location Area Design Scheme
841
vý³uryv²³² 6yy¸s³urpryy²h¼r výv³vhyvªrqh²úIPUDI8GV9@9ú UurýZrqr³r¼·výr ³Z¸³u¼r²u¸yq½hyÃr²³Ghýq ³C i' r`h·vývýt³ur vý³r¼pryy³¼hssvp½hyÃr²6y¸¸¦ v³r¼h³r² Ãý³vyhyy¸s³urpryy²h¼r výpyÃqrq Dýrhpuv³r¼h³v¸ý³ursv¼²³þIPUDI8GV9@9´pryy s¼¸· ³ur pryy yv²³ v² ²ryrp³rq hýq h ýrZ G6 v² p¼rh³rq Zv³u ³uv² pryy Uurý hý výýr¼ y¸¸¦svýq²³ur ýrvtui¸Ã¼Zv³u³uruvtur²³¦¼rqvp³rq³¼hssvps¼¸·³urýrvtui¸Ã¼ yv²³²¸s ³ur pryy² ³uh³ h¼r výpyÃqrq s¸¼ ³ur pü¼rý³ G6 Uur ¦¼rqvp³rq vý³r¼pryy ³¼hssvp ¸s ³uv² ýrvtui¸Ã¼ v²Ã²rq ³¸phypÃyh³r³ur·h`v·Ã·ý÷ir¼ ¸spryy²s¸¼³urpü¼rý³G6Ds ³ur ý÷ir¼¸spryy²v²ý¸³¼rhpurq ³uv²·h`v·Ã·ý÷ir¼³ur ýrZpryyv²výpyÃqrqvý³¸³ur pü¼rý³ G6 Uur výýr¼ y¸¸¦ ³r¼·výh³r² Zurý ³ur¼r v² ý¸ ýrvtui¸Ã¼ ³uh³ phý ir vý pyÃqrq¸¼³ur·h`v·Ã·ý÷ir¼¸spryy²v²¼rhpurqs¸¼³urpü¼rý³G6 "" @`¦r¼v·rÿ³hySr²Ãy³² Xrp¸·¦h¼r³ur ¦r¼s¸¼·hýpr ¸s @U7G69 ³¸ U7G69 hýq Q7G69i' òvýt ³Z¸ ²r³²¸sr`¦r¼v·rý³²A¸¼rh²' p¸·¦h¼v²¸ý Zr òrq ³ur²h·rr`¦r¼v·rý³²hýqqh³h²r³² h²výb&dDý³ursv¼²³²r³¸sr`¦r¼v·rý³²¼¸hq qh³hh¼r òrq³¸p¸·¦Ã³rvý³r¼pryy³¼hssvp výh¼rhy·r³¼¸¦¸yv³hýh¼rhUur²rqh³hh¼r¸i³hvýrq s¼¸·h²phýýrquh¼qp¸¦' ·h¦hýq phýirq¸Zýy¸hqrqs¼¸·³urZri²v³r)u³³¦)òr¼²rprth³rpurqÃr¼qhy·r³¼¸¦¸y³`³ βm
(7%/$' 7%/$' 3%/$'
·h`
Avt" I÷ir¼¸s vý³r¼y¸ph³v¸ý h¼rh·¸½r·rý³² s¸¼ 8T2"$ ·
Uurý Zr ¼Ãý @U7G69 U7G69hýq Q7G69 ³rpuýv¹Ãr²³¸ qr²vtý³urG6² Dý ³ur²rp¸ýq²r³²¸sr`¦r¼v·rý³² Zròrq·¸ivyv³' qh³hhi¸Ã³$%¼rhy ¦r¸¦yr yv½výtvý ³ur ²h·r ·r³¼¸¦¸yv³hý h¼rh Uur' uh½r qvssr¼rý³ ¦¼¸sr²²v¸ý² hýq qvssr¼rý³ ·¸ivyv³' puh¼hp³r¼v²³vp²Xr ³¼hpr³ur·¸½r·rý³²¸s³urvý³r¼½vrZrq ¦r¸¦yri' òvýt ³urv¼·¸ ivyv³' qh³h¸ý³urG6²qr²vtýrqi' @U7G69 U7G69 hýqQ7G69 hýqp¸Ãý³³ur ý÷ir¼¸svý³r¼G6·¸½r·rý³²³urý p¸·¦h¼r³ur¼r²Ãy³²
©#!
ÿ 6²yÕuhxhýqA 7êyÃph
Dý Avt " ³ur rssrp³ ¸s ·h` ¸ý ³ur ý÷ir¼ ¸s vý³r¼y¸ph³v¸ý h¼rh ·¸½r·rý³² β· v² ²u¸ZýCr¼r³ur qvh·r³r¼¸spryy²pryy²vªrû8Tüv² "$·Uury¸Zhýq uvtu ³u¼r²u ¸yq² s¸¼ r`¦rp³rq vý³r¼pryy ³¼hssvp ½hyÃr² h¼r qr³r¼·výrq h² ³G 2 $ hýq ³C 2 ! ¼r ²¦rp³v½ry' 6²v³²u¸Zývý Avt " ³ur ýrZ @U7G69²pur·r¦r¼s¸¼·²ir³³r¼³uhý U7 G69 hýq Q7G69 irphòr v³ trýr¼h³r² yr²² ³ur vý³r¼G6 ·¸½r·rý³² Ds ·h`v·Ã· ý÷ir¼ ¸s pryy² ·h` v² t¼rh³r¼ ³uhý @U7G69 hýq U7G69 ²pur·r² ¦¼¸qÃpr ²h·r¼r²Ãy³²Uur ¼rh²¸ýv²³uh³³urqvtv³vªrq·h¦Ã²rqvý ³urr`¦r¼v·rý³phýp¸ý³hvý yv·v³rqý÷ir¼¸spryy²vs³ur8T2"$·Dýp¼r·rý³výt³ur ²vªr¸s³urG6²qrp¼rh²r² ³ur ý÷ir¼ ¸s ¦¸²²viyr G6² vý ³ur p¸½r¼htr h¼rh Uur¼rs¸¼r ³Z¸ ³¼hssvpih²rq G69 ²pur·r²¦¼¸qÃpr²h·r¼r²Ãy³²
βm (7%/$' 7%/$' 3%/$'
Avt# I÷ir¼¸s vý³r¼y¸ph³v¸ý h¼rh·¸½r·rý³² s¸¼
CS (m)
·h`2
Dý Avt#³ur rssrp³¸s8ryyqvh·r³r¼8T ¸ý ³urý÷ir¼¸svý³r¼y¸ph³v¸ýh¼rh·¸½r ·rý³² β· v²²u¸Zý Cr¼r³ur ·h` ½hyÃrv²Uur ³u¼r²u¸yq² ³G hýq³ur³Ch¼rqr³r¼ ·výrqs¸¼rhpuqvssr¼rý³8T ½hyÃr²r¦h¼h³ry' Uur v·¦¼¸½r·rý³vý β· ·hqri' @U7 G69 ¸½r¼ U7G69 hýq Q7G69 phý ir hy²¸ ²rrý vý Avt # Uur ýrZ ²pur·r ²³vyy trýr¼h³r²yr²²vý³r¼G6·¸½r·rý³²³uhý¸³ur¼²pur·r²Zurý³urpryy²vªr8Tpuhýtr²
#
8¸ÿpyòv¸ÿ
Uuv²¦h¦r¼uh²p¸ý²vqr¼rqhýqhqq¼r²²rq ³ur v²²Ãr¦h¼³v³v¸ývýt³ur p¸½r¼htr h¼r vý³¸h ý÷ir¼¸sG6²rhpup¸ý³hvývýt h t¼¸Ã¦ ¸spryy²²Ãpu ³uh³³ur¼rtv²³¼h³v¸ý³¼hssvpv² ·výv·v²rqXrqr½ry¸¦rq hýqhýhy'ªrqh ýrZ²³h³vp³¼hssvpih²rqy¸ph³v¸ýh¼rh qr²vtý ²pur·rýh·rq@U7G69rýuhýprq U7G69ü Zuvpu v²hý rýuhýprq½r¼²v¸ý¸s ¦¼r ½v¸Ã²y' ¦Ãiyv²urq ³¼hssvpih²rq y¸ph³v¸ý h¼rh qr²vtý U7G69ü ³rpuýv¹Ãr Uur ýrZ
An Efficient Location Area Design Scheme
843
@U7G69²pur·rqvssr¼² ²vtývsvphý³y' s¼¸· ³urU7G69²pur·ri' r`¦yvpv³y' ³hxvýt vý³¸hpp¸Ãý³³ur ¦¼¸¦r¼ ý÷ir¼¸s pryy²vý hýG6UurG6² Zv³u uvtur¼ vý³r¼pryy ³¼hs svph¼rhyy¸Zrq³¸ výpyÃqr·¸¼r pryy²³uhý³ur G6²Zur¼r ³urvý³r¼pryy³¼hssvpv²y¸Z 6²h ¼r²Ãy³³urvý³¼hG6·¸½r·rý³²¸s·¸ivyr²h¼r výp¼rh²rq vý¸¼qr¼³¸ qrp¼r·rý³ ³urvý³r¼G6·¸½r·rý³²Zuvpup¼rh³r¼rtv²³¼h³v¸ý³¼hssvp Xr uh½r ²u¸Zý vý ³uv²¦h¦r¼³uh³³ur @U7G69²pur·r¦r¼s¸¼·²ir³³r¼³uhý ³ur U7G69hýqQ7G69 ²pur·r² vý³ur²rý²r ³uh³v³phý ¦h¼³v³v¸ýh ·r³¼¸¦¸yv³hýh¼rh vý³¸ G6² ²Ãpu ³uh³ ³ur ¼rtv²³¼h³v¸ý ³¼hssvp trýr¼h³rq i' ·¸ivyr² v² ¼rqÃprq Pü ·r³u¸qphýirh¦¦yvrqrh²vy' ³¸ pryyÃyh¼ Zv¼ryr²² ýr³Z¸¼x² ³¸¼rqÃpr ³ur y¸ph³v¸ýæ qh³rp¸²³²Zv³u¸Ã³²¦rpvhy·¸ivyrqr½vpr²h¼rýrrqrq
Srsr¼rÿpr² 1. X¸ýtWXT GrÃýt W8H) G¸ph³v¸ý Hhýhtr·rý³ s¸¼ Ir`³Brýr¼h³v¸ý Qr¼²¸ýhy 8¸··Ãývph³v¸ý²Ir³Z¸¼x² D@@@ Ir³Z¸¼xHhthªvýr W¸y"©!ü ©û!# 2. TÃi¼h³h S a¸·h'h 6Á) G¸ph³v¸ý Hhýhtr·rý³ vý H¸ivyr 8¸·¦Ã³výt Q¼¸prrqvýt² ¸s ³ur68TD@@@ Dý³r¼ýh³v¸ýhy 8¸ýsr¼rýpr ¸ý 8¸·¦Ã³r¼ T'²³r·² hýq 6¦¦yvph³v¸ý² 7rv¼Ã³ !ü !©&û!©( 3. 6¼hÃw¸GQqr Hh¼phES7)Qhtvýt hýq G¸ph³v¸ý V¦qh³r 6yt¸¼v³u·² s¸¼ 8ryyÃyh¼ T'² ³r·² D@@@ U¼hý²hp³v¸ý² PýWruvpÃyh¼Urpuý¸y¸t'W¸y#(!ü%%û%# 4. CX CZhýt CX 8uhýt HA U²rýt 88) 6 9v¼rp³v¸ý 7h²rq G¸ph³v¸ý V¦qh³r Tpur·r Zv³u h GvýrQhtvýt T³¼h³rt' s¸¼ Q8T Ir³Z¸¼x² D@@@ 8¸·· Gr³³r¼² W¸y # !ü #(û$ 5. Th¼h'qh¼ 8V Fryy'P@S¸²r8)Pýr9v·rý²v¸ýhyG¸ph³v¸ý 6¼rh9r²vtýD@@@ U¼hý²hp³v¸ý²PýWruvpÃyh¼ Urpuý¸y¸t'W¸y #( !ü%!%û%"! 6. Xhýt X 6x'vyqvª DA T³ir¼ BG) 6ý P¦³v·hy Qhtvýt Tpur·r s¸¼ Hvýv·vªvýt Tvt ýhyvýt 8¸²³²Výqr¼ 9ryh'7¸Ãýq² D@@@8¸··Ãývph³v¸ý² Gr³³r¼² W¸y $ !ü#"û#$ 7. 8h'v¼pv@6x'vyqvªDA)P¦³v·hy G¸ph³v¸ý 6¼rh 9r²vtý ³¸ Hvýv·vªrSrtv²³¼h³v¸ý Tvtýhy yvýtU¼hssvp vý Xv¼ryr²²T'²³r·² D@@@U¼hý²hp³v¸ý²PýH¸ivyr 8¸·¦Ã³výtW¸y ! !"ü &%û©$
A Simple Pipelined Scheduling for Input-Queued Switches Sang-Ho Lee1 and Dong-Ryeol Shin2 1
2
Samsung Electronics, System LSI Division, Korea,
[email protected], http://www.samsung.com Sungkyunkwan University, Network Research Laboratory, Korea
[email protected], http://nova.skku.ac.kr
Abstract. Input-queued switch is useful for high bandwidth switches and routers because of lower complexity and fewer circuits than outputqueued switch. However, it suffers HOL-blocking, which limits the throughput to 58%. To overcome HOL-blocking problem, many inputqueued switches are controlled by sophisticated scheduling algorithms at centralized schedulers which restrict the design of the switch architecture. In this paper, we propose a simple scheduler called Pipelined and Prioritized Round Robin(PPRR) which is intrinsically distributed by each input-port. An iterative prioritized round robin scheduling algorithm in a pipelined fashion is provided. The proposed algorithm has less complexity and yet comparable performance with respect to other algorithms such as iSLIP and RPA. The effectiveness of PPRR is demonstrated with simulations under uniform and bursty traffic conditions.
1
Introduction
Input-queued switch with a FIFO queue in each input-port suffers the HOLblocking problem which limits the throughput to 58% [1]. A lot of scheduling algorithms have been suggested to improve the throughput. In order to overcome the performance reduction due to HOL-blocking, separate queues are required at each input-port for different output-ports called Virtual Output Queue(VOQ). The recently appeared algorithm is based on bipartite matching problem, known either as a maximum weighted matching (MWM) or a maximum sized matching (MSM), depending on the value of weight. MWM find the matching that provides maximum total weight. MSM selects the match containing the maximum number of pairs. It is a special case of the MWM with same weight. One of MWM policies , Reservation with Preemption Acknowledgement (RPA)[2] belongs to MWM, whereas PIM and iSLIP are typical examples of MSM. PIM is a three-phase scheduling algorithm which uses parallelism, randomness, and iteration to achieve higher throughput. The switch is very efficient and its throughput can reach 98% in four iterations. Some variations of PIM such as iSLIP appeared recently [3, 4], which leads to the optimal and fair usage of the A. Yazici and C. S ¸ ener (Eds.): ISCIS 2003, LNCS 2869, pp. 844–851, 2003. c Springer-Verlag Berlin Heidelberg 2003
A Simple Pipelined Scheduling for Input-Queued Switches
845
switch bandwidth under a variety of traffic patterns. Unfortunately, these policies are not simple to implement, requiring a computational complexity, O(N 2 logN ) [5]. RPA is based on reservation rounds where input ports indicate their urgency, possibly overwriting less urgent request by other input ports, and an acknowledgement round to select actual transfer toward desired output ports. RPA is able to deal with multiple traffic classes and its complexity is O(N 2 ) [5]. In this paper, we propose a different approach to three-steps matching used in PIM and iSLIP or reservation scheme, RPA. The arbitration in this paper is solved by applying pipelined and prioritized round robin scheme rather than bipartite method, maximum matching, reservation-acknowledge method. Simulation results show that the proposed approach achieves good performances in simple architecture, less complexity, which compares favorably with other algorithms such as iSLIP and RPA. This paper is organized as follows.Section 2 gives a basic principle of the proposed scheduling method. Section 3 shows the discussion and comparison of results obtained through the simulations. A simple hardware implementations is covered in section 4. Section 5 provides concluding remarks.
2 2.1
Proposed Scheduling Algorithm Motivation
We propose an architecture called Pipelined and Prioritized Round Robin (PPRR) Scheduler which is an input-port controller that includes a cell scheduler to avoid HOL-blocking and to reduce delay or jitter. The algorithm in PPRR is simple. Like other VOQ-based scheduling methods, PPRR maintains multiple virtual queues at the input port for each output. In an iSLIP if two or more cells routed to the same output in a time slot interval, three steps are used to resolve the conflict among inputs using input and output control units and round robin pointer. A recently proposed method, RPA has a similar idea but different scheduling scheme compared to PPRR. Actually PPRR does not require a bipartite matching among input and output ports. PRRR instead composes of two parts. The first one is to maintain multiple virtual queues, which is the same as iSLIP and RPA. The second one is sequential comparison like RPA but the scheduling is completed with one reservation round to reduce complexity, whose scheduling algorithm is based on Prioritized Round Robin(PRR). One of important issues here is the implementation complexity. Most proposed schedulers are implemented as centralized modules. The size of switch is restricted by centralized scheduler. PPRR scheduling is distributed at each input port. This guarantees the simplicity of implementation. 2.2
Prioritized Round Robin
The incoming cells wait in the buffer of its own designated output port, i.e., virtual output queue. The input scheduler selects a virtual queue and transmits a next cell in the header of selected virtual queue into crossbar switch. The
846
S.-H. Lee and D.-R. Shin
scheduling algorithm is based on Prioritized Round Robin(PRR). To apply PRR, each VQ has a priority counter depending on the queue length or waiting time and counting up as many as its priority weight at each time slot. The priority depends on the queue length, queue occupancy time, or some weights. To avoid HOL-blocking, each scheduler should not select the same destined VQ among input ports. The schedulers in the PRR scheduling do not schedule simultaneously but sequentially among input ports. This algorithm is implemented with N schedulers interconnecting the N input ports which determine the input-output port matches in a round robin and sequential manner. One input scheduler compares N VOQs to select one with highest priority. The next input scheduler then compares N-1 VOQs with which have been previously unselected VOQ index, and continues this procedure until the final input scheduler reaches. It is of centralized type and takes time to calculate, which may be significant computational burden at high speed switch. 2.3
Pipelined PRR
We propose a Pipelined PRR (PPRR) to reduce scheduling time and complexity. Each input port compares its own VOQs and selects a VOQ index of higher priority and forwards VOQ index with lower value to the next input port. In order to share temporary scheduling information between neighboring input schedulers, each input port is connected with its neighbor in a ring structure. The input-port has two operation modes, called master mode and slave mode. At any scheduling time, only one input can be a master and the others become slave. To guarantee the fairness among input ports, the master is determined by round robin. Fig.1 shows interconnection and its operation.
Send unselected VOQ Index
Master
Slave
Slave
Master Crossbar
Crossbar
Slave
Slave
Slave
Slave
(a) Slot Time
n
(b) Slot Time
n+1
Fig. 1. PPRR’s interconnection and it’s operation
Let i and k be index of input port and output port, respectively, and p(i, k) a priority of VOQ corresponding to output port k and input port i. The master
A Simple Pipelined Scheduling for Input-Queued Switches
847
and slave operations are shown in the following pseudo code. The output port index is denoted by VOQ index. Pseudo code of Master Operation /*** Initialize Register, PR ***/ PR.PRIORITY = 0; PR.INDEX = 0; /** Start Schedule **/ for(k=1; k = PR.PRIORITY) { VQO @ [4] >@ >@
S. E. Hansen and E. T. Atkins, “Automated System Monitoring and Notification With Swatch”, Proceedings of the Seventh Systems Administration Conference (LISA VII), 1993. S. Axelsson, U. Lindqvist, U. Gustafson and E. Jonsson, “An Approach to UNIX Security Logging”, Proc. 21st National Information Systems Security Conference, SS ± %6FKQHLHUDQG-.HOVH\³6HFXUH$XGLW/RJVWR6XSSRUW&RPSXWHU)RUHQVLFV´,Q$&0 7UDQVDFWLRQVRQ,QIRUPDWLRQDQG6\VWHP6HFXULW\9RO1R0D\SS± 86$ M. Bishop, “A Standard Audit Trail Format”, In Proc. 18th National Information Systems Security Conference", 1995, pp. 136–145. &- $QWRQHOOL 0 8QG\ DQG 3 +RQH\PDQ ³7KH 3DFNHW 9DXOW 6HFXUH 6WRUDJH RI 1HWZRUN 'DWD´ ,Q 3URFHHGLQJV RI WKH 86(1,; :RUNVKRS RQ ,QWUXVLRQ 'HWHFWLRQ DQG 1HWZRUN0RQLWRULQJ$SULO86$ -RKQ .HOVH\ DQG %UXFH 6FKQHLHU ³0LQLPL]LQJ %DQGZLGWK IRU 5HPRWH $FFHVV WR &U\SWRJUDSKLFDOO\ 3URWHFWHG $XGLW /RJV´ LQ 3URFHHGLQJV RI 5HFHQW $GYDQFHV LQ ,QWUXVLRQ'HWHFWLRQ
On Fair Bandwidth Sharing with RED Diego Teijeiro-Ruiz, Jos´e-Carlos L´opez-Ardao, Manuel Fern´andez-Veiga, C´andido L´opez-Garc´ıa, Andr´es Su´arez-Gonzalez, Ra´ul-Fernando Rodr´ıguez-Rubio, and Pablo Argibay-Losada Telematics Engineering Department, University of Vigo (Spain) ETSET Telecomunicacion, Campus Universitario 36200 Vigo (Spain) {diego, jardao, mveiga, candido, asuarez, rrubio, pargibay}@det.uvigo.es
Abstract. One weakness of the RED algorithm typical of routers in the current Internet is that it allows unfair bandwidth sharing when a mixture of traffic types shares a link. This unfairness is caused by the fact that, at any given time, RED imposes the same loss probability on all flows, regardless of their bandwidths. In this paper, we propose Random Rate-Control RED (RRC-RED), a modified version of RED. RRC-RED uses per-active-flow accounting to enforce on each flow a loss rate than depends on the flow’s own rate. This papers shows than RRC-RED provides better protection than RED and its variants to solve that problems (like FRED, CHOKe or RED-PD), and, moreover, it is easier to implement and lighter in complexity.
1
Introduction
The spectacular growth of Internet during the decade has made congestion control one of the most active research fields. Congestion is mainly caused by the lack or under-use of three essential resources in network nodes: buffer space, bandwidth and processing capacity. To deal with congestion, TCP uses an end-to-end mechanism where the sending TCP entity adjust its sending rate according to the information obtained monitoring the connection itself. In Drop-Tail routers - a queueing discipline where the last arrival will be the first candidate to be sacrificed - packet discarding due to buffer overflow represents an implicit congestion notification. The main problem with this kind of queue management policy is finally the inexistence of any specific step to avoid congestion, since packet discarding only occurs when congestion is already a reality. A simple solution to this problem could be Random Drop [1], although it must be combined with some other mechanism to early notify whichever detected congestion risk in order to take the right preventive actions. This has led to what is called AQM (Active Queue Management). AQM techniques are based on routers detecting the incipient congestion early, and advertising end-hosts to slow their sending rate before packets begin to be dropped. AQM results are smaller mean queue sizes and, consequently, smaller end-to-end delays. The most popular AQM techniques are ECN (Explicit Congestion Notification) [2] and RED (Random Early Detection) [3]. We shall focus in the latter because its A. Yazici and C. S¸ener (Eds.): ISCIS 2003, LNCS 2869, pp. 892–899, 2003. c Springer-Verlag Berlin Heidelberg 2003
On Fair Bandwidth Sharing with RED
893
widespread use today, and the advantage of needing no modifications at all in the hostend comunication software, where (TCP) sources will continue working the standard way after detecting the loss of a packet. RED’s main goal is to discard packets from a particular flow proportionally to its bandwidth use. This should imply a fair share of the remaining bandwidth [3]. However, situations revealing some kind of RED misbehaviour have been identified. In [5], for example, authors highlight the fact that when several TCP connections have different buffer occupancy in a RED router, dropping packets proportionally does not always guarantee fair bandwidth share.This problem may be aggravated by the presence of UDP flows, because this kind of traffic ignores congestion control signals. Also, in [4] the authors point out that RED increases the packet dropping probability, so if a packet is dropped when a connection is in the slow-start process, it will suffer specially to recover its transmission window. Over the last few years some solutions have been proposed to mitigate this undesirable RED behaviour. In [5] a modification called Flow RED (FRED) adds some per flow state information to the basic mechanism to prioritize those flows with smaller queue occupancy, and to control those users that do not react to congestion signals. Nevertheless, FRED’s improvements are not predictable and depend strongly on the flows’ arrival times. Another interesting solution is the CHOKe algorithm [6], where every time a new packet arrives, another one from the queue is chosen randomly, and if both belong to the same flow, both are discarded. This algorithm is based on the assumption that the more active a flow is the more the packets it will have in the queue. However, CHOKe does not work properly when the number of flows is high, or in the presence of UDP flows. Finally, Floyd’s solution - RED with Preferential Dropping [7] - combines some of the previous ideas. In RED-PD, RED’s drop history is used to identify the flows getting most of the bandwith, because, due to the probabilistic nature of the drops, discarded packets can be considered random samples of the incoming traffic [8]. With this information, and a RTT estimation, the flows using more bandwidth than allowed are identified and penalized until they come back into the right range. In this paper we propose a new algorithm based on RED, called Random Control Rate - RED (RRC-RED), which obtains a reasonable bandwidth fair share, even better than some of the previous solutions, but with the advantage of being a very simple solution, with less complexity and adding little per flow state information at routers. The paper is organized as follows: In section 2, RED’s basics and its main drawbacks are reviewed. In Section 3 we describe the proposed algorithm and some implementation details. Performance evaluation is treated in section 4. And, finally, Section 5 summarizes the work presented and some conclusion remarks.
2
RED Algorithm
The RED algorithm tries to early detect the congestion, and to do so it uses a particular average queue size estimation (avg) and some threshold values. To calculate avg it uses a low-pass filter based on an EWMA estimator (Exponential Weighted Moving Average) over the instantaneous size of the queue (instqueue ) every time a packet arrives at the node:
894
D. Teijeiro-Ruiz et al.
avg = avg ∗ wq + instqueue ∗ (1 − wq )
(1)
where wq is a configuration parameter. If the average queue size is below a lower threshold (minth ), the newly arrived packet is never discarded. When the average queue size exceeds minth , but is smaller than an upper threshold (maxth ), RED drops the packet with some probability Pdrop . This probability grows proportionally to the average queue size (avg), according to: Pdrop = (avg − minth )/(maxth − minth ) ∗ Pmax
(2)
When the average queue size exceeds the upper limit, maxth , the incoming packet will be discarded with probability 1. This way, RED can detect an incipient and persistent congestion, besides allowing routers to detect transient bursts. Finally, random packet drop in RED avoids the undesirable effect of global sinchronization. 2.1
RED Drawbacks
As we have pointed out earlier, RED’s main goal is to implement a packet dropping strategy that arises from the bandwidth consumption proportionality among the different packet flows sharing the buffer. This is achieved by discarding any incoming packet with the same probability (assuminging that the average queue size does not change significantly). This way, connections with greater rates will also have greater percentange of discarded packets. However, from a connection point of view, the instantaneous packet dropping rate during a short period of time is independent of its bandwidth use. When there is risk of congestion, that is, the average queue size exceeds minth , the dropping probability has a non zero value for every connection, contributing to a non-fair share of the link bandwidth. In [5] some possible reasons are identified: – The fact that every connection see the same loss rate implies that even the connection that receives much less bandwidth than it should will suffer from losses. Moreover, such unfairly treated connection will eventually also have to reduce its transmission window to half the current size if one of its packets is discarded. – The acceptance of a new packet implies an increase in the dropping probability of future packets in the other connections, even when those consume less bandwidth. This causes undesirable temporary drops that break the proportionality rule above mentioned, even between identical flows. – If a particular connection does not obey to congestion signals, this may lead to a collective high drop rate, and represents a clear proof of the RED incapability to offer fair bandwidth share to adaptive connections in presence of aggresive users, even when congestion is not severe.
3
RRC-RED Algorithm (RED with Random Rate-Control)
Our algorithm (RED with Random Rate-Control) represents another attempt to improve bandwidth fairness when RED is used. Against other solutions, the main new feature of
On Fair Bandwidth Sharing with RED
895
RRC-RED lies in its simple flow rate estimation method. Then, at the time to discard a packet, estimated flow rates will determine among a number of randomly chosen flows from which one we must eliminate the packet. Flows rate estimations will only be calculated at the time to (early) discard a packet, and is computed dividing the number of bytes sent by a particular flow by the time such flow has been sending data. Thus, the main difference between RED and RRC-RED concerns to the packet that must be discarded, not to the decision or opportunity to take the dropping action. While RED discards the incoming packet, RRC-RED compares the average rate of the flow this packet belongs to with those of N-1 other flows randomly chosen among the ones which have a packet enqueued at that time, discarding the last arrived packet from the flow with the largest flow rate at that moment. Choosing the last arrived packet is well suited for TCP, because doing so avoids unnnecesary retransmissions, as well as for UDP, because the flows sending at a higher rate will have their packets discarded with more probability as they will trigger the early drop algorithm more often. Moreover, if the flow the eliminated packet belongs to appears more than once in the randomly selected N set, the algorithm discards, if possible, many packets as times it was selected. The algorithm accumulates the number of bytes sent by each flow, and the rates estimations only occurs when a packet has to be discarded by the RED algorithm. Thus the complexity that the algorithm adds to RED is very small. Moreover, these calculations, as well as RED does, can be done in parallel with packet delivery without having a significant influence in the processing capacity of the router. After many simulations, we have observed that increasing N does not improve significantly the results obtained, moreover if we think about maintainning in a low level the computational load of the algorithm In fact, only by taking two flows randomly, RED is improved with a minor increase of the complexity. In section 4 we present the results obtained with our algorithm in different situations, compared to RED and to different solutions proposed in the literature.
4
RRC-RED Evaluation
In this section we present the results obtained with the proposed algorithm in several scenarios, and compare its performance with the ones achieved by RED and the other mentioned solutions. We have used the Network Simulator to draw the results. To
SOURCE
3
8ms
SOURCE
5ms
4ms SOURCE
4
2
SOURCE
1
100Gbps
1ms
ROUTER
5
2ms 45Mbps
6
SINK
Fig. 1. Scene 1
896
D. Teijeiro-Ruiz et al.
45000
45000 flow 0 flow 1 flow 2 flow 3
40000 35000
flow 0 flow 1 flow 2 flow 3
40000 35000 Throughput (kbps)
Throughput (kbps)
30000 25000 20000 15000 10000
30000 25000 20000 15000 10000
5000
5000
0
0 0
2
4
6
8
Time (sec)
(a) RRCRED
10
0
2
4
6
8
10
Time (sec)
(b) RED
Fig. 2. Comparaci´on entre RRC-RED y RED
make the comparisons easy, we will evaluate RRC-RED performance using the same network scenario described in [3], and represented in Figure 1: Four TCP conections, with different RTTs and start times, are considered. Windows’ sizes are limited by the smaller link capacity and by RTT, and they will have, respectively, 33, 67, 112 and 78 packets. Simulations have been made with a fixed packet length of 1000 bytes. And inside the router, the buffer size is 1000 packets, minth = 30, maxth = 80, wq = 0.002, and maxp = 1/10. As in [3], the start times of the flows are separated 20 miliseconds, and we assume that all sources have always data to send. In Figure 2 we can see a comparison between RED and RRC-RED.As this figure shows, RRC-RED rapidly reaches a situation where the four flows have very similar throughputs. On the contrary, RED is unable to reduce the rate of the first flow (flow 0) once this has accomodated its window size: in RED, the first flow arriving at the router uses all the resources, so it fixes its window size to take the most of that capacity; nevertheless, the rest of the flows, which will appear every 20 ms, will see a much smaller bandwidth, because in that moment flow 0 is monopolizing the link. With this situation the random drop mechanism begins to work, having a bigger impact on the flows still at the slow-start phase, since they still are adjusting their window size. It is obvious that when this point is reached, as shown in figure 2, it is difficult to reduce the most active flow rate, and to increase the rate of the less active ones, so the rate differences among all the flows are significant. Our algorithm tries to avoid, as much as possible, this undesirable effect, so fairness in bandwidth share can be reached soon. This speed of convergence does not discriminate between longlived or short-lived connections, offering a fair share whatever the situation is, something that RED can not do satisfactorily. In the next comparison we include the rest of the proposals (RED-PD, FRED and CHOKe), but use 8 TCP flows in a longer simulation with the same scenenario (we duplicate the original 4 sources). In figure 3 we show the last seconds of each simulation, where we can observe again the right performance of RRC-RED. If we choose any flow and follow its evolution over time, we can observe these behaviors: when its throughput is high, it is probable that sooner or later it will be the victim of the dropping algorithm, so its rate will begin to descend slowly. After reducing enough its rate, this flow will not be the goal of the discards for a period of time, so its rate will increase again while other flows see their
On Fair Bandwidth Sharing with RED
9000
9000
7000
3000
5000
7000
4000 3000
6000 5000
7000
4000 3000
6000 5000
2000
2000
1000
1000
1000
1000
0
0 40
42
44
46
48
40
50
42
Time (sec)
44
46
48
42
Time (sec)
(a) RRCRED
44
46
48
(c) PD
4000 3000
0 42
Time (sec)
(b) RED
5000
1000 40
50
6000
2000
0 40
50
7000
3000
2000
flow 0 flow 1 flow 2 flow 3 flow 4 flow 5 flow 6 flow 7
8000
4000
2000
0
9000 flow 0 flow 1 flow 2 flow 3 flow 4 flow 5 flow 6 flow 7
8000
Throughput (kbps)
4000
6000
9000 flow 0 flow 1 flow 2 flow 3 flow 4 flow 5 flow 6 flow 7
8000
Throughput (kbps)
5000
7000 Throughput (kbps)
Throughput (kbps)
6000
9000 flow 0 flow 1 flow 2 flow 3 flow 4 flow 5 flow 6 flow 7
8000
Throughput (kbps)
flow 0 flow 1 flow 2 flow 3 flow 4 flow 5 flow 6 flow 7
8000
897
44
46
48
40
50
42
Time (sec)
RED-
44
46
48
50
Time (sec)
(d) FRED
(e) CHOKe
Fig. 3. Comparison between RRC-RED, RED, RED-PD, FRED y CHOKe
9000
9000
7000
3000
6000 5000
7000
4000 3000
6000 5000
7000
4000 3000
6000 5000
2000
2000
1000
1000
1000
1000
0 40
42
44
46
48
Time (sec)
(a) RRCRED
50
0 40
42
44
46
48
50
42
44
46
48
Time (sec)
(c) PD
RED-
50
6000 5000 4000 3000 2000 1000
0 40
Time (sec)
(b) RED
7000
3000
2000
flow 0 flow 1 flow 2 flow 3 flow 4 flow 5 flow 6 flow 7
8000
4000
2000
0
9000 flow 0 flow 1 flow 2 flow 3 flow 4 flow 5 flow 6 flow 7
8000
Throughput (kbps)
4000
9000 flow 0 flow 1 flow 2 flow 3 flow 4 flow 5 flow 6 flow 7
8000
Throughput (kbps)
5000
7000 Throughput (kbps)
Throughput (kbps)
6000
9000 flow 0 flow 1 flow 2 flow 3 flow 4 flow 5 flow 6 flow 7
8000
Throughput (kbps)
flow 0 flow 1 flow 2 flow 3 flow 4 flow 5 flow 6 flow 7
8000
0 40
42
44
46
48
Time (sec)
(d) FRED
50
40
42
44
46
48
50
Time (sec)
(e) CHOKe
Fig. 4. Comparison with identical flows
rates reduced. This way, each flow’s rate will be fluctuating slightly with time, depending on which state it is. The rate control of each flow in RRC-RED manintains very close the throughput values, something that the rest of the tested solutions could not achieve. In the graphics of figure 3, we can see how RED’s algorithm experiments the same problems as in the previous example. And something similar happens to RED-PD and FRED. FRED hardly improves RED’s performance, while RED-PD, although it is slightly better, penalizes flow 0 from the beginning with a very tiny window. The only solution that works reasonably well is CHOKe, although the fairness is clearly worse than with RRC-RED and, besides, it discards an important number of packets. In another interesting experiment, we play with a number n of identical sources (8 in this example). We use the same topology as in the previous cases, but now the starting times are separated 0.5 secs, all the windows have the same size (100 packets), and the delay introduced by the link is always 2 ms. Looking at figure 4, we can observe the right evolution of RRC-RED over time. While disciplines such as RED or RED-PD reach a steady state, this does not occur to RRC-RED, where the most active flows are forced to reduce their rate, allowing the less active flows to get more bandwidth. Here we can see the failure of CHOKe when it tries to make rates look similar. We also see FRED’s nice performance. Up to now, we have only compared TCP flows. Let us look at the results obtained in presence of UDP flows, which are unresponsive to any congestion control mechanism. In this example,
898
D. Teijeiro-Ruiz et al.
18000
18000
14000
6000
10000 8000
14000
6000
12000 10000 8000
14000
6000
12000 10000 8000
4000
4000
4000
2000
2000
2000
2000
0
0 40
42
44
46
48
Time (sec)
50
40
42
44
46
48
50
42
(b) RED
44
46
48
50
(c) PD
12000 10000 8000 6000 4000 2000 0
40
Time (sec)
Time (sec)
(a) RRCRED
14000
0 40
flow 0 flow 1 flow 2 flow 3 flow 4 flow 5 flow 6 flow 7 flow 7
16000
6000
4000
0
18000 flow 0 flow 1 flow 2 flow 3 flow 4 flow 5 flow 6 flow 7 flow 7
16000
Throughput (kbps)
8000
12000
18000 flow 0 flow 1 flow 2 flow 3 flow 4 flow 5 flow 6 flow 7 flow 8
16000
Throughput (kbps)
10000
14000 Throughput (kbps)
Throughput (kbps)
12000
18000 flow 0 flow 1 flow 2 flow 3 flow 4 flow 5 flow 6 flow 7 flow 8
16000
Throughput (kbps)
flow 0 flow 1 flow 2 flow 3 flow 4 flow 5 flow 6 flow 7 flow 7
16000
RED-
42
44
46
48
50
40
Time (sec)
(d) FRED
42
44
46
48
50
Time (sec)
(e) CHOKe
Fig. 5. Comparison with TCP and UDP flows
we use the same topology of the previous examples but we add an UDP source which generates packets of 1000 bytes. As shown in figure 5, none of the proposed solutions can avoid the UDP flow from getting more bandwidth than the other flows. As we have seen in the previous examples, only RRC-RED and CHOKe get a fair share among the TCP flows while they reduce the UDP flow rate as much as possible. Again, the main difference between RRC-RED and CHOKe is that the latter discards much more packets from the TCP flows, firing more retransmissions. To conclude with the simulation study, we present in table 1 the results obtained when we have a large number of flows competing for the bandwidth, because as we explained in the introduction, some of the algorithms proved here did not work fine with a large number of flows. We use again the mean deviation of the flows’ throughput to show the performance of each algorithm. Finnally, we will comment some details
Table 1. Large # of flows Algorithm RRC-RED RED RED-PD FRED CHOKE M.Deviation (103 ) 4.14 33.24 40.07 18.35 6.69
about the complexity of the studied algorithms. RRC-RED adds only one arithmetic operation, with two variables per flow, every time the algorithm is executed, and one two arithmetic operations and one comparison in an early drop. FRED, unlike RRC-RED, mantains two variables per flow, making 8 comparisons and 6 arithmetic operations more than RED, each time a packet is enqueued and dequeued. RED-PD uses a list where the last drops from each flow in some period of time are stored, which can contribute to a large increase of the information stored in a router when the number of flows and/or the number of drops is high. Lastly, CHOKe is the lighter solution because it does not need per flow information and it only does one more comparison than RED.
On Fair Bandwidth Sharing with RED
5
899
Conclusions
The problem of congestion control in the current Internet has been broadly studied during the last years. One of the most successful solutions in this area has been RED. However, in several situations, RED has revealed itself as a mechanism incapable of guaranteing bandwidth’s fair share. There have been a number of proposals to attenuate this malfunction, but none of them revolve this satisfactorily. In this paper we present a new algorithm, RRC-RED, a mechanism that uses a rate estimator of the incoming flows to detect those consuming more bandwidth, and, thus, to discard preferably packets from those flows, in order to control the bandwidth they receive and to guarantee a fair share among all the present flows. By means of simulation, we have tested the proposed algorithm, and it has been compared with other solutions proposed in literature.
References 1. A. Mankin, K. K. Ramakrishnam., editors for the IETF Performance and Congestion Control Working Group, Gateway Congestion Control Survey, RFC 1254 (1991) 2. Floyd., S. TCP and Explicit Congestion Notification. ACM Computer Communication Review (1994), vol. 24, 10–23 3. S. Floyd y V. Jacobson. Random Early Detection Gateways for Congestion Avoidance. IEEE/ACM Transactions on Networking (1993), vol. 1, 397–413 4. M. May, C. Diot, B. Lyles. Reasons not to Deploy RED, en Proceedings of the IEEE/IFIP IWQoS (1999) 5. D. Lin and R. Morris, Dynamics of Random Early Detection, en Proceeding of ACM Sigcomm ACM, (1997) 127–137 6. R. Pan, B. Prabhakar, and K. Psounis. CHOKe,A StatelessActive Queue Management Scheme for Approximating Fair Bandwidth Allocation. In IEEE INFOCOM (2000) 7. Ratul Mahajan and Sally Floyd, RED with Preferential Dropping (RED-PD).E n Proc. ACM 9th International Conference on Network Protocols (ICNP) (2001) 8. S. Floyd and K. Fall. Router mechanisms to support end-to-end congestion control. LBL Technical Report (1997)
PES: A System for Parallelized Fitness Evaluation of Evolutionary Methods Onur Soysal, Erkin Bah¸ceci, and Erol S ¸ ahin Kovan research group Department of Computer Engineering Middle East Technical University 06531 Ankara, Turkey {soysal, erkinb, erol}@ceng.metu.edu.tr http://kovan.ceng.metu.edu.tr
Abstract. The paper reports the development of a software platform, named PES (Parallelized Evolution System), that parallelizes the fitness evaluations of evolutionary methods over multiple computers connected via a network. The platform creates an infrastructure that allows the dispatching of fitness evaluations onto a group of computers, running both Windows or Linux operating systems, parallelizing the evolutionary process. PES is based on the PVM (Parallel Virtual Machine) library and consists of two components; (1) a server component, named PES-Server, that executes the basic evolutionary method, the management of the communication with the client computers, and (2) a client component, named PES-Client, that executes programs to evaluate a single individual and return the fitness back to the server. Performance of PES is tested for the problem of evolving behaviors for a swarm of mobile robots simulated as physics-based models, and the speed-up characteristics are analyzed.
1
Introduction
Evolutionary Robotics[6] is a new approach for designing autonomous robots. It considers autonomous robots as artificial organisms that can be selectively “bred” based on their fitness for the tasks that they are being designed for. An active research track in Evolutionary Robotics is the development of behaviors for simulated robot systems. In these studies, the goal is to evolve controllers that use the sensory information of the robot to control robot’s actuators such that the robot accomplishes a desired task. Initially a population of genotypes that encode the controllers is given. Then the robots are simulated under the control of the controller specified by the genotype and its fitness is evaluated. The fittest controllers are then allowed to reproduce as defined by a set of genetic operators, and the process is repeated. Evolving behaviors in simulated robot systems require large amounts of computational power, mainly due to the evaluation of different individuals. Therefore the computational requirements of A. Yazici and C. S ¸ ener (Eds.): ISCIS 2003, LNCS 2869, pp. 900–907, 2003. c Springer-Verlag Berlin Heidelberg 2003
PES: A System for Parallelized Fitness Evaluation
901
using evolutionary methods tend to be proportional to the computational requirements of evaluating a single individual. This creates a major bottleneck for evolutionary methods. Recently there have been projects[3,5] that aim to create platforms that can use the idle processing power of computer systems that are connected over a network. SETI@home[3] is a scientific experiment that uses Internet-connected computers in the Search for Extraterrestrial Intelligence (SETI). The downloaded program runs as a background task or a screen-saver getting chunks of radio telescope data from a server and performing a computation intensive processing of the data to seek signs of artificial signals and reporting the results back to the server. This platform made it possible for the SETI project to utilize a total computing power that equals or surpasses supercomputers. This paper reports the development of a platform, named PES (Parallellized Evolution System), that parallelizes an evolutionary method on a group of computers connected via a network. In the next section, we describe PES and its specifications. Section 2 describes the implementation of PES, explaining server and client components of the system. Section 3 analyzes the speed-ups obtained using PES for the problem of evolving behaviors for a swarm of simulated mobile robots. Section 4 summarizes the results and discusses the shortcomings of the system and future development directions of the system.
2
PES
PES1 is a platform to parallelize evolutionary methods on a group of computers connected via a network. It separates the fitness evaluation of genotypes from other tasks (such as selection and reproduction) and distributes these evaluations onto a group of computers to be processed in parallel. PES consists of two components; (1) a server component, named PES-Server, that executes the evolutionary method, the management of the communication with the client computers, and (2) a client component, named PES-Client, that executes programs to evaluate a single individual and return the fitness back to the server. Figure 1 shows the structure of a PES system. PES provides the user with an easy interface that relieves him from dealing with the communication between server and client processes. PES-Client is developed for both Windows and Linux, enabling the PES system to harvest computational power from computers running either of these operating systems. An easy-to-use framework for implementing evolutionary methods, and the interoperability of the system distinguishes PES from other systems available and makes it a valuable tool for evolutionary methods with large computational requirements. PES uses PVM (Parallel Virtual Machine)[4]2 , a widely utilized message passing library in distributed and parallel programming studies, for communi1 2
Documentation and source code of PES is freely available at http://kovan.ceng.metu.edu.tr/software/PES/. Available at http://www.csm.ornl.gov/pvm/pvm home.html.
902
O. Soysal, E. Bah¸ceci, and E. S ¸ ahin PES−S (Linux) Evolutionary Algorithm Create individuals Dispatch individuals Collect fitness values
PES−C (Linux)
PES−C (Win)
Evaluate individual Send the fitness back
Evaluate individual Send the fitness back
PES−C (Win)
Evaluate individual Send the fitness back
Fig. 1. Sketch of PES system is shown. The PES-Server runs on a Linux machine and handles the management of the evolutionary method. It executes the selection and reproduction of the individuals (genotypes) which are then dispatched to a group of PES-Clients (running both Windows and Linux systems). The individuals are then evaluated by the clients and their fitnesses are sent back to the server.
cation between the server and the clients. We have also considered MPI[2] as an alternative to PVM. MPI is a newer standard that is being developed by multiprocessor machine manufacturers and is more efficient. However PVM is more suitable for our purposes since (1) it is available in source code as free software and is ported to many computer systems ranging from laptops to CRAY supercomputers, (2) it is inter-operable, i.e. different architectures running PVM can be mixed in a single application, (3) it does not assume a static architecture of processors and is robust against failures of individual processors. PES wraps and improves PVM functionality. It implements a time-out mechanism to detect processes that have crashed or have entered an infinite loop. It provides ping, data and result message facilities. Ping messages are used to check the state of client processes. Data messages are used to send task information to client processes and result packages are used to carry fitness information from clients. Now we will describe the PES-Server and PES-Clients. 2.1
PES-Server
PES-Server provides a generic structure to implement evolutionary methods. This structure is based on Goldberg’s basic Genetic Algorithm[1] and is designed to be easily modified and used by programmers. The structure assumes that fitness values are calculated externally. In its minimal form, it supports tournament selection, multi-point cross-over and multi-point mutation operators. PES-Server maintains a list of potential clients (computers with PES-Client installed), as specified by their IP numbers. Using this list, the server executes an evolutionary method and dispatches the fitness evaluations of the individuals
PES: A System for Parallelized Fitness Evaluation
903
to the available clients. The assignment passes the location of the executable to be run on the client as well as the parameters that represent that particular individual and the initial conditions for the evaluation. Then it waits for the clients to complete the fitness evaluation and get the computed fitness values back. PES-Server contains fault detection and recovery facilities. Using the ping facility the server can detect clients that have crashed and assign the uncompleted tasks to other clients. In its current implementation, the server waits for the results of fitness evaluations from all the individuals in a generation before dispatching the individuals from the next generation. 2.2
PES-Client
PES-Client acts as a wrapper to handle the communication of the clients with the server. It fetches and runs a given executable (to evaluate a given individual) with a given set of parameters. It returns the fitness evaluations, and other data back to the server. Client processes contain a loop that accepts, executes and sends results of tasks. Client processes reply to ping signals sent by the PES-Server to check their status. Crashed processes are detected through this mechanism. PES-Clients are developed for single processor PC platforms running Windows and Linux operating systems. Note that to use clients with both operating systems the fitness evaluating program should be compilable on both systems. In its current implementation, these clients have the fitness evaluation component embedded within them to simplify the communication. Yet, once the clients are installed, the fitness evaluation component of the system can be updated using scp (secure copy) utility from the server.
3
Experimental Results
The PES platform is developed as part of our work within the Swarm-bots project3 [7,8], for evolving behaviors for a simulated swarm of mobile robots. We have conducted experiments to evolve behaviors for clustering a swarm of robots and analyzed the speed-up and efficiency of the PES system in this task. The swarm of robots and their environment are modeled using ODE (Open Dynamics Engine), a physics-based simulation library, Figure 2(a). The parameters of this simulation and parameters of the controller (network weights) are passed to the PES-Client from the PES-Server. The simulator constructs the world and runs it, by solving differential dynamic equations. Movements of the robots are determined by their controller as specified by the genotype. This controller uses the sensors of the robots and moves the robots for 2000 time steps. Then a fitness value is computed based on a measure of clustering achieved. This fitness value is then returned to PES-Server, Figure 2(b). 3
More information can be found at http://www.swarm-bots.org.
904
O. Soysal, E. Bah¸ceci, and E. S ¸ ahin
PES-S Fitness value
Network weights
Descriptions World file Robot file
Simulator Kernel Network weights
Initialize Sensory input
Simulation Loop
Controller
Motor speed Sound emission
(a)
Fitness value
PES-C
(b)
Fig. 2. (a) A snapshot of the environment being simulated: Three mobile robots, distributed in an arena are enclosed by walls, can be seen. The rods emanating out of the robot bodies visualize the proximity sensing capabilities of the robots. (b) Architecture of PES-Client being used.
3.1
The Experimental Set-up
We have installed PES-Clients to 12 PC’s of a student laboratory at the Computer Engineering Department of Middle East Technical University, Ankara, Turkey. During the experiments, these computers were being used by other users and each of them had different workloads that varied in time. The population size is set as 48, requiring that 48 fitness evaluations take place during each generation. 30 generations were evolved. Figure 3 plots the load of the 12 processors in time, during the evaluation of five generations. PES-Server waits for fitness evaluations of all the individuals in a generation before the selection and reproduction of the individuals of the next generation. In the plot, the vertical lines separate the generations. Within each generation, 48 fitness evaluations are calculated, which are visible as dark diamonds or dark horizontally stretched hexagons. It can be seen that the fitness evaluation time varies between different cases. There are two major causes of this. First, each processor has a different and varying workload depending on the other users of that computer. Second, the physics-based simulation of the swarm of robots slows down dramatically as robots collide with each other and the walls in the environment. As a result, the computation required by different simulations vary drastically. In order to analyze the speed-ups achieved through the use of the PES system and its efficiency, we have repeated the evolution experiment using 1, 2, 3, 6 and 12 processors. The data obtained is used to compute the following two measures: Sp =
Time required for single machine Time required for p machines Speed-up with p processors Ep = p
PES: A System for Parallelized Fitness Evaluation
905
Processor Load Graph 12
Processors
10 8 6 4 2 0
0
10
20
30 40 50 time(seconds)
60
70
Fig. 3. The load of the 12 processors during 5 generations of evolution. See text for discussion.
The results are plotted in Figure 4(a,b). Ideally Sp should be linear to number of processors and Ep should be 1. The deviance is a result of the requirement that all individuals need to be evaluated before moving on to the next generation. As a consequence of this, after the dispatching of the last individual in a generation, all-but-one of the processors has to wait for the last processor to finish its evaluation. This causes a decrease in the speed-up and efficiency plots. Note that, apart from the number of processors, these measures also depend on two other factors: (1) the ratio of total number of fitness evaluations in a generation to the number of processors, (2) the average duration of fitness evaluations and their variance.
(a)
(b)
Fig. 4. (a) Speed-up is plotted against number of processors. (b) Efficiency is plotted against number of processors.
906
O. Soysal, E. Bah¸ceci, and E. S ¸ ahin
In Section 2, we described a ping mechanism that is implemented to check whether a processor has crashed or not. This mechanism is crucial since we envision PES to harvest idle processing powers of existing computer systems and cannot make assumptions about the reliability of the clients. Figure 5 shows the ping mechanism at work during an evolution where we had 20 fitness evaluations in each generation that run on 4 processors. Similar to the plot in Figure 3, this plot shows the loads of the processors. The numbers in the hexagons are labels that shows the index of the individual being evaluated. The continuous vertical bars separate the generations. the dotted vertical lines that are drawn at 15, 30, 45, and 60 seconds mark the pings that check the status of the processors. In this experiment, processor 2 crashed while it was evaluating individual 1. PES-Server detected this crash at the first ping (at time 15) and assigned the evaluation of individual 1 to processor 1, and removed processor 2 from its client list.
Processor Load Graph 5
4
3
3
9
4
8
12
15 18
11 14 17
19
Processors
2
6
2
5
8
11 14 16
1
4
7
10
13 17 19
2
1
5
3
8
6
12
15
10 13 16
19
18
2
5 8 9
1
3
6 11
0
4
7 10
Timeout 2
1
Task Restart
1
0
0
0
5
7
10
10
13
16
1
20
0 3
6
9
30
12 15 18
40 time(seconds)
0
4 7
50
9
11 14 17
60
70
Fig. 5. Load of each processor during a run in which a processor fails. See text for discussion.
4
Conclusion
In this paper, we have reported a new parallel processing platform for running evolutionary methods. Unlike existing parallel processing platforms, PES provides a generic framework within which evolutionary methods can be easily implemented. PES provides an interface which relieves its user from the details of communication and the status of the clients. PES adds better fault tolerance mechanisms to PVM allowing crashed processes to be detected. The user only needs to program a separate client program with a few constraints and should
PES: A System for Parallelized Fitness Evaluation
907
customize the server program to meet its requirements while leaving the rest to PES. The analysis showed that although evolutionary methods lend themselves easily to parallelization, non-incremental selection and reproduction (that is the selection and reproduction occurring only after all the individuals are evaluated) is a major source of wasting processing time. The use of incremental selection and reproduction methods would eliminate this problem. Acknowledgments. This work was partially funded by the SWARM-BOTS project, a Future and Emerging Technologies project (IST-FET) of the European Community, under grant IST-2000-31010. The information provided is the sole responsibility of the authors and does not reflect the Community’s opinion. The Community is not responsible for any use that might be made of data appearing in this publication. This work is also funded by Middle East Technical University within the “Sanal Robot Kolonisi” project under grant BAP-2003-07-2-00-01.
References 1. Goldberg D. E., Genetic Algorithms in Search, Optimization, and Machine Learning. Reading, MA: Addison-Wesley, 1989. 2. Hempel, R., The MPI Standard for Message Passing. High–Performance Computing and Networking, International Conference and Exhibition, Proceedings, Volume II: Networking and Tools. Ed. Gentzsch, Wolfgang and Harms, Uwe. 247–252,1994.SV, LNCS vol797, 1994. 3. Korpela E., Werthimer D., Anderson D., Cobb J., Lebofsky M., SETI@home: An Experiment in Public-Resource Computing Communications of the ACM, Vol. 45 No. 11, pp. 56–61, November, 2002. 4. Kowalik J. (ed), PVM: Parallel Virtual Machine: A Users’ Guide and Tutorial for Networked Parallel Computing. MIT Press Scientific and Engineering Computation, 1994. 5. Mukherjee S., Mustafi J., Chaudhuri A., Grid Computing: The Future of Distributed Computing for High Performance Scientific and Business Applications Lecture Notes in Computer Science, Vol. 2571, pp 339–342, 2002. 6. Nolfi S., Floreano D., Evolutionary Robotics: The Biology, Intelligence, and Technology of Self-Organizing Machines, MIT Press/Bradford Books, 2000. 7. S ¸ ahin E., Labella T.H., Trianni V., Deneubourg J.-L., Rasse P., Floreano D., Gambardella L.M., Mondada F., Nolfi S., Dorigo M., SWARM-BOT: Pattern Formation in a Swarm of Self-Assembling Mobile Robots. In A. El Kamel, K. Mellouli, and P. Borne, editors, Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, Hammamet, Tunisia, October 6–9, 2002. Piscataway, NJ: IEEE Press. 8. Trianni V., Labella Th.H., Gross R., Sahin E., Rasse Ph., Deneubourg J.-L. and Dorigo M. Evolving Aggregation in a Robotics Swarm. Accepted to the European Conference on Artificial Life (ECAL’03), 2003. 9. Vinschen C., Faylor C., Delorie D. J., Humblet P., Noer G., Cygwin User’s Guide.
Design and Evaluation of a Cache Coherence Adapter for the SMP Nodes Interconnected via Xcent-Net§ 1
1
Sangman Moh , Jae-Hong Shim1, Yang-Dong Lee , 2 Jeong-A Lee , and Beom-Joon Cho2 1School of Internet Eng.,
[email protected] 2School of Computer Eng., Chosun Univ. 375 Seoseok-dong, Dong-gu, Gwangju, 501-759 Korea
Abstract. This paper presents the design and evaluation of a cache coherence adapter for the cache-coherent non-uniform memory access multiprocessor system in which symmetric multiprocessor (SMP) nodes are interconnected via Xcent-Net. The SMP node is a 4-way symmetric multiprocessor subsystem based on the Intel Xeon processors, and Xcent-Net is a dual, adaptive-routed, virtual cut-through multistage interconnection network composed of hierarchical crossbar routers. The cache coherence adapter contains a directory and a remote access cache to support the directory-based cache coherence protocol customized for the SMP nodes on Xcent-Net. For any cache design, cache size and cache line size are crucial design parameters in terms of performance and implementation, and thus they are extensively evaluated. According to the simulation results, it is shown that a 128-Mbyte remote access cache with 64byte lines is the best choice, where the average data access latency is 9.4 µs and the effective bandwidth is 6.8 Mbytes/s per node.
1 Introduction Scalable multiprocessor systems are suitable for general-purpose multiuser applications. A non-uniform memory access (NUMA) multiprocessor is a shared memory system in which the access time varies with the location of memory words. As a variation of NUMA architecture, a cache-coherent non-uniform memory access (CCNUMA) model is defined with distributed shared memory and cache coherence mechanism [1-2]. A major concern of CC-NUMA is the cache-coherent memory access transparent to all the processors [3-5]. Latency tolerance for remote memory access is also a major limitation of CC-NUMA systems [6]. A CC-NUMA system with symmetric multiprocessor (SMP) nodes is a hierarchically structured multiprocessor machine, which has two layers of architecture: SMP and CC-NUMA models. In such a system, since SMP nodes are interconnected via an interconnection network, the access time to remote memories is longer than that to the local memory. In most SMP-based CC-NUMA multiprocessor systems, a direc¶
This study was supported in part by research funds from Chosun University, 2003.
$@>@>@>@>@>@ 7KHH[DFWSURWRFROVGLIIHU LQ WKH VFRSH RI WKH UHSOLFDWLRQ RU LQ XVH RI FRPPXQLFDWLRQ SDUDGLJPV DV ZULWH EURDGFDVW>@ +RZHYHUWKHZULWHXSGDWHVWUDWHJ\WXUQVWREHYHU\PHVVDJHLQWHQVLYH
7KLVZRUNKDVEHHQSDUWLDOO\VXSSRUWHGE\WKH6WDWH&RPPLWWHHIRU6FLHQWLILF5HVHDUFKJUDQWQR7&
$@>@ 3&*LVSUREDEO\WKHPRVWUHOD[HGFRQVLVWHQF\PRGHODOORZLQJ IXOOWUDQVSDUHQF\RIWKHFRKHUHQFHRSHUDWLRQIURPWKHSRLQWRIYLHZRIWKHDSSOLFDWLRQ SURFHVVDQGVWLOOXVHIXOIRUVROYLQJVHYHUDOFODVVHVRIFRPSXWDWLRQDOSUREOHPVVHH>@ DVDQH[DPSOH 7RWKHEHVWRIRXUNQRZOHGJHWKLVLVWKHILUVWLQYDOLGDWLRQEDVHGFR KHUHQFHSURWRFROIRUWKHSURFHVVRUFRQVLVWHQF\PRGHODQGWKHUHIRUHLWIXOO\DOORZVWR H[SORLW WKH HIILFLHQF\ RI WKH PRGHO 0RUHRYHU WKH FRPPXQLFDWLRQ RYHUKHDG RI WKH SURSRVHG SURWRFRO LV DGGLWLRQDOO\ UHGXFHG DV QR H[SOLFLW LQYDOLGDWLRQ PHVVDJHV DUH WUDQVPLWWHG2QO\ORFDOLQYDOLGDWLRQRSHUDWLRQVDUHSHUIRUPHGWRPDLQWDLQWKHFRQVLV WHQF\UHO\LQJRQWKHLQIRUPDWLRQUHFHLYHGZLWKXSGDWHPHVVDJHV7KHFRQFHSWRIORFDO LQYDOLGDWLRQZDVIRUPHUO\SURSRVHGLQ>@WRWUDFNFDXVDOSUHFHGHQFHRIPHPRU\DF FHVVKRZHYHUWKHLQYDOLGDWLRQFRQGLWLRQSURSRVHGWKHUHGRHVQRWDSSO\WRSURFHVVRU FRQVLVWHQF\PRGHO)XUWKHUPRUHWKHSURWRFRORI>@UHVXOWVLQLQYDOLGDWLQJPRUHRE MHFWVWKDQVWULFWO\QHFHVVDU\LQFRQWUDU\WRWKHRQHSURSRVHGKHUH 7KLV SDSHU LV RUJDQL]HG DV IROORZV ,Q 6HFWLRQ ZH GHILQH WKH V\VWHP PRGHO 6HFWLRQGHWDLOVWKHQHZFRKHUHQFHSURWRFRO7KHSURWRFROLVSURYHQFRUUHFWLQ6HF WLRQ6RPHFRQFOXGLQJUHPDUNVDUHJLYHQLQ6HFWLRQ
2 %DVLF'HILQLWLRQV 2.1 6\VWHP0RGHO 7KH'60V\VWHPLVDGLVWULEXWHGV\VWHPFRPSRVHGRIDILQLWHVHW3RIVHTXHQWLDOSUR FHVVHV333QWKDWFDQDFFHVVDILQLWHVHW2RIVKDUHGREMHFWV(DFKVKDUHGREMHFW FRQVLVWVRILWVFXUUHQWVWDWHREMHFWYDOXH DQGREMHFWPHWKRGVZKLFKUHDGDQGPRGLI\ WKHREMHFWVWDWH6KDUHGREMHFW[LVLGHQWLILHGE\LWVV\VWHPZLGHXQLTXHLGHQWLILHUGH QRWHG[LG7KHFXUUHQWYDOXHRI[LVGHQRWHG[YDOXH,QWKLVSDSHUZHFRQVLGHURQO\ UHDGZULWHREMHFWVLH ZH GLVWLQJXLVK WZR RSHUDWLRQV RQ VKDUHG REMHFWV UHDG DFFHVV 2
$QLQYDOLGDWLRQEDVHGSURWRFROKDVDOVREHHQLPSOHPHQWHGLQ'DVK>@EXWWKHFRQVLVWHQF\PRGHO XVHGWKHUHLVLQFRPSDUDEOHWR3&*
-%U]H]L VNLDQG06]\FKRZLDN
DQGZULWHDFFHVV:HGHQRWHE\UL[ YWKDWWKHUHDGRSHUDWLRQUHWXUQVYDOXHYRI[DQG E\ ZL[ Y WKDW WKH ZULWH RSHUDWLRQ VWRUHV YDOXH Y WR [ (DFK ZULWH DFFHVV UHVXOWV LQ D QHZREMHFWYDOXHRI[ 7KH UHSOLFDWLRQ PHFKDQLVP LV W\SLFDOO\ XVHG WR LQFUHDVH WKH HIILFLHQF\ RI WKH '60REMHFWDFFHVVE\DOORZLQJHDFKSURFHVVWRORFDOO\DFFHVVDUHSOLFDRIWKHREMHFW +RZHYHU FRQFXUUHQW DFFHVV WR GLIIHUHQW UHSOLFDV RI WKH VDPH VKDUHG REMHFW UHTXLUHV FRQVLVWHQF\PDQDJHPHQW7KHFRKHUHQFHSURWRFROV\QFKURQL]HV HDFK DFFHVV WR UHSOL FDVDFFRUGLQJO\WRWKH'60FRQVLVWHQF\FULWHULRQ7KLVSURWRFROSHUIRUPVDOOFRPPX QLFDWLRQ QHFHVVDU\ IRU WKH LQWHUSURFHVV V\QFKURQL]DWLRQ YLD PHVVDJHSDVVLQJ ,Q WKLV SDSHUZHDVVXPHWKHFRPPXQLFDWLRQWREHUHOLDEOHLHQRPHVVDJHLVHYHUORVWRUFRU UXSWHG 2.2 3URFHVVRU&RQVLVWHQF\ ,QWKLVZRUNZHLQYHVWLJDWHSURFHVVRUFRQVLVWHQF\3&* ILUVWSURSRVHGE\*RRGPDQ LQ>@DQGIXUWKHUIRUPDOL]HGE\$KDPDGHWDOLQ>@7KLVFRQVLVWHQF\PRGHOLVVXIIL FLHQWIRUVHYHUDOFODVVHVRIGLVWULEXWHGDOJRULWKPVEXWUHTXLUHVZHDNHUV\QFKURQL]DWLRQ WKDQDWRPLFVHTXHQWLDORUFDXVDOFRQVLVWHQF\WKXVDOORZLQJIRUPRUHFRQFXUUHQF\DQG HIILFLHQF\,WJXDUDQWHHVWKDWDOOSURFHVVHVDFFHVVLQJDVHWRIVKDUHGREMHFWVZLOOSHU FHLYH WKH VDPH RUGHU RI PRGLILFDWLRQV RI HDFK VLQJOH REMHFW DGGLWLRQDOO\ UHVSHFWLQJ WKH ORFDO RUGHU RI ZULWH RSHUDWLRQ SHUIRUPHG E\ HDFK SURFHVV 0RUH IRUPDOO\ 3&* H[HFXWLRQ RI DFFHVV RSHUDWLRQV UHVSHFWV 35$0 FRQVLVWHQF\ >@ DQG FDFKH FRQVLV WHQF\>@ DVGHILQHGEHORZ /HWORFDOKLVWRU\+LGHQRWHWKHVHWRIDOODFFHVVRSHUDWLRQVWRVKDUHGREMHFWVLV VXHG E\ 3L KLVWRU\ + GHQRWH WKH VHW RI DOO RSHUDWLRQV LVVXHG E\ WKH V\VWHP + +L ZULWHKLVWRU\+:GHQRWHWKHVHWRIDOOZULWHRSHUDWLRQV+:⊂+ DQGIL
*
L = Q
QDOO\+_[GHQRWHWKHVHWRIDOOZULWHRSHUDWLRQVSHUIRUPHGRQ[+_[⊂+ %\→LZH GHQRWHWKHORFDORUGHUUHODWLRQRIRSHUDWLRQVLVVXHGE\SURFHVV3L 'HILQLWLRQ 35$0FRQVLVWHQF\3LSHOLQHG5$0>@ ([HFXWLRQRIDFFHVVRSHUDWLRQVLV35$0FRQVLVWHQWLIIRUHDFK3LWKHUHH[LVWVD VHULDOL]DWLRQ6LRIWKHVHW+L∪+:VXFKWKDW
∀
R R ∈+ L ∪ +:
∃
M = Q
R→MR⇒R6LR
)ROORZLQJIURPWKDWGHILQLWLRQ35$0FRQVLVWHQF\SUHVHUYHVWKHORFDORUGHURI RSHUDWLRQVRIHDFKSURFHVV7KHRUGHURIRSHUDWLRQVLVVXHGE\GLIIHUHQWSURFHVVHV PD\EHDUELWUDU\ 'HILQLWLRQ &DFKHFRQVLVWHQF\FRKHUHQFH>@ ([HFXWLRQRIDFFHVVRSHUDWLRQVLVFDFKHFRQVLVWHQWLIIRUHDFK3LWKHUHH[LVWVDVH ULDOL]DWLRQ6LRIWKHVHW+L∪+:VDWLVI\LQJ
/RZ&RVW&RKHUHQFH3URWRFROIRU'606\VWHPVZLWK3URFHVVRU&RQVLVWHQF\
∀ Z Z ∈∀+: ∩+ _[ ∀
[ ∈2
L = Q
Z6LZ∨
∀
L = Q
Z6LZ
7KH DERYH GHILQLWLRQ UHTXLUHV FDFKH FRQVLVWHQF\ WR HQVXUH IRU HDFK SURFHVV WKH VDPHRUGHURIRSHUDWLRQVRQHYHU\VKDUHGREMHFW 'HILQLWLRQ 3URFHVVRUFRQVLVWHQF\3&*>@ ([HFXWLRQRIDFFHVVRSHUDWLRQVLVSURFHVVRUFRQVLVWHQWLIIRUHDFK3LWKHUHH[LVWV DVHULDOL]DWLRQ 6LRIWKHVHW+L∪+:WKDWSUHVHUYHVERWK35$0FRQVLVWHQF\DQG FDFKHFRQVLVWHQF\:HVKDOOFDOOWKLVVHULDOL]DWLRQWKHSURFHVVRURUGHU
3 7KH&RKHUHQFH3URWRFRO +HUHZHSURSRVHDEUDQGQHZFRKHUHQFHSURWRFROQDPHG3&*S7KH3&*SSURWRFRO XVHVWKHZULWHLQYDOLGDWHVWUDWHJ\DQGHQVXUHVWKDWDOOORFDOUHDGVUHIOHFWWKHSURFHVVRU RUGHU RI REMHFW PRGLILFDWLRQV E\ LQYDOLGDWLQJ DOO SRWHQWLDOO\ RXWGDWHG UHSOLFDV ,I DW DQ\WLPHSURFHVV3LXSGDWHVREMHFW[LWGHWHUPLQHVDOOORFDOO\VWRUHGUHSOLFDVRIRE MHFWVWKDWFRXOGKDYHSRVVLEO\EHHQPRGLILHGEHIRUH[DQGGHQLHVDQ\IXUWKHUDFFHVVWR WKHPLQYDOLGDWHV SUHYHQWLQJ3LIURPUHDGLQJLQFRQVLVWHQWYDOXHV$Q\DFFHVVUHTXHVW LVVXHG WR DQ LQYDOLGDWHG UHSOLFD RI [ UHVXOWV LQ IHWFKLQJ LWV XSWRGDWH YDOXH IURP D PDVWHUUHSOLFDRI[7KHSURFHVVKROGLQJDPDVWHUUHSOLFDRI[LVFDOOHG[¶VRZQHUDQG VKDOOEHGHQRWHGIRUVLPSOLFLW\[RZQHU$FFRUGLQJWRWKHKRPHEDVHGDSSURDFKWKH SURWRFRO VWDWLFDOO\ GLVWULEXWHV WKH RZQHUVKLS RI VKDUHG REMHFWV DPRQJ DOO SURFHVVHV VWDWLF RZQHUVKLS LH WKH [RZQHU LV DVVLJQHG D SULRUL DQG QHYHU FKDQJHV DOWHUQD WLYHO\ LW LV FDOOHG KRPHQRGH RI [ )URP QRZ RQ ZH VKDOO DVVXPH WKDW WKHUH LV D KRPHQRGHIRUHDFKREMHFW[DQGWKHLGHQWLW\RI[RZQHULVNQRZQ 7KH 3&*S SURWRFRO LQWURGXFHV GLIIHUHQW VWDWHV RI DQ REMHFW UHSOLFD ZULWDEOH GHQRWHG:5 UHDGRQO\52 DQGLQYDOLG,19 7KHFXUUHQWVWDWHRI[VKDOOEHGH QRWHG[VWDWH2QO\WKHPDVWHUUHSOLFDFDQKDYHWKH:5VWDWHDQGWKLVVWDWHHQDEOHVWKH RZQHUWRSHUIRUPLQVWDQWDQHRXVO\DQ\ZULWHRUUHDGDFFHVVWRWKDWUHSOLFD0HDQZKLOH HYHU\ SURFHVV LV DOORZHG WR LQVWDQWDQHRXVO\ UHDG WKH YDOXH RI DQ\ ORFDO 52 UHSOLFD +RZHYHUWKHZULWHDFFHVVWRDQ\52UHSOLFDUHTXLUHVDGGLWLRQDOFRKHUHQFHRSHUDWLRQV 0HDQZKLOHWKH,19VWDWHLQGLFDWHVWKDWWKHREMHFWLVQRWDYDLODEOHORFDOO\IRUDQ\DF FHVV7KXVDQDFFHVVWRWKH,19UHSOLFDUHTXLUHVWKHFRKHUHQFHSURWRFROWRIHWFKWKH YDOXHRIWKHPDVWHUUHSOLFDRIWKHDFFHVVHGREMHFWDQGXSGDWHWKHORFDOUHSOLFD (DFK SURFHVV 3L PDQDJHV D YHFWRU FORFN 97L $ YHFWRU FORFN LV D ZHOONQRZQ PHFKDQLVPXVHGWRWUDFNWKHGHSHQGHQF\RIHYHQWVLQGLVWULEXWHGV\VWHPV>@+HUHLW LV LQWHQGHG WR WUDFN WKH ORFDO SUHFHGHQFH RI ZULWH RSHUDWLRQV SHUIRUPHG E\ GLVWLQFW SURFHVVHV7KHLWKFRPSRQHQWRIWKH97LLQFUHDVHVZLWKHDFKZULWHDFFHVVSHUIRUPHG E\3L0RUHRYHUUHSOLFD[KDVEHHQDVVLJQHGDVFDODUWLPHVWDPSGHQRWHGE\[WV7KLV WLPHVWDPSLVXVHGWRUHIOHFWWKHPRGLILFDWLRQWLPHRIWKHORFDOUHSOLFDRI[DQGLVXS GDWHGRQHDFKZULWHRSHUDWLRQSHUIRUPHGWRWKDWUHSOLFD ,Q RUGHU WR NHHS WKH PDVWHU UHSOLFD XSWRGDWH WKH KRPHQRGH [RZQHU FROOHFWV PRGLILFDWLRQV SHUIRUPHG E\ RWKHUSURFHVVHVKRZHYHUVXFKZULWHVGRQRWLQFUHDVH[WV
-%U]H]L VNLDQG06]\FKRZLDN
7KHUHDUHIRXUEDVLFSURFHGXUHVLQYRNHGE\WKHSURWRFRODW3L LQFL97L ±LQFUHPHQWV97L>L@ ORFDOBLQYDOLGDWHL; ±LQYDOLGDWHVDOOUHSOLFDVZLWKLGHQWLILHUVEHORQJLQJWRWKH VHW;LHIRUDOO[[LG∈;SHUIRUPV[VWDWH:=,19DQG[WV:=97L>L@ RZQHGBE\LM :=^[LG[RZQHU M`±UHWXUQVWKHVHWRILGHQWLILHUVRIREMHFW UHSOLFDVORFDOO\DYDLODEOHDW3LRZQHGE\DJLYHQSURFHVV3M PRGLILHGBVLQFHLW :=^[LG[WV>W`±UHWXUQVWKHVHWRILGHQWLILHUVRIREMHFW UHSOLFDV DW 3L KDYLQJ PRGLILFDWLRQ WLPHVWDPSV QHZHU WKDQ JLYHQ W GXULQJ FRPPXQLFDWLRQZLWKVRPHSURFHVV3MWKLVSURFHGXUHLVXVHGWRGHWHFWREMHFWV SRVVLEO\RYHUZULWWHQRULQYDOLGDWHGUHFHQWO\ E\ 3L ± LQ IDFW 3M FDQ EH LQWHU HVWHGRQO\LQPRGLILFDWLRQV RI REMHFWV QRW RZQHG E\ LWVHOI VLQFH WKH PDVWHU UHSOLFDV DUH DOZD\V NHSW XSWRGDWH WKHUHIRUH 3L ZLOO DFWXDOO\ XVH DQRWKHU VHWRXWGDWHGLWM :=PRGLILHGBVLQFHLW ?RZQHGBE\LM $FWXDOO\HDFKSURFHVV3LPDQDJHVWZRYHFWRUFORFNV97L±ZKLFKMWKYDOXHUH IOHFWVWKHQXPEHURIPRGLILFDWLRQVSHUIRUPHGE\3MDQGNQRZQWR3L3LZLOOXVHWKLV YDOXHWRDVN3MIRUPRUHUHFHQWPRGLILFDWLRQV DQG97VHQWL±ZKLFKMWKYDOXHUHIOHFWV WKHQXPEHURIPRGLILFDWLRQVSHUIRUPHGE\3LZKLFKKDVDOUHDG\EHHQFRPPXQLFDWHG WR3M3LZLOOXVHWKLVYDOXHWRUHPHPEHUWKHODWHVWPRGLILFDWLRQ3MFDQEHDZDUHRI $FWLRQV RI WKH 3&*S SURWRFRO DUH IRUPDOO\ SUHVHQWHG LQ )LJ DQG FDQ EH GH VFULEHGE\WKHIROORZLQJUXOHV 5XOH ,I SURFHVV 3L ZDQWV WR DFFHVV REMHFW [ DYDLODEOH ORFDOO\ LH LQ DQ\ RWKHU VWDWH WKDQ ,19 IRU UHDGLQJ WKH UHDG RSHUDWLRQ LV SHUIRUPHG LQVWDQWDQH RXVO\ 5XOH ,I3LZDQWVWRDFFHVVD:5UHSOLFDRI[IRUZULWLQJWKHZULWHRSHUDWLRQLV SHUIRUPHGLQVWDQWDQHRXVO\ 5XOH ,I 3L ZLVKHV WR JDLQ D UHDG DFFHVV WR REMHFW [ XQDYDLODEOH ORFDOO\ [VWDWH ,19 LHDUHDGIDXOWRFFXUVWKHSURWRFROVHQGVUHTXHVWPHVVDJH 5(4L [LG 97L>[RZQHU@ WR [RZQHU VD\ 3N 7KH RZQHU VHQGV EDFN XS GDWHPHVVDJH5B83'[97N>N@,19VHW ZKHUH,19VHWFRQWDLQVLGHQWLILHUV RIREMHFWVPRGLILHGE\ 3NVLQFH97L>N@ZKLFKZDVUHFHLYHGLQ 5(4 DQG SRVVLEO\QRW\HWXSGDWHGE\3LH[FOXGLQJREMHFWVRZQHGE\3LDQGWKHUH TXHVWHG [DV[LVJRLQJWREHXSGDWHGRQUHFHLSWRIWKH5B83'PHVVDJH LH ,19VHW:=RXWGDWHGN97L>N@L ?^[LG` 2Q UHFHLSW RI 5B83' ILUVW ORFDOBLQYDOLGDWHL,19VHW LVSURFHVVHG$ORFDOUHSOLFDRI[LVFUHDWHGZLWK WKH YDOXH UHFHLYHG LQ 5B83' DQG VWDWH VHW WR 52 )LQDOO\ WKH UHTXHVWHG UHDGRSHUDWLRQLVSHUIRUPHG 5XOH ,I3LZLVKHVWRSHUIRUPDZULWHDFFHVVWRDUHSOLFDRI[LQHLWKHU,19RU52 VWDWH LH D ZULWHIDXOW RFFXUV WKH SURWRFRO LVVXHV D ZULWH UHTXHVW :B83'L [ 97L>L@ ,19VHW WR [RZQHU ± 3N 7KH ,19VHWFRQWDLQV LGHQWLILHUV RI REMHFWV PRGLILHG E\ 3L VLQFH WKH SUHYLRXV XSGDWH VHQW WR 3N LH ,19VHW:=RXWGDWHGL97VHQWL>N@N 2Q WKLV UHTXHVW 3N SHUIRUPV ORFDOBLQYDOLGDWHN,19VHW DQG VHQGV EDFN WR 3L DQ DFNQRZOHGJPHQW PHV VDJH $&.W ZKHUH W LV WKH FXUUHQW WLPHVWDPS RI WKH PDVWHU UHSOLFD RI [ 7KHQZL[ YLVSHUIRUPHGDQG3LµVFORFNLVDGYDQFHGZLWKLQF97L 7KHOR FDOUHSOLFDRI[DW3LLVWLPHVWDPSHGZLWKW
/RZ&RVW&RKHUHQFH3URWRFROIRU'606\VWHPVZLWK3URFHVVRU&RQVLVWHQF\
)LJ3&*SSURWRFRORSHUDWLRQV
)LJ SUHVHQWV D VDPSOH H[HFXWLRQ RI WKH FRKHUHQFH SURWRFRO RSHUDWLRQV LQ D V\VWHPRISURFHVVHV33DQG3VKDULQJGLIIHUHQWREMHFWV[\ERWKRZQHGE\3 3 LQLWLDOL]HVWKHYDOXHVRI[DQG\WRZLWKZ[ DQGZ\ RSHUDWLRQVUHVXOWLQJLQ WZRVXEVHTXHQWLQF97 1RWHWKDWWKHYDOXHRI97LVVKRZQRQWKH)LJXUHEHORZWKH WZRZULWHRSHUDWLRQV$IWHUWKDWDUHDGUHTXHVWIURP3DUULYHVDW3$WWKLVPRPHQW 3 GHWHUPLQHV WKH VHW RXGDWHG97>@ PRGLILHGBVLQFH ?∅ UHWXUQLQJ ^[LG \LG` DQG UHVSRQGV ZLWK 5B83'\97>@ ,19VHW ^[LG` 7KH LG RI \ LV QRW LQFOXGHG LQ WKH ,19VHW EHFDXVH \ LV EHLQJ XSGDWHG 2Q UHFHLSW RI WKDW PHVVDJH 3 LQYDOLGDWHV[DQGXSGDWHVWKHVHQGHUSRVLWLRQRILWVRZQYHFWRUFORFN97>@WR7KDW YDOXHLVEHXVHGWRFRPSRVHWKHQH[WUHTXHVWPHVVDJHWR3,QWKHVDPSOHH[HFXWLRQ 3 DOVRUHFHLYHV:B83'\97>@ ,19VHW ∅ IURP37KLVPHVVDJHFDXVHVWKH XSGDWH RI \YDOXH DQG 97>@ +RZHYHU \WV UHPDLQV XQFKDQJHG )LQDOO\ WZR UHDG DFFHVVHVDUHLQYRNHGRQ3WKHILUVWRQHU[ LVLVVXHGWRWKHSUHYLRXVO\LQYDOLGDWHG [ ZKLOH WKH VHFRQG U\ LV SHUIRUPHG ORFDOO\ RQ FXUUHQWO\ DYDLODEOH UHSOLFD RI \ 1RWHWKDWYDOXHRI\KDVQRWEHHQLQYDOLGDWHGGXULQJWKHXSGDWHRI[7KLVLVGXHWR WKHIDFWWKDW\WVUHPDLQVHTXDOWRDW\RZQHULQGLFDWLQJWKDWWKHORFDOYDOXHRI\DW3 LV VWLOO SURFHVVRU FRQVLVWHQW ,W FDQ EH HDVLO\ VHHQ LQ WKLV H[DPSOH WKDW WKH SURFHVVRU FRQVLVWHQWRSHUDWLRQU\ LVQRWFDXVDOO\FRQVLVWHQWDOVRQRWVHTXHQWLDOO\FRQVLVWHQW
-%U]H]L VNLDQG06]\FKRZLDN
)LJ$VDPSOHH[HFXWLRQRISURWRFRORSHUDWLRQV
4 &RUUHFWQHVVRIWKH3URWRFRO :H DUJXH WKDW WKH FRKHUHQFH SURWRFRO SUHVHQWHG LQ 6HFWLRQ DOZD\V PDLQWDLQV WKH VKDUHGPHPRU\LQDVWDWHZKLFKLVSURFHVVRUFRQVLVWHQW7RSURYHWKLVFODLPZHVKDOO VKRZWKDWLQDQ\H[HFXWLRQRIDFFHVVRSHUDWLRQVWKH3&*SSURWRFROVHULDOL]HVWKHRS HUDWLRQV LQ WKH SURFHVVRU RUGHU )LUVW ZH UHPDUN WKDW LQ D JLYHQ H[HFXWLRQ RI 3&*S SURWRFRORQ3LWKHUHLVH[DFWO\RQHWRWDORUGHULQJ→LRI+LDVSURFHVVHVDUHVHTXHQ WLDO 7KHUHFDQEHKRZHYHUVHYHUDOVHULDOL]DWLRQV6LRI+L∪+:SRVVLEOH 'HILQLWLRQ /HJDOVHULDOL]DWLRQ 6HULDOL]DWLRQ6LRI+L∪+:LVOHJDOLILWSUHVHUYHVORFDORUGHULQJ→L→L⊆6L LH
∀
R R ∈+ L
R→LR⇒R6LR
,QWKHUHVWRIWKLVVHFWLRQIRUHDFK+L∪+:ZHFKRRVHDUELWUDULO\DVHULDOL]DWLRQ WKDW LV OHJDO DQG FDQ EH FRQVWUXFWHG E\ H[WHQGLQJ →L ZLWK UHVXOWV RI UHFHSWLRQ RI 5B83'DQG:B83'PHVVDJHVSURFHVVHGLQDJLYHQH[HFXWLRQRI3&*S:HGHQRWHLW E\vL:HVKRZWKDWVXFKVHULDOL]DWLRQVSUHVHUYHERWK35$0DQGFDFKHFRQVLVWHQF\ LH IXOILOO WKH SURFHVVRU FRQVLVWHQF\ FRQGLWLRQ RI 'HILQLWLRQ :H VWDUW E\ SURYLQJ WKDWHDFKvLSUHVHUYHVFDFKHFRQVLVWHQF\%\RLW[ ZHGHQRWHDQRSHUDWLRQRQ[SHU IRUPHGE\3LDWWLPHW
/RZ&RVW&RKHUHQFH3URWRFROIRU'606\VWHPVZLWK3URFHVVRU&RQVLVWHQF\
/HPPD,I3NLVWKHKRPHQRGHIRUREMHFW[[RZQHU N WKHQ
∀
∀ Z v Z ⇔Z v Z
Z Z ∈+: ∩ + _ [ L = Q
L
N
3URRI )LUVWOHWXVFRQVLGHUWZRZULWHVZ ZL[ DDQGZ ZL[ ELVVXHGWR[E\WKH VDPHSURFHVV7KHFDVHL NLVWULYLDOWKHUHIRUHZHDVVXPHL≠N,Q3&*SHYHU\ZULWH ZL[ LVVXHGWRDQRQPDVWHUUHSOLFDUHTXLUHVWRXSGDWHWKHPDVWHUUHSOLFDDWWKHKRPH QRGH DQG HYHU\ XSGDWH PXVW EH DFNQRZOHGJHG EHIRUH LVVXLQJ QH[W ZL[ 7KHUHIRUH WKHRUGHURIDQ\WZRZULWHRSHUDWLRQVLQvNDJUHHVZLWKWKHORFDORUGHURIWKHLVVXHU LH ZL[ D→LZL[ E ⇔ ZL[ DvNZL[ E DQG IURP WKLV ZL[ D vL ZL[ E ⇔ ZL[ DvNZL[ E 1RZOHWXVFRQVLGHUZ ZM[ DZ ZL[ EDQGZM[ DvLZL[ E7KHUHVXOWRI DQ\ ZMW [ Y IRU M≠L≠N LV PDGH YLVLEOH WR 3L LQ 3&*S RQO\ E\ UHFHSWLRQ RI 5B83' GXULQJULW
[ YZKHUHW
!W SRVVLEOHRQO\LI[ZDVLQYDOLGDWHGDW3LEHIRUHW
2SHUD WLRQULW
[ YZLOOWKHUHIRUHUHWXUQDYDOXHEURXJKWIURPPDVWHUUHSOLFD RI [ HQIRUFLQJ RUGHULQJZKLFKFRQIRUPVWRvN @ 0 $KDPDG 3: +XWWR DQG 5 -RKQ ,PSOHPHQWLQJ DQG 3URJUDPPLQJ &DXVDO 'LVWULEXWHG6KDUHG0HPRU\3URFWK,QW¶O&RQIRQ'LVWULEXWHG&RPSXWLQJ0D\ SS± >@ 0 $KDPDG 5$%D]]L 5-RKQ 3.RKOL DQG *1HLJHU 7KH SRZHU RI SURFHVVRU FRQVLVWHQF\7HFKQLFDO5HSRUW*,7&&*HRUJLD,QVWLWXWHRI7HFKQRORJ\$WODQWD 'HFHPEHU >@ &$P]D $/&R[ 6'ZDUNDGDV 3.HOHKHU +/X 55DMDPRQ\ :<X DQG :=ZDHQHSRHO 7UHDG0DUNV 6KDUHG 0HPRU\ &RPSXWLQJ RQ 1HWZRUNV RI :RUNVWDWLRQV,(((&RPSXWHU )HEUXDU\SS± >@ & $P]D $/ &R[ DQG : =ZDHQHSRHO 'DWD 5HSOLFDWLRQ 6WUDWHJLHV IRU )DXOW 7ROHUDQFH DQG $YDLODELOLW\ RQ &RPPRGLW\ &OXVWHUV 3URF ,QW¶O &RQI RQ 'HSHQGDEOH 6\VWHPVDQG1HWZRUNV'61 -XQH >@ 5&KULVWRGRXORSRXORX5$]LPLDQG$%LODV'\QDPLFGDWDUHSOLFDWLRQ$QDSSURDFK WRSURYLGLQJIDXOWWROHUDQWVKDUHGPHPRU\FOXVWHUV3URFWK,(((6\PSRVLXPRQ+LJK 3HUIRUPDQFH&RPSXWHU$UFKLWHFWXUH+3&$ )HEUXDU\ >@ . *KDUDFKRUORR ' /HQRVNL - /DXGRQ 3 *LEERQV $ *XSWD DQG -+DQQHVV\ 0HPRU\&RQVLVWHQF\DQG(YHQW2UGHULQJLQ6FDODEOH6KDUHG0HPRU\0XOWLSURFHVVRUV 3URFWK,QW¶O6\PSRVLXPRQ&RPSXWHU$UFKLWHFWXUH0D\SS± >@ - 5 *RRGPDQ &DFKH FRQVLVWHQF\ DQG VHTXHQWLDO FRQVLVWHQF\ 7HFKQLFDO 5HSRUW ,(((6FDODEOH&RKHUHQW,QWHUIDFH:RUNLQJ*URXS0DUFK
/RZ&RVW&RKHUHQFH3URWRFROIRU'606\VWHPVZLWK3URFHVVRU&RQVLVWHQF\ >@ 0 +HUOLK\ DQG - :LQJ /LQHDUL]DELOLW\ $ &RUUHFWQHVV &RQGLWLRQ IRU &RQFXUUHQW 2EMHFW$&07UDQVDFWLRQVRQ3URJUDPPLQJ/DQJXDJHVDQG6\VWHPV -XO\ SS± >@ / +LJKDP DQG - .DZDVK %RXQGV IRU 0XWXDO ([FOXVLRQ ZLWK RQO\ 3URFHVVRU &RQVLV WHQF\3URFWK,QW¶O6\PSRVLXPRQ'LVWULEXWHG&RPSXWLQJ',6& 2FWREHU /1&66SULQJHUSS± >@ 5 -RKQ DQG 0 $KDPDG &DXVDO 0HPRU\ ,PSOHPHQWDWLRQ 3URJUDPPLQJ 6XSSRUW DQG ([SHULHQFHV7HFKQLFDO5HSRUW*,7&&*HRUJLD,QVWLWXWHRI7HFKQRORJ\ >@ 5-/LSWRQDQG-66DQGEHUJ³35$0D6FDODEOH6KDUHG0HPRU\´7HFKQLFDO5HSRUW &6753ULQFHWRQ8QLYHUVLW\6HSWHPEHU >@ 0&/LWWOHDQG6.6KULYDVWDYD,QWHJUDWLQJ*URXS&RPPXQLFDWLRQZLWK7UDQVDFWLRQV IRU,PSOHPHQWLQJ3HUVLVWHQW5HSOLFDWHG2EMHFWV/HFWXUH1RWHVLQ&RPSXWHU6FLHQFHYRO SS± >@ ) 0DWWHUQ 7LPH DQG *OREDO 6WDWHV RI 'LVWULEXWHG 6\VWHPV 3URF ,QW¶O :RUNVKRS RQ 3DUDOOHODQG'LVWULEXWHG$OJRULWKPV >@ 05D\QDO6HTXHQWLDO&RQVLVWHQF\DV/D]\/LQHDUL]DELOLW\7HFKQLFDO5HSRUW3, ,5,6$5HQQHV-DQXDU\ >@ -66DQGEHUJ³'HVLJQRI35$01HWZRUN´7HFKQLFDO5HSRUW&6753ULQFHWRQ 8QLY$SULO
Minimizing Communication Cost in Fine-Grain Partitioning of Sparse Matrices Bora U¸car and Cevdet Aykanat Department of Computer Engineering, Bilkent University, 06800, Ankara, Turkey {ubora,aykanat}@cs.bilkent.edu.tr
Abstract. We show a two-phase approach for minimizing various communication-cost metrics in fine-grain partitioning of sparse matrices for parallel processing. In the first phase, we obtain a partitioning with the existing tools on the matrix to determine computational loads of the processor. In the second phase, we try to minimize the communicationcost metrics. For this purpose, we develop communication-hypergraph and partitioning models. We experimentally evaluate the contributions on a PC cluster.
1
Introduction
Repeated matrix-vector multiplications (SpMxV) y = Ax that involve the same large, sparse, structurally symmetric or nonsymmetric square or rectangular matrix are kernel operations in various iterative solvers. Efficient parallelization of these solvers requires matrix A to be partitioned among the processors in such a way that communication overhead is kept low while maintaining computational load balance. Because of possible dense vector operations, some of these methods require symmetric partitioning on the input and output vectors, i.e, conformal partitioning on x and y. However, quite a few of these methods allow unsymmetric partitionings, i.e., x and y may have different partitionings. The standard graph partitioning model has been widely used for one-dimensional (1D) partitioning of sparse matrices. Recently, C ¸ ataly¨ urek and Aykanat [3,4] and others [9,10,11] demonstrated some flaws and limitations of this model and developed alternatives. As noted in [10], most of the existing models consider minimizing the total communication volume. Depending on the machine architecture and the problem characteristics, communication overhead due to message latency may be a bottleneck as well [8]. Furthermore, maximum communication volume and latency handled by a single processor may also have crucial impacts on the parallel performance. In our previous work [17], we addressed these four communication-cost metrics in 1D partitioning of sparse matrices. The literature on 2D matrix partitioning is rare. The 2D checkerboard partitioning approaches proposed in [12,15,16] are suitable for dense matrices or sparse matrices with structured nonzero patterns that are difficult to exploit. In
This work was partially supported by The Scientific and Technical Research Council of Turkey under project EEEAG-199E013
A. Yazici and C. S ¸ ener (Eds.): ISCIS 2003, LNCS 2869, pp. 926–933, 2003. c Springer-Verlag Berlin Heidelberg 2003
Minimizing Communication Cost in Fine-Grain Partitioning
927
particular, these approaches do not exploit sparsity to reduce the communication volume. C ¸ ataly¨ urek and Aykanat [3,6,7] proposed hypergraph models for 2D coarse-grain and fine-grain sparse matrix partitionings. In the coarse-grain model, a matrix is partitioned in a checkerboard-like style. In the fine-grain model, a matrix is partitioned on nonzero basis. The fine-grain model is reported to achieve better partitionings than the other models in terms of the total communication volume metric [6]. However, it also generates worse partitionings than the other models in terms of the total number of messages metric [6]. In this work, we adopt our two phase approach [17] to minimize the four communication-cost metrics in the fine-grain partitioning of sparse matrices. We show how to apply our communication hypergraph model to obtain unsymmetric partitioning and develop novel models to obtain symmetric partitioning.
2
Preliminaries
A hypergraph H = (V, N ) is defined as a set of vertices V and a set of nets. Every net ni is a subset of vertices. Let dj denote the number of nets containing a vertex vj . Weights can be associated with vertices. Π = {V1 , · · · , VK } is a K-way vertex partition of H = (V, N ) if each part Vk is non-empty, parts are pairwise disjoint, and the union of parts gives V. In Π, a net is said to connect a part if it has at least one vertex in that part. Connectivity λi of a net ni denotes the number of parts connected by ni . A net nj is said to be cut if λj > 1 and uncut otherwise. The set of cut and uncut nets are called external and internal nets, respectively. In Π, the weight of a part is the sum of the weights of the vertices in that part. In the hypergraph partitioning problem, the objective is to minimize the cutsize: cutsize(Π) = (λi − 1). (1) ni ∈N
This objective is referred to as the connectivity−1 cutsize metric [14]. The partitioning constraint is to satisfy a balancing constraint on the part weights, i.e., Wmax ≤ (1 + )Wavg
(2)
where Wmax is the weight of the part with the maximum weight and Wavg is the average part weight, and is an imbalance ratio. This problem is NP-hard [14]. A recent variant of the above problem is called multi-constraint hypergraph partitioning [3,7,13]. In this problem, a vertex has a vector of weights. The partitioning objective is the same as above, however, the partitioning constraint is to satisfy a set of balancing constraints, one for each type of the weights. In the fine-grain hypergraph model of C ¸ ataly¨ urek and Aykanat [6], an M ×N matrix A with Z nonzeros is represented as a hypergraph H = (V, N ) with |V| = Z vertices and |N | = M + N nets for 2D partitioning. There exists one vertex vij for each nonzero aij . There exists one net mi for each row i and one net nj for each column j. Each row-net mi and column-net nj contain all vertices vi∗ and v∗j , respectively. Each vertex vij corresponds to scalar multiplication aij xj . Hence, the computational weight associated with a vertex is 1. Each row-net mi represents the dependency of yi on the scalar multiplications with ai∗ ’s. Each
928
B. U¸car and C. Aykanat
column-net nj represents the dependency of scalar multiplications with a∗j ’s on xj . With this model, the problem of 2D partitioning a matrix among K processors reduces to the K-way hypergraph partitioning problem. In this model, minimizing the cutsize while maintaining the balance on the part weights corresponds to minimizing the total communication volume and maintaining the balance on the computational loads of the processors. An external column-net represents the communication volume requirement on a x-vector entry. This communication occurs in expand phase, just before the scalar multiplications. An external row-net represents the communication volume requirement on a y-vector entry. This communication occurs in fold phase, just after the scalar multiplications. C ¸ ataly¨ urek and Aykanat [6] assign the responsibility of expanding xi and folding yi to the processor that holds aii to obtain symmetric partitioning. Note that for the unsymmetric partitioning case, one can assign xi to any processor holding a nonzero in column i without any additional communication-volume overhead. A similar opportunity exists for yi . In the symmetric partitioning case, however, xi and yi may be assigned to a processor holding nonzeros both in the row and column i. In this work, we try to exploit the freedom in assigning vector elements to address the four communication-cost metrics. A 10 × 10 matrix with 37 nonzeros and its 4-way fine-grain partitioning is given in Fig. 1(a). In the figure, the partitioning is given by the processor numbers for each nonzero. The computational load balance is achieved by assigning 9, 10, 9, and 9 nonzeros to processors in order.
1
2
3
4
1
5
6
4
7
8
4
2 3
1
1
1
4
1
1
1
5
2
3
6
8
2
9 10
1 2
4
4
4
3
2
4
3
2
2
5
7
P1 P2 P3 P 4
8 10
4
P1
4
2
P2
5
P3
7
P4
9
3 2
(a)
1
3 4
3
1
3
3 3
2
4
1
2
7
9 10
2
(b)
(c)
Fig. 1. (a) A 10 × 10 matrix and a 4-way partitioning, (b) communication matrix Cx , and (c) communication matrix Cy
Minimizing Communication Cost in Fine-Grain Partitioning
3
929
Minimizing the Communication Cost
Given a K-way fine-grain partitioning of a matrix, we identify two sets of rows and columns; internal and coupling. The internal rows or columns have nonzeros only in one part. The coupling rows or columns have nonzeros in more than one part. The set of x-vector entries that are associated with the coupling columns, referred to here as xC , necessitate communication. Similarly, the set of y-vector entries that are associated with the coupling rows, referred to here as yC , necessitate communication. Note that when symmetric partitioning requirement arises we add to xC those x-vector entries whose corresponding entries are in yC and vice versa. The proposed approach considers partitioning of these xC and yC vector entries to reduce the total message count and the maximum communication volume per processor. The other vector entries are needed by only one processor and should be assigned to the respective processors to avoid redundant communication. The approach may increase the total volume of communication of the given partitioning by at most max{|xC |, |yC |}. We propose constructing two matrices Cx and Cy , referred to here as communication matrices, that summarize the communication on x- and y-vector entries, respectively. Matrix Cx has K rows and |xC | columns. For each row k we insert a nonzero in column j if processor Pk has nonzeros in column corresponding to xC [j] in the fine-grain partitioning. Hence, the rows of Cx correspond to processors in such a way that the nonzeros in the row k identify the subset of xC -vector entries that are needed by processor Pk . Matrix Cy is constructed similarly. This time we put processors in columns and yC entries in rows. Figure 1(b) and (c) show communication matrices Cx and Cy for the sample matrix given in (a). 3.1
Unsymmetric Partitioning Model
We use row-net and column-net hypergraph models for representing Cx and Cy , respectively. In the row-net hypergraph model, matrix Cx is represented as hypergraph Hx for columnwise partitioning. The vertex and net sets of Hx correspond to the columns and rows of matrix Cx , respectively. There exist one vertex vj and one net ni for each column j and row i, respectively. Net ni contains the vertices corresponding to the columns which have a nonzero in row i. That is, = 0. In the column-net hypergraph model Hy of Cy , the vertex vj ∈ ni if Cx [i, j] and net sets correspond to the rows and columns of the matrix Cy , respectively, with similar construction. Figure 2(a) and (b) show communication hypergraphs Hx and Hy . A K-way partition on the vertices of Hx induces a processor assignment for the expand operations. Similarly, a K-way partition on the vertices of Hy induces a processor assignment for the fold operations. In unsymmetric partitioning case, these two assignment can be found independently. In [17] we showed how to obtain such independent partitionings in order to minimize the four communication-cost metrics. The results of that work are immediately applicable to this case.
930
B. U¸car and C. Aykanat n1 5
1 2
n2 10
n4
4
n3 7
y4
n4
n1
7
n3
x[4] y[4]
x[7] y[7]
9
n2 8
5
(a)
(b)
x2
y1
x3
x4
y2
(c)
Fig. 2. Communication hypergraphs: (a) Hx , (b) Hy , and (c) a portion of H
3.2
Symmetric Partitioning Model
When we require symmetric partitioning on vectors x and y, the partitionings on Hx and Hy can not be obtained independently. Therefore, we combine hypergraphs Hx and Hy into a single hypergraph H as follows. For each part Pk , we create two nets xk and yk . For each xC [i] and yC [i] pair, we create a single vertex vi . For each net xk , we insert vi into its vertex list if processor Pk needs xC [i]. For each yk , we insert vj into its vertex list if processor Pk contributes to yC [j]. We show vertices v4 and v7 of H in Fig. 2(c). Since communication occurs in two distinct phases, vertices have two weights associated with them. The first weight of a vertex vi is the communication volume requirement incurred by xC [i]; hence we associate weight di − 1 with the vertex vi . The second weight of a vertex vi is the communication volume requirement incurred by yC [i]; as in [17] we associate a unit weight of 1 with each vi . In a K-way partition of H, an xk -type net connecting λxk parts necessitates λxk − 1 messages to be sent to processor Pk during the expand phase. The sum of these values thus represents the total number of messages sent during the expand phase. Similarly, a yk -type net connecting λyk parts necessitates λyk − 1 messages to be sent by Pk during the fold phase. The sum of these values represents the total number of messages sent during the fold phase. The sum of the connectivity − 1 values for all nets thus represents the total number of messages. Therefore, by minimizing the objective function in Eq. 1, partitioning H minimizes the total number of messages. The vertices in part Vk represent the x-vector entries to be expanded and the respective y-vector entries to be folded by processor Pk . The load of the expand operations are exactly represented by the first components of the vertex weights if for each vi ∈ Vk we have vi ∈ / xk , the weight of a vertex for the expand phase will be xk . If, however, vi ∈ one less than the required. We hope these shortages to occur, in some extent, for every processor to cancel the diverse effects on the communication-volume load balance. The weighting scheme for the fold operations is adopted with the rationale that every yC [i] assigned to a processor Pk will relieve Pk from sending
Minimizing Communication Cost in Fine-Grain Partitioning
931
a unit-volume message. If the net sizes are close to each other then this scheme will prove to be a reasonable one. As a result, balancing part sizes for the two set of weights, e.g., satisfying Eq. 2, will relate to balancing the communicationvolume loads of processors in the expand and the fold phases, separately. For the minimization of the maximum number of messages per processor metric we do not spend explicit effort. We merely rely on its loose correlation with the total number of messages metric. In the above discussion, each net is associated with a certain part and hence a processor. This association is not defined in the standard hypergraph partitioning problem. We can enforce this association by adding K special vertices, one for each processor Pk , and inserting those vertices to the nets xk and yk . Fixing those special vertices to the respective parts and using partitioning with fixed vertices feature of hypergraph partitioning tools [1,5] we can obtain the specified partitioning on H. However, existing tools do not handle fixed vertices within multi-constraint framework. Therefore, instead of obtaining balance on the communication-volume loads of processors in the expand and the fold phases separately, we add up the weights of vertices and try to obtain balance on aggregate communication-volume loads of processors.
4
Experiments
We have conducted experiments on the matrices given in Table 1. In the table, N and N N Z show, respectively, the dimension of the matrix and the number of nonzeros. The Srl.Time column lists the timings for serial SpMxV operations in milliseconds. We used PaToH [5] library to obtain 24-way fine-grain partitionings on the test matrices. In all partitioning instances, the computational-load imbalance were below 7 percent. Part.Mthd give the partitioning method applied: PTH refers to the fine-grain partitioning of C ¸ ataly¨ urek and Aykanat [6], CHy refers to partitioning the communication hypergraphs with fixed vertices and aggregate vertex weights. For these two methods, we give timings under column Part.Time, in seconds. For each partitioning method, we dissect the communication requirements into the expand and fold phases. For each phase, we give the total communication volume, the maximum communication volume handled by a single processor, the total number of messages, and the maximum number of messages per processor under columns tV, xV, tM, and xM, respectively. In order to see whether the improvements achieved by method CHy in the given performance metrics hold in practice, we also give timings, the best among 20 runs, for parallel SpMxV operations, in milliseconds, under column Prll.Time. All timings are obtained on machines equipped with 400 MHz Intel Pentium II processor and 64 MB RAM running Linux kernel 2.4.14 and Debian GNU/Linux 3.0 distribution. The parallel SpMxV routines are implemented using LAM/MPI 6.5.6 [2]. To compare our method against PTH, we opted for obtaining symmetric partitioning. For each matrix, we run PTH 20 times starting from different random seeds and selected the partition which gives the minimum in total communication volume metric. Then, we constructed the communication hypergraph with respect to PTH’s best partitioning and run CHy 20 times, again starting from
932
B. U¸car and C. Aykanat
different random seeds, and selected the partition which gives the minimum in total number of messages metric. Timings for these partitioning methods are for a single run. In all cases, running CHy adds at most half of the time required by PTH to the framework of fine-grain partitioning. In all of the partitioning instances, CHy reduces the total number of messages to somewhere between 0.47 (fom12) and 0.74 (CO9) of PTH. In all partitioning instances, CHy increases the total communication volume to somewhere between 1.32 (creb) and 1.86 (pds20) of PTH. This is expected, because a vertex vi may be assigned to a part Vk while Pk does not need any of xC [i] or yC [i]. However, reductions in parallel running times are seen for all matrices except lpl1. The highest speed-up achieved by PTH and CHy is 5.96 and 6.38, respectively, on fxm3. Table 1. Communication patterns and running times for 24-way parallel SpMxV Matrix CO9 creb ex3s1 fom12 fxm3 lpl1 mod2 pds20 pltex world
5
Size Part. N N N Z Mthd Time 10789 249205 PTH 11.43 CHy 0.66 9648 398806 PTH 26.71 CHy 2.51 17443 679857 PTH 48.88 CHy 13.58 24284 329068 PTH 22.07 CHy 13.76 41340 765526 PTH 37.73 CHy 0.29 39951 541217 PTH 27.04 CHy 5.09 34774 604910 PTH 32.83 CHy 2.18 33874 320196 PTH 18.65 CHy 6.09 26894 269736 PTH 14.29 CHy 1.11 34506 582064 PTH 30.84 CHy 2.50
Expand Phase tV xV tM xM 2477 524 290 21 5121 318 223 22 9344 750 490 23 13047 715 313 23 7964 602 312 22 19537 1128 195 23 7409 559 228 23 16713 983 96 10 1843 279 212 23 3299 215 142 16 7646 1062 226 20 15079 892 166 22 5015 845 267 23 10523 656 181 23 5373 557 299 23 14066 794 177 18 1883 172 167 16 4533 311 89 16 4934 794 300 23 10679 656 181 23
Fold Phase tV xV tM 4889 473 358 7367 714 259 12660 871 504 16157 1068 341 26434 1762 356 36347 2252 270 21208 1143 228 28151 1541 119 2662 282 236 4027 456 156 13752 961 253 20582 1507 186 9421 1135 278 14142 1517 198 13548 956 317 21302 1436 201 7065 508 273 8828 782 139 9710 1295 316 14854 1745 205
Prll. Srl. xM Time Time 22 4.55 12.54 18 4.05 23 6.64 19.30 20 5.92 20 8.39 33.52 16 7.91 13 5.03 19.86 8 4.03 17 6.39 38.13 15 5.97 17 5.73 29.81 12 5.83 22 6.92 30.81 17 5.92 19 5.23 17.82 13 4.95 20 4.27 14.64 10 3.63 23 7.35 29.63 18 6.05
Conclusion and Future Work
We showed a two-phase approach that encapsulates various communication-cost metrics in 2D partitioning of sparse matrices. We developed models to obtain symmetric and unsymmetric partitioning on input and output vectors. We tested performance of the proposed models on practical implementations. In this work, a sophisticated hypergraph partitioning tool that can handle fixed vertices in the context of multi-constraint partitioning was needed. Since the existing tools do not handle this type of partitioning, we are considering to develop such a method.
Minimizing Communication Cost in Fine-Grain Partitioning
933
References 1. Charles J. Alpert, Andrew E. Caldwell, Andrew B. Kahng, and Igor L. Markov. Hypergraph partitioning with fixed vertices. IEEE Transactions on ComputerAided Design, 19(2):267–272, 2000. 2. Greg Burns, Raja Daoud, and James Vaigl. LAM: an open cluster environment for MPI. In John W. Ross, editor, Proceedings of Supercomputing Symposium ’94, pages 379–386. University of Toronto, 1994. ¨ V. C 3. U. ¸ ataly¨ urek. Hypergraph Models for Sparse Matrix Partitioning and Reordering. PhD thesis, Bilkent Univ., Computer Eng. and Information Sci., Nov 1999. ¨ V. C 4. U. ¸ ataly¨ urek and C. Aykanat. Hypergraph-partitioning based decomposition for parallel sparse-matrix vector multiplication. IEEE Transactions on Parallel and Distributed Systems, 10(7):673–693, 1999. ¨ V. C 5. U. ¸ ataly¨ urek and C. Aykanat. PaToH: A multilevel hypergraph partitioning tool, ver. 3.0. Tech. Rep. BU-CE-9915, Computer Eng. Dept., Bilkent Univ., 1999. ¨ V. C 6. U. ¸ ataly¨ urek and Cevdet Aykanat. A fine-grain hypergraph model for 2d decomposition of sparse matrices. In Proceedings of International Parallel and Distributed Processing Symposium (IPDPS), 8th International Workshop on Solving Irregularly structured Problems in Parallel (Irregular 2001), April 2001. ¨ V. C 7. U. ¸ ataly¨ urek and Cevdet Aykanat. A hypergraph-partitioning approach for coarse-grain decomposition. In Proceedings of Scientific Computing 2001 (SC2001), pages 10–16, Denver, Colorado, November 2001. 8. J. J. Dongarra and T. H. Dunigan. Message-passing performance of various computers. Concurrency—Practice and Experience, 9(10):915–926, 1997. 9. B. Hendrickson. Graph partitioning and parallel solvers: has the emperor no clothes? Lecture Notes in Computer Science, 1457:218–225, 1998. 10. B. Hendrickson and T. G. Kolda. Graph partitioning models for parallel computing. Parallel Computing, 26:1519–1534, 2000. 11. B. Hendrickson and T. G. Kolda. Partitioning rectangular and structurally unsymmetric sparse matrices for parallel processing. SIAM Journal on Scientific Computing, 21(6):2048–2072, 2000. 12. B. Hendrickson, R. Leland, and S. Plimpton. An efficient parallel algorithm for matrix-vector multiplication. Int. J. High Speed Comput., 7(1):73–88, 1995. 13. G. Karypis and V. Kumar. Multilevel algorithms for multi-constraint hypergraph partitioning. Tech. Rep. 99-034, University of Minnesota, Dept. Computer Science/Army HPC Research Center, Minneapolis, MN 55455, November 1998. 14. T. Lengauer. Combinatorial Algorithms for Integrated Circuit Layout. Wiley– Teubner, Chichester, U.K., 1990. 15. J. G. Lewis, D. G. Payne, and R. A. van de Geijn. Matrix-vector multiplication and conjugate gradient algorithms on distributed memory computers. In Proceedings of the Scalable High Performance Computing Conference, 1994. 16. A. T. Ogielski and W. Aiello. Sparse matrix computations on parallel processor arrays. SIAM Journal on Numerical Analysis, 1993. 17. Bora U¸car and Cevdet Aykanat. Encapsulating multiple communication-cost metrics in partitioning sparse rectangular matrices for parallel matrix-vector multiplies. Siam J. Sci. Comput., Submitted to, 2002.
Scalability and Robustness of Pull-Based Anti-entropy Distribution Model g]QXUg]NDVDS Koç University, Department of Computer Engineering, Istanbul, Turkey RR]NDVDS#NXHGXWU
Abstract. There are several alternative mechanisms for disseminating information among a group of participants in a distributed environment. An efficient model is to use epidemic algorithms that involve pair-wise propagation of information. These algorithms are based on the theory of epidemics which studies the spreading of infectious diseases through a population. Epidemic protocols are simple, scale well and robust again common failures, and provide eventual consistency as well. They have been mainly utilized in a large set of applications for resolving inconsistencies in distributed database updates, failure detection, reliable multicasting, network news distribution, scalable system management, and resource discovery. A popular distribution model based on the theory of epidemics is the anti-entropy. In this study, we focus on pull-based anti-entropy model used for multicast reliability as a case study, demonstrate its scalability and robustness, and give our comparative simulation results discussing the performance of the approach on a range of typical scenarios. Keywords: Epidemic algorithms, pull-based anti-entropy, loss recovery for multicast, Bimodal Multicast.
1 Introduction There are several alternative mechanisms for disseminating updates/information among a group of participants in a distributed environment. Unicast involves the transmission of a separate message for each participant. Both multicast and broadcast can be effectively used with push mode of dissemination, and both modes would be very efficient if the underlying network supports them. Otherwise, flooding for broadcast or application-level dissemination trees for multicast can be utilized. Another alternative is to use epidemic algorithms that involve pair-wise propagation of updates. Epidemic algorithms are based on the theory of epidemics which studies the spreading of infectious diseases through a population. Epidemic communication mechanisms were first proposed for spreading updates in a replicated database [1]. The aim in this case is to infect all replicas with new updates as fast as possible. Epidemic protocols are simple, scale well and robust again common failures, and provide eventual consistency as well. They combine benefits of efficiency in hierarchical data dissemination with robustness in flooding protocols. Epidemic communication allows temporary inconsistencies in shared data among participants, in ex$@$FFRUGLQJWRWKHWHUPLQRORJ\RIHSLGHPLRORJ\DVLWHKROGLQJLQIRUPDWLRQRUDQ XSGDWHLWLVZLOOLQJWRVKDUHLVFDOOHGLQIHFWLYH$VLWHLVFDOOHGVXVFHSWLEOHLILWKDVQRW \HWUHFHLYHGDQXSGDWH,QWKHDQWLHQWURS\SURFHVVQRQIDXOW\VLWHVDUHDOZD\VHLWKHU VXVFHSWLEOH RU LQIHFWLYH ,Q WKLV PRGHO D VLWH 3 SLFNV DQRWKHU VLWH 4 DW UDQGRP DQG H[FKDQJHVXSGDWHVZLWK4)RUH[FKDQJLQJXSGDWHVWKHUHDUHWKUHHDSSURDFKHV>@ ~ Push approach: P only propagates its updates to Q. In this case, updates can be propagated only by infective sites. If many sites are infective, the probability of each one selecting a susceptible site is relatively small. As a result, a particular site may remain susceptible for a long period simply because it is not selected by an infective site. Hence, this property limits the speed of spreading updates. ~ Pull approach: P only gets new updates from Q. When several sites are infective, this approach works better. In this case, spreading updates is triggered by susceptible sites. It would be highly possible that such a site will contact an infective one to pull in the updates and become infective as well. ~ Push-pull approach: This is a hybrid of pull and push approaches where P and Q send updates to each other. 2QH RI WKH IXQGDPHQWDO UHVXOWV RI HSLGHPLF WKHRU\ VKRZV WKDW VLPSOH HSLGHPLFV HYHQWXDOO\LQIHFWWKHHQWLUHSRSXODWLRQ,IWKHUHLVDVLQJOHLQIHFWLYHVLWHDWWKHEHJLQ QLQJ XSGDWHV ZLOO HYHQWXDOO\ EH VSUHDG DFURVV DOO VLWHV XVLQJ HLWKHU IRUP RI DQWL HQWURS\)XOOLQIHFWLRQLVDFKLHYHGLQH[SHFWHGWLPHSURSRUWLRQDOWRWKHORJDULWKPRI WKHSRSXODWLRQVL]H In this study, we focus on pull-based anti-entropy model for reliability of multicast communication, and demonstrate its scalability and robustness. We give our comparative simulation results discussing the performance of the model on a range of typical scenarios. Our case study, namely Bimodal Multicast protocol [11], utilizes the model for multicast loss recovery. Periodically, every site chooses another site at random and exchanges information to see any differences and achieve consistency. The information exchanged includes some form of message history of the group members. The paper is organized as follows. Section 2 describes the concept of multicast loss recovery and our case study of using pull-based anti-entropy for this purpose. Section 3 presents simulation settings, analysis and results of our study. Section 4 summarizes conclusions and discusses future work.
gg]NDVDS
2 Pull-Based Anti-entropy for Multicast Reliability Loss recovery is an essential part of a multicast service offering reliability. Overall, the key issue is to reduce the number of feedback messages that are returned to the sender. In the nonhierarchical feedback control approach [10], a model that has been adopted by several wide-area applications is referred to as feedback suppression. A well-known example is SRM protocol [12]. In SRM, when a receiver detects that it missed a message, it multicasts its feedback to the rest of the group. Multicasting feedback allows another group to suppress its own feedback. A receiver lacking a message schedules a feedback with some random delay. There are a number of other alternatives such as forward error correction (FEC) as utilized by PGM reliable multicast protocol [13] for ensuring reliability as well. Further discussion on the approaches for loss recovery and scalability in multicasting is given in our prior study [14] where large-scale behavior of epidemic loss recovery is analyzed on network topologies consisting of around thousand nodes. Our case study for evaluating the behavior of pull-based anti-entropy model, Bimodal Multicast [11], or Pbcast (Probabilistic Multicast) for short, is a novel option in the spectrum of multicast protocols that offers throughput stability, scalability and a bimodal delivery guarantee as the key properties. The protocol is inspired by prior work on epidemic protocols [1], Muse protocol for network news distribution [15], and the lazy transactional replication method of [4]. Bimodal Multicast has been shown to exhibit stable throughput under failure scenarios that are common on real large-scale networks [11]. In contrast, this kind of behavior can cause other reliable multicast protocols to exhibit unstable throughput. Bimodal Multicast consists of two sub-protocols, namely an optimistic dissemination protocol, and a two-phase antientropy protocol. The former is a best-effort, push mode, hierarchical multicast used to efficiently deliver a multicast message to its destinations. This phase is unreliable and does not attempt to recover a possible message loss. If IP multicast is available in the underlying system, it can be used for this purpose. Otherwise, a randomized dissemination protocol can play this role. The second stage of the protocol is responsible for message loss recovery. It is based on a pull-based anti-entropy protocol that detects and corrects inconsistencies in a system by continuous gossiping. The two-phase anti-entropy protocol progresses through unsynchronized rounds. In each round: ~ Every group member randomly selects another group member and sends a digest of its message history. This is called a gossip message. ~ The receiving group member compares the digest with its own message history. Then, if it is lacking a message, it requests the message from the gossiping process. This message is called solicitation, or retransmission request. ~ Upon receiving the solicitation, the gossiping process retransmits the requested message to the process sending this request. Details and the theory behind the anti-entropy protocol are given in [11] for the interested reader.
Scalability and Robustness of Pull-Based Anti-entropy Distribution Model
937
3 Simulations, Analysis, and Results We have developed a simulation model for Bimodal Multicast protocol and its pullbased anti-entropy mechanism on ns-2 network simulator [16]. The simulation model consists of three modules: unreliable data dissemination using IP multicast protocol, pull-based anti-entropy protocol, and FIFO message ordering for applications. Details of our implementation are given in [17]. In this section, we first describe the simulation settings, and then discuss results demonstrating the scalability and robustness of the dissemination model. 3.1 Simulation Settings The simulation scenarios consist of two representative cases, namely transit-stub dense topologies, and clusters connected by a bottleneck link. Transit-stub topologies approximate the structure of the Internet that can be viewed as a collection of interconnected routing domains where each domain can be classified as either a stub or a transit domain. Stub domains correspond to interconnected LANs and the transit domains model WAN or MANs [18]. The first scenario of transit-stub dense topologies is considered for various group sizes where all nodes are members of the multicast group. A certain link noise probability is set on every link that forms a randomized system-wide noise. An example application for this scenario would be a multicast based distance education session in a wide area setting consisting of multiple local campus networks, where one server would multicast the content to most of the nodes in the network. The second case includes scenarios where LANs connected by long distance links and networks where routers with limited bandwidth connect group members. Such configurations are common in today’s networks as well. With these in mind, we constructed topologies with fully-connected clusters, and a single link connecting those clusters. All nodes are members of the multicast group where there is one sender located on the first cluster. There is 1% intra-cluster noise formed in both clusters, and a varying high drop rate is injected on the link connecting the clusters which make it to behave as a bottleneck link. We obtain our results from several runs of simulations, each run consisting of a sequence of 35000 multicast data messages transmitted by the sender. 3.2 Loss Recovery Overhead, Scalability, and Robustness for Anti-entropy We investigate loss recovery overhead distribution of Bimodal Multicast on the scenarios mentioned in the previous section. For the transit-stub dense topologies, message rate of the multicast source is 50 messages per second. Fig.1 shows request messages received by each receiver in the multicast group for group and network sizes of 40 and 120, and system-wide noise rate of 0.01 on all links. It is observed that the load of overhead traffic is balanced over group members, a desirable property that avoids overloading one member or a portion of members with high overhead traffic. Increasing participant size does not have a significant effect on the distribution only causing a negligible increase in the number of requests received by each group re-
gg]NDVDS
Fig. 1. Request messages received by each group member, Bimodal Multicast, transit-stub dense, noise rate: 0.01
ceiver. Likewise, Fig.2 demonstrates analysis results where system wide-noise rate is increased to 0.1. It is obvious that increasing noise rate by a multiple of 10 on every link would increase the number of message drops experienced on the network. In fact, when compared with Fig.1, overhead does not increase linearly with the noise rate, and on a typical member, the number of overhead messages remains almost constant and even decrease in some cases as group size scales up. Comparative simulation results with nonhierarchical feedback control approach for this scenario are given in section 3.3. For clustered topologies, message rate of the multicast source is 100 messages per second. Fig.3 shows request messages received by each receiver in the multicast group where group size is 60 and 120 respectively for two different bottleneck noise (25% and 50%). The first half of the group members reside in cluster 1 and the rest in cluster 2. Due to the high bottleneck noise rate, a large percentage of messages get dropped during transmission from cluster 1 to cluster 2. The resultant overhead distribution on group members shows the effect of bottleneck noise where the members in the first cluster take a major part in the loss recovery process. However, the over
Fig. 2. Request messages received by each group member, Bimodal Multicast, transit-stub dense, noise rate: 0.1
Scalability and Robustness of Pull-Based Anti-entropy Distribution Model
939
head on group members within clusters is balanced and even decreases as group size doubles from 60 to 120.
Fig. 3. Request messages received/group member, Bimodal Multicast, clustered networks
We explore the impact of an increase in group size, and randomized noise over network links on transit-stub topologies. Fig.4 shows average loss recovery overhead in the form of requests received and sent on all receivers of the multicast group where system-wide noise rate and group size vary. As group size increases, protocol overheads remain stable. On the other hand, the effect of increasing noise rate by a multiple of 10 (1% to 10%) on all links has Fig. 4. Overhead percentage vs group size, transit-stub dense been measured. In particular, overhead percentage in the form of request messages received causes a slight increase which is below linear. These two observations indicate the scalability of the pull-based anti-entropy loss recovery as group size and noise rate expand. 3.3 Comparison with Nonhierarchical Feedback Control Recent studies [19,20,11] have shown that, for the SRM protocol with its nonhierarchical feedback control mechanism for loss recovery, random packet loss can trigger high rates of overhead messages. In addition, this overhead grows with the size of the system. For examining this effect, and demonstrating the superiority of anti-entropy as a method for multicast loss recovery, we have conducted a set of simulations by
gg]NDVDS
using SRM’s feedback control model. On transit-stub dense scenarios, comparative analysis results for Bimodal Multicast and SRM are displayed in Fig.5, left and right,
Fig. 5. Overhead for protocols, transit-stub dense: a) Bimodal Multicast, b) SRM.
respectively. For a fixed noise rate on links (either 1% or 10%), when group size scales up, overheads remain very stable for Bimodal Multicast, whereas for SRM number of request messages received grows with an increase of the number of participants in multicast group. Furthermore, number of request messages received by group members is quite low for Bimodal Multicast when compared to SRM. This is mainly due to the pair-wise propagation of information among group members when anti-entropy is used. On the other hand, SRM uses multicast dissemination during loss recovery in order to provide high reliability. Likewise, we analyze the effect of growth in system-wide noise on the loss recovery overhead as the system scales up (due to page limitation, we do not include this graph). Scalable behavior of anti-entropy loss recovery has been observed for group sizes up to 120, whereas nonhierarchical feedback control reveals increase in the overhead as group size scales up, and a rapid increase has been observed for group sizes larger than 100.
4 Conclusions and Future Work Anti-entropy distribution model provides eventual data consistency, scalability, and independence from communication topologies. It also transparently handles failures and can work with minimal connectivity. This study yields conclusions on scalability and robustness of pull-based anti-entropy distribution model used for reliable multicasting as a case study. Transit-stub dense topologies, and clusters connected by a bottleneck link are the representative scenarios analyzed. We show that anti-entropy loss recovery produces balanced overhead distribution among group receivers and it is scalable with an increase in group size, multicast message rate and system-wide noise rate. Comparison of the model with nonhierarchical feedback control has been conducted as well. As a further research direction, scalability and robustness of multicast loss recovery would be indispensable for multicast services over wireless networks as well [21]. Wireless multicast applications cover a wide range including mobile commerce, distance education, location-based services, and military command
Scalability and Robustness of Pull-Based Anti-entropy Distribution Model
941
and control. These applications would especially benefit from multicast communication support with efficient provision of reliability guarantees.
References 1.
2. 3. 4. 5. 6. 7. 8.
9. 10. 11. 12.
13. 14. 15. 16. 17.
18. 19. 20. 21.
Demers, A., Greene, D., Hauser, C., Irish, W., Larson, J., Shenker, S., Sturgis, H., Swinehart, D. and Terry, D., Epidemic Algorithms for Replicated Database Maintenance, Proc. of the Sixth ACM Symp. on Principles of Distributed Computing, 1–12p, 1987. Birrell, A.D., Levin, R., Needham, R.M. and Schroeder, M.D., Grapevine, An Exercise in Distributed Computing, Communications of the ACM, 25(4), 260–274p, 1982. Golding, R.A. and Taylor K., Group Membership in the Epidemic Style, Technical Report, UCSC-CRL-92-13, University of California at Santa Cruz, 1992. Ladin, R., Lishov, B., Shrira, L. and Ghemawat, S., Providing Availability using Lazy Replication, ACM Transactions on Computer Systems, 10(4), 360–391p, 1992. Guo, K., Scalable Message Stability Detection Protocols, Ph.D. dissertation, Cornell University Dept. of Computer Science, 1998. van Renesse, R., Minsky, Y. and Hayden, M., A Gossip-style Failure Detection Service, Proceedings of Middleware’98, 55–70p, 1998. Xiao, Z. and Birman, KP., 2001, A Randomized Error Recovery Algorithm for Reliable Multicast, Proceedings, IEEE Infocom 2001. van Renesse, R., Birman, K.P., Vogels, W., Astrolabe: A Robust and Scalable Technology for Distributed System Monitoring, Management, and Data Mining, ACM Transactions on Computer Systems, Vol. 21, No. 2, pp. 164–206, May 2003. Bailey, N.T.J., The Mathematical Theory of Infectious Diseases and its Applications, second edition, Hafner Press, 1975. Tanenbaum, A.S. and van Steen, M., Distributed Systems: Principles and Paradigms, Prentice Hall, ISBN 0-13-088893-1, 2002. Birman, K.P., Hayden, M., Ozkasap, O., Xiao, Z., Budiu, M. and Minsky, Y., Bimodal Multicast, ACM Transactions on Computer Systems, 17(2), 41–88p., May 1999. Floyd, S., Jacobson, V., Liu, C., McCanne, S. and Zhang, L., A Reliable Multicast Framework for Light-weight Sessions and Application Level Framing, IEEE/ACM Transactions on Networking, 5(6), 784–803p., 1997. Gemmell, J., Montgomery, T., Speakman, T., Bhaskar, N., and Crowcroft, J., The PGM Reliable Multicast Protocol, IEEE Network, Jan/Feb 2003. Ozkasap, O., Large-Scale Behavior of End-to-end Epidemic Message Loss Recovery, QofIS2002, Zurich, LNCS, Springer Verlag, Vol.: 2511, pp. 25–35, October 2002. Lidl, K., Osborne, J. and Malcome, J., Drinking from the Firehose: Multicast USENET News, USENIX Winter 1994, 33–45p., 1994. Bajaj, S., Breslau, L., Estrin, D., et al., Improving Simulation for Network Research, USC Computer Science Dept. Technical Report 99–702, 1999. Ozkasap, O., Scalability, Throughput Stability and Efficient Buffering in Reliable Multicast Protocols, Technical Report, TR2000-1827, Dept. Comp. Sci., Cornell University, 2000. Calvert, K., Doar, M., and Zegura, E.W., Modeling Internet Topology. IEEE Communications Magazine, June 1997. Liu, C., Error Recovery in Scalable Reliable Multicast, Ph.D. dissertation, University of Southern California, 1997. Lucas, M., Efficient Data Distribution in Large-Scale Multicast Networks, Ph.D. dissertation, Dept. of Computer Science, University of Virginia, 1998. Varshney, U., Multicast over Wireless Networks, Communications of the ACM, vol.45, no.12, pp.31–37, December 2002.
Extended Partial Path Heuristic for Real-Time Staging in Oversubscribed Networks 0RKDPPHG(OWD\HE$WDNDQ'R÷DQDQG)VXQg]JQHU 1
Department of Electrical Engineering, The Ohio State University, 2015 Neil Avenue, Columbus, Ohio 43210, USA ^HOWD\HER]JXQHU`#HHHQJRKLRVWDWHHGX 2 Department of Electrical and Electronics Engineering, Anadolu University, 26470 Eskisehir, Turkey DWGRJDQ#DQDGROXHGXWU Abstract. In this paper we propose the use of the PPH as a base for an offline schedule method to detect unsatisfiable requests in a staging system. This property allows a reduction in the overhead of transferring data-items with deadlines in a real-time environment, through paths rearrangement. The EPP heuristic allows early detection of eventually, non-satisfied, requests based on their urgencies. Early detection and dropping of these requests also reduces the latencies of the satisfied requests. We show analytically and experimentally that the performance of the EPP heuristic is at least equivalent to the previous PPH proposed for data-staging problem in real-time environments.
1
Introduction
In today’s world of advanced technology, numerous amounts of scientific distributed applications are emerging as critical part of the community needs. The proliferation of the Internet, the development of wireless and satellite systems and networks, the advent of high performance computing and the advancement in measurements and imaging instrumentations have all created the opportunity for the development of modern distributed services in several fields. These advancements have also created and inspired new approaches to support the transportation, management, access, and analysis of large-scale data objects by potentially large number of end points. The Data Staging problem, for example, has gained its importance recently with the emergence of a class of new applications by which relatively large scale blocks of information is to be transferred from one location to another across a network of limited resources in real-time fashion. The data staging problem requires the design of efficient methods and schemes for transferring data blocks or objects to different destinations to mitigate the effect of consumption of network resources in case of scarcity of these resources. The real-time aspects of the distributed applications utilizing a data staging system can further complicate the data staging problem. The challenge, then, becomes meeting requests deadlines in scarce resource environments.
$ 'UL − W @ − 3O UL W , where 'UL is the deadline of the request and 3O UL W is the path length of request UL at time W ≥ which is the total time elapsed from the start of the data staging in the network. Definition 2. (the total set) The set S = { UL }, is the total set of all requests present during a period >W WQ @ ; called the scheduling period. We assume set S does not change during this period. Definition 3. (schedule step): We say a schedule step of length >WD WE @ is performed
on UL iff ∃ UL ∈ S (the total set) such that UL is transmitted for one hop of a length WE − WD >0 along the path 3O UL WD while no other request in S is transmitted. The times WD and WE are the start time of data transfer and the finish time of data transfer respectively.
Definition 4. (satisfiability) A request UL ∈ S is satisfied by a schedule step iff ∃ a schedule step that is performed on UL such that 3O UL W = 0 at time W and 8 J UL W ≥ 0. The request is dropped otherwise, i.e. 3O UL W >0 and 8 J UL W WD WE @ , iff 8 J UL WD = 8 J UM WD and 3O UL W , 3O UM W >0 ∀
W ∈ >WD WE @ . The requests are said to be zero competing iff they are competing
with zero urgency values. A group of competing requests is formed if any two that belong to the group are themselves competing. Definition 7. (selection process) A random selection is performed on a group of competing requests iff a schedule step is performed randomly on one of them. Lemma 1. (urgency reduction) Iff a schedule step of length >WD WE @ is performed on request UL , then, 8 J UL WD = 8 J UL WE and 8 J UM WD > 8 J UM WE , ∀
and UL ≠
UM .
Lemma 2. (dropping condition): A request
UL
UL , U M ∈ S
is dropped iff a schedule step is
performed on request UX ∈ S, UX ≠ UL and UL is zero competing with at least one request U M
UL
Lemma 3. (competing steps): Two requests
and
UM
with urgencies
8 J UL W > 8 J UM W at time W = W , if reach a competing point at W S > W then number of schedule steps performed on
8 J UL W − 8 J UM W WE − W D
UM
prior
to
W S is
given
by
, where WE − W D is the size of the schedule step.
Lemma 4. (maximum reduction): A requests UL that is competing at time W S with request most
U M , if reaches a zero competing point with it at time WX ≥ W S , will have at
8 J UL W S
schedule steps performed on either request prior to WX , where
WE − W D WE − WD is the size of the schedule steps.
Lemma 5. (path incrementing) If request
UL
is zero competing in a schedule
performed on a total set S with given path lengths then, if the path of
UL
is
0(OWD\HE$'R÷DQDQG)g]JQHU
incremented during the scheduling period UL will also reach a zero competing point during the scheduling period. Theorem 1. (general case): For a stable system of data staging (i.e., stable network and non-changing request conditions), the number of satisfied requests in a schedule generated by the EPP is larger than the number of satisfied requests in PPH in general. Corollary 1. (special case) For a stable system of data staging, if no path change is encountered (e.g. the case of tree networks) and if non-random selection process is adopted for the schedule, then a request that is satisfied by the PPH is also satisfied by the EPP.
6
Simulation Results
The simulation was performed on an environment of 30 nodes forming an irregular network. Each node represents a machine with fixed capacity of 1unit of storage and each is connected to at most three other nodes. The delay across the links was uniformly set to 20 units of time and. The experiment includes running random generated requests on random topologies. The results shown in Table 1 and Table 2 represent the behavior of both PPH and EPP in this environment. It is clear from the tables that EPP was always superior to PPH. Table 1 shows the performance when the deadlines of all requests are set to a constant 5000 unit of time (250 times the link length) and the load of the system is altered from 50 to 600 requests. It is clear that at heavy load EPP outperforms PPH by a big margin due to its ability to schedule on long extensions. Table 2 shows the deadline response for both EPP and PPH fixing the load to 100 requests and altering the uniform deadlines of the requests. The EPP also demonstrated superiority in such environment with the given inputs.
Table 1. The performance of the EPP and the PPH when deadline is fixed for all requests at 5000 time unit (250 times the link delay) Load PPH Satisfiability (%) EPP Satisfiability (%)
50 100 100
100 94 100
200 42 57
300 27 39
400
600 0.167 19.66
Table 2. The performance of the EPP and the PPH when deadline is fixed for all requests at 5000 time unit (250 times the link delay) Deadline PPH Satisfiability (%) EPP Satisfiability (%)
1000 0 28
1500 17 39
2000 33 53
2500 44 63
3000 49 74
4000 76 95
5000 94 100
15000 100 100
Extended Partial Path Heuristic for Real-Time Staging in Oversubscribed Networks
7
951
Conclusions
,W ZDV GHPRQVWUDWHG LQ WKLV SDSHU ERWK DQDO\WLFDOO\ DQG H[SHULPHQWDOO\ WKDW XQGHU FHUWDLQFRQGLWLRQVWKH33+LVDORZHUERXQGRISHUIRUPDQFHIRUWKH(33:LWKIHZHU LQYRFDWLRQV WR 'LMNVWUD¶V (33 ZLOO LQIOLFW OHVV RYHUKHDG WR WKH VFKHGXOHU DQG ZLOO EH DEOHWRDFKLHYHKLJKHUSHUIRUPDQFH%HFDXVHWKH33+DWWHPSWVFRQWLQXRXVO\WRORRN IRU RWKHU SDWKV IRU UHTXHVWV ZKLFK PD\ VDWLVI\ D VSHFLILF UHTXHVW WKLV VDWLVILDELOLW\ ZLOOEHDWWKHH[SHQVHRIRWKHUUHTXHVWV7KH (33 LV D ORZHURYHUKHDG KHXULVWLF WKDW HIILFLHQWO\KDQGOHUHTXHVWVZLWKHTXDOSULRULWLHV,IWKHJRDOLVWRPD[LPL]HDZHLJKWHG VXPRISULRULWLHVWKHQ(33PD\KDYHDORZHUSHUIRUPDQFHWKDQ33+)RUWKLVFDVHWKH (33PXVWEHFRPELQHGZLWKDPHFKDQLVPWKDWLPSURYHVWKHRYHUDOOSHUIRUPDQFHE\ PD[LPL]LQJ ERWK WKH QXPEHU RI VDWLVILHG UHTXHVWV DQG WKH ZHLJKWHG VXP RI WKHLU SULRULWLHV7KLVPHFKDQLVPLVSURSRVHGLQ>@DQGUHIHUUHGWRDVFRQFXUUHQWVFKHGXOLQJ 7KH 33+ LV H[SHFWHG WR KDYH D EHWWHU SHUIRUPDQFH LQ D QRQVWDEOH HQYLURQPHQW E\ ZKLFK G\QDPLFDOO\ FKDQJLQJ QHWZRUN DQG FRQWLQXRXVO\ LQFRPLQJ UHTXHVWV DUH DVVXPHG 7KH (33 FRPSOH[LW\ PXVW EH IXUWKHU VWXGLHG LQ FDVHRI D ODUJH QXPEHU RI UHTXHVWVDWZKLFKWKHRIIOLQHVFKHGXOHLVRIDKLJKFRPSOH[LW\
References 7KH\V07%HFN16LHJHO+--XUF]\N0$0DWKHPDWLFDO0RGHODQG6FKHGXOLQJ +HXULVWLFV IRU 6DWLVI\LQJ 3ULRULWL]HG 'DWD 5HTXHVWV LQ DQ 2YHUVXEVFULEHG &RPPXQLFDWLRQ 1HWZRUN,(((7UDQV3DUDOOHODQG'LVWULEXWHG6\VWHPV9RO1R -RQHV 3 & /RZH 7 - 0XOOHU * ;X 1 @ZKLFKZDVSURSRVHGIRUIRUPDOL]LQJXVHFDVHV 7KH VWUXFWXUH RI 6031 LV GHYLVHG WR UHIOHFW WKH RUJDQL]DWLRQ RI WKH IOH[LEOH VRIW ZDUHV\VWHPZKLFKLVVWDWHGDVDFROOHFWLRQRIVHUYLFHV$6031FRQVLVWVRIDVHWRI ORJLFDOO\LQWHJUDWHGVHUYLFHQHWV61L HDFKRIZKLFKUHSUHVHQWVDQLQGLYLGXDOVHUYLFH PRGHOZLWK3HWULQHWV'HSHQGHQFLHVDPRQJ61VFDQEHUHSUHVHQWHGE\VKDUHGSODFHV DQGVKDUHGWUDQVLWLRQV7KHIRUPDOGHILQLWLRQVRI61DQG6031DUHJLYHQDVIROORZV Definition 1 For a character set Σ, a service net (SN) is a 6-tuple SN = (P, T, F, G, L, M0), where P, T, and M0 are the same as those of P/T Net [5], F represents the set of arc information and arc descriptions which indicate the amount and type of a token flow, G is the set of guard expressions in transitions, L : P ∪ T →∑+ is a label function that associates a distinct label taken from strings (∑+) with each place and transition of P and T, respectively. Definition 2 Service-oriented Modular Petri nets (SMPN) are defined as SMPN = { SNi | i = 1…n} satisfying the following conditions: Pi, Ti, and Fi should be disjoint for SNi, The same label should not be used for both places and transitions, M0 for shared places should be the same. ,Q 6031 PRGHO HDFK VHUYLFH LV GHVFULEHG LQ DQ LQGHSHQGHQW PRGHO 6LQFH HDFK VHUYLFHPRGHOKDVDVLQJOHRUPXOWLSOHFRQWUROWKUHDGVDVHUYLFHPRGHOLVGHVFULEHGDV DQLWHUDWLYHIRUPFigure 2 shows a SMPN model of the basic call processing (BCP) model [6] is composed of three service nets. Figure 2 (a) and (b) show service nets of originating and terminating parties, respectively. Figure 2(c) describes a process of connection establishment between an originating party and a terminating party. ,Q RUGHU WR SHUIRUP D IXQFWLRQDOLW\ RI D VHUYLFH UHVRXUFHV VXFK DV KDUGZDUH VRIWZDUH DQGGHYLFHVDUHXWLOL]HG%HKDYLRUVRIDVHUYLFHDUHFORVHO\UHODWHGZLWKWKHUHVRXUFHV RI WKH V\VWHP :KHQ UHVRXUFHV DUH VKDUHG ZLWK VHYHUDO VHUYLFHV D VHUYLFH PD\ QRW RFFXUGXHWRWKHODFNRIDYDLODEOHUHVRXUFHVIn Figure 2, Resource place represents
Modeling and Analysis of Service Interactions in Service-Oriented Software
1047
the resource of a telephone system, which has a single capacity for processing a call at a time. 3,&
3,& RW
RW
5HVRXUFH
'3
RW
5HVRXUFH
'3
GW
3,& RW
3,& RW
GW GW
RW RW
'3
RW
GW
RW
1RW5HVRXUFH
3,&
GW
RW
'3
GW
GW
'3
RW
GW
5HVRXUFH
'3
'3
GW RW
'3
GW
GW
GW RW
3,&
GW
3,&
RW
'3
RW
GW
GW
RW
'3
'3 GW
'3
RW
RW
GW
3,&
RW
GW
'3
RW
RW
RW
GW
3,&
RW
'3
RW
GW
5HVRXUFH
RW
5HVRXUFH
GW
'3
GW
'3
GW RW
3,&
3,&
RW
RW
'3
GW
RW
GW
GW
RW
D DVHUYLFHQHWRIDQRULJLQDWLQJSDUW\
E DVHUYLFHQHWRIDWHUPLQDWLQJSDUW\
F LQWHUDFWLRQVRIWZRSDUWLHV
Fig. 2. The SMPN model of the BCP model
4 Service Interaction Analysis The goals of service interaction analysis are to check consistency and completeness of each service and to check abnormal interactions among services in off-line or online manner. Service interaction analysis is performed when introducing a new service into a system as well as when customers dynamically change or customize their services on their demands. Before checking service interaction analysis, service models are composed with the basic system model. Service models are merged by combining shared places (or shared transitions) with the same label into a composed place (or a composed transition). In the composed model, a reachability graph is generated for checking abnormal situations such as deadlock, livelock, unreachable transitions, and so on. Service interaction analysis is mainly composed of consistency analysis for a service net and conflict analysis among service nets. For checking consistency of a single service, a service net is composed to the basic model. In the reachability analysis of the composed model, there are following types of inconsistency.
1048
W.J. Lee
z deadlock : the behavior of a service may cause the basic services to be blocked. In worst case, a system does not work any more. Deadlock situations can be easily found by checking a dead marking. z livelock : due to the abnormal interaction with the basic service, a system may fall into a trap or an infinite loop. Livelock situations can be found by checking a loop without containing the initial marking. Once in a while, livelock may be intentionally added into a service net. z meaningless service : the functionality of a service will not occur due to unintended interaction. This service has no effect in the behaviors of a system. Meaningless service situations can be found by checking the reachability of transitions contained in a service net. In order to check conflicts among services, two service nets are combined into a composed model. In the reachability graph of the composed model, conflict types among services may be categorized as follows. z non-deterministic situation : at some situation, two or more services are simultaneously enabled. Non-deterministic situation can be found by checking whether there exists a marking that has two or more branches. z deadlock or livelock : due to unintended interactions among services, abnormal situations such as deadlock and livelock can occur. z inclusion of a service : one service can not occur due to prevention of the other service. This situation can found by checking the reachability of all transitions contained service nets. '3 > U
'3
> U LQ@ [!
[!
RW
RFV
1RW2&6
[!
RFV
U[!
U[! RFV
[!
'3
LQ@ RFV
U[! ,I[LQ/LVW U LQ /LVW!
> DEG[ @
[!
RW
DEV! > DEGDEV @
U! DEG
,I DE V R UJ ! LQ$OO U
RUJ
HOVH
DEV! DEG
DEV!
U
DE V
$OO!
$%'
2&6 D 2ULJLQDWLQJ&DOO6FUHHQ6HUYLFH
E $EEUHYLDWHG'LDOLQJ6HUYLFH
Fig. 3. Service nets of OCS and ABD services
5 Case Study In intelligent networks, telecommunication systems are enhanced by a large and steadily growing number of supplementary services such as CF (Call Forwarding), CW (Call Waiting), 2&6 2ULJLQDWLQJ &DOO 6FUHHQ $%' $EEUHYLDWHG 'LDOLQJ 7&67HUPLQDWLQJ&DOO6FUHHQ DQGVRRQ,QWKLVVHFWLRQZHGHVFULEH2&6DQG$%' VHUYLFHV LQ 6031 DQG FKHFN DEQRUPDO LQWHUDFWLRQV DPRQJ WKHP 6LQFH HDFK WHOH
Modeling and Analysis of Service Interactions in Service-Oriented Software
1049
FRPPXQLFDWLRQVHUYLFHLVSHUIRUPHGLWVEHKDYLRUE\LQWHUDFWLQJZLWK%&3PRGHOYLD '3QRGHVLWLVLQGHSHQGHQWO\GHVFULEHGDVDVHUYLFHQHWE\VKDULQJ'3SODFHVRIWKH %&3PRGHO)LJXUHVKRZVWKHH[DPSOHVRIGHVFULELQJ2&6DQG$%'VHUYLFHV$Q 2&6VHUYLFHFKHFNVZKHWKHUWKHVFUHHQLQJOLVWLQFOXGHVDGLDOHGQXPEHURUQRW$QG $%'VHUYLFHWUDQVODWHVDQDEEUHYLDWHGQXPEHULQWRWKHUHJLVWHUHGSKRQHQXPEHULILW LVUHJLVWHUHGSUHYLRXVO\2WKHUZLVHLWUHWXUQVWKHLQSXWQXPEHUYLD'3 The consistency and completeness analysis for general properties can be more efficiently applied to the Petri net slices [7] than SMPN model. Since a SMPN model has shared places that are similar to global variables, it is difficult to analyze a SMPN model in a modular approach. The Petri net slices have only shared transitions, which are obtained by restructuring the SMPN model in the perspective of control threads. Figure 4 shows the example of Petri net Slices generated from the composed model of the originating party of BCP, OCS, and ABD services. For simplicity, we omitted the guard conditions in Figure 4. Each slice has its own control token and is performed its behavior by synchronizing with other slices via shared transitions. 3,& GW
RW RW
RFV RW
'3
RW
DEG
DEG
RFV
2&6
3,&
3,&
D 6OLFH
RFV
RW
DEG
'3
RW
'3
RFV RW
RW
RFV RW
RW RFV
1RW2&6
3,&
RFV
RW
'3
'3
RW
E 6OLFH
RW
RW
RW
'3
RW
RW RW
1RW5HVRXUFH
GW
'3
3,& RW
'3
GW RW
GW
5HVRXUFH
RW
'3
GW
GW
RW
RW
RW
RW
'3
RW
RW
GW
3,& RW
F 6OLFH
G 6OLF H
Fig. 4. Petri nets Slices of the SMPN model shown in Fig. 2
A SMPN model is said to be inconsistent if there exists a set of transitions that are never enabled. This type of flaws is analogous to an unreachable code in programs. Since services are expected to reflect genuine needs, it is reasonable to require that SMPNs do not contain transitions that are never enabled. Another type of inconsis-
1050
W.J. Lee
tency occurs when there are deadlocks. These inconsistencies can be detectable by generating a compositional reachability graph (RG) of Petri net slices. In compositional analysis, after each slice is locally analyzed by generating its local RG, a pair of slices are composed and analyzed incrementally. For example, in local analysis of slice 4, we can find a situation which ocs1, abd1, and ot7 transitions can be enabled in non-deterministic order, as shown in the shaded box of Figure 4. That is, since OCS service and ABD service can be performed in arbitrary order, it is possible that the OCS function has no effect on the abbreviated form of a screening number.
6 Conclusion and Future Work :H KDYH GHVFULEHG WKH VHUYLFHRULHQWHG VRIWZDUH DSSURDFK DQG SURSRVHG D JHQHUDO IUDPHZRUN IRU PRGHOLQJ DQG DQDO\]LQJ VHUYLFHV DQG VHUYLFH LQWHUDFWLRQV E\ XVLQJ D PRGXODU 3HWUL QHWV ,Q WKH SUHVHQW FLUFXPVWDQFH WKDW WKH VHUYLFHRULHQWHG FRQFHSW LV JUDGXDOO\DFFHSWHGDQGDSSOLHGLQWKH,QWHUQHWEDVHGDSSOLFDWLRQVVHUYLFHLQWHUDFWLRQ SUREOHPV ZKLFK PD\ EH FRQVLGHUHG DV RQH RI WKH KDUGHVW REVWDFOHV IRU DSSO\LQJ VHUYLFHRULHQWHGDSSURDFKWRUHDOSUREOHPVDUHZLGHO\DQGURXJKO\GLVFXVVHGHYHQLI LQWKHORJLFDOYLHZSRLQWV As future work, we will develop an analyzing environment for service-oriented software, which is composed of detailed analysis methods and its supporting tools. And it also includes an analysis method for dynamic service interactions.
References K. Bennett, P. Layzell, D. Budgen, P. Brereton, L. Macaulay, M. Munro: 6HUYLFHEDVHG VRIWZDUHWKHIXWXUHIRUIOH[LEOHVRIWZDUH 7th APSEC(2000) 214–221 2. M. Aoyama, S. Weerawarana, H. Maruyama, C. Szyperski, K. Sullivan, D. Lea: :HE VHUYLFHVHQJLQHHULQJSURPLVHVDQGFKDOOHQJHV24rd ICSE(2002) 647–648 3. Dirk O. Keck, Paul J. Kuehn: The Feature and Service Interaction Problem in Telecommunications Systems: A Survey. IEEE Trans. on Soft. Eng.(1998) 4. Woo Jin Lee, Sung Deok Cha, Yong Rae Kwon: Integration and Analysis of Use Cases Using Modular Petri Nets in Requirements Engineering. IEEE Trans. on Soft. Eng.(1998) 1115–1130 5. W. Reisig, Petri nets : An Introduction. Springer-Verlag(1985) 6. ITU-T, Intelligent Network CS-1 Refinements Recommendations. Q.1211-Q1215 Geneva(1995) 7. W.J. Lee, S.D. Cha, H.N. Kim, Y.R. Kwon: A Slicing-based approach to enhance Petri Net Reachability Analysis. Journal of Research and Practice in Information Technology(2000) 131–143 8. G. Piccinelli, M. Salle, C. Zirpins: 6HUYLFHRULHQWHGPRGHOLQJIRUHEXVLQHVVDSSOLFDWLRQV FRPSRQHQWVTenth IEEE International Workshops on WET ICE( 2001) 12–17 9. P. Layzell: Addressing the software evolution crisis through a service-oriented view of software: a roadmap for software engineering and maintenance research. IEEE International Conference on Software Maintenance (2001) 10. D. Harel: Statecharts : A Visual Formalism for Complex Systems. Science of Computer Programming(1987) 1.
Designing Reusable Web-Applications by Employing Enterprise Frameworks Marius Dragomiroiu1, Robert Gyorodi2, Ioan Salomie1, and Cornelia Gyorodi2 2
1 University of Limerick, ECE Department, Limerick, Ireland University of Oradea, Computer Science Department, Oradea, Romania ^PDULXVGUDJRPLURLXUREHUWJ\RURGL`#XOLH
Abstract. Given the complexity of web-applications, the associated development process should focus on improving their reusability and flexibility. Enterprise Frameworks encapsulate reusable, tailorable software solutions as a collection of collaborative components, assuring integration of new components and reducing the complexity of the systems. The paper presents a development process of enterprise frameworks for web-applications and proposes a compositional design pattern targeting to increase framework flexibility and reusability.
1
Introduction
Building web-applications is a complex and time–consuming process. Webapplications typically evolve over time, by domain extensions or by an evolution in the quality of provided services. Therefore, the development process should focus on improving reusability and flexibility of the application in order to reduce the resources and time spent for it’s building, maintaining or extending. A solution to meet these concerns is carried out by employing software frameworks in the development process. Enterprise Frameworks ensure reuse not only for building blocks but also for entire system architectures including their design [1]. Unfortunately, due to the complexity of the development process, Enterprise Frameworks do not always fulfil their promises. Therefore, this paper proposes an approach in developing enterprise frameworks for web-applications aiming to improve their reusability and flexibility. The remainder of the paper is structured as follows: section two presents the application domain together with architectural analysis, section three describes the development approach of the enterprise framework, and section four summarizes the benefits of the proposed enterprise framework design. The paper concludes in section five.
2
Domain Analysis
Domain analysis identifies domain main features by capturing and organizing the associated domain knowledge. For architectural design, decomposing and organizing the domain functionalities in layers is an effective mean for managing complexity [2]. The domain layering is based on both the architectural and functional decomposition. $ 80
(msg: “IDS265-Wep cgi cgitest”; content: “cgitest.exe|0d 0a|user”; nocase; flags: “PA”;offset:4;)
HEADER HEADER: alert – warning level protocol – tcp/udp/icmp s_port – port number/any direction operator - < / > d_port – port number/any
OPTION OPTION: msg – warning message contents – data content dsize – data size offset – data start point depth – data end point nocase – no identification of upper/lower character flags – TCP flags
Fig. 3. Syntax of Intrusion Pattern
In general, the process sequence of intrusion analysis and detection is as the following: receiving packet, comparison of protocol, comparison of port, comparison of flag and other minor options, and the comparison of packet contents [7]. In this process, the most time-exacting phase (spending over 90% of the total) is the ’packet contents comparison’ process. In our implemented system, NIDS compares service port information from the onset. This means that the sequence of intrusion analysis is reduced by omitting the flag and packet data(data/payload) comparison sequences, thus requiring less computer resources when compared to a search of all 3+4+5+6+7 services classes. Therefore, when intrusion pattern database is used with patterns that are efficient to a specific OS, efficiency in detecting intrusions will increase. This is due to a decrease in computer operations required for matching the patterns. NIDS through this reduction is able to efficiently monitor and analyze more packets
Performance Improvement Scheme of NIDS
1103
Service Class 1
Common patterns Valid to all OS
Service Class
OS Class
2
SunOS patterns
Common Patterns valid to all OS
SunOS patterns
3
HP-UX patterns
4
AIX patterns
5
DG/UX patterns
6
LINUX patterns
7
Windows patterns
SunOS pattern in other services
Separating
Legacy IDS Pattern DB
Separated Pattern DB
Fig. 4. Benefits of Separated Pattern Database
4
Experimental Results
A series of test scenarios were set in place to accurately test the efficiency of the intrusion sequence process for searching the exact pattern within the intrusion pattern database. All patterns used in this experiment were collected from actual system and network vulnerability located in various public Web sites [5-8]. 4.1
Objects of Test
Three pattern databases were loaded in memory to proceed in the test. All the conditions concerning objects are identical except for the pattern databases loaded in the memory. The following thing are the pattern databases that were loaded in the computer memory. – Linear Model: linear shaped pattern database without separating pattern database – Service Model: Pattern database divided by service – OS Model: Pattern database classified by service and OS 4.2
Attack Sequence Processing Efficiency Test
Purpose: Measure each object’s efficiency concerning attack sequence amongst pattern database Methods: Produce the packet containing attack signature with implemented attack program. Let the packet flow in the network for one minute. Measure the number of transferred packets and the number of packets that each object detected as an intrusion. 112 patterns used in the test are equally divided as pattern DB l Repeat the test 10 times, calculate average number.
1104
J.-M. Kim et al.
Test Environment: Two Workstations and two PCs will be connected via the Intranet, which is a component of Internet. IDS and attack programs will be located in each Workstation at the time that the packet generator is loaded into located in the PC. The packet generator will be used for the first scenario and the attack program for the second scenario.
Table 3. Pattern Database used in experiment Model
DB
Number of patterns
Linear Model
Single DB
112
HTTP DB
30
FTP DB
27
SMTP DB
26
Service Model Etc
29
HTTP
15
FTP
15
SMTP
16
Service Etc
19
UNIX
12
OS Model
4.3
Linux
12
Windows
11
OS Etc
12
Results and Analysis
Test results are shown in Fig. 5. In the ’attack sequence’ process efficiency test, the OS model detected the highest number of packets. The OS model is 3.7 times more efficient than the linear model and 30.8% more efficient than the service model. This proves the efficiency of the model. However, the number didn’t reach the estimate of 35%, that was calculated using the separated pattern database model effect. This is due to the insufficient number of intrusion patterns, expanding overhead of using message queue. Accurate analysis in attack patterns and separating pattern database effectively will bring out more satisfactory results.
5
Conclusion
In this paper we has presented an improved method to detect intrusion among specific hardware platforms. While this paper used patterns classified using only the OS information of the target host; other important factors such as variety /
Performance Improvement Scheme of NIDS
1105
Detection Numbers
50,000 •The number of
40,000
transmitted packets = 130,000
30,000 20,000 10,000
Linear Model
Service Model
OS Model
Fig. 5. Experimental Results
different versions of OS, application versions, and patches, should be considered relevant. Obtaining further information about monitoring hosts and classifying intrusion pattern database’s accurately would be highly constructive. It may be stated that by arranging intrusion patterns differently in consideration of network traffic and by hacking trends, there will be an increase in the efficiency of IDS. With the goal of efficiency and security, this area will continue to be tackled in research initiatives, throughout the world.
References 1. H. Debar, M. Dacier, and Andreas Wespi, Towards a Taxonomy of Intrusion Detection Systems, IBM Research Division, Zurich Research Lab., Research Report RZ 3030, June 1998. 2. T.F. Lunt, ”Automated Audit Trail Analysis and Intrusion Detection: A Survey”, Proc. of 11th National Computer Security Conf., 1998. 3. B.Mukherjee, T.L. Heberlein, and K.N.Kevitt, ”Network Intrusion Detection”, IEEE Network, 8(3):26-41, May/June 1994 2001 4. R. Heady, G.Luger, A.Maccabe, and M.Servilla, ”The Architecture of a Network Level Intrusion Detection System”, Technical Report, Computer Science Department, University of New Mexico, August 1990. 5. S. Kumar, Classification and Detection of Computer Intrusions, Purdue University, Aug, 1995 6. COAST, ”Intrusion Detection”, http://www.cs.purdue.edu/coast/intrusion-detection/welcome.html 7. SNORT, ”The Open Source Network Intrusion Detection System” http://www.snort.org/ 8. WHITEHATS, ”Whitehats Network Security Resource”, http://www.whitehats.com/
Author Index
Acan, Adnan 968 Afsarmanesh, Hamideh 75 Aggarwal, J.K. 405, 430 Ahn, Byung Ha 155, 364 Ak¸calı, Elif 163 Akın, H. Levent 521, 529 Al-Hamdani, Abdullah 9 Alatan, A. Aydın 474 Alhajj, Reda 308 Alpaydın, Ethem 521 Alptekin, Ozan 885 Altılar, D. Turgay 731 ´ Alvarez-Guti´ errez, Dar´ıo 228 Anagnostopoulos, Christos 35 Anagnostopoulos, Ioannis 35 Andreasen, Troels 268 Anuk, Erhan 885 Argibay-Losada, Pablo 892 Ashourian, Mohsen 659 ¨ Aslıhak, Umit 836 Atalay, Reng¨ ul C ¸ etin 316, 611 Atalay, Volkan 505, 611 Aykanat, Cevdet 457, 926 Bae, Yongeun 763 Bah¸ceci, Erkin 900 Barenco Abbas, Cl´ audia Jacy 786 Basci, Faysal 115 Baykal, Nazife 794 Becerikli, Yasar 601 Beigy, Hamid 755, 960 Benslimane, Abderrahim 643 Besharati, Farhad 537 Beydeda, Sami 1000 Bicakci, Kemal 794 Bilgen, Semih 771 Bontchev, Boyan 356 Braake, Hubert A.B. te 324 Brzezi´ nski, Jerzy 916 Bulskov, Henrik 268 Buzluca, Feza 836 C, John Felix 747 Callens, Bert 204, 260 Cambazoglu, B. Barla 457
C ¸ apar, Abdulkerim 447 Capilla, Rafael 1035 Carchiolo, Vincenza 19 Celikel, Ebru 187 Cera, Christopher D. 397 C ¸ etin, Atılım 465 Cho, Beom-Joon 908 Cho, Jun-Ki 667 Cho, Kyungwoon 276 Cho, Seong-Yun 482 Cho, You-Ze 877 Choi, Jongmoo 413 Choi, Kang-Sun 667 Choi, Munkee 155, 364 Chountas, Panagiotis 123 Chung, Bong-Young 691 Chung, Ilyong 763 Cinsdikici, Muhammed 439 Dalkılı¸c, Mehmet Emin 187, 802 Dayıoˇ glu, Burak 885 Demir¨ oz, Bet¨ ul 952 Dhavachelvan, P. 992 Dikmen, Onur 521 Dimitrios, Vergados 35 Din¸cer, B. Taner 244 Dindaroˇ glu, M. Serdar 474 Doˇ gan, Atakan 942 Dragomiroiu, Marius 1051 Duin, Robert P.W. 505 Dursun, Taner 819 Duygulu, Pınar 513 Dynda, Vladim´ır 67 Ekambaram, Anand 196 Ekinci, Murat 421 Eltayeb, Mohammed 942 Erciye¸s, Kayhan 802 Erdogan, Nadia 284, 348 Ermi¸s, Umut 885 Ersak, Aydın 474 Ert¨ urk, Sarp 497 Etzold, Thure 252 Fern´ andez-Veiga, Manuel
651, 892
1108
Author Index
Garc´ıa Villalba, Luis Javier 786 Garibay, Ivan I. 584 Garibay, Ozlem O. 584 Gayo-Avello, Daniel 228 Gayo-Avello, Jos´e 228 Gedikli, Ey¨ up 421 Gelenbe, Erol 1 George, Roy 984 G¨ okı¸sık, Demeter 771 G¨ okmen, Muhittin 381, 447 G¨ oktoˇ gan, Ali Haydar 576 G¨ okt¨ urk, Erek 619 Gorur, Abdulkadir 300 Gruhn, Volker 1000 G¨ ull¨ u, M. Kemal 497 G¨ und¨ uz, S ¸ ule 332 Gurgen, Fikret S. 553 Gursoy, A. 316 Gusak, Oleg 860 Gyorodi, Cornelia 1051 Gyorodi, Robert 1051 Hallez, Axel 260 Han, JungHyun 397 Han, Su-Young 482 Helmer, Sven 220 Heo, Yeong-Nam 51 Hertzberger, L.O. (Bob) 75 Ho, Yo-Sung 659, 675, 683, 699 Huh, Chang-Won 691 Hwang, Min-Cheol 667 Hyusein, Byurhan 236 In, Yongho 389 I¸sikyildiz, G¨ ur¸ce 576 Jalili-Kharaajoo, Mahdi 537 Jang, Kyunghun 627, 707 Jeon, Byeungwoo 723 Jeong, Jae-Hwan 707 Jeong, Wook-Hyun 675 Ji, Kwang-Il 707 Jo, Geun-Sik 91, 99 Jung, Jason J. 91, 99 Jung, Kyung-Yong 91 Jung, Sang-Hwan 877 Kahraman, Fatih 381 Kaletas, Ersin Cem 75 Kalinli, Adem 568
Kanak, Alper 852 ˙ Kanbur, Inan 885 Kang, Chiwon 1090 Kang, Minho 155, 364 Kantarcı, Aylin 635 Kaplan, Kemal 529 Karaoˇ glan, Bahar 244 Kaya, Mehmet 308 Kayafas, Eleftherios 35 Kaymak, Uzay 324 Kılı¸c, H¨ urevren 107 ¨ Kılıc, Ozlem 447 Kim, Bo Gwan 155, 364 Kim, Do-Hyeon 877 Kim, Eun-ser 1069 Kim, Jae-Myung 1098 Kim, Jae-sung 1084 Kim, Jong-Hyuk 576 Kim, Jong-Seob 1098 Kim, Jong-Seok 51 Kim, Jongsun 413 Kim, JungJa 389 Kim, Kuinam J. 1076, 1098 Kim, Meejoung 691 Kim, Sang-ho 1069, 1084 Kim, Seongyeol 763 Kim, Seungcheon 779 Kim, Soo-Won 691 Kim, Sungnam 83 Kim, Tae-Chan 691 Kim, Taeseong 397 Kim, Tai-hoon 1069, 1084 Kim, Won Hee 155, 364 Kim, Yong-Guk 490 Kim, Yoon 627 Kim, Youngho 389 Klopotek, Mieczyslaw A. 139 Knappe, Rasmus 268 Ko, Sung-Jea 627, 667, 707 Kocak, Taskin 115 Kocatas, A. 316 Kodogiannis, Vassilis 123 K¨ ose, Hatice 529 Koh, Kern 276 Korkmaz, Emin Erkan 561 Kurt, Binnur 381 Kuruoglu, Mehmet Ercan 131 Kwon, Taekyoung 869 Lee, Dong Chun
1076, 1090
Author Index Lee, Dong Hoon 828 Lee, Gi-sung 1090 Lee, Hyeong-Ok 51 Lee, HyungHyo 811 Lee, Jeong-A 908 Lee, Jeong-Woo 683 Lee, Jeyun 723 Lee, Jong-Hee 1060 Lee, Keun-Wang 1060 Lee, Kwang-Hyoung 1060 Lee, Kyu-Ho 1098 Lee, Okbin 763 Lee, Sang-Ho 844 Lee, Sangjin 828 Lee, Sung-Oh 490 Lee, Won Suk 292 Lee, Woo Jin 1043 Lee, Yang-Dong 908 Lee, Yeijin 763 Lee, YoungRok 811 Leem, Choon-seong 1069, 1084 Li, Yue 739 Lim, Jongin 828 Ling, Sea 1008 Liu, Zhijian 984 Longheu, Alessandro 19 L´ opez-Ardao, Jos´e-Carlos 651, 892 L´ opez-Garc´ıa, C´ andido 651, 892 Loumos, Vassili 35 Macedo Mourelle, Luiza de 27, 43 Malgeri, Michele 19 Mangioni, Giuseppe 19 Mansour, Nashat 1019, 1027 Matth´e, Tom 204 Meri¸cli, C ¸ etin 529 Meybodi, M.R. 755, 960 Minogiannis, Nikolaos 59 Moerkotte, Guido 220 Moh, Sangman 83, 908 Montagne, Eur´ıpides 196 Moon, Ji-Young 699 Moon, Jongsub 828 Moussaoui, Omar 643 Musa, Mohamed E.M. 505 Nachouki, Gilles 147 Nar, Fatih 465 Nedjah, Nadia 27, 43 Nettleton, Eric 576
Neumann, Thomas 220 Ninos, Frankiskos 59 Noh, BongHam 811 Noh, Si Choon 1076 ¨ Orencik, B¨ ulent 819 ¨ Ozarar, Mert 611 ¨ ¨ Ozcanlı, Ozge Can 513 ¨ Ozgit, Attila 885 ¨ uner, F¨ Ozg¨ usun 942 ¨ ¨ Ozkasap, Oznur 852, 934 ¨ Ozsu, M. Tamer 332 Oh, Eunseuk 51 Okay, Nesrin 553 Oliver, Neal 860 Oz, Cemil 545 Ozsoyoglu, Gultekin 9 Ozturk, Zafer Ziya 179 Paker, Yakup 731 Papernick, Norman 513 Park, Chun-Su 627 Park, Gwi-Tae 490 Park, Jihun 405, 430 Park, Jiyoung 413 Park, Joong Kyu 1090 Park, Kang Ryoung 372 Park, Kyoung 83 Park, Nam Hun 292 Park, Sangho 405 Patel, Ahmed 236 Patrikakis, Charalampos 59 Petrounias, Ilias 123 Poernomo, Iman 1008 Polat, Faruk 619 Pyun, Jae-Young 707 Qian, Depei
739
Randle, Jeremy 576 Regli, William C. 397 Ridder, Dick de 505 Ridley, Matthew 576 Rodr´ıguez-Rubio, Ra´ ul-Fernando 651, 892 Rompotis, Andreas 59 Ryu, Yeonseung 276, 715 S, Valli 747 ¨ ur Saˇ glam, Ozg¨
802
1109
1110
Author Index
Sagiroglu, Seref 568 S ¸ ahin, Erol 900 Sahingoz, Ozgur Koray 284, 348 Salomie, Ioan 1051 Sauv´e, Jacques Philippe 786 Schmidt, Heinz 1008 S ¸ ehito˘ glu, Onur Tolga 976 S ¸ en, Nigar 340 Sever, Hayri 300 Shim, Jae-Hong 908 Shin, Dong-Ryeol 844 Shin, Ho-jun 1069 Silva Coelho, Fl´ avia Est´elia 786 Sogukpinar, Ibrahim 131 Sohn, Taeshik 828 Sohraby, Khosrow 860 Song, Sanghoon 869 Song, Won Jay 155, 364 Soysal, Onur 900 Srinivasan, Bala 212 Su´ arez-Gonzalez, Andr´es 651, 892 Sukkarieh, Salah 576 Sydow, Marcin 139 Syriani, Joe Abboud 1019 Szychowiak, Michal 916 Taniar, David 212 Tasaltin, Cihat 179 Ta¸sdemir, Kadim 447 Tchier, Fairouz 171 Teijeiro-Ruiz, Diego 651, 892 Temurtas, Fevzullah 179, 545 Temurtas, Hasan 179, 545 Terzioglu, Hakan 115
Tolun, Mehmet R. 252, 300 Topaloglu, N. Yasemin 1035 Tourn´e, Koen 204 Tr´e, Guy de 204, 260 T¨ umer, Borahan 952 Tunalı, Turhan 439, 635 U¸car, Bora 926 ¨ coluk, G¨ U¸ okt¨ urk 561, 592, 976 ¨ or, Alper 163 Ung¨ Uludaˇ g, Mahmut 252 Uma, G.V. 992 Uskarcı, Algan 474 Vassileva, Dessislava 356 Verkade, Jean Paul 324 Verstraete, J¨ org 260 Waluyo, Agustinus Borgy Wishart, Stuart 576 Won, Yonggwan 389 Won, Youjip 276 Wu, Annie S. 584
212
Yalabik, Ne¸se 340 Yi, Juneho 413 Y¨ ondem, Meltem Turhan 592 Yoon, Ki Song 869 Yumlu, M. Serdar 553 Yumusak, Nejat 179, 545 Zeineddine, Rabih 1027 Zhang, Wenjie 739 Zhou, Quming 430 Zyulkyarov, Ferad 236