Smart Spaces and Next Generation Wired Wireless Networking: Third Conference on Smart Spaces, ruSMART 2010, and 10th International Conference, NEW2AN ... Networks and Telecommunications)

Lecture Notes in Computer Science Commenced Publication in 1973 Founding and Former Series Editors: Gerhard Goos, Juris ...

Author: Sergey Balandin | Roman Dunaytsev | Yevgeni Koucheryavy

113 downloads 780 Views 11MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

Lecture Notes in Computer Science Commenced Publication in 1973 Founding and Former Series Editors: Gerhard Goos, Juris Hartmanis, and Jan van Leeuwen

Editorial Board David Hutchison Lancaster University, UK Takeo Kanade Carnegie Mellon University, Pittsburgh, PA, USA Josef Kittler University of Surrey, Guildford, UK Jon M. Kleinberg Cornell University, Ithaca, NY, USA Alfred Kobsa University of California, Irvine, CA, USA Friedemann Mattern ETH Zurich, Switzerland John C. Mitchell Stanford University, CA, USA Moni Naor Weizmann Institute of Science, Rehovot, Israel Oscar Nierstrasz University of Bern, Switzerland C. Pandu Rangan Indian Institute of Technology, Madras, India Bernhard Steffen TU Dortmund University, Germany Madhu Sudan Microsoft Research, Cambridge, MA, USA Demetri Terzopoulos University of California, Los Angeles, CA, USA Doug Tygar University of California, Berkeley, CA, USA Gerhard Weikum Max-Planck Institute of Computer Science, Saarbruecken, Germany

6294

Sergey Balandin Roman Dunaytsev Yevgeni Koucheryavy (Eds.)

Smart Spaces and Next Generation Wired/Wireless Networking Third Conference on Smart Spaces, ruSMART 2010 and 10th International Conference, NEW2AN 2010 St. Petersburg, Russia, August 23-25, 2010 Proceedings

13

Volume Editors Sergey Balandin Nokia Research Center Office of CTO Itamerenkatu 11-13 00180 Helsinki, Finland E-mail: [email protected] Roman Dunaytsev Tampere University of Technology Department of Communications Engineering Korkeakoulunkatu 10 33720,Tampere, Finland E-mail: [email protected] Yevgeni Koucheryavy Tampere University of Technology Department of Communications Engineering Korkeakoulunkatu 10 33720, Tampere, Finland E-mail: [email protected]

Library of Congress Control Number: 2010931611 CR Subject Classification (1998): C.2, H.4, D.2, K.6.5, D.4.6, H.3 LNCS Sublibrary: SL 5 – Computer Communication Networks and Telecommunications ISSN ISBN-10 ISBN-13

0302-9743 3-642-14890-5 Springer Berlin Heidelberg New York 978-3-642-14890-3 Springer Berlin Heidelberg New York

This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. springer.com © Springer-Verlag Berlin Heidelberg 2010 Printed in Germany Typesetting: Camera-ready by author, data conversion by Scientific Publishing Services, Chennai, India Printed on acid-free paper 06/3180

Preface

We welcome you to the joint proceedings of the anniversary 10th NEW2AN 2010 (Next Generation Teletraffic and Wired/Wireless Advanced Networking) and Third ruSMART 2010 (Are You Smart) conferences held in St. Petersburg, Russia during August 23–25, 2010. Originally the NEW2AN conference was launched by ITC (International Teletraffic Congress) in St. Petersburg in June 1993 as an ITC-Sponsored Regional International Teletraffic Seminar. The first implementation was entitled “Traffic Management and Routing in SDH Networks” and was hosted by R&D LONIIS. In 2002 the event received its current name, NEW2AN. In 2008 NEW2AN was joined by a new counterpart in smart spaces, ruSMART, hence boosting interaction between researchers, practitioners and engineers from different areas of ICT. This year, in its 10th implementation, NEW2AN/ruSMART is an established conference with a unique cross-disciplinary mix of telecommunications science in Russia. NEW2AN/ruSMART has always featured outstanding keynotes from universities and companies from Europe, USA and Russia. The 10th NEW2AN technical program addressed various aspects of nextgeneration network architectures. New and innovative developments for enhanced signaling protocols, QoS mechanisms, cross-layer optimization, traffic characterization were also addressed within the program. In particular, issues of QoS in wireless and IP-based multiservice networks were dealt with, as well as financial aspects of future networks. It is also worth mentioning that emphasis was place on wireless networks, including, but not limited to, cellular networks, wireless local area networks, personal area networks, mobile ad hoc networks, and sensor networks. The Third Conference on Smart Spaces, ruSMART 2010, provided a forum for academic and industrial researchers to discuss new ideas and trends in the emerging area of smart spaces that creates new opportunities for making fully customized applications and services for the users. The conference brought together leading experts from top affiliations around the world and there was active participation and strong interest from Russian attendees with a good reputation for high-quality research and business in innovative service creation and applications development. This year the technical program of both conferences benefited from two keynote speakers: – Vladimir Gorodetsky, St. Petersburg Institute for Informatics and Automation of Russian Academy of Sciences, Russia – Ian Oliver, Nokia Research Center, Finland – Alexey Vinel, St. Petersburg Institute for Informatics and Automation of Russian Academy of Sciences, Russia

VI

Preface

We wish to thank the Technical Program Committee members of both conferences and associated reviewers for their hard work and important contribution to the conference. This year the conferences were organized in cooperation with the FRUCT Program, ITC (International Teletraffic Congress), IEEE, Tampere University of Technology, Popov Society, and supported by NOKIA, Nokia Siemens Networks and Sententia Ltd. The support of these organizations is gratefully acknowledged. Finally, we wish to thank many people who contributed to the organization. In particular, Jakub Jakubiak (TUT) carried a substantial load of the submission and review process, the website maintenance, did an excellent job on the compilation of camera-ready papers and in the interaction with Springer. Many thanks go to Natalia Avdeenko, Ekaterina Antonyuk and Ekaterina Akimova (Monomax Meetings & Incentives) for their excellent local organization efforts and the conference’s social program preparation. We believe that the work done for the 10th NEW2AN and Third ruSMART conferences provided an interesting and up-to-date scientific program. We hope that participants enjoyed the technical and social conference program, the Russian hospitality and the beautiful city of St. Petersburg. June 2010

Sergey Balandin Roman Dunaytsev Yevgeni Koucheryavy

Organization NEW2AN International Advisory Committee Nina Bhatti Igor Faynberg Jarmo Harju Andrey Koucheryavy Villy B. Iversen Paul K¨ uhn Kyu Ouk Lee Mohammad S. Obaidat Michael Smirnov Manfred Sneps-Sneppe Ioannis Stavrakakis Sergey Stepanov Phuoc Tran-Gia Gennady Yanovsky

Hewlett Packard, USA Alcatel Lucent, USA Tampere University of Technology, Finland ZNIIS R&D, Russia Technical University of Denmark, Denmark University of Stuttgart, Germany ETRI, Republic of Korea Monmouth University, USA Fraunhofer FOKUS, Germany Ventspils University College, Latvia University of Athens, Greece Sistema Telecom, Russia University of W¨ urzburg, Germany State University of Telecommunications, Russia

NEW2AN Technical Program Commitee TPC Chair Roman Dunaytsev Mari Carmen Aguayo-Torres Ozgur B. Akan Khalid Al-Begain Sergey Andreev

Tampere University of Technology, Finland

University of Malaga, Spain METU, Turkey University of Glamorgan, UK State University of Aerospace Instrumentation, Russia Tricha Anjali Illinois Institute of Technology, USA Konstantin Avrachenkov INRIA, France Francisco Barcelo UPC, Spain Sergey Balandin Nokia, Finland Thomas M. Bohnert SAP Research, Switzerland Torsten Braun University of Bern, Switzerland Chrysostomos Chrysostomou University of Cyprus, Cyprus Nirbhay Chaubey Institute of Science and Technology for Advanced Studies and Research (ISTAR), India Ibrahim Develi Erciyes University, Turkey Eylem Ekici Ohio State University, USA Sergey Gorinsky Washington University in St. Louis, USA Markus Fidler NTNU Trondheim, Norway Giovanni Giambene University of Siena, Italy Stefano Giordano University of Pisa, Italy Ivan Ganchev University of Limerick, Ireland Vitaly Gutin Popov Society, Russia

VIII

Organization

Andreas Kassler Maria Kihl Tatiana Kozlova Madsen Yevgeni Koucheryavy Jong-Hyouk Lee Vitaly Li Leszek T. Lilien Saverio Mascolo Maja Matijaˇsevic Paulo Mendes Pedro Merino Ilka Miloucheva Dmitri Moltchanov Edmundo Monteiro Seán Murphy Marc Necker Nitin Nitin Jaypee Mairtin O’Droma Evgeni Osipov George Pavlou Veselin Rakocevic Simon Pietro Romano Alexander Sayenko Dirk Staehle Sergei Semenov Burkhard Stiller Weilian Su Arvind Swaminathan Dmitry Tkachenko Vassilis Tsaoussidis Christian Tschudin Andrey Turlikov Kurt Tutschku Alexey Vinel Lars Wolf

Karlstad University, Sweden Lund University, Sweden Aalborg University, Denmark Tampere University of Technology, Finland (Chair) INRIA, France Kangwon National University, Republic of Korea Western Michigan University, USA Politecnico di Bari, Italy University of Zagreb, FER, Croatia INESC Porto, Portugal University of Malaga, Spain Salzburg Research, Austria Tampere University of Technology, Finland University of Coimbra, Portugal University College Dublin, Ireland University of Stuttgart, Germany University of Information Technology, India University of Limerick, Ireland Lulea University of Technology, Sweden University of Surrey, UK City University London, UK Universita’ degli Studi di Napoli “Federico II”, Italy Nokia Siemens Networks, Finland University of W¨ urzburg, Germany Nokia, Finland University of Z¨ urich and ETH Z¨ urich, Switzerland Naval Postgraduate School, USA Qualcomm Inc., USA IEEE St. Petersburg BT/CE/COM Chapter, Russia Demokritos University of Thrace, Greece University of Basel, Switzerland State University of Aerospace Instrumentation, Russia University of Vienna, Austria SPIIRAN, Russia Technische Universit¨ at Braunschweig, Germany

NEW2AN Additional Reviewers V.V. Govindaswamy C. Hoene J. Jakubiak

N. Nitin S. Poryazov M. Smirnov

N. Vassileva

Organization

ruSMART Executive Technical Program Committee Sergey Boldyrev Nikolai Nefedov Ian Oliver Alexander Smirnov Vladimir Gorodetsky Michael Lawo Michael Smirnov Dieter Uckelmann Cornel Klein Maxim Osipov

Principle Scientist, Nokia Research Center, Helsinki, Finland Senior Research Engineer, Nokia Research Center, Zurich, Switzerland Principle Scientist, Nokia Research Center, Helsinki, Finland Head of laboratory, SPIIRAS, St. Petersburg, Russia Head of laboratory, SPIIRAS, St. Petersburg, Russia Professor, Center for Computing Technologies (TZI), University of Bremen, Germany Fraunhofer FOKUS, Germany LogDynamics Lab, University of Bremen, Germany Program Manager, Siemens Corporate Technology, Germany Siemens CT, Embedded Linux, Russia

ruSMART Technical Program Committee Juha Laurila Sergey Balandin Alexey Dudkov Didem Gozupek Kim Geunhyung Reto Krummenacher Prem Jayaraman Michel Banˆ atre Sergei Bogomolov Gianpaolo Cugola Dimitri Konstantas Markus Taumberger Bilhanan Silverajan Aaron J. Quigley Pedro Merino Victor Govindaswamy

Nokia Research Center, Switzerland Nokia Research Center, Finland University of Turku, Finland Bogazici University, Turkey Dong Eui University, Republic of Korea STI Innsbruck, Austria Monash University, Australia IRISA, France LGERP R&D Lab, Russia Politecnico di Milano, Italy University of Geneva, Switzerland VTT, Finland Tampere University of Technology, Finland University College Dublin, Ireland University of Malaga, Spain Texas A&M University, USA

ruSMART Additional Reviewers M. Baqer J. Jakubiak

Y. Koucheryavy

IX

Table of Contents I

ruSMART

Smart Spaces Use Cases Extending Context Spaces Theory by Proactive Adaptation . . . . . . . . . . . Andrey Boytsov and Arkady Zaslavsky

1

Context-Awareness Enabling New Business Models in Smart Spaces . . . . Boris Moltchanov, Christian Mannweiler, and Jose Simoes

13

A Middleware Architecture for Safety Critical Ambient Intelligence Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Antonio Coronato and Giuseppe De Pietro

26

Ranking Algorithm by Contacts Priority for Social Communication Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Natalia Dudarenko, Juwel Rana, and K˚ are Synnes

38

CrysP: Multi-Faceted Activity-Infused Presence in Emerging Social Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Saguna, Arkady Zaslavsky, and Dipanjan Chakraborty

50

Smart-M3 Platform Multichannel System of Audio-Visual Support of Remote Mobile Participant at E-Meeting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Andrey L. Ronzhin, Victor Yu Budkov, and Alexey A. Karpov On-the-Fly Ontology Matching in Smart Spaces: A Multi-model Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Alexander Smirnov, Alexey Kashevnik, Nikolay Shilov, Sergey Balandin, Ian Oliver, and Sergey Boldyrev

62

72

Access Control in Personal Localized Semantic Information Spaces . . . . . Alexey Koren and Alexander Buntakov

84

Integration of an Answer Set Engine to Smart-M3 . . . . . . . . . . . . . . . . . . . . Vesa Luukkala and Jukka Honkola

92

A Video Monitoring Model with a Distributed Camera System for the Smart Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Andrey L. Ronzhin, Maria Prischepa, and Alexey Karpov

102

XII

Table of Contents

Smart Spaces Solutions Physical Approach in Smart Homes: A Proposition and a Prototype . . . . Michele Dominici, Giulio Zecca, Frédéric Weis, and Michel Banˆ atre SMEO: A Platform for Smart Classrooms with Enhanced Information Access and Operations Automation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Akshey Jawa, Sudip Datta, Sujeeth Nanda, Vishal Garg, Vasudeva Varma, Suresh Chande, and Murali Krishna Punaganti Venkata Creative Approach to the Design and Prototyping of Experimental Smart Spaces, Case Studies from the Interactive Environments Minor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tomasz Jaskiewicz, Walter A. Aprile, and Aadjan van der Helm Event Recognition via Energy Efficient Voting for Wireless Sensor Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mohamed Baqer

II

111

123

135

148

NEW2AN

Performance Evaluation Evaluation of Multi-service CDMA Networks with Soft Blocking . . . . . . . Villy B. Iversen Design of MPLS over DWDM Architecture for Unicast and Anycast Flows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jakub Gladysz and Krzysztof Walkowiak Comparative Analysis of Sleep Mode Control Algorithms for Contemporary Metropolitan Area Wireless Networks . . . . . . . . . . . . . . . . . Alexey Anisimov, Sergey Andreev, Olga Galinina, and Andrey Turlikov Determining Normalized Measure of Dispersion for Evaluating the Performance of IEEE 802.11 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Faisal Iradat and Sayeed Ghani Development of a System to Promote Eco-Driving and Safe-Driving . . . . Ryosuke Ando, Yasuhide Nishihori, and Daisuke Ochi

160

172

184

196 207

Performance Modeling Oligopoly Game Modeling for Cognitive Radio Environments . . . . . . . . . . Ligia C. Cremene, D. Dumitrescu, and Réka Nagy

219

Table of Contents

Reducing the Number of States for Markovian Model of Optical Slotted Ring Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mateusz Nowak and Piotr Pecka Performance Modelling of Automatic Identification System with Extended Field of View . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Troels Laursen, Hans Peter Mortensen, Nikolaj Bisgaard Pedersen, Ulrik Wilken Rasmussen, Tatiana K. Madsen, and Jens Dalsgaard Nielsen Performance Measures Computation for a Single Link Loss Network with Unicast and Multicast Traffics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Irina Gudkova and Olga Plaksina Realization of a Method of Matrix-Geometric Progression . . . . . . . . . . . . . Yuri Ryzhikov

XIII

231

242

256

266

Delay-/Disruption-Tolerant Networking and Overlay Systems Mobility Pattern Based Routing Algorithm for Delay/Disruption Tolerant Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ManKyu Park, SeokHong Min, SangHo So, DeockGil Oh, ByungChul Kim, and JaeYong Lee

275

Heuristic Congestion Control for Message Deletion in Delay Tolerant Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lertluck Leela-amornsin and Hiroshi Esaki

287

The FDPOO: A Flexible Delivering Platform with at-One-time and On-time guarantees in Opportunistic Networks . . . . . . . . . . . . . . . . . . . . . . Yuan-Tse Yu, Chung-Ming Huang, and Hsing-Cheng Wang

299

A Distributed Telematics Peer-to-Peer (D-TP2P) Networking System for Service Discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chung-Ming Huang, Chia-Ching Yang, and Chun-Yu Tseng

311

Empirical Predictor of TCP Throughput on a Multihop Wireless Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Anna Chaltseva and Evgeny Osipov

323

A Framework for Connectivity in Inter-working Multi-hop Wireless Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Oladayo Salami, Antoine Bagula, and H. Anthony Chan

335

XIV

Table of Contents

Integrated Wireless Networks Practical Vertical Handover Mechanisms between Satellite and Terrestrial Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . MinSu Shin, ManKyu Park, DeockGil Oh, ByungChul Kim, and JaeYong Lee Geo-aware Handover of Mission Agents Using Opportunistic Communication in VANET . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Edison Pignaton de Freitas, Tales Heimfarth, Fl´ avio Rech Wagner, Armando Morado Ferreira, Carlos Eduardo Pereira, and Tony Larsson

353

365

Comparison and Evaluation of the Most Efficient Mobility Management Protocols for Integrated Wireless Networks . . . . . . . . . . . . . . . . . . . . . . . . . . Asimakis Lykourgiotis and Stavros Kotsopoulos

377

Functional Model and Service Scenarios for QoS Enabled Mobile VoIP Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kyu Ouk Lee, Sang Soo Lee, and Tae Whan Yoo

389

Heuristic Approaches to the Multi-objective Network Design and Optimization for Wireless Data Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . Chutima Prommak and Naruemon Wattanapongsakorn

398

Resource Management Call Admission Control in Single and Two-Tier Cellular Networks . . . . . . Manfred Schneps-Schneppe and Janis Sedols

411

Joint Admission Control and Resource Allocation for Multiuser Loading in LTE Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . V. Venkatkumar, T. Haustein, and M. Faulkner

421

Data Traffic Scheduling Algorithm for Multiuser MIMO-OFDM System with the Throughput Maximization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jerzy Martyna

436

Multimedia Communications Scalable Video Coding Based on Three-Dimensional Discrete Pseudo Cosine Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Eugeniy Belyaev, Timofey Sukhov, and Liu Kai

448

NTRULO: A Tunneling Architecture for Multimedia Conferencing over IP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A. Amirante, T. Castaldi, L. Miniero, and S.P. Romano

460

Table of Contents

XV

Seamless Streaming Service Session Migration Support Architecture for Heterogeneous Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Geun-Hyung Kim and Bong-Hwan Lee

473

Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

485

Extending Context Spaces Theory by Proactive Adaptation Andrey Boytsov and Arkady Zaslavsky Department of Computer Science and Electrical Engineering Luleå University of Technology, SE-971 87 Luleå {Andrey.Boytsov,Arkady.Zaslavsky}@ltu.se

Abstract. Context awareness is one of the core features of pervasive computing systems. Pervasive systems can also be improved by smart application of context prediction. This paper addresses subsequent challenge of how to act according to predicted context in order to strengthen the system. Novel reinforcement learning based architecture is proposed to overcome the drawbacks of existing approaches to proactive adaptation. Context spaces theory is used as an example of how existing context awareness systems can be enhanced to achieve proactive adaptation. This recently developed theory addresses problems related to sensors uncertainty and high-level situation reasoning and it can be enhanced to achieve efficient proactive adaptation as well. This article also discusses implementation options and possible testbed to evaluate the solutions. Keywords: Context awareness, context prediction, context spaces theory, pervasive computing, reinforcement learning, proactive adaptation, act-ahead adaptation.

1 Introduction Pervasive computing is a paradigm where computing systems are integrated into the everyday life in a non-intrusive, graceful and transparent manner. Some implementations of pervasive computing paradigm include smart homes, elderly care systems, smart mobile devices, GPS navigators, RFID tracking systems, social networks. Context awareness is one of the basic features of pervasive computing. Context prediction is also recognized as a challenge and an opportunity. However, for context prediction and especially for acting on predicted context there is a definite lack of universal approach to the problem. This paper proposes and motivates the approach for contextual act-ahead adaptation – proactive adaptation to predicted context. The strengths and challenges of proposed approach are discussed. The article is structured as follows. Section 2 describes related work and current challenges of proactive adaptation to predicted context. Section 3 further elaborates identified challenges and proposes and explains the solution approach. Section 4 describes the essentials of context spaces theory and introduce ECORA framework. Sections 5 and 6 discuss the integration of proposed approach into context spaces S. Balandin et al. (Eds.): ruSMART/NEW2AN 2010, LNCS 6294, pp. 1–12, 2010. © Springer-Verlag Berlin Heidelberg 2010

2

A. Boytsov and A. Zaslavsky

theory and introduce CALCHAS – Context Aware aCt aHead Adaptation System. Section 7 introduces a sample testbed scenario. Section 8 makes a summary, provides future research plans and concludes the paper.

2 Context Prediction and Acting on Predicted Context Most of context prediction approaches use machine learning techniques. Predictive models that were applied to context prediction task include sequence predictors [5], neural networks [22], Bayesian networks [17], branch predictors [16], decision tree learning [11], time series prediction [19], Markov models [7], trajectory extrapolation [1]. Some authors actually do recognize acting on predicted context as a specific problem ([3], [4]), but still there is not much research done for the challenge of proactive adaptation. Generally most of the ways to act on predicted context can be classified in two groups. 1. Rule-based engines. For every particular prediction result there is a rule that defines an action. Improving behavior of the system is achieved by improving efficiency of context prediction. 2. Learning by example. That approach was applied mostly in smart home environments. Pervasive computing system tracks user actions and then it starts executing the actions for the user. On the basis of studying and analyzing existing approaches, summary representation can look like expression set (1). state(t)=SenseAndProcessData(t) prediction(t)=PredictionModel(state(t),history(t),prediction(t-1 ) history(t+1)=addToHistory(state(t),history(t) ) action(t)=actOnPrediction(state(t),prediction(t),history(t)) .

(1)

Where state(t) is entire context state at the time t, including the results of sensor data acquisition, validation and processing. Entire aggregated history up to time t, but not including time t, is stored in history(t). Prediction results at time t for time t+1 and maybe subsequent steps are referred to as prediction(t). Usually they depend on the current state and on the dependencies learned from history. Sometimes previous prediction results can influence them as well, if time horizons of prediction attempts do overlap. System is acting on predicted context, so action at time t ܽܿ‫݊݋݅ݐ‬ሺ‫ݐ‬ሻ depends on prediction results and current state. In case learning by example is implemented, action also has some learning mechanisms and therefore depends on history. The model presented in (1) has a very serious drawback – it cannot handle mutual dependency between system actions and prediction results. Those drawbacks and solution opportunities will be addressed in section 3. Notable exceptions that do not fall in that model are the applications of Markov decision processes and Q-learning based approaches to context prediction [24],[6],[13]. They will be mentioned further in section 3 when discussing reinforcement learning solutions.

Extending Context Spaces Theory by Proactive Adaptation

3

3 Proactive Adaptation as Reinforcement Learning Task Sometimes the system has to provide results of context prediction in quite specific conditions. Here are some motivating use cases. Consider elderly care system. Users need some assistance in accomplishing everyday tasks. System is able to give an advice, call a caregiver or just do nothing if the user is performing quite well. The task is to predict the probability of successful outcome of the activity and to act to maximize that probability, but not at the cost of wasting the time of the caregiver or annoying the user with unnecessary advices. Use case is partially based on [10]. Another motivating use case can be a smart home, where user satisfaction can be estimated according to both direct questioning and the number of user manual interventions into a configuration of home parameters. The task is to predict user’s cumulative satisfaction and act to increase it. Also the system should do its best to behave in non-intrusive manner and not to annoy the user with too many questions. One more example might be prediction and prevention of memory shortage in pervasive system. If memory shortage was predicted with the certainty above the threshold, the system takes actions to avoid memory shortage condition (e.g. searches for additional resources to share the work). But if those preventive actions are successful, the predictor would have learning problem. Formally if there was no memory shortage, it is a prediction failure. In reality most likely it is not, but we cannot claim it for sure. Actually, when the system starts decision making, predictor becomes unable to learn properly - predictor cannot distinguish between prediction failure and successful prevention. For all those cases, neither rule-based system, nor learning by example can solve the problem completely. The solution for this and many similar use cases is to take decision making process into account while making predictions. The dependencies for those use cases can be described like the expressions (2). state(t)=SenseAndProcessData(t) prediction(t)=PredictionModel(state(t),history(t),prediction(t-1) , action(t)) history(t+1)=addToHistory(state(t),history(t) ) action(t)=actOnPrediction(state(t),prediction(t),history(t)) .

(2)

The meaning of state(t), history(t), action(t) and other elements in (2) is the same as in expressions (1). The difference between (2) and (1) is marked in bold. The additional dependency makes most current proactive adaptations methods inapplicable. System acts on predicted context, but in turn predicted context depends on system actions. Most of the context prediction approaches we mentioned previously use predictive models, which do not take into account the mutual dependency between system actions and prediction results. Usually this problem is avoided by completely splitting the context in two independent parts: the one that is affected by actions and the one that is predicted. For example, if the system intends to proactively open the door before the user comes into the room, the choice whether the system opens the door or leaves it closed will not change user intentions and, therefore, prediction results. Learning by example is also the way to avoid mutual dependency problem, but this approach has very limited applicability: it works only if possible user actions and

4


system actions significantly overlap and only with the assumption that imitating user actions grants acceptable effectiveness of the system. We propose an enhanced method to solve mutual dependency problem between actions and predictions. That problem can actually be viewed as reinforcement learning task. Reinforcement learning problem is a problem faced by an agent that should find an acceptable behavior in a dynamic environment and learn from its trial and errors [12]. Reinforcement learning in application to pervasive computing task is almost not researched. Notable exceptions are Markov decision processes and Q-learning approaches [24],[6],[13]. However, as it will be discussed further in this section, Markov decision processes have very limited applicability due to the features of pervasive computing systems. Recent advancements in reinforcement learning include the approaches, which to the best of our knowledge were not applied to pervasive computing at all, including predictive state representation [23], most cases of neural network control [8], different applicable cases of actor-critic model (like [9]), selforganizing maps and Q-learning integration [20] and many more. To the best of our knowledge, there was no attempt to address particular features of reinforcement learning in pervasive computing area. Those features are: • Continuous action spaces. Actuators sometimes provide the choice from practically continuous range of values. • Continuous state spaces. Many kinds of sensors produce values from practically continuous range as well. However, this problem can be resolved by the means of situation awareness. • Mobility of nodes. Mobility of nodes in pervasive computing system can cause the loss of connection to certain sensors and actuators. Or, on the contrary, new connections to different sensors and actuators can be established. • High dimensionality. The count of sensors and actuators in pervasive computing system can be very high. • High heterogeneity of data. Discrete-valued, continuous-valued or event-based sensors and actuators can appear in any combination. • Ability to incorporate prior knowledge. Sometimes common sense or expert estimations can give some sketches of good acting strategies. Ability to incorporate them can significantly reduce learning time. • Explicit prediction result can also provide some insight into the problem. • Limited exploration capabilities. Pervasive system cannot just do a random thing to see what happens. • Limited time for decision making. • Goals of the user can change instantly. The features mentioned above seriously limit the scope of reinforcement learning solutions that we can try. In particular, those features mean that Markov decision process – the dominating model for reinforcement learning – can be used only in limited set of special cases. The main reason here is MDPs’ discrete state and action spaces, which might be not suitable for pervasive computing solutions. In sections 4-6 we will discuss in more details, how we can address pervasive computing challenges presented in the list above.


5

4 Context Spaces Theory – Main Concepts Some features of proactive adaptation in pervasive computing can be addressed by improving context spaces theory. The theory of context spaces [14] is a recently developed approach for context awareness and reasoning which addresses the problems of sensors uncertainty and unreliability. It also deals with situation reasoning and the problems of context representation in a structured and meaningful manner. Context spaces theory is designed to enable context awareness in clear and insightful way. This theory uses spatial metaphors for representing context as a multidimensional space. To understand context spaces theory we need to introduce several new terms. Any kind of data that is used to reason about context is called context attribute. Context attribute usually corresponds to a domain of values of interest, which are either measured by sensors directly or calculated from other context attributes. It can be either numerical value or a value from pre-defined set of non-numerical options. Context state represents the set of all relevant context attributes at a certain time. A set of all possible context states constitutes application space. Therefore, application space can be viewed as a multi-dimensional space where the number of dimensions is equal to the number of context attributes in the context state. The state of the system is represented by a point in the application space and the behavior of the system is represented by a trajectory moving through the application space over time. Situation space is meant to represent real life situation. It can be defined as subspace of the application space. So if context state is in the subspace representing situation S, it means that situation S is occurring. Situation S has the level of occurrence certainty. It depends on the probability of the context state to be within S and it also depends on the subspace within S. See Figure 1 for simple illustration.

Fig. 1. Context spaces theory

In context spaces theory several methods were developed for reasoning about the context. Bayesian reasoning [21] or Dempster-Schafer algorithm [18] are used to get overall confidence in the fact that a situation is occurring. Algebraic operations on situations and some logic-based methods were developed for reasoning in terms of situations [14].

6


Some solutions were developed to integrate various context prediction methods into the theory of context spaces [2]. However, those prediction methods are based on pure forecasting and do not take into account decision making aspects. Context spaces theory was implemented in ECORA [15] – Extensible Context Oriented Reasoning Architecture. ECORA is a framework for development of contextaware applications. That framework provides its functionality as a set of Java classes to be integrated into the prototypes of context-aware systems. The presented research introduces proactive adaptation methods that can be used to extend the context spaces theory and enhance ECORA-based applications.

5 Integrating Proactive Adaptation into Context Spaces Theory Context spaces theory has all the necessary capabilities to ensure context awareness. However, when it comes to proactive adaptation, some enhancements have to be made. At first, the actuators need to be introduced into the model. Actually, those actuators can constitute a separate space in the manner much like the context space itself. That actuator space can be treated as action space for reinforcement learning. Considering different meaning of sensors and actuators, it seems that the better solution is to separate the spaces for sensors and actuators. Situation spaces in the space of actuators are not likely to have any value (or, in case a set of discrete actions can be elicited, those situations will just look like points, not spaces). In case of the lost connection to actuator, the corresponding axis is removed. Most likely, it will take some time for the system to learn how to act in new conditions, but previously learnt data can provide a good starting point and reduce adaptation time. Similar refers to introducing new actuators – combining old data with exploration can provide adaptation to new capabilities. Another question is how to define reinforcement learning state space for context spaces theory. Having discrete state space is likely to make the task simpler. One solution is to take situations as states for reinforcement learning and trigger the state if the certainty of situation occurring is above some threshold. To fit the Markov property (that is essential for different kinds of Markov decision processes), situation decomposition algorithm presented in [2] can be used. That kind of situation decomposition approach can also help to overcome node mobility problem from sensor point of view: if the sensor loses the link or if new sensor is introduced, situation properties are recalculated (see [14] for the details), but it is transparent for the upper layer, including the proactive adaptation. Also taking context space as state space for reinforcement learning is an option. One more option is to take the degrees of certainty in different situations as continuous state space. To summarize, context spaces theory can be enhanced to incorporate proactive adaptation solutions. Those context spaces theory enhancements can deal with one of the main features of pervasive computing – the mobility of sensors and actuators – using geometrical metaphors and situation awareness. Addressing other features of proactive adaptation depends mostly on exact reinforcement learning method and the architecture of proactive adaptation solutions of upper layers.


7

6 CALCHAS Prototype The proposed approach can be implemented by creating a general-purpose middleware for context prediction and proactive adaptation based on context spaces theory. CALCHAS (Context Aware Long-term aCt aHead Adaptation System) prototype is now under development. Integrated with ECORA, it will allow achieving efficient proactive adaptation for context spaces theory. The architecture was designed to address all the features of pervasive computing area that were mentioned in section 3. That middleware solution should incorporate a library of different context prediction and proactive adaptation methods. Middleware development is in progress and the prototype is being implemented. We propose the following architecture for CALCHAS (see Figure 2).

Fig. 2. CALCHAS general architecture

The architectural blocks have following purpose. Run-time goal and efficiency engine translates user input into exact goals for pervasive system in terms of context and timing. Sensor fusion and low-level validation block provides preliminary low-level processing of context data. Retraining database provides bulk of recent low-level data on request, in case there is a need to retrain the model (e.g. due to goal change) or

8


Fig. 3. CALCHAS adaptation engine

provide faster model start. Feedback is implicit – every actuator action will affect future state of the system, which will in turn be tracked by sensors. Adaptation and prediction engine is responsible for inferring adaptation sequences – sequences of actions for the actuator to do. Also it is responsible for providing explicit prediction results. Adaptation engine has the following structure (see Figure 3). All the translation blocks are used to translate between the format of internal model (e.g. states of Markov decision process, its reward functions) and the format of the outer system (e.g. vectors of values of sensor data and exact commands for actuators). The adaptation core is responsible for generating the actions according to the model and learning engine is responsible to adapt the model to new data. All translation blocks are heavily dependent on specific task, but for most of other blocks general purpose solutions can be developed. Supplied with a comprehensive library of reinforcement learning methods, the system can facilitate the development of pervasive computing applications, which will be aware of current and predicted context and which will be capable of smart proactive adaptation to predicted context.

7 Sample Scenario for Proactive Adaptation Methods In order to validate proactive adaptation solutions, we introduce “Moonprobe” model. “Moonprobe” is the result of subsequent development of application scenario presented in [2]. The “Moonprobe” model is an imitation of a vehicle in 2D environment. The aim of the vehicle is to reach destination point safely and not crash or run out of fuel on its way. “Moonprobe” was developed using XJ Technologies AnyLogic modeling tool [25].


9

Although not representing pervasive system on its own, the “Moonprobe” model deals with the problems that are common for every pervasive system like sensor uncertainty, sensor failures, acting under uncertainty, fusing heterogeneous sensor data. The moonprobe has a set of specific situations to be aware of. These are sensor failure conditions, progress of going to the destination point and safety conditions. “Moonprobe” uses around 20 context parameters to reason about the situations. Also the moonprobe has some actuators to trigger: engines can help the probe to land safely or to keep the speed. However, working engines require fuel, and the amount of fuel is limited. The actions based on actuators can be considered continuous (direction and power of engine) as well as discrete (like “do nothing”, “keep velocity”, “dampen the fall” etc.). In a very simplified manner the model can be described by the following set of equations (see formula (3)).

N

If probe is in the air: 0 If probe is on the ground: projection of M g on the axis perpendicular to the ground

Fen

(3)

|Fen|

Where t is time, X is probe coordinate, V is probe velocity, M is mass of the probe (known system parameter), N is support reaction (directed perpendicular to the ground), Fl – fuel level remained, g – gravity, FCR – fuel consumption rate (known system parameter), Fen – engine force vector (value and direction set by probe). In some cases probe can leave the ground. When probe lands, it will experience ground collision. Ground collisions are dangerous for the probe. Engines can be used to slow down the landing and avoid the crash.

Fig. 4. “Moonprobe” system architecture

The architecture of the model is following (see Figure 4). System consists of several components. •

“Environment and probe” component. It implements the physical model of the system. Also it monitors what happens to the probe (crash, sensor outage etc.).

10

• • •


“Sensors” component. Sensors measure the data from the probe in certain moments of time and introduce some errors. “Controlling engine” component. It takes sensed data and provides control information. All proactive adaptation logic is embedded there. “Main” component provides overall assessment of experimental results.

The design of exact learning approaches in CALCHAS is in progress. Rule-based decision making engine with quantified actions was introduced into “Moonprobe” model to act as a benchmark for different reinforcement learning solutions. Rules were defined based on common sense. Screen dump of running “Moonprobe” is depicted on Figure 5.

Fig. 5. "Moonprobe" system working

8 Conclusion and Future Work In this work we addressed the task of proactive adaptation to predicted context and proposed reinforcement learning based architecture, which will allow pervasive computing systems to address more complicated use cases. The features of proactive adaptation task in pervasive computing systems were identified. The architecture of proactive adaptation solution – CALCHAS system – was developed to address them. Context spaces theory was used as particular example of integration between context awareness system and proactive adaptation solution. Context spaces-based testbed for proactive adaptation solution was implemented. Our future plans are to implement the entire middleware with a comprehensive library of available reinforcement learning approaches. Also the plan is to extend reinforcement learning solutions methods to suit the core challenges pervasive computing area.


11

References 1. Anagnostopoulos, C., Mpougiouris, P., Hadjiefthymiades, S.: Prediction intelligence in context-aware applications. In: Proceedings of the 6th international conference on Mobile data management, Ayia Napa, Cyprus, pp. 137–141. ACM, New York (2005) 2. Boytsov, A., Zaslavsky, A., Synnes, K.: Extending Context Spaces Theory by Predicting Run-Time Context. In: Balandin, S., Moltchanov, D., Koucheryavy, Y. (eds.) ruSMART 2009. LNCS, vol. 5764, pp. 8–21. Springer, Heidelberg (2009) 3. Cook, D.J., Augusto, J.C., Jakkula, V.R.: Ambient intelligence: Technologies, applications, and opportunities. Pervasive and Mobile Computing 5, 277–298 (2009) 4. Cook, D.J., Das, S.K.: How smart are our environments? An updated look at the state of the art. Pervasive and Mobile Computing 3, 53–73 (2007) 5. Das, S.K., Cook, D.J., Bhattacharya, A., Heierman III, E.O., Lin, T.-Y.: The role of prediction algorithms in the MavHome smart home architecture. IEEE Wireless Communications 9(6), 77–84 (2002) 6. Feki, M., Lee, S., Bien, Z., Mokhtari, M.: Context Aware Life Pattern Prediction Using Fuzzy-State Q-Learning. Pervasive Computing for Quality of Life Enhancement, 188–195 (2007) 7. Gellert, A., Vintan, L.: Person Movement Prediction Using Hidden Markov Models. Studies in Informatics and Control 15(1) (2006) 8. Hagan, M.T., Demuth, H.B.: Neural networks for control. In: Proceedings of the 1999 American Control Conference, San Diego, CA, pp. 1642–1656 (1999) 9. van Hasselt, H., Wiering, M.: Reinforcement Learning in Continuous Action Spaces. In: Proceedings of IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning (ADPRL’07), Honolulu, HI, USA, pp. 272–279 (2007) 10. Hoey, J., Von Bertoldi, A., Poupart, P., Mihailidis, A.: Assisting persons with dementia during handwashing using a partially observable Markov decision process. In: Proceedings of International Conference on Vision Systems, p. 66 (2007) 11. Hong, J., Suh, E., Kim, J., Kim, S.: Context-aware system for proactive personalized service based on context history. Expert Systems with Applications 36(4), 7448–7457 (2009) 12. Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: A survey. Journal of Artificial Intelligence Research 4, 237–285 (1996) 13. Mozer, M.C.: Lessons from an adaptive home. In: Smart environments: technologies, protocols, and applications, pp. 273–294. Wiley, Chichester (2004) 14. Padovitz, A., Loke, S.W., Zaslavsky, A.: Towards a theory of context spaces. In: Proceedings of the Second IEEE Annual Conference on Pervasive Computing and Communications Workshops, pp. 38–42 (2004) 15. Padovitz, A., Loke, S.W., Zaslavsky, A.: The ECORA framework: A hybrid architecture for context-oriented pervasive computing. Pervasive and Mobile Computing 4(2), 182–215 (2008) 16. Petzold, J., Bagci, F., Trumler, W., Ungerer, T.: Context Prediction Based on Branch Prediction Methods. Technical Report, Institute of Computer Science, University of Augsburg (2003) 17. Petzold, J., Pietzowski, A., Bagci, F., Trumler, W., Ungerer, T.: Prediction of Indoor Movements Using Bayesian Networks. In: Strang, T., Linnhoff-Popien, C. (eds.) LoCA 2005. LNCS, vol. 3479, pp. 211–222. Springer, Heidelberg (2005) 18. Shafer, G.: A Mathematical Theory of Evidence. Princeton University Press, Princeton (1976)

12


19. Sigg, S., Haseloff, S., David, K.: Minimising the Context Prediction Error. In: Proceedings of IEEE 65th Vehicular Technology Conference VTC 2007, Dublin, Ireland, pp. 272–276 (Spring 2007) 20. Smith, A.J.: Applications of the self-organising map to reinforcement learning. Neural Networks 15, 1107–1124 (2002) 21. Russell, S.J., Norvig, P.: Artificial Intelligence: A Modern Approach, 2nd edn. Prentice Hall, Englewood Cliffs (2003) 22. Lin, T., Wang, C., Lin, P.-C.: A neural-network-based context-aware handoff algorithm for multimedia computing. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP) archive 4(3), 1–23 (2008) 23. Wolfe, B., James, M.R., Singh, S.: Approximate predictive state representations. In: Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems, vol. 1, pp. 363–370. International Foundation for Autonomous Agents and Multiagent Systems, Estoril (2008) 24. Ziebart, B.D., Maas, A.L., Dey, A.K., Bagnell, J.A.: Navigate like a cabbie: probabilistic reasoning from observed context-aware behavior. In: Proceedings of the 10th international conference on Ubiquitous computing, Seoul, Korea, pp. 322–331. ACM Press, New York (2008) 25. How to Build a Combined Agent Based / System Dynamics Model Logic in AnyLogic (2008), http://www.xjtek.com/file/161

Context-Awareness Enabling New Business Models in Smart Spaces Boris Moltchanov1, Christian Mannweiler2, and Jose Simoes3 1

Telecom Italia S.p.A., via Reiss Romoli 274, 10148-Torino, Italy [email protected] 2 University of Kaiserslautern, Wireless Communications and Navigation Group 67663 Kaiserslautern, Germany [email protected] 3 Fraunhofer Institute FOKUS, Kaiserin-Augusta-Alle 31, 10589 Berlin, Germany [email protected]

Abstract. This paper describes how emerging context-awareness technologies will act as a driving force for introduction of intelligently personalized services in smart spaces. The according context-awareness framework is taken from the results of the EU FP 7 project “C-CAST”. Major novelties include a highly scalable context management architecture as well as a context prediction methodology. The paper considers business models and financial structures as well as service and application examples that might be realized in smart spaces enriched with environmental, personal, social, and network context information. From a commercial perspective on context-aware applications in smart spaces, these examples differ from traditional Telecom and IT services and demonstrate how Telecom and IT companies may strengthen and leverage their business based on context information about customers, services and their environments that is abundantly available in smart spaces. Moreover, this paper includes the description of a field trial with new applications in quasi-smart space environments. Keywords: Business model, smart space, sensor, context, prediction.

1 Introduction The number of mobile user terminals permanently connected to the web over wireless technologies as potential context sources (e.g., embedded sensors, cameras) and the sensors creating large interworking Wireless Sensor Networks (WSN) interconnected with Building Sensor Network and Automotive Sensor Networks is expected to increase significantly creating really ubiquitous systems. All these information sources create heterogeneous and very pervasive environments collectively identifiable as Smart Spaces. Users surrounded by this technology expect more sophisticated and useful services and richer multimedia content. S. Balandin et al. (Eds.): ruSMART/NEW2AN 2010, LNCS 6294, pp. 13–25, 2010. © Springer-Verlag Berlin Heidelberg 2010

14

B. Moltchanov, C. Mannweiler, and J. Simoes

In parallel, the advent of Web 2.0 technologies and their consequent application deployments revealed that user generated content, when tied to a certain geographical location and part of the augmented reality and virtual social communities became a trend in what concerns user most used services. Leveraging smart space technology with a context-aware framework will allow objects, devices, persons and entities to be characterized and interact in real time. Among all players involved in the business value chain of the innovative services, users themselves are the ones demanding for appealing services. To satisfy this demand, a mentality shift needs to occur, as the technological one occurred with the emergence of context-awareness and WSN technologies that enabled new business models and revenue schemes. The context management technology used in this work, has been designed in the European research project C-CAST [1], which aims at combining mobile distribution techniques (both unicast and multicast) and context-awareness in order to optimize multimedia content distribution. This papers shows how it can be used for new business models in smart spaces. The remainder of the paper is organized as follows. Section 2 presents the state-of-the-art in context-awareness for smart spaces. The technical and business aspects of our context management framework are presented in sections 3 and 4, respectively. Then, section 5 describes some context-aware commercial services and trials built within Telecom Italia (TI) infrastructure. Finally, section 6 concludes the paper and introduces future work.

2 State of the Art This section introduces the terminology and provides an overview of related research activities in the areas of context management. 2.1 Definitions and Terminology Context – Ryan et al. [2] referred to context as the user’s location, environment, identity and time. Dey [3] defines context as the user’s emotional state, focus of attention, location and orientation, date and time, as well as objects and people in the user’s environment. Hull et al. [4] describe context as the aspects of the current situation. Brown [5] defines context to be the elements of the user’s environment, which the computer knows about. The notion of situation is closely related to the concept of context. Zimmermann [6] defines it as “the state of a context at a certain point (or region) in space at a certain point (or interval) in time, identified by a name”. Smart Space – a smart space is any real or virtual location equipped with passive and active artifacts. These artifacts can be any kind of sensors and actuators, mobile devices, software agents, autonomous vehicles, management systems, and also human beings. In smart spaces, these artifacts have the processing and communication capabilities to interact with each other in a (mutually) beneficial way. Examples include smart homes, smart factories, smart transportation, smart grid or ambient assisted living environments [7], [8].

Context-Awareness Enabling New Business Models in Smart Spaces

15

2.2 Types of Context for Smart Spaces Context information can be derived from the physical environment or a physical state of an entity (e.g., temperature, humidity, light, noise levels, etc.) and logical (e.g., activity, friend, parent, preferences, etc.). Moreover, context may be of primary origin or derived origin, which effectively is reasoned out of primary context data. There is also meaningful context, a type considered in the literature as linguistic context. For this work, we will basically abstract from the context type, i.e. consider all types. 2.3 Context-Aware Systems for Smart Spaces The history of context-aware systems started when Want et al. [9] introduced the Active Badge Location System, considered to be one of the first context-aware applications. Ailisto et al., [10] and Dey and Abowd [11] proposed a layered conceptual architecture that augments layers for detecting and using context by adding interpreting and reasoning functionalities. In middleware-based systems, the Service-Oriented ContextAware Middleware (SOCAM) project [12] introduced an architecture for building and rapid prototyping context-aware mobile services. One further extensible centralized middleware approach designed for context-aware mobile applications is a project called Context-Awareness Sub-Structure (CASS) presented in [13]. Chen [14] presents a Context Broker Architecture (CoBrA), which is an agent-based architecture, for supporting context-aware systems in smart spaces (e.g., intelligent meeting rooms, smart homes, and smart vehicles). Other well-known examples for broker-based approaches are the SOUPA, and GAIA [15]. Many EU projects have explored contextawareness aspects, for example, SPICE [16] and MUSIC [17]. One of the main reasons of little context-awareness usage, despite the mentioned efforts, was lack of end user devices that could host the context-aware applications for users. With the wide scale availability of such devices, including the WSN sensors, it is now possible to harness the possibilities of context-awareness. There are many types of the services where context-aware smart-spaces could play a significant role, namely advertisement, e-recommendations, e-tourism, gaming, etc, opening the path for personalized content creation and distribution, enabling new business models of contextualized applications and service delivery over smart spaces.

3 Technical Aspects of Context-Aware Smart Spaces Before focusing on the business aspects, it is important to understand the technical requirements of smart spaces and how context is managed and represented, using our framework, throughout the entire value chain. 3.1 Context Management Requirements of Smart Spaces The vision of smart spaces includes intelligent environments and context-aware services that react in a sensitive and adaptive way to the presence of humans and/or objects. Telecom related applications as well as energy saving, gaming, chatting, info and payment services are important constituent parts of such smart spaces. The design of such context management system should therefore consider objectives and

16


requirements from various domains, like data management, communication, user ID management, system reliability, privacy, security, etc. The novelty here lies in the utilisation of information from smart spaces for supporting conventional, as well as community created context applications, in a way that has not been attempted before. This section gives an overview of the most common high-level objectives and requirements [7],[18]. 1) Enhance and leverage context data through • filtering mechanisms as well as evaluating and improving its (multidimensional) quality; • aggregation, fusion, inference, and prediction. 2) Efficiently exchange data and provide context services within and between different smart spaces by • establishing rules for context discovery, storage, and distribution; • designing common interfaces for communication within and across smart spaces; • specifying communication mechanisms between smart space artifacts • establishing management procedures for smart space entities owning context data. 3) Ensure end-to-end data security by • monitoring and assuring the non-violation of privacy, security, and trust within and across smart spaces; • guaranteeing the enforcement of rules and policies. These overall objectives yield a set of design requirements for the desired smart space system functionalities. The system needs to be capable of obtaining and representing context data from a wide range of sources, as well as processing and storing it according to rules and policies defined by the artifacts (users, objects, etc.) present in a smart space. Moreover, mechanisms for computation of semantically more abstract context, obtained from simple types of contexts (aggregation, fusion, reasoning, prediction, and inference) have to be made available. This includes the usage of widely used, standardized communication protocols and interfaces. A smart space has to dispose of a (preferably distributed, as described in [19]) context service directory that stores different types of information. Namely, what context services are available and at which location within a smart space. This will allow autonomous reconfigurations depending on the load and the capabilities of the smart space artifacts. Hence, a protocol for handling the arrival of new context sources joining a smart space needs to be available. Finally, mechanisms for identification, authentication and authorization guaranteeing end-to-end security and privacy of user context within and across smart spaces have to be included. 3.2 Context Management Architecture Context, “embedded” into our world and around our lives, can be exploited to enrich communication systems. The architecture proposed in this section is based on a topdown, service scenario-to-architecture component analysis. It is further scrutinized by a bottom-up synthesis of available underlying technologies for service creation and


17

deployment to potential customers within a mobile environment, which possesses already context information regarding its customers. The proposed architecture is based on the producer consumer role paradigm. The system is built upon three basic functional entities: Context Consumer (CxC), Context Broker (CxB) and Context Provider (CxP), where a CxP is a source of the context information, a CxB is an arbitral node handling the context data as a reference point for the overall context in a smart space and CxCs are consumers of context information. The decoupling of provisioning and consumption is important as it impacts on the overall scalability of the system. Further details about this implementation can be found in [20] and [21]. 3.3 Context Modeling and Representation Context information needs to be represented and modeled for being machine interpretable and exchangeable using well-defined interfaces. The goals are to support easy manipulation on many devices, including resource constrained mobile devices, achieve low overhead in keeping the model up-to-date, easy extension, efficient search and query access, having cheap and simple mechanisms for adding new types of information and scalability. In the literature, different approaches for representing contextual knowledge and semantic information can be found [22]. To improve the limitations identified, our work proposes an appropriate option for representation and exchange of context information in smart space environments is ContextML [23], which is based on the XML format. ContextML can handle several layers of abstraction in a context-aware system, as illustrated in Fig. 1, and any context-aware middleware or architecture must therefore be capable of building representations and models of these layered abstractions.

Fig. 1. Layered Context Model

18


The main context management features are context acquisition, context aggregation & fusion, context dissemination, discovery and lookup. Adapting such a structure to existing infrastructures could allow a context handling interface to be embedded into a service exposure layer as a SOA-like web service, allowing secure and controlled access to context information. Nevertheless, privacy issues need to be addressed efficiently in order for such services to become a reality. To accomplish such vision, the work done in [24] addresses some of the most relevant issues.

4 The Business Aspects of Context-Awareness in Smart Spaces This chapter shows how the outcome of the proposed context-aware system can enable new business models in smart spaces. 4.1 Types of Context Before getting into the possible business models, it is important to understand what kind of context can be made available. In this sense, we will separate context into three different categories: real-time, historical and reasoned context. Nevertheless, only real-time information is considered as context in the real meaning of this term. Table 1 provides some examples of these context types. Table 1. Types of Context Available within a Smart Space Real-Time Location Orientation

Historical Previous locations Previous orientation

Temperature Humidity Brightness Noise level

Previous temperatures Previous humidity Previous brightness Previous noise levels

Proximity

Previous proximity

Call/VAS Log Phone profile settings HW version, FW release and SW packages installed Battery level

Previous Call/VAS Logs Previous profile settings Previous devices settings

…

Reasoned Movement Movement, destination Weather Weather Weather, Time User activity, user location Friends, colleagues, customers Relationship Proximity Service type, capability, content format Service usage

4.1.1 Real-Time Context This type of context represents the data available in the cache of the CxB, meaning that the context retrieved is still valid (has not expired). This is particularly useful to trigger events based on real time actions. It can be requested per user, per context or per object (e.g., retrieve users that viewed a specific content in a determined location).


19

4.1.2 Context Prediction for Generating Higher-Level Knowledge Reasoning refers to information that can be inferred from analyzing data and combining different context information. This is usually more complex to obtain and should have a confidence level associated with it. One example for reasoning includes context prediction [25] Context prediction deals with the issue of generating awareness for future context. This implicitly includes the proactive control and adaptation of a system such as a smart space according to (expected) future user behavior. Generally, an entity of a context management system (e.g. context consumer, CxC) can query the future value of the specified context. An exhaustive prediction contains the following parameters: • • •

Name of context scope to be predicted Name of entity the scope belongs to Future point in time the context should be computed for

On the one hand, current context information of the specified entity will be queried in order to get the starting state for the prediction algorithm. On the other hand, historical data of the specified scope and as well as other (related) scopes of that entity will then be retrieved from other entities, e.g. from other CxPs or from context history databases. For example, in case of a location prediction, additional interesting context data could be speed, acceleration and orientation. For the actual prediction process, a set of algorithms for analysis of historical data are available. Suitable candidates include tendency predictors, first or higher order Markov models, hidden Markov models, or Bayesian networks. Depending on the prediction request, the most appropriate algorithm is selected. Again, relevant criteria include time horizon of prediction, type of context (i.e. expected data characteristics), performance of previous predictions for specific applications (e.g. service execution in smart space environments) as well as real time requirements. Another important aspect is the degree to which the algorithm needs supervision, i.e. human interaction. In principal, for smart space environments, prediction should be performed completely unsupervised. As such, the combination or parallel execution of different algorithms shall provide predictions with higher confidence and reliability. 4.2 Relevant Applications and Business Models Context-aware smart spaces may be used by many applications and services. The most interesting application classes that can benefit from context-awareness in smart paces are the following: • • •

sustainability – including assisted living, smart cities, traffic management, intelligent energy management, tele-working, smart remote (business) process control machine to machine communications – including smart factories, logistics, and vehicular communication e-health – including smart hospitals, personalized fitness programs, elderly support from remote, advanced national health and wellness program

20


• •

emergency management, recovery and rescue teams support – fast recovery from catastrophes, remote monitoring, support for rescue forces and efficient material allocation Gaming – including distributed and remote entertainment, education and tourism systems.

Any type of company from both the Telecom and IT industries may leverage on the potential of the presented context-aware applications in smart spaces. Some of the most promising business models applicable in this area will be described in the following sections. 4.2.1 The In-House Model One of the best ways to avoid churn is to offer users what they want and need. Using context intelligence to improve services customization and personalization became a trend and operators are starting to use context to improve or create, existing or new services, respectively. A great example of such deployment is provided by some Telcos where a service merges a simple address book into an immersive social experience. Similar ideas can be employed for enhancing user experience in smart spaces. Although very often third party applications (e.g., Facebook, Twitter) are used, there is not any sort of revenue sharing. With this approach, they improve not only customer satisfaction but also revenue generation resulting from the associated data consumption from the operators' network. Furthermore, as these services are usually associated with fidelity plans, this helps operators to rethink long-term strategies. 4.2.2 The Revenue Sharing Model This particular method can exist in two variants. The first occurs when the Telco improves its offer by using context information from external sources (e.g., social networks, traffic or news provider). This is similar to the outside-in methodology or to a white labeling product/service where the operator leverages its customer base to a new offering and earning a percentage out of it. The second approach resembles the inside-out approach, where third party applications are built not only based on operator infrastructure (using APIs) but also its context information. In this situation, although the Telecoms do not own the end user of these applications; they still earn a part of the revenue generated from this service. 4.2.3 The Advertising Model Advertising can be leveraged by a smart space infrastructure in a myriad of ways. It can be done in a non-intrusive way when the customer requests a specific service (e.g. in a shopping mall), which is then sponsored by advertising. In any case, advertisement can be targeted according to a user’s preferences and context. Therefore, advertisers are no longer limited to little customer segmentation options. In a way, this model is a mix of revenue sharing and context exposure selling as they earn money by using both their infrastructure (e.g., SMS or MMS) and their context information (e.g., location). The main challenge of this model is to provide advertisements in an unobtrusive way, simultaneously considering users’ interests for a specific moment in time. These targeted events will consequently improve the probability of service sales since the offer will match his or her needs, interests and context.


21

4.2.4 Context Exposure Selling Model Operators can extend the usage of their customers’ context to outside their domain in order to improve their long-tail experience and creating revenues from businesses build by third parties. In this model there is no strict correlation between the service offered by third party providers and the operator. In this sense, the Telco exposes context through well-defined APIs. Then, depending on what is necessary the context can be accessed by: • •

•

Request – can be done per user, per group of users, or per context. In this case, the billing is performed on a per request basis or per response basis (both models could work). Subscription – for some services, in case an application needs to keep an updated status of the context (e.g., to track movement you need to update location regularly). For this case, a fee can be applied for the subscription to certain context, to certain users, group of users or a combination of all as the updates can occur without any specific pattern and therefore is difficult to estimate a price per notification. Bundling – bundling context decreases the offer price when compared to single purchases while significantly improving or completing the service offer.

4.2.5 Self-service Model In today’s interactive networks, users are no longer just consumers but also producers. Although for now this concept is mainly associated with content, it is predictable that in the future the same tendency is reflected on service composition and execution, where people can build their own applications based on intuitive graphical interfaces that represent logical interfaces of a myriad of context enablers. In this scenario the operator would also need a service orchestrator based on context. This would pave the way to a whole new way of service creation and consequently revenue generation. In a first instance, it would allow customers to engage with the service and network providers and would accelerate new service development paying back the network operator’s investments in customer tailored services. In a way, this can be seen as the “Application Stores” business model, where customers could then publish their services, using both the operators’ infrastructure and context information. For this to happen, a revenue sharing model together with context exposure billing model should be defined to charge an intermediate or an end user. 4.3 Limitations Despite the simplicity and flexibility in which these business models can be implemented, the processing and management of personal data obviously raises security and privacy issues that need to be addressed. In this sense, users need to stay in control of the whole process, enabling them to specify when, what, why, who, where and how the data is or can be accessed. In other words, user context should be disclosed according to contextual privacy policies and settings. Even the most primitive model should allow users to opt-in and opt-out from such profiling. Currently, on an average, only 3% of an operator’s costumer base opts-in for such services. Therefore, it is necessary to improve these business models in order to attract more end users. Otherwise, largescale context exposure and distribution will never be available.

22


In addition, it is necessary to achieve a consensus and agreement among different stakeholders so that information can be provided in a standardized way. This will allow for compatibility within and across different smart spaces and it will facilitate the reusability of applications in different smart spaces. Examples include developerindependent software APIs, as well as manufacturer-independent “hardware APIs”. Moreover, context shall be efficiently acquired, encoded and transformed into a determined high-level context, distributed and provided to requesting entities for consumption, while simultaneously maintaining its real-time properties. This is a very complex and resource-consuming task requiring appropriate context sources, very efficient supporting context management systems with embedded intelligence, employment of interoperable and lightweight standards, fast communication channels and secured interfaces for context retrieval and distribution. The main difficulty in the creation of intelligent and comprehensive smart spaces is “filling” the spaces with myriads of the sensors, which require huge investments. Therefore the users’ mobile terminals should be considered as sensor devices in smart spaces.

5 Trials of New Applications and Services Quasi-smart Space Environments TI is already providing some context-aware services tied to a certain location or a territory to its customers especially for mobile users in the eTourism field, where the B2B2C model is implemented with the following elements: • • • • •

the development of a model of integrated and aggregated offer on a local basis, or "under the patronage" of public administrations responsibility; the involvement of the PAs (Public Administrations) for the sponsorship of the model and the involvement of the proper actors active on the business value chain; the collection and delivery of content (e.g., information for the enhancement of heritage artists); a strong leverage on mobile devices for the end customer and on TI’s infrastructure based on assets (connectivity, data centers, Telco Capabilities) solutions; the hypothesis of conditional business to the establishment of a collaborative model based on a local public-private integration.

The different roles and their interactions in the TI’s eTourism scenario are shown in Fig. 2 where the CA&SA Provider is responsible for integrating context information and self-adapts it into its services. To some actors in the network this context information has an economic value. Therefore, it is assumed that actors do not give away data or services without an adequate compensation. Consequently, the analysis here has been done in terms of value exchanges between the different actors. By playing the role of context enabler, TI limited its involvement to providing network data that could enable context reasoning. In any case, it could be an important distributor of context data in the market of CA&SA services. By playing the role of context


23

provider, TI would increase the involvement in the organization of these services and would trade context data from different sources. There are no sensor networks involved in the described services as abovementioned, however, mobile terminals and location-aware services can be considered as sensors. Therefore, the described environment is a first step towards a field trial smart-space facilitated by a Telecom operator. In the presented scenario, it plays the role of network operator, service provider and seller. This combination of roles puts TI in a privileged position to collect user and network context data.

CA&SA Service Provider

Fig. 2. View of different roles and their interactions in a general eTourism service scenario

The business model used in the aforementioned services was based on the bundling of a customized platform/service solution to third parties interested in a promotion of certain initiatives and events at large national, regional and local scales. These entities, (e.g., City Hall, Government, Event sponsors) interested to involve and to attract, for any reason (e.g., advertisement, social visibility, tourism), as much people as possible, have now a way to target this offer within a specific and contextualized scope. A prepared ad-hoc system usually involves context management platforms, preparation and customization of the website and its continent, where users can register and upload their content. Then different interactions are offered, voting, rating, the best content, the most popular place, as well as interactive game scenarios where every participant is invited and potentially involved. All these web portals [26] leverage User Generated Content (UGC) upload and sharing capabilities, mainly based on the user location, including the proximity to objects and other people, presence status, willingness proposition, preferences, and social contexts.

24


6 Conclusions and Future Work In this work we have presented the business potential of context-aware systems, demonstrated by a technical solution implemented at TI premises for trial concepts and scenarios. Moreover, we have highlighted context management requirements for smart spaces. As seen, most use cases involve more than two entities (a smart space consists of many active entities), therefore disrupting the traditional business models. Despite the presented limitations, we have shown that small variations to well-known models can generate positive financial flows. Furthermore, it became clear that the application of context-awareness as wide spread technology, oriented to the social usage and treating personal data, shall be further analysed for potential issues regarding privacy, business models and law regulations before it can be widely adopted and deployed in smart spaces. Nevertheless, when context-awareness and context-data are exposed as a business enabler to third parties, in a guaranteed and protected way, all entities within the value chain will be able to deliver attractive solutions. Still lacking is the definition of the right pricing structure with an appropriate revenue sharing scheme. Altogether, this work will leverage context exposure over Web2.0 interfaces and other new business propositions.

Acknowledgments This work is supported by the European ICT project “C-CAST” - Context Casting (Contract-No. ICT-216462) as well as by the project “G-Lab” (funding reference 01 BK 0808) funded by the German Ministry for Education and Research.

References 1. Context Casting (C-CAST), FP7-ICT-2007-1, http://www.ict-ccast.eu/ 2. Ryan, N., Pascoe, J., Morse, D.: Enhanced reality fieldwork: the context-aware archaeological assistant. In: Proceedings of 25th Anniversary Computer Applications in Archaeology (1997) 3. Dey, A.K., Abowd, G.D.: Towards a better understanding of context and contextawareness. In: Proceedings of Work on the What, Who, Where, When and How of Context-Awareness. ACM Press, New York (2000) 4. Hull, R., Neaves, P., Bedford-Roberts, J.: Towards situated computing. In: Proceedings of the First International Symposium on Wearable Computers (ISWC ’97), p. 146 (1997) 5. Brown, P.J.: The stick-e document: a framework for creating context-aware applications. In: Proceedings of Electronic Publishing, Palo Alto, pp. 259–272 (1996) 6. Zimmermann, A.: Context Management and Personalisation, p. 260 (2007) 7. Oh, Y., et al.: A Context Management Architecture for Large-Scale Smart Environments. IEEE Communications Magazine (March 2010) 8. Casdell-Oliver, R., Liu, W.: Representation and recognition of Situations in Sensor Networks. IEEE Communications Magazine (March 2010) 9. Want, R., Hopper, A., Falcao, V., Gibbons, J.: The active badge location system. ACM Transactions on Information Systems 10(1), 91–102 (1992)


25

10. Ailisto, H., Alahuhta, P., Haataja, V., Kylloenen, V., Lindholm, M.: Structuring context aware applications: Five-layer model & example case. In: Proceedings of Workshop on Concepts & Models for Ubiquitous Computing, Sweden (2002) 11. Dey, A.K., Abowd, G.D.: A conceptual framework and a toolkit for supporting rapid prototyping of context-aware applications. HCI Journal 16(2-4), 7–166 (2001) 12. Gu, T., Pung, H.K., Zhang, D.Q.: A middleware for building context-aware mobile services. In: Proceedings of IEEE Vehicular Technology Conference (VTC), Milan, Italy (2004) 13. Fahy, P., Clarke, S.: CASS – a middleware for mobile context-aware applications. In: Workshop on Context Awareness, MobiSys 2004 (2004) 14. Chen, H., Finin, T., Joshi, A.: An Intelligent Broker for Context-Aware Systems. In: Article, Adjunct Proceedings of Ubicomp 2003 (2003) 15. Roman, M., et al.: GAIA: A Middleware Infrastructure to Enable Active Spaces. IEEE Pervasive Computing 1(4), 74–83 (2002) 16. Service Platform for Innovative Communication Environment (SPICE), FP6 EU Project, http://www.ist-spice.org 17. Salis, N., Licciardi, C., Hallsteinsen, S., Geihs, K.: Telecom Italia’s MUSIC Exploitation: Mobile eTourism, D13.12, FP6 EU Project MUSIC (December 2009) 18. Mannweiler, C., et al.: A Robust Management Platform for Multi-Sensor Location Data Interpretation. In: Proceedings of the 19th Future Network and Mobile (2010) (to be published in June 2010) 19. Mannweiler, C., Amann, B., Schneider, J., Klein, A., Schotten, H.D.: A Distributed Context Service Architecture for Heterogeneous Radio Environments. To be published in ITG-Fachbericht der 15. ITG Fachtagung Mobilkommunikation, Osnabrück, Germany (May 2010) 20. Moltchanov, B., Knappmeyer, M., Licciardi, C.A., Baker, N.: Context-Aware Content Sharing and Casting. In: Proceedings of ICIN 2008, Bordeaux, France (October 2008) 21. Liaquat, S., Knappmeyer, M., Baker, N., Moltchanov, B.: A Federated Broker Architecture for Large Scale Context Dissemination. To be published in Proceedings of The 2nd International Symposium on Advanced Topics on Scalable Computing, in conjunction with The 10th IEEE International Conference on Scalable Computing and Communications (ScalCom-2010), Bradford, UK, June 29-July1 (2010) (to be published in July 2010) 22. Baldauf, M., Dustdar, S., Rosenberg, F.: A survey on context-aware systems. International Journal of Ad Hoc and Ubiquitous Computing 2(4), 263–277 (2007) 23. Knappmeyer, M., Liaquat, S., Frà, C., Baker, N., Moltchanov, B.: ContextML A LightWeight Context Representation and Context Management Schema. In: Proceedings of IEEE International Symposium on Wireless Pervasive Computing, Modena, Italy, May 5-7 (2010) 24. Simoes, J., Weik, P., Magedanz, T.: The Human side of the Future Internet. In: Towards the Future Internet, pp. 183–192. IOS Press, Amsterdam (2010) doi:10.3233/978-1-60750539-6-183 25. Mayrhofer, R.M.: An Architecture for Context Prediction. Dissertation, Johannes Kepler University Linz (2004) 26. http://teamlife.telecomitalia.it/lucca, http://teamlife.telecomitalia.it/lecce, http://teamlife.telecomitalia.it/venezia, http://beta.teamlife.it/bs

A Middleware Architecture for Safety Critical Ambient Intelligence Applications Antonio Coronato and Giuseppe De Pietro Institute of High Performance Computing and Networking ICAR-CNR Via P. Castellino 111, 80131, Naples - Italy [email protected], [email protected] Abstract. Ambient Intelligence (AmI) technologies are more and more often adopted in scenarios, like healthcare, environments monitoring and control, ambient assisted living, etc, which are critical for human safety and assets preservation. This requires that such a kind of applications be designed taking into account stringent dependability requirements. This paper presents a service-oriented middleware architecture for safety critical Ambient Intelligence applications, which provides services that enable the designer to develop runtime verification mechanisms; that is, it is possible to continuously monitor the verification of correctness properties defined at design time.

1

Introduction

Ambient Intelligence (AmI) is the emerging computing paradigm, which derives from Pervasive Computing, that refers to electronic environments that are sensitive and responsive to the presence of people [4]. In the past few years, many pervasive computing environments have been developed and deployed to realize smart rooms, active spaces and context-aware applications. All these environments have been devised with the aim of enabling users to derive benefit from technology without being distracted by it, in order to augment productivity, collaboration, and access to technology. A recent trend is to use these technologies in new domains like hospitals, old people’s homes, emergency scenarios, etc. But, such kinds of environment present a new criticality especially concerning the issues of expressing requirements, verifying and validating them, ensuring functional correctness, etc. These applications must be considered safety critical and, consequently, should be designed taking into account such a criticality. As an example, let us consider the application described in [14]; it mainly consists in a tele-monitoring system that processes the stream of physiological data coming from an Electrocardiogram (ECG) sensor monitor of a cardiopathic patient at home. Data must be processed in real-time; i.e. by granting ”results” within a certain temporal period. Moreover, in the case of critical values, alerts must be raised and processed by a temporal threshold. As also emphasized by the authors, functional incorrectness or run time failures may result in catastrophic consequences for the patients monitored. As a consequence, such considerations require that this class S. Balandin et al. (Eds.): ruSMART/NEW2AN 2010, LNCS 6294, pp. 26–37, 2010. c Springer-Verlag Berlin Heidelberg 2010

Safety Critical Ambient Intelligence Applications

27

of systems be designed and developed taking into account intrinsic and stringent dependability requirements. In this paper, we present a service oriented middleware architecture that provides a set of basic mechanisms for rapidly prototyping safety critical Ambient Intelligence applications. In addition to other middleware infrastructure, the one proposed in this paper presents a specific section (set of services) that enables the specification of correctness properties and their check while the system is running. So called runtime verification techniques are adopted both to tune the system in its early running and to monitor its correct behavior once deployed definitively. The rest of this paper is organized as follows. Section 2 presents our motivation and related work. Section 3 describes the middleware architecture. Section 4 focuses on runtime verification mechanisms for correctness properties. Section 5 presents a case study. Finally, Section 6 concludes the paper.

2

Motivation and Related Work

A remarkable example of safety critical AmI applications is the class of Ambient Assisted Living (AAL), which regards the realization of Smart Environments to monitor elderly and ill people. A middleware infrastructure for AAL applications is the PERSONA project (Perceptive Spaces prOmoting iNdepentent Aging) [15], which aims at developing a scalable open standard technological platform for building a broad range of Ambient Assisted Living Services. This is an ongoing project funded by the European Commission. The main technical challenge is the design of a self-organizing middleware infrastructure allowing the extensibility of component/device ensembles. The communication patterns of the infrastructure are based on distributed coordination strategies for service discovery and utilization. The current prototype of the PERSONA middleware, implemented on the OSGi platform [12], uses connectors based on UPnP, Bluetooth, and R-OSGi [18]. However, the development of such a kind of applications presents new criticalities depending on their intrinsic characteristics, as for an example the dependency of the system behavior from the user and resources location and movements. Such criticalities can partly be faced by means of formal methods [17]. On the other hand, the use of formal methods in the field of pervasive computing is a relatively new trend and most of existing applications rely on Ambient Calculus (AC ) [3] that provides interesting primitives for the designer of pervasive systems. As an example, Campbell and Ranganathan [2] have argued the need for the formal specification of pervasive computing environments and have proved the correctness of some elements of their system GAIA by means of AC. In previous papers, we have defined a methodology that enables the designer to specify formally an Ambient Intelligence or a Pervasive Computing application [5] [6]. However, formal methods are typically not confined to the specification phase. Indeed, they can be adopted to prove the correctness of the specification or the

28

A. Coronato and G. De Pietro

implementation. In particular, Model Checking, that along with Theorem Proving and Static Analysis is a technique aiming at static program verification [13], which is concerned with checking if all possible traces derived from a program (or abstract model) satisfy a property of interest. In contrast to these techniques, this paper focuses on runtime verification which is concerned with checking traces of events generated from the program run against properties described in the proposed logic. When a property is violated or validated, the program can take actions to deal with it. The technique can be used both for testing before deployment of the software, and for monitoring after deployment. In the first case, one must come up with test cases [1] that might uncover a bug. In this setting, runtime verification is considered as an auxiliary tool to automate the creation of oracles that detect errors. In the second case, runtime verification is used to monitor a program during its execution so to take actions in response to the violation of properties. With this perspective in mind, a runtime verification tool may be used to define how the program reacts to bugs, possibly steering it to the correct behavior [11]. It is useful to note that runtime verification techniques can be adopted also to tune the system. This is particularly true for pervasive applications whose behavior can strongly depend on operating conditions. As an example, an RFID reader can fail to detect a tagged user depending on i) the orientation of the antenna, ii) the power of the antenna itself, iii) the position of the tag on body, iv) the existence of an object between the tag and the reader at the time of reading, etc. All these conditions may be very difficult to verify with static techniques. In contrast, runtime verification of the prototype system in the real domain of application can make possible the collecting of traces that can be used to tune the system. It is important to note that runtime verification techniques typically concern real-time properties [16], but in this paper we extend the verification to properties related to physical locations.

3

Middleware Architecture

In this section, we describe the overall middleware architecture. Figure 1 shows the key services, layers and modules of the proposed middleware. The bottom-left section regards Human Computer Interaction. It includes mechanisms for handling natural and advanced interactions in a smart environment. Indeed, one of the grand challenges for future smart environment applications is making interaction with technology-rich environments intuitive, natural, unobtrusive, intelligent, and distraction free. Currently, two services have been developed, a TextToSpeech Service, which is in charge of synthesizing vocal indications, and a Messaging Service that is in charge of sending textual messages to the user’s mobile device [7]. The bottom-right section is about Sensing and Controlling. This section includes hardware sensors and actuators. These two sections are structured in different layers. The lowest layer includes hardware devices. The heterogeneity of the components at this layer is handled by means of software components,


29

Fig. 1. Middleware architecture

namely facilities, which hide the hardware components to upper layers. For example, the RFID Facility provides support to handle an RFID antenna; thus, each RFID antenna has its own facility that will handle interactions between the hardware component and the rest of the software system. Upon this layer, a Facility Service handles the set of homogeneous facilities that stand below; i.e. a facility service catches information coming from the set of facilities that are related to the same kind of hardware device, RFID antenna for the case of RFID. These services are in charge of capturing data coming from hardware components and handling them as an integrated set of information for the upper levels. Thus, the RFID Facility Service (one of the facility services not shown in the figure) collects information coming from any RFID antenna deployed within the environment. The middle section offers communication functionalities. In particular, in addition to RPC, an asynchronous communication mechanism is exposed by the Event Service. This service is in charge of dispatching events between software components. It supports the publish-subscribe mechanism. Further details can be found in [8]. The topmost section concerns the context. The services defined in this layer are: – Topology Service - This service provides a unique and uniform representation for the topology of the environment. In particular, locations are classified as semantic locations (buildings, floors, rooms, an area in front of a wall monitor, etc) and physical locations (the area sensed by an RFID antenna, the area covered by a WiFi access point, etc.) [9]. – Tracking Service - This service offers a mechanism to track all movements of a mobile resource or user.

30


– Location Service - This component is in charge of handling physical location information for mobile resources and users. In particular, it locates mobile users by means of the RFID technology and WiFi enabled mobile devices by means of classic location techniques [9]. – User Service- This service provides basic authentication mechanisms for users and handles the list of all users active in the environment. – Resource Service - This component extends standard mechanisms for registering and monitoring resources, like laptops, PDAs and sensors. Finally, the correctness section includes mechanisms for runtime verification. The correctness properties are defined with respect to the state of an ambient, which is a physically bounded entity of the domain. An ambient may be i) a location of the environment (e.g. a room in which only authorized people can enter); ii) a resource (e.g. a constrained mobile device that can execute only lightweight software agents); or iii) a person (e.g. a user that can invoke some services only after having been identified); etc. The correctness properties can also be defined with respect to temporal properties (i.e. real-time constraints). The set of services defined for runtime verification consists of: – RunTimeStateHolder Service - This is the component that holds and exposes the state of an ambient. – RunTimeChecker Service - This is the component that checks, on behalf of an ambient, the properties of hosted, entering and leaving ambients. – Timer Service - This is the component that holds and verifies temporal constraints. – Monitoring Service- This service provides basic mechanisms for monitoring the entire system.

4

Runtime Verification

In this section, the services for runtime verification are described. Runtime verification is divided into two parts: instrumentation and event observation. A monitor receives events from the executing program, emitted by event generators inserted during instrumentation, and dispatches them to a collection of algorithms, each of which performs a specialized trace analysis. In the following, both the structural and behavioral models are presented. The structural model is reported in figure 2 and consists of a set of software components that instrument the running system in order to be monitored by an external monitor, the Monitoring Service. It is important to note that in addition to real-time properties we are interested in correctness properties that are related to locations. 4.1

Structural Model

The detailed architecture is: RunTimeStateHolder This component supports the following structure:


31

– S - This is the state of the current ambient; – ES - This is the list of all admissible states for an entering ambient; i.e. another ambient must verify one of these states in order to enter the current ambient; – LS - This is the list of all admissible states for a leaving ambient; – HS - This is the list of all admissible states for a hosted ambient; – getState - This operation returns the current state; – setState - This operation updates the current state to a new value; – getHostedStates - This operation returns the list of admissible states for a hosted ambient; – setHostedStates - This operation sets the list of admissible states for a hosted ambient; – getEnteringStates - This operation returns the list of admissible states for an entering ambient; – setEnteringStates - This operation sets the list of admissible states for an entering ambient; – getLeavingStates - This operation returns the list of admissible states for a leaving ambient; – setLeavingStates - This operation sets the list of admissible states for a leaving ambient; RunTimeChecker This component supports the following operations: – canEnter - This is the operation that verifies whether an ambient (say for example b) can enter or not the current ambient (a); i.e., when the ambient b tries to enter into a, this method verifies whether the current status of b is admitted by a, or not; – canLeave - This is the operation that verifies whether an ambient (b) can leave or not the current ambient (a); – canHost - This is the operation that verifies whether an ambient can be hosted or not by the current ambient. Timer This component supports the following structure: – TimeToExpire - This is the time the timer has to wait before expiring; i.e., this is the temporal range to complete a real-time operation; – TimeToAlert - This is the time the timer waits before raising an alert. It holds the property (TimeToAlert < TimeToExpire). This parameter, which is optional, is used to send an alert a certain time before the timer expires; – setTimer - This is the operation that sets the timer’s parameters; – startTimer - This is the operation that starts the timer; – ariseAlert - This is the operation that raises an alert before the timer expires; – expire - This is the operation that sends a message when the timer expires.

32


Fig. 2. Pattern Structural View

4.2

Behavioral Models

The case of an action of type enter in is detailed. In particular, the ambient b wants to enter the ambient a. For this case, two alternative interaction models are specified. In figure 3(a), the ambient b itself is in charge of verifying that the condition Enabled for entering the ambient a is satisfied. Eventually, the ambient b is considered trustworthy. In contrast, in figure 3(b), the ambient a is in charge of verifying incoming ambients, but, it must be notified of this action in some way.

(a) Incoming centered responsibility

(b) Hosting centered responsibility

(c) Change of the State of a Hosted Ambient Fig. 3. Capability in - Pattern Behavioral Views

Finally, in figure 3(c) the case of change of the state for a hosted ambient is reported. In this case, the hosted ambient (b in the figure) must notify its new state to the hosting one (a). After that, the hosting ambient verifies that the new state is compliant with the states admissible for hosted ambients.


5

33

Case Study

In this section, we present a case study. The system being designed is a Pervasive Healthcare Application for the Department of Nuclear Medicine of the University Hospital ”Secondo Policlinico” of Naples. 5.1

Informal Description

Within the Department, patients are injected with a small amount of a radioactive substance in order to undergo specific examinations (e.g. Blood Volume Study, Bone Scan, Brain Scan, etc.). Once injected, patients emit radiation (gamma rays) and have to stay in a specific room to wait for the examination to be performed. The time patients have to wait depends on the kind of examination that has to be performed and the time the radioactive substance takes to propagate -within the body- and to decay to the right level. In fact, examinations can be executed only if the radiation level is in a certain range. After the examination, patients return to the waiting room until the level of radiation becomes less than a specific threshold and, so, harmless. The department consists of the following rooms: 1) Acceptance Room (AcceptanceRoom) -This is the room where patients are accepted within the department and wait for injection; 2) Injection Room (InjectionRoom) -This is the room where patients receive the injection; 3) Hot Waiting Room (HotWaitingRoom) -This is the room where patients wait for the examination after having been injected and until the radiation level reaches the correct range; and 4) Diagnostic Room (DiagnosticRoom) -This is the room where examinations are performed. Currently, patients are accompanied by nurses when moving within the department. The goal is to have an automatic system that would lead them. The system will adopt RFID readers and tags to identify and locate patients within the department. Mobile devices will also be used to send messages to patients in order to invite them to move (e.g. ”Move to the injection room”, ”exit the department”, ”you are not allowed to enter this room”, etc.). In this way, nurses will no longer be necessary to guide patients within the department. This will have two benefits: lower costs and fewer people exposed to radiation. 5.2

A Peace of the Formal Specification

First, the following classes of ambients are concerned: {Department, Room, Patient, RFIDReader, RFIDTag, MobileDevice, Message}

It is possible to specify the status of each ambient. The injection room and diagnostic room can be either ’WORKING’ or ’NON WORKING’. When ’WORKING’, it can be ’IDLE’ -no patient inside- or ’BUSY’. The patient is ’WAITING TO BE INJECTED’ (W) before having the injection and ’INJECTED’ (I) after the injection. Next, when the radioactive substance decays to the right level for the examination, he is ’READY’ (R) to be examined. After the examination his status

34


is ’EXAMINED’ (E). Finally, when the radioactive substance becomes harmless, he is ’DECAYED’ (D). It is also possible to assume that other classes of ambients are stateless. The following formulas specify these possible states: Room ∈ {NON WORKING,WORKING*IDLE,WORKING*BUSY}

(1)

Patient ∈ {W,I,R,E,D}

Now, let us focus on some dynamic aspects. As an example, it is possible to formalize the path of a patient who has to move from the hot waiting room to the diagnostic room in order to undergo the examination. Before this, some constraints regarding this movement are expressed. Indeed, concerning the hot waiting room, a patient can enter only when he is just injected or examined and can exit only when ready to be examined or decayed. Therefore, the following constraints must be guaranteed: Patient canEnter(HotWaitingRoom), iff Patient ∈ {I,E}

(2)

–A patient can enter the hot waiting room only if he is INJECTED or EXAMINED Patient canLeave(HotWaitingRoom), iff Patient ∈ {R,D}

(3)

–A patient can leave the hot waiting room only if he is READY or DECAYED HotWaitingRoom canHost(Patient), iff Patient ∈ {I,R,E,D}

(4)

–A patient can stay in the hot waiting room only if he is INJECTED, READY, EXAMINED or DECAYED

It is worth noting that, on one hand, the specification proposed so far describes in an unambiguous way part of the application domain and behavior of its users; on the other hand, constraints and properties (e.g. canEnter ) will affect the software system. One of the most important constraints for the application regards the minimum and maximum time to wait for the examination after the patient has received injection. This is specified by the following formula: T MIN τ desk ).

1 Main speaker (presenter)

Personal webcameras assigned with participants sitting at the conference table

0 1

Personal web-camera

0

s2

1 0 1

s3

0 1

c1

0 1

c3i

c4i

c5i

Touch input was used (beginning time of touchscreen using t desk is saved) A speaker in the presentation area is not observed by the video monitoring system Speaker is found in the presentation area Speaker face is not found Speaker head is directed to (the face tracking system founded presenter face) Speech activity in the presentation area is not detected A presenter gives a speech (the sound source localization system detected an activity in the presentation area) Currently there are no speakers at the conference table Currently a participant at the conference table gives a speech comment Personal web-camera i is turned-off

s1

c2i

− t slide > τ slide , where tcur is current time).

i

is turned-on

0

No participant in front of the web-camera

1

There is a participant in front of the web-camera i (the video monitoring system estimates degree of changing image background recorded before the meeting) No speech activity of the participant sitting in front of the webcamera i

0

i

1

A participant sitting in front of the web-camera i gives a speech comment (the multichannel speech activity detection system determines useful signal in the audio channel of the web-camera i )

0

Face of the participant sitting in front of the web-camera i is not found Face position of the participant sitting in front of the web-camera i is detected (the face tracking system found the participant face)

1

Let’s consider the process of graphical content compilation in the forms. Each graphical form F j on the web-page is described by the following tuple

F j = l j , u j , w j , h j , g j , where l j is upper left corner position of the form at the

Multichannel System of Audio-Visual Support of Remote Mobile Participant

69

abscissas axis, u j is upper left corner position of the form at the ordinates axis, w j is the form width, h j is the form height, g j is a graphical content of the form, which is actual and was chosen from the set G . Sizes of the forms could be changed depending on the current features of browser used in the client device. In the forms meeting ( PC ) , F2meeting ( MD ) the number of graphical component is F2meeting ( PC ) , F4 changed depending on the parameter values mentioned in Table 3. In other forms the graphical component numbers are kept during the whole meeting. Selection of the current graphical component g ∈ G for the referred forms is realized by a logicaltemporal model of compiling the graphical interface web-page. The following set of logical rules is an essence of the model: ⎧G3 , s1 ∧ s 2 ∧ s3 ∧ ¬c1 , ⎪G , ¬s ∧ ¬c ∧ (( p ∧ p ) ∨ (d ∧ d )), , ⎪ 4 2 1 1 2 1 2 g 2meeting ( PC ) = ⎨ G s c , ¬ ∧ , 3 1 ⎪ 6 ⎪⎩G9 , overwise. ⎧G1 , ( p1 ∧ p 2 ∧ (¬d 1 ∨ ¬d 2 )) ∨ ( p1 ∧ p 2 ∧ p 3 ∧ d 1 ∧ d 2 ∧ ¬d 3 ) ∨ ⎪ ( p ∧ p ∧ p ∧ d ∧ d ∧ d ∧ (t slide > t desk )), 1 2 3 1 2 3 ⎪ , ⎪ , (( ) ) ( ¬ ∨ ¬ ∧ ∧ ∨ ∧ G p p d d p p 2 ∧ ¬p 3 ∧ d 1 ∧ d 2 ∧ d 3 ) ∨ ⎪ 1 2 1 2 1 g 4meeting ( PC ) = ⎨ 2 ⎪ ( p1 ∧ p 2 ∧ p 3 ∧ d 1 ∧ d 2 ∧ d 3 ∧ (t slide < t desk )), ⎪G 9 , (¬p1 ∨ ¬p 2 ) ∧ (¬d 1 ∨ ¬d 2 ), ⎪ ⎪⎩G 4 , overwise . ⎧G1 , (( p1 ∧ p 2 ∧ (¬ d 1 ∨ ¬d 2 )) ∨ ( p1 ∧ p 2 ∧ p 3 ∧ d 1 ∧ d 2 ∧ ¬ d 3 ) ∨ ⎪ ( p1 ∧ p 2 ∧ p 3 ∧ d 1 ∧ d 2 ∧ d 3 ∧ (t slide > t desk ))) ∧ ¬ s 2 ∧ ¬c1 , ⎪ ⎪G 2 , (((¬ p1 ∨ ¬ p 2 ) ∧ d 1 ∧ d 2 ) ∨ ( p1 ∧ p 2 ∧ ¬ p 3 ∧ d 1 ∧ d 2 ∧ d 3 ) ∨ . ⎪ ( p1 ∧ p 2 ∧ p 3 ∧ d 1 ∧ d 2 ∧ d 3 ∧ (t slide < t desk ))) ∧ ¬ s 2 ∧ ¬c1 , ⎪ meeting ( MD ) = ⎨ g2 ⎪G 3 , ¬ p 3 ∨ ¬ d 3 ∧ s1 ∧ s 2 ∧ s 3 ∧ ¬c1 , ⎪G 4 , ¬ p 3 ∨ ¬ d 3 ∧ ¬ s 2 ∧ ¬c1 , ⎪ ⎪G 6 , ¬ p 3 ∨ ¬ d 3 ∧ ¬ s 3 ∧ c1 , ⎪current component is saved , overwise . ⎩

The component G5 , which consists of actual images of participants sitting at the conference table, is compiled by an analysis of states of personal web-cameras and presence of participants and faces on frame. Let’s identify an set of images from the web-cameras as W = {W1 ,W2 ,...WN } , where N W is a number of the web-cameras W

mounted on the conference table (in the developed meeting room N W = 10 ). Then the component G5 consists of the images captured by turned-on web-cameras, in which a NW

participant is detected: G5 = ∪ (Wi c2i ∧ c3i = 1) . Taking into account limited sizes of i =1

the forms used for representing the component G5 , the number of displayed participants is reduced by an analysis of his speech activity c4 i and/or presence of his face

70

A.L. Ronzhin, V.Y. Budkov, and A.A. Karpov

in the frame c5i . Particularly, the form F1meeting ( MD ) for mobile device contains up to three participant images, so both parameters are used for selection of more active participants: NW

F1meeting ( MD ) = ∪ (Wi c2i ∧ c3i ∧ c4i ∧ c5i = 1) . i =1

The proposed logical-temporal model of compilation of web-page graphical interface was tested on a personal computer and several models of smartphones donated by the Nokia company. Table 3 presents some examples of web-page content during a meeting. The sound source localization system and multichannel speech activity detection system were used for selection of source of audio stream transmitted to remote participant. Speech of a presenter is recorded by the microphone of the web-camera closest to the presentation area. The built-in microphone of the web-camera assigned with the presently active participant sitting at the conference table is used for recording his/her speech. Table 3. Content examples of the web-based application for meeting Screen of client device Personal computer

Mobile device

Presentations

Registration

Meeting state

The developed web-page layout is oriented on passive remote participants. To enhance his potentials a toolbar allowing a participant located outside the meeting room to give a question to the presenter and to share the current discussion.

Multichannel System of Audio-Visual Support of Remote Mobile Participant

71

4 Conclusion Development of the multichannel system for speech activity analysis, which is used for preparation the multimedia reports of distributed meetings, contributes to saving costs of verbatim record preparation, improving quality of teleconferences, as well as provides statistics of meetings and convenient archive access and retrieval. The developed speaker diarization system uses the algorithm for speech activity detection in the multichannel audio stream that provides appropriate quality of phrase boundaries determination and automatic selection of the web-camera of the active speaker. The proposed logical-temporal model of compilation of web-page graphical interface allows remote participants to perceive whole events in the meeting room via a personal computer or smartphones. Acknowledgments. This work is supported by The Russian Program “Research and Research-Human Resources for Innovating Russia in 2009-2013” (contract П2360), RFBR (projects 08-08-00128-а, 08-07-90002-СТ_а) and Nokia.

References 1. Yankelovich, N., Kaplan, J., Simpson, N., Provino, J.: Porta-person: telepresence for the connected meeting room. In: Proceedings of CHI 2007, pp. 2789–2794 (2007) 2. McCowan, I., Gatica-Perez, D., Bengio, S., Moore, D., Bourlard, H.: Towards computer understanding of human interactions. In: Aarts, E., Collier, R.W., van Loenen, E., de Ruyter, B. (eds.) EUSAI 2003. LNCS, vol. 2875, pp. 235–251. Springer, Heidelberg (2003) 3. Waibel, A., Steusloff, H., Stiefelhagen, R.: CHIL—Computers in the human interaction loop. In: Proc. of 5th IAMIS Workshop (2004) 4. Danielson, T., Panoke-Babatz, U., et al.: The AMIGO project: Advanced Group Communication Model for Computer-based Communication Environment. In: Proc. of CSCW’86 (1986) 5. Kaiser, E.: Can Modeling Redundancy in Multimodal, Multi-Party Tasks Support Dynamic Learning? In: CHI ’05 Workshop: CHI Virtuality 2005 (2005) 6. Op den Akker, R., Hofs, D., Hondorp, H., op den Akker, H., Zwiers, J., Nijholt, A.: Supporting Engagement and Floor Control in Hybrid Meetings. In: Esposito, A., Vích, R. (eds.) COST Action 2102 2008. LNCS (LNAI), vol. 5641, pp. 276–290. Springer, Heidelberg (2009) 7. Fredouille, C., Evans, N.: The influence of speech activity detection and overlap on speaker diarization for meeting room recordings. In: INTERSPEECH 2007, pp. 2953– 2956 (2007) 8. The 2009 (RT-09) Rich Transcription Meeting Recognition Evaluation Plan (2009), http://www.itl.nist.gov/iad/894.01/tests/rt/2009 9. Tranter, S., Reynolds, D.: An Overview of Automatic Speaker Diarization Systems. IEEE Trans. ASLP 14(5), 1557–1565 (2006) 10. Markov, K., Nakamura, S.: Never-Ending Learning System for Online Speaker Diarization. In: IEEE ASRU Workshop, pp. 699–704 (2007) 11. Pfau, T., Ellis, D., Stolcke, D.: Multispeaker Speech Activity Detection for the ICSI Meeting Recorder. In: IEEE ASRU Workshop, pp. 107–110 (2001) 12. Dines, J., Vepa, J., Hain, T.: The segmentation of multi-channel meeting recordings for automatic speech recognition. In: ICSLP 2006, pp. 1213–1216 (2006) 13. Laskowski, K., Jin, Q., Schultz, T.: Crosscorrelation based multispeaker speech activity detection. In: INTERSPEECH 2004, pp. 973–976 (2004)

On-the-Fly Ontology Matching in Smart Spaces: A Multi-model Approach Alexander Smirnov1, Alexey Kashevnik1, Nikolay Shilov1, Sergey Balandin2, Ian Oliver2, and Sergey Boldyrev2 1 SPIIRAS, 39, 14 line, 199178, St. Petersburg, Russia {smir,alexey,nick}@iias.spb.su 2 Nokia Research Center, Itämerenkatu 11-13, 00180, Helsinki, Finland {sergey.balandin,ian.oliver,sergey.boldyrev}@nokia.com

Abstract. Proper functioning of smart spaces demands semantic interoperability between knowledge processors connected to it. As a consequence it is required to develop models that would enable knowledge processors to translate on-the-fly between the internal and smart space ontologies. This paper presents the developed multi-model approach to the above problem, where the major elements of the selected approach are described in detail. Keywords: Smart spaces, ontology matching, semantic similarity, semantic interoperability.

1 Introduction Smart-M3 is an open source software platform [1] that aims to provide "Semantic Web" [2] information sharing infrastructure between software entities and various types of devices. The platform combines ideas of distributed, networked systems and Semantic Web [3]. The major application area for Smart-M3 is the development of smart spaces solutions, where a number of devices can use a shared view of resources and services [4, 5]. Smart spaces can provide better user experience by allowing a user to flexibly bring in new devices and seamlessly access all the information in the multi-device system from any of the devices. The reference model of the Smart-M3 smart spaces is shown in Fig. 1 and it is close to the Linda ideology [6, 7]. The smart space is implemented by one or several Service Information Brokers (SIBs) connected to the common space. SIB is a part of the Smart-M3 platform and it is responsible for information management within the smart space. Knowledge Processors (KPs) represent different applications that use the smart space and implement protocols defined by the Smart-M3 platform for communicating with SIBs. The information exchange is organized through transfer of information units (represented by RDF triples “subject-predicate-object”) from KPs to the smart space and back. The information submitted to the smart space becomes S. Balandin et al. (Eds.): ruSMART/NEW2AN 2010, LNCS 6294, pp. 72–83, 2010. © Springer-Verlag Berlin Heidelberg 2010

On-the-Fly Ontology Matching in Smart Spaces: A Multi-model Approach

Application

KP

73

Smart Space

SIB …

Application

KP

Knowledge Base

Fig. 1. Smart space based on Smart-M3: reference model

available to all KPs participating in the smart space. The KPs can also transfer references to the appropriate files/services into the smart space, since not all information can be presented by RDF triples (e.g., a photo or a PowerPoint presentation). As a result the information is not really transferred but shared between KPs by using smart space as a common ground. However, real life implementation of smart spaces faces a number of problems. Let’s consider a simple case study when a user is having his/her mobile device with KP running on it. The user KP (UKP) is going to make a presentation. The LCD projector represented in the smart space by Projector KP (PKP) is a key enabler of this functionality. But in order for the presentation to be shown, the UKP has to share the information about the presentation location (URI) with the PKP. As a result the following conditions have to be fulfilled: 1. 2.

The UKP has to know that the PKP is a part of the smart space; The UKP has to share the presentation’s URI in such a way that the PKP can understand it.

Even from this simple example we can see that proper functioning of the smart space requires semantic interoperability between knowledge processors connected to it. As a result it is required to develop models that would enable knowledge processors to translate on-the-fly between their internal and smart space’s ontology to a certain extent. The paper presents the developed multi-model approach for the above stated problem. Section 2 introduces the main principle of the suggested approach. Sections 3 to 5 describe the major elements of the approach in more details. Section 6 presents the developed demo use case that illustrates the approach. The main results and findings are summarized in Conclusions.

2 Multi-model Approach for On-the-Fly Matching KPs and SIB Ontology In order to analyze the existing ontology matching techniques an extensive state-ofthe-art review has been done, which covered about 20 systems/approaches/projects related to ontology matching. Among them the following ones are worth to be mentioned: GLUE System 8, 9, Falcon-AO 10, MLMA 11, Hovy 12, SKAT 13, ONION

74

A. Smirnov et al.

14, Promt 15, H-Match 16, CTX-MATCH 17, SMART 18, Cupid 19, COMA 20, Similarity Flooding Algorithm 21, AgreementMaker 22, Pattern Based Approach 23, MinSMatch 24, OntoView 25, Chimaera 19. The below proposed approach allows matching of KPs and SIB ontology for the interoperability purposes and is based on the ontology matching model illustrated by Fig. 2. The approach takes into account that the matching procedure has to be done “on-the-fly” by mobile devices with limited resources and remembering the fact that knowledge processors are responsible for performing certain concrete and welldescribed tasks, which means that the corresponding ontology generally should be small – to – medium size and describe only very limited domains. Ontology is represented as RDF triples, consisting of the following ontology elements: subject, predicate, object. Degree of similarity between two ontology elements is in the range [0, 1]. The approach consists of the following steps: 1. Compare ontology elements taking into account synonyms of both ontologies. The degree of similarity between equal elements is set to 1 (maximum value of the degree of similarity). 2. Compare all elements between two ontologies and fill the matrix M using similarity-based model described in Section 2. Matrix M is of size m to n, where m is the number of elements in the first ontology and n is the number of elements in the second ontology. Each element of this matrix contains the degree of similarity between the string terms of two ontology elements using the fuzzy string comparison method described in Section 3.2. 3. For knowledge processors, which can access Internet, e.g. WordNet or Wiktionary, the model of searching semantic distances was developed and presented in Section 3. 3.1. Compare all elements of two ontologies and fill the matrix M’. Matrix M’ is of size m to n, where m is the number of elements in the first ontology and n is the number of elements in the second ontology. Each element of this matrix represents the degree of similarity between two ontology elements. Matching model

Method class

Comparison of elements of two ontologies using synonyms

Combined

Comparison of elements of two Comparison of elements of two ontologies using similarityontologies using semanticbased method based distances search method

Linguistic

Graph-based distance improvement

Contextual

Fig. 2. Multi-model approach to on-the-fly matching KP and SIB ontology


75

3.2. Update values in matrix M, where each new value of elements of M is the maximum value of (M, M’) 4. Improve distance values in the matrix M using the graph-based distance improvement model described in Section 4. As a result the matrix M contains degrees of similarity between ontology elements of two knowledge processors. This allows determining correspondences between elements by selecting degrees of similarities which are higher than the threshold value. The next sections describe major elements of the proposed approach in details.

3 Similarity-Based Model for Matching Ontology The similarity-based model for the ontology matching is presented in Fig. 3. It contains a stemming procedure to normalize words, improved fuzzy string comparison procedure, and normalization procedure. The normalization procedure makes it possible to reduce the resulting similarity for its easier interpretation (Section 3.3). Improved FSC Stemming

Conventional Fuzzy string comparison (FSC)

Normalization

Fig. 3. Similarity-based model for the ontology matching

3.1 Stemming To improve the matching quality the application of the stemming procedure is proposed. This operation makes it possible to identify ontology elements even if they are written in different forms. The following conversions can be done: “looking” → “look”, “device” → “devic”, “vertical” → “vertic”, and “horizontal” → “horizont”. This procedure is uniquely tuned for each supported language. 3.2 Fuzzy String Comparison The basis of the string comparison algorithm is the well-known conventional algorithm that calculates occurrence of substrings from one string in the other string. The algorithm can be illustrated by the following example from matching the iCal and vCal formats. Table 1 presents the comparison of strings “DTSTART” and “START_D”. Number of substrings of the first string contained in the second string: 16 Total number of substrings: 28 The result: string “DTSTART” is contained in the string “START_D” with similarity of 0.57 or 57%. However, this algorithm does not take into account the length of the second string. As a result it was decided to introduce the following additional modifications:

76

A. Smirnov et al. Table 1. Fuzzy string comparison example

Substrings of the first string

Contained in the second string

Substrings of the first string

Contained in the second string

D

+

TST

-

T

+

STA

+

S

+

TAR

+

T

+

ART

+

A

+

DTST

-

R

+

TSTA

-

T

+

STAR

+

DT

-

TART

+

TS

-

DTSTA

-

ST

+

TSTAR

-

TA

+

START

-

AR

+

DTSTAR

-

RT

+

TSTART

-

DTS

-

DTSTART

-

1. Perform the comparison based on the above algorithm twice: FC1 = FuzzyCompare(Element1, Element2) and FC2 = FuzzyCompare(Element2, Element1). 2. Calculate the result as an aggregation of the above results in accordance with the following formula: Re’=n*FC1+(1-n)*FC2, where n is a weight, n∈[0;1]. n = 0.5 sets the same weight to the both strings, n = 0 searches only Request within Class, and n = 1 searches only Class within Request. It is proposed to set n = 0.5 . 3.3 Normalization of Results Since the similarity metrics are obtained by via different techniques they have to be normalized. This is proposed to be done via the following formula: ⎛1 Re = ⎜ − ⎜2 ⎝

1 1⎞ + ⎟ − 4 a ⎟⎠ a * Re ′+

1 1 ⎛1 ⎜ − ⎜2 ⎝

1 1⎞ + ⎟ 4 a ⎟⎠

,


77

where a is a coefficient, a∈(0;+∞). Values of a close to 0 (e. g., 0.00001) produce almost no normalization, values of a more than 1 significantly decrease the resulting similarity level. The value of a should be set empirically.

4 Model of Searching Semantic Distances of Ontology Elements For measuring the semantic distances between ontology elements we use the machine readable dictionary extracted by direct access to Wiktionary and WordNet. This dictionary includes: 1) a set of words defined in dictionary along with 2) definitions given for each word, 3) a set of synonyms for each word, and 4) a set of associated words for each word. Words associated to a word are considered as hyperlinked words occurring in the Dictionary definition given for this word. The nodes of ontology are linked to nodes representing their synonyms and associated words as this is given in the machine-readable dictionary. The links between the nodes are labeled by the distance of relations specified between the concepts represented by these nodes in the machine-readable dictionary. Weight w of a relation specified between two ontology elements t i and t j is assigned as: ⎧0,5 − t i , t j are synonyms ⎪ w = ⎨0,3 − t i , t j are associated words ⎪∞ − t , t are the same word i j ⎩

The values for the weights were evaluated based on the following principles: 1. Semantic distances between synonyms are assumed to be smaller than semantic distances between associated words; 2. Semantic distance is proposed to be calculated as inversely proportional to weights raised to a power. The power is proportional to the path between the compared words. The longer path the greater the semantic distance for the two different words is expected to be. To meet this expectation with reference to the way of the semantic distance calculation, a weight of the relation between two different words should be in the range (0, 1) and ∞. Taken into account the first principle, we empirically selected the weights: 0,5 - for the relation between the synonyms; and 0,3 - for the relation between the associated words; 3. The semantic distance between the same words is equal to 0. For searching semantic distance between elements of two ontologies, nodes of the first ontology are checked for their similarity to nodes of the second ontology. As a measure of similarity the semantic distance (Dist) is used.

Dist (t i , t j ) =

1 sj

∑ ∏ wk S k = si

78

A. Smirnov et al.

where t i , t j – ontology elements; w – weight of lexical relation existing between t i and t j ; S – a set of paths from t i to t j , where a path s is formed by any number of links that connect t i and t j passing through any number of nodes. The degree of similarity depends inversely on distance.

5 Graph-Based Distance Improvement Model The graph-based improvement model for propagation similarities from one ontology element to another is presented in Fig. 4. The main goal of this model is to propagate the degree of similarity between closely matching ontology elements to ontology elements related to them through RDF triples. Ontology 1

Ontology 2 d(xi, xj)

xi

d(ri, rj)

ri

d(x’i, x’j)

x’i

xj rj x’j

Fig. 4. Matching of two ontology model

Set X=(x1, x2, ..., xn) is the set of subjects and objects in the ontology of two knowledge processors. Set Dx = (d(xi, xj), ...) is a degree of similarity between xi and xj. Set R = (r1, r2, ..., rn) is a set of predicates in the ontology of two knowledge processors. Set Dr = (d(ri, rj), ...) is a set of degrees of similarity between ri and rj. Constant Tr is a threshold value that determines whether two ontology elements mapped to each other or not. The following algorithm allows propagating similarity distance to RDF subjects and objects. d(xi, xj) = maximum(Dx) while (d(xi, xj) > Tr) do for each d(x’i, x’j) as xi rm x’i and xj rl x’j do if d(rm, rl) > Tr then d(x’i, x’j) = endif endfor Exclude d(xi, xj) from Dx d(xi, xj) = maximum(Dx) endwhile

2

d (x i , x j ) ∗ d (x i′ , x ′j )


79

The following algorithm allows propagating similarity distance to RDF predicates. for each d(xi, xj) > Tr do for each d(x’i, x’j) > Tr as xi rm x’i and xj rl x’j do d(rm, rl) = 3 d (x i , x j ) ∗ d (x i′ , x ′j ) ∗ d (r m, r l

)

endfor endfor

6 Smart-Room Case Study This is an extended scenario originally proposed in one of our previous papers [6]. The meeting takes place in an “intelligent room” including LCD projector, whiteboard, and access to Internet-based translation service. Users that are planning to make presentation have the special knowledge processor (called User Knowledge Processor or UKP) on their Nokia MAEMO devices, that implements the required functionality as described below. Upon event of user entering to the smart space meeting room, at least the following information from users’ mobile devices become accessible for other smart space UKPs: -

the user profile information (name, photo, domain of interests, e-mail, and phone number, etc.); presentation information (title, keywords, URI).

Before the meeting starts the agenda has to be automatically built and shown on the whiteboard including the speakers’ names, photos, and presentation titles. The current presentation is highlighted on the screen. All meeting participants can see the detailed agenda on the screens of their personal mobile devices. Users can change their user profile items. When the user changes the information about his/her presentation in the profile, UKP changes the appropriate rules in the smart-space. A GUI interface for visualizing the detailed agenda in the screen of participants’ Nokia MAEMO devices has been implemented as well. At the scheduled time the appropriate presentation starts automatically, i.e. the LCD projector is switched ON and the appropriate presentation is shown. The user can control the slideshow directly from the mobile device. Five minutes before the presentation ending time and when the presentation time is about to be over the whiteboard reminds the speaker about the time restrictions. The architecture of this case study is presented in Fig. 5. It includes the following knowledge processors: KP1 – whiteboard (PC); KP2 – projector (PC); KP3 – translator (PC + Internet service); KP4 – coordinator (PC); KP5… N – UKPs (e.g. Nokia N810). The application of the matching procedure is shown in Fig. 6. Let’s consider the following example. The user having mobile device with a knowledge processor (User KP or UKP) running on it and he/she is going to give a lecture. The corresponding ontology is presented in Fig. 7.

80

A. Smirnov et al. Whiteboard

Projector KP1

KP2

Participants’ UKPs

Translator

SIB

KP5

KP3

KP…

Coordinator KP4

KPN

Fig. 5. The case study architecture

The LCD projector represented in the smart space by Projector KP (PKP) is capable of this function, as it is shown by ontology of the projector KP in Fig. 8. This ontology currently located in the smart space. In order for the presentation to be shown UKP has to share the information about the presentation location (URI) with PKP. Participants’ UKPs

Projector PKP SIB

UKP Ontology

PKP

Ontology Matching

Ontology

Matching

Fig. 6. The extended case study architecture user gives

lecture part of

current slide

is of has

type

is

number

file location

Fig. 7. Ontology of user KP

For this purposes the UKP and PKP ontology have to be merged. The element “lecture” from UKP is merged with the “presentation” from PKP as a result of the semantic-based distances search method. Distance between these elements is 0.3 since


projector

81

has

state

show

presentation part of

slide

is of has

is

number

type URI

synonym

file location

synonym

URL

Fig. 8. Ontology of projector KP

they are synonyms according to the Wiktionary. The element “file location” from UKP is merged with the “URI” from PKP as a result of checking the synonyms rules of the PKP ontology. The element “Current slide” is merged with the element “Slide”. The degree of similarity between these elements estimated via the fuzzy string comparison method is 0.59. As a result of this matching UKP knows that the following rule has to be added to the smart space to start the lecture: (“URI”, “is”, “http://myexample.com/presentation1.ppt”).

7 Conclusion The paper proposes the multi-model approach to on-the-fly ontology matching in smart spaces. The approach has been developed integrating the most efficient techniques applicable to Smart-M3 and based on analysis of the state of art. It takes into account that the matching has to be done “on-the-fly” by mobile devices with limited capacities and uses the fact that knowledge processors are responsible for performing certain concrete and well-described tasks. The experiments showed that the matching procedure based on the proposed algorithm (without using on-line dictionaries) takes less than a second on a Nokia N810 mobile device. Any measurement of similarity comes with a certain probability. In this study the minimal probability when the nodes are considered to be the same is captured by comparison to a certain threshold value. The state-of-the-art study discovered that at the moment there are no methods for automatic (without involvement of experts) setting of the threshold value for the matching procedure. Hence, within scope of this project we had to spend significant efforts to minimize as much as possible the need for the expert involvement in the future. As a result, the developed model assumes involvement of experts only at two stages of the design phase and no expert involvement at the run time. First, the experts are obviously needed at the development of the ontology for KPs. At this stage they also define synonyms for the appropriate domain that will be used during matching.

82

A. Smirnov et al.

Another stage when the experts are required is at the setting of the threshold value. This could be done either during the ontology design or postponed till the designed ontology is matched against some other existing ontology from the appropriate domain. For example, it could be specially developed testbed ontology, or accumulated ontology from the real world, depending on the availability. If the threshold is defined when the KP ontology is designed, its quality would be usually worse than when its definition is postponed because of the lack of the matching results with other existing ontology. However, it is not always possible to postpone since the testbed or real world ontology is not be available in many cases, e.g. in case of developing KPs for new domains. In this study we could not find fully automated solution for defining threshold value, but we are continuing to think in this direction and your ideas on how it can be solved are very welcome.

Acknowledgement The presented work is a result of the joint project between SPIIRAS and Nokia Research Center. Part of the implementation work has been also supported by Open Innovations Framework Program FRUCT – www.fruct.org.

References 1. SourceForge, Smart-M3 Retrieved: 10.4, (2010), http://sourceforge.net/projects/smart-m3 2. Semantic Web site, http://www.semanticweb.org (retrieved 10.4.2010) 3. Smart-M3 Wikipedia page, http://en.wikipedia.org/wiki/Smart-M3 (retrieved 10.4.2010) 4. Oliver, I., Honkola, J.: Personal Semantic Web Through a Space Based Computing Environment. In: Middleware for Semantic Web 08 at ICSC’08, Santa Clara, CA, USA (2008) 5. Oliver, I., Honkola, J., Ziegler, J.: Localized Space Based Semantic Webs. In: WWW/Internet Conference, Freiburg, Germany (2008) 6. Carriero, N., Gelernter, D.: Linda in context. Artificial Intelligence and Language Processing 32(4), 444–458 (1989) 7. Wells, G.: Coordination Languages: Back to the Future with Linda. In: Proceedings of the Second International Workshop on Coordination and Adaptation Techniques for Software Entities (WCAT’05), Glasgow, Scotland, pp. 87–98 (2005) 8. AnHai, D., Jayant, M., Pedro, D., Alon, H.: Ontology Matching: A Machine Learning Approach. In: Handbook on Ontologies in Information Systems, p. 660. Springer, Heidelberg (2004) 9. Doan, A., Madhavan, J., Domingos, P., Halevy, A.: Learning to map between ontologies on the semantic web. In: Proceedings of the 11th international conference on World Wide Web, pp. 662–673 (2002) 10. Hu, W., Jian, N., Qu, Y., Wang, Y.G.: A Graph Matching for Ontologies. In: K-CAP Workshop on Integrating Ontologies, pp.43–50 (2005) 11. Alasoud, A., Haarslev, V., Shiri, N.: An Effective Ontology Matching Technique. In: 17th International Symposium ISMIS 2008, Toronto, Canada, May 20-23, pp. 585–590 (2008) 12. Hovy, E.: Combining and standardizing largescale, practical ontologies for machine translation and other uses. In: The First International Conference on Language Resources and Evaluation (LREC), Granada, Spain, pp. 535–542 (1998)


83

13. Mitra, P., Wiederhold, G., Jannink, J.: Semi-automatic Integration of Knowledge Sources. In: 2nd International Conference on Information Fusion (FUSION 1999), Sunnyvale, CA (July 6-8, 1999) 14. Mitra, P., Kersten, M., Wiederhold, G.: Graph-Oriented Model for Articulation of Ontology Interdependencies. In: Zaniolo, C., Grust, T., Scholl, M.H., Lockemann, P.C. (eds.) EDBT 2000. LNCS, vol. 1777. Springer, Heidelberg (2000) 15. Noy, N., Musen, M.: Anchor-PROMPT: Using Non-Local Context for Semantic Matching. In: Workshop on Ontologies and Information Sharing at the Seventeenth International Joint Conference on Artificial Intelligence (IJCAI-2001), Seattle, USA (2001) 16. Castano, S., Ferrara, A., Montanelli, S.: H-Match: an Algorithm for Dynamically Matching Ontologies in Peer-based Systems. In: Proc. of the 1st VLDB Int. Workshop on Semantic Web and Databases (SWDB 2003), Berlin, Germany (2003) 17. Serafini, L., Bouquet, P., Magnini, B., Zanobini, S.: An algorithm for matching contextualized schemas via SAT. Technical report, DIT University of trento, Italy (2003) 18. Noy, N., Musen, M.: SMART: Automated Support for Ontology Merging and Alignment. In: 12th Workshop on Knowledge Acquisition, Modeling, and Management, Banff, Alberta (1999) 19. McGuinness, D.L., Fikes, R., Rice, J., Wilder, S.: An Environment for Merging and Testing Large Ontologies. In: Proceedings of the Seventh International Conference on Principles of Knowledge Representation and Reasoning (KR’2000). Breckenridge, Colorado, USA (2000), http://www.ksl.stanford.edu/software/chimaera/ (retrieved, 10.4.2010) 20. Aumueller, D., Do, H., Massmann, S., Rahm, E.: Schema and Ontology Matching with COMA++. In: Proceedings of the 2005 ACM SIGMOD International Conference on Management of Data, pp. 906–908 (2005) 21. Melnik, S., Garcia-Molina, H., Rahm, E.: Similarity flooding: a versatile graph matching algorithm and its application to schema matching. In: Proceedings of the 18th International Conference on Data Engineering, USA, pp. 117–128 (2002) 22. Cruz, I., Antonelli, F., Stroe, C.: Efficient Selection of Mappings and Automatic Qualitydriven Combination of Matching Methods. In: The Fourth International Workshop on Ontology Matching, Washington DC, USA (2009) 23. Ritze, D., Meilicke, C., Šváb-Zamazal, O., Stuckenschmidt, H.: A pattern-based ontology matching approach for detecting complex correspondences. In: The Fourth International Workshop on Ontology Matching, Washington DC, USA (2009) 24. Giunchiglia, F., Maltese, V., Autayeu, A.: Computing Minimal Mappings. In: The Fourth International Workshop on Ontology Matching, Washington DC, USA (2009) 25. Klein, M., Kiryakov, W., Ognyanov, D., Fensel, D.: Ontology Versioning and Change Detection on the Web. In: 13th International Conference on Knowledge Engineering and Knowledge Management (EKAW02), Sigiienza, Spain (2002) 26. Smirnov, A., Kashevnik, A., Shilov, N., Oliver, I., Lappetelainen, A., Boldyrev, S.: Anonymous Agent Coordination in Smart Spaces: State-of-the-Art. In: Balandin, S., Moltchanov, D., Koucheryavy, Y. (eds.) ruSMART 2009. LNCS, vol. 5764, pp. 42–51. Springer, Heidelberg (2009)

Access Control in Personal Localized Semantic Information Spaces Alexey Koren and Alexander Buntakov State University of Aerospace Instrumentation, Bolshaya Morskaya 67, Saint-Petersburg, Russia {alexey.koren,alexander.buntakov}@extendedmobile.net

Abstract. Leading by global convergency of devices and software from different domains (mobile, desktop, web) information exchange platforms are also evolve into form of smart spaces, personal localized semantic information clouds. New architectures appear, fulfilling needs of crossplatform interaction, semantic reasoning and many other novel functions. In this paper we discuss access control problem for such kind of platforms. Several achievements are described briefly highlighting features that are most interesting in given context, requirements for access control framework are suggested and finally we introduce model of access control framework for one of the most promising smart spaces architecture. Keywords: Smart spaces, access control, personal semantic web.

1

Introduction

Despite of tremendous growth of World Wide Web in last decades and appearance of huge number of social services that affect volumes of information shared on the Web, most information sources are still private and open to limited number of trusted users. That is in human nature and most probably situation will not change in short time. People are about to create personal spaces and share it only with chosen person. On the other hand there are blogs, online diaries, social networks where exactly the same people publish a lot of information with everybody. But during the sharing people still want to hold ownership and control of all published information. Explaining in terms of Tim Berners-Lee’s idea of Giant Global Graph [1]: actors (both corporate or individuals) want personal subgraph of it to be under their control. One of the most promising architectures for fulfilling such need is localized personal semantic spaces (we will use term ”smart space” in this paper) described in [2, 3]. It takes advantage of many concepts, which are now popular in discussions about semantic technologies: automated reasoning, context-awareness, personalization etc. Such technologies and platforms emerge rapidly and we can say that it is predictable and fits common trend of device and software domains convergency [4]. Basic architecture is simple and it is shown on figure 1. Each smart space obtains a shared place for information exchange called ”whiteboard”. This is a S. Balandin et al. (Eds.): ruSMART/NEW2AN 2010, LNCS 6294, pp. 84–91, 2010. c Springer-Verlag Berlin Heidelberg 2010

Access Control in Personal Localized Semantic Information Spaces

85

canvas for creating semantic subgraph. For representing information open standards such as RDF [5] and OWL [6] are used. Agents, also called ’knowledge providers’ (KPs), connect to SIB during simple protocol allowing ”Insert”, ”Retract”, ”Query”, ”Subscribe” and ”Unsubscribe” actions, where: Insert - insert a set of triples; Retract - remove a set of triples; Query - query information store for a set of triples; Subscribe - set up a subscription for a given query; Unsubscribe - terminate processing of the given subscription. This set of actions could be easily represented as traditional [SELECT, INSERT, REMOVE] or [GET, PUT, DELETE] actions set, which seems more clear and strict for defining access.

Fig. 1. Smart space architecture

Agent could potentially connect to any number of spaces, combining information from them for more complex reasoning or simple aggregation for user. 1.1

Smart Spaces’ Security Requirements

Such way of organizing information makes traditional access control methods not applicable for providing security in pervasive highly dynamic systems filled with semantic information. New model of information interaction imposes new security requirements. We consider following as basic ’must have’ features for smart spaces oriented access control methods: 1) Access control on information (RDF) level. Security administrator should be able to restrict or permit explicitly any atomic piece of information, in terms of RDF (as it is standard de-facto for semantic technologies) - any particular triple or a set of triples. 2) Flexible semantic policies definition. From the other hand ability to define access policies on high semantic level is also important. It is more handy way to manage security permissions when it is no need to work on information pieces

86

A. Koren and A. Buntakov

level but on information types level. For example instead of restricting access to every person’s credit card information for anonymous it is more correct way (both from usability and security points of view) to restrict access to all triples of type [*, ”hasCreditCardNumber” ,*]. On higher level it should be possible to group different triple patterns into sets according to its semantics and work with such semantic categories instead of single patterns. In this example all personal financial information which could consist of several card numbers, bank accounts etc. may be combined in one group and administrator will permit or restrict access to this high-level object. Common way for organizing this is using ontologies. 3) Context-awareness and dynamic permission management. In novel ubiquitous multi-agent systems environmental conditions are changing very rapidly. And the same time state of environment sometimes affects permissions of agents. Most usual example is location - mobile agents located in particular area, lets say hotel main hall, may be able to access hotel’s internal services and the same time agents located outside the building are not permitted. Security system must handle all possible context situation gracefully, associating rights to agents in real-time after context changing. 4) Support for reasoning. Many semantic platform gain advantage of reasoning process. During reasoning new portions of information may appear in the system. After this security system must perform secure reasoning to determine what security attributes should be assigned new objects. It should be done not only to prevent unwanted access to private information but also for measuring trust for newly appeared information: may other users take into account these information in some sensitive reasoning processes or not? Lets look at one more example from financial world. Knowledge providers perform reasoning to recalculate estimations of air transport company after messages about planes’ crashes. If somebody will add untrusted information about several incidents - it should not affect stock prices before information will be confirmed officially. 5) Simple lightweight design. It is worth to note that multi-agents environment such as smart spaces are often consist of small devices restricted on energy, memory and/or processing power like mobile phones or consumer electronics. In order to minimize resource consumption for security subsystem it is very important to reuse as much as possible existing infrastructure and make security solution as resource-efficient as possible.

2

Related Work

Rebecca Montanari, Alessandra Toninelli et al. in a series of works dedicated to security for mobile agents [7, 8] achieved significant results in observing area of semantically-rich context-aware security policies based on rules and onthologies. Authors proposed hybrid approach which combines description logic and logical programming for definition of security policies for ubiquitous dynamically changing multi-agent environments. Benefits of this approach is very well-reasoned and these results affects architecture proposed in this paper pretty much.


87

One more example of using logical programming of context-aware security policies is Cerberus [9], security service of Gaia project. Cerberus is a ubiquitous security mechanism integrating context-awareness and reasoning over security policies for providing access control in ubiquitous computing environments. Interesting idea is to use confidence values for different types of authentication and several probabilistic methods to combine them to net value. But theoretically interesting idea is difficult for implementation in real world. For every system security administrator should manually define confidence levels for every authentication method and their combinations. For really complicated system it might be painful and lead to multiple vulnerabilities (or will require development of verification process). And in most cases confidence levels will anyway be created by administrator which uses Boolean logical clauses to determine permissions in every case, so it is more convenient to don’t use fuzzy conditions instead of natural logical programming and strict statements for defining security. Covington et al. proposed a mechanism of environmental roles [10] as a method of assigning security permissions to subjects and groups basing on state of system (environment). An environmental role can be activated if set of environmental variables (day of week, time, location etc.) satisfies a policy defined by security administration. This set is stored in a role definition. Such kind of encapsulation seems reasonable in many cases but sometimes it could be not very handy in situation when there are many roles which are pretty similar in general but different in details. In this case administrator should either place piece of logic into role definition (not good semantically and from encapsulation point of view) or create a very big set of environmental roles each of them is a small modification of base one. That is one more argument for separation logic of assigning roles basing on system’s state and roles itself. This way usage of environmental variables together with logically programmed security policies is a better way. There were several attempts to describe security infrastructures for RDF stores. In [11] authors propose a policy based access control method for an RDF stores. Idea is clear and simple and most applicable architecture detail is that all metadata and security policies is stored into RDF store itself. In [12] it is proposed to store security object for every triple or triple pattern. This object will define access rights for given triple. Also two-level conflict resolution policy mechanism is described. Main disadvantage of RDF-based security approaches (in our case) is that there is no possibility to define policies on high semantic level using ontologies and context.

3 3.1

Access Control Model Access Control on Low Level

Main functionality of smart spaces is to provide information and knowledge exchange in unified open format. Thereby main functionality of its access control system is to manage access to these information and knowledge. As vice as information mainly represented in RDF format - security will also work with RDF format and RDF stores at lowest level.

88


Common representation of RDF data set is oriented graph. Security check for given user and given may be presented as taking a subgraph in accordance with user’s access rights for every vertex and edge. We can manage access to RDF graph by assigning access right to different triple templates. Vertices are represented as triples like [John Smith, ?, ?] or [?, ?, John Smith], which corresponds to ’subject’ or ’object’ position of Mr. Smith in triple accordingly. Edges are represented as triples like [?, foaf:knows, ?]. Given triple describes simple relation between two subjects in social network using FOAF ontology [13]. Managing access to such kind of triple it is easy to make all contacts of person invisible to any group of users, let say ’anonymous’, ’banned’ or ’not friends’. Also it is possible to define access to any particular triple using same notation: [John Smith, ”salaryIs”, ”2000000 euro cash”]. Triple templates may be grouped together by any thematic, access or other property to simplify policy definition and changing. For managing access in dynamically changing environments it is most reasonable to use role-based access control model (RBAC) [14], instead of mandatory or discretionary as most flexible. We suppose P to be set of possible actions [PUT, GET, REMOVE] for all triple templates and template groups. Roles could be assigned to user in two ways: statically and dynamically. We don’t distinguish common roles and environmental or context-based roles explicitly. Secure reasoner is just assigning common roles to user basing on its context and/or state of environment. User can not only gain roles during environment check, but also lose some of roles statically assigned to him/her earlier. Consider the following example. Mr. Smith could be member of ’sales managers’ groups and thereby have access to some sensitive company information. But if he will trying to access this information from public computer or through insecure connection, then during security reasoning process he will temporary lose ’sales managers’ privileges and gain weaker privileges from role ’sales manager in untrusted environment’ and all security check for his queries will be done against this role, instead of his membership statically assigned before. It will have place until he will return to trusted corporate network or will set up VPN connection. 3.2

High-Level Policies Definition

Common way to define high-level security constraints is using ontologies (most likely in OWL format). Security administrator can describe any clause relevant to his/her domain in semantic way, like ’colocatedWithUser’, ’customerOfOurCompany’ or ’memberOfDependentProject’. Roles should also be described in OWL because ownership of some roles could be part of another role’s or policy’s description. Alessandra Toninelli et al in [8] suggest architecture of policy framework for pervasive computing to exploit logical programming [LP] in addition to description logic [DL] for encoding strict and clear rules operating high-level constraints and entities derived from ontology. In such manner we separate and encapsulate 3 main domains:


89

1) Description of all possible constraints, entities, actors, environment states etc. given in form of ontology; 2) Set of roles, independent from any environmental or other constraints, defining access right for different types of users and/or automated agents; 3) Set of logical rules, defining which user will grant which permissions in current context. At the same time all of 3 domains could be expressed in RDF form and be put into regular RDF storage without need to upgrade its structure for storing new data formats, that is important for small weak devices. 3.3

Support for Reasoning

Dynamic rules-based policy also should be applied to newly added triples. If triple fits any template defined in access policy it automatically gains according information group membership and access constraints. In case when triple doesn’t fit any template its access constraints may be chosen basing on user’s roles set or it can be assigned to default category. Trust maintenance is organized via adding ’trust level’ attribute to information groups or triple templates. Once system controls access to information group it could guarantee that information in given group may be changed only by authorized entities (users or automated agents) with proper level of trust. If triple appears as a result of reasoning process - KP should take trust level of all information considered during the process and set the lowest level from presented to resulting triples. Problems may appear when KP aggregates information from different smart spaces to perform reasoning. They most probably will have totally different trust policies which have no common semantics. In this case KP could treat resulting information as having trust lowest level from different spaces, because at the time when they don’t have common semantics in general and can not be comparable, KP knows meaning of every trust level and is able to decide is information combined from pieces with given trust levels trusted enough or not. Consider example with 2 news resources, each has authorized information from its journalists, verified by editors, and forum where any guy could leave his/her message. Thus first type of information is trusted (has trust level = ”1”) for both resources and forum threads has no trust in general (trust level = ”0”). Mathematically they are equal but let us say first resource is very old and respectful periodical and second one has doubtful reputation. In this case user will treat information appeared by reasoning on main news lines of both resources as not trusted enough, as he knows reputation of both resources. 3.4

Summary

Summarizing all details described above we see effective access control framework for dynamic, localized space based semantic webs as it shown on figure 2.

90


Fig. 2. Smart space architecture with security modules

All system information is stored into regular RDF store. Logical programming rules are not RDF-based itself usually, but they can be serialized to unify storage mechanism. Context is integrated with secure reasoning module which analyzes system’s state and assign or remove roles to users. We don’t analyze authentication and authorization methods in this paper that’s why there are no changes at all at the agents’ side. Trust maintenance need some changes on agents’ side but it also not actually a part of access control architecture.

4

Conclusion and Future Work

This paper describes our vision smart spaces access framework architecture. Multi-agent pervasive environment and semantic personalized webs are extremely new and intensively developing areas. All trends in World Wide Web, mobile devices and consumer electronics faster or slower but going in these directions. As soon as pure information now is the only possible level of interaction, research and development in area of security should also evolve to fulfill novel needs. In this paper we done review of related work in these areas and tried to build solution utilizing some of achievements to apply them to providing access control in smart spaces based environments. Practically, this paper leaves too much interesting questions out of scope. Authentication and authorization in multi-spaces environment is very interesting area for research as well as advanced support of reuse and reasoning over information from different sources, which can be described using different ontologies and have totally different trust levels. Solution given in this paper is far too simplistic and we will work further to make this concept-level model to work in dynamic environment with high level of uncertainty.


91

Acknowledgement The authors would like to thank FRUCT (Finnish-Russian University Collaboration in Telecommunications) program [15] for support and assistance provided and also would like to acknowledge importance of the financial help provided by Nokia and Nokia Siemens Networks.

References 1. Shadbolt, N., Berners-Lee, T., Hall, W.: The Semantic Web Revisited. IEEE Intelligent Systems 21(3), 96–101 (2006) 2. Oliver, I., Honkola, J., Ziegler, J.: Dynamic, Localised Space Based Semantic Webs. In: WWW/Internet Conference, Freiburg, Germany (2008) 3. Oliver, I., Honkola, J.: Personal Semantic Web Through A Space Based Computing Environment. In: Middleware for Semantic Web 08 at ICSC’08, Santa Clara, CA, USA (2008) 4. Koren, A., Balandin, S.: The Road to Smart Spaces: Convergence of Consumer Electronics with a Change of Content Management and Application Design Principles. In: Proceedings of 6th FRUCT Seminar, Helsinki, Finland (2009) 5. Resource Description Framework (RDF): Concepts and Abstract Syntax. In: W3C Recommendation, http://www.w3.org/TR/rdf-concepts/ (retrieved 03-08-2009) 6. OWL Web Ontology Language Overview, W3C Recommendation, http://www.w3.org/TR/owl-features/ (retrieved 03-08-2009) 7. Montanari, R., Toninelli, A., Bradshaw, J.M.: Context-Based Security Management for Multi-Agent Systems. To be published Proc. of the Second IEEE Symposium on Multi-Agent Security and Survivability. IEEE Press, Philadelphia (2005) 8. Toninelli, A., Kagal, L., Bradshaw, J.M., Montanari, R.: Rule-based and Ontologybased Policies: Toward a Hybrid Approach to Control Agents in Pervasive Environments. In: Proc. of the Semantic Web and Policy Workshop (SWPW), in conj. with ISWC 2005, Galway, Ireland (2005) 9. Al-Muhtadi, J., Ranganathan, A., Campbell, R., Mickunas, M.D.: Cerberus: A Context-Aware Security Scheme for Smart Spaces. In: Proceedings of 1st IEEE Annual Conference on Pervasive Computing and Communications (PerCom 2003), Fort Worth, TX, pp. 489–496 (2003) 10. Covington, M.J., Long, W., Srinivasan, S., Dey, A.K., Ahamad, M., Abowd, G.D.: Securing Context-Aware Applications Using Environment Roles. In: SACMAT’01, May 3-4, Chantilly, Virginia, USA (2001) 11. Dietzold, S., Auer, S.: Access Control on RDF Triple Stores from a Semantik Wiki Perspective 12. Jain, A., Farkas, C.: Secure Resource Description Framework: an Access Control Model. In: SACMAT’06, Lake Tahoe, California, USA (2006) 13. FOAF Vocabulary Specification, http://xmlns.com/foaf/spec/ (retrieved, 10-042010) 14. Sandhu, R., et al.: Role based access control models. IEEE Computer 29(2) (February 1996) 15. Finnish-Russian University Collaboration in Telecommunications (FRUCT) program official webpage, http://www.fruct.org (retrieved, 10-04-2010)

Integration of an Answer Set Engine to Smart-M3 Vesa Luukkala and Jukka Honkola Nokia Research Center, Itämerenkatu 11, Helsinki, Finland [email protected], [email protected]

Abstract. We present our work in integrating an answer-set engine smodels to Smart-M3 interoperability platform. This gives user a possiblity of programming and trialing logic program reasoning in Smart-M3 environment. The smodels programming model guarantees that evaluation of rules is terminating. We add reactive features on top of the answer-set programming model and discuss its implications.

1

Introduction

Nowadays, a lot of devices with considerable computing power and connectivity mechanisms are present in the everyday environment. The potential of these devices has also increased the interest in getting them to interoperate. Currently, interoperability standards mostly apply for single domains, such as UPnP and DLNA for home entertainment, or are controlled by a single company, such as the Apple ecosystem. The domain specific interoperability standards are challenging for nomadic devices such as smart phones, which ideally should be able to interoperate with any device it can connect to at any given time. A nomadic device will currently have to implement multiple standards to be able to participate in the different domains. Furthermore, the existing standards often target specific use cases instead of attempting to specify a more general interoperability framework. One initiative to target a generic interoperability framework is the semantic web [1]. The goal of the semantic web is to provide machine readable information about the World Wide Web in order to allow automating many tasks that the web is currently used for manually. Eventually this would result in one giant global graph describing the resources of the web in RDF [10] according to ontologies defined in dedicated languages such as in OWL [9]. However, the web is not the best platform for sharing the rapidly changing, dynamic local information about the immediate environment of a device. Smart-M3 is an interoperability platform that allows devices to easily share and access local semantic information, while also allowing the more global semantic information to be available. The information is represented by using same mechanisms as in semantic web, thus allowing easy exchange of global and local information. The RDF representation also allows extremely easy linking of S. Balandin et al. (Eds.): ruSMART/NEW2AN 2010, LNCS 6294, pp. 92–101, 2010. c Springer-Verlag Berlin Heidelberg 2010

Integration of an Answer Set Engine to Smart-M3

93

data also between different ontologies, thus making cross-domain interoperability straightforward. While the used semantic web technologies have been designed to allow reasoning over the information, Smart-M3 currently supports only limited reasoning. Queries in Smart-M3 understand subclass relations, and the query engine also understands the owl:sameAs property. The implemented reasoning mechanisms in Smart-M3 are currently domain-independent, that is, they can take into account properties defined in RDFS or OWL, but not information defined in domain ontologies. However, in many applications creating domain-specific reasoners would be useful. We present an approach where we use logic programming for defining the domain specific reasoning rules. As each RDF triplet can be interpreted as a fact of arity two, there is a straightforward link between RDF content and rulebases. Therefore, various logic programming approaches can be used in a natural manner to deduce new information by means of user constructed rules executed over the fact base. The logic programming approach is useful in creating domain specific instructions for searching and abstracting over the RDF content. Also searching for configurations and policies are natural to express using logic programming means. Furthermore it is possible to generate consitency rules based on ontologies. In short, a rule mechanism can be used to provide additional functional semantics to accompany ontologies. Rules can also be thought of as views in the underlying RDF-base. In this way the logic programming mechanism provides a flexible alternative to query languages like SPARQL [12] or Wilbur Query Language [5]. This approach is not new, for instance Wielemaker et. al. [13] have proposed Prolog to be fundament for application development for semantic web for similar reasons. The smodels [6] is a system for answer set programming. It consists of lparse front end which transforms rule language to input suitable for smodels, which is a solver based on stable model semantics of logic programming. Here we refer to the combination of lparse front end and smodels solver simply as smodels. smodels allows constructing programs using extended logic programming concepts, atoms and inference rules, but by language construction the programs are always decidable. ssls 1 is a command-line tool giving access to Smart-M3 operations with some additional convenience features as well as a possiblity of interfacing external tools to operate on the results of the operations. We use ssls to integrate smodels with Smart-M3 as a node of the system. We do not expect this work to provide a complete programming environment for applications (even though it in principle could be one), but rather as a component in Smart-M3 technology stack. In section 2 we introduce the Smart-M3 interoperability platform, in section 3 we present the integration between smodels and ssls, in section 4 we give an example of use and finally in section 5 we present our conclusions and possibilities for future work. 1

Available at http://sourceforge.net/projects/ssls

94

2

V. Luukkala and J. Honkola

Smart-M3

Smart-M3 is a space-based interoperability platform consisting of two main components: semantic information broker (SIB) and knowledge processor (KP). A smart space is defined as a named search extent of information, where the information is stored in one or more SIBs. The architecture of the system is presented in Figure 1. In the simplest case, one SIB will store all information in a smart space, but there is a possibility of connecting multiple SIBs to make up a smart space. The SIBs making up a smart space will be connected with a protocol that provides distributed deductive closure [2]. Thus, any KP sees the same information content in the smart space regardless of the SIB it connects to2 . The information in the smart space is stored as an RDF graph, usually according to some defined ontology. The use of any specific ontology is not mandated, however. Neither is the consistency of the stored information guaranteed in any way. The KPs may modify and query the information using the insert, remove, update, query and subscribe operations by using the Smart Space Access Protocol (SSAP). Join Leave

: Join a KP to a named space : Leave a named space. After leaving, no more operations may be performed until a join operation Insert : Atomically insert a graph in the space Remove : Atomically remove a graph from the space Update : Atomically update a graph in the SIB. Update is a combination of remove followed by insert, executed atomically Query : Query for information in the space using any supported query language Subscribe : Set up a persistent query in the space; changes to the query results are reported to the subscriber Unsubscribe : Cancel an existing subscription The communication between KPs and SIB may be implemented by using any SOA style service network such as the Network on Terminal architecture [7], or by using a suitable transport protocol such as XMPP or TCP/IP. This approach allows using already deployed communication mechanisms and protects previous investments. Furthermore, it also means that KPs can connect to smart spaces by using the mechanism most suited to them. 2

Distributed deductive closure is not implemented in the current Smart-M3 implementation.


95

Agent Agent

Agent

Agent

M3 Space SIB SIB SIB

Fig. 1. A diagram of the general system architecture: Agents, M3 spaces and SIBs

2.1

Notion of Application

The notion of application in a smart space differs from the concept of traditional application. Instead of a monolithic application running on a single screen, the smart space applications are better seen as scenarios that can be executed to meet the goals of the user. The scenario emerges from the observable actions, taken by the participating knowledge processors, based on the information in the smart space, and also from the use of available services. The scenarios may also be transient: the scenario will change as the participating KPs join and leave the smart space, and also as services become available or unavailable. One of the targets of Smart-M3 has been to make combining the scenarios easy. This is achieved by the loose coupling between the KPs, as they only communicate by modifying and querying the information in the smart space. Thus, any effect that appearance or disappearance of KPs may have on the rest of the environment is limited to the information seen in the smart space. KPs may, of course, communicate with each other by any available means. but this is outside the scope of Smart-M3. A KP understands its own, non-exclusive set of information. This information may be thought of as the ontology of a KP. It is rarely defined formally, and is mostly an implicit concept which can be useful in some situations. Overlap of the sets of information understood by KPs is an essential precondition for achieving interoperability, as otherwise the KPs will never see each others actions. Smart-M3 has been used as an interoperability platform for a number of case studies. The case studies illustrate the applicability of the concept to a variety of areas. A case study of cross domain interoperability is described in [4]. The scenario involved a wellness domain represented by SportsTracker application, home entertainment domain represented by music streaming using a UPnP service, gaming domain represented by SuperTux game, and telecom domain represented by a phone call observer application. All domains share information using a M3 smart space, resulting in improved user experience compared to the case where the components operate independently. For example, when a call is received, the

96


game and music player see the information that a call is ongoing in the smart space and can take appropriate action, in this case pause the music and game. When the call ends, music continues automatically. Furthermore, the played music changes according to the state of the game, for example, when the player loses lives, the music changes appropriately. Finally, the game may award extra lives if the player has been exercising lately. Other case studies include a building automation case [3], a smart meeting room case [11], a home sensor network case [8], and a health care case developed by University of Bologna. In the building automation case, information from the installed sensors and systems is shared for others by using Smart-M3, allowing also devices that do not implement the oBIX or other building automation protocols to access the information. In the smart meeting room case, Smart-M3 is used to coordinate access to the resources of a meeting room, such as a projector for presentations. The home sensor network case is remarkable in that all the hardware used is commercially available. The demo shows input from several sensors on multiple devices, such as Apple iPhone, Nokia N810 and laptop. The health care case uses information from temperature and humidity sensor and patient’s heartrate monitor to determine if the external conditions may be hazardous for the patient. Furthermore, a demonstration showing a combination of the home sensor network and healthcare demos where the information from the sensors used in home sensor network demo was used in the healthcare system, illustrating the ease of creating mash-up applications, has been created. All of these scenarios have been implemented by using various kinds of wrappers on top of SSAP protocol and ontological concepts on procedural languages, like C, C# and Python. All of these scenarios contain KPs whose essential behaviour can easily be expressed as a set of rules and a part of the effort involves re-implementing a specific ruleset and its support mechanisms in the host language.

3

Integrating smodels Using ssls

For this work the purpose of ssls is to integrate the smodels system as a node local rule execution environment. This allows us to easily trial various functionalities, which may later on be integrated to SIB itself. We assume that at least some part of the SIB contents is fetched to the node for the duration of the rule execution. The basic operation of rule execution is that given a command, ssls fetches the contents of the SIB and generates a rulebase from them. The rules themselves have been defined locally in a file or within ssls and they are executed against the rulebase using smodels. The result will be a set of triples to be inserted in the SIB and set of triples to be deleted from the SIB, which are then committed to


97

the SIB. There are mechanisms for limiting the amount of fetched information as well as adding a degree of reactivity, which are described later on in this section. The conversion of the SIB contents is straghtforward: the SIB is queried for all triples and they are converted from subject, predicate, object triplet to fact predicate(subject, object). In order to satisfy the lparse syntax checker all URIs are quoted, which poses some problems for distinguishing between literals and URIs. This is solved by doublequoting the URIs with < and >. For example, a triplet :abcd-2ew4, rdf:type, wp1:Person would produce a fact ""("", "<wp1:Person>"). The rules are written in a text file, which is given to ssls as a parameter. It is also possible to define rules within the ssls interpreter, but only a subset of lparse language features are supported. The result of the smodels execution over the fact base and the rules is parsed for predefined facts which trigger adding content to insert and delete buffers, which are then committed to SIB. The facts for adding triples to insert and delete buffers are correspondingly i and d, both of arity three. Note that both i and d are ssls commands for adding triples to the buffers, which are then committed by the SSAP command update. After all generated facts have been parsed the update command is issued which will cause atomically the deletions to occur, followed by the insertions. In addition to the above i and d there are specific facts which cause ssls to print their arguments. During the rule execution several facts beside the ones which are interpreted (described above) may have been deduced. Unless these facts have been “marshalled” by means of i and d to the SIB, they will disappear after the rule execution. In this way the node with the rule engine is stateless. Of course it may be that the SIB is modified inbetween two rule executions and thus the results will be different for the same rules. However, it may be useful to have a hidden state, which is not published to the SIB. This may be due to security issues or performance reasons. This can be achieved by adding facts to the file with rules, but there is also a mechanims which allows to create a new persistent fact by means of rules themselves. If the fact ssls_fact with arity two is deduced, ssls calls its own rule command to add a new fact to its internal rulebase with the given arguments. The downside of introducing hidden state is that the state is local and the distribution of rules which use ssls_fact may be harder. Above we mentioned that the whole SIB is transferred to the node as the factbase for rule execution. Since the transferral is done by means of the SSAP query command, it is possible to limit the amount of information transferred. ssls contains a view mechanims which allows constructing a query with a set of triple patterns. So far we do not have any mechanism to check whether a view enables all rules, so this is an ad-hoc mechnism of reducing the amount traffic and requires an understanding of the particular rules to be safe. Finally, we may automatically trigger execution of rules by using the SSAP subscription mechanism. The mechanism allows setting up a persistent query and whenever there are changes in SIB pertaining to the query, the node is informed

98


about them. ssls allows naming subscriptions and whenever a subscription with its name prefixed by smls fires, the rule execution is initiated. This provides for a simple reactive mechanism to execute the rules. Note that so far each single execution of the rule-engine is decidable, but the above mechanism together with the notion of state (either in SIB or locally in the node) makes it impossible to determine the behaviour of such a chaining of single rule-engine executions. On the other hand this enables more generic programming, but the required mechanisms and sensibility of this approach needs further research.

4

A Rule Example

We have chosen smodels over other similar alternatives (like Prolog) for couple of reasons: firstly, the execution is decidable, removing a whole class of errors. Secondly, the execution order of the rule body is not relevant. This enables future distribution strategies for rule execution. Thirdly, the system is well established with several implementations. Finally, the smodels language contains several features, which are very useful in constructing rules, namely: possiblity of choosing a defined number of facts, a negation which works as set negation and weighing facts. const threshold = 10. % Rule 1 3 { power_save_on(X),i(X,"<mob:pmode>","low"),d(X,"<mob:pmode>","high") } 3 :""(X,"<mob:Device>"), "<mob:battery_level>"(X,Y), not connection_active(X), lt(Y,threshold). % Rule 2 connection_active(X) :1 {"<mob:gsm>"(X,"on"), "<mob:wlan>"(X,"on")}. % Rule 3 3 { power_save_off(X),i(X,"<mob:pmode>","high"),d(X,"<mob:pmode>","low") } 3 :""(X,"<mob:Device>"), "<mob:battery_level>"(X,Y), ge(Y,threshold). Fig. 2. smodels rules for a simple power manager

We give a simplistic example highlighting the two first features. In this case we have a mobile device, which monitors its battery level and if the level falls


99

under a preset value, puts the device to a power saving mode unless a connection is active. We define connection as either GSM being active or a WLAN connection being active. Figure 2 shows three rules to achieve this. Rule 1 derives three facts power_save_on, i and d for a device instance X, if the battery level is less than threshold and the fact connection_active does not exist in the rulebase. The head has three facts surrounded by minimum and maximum bounds, here both are three. This means that if the body matches, smodels should choose precisely three facts from the three possible choices. Here this is used as shorthand to avoid writing the same body for three different heads, but with different bounds we could choose subset(s) of the facts. Rule 2 states that we can derive connection_active if at least one of the listed possibilities exists. Rule 3 acts as a complement of Rule 1, turning power saving off when needed. Note, that power_save_on and power_save_off facts are not used anywhere in here. They are redundant as we expect that the information produced by i and d will trigger the appropriate behaviour by some other node in the system. However, if there will be other rules, these two facts can be used as components in those. In a way these facts act as an interface that this set of rules is offering for other rules.

5

Conclusions and Future Work

We have presented an integration of an answer-set programming system smodels to Smart-M3. The mechanism we have presented is simple and not specific to smodels, any other logic-programming system could be integrated by similar means. smodels does not as such contain any reactive features but Smart-M3 applications are by their nature reactive. This is why we have introduced a simple reactive mechanism in our system, which allows us the possiblity of implementing those kinds of programs using logic programming mechanisms. This comes at a price as the behaviour of the system when chaining several rule executions may not be easily predictable. One of our future research items is to understand and solve the challenges arising from this. As Smart-M3 does not currently have advanced reasoning capabilities, we use this approach to trial possible solutions to be included in the Smart-M3 system. One interesting possibility is to embed the rule engine in the SIB, and define an ontology for expressing the rulesets. This would allow executing rules defined by KPs in the SIB. For KPs with limited computing power this could be used as a mechanism to offload some of the computation from a small device to a more powerful device. Furthermore, as mentioned in section 2.1, in most scenarios there seems to be a need to utilize reasoning which is well suited to be expressed succinctly using rules. This reasoning could be performed conveniently in the SIB with the embedded rules engine. However, the exact benefits, e.g. less network traffic, and tradeoffs, e.g. performance of SIB, need to be further researched.

100


The previous issues yield more particular technical research topics and work items pertaining to our rule based approach, such as: – how to derive a suitable query expression from the rules to reduce the amount of traffic needed – how to parallelize the execution of rules – implement publishing and reading in rules from SIB, enabling reflective creation of rules – how to derive rules from an ontology to enforce consistency of SIB contents – implement an ssls primitive to start subscriptions from the rules We also aim to reimplement parts of the existing scenarios using the rule based approach.

Acknowledgement This work was supported by SOFIA (Smart Objects For Intelligent Applications) project and funded by Tekes (the Finnish Funding Agency for Technology and Innovation) and the European Commission, and DIEM (Devices and Interoperability Ecosystem) funded by Tekes.

References 1. Berners-Lee, T., Hendler, J., Lassila, O.: The semantic web. Scientific American (May 2001) 2. Boldyrev, S., Oliver, I., Honkola, J.: A mechanism for managing and distributing information and queries in a smart space environment. In: Proceedings of MDMD 2009 (May 2009) 3. Främling, K., Oliver, I., Honkola, J., Nyman, J.: Smart spaces for ubiquitously smart buildings. In: Proceedings of UBICOMM 2009 (October 2009) 4. Honkola, J., Laine, H., Brown, R., Oliver, I.: Cross-domain interoperability: A case study. In: Balandin, S., Moltchanov, D., Koucheryavy, Y. (eds.) ruSMART 2009. LNCS, vol. 5764, pp. 22–31. Springer, Heidelberg (2009) 5. Lassila, O.: Programming Semantic Web Applications: A Synthesis of Knowledge Representation and Semi-Structured Data. PhD thesis, Helsinki University of Technology (November 2007) 6. Niemelä, I., Simons, P.: Smodels – an implementation of the stable model and wellfounded semantics for normal logic programs. In: Fuhrbach, U., Dix, J., Nerode, A. (eds.) LPNMR 1997. LNCS, vol. 1265, pp. 420–429. Springer, Heidelberg (1997) 7. Network on terminal architecture (November 2008), http://www.notaworld.org 8. Openm3 demonstration release, http://sourceforge.net/projects/smart-m3/files/OpenM3.tar.gz/download 9. Web ontology language, http://www.w3.org/2004/OWL/ 10. Resource description framework, http://www.w3.org/RDF/ 11. Smirnov, A., Kashevnik, A., Shilov, N., Oliver, I., Balandin, S., Boldyrev, S.: Anonymous agent coordination in smart spaces: State-of-the-art. In: Balandin, S., Moltchanov, D., Koucheryavy, Y. (eds.) ruSMART 2009. LNCS, vol. 5764, pp. 42–51. Springer, Heidelberg (2009)


101

12. W3c recommendation: Sparql query language for rdf, http://www.w3.org/TR/2008/REC-rdf-sparql-query-20080115 13. Wielemaker, J., Hildebrand, M., van Ossenbruggen, J.: Using Prolog as the fundament for applications on the semantic web. In: Heymans, S., Polleres, A., Ruckhaus, E., Pearse, D., Gupta, G. (eds.) Proceedings of the 2nd Workshop on Applicatiions of Logic Programming and to the web, Semantic Web and Semantic Web Services, Porto, Portugal. CEUR Workshop Proceedings, vol. 287, pp. 84–98. CEUR-WS.org (2007)

A Video Monitoring Model with a Distributed Camera System for the Smart Space Andrey L. Ronzhin, Maria Prischepa, and Alexey Karpov SPIIRAS, 39, 14th line, St. Petersburg, Russia {ronzhinal,prischepa,karpov}@iias.spb.su

Abstract. The developed model of video monitoring based on distributed camera system is intended for automation of registration of meeting participants. The problem of detecting position of participants simultaneously attended in the room and their faces is solved by joint use of wide-angle, omni-directional and PTZ cameras located on walls and ceiling of the room, as well as implementation of face detection technology for photographing the zones of possible occurrence of participants faces. The quality of photos was estimated by accuracy of pointing the PTZ camera and the diagonal value of a face in the captured image. The deviation of face in the frame relatively the center averaged 8% and the square of face was no less than 7% of the image. Also it was observed that the mean time of registration increases, when the number of participants increases and the level of illumination decreases. Further development of the model is aimed at the automation of capturing the main speaker, discussion, vote and other key phases of events. Keywords: Multi-camera tracking, pointing PTZ camera, face detection, meeting support technology, smart space.

1 Introduction Due to the rapid development of broadcast networks, advances in processing and transmitting multimedia information, creation of Internet applications for videoconferencing and distant education, the so-called systems of E-meeting and E-lecture become more popular in business, academic institutes and universities at organization of distributed events. Such systems allow us to save on transportation costs, to choose an individual way of learning and to provide convenient facilities for searching and accessing information. Technological advances in recording, analysis, searching and transmission of multimedia data are already used in several research systems for video conferencing. Below, we analyze the range of problems holding the development of remote learning and collaboration, and methods for their solutions implemented so far in research systems only. Distributed events organized via specialized web-applications are real alternative to traditional meetings and lectures, where participants interact "face to face." Development of multimedia technologies allows video conference systems not only recording and output of audio and video data, but they also use more sophisticated S. Balandin et al. (Eds.): ruSMART/NEW2AN 2010, LNCS 6294, pp. 102–110, 2010. © Springer-Verlag Berlin Heidelberg 2010

A Video Monitoring Model with a Distributed Camera System for the Smart Space

103

techniques, such as analysis, pattern recognition and structuring of multimedia data, that certainly enhances services for participants and leads to new ways of access to events in real-time and archive processing [1]. The most of the research, conducted in this area, was devoted to development of software and hardware for audio- and videoprocessing. At present, panoramic camera, tablet for handwritten notes, whiteboard, tools for multimedia presentations, technology of synchronization and integration of data and an analysis of behavior and ways of cooperation between the participants during the event are widely employed. The automatic analysis of audio-visual data of meetings or lectures is complicated by the fact that such events are usually held in large halls with lots of participants, who arbitrarily change positions of their body, head and gaze. To improve the capturing and tracking of many participants, arrays of microphones, panoramic cameras, intelligent cameras (PTZ - Pan / Tilt / Zoom) and distributed camera systems are used. In the system Cornell Lecture Browser [2], two video cameras with subsequent synchronization and integration of video streams are used. In the eClass, also known as Classroom 2000 [3], videos of lectures were combined with the handwritten notes on a whiteboard. The system AutoAuditorium [4] detects participants by the PTZcamera, as well as carries out an automatic selection of video from one of three cameras, installed in the room. In the system [5], it is proposed to use the spatial sound source localization to control the PTZ-cameras and to track speakers and listeners. Also a list of rules and recommendations for professional operators, assisting to select optimal positions of the cameras in the hall, is defined. In the system [6], eight PTZcameras are controlled by video stream analysis, sound source and data localization from ultrasonic motion sensors. In the system FlySpec [7], the PTZ and omnidirectional cameras, management of which is remotely carried out by several participants with the automatic control system adjusting the direction of PTZ-cameras, are applied. Application of the panoramic camera allows us to capture images of all the events taking place in the room and to find locations of each participant [8]. Motion sensors [9] and microphone arrays are used additionally to video monitoring, in order to detect positions of participants and the active speaker. Sound source localization with the microphone arrays is effective in small lecture or conference rooms only. To record audio signals in a large room, participants and speaker often use personal microphones [4] or apply a system of microphone arrays, distributed in the room [6]. Various approaches have been proposed to record presentation slides projected on the screen during the event. Some systems require presentation slides loaded in advance or special software installed on user's computer, which says the number of the current slide to the server. In papers, in order to capture and record the presentation slides, a separate video camera, directed to the projection screen, is used. Another approach is to automatically capture video data directly from the VGA output of the computer or the projector, and to save data in a separate thread [8]. Whiteboards and blackboards play important role in meetings and lectures for making handwritten notes using a special pen or a hand with the sensor surface [10]. In the paper [11], image from the board is captured by the PTZ-camera. The system of image capturing, presented in [12], automatically identifies most interesting areas of the board and scales them for a better representation of information. Automatic handwriting recognition is used in the system [13] for processing sketches made on whiteboards.

104

A.L. Ronzhin, M. Prischepa, and A. Karpov

2 The Video-Monitoring Model of Meeting Participants The given research is focused on solution of the following problems: (1) finding and tracking active speaker with PTZ-cameras; (2) analysis of human beings’ motion for gesture recognition and activity classification; (3) registration of event participants; (4) detection and identification of users’ faces; (5) detection of gaze direction to find an object of attention of participants. The efficiency of events and conferences is improved with the development of hardware for recording and output of multimedia data. At the same time, a new range of unresolved issues was identified; so, development of methods for searching, tracking and recognition of objects by means of multichannel processing still remains very important. While designing the smart meeting room, ergonomic aspects of use of multimedia equipment have been taken into account, as well as location of audio and video recorders has been chosen that which enables to capture many participants at once. Choosing the number of cameras, their location, resolution, viewing angles and other settings for the video monitoring system, coverage zone, the number of objects that require simultaneous monitoring, degree of details of analyzed objects, the level of illumination have to be taken into account. Figure 1 shows places of seats, multimedia equipment (TV set, Projector), five AXIS Internet-cameras (PTZ-camera Cam3, Cam5, wireless cameras Cam2, Cam4, camera with wide angle lens Cam1, installed on the ceiling in the center of the room),10 personal webcams Logitech AF Sphere, four groups of lighting. Three microphone arrays located in the center of the left and right walls and over the sensor plasma (touch screen) serve for sound sources localization and recording phrases of participants. Each array of microphones has T-shaped configuration [14]. The aim of this paper is description of the developed system for video monitoring; so, the microphones are not shown in figure. Analysis of model-algorithmic software for processing multi-channel audio streams is used at development of multimodal information kiosk and smart rooms [15]. The conference table with installed personal webcameras for placement of participants of small meetings (round tables) up to 10 people is located on the left side of the hall. The right side of the hall contains rows of seats, which can accommodate up to 32 participants, tracking of which is implemented by the distributed system of cameras and microphone arrays. Three non-overlapping areas were identified, which require non-stop video monitoring: (1) the main speaker zone, (2) zone of participants, sitting at the conference table, (3) zone of participants in seats. Video monitoring of all three zones is carried out with the camera Cam1 having the viewing angle of 140 degrees and the resolution of 2048x1536 pixels. The stationary cameras Cam2, Cam4 have the viewing angle of 74 degrees and the resolution of 1280x1024 pixels. Two cameras Cam3, Cam5 with viewing angles of 51.6 degrees and the resolution of 704x576 pixels, equipped with the functions of tilt, rotation and scaling make capturing of any point of the space in the smart room.

m3 Ca

C C am

C

C

C

Third group of light

Second group of light

5

7150мм

105

C

C

m4 Ca


Cam2

Fig. 1. Scheme of the smart room and positions of cameras

The camera Cam2 installed at the height of 2 meters on the opposite wall of the projection screen is used for recording audio-visual data of events. The camera Cam1 has the wide-angle lens and locates on the ceiling in the center of the room. It covers the entire space of the room and allows to determine coordinates of all the participants in the horizontal plane. The stationary camera Cam4 was installed on the left from the door and designed in such a way, in order to cover all the users on seats at once. PTZcamera Cam3 and camera Cam2 were installed near each other and work simultaneously. The analysis of frames from the camera Cam2 allows to determine position of an observed object and to generate a command for rotation of the camera Cam3 in needed direction. The PTZ-camera Cam5 was installed in the center of the left wall at the height of 2 meters from the floor level and serves to capture video-data of the active speaker. The developed multi-functional model carries out the video monitoring of the event and allows us to automate main modes, such as: registration of participants, taking pictures of the speaker, active participants in the room during the discussion and vote. Based on the key stages of the events, different modes of the multifunctional model have been developed. Table 1 describes main functions in three basic modes: observation, registration of participants, event support. There are 3 cases in the mode “Event support”. The first case is the “Speaker’s talk”, where the system tracks movements of the speaker analyzing changes in neighboring frames received from the camera Cam1. On the basis of the speaker coordinates in the horizontal plane, angles of rotation and zoom are calculated to control the camera Cam5. The mode "Discussion" begins at the end of the speaker’s talk. In this mode, the control system of the intelligent room is managed by the sound sources localization module and determines active participants in the seat zone and directs the

106


camera Cam3 there. The camera Cam5 keeps tracking of the presenter during the given mode. In the mode "Vote", the camera Cam1 determines participants in the seat zone, and counts raised hands of participants of the event using the camera Cam4. At the same time, the camera Cam3 with a minimum scale factor records the process of voting. The camera Cam2 directed to the projection screen is used for recording of the event. Table 1. Camera’s functions in various modes of video monitoring Camera

Mode Observation

Cam1

Tracking changes in the entrance zone

Registration

Tracking of movements and detection of seated participants

Speaker’s talk Tracking of movements of speaker

Event support Discussion

Vote

Detection of seated participants

Detection of seated participants

Cam4

Detection of faces of seated participants

Identification of the active speakers

Calculation of votes

Cam3

Photographing of the participants

Recording of the discussion of active speakers

Recording of the voting process

Cam5 Cam2

Recording of an active speaker

Recording of an active speaker

Recording of the screen and the room

Recording of the screen and the room

Figure 2 shows the scheme of transitions from one mode to others, which are initiated by the operator with the help of the control system of the intelligent room. The system changes its mode to "Observation", if a moving object is found in the entrance zone, located in front of the entrance door. Also, the system performs tracking of user movements and monitoring of seats. The system changes its mode to "Registration", when one of the participants sits in the chair. The system changes its mode to "Event support" with the sub-mode “Speaker’s talk”, when all the participants of the event have been registered and the command on appearance of the first speaker has been received. When the presentation of the speaker has been completed, the system changes the mode to "Discussion". The system transmit to the mode “Speaker’s talk”, when the command on starting a new presentation has been received. The mode


107

Fig. 2. Scheme of mode transitions of the video monitoring model

“Vote” starts, when the system finds raised hands. The model goes to the sleep mode, when the system receives a command on the end of event and this mode works until starting of a new event. In the mode "Registration", three cameras are used for monitoring of each seated participant, so the algorithm can be divided into 3 phases. At the beginning of the first phase, the frame is captured by the camera Cam1, after which a cycle with the number of seats in the hall starts and a fragment from the frame corresponding to the concrete seat is extracted. Then, for each fragment colors distribution histogram is calculated. Decision on presence of a person in the chair is made depending on the value of correlation between histograms of the current fragment and of the original fragment, created for the given empty seat in advance. After processing all the seats in the room the frame is captured by the camera Cam4, in which the search of a face is performed. The zone of possible occurrence of a face increases by 60 pixels, if a face is not found during the processing of 30 frames and the second search is made. Then, the Cam3 is directed to each person, so that to make picture of his/her face with a large resolution. If a face is found, the control system receives the photo of the participant and the number of his/her chair. The model goes into the standby mode after processing of all the participants.

3 Experimental Results Series of experiments with different numbers of participants (1, 3, 5, 10) and four variations of illumination: (1) groups of light 1, 2, 3, 4, are turned on; (2) groups of light 1, 2, 4 are turned on; (3) groups of light 3, 4, are turned on; (4) group of light 4 is turned on, were made for testing the algorithm. The results were evaluated by two criteria: (1) the accuracy of the model, which is equal to the ratio of the total number of registered participants to the number of participants in the chair zone; (2) the average time of registration.

108


The results of preliminary experiments show that the accuracy deteriorates and the average time of registration increases, when the number of participants increases and the level of illumination decreases. The model performance was higher with lighting options 1 and 3 that can be justified by the given location of groups of light; for example, when the projector for displaying presentations is used, the first two groups of light should be switched off for better visibility of slides. Two classes of problems affecting the accuracy of the algorithm of the registration have been revealed: (1) the system-independent errors such as personal hairstyles of participants, active movements, (2) technical limitations, such as the channel bandwidth transmitted by the video stream, as well as the technical characteristics of the PC, where the model is installed. Figure 3 shows an example of the model work in the mode "Registration", where (1), (2), (3) are frames taken from Cam1, Cam4, Cam3, respectively. Here the camera Cam1detects 6 participants in the zone of seats (one false object was detected as a participant). Seat numbers with participants were used for the analysis of the respective zones in the frame recorded by the camera Cam4. In this example, the frame analysis shows the presence of four faces in the areas assigned to these seats. The resulting list of seats controls the PTZ-camera Cam3, which is consistently directed in such a way, in order to photograph the face of each participant with a required quality. Several Frames are recorded and a face is searched in them, since the participants randomly change their positions. The frame is stored only after detection of an object with the dimensions that satisfy with the model of a face.

Fig. 3. Model work in the mode “Registration”

The quality of photos taken by the model in the mode "Registration" depends on the accuracy of the PTZ-camera Cam3 and can be estimated by two criteria: (1) precision of pointing the camera to the face, (2) the dimensions of the photos. The first test shows: how accurately were selected angles of rotation, and the second test is responsible for the correctness of calculation of the scaling factor. The deviation D between the center of the face and the center of the frame is calculated. The deviation is determined in percents to avoid the impact of changes in scale. The relative size R of face is calculated as the ratio of the diagonal of the face image to the diagonal of frame. In the case, if no face in the frame, the deviation is set to 100% and the relative size of the face is 0%. Relative and absolute squares of the face (S and A, respectively) were calculated in addition to the relative size of the face and the deviation. Table 2 shows the results of evaluation of photographing accuracy.


109

Table 2. Assessment of face detection accuracy of the model in the "Registration" mode R,% Min Max 27

53

D,%

S,%

A, Mpixel

Mean

Min

Max

Mean

Min

Max

Mean

Min

Max

Mean

38

0

23

8

7

29

14

0.03

0.12

0.06

These experiments show that the majority of errors in the camera pointing are connected with detection of fake face objects. As the result, the camera focuses not on a face, but on an object, which looks like a human being’s face. A minimum size of a face is taken into account for the face detection in order to overcome this limitation. Most photos from false targets have been identified after this modification of the algorithm. Another problem is motion of people during the photographing that leads to blurred photos. In most cases, recording a series of frames and subsequent filtration allow to get photos with the optimal quality. For camera status monitoring, all the information is sent to the control system of the intelligent room, in which the dialog window displays registered objects, including participants seated at the conference table and in the zone of seats, as well as moving objects. Photos of the registered participants are displayed in the dialog window in accordance with the occupied seats.

4 Conclusion Tracking of moving objects is difficult and important task, solution of which allows automating support of meetings, teleconferences and other events. In the developed model of video monitoring, the technologies for detecting and tracking the participants are implemented and investigated. The developed multi-functional system of Internet cameras is able to work in three modes: “Registration”, “Observation” and “Event support”. Currently, the mode is manually selected by the operator, as well as the analysis of events, factors, equipment states and other situational information, which determines the transition to a particular stage and time of the switching on the cameras, is made. All the modes of operation have been tested in the course of lectures and meetings in the smart room of SPIIRAS. A particular attention was paid to the mode “Registration”. The influence of lighting level, the number of participants and their positions on the accuracy and speed of face searching and the subsequent photographing has been analyzed. At the next stage of the development of the model, other work modes will be tested and a method for automatic selection of the modes of cameras will be offered, as well as mathematical and software tools will be analyzed, and a system for automatic counting of the votes based on recognition of gestures will be investigated. Acknowledgments. This work is supported by The Russian Program “Research and Research-Human Resources for Innovating Russia in 2009-2013” (contract П2360), RFBR (projects 08-08-00128-а, 08-07-90002-СТ_а) and Nokia.

110


References 1. Erol, B., Li, Y.: An overview of technologies for e-meeting and e-lecture. In: IEEE International Conference on Multimedia and Expo., pp. 6–12 (2005) 2. Mukhopadhyay, S., Smith, B.: Passive capture and structuring of lectures. ACM Multimedia, 477–487 (1999) 3. Brotherton, J.A., Bhalodia, J.R., Abowd, G.D.: Automated capture, integration, and visualization of multiple media streams. IEEE Multimedia (1998) 4. Bianchi, M.: AutoAuditorium: A fully automatic, multicamera system to televise auditorium presentations. In: Joint DARPA/NIST Smart Spaces Tech. Workshop (1998) 5. Rui, Y., Gupta, A., Grudin, J., He, L.: Automating lecture capture and broadcast: Technology and videography. ACM Multimedia Systems Journal (2004) 6. Kameda, Y., Nishiguchi, S., Minoh, M.: CARMUL: Concurrent automatic recording for multimedia lecture. In: ICME (2003) 7. Liu, Q., Kimber, D., Foote, J., Wylcox, L., Boreczky, J.: FLYSPEC: A multi-user video camera system with hybrid human and automatic control. ACM Multimedia (2002) 8. Lee, D.S., Erol, B., Graham, J., Hull, J.J., Murata, N.: Portable meeting recorder. ACM Multimedia, 493–502 (2002) 9. Sun, X., Foote, J., Kimber, D., Manjunath, B.: Panoramic video capturing and compressed domain virtual camera control. ACM Multimedia, 229–238 (2001) 10. Friedland, G., Knipping, L., Rojas, R., Tapia, E.: Teaching with an intelligent electronic chalkboard. In: ACM Workshop on Effective Telepresence (2004) 11. Saund, E.: Image mosaicing and a diagrammatic user interface for an office whiteboard scanner. Technical Report, Xerox Palo Alto Research Center (1999) 12. Onishi, M., Izumi, M., Fukunaga, K.: Production of video image by computer controlled camera operation based on distribution of spatiotemporal mutual information. In: ICPR, pp. 102–105 (2000) 13. Wienecke, M., Fink, G., Sagerer, G.: Towards automatic video-based whiteboard reading. In: ICDAR (2003) 14. Maurizio, O., Piergiorgio, S., Alessio, B., Luca, C.: Speaker Localization in CHIL Lectures: Evaluation Criteria and Results. In: Renals, S., Bengio, S. (eds.) MLMI 2005. LNCS, vol. 3869, pp. 476–487. Springer, Heidelberg (2006) 15. Ronzhin, A.L., Budkov, V.Y.: Multimodal Interaction with Intelligent Meeting Room Facilities from Inside and Outside. In: Balandin, S., Moltchanov, D., Koucheryavy, Y. (eds.) ruSMART 2009. LNCS, vol. 5764, pp. 77–88. Springer, Heidelberg (2009)

Physical Approach in Smart Homes A Proposition and a Prototype Michele Dominici, Giulio Zecca, Frédéric Weis, and Michel Banˆ atre INRIA, Rennes-Bretagne Atlantique, Campus Universitaire de Beaulieu 35042 Rennes Cedex, France {Michele.Dominici,Giulio.Zecca,Frederic.Weis,Michel.Banatre}@inria.fr http://www.irisa.fr/

Abstract. The state-of-the-art solutions in the Smart Home domain rely on a logical approach, based on a heavy instrumentation of the environment and on complex techniques that analyze very large amounts of data, collected by sensing devices disseminated in the environment. These aspects negatively influence the delivery of transparent and effective user services. This paper proposes the adoption of a physical approach, in which the information and the computations are carried and executed directly by augmented physical entities, allowing small, meaningful exchanges of data that need less complicated processing. The result is a seamless delivery of useful, transparent and effective user services. The paper describes this approach in detail, analyzes some drawbacks of the state-of-the-art Smart Homes, shows how the physical approach can address those issues and illustrates a prototype that demonstrates the feasibility of the proposed solution. Keywords: Smart Homes; Physical approach; Ubiquitous Computing; Augmented objects; Home automation.

1

Introduction

The 20th century saw a dramatic revolution in domestic technology, a revolution that culminated at the close of the century with the emergence of the concept of Smart Home. A Smart Home is a residence equipped with computing and information technology devices conceived to collaborate in order to anticipate and respond to the needs of the occupants, working to promote their comfort, convenience, security and entertainment while preserving their natural interaction with the environment [1]. When talking about natural interaction, one of the most precious resources is user attention: during their activities, users should be assisted invisibly, reducing interruptions and explicit interactions with the system as much as possible.

The authors would like to thank EDF that has funded this research.

S. Balandin et al. (Eds.): ruSMART/NEW2AN 2010, LNCS 6294, pp. 111–122, 2010. c Springer-Verlag Berlin Heidelberg 2010

112

M. Dominici et al.

A possible way to achieve the goals of this scenario is the application of the human-computer interaction paradigm called Ubiquitous Computing. The aim of this paradigm is to seamlessly and invisibly integrate in the physical environment a multitude of digital devices that provide services to the users without asking for their attention [2]. To this end, Ubiquitous Computing applications are typically context-aware. It has already been highlighted in [3] that the notion of context is not unambiguous and depends on the application domain. Generally speaking, it can be defined as the execution environment of applications, represented as a collection of information regarding the digital, the physical and the user-related environments. When applications are aware of the context, they can easily provide much more useful services, because they “penetrate” their environment and recognize the situations. For example, if a PDA application can sense that a user is currently driving his car, it can automatically switch from visual to audio output [4]. The idea of using the Ubiquitous Computing paradigm in the Smart Home domain is not new. However, the state-of-the-art solutions only partially adhere to its principles. Often the adopted approach consists in a heavy deployment of sensor nodes, which continuously send a lot of data to a central elaboration unit, in charge of the difficult task of extrapolating meaningful information using complex techniques. This is what we call a logical approach. In this paper, we propose instead the adoption of a physical approach, in which the information is spread in the environment, carried by the entities themselves, and the elaboration is directly executed by these entities “inside” the physical world. This allows performing meaningful exchanges of data that will thereafter need a less complicate processing compared to the current solutions. The result is a Smart Home that can, in an easier and better way, integrate the context in its functioning and thus seamlessly deliver more useful and effective user services. Our contribution aims at implementing the physical approach in a domestic environment, showing a solution for improving both comfort and energy savings. The goal of our proposal is twofold. From the one side, it aims at showing the open issues and possibilities in this domain of the ubiquitous computing. From the other side, we present the implementation of a prototype, which intends to be a first step towards the distribution of the information in the environment following a physical approach. Our work is divided as follows. In Section 2 we describe the physical and the logical approaches to the Ubiquitous Computing, explaining how the proposed physical approach can address some open issues of the logical one. In Section 3 we introduce some examples of the logical approach in the Smart Home domain, showing its limits. In Section 4 we describe in detail how the physical approach can be advantageously used in the Smart Home domain, we talk about the rationale that brought us to such kind of solution and we present a prototype that we developed, together with the technologies adopted to implement it. Finally, in Section 5 we draw the conclusions of our study and then lay the foundations for our future research.

Physical Approach in Smart Homes

2

113

Logical vs. Physical Approach in Ubiquitous Computing

As we introduced in the previous Section, we can identify two main approaches to the Ubiquitous Computing: the logical and the physical one. In the logical approach, the participating entities are well identified and the elaboration of the collected data is centralized, decoupling the representation and the elaboration from the physical world. In the physical approach, the information is carried by the entities themselves, and the computation is directly executed by them “inside” the physical world. As an example of logical approach in the Smart Home domain, we can choose the monitoring of the heating conditions in a room. Consider a group of sensors that frequently send surveys about the temperature (see Fig. 1.a), and compare that solution to the corresponding physical one, made of an augmented heater that can publish sensitive information about its status (goal temperature, consumption, heating mode, etc.) to other devices in its proximity (see Fig. 1.b). The logical approach relies on a heavy deployment of sensors in the environment, which continuously collect a lot of information that is sent to a central elaboration unit to extrapolate meaningful data. Following a physical approach, information is spread in the environment, distributed closer to the real entities that compose the system, and the objects can carry different kinds and granularity of information [5]. With a physical approach, sensitive information is brought directly in the objects and devices composing the environment. This allows having meaningful exchanges of data that will thereafter need a less complicate computation compared to a complete feedback from the sensors in the environment. What makes this approach doable is the large number of common objects than can be modified to provide augmented information without changing the way a user employs it. The above-cited example of the heater shows how easier it is to exploit meaningful information provided by the entity itself, instead of having to parse raw data in order to extract a context representation (see Fig. 1.b).

3

Current Approach in the Smart Home Domain

In this section, we identify three main structural and functional components of a Smart Home and present some state-of-the-art prototypes from the point of view of these dimensions. The goal is to characterize the logical approach in the Smart Home scenario and show its limits. 3.1

Characterization of Smart Home Solutions

We can identify three main components of a Smart Home system: Architecture – To reach the goal of assisting and helping inhabitants, Smart Homes are instrumented with the necessary technological equipments. These can

114

M. Dominici et al.

Fig. 1. Logical (a) and physical (b) approach

include communication infrastructures, processing units, sensing and actuating devices (often equipped with computation and wireless communication capabilities), “augmented” appliances, etc. Sensing devices are used to capture context information from the environment and actuating devices are in charge of reflecting on the environment the decisions of the application logic or to provide notifications to the users. Sensing capabilities can include user detection, localization [6, 7] and identification [8], user movements and activity recognition [9–11], as well as monitoring of the physical condition of the house (temperature, light, humidity, etc.). Actuators include lights, heating, ventilation, and air conditioning systems, alarm triggering mechanisms, but also displays, voice synthesizers and other user output devices. Augmented (or smart ) devices are everyday appliances equipped with computational capabilities and communication interfaces. We will refer to these aspects as the architecture of prototypes of Smart Homes. The typical architecture of systems adopting the logical approach is characterized by a heavy instrumentation of the environment (in terms of number of deployed sensing devices), of the users (in terms of imposed constraints and equipments, provoked distractions and privacy invasion) and by the presence of a single, central processing unit. Context Acquisition – The systems have to collect data from the sensing devices in order to use them, in a later phase, to gain some knowledge about the context. The choices that can be made to support this task mostly concern the sensor query algorithms, including the raw sensor data preprocessing, filtering and in-network aggregation techniques [12]. We will refer to these aspects of the


115

systems as the context acquisition choices. Generally, systems adopting the logical approach continuously collect all the available raw sensor readings without preprocessing them, delegating their elaboration to the central processing unit. Context Representation and Exploitation - This functional block of Smart Home systems is in charge of promoting the acquired sensing device data to exploitable context information, in order to learn about behavior, choices and preferences of the users and possibly to predict the context changes. The complexity of this task directly depends on the choices adopted in the previous dimensions. Possible approaches consist in building and reasoning on model-based context representations [13, 14] or in directly analyzing raw sensor readings [9–11, 15]. Systems that adopt the logical approach generally analyze the collected raw sensor readings using machine learning techniques with some use of probability, like Bayesian rules, Markov chains, pattern recognition, adaptive control and so on. 3.2

Critical Analysis of State-of-the-Art Smart Homes

We now introduce some prototypes of Smart Homes that adopt the logical approach and analyze them from the point of view of the proposed characterization, highlighting the emerging drawbacks. Starting from a large literature, we selected three projects that, in our opinion, represent good examples that show the principles of the logical approach and its limits. They in fact share common aspects that are typical of the logical approach: i) they adopt an architecture that includes infrared motion detectors, used to localize inhabitants, and a single central unit; ii) to acquire context, the central unit of each prototype continuously collects raw sensor readings from the whole house; iii) no structured representation of the context is built and its exploitation is made applying machine learning and statistical techniques directly to the raw sensor readings. The PROSAFE project [16, 17] proposes a prototype system capable of monitoring elderly or disabled people and of triggering an alarm when a dangerous situation is detected. The approach is based on thresholds, used for calculating and analyzing user habits. The testing of the prototype showed a high frequency of false alarms, the most impacting cause being the difficulty to calculate trustworthy thresholds and to build a truthfully representative model of user behavior, which can be in practice very unpredictable. The ERGDOM project [18, 19] proposes a solution that manages the house heating system in order to create a comfortable environment for the inhabitants while reducing energy consumption and user intervention. The system predicts user occupation and calculates the comfort temperature analyzing the history of detections and user choices. Tests of the ERGDOM prototype showed that, due to technological limits of the sensors, the system fails to correctly detect user presence in many situations. The Adaptive House [20, 21] aims to automatically manage the lighting, heating and ventilation systems in an energy-efficient way. The core of the system is the capability of determining and predicting the user occupation of each area of the house. Tests performed on the prototype showed that, due to the complexity

116

M. Dominici et al.

of the sensor reading analysis, prediction errors and unmotivated actions of the system are frequent and cause user discomfort and frustration. Analyzing the results of the proposed prototypes, we understand that the logical approach, based on the analysis of continuously collected raw sensor readings, shows important drawbacks in terms of complexity of instrumentation, high cost, low reliability and usability. In the following section, we show how a physical approach can deal with these limitations.

4

Physical Approach in Smart Homes: Benefits and Feasibility

In this section, we propose to adopt the physical approach to the Smart Home domain in order to overcome the highlighted issues. To this end, we present the numerous benefits of this approach and show its feasibility illustrating a prototype. 4.1

Physical Approach in a Smart Home Scenario

In a classical approach, all the intelligence of the Smart Home is condensed in a single entity that takes care of every device in the house. The sensors distributed in the environment have to send back all the gathered data to the central entity, that takes all the burden of parsing the sensitive information and infer the policies to be implemented. Our vision is instead focused on a physical approach, where every device carries a part of the global intelligence: every single entity can analyze the part of information sensitive for its goal, derive useful data, and communicate meaningful information to the other devices. Considering a home scenario, we discover that many of the daily living appliances and objects can be augmented with computational, communication or sensing capabilities without much effort. This results in a new potential for application development and deployment, without an increase in the perceived “complexity” of the environment from the inhabitant point of view. Appliances that can be augmented with communication and computational capabilities in a typical home scenario include, among others, electric heaters, lights and white goods (ovens, refrigerators, washing machines, dishwashers, etc.). Even parts of the house like doors (see Sect. 4.2), floors [7, 8] or electrical systems [6] can be augmented with sensing capabilities. Leveraging these smart objects, we can move the complexity of the system from a central unit (where it is localized in the prototypes showed in Sect. 3.2) to several, distributed devices. The application logic is thus performed directly by the entities of the real world. This approach has the following important characteristics and advantages: 1. There is no more need for an exhaustive collection of information coming from sensors disseminated in the whole house, but a local processing can be performed by the smart appliances. This allows using lighter and less expensive communication infrastructures. Furthermore, this results in a decentralization of the system, which allows users to interact directly with the augmented appliances, reducing the need for a central management unit.


117

2. The fact that information is collected locally and comes directly from the physical objects enhances its value. A new importance can be given to every communication and new criteria can be adopted to filter the events: information acquires a new, physical meaning. Therefore, the computation, the application logic and the entire system “enter” inside the physical world, becoming a part of it. The processing of sensor readings and events becomes easier, more immediate and there is no more need of complex machine learning and statistical analysis techniques. 3. The environment and inhabitants are instrumented in a lighter way but, exploiting the augmented objects, the system is still able to retrieve the needed context information. The system is therefore characterized by an easier and less costly deployment and by a higher degree of acceptance by the inhabitants. The design choices involved in the adoption of the physical approach mostly refer to the instrumentation and depend on the desired user services and scenarios. The selection of the appliances to instrument and in which way and the choices of the equipments used to sense the environment and to recognize user activities have to consider the possible user scenarios and situations. The next sections present a prototype that we realized following the principles of the physical approach and discuss the issues encountered during its design and implementation. The prototype has been realized in collaboration with EDF (Electricité De France) group, the leading electricity producer in France. 4.2

Analysis of the Constraints

The realized prototype had to face the problem of the management of the heating system in a domestic environment; a few issues arose when we evaluated the requirements. First of all, thanks to some of our previous works in the ubiquitous computing domain [22–24], we could state that a normal user is not keen on being instrumented for accomplishing everyday tasks. Furthermore, having to make explicit actions or movements to trigger the response of the system would distract the user from acting as naturally as possible. Planning the interaction of the user with the environment had thus to consider these constraints to build a usable system where the transition from the real to the virtual world is quite seamless. Having in mind these criteria, we moved on searching a way to bring some of the intelligence closer to the physical world. The goal was obtaining a home automation system that could autonomously manage the main consumer of energy in the domestic environment: the heating system. Indeed, a clever management of the heaters can bring to significant energy saving just by optimizing the timers to switch them on and off. To obtain such a goal, the devices must interact one with the other, so we needed communication capabilities. In order to exploit the spatial configuration and relative positions of the devices, we relied on a light version of a tuple communication engine, SPREAD [3]. This allows any device to communicate ordered information to other entities within its communication

118

M. Dominici et al.

range (or bubble) and thus publish useful content in its vicinity. This is a first, basic step to bring capabilities directly into the objects in the environment: as we explained in the above section 4.1, there are a lot of commonly used objects that can be augmented to bring meaningful information that are directly accessible. Once the possibility of information exchange was in place, we needed the objects to provide useful data to describe the environment and the current context. To manage the heating system with a good granularity, we wanted to know the number of users in a room, in order to associate an absence timer that will gradually lower the heating as the room remains empty. A timer has been coupled with each heater, so that the temperature does not change rapidly, but gracefully. This choice has a double overall impact: on the one hand, it avoids continuously switching heaters on and off as a user goes in and out of a room several times, which brings to a energy waste; on the other hand, it increases the perceived comfort, since the room temperature will not instantly decrease just because the user leaves it for a few minutes. In order to fulfill the global goal, different technologies have been analyzed, but we were looking for simple and cost-effective solutions. For this reason we finally adopted a quite novel approach based, on the contrary, on a well-known technology: Infrared (IR) sensors. This approach allowed us to augment a simple and common object, the door, by hiding IR sensors in the frame, which is another step for accomplishing what we are proposing, i.e., exploiting objects in the environment to provide meaningful information. Once provided the infrastructure, we had to take care of the real core of the system, made by the heaters. A way to implement a physical approach to analyze the context was by providing heaters augmented with a little intelligence, needed both to compute the status and to communicate with other neighbors. We equipped each heater with the capability of knowing its location, the ongoing consumption, the temperature and some configuration parameters. Thus, differently from the classical Home Automation systems, part of the intelligence and decision mechanism of our system is executed by the distributed devices. In a “pure” physical approach, the augmented heaters would receive the user presence information directly from the IR sensors. Because of time and resource constraints, however, we decided to use an IR sensor technology that is not equipped with wireless communication interface or computation capabilities. Thus, we introduced an additional device that we call box, which controls the heaters and receives, interprets and publishes the signals from the infrared sensors. This choice doesn’t influence the validity of our prototype and should be seen as a limit that we will overtake in our future work. Nevertheless, the box can perform important tasks that will be kept in our future prototypes: first of all, it can publish in the environment useful information about, e.g., electricity price or meteorological conditions, that it can retrieve from the Internet, acting as a gateway; second, it is suited to our demonstration purposes because it allows having a coherent view of the number of users in each room of the habitat (see Fig. 2). We used it to provide a graphical interface to ease the visual comprehension of the system at runtime, where each room is shown together with


119

the number of occupants and the heating mode of the corresponding heater. In order to represent in a simple way the actual heating mode, we chose a color for each heating level (from colder to hotter: blue, green, orange and red). Therefore, at every instant, a user can visually figure out the temperature range just by looking at the interface (see Fig. 2). Furthermore, every heater has a LED in front of it to show the same color code, so as to ease the user in intuitively accessing heating information distributed in the environment.

Fig. 2. Overview of the prototype

4.3

Prototyping

Implementation. Having in mind the principles illustrated in the previous Sections, we decided to implement a prototype to show which capabilities of the physical approach can be put into practice. The configuration of the system was thus put in place as follows: in the environment we have three rooms, and to enter a room a user has to go through the door, which has two couples of infrared (IR) sensors to detect the movement. One couple is in fact adequate to detect someone crossing the IR flow, but not enough to understand the direction of the user. So, if for example the system detects that the IR couple A has been crossed

120

M. Dominici et al.

before the IR couple B, this means that a user is going in the direction from A to B (see Fig. 2). The heater receives then just a tiny communication about the state change, and applies the agreed policy for changing the heating mode, computing locally the necessary information. An additional feature allows the user to retrieve information on his Personal Digital Assistant (PDA) just by entering the communication bubble of a heater, leaving the possibility of changing the heating mode at will. The devices distributed in the environment are: 1. Infrared Sensors – the movement sensors, directly connected to the box, to detect when a user passes through a door, and in which direction. 2. Heaters – the actuators. In our configuration, each room has a single heater, but this is not a constraint, since the wireless communication capabilities allow a good fine-tuning of their behavior. 3. PDA – the portable device used for having general information, statistics, and to remotely modify the behavior of the heater. In the next future, a common Smartphone could play this role. 4. Box – the central entity that collects readings from IR sensors, sends control messages to heaters and logs all the activity. For the evolution of our solution this component could disappear or just act as a gateway to the Internet. It should be noticed that the real implementation followed some practical constraints of resources and time, but it has to be seen as the first step towards the realization of our vision. Hardware. The choice of the hardware has been made according to our needs, so the box is simulated by a common Laptop Computer, each heater is a mini-pc Norhtec Microclient Jr, and the infrared sensors are Cedes ELS-300 coupled with an acquisition module LabJack U12. In addition to that, an optional functionality is offered by an HP iPAQ 214 Enterprise PDA. The PDA can be used to obtain information of a single heater in proximity. This is possible by combining the power of the 802.11 Wi-Fi communication with the spatial hints provided by RFID tags coupled to each heater.

5

Conclusions and Open Issues

In this paper we proposed a novel way of using a physical approach to develop a Smart Home. As a first step, we analyzed the existing solutions, and we realized that most of them were based on a logical approach. This method relies on lots of sensors that send raw data to a central entity, which in turn takes care of digitally representing the environment. In this case the physical world is not directly coupled with its representation. The physical approach, instead, exploits devices that carry their own part of information, so that meaningful communication occurs among entities and local computation is performed whenever a relevant event is detected. Moreover, we exploit physical properties of the environment and physical proximity between entities to enrich communications and events with an additional value, which simplifies information processing.


121

We developed a first prototype using simple, cost-effective devices and augmented everyday objects distributed in the environment. This led to achieve natural, context-sensitive and implicit interaction with the inhabitants, which in our proposition are not instrumented with wearable sensors or tags and don’t have to face the privacy concerns of video cameras or microphones as in other solutions. Our work lays the foundations for further enhancements of the implementation, so as to strengthen the validity of our vision. The physical approach demonstrated to be not only feasible, but also suitable for the Home Automation domain, validating our research and motivating us to continue delving in the same direction. We will explore new techniques and technologies that allow exploiting the contextual information implicit in domestic interactions and events. We are particularly interested in modeling and recognizing inhabitant activities, which can allow realizing transparent and assistive user services.

References 1. Harper, R.: Inside the smart home. Springer, London (2003) 2. Weiser, M.: The computer for the 21st century. SIGMOBILE Mob. Comput. Commun. Rev. 3(3), 3–11 (1999) 3. Couderc, P., Banˆ atre, M.: SPREADing the web. In: Conti, M., Giordano, S., Gregori, E., Olariu, S. (eds.) PWC 2003. LNCS, vol. 2775, pp. 375–384. Springer, Heidelberg (2003) 4. Schiele, G., Handte, M., Becker, C.: Good manners for pervasive computing–an approach based on the ambient calculus. In: Proc. Fifth Annual IEEE International Conference on Pervasive Computing and Communications Workshops PerCom Workshops ’07, March 19-23, pp. 585–588 (2007) 5. Lee, H., Choi, J.S., Elmasri, R.: A classification and modeling of the quality of contextual information in smart spaces. In: PERCOM ’09: Proceedings of the 2009 IEEE International Conference on Pervasive Computing and Communications, pp. 1–5. IEEE Computer Society, Washington (2009) 6. Patel, S.N., Truong, K.N., Abowd, G.D.: PowerLine positioning: A practical subroom-level indoor location system for domestic use. In: Dourish, P., Friday, A. (eds.) UbiComp 2006. LNCS, vol. 4206, pp. 441–458. Springer, Heidelberg (2006) 7. Valtonen, M., Maentausta, J., Vanhala, J.: Tiletrack: Capacitive human tracking using floor tiles. In: Proc. IEEE International Conference on Pervasive Computing and Communications PerCom 2009, March 9-13, pp. 1–10 (2009) 8. Orr, R.J., Abowd, G.D.: The smart floor: a mechanism for natural user identification and tracking. In: CHI ’00: CHI ’00 extended abstracts on Human factors in computing systems, pp. 275–276. ACM, New York (2000) 9. Gu, T., Wu, Z., Tao, X., Pung, H.K., Lu, J.: epSICAR: An Emerging Patterns based approach to sequential, interleaved and Concurrent Activity Recognition. In: Proc. IEEE International Conference on Pervasive Computing and Communications PerCom 2009, March 9-13, pp. 1–9 (2009) 10. Gu, T., Wu, Z., Wang, L., Tao, X., Lu, J.: Mining emerging patterns for recognizing activities of multiple users in pervasive computing. In: 6th Annual International Mobile and Ubiquitous Systems: Networking & Services, MobiQuitous, MobiQuitous ’09, July 2009, pp. 1–10 (2009)

122

M. Dominici et al.

11. Zhang, Z.: Ubiquitous human motion capture using wearable micro-sensors. In: Proc. IEEE International Conference on Pervasive Computing and Communications PerCom 2009, March 9-13, pp. 1–2 (2009) 12. Yao, Y., Gehrke, J.: The cougar approach to in-network query processing in sensor networks. SIGMOD Rec. 31(3), 9–18 (2002) 13. Boytsov, A., Zaslavsky, A., Synnes, K.: Extending context spaces theory by predicting run-time context. In: Balandin, S., Moltchanov, D., Koucheryavy, Y. (eds.) NEW2AN ’09 and ruSMART ’09. LNCS, vol. 5764, pp. 8–21. Springer, Heidelberg (2009) 14. Weis, T., Becker, C., Br¨ andle, A.: Towards a programming paradigm for pervasive applications based on the ambient calculus. In: International Workshop on Combining Theory and Systems Building in Pervasive Computing, part of PERVASIVE, Citeseer (2006) 15. R¨ omer, K., Mattern, F.: Event-based systems for detecting real-world states with sensor networks: a critical analysis. In: Proceedings of the 2004 Intelligent Sensors, Sensor Networks and Information Processing Conference, December 2004, pp. 389– 395 (2004) 16. Campo, E., Chan, M.: Detecting abnormal behaviour by real-time monitoring of patients. In: Proc. of the AAAI-02 Workshop Automation as Caregiver, pp. 8–12 (2002) 17. Chan, M., Hariton, C., Ringeard, P., Campo, E.: Smart house automation system for the elderly and the disabled. In: IEEE International Conference on Systems, Man and Cybernetics, Intelligent Systems for the 21st Century, October 1995, vol. 2, pp. 1586–1589 (1995) ´ Chan, M., Estève, D.: L’apprentissage des modes de vie: une base 18. Campo, E., indispensable au développement d’un habitat “intelligent”. Annals of Telecommunications 58(5), 850–865 (2003) 19. Bonhomme, S.: Méthodologie et outils pour la conception d’un habitat intelligent. PhD thesis, Institut National Polytechnique de Toulouse (2008) 20. Mozer, M.C.: Lessons from an adaptive home. In: Cook, D.J., Das, S.K. (eds.) Smart Environments, pp. 271–294. John Wiley & Sons, Inc., Chichester (2004) 21. Mozer, M.: The neural network house: An environment that adapts to its inhabitants. In: Proceedings of the American Association for Artificial Intelligence Spring Symposium on Intelligent Environments, Citeseer, pp. 110–114 (1998) 22. Le Bourdon, X., Couderc, P., Banˆ atre, M.: Spontaneous hotspots: Sharing contextdependant resources using bluetooth. In: International Conference on Autonomic and Autonomous Systems (ICAS’06) (July 2006) 23. Couderc, P., Becus, M., Banˆ atre, M.: Ubi-board: A smart information diffusion system. In: Balandin, S., Moltchanov, D., Koucheryavy, Y. (eds.) NEW2AN 2008. LNCS, vol. 5174, pp. 318–329. Springer, Heidelberg (2008) 24. Couderc, P., Banˆ atre, M.: Ubi-check: a pervasive integrity checking system. In: Balandin, S., Moltchanov, D., Koucheryavy, Y. (eds.) ruSMART 2009. LNCS, vol. 5764, pp. 89–96. Springer, Heidelberg (2009)

SMEO: A Platform for Smart Classrooms with Enhanced Information Access and Operations Automation Akshey Jawa1, Sudip Datta2 , Sujeeth Nanda1 , Vishal Garg3 , Vasudeva Varma4 , Suresh Chande5 , and Murali Krishna Punaganti Venkata5 1

Centre for IT in Building Science, IIIT Hyderabad, Hyderabad, India {aksheyjawa,sujeeth}@research.iiit.ac.in 2 Search and Information Extraction Lab, IIIT Hyderabad, Hyderabad, India [email protected] 3 Centre for IT in Building Science, IIIT Hyderabad, Hyderabad, India [email protected] 4 Search and Information Extraction Lab, IIIT Hyderabad, Hyderabad, India [email protected] 5 Nokia Devices R&D Helsinki, Finland {suresh.chande,murali-krishna.punaganti}@nokia.com

Abstract. This paper explores the area of creating smart spaces using mobile Internet devices to develop intelligent classrooms. We propose a suite of applications that leverage the recent progress made in the domains of mobile computers and information access, resulting in a richer classroom experience where knowledge is dispensed with greater efficiency while also making it possible to access it efficiently from the Internet as well as peers. We attempt to achieve this by introducing novel means of classroom teaching and management and an intelligent information access system while harnessing the computational capabilities of mobile Internet devices. Though the current work is still in progress, this paper elaborates on our attempts in reducing the gap between technological progress in various domains and the existing pedagogy. Keywords: Smart Spaces, Intelligent Classrooms, Teaching Methodologies, Information Access.

1

Introduction

The last couple of years have seen advent of a number of novel and powerful mobile computers which have revolutionized the mobile market. A couple of years back mobile devices had low processing power, limited multimedia functionalities, poor web connectivity, and restricted memory and storage capacities, S. Balandin et al. (Eds.): ruSMART/NEW2AN 2010, LNCS 6294, pp. 123–134, 2010. c Springer-Verlag Berlin Heidelberg 2010

124

A. Jawa et al.

leaving a wide gap between the capabilities of a mobile device and that of a desktop PC. In recent times, handy mobile Internet devices with higher processing powers and availability of a large array of features has largely bridged this gap for a number of functionalities, opening new vistas for application of these capabilities. At the same time, rapid growth of Internet has resulted in an unprecedented increase of content on the web. Availability of large amounts of information on the web resulted in a pressing need for development of ‘Information Access’ systems that would assist users with an information need in identifying relevant information and producing it in a concise form. This led to research and development in various modes of Information Access such as Information Retrieval, Text Categorization, Text Summarization, among others. Over the years, these generic systems have matured considerably leading to opportunities for their application in new scenarios and systems. While there has been remarkable growth in technology, the classroom pedagogy has been largely oblivious of these technological gains, with the instructors still employing boards and presentations while the students take notes on paper notebooks. In the current teaching methodology, a students learning is constrained by the knowledge of the instructor while the opportunities for learning from peers remain largely untapped. Also, the vast pool of knowledge that exists on the web is not effectively consumed because it is not available on the go, coupled with issues of retrieving the needed information in a precise and swift manner. In this work we attempt to address the stated issues by introducing smart mobile Internet devices in a classroom environment where efficient classroom teaching and management, and intelligent information access can be provided using the 3Cs of the device namely, context, communication and computation. We next elaborate on the 3Cs of the device, claiming that their presence can be utilized to harness the technological growth in enhancing learning. The context of classroom environment helps us in knowing that only documents that are academic in nature should constitute the repository. Moreover, knowledge about the topic to be discussed in next class can be taken as a factor to determine the temporal importance and relevance of a document. In fact, the context of which user makes a query is used as a parameter to customize the choice and ordering of results. Communication capabilities of the device are used extensively for the project. To cite a few examples, it is used to connect devices over remote framebuffer protocol using VNC. Similarly, course content sharing both in real time as well as through archive happens over the network. From the computation perspective, the device’s touchscreen capability allows the instructor to annotate over the slides during the presentation, students to view slides, make notes and pass their queries vocally to the instructor, making it particularly useful when they are remotely connected. In future, with improved Optical Character Recognition (OCR) capabilities, annotations will be converted into text, thus making it searchable.

SMEO: A Platform for Smart Classrooms

125

Recent years have seen a number of attempts to technologize the classroom pedagogy. Some of these systems, like the closed source Blackboard1 system, have been commercially used in niche classroom management market, providing functionalities for course and content management system. Classroom presenter [1], [2] is an open source research oriented system largely focusing on distance learning. It also provides functionalities for lecture archiving, including voice and annotations. One drawback of this system is that it does not work on mobile devices. Ubiquitous [3], is a system that attempts to get over this drawback by using platform independent web-based front end while still depending on classroom presenter in the back end. DyKnow [4], another closed source commercial product, in addition to classroom teaching functionalities, also provides the instructor with ability to monitor student devices. Online conferencing systems like Dimdim2 are also occasionally used in classrooms providing functionalities like collaborative document viewing and annotations with voice streaming. In this paper we propose SMEO, a system that in addition to classroom teaching and management using mobile devices also provides functionalities for smart information access. While, the classroom teaching and management module provides tools to stream and archive lecture, and to manage attendance, classroom facilities, queries, and quizzes, the most unique contribution of this work is integration of smart information access techniques which to the best of our knowledge is the first such attempt. Focusing on contextual personalized search, summarization and collaborative learning, it aims at providing improved means of learning in a classroom environment. While the current work is still under progress, we present the inspiration for this work along with it current status, besides stating the goals of this work. The remainder of this paper is organized as follows: in Section 2 we discuss the architectural setup that we used. We follow this with a broad outline of the proposed platform in Section 3. Section 4 elaborates our ideas on improving classroom teaching and management using the mobile computer devices while Section 5 expands on how intelligent information access techniques could enhance learning. Finally, in Section 6 we state the conclusions drawn from this work along with suggestions for future improvements.

2

Architecture

SMEO requires the following infrastructure to be present in the classroom: 1. 2. 3. 4. 5. 6. 7. 1 2

A Linux server for handling the classroom teaching and management module A Linux server for running the information access module A projector High speed Wi-Fi Internet connection A mobile computer for the instructor A mobile computer for each student

http://www.blackboard.com http://www.dimdim.com

126

A. Jawa et al.

Fig. 1. Architectural Setup

As shown in Fig. 1 the classroom teaching and management Linux server is directly connected to the projector, so the slides which the instructor wants to project on to the projector are kept on this server. But the instructor views, controls and annotates the slides from the mobile computer which is connected to this server through VNC which shows the remote computer’s display on the local computer which is the mobile computer in this case. All the communication between the instructor and student mobile computers, whether it is student queries to the instructor or conducting of quiz by the instructor, happens through the classroom teaching and management server. The information access module, on the other hand is responsible for running all search related jobs, such as crawling, indexing, maintenance of profiles, among others. This should be accessible to the devices and connected to the Internet. Both the servers are internally connected because sharing of information among them, is required at various steps, such as extending the operations of search and summarization to the course content.

3

The Application

SMEO is a platform for creating smart spaces in a classroom environment providing a suite of applications on high performance mobile Internet devices that can broadly be categorized into - Classroom teaching and management: focusing on smarter delivery of lectures and Intelligent information access: focusing on smarter means to access information. We attempt to enhance classroom learning by providing relevant applications from both these class of applications through a simple interface to the instructor and the students. In the following sections we elaborate on the methodology and work-flow of each of these applications. Because of the large number of tools and limited space we show the screenshots of only few of these applications. The set of applications and the underlying functionalities to which an individual has access, varies according to their privileges, depending on whether the


Fig. 2. Instructor’s Interface of Applications

127

Fig. 3. Manage Lecture Display

individual accessing the application is an instructor or a student. The start interface for the instructor is shown in Fig. 2. Student interface, though similar, is not shown because of paucity of space.

4

Classroom Teaching and Management

Once SMEO authenticates a user through a pre-registered username and password, the classroom teaching and management module provides interface for all interactions between the user and the application. There exist separate interfaces for instructors and students and corresponding to their roles and privileges a set of applications are displayed. While we discuss most applications provided by SMEO (from instructor’s perspective as well as student’s perspective) in this section, we devote a separate section for the information access functionalities because of the uniqueness of its applicability in a classroom environment. 4.1

Lecture Delivery/Attendance

This module allows the instructor to use the mobile computer to navigate the slides and annotate over them. The lecture is streamed to the students in real time and is also archived so that students can download it for offline use. Students can also make private annotations which can serve as their notes. Before the teacher starts the lecture, he needs to upload the presentation to the classroom management server. The system ensures that the instructor has filled meta-fields of title and keywords, which are required by the information access module, before uploading. Fig. 3 shows the screen for managing lectures. The instructor can either add or remove a lecture, publish (making it accessible to students) or unpublish it, or request retrieval and indexing of documents related to the current lecture. As soon as lecture is published, crawling and indexing of documents related to the topics to be covered, is started. Thus, an instructor is required to publish the lecture content, a couple of hours before the lecture to allow crawling and indexing of relevant documents, ahead of the lecture, enabling the availability of updated information access applications. When the teacher wants to take a lecture, he connects to the server using VNC (Virtual Network Computing) system. Next, controlling the server through VNC,

128

A. Jawa et al.

Fig. 4. Lecture streaming

instructor opens the presentation on the server with FreOffice (koffice port for Maemo). The original version of FreOffice did not have the capability to annotate but the extended version on which we are working, allows annotations on slides. The transmission of server’s display, annotations and audio over the network is explained in Fig. 4. Through VNC, when the instructor annotates the slides by scribbling over them using the stylus, each annotation stroke is stored as a separate file. The directory in which the annotation stroke files are created is kept in sync with a directory on student side using rsync. Also, instructor’s voice is streamed to student devices, besides being recorded for archival purposes. On the server side, an XML file is maintained, which stores the names of the annotation stroke files and the time stamps of the annotation stroke with the associated slide number. When the lecture finishes, this XML file and the annotation stroke files are pushed to the archive so that students or the instructor can download them for offline use. When working in offline mode, our extended FreOffice parses the XML containing the time stamps and the annotation stroke file names, and displays them over the slides in a time sequence. Students are also allowed to make their own annotations which can be shared among peers, thus allowing community sharing of notes. For reducing clutter on the displayed presentation, students can switch on/off one or more of the annotation layers. If a student only wants to view the instructor’s annotations with the slides, he can simply connect to the VNC server running on the server machine in ‘view only mode’. This can be done anytime during the course of a lecture. This allows a student to observe activities on the display of the server (which is also displayed on the projector screen) by running VNC server in ‘view-only’ mode. Note: The tool described in this Section can also be used by the instructor to correct assignments by making annotations or by recording vocal comments,


129

which are sent to students once the correction is done. This makes the assignment evaluation more interactive and swift. 4.2

Intelligent Information Access

This is a web based suite of tools for efficiently accessing contextual information available on the web along with that generated in the course. It has tools for contextual personalized search, summarization and collaborative learning, enabling access to relevant information in a well presented format. While it provides an interface for quick search and accessing other ‘information access’ services to students, the instructor’s interface is supplemented with options to specify indexing process, corresponding ‘seed topics’, besides other functionalities. Services provided by the information access suite are further elaborated in Section 5. 4.3

Classroom Question Answering

This application provides an efficient question answering mechanism for the classroom. During the lecture, students can send requests to ask queries. These requests are queued on the server. The instructor can look at the set of queued queries and approve acceptance of one request at a time. When a request is approved, a message “Ask your question!” is flashed on the referred student’s device. When the question is asked, the voice is forwarded to all the students devices which are part of that course. This enables the discussion in the class to be audible, making it suitable for large classes to be handled well, as many times the queries as well as the responses are very useful learning experience for the listeners. Since we have the voice channel open between several devices, it makes the overall session audible to all the students connected to the lecture. 4.4

Design/Attempt Quiz

This module helps the instructor to create a password protected quiz and share it with the students. It also allows the student to open the quiz, answer it using a user friendly quiz form and view the grades once it is graded. An instructor can create a quiz using the ‘quiz creator’, either on laptop or a mobile computer. It creates an encrypted file containing the questions and the answers in an XML format, which is uploaded on the server and a start time for the quiz is announced. The encryption key is known only to the instructor. The instructor then uploads this file on to the server and announces a start time for the quiz. Students download the quiz file at their convenience and at the start time, the key is made public. This key needs to be fed to the quiz client for accessing the quiz. The quiz client decrypts the file using the key, and creates a user friendly form to allow the students to answer the quiz and submit the answers to the server. It allows the students to mark certain questions for review,

130

A. Jawa et al.

filter questions and search questions. Another thing to note is that this quiz XML follows Moodle quiz XML schema so it can also be used with Moodle [5]. At the finishing of the quiz duration the grades are calculated by the client, sent to the server for archiving and displayed to the student on his mobile computer. 4.5

Classroom Feedback

One unique feature of our project is the feedback system. It allows the instructor to have real-time feedback on how many students are understanding the presentation. This feedback is integrated with our extended FreOffice. It allows the student to rate any particular slide for its difficulty. This gives the instructor a real-time feedback if students are finding some topic difficult to comprehend. Students also have an option to give feedback for the whole lecture. The difficulty rating is kept anonymous, but for the lecture feedback student has a choice to reveal his identity. 4.6

Facility Control

This module enhances classroom comfort by controlling classroom facilities, like fans, lights, A/Cs, projectors, window films, and dimmable lights, automatically or by minimal manual intervention. We are working on a building automation gateway that connects with any building automation backends which can work with known building automation protocols such as LonTalk, or bacnet, and can connect to various facilities through relays, Wi-Fi, Zigbee [6] or IR. The gateway, which is an embedded server, stores the coordinates of available facilities and different mode settings for the classroom in an XML file which the instructor can download. Using the XML file, the instructor side application constructs a map of the classroom, showing the facilities and their on/off status. Clicking on any of the buttons in the map switches the appliance on or off and also changes the color of the button. The instructor can also select one of the various modes, “Presentation” mode, “All lights off” mode, etc, to switch on/off a number of appliances at a single click. The gateway also has an access control mechanism which does not allow anybody other than the instructor to control the facilities. For making the classroom energy efficient, the gateway dynamically controls the operating parameters based on the current needs and also uses feed forward and feedback controls. The devices help in getting a feedback in terms of number of people present and their comfort requirements which helps the system to optimize the energy consumption while still maintaining the comfort requirements. Methods such as demand based fresh air, enthalpy based control, etc are also used. 4.7

Remote Classroom Presence

Another interesting feature of the system is that the students or the instructors can attend or give a lecture from any corner of the world. They just need a


131

working Internet connection on their mobile computers which can connect them to the classroom over a Virtual Private Network. This makes all the above mentioned tools accessible to them. This feature can also allow the subject matter experts to give lectures remotely or even take a whole course remotely. Similarly, students from different universities can be given access to attend the lectures remotely. 4.8

Classroom Attendance

This is a simple application that allows the instructor to know the number of students connected to the server, either remotely or locally.

5

Intelligent Information Access

The most unique aspect of this project is the incorporation of ‘Intelligent Information Access’ in a classroom environment and its integration with classroom management modules. By ‘Intelligent Information Access’ we imply the ability to retrieve, extract and access the information as needed in the context of a classroom. We provide the stated capability through a wide-range of information access techniques, primarily focused on contextual search, personalization and summarization technologies while also exploring the domain of collaborative learning. We next elaborate on each of these techniques along with a discussion on their adaptation to the classroom environment. 5.1

Corpus Building and Contextual Search

By contextual search we imply that the system is aware that it is catering to queries given in a classroom environment and hence has to produce results that are academic in nature. Moreover, the system also takes into consideration the courses currently being studied by a student while producing search results for a user query. In fact, it attempts to provide results that are contextually more relevant to the topics covered in recent classes. To achieve this we used a twopronged approach of selectively constructing a corpus of relevant documents and its deep-indexing which is complemented by personalization (explained in section 5.2) of results. This module also builds the corpus by indexing relevant documents retrieved from the web along with course content stored in classroom management folder, making it searchable. The first step to contextual search is corpus building. It is depicted in Fig. 5 and works in the following manner: as mentioned in Section 4.1, with each lecture presentation, the instructor must fill the meta-fields of title and keywords. These keywords serve as the ‘seed topics’ for retrieving other related topics. These topics are given to Wikipedia Miner [7], a toolkit for navigating and making use of the structure and content of Wikipedia, to retrieve semantically related topics. A combination of ‘seed’ and related topics, serving as queries, are submitted to generic search engines and top-hits are crawled. These documents are then

132

A. Jawa et al.

Fig. 5. Contextual Search: Flow Diagram

deeply indexed for classroom consumption. By deep-indexing we imply that the index contains much richer information than a conventional inverted index, such as named entities, other semantic entities and meta-information like the course and lecture associated with it. This index can now be searched and the search returns much more contextually relevant results. 5.2

Personalization

Personalization aims to sort search results based on the specific needs and interests of individuals, instead of producing same results to anyone giving an identical query. Personalization of search tools adds an extra step to the search process. Thus, the query is first matched against the corpus of documents to retrieve relevant results. This is followed by matching results with user profiles to sort the results according to user preferences. The profile of a user contains explicitly gathered information such as the content of courses done in the past and those that are currently being pursued along with implicitly gathered dynamic information such as search history consisting of queries and the corresponding hits. This information is used by the system to customize the search results so as to produce more relevant results for a user. We also attempt at extending personalization to a group level because individuals with similar profiles might benefit from the search history of each other. Thus we also attempt at maintaining group profiles and customizing results for users with similar profiles at a group level, as well. 5.3

Summarization

Document summarization is defined as an automatic procedure aimed at extraction of information from multiple text documents. It is the process of condensing text information to its most essential facts. Thus, a well composed and coherent summary is a solution for information overload. In the current work, while we do not make claims about improving the document summarization algorithms, we worked on employing existing summarization systems in a smart way. While we used [8] in this work, any other high precision generic multi document summarizer can be substituted as a replacement to our summarizer. Besides providing an interface for conventional multi-document summarization we also provided two novel manifestations of text summarizers. In the first of these, a user expresses information need in the form of a query for which the contextual search engine displays results. The user can seek a summary of any subset of these


133

results by selecting check-boxes provided next to the results and clicking on the summarize button. In the second case a user expresses information need in the form of a set of questions, and the system produces a single text as the summarized version of a single or multiple documents containing the relevant information. That is, given a user’s information need the system identifies a set of relevant documents from the corpus and further creates, from the document set, a summary which answers the expressed information need. 5.4

Collaborative Learning

In this module we explored group learning aspects wherein knowledge acquired by one student could be utilized by the student community using the system. In the current work we introduced some simple yet effective features for collaborative learning. For example, we took explicit (in the form of link recommendation) and implicit (in the form of page hits for a given query) feedback from the search module to improve the results. We also collected statistics such as the most queried terms and the most viewed documents for each course, over different time periods, to make recommendations. We also had a feature for collaborative bookmarking and their tagging. Making this information searchable came handy while looking for links pertaining to topics identified by tags. We also attempted to make the annotations made by students, searchable. This would have helped students in getting better insights to the text mentioned in a slide utilizing the comprehension of their peers. Unfortunately, this was hampered by the limited OCR capabilities of the mobile computers that we used in our work. Future devices, with improved OCR capabilities should make it feasible to convert annotations into text which can easily be made searchable, thereby greatly enhancing the scope of collaborative learning. We believe that we only touched the surface of group learning and there is considerable scope of enhanced learning in this domain.

6

Conclusion and Future Work

We believe that the system which we have described is yet another step in the direction of digitizing the classrooms. Since the whole system is open source it can be extended to support the future needs of the classrooms. There still remain a number of issues and challenges to overcome in the attempt to digitize classrooms. While technical advances will happen and newer functionalities and features will be introduced, probably the greatest challenge is the introduction and acceptance of such means of teaching and learning in the classroom setup. But, we should remember that it was not too long ago that the community was apprehensive about the success of teaching through presentation slides but now their use is widespread. While presentation slides only made classroom teaching more structured and orderly, development of classrooms on our lines might influence and enhance learning more drastically. We believe that further experiments and evaluative studies are required in this direction to ascertain their influence.

134

A. Jawa et al.

In the current work we proposed a prototype for improving and enhancing classroom teaching and learning, there still remain some critical issues that need to be worked on, before the success of current work can be ensured. As the next step, we need to work on issues such as ensuring confidentiality and reliability of the system while identifying alternative means of working, in cases of device failures. We also need to employ the system in an actual classroom environment to determine the resource requirements of the system and to resolve issues of scalability, should it arise. As the mobile devices become more powerful and technologies like OCR help replace pen and paper with digital pen (stylus) and digit paper (touch screen) we hope that the classrooms of tomorrow, in addition to being user friendly, environmental friendly and energy efficient, will be fully digital, remotely accessible and will have efficient techniques for retrieving and presenting relevant information.

References 1. Anderson, R., Anderson, R., Simon, B., Wolfman, S.A., VanDeGrift, T., Yasuhara, K.: Experiences with a tablet pc based lecture presentation system in computer science courses. In: SIGCSE ’04: Proceedings of the 35th SIGCSE technical symposium on Computer science education, pp. 56–60. ACM, New York (2004) 2. Anderson, R.J., Anderson, R.E., Linnell, N., Pervaiz, M., Saif, U., Videon, F.: Collaborative technologies in international distance education. In: CSCWD, pp. 522– 527 (2009) 3. Wilkerson, M., Griswold, W.G., Simon, B.: Ubiquitous presenter: increasing student access and control in a digital lecturing environment. In: SIGCSE ’05: Proceedings of the 36th SIGCSE technical symposium on Computer science education, pp. 116– 120. ACM, New York (2005) 4. Dave, B.: Pushing forty(courses per semester):pen-computing and dyknow tools at depauw. In: The Impact of Tablet PCs and Pen-based Technology on Education, pp. 31–39. Purdue University Press (2006) 5. Dougiamas, M., Taylor, P.: Moodle: using learning communities to create an open source course management system. In: Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications, AACE 2003, vol. (1), pp. 171–178 (2003) 6. Fang, A., Xu, X., Yang, W., Zhang, L.: The Realization of Intelligent Home by ZigBee Wireless Network Technology. In: PACCS, pp. 81–84. IEEE Computer Society, Los Alamitos (2009) 7. Milne, D., Witten, I.: An open-source toolkit for mining wikipedia (2009) 8. Bysani, P., Bharat, V., Varma, V.: Modeling novelty and feature combination using support vector regression for update summarization. In: 7th International Conference on Natural Language Processing, NLP Association of India (2009)

Creative Approach to the Design and Prototyping of Experimental Smart Spaces, Case Studies from the Interactive Environments Minor Tomasz Jaskiewicz1, Walter A. Aprile2, and Aadjan van der Helm2 1

Faculty of Architecture, Delft University of Technology Julianalaan 134, 2628 BL, Delft, The Netherlands +31 15 2785954 [email protected] http://www.hyperbody.nl 2 Faculty of Industrial Design Engineering, Delft University of Technology Landbergstraat 15,2628 CE, Delft, The Netherlands +31 15 2784966 [email protected], [email protected] http://studiolab.io.tudelft.nl

Abstract. This paper attempts to challenge the established practice of design and engineering of smart environments in two ways. Its first premise is that smart spaces should be formed by systems that comprise not only home or office appliances, but also embedded building component agents. The second premise of this paper is a consequent call for an interdisciplinary approach to creation of smart spaces. Experimental undergraduate course is used to provide a set of case studies to illustrate benefits and threats of such approach, as well as the relevance of the integration of iterative prototyping already in the earliest phases of design processes. Keywords: Context-aware systems, interactive architecture, interaction design, swarm modularity, prototyping.

1 Introduction 1.1 Smart Buildings The sister notions of smart homes, smart buildings or more generally; smart spaces or smart environments have become synonymous to the concept of "a physical world that is richly and invisibly interwoven with sensors, actuators, displays, and computational elements, embedded seamlessly in the everyday objects of our lives, and connected through a continuous network"[1]. The common justification for much of the research in this domain, as well as for numerous commercial projects in the field of home automation is the technological facilitation of everyday chores of typical users of such environments. Consequently, a “smart home” can be defined as “a residence equipped with computing and information technologies which anticipates and responds to the needs of the occupants, working to promote their comfort, convenience, S. Balandin et al. (Eds.): ruSMART/NEW2AN 2010, LNCS 6294, pp. 135–147, 2010. © Springer-Verlag Berlin Heidelberg 2010

136

T. Jaskiewicz, W.A. Aprile, and A. van der Helm

security and entertainment through the management of technology within the home and connections to the world beyond”[2]. In larger and non-residential buildings, building energy efficiency, security, as well as in many cases also building flexibility become main drives, whereas user comfort maintains its relevance, but is not paramount. Overall, it is commonly acknowledged that “the essence of the smart building is comprised of advanced and integrated systems for building automation, life safety and telecommunications systems. (…) smart buildings are integrating these systems from a physical and logical perspective.” [3]. Definitions given to this point do not fully justify the use of the term “smart”. It is commonly acknowledged that users should be able to interact with building systems in “an intelligent and unobtrusive way” and in order to achieve this kind of interaction “these environments should be aware of the needs of people, customizing requirements and forecasting behaviours” [4]. Achieving this is the goal of the domain of ambient intelligence (AmI), which provides solutions for achieving the said “smartness”, often through application of multi-agent, adaptive computational models. There are many clichés that can be commonly encountered around the domain of automated/smart buildings. These can be illustrated by a quote from the popular publication “Smart Homes for Dummies”; “Your smart home can seep into all aspects of your life. It helps you to choose day-to-day tasks that can take up so much time, such as opening the draperies, dimming the lights, and flipping on the Weather Channel to see whether the kids have a snow day.” [5]. The fallacy in this example is the assumption that an activity of dimming the lights or changing channels on television is indeed a notable burden to home inhabitants. Typically, solutions attempting to automate such trivial tasks tend to be over-complex, relatively costly and can ultimately create more annoyance than actual benefit to the users. Clearly, the real drive behind many projects, including do-it-yourself smart homes hinted by “Smart Homes for Dummies” is their technological “coolness”, rather than real daily-life benefits for their users. This paper attempts to challenge the established practice of design and engineering of smart environments in two ways. Its first premise is that smart spaces should be formed by systems that comprise not only home or office appliances, but also embedded building component agents. It is common to neglect architectural building components as potential smart embedded agents of building systems (with the exception of few unique experimental projects such as “The Media House”[6]). Such approach is not only expected to deliver unprecedented range of sensing and actuating possibilities to enhance smart building systems. It also opens new doors between the domains of embedded systems and architectural design. The second premise of this paper is a consequent call for an interdisciplinary approach to creation of smart spaces. It is clear to us that in order to design and engineer successful smart environments it is necessary to integrate knowledge of different disciplines in order to address not only technological, but also cultural, social, structural and aesthetical aspects of such systems that are all inherently interdependent on each other. 1.2 Designing Buildings as Systems Considering buildings to be dynamically performing entities, engaging multiple users and external factors in interactive exchange of information means that architecture has

Creative Approach to the Design and Prototyping of Experimental Smart Spaces

137

to be treated as a system. Thinking of architecture as a system inevitably opens the issue of drawing border, that’s to say defining where the system ends. Contemporary design of complex entities brings in the concept of systems being made up of hierarchies and graphs of interacting smaller and bigger systems. Defining the edges and the reach of the system depends entirely on the designer’s purpose: there are no “natural” borders, since what could be outside of a system from an electrical engineer’s point of view may very well be a key constituent from the point of view of an economist. The systemic way of dealing with architecture and architectural design implies taking the approach of treating architectural constructs not as finite singularities, but as multidimensional multiplicities. These multiplicities or “systems of systems” are the field of inquiry of systems theory and cybernetics (the study of regulatory systems). These fields have much to give to architecture. Traditionally, architects are limited to providing only a very limited number of variants of their designs for each project, with no more than few scenarios of how it could change over time. Creating buildings as systems means delivering wide and flexible ranges of design solutions depending on multiple parameters such as environmental conditions or the development of socio-spatial ecologies. It also means accepting the transitory nature of a building, its site and its pattern of use: it is a way of increasing the flexibility and the future survivability of the chosen architectural solution. One may be tempted to create a building system from a constrained set of logical or physical elements and a mesh of formal relations between them. However, the intricacy of problems that architecture has to deal with makes that simplification unlikely to succeed, as only a very limited amount of design factors can be accounted for in this way. Therefore, architecture has to be seen as a complex system - a system composed of a very high number of interconnected parts. Such parts can be nested (sets of parts may constitute higher order parts). What is also important to note here is that such complex systems may exhibit properties (including behavior) not evident from the properties of its individual parts. Architectural systems, particularly when their users are brought into the picture, are prime candidates for phenomena of “emergence”1, in which very complex behavior materializes out of very simple rules. We can also expect “strange loops” in buildings, a phenomenon where an abstract look crosses levels of abstraction as in an Escher drawing. For instance, in a building the lowest level of architectural detail (a molding, a handle) can influence sharply emotions and development of the building users. Conversely, the moods, beliefs and behaviors of inhabitants can destroy or modify even the best built apartment block. 1.3 Problem There are no established methodologies for creative design of architectural structures as complex interactive systems, however many lessons can be learned from the domains of human-computer interaction and interaction design [7], [8], [9], [10], [11]. 1

For a history of the fortune of the concept, see Corning, P. A. (2002). “The re-emergence of emergence: A venerable concept in search of a theory”. Complexity, 7(6), 18-30.

138


Numerous relevant projects exist that explore the overlap of art, design and technology by means of creation of complexly interactive spatial installations, such as Philip Beesley’s “Hylozoic Soil” or Usman Haque’s “Open Burble”, typically developed for art-technology festivals, exhibitions and other public events. However, in such projects emphasis is put on creativity and rarely on direct practical applicability of proposed solutions. These projects are being developed following a subjective artistic vision or concept. On the other hand, typical engineering projects involved in creation of “smart spaces” are limited to technological facilitation of generic user activities, without consideration to the larger context of cultural or social problems that may be of significant relevance to the ultimate success of these projects. What’s more, typically “smart spaces” augment existing building spaces with technology, rather than being inherently integrated into building structures. In this way they are not having an active, conscious role in creation of resulting architectural qualities. A new approach is needed to merge these two trends, promote and encourage a creative attitude to design and engineering of “smart spaces”. Such an approach requires integration of knowledge and skills from disparate fields, including but not limited to; human-computer interaction (HCI), interaction design (IxD), architectural design, industrial design, embedded systems and structural engineering. A novel strategy should facilitate out-of-the-box design of “smart spaces” with emphasis on both; inventive problem posing and their creative solving using state-of-the-art technology.

2 Case Study Project At the Delft University of Technology, a “minor” is a six-month project that occupies groups of students in the last semester of their BA. The university encourages the creation of minors that include students and staff from more than one faculty. In the particular case of the Interactive Environments minor, students came from Architecture, Industrial Design and Engineering, while the teaching staff was from Architecture (research group Hyperbody2) and Industrial Design (ID-StudioLab3). It has been coordinated, co-organized, and co-taught by the authors. The main purpose of the programme was to create an experimental framework where interdisciplinary researchers, practitioners and students could be brought together to work on the ideas of embedding interaction in architecture. Rather than working on these ideas on a purely theoretical level, the goal was to design and build in full-scale an operational, spatial interactive environment prototype. Without neglecting conceptual validity and development, we decided to insist on the concept of “prototyping an environment”, as a way to bring as soon as possible into the design process as much reality as possible, a very important practice when dealing with speculative forms and new and uncertain 2

Hyperbody is a research group at the Faculty of Architecture at the Delft University of Technology directed by prof. ir. Kas Oosterhuis. The goals set for the group’s research are to explore techniques and methods for designing and building of non-standard, virtual and interactive architectures. Cutting edge techniques and methods are taught and applied by researchers and students.” http://www.hyperbody.nl 3 “The ID StudioLab is a group of designers, researchers, and students in the department of Industrial Design […] The StudioLab aims at integrating multidisciplinary knowledge on user, technology, and the product usage context.” http://studiolab.io.tudelft.nl


139

technologies [12]. This setup has become an ultimate test field for verifying the applicability of various design methods for design and creation of architectural interactive systems. 2.1 Minor Setup, Brief and Design Process The minor has been a semester long activity, divided in two parts. In the first two months students have been engaged in various skill- and knowledge building assignments, while teachers have been closely monitoring the students, as well as exchanging knowledge and design approaches. Throughout the whole semester, five Delft teachers4 have been continuously involved in the project; their backgrounds included architecture, industrial design, computer science, interaction design, new media and digital music. In addition to the regular faculty, during the course many other specialists were invited as lecturers, workshop teachers and consultants5. The brief given to the students was “the lounge space”: a building or a part of a building where people interact casually and informally, listen to music, eat, drink and relax. In the first part of the semester students were engaged in a series of short design and prototype assignments where they could build up their skills and experiment on simple interactive objects for specific functional or experiential purposes. Gradually, the scale of their design was shifted from individual objects to networks of objects interacting with each other and with their users. In parallel to these assignments, students were also conceptually designing an interactive lounge environment, based on the ideas explored in their quick prototypes. The rationale behind this course organization was, on one hand to allow informed and playful exploration of a variety of concepts, while achieving situated learning of technological and technical abilities in a designerly context (i.e. not technology driven). The first half of the semester ended with the delivery by each of the three student groups of a general design for an interactive lounge installation. During the second half of the semester, the three groups were helped by the teaching staff to realize their design visions as small pavilion structures. As we have mentioned, students arrived to the minor with very diverse backgrounds. Among 21 students, one third were students of architecture, one third of industrial design and engineering and the remaining ones were studying at faculties including aerospace engineering, computer science, applied physics and mechanical engineering. This mix of experiences allowed (or even mandated) a process of creative rethinking of the design approach. One of the early issues was the lack of compatibility between very directed, waterfall type engineering and scientific development processes and more iterative, loop-oriented approaches typical of designers and 4

Main studio teachers were: Walter Aprile, Aadjan van der Helm, MarkDavid Hosale, Tomasz Jaskiewicz, Dieter Vandoren. 5 Responsible professors and guest critics of the course are: prof. Kas Oosterhuis and prof. David Keyson. Guest consultants and workshop teachers include: Ruairi Glynn(interactivearchitecture.org), Jerome Decock (lab[au]), Denis Oudendijk(Refunc), Jan Korbes(Refunc), Dan Overholt, Daan Roosegaarde(Studio Roosegaarde), Marco Verde, Kees Baardolf, Rob Luxen, guest lecturers include: prof. Tetsuo Tomiyama, Alisa Andrasek, Axel Killian, Philippe Rahm, Christian Derickx.

140


architects. Just like the groups had to work out their social structure and process, a common language of interactive architecture had to emerge from the process, to make collaboration possible. It was empirically proven very early in the process that the typical architectural design approach is not suitable for an interactive design assignment. Even though the function of the space that students had to create has been defined as a “lounge”, the interaction requirements made it impossible to design fixed pavilion forms that would allow for maintaining a continuous reciprocal interaction between users and the pavilion. As an alternative process, the students have proceeded by defining lists of user activities that their pavilions should accommodate. Following that, lists of specific spatial affordances were created. To help with achieving design coherence, each group defined a concept that would in an abstract way guide the experience that each installation was trying to provide for their users. These mottos, later on in the process put aside, were “parallel universe”, “artificial nature” and “immersive ecology”.

Fig. 1. Students engaged in building a “distributed ecology” of interacting robotic agents during workshop with Ruairi Glynn

A big change could be clearly observed throughout the skill-building workshops in the way students were thinking and talking about their projects. In the beginning of the course their descriptions and ideas about their design’s behaviour were rather naively approaching stated problems, even the most technologically demanding ones were not going beyond the reactive manner of operation that could be entirely described using a series of if... then... statements. Gradually the students have gotten to understand the systemic nature of their designs. An important trigger for this understanding was a three day workshop with Ruairi Glynn6, where the students in sub-groups had to design small, light-sensitive robotic “creatures” and build a “performative ecology” out of all robots combined together (see Figure 1). The horizons of the students became broader, and their thoughts more complex, after a group visit to the Ars Electronica exhibition. This provided a series of very important common references of classic interactive pieces and a useful snapshot of the state of the art. While of course a common background can be acquired also from a textbook, a visit to an event could prove capable of forming much stronger memories – not 6

Ruairi Glynn is a lecturer at the Bartlett School of Architecture and Central Saint Martin college in London, he runs the online blog interactivearchitecture.org.


141

to mention the fact that interactive pieces, obviously, are made to be interacted with, not so much read about. Gradually the descriptions of projects became more oriented towards explaining the experiences of users in their space and attempting to define those experiences through a multitude of localized interactions between users and the designed installation components, among the components and, what’s important, also among the users. In this way all designs have drifted towards becoming ecologies in which both users and installation components are actors. The step in which student projects have shifted from top-down and centralized ideas of interaction to the understanding of their projects as complex, adaptive ecologies has been in all three cases a noticeable breakthrough. The profoundness of the considered interactions and resulting formal complexity of projects has quickly increased and the projects presented at the midterm review were very satisfactory. 2.2 Case 1 - Odyssey The project of the group of the students7 who named themselves the “Odyssey” team, envisioned a setup where the space of the installation has a clearly differentiated outside and inside character, creating a surreal feeling of an immersive, continuous space on the inside and a crude, mechanical form on the outside of the pavilion (see Figure 2). The behaviour of the pavilion is organized by the concept of virtual waves emitted by people entering the space of the pavilion. These waves would be received by the kinetically actuated structure of the installation, the distributed ambient lighting elements and 40 small actuators embedded in the floor of the space. The centrally placed, large sitting cushion connected to the ceiling of the pavilion would act as a heart of the system, collecting signals from all dynamic installation components and based and translating their

Fig. 2. Design, building and user testing of the “Odyssey” group’s prototype 7

Students in the Odyssey group are: Govert Flint, Lieke Kraan, Bob Groeneveld, Jesse Timmermans, Merijn Pen, Thomas van Oekelen, Melisa Garza Vales.

142


behaviour into continuously generated sound, while occasionally creating an energy explosion and sending out its own wave. It became empirically verified upon the completion of the project that as envisioned, this behaviour leads to the perception of the space as intelligent and that through repeated interactions users can learn how to meaningfully engage in a dialogue with the Odyssey installation. 2.3 Case 2 – GEN The project of the second group8, called “GEN”, includes a static canopy structure, kinetically actuated vertical spatial separators that resemble leaves and a complex pneumatic floor “landscape” (see Figure 3). The group intended to create an installation that would display a non-repetitive, yet cyclical behaviour in which the space would constantly try to create conditions that it perceives as most favourable by its occupants. By tracking the faces of people inside and outside the installation area, the installation maintains a continuous record of occupant activity. Components of the installations that are in use (e.g. being sat on) are seen as successful and gain virtual “kudos”. In the process they can trade kudos among each other to overrule the behaviour of their neighbours. The kudo economy is a way to optimize the system in a computationally light and fairly robust way. The installation also included a small amount of random perturbation factors, to make sure that new behavioural patterns can spontaneously evolve from the process.

Fig. 3. Design, building and the final prototype of the “GEN” group

2.4 Case 3 – sCAPE The third project, for which the students9 chose the name “sCAPE” aims at creating a pavilion based on a fully distributed system. “LEDwork” computation and display nodes can be freely connected to each other to create a three dimensional spatial network that is supported by and integrated with the pavilion (see Figure 4). Each element has its own simple behaviour and can locally exchange information with other elements it is physically connected to. The basic elements are light-emitting ones, free to be moved around. All other parts of the installations are mutations of those. The 8

Students in the GEN group are: Altynay Imanbekova, Joris Hoogeboom, Ben van Wijk, Hanneke Hoogewerf, Tino de Bruijn, Patrick Pijnappel, Alexander de Mulder. 9 Students in the sCAPE group are: Iris van Loon, Marieke Dijksma, Teun Verkerk, Fons van der Berg, Tom Goijer, Cees-Willem Hofstede.


143

Fig. 4. Design and building of the “sCAPE” group’s prototype

structure of the pavilion is built out of struts and nodes, where each node is regarded as a LEDwork node fixed in place that can be connected to all mobile nodes. Other fixed nodes include additional lighting elements, sound emitting components and more. The intention is to create a dynamic ecosystem in which the visitor to the installation can submerge himself completely. The family mutation resemblance among the components of the system has been observed to stimulate the right behaviours in the users, such as rearranging the LEDwork nodes or attaching them to the pavilion walls for recharging (see Figure 5).

Fig. 5. Autonomous node prototypes of the “sCAPE” group’s design

The second part of the semester has been entirely devoted to building the installations in full-scale. It has been explicitly stated that projects upon which the designs were based were not to be seen as final blueprints, but to the contrary, they were supposed to be modified and evolve throughout the entire building and testing process.

144


The construction of interactive environments cannot be, currently, detached from the process of prototyping. The technology of interaction is developed and integrated at construction time: the traditional split between architecture, engineering and design cannot hold at this level of innovation. If the first quarter could be described as an iterative skill building period, the second quarter has been devoted to the iterative building of complete design prototypes. Students had to fit their installations in a tight budget, thus they had to experiment with various materials, many of which were recycled waste or leftovers from industrial production. Their designs had to be constantly adjusted, based on the structural and aesthetic properties of the used materials. For this, invaluable support was provided by quest teachers Denis Oudendijk and Jan Korbes from the Refunc (http://www.refunc.nl) group. The interactions have been tested and adjusted on the fly, based on ongoing experiments with embedding sensors, actuators and effectors into the built structures. To facilitate the prototyping process, the groups were further subdivided and each subgroup of two or three students that were made responsible for a different part of the installation. In this way, installation elements were prototyped individually and gradually integrated into a bigger system. The process of building the installations was thus interactive in itself. By interacting with the materials and component prototypes, students were adapting their designs and adjusting what was being built. In this way many creative solutions were found for difficult problems. All installations were finished in time, and are already partly operational while still being built.

3 Evaluation 3.1 Multi-agent Design Systems What can be observed in the developed installations is the logical application of distributed systems to interactive architectural applications. Each interactive installation component may be seen as a node in a complex network of interactions with other components and users. Such strategy allows developing installations iteratively and permits simultaneous working on their different parts. What’s more, distributed systems yield many advantages over centralised ones when it comes to resulting behaviours. An interactive installation without a centralized controlling core naturally becomes developed as an adaptive system, which in turn results in its nondeterministic operation. Applying such models to architecture means that buildings can be seen as distributed networks of dynamically interrelated building components. Such networks in their principles are easily extensible and adaptable. However, their most important feature is capability of exhibiting what is referred to as “swarm intelligence”. It is a holistic concept of intelligent behaviour that can emerge from systems which consist of a high number of simple interconnected agents. Those agents in case of buildings are active building components communicating with each other, with users of building spaces and with their environment. Complex adaptive system models applied to architectural design can lead to formulating a new methodology for creation of architectural processes. Such a new approach could lead to combining multiple current, yet still separate trends that are


145

currently emerging on the border of architecture and information technology and to creating architecture that is inherently dynamic and adaptive in the most efficient manner. 3.2 Computer Aided Interactive Architectural Design It has been observed among the students of the Interactive Environments Minor that despite general proficiency with the use of computer aided design (CAD) software; the use of this software did not have any explicit effect on the course of student design processes. It became clear that students would greatly benefit from an integrated software toolkit that would have allowed them to create a virtual model of their installations and test the designed interactions and behaviours, as well as structural performance of designed installations virtually on that model before its physical realisation. Looking at commonly used digital design tools in architecture, even though achieving interactive quality of architecture in itself is not a direct goal for most practicing architects, contemporary digital design tools to some extent already allow creation of building designs as dynamic systems. An example here can be a parametric building design, storing all design data in shared building information model (for example in Gehry Technologies Digital Project CAAD environment). As a result of employment of those new technological means, we can already witness a radical shift in architectural design methodology. Linear and centrally steered “top-down” designing becomes gradually replaced or supplemented by creation of parametric, procedural and relational digital design models. Consequently, those models become final design products themselves, making traditional, fixed plan drawings less important or even completely redundant. We can imagine that in the next future, mass production will be replaced by the mass customization of building elements, made possible with datadriven manufacturing technologies and building environment. In many cases, it will not make a difference whether a building consists of repetitive modular components, or if its every single component is a different custom part. Ultimately after being built buildings are often also equipped with sophisticated control systems that dynamically govern HVAC installations, lighting, security, accessibility, media systems, and many other aspects of building’s functionality. This shows that although they are hardly interactive, contemporary buildings are already commonly designed, constructed and used as dynamic systems. However the structure of these systems tends to repress any degree of indeterminacy, thus interactive qualities cannot emerge.

Fig. 6. Swarm Toolkit design platform prototype

146


The research work of one of us (Jaskiewicz) required the development of a design software application prototype under the working name of SwarmToolkit. The prototype includes a graphical interface and an open framework allowing virtual creation of autonomously operating elements and execution of their behaviour in real-time10. This solution allowing for integrated design and behavioural operation of a virtual model would seamlessly complement the workflow in projects similar to ones presented, allowing for a radical increase of component and installation design. Direct link to component fabrication can be established. What’s more, the virtual model can operate alongside the physical prototype in a real-time feedback loop, making it possible to fine tune interactions and instantly simulate proposed modifications. Controllers embedded in the physical installation may be early on in the prototyping process controlled by this software and ultimately they can be detached and operate autonomously, communicating only locally with each other11. 3.3 Future Prospects Interactive architecture as a domain is still in its infancy. Only continuous exploration of new methods, tools and techniques can ultimately lead to creation of full featured interactive buildings. Developing interactive installations in an iterative adaptive manner has proven successful and approaching interactive architecture as an adaptive multi-agent system appears as a promising strategy. In his manifesto Bruce Mau has written: "Make your own tools. Hybridize your tools in order to build unique things. Even simple tools that are your own can yield entirely new avenues of exploration. Remember, tools amplify our capacities, so even a small tool can make a big difference.” [13]. It becomes clear that architecture, as never before, is tied to digital tools that we develop to open up new possibilities for architectural design. Nevertheless, it’s not the digital tools that are the true essence of the new kind of “digital architecture”. It is architecture’s ability to read, process and communicate information that’s about to revolutionise the architectural profession.

References 1. Cook, D., Das, S.: Smart Environments: Technology, Protocols and Applications. WileyInterscience, Hoboken (2004) 2. Aldrich, F.K.: Smart Homes: Past, Present and Future. Inside the Smart Home, For Dummies (2003) 3. Sinopoli, J.: Smart Buildings (2006) 4. Ramos, C.: Ambient Intelligence – A State of the Art from Artificial Intelligence Perspective. In: Neves, J., Santos, M.F., Machado, J.M. (eds.) EPIA 2007. LNCS (LNAI), vol. 4874, pp. 285–295. Springer, Heidelberg (2007) 10

By real-time is meant an iterative process operating at more than 15 iterations per second, therefore perceived as continuous operation. 11 Arduino microcontrollers are used in the current setup. They can operate as devices slaved to a PC through the Firmata protocol, or they can be directly programmed to operate autonomously. Scripts written to control parts of the virtual model in Swarm Toolkit can be translated to Arduino code and seamlessly ported to the fully autonomous physical node. The solution is not microcontroller specific, other microcontrollers can be in further versions integrated in the system.


147

5. Briere, D., Hurley, P.: Smart Homes For Dummies (2007) 6. Guallart, V.: Media House Project: The House Is The Computer The Structure The Network. Actar (2005) 7. Laurel, B.: The Art of Human-Computer Interface Design. Addison-Wesley Professional, Reading (1990) 8. Lidwell, W., Holden, K., Butler, J.: Universal Principles of Design. Rockport Publishers (2003) 9. Sharp, D.H., Rogers, P.Y., Preece, D.J.: Interaction Design: Beyond Human-Computer Interaction. Wiley, Chichester (2007) 10. Buxton, B.: Sketching User Experiences: Getting the Design Right and the Right Design. Morgan Kaufmann, San Francisco (2007) 11. Moggridge, B.: Designing Interactions. The MIT Press, Cambridge (2007) 12. Kelley, T.: Prototyping is the shorthand of innovation 12, 35–42 (2001) 13. Mau, B.: An Incomplete Manifesto for Growth (1998)

Event Recognition via Energy Efficient Voting for Wireless Sensor Networks Mohamed Baqer Department of Computer Engineering, College of Information Technology, University of Bahrain, Kingdom of Bahrain [email protected]

Abstract. Physical world event monitoring and detection are two of the most dominating applications of wireless sensor networks. Combining sensor networks with artificial intelligence and pattern recognition is envisaged to deliver smart application to end-users. In this paper, a new distributed energy efficient pattern recognition approach is proposed to first depict events of interest as patterns and then detect the occurrence of events of interest, namely: the Voting with Random Node Selection (VRNS) approach. Sensor nodes may leverage their processing and storage capabilities by collectively handling sensory data to translate local sensory information of individual sensor nodes into meaningful results that depict the global status of the environment. In contrast to classical pattern recognition approaches, the VRNS approach operates within the constrained sensor network’s resources and infrastructures. Instead of utilising the entire sensor network, events of interest are recognised by employing an arbitrary sample of sensor nodes. The results show that the VRNS approach is capable of prolonging the longevity of the network’s lifetime, recognising event patterns effectively, and avoiding over-utilisation hotspots in the network. Keywords: Wireless sensor networks, energy conservation, lifetime, pattern recognition, voting, distributed detection.

1

Introduction

Wireless sensor network research is a discipline that prospered with the increase in users that need to understand their physical world and be alerted with changes in their surroundings. Sensor nodes are envisaged to bridge the physical world with the human-driven context of the Internet [1], [2], [3]. Battery operated sensor nodes are being deployed in the environment to ubiquitously convey sensory information [4]. Sensory applications are event-driven applications used for inferring the physical world’s status from sensory data [5], [6]. Sensor networks are envisaged to facilitate smart living by seamlessly providing a constant flow of information about the environment to end-users and machines to improve our lifestyle. Information from sensor networks will allow us to depict our physical S. Balandin et al. (Eds.): ruSMART/NEW2AN 2010, LNCS 6294, pp. 148–159, 2010. c Springer-Verlag Berlin Heidelberg 2010

Event Recognition via Energy Efficient Voting for Wireless Sensor Networks

149

world surroundings in highly sophisticated relationships and to optimise these relationships to attain efficient and effective performance. The processing and storage capabilities of sensor nodes facilitate the implementation of complex sensory applications and allow sensory data to be processed within the network close to their sources. Generally, replacing or recharging the sensor node’s battery is infeasible; hence the services that the sensor node provides are restricted by the depletion of the sensor node’s battery. A typical Mica2 sensor node has about 4Kbytes of RAM and 128Kbytes of flash memory and operates on two AA batteries [7]. Thereby, sensory applications need to be designed whilst considering the resource constraints of the sensor nodes. Two primary energy conservation models are employed in sensor networks, namely: information processing and sleep mode strategies. Communication is the highest energy consumer in sensor networks [8]. The processing capability of sensor nodes is utilised to process sensory data whilst being routed to its destination, and consequently reduce the amount of energy consumed in communication [9], [10]. Moreover, information processing approaches are utilised to produce global results beyond the capability of any individual sensor node. Innetwork processing and collaborative signal and information processing (CSIP) approaches are commonly used to establish node collaboration for processing data whilst en-route to the base-station [9], [10]. The design of sensor node hardware incorporates various energy modes: active, idle, and sleep modes [11], [7]. It is generally desirable to switch off sensor nodes by setting them to the sleep mode since non-negligible amounts of energy are consumed in the active and idle modes [12], [13]. In sensor networks a sensory event pattern could be a weather map, the behaviour data of sea birds, or stress levels in a bridge [14], [15]. Pattern recognition approaches may be employed for automatic extraction and discovery of valuable information from the constant flow of sensory data [16]. Furthermore, pattern recognition techniques can detect interesting phenomena in the environment by using spatio-temporal patterns that depict the global status of events of interest. Using sensory patterns instead of handling individual sensory data points provides a practical method for capturing multidimensional and complex events of the physical environment [17], [18]. Classical pattern recognition approaches, however, are designed for typical computers with ample processing, storage, and power supply, and are not designed for the ad hoc and dynamic nature of the sensory infrastructure. Therefore, it is necessary to examine the pattern recognition approaches beyond their classical algorithm efficiency to incorporate their demand on the resources of the sensory infrastructure.

2

Related Work

Pattern recognition and neural network techniques may detect phenomena of interest in the environment by using spatio-temporal patterns that encapsulate the features and attributes of events of interest. Pattern recognition approaches for sensor networks operate to detect large non-local events. Sensor nodes within

150

M. Baqer

a network collaborate amongst each other to translate their local sensory data into global patterns. Two neural networks for data processing using sensor networks were proposed by Kulakov et al. [19]. The authors argue that classical neural network algorithms are well suited for sensor networks for data dimensionality reduction. Adaptive Resonance Theory (ART) and FuzzyART are implemented using sensor nodes for digital signal and analogue signal processing, respectively. A complete ART neural network is implemented on a single sensor node to cluster data from various sensor modules. Input sensory data are categorised based on their similarities to the stored reference patterns. Moreover, the authors then propose topologies that support different distributed data fusion schemes for in-network data processing. In essence, this approach functions as an information fusion approach commonly used in sensor networks. Although the authors discuss neural networks as a distributed approach, the implementation of the ART neural network is limited to individual nodes. The approach uses three topologies to improve data reduction, however, it does not discuss the time overhead required to cluster the data using the various topologies, and the correctness of the sensory data clusters. The user may not be able to correctly capture the status of the physical environment due to information loss resulting from the dimensionality reduction of sensory data. Finally, the approach consumes additional processing power and time by processing all of the sensory data despite their correlation or redundancy. Oldewurtel et al. [20] employed Hopfield Network and self organising maps (SOM) to process sensory data within the network. First, Hopfield Network (HN) was implemented on a single node to make the most of the parallelism and low computation costs of neural networks to replace resource demanding traditional signal processing algorithms. The node uses HN to deal with erroneous data and loss of data to avoid wasting energy in retransmissions. Information is processed within the network by means of node collaboration using SOM and in-network processing. SOM is implemented in a distributed manner over the network to reduce the dimensionality of the sensory data and thus reduce the energy consumed for transmissions. Finally, an in-network data processing approach is proposed as an alternative to the SOM approach. The authors utilised in-network approaches for data reduction and filtering and employed several approaches to process sensory data. Despite the analogy presented in the aforementioned papers between sensor networks and neural networks, the implementation of neural networks on sensor nodes is not trivial. The authors have implemented off-the-shelf neural network algorithms on sensor networks without incorporating any energy conservation mechanisms. For instance, SOM and ART algorithms require a fully-connected network which may not be feasible in sensor networks and demands a significant amount of network communication. Consequently, these algorithms consume non-negligible amounts of energy and may reduce the lifetime of the sensor network drastically.


151

Previously we have proposed the Voting Graph Neuron, VGN, approach as an energy efficient distributed event recognition model for wireless sensor networks in [21]. Initial research with respect to designing pattern recognition approaches that can operate within the resource-constraint and energy limitations of sensor networks was proposed. Event patterns that describe events of interest were employed for event monitoring. The VGN approach proposed a model for node collaboration where sensor nodes may exchange their results to reach a global consensus that reflects the status of the environment. The VGN approach’s performance was affected by the similarity amongst patterns. Patterns of high similarity consumed more energy than other patterns. Moreover, the model involved all nodes in every pattern recognition cycle which reduced the lifetime of the network. Finally, the VGN approach suffered from performance hotspots that showed that some sensor nodes are over-utilised more than others. This paper contributes by proposing a pattern recognition approach that suits the resource-constrained infrastructure of sensor networks. First, sensory event patterns are proposed and defined for sensory applications. Classical pattern recognition approaches do not take into consideration the dynamic and ad hoc nature of sensor networks and were originally designed for computers with ample processing and storage capabilities. Pattern storage and recognition is attained in a distributed manner to balance the operations load evenly throughout the network. As a result, the energy consumption is reduced by reducing the amount of communicated sensory data and balancing the sensor node utilisation in the network.

3

The VRNS Approach

The VRNS approach is proposed to perform distributed in-network pattern recognition. Events of interest are depicted as reference patterns and stored in a distributed manner in the sensor network. The pattern recognition operations are preformed in a distributed manner where each sensor node produces votes - the result of matching the local sensory measurement with the locally stored sub-patterns. The global event is matched with the predefined pattern of interest that receives the largest number of votes. The following present the main components and operations of the VRNS approach. 3.1

Sensory Event Patterns

The VRNS approach assumes that a sensory event pattern is represented by a M × N matrix. Each cell of the matrix is associated with one sensor node and corresponds to a specific region of the monitored area. Moreover, the number of active sensor nodes is equal to the number of cells (M × N ). In scenarios where the number of deployed sensor nodes might be greater than M × N, cell leaders may be elected to represent the cells and aggregate the sensory data of sensor nodes belonging to their cell [9]. Consequently, a matrix of sensor nodes of size M × N is created to correspond to event patterns. The matrix, however, can be

152

M. Baqer

of regular or irregular shape depending on the deployment scheme. Therefore, an event pattern may be defined as an M × N snapshot matrix of sensory data at a particular time t in a specific area A monitored by M × N matrix of sensor nodes, as illustrated in Figure 1.

Fig. 1. Mapping the status of the real world using event patterns

3.2

Pattern Storage

A set of reference patterns are used as a training set to detect events in the environment. In classical approaches the natural choice will be to store the training patterns in a centralised manner in the base-station. In sensor networks, as a result of the limited storage capacity of individual sensor nodes, a single sensor node cannot store all reference patterns, and storing the reference patterns at the base-station is not desirable due to the problems associated with centralised designs in sensor networks [22], [10]. The VRNS approach overcomes this problem by partitioning the patterns into sub-patterns. A sub-pattern is the fundamental pattern element comprising sensory information of a particular location. Each sub-pattern represents the local physical world status and is assigned to a dedicated sensor node deployed in same geographical location. The implementation details of the memory and data structures used for storing the patterns are out of the scope of this research. Nevertheless, the patterns should be stored in a manner which optimises the space and time required to store and to retrieve them, respectively. 3.3

Pattern Recognition

Event patterns are recognised by a voting mechanism. Input event patterns are recognised and matched to the winning reference pattern in a distributed manner. Each sensor node produces a vote set comprising the labels of reference


153

patterns that matches the local sensory measurements. Votes sets from various nodes in the network are collected and aggregated into vote polls. An information aggregation approach that allows the fusion of data enroute should be utilised to improve the longevity of the network [23], [24]. Amongst m stored patterns, the winning reference pattern is the pattern from the stored training set of pattern label Li that received the highest number of votes and the received votes count exceeds a threshold value, as depicted in Formula 1. Li if maxm i=1 (P olli ) ≥ α · M M atch = (1) Lm+1 otherwise The threshold value α provides a guarantee that the winning reference pattern received sufficient votes from the sensor nodes of the network. For instance, setting α=0.5 or α=1.0 implements simple majority and unanimity voting, respectively. On the other hand, the input pattern is rejected by setting the winning vote to Lm+1 . Furthermore, input patterns may be rejected or mismatched when the correct pattern label does not receive the largest number of votes and the number of votes received by any label is less than the confidence threshold value. Input patterns are rejected when none of the vote polls have satisfied the confidence threshold. Formula 2 describes the states of the pattern recognition conditions of the VRNS approach in terms of the number of votes and the value of α, where Pollcorrect and Pollincorrect ≥ α · M. ⎧ P ollcorrect > P ollincorrect ⎨ match Result = mismatch P ollcorrect ≤ P ollincorrect (2) ⎩ reject otherwise 3.4

Node Selection

The node selection technique incorporated in the VRNS approach leverages the benefits of the sleep mode. The sensor nodes may be switched entirely or partially by switching the radio antenna to conserve energy. In either situation, significant amounts of energy can be conserved. The VRNS employs a subset of sensor nodes to recognise patterns. A sample of sensor nodes is selected from the network instead of involving the entire network in the voting and pattern recognition process. Thereby, the selected nodes will actively cooperate to recognise the input patterns whilst setting the rest of the network into the sleep mode. Additionally, the sensor nodes enters the sleep mode after communicating their vote sets. There have been many node selection techniques proposed for sensor nodes which may be integrated with the VRNS approach. The node selection technique, however, should be implemented in a distributed manner to avoid any dependency on any centralised entity. As a result, each sensor node should be able to autonomously determine whether to participate in the voting process or to enter the sleep mode. Moreover, the selection criteria should be highly arbitrary and

154

M. Baqer

nodes should be selected in a fair manner from the entire network to attain uniform energy consumption across the sensor network.

4

Results

In order to explore the efficiency of the VRNS approach, a number of simulations were conducted using MATLAB [25] and tested using patterns from the MNIST pattern library [26]. Patterns of the MNIST are patterns of digit patterns of numbers from 0 to 9. The simulation evaluated the VRNS approach with respect to pattern recognition performance and energy consumption. The advantage of employing performance patterns is to exploit them to visually discover locations of unbalanced energy consumption. The simulation results whilst using a training set of 512 patterns are presented in this paper. 4.1

Pattern Recognition Performance

An input pattern may be matched, mismatched, or rejected. In the following simulations, the true positive and true negative tests were used to explore the pattern recognition performance. The true positive experiments explored the ability of the VRNS approach to recognise unknown input patterns when they are selected from the training set. On the other hand, the true negative experiments simulate the ability of the VRNS approach to reject patterns. In the true negative experiments, the unknown input patterns were selected from patterns not included in the training set. Therefore, it is desirable to attain maximum true positive results while reducing the true negative outputs. True Positive. Figure 2 comprises 6 sub-graphs which depict having the input pattern affected by 0% to 50% variability (noise). Each sub-graph plots the successful recognition rate with the increase in the number of selected nodes, from 10% to 100%. The results show that the pattern recognition performance decreases with the increase in the amount of variability and more variability is tolerated whilst using small α values. In the figure, sub-graphs depicting low variability influence show better pattern recognition performance than sub-graphs with high variability influence. Moreover, the sub-graphs show that a variety of pattern recognition performances is attained whilst using various α values. The VRNS approach, whilst selecting all sensor nodes, α of 1.0, fails entirely when 10% or more of any input pattern is affected by variability, as shown in Figure 2-2. Other α values show better variability robustness. For instance, variability of 20% is shown in Figure 2-3 and when using an α value of 0.5, the pattern matching the VRNS approach attains pattern recognition performance above 0.6. When the variability is 30%, however, the approach whilst using α of 0.5 is entirely unable to recognise any patterns as shown in Figure 2-4. All α values fail to recognise any patterns when the variability is ≥ 50%.


155

Fig. 2. True positive for a sensor network storing 512 patterns while various α and values. Figures 1 to 6 depict the network true positive performance while having 0.0% till 50% variability incremented by 10%.

True Negative. In event monitoring and detection settings, not all events occurring in the environment are of interest to end-users. The VRNS approach is designed to reject patterns when the number of votes received by any reference pattern does not satisfy the confidence threshold of α. The simulated network utilised digit patterns of number 3 as a training set and used patterns of number 1 as input test patterns. The true negative simulations follow the simulation settings as the true positive settings. Figure 3 shows that the pattern recognition performance of 1.0 is attained when the VRNS approach utilises α value of 1.0, regardless of the amount of variability. The pattern recognition performance whilst using other α values improves with the increase in the amount of variability. For instance, utilising α value of 0.5, the pattern recognition starts improving when 20% variability is affecting the patterns. All α values except 0.1 attain pattern matching of 1.0 when the variability is ≥ 50%. The value of α of 0.1 is significantly small to reject any input pattern and hence fails to successfully recognise any patterns. Node Selection and Pattern Recognition Performance. The number of nodes selected to recognise events is an important factor influencing the longevity of the sensor network. Instead of using all nodes, as in the base-station model, the sample size specifies the number of nodes randomly selected to recognise the event patterns. The true negative and true positive simulation results of only using 20% of nodes are approximately equal to the results attained when all

156

M. Baqer

Fig. 3. True negative for a sensor network storing 512 patterns of number 3 and tested with patterns of number 1 while utilising various α and values. Figures 1 to 6 depict the network true positive performance while having 0.0% to 50% variability incremented by 10%.

nodes (100%) are used to recognise patterns, as shown in all Figures 2 and 3. The figures show that there is no improvement attained while using all sensor nodes instead of only 20% of the nodes. In addition to attaining a pattern recognition performance equal to utilising all nodes by using only 20% of the network, the network can significantly prolong the lifetime of the network by 5 times by switching off the rest of the sensor nodes. 4.2

Energy Consumption

The average node communication, ANC, is proposed to provide energy consumed in communication. In addition to studying the performance of the VRNS approach in respect to classical pattern recognition measurement, it is essential to study the energy consumption of the approach. The ANC measures the amount of energy consumed in communicating vote sets for recognising event patterns. ANC Simulations. Figure 4 shows the ANC energy distribution in the sensor network depicted in colour maps. In each of these figures, an increasing trend in the ANC value with the increase in the sample of selected sensor nodes can be observed, as seen in the colour maps 1 to 4 of each of the figures. The ANC value is increased with the increase in the sample size as the result in the increase in the probability of reselecting nodes to recognise patterns. When the sample


157

size is large enough to include all nodes of the network, all nodes have ANC = 1.0. This means that all nodes are chosen to communicate their vote sets for each event pattern. The figures show that the node selection technique of the VRNS approach provides fairness in the communication and processing loads amongst nodes. The majority of nodes have approximately equal ANC values proportional to the sample size. Sensor nodes are evenly selected and rarely over or under-utilised and the network does not suffer from hotspots.

(1)

(2)

(3)

(4)

Fig. 4. ANC of 32×32 nodes storing 512 patterns and employing a sample of sensor nodes of sizes 64, 128, 512, and 1024 nodes

5

Conclusion

The VRNS approach was proposed to provide energy efficient and decentralised pattern recognition for event monitoring applications using sensor nodes by employing in-network information processing and sleep-mode techniques. The pattern storage and pattern recognition operations are evenly performed over the network. Moreover, energy distribution maps are utilised to visually evaluate the energy consumption performance of sensor networks, namely the ANC metric. These patterns are beneficial to highlight any redundancy or hotspots in the sensor network. The ANC results show that the VRNS approach provides balanced node utilisation and thereby is able to prolong the sensor network’s lifetime. The effectiveness of the energy conservation is further improved by using a sub-set of randomly selected sensor nodes instead of involving the entire network. Consequently, the VRNS approach attained an effective pattern recognition performance whilst prolonging the lifetime of the network. Acknowledgments. The author would like to thank Dr. A. I. Khan and Prof. Bala Srinivasan from Monash University for their input and review of this work.

158

M. Baqer

References 1. Estrin, D., Culler, D., Pister, K., Sukhatme, G.: Connecting the physical world with pervasive networks. IEEE Pervasive Computing 1(1), 59–69 (2002) 2. Tolle, G., Polastre, J., Szewczyk, R., Culler, D., Turner, N., Tu, K., Burgess, S., Dawson, T., Buonadonna, P., Gay, D., Hong, W.: A macroscope in the redwoods. In: Proc. ACM Conference on Embedded Networked Sensor Systems (SenSys’05), San Diego, California, November 2005, pp. 51–63 (2005) 3. Baqer, M., Kamal, A.: S-sensors: Integrating physical world inputs with social networks using wireless sensor networks. In: Proceedings of the 5th International Conference on Intelligent Sensors, Sensor Networks and Information Processing (ISSNIP), pp. 213–218. IEEE, Los Alamitos (2009) 4. Akyildiz, I., Su, W.S., Sankarasubramaniam, Y., Cayirci, E.: A survey on sensor networks. IEEE Communications Magazine 19(2), 102–114 (2002) 5. R¨ omer, K., Mattern, F.: The design space of wireless sensor networks. IEEE Wireless Communications 11(6), 54–61 (2004) 6. Karl, H., Willig, A.: Protocol and Architecture for wireless sensor networks. Wiley, Chichester (June 2005) 7. Crossbow: Mica2 datasheet, http://www.xbow.com/Products/Product pdf files/Wireless pdf/ MICA2 Datasheet.pdf (last accessed June 21, 2009) 8. Raghunathan, V., Schurgers, C., Sung Park Srivastava, M.: Energy-aware wireless microsensor networks. IEEE Signal Processing Magazine 19(2), 40–50 (2002) 9. Krishnamachari, B., Estrin, D., Wicker, S.: The impact of data aggregation in wireless sensor networks. In: Proceedings of the 22nd International Conference on Distributed Computing Systems (ICDCSW ’02), pp. 575–578. IEEE Computer Society Press, Los Alamitos (2002) 10. Madden, S., Franklin, M.J., Hellerstein, J.M., Hong, W.: Tag: a tiny aggregation service for ad-hoc sensor networks. In: 5th Annual Symposium on Operating Systems Design and Implementation (OSDI), pp. 131–146 (2002) 11. Ye, W., Heidemann, J., Estrin, D.: An energy-efficient mac protocol for wireless sensor networks. In: Proceedings of the 21st International Annual Joint Conference of the IEEE Computer and Communications Societies (INFOCOM), vol. 3, pp. 1567–1576. IEEE, Los Alamitos (2002) 12. Cao, Q., Abdelzaher, T.F., He, T., Stankovic, J.A.: Towards optimal sleep scheduling in sensor networks for rare event detection. In: Proceedings of the Fourth International Conference on Information Processing in Sensor Networks (ACM/IEEE IPSN), April 2005, pp. 129–143. IEEE, Los Alamitos (2005) 13. Keshavarzian, A., Lee, H., Venkatraman, L.: Wakeup scheduling in wireless sensor networks. In: Proceedings of the seventh ACM international symposium on Mobile ad hoc networking and computing (MobiHoc ’06), pp. 322–333. ACM, New York (2006) 14. Mainwaring, A., Culler, D., Polastre, J., Szewczyk, R., Anderson, J.: Wireless sensor networks for habitat monitoring. In: WSNA ’02: Proceedings of the 1st ACM International Workshop on Wireless Sensor Networks and Applications, pp. 88–97. ACM Press, New York (2002) 15. Kim, S., Pakzad, S., Culler, D., Demmel, J., Fenves, G., Glaser, S., Turon, M.: Wireless sensor networks for structural health monitoring. In: SenSys ’06: Proceedings of the 4th International Conference on Embedded Networked Sensor Systems, pp. 427–428. ACM Press, New York (2006)


159

16. Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, Heidelberg (2006) 17. Xue, W., Luo, Q., Chen, L., Liu, Y.: Contour map matching for event detection in sensor networks. In: Proceedings of the 2006 ACM SIGMOD international conference on Management of data (SIGMOD ’06), pp. 145–156. ACM, New York (2006) 18. Li, M., Liu, Y., Chen, L.: Nonthreshold-based event detection for 3d environment monitoring in sensor networks. IEEE Educational Activities Department 20, 1699– 1711 (2008) 19. Kulakov, A., Davcev, D.: Distributed data processing in wireless sensor networks based on artificial neural-networks algorithms. In: In proceedings of the 10th IEEE Symposium on Computers and Communications (ISCC’05), pp. 353–358. IEEE, Los Alamitos (2005) 20. Oldewurtel, F., M¨ ah¨ onen, P.: Neural wireless sensor networks. In: Proceedings of the International Conference on Systems and Networks Communication (ICSNC ’06), pp. 28–36. IEEE, Los Alamitos (2006) 21. Baqer, M., Khan, A.I.: Energy-efficient pattern recognition approach for wireless sensor networks. In: Proceedings of the 3rd International Conference on Intelligent Sensors, Sensor Networks and Information (ISSNIP’07), pp. 509–514. IEEE Computer Society, Los Alamitos (2007) 22. Ye, W., Heidemann, J., Estrin, D.: An energy-efficient mac protocol for wireless sensor networks. IEEE 3, 1567–1576 (2002) 23. Madden, S., Franklin, M.J., Hellerstein, J.M., Hong, W.: Tag: a tiny aggregation service for ad-hoc sensor networks. In: 5th Annual Symposium on Operating Systems Design and Implementation (OSDI), pp. 131–146 (2002) 24. Boulis, A., Ganeriwal, S., Srivastava, B.: Aggregation in sensor networks: an energy - accuracy tradeoff. In: Proceedings of the International Workshop on Sensor Network Protocols and Applications, May 2003, pp. 128–138 (2003) 25. MATLAB: Matlab, http://www.mathworks.com/ (last accessed July 2009) 26. MNIST: Mnist database, http://yann.lecun.com/exdb/mnist/ (last accessed July 2009)

Evaluation of Multi-service CDMA Networks with Soft Blocking Villy B. Iversen Department of Photonic Engineering, Technical University of Denmark Building 343, 2800 Kongens Lyngby, Denmark [email protected]

Abstract. This paper presents a new approach for calculating the capacity of multi-service CDMA networks with soft blocking using general insensitive traffic models. The system is modeled as a multi-rate traffic flow model with state dependent blocking probabilities which have to fulfil specific requirements for maintaining reversibility of the model. By a new state-based algorithm performance measures are obtained for each individual service. An example for solving a realistic case is given. The granularity of the basic bandwidth unit can be chosen arbitrarily. Keywords: CDMA, soft blocking, state-based algorithm, granularity, BPP multi-rate traffic.

1

Introduction

Universal Mobile Telecommunication System (UMTS) networks, being new entrants on the market, are trying to prove their existence and secure a position in the future. One of the innovations UMTS brings is multi-band Wide-Band Code Division Multiple Access (WCDMA), which allows many users to transmit simultaneously within the same frequency band. Novelty in WCDMA compared with GSM networks systems is that single-user transmission in the system to all other users located around the air-interface is noise. Every time a new call is accepted, the signal-to-noise ratios for all other users are decreased. Because of that, WCDMA systems have no hard limits on call capacity. In order to preserve the quality of already accepted calls, a new call should not be accepted by the system if this call increases the noise for any other existing call above the tolerable level. Staehle & M¨ ader [11] have dealt with the same problem in a different approximative way. This paper presents a powerful and flexible model for evaluation of blocking probabilities of WCDMA system and related teletraffic parameters as traffic, call and time congestion. At the end, the paper gives an example of how to use the described model and algorithms. An earlier model was described in a conference paper [8]. In this paper we have modified the state dependent blocking probabilities and introduced a new numerical stable algorithm for BPP (Poisson - Binomial - Pascal) multi-rate traffic, which is a generalization of the classical recurrence Erlang-B formula, and a generalization of earlier algorithms S. Balandin et al. (Eds.): ruSMART/NEW2AN 2010, LNCS 6294, pp. 160–171, 2010. c Springer-Verlag Berlin Heidelberg 2010

Evaluation of Multi-service CDMA Networks with Soft Blocking

161

by Fortet and Grandjean (otherwise known as Kaufman and Roberts’ algorithm) and Delbrouck. This new algorithm includes global-state-dependent blocking probabilities. In section 2 we review the basic principles of calculating the capacity of WCDMA for services with different bandwidth and Quality-of-Service (QoS) requirements. In section 3 we introduce state dependent blocking probabilities as they appear in WCDMA. Section 4 presents a new compact recursive algorithm. New results on conditions for product-form and reversibility are presented, and the above algorithm is generalized to global-state-dependent blocking probabilities. A realistic large scale case is presented in section 5, and conclusions are given in section 6.

2

WCDMA Capacity

WCDMA networks are designed to provide multi-rate services with diverse Quality-of-Service (QoS) parameters as different speed and bit error rates. In the following we consider a single UMTS cell for capacity analysis. Influence from neighbor cells is modeled by the noise, which is expressed by neighbor cell interference ratio ι introduced below. The usual way to describe WCDMA capacity is through signal and noise power levels and their ratio. Such modeling comes very close to physical network layer and has been used by radio engineers ([5] [12]). However, for teletraffic calculations, it is preferable to convert all power levels and decibels into channels, offered traffic and blocking probabilities. Furthermore, only the up-link traffic direction (from mobile station towards the base station) will be considered. The down-link can be modeled in similar way. For that we need to express UMTS service definition by bandwidth resources needed from the particular traffic channel by the considered service. The relevant WCDMA cell capacity defining parameters are the following [6]: – WCDMA chip rate, W = 3.84 Mcps. – Noise-rise is defined as the ratio of the total received wide-band power to the noise power. It is functionally related to the cell up-link utilization factor ηUL ([6] formula 8.9: Noise rise = 1/(1 − ηUL )). The value of 3 dB noise-rise corresponds to ηUL = 50% utilization. – Neighbor cell interference ratio ι: cell interference ι = other own cell interference Interference ratio depends on cell environment, type of antenna used, and other factors. Common assumed values are 0.55 or 0.65. This reduces the cell capacity n by a factor 1/(1 + ι). In this paper we include other cell interference by state dependent blocking probabilities bx , discussed further below. Service classes in UMTS are described by following three parameters:

162

V.B. Iversen

– Service bit-rate Rj . – Signal energy per bit divided by noise spectral density level Eb /N0 that is required to provide predefined QoS. – Activity factor νj . For data services this equals 1, for voice 0.67 (50% voice activity plus overhead). Let N denote the total number of active calls (or users) in the cell for all services. The mathematical relation between above entities is ([6] formula 8.12): ηUL =

N j=1

1 1+

Eb N0

W Rj νj

.

(1)

j

Let us denote the service-related entities product by κj : Eb κj = · Rj · νj . N0 j

(2)

Then the maximum number of channels of service j (alone) in one cell is: W Nmax,j = ηUL 1 + . κj

(3)

Let us denote by Rs,j the spread bit rate of service j, which is the proportion of W utilized by one connection of service Rj (the chip rate of the connection): κ2j W 1 . (4) Rs,j = = κj − Nmax,j ηUL κj + W The formula above provides the technique to obtain unified measure unit for calculating traffic in multi-service and multi-QoS WCDMA systems. An example of calculations using above formulæ is shown in the Table 1. In the following we need to specify the total capacity and bandwidth requirement by a basic bandwidth unit (BBU) which we name a channel. The higher the required accuracy, the smaller is the unit we have to choose. As an approximation we may choose the unit-channel to be equal to 13.5 kcps, and Table 1. Capacity of a WCDMA cell. Assumptions for calculations for cell parameters: ηUL = 0.5 and W = 3840 kcps. Eb Eb j Service Rj [kbps] νj [dB] N κj Nmax,j Rs,j [kcps] N0 0 j

1 2 3 4

Voice Voice/moving Data Data

7.95 7.95 32 64

0.67 0.67 1 1

4 7 3 2

j

2.51 5.01 2.00 1.58

13.38 144.0 26.70 72.4 63.85 30.6 101.43 19.4

27 53 126 198


163

then approximate the bandwidth of the above services by 2, 4, 3, 5, 10, 16, 32 channels, respectively. The cell capacity is then 3840/13.5 = 284 channels. In principle, the performance of a system should be independent of the granularity chosen.

3

State-Dependent Blocking

The blocking probability of a call attempt should be independent of the granularity chosen if the Basic Bandwidth Unit is a integral multiple of the largest common divisor of the bandwidth demands. In practice a rough granularity may waste more bandwidth than a fine granularity because we will overbook the capacity needed. So if a call with a given granularity is a single-slot connection, then with the double (finer) granularity the same call becomes a two-slot connection. Denoting by bx,x+1 the blocking probability for a call arriving in state x and requiring one slot, then for these two cases the acceptance probability for a call attempt becomes (1 − bx,x+2 ) respectively (1 − bx,x+1 )(1 − bx+1,x+2 ) In CDMA, the capacity limiting factor is the power transmitted by user terminals and antennas. The total power is made up of contributions from own cell, other cells, and background noise. The contribution from other cells is typically around 35–40 % (with i = 0.55 – 0.65), and it is often assumed to be an independent Log-Normal distributed random variable. Therefore, the blocking probability of a new connection is a random variable depending on the state of the system (own cell, neighbor cells) and the bandwidth (noise-rise) of the new connection. We have to assume that the neighbor cell interference is varying slowly as compared with the mean holding times, so that we don’t risk that already accepted connections run out of capacity. Thus we assume that if we are in state x, then the neighbor cell interference at most occupy the bandwidth n − x. If the variations were fast, then we would not be able to guarantee any quality-ofservice. If our own cell is in state x then the remaining capacity is n − x and the bandwidth d is either available for the new connections or used by connections in neighboring cells. Let us denote the distribution function of the random variable describing the neighbor cell interference by F (x) (density function f (x). Conditioning on being in state x we know the inter-cell interference is within the interval [0, n − x]. If we request the bandwidth d, then a call attempt will be blocked if the neighbor cell interference is within [n − x − d, n − x]: n−x f (u) du (5) bd (x) = u=n−x−d F (n − x) F (n − x) − F (n − x − d) = (6) F (n − x) F (n − x − d) =1− . (7) F (n − x)

164

V.B. Iversen

In a similar way we get for a single-slot call in state x + d: bd (x + d) = 1 −

F (n − x − 2d) . F (n − x − d)

(8)

A call attempt requesting bandwidth 2d in state x experiences the blocking probability: F (n − x − 2d) b2d (x) = 1 − . (9) F (n − x) It is easy to se that this is the same as the blocking probability for two call attempts each requesting one bandwidth unit d: F (n − x − 2d) F (n − x) F (n − x − d) F (n − x − 2d) = · F (n − x) F (n − x − d) = {1 − bd (x)} · {1 − bd (x + d)}

1 − b2d (x) =

(10) (11) (12)

Thus a two-slot call attempt has the same blocking probability as two single-slot calls. The same results was obtained by a different approach for multi-rate traffic in [9].

4

Algorithms for Traffic Performance

We consider the BPP traffic paradigm. BPP traffic models include Binomial (finite source, Engset) traffic, Poisson arrival traffic, and Pascal traffic. Binomial traffic is more smooth than Poisson traffic. Pascal traffic is more bursty than Poisson and is equivalent to Poisson batch arrival with geometric batch size distribution. The state of the system x is defined as the number of busy channels. The arrival rate of service j (j = 1, 2, . . . , N ) is supposed to depend on the number of connections of this type. So the arrival rate of service j is λj (x) when we have x channels occupied by this type. The bandwidth demand of service j is dj , so that each connection requires dj channels (= Basic Bandwidth Units = BBU). The holding time of a connection of type j is exponentially distributed with rate dj μj (the factor dj is chosen only for convenience, the connection terminates all channels at the same time). In the mode considered (reversible scheduling), this model will be insensitive to the service time distribution, so that results obtained for exponential distributions are valid for more general distributions. For service j, the offered traffic in channels is Aj = λj /μj , and each connection needs di channels. Finite source traffic is characterized by number of sources Sj and offered traffic per idle source βj . Alternatively, we in practice often use the offered traffic Aj and the peakedness Zj . We have the following relations between the two representations: Aj = Sj ·

βj , 1 + βj

Zj =

1 , 1 + βj

(13)


βj =

γj 1 − Zj = , dj μj Zj

Sj =

Aj . 1 − Zj

165

(14)

dj is the data rate required by service j, and bj (x) is the blocking probability in state x for a call of type j requesting dj channels (soft blocking), which characterizes the current other-cell interference level. We have: ⎧ 0 x

Smart Spaces and Next Generation Wired Wireless Networking: Third Conference on Smart Spaces, ruSMART 2010, and 10th International Conference, NEW2AN ... Networks and Telecommunications)

Smart Spaces and Next Generation Wired Wireless Networking: 9th International Conference, NEW2AN 2009 and Second Conference on Smart Spaces, ruSMART ... Networks and Telecommunications)

Smart Spaces and Next Generation Wired Wireless Networking - New2an 2011, Rusmart 2011

Wireless Advanced Networking: 8th International Conference, NEW2AN 2008 and 1st Russian Conference on Smart Spaces, ... Networks and Telecommunications)

Smart Wireless Sensor Networks

Smart Wireless Sensor Networks

NETWORKING 2008 Ad Hoc and Sensor Networks, Wireless Networks, Next Generation Internet: 7th International IFIP-TC6 Networking Conference Singapore, May

Advanced Wired and Wireless Networking

Third-generation systems and intelligent wireless networking

Next Generation Wireless Systems and Networks

Smart Sensing and Context Third European Conference EuroSSC

Wired-Wireless Multimedia Networks and Services Management: 12th IFIP IEEE International Conference on Management of Multimedia and Mobile Networks and ... Networks and Telecommunications)

Wired-Wireless Multimedia Networks and Services Management: 12th IFIP IEEE International Conference on Management of Multimedia and Mobile Networks ... Networks and Telecommunications)

Function spaces: The fifth conference

Function spaces, the second conference

Hilbert Spaces and Operators on Hilbert Spaces

Cases on Telecommunications And Networking

NETWORKING 2007, Ad Hoc and Sensor Networks, Wireless Networks, Next Generation Internet, 6 conf

Orlicz Spaces and Modular Spaces

Smart Antenna Systems and Wireless LANs

Wireless Advanced Networking: 7th International Conference, NEW2AN 2007, St. Petersburg, Russia, September

IP-Based Next-Generation Wireless Networks: Systems, Architectures, and Protocols

IP-Based Next-Generation Wireless Networks: Systems, Architectures, and Protocols

Wireless Advanced Networking: 6th International Conference, NEW2AN 2006, St. Petersburg, Russia, May 29-June 2,

10th European Conference on Mixing

Smart Business Networks

Mechanisms for Autonomous Management of Networks and Services: 4th International Conference on Autonomous Infrastructure, Management, and Security, ... Networks and Telecommunications)

Smart Phone and Next Generation Mobile Computing (Morgan Kaufmann Series in Networking)

Next Generation Wireless Applications

Wired Wireless Internet Communications: 8th International Conference, WWIC 2010, Lulea, Sweden, June 1-3, 2010. Proceedings (Lecture Notes in Computer ... Networks and Telecommunications)

Smart Spaces and Next Generation Wired Wireless Networking: Third Conference on Smart Spaces, ruSMART 2010, and 10th International Conference, NEW2AN ... Networks and Telecommunications)

Smart Spaces and Next Generation Wired Wireless Networking: 9th International Conference, NEW2AN 2009 and Second Conference on Smart Spaces, ruSMART ... Networks and Telecommunications)

Smart Spaces and Next Generation Wired Wireless Networking - New2an 2011, Rusmart 2011

Wireless Advanced Networking: 8th International Conference, NEW2AN 2008 and 1st Russian Conference on Smart Spaces, ... Networks and Telecommunications)

Smart Wireless Sensor Networks

Smart Wireless Sensor Networks

NETWORKING 2008 Ad Hoc and Sensor Networks, Wireless Networks, Next Generation Internet: 7th International IFIP-TC6 Networking Conference Singapore, May

Advanced Wired and Wireless Networking

Third-generation systems and intelligent wireless networking

Next Generation Wireless Systems and Networks

Smart Sensing and Context Third European Conference EuroSSC

Wired-Wireless Multimedia Networks and Services Management: 12th IFIP IEEE International Conference on Management of Multimedia and Mobile Networks and ... Networks and Telecommunications)

Wired-Wireless Multimedia Networks and Services Management: 12th IFIP IEEE International Conference on Management of Multimedia and Mobile Networks ... Networks and Telecommunications)

Function spaces: The fifth conference

Function spaces, the second conference

Hilbert Spaces and Operators on Hilbert Spaces

Cases on Telecommunications And Networking

NETWORKING 2007, Ad Hoc and Sensor Networks, Wireless Networks, Next Generation Internet, 6 conf

Orlicz Spaces and Modular Spaces

Smart Antenna Systems and Wireless LANs

Wireless Advanced Networking: 7th International Conference, NEW2AN 2007, St. Petersburg, Russia, September

IP-Based Next-Generation Wireless Networks: Systems, Architectures, and Protocols

IP-Based Next-Generation Wireless Networks: Systems, Architectures, and Protocols

Wireless Advanced Networking: 6th International Conference, NEW2AN 2006, St. Petersburg, Russia, May 29-June 2,

10th European Conference on Mixing

Smart Business Networks

Mechanisms for Autonomous Management of Networks and Services: 4th International Conference on Autonomous Infrastructure, Management, and Security, ... Networks and Telecommunications)

Smart Phone and Next Generation Mobile Computing (Morgan Kaufmann Series in Networking)

Next Generation Wireless Applications

Wired Wireless Internet Communications: 8th International Conference, WWIC 2010, Lulea, Sweden, June 1-3, 2010. Proceedings (Lecture Notes in Computer ... Networks and Telecommunications)

Recommend Documents