Web Information Systems and Technologies: 5th International Conference, WEBIST 2009, Lisbon, Portugal, March 23-26, 2009, Revised Selected Papers (Lecture Notes in Business Information Processing)

Lecture Notes in Business Information Processing Series Editors Wil van der Aalst Eindhoven Technical University, The N...

Author: José Cordeiro | Joaquim Filipe

160 downloads 947 Views 5MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

Lecture Notes in Business Information Processing Series Editors Wil van der Aalst Eindhoven Technical University, The Netherlands John Mylopoulos University of Trento, Italy Norman M. Sadeh Carnegie Mellon University, Pittsburgh, PA, USA Michael J. Shaw University of Illinois, Urbana-Champaign, IL, USA Clemens Szyperski Microsoft Research, Redmond, WA, USA

45

José Cordeiro Joaquim Filipe (Eds.)

Web Information Systems and Technologies 5th International Conference, WEBIST 2009 Lisbon, Portugal, March 23-26, 2009 Revised Selected Papers

13

Volume Editors José Cordeiro Joaquim Filipe Department of Systems and Informatics Polytechnic Institute of Setúbal Rua do Vale de Chaves, Estefanilha, 2910-761 Setúbal, Portugal E-mail: {jcordeir,j.filipe}@est.ips.pt

Library of Congress Control Number: 2010924048 ACM Computing Classification (1998): H.3.5, J.1, K.4.4, I.2 ISSN ISBN-10 ISBN-13

1865-1348 3-642-12435-6 Springer Berlin Heidelberg New York 978-3-642-12435-8 Springer Berlin Heidelberg New York

This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. springer.com © Springer-Verlag Berlin Heidelberg 2010 Printed in Germany Typesetting: Camera-ready by author, data conversion by Scientific Publishing Services, Chennai, India Printed on acid-free paper SPIN: 06/3180 543210

Preface

This book contains a selection of the best papers from WEBIST 2009 (the 5th International Conference on Web Information Systems and Technologies), held in Lisbon, Portugal, in 2009, organized by the Institute for Systems and Technologies of Information, Control and Communication (INSTICC), in collaboration with ACM SIGMIS and co-sponsored by the Workflow Management Coalition (WFMC). The purpose of the WEBIST series of conferences is to bring together researchers, engineers and practitioners interested in the technological advances and business applications of Web-based information systems. The conference has four main tracks, covering different aspects of Web information systems, including Internet Technology, Web Interfaces and Applications, Society, e-Communities, e-Business and e-Government. WEBIST 2009 received 203 paper submissions from 47 countries on all continents. A double-blind review process was enforced, with the help of more than 150 experts from the International Program Committee; each of them specialized in one of the main conference topic areas. After reviewing, 28 papers were selected to be published and presented as full papers and 44 additional papers, describing work-inprogress, published and presented as short papers. Furthermore, 35 papers were presented as posters. The full-paper acceptance ratio was 13%, and the total oral paper acceptance ratio was 36%. Therefore, we hope that you find the papers included in this book interesting, and we trust they may represent a helpful reference for all those who need to address any of the research areas mentioned above.

January 2010

José Cordeiro Joaquim Filipe

Organization

Conference Chair Joaquim Filipe

Polytechnic Institute of Setúbal/INSTICC, Portugal

Program Co-chairs José Cordeiro

Polytechnic Institute of Setúbal/INSTICC, Portugal

Organizing Committee Sérgio Brissos Marina Carvalho Helder Coelhas Vera Coelho Andreia Costa Bruno Encarnação Bárbara Lima Raquel Martins Elton Mendes Carla Mota Vitor Pedrosa Vera Rosário José Varela Pedro Varela

INSTICC, Portugal INSTICC, Portugal INSTICC, Portugal INSTICC, Portugal INSTICC, Portugal INSTICC, Portugal INSTICC, Portugal INSTICC, Portugal INSTICC, Portugal INSTICC, Portugal INSTICC, Portugal INSTICC, Portugal INSTICC, Portugal INSTICC, Portugal

Program Committee Christof van Nimwegen, Belgium Ajith Abraham, USA Isaac Agudo, Spain Abdullah Alghamdi, Saudi Arabia Rachid Anane, UK Margherita Antona, Greece Matteo Baldoni, Italy Cristina Baroglio, Italy David Bell, UK Orlando Belo, Portugal Ch. Bouras, Greece Stéphane Bressan, Singapore Tobias Buerger, Austria Maria Claudia Buzzi, Italy

Elena Calude, New Zealand Nunzio Casalino, Italy Sergio de Cesare, UK Maiga Chang, Canada Shiping Chen, Australia Dickson Chiu, China Isabelle Comyn-wattiau, France Michel Crampes, France Daniel Cunliffe, UK Alfredo Cuzzocrea, Italy Steven Demurjian, USA Y. Ding, USA Schahram Dustdar, Austria Barry Eaglestone, UK

VIII

Organization

Atilla Elci, Turkey Vadim Ermolayev, Ukraine Josep-lluis Ferrer-gomila, Spain Filomena Ferrucci, Italy Giovanni Fulantelli, Italy Erich Gams, Austria Dragan Gasevic, Canada Nicola Gessa, Italy José Antonio Gil, Spain Karl Goeschka, Austria Stefanos Gritzalis, Greece Vic Grout, UK Francesco Guerra, Italy Aaron Gulliver, Canada Abdelkader Hameurlain, France Ioannis Hatzilygeroudis, Greece Stylianos Hatzipanagos, UK Dominic Heutelbeck, Germany Pascal Hitzler, Germany Wen Shyong Hsieh, Taiwan Christian Huemer, Austria Alexander Ivannikov, Russian Federation Kai Jakobs, Germany Ivan Jelinek, Czech Republic Qun Jin, Japan Carlos Juiz, Spain Michail Kalogiannakis, Greece Jaakko Kangasharju, Finland George Karabatis, USA Frank Kargl, Germany Roland Kaschek, New Zealand Sokratis Katsikas, Greece Ralf Klamma, Germany Agnes Koschmider, Germany Tsvi Kuflik, Israel Daniel Lemire, Canada Tayeb Lemlouma, France Kin Li, Canada Claudia Linnhoff-Popien, Germany Pascal Lorenz, France Vicente Luque-Centeno, Spain Cristiano Maciel, Brazil Michael Mackay, UK Anna Maddalena, Italy George Magoulas, UK

Ingo Melzer, Germany Panagiotis Metaxas, USA Debajyoti Mukhopadhyay, India Ethan Munson, USA Andreas Ninck, Switzerland Alex Norta, Finland Dusica Novakovic, UK Andrea Omicini, Italy Kok-leong Ong, Australia Jose A. Onieva, Spain Jun Pang, Luxembourg Laura Papaleo, Italy Eric Pardede, Australia Kalpdrum Passi, Canada Viviana Patti, Italy Günther Pernul, Germany Josef Pieprzyk, Australia Luís Ferreira Pires, The Netherlands Thomas Risse, Germany Danguole Rutkauskiene, Lithuania Maytham Safar, Kuwait Alexander Schatten, Austria Jochen Seitz, Germany Tony Shan, USA Quan Z. Sheng, Australia Keng Siau, USA Miguel Angel Sicilia, Spain Marianna Sigala, Greece Pedro Soto-Acosta, Spain J. Michael Spector, USA Martin Sperka, Slovak Republic Eleni Stroulia, Canada Hussein Suleman, South Africa Junichi Suzuki, USA Ramayah T., Malaysia Taro Tezuka, Japan Dirk Thissen, Germany Arun Kumar Tripathi, Germany Th. Tsiatsos, Greece Michail Vaitis, Greece Juan D. Velasquez, Chile Maria Esther Vidal, Venezuela Viacheslav Wolfengagen, Russian Federation Lu Yan, UK

Organization

Auxiliary Reviewers Michelle Annett, Canada David Chodos, Canada Zafer Erenel, Turkey Nils Glombitza, Germany Mehdi Khouja, Spain Xitong Li, China Antonio Molina Marco, Spain Sergio Di Martino, Italy Bruce Matichuk, Canada

Christine Mueller, New Zealand Eni Mustafaraj, USA Parisa Naeimi, Canada Stephan Pöhlsen, Germany Axel Wegener, Germany Christian Werner, Germany Jian Yu, Australia Donglai Zhang, Australia

Invited Speakers Peter A. Bruck Dieter A. Fensel Ethan Munson Mats Daniels

World Summit Award, Austria University Innsbruck, Austria University of Wisconsin – Milwaukee, USA Uppsala University, Sweden

IX

Table of Contents

Part I: Internet Technology Collaboration and Human Factor as Drivers for Reputation System Effectiveness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Guido Boella and Marco Remondino

3

Agent-Oriented Programming for Client-Side Concurrent Web 2.0 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mattia Minotti, Giulio Piancastelli, and Alessandro Ricci

17

The SHIP: A SIP to HTTP Interaction Protocol: Advanced Thin-Client Architecture for IMS Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Joachim Zeiß, Rene Gabner, Sandford Bessler, and Marco Happenhofer Efficient Authorization of Rich Presence Using Secure and Composed Web Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Li Li and Wu Chou

30

44

Part II: Web Interfaces and Applications Information Supply of Related Papers from the Web for Scholarly e-Community . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Muhammad Tanvir Afzal When Playing Meets Learning: Methodological Framework for Designing Educational Games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Stephanie B. Linek, Daniel Schwarz, Matthias Bopp, and Dietrich Albert SiteGuide: A Tool for Web Site Authoring Support . . . . . . . . . . . . . . . . . . . Vera Hollink, Viktor de Boer, and Maarten van Someren

61

73

86

ArhiNet – A Knowledge-Based System for Creating, Processing and Retrieving Archival eContent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ioan Salomie, Mihaela Dinsoreanu, Cristina Pop, and Sorin Suciu

99

Optimizing Search and Ranking in Folksonomy Systems by Exploiting Context Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Fabian Abel, Nicola Henze, and Daniel Krause

113

XII

Table of Contents

Adaptation of the Domain Ontology for Different User Profiles: Application to Conformity Checking in Construction . . . . . . . . . . . . . . . . . Anastasiya Yurchyshyna, Catherine Faron-Zucker, Nhan Le Thanh, and Alain Zarli The RDF Protune Policy Editor: Enabling Users to Protect Data in the Semantic Web . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Fabian Abel, Juri Luca De Coi, Nicola Henze, Arne Wolf Koesling, Daniel Krause, and Daniel Olmedilla

128

142

An Unsupervised Rule-Based Method to Populate Ontologies from Text . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Eduardo Motta, Sean Siqueira, and Alexandre Andreatta

157

Web Spam, Social Propaganda and the Evolution of Search Engine Rankings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Panagiotis Takis Metaxas

170

Part III: Society, e-Business and e-Government Making the Invisible Visible: Design Guidelines for Supporting Social Awareness in Distributed Collaboration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Monique Janneck

185

Interaction Promotes Collaboration and Learning: Video Analysis of Algorithm Visualization Use during Collaborative Learning . . . . . . . . . . . . Mikko-Jussi Laakso, Niko Myller, and Ari Korhonen

198

Modelling the B2C Marketplace: Evaluation of a Reputation Metric for e-Commerce . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Anna Gutowska and Andrew Sloane

212

Part IV: Web Intelligence Using Scientific Publications to Identify People with Similar Interests . . . Stanley Loh, Fabiana Lorenzi, Roger Granada, Daniel Lichtnow, Leandro Krug Wives, and José Palazzo Moreira de Oliveira

229

Website-Level Data Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jianqiang Li and Yu Zhao

242

Anti-folksonomical Recommender System for Social Bookmarking Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Akira Sasaki, Takamichi Miyata, Yasuhiro Inazumi, Aki Kobayashi, and Yoshinori Sakai Classifying Structured Web Sources Using Support Vector Machine and Aggressive Feature Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hieu Quang Le and Stefan Conrad

256

270

Table of Contents

Scalable Faceted Ranking in Tagging Systems . . . . . . . . . . . . . . . . . . . . . . . José I. Orlicki, J. Ignacio Alvarez-Hamelin, and Pablo I. Fierens

XIII

283

Answering Definition Questions: Dealing with Data Sparseness in Lexicalised Dependency Trees-Based Language Models . . . . . . . . . . . . . . . . Alejandro Figueroa and John Atkinson

297

Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

311

Part I Internet Technology

Collaboration and Human Factor as Drivers for Reputation System Effectiveness Guido Boella and Marco Remondino University of Turin, Department of Computer Science, Cso Svizzera 185, Turin, Italy {guido,remond}di.unito.it

Abstract. Reputation management is about evaluating an agent's actions and other agents' opinions about those actions, reporting on those actions and opinions, and reacting to that report thus creating a feedback loop. This social mechanism has been successfully used, through Reputation Management Systems (RMSs) to classify agents within normative systems. Most RMSs rely on the feedbacks given by the member of the social network in which the RMS itself operates. In this way, the reputation index can be seen as an endogenous and self produced indicator, created by the users for the users' benefit. This implies that users’ participation and collaboration is a key factor for the effectiveness a RMS. In this work the above factor is explored by means of an agent based simulation, and is tested on a P2P network for file sharing. Keywords: Reputation system, Agent based simulation, Peer to peer network.

1 Introduction In everyday's life, when a choice subject to limited resources (like for instance money, time, and so on) must be done, due to the overwhelming number of possibilities that people have to choose from, something is needed to help them make choices. People often follow the advice of others when it comes to which products to by, which movies to watch, which music to listen, which websites to visit, and so on. This is a social attitude that uses others’ experience They base their judgments of whether or not to follow this advice partially upon the other person's reputation in helping to find reliable and useful information, even with all the noise. Using and building upon early collaboration filtering techniques, reputation management software gather ratings for people, companies, and information sources. Since this is a distributed way of computing reputation, it is implicitly founded on two main assumptions: 1 - the correctness of shared information and 2 - the participation of users to the system. While the negation of the first could be considered as an attack to the system itself, performed by users trying to crash it, and its occurrence is quite rare, the second factor is often underestimated, when designing a collaborative RMS. Users without a vision of the macro level often use the system, but simply forget to collaborate, since this seems to cause a waste of time. The purpose of the present work is to give a qualitative and, when possible, quantitative evaluation of the collaborative factor in RMSs, by means of an empirical J. Cordeiro and J. Filipe (Eds.): WEBIST 2009, LNBIP 45, pp. 3–16, 2010. © Springer-Verlag Berlin Heidelberg 2010

4

G. Boella and M. Remondino

analysis conducted via an agent based simulation. Thus, the main research question is: what’s the effectiveness of a RMS, when changing the collaboration rate coming from the involved users? In order to answer this question, in the paper an agent based model is introduced, representing a peer-to-peer (P2P) network for file sharing. A basic RMS is applied to the system, in order to help users to choose the best peers to download from. In fact, some of the peers are malicious, and they try to exploit the way in which the P2P system rewards users for sharing files, by uploading inauthentic resources when they do not own the real ones. The model is described in detail and the results are evaluated through a multi-run ceteris paribus technique, in which only one setting is changed at a time. In particular, the most important parameters which will be compared, to evaluate the effectiveness of the RMS are: verification of the files, performed by the users and negative payoff, given in case a resource is reported as being inauthentic. The verification of the files, i.e. users’ the collaboration, is an exogenous factor for the RMS, while the negative payoff is an endogenous and thus directly controllable factor, from the point of view of a RMS’s designer. The P2P framework has been chosen since there are many works focusing on the reputation as a system to overcome the issue of inauthentic files, but, when evaluating the effectiveness of the system, the authors [1] usually refer to idealized situations, in which users always verify the files for authenticity, as soon as they start a download. This is obviously not the case in the real world: first of all, most resources require to be at least partially owned, in order to be checked. Besides, some users could simply decide not to check them for long time. Even worse, other users could simply forget about a downloaded resource and never check it. Last but not least, other users might verify it, but simply not report anything, if it’s not authentic.

2 Reputation and P2P Systems Since uploading bandwidth is a limited resource and the download priority queues are based on a uploading-credit system to reward the most collaborative peers on the network, some malicious users create inauthentic files, just to have something to share, thus obtaining credits, without being penalized for their behavior. To balance this, RMSs have been introduced, which dynamically assign to the users a reputation value, considered in the decision to download files from them or not. RMSs are proven, via simulation, to make P2P networks safe from attacks by malicious peers, even when forming coalitions. In networks of millions of peers attacks are less frequent, but users still have a benefit from sharing inauthentic files. It’s not clear if RMSs can be effective against this selfish widespread misbehavior, since they make several ideal assumptions about the behavior of peers who have to verify files to discover inauthentic ones. This operation is assumed to be automatic and with no costs. Moreover, since the files are usually shared before downloading is completed, peers downloading inauthentic files unwillingly spread them if they are not cooperative enough to verify their download as soon as possible. In the present work, the creation and spreading of inauthentic files is not considered as an attack, but as a way in which some agents try to raise their credits, while not possessing the real resource that's being searched by others. A basic RMSs is introduced, acting as a

Collaboration and Human Factor as Drivers for Reputation System Effectiveness

5

positive or negative reward for the users and human factor behind the RMSs is considered, in the form of costs and benefits of verifying files. Most approaches, most notably EigenTrust [2], assume that verification is made automatically upon the start of download of the file. By looking as we do at the collaboration factor in dealing with RMSs, we can question their real applicability, an issue which remains unanswered in the simulation based tests made by the authors. To provide an answer to this question it is necessary to build a simulation tool which aims at a more accurate modeling of the users’ behavior rather than at modeling the reputation system in detail.

3 Model Framework We assume a simple idealized model of reputation, since the objective is not to prove the effectiveness of a particular algorithm but to study the effect of users’ behavior on a reputation system. We use a centralized system which assumes the correctness of information provided by users, e.g., it is not possible to give an evaluation of a user with whom there was no interaction. When verifying a file, the agents give a negative payoff to the agent uploading it, in case it’s inauthentic. In turn, the system will spread it to the agents (if any) who uploaded it to the sender. There are two reputation thresholds: the first and higher one, under which it’s impossible to ask for resources to other agents, the second, lower than the other, which makes it impossible even to share the owned files. This guarantees that an agents that falls under the first one (because she shared too many inauthentic files), can still regain credits by sharing authentic ones and come back over the first threshold. On the contrary, if she continues sharing inauthentic files, she will fall also under the second threshold, being de facto excluded from the network, still being a working link from and to other agents. The agents are randomly connected on a graph and feature the following parameters: Unique ID, Reputation value, set of neighbors, set of owned resources, set of goals (resources), set of resources being downloaded, set of suppliers (by resource). At each time step, agents reply to requests for download, perform requests (according to their goals) or verify files. While an upload is performed – if possible each time another agent makes a request, requesting a resource and verification are performed in alternative. Verification ratio is a parameter for the simulation and acts stochastically on agents’ behavior. All agents belong to two disjoint classes: malicious agents and loyal ones. They have different behaviors concerning uploading, while feature the same behavior about downloading and verification: malicious agents are simply agents who exploit for selfishness the weaknesses of the system, by always uploading inauthentic files if they don’t own the authentic ones. Loyal agents, on the contrary, only upload a resource if they own it. A number of resources are introduced in the system at the beginning of the simulation, representing both the owned objects and the agents' goals. For coherence, an owned resource can't be a goal, for the same agent. The distribution of the resource is stochastic. During the simulation, other resources (and corresponding goals) are stochastically distributed among the agents. Each agent (metaphorically, the P2P client) keeps track of the providers, and this information is preserved also after the download is finished.

6


In order to test the limits and effectiveness of a reputation mechanism under different user behaviors an agent based simulation of a P2P network is used as methodology, employing reactive agents to model the users; these have a deterministic behavior based on the class they belong to (malicious or loyal) and a stochastic idealized behavior about verifying policy. Their use shows how the system works at an aggregate level. However, reactive agents can also be regarded as a limit for our approach, since real users have a flexible behavior and adapt themselves to what they observe. We built a model which is less idealized about the verifying factor, but it’s still rigid when considering the agents’ behavior about sending out inauthentic files. That’s why we envision the necessity to employ cognitive agents based on reinforcement learning techniques. Though, reactive agents can also be a key point, in the sense that they allow the results to be easily readable and comparable among them, while the use of cognitive agents would have moved the focus from the evaluation of collaborative factor to that of real users’ behavior when facing a RMS, which is very interesting, but beyond the purpose of the present work. In future works, this paradigm for agents will be considered.

4 Model Specifications and Parameters The P2P network is modeled as an undirected and non-reflexive graph. Each node is an agent, representing a P2P user. Agents are reactive: their behavior is thus determined a priori, and the strategies are the result of the stimuli coming from the environment and of the condition-action rules. Their behavior is illustrated in next section. Formally the multi agent system is defined as MAS = , with Ag set of nodes and Rel set of edges. Each edge among two nodes is a link among the agents ; with and belonging to Ag. Each agent and is indicated by the tuple features the following internal parameters: – Unique ID (identifier), – Reputation value (or credits) N( ), – Set of agent’s neighbors RP( ), – Set of owned resources RO( ), – Set of goals (resource identifiers) RD( ), – Set of resources being downloaded P( ), – Set of pairs < supplier; resource >. A resource is a tuple , where Name is the resource identifier and Authenticity is a Boolean attribute indicating whether the resource is authentic or not. The agent owning the resource, however, does not have access to this attribute unless he verifies the file. The resources represent the object being shared on the P2P network. A number of resources are introduced in the system at the beginning of the simulation; they represent both the owned objects and the agents' goals. For coherence, an owned resource can't be a goal, for the same agent. The distribution of the resource is stochastic. During the simulation, other resources are stochastically introduced. In this way, each agent in the system has the same probabilities to own a resource, independently from her inner nature (malicious or loyal). In the same way also the


7

corresponding new goals are distributed to the agents; the difference is that the distribution probability is constrained by its being possessed by an agent. Formally R be the set of all the resources in the system. We have that: RD( ) ⊆ R, RO( ) ⊆ R, RD( )∩ RO( ) = Ø .

(1)

Each agent in the system features a set of neighbors N( ), containing all the agents to which she is directly linked in the graph: N( ) = {

∈ Ag |
∈ Rel}.

(2)

This information characterizes the information of each agent about the environment. The implemented protocol is a totally distributed one, so looking for the resource is heavily based on the set of neighbors. In the real word the shared resources often have big dimensions; after finding the resource, a lot of time is usually required for the complete download. In order to simulate this the set of the "resources being downloaded" (Ris) introduced. These are described as Ris = , where ID is the resource identifier, completion is the percentage already downloaded and "check status" indicates whether the resource has been checked for authenticity or not. In particular, it can be not yet verified, verified and authentic and verified and inauthentic: check status ∈ {NOT CHECKED;AUTH; INAUTH}.

(3)

Another information is ID of the provider of a certain resource, identified by P( ). Each agent keeps track of those which are uploading to him, and this information is preserved also after the download is finished. Real systems allow the same resource to be download in parallel from many providers, to improve the performance and to split the bandwidth load. This simplification should not affect the aggregate result of the simulation, since the negative payoff would reach more agents instead of just one (so the case with multiple provider is a sub-case of that with a single provider). 4.1 The Reputation Model In this work we assume a simple idealized model of reputation, since the objective is not to prove the effectiveness of a particular reputation algorithm but to study the effect of users' behavior on a reputation system. We use a centralized system which assumes the correctness of information provided by users, e.g., it is not possible to give an evaluation of a user with whom there was no interaction. The reason is that we focus on the behavior of common agents and not on hackers who attack the system by manipulating the code of the peer application. In the system there are two reputation thresholds: the first and higher one, under which it’s impossible to ask for resources to other agents, the second, lower than the other, which makes it impossible even to share the owned files. This guarantees that an agents that falls under the first one (because she shared too many inauthentic files), can still regain credits by sharing authentic ones and come back over the first threshold. On the contrary, if she continues sharing inauthentic files, she will fall under the second threshold, being de facto excluded from the network, still being a working link from and to other agents.

8


4.2 The User Model Peers are reactive agents replying to requests, performing requests or verifying files. While upload is performed each time another agent makes a request, requesting a file and verification are performed (in alternative) when it is the turn of the agent in the simulation. All agents belong to two disjoint classes: malicious agents and loyal agents. The classes have different behaviors concerning uploading, while they have the same behavior concerning downloading and verification: malicious agents are just common agents who exploit for selfishness the weaknesses of the system. When it is the turn of another peer, and he requests a file to the agent, he has to decide whether to comply with the request and to decide how to comply with it. - The decision to upload a file is based on the reputation of the requester: if it is below the "replying threshold", the requestee denies the upload (even if the requestee is a malicious agent). - The "replyTo" method refers to the reply each agent gives when asked for a resource. When the agent is faced with a request he cannot comply but the requester's reputation is above the "replying threshold", if he belongs to the malicious class, he has to decide whether to create and upload an inauthentic file by copying and renaming one of his other resources. The decision is based depending on a parameter. If the resource is owned, she sends it to the requesting agent, after verifying if her reputation is higher than the "replying threshold". Each agent performs at each round of simulation two steps: 1) Performing the downloads in progress. For each resource being downloaded, the agents check if the download is finished. If not, the system checks if the resource is still present in the provider's "sharing pool". In case it's no longer there, the download is stopped and is removed from the list of the "owned resources". Each file is formed by n units; when 2/n of the file has been downloaded, then the file gets automatically owned and shared also by the agent that is downloading it. 2) Making new requests to other peers or verifying the authenticity of a file downloaded or in downloading, but not both: a) When searching for a resource all the agents within a depth of 3 from the requesting one are considered. The list is ordered by reputation. A method is invoked on every agent with a reputation higher than the "requests threshold", until the resource is found or the list reaches the ending point. If the resource is found, it's put in the "downloading list", the goal is cancelled, the supplier is recorded and linked with that specific download in progress and her reputation is increased according to the value defined in the simulation parameters. If no resource is found, the goal is given up. b) Verification means that a file is previewed and if the content does not correspond to its description or filename, this fact is notified to the reputation system. Verification phase requires that at least one file must be in progress and it must be beyond the 2/n threshold described above. An agent has a given probability to verify instead of looking for a new file. In case the agent verifies, a random resource is selected among those “in download” and not checked. If authentic, the turn is over. Otherwise, a "punishment" method is


9

invoked, the resource deleted from the "downloading" and from the "owned " lists and put among the "goals" once again. The RMS is based on the "punishment" method which lowers the supplier's reputation, deletes her from the "providers" list in order to avoid cyclic punishment chains, and recursively invokes the "punishment" method on the punished provider. A punishment chain is thus created, reaching the creator of the inauthentic file, and all the aware or unaware agents that contributed in spreading it.

5 Results The simulation goes on until at least one goal exists and/or a download is still in progress. In the following table a summary of the most important parameters for the experiments are given: Table 1. The main parameters

Parameters Total number of agents Total number of graph edges Initial reputation (credits) for each agent Percentage of loyal agents Total number of resources at the beginning Threshold to share owned resources Threshold for requesting resources Number of turns for introduction of new resources Number of new resources to introduce Total number of steps

Value 50 80 50 50% 50 25 10 1 3 2000

In all the experiments, the other relevant parameters are fixed, while the following ones change: Table 2. The scenarios

Parameters Positive payoff Negative payoff Verification percentage

Exp 1 1 3 30%

Exp 2 1 3 40%

Exp 3 1 4 30%

Exp 4 1 8 30%

Exp 5 1 0 30%

Exp 6 1 2 40%

A crucial index, defining the wellbeing of the P2P system, is the ratio among the number of inauthentic resources and the total number of files on the network. The total number is increasing more and more over time, since new resources are introduced iteratively. Another measure collected is the average reputation of loyal and malicious agents at the end of the simulation; in an ideal world, we expect malicious ones to be penalized for their behavior, and loyal ones to be rewarded. The results were obtained by a batch execution mode for the simulation. This executes 50 times the simulation with the same parameters, sampling the inauthentic/total ratio every 50 steps. This is

10

G. Boella and M. Rem mondino

to overcome the sampling effect; many variables in the simulation are stochastic, so h level of confidence for the produced results. In 2000 tuurns, this technique gives an high we have a total of 40 samp ples. After all the executions are over, the average for eeach time step is calculated, and represented in a chart. In the same way, the grand averrage f loyal and malicious agents is calculated, and represennted of the average reputations for in a bar chart. In Fig. 1, the chart with the trend of inauthentic/total resources is represented for the results coming from experiments 1, 2, 3, 5 and 6. The resultss of experiment 4 is discussed laater.

Fig. 1. Inauthentic/total ratio

Experiment 5 depicts thee worst case: no negative payoff is given: this is the casee of a P2P network without a RMS R behind it. The ratio initially grows and, at a certtain point, it gets constant overr time, since new resources are stochastically distribuuted among all the agents with th he same probability. In this way also malicious agents hhave new resources to share, and a they will send out inauthentic files only for thhose resources they do not own. In the idealized world modeled in this simulation, siince nd 50 loyal, and since the ones with higher reputation are agents are 50 malicious an preferred when asking for a file, it’s straightforward that malicious agents’ reputattion fly away, and that an high h percentage of files in the system are inauthentic (abbout 63%). Experiment 1 shows how a simple RMS, with quite a light punishing factorr (3) wer the percentage of inauthentic files in the network oover is already sufficient to low time. We can see a positivee trend, reaching about 28% after 2000 time steps, which is an over 100% improvement compared to the situation in which there was no punishment for inauthenticc files. In this experiment the verification percentage is at 30%. This is quite low, sincce it means that 70% of the files remain unchecked foreever (downloaded, but never ussed). In order to show how much the human factor can influence the way in which a RMS works, in experiment 2 the verification percenttage 4 leaving the negative payoff still at 3. The resullt is has been increased up to 40%, surprisingly good: the inau uthentic/total ratio is dramatically lowered after few tuurns (less than 10% after 200), reaching r less than 1% after 2000 steps. Since 40% of ffiles checked is quite a realistic percentage p for a P2P user, this empirically proves that eeven the simple RMS proposeed here dramatically helps in reducing the numberr of inauthentic files. In order to assign a quantitative weight to the human factor, in

Collaboration and Human n Factor as Drivers for Reputation System Effectiveness

11

experiment 3, the negativee payoff is moved from 3 to 4, while bringing back the verification percentage to 30%. Even with a higher punishing factor, the ratio is woorse ning that it’s preferable to have a higher verification rrate, than in experiment 2, mean compared to a higher negaative payoff. Experiment 6 shows the opposite trend: the negative payoff is lighterr (2), but the verification rate is again at 40%, ass in experiment 2. The trend is very similar – just a bit worse - to that of experiment 33. In particular, the ratio of inautthentic files, after 2000 turns, is about 16%. At this poinnt, it gets quite interesting to fin nd the “break even point” among the punishing factor and the verification rate. After some empirical simulations, we have that, compared w with 40% of verification and 3 negative payoff, if now verification is just at 30%, the negative payoff must be seet to a whopping value of 8, in order to get a comparaable trend in the ratio. This is do one in experiment 4 (Fig. 2): after 2000 turns, there’s 1% % of inauthentic files with a negaative payoff of 3 and a verification percentage of 40%, and about 0.7 with 8 and 30% reespectively. This clearly indicates thaat collaboration factor (the files verification) is crucial foor a RMS to work correctly and give the desired aggregate results (few inauthentic files oover a P2P network). In particular, a slightly higher verification rate (from 30% to 400%)

Fig g. 2. Weighting the collaboration factor

Fig. 3. 3 Final average reputations for the agents

12


weights about the same of a heavy upgrade of the punishing factor (from 3 to 8). This can be considered as a quantitative result, comparing the exogenous factor (resource verification performed by the users) to the endogenous one (negative payoff). Besides considering the ratio of inauthentic files moving on a P2P network, it’s also crucial to verify that the proposed RMS algorithm could punish the agents that maliciously share inauthentic files, without involving too much unwilling accomplices, which are loyal users that unconsciously spread the files created by the former ones. This is considered by looking at the average reputations, at the end of simulation steps (Fig. 3). In the worst case scenario, the malicious agents, that are not punished for producing inauthentic files, always upload the file they are asked for (be it authentic or not). In this way, they soon gain credits, topping the loyal ones. Since in the model the users with a higher reputation are preferred when asking files, this phenomenon soon triggers an explosive effects: loyal agents are marginalized, and never get asked for files. This results in a very low average reputation for loyal agents (around 70 after 2000 turns) and a very high average value for malicious agents (more than 2800) at the same time. In experiment 1 the basic RMS presented here, changes this result; even with a low negative payoff (3) the average reputations after 2000 turns, the results are clear: about 700 for loyal agents and slightly more than 200 for malicious ones. The algorithm preserves loyal agents, while punishing malicious ones. In experiment 2, with a higher verification percentage (human factor), we see a tremendous improvement for the effectiveness of the RMS algorithm. The average reputation for loyal agents, after 2000 steps, reaches almost 1400, while all the malicious agents go under the lower threshold (they can’t either download or share resources), with an average reputation of less than 9 points. Experiment 3 explores the scenario in which the users just check 30% of the files they download, but the negative payoff is raised from 3 to 4. The final figure about average reputations is again very good. Loyal agents, after 2000 steps, averagely reach a reputation of over 1200, while malicious ones stay down at about 40. This again proves the proposed RMS system to be quite effective, though, with a low verification rate, not all the malicious agents get under the lower threshold, even if the negative payoff is 4. In experiment 6 the verification percentage is again at the more realistic 40%, while negative payoff is reduced to 2. Even with this low negative payoff, the results are good: most malicious agents fall under the lowest threshold, so they can’t share files and they get an average reputation of about 100. Loyal agents behave very well and reach an average reputation of more than 900. Experiment 4 is the one in which we wanted to harshly penalize inauthentic file sharing (negative payoff is set at 8), while leaving an high laxity in the verification percentage (30%). Unlikely what it could have been expected, this setup does not punish too much loyal agents that, unwillingly, spread unchecked inauthentic files. After 2000 turns, all the malicious agents fall under the lowest threshold, and feature an average reputation of less than 7 points, while loyal agents fly at an average of almost 1300 points. The fact that no loyal agent falls under the “point of no return” (the lowest threshold) is probably due to the fact that they do not systematically share inauthentic files, while malicious agents do. Loyal ones just share the inauthentic resources they never check. Malicious agents, on the other side, always send out inauthentic files when asked for a resource they do not own, thus being hardly punished by the RMS, when the negative payoff is more than 3.


13

5.1 Comeback Mode: Whitewashing A "whitewashing" mode is implemented and selectable before the simulation starts, in order to simulate the real behavior of some P2P users who, realizing that they cannot download anymore (since they have low credits or, in this case, bad reputation), disconnect their client, and then connect again, so to start from the initial pool of credits/reputation. When this mode is active, at the beginning of each turn all the agents that are under a given threshold reset it to the initial value, metaphorically representing the disconnection and reconnection. In experiments 7, 8 and 9 this is tested to see if it affects previous results. In Fig. 4, the ratio among inauthentic and total resources is depicted, and in Fig. 5 the final average reputation for agents, when whitewashing mode is active.

Fig. 4. Inauthentic/total ratio in whitewashing mode

Even with CBM activated, the results are very similar to those in which this mode is off. They are actually a bit worse when the negative payoff is low (3) and so is the verification percentage (30%): the ratio of inauthentic files in the network is quite high, at about 41% after 2000 turns versus the 27% observed in experiment 1, which had the same parameters, but no CBM. When the verification percentage is increased to 40%, though, things get quite better. Now the ratio of inauthentic files has the same levels as in experiment 2 (less than 1% after 2000 steps). Also with a lower verification percentage (again at 30%), but leaving the negative payoff at 4, the figure is almost identical to the one with the same parameters, but without a CBM. After 2000 turns, the inauthentic files ratio is about 12%. The experiments show that malicious agents, even resetting their own reputation after going below the lowest threshold, can’t overcome this basic RMS, if they always produce inauthentic files. This happens because, even if they reset their reputation to the initial value, it’s still low compared to the one reached by loyal agents; if they shared authentic files, this value would go up in few turns, but since they again start spreading inauthentic files, they almost immediately fall under the thresholds again.

14


Fig. 5. Final average reputations for the agents, in whitewashing mode

5.2 Scaling Issue Changing the numbers of agents, agent based models can suffer from scaling issues, meaning that the results obtained with a certain number of agents don’t hold when this number changes significantly. To verify this, two more experiments were carried on. The number of agents is increased to 150 (three times the previous value). Coherently, the number of edges, the initial pool of resources and the number of resources introduced at each turn are also tripled. The trend is very similar: with a low negative payoff (3) and a low verification rate (30%), we have just a slightly higher ratio of inauthentic files. The same comparison is carried on with the average final reputations. Again, the results are very similar, even if the system with more agents has a slightly worse aggregate behavior than the smaller one. Generally speaking we conclude that the difference is very low, so the scaling issue is not influencing the results shown, on a 1:3 basis. In future works this study will be extended to even more agents.

6 Discussion and Conclusions The main purpose of the work is to show, by means of an empirical analysis based on simulation, how the collaboration coming from the agents in a social system can be a crucial driver for the effectiveness of a RMS. As a test-bed we considered a P2P network for file sharing and, by an agent based simulation, we show how a basic RMS can be effective to reduce inauthentic files circulating on the network. In order to enhance its performance, though, the collaboration factor, in the form of verifying policy, is crucial: a 33% more in verification results in about thirty times less inauthentic files on the network. While a qualitative analysis of this factor is straightforward for the presented model, we added a quantitative result, trying to weight the exogenous factor (the verification rate) by comparing it to the endogenous one (the negative payoff). We showed that a 33% increase in verification percentage leads to similar results obtained by increasing the negative payoff of 66%. Again, the collaboration factor proves to be crucial for the RMS to work efficiently.


15

First of all in this paper we look for the factors controlled by users determining the effectiveness of RMSs in P2P network. Two are the critical points: the decision of sharing inauthentic files and the decision to verify or not the downloaded files. While the benefit of not verifying is determined by the saved time (verifying is incompatible with making new searches and starting new downloads), the benefit of spreading inauthentic files must be confirmed by the simulation. Without the mechanism for punishing malicious agents, inauthentic files increase sharply, and at the end of simulation the reputation of malicious agents is much higher than loyal ones', and this is reached very soon. Thus, producing inauthentic files is a worthy strategy, under no enforcement mechanism. However, the behavior of malicious agents strikes back against them: they are not attackers and have the goal of downloading resources, as well. Here we assume that if a file is verified, then the reputation of the uploader is decreased immediately, due to the lower cost of this action. A more fine grained model should consider also this human factor. Analogously we do not consider the possibility to punish peers without first receiving and checking a file - a behavior which should be prevented by the software itself - as well as we do not consider the possibility of punishing even if the file is authentic. As stated in the Introduction, our goal is to model the behavior of normal user, not of hackers attacking the system. The second question of the work is: how to evaluate the role of these factors by means of agent based simulation? Which factors have most impact? The simulation framework for reputation gives interesting results: the key factor to lower the number of inauthentic files in a file sharing P2P system is the proportion of verifications made by peer. Even a low figure like 30% sharply limits the behavior of malicious agents when we do not consider the possibility of whitewashing after comeback. The role of punishment, in terms of reputation points, has instead a more limited impact. Surprisingly, even when whitewashing is allowed, the number of inauthentic files in the system can be limited if peers verify files 40% of the times. The same results cannot be achieved by increasing the figure of the punishment and decreasing the proportion of verifications. The moral of our study is that a mechanism for stimulating users to check the authenticity of files should be promoted, otherwise the entire file sharing system is flooded by inauthentic files. In contrast, usual approaches to RMS [1] consider verification as automatic, thus ignoring the human factor: since we show that verification has a sharp effect according to its rate, it cannot be ignored in simulating the effect of a RMS. Thus, we identify the conditions when even a simple RMS can dramatically reduce the number of inauthentic files over a P2P system and penalize malicious users, without banishing them from the network, as proposed in other models based on ostracism. The proposed model is simplistic. The reader must be aware of several limitations, which are the object of ongoing work. Resources are not divided in categories; inauthentic files, in reality, are mostly found in newer resources. Thus, we are aiming at using real data to differentiate the kinds of resources, distinguishing in particular newly requested resources. At present we distinguish malicious agents from loyal agents based on their response, but all the agents have the same behavior for other aspects, e.g. they verify with the same proportion. It could be useful to simulate using different parameters in each agent of the two classes. Bandwidth is not considered:

16


thus all downloads proceeds at the same rate, even if the decision to upload to a peer is based on his reputation. Moreover, a peer decides to upload on the basis of which agent has the highest reputation. It is well known that this algorithm can create unbalance among peers, but we abstract here from this problem, since we are not proposing a new P2P mechanism but checking the efficacy of a RMS on the specific problem of inauthentic files. Note, however, that this strategy has a negative effect when malicious peers get high reputation, but if the reputation system is well tuned, malicious agents should never get high reputation. Finally, we allow agents to disconnect and reconnect, but this whitewashing happens without changing their position on the graph, or behavior. The real improvement in our ongoing work, is moving from reactive agents to more sophisticated ones, able to learn from what happened in the network. While in the current model agents stochastically decide whether to upload an inauthentic file or not, or to verify or not, cognitive agents adapt to the circumstances, by considering how many objectives they can achieve using their current strategy, and by looking for new alternatives. Modeling adaptive agents allows to check for further vulnerabilities, like what happens when agents produce inauthentic files at a variable rate which does not decrease too much their reputation. Acknowledgements. The authors wish to gratefully acknowledge Mr. Gianluca Tornese, who initially contributed to the implementation of the model described in the present work.

References 1. Josang, A., Ismail, R., Boyd, C.: A survey of trust and reputation systems for online service provision. Decis. Support Syst. 43(2), 618–644 (2007) 2. Kamvar, S.D., Schlosser, M.T., Garcia-Molina, H.: The eigentrust algorithm for reputation management in p2p networks. In: WWW 2003: Proceedings of the 12th international conference on World Wide Web, USA, pp. 640–651. ACM Press, New York (2003)

Agent-Oriented Programming for Client-Side Concurrent Web 2.0 Applications Mattia Minotti, Giulio Piancastelli, and Alessandro Ricci DEIS, Alma Mater Studiorum — Università di Bologna, Italy [email protected], [email protected]

Abstract. Using the event-driven programming style of JavaScript to develop the concurrent and highly interactive client-side of Web 2.0 applications is showing more and more shortcomings in terms of engineering properties such as reusability and maintainability. Additional libraries, frameworks, and AJAX techniques do not help reduce the gap between the single-threaded JavaScript model and the concurrency needs of applications. We propose to exploit a different programming model based on a new agent-oriented abstraction layer, where first-class entities – namely agents and artifacts – can be used, respectively, to capture concurrency of activities and their interaction, and to represent tools and resources used by agents during their activities. We specialise the model in the context of client-side Web development, by characterising common domain agents and artifacts that form an extension of an existing programming framework. Finally, we design and implement a simple but significant case study to showcase the capabilities of the model and verify the feasibility of the technology. Keywords: Concurrent Programming, Agent-Oriented Programming, Web 2.0.

1 Introduction One of the most important features of the so-called Web 2.0 is a new interaction model between the client user interface of a Web browser and the server-side of the application. Such rich Web applications allow the client to send multiple concurrent requests in an asynchronous way, avoiding complete page reload and keeping the user interface live and responding. Periodic activities within the client-side of the applications can be performed in the same fashion, with clear advantages in terms of perceived performance, efficiency and interactivity. The client user interface of rich applications is programmed with extensive use of JavaScript and AJAX techniques. Being JavaScript a single-threaded language, most of those programs are written in an event-driven style, in which programs register callback functions that are triggered on events such as timeouts. A single-threaded event loop dispatches the appropriate callback when an event occurs, and control returns back to the loop when the callback completes. To implement concurrency-intensive features, still keeping the interface responsive, programmers must chain callbacks together— typically, each callback ends by registering one or more additional callbacks, possibly with a short timeout. However, this style of event-driven programming is tedious, bugprone, and harms reusability [3]. J. Cordeiro and J. Filipe (Eds.): WEBIST 2009, LNBIP 45, pp. 17–29, 2010. c Springer-Verlag Berlin Heidelberg 2010

18

M. Minotti, G. Piancastelli, and A. Ricci

The limitations of the JavaScript programming model have been faced by introducing libraries that attempt at covering event-based low-level details behind the emulation of well-known concurrent abstractions. Concurrent.Thread [7] builds a thread abstraction on top of the event-driven JavaScript model, converting multi-threaded code into an efficient continuation-passing style. The WorkerPool API in Google Gears1 simulates a collection of processes that do not share any execution state, thus can not directly access the browser DOM of Web pages. Albeit working in practice, neither approach is feasible to support a sound programming model for concurrency in Web 2.0 applications [6]. Frameworks such as Rails,2 with its Ruby-to-JavaScript compiler plug-in, and Google Web Toolkit (GWT)3 approach the problem from a different angle, by adopting the programming model of the single language employed for implementing both the client- and server-side of the application. Even if the application development benefits from the use of object-oriented languages in terms of decomposition and encapsulation, the limitations of the target execution environment make such solution less effective: for example, GWT does not allow the use of Java threads, and only offers a timer abstraction to schedule tasks (implemented as methods) within the single-threaded JavaScript interpreter. We argue that, despite these several efforts, Web application client-side technologies do not offer suitable abstractions to manage the coordination and interaction problems of typical distributed applications in a simple yet complete fashion. With the aim of supporting the development of Web applications as much similar as possible to distributed desktop applications, JavaScript and AJAX are not enough: a different programming model is needed, so as to provide mechanisms and abstractions really oriented to collaboration and cooperation among concurrent computational entities. In this sense, even the object-oriented paradigm promoted by GWT shows its shortcomings. Indeed, mainstream object-oriented programming languages such as Java are currently undergoing a concurrency revolution [13], where (i) support for multi-threading is extended by synchronisation mechanisms providing a fine-grained and efficient control on concurrent computations, and (ii) it is becoming more and more important to also provide more coarse-grained entities that help build concurrent programs from a set of higher-level abstractions. We believe that, to effectively face challenges such as distributed computation, asynchronous communication, coordination and cooperation, agents represent a very promising abstraction, natively capturing and modeling concurrency of activities and their interaction; therefore, they can be considered a good candidate for defining a concurrent programming model beyond languages and technologies currently available within browsers on the client-side of Web applications. The agent abstraction is meant to model and design the task-oriented parts of a system, encapsulating the logic and control of such activities. Not every entity in the system can (or should) be modeled in this way, though; we also introduce the companion artifact abstraction, as the basic building block to design the tools and resources used by agents during their activities. 1 2 3

http://gears.google.com http://www.rubyonrails.org/ http://code.google.com/webtoolkit/

Agent-Oriented Programming for Web 2.0

19

Accordingly, in this paper we first describe the agent and artifact abstractions as defined by the A&A (Agents and Artifacts) programming model [8], recently introduced in the field of agent-oriented programming and software engineering; then, we specialise the model in the context of client-side Web application development, by characterising common domain agents and artifacts that form a small extension of an existing programming framework based on A&A. Further, we describe the design and implementation of a simple but significant case study to showcase the capabilities of the framework, and conclude with some final remarks, including next steps of this work.

2 The Agents and Artifacts Programming Model In the A&A programming model, the term “agent” is used to represent an entity “who acts” towards an objective or task to do pro-actively, and whose computational behaviour accounts for performing actions in some kind of environment and getting information back in terms of perceptions. Differently from the typical software entity, agents have no interface: their interaction with the environment takes place solely in terms of actions and perceptions, which concerns in particular the use of artifacts. The notion of activity is employed to group related actions, as a way to structure the overall behaviour of an agent. The A&A model is currently accompanied with the simpA framework4 as a Java technology to prototype concurrent applications using the agent and artifact abstractions as basic building blocks [11]. To define a new agent template in simpA, only one class must be defined, extending the alice.simpa.Agent base class provided by the framework API. For example, on the client-side of a Web e-mail application, an agent that fetches all the messages from an account, and periodically checks if new messages arrived, may be structured as follows: public class MailAgent extends Agent @ACTIVITY_WITH_AGENDA( todos={ @TODO(activity="setup", persistent=false), @TODO(activity="fetch", persistent=false, pre=completed(setup)), @TODO(activity="check", persistent=true) } ) void main() {} @ACTIVITY void setup() { /* ... */ @ACTIVITY void fetch() { /* ... */ @ACTIVITY void check() { /* ... */

{

} } }}

Agent activities in simpA can be either atomic or structured, composed by some kinds of sub-activities. Atomic activities are implemented as methods with the @ACTIVITY 4

http://simpa.sourceforge.net

20


annotation, with the body of the method defining the computational behaviour of the agent corresponding to the accomplishment of the activity. Structured activities introduce the notion of agenda to specify the hierarchical set of the potential sub-activities composing the activity, referenced as todo in the agenda. Each todo names the subactivity to execute, and optionally a pre-condition. When a structured activity is executed, the todos in the agenda are executed as soon as their pre-conditions hold, but if no pre-condition is specified, the todo is immediately executed. Thus, multiple subactivities can be executed concurrently in the context of the same (super) activity. A structured activity is implemented by methods with an @ACTIVITY WITH AGENDA annotation, containing todo descriptions as a list of @TODO annotations. A todo can be specified to be persistent—in this case, once it has been completely executed, it is reinserted in the agenda so as to be possibly executed again. This is useful to model cyclic behaviour of agents when executing some activities. In the A&A programming model, the artifact abstraction is useful to design passive resources and tools that are used by agents as basic building blocks of the environment. The functionality of an artifact is structured in terms of operations whose execution can be triggered by agents through artifact usage interface. Similarly to the interface of objects or components, the usage interface of an artifact defines a set of controls that an agent can trigger so as to execute operations, each one identified by a label and a list of input parameters. Differently from the notion of object interfaces, in this use interaction there is no control coupling: when an agent triggers the execution of an operation, it retains its control (flow) and the operation execution on the artifact is carried on independently and asynchronously. The information flow from the artifact to agents is modeled in terms of observable events generated by artifacts and perceived by agents; therefore, artifact interface controls have no return values. An artifact can also have some observable properties, allowing to inspect the dynamic state of the artifact without necessarily executing operations on it. Artifact templates in simpA can be created by extending the base alice.simpa. Artifact class. For example, an artifact representing a Web page, storing its DOM model in an observable property, and exposing an operation to change an attribute of an element in the model, may be structured as follows: public class Page extends Artifact { @OBSPROPERTY Document dom; @OPERATION void setAttribute(String id, String attr, String val) { Element e = dom.getElementById(id); e.setAttribute(attr, val); updateProperty("DOM", dom); } }

Each operation listed in the artifact user interface is defined as a method with an @OPERATION annotation. Besides the usage interface, each artifact may define a linking interface, applying the @LINK annotation on operations that are meant to be invoked by other artifacts.


21

1b

1a

sense with-my-filter

use opX(params) 2a

opX EXECUTION

opX EXECUTION TRIGGERED

2b

SENSOR XYZ

my_event1

e e

op_exec_completed

Fig. 1. Abstract representation of an agent using an artifact, by triggering the execution of an operation (left, step 1a) and observing the related events generated by the operation execution (right, step 1b)

Thus, it becomes possible to create complex artifacts as a composition of simpler ones, assembled dynamically by the linking mechanism. An artifact typically provides some level of observability, either by generating observable events during execution of an operation, or by defining observable properties using the @OBSPROPERTY annotation. An observable event can be perceived by the agent that has used the artifact by triggering the operation generating the event; changes to observable properties, triggered by the updateProperty primitive, can be sensed by any agent that has focussed on the artifact, without necessarily having acted on it. Besides agents and artifacts, the notion of workspace completes the basic set of abstractions defined in A&A: a workspace is a logic container of agents and artifacts, and it is the basic means to give an explicit (logical) topology to the working environment, so as to help scope the interaction inside it. We conclude this section by focussing on the main ingredients of the agent-artifact interaction model: a more comprehensive account and discussion of these and other features of agents, artifacts and workspaces – outside the scope of this paper – can be found in [8,11]. 2.1 Agent-Artifact Interaction: Use and Observation The interaction between agents and artifacts strictly mimics the way in which people use their tools. For a simple but effective analogy, let us consider a coffee machine. The set of buttons of the coffee machine represents the usage interface, while the displays that are used to show the state of the machine represent artifact observable properties. The signals emitted by the coffee machine during its usage represent observable events generated by the artifact. Interaction takes place by means of a use action (stage 1a in Figure 1, left), performed by an agent in order to trigger the execution of an operation over an artifact; such an action specifies the name and parameters of the interface control corresponding to the operation. The observable events, possibly generated by the artifact while executing an operation, are collected by agent sensors, which are the parts of the agent

22


conceptually connected to the environment where the agent itself is situated. Besides the generation of observable events, the execution of an operation results in updating the artifact inner state and possibly artifact observable properties. Finally, a sense action is executed by an agent to explicitly retrieve the observable events collected by its sensors. It must be noted that, dually to indirect interaction through artifacts, agents can also directly communicate with each other, through a message passing mechanism provided by the framework.

3 An Agent-Oriented Model for Client-Side Web Programming The main objective of our programming model based on agent and artifact abstractions is to simplify development of those parts on the client-side of Web applications that involve elements of concurrency, by reducing the gap between design and implementation. At the design level, it is first necessary to identify the task-oriented and functionoriented parts of the client-side system; then, such organisation drives to the definition of agents and artifacts as depicted by the A&A model. At the implementation level, the design elements can be directly realised using the simpA framework. The notion of activity and the hierarchical activity model adopted in the agent abstraction allow a quite synthetic and readable description of possibly articulated behaviours, dealing with coordination management on a higher level than threads, timeouts, and callbacks. The adopted model of artifacts permits to specify possibly complex functionalities to be shared and concurrently exploited by multiple agents, abstracting away from low-level synchronisation mechanisms and primitives. In the domain of client-side Web development, all computation takes place in the browser environment, that can be straightforwardly mapped onto the workspace abstraction; agents and artifacts downloaded as the client-side part of Web applications will join this workspace, interacting and cooperating in a concurrent fashion. Among those agents and artifacts, there are a number of recurring elements in the majority of applications; here follows a description of the main elements that we identified, along with the role they will play in our programming model. Page. The page is typically represented by an accessible, tree-like, standardised DOM object, allowing to dynamically update its content, structure, and visualisation style; also, the page generates internal events in response to user’s actions. The direct mapping of the DOM API onto operations, and of the response to user’s actions onto observable events, suggests the artifact as the most natural abstraction to model pages. HTTP Channel. An entity is needed to perform transactions in the HTTP protocol, allowing to specify the operation to execute (e.g. GET, POST, PUT, DELETE), set the header values and possibly a payload in the request; such an entity also has to receive responses and make them available to the other entities in the system. The channel does not account for autonomous actions, but it is supposed to be used by a different, proactive entity; it is therefore modeled as an artifact, so that asynchronous communication towards the server can be ensured by the agent-artifact interaction model.


23

Common Domain Elements. Common components that are frequently and repeatedly found in specific application domains can also be modeled in terms of agents and artifacts. As an example, consider the shopping cart of any e-commerce Web site: since its control logic and functionalities are almost always the same, it could be implemented as a client-side component, in order to be used by a multiplicity of e-commerce transactions towards different vendors, and allowing comparisons and other interesting actions on items stored for possibly future purchase. Given the possibly complex nature of the computations involved, such common domain elements would probably need to be modeled as a mix of both agents and artifacts. There also are some important issues in client-side Web application development that, due to the lack of space, we only intend to acknowledge without providing a complete description of their possible modeling and programming solution. First, security needs to be addressed, in order to manage read/write permissions on the file system, mobile code authentication, origin control, and access authorisation; to this purpose, the RBAC model [12] provided by simpA can be exploited. Then, albeit architecturally deprecated, also cookies need to be taken into account as a popular mechanism to allow data persistence between working sessions; they can be devised as another particular kind of artifact template. 3.1 From simpA to simpA-Web simpA-Web is a thin layer that exploits classes and mechanisms of the simpA framework to define agent and artifact templates oriented to client-side Web development. Whereas simpA supports A&A abstractions, simpA-Web offers specific agents and artifacts representing the common elements comprised by the programming model explained above. For example, simpA-Web provides implementations for the HTTP Channel and the Page artifacts. The HTTP Channel artifact represents a HTTP working session used by a part of the client-side Web application. The artifact allows communication through the HTTP protocol, and is able to store and manage both local and remote variables. The user interface of HTTP Channel exposes three operations: – setDestination stores the URI of the server to which the session represented by this artifact will communicate. This operation takes the destination URI as a parameter, and generates a DestinationChanged observable event. – setHeader adjusts a HTTP header for subsequent requests. It takes the header name and value as parameters, and generates a HeaderChanged observable event. – send transmits a HTTP request to the server identified by the stored URI. It takes the HTTP payload as a parameter, and generates three events: RequestSent, as soon as the HTTP request has been committed; Response, containing headers and body, when a HTTP response is received; Failure, if no response has been sent back. The Page artifact represents the interface to access the page visualised by the browser, encapsulating its events and its main characteristics. The Page artifact features an observable property called DOM, representing the Document Object Model of the whole

24


page. The artifact interface exposes six operations, each generating a corresponding observable event at the end of a successful execution: – changeDOM substitutes the old DOM with a new object taken as a parameter, thus changing the representation of the whole page. – setElement changes the content of a DOM element identified by the first id parameter to the string passed as second parameter. – setAttribute does the same as the previous operation, but on a property of a DOM element. – addChild appends a DOM element as a child to an element with a given id. The presented entities in the simpA-Web layer represent a common agent-oriented infrastructure deployed as an additional library alongside simpA, so as to free clientside Web application developers from dealing with such a support level and let them focus on application domain issues.

4 A Sample Agent-Oriented Web 2.0 Application To verify the feasibility of our A&A-based programming model and test-drive the capabilities of the simpA-Web framework, we designed a sample Web application to search products and compare prices from multiple services. The characteristics of concurrency and periodicity of the activities that the client-side needs to perform make this case study a significant prototype of the typical Web 2.0 application. We imagine the existence of N services (of type A) that offer product lists with features and prices, codified in some standard machine-readable format. Each type A service lets users download an agent (typically programmed by the service supplier) that allow product searching on that service; each agent communicates with the corresponding service using its own protocol, possibly different from the protocols of the other agents in the system. We further imagine the existence of an additional service (of type B) offering a static list of type A services or allowing to dynamically search the Web for such services. The client-side in this sample Web application needs to search all type A services for a product that satisfies a set of user-defined parameters and has a price inferior to a certain user-defined threshold. The client also needs to periodically monitor services so as to search for new offerings of the same product. A new offering satisfying the constraints should be visualised only when its price is more convenient than the currently best price. The client may finish its search and monitoring activities when some userdefined conditions are met—a certain amount of time is elapsed, or the user interrupts the search with a click on a proper button in the page displayed by the browser. Finally, if such an interruption took place, by pressing another button it must be possible to let the search continue from the point where it was blocked. It’s worth remarking that the sample application should be considered meaningful not for evaluating the performance or efficiency of the framework – which is part of future work – but for stressing the benefits of the agent-oriented programming model in facing the design of articulated and challenging application scenarios. The main value in this case is having a small set of high-level abstractions that make it possible to


25

keep a certain degree of modularisation, separation of concerns, and encapsulation in designing and programming applications, despite of the complexities given by aspects such as concurrency, asynchronous interactions, openness/dynamism, distribution. 4.1 Design From the above description, it is fairly easy to use high-level abstractions such as agents and artifacts in order to represent the different computational elements in the client-side of the application. We define the Application Main (AM) Agent as the first agent to execute on the client, with the responsibility to setup the application once downloaded. Immediately after activation, the agent populates the workspace with artifacts and other agents needed to perform the product search as defined by the requirements. After validating userdefined data for product querying, the agent spawns another agent to interface with the B service so as to get a list of A services and commence the product search. The AM Agent also needs to control results and terminate the research when a suitable condition is verified; moreover, it has to manage search interruption and restarting as a consequence of user’s explicit commands. The task of the Service Finder (SF) Agent is to use the type B service so as to find a list of type A services, and to concurrently download search agents from their sites, asking them to start their own activity. The SF Agent also has to periodically monitor the type B service for new type A services to exploit. The Product Finder (PF) Agent instances interact with their own type A service by a possibly complex communication protocol, so as to get a list of products satisfying the user-defined parameters. After, each agent passes the resulting list to another element in the workspace dealing with product data storage. Each PF Agent also periodically checks its service in order to possibly find new products or removing from the list products that are no more available. The Product Directory (PD) Artifact stores product data found by the PF agents, and applies filtering and ordering criteria to build a list of available products, possibly containing only one element. The artifact also shows a representation for its list on the page in the browser, updating results as they arrive. Finally, an additional Pausing Artifact is introduced to coordinate the agents in the workspace on the interruption or restarting of the research activities in response to a user’s command. While the control logic of search interruption and restarting is encapsulated in the AM agent, the agent does not directly know all the PF agents; in such a case when direct communication between agents is unfeasible, an intermediate coordination element is needed to ask agents to perform the actions requested by the user. The complete architecture of the client-side application is depicted in Figure 2, where interactions with pre-defined artifacts and agents in the browser are shown. In particular, it must be noted that the Service Finder and Product Finder agents use their own instances of the HTTP Channel artifact to communicate through the Web with the correspondent sites; also, the Product Directory artifact links the Page artifact in order to visualise the product list by directly use the DOM API operations that are made available through the Page artifact linking interface.

26


Fig. 2. The architecture of the sample client-side Web application in terms of agent, artifacts, and their interactions

It’s worth noting that we chose to place all the agents and artifacts described above on the client side to maximise the distribution of the workload (with respect to the server-side): however, it is straightforward to devise different design solutions where some of the agents (and artifacts) run on the server side and interact with agents on the client side by using proper artifacts like the HTTP Channels. 4.2 Implementation Since we wanted our application case study to exploit existing Web clients, so as to verify the feasibility of current technologies w.r.t. our new model, resorting to design or implementation compromise when needed, we had to face some discrepancies that emerged during the implementation phase. As we could not use JavaScript, due to its single-threaded nature, as a language for the client-side of our sample Web application, we adopted the other mechanism for code mobility that browsers make available, that is Java applets. Applets allow to transfer code from a server to a browser and have it executed within a controlled secure environment known as sandbox; in particular, signed applets drop much of the security constraints of the sandbox, for instance allowing Java classes to open their own connections towards multiple servers. Furthermore, the Java Virtual Machine invoked by the browser does not force any restriction on the number of threads that a program may spawn, thus providing a truly concurrent environment where to execute our application. Access to the page DOM representation in the browser is granted by a suitable JavaScript library called LiveConnect,5 that can be conveniently invoked from the JVM multi-threaded execution context. Another issue is that browsers start without any notion of agents or artifacts: for this reason, the simpA-Web framework has to be entirely downloaded at the same time as 5

http://developer.mozilla.org/en/LiveConnect


27

the agents and artifacts belonging to the real client-side application. In particular, we exploit the applet mechanism allowing a specific class to start as soon as the JAR files have been completely fetched from the Web site in order to activate the Application Main agent. The Page artifact is programmed to access the DOM API by using the Rhinohide6 library, a convenient wrapper around LiveConnect that offers a simpler and more complete support for page event management. The HTTP Channel artifact does not exploit the HTTP protocol support in the browser, instead relying on functionalities offered by the Java standard library. As a relevant code snippet, we show the Product Finder Agent interacting with the HTTP Channel artifact to get the product list from its type A service. public class ProductFinder extends Agent { @ACTIVITY void find() { ArtifactId http = lookupArtifact("HTTP_" + getId()); String url = getServiceURL() + getQuery(); // Make the Request use(http, new Op("setDestination", url)); SensorId sid = getSensor("s0"); use(http, new Op("send", "GET", ""), sid); // Sense the Response Perception p = sense(sid, "Response"); String body = p.stringContent(0); // ... } }

The source code of the application – including a PHP back-end simulating type A and B services, used to test the system – can be downloaded from simpA web site.

5 Next Steps simpA agent model is very simple, providing a support for a quite limited set of concepts – activity, actions, perceptions – that typically characterise the agent abstraction [14]. Also, being a framework based on the Java language, such concepts are not directly captured at a language level, but indirectly through an OO API. Actually the A&A meta-model is not bound to any specific concrete programming model or architecture for defining agents. Then, the next step is to investigate the adoption of agentoriented programming languages that provide a support for a stronger notion of agency, such as those introduced in literature for programming intelligent agents [15]. Generally speaking, these agent programming languages make it possible to program an agent in terms of high-level concepts, such as beliefs, goals (tasks) and plans to achieve goals (or fulfill tasks). In particular, they make it possible to easily program agents as software components that integrate both a pro-active behaviour – that is a behaviour that is oriented to the achievement of some goal choosing autonomously the best actions to 6

http://zelea.com/project/textbender/o/rhinohide

28


do for that – and a reactive behaviour – that is being able to promptly react to events occurring in the environment (such as the user inputs or the arrival of messages from the network, for example) and accordingly consider them for the next course of action. In that perspective a program, keeping the A&A conceptual model, is conceived as a collection of intelligent agents interacting in a shared environment, by directly communicating by means of message passing and by constructing and using artifacts, as shared resources and tools useful for their tasks. In other works [9,10] we described the concrete application of such a perspective for multi-agent system programming using concrete technologies, namely Jason platform for programming the agents [2] and CArtAgO framework for programming the artifactbased environments where agents are situated [10]. Actually also simpA is based internally on CArtAgO for defining, executing, managing artifacts. So a next step of our work will be apply this perspective also for Web 2.0 applications, in particular experimenting the use of Jason for programming (intelligent) agents populating the applications.

6 Concluding Remarks Current languages and techniques used to develop the client-side of Web 2.0 applications have clearly shown their shortcomings in terms of programming model and code organisation. Even for a mildly complex application such as the presented case study, a possible JavaScript-based approach is not feasible to effectively manage concurrent and periodical activities so as to promote engineering properties such as maintainability, extendibility, and reusability. In particular, the use of callbacks and timeouts to fit the single-threaded event-based programming style of JavaScript constrains developers to work at such a low abstraction level that a high degree of flexibility and a solid design are hard to achieve within reasonable amounts of time and effort. In this scenario, toolkits such as GWT appear only to partially help: while they may reduce the time-to-market and improve the overall structure and organisation of applications, their programming abstractions still lack the expressivity to represent the coordinating and cooperating entities of typical concurrent systems. Agents have been shown to natively capture concurrency of activities and their interaction, both at the design and the implementation level. In the pre-AJAX era, agents had already been used as the basic building block for client-side concurrent Web programming [5]. This suggests that agent-oriented programming models such as A&A may effectively help construct the kind of highly interactive applications that populate the Web 2.0 and exploit concurrency as their primary computational paradigm. As a significant prototype of this kind of applications, the presented case study has been designed and implemented by employing a set of engineering best practices and principles (including modularisation, separation of concerns, encapsulation) that are directly supported by the A&A model, but that are hardly represented in the current technologies for client-side Web development. The integration difficulties that agent-oriented technologies such as simpA have to overcome in order to seamlessly work on modern Web clients should not be considered as diminishing the value of the programming model, but as a temporary accident due to the lack of proper support for the concurrency paradigm in the mainstream.


29

As a final note, the use of agents to represent concurrent and interoperable computational entities already sets the stage for a possible evolution of Web 2.0 applications into Semantic Web applications. From the very beginning [1,4], research activity on the Semantic Web has always dealt with intelligent agents – as mentioned in Section 5 – capable of reasoning on machine-readable descriptions of Web resources, adapting their plans to the open Internet environment in order to reach a user-defined goal, and negotiating, collaborating, and interacting with each other during their activities. Concurrency and intelligence being two orthogonal programming concepts, agentoriented models that combine both aspects are likely to be adopted for future mainstream technologies.

References 1. Berners-Lee, T., Hendler, J., Lassila, O.: The Semantic Web. Scientific American (2001) 2. Bordini, R., Hübner, J., Wooldridge, M.: Programming Multi-Agent Systems in AgentSpeak Using Jason. John Wiley & Sons, Ltd., Chichester (2007) 3. Foster, J.S.: Directing JavaScript with Arrows. In: ICFP 2008: The 13th ACM SIGPLAN International Conference on Functional Programming, Victoria, British Columbia, Canada (2008) (poster paper) 4. Hendler, J.: Agents and the Semantic Web. IEEE Intelligent Systems 16(2), 30–37 (2001) 5. Hong, T.W., Clark, K.L.: Concurrent programming on the Web with Webstream. Technical report, Department of Computing, Imperial College, London (2000) 6. Lee, E.A.: The problem with threads. IEEE Computer 39(5), 33–42 (2006) 7. Maki, D., Iwasaki, H.: A portable JavaScript thread library for Ajax applications. In: OOPSLA 2007: Companion to the 22nd ACM SIGPLAN conference on Object oriented programming systems and applications, Montreal, Quebec, Canada, pp. 817–818 (2007) 8. Omicini, A., Ricci, A., Viroli, M.: Artifacts in the A&A meta-model for multi-agent systems. Autonomous Agents and Multi-Agent Systems 17(3), 432–456 (2008) 9. Ricci, A., Piunti, M., Acay, L.D., Bordini, R., Hubner, J., Dastani, M.: Integrating artifactbased environments with heterogeneous agent-programming platforms. In: Proceedings of 7th International Conference on Agents and Multi Agents Systems, AAMAS 2008 (2008) 10. Ricci, A., Piunti, M., Viroli, M., Omicini, A.: Environment programming in CArtAgO. In: Bordini, R.H., Dastani, M., Dix, J., El Fallah-Seghrouchni, A. (eds.) Multi-Agent Programming: Languages, Platforms and Applications, vol. 2, pp. 259–288. Springer, Heidelberg (2009) 11. Ricci, A., Viroli, M.: simpA: an agent-oriented approach for prototyping concurrent applications on top of Java. In: PPPJ 2007: The 5th international symposium on Principles and practice of programming in Java, Lisboa, Portugal, pp. 185–194 (2007) 12. Sandhu, R.S., Coyne, E.J., Feinstein, H.L., Youman, C.E.: Role-based access control models. IEEE Computer 29(2), 38–47 (1996) 13. Sutter, H., Larus, J.: Software and the concurrency revolution. ACM Queue: Tomorrow’s Computing Today 3(7), 54–62 (2005) 14. Wooldridge, M.: An Introduction to Multi-Agent Systems. John Wiley & Sons, Ltd., Chichester (2002) 15. Wooldridge, M.J., Jennings, N.R.: Intelligent agents: Theory and practice. The Knowledge Engineering Review 10(2), 115–152 (1995)

The SHIP: A SIP to HTTP Interaction Protocol Advanced Thin-Client Architecture for IMS Applications Joachim Zeiß1 , Rene Gabner1 , Sandford Bessler1 , and Marco Happenhofer2 1

ftw. Telecommunications Research Centre Vienna Doanu-City-Straße 1, 1220 Vienna, Austria {zeiss,gabner,bessler}@ftw.at http://www.ftw.at 2 Vienna University of Technology Favoritenstraße 9/388, 1040 Vienna, Austria [email protected] http://www.tuwien.ac.at

Abstract. IMS is capable of providing a wide range of services. As a result, terminal software becomes more and more complex to deliver network intelligence to user applications. Currently mobile terminal software needs to be permanently updated so that the latest network services and functionality can be delivered to the user. In the Internet, browser based user interfaces assure that an interface is made available to the user which offers the latest services in the net immediately. Our approach combines the benefits of the Session Initiation Protocol (SIP) and those of the HTTP protocol to bring the same type of user interfacing to IMS. SIP (IMS) realizes authentication, session management, charging and Quality of Service (QoS), HTTP provides access to Internet services and allows the user interface of an application to run on a mobile terminal while processing and orchestration is done on the server. A SHIP enabled IMS client only needs to handle data transport and session management via SIP, HTTP and RTP and render streaming media, HTML and Javascript. SHIP allows new kinds of applications, which combine audio, video and data within a single multimedia session. Keywords: distributed mobile computing, HTTP, IMS thin client, mobile client s/w virtualization, mobile widgets, SaaS, SIP.

1

Introduction

The IP Multimedia Subsystem (IMS) [1] is expected to provide convergent applications to mobile terminals using the Session Initiation Protocol (SIP) [2]. The IMS architecture envisions mobile terminals supporting SIP. SIP messages are routed from a user terminal across the IMS infrastructure to a specific application server in the system. A service application or component is triggered which, in turn, orchestrates service enablers (such as presence, location, etc.). Besides SIP, currently, additional protocols are required to be supported by an IMS capable client: the XML Configuration Access Protocol (XCAP) [3] for configuration J. Cordeiro and J. Filipe (Eds.): WEBIST 2009, LNBIP 45, pp. 30–43, 2010. c Springer-Verlag Berlin Heidelberg 2010

SHIP Advanced IMS Thin Client

31

of XML documents on XML Document Management Server (XDMS), the Message Session Relay Protocol (MSRP) [4] for stateful messaging, and of course HTTP [5] - for accessing the services via web front-ends. From the observation above, we see the need to simplify the interactions with service applications, to reduce the number of required protocol stacks and the amount of processing in the terminal. The lack of IMS terminals on the market is a major obstacle for IMS/NGN roll-out. Like existing SIP clients, current IMS clients support in general a nonextensible set of applications like VoIP, video, chat, presence, and address book functions. Interlinking the SIP and HTTP protocols, as presented in this paper, however, will provide the basis for an open model of SIP/IMS terminals. With this model new services and enablers become available to the mobile terminal instantly; there is no need to deploy additional client software or even upgrade the operating system to support new network capabilities and services. We propose a thin client architecture for IMS applications by introducing a distributed MVC software pattern [6]. The user interface components (view and control) run on the terminal and interact with processing components (model) executed on the application server. The components communicate via HTTP session controlled by SIP, concurrently with voice or video transport inside the same dialog. Generally, the SHIP solution follows the Software as a Service (SaaS) model already envisioned by Bennett et al. [7] to unburden the user of software maintenance, ongoing operation, and support. Our architecture enables on-demand pricing of applications, a better protection of intellectual property for the software vendor and the network operator to control services and act as application service provider. The main objective of this work is to present a solution for a better exploitation of the SIP signaling towards a seamless, secure, user friendly and provider-efficient deployment of IMS services. Such a solution would satisfy a number of wishes of network operators and users: – User micro-portal: IMS subscribers can connect to a user-friendly, appealing personalized homepage in which relevant information and their commonly used functions are presented. The users can manage their own profile data, application preferences and privacy settings. – Single sign-on: a user authenticated to IMS doesn’t need to authenticate again to perform actions for them she is authorized or consume services to which she is subscribed. – Web based user interface: in order to avoid the development and installation of client applications, web and web 2.0 technology could be used in the browser. – Asynchronous (HTTP push) events dispatched by applications towards user terminals are supported. The SHIP (SIP/HTTP interaction protocol) architecture is a blending of the SIP and HTTP protocols. IMS authentication is used to manage HTTP sessions within a SIP dialog. This approach leads to a novel, web based, universal IMS terminal.

32

1.1

J. Zeiß et al.

Related Work

Addressing the single-sign-on feature, 3GPP has proposed the Generic Bootstrapping Architecture (GBA) [8] [9]. This framework introduces two new functions called Bootstrapping Server Function (BSF) and Network Authentication Function (NAF). The BSF has access to the Home Subscriber Server (HSS) and can download user profiles and credentials. The NAF offers services to the User Equipment (UE). In order to use the service, the UE has first to authenticate on the BSF and the NAF can later check the UE authentication. This requires an implicit authentication at the network; our approach realizes the authentication explicit. The Sun JSR 281 specification [10] describes the IMS API that enables IMS applications to be installed on a JSR281-enabled device. The API assumes the existence of basic capabilities such as Push-to-Talk over Cellular (PoC), presence, instant messaging, etc. as well as the access to SIP headers and message bodies. Complementary to our approach, the use of JSR 281 implies the development and deployment of each new application on the terminal. Thus often CPU-intensive and complex orchestration of IMS enablers has to be done at the client’s terminal. In the EU FP6 project Simple Mobile Services (SMS) the authors propose to transport serialized messages coded in JSON (scripts) via SIP messages, taking advantage of the asynchronous character of SIP. This approach uses the signaling infrastructure (SIP proxies) to transport user data. Our approach uses the infrastructure only to establish a session and transports the user data point to point and reduces so the load on the SIP infrastructure. 1.2

Structure of This Paper

The rest of the paper is organized as follows: section 2 describes the proposed system architecture, section 3 explains the mechanisms to integrate bi-directional HTTP transport into a SIP session, section 4 explains the implementation of the SHIP concept on clients and servers and section 5 describes a design example for a typical SHIP based application. Section 6 concludes our work and gives further research directions.

2

System Architecture

In the following we assume that the environment for deploying our solution is that of an IMS service provider. First we introduce the SHIP concept and analyze the functionality required by the (mobile) terminal as well as on the server. Section 2.2 explains the integration of SHIP into an IMS overlay network. 2.1

Genaral Architecture

The basic concept of SHIP is to combine and manage HTTP sessions with SIP. A SIP INVITE message is used to negotiate a TCP channel for HTTP connections.


33

To achieve this, the media capabilities of a User Agent to handle HTTP requests are extended. A detailed explanation of the session setup is given in section 3. Of course it is still possible to establish additional media sessions (e.g. voice and video) within the same SIP dialog.

Fig. 1. SHIP architecture

Figure 1 gives an overview of the involved functions. It illustrates the SHIP terminal on the left hand side and a SHIP server on the right hand side. Both components are connected via a mobile IP network. A traditional User Agent for voice and audio calls, called ‘SIP basic services’ is depicted in Figure 1 . For these services a client uses both a SIP and a RTP stack. The SHIP terminal reuses the existing functionalities while adding additional capabilities. The SHIP logic uses SIP to establish a dialog that negotiates a TCP channel for HTTP traffic. The processing logic establishes, manages and terminates the SIP dialog for the SHIP session. It also ensures that each HTTP request is associated with a secret session key. The SHIP logic is the only component, which is aware of the 1:1 relation between a SIP dialog and a SHIP session. The Web front-end (e.g. a web browser or html rendering engine) uses the SHIP logic to send and receive HTTP messages and to display the HTTP part of the service. It is possible to access services which offer audio, video and data in any combination with only one user agent. This approach makes the SHIP architecture flexible for service developers and can be used to run the user interface at the user terminal, whereas service logic is being processed at the server.

34

J. Zeiß et al.

At the server side HTTP and RTP traffic as well as SIP signaling are terminated. The RTP media is being processed by the SIP basic services component and the HTTP traffic by the SHIP logic component. SIP signaling is terminated in one or both components, depending on the requested media type. Since the SIP signaling and the HTTP traffic of the client terminates at the server, the SHIP logic can correlate both sessions with a unique secret session key negotiated via SIP. The SHIP logic assigns the key to each incoming valid HTTP request and forwards the request to the application. The latter can be a valueadded service as depicted in Figure 1, or a proxy which forwards the request with the user identification to a 3rd party application in a trusted domain. A SHIP server consists of two main components the SIP Session Manager (SSM) and an HTTP proxy. The SSM is the endpoint for SIP messages sent by a client to establish a SHIP session. It stores a unique session ID for each SHIP session. Subsequent HTTP requests from the client’s terminal containing a valid session ID are forwarded by the HTTP proxy. The proxy optionally replaces the SHIP session ID with the ‘P-Asserted-Identity’ [11] header value to assure the identity of the requester to subsequent AS. 2.2

Integration of SHIP in IMS

This section shows a possible integration of SHIP into a mobile provider’s IMS. Figure 2 depicts the provider’s IMS core, the SHIP server and IMS service enablers, like presence, location or group management. The enablers provide beneficial IMS features, which can be easily used and combined by SHIP services without the need of deploying enabler specific software on the user terminals. SHIP clients and 3rd party providers can connect to the SHIP server. The IMS core is the point of contact for any SIP based communication from and to the end-user. SIP requests, addressing a SHIP service, received at the IMS Call Session Control Functions (CSCFs) 0 are routed to the SHIP server, by executing Initial Filter Criteria (IFC) or by resolving the provided SIP URI. The connection from the SHIP terminal to the IMS core and the SHIP server is provided by the PLMN using HSDPA, WLAN, etc. 3rd party content and service provider are connected via Internet. However, it is possible to build more sophisticated services using a SHIP terminal as shown in section 5. Similar to enabler services, 3rd party data and applications could be used to create valuable services. The existence and correctness of the ‘P-Asserted-Identity’ header provided by the IMS overlay network is essential for the functionality and authentication, because the involved components may authenticate the subscriber by using this header. To realize correctness, all involved SIP entities and HTTP entities which receive or forward SIP or HTTP requests, have to guarantee that this header is not manipulated and therefore they have to be in a domain of trust. In the case of IMS this could be the home network of the subscriber or just a trusted 3rd party service provider.


35

Fig. 2. SHIP in IMS

3

Managing the HTTP Session

SIP uses the Session Description Protocol (SDP) [12] to negotiate the parameters of a session like protocol, media type, IP addresses and port numbers. Media types in SDP are: video, audio, application and data, with UDP as transport protocol. Currently, it is not possible to specify HTTP interactions with SDP since HTTP relies on TCP as transport protocol. In [6] the authors specified an extension to SDP to enable the transport of fax data using SIP. For that purpose an extension is defined that expresses TCP data as a media type. According to [13] in which a TCP connection for transmitting a fax is announced via SDP (see Figure 3(a)) we propose a similar extension for HTTP sessions. If the calling party offers the SDP of Figure 3(a) and the called party answers with the SDP of Figure 3(b), then the called party would start a TCP connection form port 9 (IP address 192.0.2.1) to port 54111 (IP address 102.0.2.2). To handle a HTTP connection we propose the configuration depicted in Figure 4(a) and Figure 4(b). In our case media type m changes from image to application and protocol from t38 to http. Furthermore, we reuse the encryption key parameter k of the SDP to store a session key. For a User Agent, which is able to handle HTTP request and response messages, the associated SDP message includes the server and client part (Figure 5) to establish bi-directional HTTP data transport.

36

J. Zeiß et al.

(a)

(b)

Fig. 3. (a) TCP Client Part. (b) TCP Server Part.

(a)

(b)

Fig. 4. (a) HTTP Server Part. (b) HTTP Client Part.

Fig. 5. HTTP bi-directional

The encryption of the HTTP stream should be handled by SSL/TLS [14]. The SDP also transports a session key, to be added as custom header to each HTTP request. It maps the incoming HTTP requests to their corresponding SIP session. This mapping mechanism authenticates the HTTP request originator. An IP address and port inspection could also identify the user. Figure 6 shows the message flow of a SHIP session. The calling party acts as HTTP client and SIP User Agent, the called party as HTTP server and SIP User Agent. In this scenario the called party assigns a session key ‘key123’ to the SIP Call-ID ‘call1’. The called party sends the session key in the 200 OK SIP response to the calling party. The HTTP server maps all incoming HTTP requests with session key ‘key123’ to call-ID ‘call1’. The validity of a HTTP session is limited by the validity of the corresponding SIP session. If the SIP session is closed or no longer valid, the HTTP servers will not accept any request with key ‘key123’. Both parties are able to start additional video and audio streams using re-INVITE. It is also possible to add an HTTP stream to a video and audio stream, so that the multimedia session can cover audio, video and data (application).


37

Fig. 6. Authenticated HTTP Dialog within a SIP Session

4

System Implementation

Our implementation is based on the IMS Communicator (http:// imscommunicator.berlios.de/) and the SIP communication server from Oracle. Our extended version of the IMS Communicator performs the IMS registration, and the SIP session establishment. From the received SDP data, it takes the IP address, port and session key information and forwards it to the local running HTTP proxy, which realizes the SHIP logic in Figure 1. The local web browser is configured to communicate via this HTTP proxy. At the remote side the SIP server handles both SIP and HTTP. SIP functions are implemented as SIP Servlets [15], the HTTP part is implemented using HTTP Servlet technology. To achieve SHIP functionality the SIP Servlet as well as the HTTP Servlet share a common session object provided by the SIP server.

38

4.1

J. Zeiß et al.

Session Setup

When the user accesses a webpage (e.g. personalized access to subscribed services), the browser sends the request to the local HTTP proxy. If a SHIP session already exists, the proxy adds the corresponding session key to the HTTP request and forwards the request to the SIP communication server. If no SHIP session is running, the proxy triggers the IMS Communicator to establish the SIP dialog of the new SHIP session. For this purpose a SIP INVITE message is sent to a predefined SIP service URL including SDP data depicted in Figure 5. Initial Filter Criteria (IFC) evaluate the corresponding SIP server. The ’P-Asserted-Identity’ header assures the application server of the callers’ identity. The incoming SIP INVITE message triggers a SIP Servlet at the application server which generates the SHIP session key. This ID is stored for later usage by HTTP Servlets. A SIP 200 OK response message containing session key as well as HTTP server IP and port number is sent in reply to the INVITE. When the IMS Communicator receives the 200 OK it forwards the relevant SDP parameters to the HTTP proxy module. To setup the HTTP part of the SHIP session, the HTTP proxy takes the IP address and port number of the communication server and prepares a connection for later HTTP requests. Now, the SHIP session is valid and HTTP requests can flow in both directions. Incoming HTTP requests at the SHIP server trigger an HTTP Servlet which inspects the authentication header as well as the originating IP address and port. It compares the session key to the previously stored value. In case of matching session key values, the session key header is replaced by the P-Asserted-Identity header of the linked SHIP session. Next, the server forwards the request to its final destination. If there is no matching key, the Servlet rejects the HTTP request with an HTTP 401 error response. Since the application server correlates all incoming traffic with a certain session key to the identity of the called party and no further authentication is used to access value-added services, it might become a target for ‘man in the middle’ attacks. If one eavesdrops the HTTP (or SIP) traffic he can find out the session key and use it for service requests on behalf of the original called party. Therefore, it is required to use TLS [16] to secure the HTTP transport and IPSec [17] to secure SIP. Since the proposed mechanism is applied for each TCP connection, the HTTP performance deteriorates, since each request has to be processed sequentially. Most web browsers start several TCP connections to decrease the latency due to the processing of a webpage composed of several resources. To improve the performance, the proxy module of the IMS Communicator has been modified so that several TCP connections are established and are also accepted by the HTTP protocol stack within the SIP application server, without considering the TCP port announced earlier in the SDP of the IMS Communicator.

5

Application Example

This section discusses a typical SHIP based application example. The Multimedia Call Center presented is a rich call scenario where communication takes


39

place via voice and visual components derived from an associated HTML connection. Thus, we combine two different media types - data and voice - to one session, referred to as ‘rich call’ (optional video). In call centers there are often different experts for different concerns or questions. To forward calls to the right expert, Interactive Voice Response (IVR) is used. The system informs the caller, which key to press or which keyword to speak in order to proceed. The IVR system detects the selections and finally forwards the call to the corresponding expert. This approach reduces the number of call agents, who only categorize and forward calls. But there are also drawbacks. After listening to a long list of options people might not be able to recall their best choice, or cannot decide which option to choose. Some sorts of dialogues cannot be implemented using IVR systems at all, as there are too many options confusing the customer just by listening to them. To reduce this problem most IVR systems query the caller several times with small questions. But in general, complex, voice-only, IVR dialogues are hard to follow for humans. We believe that serving customer requests can be simplified, if graphical user interfaces are combined with voice communication. Callers are guided through the menus by visual input forms optionally in combination with voice instructions like ‘please assign a category to your question, if possible’. Indeed this approach does not require any voice recognition, because the choices made by the caller are transmitted as data to the call center. Therefore, this type of rich call applications, can realize much more complex dialogues as today’s IVRs. Processing the Personal Identification Number (PIN) or determining the right agent are just two possible examples. Figure 7 shows the architecture of a call center using SHIP. There is a group of call agents with partly different skills. The Call Center AS (SHIP server) terminates SIP and HTTP signaling as well voice data. It might access a database,

Fig. 7. Call Center with SHIP architecture

40

J. Zeiß et al.

Fig. 8. Callflow in a SHIP enabled Call Center

where account data are stored. For each SHIP caller the AS stores the service data (account data, selected question category, etc.), already entered by the caller. The call agent has access to these service data, as soon as he accepts the call. Figure 8 shows the call flow for this scenario. Alice sends a SIP INVITE message (1) addressed to the call center’s SIP URI. The IMS core routes the request to the application server. It is worth to mention, that the request includes two different media types inside the SDP part, one for audio and one for data (HTTP). If both media types are present, the AS detects the client’s SHIP capabilities. The AS agrees to the client’s connection parameters (2 & 3) and starts a session. Both, audio and data terminate at the AS. If the SDP data media type were missing - indicating that the client cannot handle HTTP - a voiceonly call would be established. Since both client and server are SHIP enabled, the client sends an HTTP request (4) associated to the previously established SHIP session to the server. The server responds with an HTML form (5) where Alice can enter her account data, and specify her concerns. In traditional IVR systems these data are transferred via voice. The major benefit of our solution is that the system assists Alice utilizing text and audio to fill up the form properly. The client sends Alice’s service data to the AS (6 & 7) which generates a ticket for this request and assigns this request to the correct group of call center agents. The way the AS assigns the call to a certain agent, is out of scope of this document. Before the AS sends an INVITE message (8) to an agent (Bob), it correlates the


41

Fig. 9. End-to-End data communication

call from Alice with the new one. Note that the correlation includes all service data of Alice which become accessible for Bob from now on. From the SIP and RTP perspective, the AS acts as a Back-2-Back User Agent (B2BUA), as defined in [2] Regarding HTTP the AS acts as an HTTP server, handling both parties within one joint session. This call center system requires that all call agents use SHIP enabled clients. Thus the call can be established with voice and HTTP media (9 & 10). Bob’s client is aware of the AS’ SHIP capabilities, because it received the corresponding SDP data media type within the INVITE request (8). As a result Bob’s call agent sends an HTTP request (11). The AS responds with Alice’s service data (12) (e.g. an HTML form with all account data and the recent bills). During this session, the associated HTTP connection supports the call center agent by displaying certain aspects of Alice’s bill on her mobile’s display. The following Figure 9 shows how Bob uses the data channel to display billing data at Alice’s SHIP client. For this reason he updates the session at the AS with a new HTTP request (1 & 2) (e.g. ‘show bill’). Alice’s client has to update its screen using AJAX or SHIP based HTTP push) (3 & 4). With this mechanism it is possible to push images and websites to the other party or to send data from the caller to the call agent.Parts of the functionality described above could also be implemented as a web application guided by voice (transferred over HTTP), in connection with a third party voice call. But our approach reuses the IMS infrastructure to achieve QoS, and to validate the identity of the users.

6

Concluding Remarks

In this paper we show a new way of approaching application development in IMS. The basic idea is to blend the SIP and HTTP protocols in order to achieve authenticated and authorized web based communication, while keeping the advantages of IMS (charging, QoS, roaming, usage of service enablers, etc.). Our

42

J. Zeiß et al.

first prototype implementation could proof the concept. For a large scale deployment, we will discuss the required changes for SDP standard within the respective standardization groups. Instead of executing a monolithic stand-alone application on the mobile terminal we propose to distribute the software between client and server following the SaaS model. In our application architecture GUI components are executed on the client side, whereas functional components, data processing and enabler orchestration reside on the application server. Interactions between terminal and server are entirely controlled by SIP. Bi-directional HTTP traffic to transport GUI and user events is managed by a SIP session. We reduced the protocols required to be supported by an IMS client to SIP (incl. RTP) and HTTP. There is no need for deploying enabler or application specific code on the mobile terminal. We discussed an application example demonstrating the capabilities of our approach, which is currently being implemented by the BACCARDI project (www.ftw.at). This project investigates future opportunities and enhancements of IMS and was initiated by the Telecommunication Research Center Vienna (ftw.) and its industrial partners. In the future we will compare the possibilities of introducing a HTTP push mechanism controlled by either SIP Subscribe/Notify messages or a Call-Id related SIP MESSAGE message. In both cases the client is informed about changes of previously downloaded content (occurring during the SIP session). Furthermore, we are looking at integrating our solution into currently available mobile widget engines such as the Symbian Web runtime (WRT) . The SHIP concept is not only a simplification of IMS protocol handling but opens new opportunities to service providers, service developers, network operators and users. Service providers can offer their Internet services to the community of IMS subscriber and reuse the IMS infrastructure for charging, QoS and authentication. Service developers can implement the services, as they did for the Internet community, without any special knowledge about deploying telco services. The network operators might profit from this approach, because they can sell more services, like those which are deployed for the Internet. Finally there is no need for the user to install and update service specific software. Acknowledgments. This work has been supported by the Austrian Government and the City of Vienna within the competence center program COMET. We thank our colleagues in the BACCARDI project, especially Joachim Fabini and Rudolf Pailer for the fruitful discussions and contributions.

References 1. 3GPP TS23.228, IP Multimedia Subsystem (IMS); Stage 2 (2009) 2. Rosenberg, J., et al.: SIP: Session Initiation Protocol, RFC 3261 (2002) 3. Rosenberg, J.: The Extensible Markup Language (XML) Configuration Access Protocol (XCAP), RFC 4825 (2007) 4. Campbell, B., Mahy, R., Jennings, C.: The Message Session Relay Protocol (MSRP), RFC 4975 (2007)


43

5. Fielding, R., et al.: Hypertext Transfer Protocol – HTTP/1.1, RFC 2616 (1999) 6. Gamma, E., Helm, R., Johnson, R., Vlissides, J.: Design Patterns. Addison-Wesley, Reading (1995) 7. Bennett, K., Layzell, P., Budgen, D., Bereton, P., Macauley, L.A., Munro, M.: Service-Based Software: The Future for Flexible Software. In: Asia-Pacific Software Engineering Conference, December 5-8 (2000) 8. 3GPP TS33.220, Generic Authentication Architecture (GAA); Generic bootstrapping architecture (2008) 9. Sher, M., Magedanz, T.: Secure Access to IP Multimedia Services Using Generic Bootstrapping Architecture (GBA) for 3G & Beyond Mobile Networks, Q2Swinet (2006) 10. Java Specification, IMS Services API JSR-281 (2006), http://jcp.org/en/jsr/detail?id=281 11. Jennings, C., Peterson, J., Watson, M.: Private Extensions to Session Initiation Protocol (SIP) for Asserted Identity within Trusted Networks, RFC 3325 (2002) 12. Handley, M., Jacobson, V.: SDP: Session Description Protocol, RFC 2327 (1998) 13. Yon, D., Camarillo, G.: TCP-Based Media Transport in the Session Description Protocol, RFC 4145 (2005) 14. Lennox, J.: Connection-Oriented Media Transport over the Transport Layer Security (TLS) Protocol in the Session Description Protocol (SDP), RFC 4572 (2006) 15. Java Specification, Java SIP Servlet Specification v1.1, JSR-289 (2008), http://jcp.org/en/jsr/detail?id=289 16. Dierkse, T., Allen, C.: The TLS protocol Version 1.0, RFC 2246 (1999) 17. Kent, S., Atkinson, R.: Security Architecture for the Internet Protocol, RFC 2401 (1998)

Efficient Authorization of Rich Presence Using Secure and Composed Web Services Li Li and Wu Chou Avaya Labs Research, Avaya Inc., 233 Mount Airy Road Basking Ridge, New Jersey 07920, U.S.A. {lli5,wuchou}@avaya.com

Abstract. This paper presents an extended Role-Based Access Control (RBAC) model for efficient authorization of rich presence using secure web services composed with an abstract presence data model. Following the information symmetry principle, the standard RBAC model is extended to support context sensitive social relations and cascaded authority. In conjunction with the extended RBAC model, we introduce an extensible presence architecture prototype using WS-Security and WS-Eventing to secure rich presence information exchanges based on PKI certificates. Applications and performance measurements of our presence system are presented to show that the proposed RBAC framework for presence and collaboration is well suited for real-time communication and collaboration. Keywords: Role-based aaccess control (RBAC), Presence authorization, Web services, security.

1 Introduction A presence system consists of a set of presentities and watchers who exchange presence information, either directly or through some presence servers. In communication and collaboration, presence information typically indicates a person’s availability and willingness to communicate. Presence may also include a person’s physical location and other environmental information. For example, many internet IM systems offer certain online presence information, such as “available, busy, and away.” But presence is not limited to persons, and it can be extended to any entity that can communicate. In general, presence information is exchanged between a “presentity” who discloses its presence states and a “watcher” who receives the presence states. To preserve the privacy, a presentity should have the ability to only disseminates presence information to the authorized watchers. To support this, the watchers are required to subscribe to the presentity before receiving the presence information. This subscription mechanism also allows both presentity and watcher to terminate presence exchange as appropriate. When used properly, presence information can significantly reduce the uncertainty and cost in traditional communication and collaboration in a way that is nonintrusive to the participants. It has been shown to improve social connections (keep in touch with family and friends) and collaboration within teams and organizations. For this J. Cordeiro and J. Filipe (Eds.): WEBIST 2009, LNBIP 45, pp. 44–57, 2010. © Springer-Verlag Berlin Heidelberg 2010

Efficient Authorization of Rich Presence Using Secure and Composed Web Services

45

reason, industrial standard organizations, e.g. IETF [6] and Parlay [15], are developing standard architectures and protocols for rich presence. Rich presence contains much more elaborated presence information than typical online presence. For example, Parlay X Web Service Presence specifies 10 presence attributes, including Activity, Place, Sphere, Privacy, etc. where each attribute can have 20 to 30 values. Some of the presence values disclose a presentity’s private life, such as if he is in a bar, prison or sleeping. Albeit being useful, rich presence information inevitably raises the privacy concerns about the presentity. Negatively correlated with presence, privacy is a highly contextual and subjective issue and must be dealt with carefully to achieve a balance between ease of collaboration and personal freedom. On one hand, individual presentities must be able to determine what presence information should be disclosed to which watcher under what context. For example, a person tends to expose different presence information to friends, parents, co-workers and managers. On the other hand, an organization often needs to enforce some central presence policies and leave the rest to the discretion of its members. For instance, a company may prohibit its employees from exposing location information to outsiders. An online community may mandate a minimal set of presence requirements and leave the rest to its members. To address these issues, a presence system should provide an intuitive, flexible and efficient method for presentities to authorize the disclosure of presence information based on their needs. Our observation is that the presence authorization to large extent is based on the social relations between watchers and presentites. A relation can be characterized as the role of a watcher from the perspective of a presentity. The notion of role is an intuitive concept, meaningful to people without technical background. For this reason, we adopt Role-Based Access Control (RBAC) and apply the NIST standard RBAC model [18] to presence authorization. RBAC models have been traditionally used in enterprises to control access to sensitive resources. RBAC is a natural fit for enterprises because the business functions are partitioned into roles that can be assumed by different people. The basic RBAC model assigns individual users to roles which are associated with permissions. Therefore, a user can only access resources permitted by the roles assigned to him. Roles can be organized into hierarchies in which the permissions can be inherited. Constraints can also be added to RBAC models to prevent undesired configurations. One such constraint is to enforce Separation of Duty (SOD) such that mutually exclusive roles are not assigned to the same user at the same time. RBAC thus provides a valuable level of abstraction and modularization of authorization policies based on the Least Privilege Principle that would be difficult to achieve by other access control models. However, the standard RBAC model needs to be augmented because presence authorization raises special issues that are not covered in the standard model. These issues are discussed below. First, the concept of role in presence authorization is different from the standard RBAC model. In presence system, roles represent relations between two users (watcher and presentity), whereas in the standard RBAC model they represent functions of one user within an enterprise. This implies that role assignment in presence system depends on two users, instead of one as in the standard RBAC model. Furthermore, in

46

L. Li and W. Chou

presence system, the role assignment may vary depending on other contextual factors, such as the time and nature of the engagement, whereas in enterprises the role assignments do not change very often. Second, the standard RBAC model does not support cascaded authorization, where a central authority establishes common authorization rules and its subordinates can refine them. Cascaded authority is necessary to assert certain global control while permitting personal choices in organizational and social environment. Third, the standard RBAC model does not define the user authentication that is critical for secure presence systems. Without authentication, a watcher’s true identity cannot be established and role assignments based on false identity would grant the watcher unauthorized access to presence information. Forth, the standard RBAC model does not define permission representation that has special importance in presence systems. Under the information symmetry principle, the internal permissions of a presentity should match the external presence information exposed to the watchers to eliminate potential information gaps. To address the first issue, we extended the standard RBAC model by incorporating social relation and context into role assignment. For the second issue, we add new features and constraints for role hierarchy to support cascaded authority. To resolve the third issue, we adopt the PKI security protocol for user authentication. On the fourth issue, we propose an abstract presence data model based upon Parlay X Web Service Presence and RFC5025 authorization [17] to ensure data integrity. Besides privacy issues discussed above, another important issue in presence systems is the interoperability between heterogeneous watchers and presentities. To tackle this issue, we adopt the web service architecture by abstracting each watcher and presentity as a set of web services. In particular, we adopt WS-Eventing [23] and model presentities as event sources and watchers as event sinks. The presence updates are conveyed as asynchronous events that a watcher can subscribe. Notice a web service endpoint can be both a presentity and watcher. The proposed presence data model is used to extend the generic WS-Eventing services into a presence protocol to reduce the running services on the endpoint. WS-Eventing is also composed with WSSecurity [22] to ensure end-to-end message authenticity, integrity and confidentiality. The rest of the paper is organized as follows. In Section 2, we survey some related work in RBAC, presence authorization and web services. In section 3, we outline the web service architecture for presence in which the RBAC model operates. Section 4 presents the presence data model and its implications. Section 5 discusses the extensions to the standard RBAC model and relevant security mechanisms. In section 6, we discuss a prototype presence architecture based on secure web services using WSSecurity with PKI and UDDI servers. Within this architecture, we study the performance and applications of the proposed presence system for web services enabled softphones. We conclude in section 7 with some thoughts on future work.

2 Related Work Privacy aware design has been an active research area in recent years [9][12][11][10]. A recurring principle in these researches is “information symmetry” such that parties participating in information exchange should have equal awareness and control over the information flows.


47

The concept of role has shown up in several presence architectures, but they fall short of formally adopting RBAC models. Hong et al [9] proposes an analytical privacy risk model which considers the roles and social relationships between presentity and watcher as one of the input parameters to the model. The authors recommend “push” mode delivery of presence information to lower privacy risks, because the presentity initiates the data transmission. Lederer et al [12] discusses five pitfalls in privacy aware design based on their experience of developing a GUI for setting presence preferences. A preference rule has three parts: a role, situations and a face, meaning that if someone of this role (roommate, parent, boss) requests in these situations, it should show the presence states defined in the face. This clearly is a RBAC based approach, but the author does not make any reference to RBAC. Their usability studies showed that users expressed discomfort with hiding presences behind faces, which is a permission definition issue in RBAC models. The general lessons are that the privacy component should be simple, transparent, intuitive and easy to change. The paper recommends the practice of plausible deniability – people can choose to ignore presence requests or customize information disclosure without having to explaining why. Jorns [10] provides an overview of location based systems in the face of privacy and proposes a pseudonym based scheme to enhance privacy in a mobile communication system. The scheme is based on “symmetric information data arrangement”, where the presentity has control over to whom the location information is disclosed using pseudonym management. This is in contrast to “asymmetric arrangement” where a presentity has no such control. Hengartner et al [21] proposes a digital certificate based access control framework for sharing personal location information in a heterogeneous environment. The authorization is achieved by checking the trust delegation relations expressed in certificates. They also use “local names” to designate a group of people in a single certificate instead of creating certificates for each person. Local names can be regarded as roles in our RBAC approach. RBAC model has been standardized by NIST [18], and it is still a very active research area for improvements [3][13][14][24]. Traditionally, it is used for enterprise resource protection. There are some recent efforts to extend RBAC to incorporate privacy [13][14] and geospatial and temporal factors [3]. On one hand, however, these generic models are too complex for presence, and on the other, they do not address the specific issues in presence systems. For instance, the privacy aware RBAC model [14] does not consider the relational information in role assignment. Some of the principles followed by this model, such as Purpose Specification Principle and Use Limitation Principle, are difficult to enforce in a presence system, because once a presence event is transmitted to a watcher (a person), the presence system has no control over what the person can do with that information. If we were to enforce these principles to limit the purposes and usage of presence information, the presence system would be less useful, since a person will not be able to review and retransmit the received presence information at will. Furthermore, the data integrity or cascaded authority in web service based presence systems are not addressed in these models. For this reason, we choose to start with the standard NIST RBAC model.

48

L. Li and W. Chou

Godefroid et al [8] developed a framework to ensure the correctness of presence authorization rules using model checking techniques. Each rule has a condition and an action. Conditions are defined in terms of the presentity’s state, such as “user i’s door is open” and the action includes invite, accept, reject, etc. However, these rules are not based on RBAC models. There are many research work based on the SIP presence framework [16][17][2][19]. IETF RFC5025 [17] defines presence authorization rules for SIP Instant Messaging and Presence (SIMPLE). This specification is based on a three level presence data model. The top level consists of three entities: device, service and person. The second level is the attributes: activity, place, mood, etc., and the third level is a range of presence values. The authorization rule consists of condition, action and transformations, and it is applied to each presence subscription. The condition checks the identity of the watcher and the sphere of the presentity. If the condition matches, the action and transformations are carried out. The actions (block, confirm, polite-block and allow) determine the status of the subscription. The transformations define which presence entities and attributes will be included. However, the authorization rules of [17] are user centric instead of role based and the transformations do not apply to presence values. RFC3856 [16], the protocol for SIP presence subscription, does not provide a mechanism for a watcher to indicate the presence preference in subscriptions and to know the authorized presence attributes. This information asymmetry may lead to information overload to the watchers. In particular, Singh et al [19] outlines some authentication mechanisms within SIP architecture, including using PKI certificates. Parlay X Presence Web Service [15] defines a two level presence data model. The top level is a set of attribute types and the second level is a set of values about the presence state of a person. The presence web service includes interfaces for watchers to subscribe or poll presence data. Similar to SIMPLE, a watcher can request a subset of presence attributes, and the presentity can grant permissions (yes or no) to the requested attributes. When the presentity changes the authorization of an existing presence subscription, the service will notify watchers accordingly. However, the presence protocol is coupled with the presence data model, making reusing them difficult. In this standard, neither watcher nor presentity can select presence values from the attributes. Also, the standard does not specify a standard presence authorization process. WS-Security [22] is an OASIS standard for securing end-to-end web services interactions. It provides support for user name tokens and password, message timestamp, digital signature of message, and message encryption based on a variety of cartographical schemes, including PKI architecture. WS-Eventing [23] is a W3C submission and in the process of becoming a W3C standard recommendation. It defines a subscribe/notify web service interaction pattern for managing event subscriptions between event sources and event sinks. It has been applied in web service oriented communications as shown in [4][5]. UDDI [20] is another OASIS standard for web services to publish their structural information so that services can be discovered and consumed. A UDDI repository at the minimum contains the name and URI of web services, but it is extensible to store other type of information.


49

Fig. 1. High level Web service based presence protocol

3 Outline of Web Service Based Presence System In our web services based presence framework, watchers and presentities interact over the Web though the web service protocols. The high level interaction sequence that sets up presence exchange is depicted in the following diagram (Fig. 1). In this architecture, a presentity s first publishes its web services onto the Web (UDDI registry or Web server). A watcher w interested in s can discover the presentity’s presence data model d over the Web, subject to the authorization approval. To obtain the presence updates, the watcher w subscribes to d with a request q indicating the desired presence information. The request q is then subject to the authorization process. Upon approval, a filter f is returned along with the current presence states v of s and a reference m to the subscription. Subsequent presence event from s will be delivered to w only if it matches f. Either s or w can cancel the subscription m to terminate the presence session. This sequence of operations is the same regardless if a presence server is deployed between w and s or there are peer-to-peer connections between them. For the same reason, the RBAC models can be deployed either in the presence server or in the presentity or both. If a presence server is used, the presentity can delegate part or all authorization to the server. For clarity, this diagram only illustrates the deployment of RBAC model on the presentity. Regardless where the RBAC model is installed, the authorization process will be the same.

4 Presence Data Model The presence data model is a tree of finite depth. The schema of this model is depicted by the following diagram (Fig. 2), where the root node represents the presentity and the children represent the entities (device, service, person, etc.), attributes (activity, place, mood, etc.), and values (busy, work, happy, etc.). A presence data model is used for three purposes: 1) it is published as part of the presence web services to indicate what presence events are available at a presentity; 2) it is used by RBAC model to define presence authorization; 3) it is used to design an efficient filter dialect to filter presence events for both watchers and presentities.

50

L. Li and W. Chou

Fig. 2. Schema of the tree based presence data model

Fig. 3. The dependences on the presence data model in a presence system

To illustrate these points, the following (UML) diagram (Fig. 3) depicts the relations between the presence data model and its stake holders in the presence system. These dependences ensure that all stake holders use the same presence data model. Without such data integrity, a RBAC model’s permission may be out of sync with the published presence data. For the presentity, it unwillingly leaks unauthorized presence data when the permitted data is more than the published one. For the watcher, it leads to frustration as published presence is never authorized when the published data is more than the permitted one. By using the same presence data model in the external presence web services and the internal RBAC model, it achieves data integrity within the presence system. When shared, the presence data model provides the same view on presence information to both presentities and watchers. This shared view increases symmetric information flow for privacy protection, because the watchers and presentities have equal knowledge and control over presence information dissemination. Furthermore, the presence data model decouples the logic of the RBAC model and the presence protocol from the content of presence data models. As a result, the same logic applies to different presence models. This flexibility enables a presence system to support one or more presence data models. For example, one presence system may elect one presence data model based on Parlay X Presence Web Service, another based on SIPMLE, and a third one based on the union of both. Similarly, a presence data model can be profiled by any presentity to represent a subset of presence information he is willing to expose. Moreover, the presence data model allows us to use a generic WS-Eventing service instead of more specialized Parlay X Presence Web Service as the presence protocol. By extending WS-Eventing, we decouple the presence data model from the presence protocol. At one hand, we can reuse WS-Eventing for different purposes by defining different extensions. On the other hand, we can enhance or change the presence protocol without changing the presence data model. Such approach leads to a very small,


51

efficient, extensible and modular presence system that provides a generic methodology for presence authorization in collaboration and social networks.

5 Extended RBAC Model Our extended RBAC model M consists of the following sets of entities: presentities (S), watchers (W), contexts (C), roles (R), presence data model (D), permission actions (A), and data model transformations (T). The sets S and W correspond to the users (U) of the standard RBAC model. The sets P and R have the same meaning as in the standard RBAC model. The following additional entities are introduced in our extended RBAC model. The set C denotes a set of application dependent contexts. The set D is a set of presence data models published by presentities in S. The set T is a set of transformations that decorates trees in D. These decorated trees are called “permission tree” because the nodes are decorated with permission actions defined in the set A [17], e.g. allow (A), block (B), polite-block (L) and confirm (I). In permission trees, a node without action will inherit its parent’s action, whereas a node’s own action overrides its parent’s action. The authorization specification of M is defined by the following formulas. W × S × C→[Role]→R→[Granted]→ C × T(D)

(1)

f(w, s, c) = Requested(w, s, c) ∩ Granted(Role(w, s, c))

(2)

Received(w, s, c)=f(w, s, c) ∩Presence(s))

(3)

Formula 1 specifies the RBAC model by mapping a pair of watcher and presentity in a context into a role. Each role is associated with a permission tree under a context. The key of authorization is to calculate the filter predicate f, which is given by Formula 2. Here Requested(w,s,c) denotes the requested presence information from watcher w to presentity s in context c. This predicate is represented as a special filter (permission tree) in WS-Eventing subscribe request. The filter predicate f for this request is the intersection of the request and the granted permission derived from Formula 1. Formula 3 shows that the presence events from s are filtered by f before received by w. To elaborate the authorization process, Fig. 4 shows that a watcher w, subscribing certain presence data represented as a tree q, is assigned to role r. The role r is associated with a permission tree t decorated with some actions. In this case, the watcher w requests presence values v11 and v12 of the attribute a1 and all values of a2, whereas the permission tree grants access to values v11 of the attribute a1 and all values of a2 only if it is confirmed by s. Assuming the presentity rejects attribute a2, then we have f={a1/v11}. If there is a presence event {a1/v11,a1/v12}, only {a1/v11} will be delivered to the watcher after the filtering. The calculated filter f is stored with the presence subscription on the presentity side and sent back to the watcher in the WS-Eventing subscribe response as well. By reviewing the filter, the presentity knows what presence will be disclosed and the watcher knows what presence will be received. In other words, the shared filter

52

L. Li and W. Chou

Fig. 4. A RBAC model with watcher request and role permission represented as trees created from the same data model

guarantees that no presence will ever leak to the watcher without the consensus of the presentity. By the same token, no granted presence will be concealed from the watcher. The only exception to this rule is polite-block, where the presentity hides its presence from the watcher. However, this is a well accepted privacy practice following the Plausible Deniability Principle [12] that favors privacy over presence. This extended RBAC model is particularly suitable for creating cascaded authorities using role hierarchy. Our goal is, given a RBAC model M1 representing a central authority, to derive a new model M2 that maintains all the mandatory permissions of M1. To support this, we propose new constraints on standard role hierarchy to prevent the mandatory permissions from being overridden. Since permission is domain dependent, we introduce a new attribute, final (F), to decorate the permission trees. This attribute separates nodes that must be inherited by M2 from those that can be overridden, thus providing protection as well as flexibility in cascaded models. More formally, let M1 = (R1, C1, T1(D1), A1) and M2 = (R2, C2, T2(D2), A2), representing two RBAC models. M2 is a valid derivation of M1 iff the following conditions hold:

D2 ⊆ D1 ∧ A 2 ⊆ A1 ∧ C2 ⊆ C1

∀ (c , t2 ) ∈ C

2

(4)

× T 2 ( D 2 ) ∃ ( c , t1 ) ∈ C 1 × T 1 ( D 1 )

( d ( t1 ) = d ( t 2 ) → ( r o l e ( c , t1 ) ∈ j u n i o r ( r o l e ( c , t 2 )

(5)

∧ f i n a l ( t1 ) ∈ t 2 ) ) By the first condition, the presence data models, actions and contexts of M2 are sanctioned by M1. The second condition ensures that if a tree t2 in M2 has the same data model as a tree t1 in M1, then t2’s role inherits t1’s role and all final nodes of t1, if any, are inherited by t2 as well. The following diagram (Fig. 5) illustrates two models: M1 with role manager and M2 with role director that satisfy these conditions. In this case, M1 marks a1:A (allow attribute a1) as final but leaves a2 open. Model M2 changes a2 from I (confirm) to A (allow), and adds a new node a3. If t2 had a node a1:B (block attribute a1), it would violate the conditions. This derivation process can be repeated as long as necessary.


53

Fig. 5. Cascaded RBAC models using constrained role hierarchy

This role hierarchy can be interpreted by standard RBAC semantics, as either permission hierarchy or activation hierarchy [18]. To enable efficient computation of permissions, the cascaded models can be flattened into one final RBAC model whose permission trees are the unions of all permission trees along the inheritance relations. The permission tree of a senior role will inherit nodes from the junior role if it does not have them, and it will overwrite nodes from the junior role by its own. Following this rule, the collapsed tree for the role director in M2 is {a1:A, a2:A, a3:I}. As we pointed out before, the watchers must be authenticated before being authorized by the RBAC model. Similarly, the watchers need to ensure that the presence data is indeed from the presentity. Because the watchers and presentities are two independent endpoints, we adopt the WS-Security PKI security protocols. Assume that each watcher and presentity has a pair of private and public keys issued by some Certificate Authority (CA) that they both trust. A watcher w can sign and encrypt a presence subscription message to presentity s as follows: C = encrypt(sign(M, private_key(w)), public_key(s)) Once presentity s receives this message, it decrypts and verifies the message as follows: M = verify(decrypt(C, private_key(s)), public_key(w)) By verifying the signature, presentity s can check if the message is indeed coming from watcher w. After w is authenticated, it is subject to the RBAC model for authorization process. By encrypting the message, watcher w ensures that only s can understand the message. The same security process can be applied to any message exchanged, to ensure the complete message authenticity, integrity and confidentiality in the presence system. It should be pointed out that WS-Security ensures end-to-end security that is agnostic of transport protocols. This level of security allows sensitive presence information to be passed between value-add 3rd party applications without sacrificing message integrity and confidentiality.

6 Implementation Our prototype presence system consists of a UDDI server (jUDDI), a Certificate Authority (CA) server (EJBCA running on JBoss), presentities and watchers. Each presentity and watcher in our system is a softphone hosting an identical set of web

54

L. Li and W. Chou

Sender plain SOAP Message

plain SOAP Message

private key

WS-Security Interceptor

Add Timestamp, user name, signature, encryption

decryption, verification

secure SOAP Message

secure SOAP Message

Receiver

public key

UDDI

Fig. 6. SOAP message processing using WS-Security

services (WS-Eventing and WS-Security). PKI certificates (public and private keys) issued by the CA are administrated to each softphone. Each softphone publishes or deletes its web services and public key in the UDDI server, when the user logs in or off the softphone. The UDDI server thus acts as an on/off presence server and a public key repository. To start a presence subscription, the watcher’s softphone finds the presentity web service from the UDDI and invokes them accordingly. We developed WS-Eventing and utilized the WS-Security API provided by gSOAP [7] to support the security features. Fig. 6 illustrates how the WS-Security interceptor processes the incoming and outgoing SOAP messages. The web services were deployed to the Google Android mobile phone platform [1], on which we also developed the presence related GUI for user interactions. Our prototype system has four roles: manager, peer, subordinate and anonymous, with some predefined permissions against the Parlay X Presence. To demonstrate the approach, four presence attributes: Activity, Privacy, Place and Sphere, of the Parlay X Presence data model are selected. The role anonymous, inherited by other roles, grants everyone access to the presence data model and some rudimentary presence information. Fig. 7 shows screenshots of the presence subscription GUI and the contact list GUI with presence information, as seen on the softphones. The presence information is combined with the call management of the softphone to enable presence aware communication. One application is presence aware call that can put an active call on hold when it is not answered and then it automatically reconnects when the callee’s presence state changes to available. This avoids the need for the caller to frequently poll the callee with phone calls, which makes the life easier for the caller and reduces unnecessary network traffic as well. The performance measurement of the presence system is summarized in the following table (Table 1). The time (millisecond) were averaged over 10 to 100 trials on a notebook computer with 1.79GHz CPU and 1G RAM running Windows XP Professional in a LAN environment. The “Watcher” column is the round-trip latency of presence subscription messages measured at the watcher side, with and without WSSecurity. Similarly, the “Presentity” column is the total subscription processing time on the presentity side, with and without WS-Security (the gSOAP version we used did


55

Table 1. Performance measurement of the prototype in millisecond (ms)

Component with WS-Security no WS-Security WS-Eventing

Watcher 20.35 17.27 N/A

Presentity 7.45 5.77 4.76

Fig. 7. Screenshot of presence subscription interface used by the watchers (left) and the screenshot of a watcher’s contact list showing the presence state of his contact (right)

not implement encryption so the process only included username token, timestamp and digital signature). The total processing time included SOAP engine and WS-Eventing module. The “WS-Eventing” was the time spent in executing the WS-Eventing code, which included the presence authorization. These results show that the overall performance is acceptable for real-time telecommunication.

7 Conclusions This paper presented an extended RBAC model for presence authorization and a presence architecture using this model with secure web services for privacy protection. We proposed a data centric approach and showed that it is capable of decoupling web service protocols, authorization process and presence information while ensuring the data integrity and symmetric information flow in presence systems. In this approach, the standard RBAC model is extended in two ways to address the needs of web services based presence systems. The first is to incorporate social relation, context and presence data model to support efficient presence authorization process. The second is to introduce the constraints for cascading RBAC models using role hierarchy to support central authority. A prototype presence system was implemented consisting of UDDI registry, PKI CA server, and web services enabled softphones running WS-Eventing

56

L. Li and W. Chou

and WS-Security. Applications of the proposed presence architecture to enhance communication were also demonstrated. The experimental results indicated that the system performance is well suited for real-time communication and collaboration. For future work, we will study the combination of RBAC models with the context of enterprise organizations and social networks, to enable presence aware communication and collaboration efficiently without loosing privacy. Acknowledgements. The authors would like to thank Wei Zhang and Yanbing Yu for their contributions to this project.

References 1. Android - An Open Handset Alliance Project, http://code.google.com/android/ 2. Beltran, V., Paradells, J.: Middleware-Based Solution to Offer Mobile Presence Services. In: Mobileware 2008 (February 2008) 3. Chen, L., Crampton, J.: On Spatio-Temporal Constraints and Inheritance in Role-Based Access Control. In: ASIACCS 2008, March 2008, pp. 205–216 (2008) 4. Chou, W., Li, L., Liu, F.: Web Services Methods for Communication over IP. In: ICWS 2007, Salt Lake City, July 2007, pp. 372–379 (2007) 5. Chou, W., Li, L.: WIPdroid – a two-way web services and real-time communication enabled mobile computing platform for distributed services computing. In: Proceedings of International Conference on Services Computing 2008, July 2008, vol. 2, pp. 205–212 (2008) 6. Day, M., Rosenberg, J., Sugano, H.: A Model for Presence and Instant Messaging. RFC 2778 (February 2000) 7. gSOAP, http://gsoap2.sourceforge.net/ 8. Godefroid, P., Herbsleb, J.D., Jagadeesan, L.J., Li, D.: Ensuring Privacy in Presence Awareness Systems: An Automated Verification Approach. In: Proceedings of the 2000 ACM conference on Computer supported cooperative work, pp. 59–68 (2000) 9. Hong, J.I., Ng, J.D., Ledere, S., Landay, J.A.: Privacy Risk odels for Designing PrivacySensitive Ubiquitous Computing Systems. In: DIS 2004, August 1-4, pp. 91–100 (2004) 10. Jorns, O.: Privacy Enhancing Architectures Overview. In: Intensive Program on Information and Communication Security: Secure Embedded Systems (IPICS 2004), November 25 (2004) 11. Langheinrich, M.: Privacy by Design – Principles of Privacy-Aware Ubiquitous Systems. In: Proceedings of the 3rd international conference on Ubiquitous Computing, pp. 273–291 (2001) 12. Lederer, S., Hong, J.I., Dey, A.K., Landay, J.A.: Personal privacy through understanding and action: five pitfalls for designers. Personal and Ubiquitous Computing 8(6), 440–454 (2004) 13. Ni, Q., Trombetta, A.: Privacy-aware Role Based Access Control. In: SACMAT 2007, June 2007, pp. 41–50 (2007) 14. Ni, Q., et al.: Conditional Privacy-Aware Role Based Access Control. In: Biskup, J., López, J. (eds.) ESORICS 2007. LNCS, vol. 4734, pp. 72–89. Springer, Heidelberg (2007) 15. Parlay, X.: Draft ETSI ES 202 391-14 v0.0.8 (2007-06), Open Service Access (OSA), Parlay X Web Services, Part 14: Presence, Parlay X 2 (2007)


57

16. Rosenberg, J.: Request for Comments: 3856, A Presence Event Package for the Session Initiation Protocol (SIP) (August 2004) 17. Rosenberg, J.: Request for Comments: 5025, Presence Authorization Rules (December 2007) 18. Sandhu, R., Ferraiolo, D., Kuhn, R.: The NIST Model for Role-Based Access Control: Towards A Unified Standard. In: Proceedings of 5th ACM Workshop on Role Based Access Control, July 26-27 (2000) 19. Singh, V.K., Schulzrinne, H.: A Survey of Security Issues and Solutions in Presence (2006), http://www1.cs.columbia.edu/~vs2140/presence/ presencesecurity.pdf 20. UDDI Version 2.04 API Specification, UDDI Committee Specification (July 19, 2002) 21. Hengartner, U., Steenkiste, P.: Implementing Access Control to People Location Information. In: SACMAT 2004, pp. 11–20 (June 2004) 22. Web Services Security: SOAP Message Security 1.1 (WS-Security 2004), OASIS Standard Specification, February 1 (2006) 23. Web Services Eventing (WS-Eventing), W3C Member Submission, March 15 (2006) 24. Zhang, Y., Joshi, J.B.D.: UAQ: A Framework for User Authorization Query Processing in RBAC extended with Hybrid Hierarchy and Constraints. In: SACMAT 2008, pp. 83–91 (June 2008)

Part II Web Interfaces and Applications

Information Supply of Related Papers from the Web for Scholarly e-Community Muhammad Tanvir Afzal Institute for Information Systems and Computer Media Graz University of Technology, Infelldgasse 16c, 8010 Graz, Austria [email protected]

Abstract. Current search engines require the explicit specification of queries in retrieving related materials. Based on personalized information acquired over time, such retrieval systems aggregate or approximate the intent of users. In this case, an aggregated user profile is often constructed, with minimal application of context-specific information. This paper describes the design and realization of the idea of ‘Links into the Future’ for discovering related documents from the Web, within the context of an electronic journal. The information captured based on an individual’s current activity is applied for discovering relevant information along a temporal domain. This information is further pushed directly to the users’ local contexts. This paper as such presents a framework for the characterization and discovery of highly relevant documents. Keywords: Information supply, Links into the Future, Personalized search, Digital journals.

1 Introduction As the number of electronic publications expands, acquiring relevant information to suit particular user needs becomes a major challenge. The searching for related or relevant material has to duly consider the task at hand. This research then focuses on the identification of relevant documents within a scholarly publishing environment. As opposed to the retrieval of a million hits as provided by generic search engines, we explore the direct provision of relevant information only. Andrei Broder, Yahoo’s Vice-President and research fellow, elaborated information supply paradigm in his talk at Stanford [5]. He explained that the future of search will undergo a shift from information retrieval to information supply. Users will be provided the most relevant information according to their current need. The information discovery in such a systems will be based on users’ local context and tasks at hand. In the context of information supply paradigm, we propose a feature called Links into the Future. When a user is looking on a particular content on the Web, this feature provides users with the most relevant information that has been made available on the Web after the publication date of the focused content. This can be realized in different contexts and scenarios. For example a user is reading particular news; Links J. Cordeiro and J. Filipe (Eds.): WEBIST 2009, LNBIP 45, pp. 61–72, 2010. © Springer-Verlag Berlin Heidelberg 2010

62

M.T. Afzal

into the Future provides the user with the relevant news that were made available afterwards. The discovered news may contain other news articles containing positive or negative sentiment about the focused news. If a user is reading a book, Links into the Future provides user with the books that are the extended version of the focused book, it may also provide positive and negative reviews about the focused book from different sources. In the current paper, we are talking about Links into the Future in the domain of digital journals. If a user is reading a research paper, the user is provided with the most relevant research papers that were published in the future dates. A related document could mean different things to different people; e.g. it could be a paper written in the same area, one written by the same author, or merely taking about the same research problem. We use the term ‘Links into the Future’ in this paper to describe the discovery of related papers based on the information compiled from a user context while reading a paper within an electronic journal. For a reader of an electronic journal, ‘Links into the Future’ refers to potentially useful papers to the one that he or she is reading. This idea was originally proposed by Hermann Maurer in a presentation entitled “Beyond Digital Libraries” at the “Global Digital Library Development in the New Millennium” in Beijing, China [11] and was subsequently partially realized [10]. In this paper, we consider a paper written by the same author or team of authors at a later time (then the current paper) to be related and useful, particularly if it was also in the same area of expertise. At the same time, any paper that cited this paper is also considered related and relevant. A person reading a paper published in 1995 may not directly know if the same author has published a newer paper in year 2000 in the same area or if anyone has cited this paper. Well, the reader can of course, search the Web to look for similar papers. In this case, the reader may then loose the original context, and also get distracted by the millions of hits from global search engines. There are also times when particular related items may not be found as the formulation of exact query terms can in itself be a challenge. Alternatively, the reader has an option to explore citation indices such as CiteSeer1 CiteULike2 and DBLP3 to search for related papers. This will also require a deliberate effort on the part of the reader. Additionally, results from multiple sources of information will then need to be compiled and consolidated. The implementation of such a feature within the context of the Journal of Universal Computer Science (J.UCS)4 was described in [3]. This paper then explores the extension of ‘Links into the Future’ for papers published beyond the closed database of J.UCS. The discovery of related links within a closed database has only partially addressed user concerns, in providing starting points for further exploration. By extending the notion of Links into the Future to also cover relevant papers from the Web, a more comprehensive solution can be obtained. The links discovered as such will also be more reflective of the state-of-the-art in this field. J.UCS, an electronic journal covering all areas of Computer Science with more than 1,200 peer reviewed research contributions, serves as an excellent platform for the 1

http://citeseerx.ist.psu.edu/ http://www.citeulike.org/ 3 http://www.informatik.uni-trier.de/~ley/db/ 4 http://www.jucs.org 2

Information Supply of Related Papers from the Web for Scholarly e-Community

63

discovery of related papers on the Web. Papers within J.UCS are categorized and tagged according to the ACM categories5. A number of additional categories have been recently added to J.UCS to reflect the development in Computer Science discipline. These extended categories would be referred to as topics in the rest of the paper.

2 Definition of Links into the Future A future link from paper “a” to paper “b” (Future_Link (a,b)) exists, if a semantic relationship can be established between them. For example: if paper “b” is written by the same team of authors of paper “a” and the topics of both papers are similar, then paper ‘b’ is considered to be related to paper ‘a’. Alternatively if there exists a citation from paper b to a, there is a highly likely relationship. Current systems tend to perform similarity matches without considering semantic similarity, based on the task characteristics. The equation 1 describes the definition of “Links into the Future”. (Authors (b) €€ Authors (a)

Topics (b) €€ Topics (a)) Future_Link (a, b)

Citation (b, a)

(1)

3 Related Work Most related work explores the servicing of users within a present-context, making use of limited information, captured in-vivo. Our work, on the other hand, describes the augmentation and annotation of documents created in the past with information that became available later. In this way, a research paper is not seen as a static document, but rather one that is constantly updated and kept up-to-date with relevant links. A number of past studies make use of user’s context and activity to provide them the most relevant information. For example, typical search engines return relevant results based on the small amount of information from user queries and a measure of web site popularity, rather than considering individual user interest and context [16], 2005). Spretta & Gauch employed user profiles based on user queries, search activities and the snippets of each examined result to refine search result rankings. With this context specific ranking of search results, an improvement of 34% in the rank order has been obtained. Rhodes and Maes [15] described a new class of software agents called Just-inTime Information Retrieval Agents (JITIRs), which has an ability to proactively present potentially valuable information based on a person’s local context in a nonintrusive manner. Another related work pushes the most relevant Web URLs based on the user activity and context. User context is determined by examining active personal desktop documents [6]. Similarly by observing user activity and context while reading a particular article, our notion of ‘Links into the Future’ presents the most related papers of the same team of authors within a local context. This paper discus how this concept can be extended to the WWW as a mechanism for contextual information supply for academic publications along a temporal dimension. 5

http://www.acm.org/class/1998/

64

M.T. Afzal

Existing approaches for finding related papers uses citation analysis, text similarity, bibliographic analysis and context based relatedness. For example, CiteSeer has employed three methods for finding related papers a) word vectors b) string distance 6 c) and citations [9]. PubMed on the other hand computes the relatedness between two papers using text-based similarities between paper titles, abstracts, and assigned 7 MeSH terms . For the focused paper, PubMed provides a list of related papers according to their relatedness scores. Ratprasartporn [14] have made use of context (topics) of research publications to determine the related papers. An ontology consisting of terms were utilized as a context of publications. A publication is assigned to one or more contexts with the context represents the publication topics. Digital libraries are traditionally built largely by a massive human effort. Example of these include INSPEC for engineering, PubMed for medicine and DBLP for Computer Science. Alternatively automated approaches are being employed to construct large citation indices. Examples of these efforts include CiteSeer and Google Scholar. The limitations of these automatic approaches are that human effort is often required in verifying entries in the index. Fully automated techniques have problems in disambiguating entries while traditional constructed digital libraries are limited in their number of scientific publication. Google Scholar has mistakenly identified name of places as authors of scientific publications [12]. Although Google Scholar has improved gradually, it continues to 8 find citations backward in time . Their index covers a large collection of peer9 reviewed papers . It however also considers false positive citations like citations to press releases, resumes, and links to bibliographic records for cookbooks [13]. CiteSeer claims that 80% of the publications titles can be extracted accurately while their word and phrase matching algorithm further has an error margin of 7.7% [9]. Our system is fully automated in extracting papers from the Web and from citations. It also computes a conceptually enhanced similarity score between a source paper and candidate future papers.

4 Identifying Future Links from Web The APIs for Google10 , Yahoo11 and MSN Live12 were used for the experiments13. The identification of future links from the web includes the following steps: query formulation, removing duplicates, filtering papers only, similarity algorithm and determining future links. The description of each is shown in Figure 1. 6

PubMed, http://www.ncbi.nih.gov/entrez/query.fcgi Medical Subject Headings (MeSH), http://www.nlm.nih.gov/mesh/ 8 Reference Reviews, http://www.gale.cengage.com/reference/peter/200708/SpringerLink.htm 9 About Google Scholar, http://scholar.google.at/intl/en/scholar/about.html 10 http://code.google.com/apis/soapsearch/reference.html 11 http://www.programmableweb.com/api/yahoo-search 12 http://msdn2.microsoft.com/enus/library/bb266180.aspx 13 The SOAP based search API has been used since Dec 5, 2006 with permission. 7


65

Fig. 1. Rules for finding candidate future links from the Web

4.1 Link Extraction When querying a search engine, the formulation of query terms strongly affects the results. SOAP APIs have been used by our Web search service to seek Web documents. In performing a search it was found that the use of all available semantic information was able to narrow down search space significantly. The effects of query formulation and choice of query terms is shown in Table 1. Table 1. Query Formulation Query Hermann Maurer “Hermann Maurer”

Google Hits

Yahoo Hits

MSN Live Hits

1,680,00

1,260,000

4,480,00

25,600

92,800

27,000

918

1,720

446

193

775

114

abstract references "Hermann Maurer” abstract references "Hermann Maurer” filetype:PDF

4.2 Removing Duplicates As a further pre-processing step, duplicates are filtered reducing the results by more than 50%. Documents are then downloaded in parallel into java threads. The importance of removing duplicates is shown in Table 2. 4.3 Identifying Research Papers Even by specifying document formats to be either PDF, Doc or PS and also explicitly querying with formulated query terms, the retrieved documents also contains:

66

I. II. III. IV. V. VI. VII.

M.T. Afzal

Theses supervised by the author. Curriculum Vitae, Home page and Business cards of the author. Conference programmes where the author’s name was mentioned. Documents edited by the author. Presentation files The author’s publication list. The author may be listed in the reference entries or in the acknowledgement section of a research paper.

As we are only interested in actual research papers at this point, a further filtering step was performed. This process is important in potentially automating the discovery of Web-pages and publication lists. Docments in PS and doc file formats are first converted to PDF using MiKTeX14 and Openoffice tool15 respectively. Then pdfbox16, a java library is used to convert PDF to plain text for further analysis. A heuristic approach is applied in the actual identification of research papers. The heuristic used are as follows: 1. Title of the paper followed by author name and abstract should exist in the same page. (need not be in the first page). Authors’ full name is then searched to disambiguate author names. 2. Word “reference” or “references” is found followed by proper sequence starting with one of the them “[author]”, “[1]”, “1” and (). Documents that were classified as research papers are shown in Table2 for selected authors. These authors were selected randomly from J. UCS author index for this experiment. The used heuristics were found enough to classify retrieved contents as a research paper. Paper classification module gives no false positive. 4.4 Similarity Algorithm for Checking That Papers Are in the Same Area When we considered Links into the Future for the articles published in Journal of Universal Computer Science only, the former implementation was straightforward. All articles are categorized according to ACM topics and metadata is already available. But when we locate papers from the Web, documents are not categorized according to the ACM topics and metadata cannot be expected to be found. We then performed similarity detection to automatically discover topics of documents. We measured similarity by taking dot product of vectors from the source and the candidate paper. The results were, however, not satisfactory due to the following reasons: 1) Author’s writing style was usually the same in his/her set of documents. A similar use of common terms produced impression of being a larger similarity between documents 2) Paper’s headers share the similar text such as author name, affiliation etc 3) The Reference List at the end of both documents make use of similar text. 14

http://www.miktex.org/ http://www.openoffice.org/ 16 http://www.pdfbox.org/ 15


67

Table 2. Links into the Future results for selected authors Author

Maurer H.

Abraham A.

Bulitko V

Shum S. B

Abecker A.

Focused paper in J.UCS Digital Libraries as Learning and Teaching Support vol. 1 Issue 11 A Novel Scheme for Secured Data Transfer Over Computer Networks Vol. 11 Issue 1 On Completeness of Pseudosimple Sets Vol.1 issue 2 Negotiating the Construction and Reconstruction of Organisational Memories Vol. 3 issue 8 Corporate Memories for Knowledge Management in Industrial Practice: Prospects and Challenges Vol. 3 Issue 8

Classified Papers

Search Engine

Formulated Query

After Duplicate Removal

Google

112

75

12

Yahoo

495

86

19

Google

148

62

13

Yahoo

263

87

41

Google

21

21

7

Yahoo

45

28

13

Google

103

81

11

Yahoo

546

104

26

Google

69

59

9

Yahoo

335

65

Unique paper

Actual Future Links

23

17

33

22

17

3

28

21

17

15

14

To overcome these problems, we pre-processed the text removing the paper’s header (section before abstract) and the reference section of the paper to focus only on the original text. We performed Yahoo Term Extraction17 to extract key terms. This extraction scheme has been used in the number of past studies for extracting facet terms [7] [8] and building expertise profile [4]. In our case, the results from Yahoo Term Extraction was seen to be not convincing until we removed the header and the references sections. The similarity measured on these terms was able to filter the most relevant papers as can be seen in Table 2 and Figure 2. For example, in Table 2, for the author “Vadim Bulitko”, the relevant papers are only 3 out of 17 unique candidate papers found by the paper classification module. The manual inspection revealed that these three were the only papers in the same area. 4.5 Determining Future Links A paper published later is considered a Link into the Future for a paper if it is also written by the same team of authors and is content wise in a similar area or cites the previously written paper. Papers determined to be relevant from multiple sources are then compiled and consolidated as annotations to each paper residing on the J.UCS server. 17

http://developer.yahoo.com/search/content/V1/termExtraction.html

68

M.T. Afzal

4.6 Case Study Figure 2 represents an example of a source paper and its candidate future papers. All of these candidates are acquired from Web by using SOAP APIs as discussed earlier. Candidates “C1, C2, C7, C11, C18” were published within J.UCS. The remaining 18 papers were published outside J.UCS. Figure 2 has been created by using “Graphviz” toolkit. The link distance between source “S1” and candidate “Cn” node is inversely proportional to the term similarity. The figure is further annotated using key terms from the associated papers. Based on the visual representation it is possible to manually ascertain a threshold for candidate papers that belong to the same area. Threshold for this example has been represented by a dotted circle from source paper to the candidate future papers. In this way, it filters 17 papers out of 23. Here the source paper belongs to the topical areas of E-Learning, digital libraries and teaching support. It is obvious that the papers within the closed circle also belong to the topics of source paper. The threshold can be altered to refine the closeness of fit of target documents based on usage or application.

Fig. 2. Similarity measure score for a source paper and its candidate future links

Figure 3 represents the user interface for this feature. The user viewing the source paper entitled “Digital Libraries as Learning and Teaching Support” at18, clicked on “Links into the Future” button and was shown the screen as in Figure 3. In Figure3, the future links from J.UCS database (based on metadata similarity and citations) are

18

http://www.jucs.org/jucs_1_11/digital_libraries_as_learning.


69

consolidated with the future links extracted and filtered from web (as shown in figure 2). Readers are encouraged to explore this feature in Journal of Universal Computer Science (http://www.jucs.org).

Fig. 3. Links into the Future interface

This feature is currently fully implemented for the J.UCS papers and it suggests future related papers that are also published in J.UCS or cited in J.UCS papers. As we are also extending Links into the Future for documents published outside J.UCS, this prototype is being updated.

5 Citation Mining In the previous sections, we discussed the discovery of Links into the Future from web documents. Additionally, we are also calculating Links into the Future based on citations as mentioned in eq. 1. We have developed a new technique for citation mining named as Template based Information Extraction using Rule based Learning ( TIERL). The details about this technique can be found in [1]. We have extracted all references from papers published within J.UCS. These were 15,000 in number and citations were found where J. UCS papers cited other J. UCS papers [1]. After extraction of future links for the focused paper, the future link ontology [2] is updated (see Figure 4).

70

M.T. Afzal

Fig. 4. Future link ontology

6 Discussion To evaluate the research, we have compared the citations extracted from our technique with citations extracted from CiteSeer. The dataset used for evaluation comprised of citations from J.UCS to J.UCS papers. We performed this experiment in the month of May, 2008. There were 92 unique papers that were cited by other J.UCS papers and numbers of citations for these 92 papers were 151. Citeseer indexed 67 papers (73%) out of 92 focused papers and citations found by CiteSeer were only 38 (25%) out of 151. It was due to citations that were in non-compliant formats in the original papers. Our local heuristics employed for J.UCS gave better results. Our technique was able to disambiguate authors by looking for author’s full name in the text of paper. This approach also avoids the mistaken identity of names of places as author of scientific publications as discussed earlier. Table 2 highlights the retrieval results for Links into the Future that was determined from the Web for the paper “Digital Libraries as Learning and Teaching Support” published in J.UCS vol. 1 Issue 11. When user performs a query on search engines, he/she is returned with millions of hits as can be seen in Table 1. The best formulated query for finding PDF/PS/Doc documents was applied to reduce the results to a few hundreds of documents. The process of removing duplicates then reduced the number of documents by up to 50%. Our heuristic rules filtered 12 out of 75 as papers from Google’s results and 19 out of 86 from Yahoo respectively. The similarity based on key terms was then further applied to select 17 papers out of 23 unique papers. Although a user has an option to explore citation indexes to search for related papers. But there are two issues 1) times when papers do not exist on these citation indexes like the source paper in our case study was not indexed by CiteSeer. While Google Scholar indexes it but suggests hundreds of related papers. 2) A deliberate effort is thus needed to find related papers outside the user’s local context.


71

7 Conclusions This work describes the extension of the idea of Links into the Future to cover research papers from the Web. The results are promising in providing candidates for future links. The formulated query enables us to retrieve relevant contents from the Web. Furthermore, a set of heuristics helped to filter unique research papers from the retrieved contents. The key term similarity detection has additionally discovered the most relevant papers for a focused paper. The discovered Links into the Future are supplied to users of a digital journal. This information supply is based on user local context and task at hand. As further works, we are currently developing a tool based on sentiment analysis of citations to evaluate the context of citations. We are also further exploring the discovery of future related papers from digital libraries like DBLP and CiteSeer.

References 1. Afzal, M.T., Maurer, H., Balke, W.T., Kulathuramaiyer, N.: Improving Citation Mining. In: The First International Conference on Networked Digital Technologies. IEEE Press, Ostrava (2009) (accepted) 2. Afzal, M.T., Abulaish, M.: Ontological Representation for Links into the Future. In: International Conference on Computer Science and Convergence Information Technology, pp. 1832–1837. IEEE (CS) Press, Gyeongju-Korea (2007) 3. Afzal, M.T., Kulathuramaiyer, N., Maurer, H.: Creating Links into the Future. Journal of Universal Computer Science 13, 1234–1245 (2007) 4. Aleman-Meza, B., Decker, S.L., Cameron, D., Arpinar, I.B.: Association Analytics for Network Connectivity in a Bibliographic and Expertise Dataset. In: Cardoso, J., Lytras, M.D. (eds.) Semantic Web Engineering in the Knowledge Society. IGI Global (2008) 5. Broder, A.: The Future of Web Search: From Information Retrieval to Information Supply. In: Etzion, O., Kuflik, T., Motro, A. (eds.) NGITS 2006. LNCS, vol. 4032, p. 362. Springer, Heidelberg (2006) 6. Chirita, P.-A., Firan, C.S., Nejdl, W.: Pushing Task Relevant Web Links down to the Desktop. In: 8th ACM International Workshop on Web Information and Data Management, pp. 59–66. ACM Press, Arlington (2006) 7. Dakka, W., Dayal, R., Ipeirotis, P.: Automatic discovery of useful facet terms. In: SIGIR Workshop on Faceted Search. ACM Press, Seattle (2006) 8. Dakka, W., Ipeirotis, P.: Automatic extraction of useful facet hierarchies from text databases. In: 24th International Conference on Data Engineering, pp. 466–475. IEEE Press, Mexico (2008) 9. Giles, C.L., Bollacker, K.D., Lawrence, S.: CiteSeer: An Automatic Citation Indexing System. In: 3rd ACM Conference on Digital Libraries, pp. 89–98. ACM Press, Pittsburgh (1998) 10. Krottmaier, H.: Links to the Future. Journal of Digital Information Management 1, 3–7 (2003) 11. Maurer, H.: Beyond Digital Libraries, Global Digtial Library Development in the New Millenium. In: NIT Conference, pp. 165–173. Tsinghua University Press, Beijing (2001) 12. Postellon, D.C.: Hall and Keynes join Arbor in the citation indices. Nature 452, 282 (2008)

72

M.T. Afzal

13. Price, G.: Google Scholar Documentation and Large PDF Files, http://blog.searchenginewatch.com/blog/041201-105511 (accessed 22, July 2009) 14. Ratprasartporn, K., Ozsoyoglu, G.: Finding Related Papers in Literature Digital Libraries. In: Kovács, L., Fuhr, N., Meghini, C. (eds.) ECDL 2007. LNCS, vol. 4675, pp. 271–284. Springer, Heidelberg (2007) 15. Rhodes, B.J., Maes, P.: Just-in-time information retrieval agents. IBM Syst. J. 39, 685–704 (2000) 16. Speretta, M., Gauch, S.: Personilized Search Based on User Search Histories. In: IEEE/WIC/ACM International Conference on Web Intelligence, pp. 622–628. IEEE (CS) Press, Compiègne (2005)

When Playing Meets Learning: Methodological Framework for Designing Educational Games Stephanie B. Linek1, Daniel Schwarz2, Matthias Bopp3, and Dietrich Albert1 1

Institut of Psychology, University of Graz, Universitätsplatz 2, 8010 Graz, Austria [email protected], [email protected] 2 Takomat GmbH, Neptunplatz 6b, 50823 Cologne, Germany [email protected] 3 Center for Advanced Imaging, Brain Research Institut, University of Bremen Hochschulring 18, 28359 Bremen, Germany [email protected]

Abstract. Game-based learning builds upon the idea of using the motivational potential of video games in the educational context. Thus, the design of educational games has to address optimizing enjoyment as well as optimizing learning. Within the EC-project ELEKTRA a methodological framework for the conceptual design of educational games was developed. Thereby state-of-the-art psycho-pedagogical approaches were combined with insights of media-psychology as well as with best-practice game design. This science-based interdisciplinary approach was enriched by enclosed empirical research to answer open questions on educational game-design. Additionally, several evaluation-cycles were implemented to achieve further improvements. The psycho-pedagogical core of the methodology can be summarized by the ELEKTRA’s 4Ms: Macroadaptivity, Microadaptivity, Metacognition, and Motivation. The conceptual framework is structured in eight phases which have several interconnections and feedbackcycles that enable a close interdisciplinary collaboration between game design, pedagogy, cognitive science and media psychology. Keywords: Game-based learning, Methodology, Microadaptivity, Media psychology.

1 Introduction Game-based learning is a relatively new research area and so far there exist no concrete systematic recommendations for the conceptualization of an integrated design of educational games. In the following, a newly developed conceptual framework for the creation of educational (adventure-)games will be outlined and illustrated by several concrete examples and empirical (evaluation) studies. The proposed methodology was developed and successfully used in the EC-project ELEKTRA (Enhanced Learning Experience and Knowledge Transfer). The described process can serve as a model for other contexts of game based-learning as well as the creation of serious games. J. Cordeiro and J. Filipe (Eds.): WEBIST 2009, LNBIP 45, pp. 73–85, 2010. © Springer-Verlag Berlin Heidelberg 2010

74

S.B. Linek et al.

1.1 When Playing Meets Learning: The Appeal of Game-Based Learning Game-based learning rests upon the idea of using the motivational and immersive potential of conventional video games in the educational context. Even though there are several publications on games [1], game-play [2], and game-based learning [3], the divers contributions are often rather unconnected and an overall framework for the creation of educational games is still missing. The main problem in this context is the seldom collaboration between psychopedagogical scientist and industrial game designers. For an appropriate serious game design, both, the creativity of game designers as well as the expertise of psychopedagogical scientists are necessary. A first step in this direction was made within the EC-project ELEKTRA which will be described in the next subchapter. 1.2 Interdisciplinary Research-Project ELEKTRA ELEKTRA (Enhanced Learning Experience and Knowledge Transfer) was an ECproject under FP6 on game-based learning. The aim of this interdisciplinary research project was twofold: On the one hand it aimed at the development of a state-of-the-art educational adventure-game to make learning as exciting as leading-edge computer games. For this practical aim the so-called ELEKTRA-demonstrator was developed which comprises the first chapter of an educational adventure-game on the learning domain physics/optics. On the other hand a general methodology about the conceptual design and production of digital learning games should be established. This second aim was accomplished by the ELEKTRA methodology which will be described in this article. The core idea of producing effective and motivating digital game-based e-learning experiences for young children relies on an interdisciplinary approach which combines state-of-the-art research in cognitive science, pedagogical theory and neuroscience with best industrial practice in computer game design. The developed methodology builds not only a framework for structuring and supporting the interdisciplinary cooperation, but also inherent several interrelated phases and evaluation-cycles that enable continuous improvements and enhancements of the educational game design. 1.3 The ELEKTRA Methodology: Overview On a general level, the ELEKTRA methodology does not reinvent the wheel but shares a lot of elements with usual instructional design models that many readers might be familiar with (e.g., [4]). In particular the proposed methodology can be seen as an adaption of the Dick and Carey System Approach Model [5] – revised for the purpose of making a state-of-the-art digital learning game. The base of the developed methodology can be summarized by the ELEKTRA’s 4Ms: Macroadaptivity, Microadaptivity, Metacognition, and Motivation. Within the ELEKTRA-project we identified these 4Ms as the pivotal elements of an (exciting) educational game (independent of the concrete learning content and storyline/genre of the game). In order to manage the workflow within the interdisciplinary collaboration a framework with eight phases was developed:

When Playing Meets Learning

75

Phase 1: Identify instructional goals Phase 2: Instructional analysis Phase 3: Analyze learners and context of learning Phase 4: Write performance objectives and overall structure of the game Phase 5: Learning game design Phase 6: Production and development Phase 7: Evaluation of learning Phase 8: Revise instruction Revise instruction

8

- Analyse user validation - Recommendations - Revise and update: Instructional goals, user requirements, instructional analysis, Learning Game Design, Production and Development via RAD-approach

M1 Instructional analysis

M2

2

M3

- Create Knowledge structure

Didactic Design

1 Identify instructional goals

- Game chapters - Categorize learning Objectives - Triple consistency - Learning methods - Performance structure

adaptivity, Metacognition, Microadaptivity (pedagogical rules, adaptive elements))

- Game world and mechanics - GpS - Background story - Characters - StS

Design In-Game Assessment

Production and 6 Development - Technology development - Content production - Game release

RES 422

Evaluation of learning

7

- User validation - Scientific validation design - Testing (functional + pedagogical)

(e.g. Logfiles)

- Microadaptivity (Update learner model, Assessment, adaptivity)

What to learn

How to learn DESIGN

CONCEPTION

C

M4

- Assessment + Validation instruments

3

- User requirements -User preferences - User‘s entry skills

CONTENT

5 Design storybased game world

- LeS (Macro-

Write performance 4 Objectives + overall structure of the game

Analyse learners and context of learning

Learning Game Design

L

LEARNING

How to make PRODUCTION DEVELOPMENT

T

How to evaluate VALIDATION

TECHNOLOGY

Fig. 1. Overview on the eight phases of the model

Even though these phases are numbered from one to eight, they do not follow a linear order but have several interconnections and feedback cycles. Figure 1 illustrates the workflow within the eight phases of the model. The ELEKTRA’s 4Ms are mainly addressed in phase 5 which can be suggested as the core of the methodology: the learning game design. But also the other phases relate to the 4Ms in an implicit way: The phases before feed in the learning game design, the succeeding phases rely on the learning game design and its implementation and improvements, respectively. In the following, first the ELEKTRA’s 4Ms will be characterized. Second, the eight phases will be described; thereby the focus lies on the psychological contribution within this framework. Several practical and empirical examples from the ELEKTRA-project will be given. Finally, a short resume will be provided.

2 Base of the Methodology: ELEKTRA’s 4Ms The ELEKTRA’s 4Ms include the pivotal features of a successful educational game.

76

S.B. Linek et al.

The headwords Macroadaptivity, Microadaptivity, Metacognition, and Motivation are only rough catch phrases for various elaborated concepts, models and findings. Within the ELEKTRA project the main psychological contributions regard to microadaptivity and motivation. The work on macroadaptivity and metacognition was mainly part of the pedagogical partners. 2.1 M1 - Macroadaptivity Macroadaptivity deals with the adaptive pedagogical sequencing of alternative learning situations for one learning objective. Thereby macroadaptivity refers to the instructional design and management of the available learning situation. It addresses the adaptivity between different learning situations and refers also to a diversification of learning based on Bloom’s taxonomy [6]. The macroadaptive process leads to the creation of a learning path which represents a specific combination of divers learning situations. The diversity of learning situations should strengthen deeper cognitive processing and foster long-term knowledge gain. 2.2 M2 - Microadaptivity Microadaptivity regards to adaptive interventions within a learning situation. It involves a detailed understanding of the learner’s skills and a set of pedagogical rules that determine the interventions given to the learner. Within ELEKTRA the idea behind the concept of microadaptivity [7] is to develop a system that provides hints adapted on the user’s (current) knowledge and competence state. Whereas macroadaptivity refers to traditional techniques of adaptation such as adaptive presentation and adaptive navigation on the level of different learning situations microadaptivity deals with the adaptivity within a single learning situation. The basis of the microadaptive skill assessment and the non-invasive interventions is a formal model for interpreting a learner’s (problem solving) behavior. To realize the non-invasive skill-assessment and the adaptive interventions, ELEKTRA relies on the formal framework of the Competence-based Knowledge Space Theory (CbKST; [8], [9], [10]). Originating from conventional adaptive and personalized tutoring, this set-theoretic framework allows assumptions about the structure of skills of a domain of knowledge and to link the latent skills with the observable behavior. Microadaptivity in this context means that the intervention/hint was selected on the basis of knowledge assessment via CbKST in a modified version. This microadaptive assessment procedure assigned a set of required skills as well as a set of missing (non-activated) skills to each feasible action of the player. Given the player’s action (e.g., positioning an object), the likelihoods of the competence states which contain the associated required skills are increased (one updating step for each skill in the set of required skills). Analogously, the likelihoods of the competence states which required the associated missing (non-activated) skills are decreased (one updating step for each skill in the missing skill set). Thereby, in the microadaptive assessment routine each player’s action is followed by a sequence of updates. The likelihoods of the skill states are modified consecutively for each required skill and each missing skill. These probabilities of the required skills and missing skills build up the basis for the microadaptive intervention/hint.


77

LearningSituation Related_Objects*

in_les* SkillSets_Required*

Objects

Learner

position_category* poscat_related_object*

CompetenceSet

Position_Category poscat_skills_missing*

Skills_Taught*

has_skillstate*

Has_Skill*

Includes_Skills* poscat_skills_required*

CompetenceState

incl_skills* Competence

Has_Prerequisite*

Fig. 2. Microadaptivity – integrated model

The chosen hint provides either the necessary information to solve the problem (to learn a missing skill) or the affective support (e.g., motivating or activating feedback) fitting the current progress state of the learner assessed by his/her action history. In order to provide the appropriate kind of intervention, not only the knowledge assessment but also pedagogical rules are taken into account. Within the ELKTRA-demonstrator the microadaptive interventions are presented by a non-player character (NPC) named Galileo in order to merge microadaptivity with the storyline and the overall game play. The NPC Galileo provides the intervention, usually in the form of a comment or a question, in accordance with the game story. If for example, the learner needs motivational support, Galileo encourages him/her by use of the story elements like the rescue of the female pretty NPC named Lisa.

Fig. 3. Using the non-player character named Galileo for providing microadaptive interventions

78

S.B. Linek et al.

2.3 M3 - Metacognition Regarding Flavell [11 (p. 232)] “Metacognition refers to ones knowledge concerning one‘s own cognitive processes or anything related to them, e.g., the learning-relevant properties of information or data”. Even though there exists slightly different interpretations of this original definition, metacognition is agreed to involve knowledge about one’s own knowledge as well as knowledge about one’s own cognitive processes. The ability of the ELEKTRA-demonstrator to foster metacognitive development was considered as a major challenge and an important differentiator compared to traditional educational games. The integration of a reflective pause in the game-based learning process seems at first sight contradicted to storytelling and the flow of game play. Within ELEKTRA the resolution to this dilemma is based on two pillars: First, the implementation of certitude degrees, i.e., while performing a task, the learner has to indicate the prudence and confidence he/she has in his/her performance. Second, a firm support of this kind of metacognition by the storytelling, i.e., the prudence and confidence estimation were made in a close parasocial dialog with the NPC Galileo. The metacognitive reflections are therefore tightly bound to the gaming process. Thereby the ELEKTRA-demonstrator contributes to develop not only the ability to perform, but also to understand the conditions of success, and thus, having cognitive and metacognitive goals in addition to the pure performing goal. 2.4 M4 - Motivation The fourth M named Motivation comprises several motivational concepts and related approaches used for enjoyment and learning. Motivation in this sense is only a keyword for different aspects of the storyline, the challenges and skills (flow-experience), the intrinsic motivation of the gamer, the parasocial interaction and empathy with the NPCs as well as the identification with the avatar. In general, motivation is a phrase used to refer to the reason(s) for engaging in certain activities. In the context of learning games, the creation of motivation to engage in and perform learning activities is a core element of good game design and can be suggested as the major advantage of educational games compared to other ways of e-learning. There are many aspects of games which are suggested to contribute to the gamers motivation [1], e.g., competition, parasocial interaction with the NPCs, fantasy, escapism, suspense or curiosity as well as the balance between challenges and skills (enabled by different game-levels) which in turn fosters the so-called flow-experience [12]. Within ELEKTRA we mainly focussed on the storyline and the game characters as motivational tools for learning. This includes the creation of a story that adds “sense” to specific learning activities, i.e., the learning activities are an integrative part of the story itself. Thereby the story confronts the player with certain gamechallenges/problems (e.g., riddles) that he/she can only solve when he/she first learns certain skills and the story makes it worth to do so.


79

Fig. 4. Riddle within the ELEKTRA-demonstrator: Solution requires knowledge on optics

A typical example would be that the learning activities influence the fate of the avatar or the good and bad NPCs. The crucial issue is to merge learning activities and storyline in a playful way. The usage of the storyline (including game-characters) as a motivational tool comprises several subtasks: designing a setting and a general plot, an avatar with which the players can easily identify as well as designing interesting, authentic good and bad game-characters with which the players can have an immersive parasocial interaction. Especially the latter point is of crucial importance since the game-characters are not only essential for the storyline but also a pivotal source of motivation. Thus, in ELEKTRA, we paid special attention to this issue. On the one hand we conducted several multimedia-studies on the design of the game-characters (which will be partly described in section 3). On the other hand for each of the involved game-characters a biographical background and a personality description were created in order to have authentic and credible game-characters that behave in a (psychological) coherent way throughout the whole game/storyline.

3 Description of the Eight Phases In the following the eight phases will be described. Like mentioned above it is important to note, that these phases don’t follow a simple linear order but rather comprise several interconnections and feedback cycles (see also figure 1). The psycho-pedagogical contributions within ELEKTRA regarded mainly to the phases of instructional analysis, the analysis of the learners and the context of learning, the learning game design and the evaluation phase. For these phases concrete practical and empirical examples will be given to illustrate the important part of cognitive science and media psychology in the conception of educational games.

80

S.B. Linek et al.

3.1 Phase 1: Identify Instructional Goals In this early stage, pedagogy clearly prevails the overall game design by setting some fundamental pedagogical and didactical decisions with respect to the chosen learning goals, the basic areas of learning content and the general pedagogical approach. The context of the game has to be outlined as well: Should the learning game be deployed in a class-room situation at school or should it be played at home as a spare-time activity? This decision is another important cornerstone for the general conditions of the whole design of the learning game. After the definition of learning goals, topic, target group, learning content, pedagogical approach and the context, the general framework of the game is settled. This pedagogical framework not only constitutes the learning experience in the game, but also has got a fundamental impact on the overall concept of the game design. The choice of the game genre is the first crucial design decision which is directly dependent on the learning objectives. If someone like to create for example a strategic simulation game, he/she would perhaps choose different types of learning goals than for a racing game. 3.2 Phase 2: Instructional Analysis In phase 2 the learning objectives and the related learning content are transferred into a formal knowledge structure which is called knowledge space. The theoretical background and mathematical-formal framework is delivered by the already mentioned CbKST. In this context, the main advantage of the CbKST is the clear distinction between observable behavior and the underlying skills and their interrelationships. Thereby the prerequisite relations between skills as well as between overt behaviors enable the adaptation to the actual available skills of the learner as well as the adaptation to the ongoing learning progress. In the established knowledge space all of the learning objectives are represented as an ontology of skills. Thereby the accordingly skills are structured as a map that allows analyzing the developing knowledge state of the learner and thus a learner model. In addition, it allows adapting the game environment to the individual learning needs of the player. This can take place on different levels, e.g., on the level of macroadaptivity or on the level of microadaptivity. 3.3 Phase 3: Analyse Learners and Context of Learning Phase 3 contributes to the detailed analysis of the learners and the context of learning. Thereby the characteristics of the learner group concerning entry skills, learning problems, preferences and attitudes are determined. In a learning game, these areas refer to the learning process as well as to the game play [13]. Thereby the twofold role of the target user has to be taken into account: he/she is both, a learner and a player. Entry skills for the learner could be known difficulties in the chosen learning topic. Additional, entry skills for the player could be the state of his/her game literacy. The learner analysis serves as input for a variety of game decisions: For example the NPC design, the visual style of the game, and the provision of specific learning methods. It is also used to determine the initial state of the learner model. These


81

decisions could be partly made by help of existing literature and research findings. However, with respect to the concrete game design partly additional empirical studies might be necessary. For example within ELEKTRA a focused multimedia study on the NPC-design (regarding his friendliness, the naturalism of the graphics, and the role of color) was conducted. The results of this so-called NPC-study indicate a clear preference for a colored, naturalistic NPC-design. For the NPC’s friendliness the pupils favor a NPC that was similar to their own, indicating similarity-attraction [14]. However, regarding that last finding, the data indicate also gender differences in a way, that similarityattraction hold true for female players only, whereas male players showed a general preference for the more unfriendly version of the NPC. 3.4 Phase 4: Write Performance Objective and Overall Structure of the Game On the basis of phases 1 to 3, performance objectives are laid out and, closely linked to this, the overall pedagogical structure of the game is written. This basic scenario is a kind of working paper which will go through various changes throughout the continuing revising process of the creation for the game. In particular the overall pedagogical structure should include a general description of the story of the game (including the setting, the characters, and the plot), the game-chapters as well as various situations of the game that build up the chapters. They are described in a rough way which includes their main functionality within the game and their possible sequences which can include adaptive branches. 3.5 Phase 5: Learning Game Design Phase 5 is the very core of the ELEKTRA methodology and the accordingly design of a learning game. It is the central work phase where the successful integration of learning and gaming takes place and everything comes together. The main task in this phase is to develop detailed descriptions of each situation in the game: Learning situations (LeS), gameplay situations (GpS), and storytelling situations (StS). Every situation must be described in terms of stage, possible actions, and events that happen in the environment in reaction to the player’s activities. The output is a “Game Design Document” which gives programmers (development) and artists (content production) precise instructions for the development and production of the educational game. The challenge of this design process is to design those three types of situations in such a manner that they constitute pedagogical valid learning activities that are embedded in a meaningful and exiting learning game experience for the player. In an ideal learning game experience the three essential situation types work together as ingredients of a new experience which would arrange a superior game situation from games, learning, and storytelling. This ideal is not always achievable but at least the gameplay situations, learning situations and storytelling situations should motivate, amplify and legitimate each other by embedding them into a meaningful context. The conceptual tools for the design of the situations and their sequencing are based on the already described ELEKTRA’s 4Ms. Thereby Macroadaptivity, Microadaptivity,

82

S.B. Linek et al.

Design + Arts

Digital learning game as a whole

Game Design Cognitive Science

An Ideal user experience in a digital learning game would comprise a mutual pervasion of the three identified situational components (Learning situation, Story situation, Gameplay situation) in ONE LEARNING GAME-Situation (LGS):

Pedagogy Game authoring

In Game Assessment

Learning Situation SCHOOL Learning Situation

HOMEWORK

Transfer of Learning

NEUROSCIENCE Game play Situation

Story Situation

New Learning experience

USERTESTING

Validation of Learning 1

Fig. 5. The ideal learning game situation

and Metacognition are mainly concepts of the instructional strategy of the learning situations and the appending in-game assessments, while Motivation is rather the objective of the story-based game world. 3.6 Phase 6: Production and Development There are two main work areas in the production and development phase: On the one hand programmers develop the various technologies required for the game, on the other hand, artists and producers create all the media assets that are necessary to build the game world. Roughly spoken, one can say, that the development team works on the logic of the game while the production team creates the data for it. The necessary input for the development team and the content production team are the pedagogical scenarios written in phase 4 and the Game Design Document of phase 5. During phase 6 there is a vivid exchange between the programmers of the development team and the artists and producers of the content production team. The outcome of this phase is a published release version of the game that can be tested, played and evaluated. 3.7 Phase 7: Evaluation of Learning There are two different forms of evaluation: the formative evaluation and the summative evaluation of the game. The formative evaluation is also called testing and is closely connected with the development and production work in phase 6. Ideally, the formative evaluation should take place in (monthly) timeboxes when a new testable version of the game-prototype with the latest implementations and improvements is delivered (as output of phase 6). This iterative timebox releases will undergo each time a functional and


83

psycho-pedagogical testing. The formative evaluation can concentrate on single game-elements like background-music or game characters or might deal with the implementation of a new approach like microadaptivity in ELEKTRA [15]. The evaluation results of this testing will directly feed back into early phases. Thereby, the report on technical testing describes functional bugs that manifest themselves in mistakes of the game system. The programmers then have to correct or change the according software components. The report of the psycho-pedagogical testing relates to gaming and learning experiences of the target end user. The results of the psychopedagogical evaluation forces sometimes even to go back to the design phase 5. The summative evaluation can be described as a general evaluation of the developed game and the whole process. It takes place when the iterative technical testing leads to a stable running and psycho-pedagogical meaningful version of the game. In order to analyze the learning behavior and success of the pupils in the game and their evaluation of the gaming experience as a whole, a science-based methodology is applied, using standardized questionnaires as well as logfile-information. In this context not only control variables and pre-questionnaires are considered, but also long-term effects of the learning-game experience should be assessed (e.g., to assess the longterm knowledge gain). For both, the formative as well as the summative evaluation, it is advantageously to use an integrative assessment approach by using logfile-information as well as questionnaire data (like it was done in ELEKTRA). Since these methods deliver complementary data of diverse informational impact their combination provides a more holistic view of the evaluated game and the associated psychological processes [15]. 3.8 Phase 8: Revise Instructions Subsequent to the game testing and the empirical summative evaluation, the next essential step is to interpret and exploit the evaluation results for providing recommendations for improvements and enhancements of the learning game as a whole. These recommendations have to be feed in all preceding phases, affecting all previous tasks and activities and hence, might resulting in a revision and update of the instructional goals (phase 1), instructional analysis (phase 2), user requirements and preferences (phase 3), learning game design (phase 5) as well as production and development (phase 6). Moreover, also the implementation of the evaluation itself might befall revision, e.g., in case of an emerging need for improving the assessment instruments or enhancement of the questionnaires. This in turn requires a close collaboration between scientific research and evaluation. Accordingly, research partners are responsible for selecting scientific proven evaluation instruments as well as for proposing an adequate methodology and data-analysis.

4 Conclusions The proposed methodology delivered a general conceptual framework for the creation of a broad spectrum of educational games. The applicability and validity of the methodology was firstly proven within the EC-project ELEKTRA. The ELEKTRA-demonstrator

84

S.B. Linek et al.

was evaluated empirically and proved its effectiveness for enjoyment as well as for learning. Besides this first positive evidence of the effectiveness of the proposed methodology, also the newly developed micropadaptivity-formalism was successfully tested in several empirical pilot-studies (Linek, Marte, & Albert, 2008). The proposed ELEKTRA methodology can be suggested as a first framework for designing a broad spectrum of educational games. The framework is flexible and open for new technical developments and possibilities and bears the potential to integrate new scientific psycho-pedagogical concepts. Accordingly, the described methodology can be suggested as an open framework that can be adapted to the concrete needs and aims of game-designers, scientists and the target end users. Acknowledgements. This paper is part of the ELEKTRA-project funded by the Sixth Framework Programme of the European Commission’s IST-Programme (contract no. 027986). The author is solely responsible for the content of this paper. It does not represent the opinion of the European Community. Thanks to the ELEKTRA-team for the inspiring interdisciplinary work!!!

References 1. Vorderer, P., Bryant, J.: Playing video games. Motives, responses, and consequences. Lawrence Elbaum Associates, New Jersey (2006) 2. Salen, K., Zimmerman, E.: Rules of play. Game design fundamentals. MIT Press, Cambridge (2004) 3. Prensky, M.: Computer games and learning: Digital game-based learning. In: Raessens, J., Goldstein, J. (eds.) Handbook of computer game studies, pp. 97–122. MIT Press, Cambidge (2005) 4. Brown, A., Green, T.D.: The essentials of instructional design. Connecting fundamental principles with process and practice. Pearson/Merrill Prentice Hall, Upper Saddle River (2006) 5. Dick, W., Carey, L., Carey, J.O.: The systematic design of instruction, 6th edn. Pearson/Allyn & Bacon, Boston (2005) 6. Bloom, B.: Taxonomy of educational objectives. In: Handbook I: The cognitive domain. David McKay Co. Inc., New York (1956) 7. Albert, D., Hockemeyer, C., Kickmeier-Rust, M.D., Pierce, N., Conlan, O.: Microadaptivity within complex learning situations – a personalized approach based on competence structures and problem spaces. Poster presented on the International Conference on Computers in Education (2007) 8. Albert, D., Lukas, J. (eds.): Knowledge spaces: theories, empirical research, and applications. Lawrence Erlbaum Associates, Mahwah (1999) 9. Doignon, J.P., Falmagne, J.C.: Knowledge spaces. Springer, Berlin (1999) 10. Korossy, K.: Extending the theory of knowledge spaces: A competence-performance approach. Zeitschrift für Psychologie 205, 53–82 (1997) 11. Flavell, J.H.: Metacognitive aspects of problem solving. In: Resnick, L.B. (ed.) The nature of intelligence, pp. 231–236. Lawrence Erlbaum Associates, New Jersey (1976) 12. Csikszentmihalyi, M.: Flow: The psychology of optimal experience. Harper & Collins, New York (1990)


85

13. Linek, S.: Creating flow in game-based learning: Threefold conception of challenges and skills. In: Proceedings of the International Technology, Education and Development Conference (2007) 14. Linek, S., Schwarz, D., Hirschberg, G., Kickmeier-Rust, M., Albert, D.: Designing the non-player character of an educational adventure-game: the role of personality, naturalism, and color. In: Proceedings of the International Technology, Education and Development Conference (2007) 15. Linek, S.B., Marte, B., Albert, D.: The differential use and effective combination of questionnaires and logfiles. In: Proceedings of the International Conference on Interactive Computer Aided Learning, Special Track Computer-based Knowledge & Skill Assessment and Feedback in Learning Settings (2008)

SiteGuide: A Tool for Web Site Authoring Support Vera Hollink1 , Viktor de Boer2 , and Maarten van Someren2 1

2

Centre for Mathematics and Computer Science Science Park 123, Amsterdam, The Netherlands [email protected] University of Amsterdam, Science Park 107, Amsterdam, The Netherlands {V.deBoer,M.W.vanSomeren}@uva.nl

Abstract. We present ‘SiteGuide’, a tool that helps web designers to decide which information will be included in a new web site and how the information will be organized. SiteGuide takes as input URLs of web sites from the same domain as the site the user wants to create. It automatically searches the pages of these example sites for common topics and common structural features. On the basis of these commonalities it creates a model of the example sites. The model can serve as a starting point for the new web site. Also, it can be used to check whether important elements are missing in a concept version of the new site. Evaluation shows that SiteGuide is able to detect a large part of the common topics in example sites and to present these topics in an understandable form to its users. First results of a user study indicate that Siteguide helps users to create web site designs with better structured contents and links.

1 Introduction Even the smallest companies, institutes and associations are expected to have their own web sites. However, designing a web site is a difficult and time-consuming task. Software tools that provide assistance for the web design process can help both amateur and professional web designers. Newman and Landay (2000) studied the current practices in web design and identified four main phases in the design process of a web site: discovery, design exploration, design refinement and production. A number of existing tools, such as Adobe Dreamweaver1 and Microsoft Frontpage2 provide help for the latter two phases, where an initial design is refined and implemented. These tools however, do not support collecting and structuring the content into an initial conceptual model [1]. In this paper, we present ‘SiteGuide’, a system that helps web designers to create a setup for a new site. Its output is an initial information architecture for the target web site that shows the user what information should be included in the web site and how the information should be structured. Figure 1 shows a screenshot of the SiteGuide system. An important step in the discovery phase of web site design is reviewing web sites from the same domain as the target site [2]. For instance, a person who wants to build a site for a small soccer club will often look at web sites of some other small soccer clubs. 1 2

http://www.adobe.com/products/dreamweaver http://office.microsoft.com/frontpage

J. Cordeiro and J. Filipe (Eds.): WEBIST 2009, LNBIP 45, pp. 86–98, 2010. c Springer-Verlag Berlin Heidelberg 2010

SiteGuide: A Tool for Web Site Authoring Support

87

Fig. 1. A screenshot of the SiteGuide system showing the graph representation of a number of topics for a web site of a hotel

The information architectures of the examined sites are used as source of inspiration for the new site. Reviewing example sites can provide useful information, but comparing sites manually is very time-consuming and error-prone, especially when the sites consist of many pages. The SiteGuide system creates an initial information architecture for a new site by efficiently and systematically comparing a set of example sites identified by the user. SiteGuide automatically searches the sites for topics and structures that the sites have in common. For example, in the soccer club domain, it may find that most example sites contain information about youth teams or that pages about membership always link to pages about subscription fees. The common topics are brought together in a model of the example sites. The model is presented to the user and serves as an information architecture for the new web site. SiteGuide can also be used in the design refinement phase of the web design process as a critic of a first draft of a site. The draft is compared with the model, so that missing topics or unusual information structures are revealed.

2 Functionality of the SiteGuide Tool The SiteGuide system has two main usage scenarios, shown in Figure 2. In both scenarios the user starts the interaction by inputting the URLs of the home pages of a small set of example web sites. SiteGuide then scrapes and analyzes the sites and captures their commonalities in a web site model. The model forms the suggested information architecture for the new site. In the modeling scenario the information architecture is

88

V. Hollink, V. de Boer, and M. van Someren

the end point of the interaction and is outputted to the user. In the critiquing scenario the user has already created a first draft version of his new site. SiteGuide compares the draft with the model of the example sites and outputs the differences. Figure 3 shows the structure of an example site model. A model consists of a set of topics that appear in the example sites. To communicate the model to a user, SiteGuide describes each topic with a set of characterizing features. These features explain to the user what the topic is about. They consist of key phrases which are extracted from the contents of the pages that handle on the topic as well as titles of these pages, anchor texts of links pointing to the pages and terms from the page URLs. Additionally, SiteGuide shows a link to a page that exemplifies the topic. To inform the user on how a topic should be embedded in the site, SiteGuide shows structural features that describe how the topic is represented in the pages of the example sites. It shows the average number of pages about the topic, the average number of incoming and outgoing links for those pages and links between topics (e.g., pages on topic A frequently link to pages on topic B). In the modeling scenario the topics of the model are presented to the user either as a set of natural language statements or in a graph representation as shown in the screenshot of Figure 1. There, each of the boxes corresponds to the output for one topic. The user can review the topics and decide which topics he (or she) will include in the new web site. The information architecture is also exported to an XML file, so that

example sites

build model

model

format

human readable statements

draft of new site

compare

differences

format

human readable statements

Fig. 2. The two usage scenarios of the SiteGuide system. The black arrows denote the modeling scenario. The white arrows denote the critiquing scenario.

topic A characterizing features keywords: weather, wind title: Weather conditions example page:

topic C topic B features ...

www.surf.com/weather.html ... structural features on average 2.1 pages on average 6 incoming links

features ...

topic D

topic E

features ...

features ...

...

Fig. 3. Example of an example site model. A model consists of topics that have characterizing and structural features (only shown for topic A). Frequently occurring hyperlinks between topics are denoted by arrows.


89

it can be imported into other web design tools that provide alternative visualizations, such as prototyping or wireframing tools. The output of the critiquing scenario is a set of textual statements that describe the differences between the example sites and the draft. In this scenario SiteGuide indicates which topics are found on most example sites but not on the draft site and which topics are found on the draft site but not in any of the example sites. In addition, it outputs topics that are both on the draft and the example sites, but that have different structural features.

3 Method The main task of SiteGuide is to find topics that occur on most example sites. For this, SiteGuide identifies pages of different example sites that handle on the same topic and maps these pages onto each other. A mapping can be seen as a set of page clusters. Each cluster contains all pages from example sites that handle on one topic. We design our clustering format in such a way that it is able to capture differences between web sites in the information they present and in the way the information is organized. All pages of the example sites must occur in at least one page cluster, but pages may occur in multiple clusters. In the simplest case a cluster contains one page from each example site. For example, a cluster can contain for each site the page about surfing lessons. However, if on one of the sites the information about surfing lessons is split over several pages, these pages are placed in the same cluster. It can also happen that a cluster does not contain pages from all sites, because some of the sites do not contain information on the cluster’s topic. Pages occur in more than one cluster, when they contain content about more than one topic. We developed a heuristic method to find a good mapping between a set of example sites. A space of possible mappings is defined and searched for a good mapping. Below we first explain how SiteGuide measures the quality of a mapping. Then we explain how it searches for the mapping that maximizes the quality measure. Finally, the generation of the example site model from the mapping and the comparison between the model and a draft of the new site are discussed. 3.1 Quality Measure for Example Site Mappings As in most clustering tasks, the quality of an example site mapping is better when the pages in the clusters are more similar to each other. However, in most domains pages of one site are more similar to pages on the same site that handle on other topics than to pages of other sites on the same topic. As a result, a standard clustering method would mainly form groups of pages from one example site instead of identifying topics that span multiple sites. We solve this problem by focussing on similarities between pages from different sites. We define the quality of a page cluster as the average similarity of the pages in the cluster to all other pages in the cluster from other web sites. Most web pages contain at least some text, so that text similarity measures can be used to compute the similarity between two pages. However, web pages are not standalone texts, but part of a network that is connected by links. In SiteGuide we make use of

90


the information contained in the link structure by computing the similarity between the pages’ positions in their link structures. As extra features we use the similarity between page titles, page URL’s and the anchors of the links that point to pages. Below, each of these five types of similarity are discussed in more detail. The quality of a page cluster is a combination of the five similarity measures. The quality of cluster C in mapping M is: quality(C, M ) = (wi · simi (C, M )) − α · SC simi ∈Sims

Here Sims are the five similarity measures, which are weighted with weighting parameters wi . SC is the number of example sites that have pages in cluster C. α is a parameter. The term −α · SC subtracts a fixed amount (α) for each of the SC sites in the cluster. Consequently, adding pages of a site to a cluster only improves the cluster’s score if the pages bear a similarity of at least α to the pages of the other sites in the cluster. In this way the size of the clusters is automatically geared to the number of sites that address the same topic, so that we do not need to specify the number of clusters beforehand. Text similarity between two pages is expressed as the cosine similarity between the terms on the pages [3]. This measure enables SiteGuide to identify parts of the texts that pages have in common and ignore site-specific parts. Stop word removal, stemming and tf · idf weighting are applied to increase accuracy. Anchor text similarity between two pages is defined as the cosine similarity between the anchor texts of the links that point to the pages. For the computation of page title similarity and URL similarity we use the Levenshtein distance [4] instead of the cosine similarity. Levenshtein distance is more suitable for comparing short phrases as it takes the order of terms into account and works at character level instead of term level. We developed a new measure to compute the similarity between the positions of two pages in their link structures. We look at the direct neighborhood of each page: it’s incoming and outgoing links. The link structure similarity of a cluster is the proportion of the incoming and outgoing links of the pages that are mapped correctly. Two links in different link structures are mapped correctly onto each other if both their source pages and their destination pages are mapped onto each other. 3.2 Finding a Good Mapping A naive approach for finding a mapping with a high quality score would be to list all possible mappings, compute for each mapping the quality score, and choose the one with the highest score. Unfortunately, this approach is not feasible, as the number of possible mappings is extremely large [5]. To make the problem computationally feasible, we developed a search space of possible mappings that allows us to heuristically search for a good mapping. We start our search with an initial mapping that is likely to be close to the optimal solution. In this mapping each page occurs in exactly one cluster and each cluster contains no more than one page from each example site. The initial mapping is built incrementally. First, we create a mapping between the first and the second example site. For each


91

two pages of these sites we compute the similarity score defined above. The so called Hungarian Algorithm [6] is applied to the two sites to find the one-to-one mapping with the highest similarity. Then, the pages of the third site are added to the mapping. We compute the similarity between all pages of the third site and the already formed pairs of pages of the first two sites and again apply the Hungarian Algorithm. This process is continued until all example sites are included in the initial mapping. We define five mapping modification operations that can be used to traverse the search space. Together, these operations suffice to transform any mapping into any other mapping. This means that the whole space of possible mappings is reachable from any starting point. The operations are: – – – – –

Split a cluster: the pages from each site in the cluster are placed in a separate cluster. Merge two clusters: place all pages from the two clusters in one cluster. Move a page from one cluster to another cluster. Move a page from a cluster to a new, empty cluster. Copy a page from one cluster to another cluster.

With these operations SiteGuide refines the initial mapping using a form of hill climbing. In each step it applies the operations to the current mapping and computes the effect on the similarity score. If an operation improves the score it is retained; otherwise it is undone. It keeps trying modification operations until it can not find any more operations that improve the score with a sufficient amount. The five operations can be applied to all clusters. To increase efficiency, SiteGuide tries to improve clusters with low quality scores first. 3.3 From Mapping to Model The next step is to transform the mapping into a model of the example sites. The mapping consists of page clusters, while the model should consist of descriptions of topics that occur on most of the example sites. Each cluster becomes a topic in the model. Topics are characterized by the five characterizing features mentioned in Section 2. SiteGuide lists all terms from the contents of the pages and all URLs, titles and anchor texts. For each type of term we designed a measure that indicates how descriptive the term or phrase is for the topic. For instance, content terms receive a high score if they occur in all example sites frequently in pages on the topic and infrequently on other pages. These scores are multiplied by the corresponding similarity scores, e.g., the score of a content term is multiplied by the topic’s content similarity. The result of this is that features that are more important for a topic are weighted more heavily. The terms and phrases with scores above some threshold (typically 3 to 10 per feature type) become characterizing features for the topic. The most central page in the cluster (the page with the highest text similarity to the other pages in the cluster) becomes the example page for the topic. To find the structural features of the topics, SiteGuide analyzes the pages and links of the corresponding page clusters. It determines for each site over how many pages the information on a topic is spread and counts the number of incoming and outgoing links. Furthermore, it counts how often the topic links to each other topic. The question

92


is which of these numbers indicate a stable pattern over the various example sites. Intuitively, we recognize a pattern in, for instance, the number of outgoing links of a topic, when on all sites the pages on the topic have roughly the same number of outgoing links. We have formalized this intuition: when the numbers for the various sites have low variance, SiteGuide marks the feature as a common structural feature. In the modeling scenario SiteGuide outputs for each topic the characterizing features, the example page and the common structural features. As stated in Section 2, SiteGuide can output these elements either visually or as text. The textual output consists of a series of human readable statements, such as Pages on topic A have URLs like ‘mappage’ and ‘location’. The visual output is shown in Figure 1. To allow the user to easily use the generated model as an initial interaction design within a web design process, SiteGuide also offers the option to export the model to the XML format used in the DENIM sketching tool [7]. This tool can be used to easily create prototypes for new web sites. Each topic becomes a DENIM page with the page content consisting of the characterizing features. The links between topics are used to generate site map links in DENIM. When imported into the DENIM tool, the model can also be converted to a working HTML web site, that can be used for testing. For more information about the integration of SiteGuide and DENIM see [8]. For the critiquing scenario we developed a variant of the web site comparison method which enables SiteGuide to compare the example site model to a draft of the new site. This variant uses the hill climbing approach to map the model to the draft, but it does not allow any operations that alter the example site model. In this way we ensure that the draft is mapped onto the model, while the model stays intact. Once the draft is mapped, SiteGuide searches for differences between the draft and the example site model. It determines which topics in the model do not have corresponding pages in the draft and reports that these topics are missing on the new site. Conversely, it determines which topics of the draft do not have counterparts in the example sites. Finally, it compares the structural features of the topics in the new site to the common structural features in the example site model and reports the differences.

4 Evaluation To determine whether SiteGuide can provide useful assistance to users who are building a web site, we need to answer three questions. 1) Do the discovered topics represent the subjects that are really addressed at the example sites? 2) Are the topic descriptions understandable for humans? 3) Do the example site models help users to create better setups for their web sites? To answer the first question we compared mappings created by SiteGuide to manually created example site mappings. For the second question we asked humans to interpret SiteGuide’s output. To answer the third question we present first results of a user study in which we compared setups of users that were created with and without the help of SiteGuide. We used web sites from three domains: windsurf clubs, primary schools and small hotels. For each domain 5 sites were selected as example sites. We purposely chose very different domains: the windsurf clubs are non-profit organizations, the school domain is an educational domain and the hotel domain is commercial. Table 1 shows the main properties of the three domains.


93

The SiteGuide system generated example site models for the three domains. We compared these models to gold standard models that we had constructed by hand. The gold standard models consisted of a mapping between the example sites and for each page cluster a textual description of the topic that was represented by the cluster. The textual descriptions were 1 or 2 sentences in length and contained around 20 words. For example, in the school domain one topic was described as ‘These pages contain a list of staff members of the schools. The lists consist of the names and roles of the staff members.’. Features of the gold standards are given in Table 1. To validate the objectivity of the gold standards, for one domain (hotels) we asked another person to create a second gold standard. We compared the topics in the two gold standards and found that 82% of the topics were found in both gold standards. From this we concluded that the gold standards were quite objective and are an adequate means to evaluate the output of SiteGuide. 4.1 Evaluation of the Mappings First, we evaluated the page clusters that were generated by SiteGuide to see to what extend these clusters coincided with the manually created clusters in the gold standards, in other words, to what extend they represented topics that were really addressed at the example sites. We counted how many of the clusters in the generated mapping also occurred in the gold standards. A generated cluster was considered to be the same as a cluster from the gold standard if at least 50% of the pages in the clusters were the same. We considered only topics that occurred in at least half of the example sites (frequent topics), as these are the topics that SiteGuide was looking for. The quality of mappings is expressed by precision, recall and f-measure over the page clusters. When Cgold are the clusters in a gold standard and Ctest are the clusters in a generated mapping, the measures are defined as: precision = |Ctest ∩ Cgold |/|Ctest | recall = |Ctest ∩ Cgold |/|Cgold | (2·precision · recall) f−measure = (precision + recall) In a pilot study [5] we tested the influence of the various parameters. We generated example site mappings with various weights for the similarity measures. On all three Table 1. Properties of the evaluation domains and the gold standards (g.s.): the total, minimum and maximum number of pages in the example sites, the number of topics in the g.s., the number of topics that were found in at least 50% of the sites (frequent topics) and the percentage of the pages that were mapped onto at least one other page domain

total pages

hotel surfing school

59 120 154

min-max pages 9-16 8-61 20-37

topics in g.s.

frequent topics

21 90 42

7 12 17

% pages mapped in g.s. 81% 76% 80%

94


Table 2. Quality of the example site mappings and quality of the comparison between drafts and example sites domain hotel surf school

precision

recall

f-measure

1.00 0.33 0.47

0.43 0.42 0.41

0.60 0.37 0.44

removed topics detected 0.54 0.25 0.33

added topics detected 0.52 0.26 0.88

domains giving high weights to text similarity resulted in mappings with high scores. In the hotel domain URL similarity also appeared to be effective. Increasing the minimum similarity parameter (α) meant that we required mapped pages to be more similar, so that precision was increased, but recall decreased. Thus, with this parameter we can effectively balance the quality of the topics that we find against the number of topics that are found. When the SiteGuide system is used by a real user, it obviously cannot use a gold standard to find the optimal parameter settings. Fortunately, we can estimate roughly how we should choose the parameter values by looking at the resulting mappings as explained in [5]. The scores of the example site models generated with optimal parameter values are shown in Table 2. The table shows the scores for the situation in which all frequent topics that SiteGuide has found are shown to the user. When many topics have been found we can choose to show only topics with a similarity score above some threshold. In general, this improves precision, but reduces recall. Next, we evaluated SiteGuide in the critiquing scenario. We performed a series of experiments in which the 5 sites one by one played the role of the draft site and the remaining 4 sites were example sites. In each run we removed all pages about one of the gold standard topics from the draft site and used SiteGuide to compare the corrupted draft to the examples. We counted how many of the removed topics were identified by SiteGuide as topics that were missing in the draft. Similarly, we added pages to the draft that were not relevant in the domain. Again, SiteGuide compared the corrupted draft to the examples. We counted how many of the added topics were marked as topics that occurred only on the draft site and not on any of the example sites. The results are given in Table 2. Table 2 shows that SiteGuide is able to discover many of the topics that the sites have in common, but also misses a number of topics. Inspection of the mappings demonstrates that many of the discovered topics can indeed lead to useful recommendations to the user. We give a few examples. In the school domain SiteGuide created a page cluster that contained for each site the pages with term dates. It also found correctly that 4 of the 5 sites provided a list of staff members. In the surfing domain, a cluster was created that represented pages where members could leave messages. The hotel site mapping contained a cluster with pages about facilities in the hotel rooms. The clusters are also useful in the critiquing scenario: for example, when the owner of the fifth school site would use SiteGuide, he would learn that his site is the only site without a staff list. Some topics that the sites had in common were not found, because the terms did not match. For instance, two school sites provided information about school uniforms, but on the one site these were called ‘uniform’ and on the other ‘school dress’. This


95

example illustrates the limitations of the term-based approach. In the future, we will extend SiteGuide with WordNet [9], which will enable it to recognize semantically related terms. 4.2 Evaluation of the Topic Descriptions Until now we counted how many of the generated topics had at least 50% of their pages in common with a gold standard topic. However, there is no guarantee that the statements that SiteGuide outputs about these topics are really understood correctly by the users of the SiteGuide system. Generating understandable descriptions is not trivial, as most topics consist of only a few pages. On the other hand, it may happen that a description of a topic with less than 50% overlap with a gold standard topic is still recognizable for humans. Therefore, below we evaluate how SiteGuide’s end output is interpreted by human users and whether the interpretations correspond to the gold standards. We used SiteGuide to create textual output about the example site models generated with the same optimal parameter values as in the previous section. Since we only wanted to evaluate how well the topics could be interpreted by a user, we did not output the structural features. We restricted the output for a topic to up to 10 content keywords and up to 3 phrases for page titles, URLs and anchor texts. We also displayed for each topic a link to the example page. Output was generated for each of the 34 frequent topics identified in the three domains. We asked 5 evaluators to interpreted the 34 topics and to write a short description of what they thought each topic was about. We required the descriptions to be of the same length as the gold standard descriptions (10-30 words). None of the evaluators were domain experts or expert web site builders. It took the evaluators on average one minute to describe a topic, including typing the description. By comparison, finding the topics in the example sites by hand (for the creation of the gold standard) took about 15-30 minutes per topic. An expert coder determined whether the interpretations of the evaluators described the same topics as the gold standard descriptions. Since both the evaluators’ topic descriptions and the gold standard topic descriptions were natural language sentences, it was often difficult to determine whether two descriptions described the exact same topic. We therefore had the coder classify each description in one of three classes: a description could either have a partial match or an exact match with one of the gold standard topics or have no matching gold standard topic at all. An exact match means that the description describes the exact same topic as the gold standard topic. A partial match occurs when a description describes for instance a broader or narrower topic. To determine precision the coder matched all topic descriptions of the evaluators to the gold standard descriptions. To determine recall the coder matched all gold standard descriptions to the evaluators’ descriptions in the same manner. To determine the objectivity of the coding task, we had a second expert coder perform the same task. The two coders agreed on 69% of the matches, considering the possible variety in topic descriptions, we consider this an acceptable level of inter-coder agreement. Averaged over the domains and evaluators, 94% of all evaluators’ topic descriptions were matched to one of the gold standard topics when both partial and exact matches

96

V. Hollink, V. de Boer, and M. van Someren Table 3. Results of the manual evaluation for the three domains domain hotel school surf

precision 0.90 0.78 0.81

recall 0.54 0.54 0.63

f-measure 0.68 0.64 0.71

were considered. In other words, 94% of the topics that were found by SiteGuide corresponded (at least partially) with a topic from the gold standard and were also interpreted as such. When only exact matches were counted this figure was still 73%. This indicates that for most topics SiteGuide is capable of generating an understandable description. We calculated precision and recall based on the relaxed precision and recall used in ontology matching literature [10], resulting in the same formula for precision and recall as in Section 4.1, where |Ctest ∩ Cgold | is the number of exact matches plus 0.5 times the number of partial matches. In Table 3 we display the relaxed precision, recall and f-measure values for the three domains. The average precision over all domains is 0.83, showing that most of the topics that SiteGuide found could indeed be interpreted and that these interpretations corresponded to correct topics according to our gold standard. The average recall is 0.57, which means that more than half of the topics from the gold standard were correctly identified and outputted by SiteGuide. Both precision and recall do not vary much across domains, indicating that SiteGuide is capable of identifying and displaying topics in a wide range of domains. The results in Table 3 are considerably better than those in Table 2. This shows that for a number of topics where the page overlap with the gold standard was less than 50%, the displayed topic could still be interpreted correctly by humans. 4.3 Evaluation of Web Site Design Support We conducted a small-scale user study to determine if the use of SiteGuide’s output improves the early phases of the web design process. In this study, described in more detail in [8], we asked 12 participants to make setups for three web sites: a primary school, a small hotel and a scientific conference. The 6 participants in the test group were presented with five example sites and SiteGuide’s graphical output for these sites. The participants in the control group were only presented with the example sites. The participants were asked to create for each of the three web sites a sketch of its design consisting of ‘boxes and arrows’. Afterwards, the users filled out a questionnaire. From a first analysis of the questionnaire we concluded that although it takes more effort to interpret SiteGuide’s example site models than to interpret the example sites themselves, the topics in the models are well understood. Also, inspection of the web design sketches showed that the test group incorporated a large part of the topics suggested by SiteGuide in their sketches. We assessed the quality of the sketches for the school site by presenting them doubleblind to a web design expert. The expert ranked the 12 sketches on overall quality and graded them on a 5-point scale on 5 criteria: completeness, relevancy, detailedness, content structure and link structure. We found that the sketches of the test group were ranked significantly higher than the sketches of the control group. In addition, the test


97

group had significantly higher scores on content structure and link structure. These results show that SiteGuide can really help users to create web site designs with better content and link structures.

5 Related Work Existing tools for assisting web site development help users with the technical construction of a site. Tools such as Dreamweaver1 or Frontpage2 allow users to create web sites without typing HTML. Other tools evaluate the design and layout of a site on usability and accessibility, checking for example for dead links and buttons and missing captions of figures (see for an overview [11,12]). However, none of these tools help users to choose appropriate contents or structures for their sites or critique the content that is currently used. Our approach is in spirit related to the idea of ontologies. The goal of an ontology is to capture the conceptual content of a domain. It consists of structured, formalized information on the domain. The web site models presented in this paper can be viewed as informal ontologies as they comprise structures of topics that occur in sites from some domain. However, our models are not constructed by human experts, but automatically extracted from example sites. Another related set of tools are tools that improve link structures of web sites, such as PageGather [13] and the menu optimization system developed by Hollink et al. [14]. These tools do not provide support on the contents of a site. Moreover, they need usage data, which means that they can only give advice about sites that have been online for some time. The algorithm that underlies the SiteGuide system is related to methods for highdimensional clustering, which are, for instance, used for document clustering. However, there are several important differences. The task of SiteGuide is to find topics in a set of web sites instead of an unstructured set of documents. It is more important to find topics that appear in many sites than to group pages within sites. This is reflected in the similarity measure. Another difference is that in SiteGuide the relations (links) between pages are taken into account. The extent to which relations between pages within one site match the relations in other sites contributes to the similarity between pages. Also, the final model of the sites includes relations between topics.

6 Conclusions The SiteGuide system provides assistance to web designers who want to build a web site but do not know exactly which content must be included in the site. It automatically compares a number of example web sites that are similar to the one the user would like to create and constructs a model that describes the features that the sites have in common. The model can be used as an information architecture for the new site. In addition, SiteGuide can show differences between example sites and a first version of a new site. SiteGuide was applied to example web sites from 3 domains. In these experiments, SiteGuide proved able to find many topics that the sites had in common. Moreover, the

98


topics were presented in such a way that humans could easily and quickly understand what the topics were about. First results from a user study suggest that SiteGuide can really improve web design. Users presented with the models generated by SiteGuide indeed incorporated them in their web design sketches. Moreover, sketches created by users that were presented with the models were of higher quality than those created without the models. In the near future SiteGuide will be extended with a number of new features. Semantics will be added to the similarity measure in the form of WordNet relations. Also, SiteGuide will output additional features such as style features (e.g., colors and use of images) or the amount of tables, lists, forms, etc. Finally, we will conduct more user studies to examine the influence of SiteGuide on the web design process in detail.

References 1. Falkovych, K., Nack, F.: Context aware guidance for multimedia authoring: Harmonizing domain and discourse knowledge. Multimedia Systems 11(3), 226–235 (2006) 2. Newman, M.W., Landay, J.A.: Sitemaps, storyboards, and specifications: A sketch of web site design practice. In: Proceedings of the Third Conference on Designing Interactive Systems, New York, pp. 263–274 (2000) 3. Salton, G., McGill, M.J.: Introduction to Modern Information Retrieval. McGraw-Hill, New York (1983) 4. Levenshtein, V.I.: Binary codes capable of correcting deletions, insertions and reversals. Soviet Physics Doklady 10(8), 707–710 (1966) 5. Hollink, V., Van Someren, M., De Boer, V.: Capturing the needs of amateur web designers by means of examples. In: Proceedings of the 16th Workshop on Adaptivity and User Modeling in Interactive Systems, Würzburg, Germany, pp. 26–31 (2008) 6. Munkres, J.: Algorithms for the assignment and transportation problems. Journal of the Society of Industrial and Applied Mathematics 5(1), 32–38 (1957) 7. Lin, J., Newman, M.W., Hong, J.I., Landay, J.A.: DENIM: Finding a tighter fit between tools and practice for web site design. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, The Hague, The Netherlands, pp. 510–517 (2000) 8. de Boer, V., Hollink, V., van Someren, M.: Automatic web site authoring with SiteGuide. In: Recent Advances in Intelligent Information Systems: Proceedings of the IIS 2009 conference, Krakow, Poland (2009) 9. Fellbaum, C. (ed.): WordNet: An electronic lexical database. MIT Press, Cambridge (1998) 10. Ehrig, M., Euzenat, J.: Relaxed precision and recall for ontology matching. In: Proceedings of the K-Cap 2005 Workshop on Integrating Ontologies, Banff, Canada, pp. 25–32 (2005) 11. Web Accessibility Initiative: Web accessibility evaluation tools: Overview (2008), http://www.w3.org/WAI/ER/tools/ (Last accessed: July 4, 2008) 12. Brajnik, G.: Using automatic tools in accessibility and usability assurance processes. In: Stary, C., Stephanidis, C. (eds.) UI4ALL 2004. LNCS, vol. 3196, pp. 219–234. Springer, Heidelberg (2004) 13. Perkowitz, M., Etzioni, O.: Towards adaptive web sites: Conceptual framework and case study. Artificial Intelligence 118(1-2), 245–275 (2000) 14. Hollink, V., Van Someren, M., Wielinga, B.: Navigation behavior models for link structure optimization. User Modeling and User-Adapted Interaction 17(4), 339–377 (2007)

ArhiNet – A Knowledge-Based System for Creating, Processing and Retrieving Archival eContent Ioan Salomie, Mihaela Dinsoreanu, Cristina Pop, and Sorin Suciu Technical University of Cluj-Napoca, Department of Computer Science 15 C. Daicoviciu Street, 400020 Cluj-Napoca, Romania {Ioan.Salomie,Mihaela.Dinsoreanu,Cristina.Pop}@cs.utcluj.ro, [email protected]

Abstract. This paper addresses the problem of creating, processing and querying semantically enhanced eContent from archives and digital libraries. We present an analysis of the archival domain, resulting in the creation of an archival domain model and of a domain ontology core. Our system adds semantic markup to the historical documents content, thus enabling document and knowledge retrieval as response to natural language ontology-guided queries. The system functionality follows two main workflows: (i) semantically enhanced eContent generation and knowledge acquisition and (ii) knowledge processing and retrieval. Within the first workflow, the relevant domain information is extracted from documents written in natural languages, followed by semantic annotation and domain ontology population. In the second workflow, ontologically guided natural language queries trigger reasoning processes that provide relevant search results. The paper also discusses the transformation of the OWL domain ontology into a hierarchical data model, thus providing support for the efficient ontology processing. Keywords: Archival domain model and ontology, Knowledge acquisition, Hierarchical data model, Semantic annotation, Ontology guided query, Reasoning.

1 Introduction Archives represent an important source of historical knowledge. Being part of the national heritage the documents contained in the archives are priceless, therefore efforts should be made to avoid their deterioration. Allowing the access of researches to the documents in their original form is both costly, due to mobility or security reasons, and risky, due to the exposure of the documents to factors that may alter their integrity. The use of digital copies of the original archival documents is a solution that diminishes the costs associated to the study of the documents, while preserving their original form. When organized in archives, documents have associated descriptions (metadata). While the metadata format is consistent, their content and the content of the documents are heterogeneous. Among the factors that contributed to the heterogeneity, three are directly affecting the document study. The first and the most important factor is the variety of languages in which their content and metadata are written. The second factor is the historical period in which both the document and the descriptions J. Cordeiro and J. Filipe (Eds.): WEBIST 2009, LNBIP 45, pp. 99–112, 2010. © Springer-Verlag Berlin Heidelberg 2010

100

I. Salomie et al.

originated, because the writing style changes over time. The embellishments that decorate the documents represent the third factor. Researchers also face the problem of the large quantity of available documents, thus making it hard to browse through all of them for finding the required information. Although content management systems capable of dealing with distributed data exist for years, making them able to process natural language documents is still a challenge. Addressing these issues, the ArhiNet research project [15] offers a solution to retrieval, processing and querying for relevant information from a large repository of heterogeneous documents regarding the history of Transylvania. The documents repository is created by processing archival documents from the Cluj County National Archives [16]. The ArhiNet system creates a knowledge base by processing the documents and annotating them with concepts and relations from the ArhiNet domain ontology. This way, we add a layer of machine-processable semantics over the content of raw documents contained in the archives by using (semi) automated knowledge acquisition. The obtained knowledge base, together with the document annotations, is used for solving ontology-guided natural language queries submitted by humans or software agents. The relevant query results represent documents, information and knowledge related to the semantics of the queries. Historians and archivists can use the system to ease the study of documents part of their work. The rest of the paper is organized as follows. In Section 2 we introduce related work. Section 3 presents our analysis of the archival domain, describing the document corpus and detailing our Archival Domain Model and Core Domain Ontology. The proposed system architecture and the associated workflows are described in Section 4. Section 6 contains a case study illustrating the system's functionality in the context of historical archives. The paper ends with our conclusions and future work proposals.

2 Related Work Regarding knowledge acquisition, among the most relevant approaches, we mention OntoPop [1], [2], SOBA [3] and Ontea [4]. OntoPop provides a single-step solution for (i) semantically annotating the content of documents and (ii) populating the ontology with the new instances found in the documents [1]. The solution uses domain-specific knowledge acquisition rules which link the results obtained from the information extraction tools to the ontology elements, thus creating a more formal representation (RDF or OWL) of the document content [2]. SOBA is a system designed to create a soccer specific knowledge base from heterogeneous sources [3]. SOBA performs (i) automatic document retrieval from the Web, (ii) linguistic annotation and information extraction using the Heart-of-Gold approach [5] and (iii) mapping of the annotated document parts on ontology elements [3]. Ontea performs semi-automatic annotation using regular expressions combined with lemmatization and indexing mechanisms [4]. The methodology was implemented and tested on English and Slovak content. In literature, the proposed models for managing large ontologies include XMLbased models [7], relational database models [8][9], graph-based models [10] and inmemory models [11]. For querying and reasoning, Ginseng [12], Gino [13] and PowerAqua [14] provide solutions for natural-language ontology guided user queries.

ArhiNet – A Knowledge-Based System

101

In our approach, we have adopted the OntoPop principles in the knowledge acquisition workflow, in addition considering synonym and homonym processing. Our approach performs (i) information extraction from unstructured text, (ii) domainspecific document annotation and (iii) uses reasoning on the ontologies to infer properties for the newly added instances. Our system was designed to process multilingual documents, including Latin languages, and so far it has provided good results for a corpus of Romanian documents, by using Romanian language specific resources. Moreover, the ArhiNet System also provides natural-language ontology guided query interface and introduces a hierarchical data model for efficiently storing, processing and reasoning on ontologies.

3 The Archival Domain – Corpus, Model and Domain Ontology In this section we present our analysis of the archival domain based on a corpus of documents provided by the historians from the Cluj County National Archives. 3.1 Archival Document Corpus We create our corpus by pre-processing a set of original archival documents (called ODoc documents) that capture historical facts that happened in the medieval Transylvania. The historical evolution of Transylvania is the main source of heterogeneity of the ODoc documents in the archives. Some specific factors that contribute to the heterogeneity are: (i) the documents’ language (Latin, Hungarian, German and Romanian), (ii) the institution that issued the document (different kinds of royal, local or religious authorities), (iii) whether the documents were printed or handwritten and (iv) the writing embellishments that decorate the documents. These characteristics lead to great difficulties in documents’ automatic processing for information extraction and therefore we decided to use as input for our system the document summaries created and provided by archivists. The ODoc documents are digitized, thus creating DDoc documents. Each DDoc document is manually processed by the archivists as part of their professional task. As a result of this processing, a PDoc (standing for Processed Document) document is generated. Each PDoc contains the technical data (PtDoc) and the summary (PsDoc) of the original document. The technical data included in the PtDoc may refer to the date of issue, archival fund, catalogue number or other metadata. In our case study, the document content actually represents a summary of the associated original document, expressed in natural language in an unstructured manner. Figure 1 presents the pre-processing steps that create the PtDoc and PsDoc starting from an ODoc. For the PDoc document of Figure 2, the PtDoc part contains the following technical data: the source ODoc was issued in March 25, 1320, the issue place is not known, a copy of the ODoc in Hungarian was written on paper in the 17th century, the original document is part of the Kemeny family fund, it is the 196th document from tome two of this fund, and the ODoc publication’s acronym is “D.I.R”, addressing the 13th century, volume one, pages 426-427.

102

I. Salomie et al.

Fig. 1. Historical documents pre processing steps

Fig. 2. Example of a PDoc document

In English, the PsDoc part reads: “Carol Robert, the king of Hungary, donates to Mihail and Nicolae, the sons of Albert of Juk, the Palostelek domain and the Imbuz (Omboz) forest, in the Dabaca County, for their faithful military services carried out together with the magistrate Stefan, against Moise, a rebel against the crown”. 3.2 Archival Domain Model This paper proposes a generic representation of the archival domain as illustrated in Figure 3. The archival domain is modeled starting from the raw medieval documents provided by the Cluj County National Archives (CCNA, 2008) presented in the previous subsection. These documents are hand written and contain many embellishments, making them hard to be automatically processed. Due to this difficulty, in our case studies we have used document summaries generated by the archivists (see Figure 2 for an example). Within our model, the central element is the document. Documents belong to a specific domain such as the historical domain or the medical domain. In our research we have used the historical archival domain, formally represented as domain knowledge by means of domain ontology (concepts and relations) and rules. Documents can be obtained from several data sources like external databases, Web

Fig. 3. Archival domain model


103

Fig. 4. The core of the domain ontology

sites or digitized manuscripts. Several documents may be related to one another by referring information about the same topics even if they are not containing the same lexical representations (e.g. names, events, etc.). When searching in the archival documents it is important to identify all documents that are related to a specified topic. To enable information retrieval from all relevant documents, the domain knowledge is used to add a semantic mark-up level to the documents content.The domain knowledge core (domain ontology and rules) is captured by processing and analyzing a large repository of archival documents, focusing on identifying their common concepts and relationships. Next, based on information extraction techniques applied on the PsDocs, the domain knowledge is enriched. 3.3 The Core of the Domain Ontology Ontologies capture and organize knowledge and offer support for reasoning. Our system uses the domain ontology in the processes of knowledge acquisition, document annotation and semantic querying. The development of the domain ontology is performed in two stages. During the first stage, a manual iterative process is performed to analyze the domain and to create the core of the domain ontology. The second stage is automatic and is performed by the system as it processes documents and performs reasoning on the ontology. This section presents the iterative process of core development, while the second stage is described in subsection IV/D. For designing the core of the ontology we have studied the medieval history of Transylvania and analyzed a large set of documents from the corpus together with historians and archive experts. We also considered what historians are interested to find from our system, like the owners of a certain territory for example. This way we have identified the relevant concepts and relations which are described below.

104

I. Salomie et al.

Fig. 5. ArhiNet Architecture

Places, persons, dates and events are the most relevant elements that appear in the documents. The names associated to these entities add uncertainty to reasoning about the possible synonyms and homonyms. We grouped territory types under the concept of TerritorialDivision. References to persons can appear as one or more names accompanied by a title and possibly the name of a territorial division (e.g. “Carol Robert, the king of Hungary”). Authorities play an important role because they were the only entities capable of issuing official documents. A particularity of an authority is that it can also appear in both the PsDoc and PtDoc parts of the document. The Event is also an important concept and in PsDocs it is often related to complaints, donations, occupation or recognition of property. Depending on the way the documents are phrased, it is possible to identify various elements connected to an event: the involved parties, the reason that triggered the event or its date. These elements were represented as properties of the concept Event. After several iterations and revisions, we obtained a final version of the core of our ontology about medieval Transylvanian history, illustrated in Figure 4. For the ontology development we have used Protégé [7].

4 The ArhiNet System This section presents the ArhiNet System layered architecture and three of the main workflows associated to the system’s conceptual layers.


105

4.1 System Architecture The system is structured on four interacting processing layers: (i) the raw data acquisition and representation layer, (ii) the knowledge acquisition layer, (iii) the OWL Ontology to HDM mapping layer and (iv) the knowledge processing and retrieval layer. The layers and their associated resources and processes are shown in Figure 5. The Primary DataBase (PDB) is used for raw document persistence, while the Knowledge Server (KS) is used for learning and reasoning tasks. The Raw Data Acquisition and Representation layer provides support for collecting and storing data in the Primary DataBase from multiple sources by means of Optical Character Recognition (OCR) techniques on raw documents, data import from external databases or by means of the system’s integrated user interface. The Knowledge Acquisition layer uses pattern-matching to extract relevant data from the raw documents. Based on the domain ontology and on a set of knowledge acquisition mapping rules, the documents are then semantically annotated. New concepts and instances are identified and added to the domain ontology as a result of this process. Within the OWL Ontology to HDM Mapping layer, the ontology resulting from the Knowledge Acquisition layer is mapped to a hierarchical data model that allows efficient reasoning and knowledge retrieval processes. The Knowledge Processing and Retrieval layer enables ontologically-guided natural language queries over the annotated documents and system knowledge. 4.2 Knowledge Acquisition The objective of the Knowledge Acquisition layer is to extend the domain knowledge by identifying, extracting and annotating the relevant domain-specific information from the summaries of archival documents. Knowledge Acquisition uses text mining techniques (tokenization, pattern matching and data structuring processes) applied in a pipeline fashion over the documents’ content. We adopted OntoPop [1][2] principles in designing the Knowledge Acquisition workflow (see Figure 6). Additionally, we have considered synonyms and homonyms processing. Synonyms population is required for identifying and processing ontology instances having several lexical forms with the same meaning in different documents. For example, the names “Palostelek” present in one document and “Paulusteleky” in another document have been identified and further processed as synonyms. Homonym identification and representation deals with common lexical representations for different instances (i.e. the name “Mihai” may refer either to the same person or to different persons in different documents). In the following, we describe the main activities of the knowledge acquisition workflow. Technical Data Extraction. This activity is responsible for separating the document technical data from its content (see Figure 2 for an example). Lexical Annotation. The objective of this activity is to identify and annotate the relevant lexical elements in the content based on pattern-matching rules. A pattern matching rule (see Figure 8) defines a relationship between the lexical elements and their annotation elements. The output of the lexical annotation activity consists of annotated lexical data represented in a hierarchic format of extracted words along with their annotation elements according to the pattern matching rule (see Figure 9).

106

I. Salomie et al.

Fig. 6. Knowledge Acquisition Pipeline

Knowledge Extraction. The objective of knowledge extraction activity is to use the domain ontology in order to semantically annotate the hierarchical structure of annotated lexical elements obtained in the previous activity. This activity is supported by a set of knowledge acquisition mapping rules. Each knowledge acquisition mapping rule (see Figure 10 in the Case Study for an example) defines (i) ways of associating the annotated lexical elements to ontology concepts and (ii) a set of actions for populating the ontology with instances and relations. The result of the knowledge extraction activity is an RDF structure stored in a file associated to the original document content (see Figure 11). Ontology Population. The ontology population is performed with instances and relations identified as a result of applying the knowledge acquisition mapping rules onto the lexically annotated documents, during Knowledge Extraction process. An instance is represented as follows: inst = {inst.doc.td, inst.val, inst.prop, inst.attr}, where (i) inst.doc.td represents the technical data of the document in which the instance appears, (ii) inst.val is the instance’s lexical representation, (iii) inst.prop is the set of properties in which the instance is involved and (iv) inst.attr is the set of attributes associated to the instance. Ontology population (see Figure 12) also deals with processing of synonym and homonym instances. In order to establish whether two instances are synonyms or homonyms, we defined a distance function d, taking as arguments a candidate instance (cinst) and an ontology instance (oinst), that allows us to differentiate between the following four cases: • If d(cinst, oinst) < θ and cinst.val = oinst.val then cinst is identical to oinst • If d(cinst, oinst) < θ and cinst.val ≠ oinst.val then cinst is different from oinst • If d(cinst, oinst) > θ and cinst.val = oinst.val then cinst is synonym of oinst • If d(cinst, oinst) > θ and cinst.val ≠ oinst.val then cinst is homonym of oinst


107

For adding to the ontology all the synonyms of an instance, we also use dictionaries. Due to instance population and to the addition of new properties, the structure of the ontology changes and inconsistencies may appear. We address these issues by introducing ontology management activities, aiming at (i) inferring new relations and properties as a result of ontology modification due to previous population processes and (ii) preserving ontology consistency. 4.3 OWL Ontology to Hierarchical Data Mapping The Hierarchical Data Model (HDM) Ontology Representation [ref] facilitates efficient ontology storage and efficient ontology reasoning and querying operations. Our HDM provides a generic ontology representation using hierarchies that can be persisted in a general purpose records-based structure. The OWL ontology elements are represented in the HDM using relational data entities, while the hierarchical ontology relations are represented using trees. The HDM handles the hierarchical relationships found in the internal OWL ontology structure (e.g. class inheritance). OWL uses classes, individuals, properties and datatypes to capture domain knowledge. By using the inheritance relationships, classes can be sub-classed, thus creating a tree of class structure based on parent - child relations. Individuals belonging to different classes generate a tree of class membership relations. The HDM ontology representation can be viewed as a forest of trees. Multiple OWL ontologies can be mapped in our HDM, thus enabling inheritance relationship propagation between ontologies. The HDM database support schema is static, allowing query optimization for better performance and the use of data analysis tools. The OWL to HDM mapping algorithm works as follows: (i) parses the input OWL ontology and creates the forest of trees representing the hierarchical relations contained in the OWL ontology, (ii) traverses the trees in a top-down manner, inserting each visited node into the database. This way the child-parent relationship is preserved and only add operations are performed on the hierarchical database model. 4.4 Knowledge Processing and Retrieving The aim of the knowledge processing and retrieving layer is to provide support for intelligent queries that enable searching for the most relevant information contained in archival documents. The system assists the user while entering queries in natural language by guiding his/her input according to a query grammar enhanced with ontology entities. User query triggers a complex reasoning process on the HDM represented ontology that includes synonym search, logical inferences and class-subclass and class-superclass searches. As a result, the set of query-relevant documents is identified and new relevant knowledge may be generated. Figure 7 presents the top-level design of the knowledge processing and retrieval layer, consisting of two main components: the query parser and the query processor. The query parser analyses the user input, offering suggestions based on the query grammar and ontology elements. The query grammar is specified in Extended Backus-Naur Form (EBNF). The current-word and next-word suggestions provided to the user are ranked by using a trigram model. When the user triggers the execution, the query is parsed and then translated into a hierarchy of inference engine atomic tasks.

108

I. Salomie et al.

Fig. 7. Knowledge processing and retrieval

The query processor uses the inference engine to solve the sub-queries and to ensemble the query result. The inference engine operates on the ontology persisted in the HDM and uses both inference techniques on the ontology and data mining techniques for reasoning purposes. Document searching is performed at two levels: one level relies on the technical data, which narrows the set of documents, while the other level relies on the semantic meaning of the user input query. Usually, in historical documents, several terms, such as person or location names, have different representations around a common root and it is essential to identify all documents containing these synonyms.

5 Case Study As a case study we have used a corpus of about 200 documents regarding the history of Transylvania, starting from the medieval period, supplied by the Cluj County National Archives. The original documents are found in Latin, Hungarian, German and Romanian. Each document is associated with a document summary in Romanian which highlights the events and participants. These summaries were used in our system as the raw documents, the main source of information. An example of such a document summary which is used for further exemplification throughout this section is presented in the PsDoc part of Figure 2. In order to specialize the generic knowledge acquisition workflow presented in Figure 6 for the historical domain, we have used the corpus for creating (i) the core of the historical domain ontology (see Figure 4), (ii) a set of specific pattern-matching rules for the annotation of the lexical elements and (iii) a set of specific knowledge acquisition rules for mapping the ontology concepts to the annotated lexical elements. Case Study - Lexical Annotation. We have used JAPE grammars [6] to create the set of specific pattern-matching rules for the annotation of the lexical elements. A JAPE grammar groups the rules that specify the actions to be performed when certain patterns are matched, into phases. Such a JAPE rule can be seen in Figure 8. The rule searches for instances of the child–parent relationship by looking for specific linguistic construction patterns. The presented rule, CandidatePersonComplex, finds phrasal patterns that associate titles (T) to persons (P) in order to annotate the lexical elements P and T as Complex Person (a lexical construct consisting of a name and a title).


109

Phase:PersonComplex Input:LookupTokenSpaceToken TempPersonTitleTitleComplex Options:control=appelt Rule:CandidatePersonComplex ( {TempPerson} {Token.kind==punctuation, Token.string==","}? SPACE ({TitleComplex}|{Title}) ):personComplex ͲͲ> :personComplex.PersonComplex= {kind="PersonComplex"}

Fig. 8. Example of JAPE rule

Fig. 9. Example of an LexAnnDoc XML File

For the identification of proper names in the raw documents, we used an existing Romanian gazetteer [6], that provides lists of Romanian words. We have enriched the gazetteer with additional lists that contain information specific to the addressed historical periods, such as events, kinship relations, titles, estates, etc. For achieving the objectives of the lexical annotation, we have employed the GATE API [6] to use the ANNIE [6] information extraction system. Within the process of annotating the lexical elements, the raw document is passed along with the gazetteer lists through the pipeline of JAPE grammars for extracting and structuring the relevant information. For the raw document shown in Figure 2, the result of the annotation of the lexical elements is the LexAnnDoc XML file shown in Figure 9. It contains a hierarchic structure of the identified lexical elements that are further semantically annotated by using a set of knowledge acquisition mapping rules (see Figure 10 for an example). Case Study – Semantic Annotation. The objective of this process is to associate ontology concepts to the lexical entities in the LexAnnDoc XML file and to populate the ontology with the lexical entities. The knowledge acquisition mapping rule specifies (i) how ontology concepts are mapped to the lexical elements and (ii) a set of operations that need to be performed on the ontology in order to store the new ontology elements (instance population, definition of properties and relations). The mapping rule of Figure 10 shows how the lexical tag PersonComplex, containing the child elements Person and TitleComplex, is semantically annotated with the ontology concept TitleComplex. The mapping rule also specifies the actions of (i) adding the identified lexical element identified as Person as instance of Person into the ontology and (ii) defining the hasCorrespondingTerritory relation between the added instance of Person and an instance of Territory.

110

I. Salomie et al.

For the currently processed raw document, the knowledge extraction process also generates an RDF file (see Figure 11) that contains RDF statements capturing the semantic annotations of the document. PersonComplex Person TitleComplex addInstance Person Person … hasCorrespondingTerritory Title Location

... razvratit magistru doneaza regele CarolRobert Ungariei Stefan Moise …

Fig. 10. Example of a knowledge acquisition mapping rule

Fig. 11. Example of RDF File

Case study – Ontology Population. After processing several documents within the knowledge acquisition workflow, the domain ontology was populated with new instances and properties (see Figure 12).

Fig. 12. Ontology population results


111

a)

b) Fig. 13. a) Example of a natural language ontology guided query. b) The result of the query from a).

The relevant documents and information obtained after executing the query illustrated in Figure 13a is presented in Figure 13b.

6 Conclusions and Future Work The present paper proposes a generic model of the archival domain and offers a technical solution for generating and processing semantically enhanced archival eContent. The solution follows three main workflows: (i) knowledge acquisition, (ii) OWL ontology to hierarchical database mapping and (iii) knowledge processing and retrieval. We adopt the OntoPop methodology to create knowledge acquisition mapping rules to semantically annotate the content of documents and populate the domain ontology. Synonyms and homonyms are also treaded during ontology population. The resulting OWL ontology is mapped to a hierarchical database model to allow efficient knowledge processing and retrieval. Reasoning techniques applied in knowledge processing and retrieval enable ontology-guided natural language queries aiming at the identification of relevant documents and knowledge. We tested our system on a corpus of 200 summaries of archival documents and obtained promising results. For specializing the knowledge acquisition pipeline on the archival domain we developed a set of 37 JAPE rules and the associated knowledge acquisition mapping rules. For future work, we intend to apply our solution to historical documents written in Hungarian, Latin and German and improving multilingual transparency for both document processing and document querying. Acknowledgements. This work was supported by the ArhiNet project funded by the Romanian Ministry of Education and Research.

112

I. Salomie et al.

References 1. Amardeilh, F.: Web Sémantique et Informatique Linguistique: Propositions Méthodolo giques et réalisation d’une plateforme logicielle. These de Doctorat, Universite Paris XNanterrere (2007) 2. Amardeilh, F.: OntoPop or how to annotate documents and populate ontologies from texts. In: Proceedings of the ESWC 2006 Workshop on Mastering the Gap: From Information Extraction to Semantic Representation, Budva, Montenegro, June 12. CEUR Workshop Proceedings (2006), ISSN 1613-0073 3. Buitelaar, P., Cimiano, P., Racioppa, S., Siegel, M.: Ontology-based Information Extraction with SOBA. In: Proceedings of the International Conference on Language Resources and Evaluation, pp. 2321–2324 (2006) 4. Laclavik, M., Ciglan, M., Seleng, M., Krajei, S.: Ontea: Semi-automatic Pattern based Text Annotation empowered with Information Retrieval Methods. In: Tools for acquisition, organisation and presenting of information and knowledge: Proceedings in Informatics and Information Technologies, Kosice, Vydavatelstvo STU, Bratislava, part 2, pp. 119– 129 (2007), ISBN 978-80-227-2716-7 5. Schäfer, U.: Integrating Deep and Shallow Natural Language Processing Components – Representations and Hybrid Architectures, Saarbrücken Dissertations in Computational Linguistics and Language Te, DFKI GmbH and Computational Linguistics Department, Saarland University, Saarbrücken, Germany (2007) 6. Tablan, V., Maynard, D., Bontcheva, K., Cunningham, H.: Gate – An Application Developer’s Guide (2004), http://gate.ac.uk/ 7. del Mar Roldán-García, M., Aldana-Montes, J.F.: A Tool for Storing OWL Using Database Technology. In: Proceedings of the OWLED 2005 Workshop on OWL: Experiences and Di-rections, Galway, Ireland, CEURWS.org (2005) 8. Vysniauskas, E., Nemuraite, L.: Transforming Ontology representation from OWL to relational Database. ISSN 1392 – 124x Information Technology and Control 35(3A), 333–343 (2006) 9. Zhuge, H., Xing, Y., Shi, P.: Resource Space Model, OWL and Database: Mapping and Integration. ACM Transactions on Internet Technology 8(4), Article 20 (2008) 10. Trissl, S., Leser, U.: Querying ontologies in relational database systems. In: Ludäscher, B., Raschid, L. (eds.) DILS 2005. LNCS (LNBI), vol. 3615, pp. 63–79. Springer, Heidelberg (2005) 11. Kalyanpur, A., Pastor, D.J., Battle, S., Padget, J.: Automatic Mapping of OWL Ontologies into JAVA. In: Proceedings of the Sixteenth International Conference on Engineering & Knowledge Engineering (SEKE 2004), Banff, Alberta, Canada (2004) 12. Bernstein, A., Kaufmann, E., Kaiser, C., Kiefer, C.: Ginseng: A Guided Input Natural Language Search Engine for Querying Ontologies. In: 2006 Jena User Conference, Bristol, U.K. (2006), http://www.ifi.uzh.ch/ddis/staff/goehring/btw/files/ Bernstein_JenaConf_2006.pdf 13. Bernstein, A., Kaufmann, E.: GINO – A Guided Input natural language Ontology Editor. In: Cruz, I., Decker, S., Allemang, D., Preist, C., Schwabe, D., Mika, P., Uschold, M., Aroyo, L.M. (eds.) ISWC 2006. LNCS, vol. 4273, pp. 144–157. Springer, Heidelberg (2006) 14. Lopez, V., Motta, E., Sabou, M., Fernandez, M.: Question Answering on the Real Semantic Web. In: 6th International and 2nd Asian Semantic Web Conference (ISWC 2007+ASWC 2007) (2007) 15. The ArhiNet Research Project, http://dsrl.coned.utcluj.ro/ 16. Cluj County National Archives (CCNA), http://www.clujnapoca.ro/arhivelenationale/

Optimizing Search and Ranking in Folksonomy Systems by Exploiting Context Information Fabian Abel, Nicola Henze, and Daniel Krause IVS – Semantic Web Group, Leibniz University Hannover Appelstr. 4, 30167 Hannover, Germany {abel,henze,krause}@kbs.uni-hannover.de

Abstract. Tagging systems enable users to annotate resources with freely chosen keywords. The evolving bunch of tag assignments is called folksonomy and there exist already some approaches that exploit folksonomies to improve resource retrieval. In this paper, we analyze and compare graph-based ranking algorithms: FolkRank and SocialPageRank. We enhance these algorithms by exploiting the context of tags, and evaluate the results on the GroupMe! dataset. In GroupMe!, users can organize and maintain arbitrary Web resources in self-defined groups. When users annotate resources in GroupMe!, this can be interpreted in context of a certain group. The grouping activity itself is easy for users to perform. However, it delivers valuable semantic information about resources and their context. We present GRank that uses the context information to improve and optimize the detection of relevant search results, and compare different strategies for ranking result lists in folksonomy systems. Keywords: Social media, Folksonomy systems, Search, Ranking, Optimization, FolkRank, GFolkRank, SocialPageRank, GRank.

1 Introduction Social interactions, participation in the content creation process, easy-to-use applications – these are among the usage characteristics of currently successful, so-called Web 2.0 applications. Users in Web 2.0 applications are more than ever active in the content life-cycle: They contribute with their opinion by annotating Web content (the so-called tagging), they add and annotate content (e.g. by using applications for sharing their bookmarks, pictures, videos, etc. with other users), they rate content, and they create content (e.g. with sorts of online diaries, so-called blogs). In this paper, we focus on the first type of applications: social tagging systems. In a social tagging system, users tag Web content, share these tags with other users of the application, and profit by the tagging activity of the whole user community by discovering / retrieving relevant Web content during browsing / as answers to search queries. The tagging activities are modeled in a folksonomy [11]: a taxonomy, which evolves over time when users (the folks) annotate resources with freely chosen keywords. Tagging systems such as Flickr1 moreover allow users to group resources (e.g., in sets or albums) which gains additional context information. 1

http://flickr.com


114

F. Abel, N. Henze, and D. Krause

Bao et al. showed that Web search can be improved by exploiting knowledge embodied in folksonomies [4]. In this paper, we introduce and evaluate different ranking strategies for folksonomy systems. The main contributions in this paper can be summarized as follows. – We propose a trainable algorithm, which exploits the context gained by grouping resources in folksonomy systems and which improves search for resources: GRank. – We compare existing ranking algorithms for folksonomies: FolkRank and SocialPageRank. We extend these algorithms and propose (a) group-sensitive FolkRank algorithms, and (b) a topic-sensitive SocialPageRank algorithm, and evaluate their quality with respect to search tasks. – We demonstrate how the GRank algorithm can be optimized and applied to improve the performance of ranking algorithms that are originally not group-sensitive. The paper is organized as follows. In the next section we discuss our work with respect to related work. In Section 3, we briefly introduce the functionality of the GroupMe! system. Section 4 presents a formal definition of folksonomies, and their extension with group structures. Afterwards, we identify the characteristics of the folksonomy, which builds the dataset of the GroupMe! application. Different folksonomy-based ranking strategies are discussed in the following section. Section 6 presents our evaluation results. The paper ends with conclusions.

2 Related Work In this paper we enhance ranking algorithms for folksonomies. We extend the FolkRank algorithm introduced in [7] with the capability of exploiting additional context information gained by GroupMe! folksonomies. Furthermore, we improve SocialPageRank [4] by enabling topic-sensitive rankings. In our experiment we evaluate ranking of resources whereas in [2] we focussed on ranking tags and evaluated the quality of different graph-based ranking algorithms with respect to tag recommendations. In [13] the authors propose an approach for recommending tags, which is based on co-occurences of tags. However, our evaluations in [2] indicate that graph-based recommender algorithms are more appropriate for folksonomies than strategies as described in [13]. When designing algorithms for folksonomy systems, the basic assumption is that tags describe the content of resources very well. In [9] the authors prove this assumption by comparing the actual content of web pages with tags assigned to these websites in the Delicious2 system.

3 GroupMe! Folksonomy System The GroupMe!3 Folksonomy System [1] is a fun-to-use Web 2.0 application. It is a resource sharing system like Delicious or Bibsonomy4, offering the extended feature 2 3 4

http://delicious.com http://groupme.org http://bibsonomy.org

Optimizing Search and Ranking in Folksonomy Systems

115

Fig. 1. GroupMe! group about WEBIST ’09 available at http://groupme.org/GroupMe/group/ 2671

of grouping Web resources. These GroupMe! groups can contain arbitrary multimedia resources like websites, photos or videos, which are visualized according to their media type: E.g., images are displayed as thumbnails and the headlines from RSS feeds are structured in a way that the most recent information are accessible by just one click. With this convenient visualization strategy, the user can grasp the content immediately without the need of visiting the original Web resource. Fig. 1 shows a group about WEBIST 2009 in Lisbon, which contains the website of the conference, a video with traveling information about Lisbon, a GroupMe! group about the last WEBIST conference, etc. GroupMe! groups are created by dragging & dropping multimedia resources from various sources into a group (cf. Fig. 1). We also offer a bookmarklet to add the currently visited Web site with a single click into the GroupMe! system. Building groups is a very convenient way of aggregating content. As groups are also normal Web resources, it is possible to group groups and hence, to model hierarchies. To foster Semantic Mashups, all data collected by the GroupMe! system is available in different formats: For a lightweight integration, we offer RSS feeds, which enable users easily to get informed if new content is added to a group. Further, GroupMe! adheres to the principles of Linked Data, which enables other applications to navigate through the semantically enriched data corpus.

4 Folksonomies Formally, folksonomies are defined as tuples of folksonomy entities, i.e. users, tags, and resources, and the bindings between these entities, which are called tag assignments

116


(a) Media type distribution in the GroupMe!

(b) Distribution of tag assignments.

Fig. 2. GroupMe! data set characteristics

and denote which user has assigned which tag to a certain resource. According to [6], a folksonomy can be defined as follows. Definition 1 (Folksonomy). A folksonomy is a quadruple F := (U, T, R, Y ), where: – U , T , R are finite sets of instances of users, tags, and resources – Y defines a relation, the tag assignment, between these sets, that is, Y ⊆ U ×T ×R GroupMe! extends this folksonomy definition by the concept of groups: Definition 2 (Group). A group is a finite set of resources. A group is a resource as well. Groups can be tagged or arranged in groups, which effects hierarchies among resources. In general, tagging of resources within the GroupMe! system is done in context of a group. Hence a GroupMe! folksonomy is formally characterized via Definition 3 (cf. [1]). Definition 3 (GroupMe! Folksonomy). A GroupMe! folksonomy is a 5-tuple F := ˘ G, Y˘ ), where: (U, T, R, – U , T , R, G are finite sets that contain instances of users, tags, resources, and groups ˘ = R ∪ G is the union of the set of resources and the set of groups – R ˘ × (G ∪ {ε}), where ε is – Y˘ defines a GroupMe! tag assignment: Y˘ ⊆ U × T × R a reserved symbol for the empty group context, i.e. a group that is not contained in another group when it is tagged by a user. 4.1 Folksonomy Characteristics in GroupMe! To decide whether known folksonomy search and ranking algorithms can be improved by considering the group context, we had a closer look on the tagging and grouping behavior of our users by analyzing a snapshot of the GroupMe! dataset, which contains 1546 unique tags, 2338 resources, 352 users, 453 groups, and 2690 tag assignments. The first question was whether users made use of the feature of grouping and visualizing different media types. In Fig. 2(a) we show the distribution of the different media types in the GroupMe! system.


117

Our observation is, that users use different media types and especially multimedia documents such as images or videos. About 40% of all resources in our system are multimedia documents, where tags form the main textual description, because extraction of other metadata is barely possible. In [9] the authors show that tags describe resources very precisely and are hence a valuable input for searching and ranking. GroupMe! motivates users to tag resources by using the free-for-all tagging approach (see [10]), which enables users to tag not only their own resources, but all resources within the GroupMe! system. On a logarithmic scale (extended with zero), we plotted the number of tag assignments on the y-axis and the number of resources having this number of tags assigned on the x-axis (see Fig. 2(b)). We observed a power law distribution of the tag assignments per resource, while about 50% of all resources do not even have a single tag assignment. That means, that 50% of all resources in the GroupMe! system can hardly be found by known folksonomy based search and ranking algorithms.

5 Folksonomy-Based Ranking Algorithms In this section we present different algorithms, which target on ranking folksonomy entities. We first introduce graph-based algorithms that can be applied to arbitrary folksonomy entities (users, tags, and resources). In Section 5.2 we describe algorithms, which specifically focus on ranking resources to support e.g. traditional search functionality in folksonomy systems. Our contributions, i.e. ranking algorithms we developed, can be summarized as follows: GFolkRank & GFolkRank+ . Graph-based ranking algorithms, which extend FolkRank [7] and turn it into a group-sensitive algorithm in order to exploit GroupMe! folksonomies (see Section 5.1). Personalized SocialPageRank. Extension to SocialPageRank [4], which allows for topic-sensitive rankings. GRank. A trainable search and ranking algorithm optimized for GroupMe! folksonomies. 5.1 Universal Ranking Strategies Universal ranking strategies like FolkRan and Group-sentitive FolkRank can be used to rank arbitrary parts of a folksonomy, e.g. users, resources, tags etc. FolkRank. FolkRank [7] adapts Personalized PageRank [12] for ranking users, tags, and resources in traditional folksonomies. w ← dAw + (1 − d)p

(1)

The adjacency matrix A models the folksonomy graph GF = (VF , EF ). GF is an undirected, weighted tripartite graph, which is created from the the folksonomy (cf. Definition 1). The set of nodes is VF = U ∪ T ∪ R and the set of edges is given via

118


EF = {{u, t}, {t, r}, {u, r}|(u, t, r) ∈ Y }}. The edges are weighted according to their frequency within the set of tag assignments. For example, w(t, r) = |{u ∈ U : (u, t, r) ∈ Y }| denotes the popularity of tag t for the resource r and counts the number of users, who have annotated r with t. w(u, t) and w(u, r) are defined accordingly. A is normalized so that each row has a 1-norm equal to 1. The influence of the preference vector p is configured via d ∈ [0, 1]. Finally, the FolkRank algorithm is defined as follows (see [7]). Definition 4 (FolkRank). The FolkRank algorithm computes a topic-specific ranking in folksonomies by executing the following steps: 1. 2. 3. 4.

p specifies the preference in a topic (e.g. preference for a given tag). w0 is the result of applying the Personalized PageRank with d = 1. w1 is the result of applying the Personalized PageRank with some d < 1. w = w1 − w 0 is the final weight vector. w[x] denotes the FolkRank of some x ∈ VF .

When applying FolkRank to GroupMe! folksonomies (see Definition 3) a straightforward approach is to ignore the group dimension of GroupMe! tag assignments. Therewith, the construction of the folksonomy graph GF = (VF , EF ) has to be adapted ˘ and EF = {{u, t}, {t, r}, {u, r}|u ∈ slightly. The set of nodes is given by VF = U ∪T ∪ R ˘ ˘ U, t ∈ T, r ∈ R, g ∈ G ∪ {ε}, (u, t, r, g) ∈ Y } defines the set of edges. Computation of weights is done correspondingly to the FolkRank algorithm, e.g. w(t, r) = |{u ∈ U : g ∈ G ∪ {ε}, (u, t, r, g) ∈ Y˘ }| is the number of users, who annotated resource r with tag t in any group. Group-sensitive FolkRank (GFolkRank). The traditional FolkRank does not make use of the additional structure of GroupMe! groups. In [3] we propose different adaptations of FolkRank, which exploit group structures in folksonomies, and show that they improve the ranking quality of FolkRank significantly (one-tailed t-test, significance level α = 0.05). In this paper we present one of these strategies, which we call GFolkRank. GFolkRank interprets groups as artificial, unique tags. If a user u adds a resource r to a group g then GFolkRank interprets this as a tag assignment (u, tg , r, ε), where tg ∈ TG is the artificial tag that identifies the group. The folksonomy graph GF is extended with additional vertices and edges. The set of vertices is expanded with the set of artificial tags TG : VG = VF ∪ TG . Furthermore, the set of edges EF is augmented ˘ u has added r to by EG = EF ∪ {{u, tg }, {tg , r}, {u, r}|u ∈ U, tg ∈ TG , r ∈ R, group g}. The new edges are weighted with a constant value wc as a resource is usually added only once to a certain group. We select wc = 5.0 ≈ max(|w(t, r)|) because we believe that grouping a resource is, in general, more valuable than tagging it. GFolkRank is consequently the FolkRank algorithm (cf. Section 5.1), which operates on basis of GG = (VG , EG ). GFolkRank+ denotes a strategy that extends GFolkRank with the feature of propagating tags, which have been assigned to a group, to its resources. The weight of edges e ∈ EG , which are caused by such inherited tag assignments, is adjusted by a dampen factor df ∈ [0, 1]. For our evaluations in Section 6 we set df = 0.2.


119

5.2 Ranking Resources In contrast to the FolkRank-based algorithms, which can be utilized to rank all types of folksonomy entities – i.e. users, tags, resources, and groups – we present SocialPageRank [4] and propose GRank, which is a search and ranking algorithm optimized for GroupMe! folksonomies. Both algorithms concentrate on ranking resources. SocialPageRank. The SocialPageRank algorithm [4] is motivated by the observation that there is a strong interdependency between the popularity of users, tags, and resources within a folksonomy. For example, resources become popular when they are annotated by many users with popular tags, while tags, on the other hand, become popular when many users attach them to popular resources. SocialPageRank constructs the folksonomy graph GF similarly to FolkRank. However, GF is modeled within three different adjacency matrices. AT R models the edges between tags and resources. The weight w(t, r) is computed as done in the FolkRank algorithm (cf. Section 5.1): w(t, r) = |{u ∈ U : (u, t, r) ∈ Y }|. The matrices ARU and AUT describe the edges between resources and users, and users and tags respectively. w(r, u) and w(u, t) are again determined correspondingly. The SocialPageRank algorithm results in a vector r, whose items indicate the social PageRank of a resource. Definition 5 (SocialPageRank). The SocialPageRank algorithm (see [4]) computes a ranking of resources in folksonomies by executing the following steps: 1. Input: Association matrices AT R , ARU , AUT , and a randomly chosen SocialPageRank vector r 0 . 2. until ri converges do: (a) ui = ATRU · r i (b) ti = ATUT · ui (c) r i = ATT R · ti (d) t i = AT R · r i (e) u i = AUT · t i (f) r i+1 = ARU · u i 3. Output: SocialPageRank vector r. SocialPageRank and FolkRank both base on the PageRank algorithm. Regarding the underlying random surfer model of PageRank [12], a remarkable difference between the algorithms relies on the types of links that can be followed by the “random surfer”. SocialPageRank restricts the “random surfer” to paths in the form of resource-user-tagresource-tag-user, whereas FolkRank is more flexible and allows e.g. also paths like resource-tag-resource. Personalized SocialPageRank. SocialPageRank computes a global ranking of resources in folksonomies. With the Personalized SocialPageRank algorithm we extend SocialPageRank introduced in [4] and transform into a topic-sensitive ranking algorithm. Therefor, we introduce the ability of emphasizing weights within the input matrices of SocialPageRank so that preferences can be considered, which are possibly adapted to a certain context. For example, w(t, r) is adapted as follows:

120


w(t, r) = pref (t) · pref (r) · |{u ∈ U : (u, t, r) ∈ Y }|, where pref (·) returns the preference score of t and r respectively. The preference function pref (·) is specified in equation 2: pref (x) =

1, if there is no preference in x c > 1, if there is a preference in x

(2)

In our evaluations (see Section 6) we utilized the Personalized SocialPageRank in order to align the SocialPageRank to the context of a keyword query tq and specified a preference into tq using c = 20. GroupMe! Ranking Algorithm (GRank). The most important application of ranking algorithms is search. In Definition 6 we introduce GRank, a search and ranking algorithm optimized for GroupMe! folksonomies. Definition 6 (GRank). The GRank algorithm computes a ranking for all resources, which are related to a tag tq with respect to the group structure of GroupMe! folksonomies (see Definition 3). It executes the following steps: 1. Input: keyword query tag tq . ˘a ∪ R ˘b ∪ R ˘c ∪ R ˘ d , where: ˘q = R 2. R ˘ a contains resources r ∈ R ˘ with w(tq , r) > 0 (a) R ˘ which are contained in a group g ∈ G with ˘ (b) Rb contains resources r ∈ R, w(tq , g) > 0 ˘ that are contained in a group g ∈ G, which ˘ c contains resources r ∈ R (c) R ˘ with w(tq , r ) > 0 and r = r contains at least one resource r ∈ R ˘ ˘ with w(tq , r ) > 0 (d) Rd contains groups g ∈ G, which contain resources r ∈ R ˘ q |, where w ˘ (r) returns the GRank of re3. wR˘ q is the ranking vector of size |R Rq ˘q source r ∈ R ˘ q do: 4. for each r ∈ R (a) wR˘ q (r) = w(tq , r) · da ˘ a do: (b) for each group g ∈ G ∩ R wR˘ q (r) + = w(tq , g) · db ˘ a where r is contained in a same (c) for each r ∈ R group as r and r = r do: wR˘ q (r) + = w(tq , r ) · dc (d) if(r ∈ G) then: ˘ a where r is contained in r do: for each r ∈ R wR˘ q (r) + = w(tq , r ) · dd 5. Output: GRank vector wR˘ q w(tq , r) is the weighting function defined in Section 5.1 and counts the number of users, ˘ with tag tq in any group. When dealing with multiwho have annotated resource r ∈ R keyword queries, GRank accumulates the different GRank vectors. The parameters da , db , dc , and dd allow to emphasize and train the weights gained by (a) directly assigned tags, (b) tags assigned to a group the resource is contained in, (c) tags assigned to neighboring resources, and (d) tags assigned to resources of a group. For the evaluations in Section 6 we set da = 10, db = 4, dc = 2, and dd = 4.


121

Table 1. Feature overview of the different ranking strategies presented in Section 5.1 and Section 5.2 Ranking applicable topic- groupStrategy for sensitive sensitive FolkRank [7] u, t, r yes no GFolkRank u, t, r yes yes + GFolkRank u, t, r yes yes SocialPageRank [4] r no no Pers. SocialPageRank r yes no GRank r yes yes

5.3 Synopsis Table 1 summarizes some features of the ranking strategies presented in the previous sections. The FolkRank-based algorithms are applicable for ranking of arbitrary folksonomy entities, i.e. users, tags, and resources. Furthermore, they are topic-sensitive, which claims that they do not compute a static ranking but allow to adapt rankings to a certain context. SocialPageRank computes static, global rankings independent of the context, which is e.g. given by a keyword query. With Personalized SocialPageRank we transformed SocialPageRank into a topic-sensitive ranking algorithm. GFolkRank, GFolkRank+, and GRank denote search and ranking strategies, which exploit group structures of GroupMe! folksonomies (cf. Definition 3) and are therewith groupsensitive.

6 Evaluations The most important application for ranking algorithms is search. Therefor, we evaluated the algorithms presented in Section 5 with respect to search for resources within the GroupMe! dataset, which is characterized in Section 4.1. Topic-sensitive ranking strategies can directly be applied to the task of searching for resources, e.g. FolkRank-based algorithms can model the search query within the preference vector (see Equation 1 in Section 5.1) in order to compute a ranked search result list. In [3] we evidence that our group-sensitive ranking algorithms like GFolkRank (see Section 5.1) improve the search and ranking quality significantly (one-tailed t-test, significance level α = 0.05) compared to FolkRank. Non-topic-sensitive ranking strategies – like SocialPageRank – compute global, static rankings and therewith need a baseline search algorithm, which delivers a base set of possibly relevant resources, which serve as input for the ranking algorithm. In our search experiments we formulate the task to be performed by the ranking strategies as follows. Search Task. Given a base set of possibly relevant resources, the task of the ranking algorithm is to put these resources into an order so that the most relevant resources appear at the very top of the ranking. 6.1 Metrics and Test Set For evaluating the quality of the ranking strategies with respect to the search task we utilized the OSim and KSim metrics as proposed in [5]. OSim(τ1 , τ2 ) enables us to determine the overlap between the top k resources of two rankings, τ1 and τ2 .

122


OSim(τ1 , τ2 ) =

|R1 ∩ R2 | k

(3)

˘ are the sets of resources that are contained in the top k of ranking where R1 , R2 ⊂ R τ1 and τ2 respectively, and |R1 | = |R2 | = k. KSim(τ1 , τ2 ) indicates the degree of pairwise distinct resources, ru and rv , within the top k that have the same relative order in both rankings. KSim(τ1 , τ2 ) =

|{(ru , rv ) : τ1 , τ2 agree on order of (ru , rv ), ru = rv }| |Rτ1 ∪τ2 | ∗ (|Rτ1 ∪τ2 | − 1)

(4)

Rτ1 ∪τ2 is the union of resources of both top k rankings. When detecting whether both rankings agree on the order of two resources, we use τ1 and τ2 . τ1 corresponds to ranking τ1 extended with resources R1 that are contained in the top k of τ2 and not contained in τ1 . We do not make any statements about the order of resources r ∈ R1 within ranking τ1 . τ2 is constructed correspondingly. In our analysis we apply OSim and KSim in order to compare rankings computed by the ranking strategies with optimal rankings. The optimal rankings are based on 50 hand-selected rankings: Given 10 keywords, which were out of the set of tags T , 5 experts independently created rankings for each of the keywords, which represented from their perspective the most precise top 20 ranking of resources. Therefore, they were enabled to inspect and the entire GroupMe! dataset. By building the average ranking for each keyword, we gained 10 optimal rankings. Among the 10 keywords, there are frequently used tags as well as seldom used ones. 6.2 Base Set Detection The base set contains all search results, which are finally returned as a search result list, where the order is computed by the ranking algorithm. Hence, it is important to have a search method, which produces a base set containing a high number of relevant resources (high recall) without loosing precision. Table 2 compares different base set detection methods with each other. Basic. Returns only those resources, which are directly annotated with the search key˘ a in Definition 6). word (cf. R BasicG. Returns in addition to Basic also resources, that are contained in groups anno˘a ∪ R ˘ b in Definition 6). tated with the query keyword (cf. R

Table 2. Comparison of different procedures to determine the basic set of relevant resources. Values are measured with respect to the test set described in Section 6.1. Base Set Algorithm Basic BasicG BasicG+

Recall 0.2767 0.5165 0.8853

Precision F-measure 0.9659 0.4301 0.7815 0.6220 0.6120 0.7237


123

BasicG+ . This approach exploits group structures more extensively. It corresponds to ˘ q in Definition 6). our GRank algorithm without ranking the resources (cf. R Having a recall of nearly 90%, BasicG+ clearly outperforms the other approaches. Though the precision is lower compared to Basic, which searches for directly annotated resources, the F-measure – the weighted mean of precision and recall – certifies the decisive superiority of BasicG+ . In our experiments we thus utilize the group-sensitive BasicG+ in order to discover the set of relevant resources to be ranked. All ranking algorithms therewith benefit from the power of BasicG+ . 6.3 Experiment In our experiment we proceed as follows. For each keyword query of our test set described in Section 6.1 and each ranking strategy presented in Section 5.1 and 5.2 we perform three steps. 1. Identification of the base set of possibly relevant resources by applying BasicG+ (see Section 6.2). 2. Execution of ranking algorithm to rank resources contained in the base set according to their relevance to the query. 3. Comparison of computed ranking with the optimal ranking of the test set by measuring OSim and KSim (see Section 6.1). Finally, we average the OSim/Ksim values for each ranking strategy. 6.4 Results Figures 3(a) and 3(b) present the results we obtained by running the experiments as described in the previous section. On average, the base set contains 58.9 resources and the average recall is 0.88 (cf. Table 2). The absolute OSim/KSim values are therewith influenced by the base set detection. For example, regarding the Top 20 results in Table 3(b), the best possible OSim value achievable by the ranking strategies is 0.92, whereas the worst possible value is 0.27, which is caused by the size and high precision of the base set. OSim and KSim both do not make any assertions about the relevance of the resources contained in the Top k. They measure the overlap of the top k rankings and the relative order of the ranked resources, respectively (see Section 6.1). Asexpected,thestrategy,whichranksresourcesrandomlyperformsworse.However,due tothehighqualityofthegroup-sensitivebasesetdetectionalgorithm,theperformanceofthe random strategy is still acceptable. SocialPageRank is outperformed by the topic-sensitive rankingalgorithms.PersonalizedSocialPageRank,thetopic-sensitiveversion,whichwedevelopedinSection5.2,improvestheOSim-performanceofSocialPageRankby16%andthe KSim-performanceby35%,regardingthetop10evaluations. The FolkRank-based strategies perform best, especially when analyzing the measured KSim values. Regarding the performance of SocialPageRank within the scope of the top 10 analysis, FolkRank, GFolkRank, and GFolkRank+ improve KSim by 132%, 110%, and 102% respectively. Here, the results evaluated by the OSim metrics also indicate an increase of the ranking quality, ranging from 58% to 71%.

124


It is important to clarify that all algorithms profit from the GRank algorithm, which is applied to detect the base set of possibly relevant resources. For example, if the topicsensitive FolkRank algorithm is used without GRank then the quality would decrease significantly by 17%/13% with respect to OSim/KSim. Moreover, GRank can compete with the FolkRank-based algorithms in re-ranking the set of possibly relevant resources and produces – with respect to OSim and KSim – high quality rankings as well. For example in our top 10 evaluations, GRank performs 65%/89% (OSim/KSim) better than SocialPageRank, whereas FolkRank improves GRank slightly by 5%/25% (OSim/KSim). The promising results of GRank are pleasing particularly because GRank does not require computationally intensive and time-consuming matrix operations as required by the other algorithms. The group-sensitive ranking strategies do not improve the ranking quality significantly. However, all ranking algorithms listed in Figures 3(a) and 3(b) benefit from the group-sensitive search algorithm, which determines the basic set and which supplies the best (regarding F-measure) set of resources that are relevant to the given query. 6.5 Optimizing GRank The Grank algorithm, which is introduced in Section 5.2 and applied as base set detection algorithm in our evaluation (cf. previous sections), positively impacts the quality of the other ranking algorithms in fulfilling the task of re-ranking a set of possibly relevant resources. Given a tag t as query the GRank algorithm ranks a resource r based on four features: (a) the number of users who assigned t to r, (b) the number of user who assigned t to a group where r is contained in, (c) the number of tag assignments where t was used for a resource that is grouped together with r, and (d) the number of tag assignments where t was used for a resource that is contained in r (if r is a group resource). The influence of these features can be adjusted via corresponding parameters da , db , dc , and dd . In our evaluations presented in the previous sections we set da = 10, db = 4, dc = 2, and dd = 4 which is founded by the results shown in Fig. 4(a-d). Fig. 4(a) depicts how OSim and KSim vary if da is altered from 0 to 20 while db , dc , and dd are constantly set to 1. The best performance with respect to OSim is outputted

(a) Top 10 OSim/KSim comparison.

(b) Top 20 OSim/KSim comparison.

Fig. 3. Top 10 and top 20 OSim/KSim comparison between different ranking strategies. Basic Set is determined via BasicG+ (cf. Section 6.2). In (a) the best possible OSim is 0.95 while the worst possible OSim: 0.04. In (b) the best possible OSim is 0.92 while the worst possible OSim is 0.27.


(a)

(b)

(c)

(d)

125

Fig. 4. Varying parameters da , db , dc , and dd of the GRank algorithm. In Figure (a) the influence of direct tags (da ) is altered from 0 to 20 while db , dc , and dd are constantly set to 1. In Figure (b), (c), and (d) the weights db , dc , and dd are varied correspondingly.

for da = 3 while KSim is maximized for da = 5 indicating that the first feature (number of user who assigned the query as tag to a resource) should be weighted stronger than the other features. In contrast, the influence of the neighboring resources should be rather small as indicated by Fig. 4(c) where OSim and KSim are maximized if dc is closed to zero and therewith smaller than da , db , and dd that are equal to 1. Increasing dc with respect to da , db , and dd results in a significant degradation of the OSim and KSim metrics. An examination of the GroupMe! data set explains that observation: GroupMe! groups are often created for a specific task such as travel planing (cf. Section 3). Hence, neighboring resources, i.e. resources that are contained in the same GroupMe! group, as well as their tags might be inhomogeneous. For example, a video with travel information might be grouped together with the website of a computer science conference a user plans to attend. Fig. 4(b) reveals that the consideration of tags, which are assigned to a group the resource to be ranked is contained in, is reasonable. Setting db as high as da , dc , and dd produces the best OSim results and increasing db to 3 gains the best results regarding KSim. In comparison to tags of neighboring resources that possibly introduce noise, group tags (on average each group has approx. 3 tags) are the more appropriate feature to consider which means that setting db > dc leads to a better OSim/KSim performance. Similarly, dd > dc leads to better results as clarified when comparing Fig. 4(d) and Fig. 4(c). Using machine learning techniques one can learn and optimize the adjustment of the GRank parameters. For example, the SV M rank algorithm [8] deduces an optimal

126


model with the following parameter assignments: da = 1.18, db = 0.21, dc = 0.14, and dd = 0.26. In comparison to the setting where all parameters are equally adjusted to 1, the deduced optimal model performs 24.4% better with respect to OSim and even 44.7% better with respect to KSim metrics.

7 Conclusions Folksonomy systems are valuable sources for improving search for Web resources. In this paper, we have described, proposed, and extended different graph-based ranking strategies for folksonomy systems, and evaluated, compared and optimized their performances with respect to ranking of search results. In addition, we analyzed the effect of using additional information about the context, in which some tagging activity took place, namely the group context provided by social resource sharing systems like GroupMe! that allow users to organize resources in groups, on search and ranking. With GRank we present a trainable search and ranking algorithm that exploits the group context. Our evaluations show that by exploiting group context we improve search performance in terms of both, recall as well as overall quality (measured via F-measure). The discussed graph-based ranking strategies overall perform very well in ranking search results. They have in common that they all adapt in one way or the other the PageRank [12] ideas. However, those strategies which utilize the full folksonomy information and are topic-sensitive perform best.

References 1. Abel, F., Frank, M., Henze, N., Krause, D., Plappert, D., Siehndel, P.: GroupMe! – Where Semantic Web meets Web 2.0. In: Aberer, K., Choi, K.-S., Noy, N., Allemang, D., Lee, K.-I., Nixon, L.J.B., Golbeck, J., Mika, P., Maynard, D., Mizoguchi, R., Schreiber, G., CudréMauroux, P. (eds.) ASWC 2007 and ISWC 2007. LNCS, vol. 4825, pp. 871–878. Springer, Heidelberg (2007) 2. Abel, F., Henze, N., Krause, D.: Exploiting additional Context for Graph-based Tag Recommendations in Folksonomy Systems. In: Int. Conf. on Web Intelligence and Intelligent Agent Technology (WI-IAT 2008). ACM Press, New York (2008) 3. Abel, F., Henze, N., Krause, D., Kriesell, M.: On the effect of group structures on ranking strategies in folksonomies. In: King, I., Baeza-Yates, R. (eds.) Weaving Services and People on the World Wide Web. Springer, Heidelberg (2009) 4. Bao, S., Xue, G., Wu, X., Yu, Y., Fei, B., Su, Z.: Optimizing Web Search using Social Annotations. In: Proc. of 16th Int. World Wide Web Conference (WWW 2007), pp. 501–510. ACM Press, New York (2007) 5. Haveliwala, T.H.: Topic-Sensitive PageRank: A Context-Sensitive Ranking Algorithm for Web Search. Transactions on Knowledge and Data Engineering 15(4), 784–796 (2003) 6. Hotho, A., Jäschke, R., Schmitz, C., Stumme, G.: BibSonomy: A Social Bookmark and Publication Sharing System. In: Proc. First Conceptual Structures Tool Interoperability Workshop, Aalborg, pp. 87–102 (2006) 7. Hotho, A., Jäschke, R., Schmitz, C., Stumme, G.: FolkRank: A Ranking Algorithm for Folksonomies. In: Sure, Y., Domingue, J. (eds.) ESWC 2006. LNCS, vol. 4011, pp. 411–426. Springer, Heidelberg (2006)


127

8. Joachims, T.: Optimizing search engines using clickthrough data. In: Proc. of eighth International Conference on Knowledge Discovery and Data Mining (KDD), Edmonton, Alberta, Canada, pp. 133–142. ACM Press, New York (2002) 9. Li, X., Guo, L., Zhao, Y.E.: Tag-based social interest discovery. In: Proc. of the 17th Int. World Wide Web Conference (WWW 2008), pp. 675–684. ACM Press, New York (2008) 10. Marlow, C., Naaman, M., Boyd, D., Davis, M.: HT06, tagging paper, taxonomy, flickr, academic article, to read. In: Proc. of the 17th Conf. on Hypertext and Hypermedia, pp. 31–40. ACM Press, New York (2006) 11. Vander Wal, T.: Folksonomy (2007), http://vanderwal.net/folksonomy.html 12. Page, L., Brin, S., Motwani, R., Winograd, T.: The PageRank Citation Ranking: Bringing Order to the Web. Technical report, Stanford Digital Library Technologies Project (1998) 13. Sigurbjörnsson, B., van Zwol, R.: Flickr tag recommendation based on collective knowledge. In: Proc. of 17th Int. World Wide Web Conference (WWW 2008), pp. 327–336. ACM Press, New York (2008)

Adaptation of the Domain Ontology for Different User Profiles: Application to Conformity Checking in Construction Anastasiya Yurchyshyna1,2, Catherine Faron-Zucker1, Nhan Le Thanh1, and Alain Zarli2 1

I3S, UNSA-CNRS, 930 route des Colles, BP 145, 06903, Sophia Antipolis, France {Catherine.Faron-Zucker,Nhan-Le-Thahn}@unice.fr 2 CSTB, 290 route des Lucioles, BP 209, 06904, Sophia Antipolis, France {anastasiya.yurchyshyna,alain.zarli}@cstb.fr

Abstract. This paper presents a method for the adaptation of domain ontologies to different contexts and user profiles. It is applied to the modeling of conformity checking in construction. Considering a method we have developed to acquire and represent the expert knowledge involved in the conformity checking process, we highlight the need to reduce the gap between the expert knowledge primary captured in the domain ontology and the end user knowledge. We propose a method to contextualize the ontological knowledge initially acquired and to adapt it to different user profiles. Keywords: Adaptation of domain ontology, conformity-checking modeling, user profiles, validation by usage, semantic search.

1 Introduction Conformity checking in construction is a very complex and multidisciplinary problem. Its complexity can be explained by the following factors: (i) a great variety of heterogeneous components defining the conformity checking (e.g. modeling of construction regulations, reasoning on conformity), (ii) the interdependence of various actors of the construction domain; (iii) the large amount of non formalized expert knowledge guiding the process, (iv) the great volumes of construction data to be retrieved and maintained. The key problem in conformity checking in construction is to automate the process of checking whether a construction project (e.g. a private house, a public building, a non-building installation) is conform to a set of conformity requirements described by regulation texts. The semantic complexity of this problem requires an expressive formalism for representing the knowledge of the checking process. Recently, multiple approaches for the development of building-oriented ontologies have been developed: e-COGNOS [8], ifcOWL [12], buildingSMART [3]. Despite the variety of these approaches, these generic ontologies can be hardly used for the specific aim of conformity-checking because of many unsolved aspects. First, in most practical cases, the checking remains J. Cordeiro and J. Filipe (Eds.): WEBIST 2009, LNBIP 45, pp. 128–141, 2010. © Springer-Verlag Berlin Heidelberg 2010

Adaptation of the Domain Ontology for Different User Profiles

129

manual and, as a result, not very effective; there is no formalised approach for the automatic conformity checking. Second, the procedures of checking are not comprehensive and not transparent for non-professionals and/or for experts having different user profiles. A lot of results are deduced and/or interpreted only with the help of tacit expert knowledge, which is non-formalised or which formalisation is time-/resourceconsuming. Third, more expressive formalisms for representing domain knowledge taking part in checking are required. Finally, the existing software still imitates simple manual procedures and/or practices shared by domain experts. To address these limitations, we have developed a conformity-checking ontology, which integrates not only building-related knowledge, but also the knowledge on conformity regulation texts and the expert knowledge on checking procedures [23]. Developed with the help of domain experts, mostly from CSTB (Centre Scientifique et Technique du Bâtiment), our conformity-checking ontology was a key component of our conformity-checking model. Complex and multidisciplinary, the construction industry is a field of collaborative work and communication of multiple actors, the so-called “key players of the building-oriented market”, who form the target audience for the development of the construction sector and their needs define the innovation process of the industry. They are (i) architects generating data related to different aspects of a building; (ii) engineers responsible for generating data related to a specified facility’s system of a building; (iii) contractors dealing with process-related characteristics of a building (scheduling, cost analysis, project management, etc.); (iv) consumers of a building product; (v) building product manufacturers generating supplementary data related to a building product (e.g. physical and functional characteristics, cost); (vi) legal authorities formulating performance-oriented rules of the development of a building. Obviously, these different actors of the conformity checking in construction have different needs and understanding of the checking process. It also means that they may interpret and use the knowledge from the domain ontology in different ways (e.g. an acoustic engineer may need a very detailed “version” of the domain sub ontology concerning acoustics, in contrast to a final user interested in general non-detailed conformity recommendations). The domain tacit knowledge plays an important role in the conformity checking process. To take it into account, we developed an approach for formalizing expert knowledge, and integrate it in the conformity-checking ontology [24]. On the other hand, it is always interesting to enrich the initial ontology by the knowledge that is acquired from its usage. This is what motivates our current work on the validation of the conformity-checking ontology by integrating the results of semantic search processes. The interdependence of the actors of the application domain (i.e. conformity checking in construction) is, however, an important factor to take into account by the application of a generic domain ontology. In this context, it is a real challenge to enrich our approach for the ontology development by the knowledge of its usage by different user groups and to refine the ontology for different user profiles. This paper addresses this problem and presents the DOUP method to adapt the domain ontology to different user profiles.

130

A. Yurchyshyna et al.

The paper is organized as follows. In section 2, we discuss some ontology-based approaches for formalizing domain knowledge and particularly focus on methods for the ontology development for different user profiles. Section 3 describes our approach for an improved development of the domain ontology for different user profiles (the DOUP-method). The implementation of this approach is discussed in section 4. Finally, we describe our ongoing works and the perspectives of our research.

2 Three Motivations and Approaches to the Adaptation of Domain Ontologies for Different User Profiles The problem of the development of a domain ontology is first characterized by a large amount of tacit knowledge [18] that makes the whole domain more expressive [13]. We identify five main requirements in the development of such an ontology. First, a large amount of the knowledge to be formalized in a domain ontology is tacit indeed. For example, the conformity-checking in construction is characterised by: (i) the “know-what” knowledge of the construction industry commonly known by architects; and (ii) the “know-how” knowledge of the checking process shared by conformity experts. We particularly focus on the methods for capitalising the tacit “know-how” knowledge and expert practices that we discussed in our previous works [24]. Second, before formal representation, the domain knowledge should be first interpreted by domain experts. Even if the experts are the professionals of the domain, it is obvious that such interpretation remains rather subjective and/or partial [6]. Third, the development of the domain ontology is driven by its (future) application. For that reason, the development of a domain ontology is often a part of some more global task (e.g. conformity-checking modelling, semantic search, etc.). Moreover, the formalisms used for development should be seamlessly interconnected with the formalisms of these other connected tasks and should be based on interoperable standards. For example, the development of the buildingSMART ontology [3] is coordinated with the elaboration of the SMARTcodes™, the code provisions for compliance checking [21], as well as regulation-centric knowledge representation formalisms define the conceptual architecture of the conformance assistance framework [16]. Fourth, the expert interpretation sometimes differs from the understanding of end users, who may find the knowledge “not adequate” and “difficult to use”, but fail to express the exact meaning of the concepts used (e.g. a user may find it difficult to distinguish between “main door” and “entrance door”, but does not use these two concepts in the same way). Fifth, the domain ontology is often defined in a specified context [4], [14], which should be integrated in a collaboration process and then validated by usage. The problem of construction collaboration studied in [2] is a good example of process integration and context modelling, which are based on two aspects of the Semantic Web vision: (i) the awareness of user context (such as user profile/role, preferences, task, location, existing project conditions, etc.) for delivering pertinent information to exchange, as well as (ii) the development of corresponding web services, which allow a system to dynamically discover and invoke services.


131

In addition to handling tacit knowledge, the development of a generic domain ontology requires to take into account the practical usage of it by different user groups. Such usage may cause problems for the following reasons: (i) The interpretation of the domain knowledge by end users may be different from the interpretation of domain experts even if the ontologies are developed to share the human oriented semantics [1]. (ii) Different groups of end users may have different scopes of interest (e.g. an architect and a legal authority need different levels of detail for conformity-related knowledge). (iii) A large amount of knowledge remains tacit, and, consequently, end users might not define if they really need to use this knowledge (e.g. in the case of checking the conformity of a public building, a user may be interested only in checking its accessibility, but not its acoustic requirements, which are part of the global conformity-checking). Finally, the development of a domain ontology calls for handling different user profiles. A general approach for personalizing the user's environment and integrating the user profiles into the development of information services is discussed in [22]. The main methods for the automatic creating and application of user profiles are discussed in [12]. These methods allow integrating search results tailored to individual users to more complex systems and thus to personalize the application of such systems. In [21], the authors propose a general approach for representing the user context by assigning interest scores to existing concepts in a domain ontology. We focus on these three motivations to the development of the domain ontology for different user profiles to define our approach for the improved development of the domain ontology for conformity-checking in construction, which allows the personalisation of the domain knowledge for different user profiles.

3 Our Approach for the Adaptation of the Domain Ontology to Different User Profiles In order to adapt the domain ontology for different user profiles, we have developed the so-called DOUP-method (Domain Ontology for different User Profiles) that can be decomposed into three steps: 1. a knowledge representation and acquisition method developed for our conformity-checking model [23]. 2. a method of context modelling of the ontology by integrating the results of semantic search (the CMSS-method) [25]. 3. a method for the adaptation of the domain ontology to different user profiles (the AUP-method). 3.1 Knowledge Representation and Acquisition Method We adopt an ontological approach and semantic web technologies [5] to develop a knowledge representation and acquisition method (cf. Figure 1) that allows us to represent complex and multidisciplinary knowledge characterizing the conformitychecking process in construction. In this section, we briefly describe the main ideas of the KRA method. A more detailed presentation and corresponding examples can be found in [23].

132


Fig. 1. Knowledge representation and acquisition method

The first phase of our method aims to acquire the formal representations of conformity requirements expressed by technical construction norms. We have developed a base of accessibility queries by extracting them from the CD REEF, the electronic encyclopaedia of construction texts and regulations, edited by CSTB, and by formalizing them as SPARQL queries in collaboration with construction experts from the CSTB. The second phase aims at the semi-automatic development of an ontology oriented conformity checking on the basis of the concepts of the IFC construction model [10], which occurs in the acquired SPARQL conformity queries. These concepts are organized and described in the OWL-Lite language. The acquired ontology is then enriched by non-IFC concepts occurring in the conformity queries. The intervention of domain experts is required in this case to define new non-IFC concepts in terms of the checking ontology (e.g. GroundFloor class is defined as a IfcBuildingStorey situated at the level of entering into a building). The third phase is dedicated to the acquisition of a construction project representation oriented conformity checking. This representation is based on its initial ifcXML representation and is guided by the acquired conformity-checking ontology. We develop an XSLT stylesheet that filters this ifcXML to extract the data relative to the conformity checking ontology and organizes them as RDF triples. The acquired RDF data is then enriched with non-IFC concepts extracted from conformity queries (e.g. a project representation is enriched by using the GroundFloor concept calculated on the basis of its initial IFC-based data (e.g. IfcDoor, IfcStair, etc.) The acquired queries, however, only contain conformity constraints, they have no supplementary information to guide the checking process: e.g. the scheduling of queries. The forth phase of our method thus aims at the semantic annotation of conformity queries. We propose a special RDF annotation of a query, developed according to its tag-based context: possible values for certain tags are concepts/properties of the conformity-checking ontology. To do it, we combine two main methods of document annotation: annotation by content of the document and annotation by its external sources [17]. First, we annotate a query by its content. To do this, we define a set of key concepts of this query, which describe what is really checked by this conformity requirement. In other words, we define keyConcept tag in the RDF annotation of a query, which value is a list of


133

concepts from the conformity-checking ontology extracted from the SPARQL representation of this query. We remark also that there is a semantic correspondence between different types of knowledge used for query annotation. For example, a conformity query defining the physical dimensions of a door is annotated by a Door concept from our conformity-checking ontology. Second, we annotate a conformity query according to external sources. Such annotation allows representing different types of knowledge. First, they are characteristics of the regulation text from which the query was extracted: (i) thematic (e.g. accessibility); (ii) regulation type (e.g. circular); (iii) complex title composed of the title, publication date, references, etc.; (iv) level of application (e.g. national), (v) destination of a building (e.g. private house). Second, they are characteristics of knowledge extraction process: (i) article, (ii) paragraph from which a query was extracted. Third, it is formalized expert knowledge: tacit « common knowledge » on the process of conformity-checking that is commonly applied by domain experts: (i) knowledge on domain and sub domain of the application of a query (e.g. Stairs); (ii) knowledge on checking practice (e.g. if a room is adapted, it is always accessible). Fourth, it is the application context of a query. This group specifies the aspects of query application for certain use cases. For example, the requirements on the maximal height of stairs handrail vary from 96 cm (for adults) to 76 cm (for kids). In this case, it is important to know the destination of a building (e.g. school). Characteristics and possible values of the first two groups are automatically extracted from the CD REEF. The knowledge described by the last two groups is defined partially and/or has to be explicitly formalized by domain experts. The knowledge acquired by the KRA-method is then used in our conformitychecking model that is based on the matching of the representations of a construction project and those of conformity queries [23]. 3.2 Context Modeling of the Domain Ontology by Integrating the Results of Semantic Search According to the knowledge acquisition method described above, the conformitychecking ontology is developed with the help of domain experts and does not depend on the conformity-checking process. All concepts and relations of the ontology are defined and validated by domain experts before the checking process and can not be changed in the process of checking. Domain experts also formulate rules of definition of new concepts, context rules and, in general, they validate the whole knowledge base of the conformity-checking process. In some cases, such definitions can be partial or inadequate, and it does not represent the real usage-driven conformity-related knowledge of the checking process: even the definition of domain experts is not sufficient to represent the whole complexity of the checking knowledge. To answer this problem, we propose to acquire the knowledge on the conformity checking practices, which turns out explicit thanks to a large number of checking operations by different non-expert end users. Conceptually, our approach for integrating user-context awareness is close to [2], which integrates the following information: (i) current location; (ii) user-device profile; (iii) user identity; (iv) user’s current activity; (v) visual context; (vi) time via computer clock.

134


Fig. 2. Conformity-checking model

In the context of our research, we are mostly interested in three main context-based characteristics: (i) location and its interpretation for the project data: the current “location of checking” (e.g. if a user has already chosen to check a one-floor building, then all checking requirements on “vertical circulation” should be skipped); (ii) typical user scenario according to the mostly used scenario that corresponds to the initial algorithm of scheduling conformity queries, which is validated by construction experts [24]; (iii) user profile and its interpretation for the project data: the role of the current user (e.g. a non-professional user can only use the conformity knowledge base, while an expert is allowed to validate the formalized queries). To do this, we adopted an approach of context-based modeling of the ontology by integrating the results of semantic searches of end users. The so called CMSS method improves our CMV method described in [25] by taking into account the semantic proximity of different concepts or relations in the conformity-checking ontology, according to the interpretation of end users. The CMSS method is based on the semantic annotations of conformity queries. It aims to analyze the simultaneous choice of the queries, which are annotated by the same key concepts, and thus to define the semantic similarity between these concepts. In our knowledge acquisition method described above, the semantic annotations of conformity queries are developed according to the tag-based context: possible values for certain tags of semantic annotations of queries are concepts or properties of the conformity-checking ontology. The following example illustrates an annotation of a query by the door concept of the conformity-checking ontology.

... ... Our work on modeling the domain ontology for conformity checking in construction is conducted under the Semantic Web vision, which guarantees more advances capabilities for processing the knowledge. In particular, we provide the users with semantic search capabilities that give better results than traditional search mechanisms.


135

The semantic annotation of conformity queries allows us to propose a user more detailed selection of queries to be checked. For example, for a user interested in checking the conformity of a door, we can propose the semantic search that will give a semantically richer result: it will interrogate the domain ontology to define not only the queries annotated by Door, but also all the corresponding ones (its subclasses Entrance, EntranceDoor, FrontDoor, AccessibleEntrance). It also means that a user can obtain a semantically consistent answer about the content of the conformity query before executing it – only by its RDF annotation – and thus to identify what he really wishes to check. Technically, such semantic search is based on the execution of the following SPARQL query against a base of RDF semantic annotations. PREFIX a: PREFIX ontoCC: SELECT ?s ?nQuery ?appValue ?cCl WHERE { ?s direct::rdfs:subClassOf ?cCl FILTER(?s ôntoCC:) ?nQuery a:domainApplication ?appValue ?appValue rdf:type ?cCl FILTER (?cCl ~ 'door') } In our example, the search of “door” expression will give the list of queries, which application domain contains “door” and classifies them according to the conformitychecking ontology. In comparison to the traditional search (that results with the only answer “door”), the semantic search will detail the application domain of the retrieved queries and classify them into subclasses: (i) “door”; (ii) “entrance door”, “front door”, “entrance”; (iii) “accessible entrance” (cf. Figure 3). The advantage of such semantic search is that it is defined according to the general domain knowledge of the construction industry, formalized in the conformitychecking ontology, which is independent of an end user, but helps him to detail the search of corresponding conformity requirements and thus to refine the algorithms of their application during this process. Another advantage of our approach for the semantic search of conformity queries is that the results of the semantic search followed by the user selection of a query can then be used to validate the initial domain ontology. To illustrate these ideas, let us take three subclasses of a Door class: FrontDoor, Entrance and EntranceDoor, which are defined as equivalent in the conformitychecking ontology. They are also used as key concepts to annotate conformity queries (e.g. these three concepts annotate the query “an entrance door of any building should be accessible to disabled persons”). According to our model, for the checking of the conformity of an entrance door of a building, a construction project should be checked by the queries annotated by all these three concepts. A full list of these queries will be thus proposed to an end user. In some cases, this list turns out redundant when the end user has no interest in some specific queries (e.g. the one concerning the luminosity of an entrance door of a school). It is, therefore, important to evaluate the cohesion between the queries chosen and rejected by an end user and the corresponding key concepts annotating these queries. For example, we can notice that queries annotated by Entrance and EntranceDoor are chosen more frequently than the ones annotated by Door. Intuitively, Entrance and EntranceDoor are semantically closer than Entrance and Door (cf. Figure 4).

136


Fig. 3. Door and its subclasses

Fig. 4. Semantic distances between subclasses of Door

To propose a formal definition of the validation of the conformity-checking ontology by usage, we first define our approach on the evaluation of the concepts of the conformity-checking ontology. It is based on three main criteria [15] adapted for the conformity-checking problematic: (i) credibility degree: we suppose that all concepts and properties of the conformity-checking ontology are defined by construction experts, their definitions are pertinent and correct with a credibility degree equal to 1; (ii) cohesion degree: we suppose that our conformity-checking ontology is homogeneous: there are subclasses of a class which are declared equivalent by domain experts (e.g. door, entrance, entranceDoor); (iii) eligibility degree: concepts and relations are defined by experts and added to the conformity-checking ontology, if they are necessary for the formalization of conformity queries. Our approach of context-based validation of the conformity-checking ontology by usage is developed according to the same criteria (cf. Figure 5), in order to keep the semantic consistency of the conformity-checking ontology: (i) credibility degree: no concepts or relations can be defined by non-expert end users; (ii) cohesion degree: the distance between the equivalent concepts is then recalculated according to the frequency of their simultaneous choice by non-expert end users (e.g. Entrance and EntranceDoor are chosen more often); (iii) eligibility degree: if some classes of semantically close concepts are defined, it can be interesting to identify the concept characterizing the whole class, e.g. EntranceDoor for the class containing Entrance, AccessibleEntrance, FrontDoor, etc. By identifying the representative concept of the class, we can refine the semantic annotation of the corresponding queries (for example, annotating them only by this concept) and, consequently, the algorithms of expert reasoning (for example, we do not need to schedule queries which are annotated by the concepts of the same class).


137

Fig. 5. Towards a context-based validation of the domain ontology by usage

To model the semantic distances in the conformity-checking ontology, we base on the semantic similarity in content-based retrieval systems [9] and the approach of the “intelligent evaluation” [15] of ontological concepts. Currently, we work on the detailed development of the conceptual approach for the evaluation of the concepts of the conformity-checking ontology. 3.3 Towards Adaptation of the Domain Ontology to Different User Profiles Our method for the development and usage-based validation of the domain ontology remains still generic and not adapted to the variety of different actors of the construction domain. For this reason, it is a real challenge to propose an approach for adapting it for different user profiles: e.g. architect, electric engineer, legal authority. In order to adapt the acquired domain ontology for different user profiles, we propose to enrich our CMSS-method by personalizing it for different user profiles. Our so called AUP method for the Adaptation of the domain ontology for different User Profiles distinguishes two main steps. First, we identify the groups of users and the corresponding user profiles. For each user profile, we create a copy of the initial generic domain ontology: e.g. the conformity-checking ontology for (i) architects; (ii) electric engineers; (iii) conformity-checking experts; and (iv) non-professional end users. Second, we define the scope of interest for each user profile. To do this, we apply the CMSS-method for each group of users and modify their copy of the domain ontology according to its usage by the corresponding end users. Then we generalise or detail the domain ontology according to the scope of interest of user profiles. For example, it is important only for an architect to distinguish between different types of entrances (cf. Figure 6). It is important to underline that the AUP method guarantees the coherence and semantic consistency between the generic domain ontology and its facets developed for different user profiles. This coherence is based on the following aspects. First, the facets of the initial domain ontology developed for different user profiles are credible: the users cannot create new concepts; they can only refine the distances between the concepts of the initial ontology. Second, the distance between synonym concepts is recalculated according to the frequency of their simultaneous choice by users of the same user profile. We do not, however, aim at establishing correspondences between

138


Fig. 6. Semantic distances between subclasses of Door for different user profiles

different user profiles. Third, if users with the same profile define semantically close concepts, these concepts are grouped and associated to a representative eligible concept: the closest super class of these semantically close concepts. For example, classes Entrance, EntranceDoor and FrontDoor of the initial domain ontology are represented by class Entrance in the ontology facet for electric engineers. As a result of the personalisation of the domain ontology for different user profiles, the rules of the definition of new concepts and the algorithms of scheduling the conformity queries should be also refined, in order to satisfy the “personalised” semantics for each group of users. The usage of these adapted rules and algorithms [24] will, consequently, become more effective, and the generated recommendations on conformity – more practical.

4 Implementation: Profiling the C3R Prototype In our previous work on the development of the conformity-checking model [23], we have developed the C3R (Conformity Checking in Construction) system (cf. Figure 7) that implements expert reasoning by organizing and applying conformity queries. For the conformity checking operation itself, C3R relies on the semantic search engine CORESE [7], which answers SPARQL queries asked against an RDF/OWL Lite knowledge base; and SEWESE [19], the JSP/Servlet/Corese environment for building Semantic Web applications. The main components of the C3R prototypes are: (i) the knowledge acquisition module (query formalizer, ontology editor; construction project extractor); (ii) the reasoning module (checking reasoner enabled by the CORESE engine; query scheduler, conformity report generator); (iii) the module on capitalization of context knowledge (query base generator; annotation editor; expert reasoning explorer; formalizer of usage-based knowledge). For the C3R prototype, we have defined a conformity-checking ontology that currently comprises around 2200 concepts and 1600 properties. The conformity-checking ontology is written in OWL-Lite, which is rather expressible and, at the same time, decidable. We also define about 50 definition rules describing new concepts and properties with the help of the ones from the conformity-checking ontology.


139

Fig. 7. C3R infrastructure

To develop a base of conformity queries for the validation of our approach, we chose 9 regulation texts on the accessibility of public buildings (French regulation base). These regulation documents represent different classes of regulation texts (e.g. norm, circular) and describe the accessibility constraints of different entities: doors, roads, signalization, etc. With the help of CSTB experts, we have identified about 350 simple text conformity queries that resume these 9 regulation texts, which are partially interpreted (around 65%). Other 35% of identified queries are classified as non interpretable and are not formalized. For practical validation of our approach, we currently formalized and tested about 100 SPARQL conformity queries. To adapt the C3R prototype for different user profiles, we identified 3 user profiles: architects, engineers, and owners/non-professional end users. For each user profile, we create a facet of the conformity-checking ontology. The calculation of the semantic similarity between the concepts of the conformity-checking ontology according to the DOUP-method is not implemented yet. It is on the next step of the incremental implementation of the C3R prototype.

5 Conclusions We presented a formal method for the development of the domain ontology for different user profiles in the context of conformity checking in construction. Our semantic approach for the improved development of the domain ontology for different user profiles (the DOUP-method) comprises three components: (i) the acquisition and representation of the knowledge to model the conformity-checking process; (ii) the contextualization of the acquired ontology by integrating the results of semantic search of conformity queries; and (iii) the refinement of this modeling with the adaptation of the domain ontology for different user profiles. We also described the conceptual architecture of the C3R prototype and presented the current work on its implementation, to illustrate the feasibility of our approach.

140


One limitation of our work is that we do not establish the semantic correspondences between different facets of the domain ontology. This very interesting research problem is not taken into account by our semantic approach for the improved development of the domain ontology for different user profiles, and can be seen as a possible axis for future research. Our future works focus on the further incremental development of the conformitychecking ontology and the C3R prototype, as well as their evaluation by domain experts and end users. In particular, according to the DOUP method, we will adapt the C3R prototype for different user profiles, as well as will create different facets of the conformity checking ontology.

References 1. Aguilar-Lopez, D., Lopez-Arevalo, I., Sosa, V.: Usage of Domain Ontologies for Web Search. In: Advances in Soft Computing, vol. 50. Springer, Heidelberg (2009) 2. Aziz, Z., Anumba, C., Law, K.H.: Using Context-Awareness and Web-Services to Enhance Construction Collaboration. In: Joint International Conference on Computing and Decision Making in Civil and Building Engineering, Montréal, Canada (2006) 3. Bell, H., Bjorkhaug, L.: A buildingSMART Ontology. In: European Conference on Product and Process Modelling (ECPPM), Valencia, Spain (2006) 4. Brézillon, P.: Context Modeling: Task Model and Model of Practices. In: Kokinov, B., Richardson, D.C., Roth-Berghofer, T.R., Vieu, L. (eds.) CONTEXT 2007. LNCS (LNAI), vol. 4635, pp. 122–135. Springer, Heidelberg (2007) 5. Berners-Lee, T.: Reflections on Web Architecture. Conceptual Graphs and the Semantic Web (2001), http://www.w3.org/DesignIssues/CG.html 6. Bultman, A., Kuipers, J., van Harmelen, F.: Maintenance of KBS’s by Domain Experts, The Holy Grail in Practice. In: Logananthara, R., Palm, G., Ali, M. (eds.) IEA/AIE 2000. LNCS (LNAI), vol. 1821, pp. 139–149. Springer, Heidelberg (2000) 7. Corby, O., Faron-Zucker, C.: Implementation of SPARQL Query Language based on Graph Homomorphism. In: Priss, U., Polovina, S., Hill, R. (eds.) ICCS 2007. LNCS (LNAI), vol. 4604, pp. 472–475. Springer, Heidelberg (2007) 8. El-Diraby, T., Fiès, B., Lima, C.: D3.6: The e-COGNOS Ontology V1.1.0 - WP3. ECognos project IST-2000-28671 (2000) 9. El Sayed, A., Hacid, H., Zighed, A.: A Context-Dependent Semantic Distance Measure. In: 19th International Conference on Software Engineering and Knowledge Engineering (SEKE), Boston, USA (2007) 10. IFC specifications, http://www.iai-tech.org/ 11. Gauch, S., Speretta, M., Pretschner, A.: Ontology-Based User Profiles for Personalized Search. In: A Handbook of Principles, Concepts and Applications in Information Systems. Integrated Series in Information Systems, vol. 14 (2007) 12. Gehre, A., Katranuschkov, P.: InteliGrid Deliverable D32.2 – Ontology Services (2007), http://www.inteliGrid.com 13. Gruber, T.: Toward Principles for the Design of Ontologies used for Knowledge Sharing. In: Int. Workshop on Formal Ontology, Stanford Knowledge Systems Laboratory Report KSL-93-04 (1993) 14. Hernandez, N., Mothe, J., Chrisment, C., Egret, D.: Modeling Context through Domain Ontologies. Information Retrieval 10(2) (2007)


141

15. Karoui, L., Aufaure, M.-A., Bennacer, N.: Contextual Concept Discovery Algorithm. In: 20th International FLAIRS Conference. AAAI Press, Menlo Park (2007) 16. Kerrigan, S.L., Law, K.H.: Regulation-Centric, Logic-Based Conformance Assistance Framework. Journal of Computing in Civil Engineering, ASCE 19(1) (2005) 17. Mokhtari, N., Dieng-Kuntz, R.: Extraction et exploitation des annotations contextuelles. In: 8èmes journées Extraction et Gestion des Connaissances (EGC), Sophia Antipolis, France (2008) 18. Polanyi, M.: The Tacit Dimension. Doubleday & Co. (1966); Reprinted Peter Smith, Gloucester, Mass (1983) 19. Sewese (2008), http://www-sop.inria.fr/teams/edelweiss/wiki/ wakka.php?wiki=Sewese 20. Sieg, A., Mobasher, B., Burke, R.: Ontological User Profiles as the Context Model in Web Search. In: IEEE/WIC/ACM Int. Conference on Web Intelligence and Intelligent Agent Technology (2007) 21. Smartcodes (2008), http://www.iccsafe.org/SMARTcodes/ 22. Sutterer, M., Droegehorn, O., David, K.: UPOS: User Profile Ontology with SituationDependent Preferences Support. In: 1st Int. Conference on Advances in Computer-Human Interaction (2008) 23. Yurchyshyna, A., Faron-Zucker, C., Le Thanh, N., Zarli, A.: Ontological Approach for the Conformity-Checking Modelling in Construction. In: 10th International Conference on Enterprise Information System (ICEIS), Barcelone, Spain (2008) 24. Yurchyshyna, A., Faron-Zucker, C., Le Thanh, N., Zarli, A.: Towards the Knowledge Capitalisation and Organisation in the Model of Conformity-Checking Process in Construction. In: Lovrek, I., Howlett, R.J., Jain, L.C. (eds.) KES 2008, Part I. LNCS (LNAI), vol. 5177, pp. 341–348. Springer, Heidelberg (2008) 25. Yurchyshyna, A., Faron-Zucker, C., Le Thanh, N., Zarli, A.: Improved development of the domain ontology for different user profiles. In: 5th International Conference on Web Information Systems and Technologies (WEBIST), Lisbon, Portugal (2009)

The RDF Protune Policy Editor: Enabling Users to Protect Data in the Semantic Web Fabian Abel1 , Juri Luca De Coi2 , Nicola Henze2 , Arne Wolf Koesling3 , Daniel Krause1 , and Daniel Olmedilla2 1 3

Distributed Systems Institute / KBS, University of Hannover, D-30167 Hannover, Germany 2 L3S Research Center, University of Hannover, D-30167 Hannover, Germany eLearning Service Abteilung (elsa), University of Hannover, D-30159 Hannover, Germany {abel,henze,krause}@kbs.uni-hannover.de, {decoi,olmedilla}@L3S.de, [email protected]

Abstract. A fine-grained user-aware access control to user profile data is a key requirement for sharing user profiles among applications. Policy languages like Protune can handle access restrictions very well but are too complicated to be used by non-experts. In this paper, we identify policy templates and embed them into a user interface that enables users to specify powerful access policies and makes them aware of the current and future consequences of their policies. In a user study, we proof the effectiveness of our editor. Keywords: Policy, RDF, Access control, User interface.

1 Introduction Personalization encounters more and more attention as it promises to make programs more user friendly and content more appropriate. This additional information about the user ideally should be learned by the system in interaction with the user and stored in a user profile. While user profiles that work well within one personalization component and are implemented in one application have proved to work well, the benefit of a user profile will even increase if different applications can share its data. Especially in a service-based environment, where users invoke many different services and use them only for a short period of time, it is nearly impossible for a single service to generate such a user profile on its own as there is not enough interaction with the user. Shared user profiles enable applications to utilize additional information that has been collected by other applications and therefore need a universal data storage format. The Resource Description Framework (RDF) provides such a generic format and hence, is used in the User Modeling Service (UMService). The UMService is part of the Personal Reader Framework, a framework allowing the creation of web service-based applications. A serious problem of shared user profiles is that applications store sensitive information: While a trustful application known by the user is maybe allowed to access his bank account information, another application should not be allowed to access the same data. Therefore, an access control system is required that grants applications access to profile data only after the user has agreed. For this access control J. Cordeiro and J. Filipe (Eds.): WEBIST 2009, LNBIP 45, pp. 142–156, 2010. c Springer-Verlag Berlin Heidelberg 2010

The RDF Protune Policy Editor: Enabling Users to Protect Data

143

system, rule-based policy languages can be used very well as they allow precisely to specify which application can operate on which data at which time. Rule-based policy languages, like Protune [1, 2], can be used to define in a finegrained fashion which web service is allowed to access which data, which credentials it has to provide and so on. In the domain of user profile data this problem is shifted to the task of selecting data from an RDF graph protected by an access policy. Policy languages can deal with this task but the resulting policies are usually too complicated to be specified by non-experts. A possible approach is to use predefined policy templates that can be completed easily by users. However, users cannot be fully aware about which data is covered by a policy if they do not have an appropriate user interface. The contribution of this paper is to present a user interface that enables non-expert users to control the access to their RDF-based user profiles. Hence, we deduce access policy templates and we examine how to implement these templates in the Protune policy language. The user interface provides immediate feedback to the user which includes information about which part of the RDF data is covered by the policy and additionally which consequences the specified policy has. The paper is structured as follows: In Section 2 we present underlying techniques like the Personal Reader Framework, the User Modeling Service and policies. Section 3 introduces the usage of policies for protecting user profiles. Section 4 describes the back-end policy database used by our authorization framework. A user interface that enables users to specify these user profile policies is shown in Section 5 and evaluated in Section 6. The related work can be found in Section 7. We conclude the paper and give an outlook to future work in Section 8.

2 The Personal Reader Framework The Personal Reader Framework [3] allows the creation of modular web service-based applications (Figure 1). These applications are accessed by user interfaces (UI for short). Syndication Services implement the application logic and can be considered as the core of an application. By means of a Connector Service all Syndication Services are able to discover and access Personalization Services dynamically, which aggregate domain-specific information in a personalized way. To gather information, Personalization Services access and process Semantic Web data sources. An important feature of the Personal Reader Framework is that new services can be integrated in a plug-andplay manner, hence no centralized component has to be modified and new services can be used immediately from all other services within the framework. Both Syndication and Personalization Services are able to access and store user data which is supplied by a centralized User Modeling Service. Several applications have been implemented with the Personal Reader Framework like the Personal Publication Reader [4], the MyEar music recommender [5] or the Agent1 .

1

http://www.personal-reader.de/agent

144

F. Abel et al. RDF

UI

RDF

Personalization Service

Syndication Service

...

Connector Service

UI

Syndication Service

UI

Policy Ecitor

... Personalization Service

User access control

User Modeling Service

Fig. 1. Personal Reader Architecture

2.1 The User Modeling Service The User Modeling Service stores and aggregates user profile data from various Personalization and Syndication Services. Because these services, which can be integrated at runtime and used immediately, aim on different domains, they also use different ontologies to express their knowledge about the user. For this reason we use RDF statements to store data domain-independently. A statement contains the user as subject, a predicate from the service ontology and a value as object. Objects can also be more complicated and further RDF statements can be used to describe the objects in a more detailed way as outlined in the example scenario in section 2.2. Access Control Layer. The access control layer of the User Modeling Service has to restrict the access to the data stored in the User Modeling Service. Therefore, a user should specify which web services are allowed to access which kind of data in the user profile and in which way. The environment of the access control layer is similar to a firewall: whenever an application tries to access a specific port, if an access rule for such application and port has been specified, the specified action (allow or deny) is performed. Otherwise the firewall asks the user how to behave. The firewall is at no time aware of which applications or ports exist in a system. Similarly, as the framework allows to plugin new services immediately, the access control layer is not aware of which services will try to access which part of the user profile. Hence, specifying static access rules a priori like in other access control systems is not applicable. Our access control layer solves this issue by a deny-by-default behavior. Every service that tries to access an RDF statement is rejected if no existing policy is applicable. The service is informed why it was rejected and will report this to the user. Afterwards, the user can enter the user interface of the access control layer to grant or deny access. The user interface can take the context into account, which contains the statements a service tried to access, and hence supports the user in specifying policies by reducing the choices to the affected statements. By allowing users to specify also general policies we try to avoid that the user is overwhelmed by too much interaction with the access control layer. Keeping user interaction low enhances usability and at the same time


145

avoids that users ignore repeatedly displayed confirm messages. In the rest of the paper, we focus on granting read access. A similar approach can be used for write access requests. 2.2 Policies for Securing Data Securing RDF data is different from securing usual datasets. Because RDF datasets can be considered as graphs we take into account this graph structure in order to provide a definition of “security”. There are many possibilities to secure the data in the user profile, like black- or whitelisting of services for specific RDF statements by means of access control lists. We do not want to mark resources as “accessible” or not in an automatic way, because the user should keep full control on which resources (not) to grant access for. But we also want to relieve the user from marking each resource individually, so we need a more flexible solution. We think that policies provide such a flexible solution. In the following we examine how Protune policies can be applied to RDF statements and graphs. Let us assume that John’s user profile contains the follwing RDF statements: S1 : S2 : S3 : ... Sm−4 : Sm−3 : Sm−2 : Sm−1 : Sm :

(John, phoneNumber, 123) (John, hasFriend, Friend 1) (Friend 1, phoneNumber, 234) (John, hasFriend, Friendn ) (Friendn , phoneNumber, 345) (John, hasFriend, Mary) (Mary, phoneNumber, 456) (John, loves, Mary)

John may want to make the phone numbers of his friends publicly available, but may want to hide statement Sm or maybe even statement Sm−1 . The policy language Protune allows to define policies protecting such statements.

3 Protune Policy Templates for a User Modeling Service The policies we need must be able to specify in a declarative way the prerequisites a service has to fulfill in order to access some resource. The policy language Protune [1,2] allows to formulate a broad range of policies like access control policies, privacy policies, reputation-based policies, provisional policies, and business rules. Some languages, like e.g. KAoS or Rei [6, 7] adopt Description Logics [8] as underlying formalism, whereas Protune, which extends PAPL [9] and PeerTrust [10], exploits Logic Programming and its SLDNF-based reasoning mechanism [11]. One of the main differences between Description Logics-based (DL-based) and Logic Programming-based (LP-based) policy languages can be found in the way they deal with negation: Description Logics allow to define negative information explicitly, whereas LP-based systems can deduce negative information by means of the so-called negation as failure inference rule. Whilst consistency problems may arise in DL-based

146

F. Abel et al.

systems (since both a statement and its negation can be asserted), LP-based systems do not have to deal with consistency issues, since they only allow the user to specify positive statements. LP-based policy languages like Protune may decide whether the user should only specify allow policies (thereby relying on the negation-as-failure inference rule) or the other way around. The first approach is usually preferred, since wrongly disclosing private information is a more serious issue than not disclosing information that should be publicly available. In our framework we need both usual deny policies and deny-by-default policies: If a deny-by-default policy applies, the user is directed to the user interface to specify new policies, if a usual deny policy occurs the user is not informed since he already defined a policy. This feature allows us to implement in a very clean way the algorithm to be executed by the access control component (cf. Figure 1), namely if(a deny policy is defined) deny access else if(an allow policy is defined) allow access else deny access and ask the user

The access control component checks first whether a deny policy is applicable to the current access request and, if it is the case, denies access. If not, the system checks whether an allow policy is applicable. If this is not the case, access is denied and a message is sent to the user. The following Protune policy applies to the RDF statements example given in the previous chapter. Its intended meaning is to allow services that belong to the user-defined group trustedServices to access the telephone numbers of John’s friends, except Mary’s number. allow(access(rdfTriple(Y, phoneNumber, X))) :requestingService(S), rdfTriple(S, memberOf, ’#trustedServices’), rdfTriple(’#john’, hasFriend, Y), not Y = ’#mary’.

Predicate rdfTriple retrieves RDF triples from some RDF repository, whereas predicate requestingService accesses runtime data in order to retrieve the value of the current requesting service. The rule the policy consists of can be read as a rule of a Logic Program, i.e., allow(access(. . . )) is satisfied if predicate requestingService, all literals rdfTriple and the inequality are satisfied. Predicates which represent an action (i.e., requestingService and rdfTriple) are supposed to be satisfied if the action they represent has been successfully executed. The policy can therefore be read as follows: access to RDF triple (Y, phoneNumber, X) is allowed if the current requesting service (S) belongs to trustedServices and X is the phone number of someone who is a friend of John different than Mary. 3.1 Policy Templates for an RDF Based User Profile Since expressive policies become quickly hard to read for non-technical users we defined some general purpose policies in so-called templates:


147

1. One may group targets (in our case RDF statements or parts of them), so that the user is enabled to state, what triples should be accessible. Examples for such a group of targeted RDF statements are: Allow access to some specific phone numbers, allow access only to my own phone number or allow access only to my friends’ phone numbers. 2. Policies may also be grouped according to the requester, so that the user is enabled to state who gets access to the triples (i.e. allow access for one service or a specific group/category of services). Protune policies allow the usage of both kind of policy types to protect specific RDF statements, a specific group of statements or, in general, an arbitrary part of an RDF graph. So, it is possible to – specify RDF-predicates anywhere used in the user profile to be secured by a policy – specify RDF-object/RDF-subject types anywhere used in the user profile – specify RDF statements that contain information directly related to the user, like (John, loves, Mary), and not just information indirectly related to the user, like (Friendx , phoneNumber, xyz) – specify meta-data predicates like requester or current time Our user interface allows to define policies protecting RDF graph patterns. When defining a policy the user must instantiate such patterns and adapt them to the given context (see Figure 5). Effects on the User Interface. If there is no policy defined on an RDF statement, an incoming request is denied by default and the accessing service will point the user to the user interface to define a new policy regulating the access to the RDF statement in the future. On the other hand, no user feedback is requested if a deny policy applies to the RDF statement and the current requester. Therefore, the service needs to distinguish between default denial and policy-based denial. Protune by itself uses only positive authorizations in order to avoid conflicts. For this reason we defined a deny predicate on top of Protune to enable also the definition of deny policies. However, if we allow for both positive and negative authorizations, conflicts can arise: This is the case whenever a resource is affected by both an allow and a deny policy. To avoid such situations we designed our user interface in order to ensure that no conflict situations will arise or that they are solved in precedence. When the user defines an allow policy affecting a resource that is already covered by a deny policy, the user interface will show a dialog, notifying the user that there is a conflict. If the user does not want to allow access to the resource, the allow policy will still be defined (since in our framework deny policies have by default higher priority than allow policies), otherwise the deny policy will be modified in order to exclude from its scope the affected resource. On the other hand when the user defines a deny policy affecting a resource that is already covered by an allow policy, the user interface will show a dialog, notifying the user that there is a conflict. If the user does not want to allow access to the resource, the deny policy will simply be added (for the same reason described above), otherwise a modified version of it will be added, which excludes from

148

F. Abel et al.

its scope the affected resource. Finally, if the user model changes, new RDF statements can be automatically covered by existing policies. But the user has also the option to apply his policy only to RDF statements existing at policy creation time. As soon as a service adds RDF statements, the user will be asked by the user interface whether his policy should also apply to the new statements.

4 The Back-End Policy Database This section describes the back-end policy database used by our authorization framework. We first describe a naive implementation whose purpose is to introduce the main concepts and definitions and afterwards we discuss possible optimizations. 4.1 A Naive Implementation The UML2 diagram presented in Figure 2 shows the entities our authorization framework is concerned with which we introduced in the previous sections, namely, requesters, resources and policies: A requester is identified by some sort of identifier (e.g., its IP address or some more complex authentication mechanism), a resource (being an RDF triple) consists of subject, predicate and object, finally a policy, beside the text of the policy itself, possesses a flag indicating whether it applies only to the resources available at policy creation-time or also to resources possibly added later on. As described in Section 3, policies are distinct in allow and deny policies and dominance relationships are defined among them. An allow (resp. deny) policy can dominate and be dominated by an arbitrary number of deny (resp. allow) policies, meaning that in case of conflict the dominating policy will be enforced. In the following we will call such relationships among policies priority relationships. The set of priority relationships induces a graph structure over the set of policies. Special care must be taken in order to avoid that the graph contains loops, since otherwise all policies belonging to a loop would be dominated and none of them could be enforced. We will describe in the following how our authorization framework avoids that loops are built in the priority relationship graph. The basic functionality provided by an authorization framework is, of course, the capability of (not) authorizing requesters to access resources. Beside that, our authorization framework also offers facilities for adding and removing requesters, resources and policies. In the following we briefly sketch the internals of such functionalities. Request Evaluation. The evaluation of a request for accessing a resource takes place according to the algorithm described in Section 3: If there is a non-dominated applicable deny policy the request is rejected, otherwise if there is a non-dominated applicable allow policy the request is accepted, otherwise the user is asked whether the request should be accepted or rejected. In the last case the user’s answer is interpreted as the definition of a new policy which dominates each possibly conflicting one. Notice that the addition of such priority relationships can never make the priority relationship graph cyclic, since in such graph the newly added policy does not have incoming arcs. 2

http://www.uml.org/


149

Fig. 2. A basic UML class diagram of our authorization framework

Resource Removal. The addition of a new resource does not require to perform any further operation, whereas, in order to keep the policy repository as clean as possible, it makes sense removing all policies affecting one single resource upon removal of that resource. Notice that in our authorization framework it is not infrequent that policies apply to one single resource: As we described above, this is the case whenever the user is asked whether a request should be accepted or rejected. Requester addition and deletion are handled in a similar way as resource addition and deletion. Policy Addition. Whenever a new policy is added, it is checked whether it conflicts with already defined policies. If this is the case, the user is asked to define priority relationships between the conflicting policies: The user is not allowed to define priority relationships which would make the priority relationship graph cyclic. Policy Removal. Whenever a policy is removed, all relationships it was involved in are removed as well. 4.2 Optimizations When performing the operations listed in Section 4.1, some activities must be carried out which are computationally expensive, namely 1. 2. 3. 4.

selection of non-dominated policies which apply to a given request (cf. list item 1) identification of policies applicable to one single resource/requester (cf. list item 2) identification of conflicting policies (cf. list item 3) constraining the user to define only priority relationships which keep the priority relationship graph acyclic (cf. list item 3)

The UML diagram presented in Figure 3 is a variant of the one shown in Figure 2: A new attribute dominatingPolicyNr has been added to policies, indicating how many other policies dominate a given one, and a relationship between policies and resources (resp. requesters) has been added. A policy can affect an arbitrary number of resources (resp. requesters), on the other hand each resource (resp. requester) can be affected by an arbitrary number of policies. Both relationships have a unique attribute tracking whether the policy affects only that resource (resp. requester). In the following we will call relationships between policies and resources (resp. requesters) influence relationships. The addition of redundant information requires some overhead in order to keep such information consistent with the global state of the system. W.r.t. the list presented in Section 4.1 the following modifications are needed:

150

F. Abel et al.

Fig. 3. The full-fledged UML class diagram of our authorization framework

Resource Addition. Whenever a new resource is added, corresponding relationships with the policies it is affected by are defined. Notice that also relationships with dominated policies are defined, so that they do not need to be computed each time a dominating policy is deleted. Relationships are of course defined only with policies which apply also to resources added after policy creation-time. Resource Removal. Whenever a resource is removed, all relationships it was involved in are removed as well. Policy Addition. Whenever a new policy is added, influence relationships are defined between the new policy and the resources and requesters it affects: If the new policy affects one single resource (resp. requester) the unique attribute of the influence relationship is set. Finally, the dominatingPolicyNr attribute of the new policy is initialized according to the priority relationships defined and the one of the policies dominated by the new policy is incremented. Policy Removal. Whenever a policy is removed, all influence relationships it was involved in are removed as well. The dominatingPolicyNr attribute of all policies which were dominated by the deleted one is decremented.

5 User Interface for Defining Access Policies The web interface that enables users to specify Protune access policies operates on top of the access control layer of the User Modeling Service as outlined in Figure 1. If a service attempts to access user data for which no access policies have been defined yet, then the operation of the service is disrupted and the user is forwarded to the policy editor. The interface which is shown to the user (see Figure 4) is adapted to the context of the disrupted operation. Such a context is given by the RDF statements which the service needed to access. Thus, the overview is split into a part which outlines these RDF statements, and a part which allows the specification of corresponding access policies. RDF statements are colored according to the policies affecting them (e.g. if a statement is not affected by any policy it may be colored yellow, green statements indicate that at least one service is allowed to access, etc.). Next to such statements the interface additionally shows conflicting policies by marking affected policies and RDF statements. Warning dialogs make the user aware of critical policies. In Figure 4 the user wants to allow the access to names to all instances of a class Contact. But as the user may not


151

Fig. 4. Defining Policies - Overview

be aware that such a policy would also disclose all future user profile entries containing a name, he is explicitly prompted for validation. If the user disagrees, he will be prompted whether the policy should be refined to cover only those name instances that are currently stored in the user profile. In general, policies are edited using the interface depicted in Figure 5. This interface consists of two main parts which allow to a) define policies (top frame) and b) dynamically show the effects of the policy (bottom frame). An expert mode is also available, which allows to directly input Protune policies. Users that do not use the expert mode just have to instantiate a template consisting of four steps (see top right in Figure 5): what. The main task during creation of access policies is the specification of RDF graph patterns which identify statements that should be accessible or not. The predefined forms for defining these patterns are generated on basis of a partial RDF graph consisting of a certain RDF statement (here: (#contact1, name, ’Daniel Krause’)) and its relation to the user (#henze, hasContact, #contact1). To clarify this fact the RDF graph is presented to the user on the left hand. To determine the options within the forms schema information of domain ontologies is utilized. In the given example the property name is part of the statement from which the forms are adapted. As name is a subproperty of contactDetail both appear within the opened combo box. By clicking on add pattern or remove the user is enabled to add/remove RDF statement patterns to/from the overall graph pattern. allow/deny. The user can either allow or deny the access to RDF statements expressly.

152

F. Abel et al.

Fig. 5. Editing a policy in a detailed view

who. The policy has to be assigned to some services or category of services. For example to ContactInfo, the service trying to access user data, or to a category like Address Data Services with which ContactInfo is associated. period of validity. This parameter permits the temporal restriction of the policy. According to Figure 5 the resulting Protune policy would be (without period of validity): allow(access(rdfTriple(X, contactDetail, _))) :requestingService(S), rdfTriple(S, memberOf, ’#addressDataServices’), rdfTriple(’#henze’, hasContact, X).

Thus, Address Data Services are allowed to access all statements (X, contactDetail, Y) that match the RDF graph pattern (#henze, hasContact, X), (X, contactDetail, Y). This policy overlaps with another policy that denies the access to statements of the form (X, privateMail, Y) wherefore a warning is presented to the user. This warning also lists the statements affected by this conflict: As (#henze, privateMail, ’[email protected]’) does not suit, the pattern specified in Figure 5 (#contact5, privateMail, ’[email protected]’) is the only affected statement. By clicking on “Yes, overwrite!” the deny policy would be amended with the exception: not rdfTriple(#contact5, privateMail, ’[email protected]’). Otherwise, by selecting “No, don’t overwrite!” both policies would overlap. But as deny policies outrank allow policies (cf. section 3) the affected statement would still be protected.


153

Next to such warnings the policy editor makes the user aware of how specified policies will influence the access to RDF statements. As name, email, etc. are subproperties of contactDetail the above policy permits access to a big part of the user’s RDF graph which is consequently shown in green (see bottom of Figure 5). 5.1 Current Implementation In the current implementation3 the user interface for defining access control policies for RDF data supports already the core functionality described in Section 5. We furthermore integrated the prototype into the Personal Reader framework (cf. Section 2) and applied AC4RDF [12] together with Protune to enforce the policies defined by the users. AC4RDF is an access control mechanism for RDF stores. It rewrites queries so that they respect the access policies specified via the user interface, i.e. policies as presented in Section 3. Regarding usability issues, the main advantage of our user interface are: easy-to-use. The users do not need to learn any policy language, policies are created by specifying simple pattern. scrutability. Users can inspect the effect of the policy immediately as the RDF data is colored either red (access not allowed) or green (access allowed) awareness of effects. Whenever a change in a policy will disclose data in the future, it is not visualized in the current graph. Hence, users get a confirmation message to make the aware of the effects of the changes.

6 Evaluation of the Interface In order to ascertain the usability of the Protune policy editor, we conducted an evaluation with students. Within our evaluation, the students had to accomplish six small tasks with gradient complexity. After we read the tasks to the student in full, we took the time the student needed upon completion of the task. In all of these tasks the students had to create policies with the help of the editor’s interface. After the creation, the editor generates Protune policies from the visual creation process. Our student test group consists of five students, advanced in their study, 3 male and 2 female, coming from computer science and math. None of the students had previous knowledge of Protune. None of the students had previously used or tested the Protune policy editor. While some students already had a basic understanding of RDF and some did not we gave a short introduction into RDF in order to make all of them aware of the graph structure and the meaning of RDF triples. Every student conducted the tasks separately. Therefore, we gave him/her a 10 minute introduction into the Protune editor. The introduction was on a need-to-know basis and contained examples how to accomplish general tasks. We explained further issues in deeper detail, only if asked by the student. An introduction into Protune or formal policies was unnecessary, since the students did not need knowledge about Protune and policies itself in order to work with the editor. 3

The policy editor user interface is made available via: http://www.personal-reader.de/wp/projects/policy-editor

154

F. Abel et al.

Fig. 6. Overview of evaluation results (n=5, time measured in seconds)

After the introductory phase, the students had to fulfill the six tasks. Every task had to be fulfill after each other, i. e. the students received task two when they finished task one and so on. After the students have finished a task, the Policy editor was resetted to a starting stage. The starting state of the editor is a scenario state, in which the Policy editor shows the request for a set of RDF triples from a specific service. Those triples are based on an example dataset, we created for this scenario. We measured the time in seconds the student needed from touching the computer mouse until finishing the task. The tasks in detail are: 1. allow the access to one specific requested RDF triple for the requesting service 2. allow the access to all currently requested RDF triples for the requesting service 3. allow the access to all RDF triples of the user profile database that do contain a specific RDF predicate for a requesting service 4. allow the access to all RDF triples of the user profile database that contain a specific RDF subject; limit the access until a certain date for the requesting service 5. deny the access to all RDF triples of the user profile database that do contain a specific RDF subject, except of one given RDF triple 6. allow the access to all RDF triples of the user profile database with a specific RDF subject for a requesting service, only, if there is existing a specific RDF triple that contains this specific RDF subject as RDF object (utilizing the graph structure of RDF triples) 6.1 Results In Figure 6 the time (in seconds) is illustrated that students required in order to finish the task. The time ranged from five seconds for the most simple first task up to 50 seconds in average for the complex exercises. This was much shorter than we expected and presumably shorter than creating Protune policies by hand. Furthermore, it is remarkable that all tasks have been solved by the students. Although, the testing group was not very big, the time the students needed did not show big variance.


155

However, the students did also make small mistakes in solving the tasks, but corrected themselves after seconds. For example, in task 3, 3 of 5 students confused ”all RDF triples” (which means ”all of the user profile database”) with ”all requested triples”, which are only the triples shown in our scenario that the service requests.

7 Related Work Controlling the access to RDF data stores can be realized within the RDF query layer. As outlined above, we utilize AC4RDF [12] together with Protune in our current implementation to enforce the policies formulated by the users via the easy-to-use editor presented in this paper. Another approach to access control to RDF data is discussed in [13], where access to RDF data is restricted by defining views, which correspond to RDF subgraphs an inquirer is allowed to access. However this approach does not make use of policies and thus, cannot exploit their advantages (e. g. negotiation, declarative and intuitive structure, and facility of fine-grained adjustments) and broad range of tool support (e. g. upcoming possibilities like formulating policy rules in natural language 4 ) policy languages like Protune offer. Beside the policy language Protune, there exist other policy languages like e.g. KAoS [6] or Rei [7] that may be utilized to secure RDF statements. But we choose Protune instead for protecting the RDF Graph patterns, because in our approach we make use of properties like e.g. recursive definitions and variables and we also want to exploit advanced features like negotiations in the future. The development of Protune has a special severity in these topics. For some policy languages there are policy editors available. Because of their complexity, most of these editors are difficult to use. KPAT [14], a policy editor for KAoS, offers to constrain the creation of policies with forms to ease the usage. However, none of the editors deals with the visualization of the policies’ consequences or takes RDF graph structure into account to deduce policy templates.

8 Conclusions and Further Work In this paper we presented a user interface that enables non-expert users to control the access to their RDF-based user profiles. We used the policy framework Protune to enforce the underlying access control layer and outlined how to use policy templates to define access control policies. Furthermore, we discussed how to deal with conflicting policies and how a user interface helps to understand complex and expressive policies and their consequences. We presented the current implementation of the proposed user interface. In the evaluation section we outlined that non-expert users were able to specify complex policies efficiently after are short introduction to the system. In the future, we will implement the prototype to a live-system enabling learners to share their user profiles with e-learning web services.

4

Expressing Protune rules in ACE: http://attempto.ifi.unizh.ch/site/

156

F. Abel et al.

References 1. Bonatti, P.A., Olmedilla, D.: Driving and monitoring provisional trust negotiation with metapolicies. In: 6th IEEE International Workshop on Policies for Distributed Systems and Networks (POLICY 2005), Stockholm, Sweden, pp. 14–23. IEEE Computer Society, Los Alamitos (2005) 2. Bonatti, P.A., Olmedilla, D.: Policy language specification. Technical report, Working Group I2, EU NoE REWERSE (2005) 3. Henze, N.: Personalization services for the semantic web: The personal reader framework. In: Framework 6 Project Collaboration for the Future Semantic Web Workshop at European Semantic Web Conference ESWC 2005, Heraklion, Greece (2005) 4. Abel, F., Baumgartner, R., Brooks, A., Enzi, C., Gottlob, G., Henze, N., Herzog, M., Kriesell, M., Nejdl, W., Tomaschewski, K.: The personal publication reader, semantic web challenge 2005. In: Gil, Y., Motta, E., Benjamins, V.R., Musen, M.A. (eds.) ISWC 2005. LNCS, vol. 3729, pp. 1050–1053. Springer, Heidelberg (2005) 5. Henze, N., Krause, D.: Personalized access to web services in the semantic web. In: SWUI 2006 - 3rd International Semantic Web User Interaction Workshop, Athens, Georgia, USA (2006) 6. Uszok, A., Bradshaw, J.M., Jeffers, R., Suri, N., Hayes, P.J., Breedy, M.R., Bunch, L., Johnson, M., Kulkarni, S., Lott, J.: Kaos policy and domain services: Toward a description-logic approach to policy representation, deconfliction, and enforcement. In: 4th IEEE International Workshop on Policies for Distributed Systems and Networks (POLICY), Lake Como, Italy, IEEE Computer Society, Los Alamitos (2003) 7. Kagal, L., Finin, T.W., Joshi, A.: A policy language for a pervasive computing environment. In: 4th IEEE International Workshop on Policies for Distributed Systems and Networks (POLICY), Lake Como, Italy. IEEE Computer Society, Los Alamitos (2003) 8. Baader, F., Calvanese, D., McGuinness, D.L., Nardi, D., Patel-Schneider, P.F. (eds.): The Description Logic Handbook: Theory, Implementation, and Applications. Cambridge University Press, Cambridge (2003) 9. Bonatti, P.A., Samarati, P.: Regulating service access and information release on the web. In: ACM Conference on Computer and Communications Security, pp. 134–143 (2000) 10. Gavriloaie, R., Nejdl, W., Olmedilla, D., Seamons, K.E., Winslett, M.: No registration needed: How to use declarative policies and negotiation to access sensitive resources on the semantic web. In: Bussler, C.J., Davies, J., Fensel, D., Studer, R. (eds.) ESWS 2004. LNCS, vol. 3053, pp. 342–356. Springer, Heidelberg (2004) 11. Lloyd, J.W.: Foundations of Logic Programming, 2nd edn. Springer, Heidelberg (1987) 12. Abel, F., Coi, J.L.D., Henze, N., Koesling, A.W., Krause, D., Olmedilla, D.: Enabling advanced and context-dependent access control in rdf stores. In: Aberer, K., Choi, K.-S., Noy, N., Allemang, D., Lee, K.-I., Nixon, L.J.B., Golbeck, J., Mika, P., Maynard, D., Mizoguchi, R., Schreiber, G., Cudré-Mauroux, P. (eds.) ASWC 2007 and ISWC 2007. LNCS, vol. 4825, pp. 1–14. Springer, Heidelberg (2007) 13. Dietzold, S., Auer, S.: Access control on rdf triple stores from a semantic wiki perspective. In: Scripting for the Semantic Web Workshop at 3rd European Semantic Web Conference, ESWC (2006) 14. Uszok, A., Bradshaw, J.M., Johnson, M., Jeffers, R., Tate, A., Dalton, J., Aitken, S.: Kaos policy management for semantic web services. IEEE Intelligent Systems 19, 32–41 (2004)

An Unsupervised Rule-Based Method to Populate Ontologies from Text Eduardo Motta, Sean Siqueira, and Alexandre Andreatta Department of Applied Informatics, Federal University of the State of Rio de Janeiro (UNIRIO) Av. Pasteur, 458, Urca, Rio de Janeiro, Brazil {eduardo.motta,sean,andreatt}@uniriotec.br

Abstract. An increasing amount of information is available on the web and usually is expressed as text. Semantic information is implicit in these texts, since they are mainly intended for human consumption and interpretation. Because unstructured information is not easily handled automatically, an information extraction process has to be used to identify concepts and establish relations among them. Ontologies are an appropriate way to represent structured knowledge bases, enabling sharing, reuse and inference. In this paper, an information extraction process is used for populating a domain ontology. It targets Brazilian Portuguese texts from a biographical dictionary of music, which requires specific tools due to some language unique aspects. An unsupervised rule-based method is proposed. Through this process, latent concepts and relations expressed in natural language can be extracted and represented as an ontology, allowing new uses and visualizations of the content, such as semantically browsing and inferring new knowledge. Keywords: Information extraction, Ontology population, Natural language processing, Brazilian Portuguese.

1 Introduction This paper describes an implemented method to extract information from Portuguese texts on a rule-based unsupervised manner. The information extraction process output is represented as a domain ontology. An increasing amount of information is available on the web and is frequently expressed as text, representing semi-structured or unstructured data. However, computational processes don’t easily handle unstructured information. Information extraction (IE) seeks to structure information by adding meaning to raw data. It is defined by Moens [21] as “the identification, and consequent or concurrent classification and structuring into semantic classes, of specific information found in unstructured data sources, such as natural language text, making the information more suitable for information processing tasks.” A usual way to capture and represent knowledge of a specific field (a knowledge domain) is through the use of a domain ontology. According to Gruber [14], “an ontology defines a set of representational primitives with which to model a domain of knowledge or discourse”. The representational primitives are typically classes (or sets), attributes (or properties), and relationships (or relations among class members). J. Cordeiro and J. Filipe (Eds.): WEBIST 2009, LNBIP 45, pp. 157–169, 2010. c Springer-Verlag Berlin Heidelberg 2010

158

E. Motta, S. Siqueira, and A. Andreatta

Therefore, the outcome of an IE process can be represented as a domain ontology. Ontologies are an appropriate way to represent structured knowledge bases, enabling sharing, reuse and inference. In this work, we present the implementation of an IE process that populates a domain ontology. The knowledge domain handled by our solution is the biographical and artistic data on the music field. The source of information is a biographical dictionary of Brazilian popular music and the output of the process is a populated domain ontology. Through this process, latent concepts and relationships expressed in natural language can be extracted and represented, allowing new uses of the available content, like integrating with other information sources (based on semantic concepts) or navigating the knowledge base on a hierarchical manner. Moreover, after knowledge is represented as a populated domain ontology that is enriched with logical rules, inference engines can be used to derive new knowledge not explicitly contained in the source texts. One of the benefits of representing the textual content as a structured knowledge base is to allow browsing through dimensions like year or place of birth, type of instruments performed or even navigating through relationships among artists, like partnership. The approach of this work is to exploit information that is most relevant to the domain, such as biographical data (e.g., date and place of birth and death, and genealogical relations), artistic data (e.g., partnership) and career events (like recording or releasing a CD, composing a song, appearing on a TV show etc). To do so, we applied IE techniques to obtain semantic information from text. In particular, this work focuses on handling texts in Portuguese for which it is necessary to deal with some unique aspects while specific language tools for Portuguese should be used. The next section includes some background on IE and ontology population. Sect. 3 is a description of the dictionary structure. Sect. 4 presents the approach to this problem and the methods applied. Sect. 5 presents implementation issues and Sect. 6 summarizes some results. Finally, Sect. 7 presents related works and concluding remarks.

2 Background 2.1 Information Extraction Information extraction has been applied in the last years in different contexts and using different techniques. IE methods can be categorized in learning and knowledge engineering approaches [18]. Learning methods require an annotated corpus, which is usually expensive to obtain. Knowledge engineering methods use rules that are hand crafted and improved on an iterative process or obtained using an unsupervised technique. Information extraction can be used to obtain entities (ontology classes), attributes (ontology classes’ slots), facts (relations between classes) and events (activities or events where entities take part) [11]. Several tasks are necessary to accomplish IE, such as named entity recognition, anaphora resolution, coreference resolution, template element task, template relationship task, and scenario template production. Additionally, general-purpose natural

An Unsupervised Rule-Based Method to Populate Ontologies from Text

159

language pre-processing steps are necessary, like tokenization, paragraph and sentence splitting, PoS (part-of-speech) tagging, shallow or full parsing etc. 2.2 Ontology Population Ontologies have gained popularity in recent years due to the emergence of the semantic web, where ontologies play a central role. One of the bottlenecks of the semantic web is acquiring semantic content, i.e, semantically annotating the existing content on the web. Text content available on the web is primarily intended for human consumption which means that the semantics contained in texts is not explicitly represented but is in the reader’s mind. Automatically (or semi-automatically) acquiring domain knowledge through ontology population is an important contribution towards semantic web applications. After extracting semantic information from text and representing it as a domain ontology, new applications arise, such as integrating information, navigating the knowledge base on a structured manner and finding new relationships not explicit from the source texts. Ontology learning is the task that aims at discovering concepts and relations from text. In contrast, ontology population (OP) from text is the task of acquiring instances of concepts and their relations from text [25]. IE and OP are closely related in the sense that an ontology can be used to represent the IE process output and, on the other hand, knowledge represented in the ontology can help the IE cycle. This interaction between IE and OP is referred to as ontology driven information extraction [26].

3 Dictionary Description The Cravo Albin’s Dictionary of Brazilian Popular Music (Dicionário Cravo Albin da Música Popular Brasileira) [3] is maintained by the Cravo Albin Cultural Institute, a civil, non-profit organization headquartered in Rio de Janeiro City and established in 2001. The dictionary contains more than 5,000 entries with information on biography, artistic data, shows, video clips and related bibliography of Brazilian popular music artists. It is available on the Internet, and it is organized and accessed by artist noun entries. It is possible to search for entries, but there is no way to navigate from entry to entry, since there are no hyperlinks. Although the whole content of the dictionary is available on the Internet, it is organized much like as a paper dictionary, i.e., there is no linkage between topics or browsing facilities through its contents. The only function available is a search by main entry (artist noun). Each entry is presented on one or more HTML pages. Information is split into the following sections: full name, birth date and place, death date and place, biography, artistic data, work, discography, shows, video clips, historical and artistic data, reviews and bibliography. Fig. 1 shows a biography detail page example. According to Abiteboul et al [1] data can be classified, in terms of its structure, in three categories. Data can be structured (both schema and data types are known), semi-structured (schema or data type is known) and unstructured (schema and data type undefined). Typically, a web page contains data in the latter two categories.

160


Fig. 1. Biography detail page example

Fig. 2. Semi-structured data represented as text

Semi-structured Information. Some data as name, full name, birth place and year are presented as semi-structured information (although with variable completeness). Some examples are shown in Fig. 3. The star icon indicates birth date and place. Note that completeness and precision may vary. The first example has a full birth date (4/5/1953), birth city (Rio de Janeiro) and birth state (RJ), while the second shows only the year of birth and the last has only city and state but no date information.

Fig. 3. Semi-structured data represented as text


161

Fig. 4. Semi-strucutred data represented as text

Fig. 5. Unstructured data, plain text

Information on discography, video clips, main shows and bibliography are presented on a semi-structured manner, as text, but following a standard format (structure and style). For instance, for discography section (discografia, in Portuguese), records are listed with publication year, record company and media type, like depicted in Fig. 4. Unstructured Information. Most of information is presented as free, natural language text, like in the artistic data and biography sections, as shown in Fig. 5. The corpus used in this work is written in Portuguese, but some translated English examples are presented for the sake of readability.

4 Ontology Population Since the source information is a web site having no previous annotation, we have implemented an unsupervised semi-automatic method to generate extraction rules and templates using heuristics based on linguistic features, such as words and PoS tags. 4.1 Pre-processing The first step was to fetch web pages and to identify HTML tags to separate sections from the dictionary. During analysis of the page structure all necessary data to next steps were identified. Sections were identified through formatting marks, such as bullets and images. There was a one-to-one relationship between these marks used in HTML page and contents of the sections. These HTML tags were used together with regular expressions to determine sections boundaries. Table 1 shows some mapping examples. Each dictionary entry is composed by one or more HTML pages that have a naming convention that was used to extract the corresponding content.

162

E. Motta, S. Siqueira, and A. Andreatta Table 1. Mapping Examples Pattern Used on Source Mapped Content HTML TAG Birth date plus Birth place Regular Expression HTML TAG Biography plus Free Text Other section beginning or page end

During the extraction from HTML, non-informative tags were filtered out in order to end with plain text corresponding to each section. Plain text was then loaded on a database to allow further processing. During this stage special characters were also transformed to make them compliant to the linguistic tools that were used. For example, special UTF-8 open (“) and close (”) quotation marks were converted to ordinary quotation marks ("). 4.2 Information Extraction Information extraction requires some general purpose NLP processing like tokenization and domain specific tasks such as entity extraction. The following subsections describe the steps performed in the proposed method. General Purpose NLP. Sentence splitting, tokenization and part of speech tagging were executed. Named entities were classified (according to the domain ontology) in Artist (subclass of Person), Company (companies are divided into Record labels, Publisher companies and others) and Work (CDs, DVDs, Vinyl etc). Temporal Expression Identification. Two types of temporal expressions were handled. Type 1 is formed by expressions containing dates in numerical form, like “01/04/ 1970” or “1970, January”. This kind of expression always includes at least one numeric part. Nevertheless, many levels of precision and uncertainty may be expressed in this kind of expression, like “in the beginning of year 2000”, “circa 1870” or “in the middle of 1999”. These levels of precision and uncertainty were taken into account and represented on the ontology. Using the numeric expression as a trigger, and morphosyntactic rules, we constructed a list of patterns and classified each expression according to the ontology classes. These rules are based on word and PoS features. Table 2 shows some temporal expression patterns generated, indicating the chosen class and precision. Type 2 contains anaphors of type “in the following year” or “two years later”. This kind of expression must be resolved, linking them to the referent expression, in order to obtain a value for the temporal attribute. This is performed using a simple heuristic inspired in Mitkov’s algorithm [8]. The nearest previous date expression identified as type 1 as described above was taken as the antecedent.


163

Table 2. Mapping Examples Pattern em in entre e between and a partir de de from on circa circa

Class

Precision

Instant

Year

Defined Interval

Year

Interval, Left Open Month Around Instant

Year

Artistic Works Identification and Classification. The main feature used for identifying artistic works (e.g. songs, shows, CDs etc.) was a verb related to an artistic activity, like the verbs to perform or to record, that are related to artistic activities. In particular, due to the dictionary’s descriptive nature, verbs appear mainly in the past tense, such as gravou (recorded) or lançou (released). These verbs are used as triggers for frames defined to extract information on the object of the action (the patient, like a song or CD), who performed it (the agent, e.g., the artist involved in the action) and when such activity represented by the verb was performed (temporal adjunct). This task can be seen as a simplified approach to the semantic role labelling problem. Fig. 6 shows a frame example used to extract information for the verb lançou (released). In order to classify the artistic works, a gazetteer of work types and song genres was used in conjunction with morphosyntactic patterns. This gazetteer was constructed and pre-loaded on the ontology to support classification during information extraction. 4.3 Ontology Preparation Music Ontology [12] was selected as a base ontology to represent the specific concepts of the music domain. However, since some concepts necessary to represent the extracted information were not available on Music Ontology, a new, extended ontology was constructed based on it, adding new concepts and relationships. From the Music Ontology the concepts of MusicArtist, MusicGroup, SoloMusicArtist, MusicalManifestation were used and translated to Portuguese to support linkage to the text content. OWL-DL was chosen to represent the ontology, due to its sufficient expression power and tools available to deal with it [6]. In order to get insight on the most relevant terms (nouns and verbs) in this corpus, statistics on tokens tagged as verb or nouns were performed. Table 3 shows the top 5 verbs and top 5 nouns (common names) found in the dictionary. Due to the characteristics of events described in this dictionary, a time ontology is essential for capturing the semantic expressed in texts. Time is represented based on

164


Fig. 6. Extraction frame sample

Allen’s theory of time [4], where two kinds of entities exist: instant and interval. An instant is characterized by a “point” in time that can have different precision levels, like a specific day, year or decade etc. An interval is defined by a starting instant and an ending instant. Furthermore, textual description of historical events usually contains some vagueness like “in the beginning of year 2000” or “circa 1890”. Thus, the ontology must also be able to capture this kind of statements with vague concepts [20]. Table 3. Top 5 verbs and top 5 nouns PoS TAG

Word gravou (recorded) lançou (launched) Verb(V) participou (participated) apresentou (presented) interpretou (interpreted) ano (year) disco (record) Common Name(CN) samba música (song) parceria (partnership)

Ontology population from text demands handling imperfect information. Imperfection of information can be categorized as imprecision (vagueness), inconsistency (contradictory information) or uncertainty (trust level) [16].


165

Fig. 7. Framework overview

In the dictionary’s case, we dealt with imprecision, as we took the authority of the dictionary as trustable and assumed the content is consistent, since it is maintained by a dedicated group of music researchers. 4.4 Ontology Instantiation After all the annotations described in Sect. 4.2 were performed, the intermediate database was used to create instances of classes and relations described in the ontology. Before committing the extracted information to the ontology, a manual validation step was performed to ensure ontology consistency.

5 Implementation To implement the described approach, a framework was developed based on a subset of the conceptual model proposed by Graça et al [13]. This framework enables processing text and having multiple annotating levels. It holds all the annotation performed during the IE process that is used afterwards to instantiate the ontology concepts and relations. An overview of the framework is depicted in figure 7. Pre-processing task fetches and processes HTML pages and then store the output to the MySQL database. Information extraction tasks annotate text on multiple levels, like PoS tags, lemmas and named entities boundaries. During this phase, some information is also retrieved from the ontology, like gazetteers for work types and song genres, which were pre-loaded on the ontology by a separate process. The Ontology Instantiation step read annotated data from the database and creates classes and relations instances.

166


The solution was implemented using MySQL 5.1 and programs were developed in Java and Python. Paragraph splitting was performed based on formatting characteristics of the HTML pages. To perform sentence splitting, tokenization and PoS tagging, LX-Suite was used [5]. LX-Suite is a freely available online service for the shallow processing of Portuguese. To identify named entities, both structural hints (like parenthesis and quotation marks) and PoS tags output by LX-Suite (namely the PNM PoS tag for part of a proper name) were used. Lemmatization was performed using the NILC lexical database [22]. Protégé 3.3.1 [23] was used to create the ontology. To instantiate concept and classes, the open-source Java library Protg-OWL API was used [19].

6 Results In order to evaluate the information extraction process performance, a subset of 374 entries of the dictionary was used. This subset contains 9,912 sentences and 295,977 tokens. From these sentences, 1,102 contain the verb gravou (recorded), 943 of them contain temporal adjuncts. The other ones have no time information, as they simple state facts without temporal properties, like “Cláudia recorded a vinyl record in Japanese”. Using the proposed heuristics to generate time expression extraction templates, a set of 33 different patterns was obtained. Applying these patterns to this test set, 89.9% of time expressions were extracted (recall) and 86.3% (precision) were correctly mapped to the ontology. This result

Fig. 8. Ontology fragment sample


167

corresponds to a F1 score of 88.1% for time expressions extraction. Precision and recall was evaluated by manually tagging and inspection over this sample. These results can be compared to those from TERN (Time Expression Recognition and Normalization). Best reported results (for English) are precision of 97.3% and recall of 89.1%, corresponding to a F1 of 93.0% [2]. From a set of 260 sentences containing relative temporal expressions (anaphoric), 80.4% were correctly resolved using the proposed heuristic. A small fragment of the populated domain ontology represented as a graph is shown in Fig. 8. It shows classes for Work (Record) and Artist (SoloMusicArtist) and some of their corresponding instances.

7 Discussion This paper presented an unsupervised rule-based method for information extraction from Brazilian Portuguese texts that reaches a reasonable extraction performance with no need for training data. Although the herein proposed method is dependent on the domain and the source, it allows extracting relevant information to the domain without having to fully annotate the corpus, what would be required if a supervised method had been used. Other contributions of this work are dealing with Portuguese texts, and building a domain ontology for the popular Brazilian music history. 7.1 Related Work Other unsupervised approaches for OP are described in [17] and [9]. Hearst patterns were adapted in our method to consider also structural patterns and verb valence. Cimiano and Vlker use a dependency parser which is not available for Portuguese. A weakly supervised method is described by Tanev and Magnini [25]. Although it has better performance than the unsupervised approaches, it requires a training set. 7.2 Future Work A potential use of the populated ontology is to support a semantic browser, where the original text from the dictionary can be visualized together with the semantic labels, like described in [24]. The populated ontology also permits visualize links not explicitly expressed in the texts, like people born at the same city or year. Navigating through these discovered dimensions can reveal interesting associations among artists. Additionally, anchoring to other sources can expand information on the entries. For example, the CliqueMusic site [10] also contains information on Brazilian musicians, so that the ontology can be enriched (or at least linked to) the corresponding entries on the other database. To deal with multiple sources, the ontology must be able to handle conflicting and different trust levels. So another path for evolution is to extend the ontology to support imperfect information, as discussed in [16]. Inference engines like the one described in [15] can be applied to the OWL ontology in order to derive new knowledge. Some examples include inference using genealogical

168


relations, temporal overlapping (that can be used to determine that two artists were contemporaneous), geographical hierarchies like city contained in a state and so on. Although in the case of dictionary it was not necessary due to relatively simple and regular HTML structure, another possible evolution is to implement a wrapper induction process to generalize HTML page extraction [7].

References 1. Abiteboul, S., Buneman, P., Suciu, D.: Data on the Web. Morgan Kaufman, San Francisco (2000) 2. Ahn, D., Van Rantwijk, J., De Rijke, M.: A Cascaded Machine Learning Approach to Interpreting Temporal Expressions. In: Proceedings of NAACL HLT 2007, Rochester, NY, pp. 420–427 (2007) 3. Albin, R.: Dicionário Cravo Albin da Música Popular Brasileira, http://www.dicionariompb.com.br 4. Allen, J.: Time and Time Again - The Many Ways to Represent Time. International Journal of Intelligent Systems 6 (1991) 5. Branco, A., Silva, J.: A Suite of Shallow Processing Tools for Portuguese: LX-Suite. In: Proceedings of 11th Conference of the European Chapter of Association for Computational Linguistics, pp. 179–182 (2006) 6. Cardoso, J.: The Semantic Web Vision: Where are We. IEEE Intelligent Systems, 22–26 (September/October 2007) 7. Chang, C., Kayed, M., Girgis, M., Shaalan, K.: A Survey of Web Information Extraction Systems. IEEE Transaction on Knowledge and Data Engineering 18(10), 1411–1428 (2006) 8. Chaves, A., Rino, L.: The Mitkov Algorithm for Anaphora Resolution in Portuguese. In: Teixeira, A., de Lima, V.L.S., de Oliveira, L.C., Quaresma, P. (eds.) PROPOR 2008. LNCS (LNAI), vol. 5190, pp. 51–60. Springer, Heidelberg (2008) 9. Cimiano, P., Völker, J.: Towards large-scale open-domain and ontology-based named entity classification. In: Proceedings of RANLP 2005, Borovets, Bulgaria, pp. 166–172 (2005) 10. CliqueMusic, http://cliquemusic.uol.com.br 11. Feldman, R., Sanger, J.: The Text Mining Handbook - Advanced Approaches in Analyzing Unstructured Data. Cambridge University Press, Cambridge (2007) 12. Giasson, F., Raimond, Y.: Music Ontology Specification (2008), http://musicontology.com 13. Graça, J., Mamede, N., Pereira, J.: A framework for Integrating Natural Language Tools. In: Vieira, R., Quaresma, P., Nunes, M.d.G.V., Mamede, N.J., Oliveira, C., Dias, M.C. (eds.) PROPOR 2006. LNCS (LNAI), vol. 3960, pp. 110–119. Springer, Heidelberg (2006) ¨ 14. Gruber, T.: Ontology. In: Liu, L., Tamer Ozsu, M. (eds.) Encyclopedia of Database Systems. Springer, Heidelberg (2008) 15. Haarslev, V., Möller, R.: Racer: An OWL Reasoning Agent for the Semantic Web. In: Proceedings of the International Workshop on Applications, Products and Services of Web-based Support Systems, in conjunction with 2003 IEEE/WIC International Conference on Web Intelligence, Halifax Canada, October 13, pp. 91–95 (2003) 16. Haase, P., Völker, J.: Ontology learning and reasoning - dealing with uncertainty and inconsistency. In: da Costa, P.C.G., d’Amato, C., Fanizzi, N., Laskey, K.B., Laskey, K.J., Lukasiewicz, T., Nickles, M., Pool, M. (eds.) URSW 2005 - 2007. LNCS (LNAI), vol. 5327, pp. 366–384. Springer, Heidelberg (2008) 17. Hearst, M.: Automatic acquisition of hyponyms from large text corpora. In: Proceedings of the 14th International Conference on Computational Linguistics (COLING), pp. 539–545 (1992)


169

18. Kaiser, K., Miksch, S.: Information Extraction - A Survey. Technical Report Asgaard-TR2005-6, Vienna University of Technology, Vienna, Austria (2005) 19. Knublauch, H.: Protégé-OWL API Programmer’s Guide (2006), http://protege.stanford.edu/plugins/owl/api/guide.html 20. Mani, I., Wilson, G.: Temporal Granularity and Temporal Tagging of Text. In: AAAI 2000 Workshop on Spatial and Temporal Granularity, Austin, TX (2000) 21. Moens, M.-F.: Information Extraction: Algorithms and Prospects in a Retrieval Context. Springer, Heidelberg (2006) 22. Muniz, M., Nunes, M., Laporte, E.: UNITEX-PB, a set of flexible language resources for Brazilian Portuguese. In: Proceedings of the Workshop on Technology on Information and Human Language (TIL), São Leopoldo, Brazil (2005) 23. Protégé, http://protege.stanford.edu 24. Quan, D., Karger, D.: How to make a semantic web browser. In: Proceedings of the 13th international conference on World Wide Web (2004) 25. Tanev, H., Magnini, B.: Weakly Supervised Approaches for Ontology Population. In: Proceedings of 11th Conference of the European Chapter of the Association for Computational Linguistics: EACL 2006 (2006) 26. Yildiz, B., Miksch, S.: Motivating Ontology-Driven Information Extraction. In: Proceedings of the International Conference on Semantic Web and Digital Libraries, ICSD 2007 (2007)

Web Spam, Social Propaganda and the Evolution of Search Engine Rankings Panagiotis Takis Metaxas Wellesley College, Wellesley, MA 02481, U.S.A. [email protected] http://cs.wellesley.edu/∼pmetaxas/

Abstract. Search Engines have greatly influenced the way we experience the web. Since the early days of the web, users have been relying on them to get informed and make decisions. When the web was relatively small, web directories were built and maintained using human experts to screen and categorize pages according to their characteristics. By the mid 1990’s, however, it was apparent that the human expert model of categorizing web pages does not scale. The first search engines appeared and they have been evolving ever since, taking over the role that web directories used to play. But what need makes a search engine evolve? Beyond the financial objectives, there is a need for quality in search results. Search engines know that the quality of their ranking will determine how successful they are. Search results, however, are not simply based on well-designed scientific principles, but they are influenced by web spammers. Web spamming, the practice of introducing artificial text and links into web pages to affect the results of web searches, has been recognized as a major search engine problem. It is also a serious users problem because they are not aware of it and they tend to confuse trusting the search engine with trusting the results of a search. In this paper, we analyze the influence that web spam has on the evolution of the search engines and we identify the strong relationship of spamming methods on the web to propagandistic techniques in society. Our analysis provides a foundation for understanding why spamming works and offers new insight on how to address it. In particular, it suggests that one could use social anti-propagandistic techniques to recognize web spam. Keywords: Search engines, Web search, Web graph, Link structure, PageRank, HITS, Web spam, Social networks.

1 Introduction Search Engines have greatly influenced the way we experience the web. Since the early days of the web people have been relying on search engines to find useful information. When the web was relatively small, Web directories were built and maintained that were using human experts to screen and categorize pages according to their characteristics.

Part of this work was supported by a Brachman-Hoffman grant.


Web Spam, Social Propaganda and the Evolution of Search Engine Rankings

171

By the mid 1990’s, however, it was apparent that the human expert model of categorizing web pages would not scale. The first search engines appeared and they have been evolving ever since. But what influences their evolution? The way a user interacts with a search engine is through the search results to a query that he or she has issued. Search engines know that the quality of their ranking will determine how successful they are. If users perceive the results as valuable and reliable, they will come again. Otherwise, it is easy for them to switch to another search engine. Research in Information Retrieval has produced a large body of work that, theoretically, produces high quality search results. Yet, search engines admit that IR theory is but one of their considerations. One of the major issues that influences the quality of ranking is the effect that web spam has on their results. Web spamming is defined as the practice of manipulating web pages in order to influence search engines rankings in ways beneficial to the spammers. Spammers aim at search engines, but target the end users. Their motive is usually commercial, but can also be political or religious. One of the reasons behind the users’ difficulty to distinguish trustworthy from untrustworthy information comes from the success that both search engines and spammers have enjoyed in the last decade. Users have come to trust search engines as a means of finding information, and spammers have successfully managed to exploit this trust. From their side, the search engines have put considerable effort in delivering spamfree query results and have developed sophisticated ranking strategies. Two such ranking strategies that have received major attention are the PageRank [4] and HITS [22]. Achieving high PageRank has become a sort of obsession for many companies’ IT departments, and the raison d’être of spamming companies. Some estimates indicate that at least 8% of all pages indexed is spam [10] while experts consider web spamming the single most difficult challenge web searching is facing today[18]. Search engines typically see web spam as an interference to their operations and would like to restrict it, but there can be no algorithm that can recognize spamming sites based solely on graph isomorphism [3]. First, however, we need to understand why spamming works beyond the technical details, because spamming is a social problem first, then a technical one. In this paper we show its extensive relationship to social propaganda, and evidence of its influence on the evolution of search engines. Our approach can explain the reasons why web spamming has been so successful and suggest new ways of dealing with it. Finally, we present a framework for the long-term approach to web spam. Background. Web spamming has received a lot of attention lately [2, 3, 10, 11, 15, 17, 18, 20, 23, 26, 27, 30]. The first papers to raise the issue were [27, 18]. The spammers’ success was noted in [2, 9–11, 19]. Characteristics of spamming sites based on diversion from power laws are presented in [10]. Current tricks employed by spammers are detailed in [14]. An analysis of the popular PageRank method employed by many search engines today and ways to maximize it in a spamming network is described in [3]. TrustRank, a modification to the PageRank to take into account the evaluations of a few seed pages by human editors, employees of a search engine, is presented in [15]. Techniques for identifying automatically link farms of spam pages were presented in [38, 1].

172

P.T. Metaxas

A comprehensive treatment on social networks is presented in [36]. The connection between the Web and social networks was explicitly noted in [24, 31] and implicitly used in [4, 22]. In fact, Kleinberg’s work explores many of these connections (e.g., [21]). Identification of web communities was explored in [23, 12]. The effect that search engines have on page popularity was discussed in [7]. The rest of this paper is organized as follows. The next section gives an overview of the problem of information reliability and web spamming. Section 3 has a short introduction to the theory of propaganda detection and the next section 4 discusses the relationship between the webgraph and the trust social network. The following section 5 analyzes the evolution of search engines as their response to spam. Finally, the last section has the conclusions and a framework for the long-term approach to web spam.

2 Web Spam The web has changed the way we inform and get informed. Every organization has a web site and people are increasingly comfortable accessing it for information on any question they may have. The exploding size of the web necessitated the development of search engines and web directories. Most people with online access use a search engine to get informed and make decisions that may have medical, financial, cultural, political, security or other important implications in their lives [9, 35, 19, 26]. Moreover, 85% of the time, people do not look past the first ten results returned by the search engine [33]. Given this, it is not surprising that anyone with a web presence struggles for a place in the top ten positions of relevant web search results. The importance of the top-10 placement has given birth to a new “Search Engine Optimization” industry, which claims to sell know-how for prominent placement in search results and includes companies, publications, and even conferences. Some of them are willing to bend the truth in order to fool the search engines and their customers, by creating web pages containing web spam [10]. Spammers attack search engines through text and link manipulations: – Text Spam. This includes repeating text excessively and/or adding irrelevant text on the page that will cause incorrect calculation of page relevance; adding misleading meta-keywords or irrelevant “anchor text” that will cause incorrect application of rank heuristics. – Link Spam. This technique aims to change the perceived structure of the webgraph in order to cause incorrect calculation of page reputation. Such examples are the socalled “link-farms,” domain flooding (plethora of domains that re-direct to a target site), page “awards,” (the spammer pretends to run an organization that distributes awards for web site design or information; the awarded site gets to display the “award”, an image linking back to awarding organization, effectively increasing the visibility of the spammer’ site), etc. Both kinds of spam aim to boost the ranking of spammed web pages. So as not to get caught, spammers conceal their actions through cloacking, content hiding and redirection. Cloaking, for example, aims to serve different pages to search engine robots and to web browsers (users). For a comprehensive treatment of the spamming techniques, see [14].


173

Since anyone can be an author on the web, these practices have naturally created a question of information reliability. An audience used to trusting the written word of newspapers and books is unable, unprepared or unwilling to think critically about the information obtained from the web. A recent study [13] found that while college students regard the web as a primary source of information, many do not check more than a single source, and have trouble recognizing trustworthy sources online. In particular, two out of three students are consistently unable to differentiate between facts and advertising claims, even “infomercials.” Very few of them would double-check for validity. At the same time, they have considerable confidence in their abilities to distinguish trustworthy sites from non-trustworthy ones, especially when they feel technically competent. We have no reason to believe that the general public will perform any better than well-educated students. In fact, a recent analysis of internet related fraud by a major Wall Street law firm [9] puts the blame squarely on the questionable critical thinking skills of the investors for the success of stock fraud cases.

3 On Propaganda Theory On the outset, it may seem surprising that a technical article discusses social propaganda. This is a subject that has been studied extensively by social scientists and might seem out of the realm of computing. However, the web is a social network, influenced daily by the actions (intentional or otherwise) of millions of people. In that respect, web researchers should be aware of social theories and practices since they may have applicability in their work. We believe that a basic understanding of social propaganda can be valuable to technical people designing and using systems that affect our social interactions. In particular, it can be useful to researchers that study Web Spam. We offer here a brief introduction to the theory of propaganda detection. There are many definitions of propaganda, reflecting its multiple uses over time. One working definition we will use here is Propaganda is the attempt to modify human behavior, and thus influence people’s actions in ways beneficial to propagandists. Propaganda has a long history in modern society and is often associated with negative connotation. This was not always the case, however. The term was first used in 1622, in the establishment by the Catholic Church of a permanent Sacred Congregation de Propaganda Fide (for the propagaton of faith), a department which was trying to spread Catholicism in non-Catholic Countries [37]. Its current meaning comes from the successful Enemy Propaganda Department in the British Ministry of Information during WWI. However, it was not until 1938, in the beginning of WWII, that a theory was developed to detect propagandistic techniques. For the purposes of this paper we are interested in ways of detecting propaganda, especially by automatic means. First developed by the Institute for Propaganda Analysis [25], classic Propaganda Theory identifies several techniques that propagandists often employ in order to manipulate perception.

174

P.T. Metaxas

– Name Calling is the practice of giving an idea a bad label. It is used to make people reject and condemn the idea without examining the evidence. For example, using the term “miserable failure” to refer to political leaders such as US President George Bush can be thought of as an application of name calling. – Glittering Generalities is the mirror image1 of name calling: Associating an idea with a “virtue word”, in an effort to make us accept and approve the idea without examining the evidence. For example, using the term “patriotic” to refer to illegal actions is a common application of this technique. – Transfer is the technique by which the propagandist carries over the authority, sanction, and prestige of something respected and revered to something he would have us accept. For example, delivering a political speech in a mosque or a church, or ending a political gathering with a prayer have the effect of transfer. – Testimonial is the technique of having some respected person comment on the quality of an issue on which they have no qualifications to comment. For example, a famous actor who plays a medical doctor on a popular TV show tells the viewers that she only uses a particular pain relief medicine. The implicit message is that if a famous personality trusts the medicine, we should too. – Plain Folks is a technique by which speakers attempt to convince their audience that they, and their ideas, are “of the people,” the “plain folks”. For example, politicians sometimes are seen flipping burgers at a neighborhood diner. – Card Stacking involves the selection of facts (or falsehoods), illustrations (or distractions), and logical (or illogical) statements in order to give an incorrect impression. For example, some activists refer to the Evolution Theory as a theory teaching that humans came from apes (and not that both apes and humans have evolved from a common ancestor who was neither human nor ape). – Bandwagon is the technique with which the propagandist attempts to convince us that all members of a group we belong to accept his ideas and so we should “jump on the band wagon”. Often, fear is used to reinforce the message. For example, commercials might show shoppers running to line up in front of a store before it is open. The reader should not have much trouble identifying additional examples of such techniques used in politics or advertising. The next section discusses the relationship of propaganda to web spam, by first describing the similarity of social networks to the web graph.

4 The Webgraph as a Trust Network The web is typically represented by a directed graph [6]. The nodes in the webgraph are the pages (or sites) that reside on servers on the internet. Arcs correspond to hyperlinks that appear on web pages (or sites). In this context, web spammers’ actions can be seen as altering the contents of the web nodes (mailnly through text spam), and the hyperlinks between nodes (mainly through link spam). 1

Name calling and glittering generalities are sometimes referred to as “word games.”


175

The theory of social networks [36] also uses directed graphs to represent relationships between social entities. The nodes correspond to social entities (people, institutions, ideas). Arcs correspond to recommendations between the entities they connect. In this context, propagandistic techniques can be seen as altering the trust social network by altering one or more of its components (i.e., nodes, arcs, weights, topology). To see the correspondence more clearly, we will examine some of the propagandistic techniques that have been used successfully by spammers: The technique of testimonials effectively adds a link between previously unrelated nodes. Glittering generalities change the contents of a node, effectively changing its perceived relevance. Mislabeled anchor text is an example of card stacking. And the technique of bandwagon creates many links between a group of nodes, a “link farm”. So, we define web spam based on the spammers actions: Web Spam is the attempt to modify the web (its structure and contents), and thus influence search engine results in ways beneficial to web spammers. Table 1 has the correspondence, in graph theoretic terms, between the web graph according to a search engine and the trust social network of a particular person. Web pages or sites correspond to social entities and hyperlinks correspond to trust opinions. The rank that a search engine assigns to a page or a site corresponds to the reputation a social entity has for the person. This rank is based on some ranking formula that a search engine is computing, while the reputation is based on idiosyncratic components associated with the person’s past experiences and selective application of critical thinking skills; both are secret and changing. This correspondence is more than a coincidence. The web itself is a social creation, and both PageRank and HITS are socially inspired ranking formulas. [4, 22, 31]. Socially inspired systems are subject to socially inspired attacks. Not surprisingly then, the theory of propaganda detection can provide intuition into the dynamics of the web graph. PageRank is based on the assumption that the reputation of an entity (a web page in this case) can be measured as a function of both the number and reputation of other entities linking to it. A link to a web page is counted as a “vote of confidence” to this web site, and in turn, the reputation of a page is divided among those it is recommending2. The implicit assumption is that hyperlink “voting” is taking place independently, without prior agreement or central control. Spammers, like social propagandists, form structures that are able to gather a large number of such “votes of confidence” by design, thus breaking the crucial assumption of independence in a hyperlink. But while the weights in the web graph are assigned by each search engine, the weights in the trust social network are assigned by each person. Since there are many more persons than search engines, the task of a web spammer is far easier than the task of a propagandist.

2

Since HTML does not provide for “positive” and “negative” links, all links are taken as positive. This is not always true, but is considered a reasonable assumption. Recently, Google introduced the “nofollow” attribute for hyperlinks, as a tool for blog site owners to mark visitor opinions. It is very unlikely that spamming blog owners will use it, however.

176

P.T. Metaxas

Table 1. Graph theoretic correspondence between the Webgraph and the Trust Social Network. There is a one-to-one correspondence between each component of the two graphs. A major difference, however, is that, even though a person may feel negative trust (distrust) for some entity, there is no negative weight for hyperlinks. Graph Theory Node weight weight computation Arc semantics weight weight range

Web Graph web page or site rank (accord. to a search engine) ranking formula (e.g., pagerank) computed continuously hyperlink “vote of confidence” degree of confidence [0 . . . 1]

Trust Social Network social entity reputation (accord. to a person) idiosyncratic (e.g., 2 recommenders) computed on demand trust opinion “recommendation” degree of entrustment [distrust . . . trust]

5 Search Engine Evolution In the early 90’s, when the web numbered just a few million servers, the first generation search engines were ranking search results using the vector model of classic information retrieval techniques: the more rare words two documents share, the more similar they are considered to be. [32, 17] According to the vector model in Information Retrieval [32], documents contained in a document collection D are viewed as vectors in term space T . Each document vector is composed of term weights wik of term Tk appearing in document Di . These weights are computed as the normalized dot product of tfik · idfk , where tfik is the frequency of term Tk in document Di , and idfk is the inverse document frequency of term Tk in document collection D. Typically, idfk is computed by a logarithmic formula so that this term will not grow significantly as the number of occurrences of Tk increase. Under this formulation, rare words have greater weight than common words, because they are viewed as better representing the document contents. The term weights are then normalized to fall on a unit sphere so that longer documents will not have an advantage over shorter documents: wik =

tfik · idfk 2 2 1≤k≤t (tfik ) (idfk )

In the vector model, document similarity sim(D1 , D2 ) between document vectors D1 and D2 is represented by the angle between them, and is computed as 1≤i≤t w1i · w2i cosine normalized: 1≤i≤t w1i · w2i sim(D1 , D2 ) = 2 2 1≤i≤t (w1i ) · 1≤i≤t (w2i ) A search query Q is considered simply a short document and the results of a search for Q are ranked according to their (normalized) similarity to the query. While the exact


177

details of the computation of term weights were kept secret, we can say that the ranking formula RG1 in the first generation search engines was based in the following principle: the more rare keywords a document shares with a query, the higher similarity it has with it, resulting in a higher ranking score for this document: RG1 = f (sim(p, Q))

(1)

The first attack to this ranking came from within the search engines. In 1996, search engines started openly selling search keywords to advertisers [8] as a way of generating revenue: If a search query contained a “sold” keyword, the results would include targeted advertisement and a higher ranking for the link to the sponsor’s web site. Mixing search results with paid advertisement raised serious ethical questions, but also showed the way to financial profits to spammers who started their own attacks using keyword stuffing, i.e., by creating pages containing many rare keywords to obtain a higher ranking score. In terms of propaganda theory, the spammers employed a variation of the technique of glittering generalities to confuse the first generation search engines [25, pg. 47]: The propagandist associates one or more suggestive words without evidence to alter the conceived value of a person or idea. In an effort to nullify the effects of glittering generalities, second generation search engines started employing additionally more sophisticated ranking techniques. One of the more successful techniques was based on the “link voting principle”: Each web site s has value equal to its “popularity” |Bs | which is influenced by the set Bs of sites pointing to s. Therefore, the more sites were linking to a site s, the higher the popularity of s’s pages. Lycos became the champion of this ranking technique [28] and had its own popularity skyrocket in late 1996. Doing so, it was also distancing itself from the ethical questions introduced by blurring advertising with ranking [8]. The ranking formula RG2 in the second generation search engines was a combination of a page’s similarity, sim(p, Q), and its site’s popularity |Bs |: RG2 = f (sim(p, Q), |Bs |)

(2)

To avoid spammers search engines would keep secret their exact ranking algorithm. Secrecy is no defense, however, since secret rules were figured out by experimentation and reverse engineering. (e.g., [30, 27]). Unfortunately, this ranking formula did not succeed in stopping spammers either. Spammers started creating clusters of interconnected web sites that had identical or similar contents with the site they were promoting, a technique that subsequently became known as link farms. The link voting principle was socially inspired, so spammers used the well known propagandistic method of bandwagon to circumvent it [25, pg. 105]: With it, the propagandist attempts to convince us that all members of a group to which we belong are accepting his program and that we must therefore follow our crowd and “jump on the band wagon”. Similarly, the spammer is promoting the impression of a high degree of popularity by inter-linking many internally controlled sites that will eventually all share high ranking.

178

P.T. Metaxas

PageRank and HITS marked the development of the third generation search engines. The introduction of PageRank in 1998 [4] was a major event for search engines, because it seemed to provide a more sophisticated anti-spamming solution. Under PageRank, not every link contributes equally to the “reputation” of a page P R(p). Instead, links from highly reputable pages contribute much higher value than links from other sites. A page p has reputation P R(p) which is calculated as the sum of fractions of the reputations of the set Bp of pages pointing to p. Let Fv be the set of links out of page v, v ∈ Bp . The reputation of a page is P R(p) =

P R(v) 1−t +t N |Fv | v∈Bp

where t is the so-called “transport” factor and N is the total number of pages in the collection. That way, the link farms developed by spammers would not influence much their PageRank, and Google became the search engine of choice. HITS is another socially-inspired ranking which has also received a lot of attention [22] and is reportedly used by the AskJeeves search engine. The HITS algorithm divides the sites related to a query between “hubs” and “authorities”. Hubs are sites that contain many links to authorities, while authorities are sites pointed to by the hubs and they both gain reputation. Unfortunately, spammers again found ways of circumventing these rankings. In PageRank, a page enjoys absolute reputation: its reputation is not restricted on some particular issue. Spammers deploy sites with expertise on irrelevant subjects, and they acquire (justifiably) high ranking on their expert sites. Then they bandwagon the irrelevant expert sites, creating what we call a mutual admiration society. In propagandistic terms, this is the technique of testimonials [25, pg. 74] often used by advertisers: Well known people (entertainers, public figures, etc.) offer their opinion on issues about which they are not experts. Spammers were so aggressive in pursuing this technique that they openly promoted “reciprocal links”: Web masters controlling sites that had some minimum PageRank, were invited to join a mutual admiration society by exchanging links, so that at the end everyone’s PageRank would increase. HITS has also shown to be highly spammable by this technique due to the fact that its effectiveness depends on the accuracy of the initial neighborhood calculation. Another heuristic that third generation search engines used was that of exploiting “anchor text”. It had been observed that users creating links to web pages would come to use, in general, meaningful descriptions of the contents of a page. (Initially, the anchor text was non-descriptive, such as “click here”, but this changed in the late 1990’s.) Google was the first engine to exploit this fact noting that, even though IBM’s web page made no mention that IBM is a computer company, many users linked to it with anchor text such as “computer manufacturer”. Spammers were quick to exploit this feature too. In early 2001, a group of activists started using the anchor text “miserable failure” to link to the official Whitehouse page of American President George W. Bush. Using what became known as “Googlebomb” or, more accurately, link-bomb since it does not pertain to Google only, other activists


179

Table 2. Changes in ranking by generations of search engines, the response of the web spammers and the corresponding propagandistic techniques S.E.’s Ranking Spamming Propaganda 1st Gen Doc Similarity keyword stuffing glittering generalities 2nd Gen + Site popularity + link farms + bandwagon 3rd Gen + Page reputation + mutual admiration societies + testimonials + anchor text + link bombs + card stacking

linked the same anchor text to President Carter, filmmaker Michael Moore and Senator Hilary Clinton. Using the anchor text is socially inspired, so spammers used the propagandistic method of card stacking to circumvent it [25, pg. 95]: Card stacking involves the selection and use of facts or falsehoods, illustrations or distructions, and logical or illogical statements in order to give the best or the worst possible case for an idea, program, person or product. The ranking formula RG3 in the third generation search engines is, therefore, some secret combination of a number of features, primarily the page’s similarity, sim(p, Q), its site’s popularity |Bs | and its the page’s reputation P R(p): RG3 = f (sim(p, Q), |Bs |, P R(p))

(3)

Search engines these days claim to have developed hundreds of little heuristics for improving their web search results [16] but no big idea that would move their rankings beyond the grasp of spammers. As Table 2 summarizes, for every idea that search engines have used to improve their ranking, spammers have managed quickly to balance it with techniques that resemble propagandistic techniques from society. Web search corporations are reportedly busy developing the engines of the next generation [5]. The new techniques aim to be able to recognize “the need behind the query” of the user. Given the success the spammers have enjoyed so far, one wonders how will they spam the fourth generation engines. Is it possible to create a ranking that is not spammable? Put another way, can the web as a social space be free of propaganda? This may not be possible. Our analysis shows that we are trying to create in cyberspace what societies have not succeeded in creating in their real space. However, we can learn to live in a web with spam as we live in society with propaganda, given appropriate education and technology. We touch upon it in our concluding section.

6 Conclusions In this paper we have argued that web spam is to cyberworld what propaganda is to society. As evidence of the importance of this analogy, we have shown that the evolution of search engines can be largely understood as the search engines’ responses in defending against spam. We do not suggest here that web spam is the sole force behind the evolution of search engines, but that it is a dominant one. New search engines are developed when researchers believe they have a good answer to spam because it directly affects the quality of the search results.

180

P.T. Metaxas

Further, our findings suggests that anti-spamming techniques can be developed by mimicking anti-propagandistic methods. In a followup paper [29] we present automatic ways of recognizing trust graphs on the web based on anti-propagandistic techniques. Our idea is to propagate distrust to a spamming network whenever one of them is recognized. In the next couple paragraphs we give a short description of our results. We are considering trustworthiness to be a personal decision, not an absolute quality of a site. One person’s gospel is another’s political propaganda, and our goal is to design methods that help individuals make more informed decisions about the quality of the information they find on the web. Here is one way that people defend against propaganda in every day life: In society, distrust is propagated backwards: When an untrustworthy recommendation is detected, it gives us a reason to reconsider the trustworthiness of the recommender. Recommenders who strongly support an untrustworthy recommendation become untrustworthy themselves. This process is selectively repeated a few times, propagating the distrust backwards to those who strongly support the recommendation. The results of this process become part of our belief system and are used to filter future information. (Note that distrust is not propagated forward: An untrustworthy person’s recommendations could be towards any entity, either trustworthy or untrustworthy.) Experimental results [29] from a number of such instances show our algorithm’s ability of recognizing parts of a spamming network. Therefore, our work is complementary to the recent developments that recognize web spam based on link analysis [38, 1]. But what one should do once one recognizes a spamming network. This is a question that has not attracted the necessary attention in the past. The default approach is that a search engine would delete such networks from its indices or might downgrade them by some prespecified amount. Search engines are reportedly doing a fair amount of this [34, 10, 15]. A more effective way is personalizing the web graph a user sees, effectively increasing the task difficulty of a spammer to the level of a propagandist: As we mentioned, a spammer has an easier job than a propagandist because he/she has to influence the web graphs of a few search engines instead of the trust graphs of millions of individuals. There are clearly cases where these approaches are appropriate and effective. But in general, both of these approaches require a universal agreement of what constitutes spam. Such an agreement cannot exist; one person’s spam may be another person’s treasure. Should the search engines determine what is trustworthy and what is not? Willing or not, they are the de facto arbiters of what information users see [34]. As in a popular cartoon by Ohman & Willis, a kid responds to the old man who has been searching his entire life for the meaning of life: “[...]if it’s not on Google, you probably won’t find it.” We believe that it is the users’ right and responsibility to decide what is acceptable for them. Their browser, their window to cyberworld, should enhance their ability to make this decision. User education is fundamental: without it, people will largely trust what they see, regardless its credibility. People should know how search engines work and why, and how information appears on the web. But they should also have a trained browser that can help them determine the validity and trustworthiness of information.


181

References 1. Benczúr, A., Csalogány, K., Sarlós, T., Uher, M.: Spam Rank – Fully automatic link spam detection. In: Proceedings of the AIRWeb Workshop (2005) 2. Bharat, K., Chang, B.-W., Henzinger, M.R., Ruhl, M.: Who links to whom: Mining linkage between web sites. In: Proceedings of the 2001 IEEE International Conference on Data Mining, pp. 51–58. IEEE Computer Society, Los Alamitos (2001) 3. Bianchini, M., Gori, M., Scarselli, F.: PageRank and web communities. In: Web Intelligence Conference 2003 (2003) 4. Brin, S., Page, L.: The anatomy of a large-scale hypertextual Web search engine. Computer Networks and ISDN Systems 30(1-7), 107–117 (1998) 5. Broder, A.: A taxonomy of web search. SIGIR Forum 36(2), 3–10 (2002) 6. Broder, A., Kumar, R., Maghoul, F., Raghavan, P., Rajagopalan, S., Stata, R., Tomkins, A., Wiener, J.: Graph structure in the web. Comput. Networks 33(1-6), 309–320 (2000) 7. Cho, J., Roy, S.: Impact of search engines on page popularity. In: WWW 2004 (2004) 8. CNETNews, Engine sells results, draws fire (1996), http://news.cnet.com/2100-1023215491.html 9. Corey, T.S.: Catching on-line traders in a web of lies: The perils of internet stock fraud. Ford Marrin Esposito, Witmeyer & Glesser, LLP (2001), http://www.fmew.com/archive/lies/ 10. Fetterly, D., Manasse, M., Najork, M.: Spam, damn spam, and statistics. In: WebDB 2004 (2004) 11. Fetterly, D., Manasse, M., Najork, M., Wiener, J.: A large-scale study of the evolution of web pages. In: Proceedings of the twelfth international conference on World Wide Web, pp. 669–678. ACM Press, New York (2003) 12. Flake, G.W., Lawrence, S., Giles, C.L., Coetzee, F.: Self-organization of the web and identification of communities. IEEE Computer 35(3), 66–71 (2002) 13. Graham, L., Metaxas, P.T.: Of course it’s true; I saw it on the internet!: Critical thinking in the internet era. Commun. ACM 46(5), 70–75 (2003) 14. Gyöngyi, Z., Garcia-Molina, H.: Web spam taxonomy. In: Proceedings of the AIRWeb Workshop (2005) 15. Gyöngyi, Z., Garcia-Molina, H., Pedersen, J.: Combating web spam with TrustRank. In: VLDB 2004 (2004) 16. Hansell, S.: Google keeps tweaking its search engine. New York Times (2007) 17. Henzinger, M.R.: Hyperlink analysis for the web. IEEE Internet Computing 5(1), 45–50 (2001) 18. Henzinger, M.R., Motwani, R., Silverstein, C.: Challenges in web search engines. SIGIR Forum 36(2), 11–22 (2002) 19. Hindman, M., Tsioutsiouliklis, K., Johnson, J.: Googlearchy: How a few heavily-linked sites dominate politics on the web. Annual Meeting of the Midwest Political Science Association (2003) 20. Introna, L., Nissenbaum, H.: Defining the web: The politics of search engines. Computer 33(1), 54–62 (2000) 21. Kleinberg, J.: The small-world phenomenon: an algorithm perspective. In: STOC 2000: Proceedings of the thirty-second annual ACM symposium on Theory of computing, pp. 163– 170. ACM Press, New York (2000) 22. Kleinberg, J.M.: Authoritative sources in a hyperlinked environment. Journal of the ACM 46(5), 604–632 (1999) 23. Kumar, R., Raghavan, P., Rajagopalan, S., Tomkins, A.: Trawling the Web for emerging cyber-communities. Computer Networks 31(11-16), 1481–1493 (1999)

182

P.T. Metaxas

24. Kumar, R., Raghavan, P., Rajagopalan, S., Tomkins, A.: The web and social networks. IEEE Computer 35(11), 32–36 (2002) 25. Lee, A.M., Lee, E.B. (eds.): The Fine Art of Propaganda. The Institute for Propaganda Analysis. Harcourt, Brace and Co. (1939) 26. Lynch, C.A.: When documents deceive: trust and provenance as new factors for information retrieval in a tangled web. J. Am. Soc. Inf. Sci. Technol. 52(1), 12–17 (2001) 27. Marchiori, M.: The quest for correct information on the web: hyper search engines. Comput. Netw. ISDN Syst. 29(8-13), 1225–1235 (1997) 28. Maulding, M.L.: Lycos: Design choices in an internet search service. IEEE Expert (12), 8–11 (1997) 29. Metaxas, P.T.: Using Propagation of Distrust to find Untrustworthy Web Neighborhoods. In: Proceedings of the fourth international conference on internet and Web Applications and Services, Venice, Italy, May 24-28 (2009) 30. Pringle, G., Allison, L., Dowe, D.L.: What is a tall poppy among web pages? In: Proceedings of the seventh international conference on World Wide Web 7, pp. 369–377. Elsevier Science Publishers B. V., Amsterdam (1998) 31. Raghavan, P.: Social networks: From the web to the enterprise. IEEE Internet Computing 6(1), 91–94 (2002) 32. Salton, G.: Dynamic document processing. Commun. ACM 15(7), 658–668 (1972) 33. Silverstein, C., Marais, H., Henzinger, M., Moricz, M.: Analysis of a very large web search engine query log. SIGIR Forum 33(1), 6–12 (1999) 34. Totty, M., Mangalindan, M.: As google becomes web’s gatekeeper, sites fight to get in. Wall Street Journal CCXLI (39) (2003) 35. Vedder, A.: Medical data, new information technologies and the need for normative principles other than privacy rules. In: Freeman, M., Lewis, A. (eds.) Law and Medicine. Series Current Legal Issues, pp. 441–459. Oxford University Press, Oxford (2000) 36. Wasserman, S., Faust, K.: Social Network Analysis: Methods and Applications. Cambridge University Press, Cambridge (1994) 37. Welch, D.: Power of persuasion - propaganda. History Today 49(8), 24–26 (1999) 38. Wu, B., Davison, B.: Identifying link farm spam pages. In: Proceedings of the fourteenth international conference on World Wide Web. ACM Press, New York (2005)

Part III Society, e-Business and e-Government

Making the Invisible Visible: Design Guidelines for Supporting Social Awareness in Distributed Collaboration Monique Janneck University of Hamburg, Germany [email protected]

Abstract. Mechanisms supporting a shared representation of activities — or awareness—within a group of people are an important prerequisite for successful computer supported collaborative activities. This article highlights the design of awareness support from a social psychological viewpoint of human behaviour in and within groups. Based on this, design guidelines for awareness support in distributed collaboration—with an emphasis on promoting social awareness—are proposed and evaluated empirically. Results show that users awareness was influenced positively as predicted by the design guidelines. Keywords: Awareness, Computer-supported cooperative work, Computersupported cooperative learning, Social psychology.

1 Introduction For collaborative activities to take place, the people involved need to develop a shared understanding of the context that they are cooperating in and the tasks that they are working on. In face-to-face interaction this is an implicit process, which is commonly supported by nonverbal communication. Tools supporting collaborative activities need to provide for mechanisms enabling this kind of awareness in computer-supported interaction as well: “Awareness is an understanding of the activities of others, which provides a context for your own activities” [4]. Collaborative activities are necessarily social activities. Research focusing on computer supported cooperative learning (CSCL) or work (CSCW) generally stresses the meaning of social processes influencing shared representation and generation of knowledge within a group of people, drawing especially on social constructivist theories, e.g. [16]. However, long-known, ‘classical’ social psychological findings and theories providing a plethora of insights into interaction within and between groups have been barely drawn upon to inform the design of cooperative systems [11,17]. To start filling this gap, this article highlights the design of awareness support in CSCL/CSCW tools from a social psychological viewpoint. Based on two well-known concepts to explain intragroup behaviour—group structures and relations, and social loafing—design guidelines for awareness support in distributed collaboration are developed. In this work, the emphasis is on promoting social awareness, i.e., representing social interactions of people using a shared workspace. J. Cordeiro and J. Filipe (Eds.): WEBIST 2009, LNBIP 45, pp. 185–197, 2010. c Springer-Verlag Berlin Heidelberg 2010

186

M. Janneck

To put these design guidelines to a first empirical test, several simple awareness functions were added to an existing groupware system and evaluated by means of a quasiexperimental design comparing both subjective and objective awareness measures in a field test before and after the new awareness functions had been implemented. Results show that users’ awareness was influenced positively as predicted by the design guidelines. This article is structured as follows: Section 2 discusses selected issues of intragroup behavior as a basis for the design of awareness mechanisms, drawing on ‘classical’ social psychological research. Section 3 describes related work regarding the representation of social activities in computer-supported interaction. Section 4 introduces design guidelines for implementing social awareness support based on the concepts discussed in section 2. In section 5, an empirical implementation and evaluation of the design guidelines is described. Section 6 concludes this article with a discussion of the results and an outline for future work.

2 Social Psychological Perspectives on Group Work In the following paragraphs, two selected social psychological concepts and their meaning for computer-supported collaboration and the design of cooperative systems are discussed exemplarily: group structures and relations, and social loafing. In section 4, they will be drawn upon to devise guidelines for designing functions supporting social awareness in computer-supported learning and working groups. Group Structures and Relations. It is a well-known fact that structures and relations within groups heavily influence the group’s success concerning collaborative activities, e.g. [1,9]. Centralized or hierarchical structures of communication (like the ‘Wheel’ or the ‘Chain’, see Figure 1), dominated by single individuals, threaten to discourage other group members, thus lowering group productivity. On the other hand, structures enabling equal contributions and self-dependent work of all members (like the ‘Net’) yield better results [19]. Of course, group structures and relations conducive to productive collaboration cannot be established or even enforced through software. On the other hand,however, software design can facilitate or impede on collaboration. Awareness mechanisms can serve to make group structures more transparent.

Fig. 1. Communication Structures [19]

Making the Invisible Visible: Design Guidelines

187

Social Loafing. Group members who benefit from other members’ efforts while contributing little themselves are known as social loafers, e.g. [18]. In computer-mediated communication, social loafing is often referred to as lurking, addressing the problem of the passiveness of—usually—the vast majority of users, while only a small number of participants contribute actively, e.g. [21]. However, Takahashi et al. [28] report that many of the alleged ‘lurkers’ nevertheless made a substantial contribution to offline communication in their company by using and actively spreading information they obtained from the intranet. Exchanging and gathering information online are typical activities in distributed collaboration. Quite naturally, though, downloading data from e.g. a shared workspace will leave less traces of activity than lively online discussions. Making this kind of ‘passivity’—which might nevertheless result in activity elsewhere—more visible can be supported by awareness mechanisms. Social loafing occurs most often when people get the impression that their contribution is unidentifiable or redundant regarding the overall group result, meaning that the missing contribution will go unnoticed or remain without consequences. Social loafing is reduced when group members take on responsibility for the group result, and when their individual contributions are clear. Thus, awareness mechanisms serve to make individual activities more visible—and also more valuable.

3 Related Work: Representing Social Activities Typical awareness mechanisms include notifications of events or changes and tracking mechanisms of presence (“who is online?”) and actions within the shared workspace. Many cooperative systems support mainly awareness of tasks that are worked on within the group (e.g., notification of changes that were made to a document). The representation of social activities—or social awareness—which often manifest in cursory interactions, leaving no visible or tangible trace (comparable to ‘coffee breaks’ in face-to-face interactions), is more of a challenge to systems design, meaning to make ‘invisible’ actions (e.g., browsing the workspace and reading entries) more visible. In the following paragraphs some approaches and examples of representing social activity are portrayed. The social navigation approach aims at enabling users to find relevant information by interpreting traces that other users left behind [2], thus conveying the history of digital objects [31]. However, so far use scenarios for social navigation have been mainly explored for individual navigation in web resources (e.g. customer recommendations— “customers who bought this book also bought...”—or feedback comments on platforms such as ebay.com or amazon.com) and less in regard to collaborative activities. Erickson & Kellogg [7] call systems that make socially meaningful clues visible socially translucent. So-called social proxies intend to give an abstract, minimalist representation of real-world activities [6]. For example, the communication platform Babble [6] portrays chat partners as coloured dots within a circle, moving to the centre when they converse actively and back to the border when inactive. The audio conferencing system Talking in Circles [26] uses graphical representations of users to show who is participating in conversations: Participants, represented as coloured circles, need to be

188

M. Janneck

within ‘hearing distance’ of a speaker to be able to receive the audio transmission. This way, parallel conversations are possible while making visible who is talking and who is participating in the conversation. Perry & Donath [25] use anthropomorphic representations to display users’ activity in e.g. discussion forums over a longer period of time. These ‘stick-figures’ look bigger and brighter when the users they represent participate actively, and fade as activity ceases. The number of postings is depicted as small boxes in the figures’ ‘bellies’, and their facial expressions give some insight into the emotional tone of the users’ contributions, interpreting emotional keywords in the postings. However, Perry & Donath [25] admit that users might feel their representations to be wrong, misleading, or even caricatured, resulting in efforts to manipulate their ‘stick-figure’ by writing compensatory yet senseless postings. PeopleGarden [3] depicts message board members as flowers, growing and flourishing according to their activity. A message board whose members show low activity will look like a neglected garden with only few and dispersed plants. An active group will be represented as a prosperous garden with a variety of blooming flowers. Social proxies like PeopleGarden [3] or anthropomorphic representations [25] deliberately use emotionally appealing and evocative visualizations of social activities, in contrast to more neutral representations, e.g. [6,26]. However, Donath [3] argues that completely neutral visualizations are never possible anyway, because social activities always evoke judgments on the side of the people involved. Therefore design decisions should explicitly involve knowledge of social processes.

4 Social Awareness Design Guidelines The social psychological concepts presented in section 2 can be used to evaluate existing approaches for visualizing social activities (section 3) and to develop new ideas for designing awareness support in cooperative systems. In the following paragraphs, this is done in the form of eight proposals, four of them related to group structures and relations and another four related to social loafing. Group Structures and Relations 1. Enable Control: Visualizing social activities within a shared workspace is vital for conveying a sense of social structures and relations. However, people need to retain some control over which of their activities are made visible for other members— and how this is done—to keep their autonomy in use. Making transparent for users how their actions are being visualized helps them to prevent that information they want to keep private is made public. 2. No Customization: For the reason of transparency, awareness functions should not be customizable. 3. No Automatic Interpretations: Social activities and emotions should be recognizable, but not be interpreted by the software, for this affects users’ control and self-management in a very sensible area. This is an argument against deducing emotional states from e.g. the text of postings (cf. [25]) or even forecasting future user behaviour (cf. [10]). Automatically interpreting emotions suggests that the


189

software possesses emotional competencies and meta-knowledge concerning the social structure of a group, which cannot be sensibly assumed. Erickson [5] states concisely: “Portray actions, not interpretation”. 4. Do not Judge Actions: For the same reason, judgmental visualizations should be treated carefully despite the liveliness they undoubtedly convey. Emotionally appealing visualizations like a withering garden or a wimpy and pale stick-figure bear the danger of having a discouraging or offending effect on users and hindering rather than promoting further use in the sense of a self-fulfilling prophecy [20]. Furthermore, it is difficult to take the respective context into account: Groups may differ regarding the number and frequency of postings because they have different conventions and motives for use, but still feel a similar quality of interaction. This is also true with respect to individual users: An apparently less active user might simply use more effective strategies when navigating or searching for information, cf. [24]. Social Loafing 5. Make the Invisible Visible: Making ‘passive’ user behaviour—like read access— more visible is important to reflect its significance for collaboration. This is a challenge especially for the design of asynchronous tools, because visualizations of users’ current activities within the workspace—conveying a simple and fast impression of their presence—are not very useful in regard to asynchronous cooperation. On the other hand, detailed, personalized navigational and activity histories tend to take up much time, screen space, and attention, especially in larger groups. Furthermore, detailed activity accounts conflict with privacy and control issues. Instead, to enable users to grasp activities quickly and intuitively, 6. Aggregate Sensible and read Access Information: Instead of giving detailed and personalized records of activity, it is recommendable to visualize potentially sensible and read access information in an aggregated form (e.g., “10 of 35 people logged on last week” or “3 of 6 people viewed this posting”). Abstract or visual representations help to grasp the information without much cognitive load (compared to text-based information). However, the visualization should be non-judgmental and neutral. 7. No Anonymous Action: In contrast to passive participation (read access), active contributions (write access) should be clearly traceable and attributable to the respective authors to encourage personal responsibility and avoid social loafing. This implies that anonymous interaction is not recommendable in CSCL/CSCW tools. 8. Use Notifications Sparsely: Considering the danger of social loafing and taking into account that ‘passive’ behaviour like reading other postings, downloading files, or simply browsing the shared workspace for information are significant activities that should be visible for the group, notifications should be used sparsely. Otherwise, users might develop a reactive rather than an active mode of use: If usage is mainly triggered by the system, users might visit the workspace not because they have a personal motivation to do so, but because they expect an ‘incentive’ (i.e., new information). Furthermore, this weakens and devaluates passive forms of participation, which will typically not cause any notification. It also brings about a

190

M. Janneck

selective perception of group activities, particularly as notifications usually need to be customizable to prevent information overload. This has to be weighed against the convenience of being informed about new postings in a quick and timesaving way.

5 Design and Evaluation of an Enhanced Awareness Support The design guidelines presented in the last section were put to a first and partial test by designing and implementing additional awareness functionalities for an existing groupware system, CommSy, that showed a lack of support of social awareness in empirical evaluations [11,12]. The new awareness functions were evaluated by means of a quasi-experimental design comparing both subjective and objective awareness measures in a field test before and after the new awareness functions had been implemented. In the following sections, the software, the new awareness features, and evaluation results are presented. 5.1 CommSy The software that was used as a basis for the implementation is CommSy, a web-based system to support group work, which is used mainly at universities, schools, but also at the workplace. For a detailed description see e.g. [23,22]. People work together in shared workspaces, so-called project workspaces, providing file sharing, a collaborative editor, a shared calendar, discussion forums, to-do lists, a billboard and other typical groupware functions. CommSy’s design principles emphasize social issues in groupware use, such as negotiation of usage rules and group responsibilities. These principles are reflected in design features such as shared editing rights pertaining to almost all entries in the system, no customization, allowing no anonymous entries, and equal access rights. Furthermore, the CommSy designers object to autonomous and interpretative system functions. Thus, the overall CommSy design actually is in accordance with the key principles of several of the guidelines presented in section 4 (e.g. guidelines 3, 7, 8). Nevertheless, there were no explicit awareness functions built into the software, as the developers quite generally opposed them as system interpretations of user actions. However, empirical investigations with a large user basis (over 1500 people using the system on a regular basis were surveyed over several years [11]) showed that users suffered from a lack of social awareness: Users did not leave any traces within the system unless they actively decided to do so. ‘Passive’ participation like downloading information or reading each other’s contributions was just not visible in the system, even though this was the main use pattern in most contexts of use. As a consequence, especially the more active users experienced frustration because they felt that their contributions were not being perceived at all. Therefore it was decided in conjunction with this study to add minor awareness functions that were in accordance with the guidelines developed in this paper—but also with the main CommSy design principles.


191

5.2 Designing Awareness Functions The design requirements for the awareness functionalities can be summarized as follows, answering the three central questions posed by Gutwin & Greenberg [8]: 1. What Information should be gathered and displayed to the Group? It is necessary to visualize how contributions are received within a group of users working together. This especially applies to read access information, since written contributions are imminently visible (guideline 5). 2. How should this Information be presented? To enable transparency and shared group awareness in accordance with the CommSy design principles, awareness information must be equally accessible for all group members (guidelines 1 and 2). To avoid control over individual participation, there should be no personalized access information (guideline 6). Judgemental visualizations should be avoided, preferring abstract illustrations (guideline 4). 3. When, and for what Activities, are the different kinds of Information important? Since CommSy was developed to support mainly asynchronous cooperation, awareness functions should display past rather than current actions. The group’s interaction rhythm (i.e., how often are members expected to use the system) or important milestones of cooperation (e.g. changes on a document) serve as reference points. Based on these requirements, two visualizations of read access information were designed: An activity display on the workspace home page and detailed access information for every item in the workspace. Activity Display on the Home Page. The workspace home page already provides an overview of the latest group activities. It can be configured to display all items that were posted or changed within an adjustable time span (e.g. during the last seven days). Therefore, a glance on the home page is sufficient to gather the latest information from the group instead of having to browse the whole workspace. In addition, awareness functions conveying an impression of social presence and general level of activity were integrated into the home page. The following information was used (Figure 2): – The number of group members who have logged on within the given time span to show how many people currently participate in the group work, – the read access level (i.e., the number of page impressions) within this time span to give an impression of the activity level within the workspace, which can also be compared over time, – and, contrasting, the write access level (i.e., the number of posts) to show that the (possibly) frustratingly low level of active participation is accompanied by a much higher level of “passive” participation, which is equally vital for cooperation but often remains invisible.

192

M. Janneck

Fig. 2. Activity display on the workspace home page

Detailed Access Information. As evaluation results showed, CommSy users—especially active users—feel uncertain regarding the questions of if and how their postings will be received by other group members and if changes they make will be recognized in time. To address this uncertainty detailed access information was added for each posting, showing the overall number of group members that have viewed this particular entry and also the number of people who visited this page after it has been edited. It is visualized by a bar showing the percentage of project room members who called on this item (Figure 3).

Fig. 3. Detailed access information

A personalized read access history (as it is provided by many groupware systems) was avoided in accordance with guidelines 1 and 6 to prevent misuse as control instrument. 5.3 Evaluation The awareness features described above were evaluated by means of a user questionnaire, measuring social awareness. Results were compared with data gathered before the awareness mechanisms had been implemented to test for an increase of social awareness. 560 users filled out the pre-awareness questionnaire, 460 answered the post-awareness survey. Respondents used CommSy mainly in Higher Education contexts. They reflected on use periods of 3-6 months (one semester, typically). Project workspaces were made up of groups with an average of 25 members. Subjective Assessment of Usefulness. Respondents were asked to rate the usefulness of the new awareness information on four-category Likert scales, with positive results: Approximately 60%, respectively, agreed that the activity display on the home page and also the detailed access information had proven useful.


193

This was especially true for participants regarding themselves as more active than average, with approximately 70% positive evaluations in this user group. To investigate differences between active and passive users, Mann-Whitney-U-tests were calculated for intergroup comparisons. Both users posting a more-than-average amount of entries or considering themselves more active than average rated the awareness mechanisms significantly better than users who behaved more passively (p1. The time weight is applied to the reputation metric in a recursive algorithm (Section 3.1).

Modelling the B2C Marketplace: Evaluation of a Reputation Metric for e-Commerce

217

Transaction Value. In counting reputation ratings the value of the transactions is also taken into account counteracting users who try to build a high reputation by cooperating in many small transactions and then cheat in a very large transaction. Also, the transaction value range depends on the context to which the reputation system will be applied i.e. the maximum price of sold goods/services in the marketplace. The weight associated with the transaction value wvx is calculated using the formula below: 1 x −v(x) where x → 0 (6) wvx = 1 − γ and γ = vM ax Where, v(x) is the value of transaction x and vM ax is the transaction range i.e. the maximum value of the goods/services in the marketplace (based on the context to which the reputation system is applied). γ is used to scale v(x) and γ >1. Number of Malicious Incidents. As in [10], in the proposed metric the reputation value is reduced to the minimum when a party reaches a certain threshold of malicious incidents. Up to that threshold the appropriate weight wm(p) is applied based on the exponential function:

if 0 ≤ m < M then wm(p) = α−m (7) if m ≥ M then wm(p) = 0

where

1 where x → 0 (8) M Where, m is a number of malicious incidents of provider p that occurred within the transactions taken into calculation. M is the set threshold of the number of malicious incidents above which the reputation value is reduced to minimum. In the equation above α is used to scale wm(p) and α > 1. α=

x

Computing Aggregated Rating. The aggregated ratings are calculated with the application of the recursive algorithm used on the list of the transaction data records sorted according to the time value. The aggregated direct rating value is calculated based on the data stored in the requesting agent a database i.e. regarding its direct interactions: wtx wtx−1 AGRDa,x (p) = U Ra,x (p) · + AGRDa,(x−1) · (9) wtx + wtx−1 wtx + wtx−1 For the case where x=0 the aggregated direct rating is equal to the updated rating for that transaction. Where U Ra,j(p) is the updated rating value of transaction j with provider p calculated by agent a and x is the index of the last transaction on the list (n − 1). The aggregated indirect rating values are calculated in the same manner as above but are based on the list of the transaction data from the subset of the ”n best/most suitable users”. In addition, the weight ws is applied for each user providing information.

218

A. Gutowska and A. Sloane

Computing Updated Ratings. Updated reputation rating U Ra,x (p) is calculated by agent a for transaction x in which a was involved with provider p. In general, each provider is reputed by an agent after each transaction by providing a transaction rating g. This is the average of two components: fulfilling provider’s signals and customer service, where both can take values [0, 1]. In addition, appropriate weight wv based on the transaction value is applied. 3.2 Optional Reputation In addition to the parameters presented above, a user may choose to include some or all of the optional parameters into calculations, which will influence the rating value of a provider. They are: existence of trustmark seals, existence of payment intermediaries, existence of first party information, existence of privacy statements, existence of security/privacy strategies, existence of purchase protection/insurance, and existence of alternative dispute resolution and are further described in [7,16]. The optional reputation is based on the set of optional parameters (providers’ characteristics) which take values [0, 1] and is presented by the average of the above parameters which have been chosen to be included into calculation. The rating scale for optional reputation metric is [0, 1]. Optional reputation constitutes the initial reputation for newcomers as at that point there is no information of the past behaviour available.

4 Simulating B2C e-Commerce Reputation System The reputation system simulator used in this study was developed in Java and it is based on a slightly modified version of the RePast agent-based simulation toolkit [22]. In the presented simulation the market is populated by a number of agents that are divided into buyers and providers. The simulation is based on discrete time ticks. At each tick buyer agents are supposed to initiate a transaction with a provider and rate him afterwards. After the agents finished their actions the data is collected and represented graphically. In the simulation the agents may enter or leave the community with equal probability (see Simulation Parameters in Section 4.5). 4.1 Modelling the Buyers The buyers in the simulation framework differ in types. The buyer agent type is a combination of its trust disposition and its expectations. Disposition to trust and the same risk attitude refer to the fact that people have a baseline attitude when they approach any trust situation. Some of them have a higher baseline level of trust than others thus, some individuals may find it easier/more difficult to trust. The disposition to trust affects the decision of either the buyer agent wants to engage in a transaction with the provider or not (see the acceptance function in Section 4.4). Based on the above there are different types of the buyer agents in the simulation:


219

Risk Taking. This type of buyers is willing to take risk easily which means they accept the high value transactions even with the provider with low reputation. Cautious. This type of buyers is risk averse and they are very careful with their decisions. They accept the transactions only if the provider has high reputation. Conservative. In the presented framework the buyer agents have also different expectations towards the outcome of the transaction which affects the way they rate the transaction (see the rating function in Section 4.4). As in [23], there are three types of the buyers agents in this study: optimists, realists, and pessimists. Combining the two attributes discussed above the following types of buyers agents were implemented in the simulation framework: Risk Taking Optimists, Risk Taking Realists, Risk Taking Pessimists, Cautious Optimists, Cautious Realists, Cautious Pessimists, Conservative Optimists, Conservative Realists, and Conservative Pessimists. 4.2 Modelling the Providers The effectiveness of a reputation system and its metric depends on its resistance against malicious behaviours. The success of non-honest agents is its measurement for the quality of the metric [24]. Therefore there are different types of providers implemented in the framework which are called Trustworthy, Shady, Player, and Fly-By-Night. They differ in their behaviour while transacting (this is also correlated with their characteristics). The characteristics of the interest are the cheating probability (ChP ) and the range of the transaction outcomes they produce in terms of customer service and fulfilling providers’ signals (in other words the quality of services they provide). The remaining attributes constitute the optional parameters in the reputation metric and include: existence of trustmark seals, payment intermediaries, privacy statements, security/privacy strategies, purchase protection/insurance, alternative dispute resolutions as well as existence of first party information. They have been chosen based on the previously conducted survey discussed in [7]. The above characteristics/optional parameters can take values between 0 and 1 where 0 means no existence of the attribute. In this way each type of the provider has the optional reputation (OP) value based on the above which constitutes the initial reputation value for any new provider in the system. In the reputation system there would be a devoted agent that would gather the optional parameters information from the providers’ Websites. The properties of different providers are as follows: Trustworthy. This type of the providers does not cheat in the transactions (ChP=0) and provides high service quality. All the parameters mentioned above have high values (OP=0.92). Shady. This agent does not have a particular pattern in its behaviour (ChP=50). It provides false statements on its Website which results in high values of the optional parameters apart from Trustmark Seals and Payment Intermediaries (OP=0.63). The quality of the services it provides is low. Player. This type of a provider tries to build high reputation by not cheating (ChP=0). When it achieves its goal, however, it starts behaving in a malicious way (ChP:=100). When its reputation falls down below the threshold then it starts being honest again

220

A. Gutowska and A. Sloane

(ChP:=0). Player agent has got high values for First Party Information, Privacy Statements and Security Strategies (OP=0.43). When it does not cheat the services provided are of a high quality. Fly-by-Night. This agent’s goal is to cheat (ChP=100). It provides false information about the services it offers. The way of payment is direct to the bank account (OP=0.51). The quality of the services it provides is low. 4.3 The Simulation Cycle The simulation framework is highly automated where the handling of the agents, initiation of the transactions and storing the ratings are part of the framework. The simulator repeatedly iterates a cycle of events that would occur in the marketplace. The steps of a transaction are as follows: 1. The simulation engine selects a buyer agent who initiates a transaction with another provider agent. 2. The buyer agent tests if the transaction is acceptable i.e. he calculates the reputation of the provider in question based on his previous direct interactions as well as information from the buyers community (the acceptance function is described in Section 4.4) 3. If the transaction takes place, the provider agent determines the outcome of it and the buyer agent rates it and stores the ratings. The ratings depend on the buyer agent type and his expectations and may not match exactly the real outcome (the rating function is described in Section 4.4) 4.4 Modelling the Transaction and Rating Processes Transaction Acceptance Function. In the presented simulation the buyer agents have a trust disposition which allows them to make different decisions when it comes to engaging in a transaction with a provider. In this work the assumption is that no buyer agent will transact voluntarily with a non-trustworthy provider i.e. the provider with low reputation. The other factor taken into consideration while making the decision is the value of the transaction. The acceptance function therefore, is a correlation between the provider’s reputation and the value of the transaction. The higher the value of the transaction the higher the reputation should be for the buyer to engage in this transaction. As different people have different disposition to trust, in the presented framework different types of buyer agents have different acceptance functions. In this way different types of agents accept the transaction of a specific value at the different reputation level. Users’ willingness to trust, however, can be changed by experience [25]. In the proposed framework all buyer agents representing a specific type start with the same acceptance function which is affected/changed later on by the outcome of the transaction (experience) and in particular by the providers’ malicious incidents. The calculation of the acceptance threshold for a specific transaction value with a specific provider is based on the Lagrange Interpolation [26].


221

The Rating Function. In the proposed framework each buyer agent rates each transaction he has been involved in and collects these ratings (see Transaction ratings Section 3.1) in his database. In a real marketplace, different people will rate a transaction differently based on their experience and their expectations towards the transaction outcome. In the discussed simulation framework, three cases are considered (as in [23]): optimists, realists, and pessimists. When it comes to the transaction, optimists will be expecting a very positive outcome, pessimists on the other hand a rather bad outcome, and realists will come somewhere between the two extremes. The simulation framework addresses the above scenario in a way that the optimist agent will hope for the best outcome (in terms of customer service and provider’s signals) he has had so far with the provider in question, the pessimist agent will anticipate the worst one, and the realist agent will expect the average result based on his experience. If the expected outcome (expOut) is higher than the actual one (realOut), the buyer agent applies the punishment value (p) to the transaction rating (rating) which is a difference between the expected and the real outcome value. If the expected outcome value is equal or lower than the actual one, the ratings reflect the outcome. The above rules are presented below: p:= expOut - realOut if p>0 then if p0, p is an object-relevant page, otherwise, it is a noise page. Since the results of the path-query might contain not only object profile pages but also object list pages, the content-query is designed for filtering out the object list pages. It utilizes the unique ontological property names of the object type as keywords for further filtering. E.g.,, for describing a product, the unique property names include such as “function”, “feature”, and “price”. The content-query is conducted on the results of the path-query. Since the property description appears only in the object entry or detailed-profile pages, the object list pages could be removed by the content-query. We need to mention here is that since the content-query depends on the pre-defined knowledge on the object, it is optional for the given solution in this paper. 3.2 Web Page Clustering After the object relevant web pages P={p1, p2, …, pn} is obtained, this section describes a clustering technique to separate P into multiple groups. The basic idea is to adopt certain hyperlink analysis to find the HRs between web pages, and then, naturally, each web page set linked together forms a cluster. The above identified HLs can be utilized directly to construct the hierarchy of the object relevant pages. However, its underlying assumption depends too much on the existence of the link collection, and also the result’s accuracy is not good enough for object identification. In addition to utilizing the URL for HR identification, we adopt a more general assumption here for discovering the HR between web pages: when the page author utilizes a hyperlink for reference to an object, he/she prefers to select the entry page of corresponding object as the destination of the hyperlink. So, for a detailed-profile page, its in-bound hyperlinks usually come from the object entry page or other sibling pages. Since all the detailed-profile pages are contained in the out-bound page set of the object entry page, the in-bound page set of the detailed-profile pages

248

J. Li and Y. Zhao

should be the subset of the out-bound page set of the object entry page. Thus, a basic rule for identifying the HR between two pages is: if there exists a hyperlink from page pi to page pj, and at the same time, pi’s outbound page set contains pj’s inbound page set, we regard pi as an ancestor page of pj. For simplicity, pi itself is also included into its outbound page set virtually. Figure 4 is an example of using this rule.

Fig. 4. Example of HR identification L:=H; //initialize L by hyperlink matrix H lij:=1 for i=j; //add virtual hyperlinks from pages to themselves to include page itself into its outbound page set R:=0; R':=D; //R represents the result of the previous iteration, R' represents the result of the current iteration DO WHILE (R!=R') //iteration terminates when R becomes convergent { R:=R'; R':=D; // initialize R' by directory matrix D //adjust L by eliminating descendant-to-ancestor links and complementing ancestor-to-descendant (virtual) links by the resultant R of the previous iteration lij:=(~rji)& (rij|lij) for izj; FOR EACH (pi, pj), izj // if pi’s outbound page set contains pj’s inbound page set, pi is an ancestor of pj {

d ji rij':=1;

IF

z 1 AND lij==1 AND (FOR EACH 1 d k d n

HAVING lki

t l jk

)

} FOR EACH (pi, pj), izj //if two pages are ancestors of each other, remove their relation { IF rij'==1 AND rji'==1 rij':=0; rji':=0; } FOR EACH (pi, pj, pk), izjzk // derive the transitive closure of the hierarchical relation { }

IF rij'==1 AND rjk'==1 rik':=1;

}

Fig. 5. The algorithm for finding the HRs web pages

Let P={p1, p2, …, pn} denotes the resultant page set of object-relevant web page filtering, where n is the amount of object-relevant pages. H is an n×n matrix. It represents the hyperlink relationships among pages of P, where the value of hij is 1 if there exists a hyperlink from pi to pj, and 0 otherwise. The n×n matrix D represents the HRships acquired directly from syntactic URL analysis, among pages of P, where dij equals to 1 if pi is at a higher level than pj in the directory structure implied in the URL, and 0 otherwise. Thus, an iterative algorithm for finding the HRs among the web pages in P is proposed, as shown in Figure 5, where P, H and D are the inputs,

Website-Level Data Extraction

249

and the output, i.e., the resultant HRships among pages of P, are represented by n×n matrix R, where, rij=1 denotes that pi is an ancestor of pj, and rij=0 denotes there is no HR between them. The essence of the algorithm is to construct R from H and D, where the syntactical URL analysis result is prior to that of semantic hyperlink relation. L is a virtual hyperlink matrix. Its initial value is set as H. For each cycle, L is updated by eliminating the descendant-to-ancestor hyperlink relation and complementing the ancestor-todescendant hyperlink relation (i.e., virtual hyperlink). The termination condition of the iteration is that R converges. It can be proved that, from the second iteration, the amount of page pairs with lij=1 decreases monotonically, and then the iteration is convergent. After the HRs between pages in P={p1, p2, …, pn} are extracted, a graph of objectrelevant pages could be generated by considering a page as a node and a HR between pages as an edge. Then the pages are naturally separated into multiple groups. 3.3 Result Refinement Considering that both the web page filtering and clustering might raise some errors, which will be propagated to final object identification, we design a coordinated approach to refine the result of the two steps. The underlying idea is that the destination pages of the hyperlinks from the same link collection should be siblings each other. It means that if the destination pages of some hyperlinks in a link collection are judged as object relevant pages, the destination pages of other hyperlinks within the same link collection should also be object relevant. Figure 6 shows an example. Within the hierarchical structure from clustering, the page p has the children p1-p4. p also has a link collection including the hyperlinks directed to not only p1-p4, but also two other pages p5, p6. The link collection could imply that p5 and p6 should be sibling of p1-p4. Then, we coordinate the result by adding p5 and p6 as p’s children.

Fig. 6. Coordinating R with Link Collections

250

J. Li and Y. Zhao

3.4 Object Identification This section will identify each distinct object and its profile pages from those refined clusters. Since the entry page and the detailed-profile pages of each object are linked together with certain HRs, we can assume that all the profile pages of a specific object are completely contained within one cluster. Then, to find all the objects and their profile pages, we only need to consider each cluster one by one.

Fig. 7. A schematic diagram on the object identification

Depending on if the content-query is conducted, we design two policies for the object identification from each cluster: Top-down policy: If the content-query has been done for web page filtering, i.e., the object list pages have been removed, we can consider the page without parent in a cluster as the entry page. Then, all its descendant pages are naturally identified as the detailed-profile pages of corresponding object. Bottom-up policy: The bottom-up policy is employed for the case that the contentquery is not conducted (i.e., the pages without parent in a cluster may refer to object list pages). The HRs between web pages determine that the details of the object are described in the lowest level pages of each cluster. Since the website builders generally use only one or two levels of pages for describing the objects, the bottom-up policy only investigates the pages at the two levels from the bottom of each cluster. Considering that an object entry page is the exclusively parent of other detailed-profile pages, and seldom two objects share one detailed-profile page, the bottom-up policy is embodied as two rules: 1) If a page at the lowest level has more than one parent, this page is an object entry page, which means its profile is described by only one page; 2) For a page at the second level from the bottom of the cluster, if all of its direct children have only one parent, this page is an object entry page and its children are the detailed-profile pages. Figure 7 shows the application of two policies for object identification. The left part is the bottom-up policy applied on the clustering result of the object relevant web pages from only the path-query. The right part is the top-down policy conducted on


251

the clustering result of the object relevant pages from both the path- and contentquery. We can see that, no matter the content-query is conducted or not (which cause that the intermediate object relevant web page collection and clustering result are different), the final sets of the identified objects are the same. Our empirical observation shows that it is a typical scenario for website-level object identification, specifically the small granularity objects (e.g., products and people). However, for the object with large granularity, the profile pages may span more than two levels. It would make the results from the two policies different. In this case, content-query with the predefined knowledge of describing these objects is preferred.

4 Evaluation 4.1 Task and Data Set Gathering the published product information in company websites can produce many value-added applications. A task to find out all the products and their descriptions in company websites is considered. Table 1. Dataset and ground truth data Websites en cn jp

15 15 15

Total pages 0.5M 0.4M 0.3M

Products ( entry pages) 1312 1119 632

Product detailed profile pages 2753 1692 855

To set up a well-controlled experiment, a group of students are hired to help us select the dominate companies in the IT domain. We use three languages, i.e., English, Chinese, and Japanese, to verify if the approach is language dependent. For each language, 15 websites are considered. The representative companies include Symantec, McAfee, Cisco, Huawei, etc. Some global companies are picked in duplicate for different languages, e.g., the Symantec websites in three languages are selected respectively for English, Chinese and Japanese experiment. As shown in Table 1, we identify all the distinct products and their profile pages from these websites manually as ground truth data. 4.2 Evaluation Criteria The Recall and Precision are adopted here. For product-relevant web page filtering, the precision is defined as the proportion of actual product relevant pages among all web pages returned by the filtering approach. The recall is the proportion of the correctly identified product relevant web pages by the approach among all the product relevant web pages in the website. For product identification, the qualities of the identified entry page and the detailed-profile pages are evaluated respectively. The recall/precision of the entry page identification reflects how well the products are discovered in the website; the recall/precision of the detailed-profile page identification reflects how well the detailedprofile pages are discovered for each specific product.

252

J. Li and Y. Zhao

4.3 Experiment Results We apply our approach for product extraction on these company websites. During the product-relevant page filtering stage, we conducted two experiments for the pathquery, one by using whitelist keywords only, and another by using both whitelist and blacklist keywords. Then, the content-query is adopted to refine the web page filtering results from two kinds of path-query, respectively. Since our target is product mining, we define a product-specific keyword list for product-relevant page filtering. The predefined values are set as shown in Table 2. Subsequently, the steps of hierarchical web page clustering and results refinement are operated under two situations, with only path-query results and with both path- and content-queries results. Finally, the product identification is conducted. Table 3 is the evaluation results of average recall/precision for the 45 test websites. Basically, the figures verify the usability of our proposed approach. The effective results of the product entry page and detailed profile page identification demonstrate indirectly that the performance of the given web page clustering method is promising. Table 2. Parameter setting for experiments Filtering Phase: Path-query Keyword Whitelist en product cn , jp

产品商品製品, 商品

Filtering Phase: Path-query Keyword Blacklist en news, about us, bbs, forum, story, event, job, career, sitemap, service cn , , , , , , , jp , , , , , ,

新闻关于我们论坛市场活动工作机会招聘网站地图服务連絡ニュース会社情報サイトマップフォーラム伝言会社案内,サービス

Filtering Phase: Content-query Ontological Keywords en cn jp

function, specification, feature, benefit, advantage, performance

功能,特性,特点,优点,优势,特色,亮点,规格,性能機能,特徴,特長,メリット,長所,仕様,スペック, 利点

Moreover, the figures in Table 3 show that the performance of our proposed algorithm is language independent. And the positive effect of considering blacklist in path-query is a bit, while considering content-query can increase accuracy notably. Considering the incorrect results, one main reason is that some product entry pages are identified as product list pages, and then the actual product entry page is considered mistakenly as the detailed profile pages. For 11 such websites without product list page, the average recall/precision of product entry pages is 78.2/81.6. However, for the websites with product list pages, the average recall/precision of product entry pages is 80.8/86.3. Since the experiment of the later step is conducted based on the result of earlier steps, the errors are propagated from earlier steps to the later steps. So some errors in product identification is inherited from the errors happened in the steps of product relevant web page filtering and web page clustering. Additionally, the identified HL for HNP extraction in web page filtering phase could also be used for finding HRs between object relevant pages, so we can use such HRs directly for clustering (called rough clustering) and product identification.


253

Table 3. Experiment results Product identification with refinement (entry page / detailed profile pages) W WB WO WBO W WB WO WBO W WB WO WBO en Rec 95.2 93.7 85.2 83.9 76.5/79.3 77.3/79.7 80.2/81.1 78.3/82.5 78.2/82.7 79.9/83.1 80.5/85.7 82.5/86.3 Prec 79.4 81.2 82.5 89.9 72.6/80.9 75.2/81.3 81.1/82.7 83.1/83.2 74.2/83.2 75.9/85.7 81.9/87.0 87.3/88.5 cn Rec 91.4 90.9 83.1 79.5 74.3/78.8 72.9/80.2 75.6/80.3 73.2/84.8 76.1/80.2 76.3/81.5 76.9/83.4 77.3/85.3 Prec 75.1 78.3 80.0 88.8 71.1/77.5 75.4/77.9 79.9/81.2 81.0/81.9 75.9/78.0 75.8/79.1 80.4/81.6 82.1/83.6 jp Rec 94.6 93.7 86.3 81.5 79.4/76.7 74.2/77.3 75.0/79.2 75.5/80.6 80.1/77.2 75.3/79.5 79.7/85.0 81.1/90.5 Prec 80.2 81.5 81.3 89.2 72.9/79.1 75.8/79.7 82.5/82.3 84.8/83.5 73.2/80.3 77.0/82.1 85.7/82.8 86.8/84.0 Ave Rec 93.7 92.8 84.9 81.6 76.7/78.3 74.8/79.1 76.9/80.2 75.7/82.6 78.1/80.0 77.2/81.4 79.0/84.7 80.3/87.4 Prec 78.2 80.3 81.3 89.3 72.2/79.2 75.5/79.6 81.2/82.1 83.0/82.9 74.4/80.5 76.2/82.3 82.7/83.8 85.4/85.4 W: filtering with white wordlist of path query; B: filtering with black wordlist of path-query; O: filtering with ontological wordlist of content-query Object relevant web page filtering

Product identification without refinement (entry page / detailed profile pages)

Fig. 8. Comparison of product identification results

The corresponding experiments are conducted to verify the effects of the adopted new web page clustering method in Section 3.2. Figure 8 is the comparison of the final product identification results with two web page clustering methods. We can see that the performance from the proposed clustering algorithm is enhanced significantly comparing with the results from the identified HLs. Averagely, the refinement step can improve performance about 3-8% with respect to both the recall and precision.

5 Discussion With the above object-relevant data extraction algorithms, we can obtain two kinds of results from a website: (1) distinct objects and their entry pages are identified; (2) the object-relevant pages and the hierarchical relations between them are also identified. If we investigate the hierarchical relations in a bottom-up way, we can observe that the parent page of the entry page of an object usually represents the category of corresponding objects; the parent page of this category page then represents an super-category of this category; and so forth. Thus, we might extract an object taxonomical hierarchy which is embedded in the website as an additional output of our system other than the above mentioned two. Particularly, if the website for extraction is a dominated website of the interested object domain, apparently the resultant taxonomical hierarchy could be used for building ontology of this domain. For example, after product extraction from the website of NEC, a dominate manufacturer of IT products, with the identified product

254

J. Li and Y. Zhao

Fig. 9. Example of NEC website (partial)

entry pages and hierarchical relations between pages (as shown in upper part of Fig. 9), a corresponding product taxonomy could be obtained (as shown in lower part of Fig. 9). Based on the above observation, we did a simple experiment to check the quality of the resultant object taxonomical hierarchy with this method. Based on the data set used in the Section 4, after product is extracted, along hierarchal relations between pages with bottom-up fashion, we regard each ancestor page of product entry pages as representing a product category, and regard each parentchild page relation as a taxonomical relation between product categories (or a category-instance relation between a product category and a product). We get the evaluation result that the precision of product-category/taxonomical relation is 60.2%/73.4%, 61.7%/71.5%, 62.4%/74.0%, 61.4%/73.0% for English, Chinese, Japanese sites and average value, respectively. Such results show that this approach for obtaining object taxonomical hierarchy from websites is feasible.

6 Conclusions Existing solutions for web data extraction assume each given page includes several data records. It makes them inapplicable for website-level data extraction, where the relevant information of an object is distributed sparsely in the website. This paper proposes a novel approach to address this problem. It exploits HLs for web page organization in websites as a novel resource for not only the object relevant web page finding but also the object-centric page clustering. The experiment results verify the usability of the proposed approach. Additionally, further discussion and experiment


255

shows that our method is also feasible for object taxonomy extraction from websites. The major limitation is that the object-relevant keywords need to be set manually. How to collect the keywords (semi-)automatically is our future work.

References 1. Laender, A., da Silva, A., Ribeiro-Neto, B., Teixeira, J.: A Brief Survey of Web Data Extraction Tools. SIGMOD Record (2002) 2. Arocena, G.O., Mendelzon, A.O.: WebOQL: Restructuring documents, data-bases, and webs. In: Proc. of ICDE (1998) 3. Arasu, A., Garcia-Molina, H.: Extracting Structured Data from Web Pages. In: SIGMOD 2003 (2003) 4. Liu, B., Grossman, R., Zhai, Y.: Mining data records in Web pages. In: Proc. of the ACM SIGKDD (2003) 5. Chang, C., Lui, S.: IEPAD: Information extraction based on pattern discovery. In: Proc. of WWW (2001) 6. Cohen, W., Hurst, M., Jensen, L.: A flexible learning system for wrapping tables and lists in HTML documents. In: Proc. of WWW (2002) 7. Cai, D., Yu, S., Wen, J.-R., Ma, W.-Y.: VIPS: a vision-based page segmentation algorithm. Microsoft Technical Report (MSR-TR-2003-79) (2003) 8. Hammer, J., Mchvoh, J., Garcia-Molina, H.: Semistructured data: The TSIMMIS experience. In: Proc. of the First East-European Symposium on Advances in Databases and Information Systems (1997) 9. Davulcu, H., Vadrevu, S., Nagarajan, S., Gelgi, F.: METEOR: metadata and instance extraction from object referral lists on the web. In: Proc. of WWW (2005) 10. Zhu, H., Raghavan, S., Vaithyanathan, S.: Alexander Löser: Navigating the intranet with high precision. In: Proc. WWW (2007) 11. Kao, H.-Y., Lin, S.-H.: Mining web informative structures and content based on entropy analysis. IEEE Trans. on Knowledge and Data Engineering (2004) 12. Zhu, J., Nie, Z., Wen, J.-R., Zhang, B., Ma, W.-Y.: Simultaneous record detection and attribute labeling in web data extraction. In: Proc. of KDD (2006) 13. Park, J., Barbosa, D.: Adaptive record extraction from web pages. In: Proc. WWW (2007) 14. Tajima, K., Mizuuchi, Y., Kitagawa, M., Tanaka, K.: Cut as a querying unit for WWW, Netnews, and E-mail. In: Proc. of ACM Hypertext (1998) 15. Kevin, S., McCurley, A.T.: Mining and Knowledge Discovery from the Web. In: ISPAN (2004) 16. Kushmerick, N.: Wrapper induction: efficiency and expressiveness. In: AI (2000) 17. Muslea, I., Minton, S., Knoblock, C.: Hierarchical wrapper induction for semi-structured information sources Autonomous Agents and Multi-Agent Sys. (2001) 18. Baeza-Yates, R., Ribeiro-Neto, B.: Modern Information Retrieval. Addison-Wesley, Reading (1999) 19. Wong, T.-L., Lam, W.: Adapting Web information extraction knowledge via mining siteinvariant and site-dependent features. ACM Trans. Internet Techn. (2007) 20. Crescenzi, V., Mecca, G., Merialdo, P.: Roadrunner: Towards Automatic Data Extraction from Large Web Sites. In: Proc. VLDB (2001) 21. Li, W.S., Ayan, N.F., Takano, H., Shimamura, H.: Constructing multi-granular and topicfocused web site maps. In: Proc. of WWW (2001) 22. Li, W., Candan, V.K.Q., Agrawal, D.: Retrieving and Organizing Web Pages by Information Unit. In: Proc. of WWW (2001) 23. Nie, Z., Ma, Y.J., Ma, W.-Y.: Web Object Retrieval. In: Proc. of WWW (2001) 24. Zhai, Y.H., Liu, B.: Structured data extraction from the Web based on partial tree alignment. IEEE Trans. on Knowledge and Data Engineering (2006) [

Anti-folksonomical Recommender System for Social Bookmarking Service Akira Sasaki1 , Takamichi Miyata1 , Yasuhiro Inazumi2 , Aki Kobayashi3 , and Yoshinori Sakai1 1

Tokyo Institute of Technology, 2-12-1-S3-67, Ookayama, Meguro-ku, Tokyo, Japan {brnw,miyata,ys}@net.ss.titech.ac.jp 2 University of Toyama, 3190, Gofuku, Toyama-shi, Toyama, Japan [email protected] 3 Kogakuin University, 1-24-2 Nishishinjuku, Shinjuku-ku, Tokyo, Japan [email protected]

Abstract. Social bookmarking has been in the spotlight recently. Social bookmarking allows users to add several keywords called tags to items they bookmarked. Many previous works on social bookmarking using actual words for tags, called folksonomy, have come out. However, essential information of tags is in the classification of items by tags. Based on this assumption, we propose an anti-folksonomical recommendation system for calculating similarities between groups of items classified according to tags. In addition, we use hypothesis testing to improve these similarities based on statistical reliability. The experimental results show that our proposed system provides an appropriate recommendation result even if users tagged with different keywords. Keywords: Social bookmarking, Recommendation system, Collaborative filtering.

1 Introduction Recommendation systems have been widely researched [8], [7], [5], [3]. Most of them are based on ‘collaborative filtering’, which is a method for predicting a specific user’s preferences for new items using preferences obtained from many other users [1], [11], [13]. Generally speaking, collaborative filtering is defined as a method for estimating preferences for items that users have not yet found by comparing preferences of items that they have already browsed, not by using item context. We use the term ‘collaborative filtering’ in a narrow sense as a method for calculating preferences as the rating of each unknown item against preferences of previously viewed items. The algorithm is as follows: 1. Collect preferences of viewed items. 2. Calculate similarities between the focus user and others based on their preferences of commonly viewed items. 3. Calculate preference of each unknown item based on similarities between the focus user and others who have already viewed it. J. Cordeiro and J. Filipe (Eds.): WEBIST 2009, LNBIP 45, pp. 256–269, 2010. c Springer-Verlag Berlin Heidelberg 2010

Anti-folksonomical Recommender System for Social Bookmarking Service

257

As Sarwar et al. acutely pointed out, more items than users leads to poor recommendation results because of the sparsity of preference data [13]. As other recommendation systems, novel web services called social bookmarking (SBM) have appeared in recent years. SBM allows users to annotate each item with one or more keywords called ‘tags’. Niwa et al. investigated the recommendation system by using tags created by SBM users [10]. They aggregated similar tags to reduce word redundancy and made tag clusters with keywords having the same meaning. However, a keyword may have various meanings depending on the context. Golder et al. pointed out these types of tags as being polysemous [2]. For example, ‘apple’ has multiple meanings: a sweet red fruit or the consumer electronics company. If the recommendation system includes these types of tags in only one tag cluster, it cannot recommend items to users who use the tag in other contexts. Due to this problem, a vocabulary-based recommendation system may lead to inappropriate results. Therefore, we did not focus on the vocabulary of tags and instead propose tag-based collaborative filtering. In addition, we define the novel similarity between item clusters based on hypothesis testing. The rest of this paper is organized as follows. In section 2, we introduce a related study of ours. Next, we explain our recommendation system algorithm in section 3. In section 4, we evaluate and discuss our recommendation system, and we conclude in section 5.

2 Related Study In this section, we introduce a related study of conventional recommendation systems using an SBM service and based on the co-occurrence of items. 2.1 Social Bookmarking (SBM) SBM services enable users to store, organize, search, manage, and share bookmarks of web pages on the Internet. SBM has a specific feature called ‘tags’. Tags are keywords created by each SBM user for categorization of web pages. In SBM services, users can share their bookmarks with other users and also browse other users’ bookmarks. For example, users can browse a list of SBM users who bookmarked the same item as they did and also browse a list of items that are tagged with the same keyword. ‘Folksonomy’, from ‘folks’ and ‘taxonomy’, is a method of collaboratively managing tags to categorize items DFolksonomy describes the bottom-up classification systems that emerge from social tagging1. Compared to conventional ‘taxonomy’, Shirky argues that folksonomy has an advantage in flexibility for changing the consensus of how objects should be classified because tagging systems do not use a controlled vocabulary2. 1

2

A. Mathes, Folksonomies – Cooperative Classification and Communication Through Shared Metadata, http://www.adammathes.com/academic/computer-mediated-communication/folksonomies.html C. Shirky, Ontology is Overrated: Categories, Links, and Tags, http://www. shirky.com/writings/ontology overrated.html

A. Sasaki et al.

Frequency of tag name

258

Rank of tag name by frequency

Fig. 1. Tag name frequencies in descending order

2.2 Conventional Study on SBM Over the past few years, a considerable number of studies have been conducted on SBM [2] [9] [14] [4] [6]. Niwa et al. proposed a web page recommendation system based on folksonomy [10]. This system is used for calculating the ‘affinity level’, which is a scalar value representing the relationship between users and tag clusters (sets of tags for reducing word redundancy). In other words, it recommends items based on the similarities in users’ tag records. However, there is empirical evidence against this idea of recommendation. A histogram of tags attached to a particular item from an actual SBM service3 is shown in Fig. 1. We sorted the tags by unique vocabulary in descending order. The tag name frequencies clearly exhibit long-tail behavior. These are two types of words: top-ranked words (a few words used by many people) and bottom-ranked words (many words used by few people). The important point to note is that the latter situation causes word redundancy. Let us look more closely at the bottom-ranked words. We can find unusual tags like ‘toread’ (sticky note for future reference. Golder et al. also pointed it out as ‘Task Organizing’[2]), ‘web/app’ (users’ own directory hierarchy), and so on. That is, in an actual SBM service, most users’ tags are not for others’ convenience but for themselves to manage their own bookmarks for their future use. Let us now return to Niwa et al.’s study. The recommendation system of using tag names is implicitly based on the collaboration of SBM users. The system then discards the bottom-ranked words, but retains a large percentage of the total tags in the practical SBM service. We will discuss SBM tagging in detail. An example of the relationship between items and keywords is shown in Fig. 2. The word redundancy is clear. Users use the words ‘robots’ and ‘AI’ for tagging the same concept of ‘machines that can think’. In addition, there is another word redundancy. Users use the same word, ‘robot’, for tagging two different concepts, ‘machines that can think’ and ‘automated systems’. A recommendation 3

delicious, http://delicious.com/


259

Keyword space

Item space Fig. 2. Relationship between items and keywords

Table 1. Comparison between proposed method and conventional methods Conventional Conventional Proposed [10] [12] method Focused on tag user and category user and tag tag (records) item item Co-occurrcence of Rating yes no yes co-occurrence ratio not defined likelihood ratio Similarity

system using only the co-occurrence of words would never recommend items in these cases. In contrast, our proposed system works even if users tag with different keywords because it does not pay attention to the vocabulary of the tags. Instead, it is rather similar to systems based on co-occurrence of items. 2.3 Conventional System Based on Co-occurrence of Items Rucker and Polanco developed a system for recommending items by calculating similarities between folders (categories by user) of bookmarks [12]. Their system is similar to our proposed system from the viewpoint of only using sets of items. However, their system does not rank each recommended item. A comparison between our proposed system and conventional systems discussed previously is shown in Table 1. As can be seen, our system calculates similarities between item clusters by using hypothesis testing for finding similar ones. Furthermore, our system calculates the recommendation rate; therefore, it can rank each item.

260

A. Sasaki et al.

focused

query

focused focused

query

Fig. 3. SBM modeling regarding relationship among items, users, and tags

3 Proposed System 3.1 Recommendation Based on SBM by Using “Item Cluster” In this paper, we focus on ‘item clusters’, which are sets of items classified by the tags used by each user. Each user has the same number of item clusters as the number of tags he/she uses in the SBM service. When a user issues a query by selecting a tag from his/her tag records, the system searches for items to recommend by focusing on the similarities between ‘query item cluster’ corresponding to the query and ‘recommender item clusters’ corresponding to other tags in the scope of commonly bookmarked items. 3.2 Model of Item Cluster We focused on a particular tag tquery that is tagged by user uf ocused . Bs refers to all the items bookmarked by uf ocused , and Ts refers to all the items tagged tquery by uf ocused . All of the items A (bookmarked by all users in the SBM service) can be classified into three sets exclusively, as shown in Fig. 3. 1. Bookmarked by uf ocused , and tagged with tquery (Ts ) 2. Bookmarked by uf ocused , but tagged without tquery (Bs ∩ Ts ) 3. Not bookmarked by uf ocused (Bs ) We define a set of items tagged with a certain tag name, like Ts as an ‘item cluster’. Let us consider two item clusters, ‘query item cluster Ts ’ and ‘recommender item cluster To ’. Ts is an item cluster tagged with tquery by uf ocused , and To is tagged with tj by ui (note that ui is not uf ocused ). We studied the conceptual similarity between Ts and To . Here, k is the number of items that are included in both Ts and To . ms is the number of items in Ts that ui tags with a different tag name from tj . mo is the number of items in To that uf ocused tags with a different tag name from tquery . Let m = ms + mo and n = m + k. The relationship between k and n shows a conceptual similarity of the two item clusters. n and k are shown in Fig. 4 and described as n = |(Bs ∩ Bo ) ∩ (Ts ∪ To )|

(1)


k

tquery

n

261

tj

focused

Fig. 4. Item recommendation by comparing item clusters

k = |Ts ∩ To |.

(2)

Next, we look at the expected similarity of the two item clusters. Here, we assume that there are only two relationships between item clusters — similar viewpoint and different viewpoint. If two users tag items from similar viewpoints, the expected probability that both users tag the same item is assumed to be p1 . Otherwise, the expected probability is assumed to be p0 . Here, p1 > p0 . p1 and p0 can be estimated by observing all item clusters to separate them into similar and different viewpoints. Desirable items should be recommended from similar item clusters. The conceptual similarity between Ts and To is defined by 1) sim(Ts , To ) = log L(n,k,p L(n,k,p0 ) p1 1 (3) = klog p + (n − k)log 1−p 1−p , 0

0

where L(n, k, p) =

n Ck p

k

(1 − p)n−k .

(4)

The log likelihood ratio of whether the similarity of two clusters is likely to be p1 or p0 is shown in Eq. 3. Here we assume the relationship between k and n follows a binomial distribution (Eq. 4) with parameters p1 and po for similar and different viewpoints, respectively. Finally, we define the recommendation rate of each item by using similarities between the item clusters. The system selects one item i as the candidate to be recommended from Bs (i ∈ Bs ). We define i’s recommendation rate by calculating the sum of similarities between the query item cluster and each recommender item cluster that contains i. R(Ts , i) = χ(To , i)sim(Ts , To ) (5) χ(To , i) =

To ∈Tall

1 if i ∈ To ∧ sim(Ts , To ) ≥ 0 0 otherwise

where Tall is the set of all item clusters.

(6)

262

A. Sasaki et al.

1

Ta

i1 i2

Tb

2

3

䃦(T Tb , i 1 )= 1

5 4

sim(T a , Tb )= 5 Ta

䃦(T Tb , i2 )= 1

5

䃦(T Tc , i 1 )= 0

0

䃦(T Tc , i 2 )= 1

1

i1 i2

Tc

sim(T a , Tc )= 1

R( T a,

)=5 2nd. R( T a, i 2 ) = 6 1st. i1

Fig. 5. Procedure of the proposed system

3.3 Procedure of the Proposed System The recommendation algorithm is as follows: uf ocused issues a query by selecting tag name ‘tquery ’ from his/her tag records. we define it as Ts . 1. 2. 3. 4.

Calculate each To of sim(Ts , To ) as Eq. 3. Calculate each χ(To , i) as Eq. 6. Calculate each i of R(Ts , i) as Eq. 5. Sort items according to recommendation rate and recommend i whose R(Ts , i) is top threc .

Here, threc is the number of items to be recommended. Fig. 5 is an example of the procedure. Ta , Tb and Tc are item clusters. Now we consider a recommendation for Ta . That is, we define Ta as query item cluster (Ts ) and the others as recommender item clusters (To ). i1 and i2 are the items to be recommended. That is, the user who makes Ta has not bookmark these items. First, the system calculate similarity between query item cluster Ta and each recommender item clusters Tb , Tc . Next, it checks every χ. For example, i1 is included in Tb so χ(Tb , i1 ) = 1. On the other hand, Tc does not include i1 so χ(Tc , i2 ) = 0. Finally, it calculates each recommendation rate R(Ta , i1 ), R(Ta , i1 ) based on sum of products which are computed by multiplying similarity by χ. Thus, R(Ta , i1 ) = sim(Ta , Tb )χ(Tb , i1 ) + sim(Ta , Tc )χ(Tc , i1 ).

4 Experiments We performed three experiments using live data obtained from del.icio.us, which is one of the most famous SBM service sites. In these experiments, we set p0 = 0.1, p1 = 0.6. 4.1 Performance Evaluation We randomly collected data of 1,000 SBM users in August, 2006. They had bookmarked about 310,000 unique items (URLs) and had tagged items with about 260 keywords on average. Therefore, we collected about 260,000 sets of item clusters. We used all of these item clusters for the experiments.


263

Table 2. Details of item clusters obtained from experimental results Cluster 1 Cluster 2 Cluster 3 uf ocused user316 user87 user796 2006 1632 2242 |Bs | tquery web randomlink music 486 178 52 |Ts |

Cluster 4 Cluster 5 Cluster 6 Cluster 7 Cluster 8 user878 user555 user313 user190 user51 10078 17965 4782 3925 2701 javascript art History Shopping funny 424 448 209 139 147

Evaluation Method. We used the collected data for calculating similarities and for evaluating our system. We masked tag information, that is, we hid whether all items included in Bs were tagged or not and revealed them after a recommendation was made. The evaluation method was as follows. 1. Select Ts from the collected data. We defined the items included in Ts as correct class X. 2. Calculate recommendation rate of each item corresponding to Bs , and recommend items from top to threc -th. We defined these items as recommended class R. 3. Count the number of items |X|, |R|, and |R ∩ X|, then calculate the recall and precision. Recall and precision were defined as follows. recall =

|R ∩ X| |X|

precision =

|R ∩ X| |R|

(7) (8)

In addition, we used F-measure, defined as follows. F-measure =

2 · recall · precision . recall + precision

(9)

In this evaluation, we focused on the top 100 query item clusters, which were ranked by the number of |Ts |. The biggest |Bs | was 17,960, and the smallest was 865. The average was 6,758.4. The biggest |Ts | in the query item clusters was 1,991, and the smallest was 477. The average was 729.28. Note that we omitted isolated items that had been tagged only by Ts to evaluate net performance. Experimental Results. The results of the evaluation are shown in Figs. 6 and 7. The averages of recall, precision, and F-measure for each query cluster are shown in Fig. 6, and the relationship between recall and precision for item clusters 1 to 8 (Table 2) is shown in Fig. 7. Discussion. In Fig. 6, we can see the precision was 0.78 when our proposed system recommended the top 100 items ranked by their similarities. The precision decreased to 0.67 for 200 items. The F-measure, which is an important measure for recommendation,

264

A. Sasaki et al.

R

R

P

N

Precision

Fig. 6. Recall, precision, and F-measure for each number of recommended items

Cluster 1 Cluster 2 Cluster 3 Cluster 4 Cluster 5 Cluster 6 Cluster 7 Cluster 8

Recall Fig. 7. Relationship between recall and precision for each item cluster

peaked at about the top 400. Judging from the above, we may say that our proposed system is useful for at least the top 400 items, which is enough for a recommendation system. The coefficient of correlation between the number of items in the item clusters and the maximum of the F-measure is -0.34. This fact shows the robustness of our proposed


265

Table 3. Comparative experiments Comparison 1 Comparison 2 Proposed Query tag user&tag user&tag Co-occurrence not used category category number of users co-occurrence ratio likelihood ratio Similarity

system. Its robustness is also evident in Fig. 7, especially in clusters 1 to 3 and clusters 6 to 8. These clusters gave good results in spite of the various numbers of items. In some cases, we found cases of low precision, such as cluster 4, and cases of low recall, such as cluster 5. We can say with fair certainty that one of the reasons was a lack of data. We could only gather data from 1,000 people, which is less than 1% of SBM users. We found another reason by looking at the data. The name of the tag in cluster 4 was ‘javascript’; however, most of the recommender item clusters were ‘programming’. The scope of the recommender item clusters seemed to be broader than that of cluster 4. However, lack of data makes the similarities between cluster 4 and these recommender item clusters relatively high. The precision decreased because cluster 4 frequently recommended items out of the focus. The name of the tag in cluster 5 was ‘art’, and most of the recommender item cluster was ‘webdesign’. The scope of the recommender item cluster seemed to be narrower than that of cluster 5. Therefore, cluster 5 was recommended in only a part of items recommended and recall was decreased. These problems were caused by the lack of data, but there is further room for investigation. For example, we will combine item clusters to create more suitable recommender item clusters. However, a more comprehensive study on creating data lies outside the scope of this paper. 4.2 Comparison 1: Recommendation Based on Folksonomy In this section, we compare the recommendation systems based on folksonomy with our proposed system from the viewpoint of recall and precision. We show the difference between the two methods in Table 3 (Comparison 1 vs Proposed). Evaluation Method. The comparative recommendation system based on folksonomy is as follows. 1. User inputs a tag name into the system as a query. 2. System recommends items with such a tag in descending order. Experimental Results. We show an example of the comparative results in Fig. 8. This result corresponds to the item cluster which tquery is ajax, |B| is 2118 and |T | is 84. Our method clearly outperformed the system based on folksonomy. Discussion. One can safely state that the recommendation system based on item clusters can produce better results than the recommendation system based on folksonomy.

266

A. Sasaki et al.

Precision

Proposed Sum tag Jaccard

Recall

Fig. 8. Proposed system vs. system using tag names vs. system using Jaccard coefficient

One might also think that folksonomy would lead to better results than our system when the query word is used commonly. However, these results show that this may not be true. Note that ‘ajax’, which refers to javascript programming techniques, is a well-known word among web programmers. Let us look closely at the results to find why our system is more appropriate than the comparative one. In a query item cluster, the items tagged with ‘ajax’ show us high quality interfaces or programming techniques. On the other hand, in a recommender item cluster, the tagged items show us only an implementation of ajax. That is, these item clusters are based on different opinions even though the tags are the same. Moreover, recommendation system based on folksonomy cannot recommend items to users who use singular tag names such as ‘java/app’ or ‘***java***’. Our proposed system, however, is not limited by the tag name. 4.3 Comparison 2: Similarity by Jaccard Coefficient In this section, we compare the similarity by Jaccard coefficient with that based on hypothesis testing. We show the difference between the two methods in Table 3 (Comparison 2 vs Proposed). Evaluation Method. There are conventional systems for comparing the similarity of sample sets, such as the Jaccard and cosine coefficients. The Jaccard coefficient is defined as the two sample sets’ intersection divided by their union. We can define the Jaccard coefficient for our situation as follows. simJaccard (s, o) =

k , n

(10)

where n and k are the values in Eqs. (1) and (2). Therefore, we assume a comparative system replacing the Jaccard coefficient with similarities based on hypothesis testing. It can be said that the system is a conventional simple collaborative filtering system based on tags.

Precision


1.0 0.95 0.90 0.85 0.80 0.75 0.70 0.65 0.60 0

267

hyposesis testing Jaccard coef.

0.1

0.2 0.3 Recall

0.4

0.5

Fig. 9. Comparison: hyposesis testing vs. Jaccard coefficient

Fig. 10. Numerical comparison between similarities based on hypothesis testing and Jaccard coefficient

Results. We show an example of the comparative results in Fig. 8. We also show the result of evaluation using another dataset, a livedoor clips dataset in December 20084 in Fig. 9. The users who bookmarked in December 2008 numbered 25,370. They bookmarked 217,892 unique items and tagged items with about 8.59 words on average. We sorted all item clusters by |Ts | and evaluated the average of top 50 item clusters. We randomly selected 100 items as candidates of recommendation. A half of them (50 items) are in correct class and the others are not in correct class. The max |Bs | in these 50 query item clusters was 27,665, and min |Bs | was 1,176. The average was 4767.8. The max |Ts | in these 50 query item clusters was 19,710, and min |Ts | was 455. The average was 1592.84. Discussion. Our system is clearly more appropriate than the system based on the Jaccard coefficient. A comparison between hypothesis testing and the Jaccard coefficient is 4

Edge datasets, http://labs.edge.jp/datasets/

268

A. Sasaki et al.

shown in Fig. 10. Lines (1)-(a) and (b) show the same similarities by the Jaccard coefficient. On the other hand, lines (2)-(a) and (b) show the same similarities by hypothesis testing. Line (1) and (2)-(a) shows the value of 0.6 and (1) and (2)-(b) shows the value of 0.4. Then, it is open to question to equate the case of n = 4, k = 3 with the case of n = 20, k = 15. The former case would arise more often than the latter. Therefore, in the Jaccard coefficient, a small value of n leads to worse results. In other words, we have to avoid any accidental co-occurrence for a high-precision and high-recall recommendation system. Hypothesis testing shows a small value when n or k is small and a large value when both n and k are large. Then, Eq. 3 can calculate similarity except in accidental co-occurrences.

5 Conclusions We proposed a novel recommendation system using SBM data. Several conventional systems using folksonomy have focused on actual tag names. However, we focused on item clusters, which are sets of items tagged by each SBM user. We assumed SBM users’ behavior follows binomial distribution and used hypothesis testing to calculate the similarities between two item clusters. In addition, we evaluated our recommendation system. The results showed high recall and precision. We compared our proposed system with the systems using actual tag names and showed that our proposed system was more appropriate. We also compared our proposed similarity calculation based on hypothesis testing with a conventional similarity calculation and verified that our resultant similarities were better than the conventional ones.

References 1. Goldberg, D., Nichols, D., Oki, B.M., Terry, D.: Using collaborative filtering to weave an information tapestry. Communications of the ACM 35(12), 61–70 (2003) 2. Golder, S., Huberman, B.A.: The structure of collaborative tagging systems. Journal of Information Science (2005) 3. Gunduz, S., Ozsu, M.T.: A user interest model for web page navigation. In: Proceedings of International Workshop on Data Mining for Actionable Knowledge (2003) 4. Hotho, A., Jaschke, R., Schmitz, C., Stumme, G.: Information retrieval in folksonomies: Search and ranking. In: Sure, Y., Domingue, J. (eds.) ESWC 2006. LNCS, vol. 4011, pp. 411–426. Springer, Heidelberg (2006) 5. Ishikawa, H., Nakajima, T., Mizuhara, T., Yokoyama, S., Nakayama, J., Ohta, M., Katayama, K.: An intelligent web recommendation system: A web usage mining approach. In: Hacid, M.-S., Ra´s, Z.W., Zighed, D.A., Kodratoff, Y. (eds.) ISMIS 2002. LNCS (LNAI), vol. 2366, pp. 342–350. Springer, Heidelberg (2002) 6. Jaschke, R., Marinho, L.B., Hotho, A., Schmidt-Thieme, L., Stumme, G.: Tag recommendations in folksonomies. In: Kok, J.N., Koronacki, J., Lopez de Mantaras, R., Matwin, S., Mladeniˇc, D., Skowron, A. (eds.) PKDD 2007. LNCS (LNAI), vol. 4702, pp. 506–514. Springer, Heidelberg (2007) 7. Kazienko, P., Kiewra, M.: Integration of relational databases and web site content for product and page recommendation. In: International Database Engineering and Applications Symposium, IDEAS 2004 (2004)


269

8. Li, J., Zaiane, O.R.: Combining usage, content, and structure data to improve web site recommendation. In: Proceedings of Web KDD 2004 workshop on Web Mining and Web Usage (2004) 9. Mika, P.: Ontologies are us: a unified model of social networks and semantics. In: Gil, Y., Motta, E., Benjamins, V.R., Musen, M.A. (eds.) ISWC 2005. LNCS, vol. 3729, pp. 122–136. Springer, Heidelberg (2005) 10. Niwa, S., Doi, T., Honiden, S.: Web page recommender system based on folksonomy mining. In: Proceedings of the Third International Conference on Information Technology: New Generations, ITNG 2006 (2006) 11. Resnick, P., Iacovou, N., Suchak, M., Bergstrom, P., Riedl, J.: Grouplens: An open architecture for collaborative filtering of netnews. In: Proceedings of the 1994 Computer Supported Cooperative Work Conference, pp. 175–186 (1994) 12. Rucker, J., Polanco, M.J.: Siteseer: Personalized navigation for the web. Communications of the ACM 40(3), 73–75 (1997) 13. Sarwar, B.M., Karypis, G., Konstan, J.A., Riedl, J.: Item-based collaborative filtering recommendation algorithms. In: Proceedings of the 10th International World Wide Web Conference (WWW10), pp. 285–295 (2001) 14. Yanbe, Y., Jatowt, A., Nakamura, S., Tanaka, K.: Can social bookmarking enhance search in the web? In: Proceedings of the 7th ACM/IEEE-CS joint conference on Digital libraries (2007)

Classifying Structured Web Sources Using Support Vector Machine and Aggressive Feature Selection Hieu Quang Le and Stefan Conrad Institute of Computer Science, Heinrich-Heine-Universität Düsseldorf D-40225 Düsseldorf, Germany {lqhieu,conrad}@cs.uni-duesseldorf.de

Abstract. This paper studies the problem of classifying structured data sources on the Web. While prior works use all features, once extracted from search interfaces, we further refine the feature set. In our research, we use only the text content of the search interfaces. We choose a subset of features, which is suited to classify web sources, by our feature selection methods with new metrics and a novel simple ranking scheme. Using aggressive feature selection approach, together with a Support Vector Machine classifier, we obtained high classification performance in an evaluation over real web data. Keywords: Deep web, Classification, Database, Feature selection, SVM, Support Vector Machine.

1 Introduction There are a large number of websites that store information in form of structured data with attribute-value pairs [7], forming an important part of the huge Deep Web [4]. These structured web sources provide search interfaces so that users can query their databases. On the one hand, integrated access over multiple sources is needed. For examples, a user may want to compare prices of a book in different online shops; or s/he may buy air tickets and book a room in a hotel online while preparing for a trip. On the other hand, there have been researches on data integration of a relative small number of heterogeneous sources (e.g. [9,18]), as well as on large-scale search over multiple text databases (e.g. [5,15]). Consequently, projects aiming at providing integrated data access to a large number of structured web sources, such as MetaQuerier [8] and WISE [14], have been born. Building and maintaining such large-scale accessing services involves a number of tasks: source finding and categorization, schema mapping and query translation, data extraction and integration, etc. The categorization task is an integral part of these projects, as sources that have been collected must be grouped according to similarity before other tasks, such as schema mapping [12] or query interface integration [28], can be performed. In this paper, we address this important problem of web source categorization. The search interfaces of structured web sources, which serve as the “entrances” to underlying databases, are used as the main source for the categorization task. In [13], He et al. argued that the form labels of search interfaces (e.g. ‘Title’, ‘ISBN(s)’, . . . in an Amazon’s search form) are the right “representatives” for structured sources, and used J. Cordeiro and J. Filipe (Eds.): WEBIST 2009, LNBIP 45, pp. 270–282, 2010. c Springer-Verlag Berlin Heidelberg 2010

Classifying Structured Web Sources Using SVM and Aggressive Feature Selection

271

only them. Subsequently, in addition to form labels as the most important feature, Lu et al. identified and utilized other features such as form values (e.g. ‘hardcover’, ‘paperback’) and other regular text terms [19]. In these two works, features inputed to their clustering algorithms must be extracted from HTML pages by another technique (see [19]). In contrast, Barbosa et al. argued that such an extraction task is hard to automate, so they used all the text (bag-of-words) of a search interface, which is partitioned into text of the form and text of the page, together with backlinks pointing to the interface [2]. A common thing in the prior works is that features, once extracted, are all used without any further selection. However, it is not difficult to see that in a search interface, words that help in distinguishing categories (e.g. ‘author’, ‘textbooks’) mingle with many more other words. Indiscriminative or noisy terms (e.g. ‘sort by’, ‘state’) also occur inside forms, as observed in [19]. Thus, this paper investigates on how to identify features suitable for categorizing structured web sources, i.e. the feature selection (FS) problem. Our classification approach employs a filtering FS method in text categorization [24], together with a Support Vector Machine (SVM) classifier [3]. In our research, we use only the text content of a search interface. To choose a suitable subset of terms, we conducted experiments with various FS techniques, such as χ2 (CHI), Information Gain (IG) [29], Bi-normal separation (BNS) [10], as well with our methods. 1 By using aggressive feature selection with χ2 or our methods, we obtained high classification performance with the subset selected, which is significantly higher than the performance obtained when using the much larger set of all features. This result does not only show that our classification approach has its own strength, but is also a convincing evidence that extracted features should be further selected. This is our first contribution. Our second contribution is that we propose a new feature selection method. As pointed out in [2], it is prone to make clustering mistakes among domains with overlapping vocabulary (e.g. Movies and Musics), and for domains with a highly heterogeneous vocabulary. Our FS methods, with new metrics and a novel ranking scheme, aim at tackling the issues. In the mentioned experiments with different FS techniques, we obtained the best performance with the new methods. In the following section, we review related work. In Section 3, we describe the classification process. Then come its details: the feature selection method in Section 4, and the choices of weighting scheme and of SVM kernel in Section 5. Section 6 presents our experimental result and discussion. We conclude with a summary and future work in Section 7.

2 Related Work We relate our work to other categorization problems, to the researches on the same topic of structured web source (SWS) categorization and on feature selection. First, SWS categorization is related to text database classification (e.g. [5,15]) as they work with the Deep Web’s sources. A text database normally stores documents 1

We use the phrase of “FS metric” to indicate a scoring formula, and “FS method/technique” to indicate a scoring formula with a ranking scheme.

272

H.Q. Le and S. Conrad

of multiple categories. Its interfaces contain simple (e.g. single-label) forms and little information about stored documents. Therefore, to have the representative summary of a text database, query submission techniques that send queries to sources and analyze returned results are needed. In contrast, a structured database stores data objects of a single domain. Its search interfaces provide much information, such as form labels and values (describing exported schema), advertised products (i.e. data items), hence can be used to categorize the database. Furthermore, it is necessary that the domain of a structured source, which contains complex and multi-label search forms, is known before a query submission technique can be applied. Since utilizing search interfaces, SWS categorization is also related to web page categorization (e.g. [6,30]) which uses terms and links, and to text categorization (e.g. [16]), which uses only terms. However, the goal of SWS categorization is not to classify a search page (i.e. an HTML page) itself but the database, to which the page’s search form serves as an “entrance”. As a result, discriminative features extracted from or related to search forms are most important to the task. Second, as mentioned in Section 1, the works [13,19,2], together with this paper, are on the same topic of SWS categorization. While the prior works show that it is feasible to determine the domain of a source by using discriminative features extracted from source’s search interfaces, we take a step further. We refine the set of features once extracted, and use the refined set in order to increase classification performance. In addition, the prior studies employed clustering (i.e. unsupervised learning) algorithms, while we use a classifying (i.e. supervised learning) technique. One reason is that we group web databases so that other integrating tasks can be performed (see Section 1). There is no online requirement as, for example, in search engines where web documents may need to be clustered dynamically within a few seconds in response to a user’s need [30]. The emphasis is thus on accuracy. The other reason is that our goal is to categorize a large number of sources. It is appropriate that we build an initial domain hierarchy, either manually or by clustering together with manually checking, from a small number of sources; then classifying the rest so as to make use of sample data better through a learning process. Third, the problem of feature selection in text categorization has been intensively studied, e.g. in [29,20,25,10,11]. These works, as well as our methods, use the filtering approach, in which features are scored by a metric, then the highest ranked features are selected [24]. There are a number of FS metrics, such as χ2 , IG and BNS, and each has its own rationale. (A comprehensive list can be found in [24,10].) As discussed later, our metrics are designed to find terms that help in distinguishing closely related domains. In terms of the ranking technique, the standard scheme sorts features regardless of categories [24], whereas our approach ranks features with respect to the category that they represent best.

3 Classification Process In our research, each complete search interface is treated simply as a text document, i.e. a bag-of-words extracted from its HTML content. Similar to other studies [13,19,2], we assume that one web database, together with its search interfaces, belongs to one


273

category. Since a document representing a web source belongs to one category, and there are multiple categories, our problem is equivalent to a single-label multi-class text categorization problem [24]. 2 We choose Support Vector Machine, a supervised learning kernel method, as our classifier, because it works in high dimensional feature spaces and has been applied successfully to text categorization [16]. Formally, the process of categorizing documents using a SVM classifier is described as follows: Let C = {ci }ni=1 be the set of n categories; D = {dj }m j=1 be the training set of m text documents, in which each document dj is labeled with a predetermined “correct” category ci ; V be the set of all features (i.e. processed words) appearing in D. Let d∗ denote a “new” document not in D that is to be classified into some category ci in C. 1. Use a FS method to choose from V a subset of N features VF S = {tk }N k=1 . 2. Represent each document dj in D as a N -dimension vector vj = (wj1 , . . . , wjN ), in which wjk is the weight of a feature tk in the document dj according to some weighting scheme. 3. Choose a SVM kernel, then train the classifier with the m vectors v1 , . . . , vm as inputs and their predetermined categories as target outputs. 4. After the learning process has been completed, the SVM classifier is ready to classify the vectorized representation of the document d∗ . In the following section, we will describe step 1, then in Section 5 step 2 and 3.

4 Feature Selection The feature selection problem in text categorization is to identify words that are suited to categorize documents. This section describes our methods that consist of (1) new metrics, and (2) a novel ranking scheme. 4.1 Feature Selection Metrics We first define new metrics, then give their intuitive explanation and comparison with χ2 metric. Metric Definition. Let {ci }ni=1 be the set of n (n ≥ 2) categories; P (t|c) be the conditional probability that a random document in a category c contains a feature t; P (c|t) be the conditional probability that a random document containing a feature t belongs to a category c; P (c) be the probability that a random document belongs to a category c. Let c1 and c2 denote the two categories that have the first and second highest values of all probabilities P (ci |t) for i = 1, . . . , n. (In case some two categories have the same probability P (c|t), we choose the category with the higher probability P (c).) The score of a feature t is given in one of the two formulas below: 2

Our approach can be easily adapted to a multi-label case, i.e. an interface is to be classified into several categories, by transforming the problem into independent problems of binary classification between a category c and its complement c [24].

274


Top-two-category separation (T2CS): S1 (t) = P (t|c1 ) − P (t|c2 ) · P (c1 |t) − P (c2 |t) Top-two-category separation - χ2 (T2CS-CHI): S2 (t) = P (t|c1 ) · P (t|c2 ) − P (t|c2 ) · P (t|c1 ) · · P (c1 |t) · P (c2 |t) − P (c2 |t) · P (c1 |t) Metric Explanation. We now explain the intuition behind T2CS metric through an example with three categories Airfares, Books and Musics, denoted as Af, Bk and Ms respectively. When a document contains the word ‘music’, together with other words, such as ‘title’, ‘elvis’ and ‘surround’, we can normally determine quickly that its category is either Musics or Books, but not Airfares. We observe that Musics and Books are the two categories to which a random document containing ‘music’ most likely belongs, and vocabularies of which usually have a large overlap. Since the main categorization difficulty is in deciding between them, we are interested in how much ‘music’ helps to classify documents into Musics or Books. We formalize it by selecting the “top” two categories that have the first and second highest values of all probabilities P (Category|music). Next, we formulate the scoring function S1 of T2CS metric. When a document contains the word ‘music’, we tend to think that its category is likely to be Musics. In that view, we take P (M s|music) − P (Bk|music) to measure how much ‘music’ helps if the document in fact belongs to Musics, as well as how much it misleads us if the document in fact belongs to Books instead. When all documents are taken into consideration, the more frequently ‘music’ appears in Musics (i.e. the higher probability P (music|M s)) the better, but the more frequently ‘music’ appears in Books (i.e. the higher probability P (music|Bk)) the less it helps, or in other words, the more it misleads us. Hence, we evaluate the score of ‘music’ in differentiating all documents of Musics from all documents of Books, as follows: P (music|M s) · P (M s|music) − P (Bk|music) − −P (music|Bk) · P (M s|music) − P (Bk|music) , which is the same as the formula S1 (music). Note that since the sum of P (Af |music), P (Bk|music) and P (M s|music) is 1, the role of Airfares is indirectly taken into account, though it does not explicitly appear in the scoring formula. Finally, let us consider all the example words ‘music’, ‘elvis’, ‘surround’, ‘title’ again. Statistics on the dataset we used shows that ‘music’ appears very frequently in Musics, relatively frequently in Books; ‘elvis’ relatively frequently and only in Musics; ‘surround’ rarely and only in Musics; ‘title’ very frequently in Books as well in Musics. Under our scoring scheme, ‘surround’ and ‘title’ have low ranks and will usually be discarded. In other metrics like IG or χ2 , similar situation happens to ‘surround’, but not to ‘title’. The other metrics score ‘title’ high, as it may distinguish Books (and Musics) from Airfares. Meanwhile, we score ‘title’ low as its usage may increase the mis-classification between Books and Musics. Note further that in text categorization,


275

contrary to a widely held belief in information retrieval, common terms are informative, and scored in favor over rare terms [29]; and we will discuss about it more in Section 6.4. Comparing T2CS-CHI to χ2 . T2CS-CHI is a variant of T2CS metric. Besides the presence of a feature t in a document, i.e. P (c|t), is into account as in T2CS, T2CS-CHI also makes use of the information on the non-presenceof t, i.e. P (c|t). (Note that since the sum of P (t|c)and P (t|c) is 1, P (t|c1 ) − P (t|c2 ) is equal to P (t|c1 ) · P (t|c2 ) − P (t|c2 ) · P (t|c1 ) .) T2CS-CHI metric has its name from the fact that in the case of binary categorization, i.e. n = 2, it can be proved to be equivalent to χ2 metric defined in [24] as follows: 2 P (t, c) · P (t, c) − P (t, c) · P (t, c) χ2 (t) = |D| · P (t) · P (t) · P (c) · P (c) where |D| denotes the total number of documents; P (t, c) the probability that a random document contains a feature t and belongs to a category c; c the complement of a category c. In the binary case, T2CS-CHI differs from χ2 only in the number |D|, which is the same for all scores. However, in the case of multi-class categorization, T2CS-CHI and χ2 are different from each other in the way of selecting categories. That is, T2CS-CHI metric utilizes only the “top” two categories. Meanwhile, in the common approach, χ2 , as well as other metrics such as BNS, are calculated from a category and its complement. Hence, for each feature, there is a score for every category; and the final score of the feature is the maximum or average value [29]. 4.2 Feature Selection Procedure This subsection discusses how our classification method processes forms and preprocesses words, then ranks and selects features. Form Processing. Following [1] we identify terms in forms (i.e. between F ORM tags) and terms in pages (i.e. within HT M L tags). We further use terms in titles (i.e. between T IT LE tags) separately. After parsing HTML search interfaces to extract words in these three types, we do not apply a stemming algorithm to reduce words with the same stem to a common form. The reason is that, for example, word ‘book’ appears frequently in both Airfares and Books domains, while word ‘books’ is often found in Books but not in Airfares. Thus, these two words should be seen as two different discriminative features instead of being merged by a stemming algorithm. In addition, terms that appear less than some small K times in every category are to be eliminated. This technique is to remove noisy words. Feature Ranking and Selection. First, we rank and select terms in each of the three types separately. While the standard scheme of the filtering approach ranks terms by their scores and selects them top down regardless of categories [24], we approach differently. In the investigation of the standard ranking scheme with a metric, e.g. T2CS or χ2 , we put each term t of the top N terms selected into the category c, to which

276


a document containing the term t most likely belongs, i.e. P (c|t) is the highest, or in other words, into the category c which the term t represents best. We observed that some categories get many more terms assigned than other categories (e.g. Automobiles may contain around 10 times higher than Books). This imbalance may make classification more error prone, since there may be not enough discriminative features to accept or reject whether a document belongs to a category with the small number of terms. The category with the small number, in turn, usually has a highly heterogeneous vocabulary. Hence, we propose a new simple ranking scheme aiming at balancing the number of terms representing each category. It is the steps 1 and 2 of the FS procedure described as follows: 1. Compute scores of terms by a metric (our metrics or the others), and assign each term t to the category c with maximum P (c|t). 2. Sort terms in each category by their scores in descending order; then re-rank all terms together first by the relative ranks in their categories in ascending order and second by their scores in descending order. 3. Select top N ranked terms from the set of all terms, where N is a parameter determined through a cross-validation procedure. Finally, we simply put all selected terms of the three title, f orm and page types into a common feature space. In our implementation, we use the same top N parameter for the three feature types. We note that the same terms in different types will be represented as different features and weighted differently in our classification method.

5 Weighting Scheme and SVM Kernel We choose a non-weighted feature scheme for document vectors in the step 2, and a linear kernel in the step 3 of the classification process (see Section 3), for we obtain the best performance with these modest settings. Formally, let VF S = {tk }N k=1 be the set of N selected features. In the non-weighted feature scheme, a document dj is represented as a N -dimension vector of weights vj = (wj1 , . . . , wjN ), where: 1 if tk is in dj wjk = 0 otherwise. We normalize each document vector, making its absolute value 1. We observe that the distinctive words of a domain (e.g. ‘ISBN’, ‘paperback’ and ‘hardcover’ for Books domain) often appear only once in a search interface as its form labels or values. In our approach, the weight of a feature is not only determined by a weighting scheme but also through training a SVM classifier. The two observations help to explain why the non-weighted feature scheme is suitable for the categorization task at hand. Moreover, our choice of a linear kernel is similar to other studies in text categorization (e.g. [16,11]) that used a SVM classifier.


277

6 Experiments 6.1 Dataset and GPC Implementation We used the TEL-8 dataset of the UIUC Web integration repository [27], which contains the search interfaces of 447 structured web sources classified into 8 domains. After converting HTML pages to text documents and manually checking, we kept 431 sources. The other sources were not usable because the offline contents of search pages required an online update while the pages themselves no longer exist on the Web. Table 1 describes the dataset that we used. We conducted experiments in 4-fold cross-validation scheme. In the form processing step, we ignored words that appear less than three times in every category, i.e. K = 3, so as to remove noisy words. For the problem at hand, web sources are to be discovered automatically by a crawler (see, e.g. [1]). Due to the dynamic nature of the Web, we assume that there is no prior information in regard to the category of a new web source to be classified. 3 Therefore, in the implementation of FS metrics, we assigned the value (1/n) to the probability P (c) that a random document belongs to a category c, where n is the total number of categories. Table 1. Dataset of 431 web sources in 8 domains Domain # of sources Airfares 43 Automobiles 80 Books 66 CarRentals 21

Domain # of sources Hotels 34 Jobs 50 Movies 71 Musics 66

For the SVM implementation, we used the SV M multiclass [26]. Besides FS level (top N ), the only parameter for the linear kernel we use is C (trade-off between training error and margin). We determine these parameters through 10-fold cross-validation scheme on each training set.4 Because the training sets are small, we have, and at the same time are able, to use the high 10-fold scheme. 6.2 Performance Measures and Feature Selection Methods To evaluate classification performance for all categories, we use overall precision p, recall r and F-measure F1 defined in [19] as follows: n n mi mi 2·p·r ), r = ), F1 = , (pi · (ri · p= m m p+r i=1 i=1 where pi is the precision of a category ci , i.e. the ratio of the number of sources correctly classified over the number of sources that are assigned into ci ; ri is the recall of a 3

4

In the TEL-8 dataset used by us and in [13,2], the ratio between the source number of Books domain and that of Jobs domain is 66 : 50 (1.32), while in another dataset used in [19] the ratio is 85 : 20 (4.25). This substantial difference in the ratio of sources is additional support for our assumption. Specifically, the FS levels are in every 240 features, and for the C parameter the feasible set is {20000, 40000, 60000, 80000, 100000}.

278


category ci , i.e. the ratio of the number of sources correctly classified over the number of sources that should have been classified into ci ; mi is the number of sources in a category ci ; m is the total number of sources and n is the total number of categories. For single-labeled datasets, the overall recall, as well as widely used micro-precision, micro-recall and micro-F1 [24], can be proved to be equal to accuracy, which is conventionally used in text categorization researches. Hence, we choose accuracy as the single measure in our graph comparing FS methods. In addition to our two metrics, we carried out experiments with other reportedly most effective metrics: χ2 (CHI), the multi-class version of Information Gain (IG), Document Frequency (DF) [29], the binary version of Information Gain (IG2) [11], Binormal separation (BNS) [10], Odds Ratio (OR) [20]. For metrics with one value per category (χ2 , IG2, BNS, OR), we used the maximum value as the score, for it performs better than the average value across metrics, classifiers, and text collections [23]. In the next two subsections, we will discuss T2CS, T2CS-CHI, χ2 , IG and DF metrics, while omit the others for their results are either similar to or lower than that of χ2 or IG. 6.3 The Effect of Feature Selection Table 2 shows how aggressive feature selection affects classification performance. We report results at the FS level and C parameters determined by using only training sets, and the number of features which is the average value of training sets. When no FS methods are applied, i.e. using all of around 8030 features after form processing, the overall precision, recall and F-measure obtained are around 92.4%. This good result reflects a common observation that SVM classifiers can cope quite well with many redudant features [16]. When applying our methods with either T2CS or T2CS-CHI metrics together with the new ranking scheme, the three measures’ values are around 95%. Those of existing FS methods, χ2 and IG metrics with the standard ranking scheme, are respectively around 94.6% and 93.7%. The automatically determined FS levels of the best three FS methods T2CS, T2CS-CHI and χ2 are not higher than 1560 features. Thus, these FS methods improve classification performance significantly while using subsets that are much smaller than the set of all features. Table 3 presents the results of two methods “All features after form processing” (column “a”) and T2CS (column “b”), where Af, Am, Bk, Cr, Ht, Jb, Mv and Ms are abbreviation for 8 domains Airfares, Automobiles, Books, CarRentals, Hotels, Jobs, Movies and Musics respectively. In Table 3, for example, value 2 in the cell row “Cr” column Table 2. Classifcation performance Method All features after form processing T2CS T2CS-CHI χ2 IG

Precision Recall F-measure # of features 92.73 % 92.11 % 92.42 % 8030 95.46 % 95.28 % 94.84 % 94.03 %

95.13 % 94.90 % 94.43 % 93.51 %

95.30 % 95.09 % 94.63 % 93.77 %

1560 1320 1020 4650


279

Table 3. Classifcation results

Af Am Bk Cr Ht Jb Mv Ms

Af Am Bk Cr Ht Jb Mv a b a b a b a b a b a b a b 38 40 0 0 0 0 3 3 2 0 0 0 0 0 0 0 79 79 0 0 0 0 0 0 0 0 1 1 0 0 0 0 64 65 0 0 0 0 1 0 1 1 4 2 0 0 0 0 16 18 1 1 0 0 0 0 4 2 0 0 0 0 1 0 29 32 0 0 0 0 0 0 0 0 0 0 0 0 0 0 50 50 0 0 0 0 0 0 2 3 0 0 2 0 0 0 62 65 0 0 0 0 0 0 0 0 0 0 0 0 7 5 (a) All features after form processing, (b) T2CS.

Ms a b 0 0 0 0 0 0 0 0 0 0 0 0 5 3 59 61

“Af-b” means that two web sources, which in fact belong to CarRentals, have been assigned into Airfares when using T2CS method. (A cell value is the sum of the results given by validation sets.) It can be observed in Table 3 that T2CS method gives better or equal performance for all domains, specifically for two groups of closely related domains {Airfares, CarRentals, Hotels} and {Movies, Musics}. For Books, a domain with a highly heterogeneous vocabulary, T2CS method shows a further improvement over the good result when using all features. 6.4 Comparision of FS Methods Figure 1 compares the accuracy of our T2CS, T2CS-CHI methods and that of existing χ2 and IG methods at different FS levels. To draw the graph, at each FS level we choose the best accuracy among feasible parameters C in each cross-validation set, then take the average values of accuracy across all cross-validation sets. 5 It can be observed that T2CS, T2CS-CHI and χ2 methods perform similarly with high accuracy around 95% in the optimal range of 500-2000 features. Next comes IG with the accuracy around 94% at the FS level of 4000 features. DF performs worst. Consistently with prior studies in text categorization [29,23], we find that rare words are not important. (This finding is in contrast to other works of SWS categorization, which weight rare words high [19,2]). As shown in Figure 1, DF method, which simply counts the number of documents containing the feature in a whole collection, has an accuracy of 92.6% at 3340 features or an equivalent DF threshold of 10, and it maintains the accuracy around this value throughout higher FS levels. (We did not apply the technique of eliminating noisy words to this FS method.) As a result, a large number of “rare” features can be removed without any loss of performance, as the total number of features is around 39100. As can be seen in Table 2 and Figure 1, T2CS and T2CS-CHI metrics have similar performance. Meanwhile, these two metrics are different from each other only in P (c|t) (see Section 4.1). Hence, the information that a term is absent from a document is not important when selecting features with our metrics. 5

The performance with the automatically tunned FS level and C parameters can not be exactly seen from this graph.

280


100

T2CS T2CS-CHI DF IG 2 χ

98

Accuracy (%)

96 95 94 92 90 88 86 0

500 1000 1500 2000 3000 4000 Number of unique features

5000

6000

Fig. 1. Comparision of FS methods

6.5 Computational Efforts In this section, we compare the efficiency of two kernel methods Support Vector Machine and Gaussian processes [22] in applying to our problem at hand. Our experiments show that the SVM classifier provides much higher efficiency. Below is the run-time information of those experiments carried on a 2.4GHz 1.5GB RAM Pentium 4 operated by Debian GNU/Linux 4. In general, the higher the FS level and parameter C, the longer the SVM training is. As an example with FS level 2400 features and C 100000, the SVM took 10 seconds to train a set of 323 samples, and the amount of memory used was below 200MB. For T2CS method, the overall training and testing time for entire data set (including four cross-validation sets, each needs an additional 10-fold cross-validation for parameters) was 2.5 hours (around 40 minutes for each of the four folds). With regards to a Gaussian process classifier (GPC), we used the implementation in the fbm software [21]. When we conducted experiments in 2-fold cross-validation scheme, the accuracy of the GPC, reported in [17], was slightly higher than that of SVM classifier, and the run-time was acceptable but much longer than that of SVM. However when we used 4-fold scheme with training sets around 1.5 times as big and features twice as many, the GPC ran very slowly. It took the GPC more than 3 hours to train the above 323 samples at the FS level of 2400 features. Furthermore, the GPC implementation in the fbm sofware can not manage more than 10000 features. Therefore we stopped experimenting with the GPC and do not report its classification results in this paper, since we consider this GPC implementation not yet suitable for a large-scale project.


281

7 Conclusions In this paper, we study the problem of categorizing structured web sources by using their search interfaces. Our approach employs a filtering feature selection technique together with a Support Vector Machine classifier. In our research, we use only the text content of the search interfaces. We conducted experiments with our FS methods with new metrics and a novel simple ranking scheme, as well with existing FS methods. The experimental result indicates that: (1) feature selection techniques improve classification performance significantly; (2) our classification approach and the proposed FS methods are effective. Our research also points out that rare words are not important to the categorization task. For future work, in terms of the classification method, one possible improvement to our research is to identify and use more different feature types from a search interface. In terms of the feature selection technique, we plan to evaluate the effectiveness of the new methods in other text categorization problems.

References 1. Barbosa, L., Freire, J.: Searching for hidden-web databases. In: 8th ACM SIGMOD International Workshop on Web and Databases, pp. 1–6 (2005) 2. Barbosa, L., Freire, J., Silva, A.: Organizing hidden-web databases by clustering visible web documents. In: IEEE 23rd International Conference on Data Engineering, pp. 326–335 (2007) 3. Bennett, K.P., Campbell, C.: Support vector machines: hype or hallelujah? ACM SIGKDD Explorations 2(2), 1–13 (2000) 4. Bergman, M.K.: White paper - The Deep Web: Surfacing hidden value (2001), http://www.brightplanet.com 5. Callan, J.P., Connell, M., Du, A.: Automatic discovery of language models for text databases. ACM SIGMOD Record 28(2), 479–490 (1999) 6. Chakrabarti, S., Dom, B., Indyk, P.: Enhanced hypertext categorization using hyperlinks. ACM SIGMOD Record 27(2), 307–318 (1998) 7. Chang, K.C.-C., He, B., Li, C., Patel, M., Zhang, Z.: Structured databases on the Web: Observations and implications. ACM SIGMOD Record 33(3), 61–70 (2004) 8. Chang, K.C.-C., He, B., Zhang, Z.: Toward large scale integration: Building a MetaQuerier over databases on the web. In: 2nd Conference on Innovative Data Systems Research, pp. 44–55 (2005) 9. Chawathe, S., Garcia-molina, H., Hammer, J., Irel, K., Papakonstantinou, Y., Ullman, J., Widom, J.: The Tsimmis project: Integration of heterogeneous information sources. In: IPSJ Conference, Tokyo (1994) 10. Forman, G.: An extensive empirical study of feature selection metrics for text classification. Journal of Machine Learning Research 3, 1289–1305 (2003) 11. Gabrilovich, E., Markovitch, S.: Text categorization with many redundant features: using aggressive feature selection to make SVMs competitive with C4.5. In: 21st International Conference on Machine Learning, pp. 321–328 (2004) 12. He, B., Chang, K.C.-C.: Statistical schema matching across web query interfaces. In: 2003 ACM SIGMOD Conference, pp. 217–228 (2003) 13. He, B., Tao, T., Chang, K.C.-C.: Organizing structured web sources by query schemas: A clustering approach. In: 13th Conference on Information and Knowledge Management, pp. 22–31 (2004)

282


14. He, H., Meng, W., Yu, C., Wu, Z.: WISE-Integrator: A system for extracting and integrating complex web search interfaces of the Deep Web. In: 2005 International Conference on Very Large Data Bases, pp. 1314–1317 (2005) 15. Ipeirotis, P.G., Gravano, L., Sahami, M.: Probe, count, and classify: categorizing hidden web databases. In: 2001 ACM SIGMOD International Conference on Management of Data, pp. 67–78 (2001) 16. Joachims, T.: Text categorization with support vector machines: Learning with many relevant features. In: Nédellec, C., Rouveirol, C. (eds.) ECML 1998. LNCS, vol. 1398, pp. 137–142. Springer, Heidelberg (1998) 17. Le, H.Q., Conrad, S.: Classifying Structured Web Sources Using Aggressive Feature Selection. In: 5th International Conference on Web Information Systems and Technologies (2009) 18. Levy, A.Y., Rajaraman, A., Ordille, J.J.: Querying heterogeneous information sources using source descriptions. In: 22nd International Conference on Very Large Data Bases, pp. 251– 262 (1996) 19. Lu, Y., He, H., Peng, Q., Meng, W., Yu, C.: Clustering e-commerce search engines based on their search interface pages using WISE-Cluster. Data & Knowledge Engineering Journal 59(2), 231–246 (2006) 20. Mladenic, D.: Feature subset selection in text learning. In: Nédellec, C., Rouveirol, C. (eds.) ECML 1998. LNCS, vol. 1398, pp. 95–100. Springer, Heidelberg (1998) 21. Neal, R.M.: Monte Carlo implementation of Gaussian process models for Bayesian regression and classification. Technical Report 9702, Dept. of Statistics, Uni. of Toronto (1997), http://www.cs.toronto.edu/˜radford/ 22. Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. The MIT Press, Cambridge (2006) 23. Rogati, M., Yang, Y.: High-performing feature selection for text classification. In: 11th International Conference on Information and Knowledge Management (2002) 24. Sebastiani, F.: Machine learning in automated text categorization. ACM Computing Surveys 34(1), 1–47 (2002) 25. Soucy, P., Mineau, G.W.: A simple feature selection method for text classification. In: 17th International Conference on Artificial Intelligence, pp. 897–902 (2001) 26. Tsochantaridis, I., Hofmann, T., Joachims, T., Altun, Y.: Support Vector Learning for Interdependent and Structured Output Spaces. In: 21st International Conference on Machine Learning (2004), http://svmlight.joachims.org/ 27. UIUC: The UIUC Web integration repository. Computer Science Dept., Uni. of Illinois at Urbana-Champaign (2003), http://metaquerier.cs.uiuc.edu/repository 28. Wu, W., Yu, C., Doan, A., Meng, W.: An interactive clustering-based approach to integrating source query interfaces on the Deep Web. In: 2004 ACM SIGMOD International Conference on Management of Data (2004) 29. Yang, Y., Pedersen, J.O.: A comparative study on feature selection in text categorization. In: 14th International Conference on Machine Learning, pp. 412–420 (1997) 30. Zamir, O., Etzioni, O.: Web document clustering: a feasibility demonstration. In: 21st International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 46–54 (1998)

Scalable Faceted Ranking in Tagging Systems José I. Orlicki1,2 , J. Ignacio Alvarez-Hamelin2,3 , and Pablo I. Fierens2 1 3

Core Security Technologies, Humboldt 1967 1◦ , C1414CTU Buenos Aires, Argentina 2 ITBA, Av. Madero 399, C1106ACD Buenos Aires, Argentina CONICET (Argentinian Council of Scientific and Technological Research), Argentina [email protected], [email protected], [email protected]

Abstract. Nowadays, web collaborative tagging systems which allow users to upload, comment on and recommend contents, are growing. Such systems can be represented as graphs where nodes correspond to users and tagged-links to recommendations. In this paper we analyze the problem of computing a ranking of users with respect to a facet described as a set of tags. A straightforward solution is to compute a PageRank-like algorithm on a facet-related graph, but it is not feasible for online computation. We propose an alternative: (i) a ranking for each tag is computed offline on the basis of tag-related subgraphs; (ii) a faceted order is generated online by merging rankings corresponding to all the tags in the facet. Based on the graph analysis of YouTube and Flickr, we show that step (i) is scalable. We also present efficient algorithms for step (ii), which are evaluated by comparing their results with two gold standards. Keywords: Web intelligence, Tagging systems, Faceted ranking.

1 Introduction In collaborative tagging systems, users assign keywords or tags to their uploaded content or bookmarks, in order to improve future navigation, filtering or searching, see, e.g., Marlow et al. [1]. These systems generate a categorization of content commonly known as a folksonomy. Two well-known collaborative tagging systems for multimedia content are YouTube [2] and Flickr [3], which are analyzed in this paper. These systems can be represented in a tagged-graph as that shown in Figure 1. In this example, there are four users, A, B, C and D. M is the set of contents and associated tags. For example, user B has uploaded one multimedia content, song2, to which it has associated the tag-set {blues,jazz}. V is the set of recommendations; e.g., user A recommends song2 of user B, which is represented in the graph as an arrow from A to B with tags blues,jazz. Users can be ranked in relation to a set of tags, called a facet in this paper. Some applications of faceted (i.e., tag-associated) rankings are: (i) searching for content through navigation of the best users inside a facet; (ii) measuring reputation of users by listing their best rankings for different tags or tag-sets. The order or ranking can be determined by a centrality measure, such as PageRank (see, e.g., Page et al. [4] and Langville and Meyer [5]), applied to a recommendation or subscription graph. Given a facet, a straightforward solution is to compute J. Cordeiro and J. Filipe (Eds.): WEBIST 2009, LNBIP 45, pp. 283–296, 2010. c Springer-Verlag Berlin Heidelberg 2010

284

J.I. Orlicki, J.I. Alvarez-Hamelin, and P.I. Fierens M = {(A, song1, {blues}) V = {(A, song2) (B, song2, {blues,jazz}) (B, song4) (C, song3,{blues}) (B, song5) (C, song4,{jazz}) (A, song3) (D, song5,{blues}) (A, song4) (D, song6,{rock}) } (C, song6) } blues /B /D @@ ~? ~ @@ ~ @@ ~~ ~ @@ jazz ~~ @@ ~~ rock blues,jazz @@ ~ @@ ~~ @ ~~~

A@

blues,jazz

C

Fig. 1. Example of construction of a tagged graph from a set of contents M and a set of recommendations V

the centrality measure based on an appropriate facet-dependent subgraph of the recommendation graph. However, online computation of the centrality measure is unfeasible because of its high time complexity, even for small facets with two or three tags. Moreover, the offline computation of the centrality measure for each facet is also unfeasible because the number of possible facets is exponential in the number of tags. Therefore, alternative solutions must be looked for. A simple solution is to use a general ranking computed offline, which is then filtered online for each facet-related query. The use of a single ranking of web pages or users within folksonomies has the disadvantage that the best ranked ones are those having the highest centrality in a global ranking which is facet-independent. In the information retrieval case, this implies that the returned results are ordered in a way which does not take into account the focus on the searched topic. Richardson and Domingos [6] call this problem topic drift. In this paper we propose a solution to the problem of topic drift in faceted rankings which is based on PageRank as centrality measure. Our approach follows a two-step procedure: (i) a ranking for each tag is computed offline on the basis of a tag-related subgraph; (ii) a faceted order is generated online by merging rankings corresponding to all the tags in the facet. The fundamental assumption is that step (i) in this procedure can be computed with an acceptable overhead which depends on the size of the dataset. This hypothesis is validated by two empirical observations. On one hand, in the studied tagged-graphs of Flickr and YouTube, most of the tags are associated to very small subgraphs, while only a small number of tags have large associated subgraphs (see Sect. 3). On the other hand, the mean number of tags per edge is finite and small as explained in Sect. 4.1. The problem then becomes to find a good and efficient algorithm to merge several rankings in step (ii), and we present several alternatives in Sect. 4. The “goodness” of a merging algorithm is measured by comparing its results to those produced by the naive approach of applying the PageRank algorithm on facet-dependent graphs (see Sect. 5). The efficiency of an algorithm is evaluated by means of its time complexity.

Scalable Faceted Ranking in Tagging Systems

285

We concentrate our effort on facets that correspond to the logical conjunction of tags (match-all-tags-queries) because this is the most used logical combination in information retrieval (see Manning et al. [7]). The remaining of the paper is organized as follows. We discuss prior works and their limitations in Sect. 2. In Sect. 3 we explore two real examples of tagged graphs. In particular, we analyze several important characteristics of these graphs, such as the scale-free behavior of the node indegree and assortativeness of the embedded recommendation graph (see Sect. 3.1). The proposed algorithms are introduced in Sect. 4, including an analysis of related scalability issues in Sect. 4.1. We discuss experimental results in Sect. 5 and we conclude with some final remarks and possible directions of future work in Sect. 6.

2 Related Work There is abundant literature on faceted search. For example, basic topic-sensitive PageRank analysis was attempted biasing the general PageRank equation to special subsets of web pages by Al-Saffar and Heileman in [8], and using a predefined set of categories by Haveliwala in [9]. Although encouraging results were obtained in both works, they suffer from the limitation of a fixed number of topics biasing the rankings. The approach of DeLong et al. [10] involves the construction of a larger multigraph using the hyperlink graph with each node corresponding to a pair webpage-concept and each edge to a hyperlink associated with a concept. Although DeLong et al. obtain good ranking results for single-keyword facets, they do not support multi-keyword queries. Query-dependent PageRank calculation was introduced by Richardson and Domingos in [6] to extract a weighted probability per keyword for each webpage. These probabilities are summed up to generate a query-dependent result. They also show that this faceted ranking has, for thousands of keywords, computation and storage requirements that are only approximately 100-200 times greater than that of a single query-independent PageRank. As we show in Sect. 4.1, the offline phase of our facet-dependent ranking algorithms has similar time complexity. Scalability issues were also tackled by Jeh and Widom [11] criticizing offline computation of multiple PageRank vectors for each possible query and preferring another more efficient dynamic programming algorithm for online calculation of the faceted rankings based on offline computation of basis vectors. They found that their algorithm scales well with the size of the biasing page set. In this paper, we propose a different alternative to the problem of faceted ranking. Instead of computing offline the rankings corresponding to all possible facets, our solution requires only the offline computation of a ranking per tag. A faceted ranking is generated by adequately merging the rankings of the corresponding tags. Sect. 4 deals with different approaches to the merging step. Some search engines incorporate tags from collaborative tagging systems. For example, Technorati [12] improves its search engine using tags entered by bloggers and Web page designers, providing also citation count ranking information (called “Authority”) (see Weinman [13]). Tags have also been used to leverage topic-dependent ranking of egos. For example, John and Seligmann [14] explore the application of tagging systems to rank expertise of taggers in enterprise environments. Basically, they use a variation

286

J.I. Orlicki, J.I. Alvarez-Hamelin, and P.I. Fierens

of PageRank biased on the basis of the facet-related tagging activity of each user. Yeung et al. [15] also rank the expertise of taggers. They use a variation of HITS algorithm (see Kleinberg [16]) that takes into account the time when a user applied tags to a given document, giving a higher ranking to those users which “discovered” the content first. Neither of these approaches is scalable because they require a new costly computation (PageRank in the case of [14] and HITS for [15]) for each facet. Hotho et al. [17] modified PageRank (called FolkRank) to work on a tripartite graph corresponding to a folksonomy. The set of nodes of the graph consists of the union of users, tags and resources (contents). All co-occurrences of tags and users, users and resources, tags and resources become edges. Furthermore, edges are weighted with the frequency of appearance of the represented relation. One problem with FolkRank is that it works with symmetric adjacency matrices (corresponding to tags and users, users and resources, tags and resources) and, hence, weights “bounce back” during each iteration of the algorithm1. Shepitsen and Tomuro [18] solved this problem by slightly modifying the adjacency matrices, i.e., they multiplied the weight of the edge connecting a tag t to a content c by the fraction of users who used t for c over all user s who used tag t. In order to allow topic-dependent rankings, Hotho and colleagues introduced a form of topic-biasing on the modified PageRank. The problem with this approach is that the generation of a faceted ranking requires a new computation of the adapted algorithm on the graph for each new facet. Social search (see, e.g., Long et al. [19]) consists in accounting for the social interactions or contributions of other users when answering a query from a given user. In this sense, there has been some work on accounting for the tagging activity of other users. For example, Zanardi and Capria [20] study the problem of ranking tagged documents for query answer presentation in this context. After expanding the query with related tags (based on a cosine similarity metric), they produce a ranking of content based on the relevance of tags associated to each document the similarity of the document taggers to the user who made the query. The similarity between users is based on past tagging activity. Schenkel et al. [21] also take into account the strength of similarity of users tagging activity and the relatedness of tags (although with different metrics), but focus on an efficient algorithm to compute the top-k results of a query. A more complete approach is presented by Symeonidis et al. [22], where users, tags, and resources are modelled in a 3-order tensor and content is recommended to users by performing Latent Semantic Analysis and dimensionality reduction using the Higher Order Singular Value Decomposition technique. In this work, we do not address the problem of user-personalization of queries or recommendations and we focus on producing user independent rankings.

3 Two Tagging Systems: YouTube and Flickr In this section, we present two examples of collaborative tagging systems,YouTube and Flickr, where content is tagged and recommendations are made. Although these systems actually rank content, to our knowledge, no use of graph-based faceted ranking is made. 1

PageRank works with a directed graph and, hence, a node “passes on” weight only on outgoing links.


287

The taxonomy of tagging systems by Marlow et al. in [1] allows us to classify YouTube and Flickr in the following ways: regarding the tagging rights, both are selftagging systems; with respect to the aggregation model, they are set systems; concerning the object-type, they are called non-textual systems; in what respects to the source of material, they are classified as user-contributed; finally, regarding tagging support, while YouTube can be classified as a suggested tagging system, Flickr must be considered a blind tagging system. In our first example the content is multimedia in the form of favorite videos recommended by users. The information was collected from the service YouTube using the public API crawling 185,414 edges and 50,949 nodes in Breadth-First Search (BFS) order starting from the popular user jcl5m that had videos included in the top twenty top rated videos during April 2008. We only considered nodes with indegree greater than one, because they are the relevant nodes to PageRank. From this information, we constructed a full tagged graph G. We have also constructed subgraphs by preserving only those edges that contain a given tag (e.g., G(music) and G(f unny) corresponding to the tags music and f unny, respectively), any tag in a set (e.g., G(music ∨ f unny)) or all tags in a set (e.g., G(music ∧ f unny)). Table 1 presents the number of nodes and edges of each of these graphs. We must note that mandatory categorical tags such as Entertainment, Sports or M usic, always capitalized, were removed in order to include only tags inserted by users. Table 1. Sizes of YouTube, Flickr graphs and some of their subgraphs YouTube G G(music ∨ f unny) G(music) G(f unny) G(music ∧ f unny)

nodes edges Flickr 50,949 185,414 G 4,990 13,662 G(blue ∨ f lower) 2,650 5,046 G(blue) 2,803 6,289 G(f lower) 169 188 G(blue ∧ f lower)

nodes edges 30,974 225,650 5,440 14,273 3,716 6,816 2,771 6,370 280 327

In our second example the content are photos and the recommendations are in the form of favorite photos2 . The information was collected from the service Flickr using the public API crawling 225,650 edges and 30,974 nodes in BFS order starting from the popular user junku-newcleus, where only nodes with indegree greater than one have been considered. The full tagged graph G and the sample subgraphs G(blue ∨ f lower), G(blue), G(f lower) and G(blue ∧ f lower) were constructed. The number of nodes and edges of these graphs are shown in Table 1. 3.1 Analysis of the Recommendation Graphs In this section, we present a summary of the graph analysis of both collaborative tagging systems, YouTube and Flickr. The objective is to disentangle the properties of these systems in order to take advantage by new ranking algorithms. 2

Only the first fifty favorites photos of each user were retrieved.

288

J.I. Orlicki, J.I. Alvarez-Hamelin, and P.I. Fierens 1

1

0.1 0.01 P(vertex indegree)

P(vertex indegree)

0.1

0.001 0.0001 1e-05 YouTube music funny music AND funny music OR funny

1e-06 1e-07 1e-08 0.1

1

0.01 0.001 0.0001

Flickr flower blue blue AND flower blue OR flower

1e-05

10 100 vertex indegree

1000

1e-06

10000

0.1

1

10 vertex indegree

100

1000

10

100

Fig. 2. Binned indegree distribution

1

1 0.1 P(vertex outdegree)

P(vertex outdegree)

0.1 0.01 0.001 0.0001 1e-05 0.01

YouTube music funny music AND funny music OR funny 0.1

0.01 0.001 0.0001

1 vertex outdegree

10

100

1e-05 0.01

Flickr flower blue blue AND flower blue OR flower 0.1

1 vertex outdegree

Fig. 3. Binned outdegree distribution

Graph analysis was made using the tool Network Workbench [23], except for the calculation of PageRank (a C++ stand-alone software). We computed node’s indegree and outdegree distributions, the correlation of indegree of in-neighbors with indegree of nodes, the PageRank distribution, and the number of tags per edge. Figures 2-6 show the parameters for the whole YouTube and Flickr graphs and also for some specific subgraphs. All graph-analytical parameters, except those for small subgraphs like G(music ∧ f unny) were binned and plotted in log-log curves3. Node indegree, in both video and photo graphs, can be bounded by a power-law distribution: P (k) ≈ k −γ , where 2 < γ < 3 (see Figure 2). Random variables modelled by this type of heavy-tailed distributions have a finite mean, but infinite second and higher non-central moments. Furthermore, there is a non-vanishing probability of finding a node with an arbitrary high indegree. Clearly, in any real-world graph, the total number of nodes is a natural upper-bound to the greatest possible indegree. However, experience with Internet related graphs shows that the power-law distribution of the indegree does not change significantly as the graph grows and, hence, the probability of finding a node with an arbitrarily high degree eventually becomes non-zero (for more details see, e.g., Pastor-Satorras and Vespignani [24]). 3

This is the reason why some degree points appear below one (x-axis), because there exist nodes with either indegree or outdegree equal to zero.

Scalable Faceted Ranking in Tagging Systems 10

indegree of in-neighbors

indegree of in-neighbors

10

289

1

0.1

YouTube music funny music AND funny music OR funny

0.01 0.1

1

10

100

1000

10000

vertex indegree

1


0.1 0.1

1

10

100

1000

vertex indegree

Fig. 4. Binned correlation of indegree of in-neighbors with indegree

Since recommendation lists are made by individual users, node outdegree does not show the same kind of scale-free behavior than node indegree. On the contrary, each user recommends only 20 to 30 other users on average (see Figure 3). Moreover, since node outdegree is mostly controlled by individual human users, we do not expect its average to change significantly as the graph grows. Another way of looking at the human origin of tag annotations is by observing Fig. 6, which shows that only few edges have many tags and the number of edges per tag falls off quickly after approximately ten. The correlation of indegree of in-neighbors with node indegree (see Figure 4) indicates the existence of assortative (positive slope) or disassortative behavior (negative slope). Assortativeness is commonly observed in social graphs, where people with many connections relates to people which is also well-connected. Disassortativeness is more common in other kinds of graphs, such as information, technological and biological graphs (see, e.g., Newman [25]). In YouTube’s graph there is no clear correlation (small or no slope), but in Flickr’s graph there is a slight assortativeness indicating a biased preference of nodes with high indegree for nodes with high indegree (see Figure 4). We also computed the PageRank of the sample graphs, removing dangling nodes with indegree 1 and outdegree 0, because most of them correspond to nodes which have not been visited by the crawler (BFS), having the lowest PageRank (a similar approach is taken by Page et al. in [4]). Figure 5 shows that PageRank distributions are also scale-free, i.e., they can be bounded by power law distributions. Note that the power law exponents are very similar for the complete each tagged graph and its subgraphs. This fact suggests that it is possible to get a good faceted ranking by combining the ranking of each tag in the facet.

4 Algorithms for Faceted Ranking Given a set M of tagged content, a set V of favorite recommendations and a tag-set or facet F, the faceted ranking problem consists on computing a ranking of users according to facet F. The naive solution is to find a graph associated to the facet and apply the PageRank algorithm to it. This approach leads to two algorithms, called edgeintersection and node-intersection, which turn out to be too costly for online queries. Indeed, their computation requires the extraction of a subgraph which might be very

290

J.I. Orlicki, J.I. Alvarez-Hamelin, and P.I. Fierens 1

0.01

0.001

1e-04

1e-05 1e-05


0.1 P(vertex PageRank)

0.1 P(vertex PageRank)

1

YouTube music funny music AND funny music OR funny

0.01

0.001

1e-04

1e-04

0.001

0.01

1e-05 1e-05

0.1

vertex PageRank

1e-04

0.001

0.01

0.1

vertex PageRank

Fig. 5. Binned Node PageRank distribution

0.1

YouTube Flickr

P(# tags per edge)

0.01 0.001 0.0001 1e-05 1e-06 1

10

100

1000

# tags per edge

Fig. 6. Binned distribution of number of tags per edge

large in a large graph4 and the calculation of the corresponding PageRank vector. However, they serve as a basis of comparison (i.e., “gold standards”) for more efficient algorithms. We define the following algorithms for match-all-tags-queries. Edge-intersection. Given a set of tags, a ranking is calculated by computing the centrality measure of the subgraph corresponding to the recommendation edges which include all the tags of a certain facet. Node-intersection. Consider the example given in Fig. 1 under the query blues∧rock. According to the edge-intersection algorithm, there is no node in the graph satisfying the query. However, it is reasonable to return node D as a response to such search. In order to take into account this case, we devised another algorithm called nodeintersection. In this case, the union of all edge recommendations per tag is used when computing the PageRank, but only those nodes involved in recommendations for all tags are kept (hence, node-intersection). This is another possible way to obtain a subgraph having only a specific tag-related information. 4

We have observed that as the graph grows the relative frequency of tags usage converges. Similar behavior was observed for particular resources by Golder and Huberman in [26].


291

Single-ranking. A simple online faceted ranking consists of a monolithic ranking of the full graph, without any consideration of tags, which is then filtered to exclude those nodes that are not related to all tags in the facet. Winners-intersection. In this case, as well as in the next two algorithms, the offline phase consists of running PageRank on each tag-related subgraph (such as G(music)) and storing only the best-w ranked users. Given a conjunction-of-tags facet, a new graph is constructed by considering only the w “winners” corresponding to each tag and the edges connecting them. A facet-related ranking is then calculated by means of the PageRank algorithm on this reduced graph. The choice of an adequate number w is application-dependent. For this paper, we have arbitrarily chosen w = 128. We shall show that reasonably good results are obtained even for this small value of w. Probability-product. Let us recall that PageRank is based on the idea of a random websurfer and nodes are ranked according to the estimated stationary probability of such a surfer being at each node at any given time. This basic concept together with the product rule for the joint probability of independent events motivated the probability-product algorithm. This algorithm pre-computes a PageRank for each tag-related subgraph. A ranking associated with a conjunction-of-tags facet is then calculated online by multiplying, on a node-by-node basis, the probabilities corresponding to each tag in the facet. Rank-sum. Consider a recommendation graph G larger than that in Fig. 1 and the query blues ∧ jazz. Assume that the PageRank of the top three nodes in the rankings corresponding the subgraphs G(blues) and G(jazz) are as given in Table 2. Ignoring other nodes, the ranking given by the probability-product rule is a, b and c. However, it may be argued that node b shows a better equilibrium of PageRank values than node a. Intuitively, one may feel inclined to rank b over a given the values in the table. In order to follow this intuition, we devised the rank-sum algorithm which is also intended to avoid topic drift within a queried facet, that is, any tag prevailing over the others. Given a conjunction-of-tags facet, the rank-sum algorithm adds-up the ranking position of nodes in each tag-related subgraph. The corresponding facet-related ranking is calculated by ordering the resulting sums (see Table 2). Table 2. Probability-product vs. rank-sum in an example Node G(blues) G(jazz) Prob.-pr. Rank-sum a 0.75 0.04 0.03 4 0.1 0.1 0.01 3 b c 0.01 0.03 0.003 6 0.01 0.05 0.005 7 d The first two columns show the probability of each node according to PageRank on the corresponding tag-related subgraph.

4.1 Complexity Analysis of the Algorithms As noticed by Langville and Meyer in [5], the number of iterations of PageRank is fixed when both the tolerated error and other parameters are fixed. As each iteration consists of

292


a sparse adjacency matrix multiplication, the time complexity of PageRank is linear on the number of edges of the graph. Since probability-product, rank-sum and winnersintersection algorithms require the offline computation of PageRank for each tag-related subgraph, it is clear that, if the average number of tags per edge is constant or grows very slowly as the graph grows, then the offline phase of these algorithms is scalable, i.e., linear on the number of edges of the complete tagged graph. Fig. 6 shows that the distribution of tags per edge falls quickly, having a mean of ∼ 9 tags per edge for the YouTube tagged graph and ∼ 13 for the Flickr tagged graph. These are not heavy-tailed distributions and, since tags are manually added to each uploaded content, we do not expect the average number of tags per recommendation to increase significantly as the graph grows. In other words, Fig. 6 validates the hypothesis on which the scalability of the offline phase of the probability-product, rank-sum and winners-intersection algorithms is based. The time complexity of edge-intersection algorithm can be estimated by decomposing it into three phases. The first step is to find the graph associated to a facet with k tags, which takes O(k · Etag ), where Etag is the number of edges of the largest tagrelated subgraph. The second step is to compute the PageRank of the resulting graph, taking O(Ef acet ) time, where Ef acet is the number of edges of the graph and, clearly, Ef acet ≤ k · Etag . Finally, the list of Nf acet nodes of the graph must be ordered according to the PageRank, taking O(Nf acet log(Nf acet )). We have found that Nf acet is, in general, much smaller than Ef acet (e.g., see Table 1). For the node-intersection algorithm, the time complexity is the same that in the former case, but Ef acet and Nf acet are usually larger because the graph includes more edges. In the case of single-ranking, the online computation takes O(k · Ntag ) time, where Ntag is the maximum number nodes of a tag-related subgraph. Indeed, assuming that there is an ordered list of nodes related to each tag, its (ordered) intersection can be computed in time that grows linearly with the sum of the lengths of the lists. The winners-intersection, probability-product and rank-sum algorithms have the same time complexity, O(k), because they only consider the top-w ranked users of each tag in the facet and, hence, their complexity depends only on the number of tags (i.e, the number of operations for each tag is fixed by the constant w). This remark is supported because top-w is widely used in the most popular search engines.

5 Experimental Results and Discussion In this section, we compare the behavior of the algorithms presented in Sect. 4. As a basis of comparison we use two algorithms whose online computation is unfeasible, but which are reasonably good standards: edge-intersection and node-intersection. In order to quantify the “distance” between the results given by two different algorithms, we use two ranking similarity measures, OSim (Haveliwala [9]) and KSim (Kendall [27] and Haveliwala [9]). The first measure, OSim(R1 , R2 ) indicates the degree of overlap between the top n elements of rankings R1 and R2 , i.e., the number of common nodes. The second measure, KSim(R1 , R2 ) is a variant of Kendall’s distance that considers the relative orderings, i.e., counts how many inversions are in a fixed top set. In both cases, values closer to 0 mean that the results are not similar and closer to 1 mean the opposite.


293

We have run our algorithms on all facets of tag pairs extracted from the 100 most used tags5 in each of the graphs, YouTube and Flickr. For each tag pair the proposed ranking algorithms (single-ranking, probability-product, rank-sum and winnersintersection) were compared to the gold-standards (edge-intersection and nodeintersection) using OSim and KSim to measure the rankings’ similarity. Tables 3-4 present a summary of the comparisons, where we display averaged similarities for different sizes of top-sets of ranked users. Figures 7 and 8 show a more detailed summary of results for the OSim metric for the proposed algorithms. We do not include gray-scale figures corresponding to the YouTube graph because they are very similar. The x-axis in the figures corresponds to the number of nodes resulting from the basis of comparison algorithm (edge-intersection or node-intersection) and the y-axis to the top number n of nodes used to compute the similarities. The similarity results of OSim were averaged for log-log ranges. Observe that darker tones correspond to values closer to 1, i.e., more similar results. White spaces correspond to cases for which there are no data, e.g., whenever the y coordinate is greater than intersection size. Table 3. YouTube: Comparison of ranking algorithms (average similarity: OSim|KSim) Algorithm

edge-intersection top 8 top 16 top 32 Single-ranking 0.08|0.48 0.10|0.50 0.13|0.51 Winners-inters. 0.06|0.48 0.04|0.49 0.04|0.50 Prob.-product 0.72|0.71 0.80|0.78 0.86|0.83 Rank-sum 0.73|0.72 0.81|0.79 0.86|0.84

node-intersection top 8 top 16 top 32 0.31|0.53 0.34|0.55 0.39|0.56 0.10|0.49 0.08|0.50 0.08|0.51 0.42|0.59 0.52|0.66 0.67|0.74 0.41|0.58 0.50|0.64 0.67|0.72

Table 4. Flickr: Comparison of ranking algorithms (average similarity: OSim|KSim) Algorithm

edge-intersection top 8 top 16 top 32 Single-ranking 0.07|0.48 0.09|0.49 0.11|0.50 Winners-inters. 0.30|0.53 0.23|0.53 0.11|0.51 Prob.-product 0.57|0.63 0.64|0.66 0.72|0.71 0.57|0.63 0.64|0.67 0.72|0.72 Rank-sum

node-intersection top 8 top 16 top 32 0.17|0.50 0.21|0.51 0.27|0.53 0.19|0.50 0.19|0.52 0.18|0.53 0.32|0.55 0.42|0.59 0.56|0.67 0.31|0.53 0.41|0.58 0.56|0.66

As can be appreciated from Tables 3-4 and Figures 7-8, the single-ranking algorithm gave the worst results in most cases. The winners-intersection algorithm, which is based on retaining only the 128 topranked users for each tag, gives worse results than probability-product and rank-sum, even for smaller intersections. This fact is explained by the relevance of a large number of recommendations of low-ranked users when computing the PageRank in both the edge-intersection and the node-intersection cases. Also note that the winnersintersection approach gave better results for Flickr than for YouTube. A possible cause 5

Some tags like you, video or youtube which give no information were removed from the experiment.

294

J.I. Orlicki, J.I. Alvarez-Hamelin, and P.I. Fierens edge−intersection vs. winners−intersection

edge−intersection vs. single−ranking 512

512

1

256

256 128

32

0.5

16 8

Top #

64

64

Top #

128

0.75

32 16 8

0.25

4

4

2

0

2 5

13

42

153

569

2130

5

7996

13

Intersection Size edge−intersection vs. prob.−product 512

153

569

2130

7996

edge−intersection vs. rank−sum 1

256

512 256

128

0.75

128

64

64

32

0.5

16 8

Top #

Top #

42

Intersection Size

32 16 8

0.25

4

4

2

2

0 5

13

42

153

569

2130

7996

5

13

Intersection Size

42

153

569

2130

7996

Intersection Size

Fig. 7. Flickr graph: Average similarity to Edge-intersection node−intersection vs. winners−intersection

node−intersection vs. single−ranking 512

512

1

256

256 128

0.75

128 64

32

0.5

16 8

Top #

Top #

64

32 16 8

0.25

4

4 0

2 92

100

132

252

713

2482

2

9261

92

100

Intersection Size

252

713

2482

9261

Intersection Size

node−intersection vs. prob.−product 512

node−intersection vs. rank−sum 1

256

512 256

128

0.75

128

64

64

32

0.5

16 8

0.25

4

Top #

Top #

132

32 16 8 4

2

0

92

100

132

252

713

Intersection Size

2482

9261

2 92

100

132

252

713

2482

9261

Intersection Size

Fig. 8. Flickr graph: Average similarity to Node-intersection

is the assortativeness of Flickr’s graph (see Sect. 3.1). Indeed, since assortativeness implies that users with many recommendations are preferentially recommended by users with also many recommendations, the relevance of low-ranked users in the computation of the centrality measure is lower.


295

The probability-product and rank-sum algorithms exhibit a similar behavior and clearly outperform other ranking algorithms when considering the similarity to the edge-intersection and the node-intersection standards on both graphs. It is also remarkable that the similarity increases more rapidly (as the size of the ranking increases) in the cases of probability-product and rank-sum algorithms than in the of single ranking (see Tables 3-4 and Figures 7-8). On the contrary, in the case of winners-intersection algorithm, the Osim metric decreases. This is due to the limitation on the number of winners w considered.

6 Conclusions and Future Work We have proposed different algorithms for merging tag-related rankings into complete faceted-rankings of users in collaborative tagging systems. In particular, two of our algorithms, probability-product and rank-sum are feasible for online computation and give results comparable to those of two reasonable, though computationally costly, standards. A prototypic application called Egg-O-Matic is available online [28]. It includes the online method rank-sum to approximate the edge-intersection offline ranking (in a mode called “all tags, same content”), and the probability-product method to approximate the node-intersection ranking (in a mode called “all tags, any content”). A matter of future research is the possibility of reducing the complexity of the offline phase of the proposed algorithms by first clustering the tags. In this case, facets would be represented by a set of clusters of related tags. This work also opens the path for a more complex comparison of reputations, for example by integrating the best positions of a user even if the tags involved are not related (disjunctive queries) in order to summarize the relevance of a user generating content on the web. It is also possible to extend the rank-sum algorithm in Sect. 4 to merge rankings generated from different systems or search engines, producing a ranking from a broader view (cross-system ranking).

References 1. Marlow, C., Naaman, M., Boyd, D., Davis, M.: HT06, tagging paper, taxonomy, Flickr, academic article, to read. In: HYPERTEXT 2006: Proc. of the seventeenth conference on Hypertext and hypermedia, pp. 31–40. ACM Press, New York (2006) 2. YouTube (2008), http://www.youtube.com/ 3. Flickr (2008), http://www.flickr.com/ 4. Page, L., Brin, S., Motwani, R., Winograd, T.: The PageRank Citation Ranking: Bringing Order to the Web. Technical report, Stanford Digital Library Technologies Project (1998) 5. Langville, A.N., Meyer, C.D.: Survey: Deeper Inside PageRank. Internet Mathematics 1(3) (2003) 6. Richardson, M., Domingos, P.: The Intelligent Surfer: Probabilistic Combination of Link and Content Information in PageRank. In: Advances in Neural Information Processing Systems 14. MIT Press, Cambridge (2002)

296


7. Manning, C.D., Raghavan, P., Schütze, H.: Introduction to Information Retrieval. Cambridge University Press, Cambridge (2008) 8. Al-Saffar, S., Heileman, G.: Experimental Bounds on the Usefulness of Personalized and Topic-Sensitive PageRank. In: WI 2007: Proceedings of the IEEE/WIC/ACM International Conference on Web Intelligence, Washington, DC, USA, pp. 671–675. IEEE Computer Society, Los Alamitos (2007) 9. Haveliwala, T.H.: Topic-sensitive PageRank. In: Proc. of the Eleventh International World Wide Web Conference, Honolulu, Hawaii (May 2002) 10. DeLong, C., Mane, S., Srivastava, J.: Concept-Aware Ranking: Teaching an Old Graph New Moves. Icdmw, 80–88 (2006) 11. Jeh, G., Widom, J.: Scaling personalized web search. Technical report, Stanford University (2002) 12. Technorati, http://technorati.com 13. Weinman, J.: A new approach to search. Business Communications Review (October 2007) 14. John, A., Seligmann, D.: Collaborative tagging and expertise in the enterprise. In: 15th International Conference on the World Wide Web (2006) 15. Yeung, C.A., Noll, M.G., Gibbins, N., Meinel, C., Shadbolt, N.: On measuring expertise in collaborative tagging systems. In: Proceedings of the WebSci 2009: Society On-Line, Athens, Greece (March 2009) 16. Kleinberg, J.M.: Authoritative sources in a hyperlinked environment. Journal of the ACM 46(5), 604–632 (1999) 17. Hotho, A., Jäschke, R., Schmitz, C., Stumme, G.: Information Retrieval in Folksonomies: Search and Ranking. In: Sure, Y., Domingue, J. (eds.) ESWC 2006. LNCS, vol. 4011, pp. 411–426. Springer, Heidelberg (2006) 18. Shepitsen, A., Tomuro, N.: Search in social tagging systems using ontological user profiles. In: 3rd International AAAI Conference on Weblogs and Social Media (ICWSM 2009), San Jose, California, USA, Association for the Advancement of Artificial Intelligence (AAAI) (May 2009) 19. Longo, L., Barret, S., Dondio, P.: Toward social search - from explicit to implicit collaboration to predict users’ interests. In: 5th International Conference on Web Information Systems and Technologies (WEBIST 2009), Lisboa, Portugal, March 2009, pp. 693–696 (2009) 20. Zanardi, V., Capra, L.: Social ranking: uncovering relevant content using tag-based recommender systems. In: RecSys 2008: Proceedings of the 2008 ACM conference on Recommender systems, pp. 51–58. ACM, New York (2008) 21. Schenkel, R., Crecelius, T., Kacimi, M., Michel, S., Neumann, T., Parreira, J.X., Weikum, G.: Efficient top-k querying over social-tagging networks. In: SIGIR 2008: Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, pp. 523–530. ACM, New York (2008) 22. Symeonidis, P., Ruxanda, M., Nanopoulos, A., Manolopoulos, Y.: Ternary semantic analysis of social tags for personalized music recommendation. In: Proceedings of 9th International Conference on Music Information Retrieval, Philadelphia, USA (2008) 23. NWB Team, Network Workbench Tool (2006), http://nwb.slis.indiana.edu/ 24. Pastor-Satorras, R., Vespignani, A.: Evolution and structure of the Internet: A statistical physics approach. Cambridge University Press, Cambridge (2004) 25. Newman, M.E.J.: Assortative Mixing in Networks. Phys. Rev. Lett. 89(20), 208701 (2002) 26. Golder, S., Huberman, B.A.: Usage patterns of collaborative tagging systems. Journal of Information Science 32(2), 198–208 (2006) 27. Kendall, M.G.: A New Measure of Rank Correlation. Biometrika 30(1/2), 81–93 (1938) 28. Egg-O-Matic (2008), http://eggomatic.itba.edu.ar/

Answering Definition Questions: Dealing with Data Sparseness in Lexicalised Dependency Trees-Based Language Models Alejandro Figueroa1 and John Atkinson2 1

2

Deutsches Forschungszentrum für Künstliche Intelligenz - DFKI Stuhlsatzenhausweg 3, D - 66123, Saarbrücken, Germany [email protected] http://www.dfki.de/˜figueroa Department of Computer Sciences, Universidad de Concepción, Concepción, Chile [email protected] http://www.inf.udec.cl/ãtkinson

Abstract. A crucial step in the answering process of definition questions, such as “Who is Gordon Brown?”, is the ranking of answer candidates. In definition Question Answering (QA), sentences are normally interpreted as potential answers, and one of the most promising ranking strategies predicates upon Language Models (LMs). However, one of the factors that makes LMs less attractive is the fact that they can suffer from data sparseness, when the training material is insufficient or candidate sentences are too long. This paper analyses two methods, different in nature, for tackling data sparseness head-on: (1) combining LMs learnt from different, but overlapping, training corpora, and (2) selective substitutions grounded upon part-of-speech (POS) taggings. Results show that the first method improves the Mean Average Precision (MAP) of the top-ranked answers, while at the same time, it diminishes the average F-score of the final output. Conversely, the impact of the second approach depends on the test corpus. Keywords: Web Question Answering, Definition Questions, Lexical Dependency Paths, Definition Search, Definition Question Answering, n-gram Language Models, Data Sparseness.

1 Introduction Answering definition questions differ markedly from traditional factoid questions. Factoid questions require a single fact to be returned to the user. Conversely, definition questions consist in a substantially more complex response, which should describe the most relevant aspects of the topic of the question (aka. definiendum or target). By and large, typical definitional QA systems rank candidate sentences taken from multiple documents, select the top-ranked candidates, and consolidates them into a final output afterwards. Broadly speaking, answering a definition query involves a zooming process that comprises the following steps: search, document processing, sentence ranking, summarisation, and many times sense discrimination. J. Cordeiro and J. Filipe (Eds.): WEBIST 2009, LNBIP 45, pp. 297–310, 2010. c Springer-Verlag Berlin Heidelberg 2010

298

A. Figueroa and J. Atkinson

In essence, definition QA systems focus on discovering answers to definition questions by gathering a succinct, diverse and accurate set of factual pieces of information about the definiendum. In the QA jargon, these pieces of information are usually called nuggets. The following sentence, for instance, yields an answer to the query “Who is Gordon Brown?”: Gordon Brown is a British politician and leader of the Labour Party. This illustrative phrase provides the next three nuggets: British politician leader of the Labour Party Specifically, answers to questions asking about politicians can encompass important dates in their lives (birth, marriage and death), their major achievements and any other interesting items such as party membership or leadership. As in our working example, a sentence can certainly carry several nuggets. Our previous work [1] investigated the extent to which descriptive sentences, discovered on the web, can be characterised by some regularities in their lexicalised dependency paths. In particular, in sentences that match some definition patterns such as “is a”, “was the” and “became the”. For example, the next two paths can be taken from our working example: ROOT→is→politician politician→leader→of→Entity The first path acts as a context indicator (politician) signalling the type of definiendum being described, whereas the latter yields content that is very likely to be found across descriptions of several instances of this particular context indicator. One interesting facet of [1] is that they make an inference process that clusters sentences according to a set of context indicators (e.g., song, disease, author) found across Wikipedia articles, and built an n-gram LM for each particular context afterwards. Test sentences are ranked thereafter according to its respective language model. As a result, we found out that regularities in dependency paths proved to be salient indicators of definitions within web documents. In this work, we extend the research presented in [1] by studying two different ways of tackling data sparseness. The first aims at extracting contextual language models from different snapshots of Wikipedia, and the second is aimed specifically at using linguistic abstractions of some pre-determined syntactic classes. The latter strategy shares the same spirit with [2,3]. The organisation of this paper is as follows: section 2 discusses the related approach to definitional question answering, section 3 describes the acquisition process of our training material, our language models and answer extraction strategy, section 4 shows the results obtained by applying our approach and finally section 5 highlights the main conclusions and further work.

Answering Definition Questions: Dealing with Data Sparseness

299

2 Related Work Definition QA systems are usually assessed as a part of the QA track of the Text REtrieval Conference (TREC). Definition QA systems attempt to extract answers from a target collection of news documents: the AQUAINT corpus. In order to discover correct answers to definition questions, definition QA systems extract nuggets from several external specific resources of descriptive information (e.g. online encyclopedia and dictionaries), and must then project them into the corpus afterwards. Generally speaking, this projection strategy relies on two main tasks: 1. Extract external resources containing entries corresponding to the definiendum. 2. Find overlaps between terms in definitions (within the target collection) and terms in the specific resources. In order to extract sentences related to the definiendum, some approaches take advantage of external resources (e.g., W ordN et), online specific resources (e.g., Wikipedia) and Web snippets [4]. These are then used to learn frequencies of words that correlate with the definiendum. Experiments showed that definitional websites greatly improved the performance by leaving few unanswered questions: Wikipedia covered 34 out of the 50 TREC–2003 definition queries, whereas biography.com did it with 23 out of 30 questions regarding people, all together providing answers to 42 queries. These correlated words were then used to form a centroid vector so that sentences can be ranked according to the cosine distance to this vector. The peformance of this class of strategies, however, fell into a steep decline when the definiendum cannot be found in knowledge bases [5,6]. One advantage of this kind of approach is that this ranks candidate answers according to the degree in which their respective words characterise the definiendum, which is the principle known as the Distributional Hypothesis[7,8]. However, the approach fails to capture sentences containing the correct answers with words having low correlation with the definiendum. This in turn causes a less diverse output and so decrease the coverage. In addition, taking into account only semantic relationships is not sufficient for ranking answer candidates: the co-occurrence of the definiendum with learnt words across candidate sentences does not necessarily guarantee that they are syntactically dependent. An example of this can be seen in the following sentence about “British Prime Minister Gordon Brown”: According to the Iraqi Prime Minister’s office, Gordon Brown was reluctant to signal the withdrawal of British troops. In order to deal with this issue, [9] introduced a method that extended centroid vectors to include word dependencies which are learnt from the 350 most frequent stemmed co-occurring terms taken from the best 500 snippets retrieved by Google. These snippets were fetched by expanding the original query by a set of highly co-occurring terms. These terms co-occur with the definiendum in sentences obtained by submitting the original query plus some task specific clues, (e.g.,“biography”). Nevertheless, having a threshold of 350 frequent words is more suitable for technical or accurate definiendums (e.g., “SchadenFreude”), than for ambiguous or biographical

300


definiendums (e.g., “Alexander Hamilton”) which need more words to describe many writings of their several facets. These 350 words are then used for building an ordered centroid vector by retaining their original order within the sentences. To illustrate this, consider the following example[9]: Today’s Highlight in History: On November 14, 1900, Aaron Copland, one of America’s leading 20th century composers, was born in New York City. The corresponding ordered centroid vectors become the words: November 14 1900 Aaron Copland America composer born New York City. These words are then used for training statistical LMs and ranking candidate answers. Bi-term language models were observed to significantly improve the quality of the extracted answers showing that the flexibility and relative position of lexical terms capture shallow information about their syntactic relation [10]. Our previous work [1] ranks answer candidates according to n-grams (n=5) LMs built on top of our contextual models, contrary to the trend of definition QA systems that solely utilise articles in knowledge bases corresponding to the definiendum. Our context models assist in reducing the narrow coverage provided by knowledge bases for many definiendums. These n-grams models are learnt from sequences of consecutive lexicalised terms linked in dependency trees representing the sentences in each context. The contribution of this work is an analysis of two different strategies for reducing data sparseness, when using dependency path-based LMs, and thus enhancing the ranking strategy proposed in our previous work[1].

3 Our Approach In the following section, the three main parts of our method introduced in [1] are pined down: definition sentence clustering, learning language models and ranking answer candidates. 3.1 Grouping Sentences According to their Contexts Indicators In our approach, context indicators and their corresponding dependency paths are learnt from abstracts extracted from Wikipedia1 . Specifically, contextual n-gram language models are constructed on top of these contextual dependency paths in order to recognise sentences conveying definitions. Unlike other QA systems [11], definition patterns are applied at the surface level [12] and key named entities are identified using namedentity recognisers (NER)2 . Preprocessed sentences are then parsed by a lexicalised dependency parser3 , in which obtained lexical trees are used for building a treebank of lexicalised definition sentences. 1 2 3

We used the snapshot supplied in January 2008. http://nlp.stanford.edu/software/CRF-NER.shtml http://nlp.stanford.edu/software/lex-parser.shtml


301

Table 1. Some examples of grouped sentences according to their context indicators Author CONCEPT was a Entity author of children’s books . CONCEPT was a Entity author and illustrator of children’s books . CONCEPT is the author of two novels : Entity and Entity . CONCEPT is an accomplished author . CONCEPT is an Entity science fiction author and fantasy author . CONCEPT is a contemporary Entity author of detective fiction . Player CONCEPT is a Entity football player , who plays as a midfielder for Entity . CONCEPT is a Entity former ice hockey player . CONCEPT is a Entity jazz trumpet player . Disease CONCEPT is a fungal disease that affects a wide range of plants . CONCEPT is a disease of plants , mostly trees , caused by fungi . CONCEPT is a chronic progressive disease for which there is no cure . Song CONCEPT is a Entity song by Entity taken from the Entity album Entity . CONCEPT is a Entity song performed by the Entity band Entity . CONCEPT is a pop song written by Entity and Entity , produced by Entity for Entity’s first album Entity .

The treebank contains trees for 1, 900, 642 different sentences in which each entity is replaced with a placeholder. This placeholder allows us to reduce the sparseness of the data and to obtain more reliable frequency counts. For the same reason, we left unconsidered different categories of entities and capitalised adjectives were mapped to the same placeholder. From the sentences in the treebank, our method automatically identifies potential Context Indicators. These involve words that signal what it is being defined or what type of descriptive information is being expressed. Context indicators are recognised by walking through the dependency tree starting from the root node. Since only sentences matching definition patterns are taken into account, there are some regularities that are useful to find the respective context indicator. The root node itself is sometimes a context indicator, however, whenever the root node is a word contained in the surface patterns (e.g. is, was and are), the method walks down the hierarchy. In the case that the root has several children, the first child is interpreted as the context indicator. Note that the method must sometimes goes down one more level in the tree depending of the expression holding the relationship between nodes (i.e., “part/kind/sort/type/class/first of ”). Furthermore, the used lexical parser outputs trees that meet the projection constrain, hence the order of the sentence is preserved. Overall, 45, 698 different context indicators were obtained during parsing. Candidate sentences are grouped according to the obtained context indicators (see table 1). Consequently, highly-frequent directed dependency paths within a particular context are hypothesised to significantly characterise the meaning when describing an

302


instance of the corresponding context indicator. This is strongly based on the extended distributional hypothesis [13] which states that if two paths tend to occur in similar contexts, their meanings tend to be similar. In addition, the relationship between two entities in a sentence is almost exclusively concentrated in the shortest path between the two entities of the undirected version of the dependency graph [14]. Hence one entity can be interpreted as the definiendum, and the other can be any entity within the sentence. Therefore, paths linking a particular type of definiendum with a class of entity relevant to its type will be highly frequent in the context (e. g., politician→ leader→ of → ENTITY). Enriching our Treebank with POS Information. This treebank is built on top of our previous one, but it accounts for selective substitutions. Contrary to [2,3], the following syntactic categories are mapped into a placeholder indicating the respective class: DT, CC, PRP, PRP$,CD, RB, FW, MD, PDT, PRP, RBR, RBS and SYM. Additionally, the following verbs, which are normally used for discovering definitions, are mapped into a placeholder: is, are, was, were, become, becomes, became, had, has and have. The aim of these mappings is amalgamating the probability mass of similar paths, when computing our language models. For example, the following illustrative paths: was→politician→a is→politician→the is→politician→an These paths are merged into: VERB→politician→DT. The idea behind this amalgamation is supported by the fact that some descriptive phrases, including “Concept was an American politician...” and “Concept is a British politician...”, share some common structure that is very likely to convey definitions. Consolidating thus their probability mass is reasonable, because it boosts the chances of paths not seen in the training data, but that share some syntactic structures. 3.2 Building Contextual Language Models For each context, all directed paths containing two to five nodes are extracted. Longer paths are not taken into consideration as they are likely to indicate weaker syntactic/semantic relations. Directions are mainly considered because relevant syntactical information regarding word order is missed when going up the dependency tree. Otherwise, undirected graphs would lead to a significant increase in the number of paths as it might go from any node to any other node. Some illustrative directed paths obtained from the treebank for the context indicator politician are shown below: politician→affiliated→with→Entity politician→considered→ally→of→Entity politician→during→time→the politician→head→of→state→of politician→leader→of→opposition politician→member→of→chamber politician→served→during proclaimed→on→Entity


303

From the obtained dependency paths, an n-gram statistical language model (n = 5) for each context was built in order to estimate the most relevant dependency path. The probability of a dependency path dp in a context cs defines the likely dependency links that compose the path in the context cs , with each link probability conditional on the last n − 1 linked words: P (dp | cs ) ≈

l

i−1 P (wi | cs , wi−n+1 )

(1)

i=1 i−1 ) is the probability that word wi is linked with the previous Where P (wi | cs , wi−n+1 word wi−1 after seeing the dependency path wi−n+1 . . . wi−1 . In simple words, the likelihood that wi is a dependent node of wi−1 , and wi−2 is the head of wi−1 , and so forth. i−1 The probabilities P (wi | cs , wi−n+1 ) are usually computed by computing the Maximum Likelihood Estimate: i−1 PML (wi | cs , wi−n+1 )=

i count(cs , wi−n+1 ) i−1 count(cs , wi−n+1 )

Some illustrative examples are as follows: PML (Entity | politician, politician → af f iliated → with) = count(politician,politician→af f iliated→with→Entity) count(politician,politician→af f iliated→with)

= 0.875

PML (of | politician, politician → activist → leader) = count(politician,politician→activist→leader→of ) count(politician,politician→activist→leader)

= 0.1667

PML (Entity | politician, proclaimed → on) = count(politician,proclaimed→on→Entity) count(politician,proclaimed→on)

=1

i ) can frequently be greater than However, in our case, the word count c(cs , wi−n+1 i−1 c(cs , wi−n+1 ). For example, in the following definition sentence:

CONCEPT is a band formed in Entity in Entity.

The word “formed” is the head of two “in”, hence the denominator of P (wi | i−1 i−1 cs , wi−n+1 ) is the number of times wi−1 is the head of a word (after seeing wi−n+1 ). In order to illustrate how selective substitutions assist in consolidating the probability mass according to some syntactic similarities at the category level, consider the next example: PM L (a | is → politician) PM L (an | is → politician) PM L (the | is → politician) PM L (DT | is → politician)

= 0.164557 = 0.0379747 = 0.00632911 + = 0.20886081

304


The obtained 5-gram language model is smoothed by interpolating with shorter dependency paths as follows: i−1 Pinterp (wi | cs , wi−n+1 )= i−1 )+(1 − λcs ,wi−1 λcs ,wi−1 P (wi | cs , wi−n+1 i−n+1

i−n+1

i−1 )Pinterp (wi | cs , wi−n+2 )

The probability of a path P (dp | cs ) is accordingly computed by accounting for the recursive interpolated probabilities instead of raw P s. Note also that λcs ,wi−1 is comi−n+1 puted for each context cs as described in [15]. Finally, a sentence S is ranked according to its likelihood of being a definition as follows: P (dp | cs ) (2) rank(S) = P (cs ) ∀dp∈S

In order to avoid counting redundant dependency paths, only paths ending with a dependent/leave node are taken into account, whereas duplicate paths are discarded. Combining Context Models from Different Wikipedia Snapshots. Another way of tackling data sparseness is amalgamating LMs learnt from different Wikipedia4 snapshots. Following the same procedure described in section 3, two additional treebanks of dependency trees were built, and hence two extra n-gram language models were generated. Accordingly, the ranking of a candidate sentence S was computed by making allowances for average values of P (cs ) and P (dp | cs ). B B 1 1 ¯ Pb (cs ) ∗ Pb (dp | cs ) rank(S) = B B b=1

∀dp∈S

(3)

b=1

In other words, we carry out experiments by systematically increasing the size of our language models in three steps B = 1, 2, 3. In the previous equation, Pb (cs ) is the probability of the context cs in the treebank b, and by the same token, Pb (dp | cs ) is the probability of finding the dependency path dp in the context cs in the treebank b. ¯ ¯ Accordingly, rank(S) is the final ranking value, and when B = 1, rank(S) is equal to rank(S), which resembles our original system presented in [1]. 3.3 Extracting Candidate Answers Our model extracts answers to definition questions from web snippets. Thus, sentences matching definition patterns at the surface level are pre-processed5 and parsed in order to get the corresponding lexicalised dependency trees. Given a set of test sentences/dependency trees extracted from the snippets, our approach discovers answers to definition question by iteratively selecting sentences. 4

5

For this purpose, we took advantage of two additional snapshots. One corresponding to early 2007 and the other to October 2008. The former yielded 1,549,615 different descriptive sentences, whereas the latter 1,063,452. http://www.comp.nus.edu.sg/˜qiul/NLPTools/JavaRAP.html


305

Algorithm 1. Answer Extractor. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

φ = ∅; indHis = getContextIndicatorsHistogram(T ); for highest to lowest frequent ι ∈ indHis do while true do nextSS = null; forall tt ∈ T do if ind(ti )==ι then rank = rank(ti ,φ); if nextSS == null or rank > rank(nextSS) then nextSS = ti ; end end end if nextSS == null or rank(nextSS) ≤ 0.005) then break; end print nextSS; addPaths(nextSS,φ); end end

The general strategy for this iterative selection task can be seen in algorithm 1 whose input is the set of dependency paths (T ). This first initialises a set φ which keeps the dependency paths belonging to previously selected sentences (line 1). Next, context indicators for each candidate sentence are extracted so as to build an histogram indHist (line 2). Since highly frequent context indicators show more reliable potential senses, the method favours candidate sentences based on their context indicator frequencies (line 3). Sentences matching the current context indicator are ranked according to equation 3 (lines 7 and 8). However, only paths dp in ti − φ are taken into consideration, while computing equation 3. Sentences are thus ranked according to their novel paths respecting to previously selected sentences, while at the same time, sentences carrying redundant information decrease their ranking value systematically. Highest ranked sentences are selected after each iteration (line 9-11), and their corresponding dependency paths are added to φ (line 18). If the highest ranked sentence meets the halting conditions, the extraction task finishes. Halting conditions ensure that no more sentences to be selected are left and no more candidate sentences containing novel descriptive information are chosen. In this answer extraction approach, candidate sentences become less relevant as long as their overlap with all previously selected sentences become larger. Unlike other approaches which control the overlap at the word level [9,11], our basic unit is a dependency path, that is, a group of related words. Thus, the method favours novel content, while at the same time, it makes a global check of the redundant content. Furthermore, the use of paths instead of words as unit ensures that different instances of a word, that contribute with different descriptive content, will be accounted accordingly.

306


4 Experiments and Results In order to assess our initial hypothesis, a prototype of our model was built and assessed by using 189 definition questions taken from TREC 2003-2004-2005 tracks. Since our model extracts answers from the web, these TREC dataset were only used as a reference for question sets. For each question, the best 300 web snippets were retrieved by using MSN Search along with the search strategy sketched in [16]. These snippets were manually inspected in order to create a gold standard. It is important to note that there was no descriptive information for 11 questions corresponding to the TREC 2005 data set. For experiment purposes, we utilised O UR S YSTEM presented in [1] as a baseline, and all systems were provided with the same set of snippets. 4.1 Evaluation Metrics In this work, two different metrics were allowed: F-score and MAP. Following the current trends of assessments in definition QA systems, the standard F-score [17] was used as follows: Fβ =

(β 2 + 1) × P × R β2 × P + R

This takes advantage of a factor β for balancing the length of the output and the amount of relevant and diverse information it carries. In early TREC tracks, β was set to 5, but as it was inclined to favour long responses, it was later decreased to 3. The Precision (P) and the Recall (R) metrics were computed as described in the most recent evaluation by using uniform weights for the nuggets [18] in the gold standard obtained in the previous section. One of the disadvantages of the F-score is that it does not account for the order of the nuggets within the output. This is a key issue whenever definition QA systems output sentences as it is also necessary to assess the ranking order, that is, determine whether the highest positions of the ranking contain descriptive information. In order to deal with this, the Mean Average Precision (MAP) was accounted for. Despite an important number of MAP [19], those measuring the precision at fixed low levels of results were used, in particular, MAP-1 and MAP-5 sentences. Hence this precision is referred to as precision at k: mj |Q| 1 1 M AP (Q) = P recision at k | Q | j=1 mj k=1

Here Q is a question set (e.g., TREC 2003), and mj is the number of ranking sentences in the output. Accordingly, mj is truncated to one or five, when computing MAP-1 and MAP-5, respectively. This metric was selected because its ability to show how good the results are on the first positions of the ranking. Simply put, for a given question set Q, MAP-1 shows the fraction of questions that ranked a valid definition on the top.


307

Table 2. Results for TREC question sets

Size Recall Precision F(3) Score Recall Precision F(3) Score Recall Precision F(3) Score Recall Precision F(3) Score

TREC 2003 TREC 2004 TREC 2005 50 64 (64)/75 O UR S YSTEM 0.57 0.50 0.42 0.39 0.40 0.29 0.53 0.47 0.38 O UR S YSTEM II 0.46 0.46 0.42 0.32 0.38 0.29 0.43 0.44 0.38 O UR S YSTEM III 0.46 0.44 0.41 0.31 0.34 0.28 0.43 0.42 0.37 O UR S YSTEM POS 0.56 0.47 0.48 0.24 0.22 0.24 0.48 0.41 0.42

4.2 Experimental Results Table 2 highlights the obtained results. In this table, O UR S YSTEM II (B = 2) and O UR S YSTEM III (B = 3) correspond to our systems accounting for two and three treebanks, respectively. Overall, the performance was decreased in terms of recall and precision. The gradual decrease in recall may be due to the fact that averaging the two/three treebanks diminishes the value of low frequent paths, because they are not (significantly) present in all the treebanks. Therefore, whenever they match a sentence, the sentence is less likely to score high enough to surpass the experimental threshold (line 14 in algorithm 1). Here, we envisage using a strategy of inter-treebank smoothing that takes away probability mass of the high frequent paths (across treebanks) and distribute it across paths low in frequency in one of the treebanks, but absent in one of the others. The reason to the steady decrease in precision is two-fold: – The decrease in recall brings about a decrease in the allowance, – And more important, the algorithm selected misleading or redundant definitions in replacement for the definitions matched by the original system, but missed by these two extensions. This outcome homologates the fact that ranking answer candidates, according to some highly frequent words across articles about the definiendum taken from several knowledge bases, would bring about an improvement in terms of ranking, but a detriment to coverage and to the diversity of the final output. On the other hand, highly frequent paths obtain more robust estimates as they are very likely to be in all treebanks, having a positive effect in the ranking. Table 3 highlights this effect. In all question sets, O UR S YSTEM II and O UR S YSTEM III

308

A. Figueroa and J. Atkinson Table 3. Mean Average Precision (MAP)

MAP-1 MAP-5 MAP-1 MAP-5 MAP-1 MAP-5

O UR S YSTEM O UR S YSTEM O UR S YSTEM O UR S YSTEM II III POS TREC 2003 0.82 0.88 0.88 0.88 0.82 0.88 0.87 0.88 TREC 2004 0.88 0.92 0.94 0.91 0.82 0.88 0.87 0.87 TREC 2005 0.79 0.81 0.82 0.73 0.77 0.78 0.78 0.71

outperformed our original system. The increase in MAP values suggests that combining estimates from different snapshots of Wikipedia assists in determining more prominent and genuine paths. These estimates along with the preference given by algorithm 1 to these paths bring about the improvement in the final ranking, that is more genuine pieces of descriptive information tend to be conveyed in the highest positions of the rank. In general, our three improvements bettered the ranking with respect to O UR S YS TEM, however our experiments did not draw a clear distinction which is the best in this aspect. For our POS based method, results in table 3 indicates an increase with respect to the original system for two datasets, but a decrease in the case of the TREC 2005 questions set. Unlike the two previous question sets, abstracting some syntactic categories leaded to some spurious sentences to rank higher. More interestingly, table 2 emphasises the marked decline in terms of F(3)-score for two datasets, while at the same time, it remarks a substantial improvement for the TREC 2005 question set. In comparison with the results achieved by the original system sketched in table 2. This enhancement is particularly due to the increase in recall so that the amalgamation of dependency paths was useful to identify a higher number of genuine descriptive sentences. On the other hand, the addition of POS tags assisted in matching more misleading and spurious sentences, and consequently it worsened the performance in terms of precision. This might also explain the decrease in the MAP value for this question set. Given these observations, our treebanks (without POS information) were observed to cover less descriptive sentences contained in this question set. In the TREC 2003-2004 question sets, the decline might be due to the fact that different original paths are still necessary to recognise several sentences.

5 Conclusions In this work, we studied two different approaches to tackle data sparseness, when utilising n-grams language models built on top of dependency paths for ranking definition questions. Results show that the precision of the top-ranked answers can be boosted by combining contextual language models learnt from different snapshots of Wikipedia. However,


309

this can have a negative impact in the precision and the diversity of the entire output. Additionally, our experiments showed that the success of abstractions based on POS taggings depends largely upon the target corpus. Nevertheless, a study of the effects of additional features in our languages models can be done as a further work. A study similar in spirit to [20]. Acknowledgements. This work was partially supported by a research grant from the German Federal Ministry of Education, Science, Research and Technology (BMBF) to the DFKI project HyLaP (FKZ: 01 IW F02) and the EC- funded project QALL-ME FP6 IST-033860 (http://qallme.fbk.eu). Additionally, this research was partially sponsored by the National Council for Scientific and Technological Research (FONDECYT, Chile) under grant number 1070714.

References 1. Figueroa, A., Atkinson, J.: Using Dependency Paths For Answering Definition Questions on The Web. In: 5th International Conference on Web Information Systems and Technologies, pp. 643–650 (2009) 2. Cui, H., Kan, M.Y., Chua, T.S.: Unsupervised Learning of Soft Patterns for Definitional Question Answering. In: Proceedings of the Thirteenth World Wide Web Conference (WWW 2004), pp. 90–99 (2004) 3. Cui, H., Kan, M.Y., Chua, T.S.: Soft pattern matching models for definitional question answering. ACM Trans. Inf. Syst. 25 (2007) 4. Cui, T., Kan, M., Xiao, J.: A comparative study on sentence retrieval for definitional question answering. In: SIGIR Workshop on Information Retrieval for Question Answering (IR4QA), pp. 383–390 (2004) 5. Han, K., Song, Y., Rim, H.: Probabilistic model for definitional question answering. In: Proceedings of SIGIR 2006, pp. 212–219 (2006) 6. Zhang, Z., Zhou, Y., Huang, X., Wu, L.: Answering Definition Questions Using Web Knowledge Bases. In: Dale, R., Wong, K.-F., Su, J., Kwong, O.Y. (eds.) IJCNLP 2005. LNCS (LNAI), vol. 3651, pp. 498–506. Springer, Heidelberg (2005) 7. Firth, J.R.: A synopsis of linguistic theory 1930-1955. Studies in Linguistic Analysis, 1–32 (1957) 8. Harris, Z.: Distributional Structure. Distributional structure. Word 10(23), 146–162 (1954) 9. Chen, Y., Zhon, M., Wang, S.: Reranking Answers for Definitional QA Using Language Modeling. In: Coling/ACL 2006, pp. 1081–1088 (2006) 10. Belkin, M., Goldsmith, J.: Using eigenvectors of the bigram graph to infer grammatical features and categories. In: Proceedings of the Morphology/Phonology Learning Workshop of ACL 2002 (2002) 11. Hildebrandt, W., Katz, B., Lin, J.: Answering Definition Questions Using Multiple Knowledge Sources. In: Proceedings of HLT-NAACL, pp. 49–56 (2004) 12. Soubbotin, M.M.: Patterns of Potential Answer Expressions as Clues to the Right Answers. In: Proceedings of the TREC-10 Conference (2001) 13. Lin, D., Pantel, P.: Discovery of Inference Rules for Question Answering. Journal of Natural Language Engineering 7, 343–360 (2001) 14. Bunescu, R., Mooney, R.J.: A Shortest Path Dependency Kernel for Relation Extraction. In: Proceedings of HLT/EMNLP (2005) 15. Chen, S., Goodman, J.: An Empirical Study of Smoothing Techniques for Language Modeling. In: Proceedings of the 34th Annual Meeting of the ACL, pp. 310–318 (1996)

310


16. Figueroa, A., Neumann, G.: A Multilingual Framework for Searching Definitions on Web Snippets. In: Hertzberg, J., Beetz, M., Englert, R. (eds.) KI 2007. LNCS (LNAI), vol. 4667, pp. 144–159. Springer, Heidelberg (2007) 17. Voorhees, E.M.: Evaluating Answers to Definition Questions. In: HLT-NAACL, pp. 109–111 (2003) 18. Lin, J., Demner-Fushman, D.: Will pyramids built of nuggets topple over? In: Proceedings of the main conference on HTL/NAACL, pp. 383–390 (2006) 19. Manning, C.D., Raghavan, P., Schütze, H.: Introduction to Information Retrieval. Cambridge University Press, Cambridge (2008) 20. Surdeanu, M., Ciaramita, M., Zaragoza, H.: Learning to Rank Answers on Large Online QA Collections. In: Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics (ACL 2008), pp. 719–727 (2008)

Author Index

Abel, Fabian 113, 142 Afzal, Muhammad Tanvir 61 Albert, Dietrich 73 Alvarez-Hamelin, J. Ignacio 283 Andreatta, Alexandre 157 Atkinson, John 297

Li, Li 44 Lichtnow, Daniel 229 Linek, Stephanie B. 73 Loh, Stanley 229 Lorenzi, Fabiana 229

Bessler, Sandford 30 Boella, Guido 3 Boer, Viktor de 86 Bopp, Matthias 73

Metaxas, Panagiotis Takis Minotti, Mattia 17 Miyata, Takamichi 256 Motta, Eduardo 157 Myller, Niko 198

Chou, Wu 44 Conrad, Stefan

Olmedilla, Daniel 142 Orlicki, José I. 283

270

De Coi, Juri Luca 142 Dinsoreanu, Mihaela 99 Faron-Zucker, Catherine 128 Fierens, Pablo I. 283 Figueroa, Alejandro 297 Gabner, Rene 30 Granada, Roger 229 Gutowska, Anna 212 Happenhofer, Marco 30 Henze, Nicola 113, 142 Hollink, Vera 86 Inazumi, Yasuhiro

256

Janneck, Monique

185

Palazzo Moreira de Oliveira, José Piancastelli, Giulio 17 Pop, Cristina 99 Remondino, Marco 3 Ricci, Alessandro 17 Sakai, Yoshinori 256 Salomie, Ioan 99 Sasaki, Akira 256 Schwarz, Daniel 73 Siqueira, Sean 157 Sloane, Andrew 212 Suciu, Sorin 99 Thanh, Nhan Le

Kobayashi, Aki 256 Koesling, Arne Wolf 142 Korhonen, Ari 198 Krause, Daniel 113, 142 Laakso, Mikko-Jussi 198 Le, Hieu Quang 270 Li, Jianqiang 242

170

128

van Someren, Maarten

86

Wives, Leandro Krug

229

Yurchyshyna, Anastasiya Zarli, Alain 128 Zeiß, Joachim 30 Zhao, Yu 242

128

229

Web Information Systems and Technologies: 5th International Conference, WEBIST 2009, Lisbon, Portugal, March 23-26, 2009, Revised Selected Papers (Lecture Notes in Business Information Processing)

Web Information Systems and Technologies: Third International Conference, WEBIST 2007, Barcelona, Spain, March 3-6, 2007, Revised Selected Papers (Lecture Notes in Business Information Processing)

Web Information Systems and Technologies: International Conferences, Webist 2005 and Webist 2006 Revised Selected Papers

Information Security and Cryptology: 5th International Conference, Inscrypt 2009, Beijing, China, December 12-15, 2009. Revised Selected Papers

Enterprise Information Systems: 12th International Conference, ICEIS 2010, Funchal-Madeira, Portugal, June 8-12, 2010, Revised Selected Papers (Lecture Notes in Business Information Processing)

Enterprise, Business-Process and Information Systems Modeling: 10th International Workshop, BPMDS 2009, and 14th International Conference, EMMSAD 2009, ... Notes in Business Information Processing)

Business Information Systems: 12th International Conference, BIS 2009, Poznan, Poland, April 27-29, 2009, Proceedings (Lecture Notes in Business Information Processing)

Next Generation Information Technologies and Systems: 7th International Conference, NGITS 2009 Haifa, Israel, June 16-18, 2009 Revised Selected Papers ... Applications, incl. Internet Web, and HCI)

Next Generation Information Technologies and Systems: 7th International Conference, NGITS 2009 Haifa, Israel, June 16-18, 2009 Revised Selected Papers ... Applications, incl. Internet Web, and HCI)

Advances in Enterprise Engineering III: 5th International Workshop, CIAO! 2009, and 5th International Workshop, EOMAS 2009, held at CAiSE 2009, Amsterdam, ... Notes in Business Information Processing)

Enterprise Information Systems: 11th International Conference, ICEIS 2009, Milan, Italy, May 6-10, 2009, Proceedings (Lecture Notes in Business Information Processing)

Information Security and Cryptology - ICISC 2009: 12th International Conference, Seoul, Korea, December 2-4. 2009. Revised Selected Papers (Lecture Notes in Computer Science Security and Cryptology)

Business Process Management Workshops: BPM 2009 International Workshops, Ulm, Germany, September 7, 2009, Revised Papers (Lecture Notes in Business Information Processing, LNBIP 43)

Computational Intelligence: Revised and Selected Papers of the International Joint Conference IJCCI 2009 held in Funchal-Madeira, Portugal, October 2009

Enterprise Information Systems: 10th International Conference, ICEIS 2008, Barcelona, Spain, June 12-16, 2008, Revised Selected Papers (Lecture Notes in Business Information Processing)

Enterprise Information Systems: 8th International Conference, ICEIS 2006, Paphos, Cyprus, May 23-27, 2006, Revised Selected Papers (Lecture Notes in Business Information Processing)

Information Theoretic Security: 4th International Conference, ICITS 2009, Shizuoka, Japan, December 3-6, 2009. Revised Selected Papers (Lecture Notes in Computer Science Security and Cryptology)

Information Systems: Modeling, Development, and Integration: Third International United Information Systems Conference, UNISCON 2009, Sydney, Australia, ... Notes in Business Information Processing)

E-Technologies: Innovation in an Open World: 4th International Conference, MCETECH 2009, Ottawa, Canada, May 4-6, 2009, Proceedings (Lecture Notes in Business Information Processing)

Technologies for Business Information Systems

Information Systems and e-Business Technologies: 2nd International United Information Systems Conference, UNISCON 2008, Klagenfurt, Austria, April 22-25, ... Notes in Business Information Processing)

Advances in Web and Network Technologies and Information Management: AP Web WAIM 2009 International Workshops: WCMT 2009, RTBI 2009, DBIR-ENQOIR 2009, ... Applications, incl. Internet Web, and HCI)

Programming Multi-Agent Systems: 7th International Workshop, ProMAS 2009, Budapest, Hungary, May10-15, 2009.Revised Selected Papers (Lecture Notes in ... Lecture Notes in Artificial Intelligence)

Web Information Systems Engineering - WISE 2009: 10th International Conference, Poznen, Poland, October 5-7, 2009, Proceedings (Lecture Notes in ... Applications, incl. Internet Web, and HCI)

Personal Satellite Services: International Conference, PSATS 2009, Rome, Italy, March 18-19, 2009, Revised Selected Papers (Lecture Notes of the Institute ... and Telecommunications Engineering)

Information Security Applications: 10th International Workshop, WISA 2009, Busan, Korea, August 25-27, 2009, Revised Selected Papers (Lecture Notes in Computer Science Security and Cryptology)

Information Hiding: 11th International Workshop, IH 2009, Darmstadt, Germany, June 8-10, 2009, Revised Selected Papers (Lecture Notes in Computer Science Security and Cryptology)

Information Hiding: 11th International Workshop, IH 2009, Darmstadt, Germany, June 8-10, 2009, Revised Selected Papers (Lecture Notes in Computer Science Security and Cryptology)

Renewables Information 2009

Coal Information 2009

Sensor Systems and Software: First International ICST Conference, S-CUBE 2009, Pisa, Italy, September 7-9, 2009, Revised Selected Papers (Lecture Notes ... and Telecommunications Engineering)

Web Information Systems and Technologies: 5th International Conference, WEBIST 2009, Lisbon, Portugal, March 23-26, 2009, Revised Selected Papers (Lecture Notes in Business Information Processing)

Web Information Systems and Technologies: Third International Conference, WEBIST 2007, Barcelona, Spain, March 3-6, 2007, Revised Selected Papers (Lecture Notes in Business Information Processing)

Web Information Systems and Technologies: International Conferences, Webist 2005 and Webist 2006 Revised Selected Papers

Information Security and Cryptology: 5th International Conference, Inscrypt 2009, Beijing, China, December 12-15, 2009. Revised Selected Papers

Enterprise Information Systems: 12th International Conference, ICEIS 2010, Funchal-Madeira, Portugal, June 8-12, 2010, Revised Selected Papers (Lecture Notes in Business Information Processing)

Enterprise, Business-Process and Information Systems Modeling: 10th International Workshop, BPMDS 2009, and 14th International Conference, EMMSAD 2009, ... Notes in Business Information Processing)

Business Information Systems: 12th International Conference, BIS 2009, Poznan, Poland, April 27-29, 2009, Proceedings (Lecture Notes in Business Information Processing)

Next Generation Information Technologies and Systems: 7th International Conference, NGITS 2009 Haifa, Israel, June 16-18, 2009 Revised Selected Papers ... Applications, incl. Internet Web, and HCI)

Next Generation Information Technologies and Systems: 7th International Conference, NGITS 2009 Haifa, Israel, June 16-18, 2009 Revised Selected Papers ... Applications, incl. Internet Web, and HCI)

Advances in Enterprise Engineering III: 5th International Workshop, CIAO! 2009, and 5th International Workshop, EOMAS 2009, held at CAiSE 2009, Amsterdam, ... Notes in Business Information Processing)

Enterprise Information Systems: 11th International Conference, ICEIS 2009, Milan, Italy, May 6-10, 2009, Proceedings (Lecture Notes in Business Information Processing)

Information Security and Cryptology - ICISC 2009: 12th International Conference, Seoul, Korea, December 2-4. 2009. Revised Selected Papers (Lecture Notes in Computer Science Security and Cryptology)

Business Process Management Workshops: BPM 2009 International Workshops, Ulm, Germany, September 7, 2009, Revised Papers (Lecture Notes in Business Information Processing, LNBIP 43)

Computational Intelligence: Revised and Selected Papers of the International Joint Conference IJCCI 2009 held in Funchal-Madeira, Portugal, October 2009

Enterprise Information Systems: 10th International Conference, ICEIS 2008, Barcelona, Spain, June 12-16, 2008, Revised Selected Papers (Lecture Notes in Business Information Processing)

Enterprise Information Systems: 8th International Conference, ICEIS 2006, Paphos, Cyprus, May 23-27, 2006, Revised Selected Papers (Lecture Notes in Business Information Processing)

Information Theoretic Security: 4th International Conference, ICITS 2009, Shizuoka, Japan, December 3-6, 2009. Revised Selected Papers (Lecture Notes in Computer Science Security and Cryptology)

Information Systems: Modeling, Development, and Integration: Third International United Information Systems Conference, UNISCON 2009, Sydney, Australia, ... Notes in Business Information Processing)

E-Technologies: Innovation in an Open World: 4th International Conference, MCETECH 2009, Ottawa, Canada, May 4-6, 2009, Proceedings (Lecture Notes in Business Information Processing)

Technologies for Business Information Systems

Information Systems and e-Business Technologies: 2nd International United Information Systems Conference, UNISCON 2008, Klagenfurt, Austria, April 22-25, ... Notes in Business Information Processing)

Advances in Web and Network Technologies and Information Management: AP Web WAIM 2009 International Workshops: WCMT 2009, RTBI 2009, DBIR-ENQOIR 2009, ... Applications, incl. Internet Web, and HCI)

Programming Multi-Agent Systems: 7th International Workshop, ProMAS 2009, Budapest, Hungary, May10-15, 2009.Revised Selected Papers (Lecture Notes in ... Lecture Notes in Artificial Intelligence)

Web Information Systems Engineering - WISE 2009: 10th International Conference, Poznen, Poland, October 5-7, 2009, Proceedings (Lecture Notes in ... Applications, incl. Internet Web, and HCI)

Personal Satellite Services: International Conference, PSATS 2009, Rome, Italy, March 18-19, 2009, Revised Selected Papers (Lecture Notes of the Institute ... and Telecommunications Engineering)

Information Security Applications: 10th International Workshop, WISA 2009, Busan, Korea, August 25-27, 2009, Revised Selected Papers (Lecture Notes in Computer Science Security and Cryptology)

Information Hiding: 11th International Workshop, IH 2009, Darmstadt, Germany, June 8-10, 2009, Revised Selected Papers (Lecture Notes in Computer Science Security and Cryptology)

Information Hiding: 11th International Workshop, IH 2009, Darmstadt, Germany, June 8-10, 2009, Revised Selected Papers (Lecture Notes in Computer Science Security and Cryptology)

Renewables Information 2009

Coal Information 2009

Sensor Systems and Software: First International ICST Conference, S-CUBE 2009, Pisa, Italy, September 7-9, 2009, Revised Selected Papers (Lecture Notes ... and Telecommunications Engineering)

Recommend Documents