Objects, Components, Models and Patterns: 47th International Conference, TOOLS EUROPE 2009, Zurich, Switzerland, June 29-July 3, 2009, Proceedings (Lecture Notes in Business Information Processing)

Lecture Notes in Business Information Processing Series Editors Wil van der Aalst Eindhoven Technical University, The Netherlands John Mylopoulos University of Trento, Italy Norman M. Sadeh Carnegie Mellon University, Pittsburgh, PA, USA Michael J. Shaw University of Illinois, Urbana-Champaign, IL, USA Clemens Szyperski Microsoft Research, Redmond, WA, USA

33

Manuel Oriol Bertrand Meyer (Eds.)

Objects, Components, Models and Patterns 47th International Conference, TOOLS EUROPE 2009 Zurich, Switzerland, June 29-July 3, 2009 Proceedings

13

Volume Editors Manuel Oriol University of York Department of Computer Science Heslington, York, YO10 5DD, UK E-mail: [email protected] Bertrand Meyer ETH Zurich Zurich, Switzerland E-mail: [email protected]

Library of Congress Control Number: 2009929321 ACM Computing Classification (1998): D.2, H.1, K.6 ISSN ISBN-10 ISBN-13

1865-1348 3-642-02570-6 Springer Berlin Heidelberg New York 978-3-642-02570-9 Springer Berlin Heidelberg New York

This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. springer.com © Springer-Verlag Berlin Heidelberg 2009 Printed in Germany Typesetting: Camera-ready by author, data conversion by Scientific Publishing Services, Chennai, India Printed on acid-free paper SPIN: 12705689 06/3180 543210

Preface

For the past 20 years the TOOLS conferences have continuously spread new technologies in the world of object-orientation, component technology and software engineering in general. They constitute a particularly important forum for software researchers and practitioners to present work that relies on producing tools. This year’s TOOLS continued the tradition and presented a strong program that included original contributions in all major fields of the object-oriented paradigm. As in TOOLS 2008, the concept of model occupied a considerable place; but other topics such as reflection, aspects, modelling languages, debugging, and virtual machines design were also strongly represented. While most conferences had a decrease in the number of submitted papers, TOOLS made the choice of quality and decided to lower its acceptance rate to 25%, as a fixed maximum announced in the Call for Papers. Out of 67 submitted contributions, the reviewing process led to the selection of 17 as full papers. For the first time, two short papers were also accepted for presentation at the conference. The 47th edition of the conference could not have seen the light without the numerous helpers that made it happen. The quality of the work of the Program Committee (PC) deserves particular acknowledgement; not only did the PC members work hard to produce helpful reviews on time, sometimes with the help of external referees, but extensive discussions followed the initial recommendations, leading to an excellent conference program and extensive advice to the authors. The Organizing Committee worked hard to schedule the events and make the conference a pleasant event. We take the opportunity to thank everyone who made this event happen in a way or another. April 2009

Manuel Oriol Bertrand Meyer

Organization

TOOLS EUROPE 2009 was organized by the department of Computer Science, ETH Zurich.

Chairpersons Conference Chair Program Chair Workshops Chair

Bertrand Meyer (ETH Zurich, Switzerland) Manuel Oriol (University of York, UK) Alexandre Bergel (INRIA Lille, France) Johan Fabry (University of Chile, Chile) Philippe Lahire (Universit´e de Nice, France) Marcus Denker (University of Bern, Switzerland) Ilinca Ciupa, Claudia G¨ unthart, Marco Piccioni

Publicity Chair

Organizing Committee

Program Committee Patrick Albert Balbir S. Barn Mike Barnett Claude R. Baudoin Bernhard Beckert Alexandre Bergel Judith Bishop Phil Brooke Cristiano Calcagno Ana Cavalcanti Dave Clarke Bernard Coulette Jing Dong St´ephane Ducasse Gregor Engels Patrick Eugster Manuel Fahndrich Jos´e Luiz Fiadeiro Michael Franz Judit Nyekyne Gaizler Benoˆıt Garbinato Carlo Ghezzi Tudor Girba Martin Glinz

ILOG Fellow, France Middlesex University, UK Microsoft Research, USA Schlumberger, France University of Koblenz, Germany INRIA Lille, France University of Pretoria, South Africa University of Teesside, UK Imperial College, UK University of York, UK KU Leuven, Belgium IRIT/University of Toulouse, France University of Texas at Dallas, USA INRIA Lille, France University of Paderborn, Germany Purdue University, USA Microsoft Research, USA University of Leceister, UK University of California, Irvine, USA Pazmany University, Hungary University of Lausanne, Switzerland Politecnico di Milano, Italy University of Bern, Switzerland University of Zurich, Switzerland

VIII

Organization

Martin Gogolla Jeff Gray Pedro Guerreiro Joseph Kiniry Ivan Kurtev Ralf Laemmel Philippe Lahire Mingshu Li Dragos Manolescu Erik Meijer Peter M¨ uller Jonathan Ostroff Richard Paige Marc Pantel Frederic Peschanski Alfonso Pierantonio Alexander Pretschner Bran Selic Anatoly Shalyto Friedrich Steimann Perdita Stevens ´ Eric Tanter Dave Thomas Laurence Tratt Antonio Vallecillo Roel Wuyts Amiram Yehudai Andreas Zeller

University of Bremen, Germany University of Alabama at Birmingham, UK Universidade do Algarve, Portugal University College Dublin, Ireland University of Twente, The Netherlands University of Koblenz-Landau, Germany Universit´e de Nice, France Chinese Academy of Sciences, China Microsoft, USA Microsoft, USA ETH Zurich, Switzerland York University, Canada University of York, UK IRIT/University of Toulouse, France LIP6 - UPMC Paris Universitas, France Universit` a degli Studi dell’Aquila, Italy TU Kaiserslautern/Fraunhofer IESE, Germany IBM, USA St. Petersburg State University ITMO, Russia Fernuniversit¨ at in Hagen, Germany University of Edinburgh, UK University of Chile, Chile Bedara, Canada Bournemouth University, UK Universidad de M´ alaga, Spain IMEC, Belgium Tel Aviv University, Israel Saarland University, Germany

External Referees Nicolas Arni-Bloch Martin Assmann Robert Atkey Jan-Christopher Bals Stephanie Balzer Hakim Belhaouari Manuel F. Bertoa Joel-Alexis Bialkiewicz Domenico Bianculli Luca Cavallaro Antonio Cicchetti Ilinca Ciupa Marcio Cornelio

Zekai Demirezen Simon Denier Marcus Denker Mauro Luigi Drago Martin Ernst Baris G¨ uldali Simon Hudon Ethan Jackson Mikolas Janota R´obert Kitlei Tam´as Kozsik Jannik Laval David Makalsky

Alessandro Margara Marjan Mernik Dimiter Milushev Tu Peng Fiona Polack Damien Pollet Jose Proenca Lukas Renggli Leila Ribeiro Jose E. Rivera Markku Sakkinen Tibor Somogyi Clifford Sule

Organization

Yu Sun Peter Swinburne Robert Tairas M´ at´e Tejfel

Faraz Torshizi Sarvani Vakkalanka Andreas W¨ ubbeke Ye Yang

Yajing Zhao

IX

Table of Contents

Invited Presentations On Realizing a Framework for Self-tuning Mappings . . . . . . . . . . . . . . . . . . Manuel Wimmer, Martina Seidl, Petra Brosch, Horst Kargl, and Gerti Kappel

1

Programming Models for Concurrency and Real-Time (Abstract) . . . . . . Jan Vitek

17

Reflection and Aspects CIF: A Framework for Managing Integrity in Aspect-Oriented Composition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Andrew Camilleri, Geoffrey Coulson, and Lynne Blair A Diagrammatic Formalisation of MOF-Based Modelling Languages . . . . Adrian Rutle, Alessandro Rossini, Yngve Lamo, and Uwe Wolter Designing Design Constraints in the UML Using Join Point Designation Diagrams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Vanessa Stricker, Stefan Hanenberg, and Dominik Stein Stream-Based Dynamic Compilation for Object-Oriented Languages . . . . Michael Bebenita, Mason Chang, Andreas Gal, and Michael Franz

18

37

57

77

Models Algebraic Semantics of OCL-Constrained Metamodel Specifications . . . . Artur Boronat and Jos´e Meseguer Specifying and Composing Concerns Expressed in Domain-Specific Modeling Languages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Aram Hovsepyan, Stefan Van Baelen, Yolande Berbers, and Wouter Joosen

96

116

Early Crosscutting Metrics as Predictors of Software Instability . . . . . . . . Jos´e M. Conejero, Eduardo Figueiredo, Alessandro Garcia, Juan Hern´ andez, and Elena Jurado

136

Extensibility in Model-Based Business Process Engines . . . . . . . . . . . . . . . Mario S´ anchez, Camilo Jim´enez, Jorge Villalobos, and Dirk Deridder

157

XII

Table of Contents

Theory Guaranteeing Syntactic Correctness for All Product Line Variants: A Language-Independent Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Christian K¨ astner, Sven Apel, Salvador Trujillo, Martin Kuhlemann, and Don Batory A Sound and Complete Program Logic for Eiffel . . . . . . . . . . . . . . . . . . . . . Martin Nordio, Cristiano Calcagno, Peter M¨ uller, and Bertrand Meyer

175

195

Components A Coding Framework for Functional Adaptation of Coarse-Grained Components in Extensible EJB Servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Olivier Caron, Bernard Carr´e, Alexis Muller, and Gilles Vanwormhoudt A Leasing Model to Deal with Partial Failures in Mobile Ad Hoc Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Elisa Gonzalez Boix, Tom Van Cutsem, Jorge Vallejos, Wolfgang De Meuter, and Theo D’Hondt

215

231

Monitoring Reusing and Composing Tests with Traits . . . . . . . . . . . . . . . . . . . . . . . . . . . St´ephane Ducasse, Damien Pollet, Alexandre Bergel, and Damien Cassou

252

Flow-Centric, Back-in-Time Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Adrian Lienhard, Julien Fierz, and Oscar Nierstrasz

272

A Classification Framework for Pointcut Languages in Runtime Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Karl Klose and Klaus Ostermann

289

Systems Generation Fast Simulation Techniques for Design Space Exploration . . . . . . . . . . . . . Daniel Knorreck, Ludovic Apvrille, and Renaud Pacalet PyGirl: Generating Whole-System VMs from High-Level Prototypes Using PyPy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Camillo Bruni and Toon Verwaest

308

328

Short Papers Using Grammarware Languages to Define Operational Semantics of Modelled Languages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Daniel A. Sadilek and Guido Wachsmuth

348

Table of Contents

Automatic Generation of Integrated Formal Models Corresponding to UML System Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Helen Treharne, Edward Turner, Richard F. Paige, and Dimitrios S. Kolovos Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

XIII

357

369

On Realizing a Framework for Self-tuning Mappings Manuel Wimmer, Martina Seidl, Petra Brosch , Horst Kargl, and Gerti Kappel Vienna University of Technology, Austria [email protected]

Abstract. Realizing information exchange is a frequently recurring challenge in nearly every domain of computer science. Although languages, formalisms, and storage formats may differ in various engineering areas, the common task is bridging schema heterogeneities in order to transform their instances. Hence, a generic solution for realizing information exchange is needed. Conventional techniques often fail, because alignments found by matching tools cannot be executed automatically by transformation tools. In this paper we present the Smart Matching approach, a successful combination of matching techniques and transformation techniques, extended with self-tuning capabilities. With the Smart Matching approach, complete and correct executable mappings are found in a test-driven manner.

1

Introduction

In this paper we present a self-tuning approach for information integration. Our approach—the Smart Matching approach—allows the derivation of high quality executable mappings for different kinds of schemas from small, simple examples defined by the user. Seamless information exchange between different sources is an important task in nearly every engineering area of computer science. This ubiquitous challenge recurs in various application domains starting from the exchange of data between databases over the exchange of ontology instances between semantic web services to the exchange of complete models between modeling tools. Although every engineering area has its own languages, formalisms, and storage formats, the problem is always the same. Two heterogeneous schemas have to be bridged in order to transform instances of the one schema to instances of the other. Thus, a generic information integration solution for the use in different application domains is highly valuable.  

This work has been partly funded by FFG under grant FIT-IT-819584 and FWF under grant P21374-N13. Funding for this research was provided by the fFORTE WIT - Women in Technology Program of the Vienna University of Technology, and the Austrian Federal Ministry of Science and Research.

M. Oriol and B. Meyer (Eds.): TOOLS EUROPE 2009, LNBIP 33, pp. 1–16, 2009. c Springer-Verlag Berlin Heidelberg 2009 

2

M. Wimmer et al.

On the one hand, matching techniques for automatically finding semantic correspondences between schemas have been developed in the context of information integration. On the other hand, transformation techniques have been established for automatically transforming data conforming to a schema A to data conforming to a schema B in a semantic preserving way. However, the combination of matching approaches and transformation approaches is accompanied with several difficulties due to the huge gap between detected alignments, mappings, and executable transformation code. For bridging this gap, two problems have to be solved. First, state-of-the-art matching techniques do not produce executable transformation code. Second, for the automatic execution of the transformation, correctness and completeness of the found alignments are indispensable. A solution for the first problem is presented in [7], where reusable mapping operators for the automatic transformation of models are introduced. A solution for the second problem is conceptually proposed in [8], where we discussed how to improve current integration techniques by the application of self-tuning methods in a test-driven manner resulting in the Smart Matching approach. Based on this work, we implemented the self-tuning matching framework Smart Matcher. In this paper we describe how the Smart Matcher increases the completeness and correctness of alignments based on predefined examples. In particular, we present the realization of the Fitness Function for self-evaluation and the Mapping Engine for self-adaptation which are the basis of the self-tuning. This paper is organized as follows. In Section 2 we present the Smart Matching approach at a glance and the running example of this paper. Sections 3 and 4 elaborate on the two core components of the Smart Matcher namely the Fitness Function and the Mapping Engine, respectively. In Section 5 we present an evaluation of our approach. Related work is discussed in Section 6. In Section 7 we conclude and give an outlook to future work.

2

Smart Matching at a Glance

In this section, we present the Smart Matcher and its underlying conceptual architecture at a glance. The Smart Matching method combines existing integration techniques, i.e., matching and transformation approaches, by introducing a dedicated mapping layer for bridging the gap between alignments, mappings, and executable transformation code as described in [1]. Feedback-driven self-tuning improves the quality of the mappings in an iterative way [9]. For the implementation of the self-tuning capabilities, the quality of mappings has to be evaluated. Therefore, we adopt test-driven development methods from software engineering [3] for developing integration solutions. The Smart Matcher’s architecture is illustrated in Fig. 1. Basically, the application of the Smart Matcher comprises eight phases which are described in the following. 1. Develop example instances. Like in test-driven software development, we specify an expected outcome for a specific input, i.e., instances for input and output schemas representing the same real world example are developed. Those instances are necessary to verify the mappings identified in later phases in order

On Realizing a Framework for Self-tuning Mappings 2 InitialMatcher

MappingLanguage Repository

CARMappingOperators CARMappingOperators pp g Operators p CAR CARMappingOperators Mapping CARMappingOperators

e.g.,COMA++,FOAM,... COMA FOAM

3 Alignments

Mapping Engine MappingEngine load

3

7

6

feedback

load

MappingModel 4

Schema

read

TransformationEngine

SchemaB

SchemaA instanceOf

Instance

write

instanceOf

instanceOf

actual

target

Bankomat Bankomat ExampleX Example X 1

1

describe

cas cashdispens cash dispens´ Example ExampleY Y´

cashdispenr pY cashdispr cash dispr Example ExampleY describe

Real World Real World Real World Bankomat Example Bankomat

FitnessFunction B d BasedonDiffͲOperator Diff O

5

Fig. 1. The Smart Matching approach at a glance

to follow a feedback-driven integration method. The user defined input instance is later called LHS (left-hand side) instance, the output instance is named target RHS (right-hand side) instance. This is the only time during the whole Smart Matching process where human input is necessary. 2. Generate initial alignments. Existing matching tools create initial alignments which may serve as input for the Smart Matcher. Every tool which supports the INRIA alignment format [4] may be used. This phase is optional, i.e., the Smart Matcher is also able to start from an empty set of alignments. 3. Interpret initial alignments. The alignments produced in Phase 2 are transformed into an initial mapping model. This task is performed by the Mapping Engine. 4. Transform instances. In this phase, the Transformation Engine executes the initial mapping model and thereby transforms LHS instances into actual RHS instances. 5. Calculate differences. The Fitness Function compares the actual and the target RHS instances by means of their contained objects, values, and links. The differences between the instances are collected in a Diff-Model, which expresses the quality of the mappings between the source schema and the target schema. 6. Propagate differences. The differences, i.e., missing and wrong objects, values, and links, identified by the Fitness Function are propagated back to the Mapping Engine. The feedback is based on the assumption that schema elements are not appropriately mapped if differences are calculated for their instances. 7. Interpret differences and adjust the mapping model. The Mapping Engine analyzes the feedback and adapts the mapping model between source and target

4

M. Wimmer et al.

schema by searching for and applying appropriate mapping operators. Depending on the types of schemas to be integrated and the used mapping language, different kinds of mapping strategies may be used. 8. Iteration. Now Phase 4 and Phase 5 are repeated. If the comparison in Phase 5 does not detect any differences or if a certain threshold value is reached, the process stops. Otherwise, the Smart Matcher’s Mapping Engine adapts the mapping model based on the identified differences and starts a new iteration by returning to Phase 4. In the following, we elaborate on the Fitness Function necessary for selfevaluation in Phase 5 and the Mapping Engine necessary in particular for the self-adaptation in Phase 7. Therefore, the next sections comprise detailed descriptions on the design rationale and implementation of these two components based on the example depicted in Fig. 2 (a). The schema on the LHS assigns a discount with a height to each customer who is characterized by name and famName. The schema on the RHS describes similar information, namely the rebate height of a person. Only the family a person belongs to is modeled by its own class. A correct mapping between those schemas should detect that the attribute famName ought to be mapped to the class family what is not given by the mapping shown in Fig. 2 (b) which has been produced from the automatically computed alignments (cf. Fig. 2 (a)).

3

A Fitness Function for Mapping Models

The aim of the Fitness Function is to provide feedback on the quality of the mapping models to the Mapping Engine. In the transformation scenario, the quality of a mapping model indicates the correctness of the transformation of the input model to the corresponding output model. Using a test-driven approach where input and output pairs (e.g., cf. Fig. 2 (c)) are given, the generated output model is compared with a given target output model (cf. Fig. 2 (d)). The smaller the differences between the actual output model and the target output model are, the better the quality of the mapping model is. This raises the question how to compute expressive differences between actual and target instance models. When using object-oriented schemas which typically consist of classes, attributes, and references, differences on the instance level appear between objects, values, and links. In the following, we describe our heuristic-based comparison method which is sufficient to compute the necessary feedback for the mapping engine without applying expensive comparison algorithms. Comparing Objects. In general, objects have a unique ID within an instance model (cf. Fig. 2 (c), e.g., f1:Family). However, when we want to compare actual instance models with target instance models which are independently created, one may not assume that two objects with the same ID are equivalent. This is because often IDs are arbitrarily assigned, e.g., based on the order the objects are instantiated. Therefore, we cannot rely on exact ID-based

On Realizing a Framework for Self-tuning Mappings LHSSchema

RHSSchema

SchemaMatching Output

Discount height ggets

~0.7

height

~0.6

gets

Discount height

Rebate height

A2A

gets

gets R2R

Person

~1.0

Person

label

Customer

~0.45

LHSInstances

c1:Customer name =homer famName =simpson

Family label

name famName

(c)

p1:Person

((d)) RHSActual RHS Actual Instances

p1:Person f1:Family

label =simpson

RHS Target Instances RHSTargetInstances

p1:Person label =homer

label =simpson

p2:Person

name =marge famName =simpson

label =marge

c3:Customer

p3:Person

f2:Family

p3:Person

name =ned famName =flenders

label =ned

label =flenders

label =flenders

height =10%

Family label

c2:Customer

d1:Discount

belongsTo g A2A

(a) (b)

RHSTargetInstances g

label =homer

label

C2C

Customer

belongsTo

name famName

RHSSchema

InitialMappingModel C2C

Rebate

~0.9

LHSSchema

p2:Person label =simpson

r1:Rebate height =10%

5

r1:Rebate height =10%

p2:Person

f1:Family label =simpson

label =marge

p3:Person

f2:Family

label =ned ned

label =flenders = flenders

r1:Rebate height =10%

Fig. 2. Running example: (a) alignments, (b) initial mapping model (c) user-defined test instances, (d) actual vs. target instances for initial mapping model.

comparison approaches, instead we have to apply a heuristic. The weakest condition, which is necessary but not sufficient, for verifying that two instance models are equivalent is to count the objects of a particular class in each instance model and compare the resulting numbers, i.e., compute the cardinalities for a given class. Consequently, having the same amount of objects for each class of the RHS schema (abbr. with rhsSchema) in the actual instance model (abbr. with actual) and in the target model (abbr. with target), is an indication that the applied mappings between the classes of the schemas ought to be correct and complete. This consideration leads to the following condition. Condition 1 := forall classes c in rhsSchema | actual.allObjects(c).size() = target.allObjects(c).size()

Because Condition 1 is not sufficient, it is possible that in certain cases the amount of objects is the same for a given class but the mappings are not correctly defined. Assume that for example a LHS class has a mapping to a RHS class, which is instantiated twice, but actually the LHS class should be mapped to another RHS class which is also instantiated twice. However, due to the fact that the RHS classes have both the same amount of instances, no difference can be determined when considering Condition 1 only. Thus, the mapping between the classes are falsely interpreted as correct. To be sure that two objects are equal, a deep comparison is necessary, meaning that attribute values and links of the objects under consideration have to match. Using deep comparison, this kind of false mappings between classes can be detected.

6

M. Wimmer et al.

Example. Fig. 2 (d) illustrates on the LHS the actual instances generated by the interpretation of the automatically computed alignments and on the RHS the given target instances. When Condition 1 is evaluated on these two models, the cardinalities of the classes Rebate and Person are the same for the actual model and for the target model. However, the cardinality of the class Family is obviously not equivalent for both instance models. The Family objects are totally missing in the actual model. Comparing Values. Having the same cardinalities for a specific class is a prerequisite for verifying that actual and target instance models are the same. To be sure that two objects, one in the actual and the other in the target instance model, are equal, these objects must contain exactly the same attribute values. Therefore, for each attribute of the RHS schema two sets of values are computed. The first one comprises all values of a specific attribute for the actual model and the second one does the same for the target model. In case these two sets comprise the same elements, the mapping for the RHS attribute seems to be correct. Otherwise the attribute mapping is assumed to be incorrect or missing. Condition 2 := forall attributes a in rhsSchema | actual.allValues(a) = target.allValues(a)

Example. When Condition 2 is evaluated for our running example, one can easily see that for the attribute Rebate.height Condition 2 holds. However, for the attribute Person.label the computed sets are totally different, namely for the actual instances the resulting set is {simpson, simpson, flenders} which has no match with the computed set for the target instances {homer, marge, ned}. Furthermore, Condition 2 does not hold for Family.label, because in the actual model there are no Family objects that may hold such values. Comparing Links. The last indication that the actual and target instance models are equal, and consequently that the mapping model between the schemas is correct, is that the structure of the instance models is the same, i.e., the links between objects must be equivalent. In particular, equal objects must have the same amount of incoming and outgoing links, which is in general hard and expensive to prove. In order to provide a fast comparison method for links, we decided only to check if the references of the RHS schema have the same amount of instantiated links. Condition 3 := forall references r in rhsSchema | actual.allLinks(r) = target.allLinks(r)

Example. Considering our running example, Condition 3 holds for the reference Person gets Rebate. However, for the reference Person memberOf Family Condition 3 does not hold. In the actual model we have no instantiations of this reference, thus the result of the allLinks operation is the empty set, whereas in the target model the operation allLinks produces a set with three entries. Preparing the feedback for the mapping engine. After evaluating these three conditions for all RHS schema elements, the feedback for the Mapping

On Realizing a Framework for Self-tuning Mappings

7

Engine is defined as the set of all RHS schema elements which do not fulfill the aforementioned conditions. For the adaptation of the mapping model done by the Mapping Engine, we decided that it is currently sufficient to know the schema elements for which differences on the instance level have been found. Therefore, these schema elements are collected for the feedback only, and not the actual differences, i.e., objects, values, and links. Concerning our running example, the following set is computed as feedback for the Mapping Engine {Family, Person.label, Family.label, Person partOf Family}.

4

A Feedback-Aware Mapping Engine

After the Fitness Function has computed the feedback, the Mapping Engine interprets it and adapts the current mapping model to improve its quality. In order to adapt mapping models in an appropriate and efficient way, we make use of two kinds of strategies. First, local strategies are needed to find out if a given mapping operator may be applied appropriately for a set of given schema elements. Second, a global strategy is needed to guide the mapping process in general and in particular for orchestrating the local strategies. 4.1

Local Strategies

As depicted in the Smart Matcher architecture (cf. Fig. 1), the LHS instance models are transformed into RHS actual instance models. Subsequently, the difference between the actual and target instance models is computed on the RHS which is then propagated back to the Mapping Engine. Consequently, the Smart Matcher always transforms form left to right but adapts the mapping models from right to left. For adaptation purposes, the Mapping Engine has to apply new mappings which can also replace existing ones. However, before a new mapping is applied, it has to be ensured that the selected mapping operator is applicable for a given set of schema elements, i.e., the mapping model should be executable and the execution should result in consistent RHS instances. In particular, each mapping operator needs a specific LHS structure and RHS structure as well as already applied mapping operators as context. Based on these constraints, local strategies can be derived for determining if a given mapping operator can be applied in a particular situation. Therefore, we have extended our mapping operators with a isApplicable method which checks for given LHS and RHS schema elements if a mapping can be applied in the current state of the mapping model. For determining the implementation of the isApplicable methods, information inherent in the mapping language is needed. This means, the local strategies can be derived from the abstract syntax and static semantic constraints of the mapping operators which have to be assured to generate correct and executable mappings. If the isApplicable method returns true for given LHS and RHS elements, the mapping operator can be applied. Thus, with the help of the local strategies, we are able to automatically build mappings between the schemas. However, until now, we have not defined how a mapping model is actually build. Therefore, in addition to the local strategies, a global strategy is needed which guides the overall mapping process.

8

M. Wimmer et al.

4.2

Global Strategies

Before a global strategy can be derived, one first has to think about how mapping models are manually developed. Generally, mapping models are developed in a three-step process. First, some prototypical mappings are established. Second, these mappings have to be interpreted, i.e., in the transformation scenario the RHS instances are generated from the LHS instances. Third, the output of the transformation has to be verified in order to be sure that the mappings were correctly developed. For automation purposes, these steps have to be covered by the global strategy. However, what aggravates the derivation of the global strategy is that there a several variations how to proceed in each step. For example, one user may prefer to build several mappings between classes in the first place before mappings between attributes and references are created. Whereas, another user may prefer to find only one mapping between two classes and then tries to find all attribute mappings for these two classes. These examples already show that the derivation of a global strategy is not as predetermined as the derivation of the local strategies. In particular, the following variation points for global strategies exist. – Sequence of mapping operator applications. The first decision that has to be made is the sequence in which the different types of mapping operators are applied. One possibility may be first finding structural commonalities between the schemas by applying symmetric operators, followed by bridging structural heterogeneities by applying asymmetric operators. – Depth-first vs. breadth-first search. Additionally to the application sequence of mapping operators, the global strategy may decide to find all C2C mappings first and afterwards A2A and R2R mappings are explored (breadth-first ) or find iteratively one C2C mapping and for this mapping all possible A2A and R2R mappings are searched (depth-first ). – All-at-once vs. incremental evaluation. Finally, it has to be decided, when and how often mappings are evaluated. The global strategy may find several possible mappings and evaluate them all together (all-at-once) or evaluate each found mapping individually (incremental ). The all-at-once approach is similar to the waterfall model in software engineering, with its disadvantage of spending much effort when working in the wrong direction. This can be avoided with the incremental approach; however, much more evaluation steps are necessary. Because there are different ways to define a global strategy, we decided to use a state-machine based implementation approach for global strategies. The major advantage of this approach is to separate basic functionality, such as transforming models, computing differences between models, as well as searching and applying mappings, from the global strategy guiding the overall mapping process. This allows us to reconfigure the global strategy faster than compared to reconfigurations on source code level, which is especially needed to empirically analyze various configurations of the aforementioned variation points. In the following, we present the most efficient configuration explored in our empirical

On Realizing a Framework for Self-tuning Mappings a

9

b [termination condition]

Init

[[not RHS_containClass && not RHS_containAtt && RHS_containRef]]

entry / computeRHS() entry / computeLHS()

[not RHS_containClass && RHS_containAtt]

[RHS_containClass]

Select RHS Class [inC2C_BL && not inA2C_BL] inA2C BL] [not inC2C_BL]

Search 4 LHS Class [not mapFound]

Search 4 LHS Attribute

do / isApplicable()

entry / isApplicable()

[mapFound] /apply()

[mapFound] /apply()

Evaluation entry / transform() entry / evaluate()

[cMapEval2True && cMapHasAtt]

Select RHS Att

[cMapEval2False || not cMapHasFeat]

a

[not inA2A_BL] inA2A BL]

[not mapFound]

Search 4 LHS Att [cMapHasAtt]

do / isApplicable() [[mapound] d] /apply()

Evaluation entry / transform() entry / evaluate() [not cMapIsApproved && not cMapHasFeat] /setCMap_eval2False a

[ not cMapHasAtt && cMapHasRef]

[cMapIsApproved &&

b not cMapHasFeat]

Select RHS Ref [not inR2R_BL]

[not mapFound]

Search 4 LHS Ref [cMapHasRef]

do / isApplicable() [mapFound] /apply

Evaluation entry / transform() entry / evaluate() [not cMapIsApproved && not cMapHasRef] /setCMap_eval2False() a

[cMapIsApproved &&

b not cMapHasRef]

Fig. 3. Incremental depth-first global strategy implemented as UML state machine

experiments. Fig. 3 illustrates this global strategy as an UML state diagram, which is a depth-first and incremental strategy regarding to the aforementioned variation points. The depth-first search also determines the application sequence of the mapping operators. Please note that only those parts of the global strategy are illustrated which are relevant for our running example. The first task of the global strategy is to calculate the not yet correctly mapped LHS and RHS schema elements based on the output of the Fitness Function. Then, it proceeds with the establishment of mappings able to act as context mappings for other mappings. More specifically, the idea is to examine one mapping between a RHS class and an arbitrary LHS element, and then attributes and references of this RHS class are used to approve the correctness of this context mapping. This means, if a mapping for a RHS class is found and this mapping fulfills the cardinality constraint (cf. Condition 1 in Section 3), in the next steps, mappings for its features, i.e., attributes and references, are searched. If at least one feature mapping is evaluated to true, i.e., fulfills Condition 2 or 3 of Section 3, the context

10

M. Wimmer et al.

mapping is approved. For the opposite case, the context mapping is marked as false, although the cardinality constraints are fulfilled. After giving an overview on the design rational of this strategy, we discuss the three major phases of this strategy based on the state machine illustrated in Fig. 3. Finding context mappings. As already mentioned before, the local strategies search from right to left. Hence, after the Init state where not yet correctly mapped LHS and RHS schema elements are computed, a RHS class is selected in the state Select RHS Class to establish a context mapping. The first choice is to select a LHS class to apply the C2C mapping operator (cf. state Search 4 LHS Class). If a C2C mapping has been found by the local strategy (cf. isApplicable method), it is applied and the state Evaluation is entered. Otherwise, the RHS class is stored in the C2C blacklist which has as impact that this RHS class will never be applied to a LHS class in future iterations (cf. condition [not in C2C BL]). Evaluating context mappings. Within the state Evaluate, the LHS instance model is transformed into a RHS actual instance model. After the transformation, the actual and the target instance models are compared and the output of the Fitness Function is analyzed. If the applied mapping is evaluated to true, e.g., for C2C mappings it is required that the cardinality constraint is fulfilled, consists of searching for feature mappings for the selected RHS class. In case the evaluation returned false, the RHS class and the LHS element are stored in the blacklist (i.e., this combination will never be applied again) and it is switched into the state Init. Approving context mappings. When the context mapping is evaluated to true, the current state is changed to Select RHS Att in which a not yet correctly mapped attribute of the RHS class is selected. Subsequently, the strategy looks for free LHS attributes. The structure and behavior of the following states for the application of attribute mappings is analogous to the aforementioned application of context mappings. After the evaluation of the found A2A mapping (cf. state Evaluation), three cases have to be distinguished. First, the A2A mapping is evaluated to true and the RHS class has no references, which results in an approved context mapping, i.e., it is not changeable in future iterations, and in a transition to the Init state. Second, no A2A mapping has been evaluated to true and the RHS class has no references which results also in a transition to the Init state but the context mapping is marked as evaluated to false, i.e., it is changeable in future iterations. Third, the RHS class has additional references, then the state is changed to Select RHS Ref independent of the evaluation of the A2A mapping. The mapping of the RHS references and approving of the context mapping is analogous to that of attributes. Running Example. For exemplifying the execution behavior of the presented global strategy, we make use of the running example. In Section 3, we have elaborated on the output of the Fitness Function for the initial mapping model which is summarized in Iteration 0 of Fig. 4. Now, it is explained how the global

On Realizing a Framework for Self-tuning Mappings Iteration 0

Iteration 1

C2C

Rebate

Discount A2A

height

Rebate

gets

R2R

height

A2A

heeight

gets

gets

C2C

D Discount

height

gets R2R

Person

Person Customer name famName

label

C2C

C Customer

belongsTo

naame famName

Family

A2A

label

C2C

belongsTo

Family l

A2C

label

label

c2:Customer name=marge famName=simpson p

p1:Person

p1:Person

label=simpson

label=homer

p2:Person 2P

T

c3:Customer

c1:Discount

label simpson label=simpson

p3:Person 3P

p3:Person

label=flenders

label=ned

f2:Family label=flenders

r1:Rebate

r1:Rebate

height=10%

height=10%

Feedback

p1:Person

nam me =homer h fam mName =simpson

c c2:Customer nam me=marge fam mName=simpson

label=marge

p2:Person

T

Rebate

gets

label=homer

f1:Family

p2:Person

label = simpson label=simpson

f2:Family

p3:Person

f3:Family

p3:Person 3P

f2 F il f2:Family

label =ned

label=ned

label=flenders

gets

R2R

C2C

Rebate height gets R2R

Person

Person Customer name famName

label

name n f famName

Family

label

C2C

Customer

belongsTo A2C agg

belongsTo

Family

A2C

label

label

Target

Actual

c1:Customer

p1:Person p

name =homer h famName =simpson

c2:Customer name=marge famName=simpson p

p2:Person p

p1:Person p

label =homer

label=homer

f1:Family

p2:Person

label=simpson

label =marge

p3:Person p

c3:Customer

Diff

f3:Family y

p3:Person

f2:Family

label =ned

label=ned

label=flenders

r1:Rebate

name =ned = ned famName =flenders

r1:Rebate

height=10%

height=10%

d1:Discount

Feedback

height=10%

{Person.label, Family, Family.label, Person.belongsTo}

Iteration 4

Rebate

name famName

c3:Customer

p3:Person

name =ned famName =flenders

label =simpson

Diff

{Person.label}

f1:Family

p2:Person

label=simpson

label=marge

f2:Family

p3:Person

f2:Family

label=ned

label=flenders

height=10%

Feedback

l b l h label=homer

label =flenders

r1:Rebate

d1:Discount height=10% e g t 0%

T Target t

p1:Person f1 F il f1:Family

p2:Person

p3:Person p label=ned

f2:Family y label=flenders

r1:Rebate

height=10%

height=10%

{Person.label, Family, Family.label, Person.belongsTo}

C2C

Rebate height

A2A

name n f famName

A t l Actual

T

f3:Family label =flenders

gets R2R

Person

p1:Person

c2:Customer

label=marge

Feedback

r1:Rebate height=10%

belongsTo A2A

Familyy

A2C agg

c c2:Customer

label

T Target t

A t l Actual

c c1:Customer

p1:Person

p1:Person

name =homer fam mName =simpson

name=marge mName=simpson fam

label

C2C

Customer

Family

c1:Customer

label=simpson

Diff

r1:Rebate

label

label

name=marge famName=simpson

p3:Person

d1:Discount

gets

belongsTo

name =homer famName =simpson simpson

f1:Family

p2:Person

f2:Family

c c3:Customer

h i ht height

R2R

A2C agg gg

p1:Person label=homer

Discount

height g gets

C2C

f1:Family label =simpson

label =simpson

name =ned fam mName =flenders = flenders

Person C t Customer

p2:Person

T

Iterration 5

C2C

A2A gets

c c2:Customer

height=10% he i h 10%

Discount height

p1:Person

name =homer fam mName =simpson

name =marge fam mName =simpson

label=marge

Target

Actual

c c1:Customer

f1:Family y f2:Family y

T

height=10%

Feedback {Person.label, Family, Family.label, Person.belongsTo}

A2A

h height

r1:Rebate

height=10%

d1 1:Discount

gets

C2C

label=marge

r1:Rebate

Discount

h i ht height

Diff

c c3:Customer

Iterration 3

C2C

A2A

p1:Person

label =homer

nam me =ned = ned fam mName =flenders

heigght=10%

Discount height

f1:Family

label =marge

{Person.label, Family, Family.label, Person.belongsTo}

Iteration 2

Target

Actual

c1:Customer f1:Family

p2:Person

Diff

label=simpson

name =ned = ned famName =flenders

height=10%

Target

Actual

c1:Customer name =homer h famName =simpson

11

T

label = homer label=homer

f1 F il f1:Family

p2:Person

label =simpson

Diff

l b l label=marge

c c3:Customer

label = homer label=homer

f1 F il f1:Family

p2:Person

label=simpson

l b l label=marge

p3:Person

f2:Family

p3:Person

l b l label=ned d

l b l =flenders label fl d

label=ned

r1:Rebate

name =ned f mName fam N =flenders fl d

height=10%

d1:Discount

Feedback

height=10%

{}

f2:Family label=flenders

r1:Rebate height=10%

Fig. 4. The Smart Matcher in action

strategy adapts the mapping model based on this input. In addition, the necessary iterations are illustrated in Fig 4. Iteration 1 and 2. When we reconsider the computed output of the Fitness Function, we find out that only one RHS class is not correctly mapped. In particular, the class Family has currently no mapping to a LHS element. Therefore, the global strategy switches from the Init state to the state Select RHS class

12

M. Wimmer et al.

in which the Family class is selected. On the LHS schema no free classes are available now. Thus, it is not possible to apply a C2C mapping and the global strategy searches for a free attribute on the LHS to apply a A2C mapping without aggregation. The Customer.name attribute is selected and an A2C mapping is applied. Subsequently, the Evaluation state is entered, the transformation is started, and the produced actual model is compared with the given target model. The evaluation results amongst others into differences for Family instances which is an indicator that the applied mapping is not correct. Thus, the found mapping is put onto the blacklist and the Init state is entered. Because there is another configuration possibility for the A2C operator, in iteration 2, an A2C mapping with aggregation is applied which is also not correct and therefore we end up again in the Init state. Iteration 3. The Init state computes the same unmapped RHS and LHS elements as in the previous iterations. Thus, we enter again the Select RHS class state where the class Family is selected. Again, no class on the LHS is free and a LHS attribute is selected. In this iteration, we already know that Customer.name is not the corresponding element. Therefore, another free attribute is searched which results in selecting Customer.famName. After building the A2C mapping marked without aggregation between Family and Customer.famName, the Evaluation state is entered. This time, the evaluation shows that the cardinalities of the class Family are not equal, but some attribute values overlap. However, for evaluating this mapping to true, the similarity value is too low. Therefore, the next iteration is started by switching to the Init state. Iteration 4. After entering the Init state, we have again the same set of incorrectly mapped RHS elements. Thus, the same path is followed as before. But this time, the applied A2C mapping is marked with aggregation. This means, only for unique values new objects are created. The evaluation of this mapping shows that no differences may be found for the focused RHS elements and the mapping is marked as evaluated to true. Because the Family class has no further unmapped features, the state Init is entered. Iteration 5. This time, the Init state computes that only one RHS attribute is not appropriately mapped. Therefore, we directly navigate to the Select RHS Att state in which Person.label is selected. Within the next state, Customer. name is selected, because this is the only free LHS attribute. The A2A mapping is applied and the evaluation computes no difference between the actual and target models. Thus, the whole mapping process terminates.

5

Evaluation

In the project ModelCVS1 which was aimed at finding solutions for the integration of modeling tools, the need for automatic model transformations arose. 1

www.modelcvs.org

On Realizing a Framework for Self-tuning Mappings

13

Fig. 5. The development of quality indicators over time

Due to the lack of appropriate tools for metamodel matching, and the strong relatedness of metamodels and ontologies [11], models were lifted to ontologies, and ontology matching tools where applied. Unfortunately, the quality of the resulting alignments did not meet our expectations [6] and we could not deduce reliable mappings for model transformation. Consequently, the Smart Matching approach was established. Although developed with a certain application domain in mind, the result was a generic matching tool which is not restricted to metamodels only. In the following we present results of a case study where elements of the UML class diagram 1.4 metamodel are matched to elements of the UML class diagram 2.0 metamodel. Both metamodels consist of about 60 elements. The user-defined instance models describe a gas station with about 40 elements for the source model and about 30 elements for the target model. For a detailed description of the setting please refer to our project page2 . Starting from an empty alignment set, the Smart Matcher achieves a precision of 0.963 and a recall of 0.897 resulting in an f-measure of 0.929. When we translate the UML class diagram 1.4 terms into German and match it against the English UML class diagram 2.0, we obtain the same results as the Smart Matcher follows an uninterpreted mapping approach. In contrast, COMA++, the most successful system in our matching systems evaluation [6], yields alignments with a precision of 0.833, a recall of 0.583, and an f-measure of 0.683. The results of the German UML class diagram 1.4 matched with the UML class diagram 2.0 clearly indicate the relevance of similarity information for COMA++: the precision decreases to 0.368 and the recall is 0.117 with an f-measure of 0.178. Fig. 5 shows the evolution of precision, recall, and f-measure with respect to the number of Smart Matching iterations. Whereas the precision value tends abruptly towards one, recall and f-measure steadily increase. At iteration 91, the precision slightly decreases. The found mapping is interpreted as correct with respect to the provided instance models, although it is not. This is due to our heuristic for references, which is not sufficient when two references which have the same amount of links are between the same classes. For additional information on the case study, we kindly refer again to our project page. 2

http://big.tuwien.ac.at/projects/smartmatcher

14

6

M. Wimmer et al.

Related Work

The enormous demand for matching systems in order to automatically solve information integration problems led to a vast number of different approaches (for surveys cf. [5,10]). Following the classification of [10] the Smart Matcher realizes a schema-based approach where user-defined instances are employed to verify and to improve the found alignments iteratively. In this sense, the Smart Matcher is based on supervised machine learning techniques. In addition to the source and the target schema, training data consisting of input/output pairs is provided and a relationship model is generated which maps the input instances to the output instances. Several systems have been proposed which incorporate machine learning techniques in order to exploit previously gathered information on the schema as well as on the data level. For example, LSD, Autoplex, and Automatch use Naive Bayes over data instances, SemInt is based on neural networks, and iMap analyses the description of objects found in both sources by the establishment of a similarity matrix (see [5] for an overview). The probably closest approach to ours is SPICY. Like the Smart Matcher, SPICY [2] enhances the mapping generation process with a repeated verification phase by comparing instances of the target schema with transformed instances. The similarity in conceptual architecture of these two approaches contrasts to the significant differences in their realization as well as the moment a human user has to intervent. Whereas SPICY only deals with 1:1 and 1:n alignments, the Smart Matcher is able to resolve more complex heterogenities due to the integration of the CAR-language. This results in very distinct search, comparison, and evaluation strategies. Furthermore SPICY assumes that instances of the target schema are a priori available (e.g. from a Web data source). In contrast, for the application of the Smart Matching approach a human user must explicitly engineer the same instances in the source and in the target schema as the Smart Matcher implements the idea of unit testing. When SPICY terminates, it returns a ranked list of transformations which has then to be evaluated and verified by the user. When the Smart Matcher terminates, the resulting mappings are correct and complete with respect to the source and target input instances. Finally, the eTuner [9] approach automatically tunes the configurations of arbitrary matching systems by creating synthetic schemas for which mappings are known. Hence, in contrast to the Smart Matcher which tunes the search process itself, the eTuner adjusts other systems externally in order to increase the accuracy of the found alignments.

7

Conclusion and Future Work

In this paper we have presented the realization of the Smart Matcher, which implements a generic method for the automatic detection of executable mappings between a source schema and a target schema. During the whole search process, human intervention is only necessary once, namely at the beginning. The

On Realizing a Framework for Self-tuning Mappings

15

user has to define instances which represent the same real world issue in the language of the source schema as well as in the language of the target schema. Those instances allow a permanent evaluation and an incremental improvement of the found mappings by the comparison of the user given target instances with the generated actual instances. The Smart Matcher’s feedback-driven approach guarantees that the resulting mappings are sound and complete with respect to those instances. Therefore an accurate engineering of the instances is inevitable for the successful application of the Smart Matcher, but the effort spent in the preparation phase is compensated with less effort necessary in later phases. For many schemas (e.g., UML diagrams, ontologies, database schemas) the creation of the instances is supported by comfortable editors and once the instances are established for one schema, those instances may be reused in multiple integration scenarios. Furthermore, the required size of the instances does not need to be large for obtaining a satisfying quality of the mappings, as we have experienced in our case study. In future work, we will focus on the development of appropriate methods for the definition of the example instances in order to provide recommendations and in order to guide the user during the preparation phase. Furthermore, we plan more extensive comparisons with other systems, especially with those which incorporate learning techniques. Concerning the Smart Matcher itself, we plan to improve the search strategies by enhancing them with sophisticated heuristics.

References 1. Bernstein, P.A., Melnik, S.: Model Management 2.0: Manipulating Richer Mappings. In: Proc. of the 2007 ACM SIGMOD Int. Conf. on Management of Data, pp. 1–12. ACM Press, New York (2007) 2. Bonifati, A., Mecca, G., Pappalardo, A., Raunich, S., Summa, G.: Schema Mapping Verification: The Spicy Way. In: Proc. of the 11th Int. Conf. on Extending Database Technology (EDBT 2008), pp. 85–96. ACM Press, New York (2008) 3. Erdogmus, H., Morisio, T.: On the Effectiveness of Test-first Approach to Programming. IEEE Transactions on Software Engineering 31(1), 226–237 (2005) 4. Euzenat, J.: An API for Ontology Alignment. In: McIlraith, S.A., Plexousakis, D., van Harmelen, F. (eds.) ISWC 2004. LNCS, vol. 3298, pp. 698–712. Springer, Heidelberg (2004) 5. Euzenat, J., Shvaiko, P.: Ontology Matching. Springer, Heidelberg (2007) 6. Kappel, G., Kargl, H., Kramler, G., Schauerhuber, A., Seidl, M., Strommer, M., Wimmer, M.: Matching Metamodels with Semantic Systems—An Experience Report. In: Workshop Proc. of the 12th GI-Fachtagung Datenbanksysteme f¨ ur Business, Technologie und Web (BTW 2007), pp. 38–52. Verlag Mainz (2007) 7. Kappel, G., Kargl, H., Reiter, T., Retschitzegger, W., Schwinger, W., Strommer, M., Wimmer, M.: A Framework for Building Mapping Operators Resolving Structural Heterogeneities. In: Proc. of the 2nd Int. United Information Systems Conf. (UNISCON 2008). LNBIP, vol. 5, pp. 158–174. Springer, Heidelberg (2008) 8. Kargl, H., Wimmer, M.: SmartMatcher—How Examples and a Dedicated Mapping Language can Improve the Quality of Automatic Matching Approaches. In: Proc. of the 2nd Int. Conf. on Complex, Intelligent and Software Intensive Systems (CISIS 2008), pp. 879–885. IEEE Computer Scociety, Los Alamitos (2008)

16

M. Wimmer et al.

9. Lee, Y., Sayyadian, M., Doan, A., Rosenthal, A.: eTuner: Tuning Schema Matching Software Using Synthetic Scenarios. VLDB Journal 16(1), 97–122 (2007) 10. Rahm, E., Bernstein, P.: A Survey of Approaches to Automatic Schema Matching. VLDB Journal 10(4), 334–350 (2001) 11. Wimmer, M., Reiter, T., Kargl, H., Kramler, G., Kapsammer, E., Retschitzegger, W., Schwinger, W., Kappel, G.: Lifting Metamodels to Ontologies: A Step to the Semantic Integration of Modeling Languages. In: Nierstrasz, O., Whittle, J., Harel, D., Reggio, G. (eds.) MoDELS 2006. LNCS, vol. 4199, pp. 528–542. Springer, Heidelberg (2006)

Programming Models for Concurrency and Real-Time Jan Vitek Computer Science Department Purdue University [email protected]

Abstract. Modern real-time applications are increasingly large, complex and concurrent systems which must meet stringent performance and predictability requirements. Programming those systems require fundamental advances in programming languages and runtime systems. This talk presents our work on Flexotasks, a programming model for concurrent, real-time systems inspired by stream-processing and concurrent active objects. Some of the key innovations in Flexotasks are that it support both real-time garbage collection and region-based memory with an ownership type system for static safety. Communication between tasks is performed by channels with a linear type discipline to avoid copying messages, and by a non-blocking transactional memory facility. We have evaluated our model empirically within two distinct implementations, one based on Purdue’s Ovm research virtual machine framework and the other on Websphere, IBM’s production real-time virtual machine. We have written a number of small programs, as well as a 30 KLOC avionics collision detector application. We show that Flexotasks are capable of executing periodic threads at 10 KHz with a standard deviation of 1.2us and have performance competitive with hand coded C programs.

M. Oriol and B. Meyer (Eds.): TOOLS EUROPE 2009, LNBIP 33, p. 17, 2009. c Springer-Verlag Berlin Heidelberg 2009 

CIF: A Framework for Managing Integrity in Aspect-Oriented Composition Andrew Camilleri, Geoffrey Coulson, and Lynne Blair Computing Department Lancaster University LA1 4WA, UK {a.camilleri,g.coulson,l.blair}@lancaster.ac.uk http://www.infolab21.lancs.ac.uk

Abstract. Aspect Oriented Programming (AOP) is becoming increasingly accepted as an approach to deal with crosscutting concerns in software development. However, AOP is known to raise software integrity issues. For example, join point shadows may easily omit crucial join points or include inappropriate ones. In this paper, we propose an extensible framework called CIF that constrains aspect-oriented software design and composition with the intent to maintain the integrity of the final composed system. CIF controls the composition of aspects and the base application in three dimensions: where the composition occurs, how the composition is carried out and what exactly is being composed. The framework is intended to be used in a team-based software development environment. We demonstrate the applicability of the framework through an application case study.

1

Introduction

Aspect Oriented Programming [1] is becoming increasingly accepted as an approach to deal with crosscutting concerns in software development. However, the flexible approach to modularity and composition enabled by AOP has a potential downside: it can easily result in systems that behave in unintended and unanticipated ways – i.e. the integrity [2] of the system can easily become compromised (examples are given in the following paragraphs). According to our analysis, integrity violations can be conveniently grouped under three orthogonal headings: i) where an aspect-oriented composition occurs, ii) how the composition occurs and iii) what exactly is being composed. ‘Where’ refers to the quantification [3] of aspect composition as specified by a pointcut expression. Quantification is about choosing the ‘shadows’ [4] within which composition will occur. The difficulty here lies in getting the strength of the quantification right (and therefore the extent of the shadow). Over-strong (or over-specific) quantification can lead to compositions that miss crucial join points, whereas weak quantification can include unintended join points [5]. In M. Oriol and B. Meyer (Eds.): TOOLS EUROPE 2009, LNBIP 33, pp. 18–36, 2009. c Springer-Verlag Berlin Heidelberg 2009 

CIF: A Framework for Managing Integrity in Aspect-Oriented Composition

19

either case the integrity of the resulting system can clearly be compromised. It is also worth noting that threats to system integrity posed by inappropriate quantification are commonly tangled with issues of maintainability and reusability – e.g. there is often a strong desire for weak quantification to help foster aspect reuse (i.e. to avoid ‘fragile pointcuts’ [6]). ‘How’ refers to the compositional semantics employed when weaving aspects; this is especially an issue where multiple aspects are woven at a common join point. Existing AOP systems provide considerable flexibility in this area, offering for example, options such as advice ordering; execution options like ‘before’, ‘after’ or ‘around’; compositional options like ‘call’ or ‘execution’; or aspect instantiation semantics such as ‘per-instance’, ‘per-class’, ‘per-call’ or ‘per-thread’. Inadvertent misuse of these dimensions of flexibility can easily result in integrity violations. For example, if a ‘distribution’ aspect is inadvertently woven before an ‘authentication’ aspect at the same join point, then the resultant system could erroneously make unauthenticated calls. As another example, it is desirable to limit the use of ‘around’ advice: reasoning about aspects that use ‘around’ advice is hard because the advice developer has the luxury to forfeit the original join point (i.e. not call proceed()). Finally, ‘what’ refers to the functionality encapsulated in an aspect. Inadvertent inclusion of inappropriate behaviour in an aspect can again lead to integrity violations; for example, deadlock might result if an aspect unexpectedly accesses a resource shared by the base code. It is also important to have confidence that a given aspect behaves as its name would suggest. For example, it could easily lead to confusion, especially in a team-based software development environment, if an aspect called ‘logging’ failed to call proceed() or used apparently inappropriate modules such as an authentication library. In general, therefore, the libraries and other code used by an aspect should be closely related to the associated concern. Diverging from this goal at the very least makes modular reasoning considerably harder. The contribution of this paper is to propose an extensible framework, called CIF (for “Composition Integrity Framework”), that constrains aspect-oriented software design and composition with the intent to avoid threats such as the above and therefore to maximise the integrity of the composed system. CIF is abstract and independent of any tool specific concepts (and as such is generally applicable to a range of AOP environments) and is designed to be employed in team-based software development environments. The constraint mechanisms offered by the framework closely follow the above analysis in terms of ‘where’, ‘how’ and ‘what’. The rest of the paper is organised as follows. Section 2 introduces CIF’s basic abstractions. Section 3 illustrates the use of these abstractions in an application case study, and Section 4 discusses our current implementation of CIF. Section 5 surveys related work, arguing that while elements of the integrity issue have been addressed in the literature, no work has yet considered the issue sufficiently comprehensively. Finally, Section 6 offers our conclusions.

20

2 2.1

A. Camilleri, G. Coulson, and L. Blair

CIF’s Basic Abstractions: Domains, Realms and Configurations Overview

The main contribution of this paper is to propose an extensible ‘integrity framework’ called CIF for aspect-oriented software development. Our objective is to specify and enforce limits for aspects in a very general way. Our approach is to specify what is permissible rather than to explicitly restrict behaviour (this is in line with the well-accepted principle of fail-safe defaults [7]). Although our work to date has largely considered compile-time AOP, our framework is inherently agnostic as to compile-time versus dynamic AOP. In more detail, our approach deals with AOP integrity in terms of programmerdefined, concern-specific, ‘boundaries’ or ‘ringfences’ that we refer to as domains. A domain scopes a specific concern and imposes its boundaries in terms of the ‘where’, ‘how’, ‘what’ analysis discussed in Section 1. Each aspect is then allowed to operate only within the limits of one or more associated domains.

Fig. 1. Composition Integrity Framework

On top of domains, CIF provides the higher-level abstraction of realms. The simplest function of a realm is to list and organise the set of domains defined for a given system. But beyond that, realms provide a way for domains to be combined and inter-related while preserving domain autonomy. This allows us, for example, to specify which domains depend on each other (like authorization and authentication), or to specify weaving priorities in cases where a join point falls simultaneously within multiple domains. In essence, a realm provides an abstraction over a set of domains and a discipline to preserve the integrity of their composition. Finally, on top of realms, CIF offers the coarser-grained abstraction of a configuration which allows us to use the foregoing abstractions to define multiple

CIF: A Framework for Managing Integrity in Aspect-Oriented Composition

21

alternative instantiations or ‘builds’ of a whole application, each of which may have its own dedicated integrity constraints. Architecturally, the CIF approach is to avoid interfering with the either the base code of a system or its associated aspects. Instead, CIF domain, realm, and configuration definitions sit alongside the system (see Figure 1) which remains oblivious to the existence of CIF. Essentially, CIF separates out the integrity concern (or ‘meta-concern’). CIF also operates throughout the development life cycle. In particular, the domain and realm abstractions constrain the design of aspects; and at build-time, a tool called the CIF verifier (see Section 4) checks the conformance of aspect implementations to domain and realm specifications, while the configuration abstraction (again, with tool support) helps ensure that application builds or deployments maintain pre-specified integrity constraints. CIF can even deal with run-time compositional constructs such as cflow. 2.2

Domains

In this and following sections we use a BNF-like notation1 to more precisely define the three main CIF abstractions and their supporting concepts. In terms of this notation, a domain is defined in Figure 2.

Fig. 2. Domain

As can be seen, a domain consists of a shadow-region (cf. ‘where’), a concernregion (cf. ‘what’), and a set of permissable-compositional-styles (cf. ‘how’). Before discussing these in detail we first expand on the general notion of a ‘region’. A region is a set of programmatic-element-instances in the target programming language environment such as specific classes, methods, fields, etc. The intention is that a region provides a means to ‘slice’ an application in some appropriate manner using a predicate, which picks up a specific set of 1

In this notation < > refers to a tuple; and { } to a set.

22

A. Camilleri, G. Coulson, and L. Blair

programming-element-instances. For example, in the case study that will be described in Section 3, we define predicates over the Java package structure of the system: healthwatcher.view.command.* in Figure 7 matches all the programmatic-element-instances (in this case classes) scoped within healthwatcher.view.command. Given this understanding of a region, a domain’s shadow-region refers to the set of programmatic-element-instances that fall within the scope of this domain. More specifically, when an aspect is assigned to this domain (see Section 2.4), and a pointcut expression is written to specify the composition of this aspect, the shadow defined by that pointcut expression is constrained by CIF to fall within the domain’s shadow region. This is useful in a number of ways; for example, if not so constrained, a pointcut that targets the execution join points of a class may inadvertently extend over all of the class’s subclasses which may be implemented by different team members who may be unaware of such a pointcut. The concern-region defines the ‘what’ of the domain; this aims to constrain the behaviour that embodies a particular concern. For example, an aspect implementing a distribution concern can be excluded from making use of the behaviour found in a database library. Finally, the permissable-compositional-styles constituent defines the ‘how’ of the domain. This is expressed as a combination of three other constructs: advicerelated-style deals with advice types, pointcut-related-style is a boolean expression that is written in terms of primitive-pointcut s, and instantiation-related-style deals with aspect instantiation options. Advice types are useful to constrain (using advice-related-style) in a number of scenarios. For example, a synchronization aspect using a semaphore should be restricted to use before and after for wait and release join points. (It is possible to use an around but we do not need to forfeit the original join point. In general, we should always strive for the simpler semantics since it reduces the possibility of introducing errors.) The pointcut-related-style is useful because it allows us to define any implicit assumptions which an advice might have. For example in AspectJ, the semantics of the implicit this2 object varies depending on captured join point [4]. We can easily violate integrity if we change a pointcut (in a manner that changes the context accessed through thisJoinPoint) without a corresponding change in how the corresponding this object is used. The arguments of an advice can also be a source of confusion and this can be alleviated by an explicit pointcut-relatedstyle. We may constrain the exposure of the arguments at a specific join point using the args pointcut instead of opting for an advice to access them using reflection. Similar reasoning applies for the instantiation-related-style. The benefits of permissable-compositional-styles are especially visible in scenarios where the pointcut is defined separately from the advice implementation; for example, AspectJ provides the use of abstract aspects. In such scenarios it is possible that implicit assumptions regarding the runtime context are violated (as described above), because each developer may have a different intuition of 2

The same reasoning applies for the implicit target object and the arguments at a join point. In AspectJ these are all accessed through the implicit thisJoinPoint.

CIF: A Framework for Managing Integrity in Aspect-Oriented Composition

23

how composition should occur. This is especially true if we take into account the subtle differences between certain primitive pointcuts; the classic example here is call and execution (in AspectJ the reference to the implicit target object is different for these pointcuts), but another possible example is within and cflow. These subtle problems are perhaps more common in environments that provide an asymmetric paradigm for the encapsulation of concerns [8,9]. As shown in Figure 2, the permissable-compositional-styles constituent is extensible and can comprise a range of mechanisms, the choice of which is dependent on the target AOP environment in which CIF is being applied3 . To summarise, the primary purpose of a domain is to constrain its associated aspect or aspects. We say that an aspect is suitably constrained by (conforms to) a domain iff: 1. all shadows specified by pointcuts applied to the aspect are contained within the domain’s shadow-region, 2. all programmatic-element-instances used by the aspect’s advice are contained within the domain’s concern-region, 3. the aspect’s composition is constrained by its associated domain’s permissable-compositional-styles definition. Examples of the use of the domain abstraction are given in Section 3. 2.3

Realms

The pseudo-BNF for the realm abstraction is shown in Figure 3. As can be seen, a realm is a set of one or more domains plus a regime that constrains relations between the domains. This regime is a boolean expression that is written in terms of an extensible set of so-called integrity functions. The initial set of integrity functions that we have employed in our work to date is shown in Figure 4. These integrity functions are all of type boolean and can express a variety of constraints between domains. In general, we differentiate between two types of integrity functions: static and dynamic. The key difference is that static integrity functions perform checks that deal more closely to the internal integrity of CIF and thus are applied before the start of the weaving process; dynamic functions are applied during the weaving process, performing checks or possibly influencing it in some manner (more about this in Section 4). In Figure 4 all of the functions are static, except for precedence(). The function dependent() declares dependencies between domains: i.e. d1 is dependent on d2. This might be used, for example, to provide a regime for domains that respectively address authorisation and authentication – i.e. the final system can only include authorisation (as a concern) if it also includes authentication. Similarly, the function exclusive() declares which domains are mutually exclusive i.e. d1 and d2 are mutually exclusive. For example, we may 3

Another possible ‘style’ that might be applied in an AspectJ [10] environment, for example, is the declare construct.

24

A. Camilleri, G. Coulson, and L. Blair

design a system that implements distribution as an aspect and want to leave open the possibility of using alternative distribution technologies (e.g. CORBA or Java RMI). In such a case we can construct separate domains for each technology and define them to be mutually exclusive. The closure() function determines the ‘level’ to which the domain’s concernregion is to be verified against an associated aspect. For example, with level = 0 only code directly included in the aspect’s advice is verified; with level = 1, verification extends to the top level of all the modules and other programmaticelement-instances that are referenced from within the advice; with level = ∞ the closure of all code referenced directly or indirectly from the advice is verified.

Fig. 3. Realm

Fig. 4. Example Integrity Functions

Finally, the function precedence() imposes a weaving order for advice between domains that share common shadows (thus addressing an element of the ‘how’ dimension that could not be addressed in individual domain definitions). In more detail, precedence() defines the advice precedence of d1 over d2 ; the shadow set argument is used to scope the set of shadows at which the precedence relation should apply4 . To round off the discussion of realms, we can summarise by saying that the purpose of a realm is twofold. First, it gives us an abstraction over all the domains that are relevant for a specific application. Second, using regimes we can express constraints over domains with the intent to preserve the integrity of a final application. In general, regimes (and their consituent integrity functions) are useful because they allow us to go beyond the programming language based constraints expressed using domains. Moreover, the fact that new integrity functions can be added provides CIF’s main mechanism for extensibility which enables us to cover arbitrary application-specific integrity issues. An example of the use of the realm abstraction is given in Section 3. 4

Incidentally, the precedence relation expressed as a sequence of aspects provided by most AOP environments, can be viewed as a special case of our precedence() function.

CIF: A Framework for Managing Integrity in Aspect-Oriented Composition

2.4

25

Configurations

The purpose of our final abstraction, that of a configuration, is to enable us to create different application builds from a set of domains. For example, we can use configurations to define application builds that are supported by different OSs, or debugging versions, or versions that include or omit security, etc. Intuitively, a configuration provides us with an instantiation of a realm.

Fig. 5. Configuration

In more detail, a configuration, as defined in Figure 5, specifies: i) a set of all the domains that are relevant to a particular application build; ii) a congruent set of aspects that are to be associated with and constrained by those domains; and iii) a realm definition that specifies the regime constraints associated with those mappings. Essentially, a configuration embodies the following assertions: 1. that each aspect conforms to its associated domain (or domains – as we have seen, an aspect can be mapped to multiple domains), 2. that the given realm’s regime evaluates to true. An example of the use of the configuration abstraction is given in Section 3. 2.5

CIF Syntax

Figures 2, 3 and 5 defined CIF’s main concepts in an abstract manner. In practice, CIF specifications employ a concrete syntax. Due to lack of space, we do not describe this syntax formally; rather we introduce it by example. More specifically, examples of concrete domain, realm and configuration specifications are given in Figures 7, 8 and 9. These CIF specifications are related to an application case study that will be described in Section 3. As can be seen in Figures 7, 8 and 9, the various constituents of the framework (regions, styles, etc.) each have their own corresponding labeled sections in a specification, and each of these sections comprises a set of definitions in which CIF-defined attributes are assigned values within curly brackets. Thus, Figure 7 shows a domains section which defines a number of domains (in this case two) together with their corresponding attributes – i.e. shadow and concern regions from the regions section, and permissible-compositional-styles from the compositional-styles section. The regions sub-section is in turn made up of region definitions that are associated with predicate-valued attributes that specify sets of programmaticelement-instances that should fall within the region being specified (the Method qualifier specifies that methods are the only programmatic element type of interest here). Similarly, the compositional-styles section is defined in terms

26

A. Camilleri, G. Coulson, and L. Blair

of advice-related-styles (with options like before, around, etc.), pointcut-related-styles (with options such as call, execution, etc.) and instantiation-related-styles (with options like singleton, per-thread, etc.). Likewise, Figures 8 and 9 respectively define realms and configurations, which employ the same syntactic approach. The realms section depends on subsections dealing with regimes and integrity functions. The configurations section simply lists a set of configurations, each of which is defined as a list of <domain, aspect> pairs.

3 3.1

Application Case Study Case Study Overview

The purpose of the following application case study is to further motivate CIF’s abstractions, but also to clarify the way we envisage CIF being used. The application on which the case study is based is discussed in detail in [11,12]. In outline, the application is called Health Watcher and is a health service complaint logging system which embodies a distributed client server architecture. It was originally implemented in Java, but later refactored to encapsulate crosscutting concerns using AspectJ. The refactored version employs aspects that address distribution, persistence and transaction concerns. The main reason for choosing this application (besides it being a real system) as our case study is the variety of configurations in which it is available, as shown in Figure 6 (which shows 4 distinct configurations). Thus, for example, the application could be accessed either through the web with a normal browser or using an application installed on the client side (thus explaining the need for different distribution technologies). Moreover, given a specific platform, the application developers had to cater for different options in terms of transaction and persistence mechanisms (in this case relational and object-oriented databases).

Fig. 6. Health Watcher Configurations

CIF: A Framework for Managing Integrity in Aspect-Oriented Composition

3.2

27

Using CIF’s Abstractions

The Health Watcher application provides an ideal ecosystem in which to encounter integrity violations because it is difficult to manage even such a modest set of concerns, given their multiple implementations. It should be obvious that the integrity of the final system can be easily violated if the corresponding aspects are not carefully combined together. CIF is useful in this situation because it gives an abstraction over the concerns and on how they can be combined. Turning to the specifics of the CIF specification, Figure 7 defines appropriate domains for the application’s distribution concern (which we will employ as our primary example concern). Two domains are specified, one for the client side and one for the server side. The concern-region of rmiserver-Domain specifies that only methods from the java.rmi.server package can be employed within aspects associated with this domain. This serves to eliminate, for example, an EJB based distribution implementation. The shadow region specified establishes an explicit set of programmatic-element-instances that constrain the compositional shadow for this domain. This prevents the aspects associated with this domain from accidentally composing with elements that are not listed in this region.

Fig. 7. Example Domains

The compositional-styles section constrains aspect oriented compositions for the client and the server, regardless of the concrete distribution technology. In this case, the pointcut-related-styles for the client and server side are constrained to use call and execution, respectively. This prevents us from confusing the advice arguments (exposed by a pointcut that is defined separately

28

A. Camilleri, G. Coulson, and L. Blair

from the advice implementation) for arguments of intercepted methods. If the style for composition is constrained through compositional-styles then we can have a uniform compositional view for all distribution concerns and our assumptions will always be sound. For example, we can choose to explicitly expose the intercepted method arguments by constraining the use of the args pointcut, instead of relying on each advice to access them through thisJoinPoint. Finally, both domains constrain their advice-related-styles to be around because the distribution aspects need to marshal data (i.e. method arguments) on both requests and replies. Using a before or an after here would be an error, since we want to replace the original join point. The respective instantiation-related-styles are restricted to be singleton since we only want one running instance of the client and of the server. Again, having this kind of constraint capability is useful to maintain a consistency amongst the alternative concern implementations.

Fig. 8. Example Realms

Figure 8 illustrates realm definitions employed in the case study. As one can see, the regime is a conjunction of five separate predicates. The dependent() integrity function is used to state that if a transaction concern is to be employed, a persistence concern is also required (this is to ensure safe storage for uncommitted data). Similarly, the exclusive() function is used to disallow the simultaneous use of RMI and EJB distribution concerns. The closure() function specifies that the full closure of the advice code of the server-side distribution aspect(s) should be verified for conformance to the server-side domain’s concern-region.

CIF: A Framework for Managing Integrity in Aspect-Oriented Composition

29

Fig. 9. Example Configurations

Finally, Figure 9 shows two sample configurations: one employs RMI-based distribution with object-oriented database persistence and transaction management; the other employs EJB distribution, with relational database persistence and transaction management. These two configurations correspond to Figures 6(a) and 6(d). The two configurations specified here correspond to RMI-based configuration and EJB-based configuration in Figure 10 (this figure gives an overview of the whole specification process). It is important to note that the configuration specifications are not effectively specifying any additional integrity constraints. They simply allow us to create our final application. Thus, given a specific realm, the mapping in the configuration can be specified (perhaps using an appropriate GUI) by a deployer who may be ignorant of the details of implementation, but simply requires to build a system which includes a specific set of concerns. The configuration allows us to do exactly this, while making sure that there are no integrity violations along the way. 3.3

Using CIF in a Team-Based Software Development Environment

The case study also enables us to illustrate how CIF can be applied in teambased software development environments. For example, consider a development environment that employs three major roles: i) a system architect, ii) a developer, and iii) a quality assurance officer. In such an environment, the system architect role would define the fundamental architecture of the application, plus CIF abstractions to constrain the subsequent definition and composition of the aspects envisaged in the design. In our particular case, this means defining the application’s basic client-server architecture, and its associated domains (e.g. rmiclient-Domain and rmiserver-Domain, plus domains for the persistence and transaction aspects as shown in Figure 10). The system architect would also constrain the way that these domains inter-relate by defining realms and associated regimes (e.g. in our case the system architect specifies that RMI and

30

A. Camilleri, G. Coulson, and L. Blair

Fig. 10. Health Watcher with CIF

EJB communications are mutually exclusive and that transactional functionality is dependent on the presence of persistence functionality). The developer role would then implement the base modules and aspects identified by the system architect. When complete, the various modules and aspects would all be delivered to the quality assurance officer who would test the delivered aspects against their associated domains (using the CIF verifier; see Section 4) and define prototype configurations (such as our RMI- and EJB-based configurations as shown in Figure 10) that exercise the system in various application-specific ways. Finally, the quality assurance role would feed back the results of testing these configurations (such as integrity violations) for refinement to the systems architects and developers as appropriate.

4

Implementation

We have built a prototype implementation of CIF for the AspectJ language using AspectBench [13]. The implementation is logically structured in two parts: a static part and a dynamic part. The static part comprises a verifier that ensures that aspects conform to domains, and that realm and configuration specifications are self-consistent e.g. ensuring that any dependencies and mutual exclusion relations (specified using the dependent() and exclusive() integrity functions) are honoured. Our current implementation supports only “closure(0)” semantics – i.e. only toplevel code is checked for conformance with regions (regions themselves are computed with assistance from the AspectBench weaver). Other application-specific integrity functions can also be added. For example, in a team-based software development scenario where multiple architects collaborate to define domains, an ignore(Region r) function could be defined to demarcate an application-level region that should never participate in the definition of any domain.

CIF: A Framework for Managing Integrity in Aspect-Oriented Composition

31

The second part of the implementation is a dynamic verifier integrated with the AspectBench weaver, and uses callbacks to realise extensible weave-time checking behavior. Integrity functions that are integrated with this part are allowed to have side effects. For example, a weaver callback associated with the precedence() integrity function directs weaving to occur according to precedences specified by this function. Figure 11 gives a layered view of our implementation. The CORE module implements the CIF’s central abstractions (domains, realms, and configurations) and performs the above-mentioned static consistency checks. Above this, the CIFParser module reads in and parses CIF specifications. The CORE and CIFParser modules are generic and entirely independent of any particular target AOP environment, the latter being represented primarily by the Weaver module. The generic parts are sheltered from the Weaver by semi-generic modules, namely the AspectLang, LangParser, and the DynamicVerifier modules: AspectLang provides an AOP-environment-specific adaptation layer between the CORE and the target AOP environment; LangParser takes care of parsing the programming language specific aspects that will participate in the composition; and DynamicVerifier provides the above-mentioned callback mechanism for integration with the weaver.

Fig. 11. CIF Implementation Architecture

We handle dynamic pointcuts (e.g. AspectJ’s cflow) in the following manner. Such pointcuts are commonly realised by inserting runtime checks (during compilation) at points (both for entry and exit control flows) where composition might occur [14,15]. In that light, our strategy5 is simply to ensure that these checks are constrained to occur only within appropriately defined shadow regions. This gives us an ‘upper limit’ for constraining dynamic composition without the need to be too restrictive. In future implementation work we plan to extend our exploration of runtime integrity support in dynamic environments. One key strategy here is to explore integrity functions that persist at runtime. For example, we could conceive of an integrity function that ensures that ‘around’ advice in a specified domain always calls proceed() [16,17]. 5

A possible alternative strategy could be to constrain only where composition should occur i.e. control flow exit points.

32

A. Camilleri, G. Coulson, and L. Blair

We also plan to provide an automated tool that would inspect all the aspects and pointcuts in an existing system and come up with corresponding domain specifications. This would relieve us of the burden of having to specify domains from scratch; it would provide an initial definition of regions for each aspect, which can be viewed and refined (at least by specifying integrity functions) as required. Complementary to this automated tool, we also plan to provide a GUI-based workbench to better evaluate the use of CIF in realistic software development environments.

5

Related Work

Although we argue that the literature has not yet comprehensively addressed the issue of integrity in AOP systems, we nevertheless acknowledge that a considerable body of work addresses a range of concerns within this general space. In terms of the where dimension, a number of approaches attempt to restrict the shadows specified by pointcut expressions. For example, [18,19,20,21] all offer ways to abstract the set of join points accessible to pointcut expressions so as to raise the semantic level and thereby increase the precision of the quantification. XPI [22] attempts to achieve similar ends through the use of an information hiding interface called ‘cross-cutting interfaces’. A complementary approach is adopted by [23] which offers a ‘negative’ pointcutting mechanism that allows sets of join points to be explicitly excluded from quantifications. In general, although this work enables greater control over the ‘where’ dimension, it does not at all address the ‘how’ or ‘what’ dimensions. Furthermore, all the proposed approaches lack an equivalent concept for domains and therefore restrict quantification in a coarse-grained manner: i.e. they do not allow per-aspect control over the quantification shadow. This is an important omission because shadow management is naturally a per-concern issue. In terms of the how dimension, most work to date has focused on how multiple aspects woven at a common join point should relate to each other. For example, [24] and [25] provide a comprehensive catalogue of aspect interactions and suggest how unwanted interactions can be detected; [26] provides a systematic means of composing multiple aspects, and detecting and resolving potential conflicts; and [16] provides a domain-specific language to define execution constraints for aspect interactions. This work is largely complementary to ours in offering greater semantic depth in specific focus areas. However, it does not address the comprehensive range of ‘how’ issues (e.g. accommodating ordering issues, aspect instantiation semantics, or execution options) that we cover (see Section 2). Our approach can also be viewed as an integrating framework for more focused work such as the above. In terms of the what dimension, [27] discusses the notion of a permission system for aspects – i.e. to control which parts of a system a given aspect can access; however, no concrete mechanisms are proposed. Other work (e.g. [28,29,30,31,32]) attempts to classify and categorise advice (e.g. in terms of whether it is read-only or, if not, the degree of invasiveness permitted). Our

CIF: A Framework for Managing Integrity in Aspect-Oriented Composition

33

approach, on the other hand (see Section 2), focuses on a finer-grained restriction of the behaviour that is applicable for a given advice type (i.e. in terms of the modules or libraries that can be used by advice). Furthermore, although the classifications provided by [28,29] seem very general and widely applicable, they do not deal, as we do, with concern interactions (see Section 2). Also, although [30,32] provide an intuitive classification, the reports generated by these tools require a lot of manual analysis, and each change requires the generation (and subsequent analysis) of a new report. Moreover, contrasting with [31], our approach does not require annotation of the underlying system. Finally, some research attempts to verify the behaviour of aspects in terms of static code analysis [33]. However this work (so far) only supports a limited source language, making it unsuitable for most real-world scenarios. “Design by contract” approaches can also be viewed as relating to the ‘what’ dimension. For example, the approach proposed by [34] divides aspects into ‘observers’ and ‘assistants’ to facilitate modular reasoning; and [35] provides a similar classification of aspects. Also, [36] describes the notion of ‘aspect integration contracts’ which attempts to preserve integrity by equipping aspects with ‘required’ and ‘provided’ denotations. Although these approaches directly target the integrity issue (albeit only in the ‘what’ dimension) their downside is that they tend to be intrusive, verbose and difficult to manage. In constrast, CIF avoids cluttering source code by factoring integrity specification out of both base modules and aspects. Finally, MAO [17] is an interesting system that addresses both ‘what’ and ‘where’ issues, but using an approach quite different from ours (see Section 2). In brief, they adopt a language-based annotation approach that enables programmers to explicitly characterise the effects of aspects. In terms of ‘what’, the programmer can, for example, write annotations that ensure that proceed() is called, that return values are returned unchanged, or that an advice may not change arguments. In terms of ‘where’, the programmer can constrain the quantification shadow of a given aspect. However, although MAO enables wideranging control over the integrity of an AOP system, it has significant disadvantages that mainly accrue from its language-based approach (again, compare our ‘factored out’ approach). In particular, MAO annotations are potentially complex and may need to be applied to base code as well as aspect code; moreover it is always difficult to find general acceptance of new language-level constructs.

6

Conclusion

As AOP gains more ground, it becomes increasingly important to provide a means to maintain a tight grip on the compositional flexibility it provides. We feel that the CIF approach offers a very promising way to achieve this, in a very practical manner. We are encouraged with the results we have achieved so far, but we also realize that there is more work to be done. Perhaps the most significant limitation of our design that we have discovered to date is the predicate-based region definition

34

A. Camilleri, G. Coulson, and L. Blair

approach. Experience has shown that this can be tricky to get right (especially using Java-inspired package predication) and may suffer from similar weaknesses as pointcuts – i.e. over strong or over weak predication. Currently, we are experimenting with alternative predication approaches and also with the integrity function facility to help minimise this weakness: for example, a ‘monitor’ function can keep track of complete enumeration (automatically, without any manual intervention) of programmatic elements picked out by specific regions and warn us when there are changes in this set. More generally, we believe that our approach to integrity management is unique in not restricting or even changing the aspect language or the base application: CIF simply sits alongside these two components, to provide the necessarily integrity-preserving machinery. It thus cleanly separates out the integrity concern (or ‘meta-concern’). Furthermore, we believe that our proposal is complementary to several of the more partial approaches discussed in Section 5, and that these could in future work be integrated into our approach. For example, XPIs [22] can be applied alongside CIF at design time to clarify and facilitate the selection of appropriate regions and compositional styles. Similarly, it seems feasible to accommodate the execution constraints defined in [16] by encapsulating them in integrity functions. We can also include advice classifications similar to augmentation and replacement [30,31,32] in advice-related-style, thus extending them with compositional styles. Finally, it seems conceivable to relate contracts, as described by [36], to domains and to therefore provide stronger guarantees that aspects comply with concern-specific semantics.

References 1. Kiczales, G., Lamping, J., Mendhekar, A., Maeda, C., Lopes, C., Loingtier, J.M., Irwin, J.: Aspect-oriented programming, pp. 220–242 (1997) 2. Clark, D.D., Wilson, D.R.: A comparison of commercial and military computer security policies. SP, 184 (1987) 3. Filman, R.E., Friedman, D.P.: Aspect-oriented programming is quantification and obliviousness. Technical report (2000) 4. Hilsdale, E., Hugunin, J.: Advice weaving in aspectj. In: AOSD 2004: Proceedings of the 3rd international conference on Aspect-oriented software development, pp. 26–35. ACM, New York (2004) 5. McEachen, N., Alexander, R.T.: Distributing classes with woven concerns: an exploration of potential fault scenarios. In: AOSD 2005: Proceedings of the 4th international conference on Aspect-oriented software development, pp. 192–200. ACM Press, New York (2005) 6. Stoerzer, M., Graf, J.: Using pointcut delta analysis to support evolution of aspectoriented software. ICSM, 653–656 (2005) 7. Saltzer, J.H., Schroeder, M.D.: The protection of information in computer systems. Proceedings of the IEEE 63, 1278–1308 (1975) 8. JBossAOP, http://www.jboss.org/jbossaop 9. Rajan, H., Sullivan, K.J.: Classpects: unifying aspect- and object-oriented language design. In: ICSE 2005: Proceedings of the 27th international conference on Software engineering, pp. 59–68. ACM, New York (2005)

CIF: A Framework for Managing Integrity in Aspect-Oriented Composition

35

10. AspectJ, http://www.eclipse.org/aspectj 11. Soares, S., Borba, P., Laureano, E.: Distribution and persistence as aspects. Softw. Pract. Exper. 36(7), 711–759 (2006) 12. Greenwood, P., Garcia, A., Rashid, A., Figueiredo, E., Sant’Anna, C., Cacho, N., Sampaio, A., Soares, S., Borba, P., Dosea, M., Ramos, R., Kulesza, U., Bartolomei, T., Pinto, M., Fuentes, L., Gamez, N., Moreira, A., Araujo, J., Batista, T., Medeiros, A., Dantas, F., Fernandes, L., Wloka, J., Chavez, C., France, R., Brito, I.: On the Contributions of an End-to-End AOSD Testbed. In: EARLYASPECTS 2007: Proceedings of the Early Aspects at ICSE, Washington, DC, USA, p. 8. IEEE Computer Society Press, Los Alamitos (2007) 13. AspectBench, http://abc.comlab.ox.ac.uk 14. Dinkelaker, T., Haupt, M., Pawlak, R., Benavides, L.D., Gasiunas, V.: Inventory of aspect-oriented execution models. Technical report, AOSD-Europe (2006) 15. Bockisch, C., Kanthak, S., Haupt, M., Arnold, M., Mezini, M.: Efficient control flow quantification. In: OOPSLA 2006: Proceedings of the 21st annual ACM SIGPLAN conference on Object-oriented programming systems, languages, and applications, pp. 125–138. ACM, New York (2006) 16. Pawlak, R., Duchien, L., Seinturier, L.: CompAr: Ensuring Safe Around Advice Composition. In: Steffen, M., Zavattaro, G. (eds.) FMOODS 2005. LNCS, vol. 3535, pp. 163–178. Springer, Heidelberg (2005) 17. Clifton, C., Leavens, G.T., Noble, J.: MAO: Ownership and effects for more effective reasoning about aspects. In: Ernst, E. (ed.) ECOOP 2007. LNCS, vol. 4609, pp. 451–475. Springer, Heidelberg (2007) 18. Kiczales, G., Mezini, M.: Aspect-oriented programming and modular reasoning. In: ICSE 2005: Proceedings of the 27th international conference on Software engineering, pp. 49–58. ACM, New York (2005) 19. Aldrich, J.: Open modules: Modular reasoning about advice, pp. 144–168. Springer, Heidelberg (2005) 20. Gudmundson, S., Kiczales, G.: Addressing Practical Software Development Issues in AspectJ with a Pointcut Interface. In: Advanced Separation of Concerns (2001) 21. Lieberherr, K., Lorenz, D.H.: Aspectual Collaborations: Combining Modules and Aspects. The Computer Journal 46(5), 542–565 (2003) 22. Sullivan, K., Griswold, W.G., Song, Y., Cai, Y., Shonle, M., Tewari, N., Rajan, H.: Information hiding interfaces for aspect-oriented design. SIGSOFT Softw. Eng. Notes 30(5), 166–175 (2005) 23. Larochelle, D., Scheidt, K., Sullivan, K., Wei, Y., Winstead, J., Wood, A.: Join point encapsulation. In: AOSD 2003 Workshop on Software-engineering Properties of Languages for Aspect Technologies (2003) 24. Sanen, F., Truyen, E., De Win, B., Wouter, J., Loughran, N., Coulson, G., Rashid, A., Nedos, A., Jackson, A., Clarke, S.: Study on interaction issues. Technical report, AOSD-Europe (2006) 25. Katz, S., Katz, E., Havinga, W., Staijen, T., Taiani, F., Weston, N., Rashid, A., Sudholt, M., Ha Nguyen, D.: Detecting interference among aspects. Technical report, AOSD-Europe (2006) 26. Composition, reuse and interaction analysis of stateful aspects. In: AOSD 2004: Proceedings of the 3rd international conference on Aspect-oriented software development. ACM, New York (2004) 27. De Win, B., Piessens, F., Wouter, J.: How secure is aop and what can we do about it? In: SESS 2006: Proceedings of the 2006 international workshop on Software engineering for secure systems, pp. 27–34. ACM, New York (2006)

36

A. Camilleri, G. Coulson, and L. Blair

28. Sihman, M., Katz, S.: Superimpositions and aspect-oriented programming. The Computer Journal 46(5), 529–541 (2003) 29. Dantas, D.S., Walker, D.: Harmless advice. In: POPL 2006: Conference record of the 33rd ACM SIGPLAN-SIGACT symposium on Principles of programming languages, pp. 383–396. ACM, New York (2006) 30. Rinard, M., Salcianu, A., Bugrara, S.: A classification system and analysis for aspect-oriented programs. SIGSOFT Softw. Eng. Notes 29(6), 147–158 (2004) 31. Munoz, F., Baudry, B., Barais, O.: Improving maintenance in aop through an interaction specification framework. In: IEEE International Conference on Software Maintenance, ICSM 2008, September 28-October 4, pp. 77–86 (2008) 32. Zhang, D., Hendren, L.: Static Aspect Impact Analysis. Technical report, AspectBench (2007) 33. Krishnamurthi, S., Fisler, K., Greenberg, M.: Verifying aspect advice modularly. In: SIGSOFT 2004/FSE-12: Proceedings of the 12th ACM SIGSOFT twelfth international symposium on Foundations of software engineering, pp. 137–146. ACM, New York (2004) 34. Clifton, C., Leavens, G.T.: Observers and assistants: A proposal for modular aspect-oriented reasoning. In: Foundations of Aspect Languages, pp. 33–44 (2002) 35. Lorenz, D.H., Skotiniotis, T.: Extending design by contract for aspect-oriented programming. CoRR, abs/cs/0501070 (2005) 36. Lagaisse, B., Joosen, W., De Win, B.: Managing semantic interference with aspect integration contracts. In: International Workshop on Software-Engineering Properties of Languages for Aspect Technologies (2004)

A Diagrammatic Formalisation of MOF-Based Modelling Languages Adrian Rutle1 , Alessandro Rossini2 , Yngve Lamo1 , and Uwe Wolter2 1

Bergen University College, P.O. Box 7030, 5020 Bergen, Norway aru,[email protected] 2 University of Bergen, P.O. Box 7803, 5020 Bergen, Norway {rossini,wolter}@ii.uib.no

Summary. In Model-Driven Engineering (MDE) models are the primary artefacts of the software development process. The usage of these models have resulted in the introduction of a variety of modelling languages and frameworks. Many of these languages and frameworks are based on the Object Management Group’s (OMG) Meta-Object Facility (MOF). In addition to their diagrammatic syntax, these languages use the Object Constraint Language to specify constraints that are difficult to specify diagrammatically. In this paper, we argue for a completely diagrammatic specification framework for MDE, where by diagrammatic we mean specification techniques which are targeting graph-based structures. We introduce the Diagram Predicate Framework, which provides a formal diagrammatic approach to modelling based on category theory – the mathematics of graph-based structures. The development of a generic and flexible formalisation of metamodelling is the main contribution of the paper. We illustrate our approach through the formalisation of the kernel of the Eclipse Modeling Framework. Keywords: Model-Driven Engineering, Meta-Object Facility, Unified Modeling Language, Object Constraint Language, Eclipse Modeling Framework, Diagram Predicate Framework, diagrammatic specification.

1 Introduction Since the beginning of computer science, developing high-quality software at low cost has been a continuous vision. This has boosted several shifts of programming paradigms, e.g. machine code to compilers and imperative- to object oriented programming. In every shift of paradigm, raising the abstraction level of programming languages and technologies has proved to be necessary to increase productivity. One of the latest steps in this direction has lead to the usage of modelling languages in software development processes. Software models are indeed abstract representations of software systems which are used to tackle the complexity of present-day software by enabling developers to reason at a higher level of abstraction. In traditional software development processes, models are used either for sketching the architectural design at a high level of abstraction or for mere documentation purposes. On the contrary, in Model-Driven Engineering (MDE) models are first-class entities of the software development process. These models are used to generate M. Oriol and B. Meyer (Eds.): TOOLS EUROPE 2009, LNBIP 33, pp. 37–56, 2009. c Springer-Verlag Berlin Heidelberg 2009 

38

A. Rutle et al.

(parts of) the software systems by means of model-to-model and model-to-text transformations. Hence, models in MDE are required to have two important properties. Firstly, they should be intuitive for humans, such as software developers, domain experts, managers and users. This has motivated for the usage of visual modelling and specification. Secondly, the models should have a precise syntax and semantics to enable analysis, validation, verification and transformation. This has motivated for the usage of formal modelling languages. The terms “diagrammatic-”, “visual-” and “graphical-” modelling are often used interchangeably. Here, we distinguish clearly between visualisation and diagrammatic syntax and focus on precise syntax (and semantics) of models independent of their visualisation. It is first and foremost due to the graph nature of models that we focus on diagrammatic modelling. By diagrammatic modelling we mean techniques which are targeting graph-based structures. Several diagrammatic specification frameworks have emerged in the last years as an attempt to facilitate modelling. The state-of-the-art of diagrammatic modelling languages includes the Unified Modeling Language (UML) [1] and the Eclipse Modeling Framework (EMF) [2]. These languages and frameworks share the same metamodel, the Meta-Object Facility (MOF) [3], which is a metamodelling and metadata repository standard developed by the Object Management Group (OMG). However, MOF is defined only semi-formally which may not guarantee the degree of precision required by MDE [4,5,6]. In this paper, we argue for a completely diagrammatic specification framework for MDE. We introduce the Diagram Predicate Framework (DPF) [7,8,9,10], which provides a formal diagrammatic approach to modelling based on category theory. The remainder of the paper is structured as follows. Section 2 motivates the necessity of formal diagrammatic modelling languages in MDE and presents a running example which we model by a UML class diagram. In the example, we introduce some modelling challenges which are difficult to express by UML alone. Section 3 introduces the basic concepts of our approach and shows how it can be used to model the motivating example precisely. Section 4 presents a generic and flexible scheme for the formalisation of metamodelling and discusses OMG’s 4-layered architecture in view of this scheme. Then in Section 5 the approach is illustrated for the metamodel of EMF in a case-study. Section 6 discusses some related works. Finally, Section 7 concludes the paper and outlines future work.

2 Motivation An appropriate approach to object-oriented modelling is to describe models – in a first approximation – as graphs. Graphs are a well-known, well-understood and frequently used means to represent system structures or states [11]. However, the expressive power offered by present-day MOF-based modelling languages – which are meant to deal with graph-based models – may not be sufficient to express certain constraints a software system must obey. As a consequence, OMG proposes the Object Constraint Language (OCL) 2.0 [12] for the definition of constraints in models defined by these languages [13]. This is just a special case of a general pattern where diagrammatic

A Diagrammatic Formalisation of MOF-Based Modelling Languages

39

modelling languages use textual languages to define constraints that are difficult to express by their own syntax and semantics. While this solution is to some extent accepted among software developers, there are two reasons why a completely diagrammatic approach is worth investigating. Firstly, OCL and MOF-based languages such as UML class diagrams live in different conceptual and technical spaces, which makes automatic reasoning about these models challenging. For example, any modification in a model structure must be reflected in the OCL constraints which are related to the modified structure, but, defining automatic synchronisation of OCL constraints for arbitrary model modifications is not possible [14]. Moreover, the identification of classes of modifications for which an automatic synchronisation of OCL constraints is possible requires a complex machinery to be implemented by tool vendors. Secondly, due to the graph nature of models, we are interested in investigating a completely diagrammatic approach for the specification and reasoning about structural models. This is also to obey the “everything is a model” vision of MDE by having both structure and constraints in the same model-centric format. Among the efforts which are done to clarify the semantics of diagrammatic modelling languages, are the introduction of MOF 2.0 [3] as a common metamodel for languages which are defined by OMG, and the so-called 4-layered modelling architecture. In this architecture, a model in each layer conforms to (or is an instance of ) a model in a higher layer, i.e. each layer represents a metamodel for the layer below it. In particular, a metamodel specifies the abstract syntax of a modelling language by defining the set of rules (or constraints) that can be used to determine whether a given model is well-formed. However, the relationship between a model and its metamodel remains imprecise. This is because how to determine that a model is well-formed according to its metamodel is only defined semi-formally [4,5,6]. Thus, although the usage of graphs for the representation of model structures is a success story, an enhancement of the formal basis is needed to: – Express well-formed diagrammatic constraints, especially constraints which span over multiple model elements, and which go beyond the structural constraints which are formalised as graph constraints [11,15]. – Formalise the relationship – conformance or instance of – between models in different layers. This relationship is expressed by the concepts of type- and typed graphs in graph theory [11], which does not capture the type of constraints mentioned above. A natural choice for such a formal enhancement of graph theory is category theory, and in particular the sketch formalism, which can be used to define semantics of diagrams, and thus of diagrammatic models, without the needs for a supplementary textual language. In the categorical sketch formalism, models are represented as graphs, and model properties are expressed by universal properties such as; limit, colimit, and commutativity constraints [16,17]. This approach has the benefit of being generic and at a high level of abstraction, but it turns models into a complex categorical structure with several auxiliary objects [7]. The proposed formalisation approach in this paper is DPF which is a generalisation and adaptation of the categorical sketch formalism, where user-defined diagrammatic

40

A. Rutle et al.

predicate signatures are allowed to represent the constructs of modelling languages in a more direct and adequate way. In particular, DPF is an extension of the Generalised Sketches [18] formalism originally developed by Diskin et al. in [4,19,20]. DPF aims to combine the mathematical rigour – which is necessary to enable automatic reasoning – with diagrammatic modelling. 2.1 Constraints in UML The motivating example illustrates the usage of some constraints which are not expressible by UML itself. These constraints are specified in OCL. We are convinced that software developers will benefit from a diagrammatic syntax capable of expressing these constraints.

Fig. 1. A UML class diagram for the management of employees and projects

In Fig. 1, a UML class diagram of an information system for the management of employees and projects is presented. We require that the following set of rules are satisfied at any state of the system: 1. 2. 3. 4. 5. 6. 7.

An employee must work for at least one department. An employee may be enrolled in none or many projects. A department may have none or many employees. A department may control none or many projects. A project must be controlled by at least one department. An employee enrolled in a project must work in the controlling department. A set of employees working for a controlling department must not be enrolled in the same controlled project more than once.

The rules above can be categorised into two groups based on their complexity. Rules 1-5 can be forced by constraints which involve one or two model elements. These rules can be expressed by UML syntax. For the rules 6 and 7, constraints which involve more than two elements are required. This can only be achieved by using the text-based OCL constraints as, for example:

A Diagrammatic Formalisation of MOF-Based Modelling Languages

41

1 context 2

3

4 5 6

Enrolment inv rule6: self.department.employees->includesAll(self. employee) inv rule7: Let enrolments:Set(Enrolment)=Enrolment. allInstances in (not enrolment->exists(enr|enr.project=self.project and enr.department=self.department and enr.employees=self.employees))

The solution above has the following drawbacks: 1. Checking the state of the system against the model will involve two steps which are done in two different technical spaces: firstly, checking the structure and some of the constraints in UML; then, checking the rest of the constraints by an OCL engine. 2. Since some the semantics of the model is hidden in the OCL code, the model development process may become complex and error-prone in the long run. In particular, domain experts may have difficulties in understanding the OCL code – something which may force the developers to use the list of rules in a natural language instead of the OCL rules. This may lead to misunderstandings [21]. A completely diagrammatic representation of the model in Fig. 1 is described in Section 3, Fig. 2.

3 Diagram Predicate Framework DPF is a graph-based specification format that takes its main ideas from both categorical and first-order logic (FOL), and adapts them to SE needs. While in FOL the arity of a predicate is given by a collection of nodes only, the arity of a predicate in DPF is given by a graph, i.e. a collection of nodes together with a collection of arrows between nodes. Besides, the main difference between FOL and DPF is that FOL is an “element-wise” logic, i.e. variables vary over elements of sets. In contrast, DPF uses a “sort-wise” logic where variables vary over sets and mappings. Diagrammatic reasoning can provide several benefits to software engineers. As an example we consider model evolution where the difference between artefacts is typically represented and reasoned about in a diagrammatic fashion. This is to help the developer to understand the rationale behind the modifications [22]. DPF has shown to be beneficial for the formalisation of version control systems in MDE. In particular, we exploit the formal foundation for the identification of commonalities and calculation/representation of differences between models [10]. In DPF, software models are represented by diagrammatic specifications. In the remainder of the paper, we use the terms “model” and “diagrammatic specification” interchangeably. In the next sections, we will discuss the syntax and semantics of these diagrammatic specifications.

42

A. Rutle et al.

3.1 Syntax of Diagrammatic Specifications Diagrammatic specifications are structures which consist of a graph and a set of constraints. The graph represents the structure of the model. Predicates from a predefined diagrammatic signature are used to add constraints to the graph. The concepts of signatures, constraints, and diagrammatic specifications are defined as follows. Definition 1 (Signature). A (diagrammatic predicate) signature Σ := (Π, α) consists of a collection of predicate symbols Π with a mapping that assigns an arity (graph) α(p) to each predicate symbol p ∈ Π. Definition 2 (Constraint). A constraint (p, δ) in a graph G is given by a predicate symbol p and a graph homomorphism δ : α(p) → G, where α(p) is the arity of p. Definition 3 (Diagrammatic Specification). A Σ-specification S := (G(S), S(Π)), is given by a graph G(S) and a set S(Π) of constraints (p, δ) in G(S) with p ∈ Π. Table 1 shows a sample signature Σ = (Π, α). The first column of the table shows the names of the predicates. The second and the third columns show the arities of predicates and a possible visualisation of the corresponding constraints, respectively. In the fourth column, the intended semantics of each predicate is specified. The predicates in Table 1 allow to specify properties and constraints that, according to our analysis, should be part of any modelling language which is used to define structural models. These predicates are just generalisations, or general patterns, for properties that are used in different areas of modelling. For example, [mult,(m,n)] for multiplicity constraint on associations in UML class diagrams, [total] for total constraint on relations in ER diagrams and [jointly-key] for uniqueness constraint (i.e. UNIQUE) in SQL. For a given signature the semantics of nodes, arrows and predicates has to be chosen in a way which is appropriate for the corresponding modelling environment. For structural models in object-oriented development, it is appropriate to interpret nodes as sets f

→ B as multi-valued functions f : A → ℘(B). The powerset ℘(B) of and arrows A − B is the set of all subsets of B, i.e. ℘(B) = {X | X ⊆ B}. 3.2 Constraints in DPF Fig. 2 shows an example of a Σ-specification S = (G(S), S(Π)). S specifies the structural model for the information system presented in Section 2.1. G(S) in Fig. 3 is the graph of S without any constraints on it. Here we will revisit some of the rules in Section 2.1. Recall that in rule 1 an employee must work for at least one department. In S, this is forced by the constraint ([total], δ1 ) on the arrow eDeps (see Table 2). Moreover, a property of S is that the functions eDeps and dEmps are inverse to each other, i.e. ∀e ∈ Employees, ∀d ∈ Departments : e ∈ dEmps(d) iff d ∈ eDeps(e). This is forced by the constraint ([inverse], δ3 ) on eDeps and dEmps. According to rule 6, an employee enrolled in a project must work in the controlling department. While this rule has to be forced by means of OCL constraints in the motivating example, we can specify it diagrammatically in S. The rule is forced by the constraint ([subset], δ8 ) between the arrows eEmps and eDep;dEmps. Note that

A Diagrammatic Formalisation of MOF-Based Modelling Languages

43

Table 1. A sample signature Σ p

α(p)

Proposed visualisat. Intended semantics

/2

A

/2

A

[total]

1

x

[key]

1

x

[singlevalued]

1

x

/2

A

[mult,(n,m)] 1

x

/2

A

1

x

/2

A

x

$

[cover]

[inverse]

•

∀a ∈ A : |f (a)| ≥ 1

f

/ B

∀a, a ∈ A : a = a implies f (a) = f (a )

[1]

f

/ B

∀a ∈ A : |f (a)| ≤ 1

/ B

∀a ∈ A : n ≤ |f (a)| ≤ m

 2, B

∀b ∈ B : ∃a ∈ A | b ∈ f (a)

f [n,m] f

f

1d

2

A g

'

B

[INV]

/2 O

f

A

y

/ B O

[DC]

g

3 x

[subset]

1

f

$

:2 x

1

/2

y

/ B

∀a, a ∈ A : a = a implies f (a) = f (a ) or g(a) = g(a )

f / B >> 3 >[JK] >> h g > > 

∀a, a ∈ A : a = a implies f (a) = f (a ) or g(a) = g(a ) or h(a) = h(a )

A

g f

[JK]2

C

1> >

x

/2

>>y >> >

z



4 [composition]

'

∀a ∈ A : f (a) ⊆ g(a)



[]



3 [jointlykey]

 {f (a) | a ∈ A}∩ {g(c) | c ∈ C} = ∅ and   {f (a) | a ∈ A} ∪ {g(c) | c ∈ C} = B

7 B

A

g





C

y

[jointlykey]

∀a ∈ A , ∀b ∈ B : b ∈ f (a) iff a ∈ g(b)

g

x

1

/ B

[KEY]

y

[disjointcover]

f

1> >

x

3

/2

>>z >> > 

3

A

D

/ B >> >>f ;g g > [COMP] >>   A

y

C f

∀a ∈ A : f ; g(a) ≡ f (a)}



{g(b) | b ∈

C

the arrow eDep;dEmps is the composition of the arrows eDep and dEmps. We believe that the usage of the [subset] predicate can help domain experts and developers in grasping the rationale of the constraint.

44

A. Rutle et al.

Fig. 2. A diagrammatic specification S = (G(S), S(Π))

Fig. 3. The graph G(S) of the diagrammatic specification S

Finally, rule 7 forces that a set of employees working for a controlling department must not be enrolled in the same controlled project more than once. Again, it is possible to specify such a rule diagrammatically by using the constraints ([jointly-key], δ8 ), ([single-valued], δ5 ) and ([single-valued], δ7 ). The semantics of the constraint [jointly-key] and [subset] are further explained in Example 2 (Section 3.3), where instances of S are taken into consideration. 3.3 Semantics of Diagrammatic Specifications In this section we discuss the semantics of diagrammatic predicates and show how this semantics is used to check whether a graph is an instance of a diagrammatic specification. As mentioned, for object-oriented modelling it is appropriate to interpret nodes in the underlying graph G(S) of a Σ-specification S by sets, and arrows by multi-valued functions. The main idea in these modelling languages is that the semantics of a model is given by a set of instances. To reflect this idea in DPF, we consider arbitrary graphs I together with a graph homomorphism ι : I → G(S) as instances of G(S). That is, a

A Diagrammatic Formalisation of MOF-Based Modelling Languages

45

Table 2. Some diagrams (p, δ) ∈ S(Π) (p, δ) ([total], δ1 )

α(p) 1

([cover], δ2 )

1

([inverse], δ3 )

1d

δ(α(p))

x

/2

x

/2

x

$

/ Employee

dEmps

Department

eDeps

2

/ Department

eDeps

Employee

Employee m

-

Department

dEmps

y x

/ 2 Enrolment

eDep

/ Department

([single-valued], δ5 ) 1

x

/ 2 Enrolment

eDep

/ Department

1

x

/2

([single-valued], δ7 ) 1

x

/2

x

$

([total], δ4 )

([total], δ6 )

([subset], δ8 )

1

1 1> > z



4

x

Enrolment

eP ro

/ P roject

eP ro

/ P roject

eEmps

:2

-

1 Employee

Enrolment eDep;dEmps

y

([jointly-key], δ9 )

Enrolment

/2

Enrolment P

3

Employee

>>y >> >

eDep / Department PPP PPeP PPro eEmps PPP P' 

P roject

f

node A in G(S) is interpreted by the set ι−1 (A) of nodes in I; and an arrow A − →B in G(S) is interpreted by the multi-valued function from ι−1 (A) into ℘(ι−1 (B)) in I. The latter is represented by the set ι−1 (f ) of arrows in I (see Example 1). To define the concept of instance of a Σ-specification we have to fix, however, the semantics of the constraints offered by the signature Σ. We can mention at least three ways to define semantics of these constraints. In Table 1, we have used the “elementwise” language of set theory. In category theory, where the signature is limited to limit, colimit and commutativity constraints, the semantics of these constraints is mathematically “pre-defined” for any category according to the universal nature of these special constraints. In implementations of modelling languages, we rely on a less descriptive but more algorithmic way to define the semantics of constraints, e.g. we implement a validator for each constraint. However, in order to analyse and formalise MOF-based modelling languages and models which are defined by these languages, we do not need to decide for one of the mentioned possibilities. We just need to know that any of these possibilities defines “valid instance of predicates”. Definition 4 (Instances of Predicates). A semantic interpretation of a signature Σ = (Π, α) is given by a mapping that assigns to each p ∈ Π a set [[p]] of graph homomorphisms τ : O → α(p) called valid instances of p, written τ  p, where O may vary over all graphs.

46

A. Rutle et al.

x / Example 1. We consider the (arity) graph 1 2 and the following instances of this graph: ⎛ ⎞ e1 / b1 a1 ⎜ ~? ⎟ x / ⎜ ⎟ e2 ~~ 2) τ1 : ⎜ ⎟ −→ (1 ~~ ~ ⎝ ⎠ ~~ a2 b2 ⎞ ⎛ e1 / b1 a1 ⎜ ~? ⎟ ⎟ x / ⎜ e2 ~~ 2) τ2 : ⎜ ⎟ −→ (1 ~~ ~ ⎠ ⎝ ~~ e3 / b2 a2 x / Note that the instances of 1 2 are what is often called “bipartite graphs”. These instances represent multi-valued functions f1 , f2 : {a1 , a2 } → ℘({b1 , b2 }), where ℘({b1 , b2 }) = {∅, {b1}, {b2 }, {b1 , b2 }}, with f1 (a1 ) = f1 (a2 ) = f2 (a1 ) = {b1 } and f2 (a2 ) = {b1 , b2 }. We have τ1 , τ2  [total] and τ2  [cover], but τ1  [cover].

To check the validity of a constraint in a given instance of G(S), it is obviously enough to inspect only that part of G(S) which is affected by the constraint. This kind of “restriction to a subpart” is described as the pullback construction, i.e. a generalisation of δ ι the inverse image construction. Given a span p − →G← − I the pullback PO

δ ι∗

O∗

/G O ι

P.B. δ∗

/I

is given by a graph O∗ and graph homomorphisms δ ∗ : O∗ → I, ι∗ : O∗ → P . Nodes in in O∗ are given by those nodes i in I such that there exists a node n in P with δ(n) = ι(i). Analogously, the arrows fa in O∗ are given by those arrows f in I such that there exists an arrow a in P with δ(a) = ι(f ). Moreover, we have δ ∗ (in ) = i, δ ∗ (fa ) = f and ι∗ (in ) = n, ι∗ (fa ) = a, respectively. In case f is injective there is no need to distinguish between different copies of nodes or arrows from I; thus we can drop the subscripts n or a, respectively, as we have done in Example 2. Definition 5 (Instance of Specification). An instance of a diagrammatic specification S = (G(S), S(Π)) is a graph I together with a graph homomorphism ι : I → G(S), written (I, ι), such that for each constraint (p, δ) ∈ S(Π) we have ι∗ ∈ [[p]], where ι∗ : O∗ → α(p) is given by the following pullback diagram α(p) O ι∗

O∗

δ

/ G(S) O ι

P.B. δ∗

/I

A Diagrammatic Formalisation of MOF-Based Modelling Languages

47

The following example revisits the motivating example in Section 2.1 and explains the usage of Definition 5 to check whether a given graph is an instance of the specification. Example 2. Fig. 4 shows an instance ι : I → G(S) of the diagrammatic specification S from Fig. 2. In Fig. 4, we have used some “user-friendly” notations: e.g. the roundedrectangle with the name Enrolment stands for the set ι−1 (Enrolment), the arrows eEmps1 and eEmps2 represent the set ι−1 (eEmps), and, the ellipsis with the elements e2 and e3 stands for one of the subsets in ℘(ι−1 (Employee)).

Fig. 4. An instance, ι : I → G(S), of the Σ-specification S in Fig. 2

To verify that (I, ι) is indeed an instance of S, we need to construct the pullback for each constraint (p, δ) in S(Π) (see Table 2). In this example, we validate three of these constraints. First, we look at the constraint ([jointly-key], δ9 ). The pullδ

ι

ι∗

9 G(S) ← − I will be α([jointly-key]) ←− back of α([jointly-key]) −→

δ∗

9 O∗ −→ I, where ι∗ : O∗ → α([jointly-key]) is as in Fig. 5. The graph homomorphism ι∗ is a valid instance of the predicate [jointly-key] since the semantics of the constraint which is set by the predicate is not violated. The constraint here is that every enri ∈ ι−1 (Enrolment) must be uniquely identified by a triple ℘(ι−1 (Employee)), ℘(ι−1 (Department)), ℘(ι−1 (P roject)) . Moreover, consider the constraint ([single-valued], δ5 ) ∈ S(Π). Since the arrow eDep is single-valued, the target of eDep1 and eDep2 must be singleton sets, i.e. |ι−1 (eDep)| ≤ 1, This constraint is not violated since the subsets {d1 } and {d2 } are both containing only one value from ι−1 (Department). In contrast, since the arrow eEmps is not marked with [single-valued], the target of eEmps2 , for example, is the subset {e1 , e2 } which contains more than one element from ι−1 (Employee). Finally, we will look at the subset constraint ([subset], δ8 ). We can check that this constraint is not violated by verifying that eEmps1 (enr1 ) ⊆ dEmps1 ( eDep1 (enr1 )) and eEmps2 (enr2 ) ⊆ dEmps2 (eDep2 (enr2 )).

48

A. Rutle et al.

Fig. 5. ι∗ : O∗ → α([jointly-key])

4 MOF-Based Modelling Languages In the previous sections, we argued that formal diagrammatic modelling languages are necessary in order to specify models which are of the quality that MDE requires. Most of present-day’s diagrammatic modelling languages share MOF as their metamodel. However, these languages have still some issues when it comes to their formal semantics, especially considering the relationship between models in different modelling layers. Therefore in this section, we focus on the formalisation of MOF-based modelling languages. First we explain the 4-layered modelling architecture of OMG standards. Then, we show how modelling languages are represented by modelling formalisms in DPF by presenting a generic and flexible scheme for the formalisation of metamodelling. Finally, the notions of metamodels and reflexive (meta)models will be formalised in terms of DPF. 4.1 The 4-Layered Modelling Architecture OMG defines four layers of modelling (Table 3). Models in the M0 -layer are called instances which represent the state of a running system. These instances must conform to models in the M1 -layer, which are structures specifying what instances should look like. These models, in their turn, must conform to a metamodel in the M2 -layer, against which models can be checked for validity. Metamodels correspond to modelling languages, for example UML and CWM (Common Warehouse Metamodel ). The highest modelling layer (as defined by OMG) is the M3 -layer where MOF is located. A model at this layer is often called meta-metamodel; it conforms to itself and it is used to describe metamodels (Fig. 6a). The concepts presented in Section 3 allow to model directly the layers M1 and M0 as Σ-specifications and instances (I, ι) of Σ-specifications, respectively. In Fig. 6b,

A Diagrammatic Formalisation of MOF-Based Modelling Languages

49

Table 3. OMG’s modelling layers Modelling layers M3 : Meta-metamodel M2 : Metamodel M1 : Model M0 : Instance

OMG Standards and examples MOF UML language A UML model: Class Person with attributes name and address An instance of Person: “Ola Nordmann” living in “Sotraveien 1, Bergen”

ιS3

conf ormsT o



MO 3 conf ormsT o

MO 2 conf ormsT o

MO 1 conf ormsT o

M0 (a) OMG’s 4-layered modelling hierarchy.



S3 (Π3 )

Π3 _ _ _/ G(S3 )

O

ιS2 S2 (Π2 ) Π2 _ _ _/ G(S2 )

O

ιS1 S1 (Π1 ) Π1 _ _ _/ G(S1 )

O

ιS0

S0 (b) OMG’s 4-layered modelling hierarchy in DPF.

Fig. 6. OMG’s 4-layered modelling hierarchy, and its formalisation in DPF

these two layers are shown as a Σ1 -specification S1 and an instance (S0 , ιS0 ). We have chosen the names Σ1 = (Π1 , α1 ), (S0 , ιS0 ) and S1 = (G(S1 ), S1 (Π1 )) to underline the correspondence to the modelling layers of OMG. In this section, the layers M2 and M3 will be formalised. 4.2 Modelling Formalisms Each modelling language L, which is located in the M2 layer, is reflected in DPF by a diagrammatic signature ΣL and a metamodel M ML . L-models are represented by ΣL -specifications where the underlying graphs are required to be instances of the metamodel M ML . Moreover, the metamodel M ML itself is assumed to be a ΣMML specification, for an appropriate meta-signature ΣMML . We will call such a triple (ΣL , M ML , ΣMML ) for a modelling formalism FL . In Section 3 we formalised the concept of a model and of instances of a model, i.e. we defined a syntax and a semantics for diagrammatic specifications. In this sense, metamodelling means that syntax of a model is treated as a semantic entity to be specified and constrained by a another model, i.e. by a syntax, at a higher level of abstraction. Especially, we are interested to constrain, in an appropriate way, the underlying graphs

50

A. Rutle et al.

of these models. This “recursion step” within modelling (from syntax to semantics) can be formalised by the following definitions. Definition 6 (Conformance). Given the diagrammatic signatures Σ1 = (Π1 , α1 ), Σ2 = (Π2 , α2 ) and a Σ2 -specification S2 = (G(S2 ), S2 (Π2 )), a Σ1 -specification S1 = (G(S1 ), S1 (Π1 )) conforms to S2 if there exists a graph homomorphism ιS1 : G(S1 ) → G(S2 ) such that (G(S1 ), ιS1 ) is an instance of S2 . S2 (Π2 ) Π2 _ _ _/ G(S2 ) O ιS1 S1 (Π1 )

Π1 _ _ _/ G(S1 ) Definition 7 (Modelling Formalism). A modelling formalism F = (Σ1 , S2 , Σ2 ) is given by diagrammatic signatures Σ1 = (Π1 , α1 ) and Σ2 = (Π2 , α2 ), and a Σ2 specification S2 = (G(S2 ), S2 (Π2 )) called the metamodel of F . An F -specification is a Σ1 -specification S1 = (G(S1 ), S1 (Π1 )) which conforms to S2 . 4.3 Meta-formalism and Reflexive (Meta)Models As discussed above, a modelling language L can be formalised as a modelling formalism F = (Σ1 , S2 , Σ2 ), i.e. (ΣL , M ML , ΣMML ). The idea is to describe the syntactic constraints on S2 by a meta-language M L, i.e. by another modelling formalism FML = (Σ2 , S3 , Σ3 ) such that S2 conforms to S3 (see Fig. 6). Note that the same instance-model pattern is used for the definition of the relationship between instances and model; between models and metamodel; as well as between metamodels and metametamodel. In principle, this chain of metamodelling, i.e. for a modelling formalism Fi = (Σi−1 , Si , Σi ) finding a meta-formalism Fi+1 = (Σi , Si+1 , Σi+1 ) such that Si conforms to Si+1 , can continue ad infinitum. However, if at every modelling layer i + 1, we choose Σi+1 as minimal as possible, i.e. only keep the predicates that are needed to constrain the underlying graph of models at layer i, then this chain ends in an endpoint. Moreover, since Σi+1 only specifies syntactic restrictions on the underlying graph of Si , it is likely Graph

O

ιSi

ιSi Si (Πi ) Πi _ _ _ _ _ _ _/ G(Si )

O

O

ιSi−1

ιSi−1 (Π

)

i−1 i−1 Πi−1 _ _ _ _ _ _/ G(Si−1 )

S

(a) Natural endpoint for metamodelling.



Si (Πi ) Πi _ _ _ _ _ _ _/ G(Si )

(Π

)

i−1 i−1 Πi−1 _ _ _ _ _ _/ G(Si−1 )

S

(b) A reflexive metamodel.

Fig. 7. Modelling layers and reflexive metamodels

A Diagrammatic Formalisation of MOF-Based Modelling Languages

51

that Πi+1 is less expressive than Πi . Thus, the natural endpoint of such a chain of decreasing expressiveness will be a modelling formalism Fn = (Σn−1 , Graph, ∅),

where by Graph we mean • (Fig. 7a). This indicates that graph is the basic structure on which the whole framework is based. Another way to avoid progress ad infinitum is to collapse a potentially infinite chain into a single loop, i.e. to stop when Σi+1 is expressive enough to describe the restrictions which are put on the underlying graph of Si+1 (see Fig. 7b). This motivates the concept of reflexive (meta)model. Definition 8 (Reflexive Metamodel). A Σ-specification S = (G(S), S(Π)) is reflexive if S conforms to itself, i.e. if S is a (Σ, S, Σ)-specification. Fig. 7b shows a reflexive metamodel Si of a modelling formalism Fi = (Σi−1 , Si , Σi ). Note that ιSi will not be, in general, the identity morphism (see Section 5). Examples of reflexive metamodels are MOF (Fig. 6) and Ecore (Fig. 8). A formalisation of the latter is outlined in the next section. ιM

instanceOf

Ecore O

Ecore

ΠEcore _ _ _ _ _ _/ G(MEcore )

instanceOf

M odel O

O

ιS S(ΠCore ) ΠCore _ _ _ _ _ _ _/ G(S)

O

instanceOf

ιI

Instance (a) EMF’s modelling hierarchy.



MEcore (ΠEcore )

I (b) EMF’s modelling hierarchy in DPF.

Fig. 8. EMF’s modelling hierarchy and its formalisation in DPF

5 Case-Study: Formalisation of EMF In this section, we will use DPF to build a diagrammatic modelling formalism for EMF. EMF is a Java open source framework for modelling, data integration and codegeneration [2]. EMF is an implementation of Essential MOF (EMOF), which is a subset of MOF 2.0, but its metamodel is called Ecore to avoid confusion with MOF. EMF is a widely-used modelling framework because: – It is considered a low cost entry to the employment of MDE. – It has a clear semantics due to the simplicity of its metamodel. – There are many tools which are built around EMF and made available as Eclipse plug-ins. Software models in EMF are represented by core models. Core models are located in the M1 -layer of the OMG architecture. These models specify the structure of instances – the M0 -layer. That is, the types of objects that make up instance models,

52

A. Rutle et al.

Fig. 9. MEcore , the metamodel of FEM F , as a ΣEcore-specification

the data they contain and the relationships between them [23]. The metamodel of core models – Ecore – specifies the structure of the core models and is located in the M2 layer of the OMG architecture. Moreover, Ecore is reflexive, meaning that it specifies its own structure (Fig. 8a). Note that EMF has only three layers since Ecore is not intended to be the metamodel of a collection of other languages as it is the case for MOF. Fig. 8b shows EMF’s modelling hierarchy in view of DPF. EMF corresponds to a modelling formalism FEMF = (ΣCore , MEcore, ΣEcore ), where ΣCore = (ΠCore , αCore ), ΣEcore = (ΠEcore , αEcore ), and the metamodel is a ΣEcorespecification MEcore = (G(MEcore ), MEcore(ΠEcore )). Core models correspond to ΣCore -specifications S = (G(S), S(ΠCore )) which conform to MEcore. In addition, MEcore is a ΣCore -specification which conforms to MEcore. This is because MEcore is reflexive and ΣEcore is part of ΣCore . In some application contexts, we extend ΣCore with additional constraints and validators for these constraints. This is to enable software modellers to put additional constraints on core models. These additional constraints may be needed since Ecore supports only a minimal set of constraints, such as; multiplicity constraints and containment- and bidirectional references. In practice, constraints in EMF models (which are specified in OCL or Java) will consist of writing an Eclipse plug-in based on the EMF Validation Framework, or will involve the usage of the EValidator API of EMF. In both cases, models and constraints live in two different technical spaces – something which may lead to challenges as mentioned in Section 2. In the following, we describe the kernel of the metamodel MEcore in terms of DPF; in addition, we show that the the metamodel is reflexive. The kernel of MEcore, which is adopted from [23], is shown in Fig. 9. Each model element in MEcore is typed over MEcore by the graph homomorphism ιMEcore : G(MEcore ) → G(MEcore ). This typing is as follows: ιMEcore (EClassifier) = ιMEcore (EDataType) = ιMEcore (EClass) = EClassifier ιMEcore (eSuperType1 ) = ιMEcore (eSuperType2 ) = ιMEcore (eReferences) = ιMEcore (eAttributes) = ιMEcore (eStructuralFeatures) = eStructuralFeatures

A Diagrammatic Formalisation of MOF-Based Modelling Languages

53

We have some concerns about the mixture of Ecore with Java, for example in MEcore the super-type (or superclass) of EClass and EDataType is an EClassifier. In other words, each instance of the metaclass EClassifier is the super-type of either an EClass or an EDataType. In the specification of Ecore as an EMF-model, the semantics of the super-type arrows is given by the semantics of the “extends” keyword in Java. That is, something outside the modelling language is used to specify Ecore. In contrast, we incorporate this constraint into the framework by means of a single predicate [disjoint-cover] (see Table 1).

6 Related Work Diagrammatic modelling and formalisation of metamodelling have been extensively discussed in the literature. Here we discuss some of the approaches to the specification of constraints and formalisation of MOF-based modelling languages. The work in [5] exploits the higher-order nature of constructive type theory to uniformly treat the syntax of models, metamodels as well as the MOF model itself. Models are formalised as terms (token models) and can also be represented as types (type models) by means of a reflection mechanism. This formalisation ensures that correct typing corresponds to provably correct models and metamodels. The works in [6,24] use algebraic specification to give formal semantics to MOF. In [6] MOF-based metamodels are considered to be graphs with attributed objects as nodes and references as arrows. These graphs are represented by specifications in Membership Equational Logic (MEL), i.e. the logical power of MEL is essentially used to define the concept of a finite (multi)set at different places and levels. This formal semantics is made executable by using the Maude language, which directly supports MEL specifications. In this formalisation, metamodels are seen to play several roles: as data, as type or as theory; these roles are formally expressed by metamodel definition, model type, and metamodel realisation, respectively. In [24] a metamodelling framework, which is also based on Maude, is presented. In this framework, graphs are represented as terms by taking advantage of the underlying term matching algorithm modulo associativity and commutativity. The ability of Maude to execute the specifications allows the implementation of some key operations on models, such as model subtyping, type inference, and metric evaluation. Epsilon Validation Language (EVL) is one of the languages in the Epsilon model management platform [25]. EVL extends OCL conceptually (as opposed to technically) to provide a number of features such as support for constraint dependency management and access to multiple models of different metamodels and technologies. Epsilon provides a mature tool support for management of Ecore-based models. It will be interesting to investigate the formal basis of this language and compare it to DPF in a future work. Visual OCL (VOCL) [26] is an effort to define a graphical visualisation for OCL. It extends the syntax of UML and is used to define constraints on UML models. VOCL allows developers to put models together with constraints without leaving the graphical level of abstraction. However, VOCL does not extend the formal semantics of OCL. It only visualises constraints which are specifiable in OCL. An interesting and open

54

A. Rutle et al.

questions is whether the OCL constraints that are visualised can indeed be seen as sortwise constraints, and thus formalised in DPF. Alloy [27] is a structural modelling language which is capable of expressing complex structural constraints and behaviour. Model analysis in Alloy is based on the usage of FOL to translate specifications into boolean expressions which are automatically evaluated by a boolean satisfiability problem (SAT) solver. Then for a given logical formula F , Alloy attempts to find a model which satisfies F . Alloy models are checked by using the Alloy analyser which attempts to find counterexamples within a limited scope which violates the constraints of the system. Even though Alloy cannot prove the system’s consistency in an infinite scope, the user receives immediate feedback about the system’s consistency.

7 Conclusion and Future Work Independent of the importance of an appropriate and unambiguous visualisation we focus in DPF on the precise syntax (and semantics) of diagrammatic constraints. We allow diagrammatic constraints which modellers can use and reason about. An advantage of using these diagrammatic constraints is that they belong to the same conceptual space as the models which they constrain. This saves software developers from the challenges caused by mixing different technical and conceptual spaces. The paper argues that DPF is a promising candidate for an appropriate diagrammatic specification framework for MDE. Besides the diagrammatic appearance, this is mainly due to the rigorous mathematical foundation of the DPF. We have shown DPF’s usage as a formal diagrammatic framework for the definition of models and modelling languages. In particular, we have presented a generic and flexible scheme for the formalisation of metamodelling and discussed the 4-layered architecture of OMG in view of this scheme. This scheme has justified the extension of type- and typed graph concepts [11] with support for diagrammatic constraints. In view of graph transformations, a characterisation of our approach can be given by considering diagrammatic specifications as type graphs with constraints, and instances as typed graphs which satisfy those constraints. Moreover, an application of the approach is illustrated by representing EMF in DPF. The version of DPF presented in the paper distinguishes clearly between signatures and (meta)models. And it seems that only this “separation of concerns” allows for a clean and well-structured formalisation of metamodelling. In most modelling languages, however, predicates and constraints are internalised in the sense that predicates are represented by classes in the metamodel and constraints by corresponding objects. It seems that such kind of internalisation can be formalised by adapting and generalising an abstract categorical construction presented by Makkai in [18]. We intend to investigate this conjecture in one of our more foundational future work. The proposed visualisation of the predicates can be modified to meet the taste of the developers of the modelling language which uses the signature. A formalisation of parts of UML within DPF is presented in [4,28]. In a future work, we intend to define a transformation for the combination of UML with OCL to DPF. This will provide users of the framework with means to specify their models formally and diagrammatically

A Diagrammatic Formalisation of MOF-Based Modelling Languages

55

without the steep learning curve of a new visualisation or notation. To define such a transformation we need to investigate to what extend OCL constraints can be considered as sort-wise constraints. Due to space limitations we have not considered dependencies between predicates [7] even if they are highly relevant for practical applications of DPF. These dependencies provide a kind of rudimentary logic for DPF. The development of a fully fledged logic for DPF, allowing especially to reason about and to verify models, is one of our main future objectives. The development of a prototype implementation of the DPF framework based on the Eclipse platform is now in its early stage; we plan to boost this development in a near future. The final prototype will also support version control, whose underlying techniques are analysed in [10].

References 1. Object Management Group: Unified Modeling Language Specification (November 2007), http://www.omg.org/cgi-bin/doc?formal/2007-11-04 2. Eclipse Modeling Framework, http://www.eclipse.org/emf/ 3. Object Management Group: Meta-Object Facility Specification (January 2006), http://www.omg.org/cgi-bin/doc?formal/2006-01-01 4. Diskin, Z.: Mathematics of UML: Making the Odysseys of UML less dramatic. In: Practical foundations of business system specifications, pp. 145–178. Kluwer Academic Publishers, Dordrecht (2003) 5. Poernomo, I.: A Type Theoretic Framework for Formal Metamodelling. In: Reussner, R., Stafford, J.A., Szyperski, C. (eds.) Architecting Systems with Trustworthy Components. LNCS, vol. 3938, pp. 262–298. Springer, Heidelberg (2006) 6. Boronat, A., Meseguer, J.: An Algebraic Semantics for MOF. In: Fiadeiro, J.L., Inverardi, P. (eds.) FASE 2008. LNCS, vol. 4961, pp. 377–391. Springer, Heidelberg (2008) 7. Diskin, Z., Wolter, U.: A Diagrammatic Logic for Object-Oriented Visual Modeling. In: ACCAT 2007: 2nd Workshop on Applied and Computational Category Theory. ENTCS, vol. 203, pp. 19–41. Elsevier Science Publishers B.V., Amsterdam (2008) 8. Rutle, A., Wolter, U., Lamo, Y.: A Diagrammatic Approach to Model Transformations. In: EATIS 2008: Euro American Conference on Telematics and Information Systems (to appear) 9. Rutle, A., Wolter, U., Lamo, Y.: A Formal Approach to Modeling and Model Transformations in Software Engineering. Technical Report 48, Turku Centre for Computer Science, Finland (2008) 10. Rutle, A., Rossini, A., Lamo, Y., Wolter, U.: A Category-Theoretical Approach to the Formalisation of Version Control in MDE. In: Chechik, M., Wirsing, M. (eds.) FASE 2009. LNCS, vol. 5503, pp. 64–78. Springer, Heidelberg (2009) 11. Ehrig, H., Ehrig, K., Prange, U., Taentzer, G.: Fundamentals of Algebraic Graph Transformation. Springer, Heidelberg (March 2006) 12. Object Management Group: Object Constraint Language Specification (May 2006), http://www.omg.org/cgi-bin/doc?formal/2006-05-01 13. Warmer, J., Kleppe, A.: The Object Constraint Language: Getting your models ready for MDA, 2nd edn. Addison-Wesley, Reading (2003) 14. Markovi´c, S., Baar, T.: Refactoring OCL annotated UML class diagrams. Software and System Modeling 7(1), 25–47 (2008) 15. Taentzer, G., Rensink, A.: Ensuring Structural Constraints in Graph-Based Models with Type Inheritance. In: Cerioli, M. (ed.) FASE 2005. LNCS, vol. 3442, pp. 64–79. Springer, Heidelberg (2005)

56

A. Rutle et al.

16. Barr, M., Wells, C.: Category Theory for Computing Science, 2nd edn. Prentice Hall International Ltd., Hertfordshire (1995) 17. Fiadeiro, J.L.: Categories for Software Engineering. Springer, Heidelberg (May 2004) 18. Makkai, M.: Generalized Sketches as a Framework for Completeness Theorems. Journal of Pure and Applied Algebra 115, 49–79, 179–212, 214–274 (1997) 19. Diskin, Z., Kadish, B.: Generic Model Management. In: Encyclopedia of Database Technologies and Applications, pp. 258–265. Idea Group (2005) 20. Diskin, Z., Dingel, J.: Mappings, Maps and Tables: Towards Formal Semantics for Associations in UML2. In: Nierstrasz, O., Whittle, J., Harel, D., Reggio, G. (eds.) MoDELS 2006. LNCS, vol. 4199, pp. 230–244. Springer, Heidelberg (2006) 21. Bottoni, P., Koch, M., Parisi-Presicce, F., Taentzer, G.: A Visualization of OCL Using Collaborations. In: Gogolla, M., Kobryn, C. (eds.) UML 2001. LNCS, vol. 2185, pp. 257–271. Springer, Heidelberg (2001) 22. Cicchetti, A., Di Ruscio, D., Pierantonio, A.: A Metamodel Independent Approach to Difference Representation. Journal of Object Technology (Special Issue on TOOLS Europe 2007) 6(9), 165–185 (2007) 23. Budinsky, F., Merks, E., Steinberg, D.: EMF: Eclipse Modeling Framework 2.0, 2nd edn. Addison-Wesley Professional, Reading (2006) 24. Romero, J.R., Rivera, J.E., Durán, F., Vallecillo, A.: Formal and Tool Support for Model Driven Engineering with Maude. Journal of Object Technology 6(9), 187–207 (2007) 25. Epsilon: Book, http://epsilonlabs.wiki.sourceforge.net/Book 26. Visual OCL: Project Web Site, http://tfs.cs.tu-berlin.de/vocl/ 27. Alloy: Project Web Site, http://alloy.mit.edu/community/ 28. Diskin, Z., Easterbrook, S.M., Dingel, J.: Engineering Associations: From Models to Code and Back through Semantics. In: Paige, R.F., Meyer, B. (eds.) TOOLS Europe 2008. LNBIP, vol. 11, pp. 336–355. Springer, Heidelberg (2008)

Designing Design Constraints in the UML Using Join Point Designation Diagrams Vanessa Stricker1, Stefan Hanenberg2, and Dominik Stein2 1 Software Systems Engineering. Data Management Systems and Knowledge Representation Institute for Computer Science and Business Information Systems, University of Duisburg-Essen 45117 Essen, Germany [email protected] [email protected] [email protected] 2

Abstract. Design errors cause high costs during the development of software, since such errors are often detected quite late in the software development process. Hence, it is desirable to prevent design errors as early as possible, i.e. it is necessary to specify constraints on the software design. While there are a number of approaches available that permit to specify such constraints, a common problem of those is that the developers need to specify constraints in a different language than their design language. However, this reduces the acceptance of such approaches, as developers need a different language just for the purpose of specifying constraints. This paper proposes an approach that permits to specify constraints within the UML by using Join Point Designation Diagrams (JPDDs).

1 Introduction Checking constraints from the very beginning of the software development process is essential for the success of the resulting software. It is commonly accepted that errors in the early stages of software development cause very high costs later on. This also applies to recent approaches such as model-driven software development [16]. In this case, it is even more crucial to check whether the design artifacts (i.e. the models) fulfill their constraints since models are the main fragments that are used to generate the resulting software from. Disregards of any constraint at design time are most often detected not before code generation time or (even worse) when the generated code is finally being tested. Since the generation of code from non-trivial models can easily take several hours, disregards of design constraints are a critical factor for the success of a software project. Typically, a number of design constraints result from architectural decisions, or from (external) techniques that are (or have to be) applied in a project. Furthermore, software companies usually have a number of development guidelines that must be followed by the developers. While there are design constraints that cannot be expressed in a formal manner (e.g. guidelines such as “names of classes and methods M. Oriol and B. Meyer (Eds.): TOOLS EUROPE 2009, LNBIP 33, pp. 57–76, 2009. © Springer-Verlag Berlin Heidelberg 2009

58

V. Stricker, S. Hanenberg, and D. Stein

should reflect their semantics”) a large amount of constraints and guidelines can be formally expressed. Instead of defining such constraints in an external document, it is desirable to specify them such that they can be immediately deployed in order to identify corrupt model elements. There are already a number of approaches that permit to specify constraints [27]. However, in practice such approaches did not turn out to be easily applicable in actual projects. The main reason for this is that constraints need to be specified in languages that developers are not familiar with. As a consequence, developers need to learn new languages (in addition to their proper design language) in order to specify constraints. This also implies that developers need to learn a set of new tools that come with the constraint language. Such an approach is costly as developers need to be trained on the language and the tools. Furthermore, applying a new language is a potential source of errors because most often constraint languages introduce concepts that do not (or not directly) appear in the utilized design language. Furthermore, developers that check their models using some constraints (in case a constraint rejects a model) need to understand what exactly the intention of the constraint is and how the model can be adapted in a way that fulfills the constraint’s underlying motivation: in case they do not understand precisely the constraint language, it is hard for them to fix their model in order to fulfill given constraints. Software developers that are using graphical design languages such as the UML [21] are familiar with the language’s concepts as well as with its graphical notation (i.e. with the language's concrete syntax). In order to reduce the previously described problems, it is desirable to provide developers with means to specify constraints in terms of the design language (i.e. UML in this case) itself. That means that the constructs provided by the constraint language as well as their graphical representation should correspond to the constructs provided by the design language. This permits developers to communicate their design constraints in terms of constructs and notational means they already know. Such an approach would increase the ability to understand constraints as well as the ability to specify constraints. Furthermore, it would reduce the training costs as well as the potential sources of errors (caused by applying a new language). This paper presents a constraint language based on the (concrete syntax of) UML in order to increase the developers’ abilities to specify and to communicate design constraints in terms of a design language they are familiar with. This constraint language is based on Join Point Designation Diagrams (JPDDs, see [6, 24, 25, 26]) that were originally developed to model aspect-oriented software in the UML. In order to exemplify the urge for such a constraint language, section 2 presents a real-world problem scenario from a medium-sized company. Section 3 gives detailed description of the underlying problem. Section 4 introduces Join Point Designation Diagrams. Then, section 4 explains how JPDDs are being deployed for the specification of design constraints. Section 5 revisits the problem and expresses it in term of JPDDs. Section 6 presents related work. Finally, section 7 discusses and concludes the paper.

2 Motivation The following example describes a real-world scenario of a medium-sized software company that provides products for health-care insurance companies. Examples for

Designing Design Constraints in the UML Using Join Point Designation Diagrams

59

such products are information systems for insurants, configurations of insurances as well as billing. The company deploys a model-driven software development approach based on UML that requires the software developers to specify most parts of the software with UML models. The company provides its own code generator that generates about 95% of the code base. The remaining code is hand-coded at well-defined hooks provided by the generator. The company generates server-sided code for both the business logic and the middleware connections as well as client-sided code for the user interface. The product under consideration consists of about 15.000 UML classes whose transformation to code takes about eight hours. (Automated) deployment and testing takes about 3 days. The resulting application runs in a J2EE environment. Developers that specify UML models have to follow a large number of constraints which come from different origins. There are constraints that represent the best practices of the company, others are forced by the underlying architecture. Furthermore, there are technical constraints from the underlying code generator. For motivation purposes, we want to cite here a simple (and simplified) example of a constraint in use which needs to be considered by developers that are responsible for designing the user interface. The user interface is based on the Model-View-Controller pattern (MVC, [22, 2]). This pattern suggests the encapsulation of the model, the view and the controller in separate elements. The view is responsible for visualizing data, the controller handles events of the user, and the model represents the data being visualized by the view. For reasons of simplicity we will neglect the model in the pattern, i.e. we only consider controller and view in the following discussion. «view» MyView

1..* view

has

1 «controller» controller MyController

Fig. 1. Correct model of views and controllers

The company’s code generator (as well as the company’s best practices) requires that a view and a controller are represented by separate UML classes. Furthermore, the generator requires that each view is annotated with a stereotype «view» and that each controller is annotated with the stereotype «controller». Each view must be connected to a controller (and vice versa) by a bi-directional association. The connection must have the name “has” and the role names “view” and “controller” respectively. The cardinality of the association is restricted, too: while the cardinality of the controller role must be 1, the cardinality of the view role may be 1..*. Figure 1 illustrates an extraction of a class diagram that fulfills these constraints. Figure 2 illustrates a model that does not fulfill the constraints: the first view does not have the required stereotype. Furthermore, the association between the second view and the controller does not comply with the constraint (since the role name “controller” is missing). Each disregard of the above design constraints had an immensely negative impact on the subsequent development process as erroneous connections between views and controllers would make either the code generation or (even) the final program execution crash. So the company decided to install model checking facilities in order to prevent this.

60

V. Stricker, S. Hanenberg, and D. Stein

1..* view

has

FirstView

1 controller

«controller» ControllerA

«view» SecondView

1..* view

has

1

«controller» ControllerB

Fig. 2. Incorrect model of views and controllers

3 Problem Statement The company’s approach to install model checking facilities was to write Java programs that read the UML models (i.e. their XMI [20] files) and check their validity by consecutively invoking hard-coded constraint-checker functions on them. Figure 3 shows a code fragment that illustrates the specification of the constraint presented above. Furthermore, the company provided its developers with a natural language description of the constraints they had to obey. public void checkViewControllers() { Collection cls = ... get the UML classes... for(Iterator it=cls.iterator(); it.hasNext() ; ) { UMLClass c = (UMLClass) it.next(); if(c.hasSterotypeNamed(“view”) { Collection as = c.getAssociationsNamed(“has”); if (!as.isEmpty()) { ...

...check rolenames.....check cardinality...

for(Iterator it2 = as.iterator(); it2.hasNext() ;) { UMLClass controller = ...

...check for sterotype “controller”

} } else ...throw an Exception...

} ... }

Fig. 3. Constraint specified in Java

Doing so led to two new problems, though: First, the constraints in the Java programs and their natural explanations had to be synchronized. Secondly, it had to be guaranteed that there are no ambiguities in the natural language description. Otherwise developers would not be able to determine why their models were considered erroneous. Due to the complexity of a large number of constraints, a natural language description of the constraints turned out to be satisfying at all: developers were not satisfied with the natural descriptions because of their unclear semantics as well as their - from their point of view - inappropriate representation. One option to overcome these synchronization and ambiguity problems is to make all developers read and understand the constraint semantics already specified in Java. This option comes along with an immense learning effort: Developers need to learn the (Java) data model of the UML diagrams, its API and its usage. This option has not been considered as a serious solution by the company: the programming skills of the developers that mainly use the UML to express their artifacts were not considered to be strong enough. Furthermore, it turned out that developers still require too much time to

Designing Design Constraints in the UML Using Join Point Designation Diagrams

61

someUmlModel.contents ->select(c: Class | c.stereotype->exists(st | st.name='view') and not ( c.associations->exists(a | a.name='has' and a.allConnections->size=2 and a.allConnections->select(ae | ae.participant = self)->exists(ae | ae.name = 'view' and ae.isNavigable = true and ae.multiplicity.range.lower >= 1 and ae.multiplicity.range.upper select(ae | ae.participant self)->exists(ae | ae.name = 'controller' and ae.isNavigable = true and ae.multiplicity.range.lower = 1 and ae.multiplicity.range.upper = 1 and (let c1: Class = ae.participant.oclAsType(Class) in c1.stereotype->exists(st | st.name='controller') ) ) ) ) ) someUmlModel.contents ->select(c: Class | /* analogously for each controller */ )

Fig. 4. Constraint specified in OCL

understand the semantics of the constraint. I.e. a code representation did not turn out to be effective to understand and communicate design constraints among developers. Unfortunately, the option of using a common (model) constraint specification means such as the OCL [27] does not do better in that regard (Figure 4 illustrates a corresponding constraint specified by using the OCL): In order to understand OCL constraints, developers need to spend a large amount of their time to get familiar with this new (i.e. new for them) language – just as in the previous case. Apart from learning keywords, operators, syntax and semantics of OCL, they would furthermore be confronted with the full complexity of UML's meta-model – something that they usually do not encounter during daily work. Again, they would face a significant mapping problem since there are plenty of UML (meta)model elements which have no explicit visual representation in UML's concrete syntax. As a result, the representation of the constraints in terms of OCL is significantly different from the representational means they are familiar with. Although Visual OCL [1, 12] tries to overcome some of these disadvantages, it does not solve these problems. The existing UML notation elements which are used as much as possible within Visual OCL are not rich enough to express all elements defined in the OCL meta-model, which is why developers need to get used to the new notation elements and still would have to learn the semantics of the OCL constructs. Hence, while the real-world scenario described above showed the urgent need to check constraints at design time, the investigation of possible options to do so – in a synchronized, unambiguous way – points to the problems of using a second/a different language for specifying the constraints. Since the representation of the constraints differs fundamentally from the representation of the models themselves, it is hard for developers to understand the relationship between the constraint specification and their models. Consequently, the semantics of the constraints remains unclear to the developer whose model is in conflict with an existing constraint, which hinders him/her from comprehending and correcting his/her design errors. In other words: Current techniques are sufficient and mature enough to detect design errors. However, what is lacking is a means which can be used by constraint developers to easily communicate their design constraints to the application developers (so that

62

V. Stricker, S. Hanenberg, and D. Stein

they know what they have to look for), and which at the same time can be used to automatically check the compliance of the models to the constraints.

4 Specifying Design Constraints This section introduces a graphical constraint language based on the UML. Therefore, so-called Join Point Designation Diagrams (JPDDs) are introduced, which permit to specify queries on UML models. Based on that, the use of JPDDs for the purpose of specifying constraints is explained1. 4.1 Join Point Designation Diagrams (JPDDs) «Join Point Designation Diagrams» (JPDDs [24, 26]) are extensions of the UML [21] in order to specify queries on software artifacts specified in the UML. Hence, JPDDs permit developers to express selections on UML models in terms of the UML itself (i.e. its own concrete syntax). Although JPDDs provide general-purpose constructs for expressing such selections (cf. [24, 26]), the original motivation for proposing JPDDs came from the domain of aspect-oriented software development, where JPDDs have been applied to select those artifacts that need to be handled by aspects (see [6]). Due to their background, JPDDs provide means to specify static selection criteria (e.g. selection criteria on classdiagrams etc.) as well dynamic selection criteria (e.g. selection criteria on interactions, etc.). For the purpose of specifying design constraints that need to be statically checked, the dynamic selection criteria are irrelevant. Hence, the following subsections introduce constructs for specifying static selection criteria, whereas constructs for specifying dynamic selection criteria are neglected. For reasons of simplicity, we furthermore explain the semantics of JPDDs in an informal way. A specification of its semantics in OCL can be found in [26]. JPDDs are defined with help of the graphical notation provided by the UML. That notation makes use of symbols from existing and well-established graphical notations, such as interaction sequence diagrams, state diagrams, activity diagrams, and class/object diagrams. Nevertheless, in order to provide reasonable selection semantics, a small set of constructs needed to be added. This is described in the next subsection. Afterwards, the semantics of selection patterns is explained by means of an example. Finally, relationships between JPDDs are explained. 4.1.1 Extensions to the UML In order to provide reasonable selection semantics, JPDDs extend the UML by the following elements: Identifiers, parameter boxes, regular expressions, list-wildcards, and indirections. Within a selection pattern, a tuple of identifiers (see Figure 5a) may be assigned to particular elements in order to designate those parts of the queried software artifact that should be actually selected by the JPDD. JPDDs possess an export parameter box at their lower right corner (see Figure 5b) which lists the identifiers of all elements that have to be selected. 1

A tool-support for JPDDs and constraint checking using JPDDs can be found at http://www.dawis.wiwi.uni-due.de/forschung/foci/aosd/jpdds/

Designing Design Constraints in the UML Using Join Point Designation Diagrams

63

(c) Deviation Specification Means

(a) Identifiers



Fig. 7. An XML Assembly File

constraints must be checked like the verification that java elements denoted by name and linkedTo XML attributes respectively refer existing view elements and base elements and are type compatible. Once this checking step achieved, sources are automatically transformed in order to make them fully EJB compatible. Classes annotated with view.Class (respectively base.Class) inherit from functionalAspect.View class (respectively functionalAspect.Base). These classes are an EJB implementation of the patterns of Section 2.2. Let’s outline that the implementation policy has to ensure that view properties (attributes and roles) are virtual. Indeed, in conformance with the view pattern, access to these properties must be delegated to the connected base. So, the corresponding getter methods will be annotated by javax.persistence.Transient. Finally, packages can be compiled, archived with the assembly descriptor file and then deployed into an EJB server (step 3 of the development cycle).

4

Server Facilities

To support the assembly of coarse-grained view components with primary components at the execution level, we propose to extend the capacity of EJB servers with a new service. The role of this service consists in ensuring the proper execution of conceptual objects throughout their views according to the architectural choices made in Section 2.2. So, it has to address the following requirements : – support for adapting the behavior of view components to map their required elements with elements of primary system components as specified by the assembly XML descriptor. – support for managing conceptual objects made of base fragments and view fragments. As a consequence, the EJB life cycle of these fragments are closely linked (both at creation and deletion time). – support for multiple application of views.

A Coding Framework for Functional Adaptation

225

We present here an implementation of this view service using dynamic interception capacities of EJB servers for adaptation needs. These capacities are offered by extensible EJB servers such as the JBOSS/J2EE application server. JBoss is an open-ended middleware, in the sense that developers can extend middleware services by aspect introduction. The core technology is the JBoss/AOP framework [23] that intensively uses Java annotations. Adding a new service consists in adding a new aspect. Each aspect is made of its implementation and an XML description which declares necessary pointcuts and advices. Figure 8 shows the XML declaration of our new aspect.

Fig. 8. Aspect Declaration of the New Service

The pointcuts allow to intercept method calls to a view component, identified by our set of annotations. The advices consist in installing a JBOSS-AOP interceptor class called aspect.ViewInterceptor. This interceptor allows to execute some code around a method identified by one of the previous pointcuts. This code performs the mapping between required elements and provided ones using the assembly descriptor and Java introspection. This achieves a dynamic implementation of the Adapter pattern (see Section 2.2) thanks to reflexive invocation. As a consequence, the developer has no additional work other than using the previous annotation framework and writing the assembly descriptor file. Figure 9 illustrates the execution of the interception mechanism applied to the access to the view association ”location” like in Section 2.2. The getLocation method of the Search.Resource class is invoked. As this method is annotated with view.Role, the bindRole method of the ViewInterceptor interceptor is called. The interceptor retrieves the view identifier (getId()) and the name of the corresponding association role of the base class in the assembly XML descriptor file, here ”agency”. The getAgency() method can now be retrieved (using Java reflection) and dynamically invoked on this object, returning its agency. Finally, it is necessary to retrieve the corresponding view fragment (”location”) through the getView(id) call which terminates the substitution process. This example shows that this delegating behaviour is completely transparent for the developer when a view object is connected to a base object.

226

O. Caron et al.

:location

client:

:Resource

base:Car

:ViewInterceptor

ag:Agency

:XMLBinding

bindRole

getLocation()

getId() id getAttributeName(id, "Resource","location") "agency"

getBase() base

getAgency() ag getView(id)

location

location

...

Fig. 9. a Sequence Diagram of the View Service

1. Agency ag1 = (Agency) em.find(Agency.class,"key-ag1") ; 2. Search.Location location = (Search.Location) ag1.getView("SearchCar") ; 3. Collection<Search.Resource> resources = location.findAll() ; 4. // resources are view fragments of cars 5. 6. location =(Search.Location) ag1.getView("SearchClient") ; 7. resources = location.findAll() ; 8. // resources are view fragments of clients 9. ( (Search.Resource) resources.toArray()[0] ).getLocation() ; 10. 11. Car newCar=new Car() ; // => creation of view fragments ; 12. entityManager.persist(aCar) ; // the entity manager now 13. // manages automatically 14. // view fragments 15. StockManager.Stock stock = ag1.getView("StockManagerCarRental") ; 16. StockManager.Resource r = newCar.getView("StockManagerCarRental") ; 17. stock.add(r) ; // now ag1 manages newCar

Fig. 10. Using Views, an excerpt

The code excerpt of Figure 10 now illustrates how to use views. At the beginning of this code sequence, we suppose there already exists an entity Agency. By using the standard EJB entity manager em, we retrieve the existing agency ag1 (line 1). Lines 2-3 and lines 6-7 are identical except the view considered so that, the findAll method applied to the agency returns a collection of resources which are either cars or clients. Figure 9 details the execution of line 9 (treatment of a view association)

A Coding Framework for Functional Adaptation

227

The rest of the code creates a new base object and we use the StockManager component to add this car to the previous agency if its capacity allows it (this capacity control is a functionality provided by the view component). Line 11 instantiates a car, which calls the functionalAspect.Base super-class constructor that creates view fragments using the assembly descriptor. When the EJB entity manager persists an instance of the base class (line 12), all the connected view fragments are automatically managed by the entity manager. This mechanism is done because the EJB association between base and views is specified with the ”cascade persist” standard mode. In this example, we show that the persistent EJB life cycle between view fragments and base fragment are closely linked. Then, the view StockManagerCarRental is used (lines 15-17).

5

Related Works

We find studies on separation of functional concerns through reusable coarsegrained components which also starts from initial works on generic models. [13] has dealt in some detail with various implementations of reusable pluggable code components based on their framework notion. Different ways are compared (template classes, class hierarchies,. . . ). [12] proposes a mapping between ThemeUML and AspectJ. The solution is a pure AOP one where each model becomes an aspect that one can weave into a primary base. However, these studies have not been directly explored within the enterprise component server sphere. [3] proposes a MDA process which transforms PIM models to EJB components. The MDA process generates specific glue in order to reuse EJB components. This work offers a set of primitives for general model composition but does not deal with specific properties of functional concerns modeling. As far as coding is concerned, most of these approaches are model driven and generative. We ourselves already used a generative strategy to obtain EJB [5] implementation of our model templates. However, the generated EJB components were hard-wired, that is, specific to a particular system and then not reusable enough. Though the present framework is usuable in a standalone manner, it can be a valuable target technology for MDE environments. We are integrating such a targeting strategy into our proper UML CASE tool ”CocoaModeler” [22] using ”EMFScript” [26] for transformation needs. From a middleware point of view, there are a lot of works on extensible platforms [14,4]. They concentrate on non-functional concerns and facilitate the reuse of the functional core of the components throughout the corresponding services. The present work is dedicated to the structuring of this core itself according to functional (real world domain) concerns. All these works are complementary contributions to the quest for rich IS components, fully reusable whatever the concerns. More, despite the fact that we concentrated here on IS component servers, solutions are somehow generalizable to other domains, components architectures and middleware, like Corba and Fractal components that we have already experimented [22].

228

6

O. Caron et al.

Conclusion

This work provides separation of functional concerns in EJB servers by the definition of coarse-grained view components. It takes advantages of AOP facilities of an open-ended server to extend the EJB framework with new dedicated services. This extension does adopt the same writing idioms as standard EJB components: package structuring, an annotation language and XML descriptors for assembly. So it conforms to the EJB developer practice. Reusing view components is facilitated because one has only to describe the architecture assembly consisting in the specification of their connections to a base EJB standard package. The resulting architecture is compatible with the EJB standard and takes advantages of all existing J2EE services: persistence, transaction and security. As far as component architectures and middleware are concerned, it is a contribution to the reusability of coarsed-grained IS components complementary to researches on separation of non-functional concerns. Though it is a result that this framework is usable in a standalone manner by EJB developers and architects, it is also a good target technology for MDE environments. We are integrating such a targeting strategy into our MDE environment in order to automatize the transformation from modeling separation of functional concerns to platform specific code. Lastly, we have experimented the framework through a fully dynamic implementation that makes extensive use of reflection. However, reflection flexibility comes at the expense of performances. In future works, we will manage this issue by studying optimization techniques for reflection and experimenting more generative implementation strategies as we already studied for CORBA middleware [21].

References 1. Baniassad, E., Clarke, S.: Theme: An approach for aspect-oriented analysis and design. In: ICSE 2004: Proceedings of the 26th International Conference on Software Engineering, pp. 158–167. IEEE Computer Society Press, Los Alamitos (2004) 2. Bardou, D., Dony, C.: Split Objects: a Disciplined Use of Delegation within Objects. In: Proceedings of the 11th Conference on Object-Oriented Programming Systems, Languages, and Applications (OOPSLA 1996), San Jose, California, USA, October 1996, pp. 122–137. ACM Press, New York (1996) 3. Bouzitouna, S., Gervais, M.P., Blanc, X.: Model Reuse in MDA. In: International Conference on Software Engineering Research and Practice. CSREA Press (2005) 4. Bruneton, E., Riveill, M.: An architecture for extensible middleware platforms. Software - Practice and Experience 31(13), 1237–1264 (2001) 5. Caron, O., Carr´e, B., Muller, A., Vanwormhoudt, G.: A Framework for Supporting Views in Component Oriented Information Systems. In: Konstantas, D., L´eonard, M., Pigneur, Y., Patel, S. (eds.) OOIS 2003. LNCS, vol. 2817, pp. 164–178. Springer, Heidelberg (2003) 6. Caron, O., Carr´e, B., Muller, A., Vanwormhoudt, G.: An OCL Formulation of UML 2 Template Binding. In: Baar, T., Strohmeier, A., Moreira, A., Mellor, S.J. (eds.) UML 2004. LNCS, vol. 3273, pp. 27–40. Springer, Heidelberg (2004)

A Coding Framework for Functional Adaptation

229

7. Caron, O., Carr´e, B., Muller, A., Vanwormhoudt, G.: Mise en oeuvre d’aspects fonctionnels r´eutilisables par adaptation. Revue L’objet, Programmation par Aspects, Hermes Ed. 11(3), 105–118 (2005) 8. Caron, O., Carr´e, B., Debrauwer, L.: Contextualization of OODB schemas in CROME. In: Database and Expert Systems Applications, pp. 135–149 (2000) 9. Clark, T., Evans, A., Kent, S.: A Metamodel for Package Extension with Renaming. In: J´ez´equel, J.-M., Hussmann, H., Cook, S. (eds.) UML 2002. LNCS, vol. 2460, pp. 305–320. Springer, Heidelberg (2002) 10. Clarke, S.: Extending standard UML with Model Composition Semantics. In: Science of Computer Programming, vol. 44, pp. 71–100. Elsevier Science, Amsterdam (2002) 11. Clarke, S., Harrison, W., Ossher, H., Tarr, P.: Subject-Oriented Design: Towards Improved Alignment of Requirements, Design and Code. In: Procs. of the Object-Oriented Programming Systems, Languages and Applications Conference (OOPSLA), Denver (1999) 12. Clarke, S., Walker, R.J.: Towards a Standard Design Language for AOSD. In: AOSD 2002: Proceedings of the 1st International Conference on Aspect-Oriented Software Development, pp. 113–119. ACM Press, New York (2002) 13. D’Souza, D., Wills, A.: Objects, Components and Frameworks With UML: The Catalysis Approach. Addison-Wesley, Reading (1999) 14. Fleury, M., Reverbel, F.: The JBoss extensible server. In: Endler, M., Schmidt, D.C. (eds.) Middleware 2003. LNCS, vol. 2672, pp. 344–373. Springer, Heidelberg (2003) 15. Gamma, E., Helm, R., Johnson, R., Vlissides, J., Booch, G.: Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley Professional Computing, USA (1995) 16. Grundy, J., Patel, R.: Developing Software Components with the UML, Enterprise Java Beans and Aspects. In: ASWEC 2001: Proceedings of the 13th Australian Conference on Software Engineering, p. 127. IEEE CS, Los Alamitos (2001) 17. Helton, D.: Coarse-Grained Components as an Alternative to Component Frameworks. In: ECOOP Workshop on Component Oriented Programming, Lisbon, Portugal (1999) 18. Helton, D.: Closing the Gap between Business Functions and Software Components in Distributed Enterprise Systems. In: Weck, W., Reussner, R., Szyperski, C. (eds.) Proceedings of the 13th International ECOOP Workshop on Component-Oriented Programming (WCOP), Karlsruhe, Germany (October 2008) 19. JBoss Aspect-Oriented Programming, http://labs.jboss.com/portal/jbossaop 20. Kent, S.: Model Driven Engineering. In: Butler, M., Petre, L., Sere, K. (eds.) IFM 2002. LNCS, vol. 2335, pp. 286–298. Springer, Heidelberg (2002) 21. Muller, A.: Construction de syst`emes par application de mod`eles param´etr´es. PhD thesis, LIFL, Universit´e de Lille (2006) 22. Muller, A., Caron, O., Carr´e, B., Vanwormhoudt, G.: On Some Properties of Parameterized Model Application. In: Hartman, A., Kreische, D. (eds.) ECMDA-FA 2005. LNCS, vol. 3748, pp. 130–144. Springer, Heidelberg (2005) 23. Pawlak, R., Retaill´e, J.-P., Seinturier, L.: Foundations of AOP for J2EE Development. APress (2005) 24. Reenskaug, T., Wold, P., Lehne, O.A.: Working with Objects: The OORam Software Engineering Method. Prentice-Hall, Inc., Englewood Cliffs (1995)

230

O. Caron et al.

25. France, R., Georg, G., Ray, I.: Supporting Multi-Dimensional Separation of Design Concerns. In: AOSD Workshop on AOM: Aspect-Oriented Modeling with UML (March 2003) 26. Tombelle, C., Vanwormhoudt, G.: Dynamic and Generic Manipulation of Models: From Introspection to Scripting. In: Nierstrasz, O., Whittle, J., Harel, D., Reggio, G. (eds.) MoDELS 2006. LNCS, vol. 4199, pp. 395–409. Springer, Heidelberg (2006) 27. Wand, Z., Xu, X., Zhan, D.: A Survey of Business Component Identification Methods and Related Techniques. International Journal of Information Technology 2(4) (2005)

A Leasing Model to Deal with Partial Failures in Mobile Ad Hoc Networks Elisa Gonzalez Boix , Tom Van Cutsem , Jorge Vallejos , Wolfgang De Meuter, and Theo D’Hondt Programming Technology Lab - Vrije Universiteit Brussel - Belgium {egonzale,tvcutsem,jvallejo,wdmeuter,tjdhondt}@vub.ac.be

Abstract. In mobile ad hoc networks (MANETs) many partial failures are the result of temporary network partitions due to the intermittent connectivity of mobile devices. Some of these failures will be permanent and require application-level failure handling. However, it is impossible to distinguish a permanent from a transient failure. Leasing provides a solution to this problem based on the temporal restriction of resources. But to date no leasing model has been designed specifically for MANETs. In this paper, we identify three characteristics required for a leasing model to be usable in a MANET, discuss the issues with existing leasing models and then propose the leased object references model, which integrates leasing with remote object references. In addition, we describe an implementation of the model in the programming language AmbientTalk. Leased object references provide an extensible framework that allows programmers to express their own leasing patterns and enables both lease holders (clients) and lease grantors (services) to deal with permanent failures. Keywords: mobile ad hoc networks, partial failures, leasing, remote object references, language design.

1

Introduction

The recent progress in the field of wireless technology has proliferated a growing body of research that deals with mobile ad hoc networks (MANETs): networks composed of mobile devices connected by wireless communication links with a limited communication range. Such networks have two discriminating properties, which clearly set them apart from traditional, fixed computer networks [13]: intermittent connectivity of the devices in the network (called the volatile connections phenomenon) and lack of any centralized coordination facility (called the 

 

Author funded by Prospective Research for Brussels program of the Institute for the encouragement of Scientific Research and Innovation of Brussels (IWOIB-IRSIB). Postdoctoral Fellow of the Research Foundation - Flanders (FWO). Author supported by the VariBru project of the ICT Impulse Programme of IWOIB-ISRIB and the MoVES project of the Interuniversity Attraction Poles Programme of the Belgian State, Belgian Science Policy.

M. Oriol and B. Meyer (Eds.): TOOLS EUROPE 2009, LNBIP 33, pp. 231–251, 2009. c Springer-Verlag Berlin Heidelberg 2009 

232

E. Gonzalez Boix et al.

zero infrastructure phenomenon). The volatile connections phenomenon states that a disconnection should not be treated as a “failure” by default: due to the limited communication range of wireless technology, devices may move out of earshot unannounced. The resulting disconnections are usually transient : the devices may meet again requiring their connection to be re-established and allowing their collaboration to be continued where they left off. The zero infrastructure phenomenon states that it is more difficult to rely on server infrastructure (e.g. a name server for service discovery) since devices spontaneously join and disjoin the network due to their physical mobility. Our research focuses on distributed programming language support for mobile ad hoc networks. In this paper, we explore support to deal with the effects engendered by partial failures. Due to the above mentioned phenomena, it can be expected that many partial failures in MANETs are the result of temporary network partitions. However, not all network partitions are transient, e.g. a remote device has crashed or has moved out of the wireless communication range and does not return. Such permanent failures should also be dealt with by means of compensating actions, e.g. application-level failure handling code. Because it is impossible to distinguish a transient from a permanent failure [15], some arbitrary criteria should be agreed upon so that a device can determine when the logical communication with another remote device has terminated. Leasing [5] provides a solution to this problem based on the temporal restriction of resources [15]. A lease denotes the right to access a resource for a specific duration that is negotiated by the owner of a resource and a resource claimant (called the lease grantor and lease holder, respectively) when the access is first requested. The advantage of leasing is that allow both lease grantor and holder to distinguish a transient from a permanent failure by approximating permanent failures as disconnections that exceed the agreed time interval. However, to date no leasing model has been designed to operate in a mobile ad hoc network setting where leasing needs to (1) be combined with computational models that deal with transient failures, (2) provide different leasing patterns for the different kinds of collaboration that can be set up in MANETs, and (3) allow both lease holders and grantors to deal with permanent failures. This paper proposes a leasing model specially designed for MANETs. Our contribution lies in integrating leasing as a special kind of remote object reference, called leased object references, which tolerate both transient and permanent failures. The leased object references model incorporates different leasing patterns at its heart and allows programmers to schedule clean-up actions at both client and server side of a leased reference upon expiration. In addition, we describe a concrete instantiation of such a model in a distributed programming language, called AmbientTalk [13], as an extensible framework in which many leasing patterns can be expressed.

2

Leasing in Mobile Ad Hoc Networks

In this section, we discern a number of criteria that need to be exhibited by a leasing model for dealing with partial failures in MANETs and describe how

A Leasing Model to Deal with Partial Failures in Mobile Ad Hoc Networks

233

existing leasing models for distributed computing fail to deal with them. We derive these criteria from the analysis of an illustrative ad hoc networking application. Throughout this paper, we assume an object-oriented system where devices can be abstracted as a container for a set of objects implementing some functionality. Objects can be exported in the network either explicitly or implicitly by passing them as parameter or return value in a message sent to a remote object. We denote such remotely accessible objects as service objects. Service objects can be referenced from other machines by means of remote object references. 2.1

Running Example: The Mobile Music Player

Consider a music player running on mobile devices. The music player contains a library of songs. When two people using the music player enter one another’s personal area network (defined by for example the bluetooth communication range of their cellular phones), the music players set up a so-called ad hoc network and exchange their music library’s list (not necessarily the songs themselves). After the exchange, the music player can calculate the percentage of songs both users have in common. If this percentage exceeds a certain threshold, the music player can e.g. warn the user that someone with a similar musical taste is nearby. MusicPlayerServiceObject@ device A

MusicPlayerServiceObject@ device B openSession() session

session@ device B

uploadSong(song) 'ok endExchange()

Fig. 1. The music library exchange protocol

Figure 1 gives a graphical overview of the music library exchange protocol modeled via a distributed object-oriented system where communication between devices is asynchronous. The figure depicts the stages of the protocol from the point of view of the music player on the device A. In fact, this protocol is executed simultaneously on both devices. Once both devices have discovered each other, the music player running on A asks the remote peer B to start a session to exchange its library index by sending an openSession message. In response to it, the remote peer returns a new session object which implements methods that allow the remote music player to send song information (uploadSong) and to signal the end of the library exchange (endExchange).

234

2.2

E. Gonzalez Boix et al.

Analysis

This section describes a set of criteria that need to be exhibited by a leasing model to be used for dealing with partial failures in MANETs. While some of the above described criteria can be observed in different leasing models for distributed computing platforms and frameworks, to the best of our knowledge, no single leasing model exhibits all of the criteria presented in this section. Leasing an Intermittent Connection. A lease denotes a time restriction on the logical connection between lease holder and grantor. At the software level, a logical connection is represented by a communication link. Because of the volatile connections phenomenon in MANETs, communication links are often intermittent: devices often disconnect for an unknown period of time and then reconnect. However, that does not imply that the logical connection should be terminated. In our running example, once the service objects that represent the music player application have discovered one another, they need to set up a session to exchange their music libraries. Such a session is leased, such that both music players can gracefully terminate the exchange process in the face of a persistent disconnection. However, if a user moves temporarily out of range, the resulting transient disconnection should not cause the exchange to fail immediately as the user may reconnect. A leasing model for MANETs must take this into account: the disconnection of a device does not indicate that resources associated with the logical connection can already be cleared, since the communication link may be restored later. Leasing Patterns. Mobile ad hoc networking applications are conceived as a collection of several devices setting up a collaboration. As different kinds of collaboration can be set up, different kinds of leasing patterns are also possible. In our running example, devices collaborate in the exchange of their music library indexes. As long as the exchange is active, i.e. uploadSong messages are received, the session should remain active. The session could be thus exported using a lease which is automatically renewed each time it receives a message. The lease should be then revoked either explicitly when a client sends the endExchange message to indicate the end of the library exchange, or implicitly if the lease time has elapsed. Other collaborations may involve objects adhering to a single call pattern such as callback objects. For example, in asynchronous message passing schemes, callback objects are often passed along with a message in order for service objects to be able to return values. These callback objects are typically remotely accessed only once by service objects with the computed return value. In this case, a lease which is automatically revoked upon a method call is more suitable. A leasing model for MANETs should be flexible enough to allow developers to specify different leasing patterns to manage the lifetime of leases. Symmetric Expiration Handling. Leasing allows lease grantors to remain in control of the resource by maintaining the right to free the resource once the lease expires. In MANETs, both communicating parties should be aware of a lease expiration since it allows them to detect a permanent disconnection. Once a lease expires, both lease holder and grantor should be able to

A Leasing Model to Deal with Partial Failures in Mobile Ad Hoc Networks

235

properly react and do some clean-up actions in order to gracefully terminate their collaboration. In our running example, the session object is clearly only relevant within the context of a single music library exchange. If the exchange cannot be completed (e.g. due to a persistent network partition), the session object and the resources allocated during the session, e.g the partially uploaded library index, should be eventually reclaimed. A leasing mechanism for MANETs should allow both lease holder and lease grantor to deal with the termination of their logical connection. 2.3

Related Work

Leases were originally introduced as a fault-tolerant technique in the context of distributed file cache consistency [5]. Jain and Kircher introduced the concept of leasing as a software design pattern to simplify resource management in [6]. In distributed object-oriented systems, leasing has been used to describe the lifetime of remote objects in frameworks like Java RMI and .NET Remoting [10], and as a general abstraction for resource management in platforms like CORBA [2] and Jini [14]. In this section, we further evaluate these leasing mechanisms in the light of the criteria for a leasing model in MANETs. Java RMI. In Java RMI leases are tightly coupled to the distributed garbage collector (DGC) and are used as a way to manage the lifetime of remote objects in a fault-tolerant fashion. Although Java RMI integrates leasing with remote references, leases are only used to reclaim unused connected remote references. In fact, its leasing model is combined with a synchronous communication model (RPC) which does not decouple objects in time or synchronization [3], which makes it unsuitable for MANETs. Leases are transparent to the programmer as a part of the DGC and the lease duration is controlled by means of a system property. If developers need to deviate from the default leasing behaviour, the system provides an interface to the DGC based on so-called dirty and clean calls. Leasing patterns need to be built on top of these low-level operations. For example, automatic renewal of leases can be only accomplished by making lowlevel dirty calls on the remote references. Expiration handling is not provided upon lease expiration in Java RMI. If a client attempts to access a service object whose lease expired, an exception is raised. This allows clients to schedule some clean-up actions in the exception handler, but forces them to install exception handlers on every remote call. .NET Remoting framework. The .NET Remoting framework incorporates leasing in combination with the concept of sponsorship for managing the lifetime of remote objects [9]. Sponsors are third-party objects which are contacted by the framework when a lease expires to check if that party is willing to renew the lease. Clients can register a sponsor on a lease and thus decide on the lifetime of server objects. Similar to JavaRMI, the .NET Remoting framework leases are used to reclaim unused connected remote references. In contrast to the simplicity of the language constructs offered in JavaRMI, the .NET Remoting

236

E. Gonzalez Boix et al.

framework incorporates a leasing pattern at the heart of its design. Leases are automatically extended on every call on the remote object by the time specified in the RenewOnCallTime property. If that property is not set, lease renewal can be achieved by registering a sponsor. Variations on the integrated pattern need to be built on top of sponsor and lease interface abstractions. The lease interface provides methods for overriding some leasing properties (e.g. RenewOnCallTime), renewing the lease, and the registration of sponsors. However, no means are provided for explicit revocation of a lease. Expiration handling is not provided either in the .NET Remoting framework. Although the system does indeed contact sponsors upon lease expiration, there are no guarantees that the system will contact the sponsor of a specific client as it may ask several sponsors until it finds one willing to renew the lease. CORBA. Leasing has been introduced to CORBA as a technique for resource management [2]. A broad definition of resource is adopted: a resource can be practically any CORBA entity. In order to provide reusable leasing functionality for different CORBA-based applications, leasing is modeled as a dedicated CORBA service [2]. A resource is expected to implement two methods used by leases to start and stop the use of the resource. A resource claimant has to obtain a lease from a lessor in order to use such a resource. A lease offers methods to be renewed and revoked. Two types of leases are granted depending on the type of claimants. Observed claimants receive a lease which observes the claimant so that if it terminates, the lease gets cancelled. The object is periodically queried to detect if it is still alive. Due to the volatile connections phenomenon, such leases do not seem appropriate for MANETs: the claimants may only be disconnected temporarily, causing the lease to be cancelled erroneously. Notified resource claimants receive a lease which notifies the claimant as soon as it expires. The lease is then automatically renewed once at server side to give the claimant sufficient time to renew the lease if necessary. Leasing patterns need to be built on top of the above mentioned architecture, e.g. automatic renewal of leases can be accomplished by making explicit renew calls on the lease interface. Expiration handling can be achieved at client side using notified claimants. Jini. Jini is a framework built on top of Java which allows clients and services to discover and set up an ad hoc network. Jini provides support to deal with the fact that clients and services may join and leave the network at any time, unannounced. Leasing was introduced to allow clients and services to leave the network gracefully without affecting the rest of the system. However, Jini relies on the communication model of Java RMI [15]. This means that transient disconnections are not supported, i.e. a disconnection blocks the connection between objects. Although Jini’s architecture is flexible enough to accommodate a leasing model which takes into account intermittent connections, to the best of our knowledge, Jini does not implement this functionality. Jini advocates the use of leasing to mediate the access to any kind of resource: objects, files, certificates that grant the lease holder certain capabilities or even the right to request for some actions to execute while the lease is valid. By default, leases have been

A Leasing Model to Deal with Partial Failures in Mobile Ad Hoc Networks

237

only integrated in the lookup service. Services get a lease when they advertise themselves with the lookup service which must be explicitly renewed; if they cannot, the lookup service will remove the service advertisement such that it does not provide stale information. Jini provides a data structure for the systematic renewal of a set of leases. Leasing patterns can be built using a lease renewal service to implement the protocol to communicate with the remote server. Expiration handling can be achieved at client side by registering an event listener on a lease renewal set. When the lease is about to expire, an expiration warning event is generated notifying all registered listeners for that set. Table 1. Summary of related work

Java RMI .NET Remoting Framework CORBA Jini

Leasing an Intermittent Connection N (RPC)

Leasing Patterns N (need to be built)

N (RPC)

Y (integration of a leasing pattern) N (RPC) Y (using notified and observer claimants) N (reliance on JavaRMI) N (need to be built)

Symmetric Expiration Handling N (no notification upon lease expiration) N (notification not guaranteed) Y (only at client side) Y (only at client side)

Table 1 summarizes our related work. We observe that no single approach adheres to all of the criteria for a leasing model in MANETs. This shortcoming forms the main motivation for our work. In particular, no approach takes into account the intermittent connections criterion since they rely on synchronous communication by RPC, which is not designed for MANETs. However, these approaches provide a foundation on which we base our leasing model.

3

Leased Object References

To address all the criteria discussed in the previous section, we introduce our leasing model for MANETs where remote object references play the role of the lease and the service objects they refer to play the role of the resource, leading to the concept of leased object references. A leased object reference is a remote object reference that transparently grants access to a service object for a limited period of time. When a client first references a service object, a leased reference is created and associated to the service VM A

client object

VM B

server object

Fig. 2. A leased object reference

238

E. Gonzalez Boix et al.

object. From that moment on, the client accesses the service object transparently via the leased reference until it expires. Figure 2 illustrates an allocated leased reference. Each end point of the leased reference has a timer initialized with a time period which keeps track of the lease time left. When the time period has elapsed, the access to the service object is terminated and the leased reference is said to expire. The lifetime of leased references can be explicitly controlled by renewing or revoking them before they expire. Once a leased reference expires, both the client object and service object know that access to the service object is terminated. Leased object references provide dedicated parameter-passing semantics to ensure that all remote interactions to a service object are subject to leasing. Leasing an Intermittent Connection. In order to abstract over the transient disconnections inherent to MANETs, a leased reference decouples the client object and the service object it refers to in time. This means that a client object can send a message to the service object even if the leased reference is disconnected at that time. Client objects can only send messages to service objects asynchronously: when a client object sends a message to the service object, the message is transparently buffered in the leased reference and the client does not wait for the message to be delivered1 . Figure 3 shows a diagram of the different states of a leased reference. When the leased reference is connected and active, i.e. there is network connection and the lease has not yet expired, it forwards the buffered messages to the remote object. While disconnected, messages are accumulated in order to be transmitted when the reference becomes reconnected at a later point in time. When the lease expires, the client loses the means of accessing the service object via the leased reference. Any attempt in using it will not result in a message transmission since an expired leased reference behaves as a permanently disconnected remote reference.

disconnect

Connected

Disconnected

(messages are forwarded)

(messages are buffered) reconnect

expire

expire

Expired (messages are dropped)

Fig. 3. States of a leased object reference

1

We are not the first ones on providing buffering of messages, this behaviour can be also found in the Rover toolkit [7].

A Leasing Model to Deal with Partial Failures in Mobile Ad Hoc Networks

239

Leasing Patterns. Leased object references incorporate two leasing variants on leased object references which transparently adapt their lease period under certain circumstances. The first variant is a renew-on-call leased reference which automatically prolongs the lease upon each method call received by the service object. This pattern has been inspired by the renewOnCall property of the .NET Remoting framework [9]. As long as the client uses the service object, the leased reference is transparently renewed by the interpreter. The second variant is a single call leased reference which automatically revokes the lease upon performing a successful method call on the service object. Such leases are useful for objects adhering to a single call pattern, such as callback objects. As previously explained, callback objects are often used in asynchronous message passing schemes in order for service object to be able to return values. These callback objects are typically remotely accessed only once by service objects with the computed return value. Other variants are definitely possible and can be built on top of the leased object reference abstraction as we illustrate later in the implementation of a concrete instantiation of the leased object references model. Symmetric Expiration Handling. We adopt a Jini-like solution for expiration handling based on event listeners which can be registered with a leased reference at both client and server side. When the reference expires, the registered listeners are notified asynchronously. This allows client and service objects to treat a failure as permanent (i.e. to detect when the reference is permanently broken) and to perform appropriate compensating actions. At server side, this has important benefits for memory management. Once all leased references to a service object have expired, the object becomes subject to garbage collection once it is no longer locally referenced. Because both sides of a leased reference have a timer, no communication with the server is required in order for a client to detect the expiration of a leased reference. However, having a client-side and server-side timer introduces issues of clock synchronisation. Keeping clocks synchronised is a well known problem in distributed systems [12]. This issue is somewhat more manageable with leases since they use time intervals rather than absolute time and the degree of precision is expected to be of the magnitude of seconds, minutes or hours. Once the leased reference is established, the server side of the reference periodically sends the current remaining time by piggybacking it onto application-level messages. At worst, the asynchrony causes a leased reference to be temporarily in two inconsistent states: either the client-side of the reference expires while the server-side is still active, or the client-side of the reference is active while the server-side expired. In the first case, a client will not attempt a lease renewal and thus, the server-side timer will eventually expire as well. In the second case, when a client requests a lease renewal, the server will ignore it and the clientside timer will expire soon thereafter. When the server-side timer is expired, the client perceives the remote object as disconnected due to a network failure.

240

4

E. Gonzalez Boix et al.

Leased Object References in AmbientTalk

We have implemented leased object references in AmbientTalk, an objectoriented programming language designed for distributed programming in mobile ad hoc networks [13]. Before describing the concrete instantiation of the leased object reference model, we first briefly introduce AmbientTalk. We will use the mobile music player example introduced in Section 2.1 to illustrate the language and the language support for leased object references. 4.1

AmbientTalk in a Nutshell

AmbientTalk is a prototype-based object-oriented distributed language. The following code excerpt shows the definition of a simple Song object in AmbientTalk: def Song := object: { def artist := nil; def title := nil; def init(artist, title) { self.artist := artist; self.title := title; }; def play() { /* play the song */ }; }; def s := Song.new("Mika", "Relax");

A prototypical song object is assigned to the variable Song. A song object has two fields; a constructor called init in AmbientTalk, and a method play. Sending new to an object creates a copy of that object, initialised using its init method. Distributed programming. AmbientTalk is a concurrent actor-based language [1]. AmbientTalk’s actors are based on the communicating event loops model of the E language [11] in which each actor owns a set of regular objects. Objects owned by the same actor communicate using sequential message sending (expressed as o.m()), as in Java. Objects owned by different actors can only send asynchronous messages to one another (expressed as o> at:, but cannot be changed with at:put:. Growable: Instances of Interval and Array are always of a fixed size. Other kinds of collections (sorted collections, ordered collections, and linked lists) may grow after creation. Accepts duplicates: A Set filters out duplicates, but a Bag will not. This means that the same elements may occur more than once in a Bag but not in a Set. Dictionary, Set and Bag use the = method provided by the elements; the Identity variants of these classes use the == method, which tests whether the arguments are the same object, and the Pluggable variants use an arbitrary equivalence relation supplied by the creator of the collection. Contains specific elements: Most collections hold any kind of element. A String, CharacterArray or Symbol, however, only holds Characters. An Array will hold any mix of objects, but a ByteArray only holds Bytes. A LinkedList is constrained to hold elements that conform to the Link accessing protocol, and Intervals only contain consecutive integers, and not any numeric value between the bounds.

4

Experimental Process

Experience with traits [4,5] shows that they effectively support reuse. We identified the problems of Section 2 in Squeak, a dialect of Smalltalk. Our ultimate

Reusing and Composing Tests with Traits

259

Table 2. Some of the key behaviors of the main Squeak class collection Implementation kind Arrayed Array, String, Symbol Ordered OrderedCollection, SortedCollection, Text, Heap Hashed Set, IdentitySet, PluggableSet, Bag, IdentityBag, Dictionary, IdentityDictionary, PluggableDictionary Linked LinkedList, SkipList Interval Interval Sequenceable access by index Array, String, Symbol, Interval, OrderedCollection, SortedCollection not indexed LinkedList, SkipList Non-sequenceable access by key Dictionary, IdentityDictionary, PluggableDictionary not keyed Set, IdentitySet, PluggableSet, Bag, IdentityBag

goal is to redesign core libraries of Squeak, favoring backward compatibility, but fully based on traits. Our results are available in Pharo, a fork of Squeak that provides many improvements, including a first redesign of the stream library [5]. The current paper takes place in a larger effort of specifying the behavior of the main collection classes, as a preliminary of designing a new library. Our approach to reuse tests is based on trait definition and composition. In both cases, our experimental process iterated over the following steps: Protocol identification and selection. We selected the main protocols as defined by the main classes. For example, we took the messages defined by the abstract class Collection and grouped them together into coherent sets, influenced by the existing method categories, the ANSI standard, and the work of Cook [1,6]. Trait definitions. For each protocol we defined some traits. As we will show below, each trait defines a set of test methods and a set of accessor methods which make the link to the fixture. Note that one protocol may lead to multiple traits since the behavior associated to a set of messages may be different: for example, adding an element to an OrderedCollection or a Set is typically different and should be tested differently (we defined two traits TAddTest, TAddForUniquenessTest for simple element addition). Now these traits can be reused independently depending on the properties of the class under test. Composing test cases from traits. Using the traits defined in the previous steps, we defined test case classes by composing the traits and specifying their fixture. We did that for the main collection classes, i.e., often the leaves of the Collection inheritance tree (Figure 2). We first checked how test traits would fit together in one main test class, then applied the traits to other classes. For example, we defined the traits TAddTest and TRemoveTest for adding and removing elements. We composed them in the test cases for OrderedCollection.

260

S. Ducasse et al.

Then we created the test cases for the other collections like Bag, etc. However, the tests would not apply to Set and subclasses, which revealed that variants of the tests were necessary in this case (TAddForUniquenessTest and TRemoveForMultiplenessTest).

5

Selected Examples

We now show how we define and compose traits to obtain test case classes. 5.1

Test Traits by Example

We illustrate our approach with the protocol for inserting and retrieving values. One important method of this protocol is at:put:. When used on an array, this Smalltalk method is the equivalent of Java’s a[i] = val: it stores a value at a given index (numerical or not). We selected this example for its simplicity and ubiquity. We reused it to test the insertion protocol on classes from two different sub-hierarchies of the collection library: in the SequenceableCollection subclasses such as Array and OrderedCollection, but also in Dictionary, which is a subclass of Set and Collection. First, we define a trait named TPutTest, with the following test methods: TPutTest >> testAtPut self nonEmpty at: self anIndex put: self aValue. self assert: (self nonEmpty at: self anIndex) == self aValue. TPutTest >> testAtPutOutOfBounds "Asserts that the block does raise an exception." self should: [self empty at: self anIndex put: self aValue] raise: Error TPutTest >> testAtPutTwoValues self nonEmpty at: self anIndex put: self aValue. self nonEmpty at: self anIndex put: self anotherValue. self assert: (self nonEmpty at: self anIndex) == self anotherValue.

Finally we declare that TPutTest requires the methods empty, nonEmpty, anIndex, aValue and anotherValue. Methods required by a trait may be assimilated as the parameters of the traits, i.e., the behavior of a group of methods is parametrized by its associated required methods [8]. When applied to tests, required methods and methods defining default values act as customization hooks for tests: to define a test class, the developer must provide the required methods, and he can also locally redefine other trait methods. 5.2

Composing Test Cases

Once the traits are defined, we define test case classes by composing and reusing traits. In particular, we have to define the fixture, an example of the domain

Reusing and Composing Tests with Traits

261

objects being tested. We do this in the composing class, by implementing the methods that the traits require to access the fixture. Since overlap is possible between the accessors used by different traits, most of the time, only few accessors need to be locally defined after composing an additional trait. The following definition shows how we define the test case ArrayTest to test the class Array: CollectionRootTest subclass: #ArrayTest uses: TEmptySequenceableTest + TIterateSequenceableTest + TIndexAccessingTest + TCloneTest + TIncludesTest + TCopyTest + TSetAritmetic + TCreationWithTest + TPutTest instanceVariableNames: ’example1 literalArray example2 empty collection result’

ArrayTest tests Array. It uses 9 traits, defines 10 instance variables, contains 85 methods, but only 30 of them are defined locally (55 are obtained from traits). Among these 30 methods, 12 methods define fixtures. The superclass of ArrayTest is CollectionRootTest. As explained later the class CollectionRootTest is the root of the test cases for collections sharing a common behavior such as iteration. ArrayTest defines a couple of instance variables that hold the test fixture, and the variables are initialized in the setUp method: ArrayTest >> setUp example1 := #(1 2 3 4 5) copy. example2 := {1. 2. 3/4. 4. 5}. collection := #(1 -2 3 1). result := {SmallInteger . SmallInteger . SmallInteger . SmallInteger}. empty := #().

We then make the fixture accessible to the tests by implementing trivial but necessary methods, e.g., empty and nonEmpty, required by TEmptyTest: ArrayTest >> empty ^ empty

ArrayTest >> nonEmpty ^ example1

TPutTest requires the methods aValue and anIndex, which we implement by returning a specific value as shown in the test method testAtPut given below. Note that here the returned value of aValue is absent from the array stored in the instance variable example1 and returned by nonEmpty. This ensures that the behavior is really tested. ArrayTest >> anIndex ^2

ArrayTest >> aValue ^ 33

TPutTest >> testAtPut self nonEmpty at: self anIndex put: self aValue. self assert: (self nonEmpty at: self anIndex) = self aValue.

These examples illustrate how a fixture may be reused by all the composed traits. In the eventuality where a trait behavior would require a different fixture, new state and new accessors could be added to the test class.

262

S. Ducasse et al.

The class DictionaryTest is another example of a test case class. It also uses a slightly modified version of TPutTest. This trait is adapted by removing its method testAtPutOutOfBounds, since bounds are for indexed collections and do not make sense for dictionaries. The definition of DictionaryTest is the following: CollectionRootTest subclass: #DictionaryTest uses: TIncludesTest + TDictionaryAddingTest + TDictionaryAccessingTest + TDictionaryComparingTest + TDictionaryCopyingTest + TDictionaryEnumeratingTest + TDictionaryImplementationTest + TDictionaryPrintingTest + TDictionaryRemovingTest + TPutTest - {#testAtPutOutOfBounds} instanceVariableNames: ’emptyDict nonEmptyDict’

DictionaryTest uses 10 traits and defines 2 instance variables. 81 methods are defined in DictionaryTest for which 25 are locally defined and 56 are brought by the traits. For this class, a similar process happens. We define the setUp method for this class. Note that here we use a hook method classToBeTested, so that we can also reuse this test case class by subclassing it. DictionaryTest >> setUp emptyDict := self classToBeTested new. nonEmptyDict := self classToBeTested new. nonEmptyDict at: #a put: 20; at: #b put: 30; at: #c put: 40; at: #d put: 30. DictionaryTest >> classToBeTested ^ Dictionary

And similarly we redefine the required methods to propose a key that is adapted to dictionaries: DictionaryTest >> anIndex ^ #zz

5.3

DictionaryTest >> aValue ^ 33

Combining Inheritance and Trait Reuse

It is worth noting that we do not oppose inheritance-based and trait-based reuse. For example, the class CollectionRootTest uses the following traits TIterateTest, TEmptyTest, and TSizeTest (see Figure 3). CollectionRootTest is the root of all tests, therefore methods obtained from these three traits are inherited by all test classes. We could have defined these methods directly in CollectionRootTest, but we kept them in traits for the following reasons: – Traits represent potential reuse. It is a common idiom in Smalltalk to define classes that are not collections but still implement the iterating protocol. Having separate traits at hand will increase reuse.

Reusing and Composing Tests with Traits

empty nonEmpty

TSizeTest testSizeWhenEmpty

empty nonEmpty

263

TEmptyTest testifEmptyifNotEmpty testIfEmptyDo TIterateTest

collection expectedSizeAfterReject speciesClass

CollectionRootTest

OrderedCollectionTest

testCollect testDetect testAllSatisfy

T...

SetTest ArrayTest ...

TSetArithmetic testUnion testIntersection

Fig. 3. Test reuse by both trait composition and inheritance (the representation of traits shows required methods on the left and provided methods on the right)

– A trait is a first class entity. It makes the required methods explicit and documents a coherent interface. – Finally, since our plans are to design a new collection library, we will probably reuse these protocol tests in a different way.

6

Results

We now present the results we obtained by reusing tests with traits. Section 6.3 discusses the questions we faced during this work. 6.1

In the Nile Stream Library

Excluding tests, Nile is composed of 31 classes, structured around 11 traits: TStream, TDecoder, TByteReading, TByteWriting, TCharacterReading, TCharacterWriting, TPositionableStream, TGettableStream, TPuttableStream, TGettablePositionableStream, TPuttablePositionableStream. More details about the design of Nile may be found in the literature [5]. Nile has 18 test classes among which the ones listed on Table 3 use traits. The columns of Table 4 and Table 6 describe: Users: how many test case classes use each trait, either directly by composition or indirectly through both inheritance and composition; Methods: the trait balance in terms of required methods vs. provided tests5 ; Ad-hoc: defined requirements or redefined methods vs. overridden tests; Effective: the number of unit tests executed during a test run that originate from the trait in column 1. For instance, TPuttableStreamTest is composed into 7 test case classes in total, through 4 explicit composition clauses; it requires 3 methods and provides 5 unit 5

The small numbers indicate any additional non-test methods that the traits provide.

264

S. Ducasse et al.

Table 3. Trait compositions in the Nile tests (only these 6 test classes use traits) Test case class

Traits composed

CollectionStreamTest

TGettablePositionableStreamTest + TPuttablePositionableStreamTest FileStreamTest TGettablePositionableStreamTest + TPuttablePositionableStreamTest HistoryTest TGettableStreamTest + TPuttableStreamTest LinkedListStreamTest TGettableStreamTest + TPuttableStreamTest RandomTest TGettableStreamTest SharedQueueTest TGettableStreamTest + TPuttableStreamTest

Table 4. Use, structure, adaptation, and benefit of test traits in Nile Test trait

Users dir. / inh.

TGettablePositionableStreamTest TGettableStreamTest TPositionableStreamTest TPuttablePositionableStreamTest TPuttableStreamTest

2/4 5/8 2/8 2/4 4/7

Methods5 req.  prov.

12 1  10 19 01 3  5 +1

Ad-hoc mth.  tests

40 80 80 00 22  0

Effective unit tests 8 80 72 4 35

tests and an auxiliary method. The test classes provide 22 definitions for its required methods: 3 requirements implemented in 7 test cases, plus 1 redefinition of the auxiliary method. Overrides are to be expected since the fixtures often can not be defined in a completely abstract way; none of the tests had to be adapted, though, which is good. In total, the 5 tests are run for each of the 7 test cases, so TPuttableStream generates 35 unit tests. 6.2

In the Collection Library

The collection hierarchy is far richer than the stream hierarchy. It contains several behaviors, often orthogonal, that are intended to be recomposed. As previously, Tables 5 and 6 describe how we composed and defined the test traits. The coverage of the collection hierarchy is in no way complete, and we expect to define other traits to cover behavior, like identity vs. equality of elements, homogeneous collections, weak-referencing behavior. . . When writing the test traits, we decided to make do with the collection classes as they exist, so the traits are much less balanced than with Nile. For instance, TGrowableTest only applies to collections that reallocate their storage, so we tested it only for OrderedCollection. However this situation is due to a lack of time since several other collections exhibit this behavior. In contrast, TIterateTest and

Reusing and Composing Tests with Traits

265

Table 5. Trait composition in the collection hierarchy tests Test case class

Traits composed

ArrayTest

TEmptySequenceableTest + TIterateSequenceableTest + TIndexAccessingTest + TCloneTest + TIncludesTest + TCopyTest + TSetAritmetic + TCreationWithTest + TPutTest BagTest TAddTest + TIncludesTest + TCloneTest + TCopyTest + TSetAritmetic + TRemoveForMultiplenessTest CollectionRootTest TIterateTest + TEmptyTest + TSizeTest DictionaryTest TIncludesTest + TDictionaryAddingTest + TDictionaryAccessingTest + TDictionaryComparingTest + TDictionaryCopyingTest + TDictionaryEnumeratingTest + TDictionaryImplementationTest + TDictionaryPrintingTest + TDictionaryRemovingTest + TPutTest − {#testAtPutOutOfBounds} IntervalTest TEmptyTest + TCloneTest + TIncludesTest + TIterateSequenceableTest + TIndexAccessingTest + TIterateTest − {#testDo2. #testDoWithout} LinkedListTest TAddTest − {#testTAddWithOccurences. #testTAddTwice} + TEmptyTest OrderedCollectionTest TEmptySequenceableTest + TAddTest + TIndexAccessingTest + TIncludesTest + TCloneTest + TSetAritmetic + TRemoveForMultiplenessTest + TCreationWithTest + TCopyTest + TPutTest SetTest TAddForUniquenessTest + TIncludesTest + TCloneTest + TCopyTest + TSetAritmetic + TRemoveTest + TCreationWithTest − {#testOfSize} + TGrowableTest + TStructuralEqualityTest + TSizeTest StackTest TEmptyTest + TCloneTest − {#testCopyNonEmpty} StringTest TIncludesTest + TCloneTest + TCopyTest + TSetAritmetic SymbolTest TIncludesTest + TCloneTest − {#testCopyCreatesNewObject} + TCopyPreservingIdentityTest + TCopyTest + TSetAritmetic − {#testDifferenceWithNonNullIntersection}

TEmptyTest are composed by nearly all test cases, so they have a much higher reuse. We also had to adapt composed tests more: while the fixtures are defined abstractly, they have to be specialized for each tested class and, sometimes, we excluded or overrode test methods in a composition because they would not work in this particular test case. Table 6 shows that traits can achieve important code reuse. The presented results should also be interpreted with the perspective that the collection hierarchy is large and that we did not cover all the classes. For example, several kind of dictionary (open vs. closed implementation, keeping order) exist and were not covered. Therefore the results are definitively encouraging.

266

S. Ducasse et al.

Table 6. Use, structure, adaptation, and benefit of test traits in the collection hierarchy Test trait TAddForUniquenessTest TAddTest TCloneTest TCopyPreservingIdentityTest TCopyTest TCreationWithTest TDictionaryAccessingTest TDictionaryAddingTest TDictionaryComparingTest TDictionaryCopyingTest TDictionaryEnumeratingTest TDictionaryImplementationTest TDictionaryPrintingTest TDictionaryRemovingTest TEmptySequenceableTest TEmptyTest TGrowableTest TIdentityAddTest TIncludesTest TIndexAccessingTest TIterateSequenceableTest TIterateTest TPutTest TRemoveForMultiplenessTest TRemoveTest TSizeTest TStructuralEqualityTest

6.3

Users dir. / inh. 1/1 3/4 9 / 11 1/1 6/7 3/3 1/2 1/2 1/2 1/2 1/2 1/2 1/2 1/2 2/2 6 / 13 1/1 1/1 8 / 10 3/3 2/2 2/9 3/4 2/3 2/4 2/9 2/1

Methods5 req.  prov. 34 37 23 11 25 17 3  13 34 01 32 39 08 3  2 +1 3  4 +1 3  6 +3 28 53 2  1 +1 56 1  13 33 7  20 +2 43 11 24 22 24

Ad-hoc mth.  tests

30 10  1 18  0 10 12  0 30 30 30 00 30 30 01 30 30 70 22  0 51 30 41  0 32 60 49  6 12  1 21 60 14  0 20

Effective unit tests 4 28 33 1 35 21 26 8 2 4 18 16 4 8 12 104 3 1 60 39 6 180 12 3 16 18 4

What Did We Gain?

In the introduction of this paper we stated some research questions that drove this experiment. It is now time to revisit them. – Are test traits reusable in practice? If each feature was tested by a separate trait, how much test reuse could we obtain? What is a good granularity for test traits that maximizes their reusability and composition? We created 13 test classes that cover the 13 classes from the collection framework (Table 5). These 13 classes use 27 traits. The number of users for each trait ranges from 1 to 13. Globally, the traits require 29 unique selectors, and provide 150 test and 8 auxiliary methods. In the end, the test runner runs 765 unit tests, which means that on average, reuse is 4.7 unit tests run for each test written. If we do not count just tests but all (re)defined methods, the ratio to unit tests run is still 1.8.

Reusing and Composing Tests with Traits

267

Moreover, since the classes we selected often exhibit characteristic behavior, we expect that once we will cover much more cases, the reuse will increase. – How far should a test fixture be adapted to specific requirements? We had to define specific test fixtures. For example, to test a bag or a set we need different fixtures. However, we could first share some common fixtures which were abstracted using trait required methods, second we could share them between several traits testing a given protocol. It was not our intention to reuse test fixtures optimally, though. To optimize the fixture reuse, we could have followed a breadth-first approach by collecting all the constraints that hold on a possible fixture (having twice the same element, being a character...) before writing any tests. However, this approach makes the hypothesis that the world is closed and that a fixture can be really shared between different test classes. We took a pragmatic approach and wanted to evaluate if our approach works in practice. In such a context, we had to define specific fixtures but we could share and abstract some of the behavior using required methods. – How far should a test be parametrized to be reusable? What is the impact on the maintainability of test code? In the test traits, one out of 6 methods is required. Those unique 29 requirements lead to 237 implementations in the test classes to define the concrete fixtures, but often they are trivial accessors. The difficulty was in striking a balance between reusing fixtures constrained to test several aspects, or defining additional independent ones.

7

Discussion

Threats to validity. The collection hierarchy is complex and dense in terms of the represented behavior. As we showed in Section 3.2, collections possess many similar elementary facets: order and objects (OrderedCollection), order and characters (String), no order and duplication (Bag), no order and uniqueness (Set)... therefore we imagine that the potential of reuse is higher than in normal domain classes. Still we believe that lots of systems designed with interfaces in mind would benefit from our approach. For example, user interface or database mapping frameworks also often exhibit such multiple inheritance behavior. Factoring test code or grouping test data? As said in the introduction, without traits, it is still possible to test protocols and interfaces orthogonally to classes, provided the test case are parametrizable, like in JUnit 4.0 (See Section 8.2). With this approach, test cases only test a specific protocol but are applied to each domain class that should respect that protocol. Alternatively, with the traditional approaches like JUnit, generic test code can be factored through inheritance or auxiliary methods. The natural question is then what is the advantage of using traits vs. parametrized test cases. The JUnit scheme implies that generic test code must be grouped with the code that passes the test data, either all in one class, or in one hierarchy per tested protocol. Reusing generic test code is indeed a typical case of

268

S. Ducasse et al.

implementation inheritance: to group the tests for several protocols and one domain class together, one needs either multiple inheritance or delegation and lots of glue code. In contrast, with test traits defining the generic test code, it is possible to compose the tests for several protocols into a class that defines the test data for a single domain, thus nicely separating both generic and domain-specific parts of the tests code. Moreover, since the domain-specific code controls trait composition, it may ignore or redefine the generic test code on a case-by-case basis. Traits with a single use. In the current system, some test traits are not reused; for instance, the ones dealing with specific OrderedCollection behavior (e.g., removing the last element, or inserting an element at a precise position) are only used by OrderedCollectionTest, so we could have defined the methods directly in this class. Whether it makes sense or not to have such specific protocol tests grouped as a trait is a design question —which is not specific to test traits but applies to using traits in general and to interface design. We believe that it still makes sense to define such cohesive test methods as a trait. The current collection library has a large scope but misses some useful abstractions; even if a trait is not reused currently, it is still a potentially reusable cohesive group of methods. For example, there is no collection which simultaneously maintains the order of elements and their uniqueness; if a new UniqueOrderedCollection is implemented, test traits specifying the ordering and uniqueness protocols will make it easy to ensure that UniqueOrderedCollection behaves like the familiar Set and OrderedCollection. The case of implementation or specific methods. When writing test traits we focused on public methods. In Smalltalk, since all methods are public, it is difficult to know if one should be covered or not. Methods may be categorized into three groups: (i) implementation and internal representation methods, (ii) methods in a public interface specific to a given collection, and (iii) unrelated method extensions. We think that tests for the first category are not worth reusing and should be defined locally in the particular test case. As mentioned above, we believe it is worth to create a trait for the second group because it proactively promotes reuse. The last group of methods are convenience methods, so they belong to a different library and should be tested there. For example, OrderedCollection >> inspectorClass should be covered by the tests for the GUI library —where it belongs. Benefits for protocol/interface design. Our experience trying to identify common protocols highlighted a few inconsistencies or asymmetries; for instance OrderedCollection >> ofSize: and Array >> new: both create a nil-initialized collection of the given size, but OrderedCollection >> new: creates an empty collection that can grow up to the given capacity. We imagine that these inconsistencies could appear because the tests were not shared between several classes. Since protocol tests are easily reused specifications, we believe they support the definition of more consistent interfaces or uniform class behavior.

Reusing and Composing Tests with Traits

269

Designing traits for tests vs. for domain classes. When designing traits for domain classes (as opposed to test traits), we often found ourselves defining required methods that could be provided by other traits. The idea there is that the method names act as join points and glue between traits, propagating behavior from one trait to another. However, in the context of this work, it is better to design traits with required methods named to avoid accidental conflicts with methods of other traits. This way, it is easier to provide specific values and behavior in the context of the specific collection class under test. If a value or behavior must be shared, a simple redirection method does the trick. Pluggable fixtures and test traits. To support reuse between test traits, we tried to share fixtures by making them pluggable. However, it was sometimes simpler to define a separate fixture specific to a given protocol. Indeed, each protocol will impose different constraints on the fixture, and devising fixtures that satisfy many constraints at the same time quickly becomes not practical. It is also important to understand how far we want to go with pluggability; for example, we did not use the possibility to execute a method given its name (which can be trivial in Smalltalk using the perform: message). We limited ourselves to use required methods to change collection elements, because we wanted to favor readability and understandability of the tests, even at the cost of potential reuse.

8 8.1

Related Work Inheritance-Based Test Class Reuse

The usual approach to run the same tests with several fixtures is to write the tests in terms of an abstract fixture and to redefine setUp to provide various concrete fixtures. This approach can be extended with Template/Hook Methods to organize and share tests within a class hierarchy; for example, the Pier and Magritte frameworks [15] use this approach. Their test suites total more than 3200 SUnit tests when run, for only about 600 actual test methods defined; it is then interesting to understand the pros and cons of the approach. The principle is the following: the test hierarchy mirrors the class hierarchy (e.g., MAObjectTest tests MAObject). Some tests are abstract or just offer default values and define hook methods to access the domain classes and instances under test (e.g., a method actualClass returns the domain class to test, and instance and others return instances under test). This approach allows one to test different levels of abstraction. In the leaves of the hierarchy, the tests defined in the superclass can be refined and made more precise. While this approach works well, it shows the following limits: – Sometimes, test methods have to be cancelled in subclasses. – When the domain exhibits multiple inheritance behavior, this approach does not support reuse: it exhibits the same limits as single inheritance in face of need for multiple inheritance reuse. Indeed, besides copy/paste or delegation, there is no specific mechanism to reuse tests.

270

S. Ducasse et al.

This approach may be improved to reuse protocol tests. The idea is to define one test case class for each protocol and define an abstract fixture. Then for each of the classes implementing the protocol, a subclass of the test case class is created and a fixture is specifically defined. The drawback of this approach is that the fixture has to be duplicated in each of the specific subclasses for each protocol. 8.2

Parametrized Test Classes in JUnit 4.0

JUnit 4.0 provides a way to parametrize a test class intended to run a set of test cases over a different data. For example, the code excerpt given below shows a test class that verifies the interface IStack for two implementations. The method annotated with @Parameters must return a collection of test data objects, in this case instances of the stack implementations JavaStack and ArrayStack. The Parameterized test runner instantiates the test class and run its tests once for each parameter. @RunWith(Parameterized.class) public class IStackTest { private IStack stack; public IStackTest(IStack stack) { this.stack = stack; } @Parameters public static Collection stacks() { return Arrays.asList(new Object[ ][ ] { { new ArrayStack() }, { new JavaStack() } }); } @Test public void newStackIsEmpty() throws Exception { assertTrue(stack.isEmpty()); } }

When the fixtures are easy to build, the parametrized test runner is thus a very convenient alternative to the usual xUnit approach of redefining setUp in subclasses. However, this is not so clear if we consider complex fixtures composed from several interrelated domain values, like we needed in our tests, and the @Parameters annotation introduces a dependency from the test code to the implementations: to test a new IStack implementor, one must modify IStackTest — or subclass it, which is the non-parametrized approach. In contrast, test traits group the protocol-level test code in an independent entity, while test classes encapsulate details specific to the tested implementation like building the fixture. Especially, test classes control the composition of tests from several traits, and can define additional ad-hoc tests. This enables a workflow where the protocol can be documented by a generic test trait, and implementation tests can be organized in test classes that parallel the domain class hierarchy.

Reusing and Composing Tests with Traits

9

271

Conclusion

Single inheritance hampers code reuse of business and testing code. Currently, programming languages lack constructs that help reuse across different unit tests. We propose test traits, an approach based on traits which reduces code duplication and favors composition of feature testing code. Our approach is particularly adapted to test protocols in class hierarchies with many polymorphic classes, and is applicable to other trait-based languages. We applied test traits to two large and critical Smalltalk libraries; in average we reused each test 4.7 times. This experiment shows definitive advantages of our approach: test reuse across several classes, test composition, simple fixture parametrization. We will pursue our effort to fully cover the collection hierarchy, and we expect to get higher test code reuse.

References 1. ANSI. American National Standard for Information Systems—Programming Languages—Smalltalk, ANSI/INCITS 319-1998 (1998) 2. Beck, K.: Simple Smalltalk testing: With patterns, http://www.xprogramming.com/testfram.htm 3. Black, A.P., Schärli, N.: Traits: Tools and methodology. In: ICSE (2004) 4. Black, A.P., Schärli, N., Ducasse, S.: Applying traits to the Smalltalk collection hierarchy. In: OOPSLA, vol. 38, pp. 47–64 (2003) 5. Cassou, D., Ducasse, S., Wuyts, R.: Traits at work: the design of a new trait-based stream library. Journal of Computer Languages, Systems and Structures 35(1), 2–20 (2009) 6. Cook, W.R.: Interfaces and specifications for the Smalltalk-80 collection classes. In: OOPSLA, vol. 27, pp. 1–15. ACM Press, New York (1992) 7. Dedecker, J., Van Cutsem, T., Mostinckx, S., D’Hondt, T., De Meuter, W.: Ambient-oriented programming in ambientTalk. In: Thomas, D. (ed.) ECOOP 2006. LNCS, vol. 4067, pp. 230–254. Springer, Heidelberg (2006) 8. Ducasse, S., Gîrba, T., Wuyts, R.: Object-oriented legacy system trace-based logic testing. In: European Conference on Software Maintenance and Reengineering (CSMR 2006), pp. 35–44. IEEE Computer Society Press, Los Alamitos (2006) 9. Flatt, M., Finder, R.B., Felleisen, M.: Scheme with classes, mixins and traits. In: AAPLAS (2006) 10. The Fortress language specification, http://research.sun.com/projects/plrg/fortress0866.pdf 11. Godin, R., Mili, H., Mineau, G.W., Missaoui, R., Arfi, A., Chau, T.-T.: Design of class hierarchies based on concept (Galois) lattices. Theory and Application of Object Systems 4(2), 117–134 (1998) 12. Ingalls, D., Kaehler, T., Maloney, J., Wallace, S., Kay, A.: Back to the future: The story of Squeak, a practical Smalltalk written in itself. In: OOPSLA, pp. 318–326. ACM Press, New York (1997) 13. LaLonde, W., Pugh, J.: Subclassing = Subtyping = Is-a. Journal of ObjectOriented Programming 3(5), 57–62 (1991) 14. Meszaros, G.: XUnit Test Patterns – Refactoring Test Code. Addison-Wesley, Reading (2007) 15. Renggli, L.: Magritte — Meta-described web application development. Master’s thesis, University of Bern (2006)

Flow-Centric, Back-in-Time Debugging Adrian Lienhard, Julien Fierz, and Oscar Nierstrasz Software Composition Group, University of Bern, Switzerland http://scg.unibe.ch

Summary. Conventional debugging tools present developers with means to explore the run-time context in which an error has occurred. In many cases this is enough to help the developer discover the faulty source code and correct it. However, rather often errors occur due to code that has executed in the past, leaving certain objects in an inconsistent state. The actual run-time error only occurs when these inconsistent objects are used later in the program. So-called back-intime debuggers help developers step back through earlier states of the program and explore execution contexts not available to conventional debuggers. Nevertheless, even Back-in-Time Debuggers do not help answer the question, “Where did this object come from?” The Object-Flow Virtual Machine, which we have proposed in previous work, tracks the flow of objects to answer precisely such questions, but this VM does not provide dedicated debugging support to explore faulty programs. In this paper we present a novel debugger, called Compass, to navigate between conventional run-time stack-oriented control flow views and object flows. Compass enables a developer to effectively navigate from an object contributing to an error back-in-time through all the code that has touched the object. We present the design and implementation of Compass, and we demonstrate how flow-centric, back-in-time debugging can be used to effectively locate the source of hard-to-find bugs.

1 Introduction When debugging object-oriented systems, the hardest task is to find the actual root cause of a failure as this can be far from where the bug actually manifests itself [1]. In a conventional debugger, the developer is provided with an interface to explore the run-time context at a particular point in time in the execution of the program. Very often defective code leads very quickly to a run-time failure, so in these cases the run-time context is exactly what we need to identify bugs. Unfortunately there are also frequent situations where the failure occurs an arbitrary period of time after the defective code has executed, in which case the run-time context at the time of the failure contains little or no useful information about the source of the error. In a recent study, Liblit et al. examined bug symptoms for various programs and found that in 50% of the cases the execution stack contains essentially no information about the bug’s cause [2]. Back-in-time debuggers have been developed to help developers step back through earlier states of the program and explore execution contexts not available to conventional debuggers. Back-in-time debugging has gained increasing attention recently [3, 4, 5, 6]. Still, even Back-in-Time Debuggers are of only limited help, since they provide no dedicated mechanism to identify which past contexts led to the defective M. Oriol and B. Meyer (Eds.): TOOLS EUROPE 2009, LNBIP 33, pp. 272–288, 2009. © Springer-Verlag Berlin Heidelberg 2009

Flow-Centric, Back-in-Time Debugging

273

state. As a consequence, developers have to manually inspect the execution history to backtrack the origins of an object. Manual inspection of the execution history can be tedious because of the overwhelming amount of data that is typically recorded from program runs. Essentially, we need an effective way to answer the question: “Where did this object come from?”. We have previously proposed an approach to track the flow of objects in a dedicated Object Flow Virtual Machine to answer precisely such questions [7], but the prototype VM does not include any debugging support to exploit object-flows. This paper proposes the novel concept of flow-centric debugging to support an effective navigation of object-oriented execution history during debugging, and presents Compass, a proof-ofconcept flow-centric debugger based on the Object Flow VM. Compass provides several novel views to aid the developer in navigating between the historical control flow and object flow information spaces. One of these views represents the entire history of a running application as an interactive call graph using a “fisheye” view. In this view, the method execution currently in focus is represented as a large circle, and more distant nodes are correspondingly smaller. As one navigates through the call graph, nodes shift and are resized as though they were viewed through a fisheye lens. The contributions of this paper are: (i) a novel approach, called flow-centric debugging, to bridge control flow views offered by conventional debuggers and historical object state views of Back-in-Time Debuggers, and (ii) a scalable and practical implementation strategy with a proof-of-concept prototype called Compass. Outline. In Section 2 we present a simple example to motivate the need for flow-centric debugging. In Section 3 we review existing approaches to back-in-time debugging, and demonstrate how they fall short in tracing back through object flows. Section 4 describes Compass, a proof-of-concept, object-centric, Back-in-Time Debugger. Section 5 describes a realistic example that illustrates how Compass can be used effectively to discover hard-to-track software bugs. In Section 6 we present our implementation, which is based on a metamodel for a debugger that integrates control and object flow into a common approach. Section 7 presents conclusions and remarks on possible future extensions.

2 Motivating Example Let us consider a simple toy example that illustrates the shortcomings of both conventional debuggers and Back-in-Time Debuggers for tracking down a common class of software defects. Figure 1 shows a UML sequence diagram of a scenario leading to a software failure. A Bank is responsible for logging new deposits and adding interest to individual BankAccount instances. It makes a request to an InputReader instance for (1) a new deposit, which is (2) returned to the Bank. The returned value happens to be nil, but this does not lead to an immediate failure. The value is (3) returned to the Bank from a helper method. The value is (4) saved in a BankAccount and (5) stored in a private field. The Bank then triggers the BankAccount to compute interest, so it (6) reads

274

A. Lienhard, J. Fierz, and O. Nierstrasz Bank BankAccount

a BankAccount

an InputReader

an InterestCalculator

example getDeposit

readDeposit nil

1

2 nil 3 4 deposit: (deposit:nil) 5

addInterest deposit

6 nil

7

8 calculateNewDeposit (deposit : nil, interest: 0.05)

Fig. 1. Sequence diagram of the crashing example

the value with an accessor that (7) returns it. The BankAccount then (8) passes the value to the InterestCalculator instance, which blows up when it tries to compute interest for a nil value. Now consider a developer trying to debug this with a conventional debugger. We can clearly see that the failure is due to the nil value, but the debugger only gives us the run-time context back to the point where the value was (6) read from the field where it was stored. We would have to set a breakpoint where the variable is stored (5) and re-run the program. Now we can see that the value was returned (3) from the method getDeposit, but since this method context is not on the stack anymore, we have to set another breakpoint to find out where the value comes from, etc. A Back-in-Time Debugger would allow us to step back in time, but we do not know how far back in time we must travel. We have no information to tell us where to go. Although the given scenario is tiny, a real-world scenario might include many thousands of events between the failure and the point where the faulty value is computed and stored. As we shall see in the following section, present Back-in-Time Debuggers do not offer adequate mechanisms to identify the contexts that the faulty object has passed through.

Flow-Centric, Back-in-Time Debugging

275

3 Limitations of Current Back-in-Time Debuggers Next we look briefly at three examples of Back-in-Time Debuggers and summarize the mechanisms they provide for navigating through historical contexts. In particular we look at how well they support the task of tracing object flow. TOD. The Trace-Oriented Debugger [4] is a Back-in-Time Debugger for Java, implemented as an Eclipse plugin, that improves the performance by using a high-speed distributed database backend. Omniscient Debugger. The Omniscient Debugger [3] is a Back-in-Time Debugger for Java, which instruments bytecode at load time, and offers a user interface to step through the generated traces and inspect the state of objects at any time. Unstuck. The Unstuck debugger [5] is a Back-in-Time Debugger for Squeak Smalltalk1 . These approaches adopt a common perspective of the program execution history. All three debuggers provide views that are stack-oriented and state-centric. The user can browse and step through the method execution trace (run-time control flow) and inspect the past state of objects. Figure 2 shows a debugging session of our running example introduced in Section 2 with the Trace-Oriented Debugger (TOD). The stack is displayed in view (A) and the current values of variables are shown in view (D). With the buttons (B) the user can step forward and backward in the recorded control flow. This perspective is a natural extension from conventional debuggers, which show the current call stack and a snapshot of the current state of objects. Back-in-time debuggers have extended these views with historical run-time data. The call stack is extended to the complete execution trace, and the object inspector is extended to all previous values of an object’s fields. Considering our example, with this perspective we can identify the faulty value of the deposit field in the bank account instance. Furthermore, we can jump to any past method execution. However, none of the three debuggers provides sophisticated support to link the symptom of the bug (null pointer exception) with its root cause (where the null value is introduced). Hence, we do not know how far back we need to jump. If we knew the origin of null, the root cause could be identified in one step. In practice, however, bugs tend to be more complex as faulty variables may infect other variables [8,9]. Therefore, to trace back the infection chain, often several objects have to be backtracked. The features that the TOD, Omniscient Debugger, and Unstuck provide to backtrack object flow are ad hoc: – TOD: a “why?” hyperlink (see Figure 2 (D)) allows the focus to be moved to the statement that assigned the current value – Omniscient Debugger offers similar functionality like TOD – Unstuck: allows one to color all methods in the call trace in which a selected object was used 1

http://squeak.org

276

A. Lienhard, J. Fierz, and O. Nierstrasz

C

B

A

D

Fig. 2. TOD. Control flow (A) can be stepped (B), and stays in sync with events in the source code (C). Current objects are also shown (D).

The features provided by TOD and Omniscient Debugger support only one backtracking step (from a field value to where it was assigned or from a method parameter to where the method was called). Hence, to get to the origin of the value, manual steps are required. This includes figuring out by which method execution an object was returned, which can be hard if loops are involved. Therefore, backtracking non-trivial object flows can quickly become a tedious task. The coloring feature provided by Unstuck (see Figure 3) gives a good overview of how often an object was used but it provides only limited support for tracking object flows. For objects that are often used, a lot of code gets highlighted even if the code is not in the path of the object that the developer is interested in. This limitation becomes especially obvious if one wants to backtrack the flow values like null, true, and false. Since these values occur in a very large number of methods, the highlighting of Unstuck is of little use. Further related work. Omnicore’s CodeGuide2 and RetroVue3 , two commercial products, provide similar stack-oriented functionality like the three above presented approaches. Probably the most interesting earlier related work is ZStep95, a reversible debugger for Lisp. A notable feature of ZStep95 is to directly access from graphical 2 3

http://www.omnicore.com http://www.visicomp.com

Flow-Centric, Back-in-Time Debugging

277

Fig. 3. Unstuck. The method trace (at left) stays in sync with the source code view (at right). Methods of the currently selected object are highlighted (reprinted from reference [5]).

Control flow space

? Object flow space

Fig. 4. A debugger must offer mechanisms to navigate between control and object flow views

objects the code that drew them [10]. Approaches, such as the Data Display Debugger [11] and Reference Patterns [12], explicitly focus on graphical state-centric views. Most other back-in-time debugging related work is focused on improving performance and addressing the memory explosion problem rather than researching ways to exploit the gathered data [13, 14, 15]. Problem summary. Since existing Back-in-Time Debuggers adopt the traditional stack-oriented and state-centric views and introduce only minimal innovations to take history into account, they fail to capture the orthogonal issue of object flow. As Figure 4 illustrates, the problem we face is to link the control flow and object flow spaces. What we are missing is a means to link these spaces and to provide views to navigate between them.

278

A. Lienhard, J. Fierz, and O. Nierstrasz

4 Compass: A Flow-Centric Back-in-Time Debugger Compass4 is a proof-of-concept debugger for object-oriented programs. We chose the name “Compass” because its goal is to be an instrument that provides directions to effectively navigate the large information space. Compass is implemented using the Object Flow VM, which efficiently tracks object-flow at run-time [7]. The Compass user interface is shown in Figure 5. We briefly introduce each of the views provided by Compass, and then we focus on the views that are especially interesting from the perspective of navigating between the object and control flow spaces.

8

1

2

3

4

5

6

9

7

Fig. 5. The Compass debugger

1. The method trace view visualizes the entire run-time control flow as a tree of nodes in a fisheye view. Object flows are shown as green arrows between nodes. 2. Call stack of the currently selected method execution. 3. Lists each reference transfer of the flow of the object selected by the developer, by, e.g., double-clicking on an item in the sub-method statements list (5). 4. Source code of the selected method and buttons to step forward and backward through the method trace. 5. List of reference transfers and message sends of current method execution. 6. Variable inspector panes. 7. Lists of control flow statements the method depends on. 8. Displays changes in relationships between objects in the subtrace of the selected method execution. 9. Lets the developer go back and forth, or jump to bookmarked method executions. We now focus on how these views present control flow and object flow, and how they enable navigation between these two spaces. 4

http://scg.unibe.ch/Research/ObjectFlow/

Flow-Centric, Back-in-Time Debugging

279

Fig. 6. A method trace view synchronized with a stack view. Mousing over a node pops up a tool tip identifying the method execution. Clicking on it shifts the focus to that node.

4.1 Navigating Method Execution Traces Compass provides two synchronized views of method execution traces. The method trace view ((1) in Figure 5) shows the entire method execution trace as a tree, and view (2) offers a more conventional stack-oriented view of a single slice of the trace, focused on a given method execution. The challenge in presenting the entire trace as a tree is that the obvious approach of using a conventional tree list does not scale well. For this reason Compass offers a novel approach to navigating traces based loosely on the notion of a fisheye view [16]. The key idea is to present method executions not as text but as circles, showing the currently selected method execution as a large circle in the center of the view. More distant method executions are shown as progressively smaller circles, progressively closer to the edge of the view. In Figure 6 we see the currently selected method execution simultaneously highlighted in the method trace view and the call stack view. Note that we sacrifice the textual feedback of a tree list view for a high level graphical overview of the method execution tree, but we lose little since the synchronized stack view gives us detailed information of the method execution in focus. We can rapidly navigate to nearby nodes in the tree by clicking on them. Since nodes in neighbouring branches are not represented in the stack view, the fisheye view offers tooltips with the name of each method execution as the mouse hovers over its node.

280

A. Lienhard, J. Fierz, and O. Nierstrasz

The layout of the method trace tree is not strictly a conventional fisheye view because we independently scale the x and y coordinates of each node. We compute the x coordinate as a function ft of the timestamp t of a given node, as shown in the following equation, where tf ocus is the start timestamp of the currently selected method execution, Ct is a constant configured in the tool, and w is the width of the view:  w + ft · w2 (t ≥ tf ocus ) 1 ft = 1 − x = w2 w |tf ocus − t| · Ct + 1 (t < tf ocus ) 2 − ft · 2 The value of ft is always in the interval [0, 1); the more ft tends to 0, the closer the node is positioned horizontally to the focused node. The y coordinate is computed similarly, but using the nesting level (depth of the call stack) of method executions instead of their timestamp. The size s of each circular node is computed as follows: s = (1 − ft ) · sf ocus where the constant sf ocus is the size of the focused method execution. Both constants sf ocus and Ct , where the latter defines the relative distance between nodes, can be adjusted by the user. When the focused node changes, the transition from the old to the new state of the method trace view is animated. The smooth transition is intended to help the user follow how the point of view changes. This proved especially useful when backtracking object flows that jump between a large number of method executions. 4.2 Navigating Object Flows Object flow is visualized as follows: for a given method execution selected in either the method trace view (1) (Figure 5) or the stack view (2), all object flow events are listed in view (5). An object flow is selected by double-clicking on one of these events. The entire object flow is then simultaneously displayed textually as a list in view (3), and graphically as green arrows connecting nodes in the method trace view (1), as shown in Figure 7. An object flow answers the question “Where did this object come from?” by displaying the path of reference transfers from the instantiation event to the selected reference. One may then navigate through the object flow by using either the textual or graphical views. The beginning of an object flow represents the creation event. An arrow from a parent to a child node usually means that an object reference was passed as a method argument, and an arrow from a child to its parent node means that the reference was returned from that method. Arrows between more distant nodes mean that the first node wrote the object to a field and that the second node read it. Arrows between parent and child nodes may also indicate that a field was written and then read, but this is relatively rare.

Flow-Centric, Back-in-Time Debugging

281

Fig. 7. Object flow visualized in method trace. Synced with the stack view at bottom left and the list of reference transfers of the selected object flow at bottom right.

To directly navigate to where the object was instantiated one can select the last item in the textual list. (Like the call stack, which grows from bottom to top, the most recent transfer of the object is at the top of the list and its instantiation is at the bottom of the list.) Often, not only the flow of an object is of interest but also where one of its field values comes from. Compass and the Object Flow VM treat objects and values of primitive type the same. Hence, to track back the origin of a field value (even if it is nil) one simply changes the focus of Compass to this value. For more details about the other views and the design and implementation of Compass, the interested reader is referred to the Masters thesis by Fierz [17].

5 Flow-Centric Debugging in Action In this section we demonstrate how Compass can be used in practice with an example of an actual, hard-to-debug defect in a multi-threaded application. The defect occurred in Pharo Smalltalk5 in the software update feature, which downloads and installs both kernel and library updates. The example has been simplified for presentation purposes, but otherwise reflects the actual defect. Problem. The example application downloads and installs software updates using two concurrent processes. The download process is handled by an UpdateDownloader class. 5

http://pharo-project.org

282

A. Lienhard, J. Fierz, and O. Nierstrasz

It puts updates into a queue shared with an installer process and puts a special marker in the queue to indicate that all updates have been downloaded: self getUpdates withIndexDo: [ :code :i | aSemaphore wait. aQueue nextPut: i. aQueue nextPut: code ]. aQueue nextPut: ''. aQueue nextPut: #finished

The installer process is handled by an UpdateLoader class, which simply gets updates from the shared queue and evaluates them: [ this := docQueue next. nextDoc := docQueue next. nextDoc ∼= #finished ] whileTrue: [ Compiler evaluate: nextDoc. docQueueSema signal ].

Unfortunately the application freezes unexpectedly. If we interrupt the application, a traditional debugger tells us that the update loader is blocked on an empty queue. This suggests that the downloader has not only stopped putting updates into the queue, but it has also failed to put the “finished” marker into the queue. Inspecting the stack trace and reading the source code does not help us to get any further in tracking down the source of the problem. Solution. In Compass, when more than one process is traced, the method trace view simply displays more than one root method execution. Starting Compass, we find ourselves in the context of the block method execution of next in the shared queue. Stepping up the stack one level, we find ourselves in the UpdateLoader method responsible for sending the blocking request (Figure 8). We see here that next has been sent 5 times, after which the method blocks on the shared queue. Therefore, this context is clearly interesting and we bookmark it so that we can easily return here. Now we want to understand where the objects in the shared queue are coming from, so we double-click the statement return '4+4' to backtrack the flow of the string '4+4' (Figure 9).

Fig. 8. Execution of the update loader

Flow-Centric, Back-in-Time Debugging

283

Fig. 9. Object flow of a element in the queue

Fig. 10. Loop on downloaded updates

Fig. 11. Side effects of the executed update code

We follow back the object flow, which brings us from the update loader process to the execution of withIndexDo: in the download process (see Figure 10). We can clearly see that a fifth update has been downloaded, but then the downloader stops abruptly. Something has interrupted this process, but we do not know what. The other active process is the update loader, so we follow our bookmark back to there. We note (Figure 8) that updates read from the queue are evaluated by the compiler, so now we step into the last evaluation and explore what happened during the last update. Quickly we arrived at the cleanup method of the update loader and remark something unusual in the side effects graph (right side of Figure 11), namely that the update loader used to have a reference to the downloader, but now it points to nil. Furthermore, the downloader no longer refers to its process. Stepping further we discover that the cleanup method terminates the downloader and nils it out (Figure 12). Evidently this update has erroneously initiated the cleanup procedure, which is only supposed to be invoked when all updates have been downloaded. Alternative solution. Of course, there are other ways to home in on the defect. For example, the update loader initializes and starts the downloader as follows:

284

A. Lienhard, J. Fierz, and O. Nierstrasz

retrieveUpdatesOntoQueue: aQueue withWaitSema: aSemaphore self downloader ifNotNil: [ self downloader terminate ]. self downloader: (UpdateDownloader new initializeWith: aQueue with: aSemaphore). self downloader start

Fig. 12. Cleanup method that stops the download process

Fig. 13. The forward flow of the downloader object

We can then use another feature of Compass, the forward flow explorer, to see what happens with the downloader afterwards (Figure 13). The forward flow shows us that the downloader is later instructed to terminate itself. When we shift our focus to this point, the run-time stack tells us that this request has originated from one of the downloaded updates.

6 Implementation: A Flow-Centric Debugging Metamodel Existing approaches commonly collect run-time data by introducing sensors into the bytecode to emit events whenever a method is entered or exited, or when a variable is assigned. The result is a trace of events organized into a control flow. The data for the user interface is then obtained by querying this trace. In contrast, we implement our new approach to flow-centric debugging by making the relevant metamodel explicit. In order to effectively exploit the opportunities offered by Back-in-Time Debuggers, we must connect these views at the metamodel level. The metamodel is populated by run-time data gathered by the Object Flow VM, which tracks object-flow by creating an alias whenever an object reference is transferred [7]. Figure 14 presents our proposal for a combined metamodel. The upper portion of the figure shows our metamodel for representing control flow information [18]. The lower portion presents our metamodel for expressing object flow.

Flow-Centric, Back-in-Time Debugging

285

Fig. 14. A metamodel combining control flow and object flow

Control flow. The execution trace is modeled by the entity MethodExecution, which points to the caller and holds a timestamp to capture the order and nesting of method executions in the control flow. Each method execution is associated with the entity Method, its static counterpart. The code of a method is represented as a list of statements (we could also use an abstract syntax tree representation, but a flat list is sufficient for our purpose). This control flow metamodel provides the means to step through the runtime control flow and to map each method execution to the source code statement from which it was called (notice the association between MethodExecution and the Send statement). Hence, one main reason for capturing single statements is to be able to map dynamic events, like method executions, to the source code. Object flow. Object-flow is captured by representing every object reference created at run-time as an instance of the entity Alias. A new alias is created whenever an object is created, passed as a parameter to a method, returned from a method, written to a field, or read from a field. The flow of an object is then given by all its aliases. Through the origin association, each alias points to the alias from which it originates. This allows us to track the flow of an object from a specific point in the program execution back to

286

A. Lienhard, J. Fierz, and O. Nierstrasz

where it was instantiated. Furthermore, aliases capture historical object state, modeled by the predecessor association. The combined control flow and object flow metamodel establishes the links between these two: each alias is linked to the control flow that caused the alias to be created. That is, each alias captures the method execution in which it is created and it captures the static statement that caused its creation. This linking in our metamodel allows us to precisely map object flow to control flow. Hence, the metamodel not only supports stepping in the execution trace, but it also directly supports navigating along the flow of objects. Scalability. Benchmarks of our Object Flow VM have shown significant improvements over existing approaches. By only keeping track of still-relevant past data, memory consumption stays within reasonable bounds and the run-time overhead is reduced. Our solution makes use of the garbage collector to release the objects that are not referenced anymore in memory and that are not relevant anymore in the program’s history. The relevance of a historical data point depends, among other properties, on the flow of objects. For instance, a method context (and its associated call stack) is retained in history as long as objects exist in memory that were passed through this method context. The Object Flow VM tracks object flow and hence the Compass debugger can directly access this information without requiring a static analysis. For a more detailed description of the Object Flow VM and its performance characteristics we refer the interested reader to our previous work [7, 19].

7 Conclusions and Future Work We have proposed flow-centric, back-in-time debugging as a novel way to improve the effectiveness of Back-in-Time Debuggers by tracking the flow of object references back through the method executions responsible for their flow. We have presented Compass, a proof-of-concept prototype that offers several novel views to aid the developer in navigating between historical control flow and object flow views. We have presented a typical scenario that demonstrates how empirically hard-to-find defects can be effectively tracked down with flow-centric debugging. Finally we have formalized our approach by presenting a simple metamodel that integrates conventional control-flow analysis of running applications with object flow analysis. Although Compass was not especially designed for debugging multi-threaded applications, it turns out to be remarkable effective since object flows capture precisely the interesting interactions between processes. The fisheye view displays the method executions of each relevant process as a tree. These trees then interleave. The view could be further improved by better distinguishing the different processes, yet keeping the property of revealing object reference transfers between processes. We have not yet carried out an empirical study to evaluate the effectiveness of flowcentric debugging. We hypothesize that Compass will considerably speed up the process of identifying the root cause of typical bugs in object-oriented programs compared to the identical tool lacking object-flow navigation features.

Flow-Centric, Back-in-Time Debugging

287

Acknowledgments We gratefully acknowledge the financial support of the Swiss National Science Foundation for the project “Bringing Models Closer to Code” (SNF Project No. 200020121594, Oct. 2008 - Sept. 2010). We would also like to thank David R¨othlisberger and Tudor Gˆırba for their help in reviewing drafts of this paper.

References 1. Zeller, A.: Why Programs Fail: A Guide to Systematic Debugging. Morgan Kaufmann, San Francisco (2005) 2. Liblit, B., Naik, M., Zheng, A.X., Aiken, A., Jordan, M.I.: Scalable statistical bug isolation. In: Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation (PLDI 2005), pp. 15–26. ACM, New York (2005) 3. Lewis, B.: Debugging backwards in time. In: Proceedings of the Fifth International Workshop on Automated Debugging (AADEBUG 2003) (October 2003) 4. Pothier, G., Tanter, E., Piquer, J.: Scalable omniscient debugging. In: Proceedings of the 22nd Annual SCM SIGPLAN Conference on Object-Oriented Programming Systems, Languages and Applications (OOPSLA 2007), vol. 42(10), pp. 535–552 (2007) 5. Hofer, C., Denker, M., Ducasse, S.: Design and implementation of a backward-in-time debugger. In: Proceedings of NODE 2006, Gesellschaft f¨ur Informatik (GI), September 2006. Lecture Notes in Informatics, vol. P-88, pp. 17–32 (2006) 6. Maruyama, K., Terada, M.: Debugging with reverse watchpoint. In: Proceedings of the Third International Conference on Quality Software (QSIC 2003), Washington, DC, USA, p. 116. IEEE Computer Society, Los Alamitos (2003) 7. Lienhard, A., Gˆırba, T., Nierstrasz, O.: Practical object-oriented back-in-time debugging. In: Vitek, J. (ed.) ECOOP 2008. LNCS, vol. 5142, pp. 592–615. Springer, Heidelberg (2008) 8. Zeller, A.: Isolating cause-effect chains from computer programs. In: SIGSOFT 2002/FSE10: Proceedings of the 10th ACM SIGSOFT symposium on Foundations of software engineering, pp. 1–10. ACM Press, New York (2002) 9. Cleve, H., Zeller, A.: Locating causes of program failures. In: ICSE 2005: Proceedings of the 27th international conference on Software engineering, pp. 342–351 (2005) 10. Lieberman, H., Fry, C.: ZStep 95: A reversible, animated source code stepper. In: Stasko, J., Domingue, J., Brown, M.H., Price, B.A. (eds.) Software Visualization — Programming as a Multimedia Experience, pp. 277–292. The MIT Press, Cambridge (1998) 11. Zeller, A., L¨utkehaus, D.: DDD — a free graphical front-end for Unix debuggers. SIGPLAN Not. 31(1), 22–27 (1996) 12. De Pauw, W., Sevitsky, G.: Visualizing reference patterns for solving memory leaks in Java. In: Guerraoui, R. (ed.) ECOOP 1999. LNCS, vol. 1628, pp. 116–134. Springer, Heidelberg (1999) 13. Feldman, S.I., Brown, C.B.: Igor: a system for program debugging via reversible execution. In: Proceedings of the 1988 ACM SIGPLAN and SIGOPS workshop on Parallel and distributed debugging (PADD 1988), pp. 112–123. ACM, New York (1988) 14. Boothe, B.: Efficient algorithms for bidirectional debugging. In: Proceedings of the ACM SIGPLAN 2000 conference on Programming language design and implementation (PLDI 2000), pp. 299–310. ACM, New York (2000) 15. Xu, G., Rountev, A., Tang, Y., Qin, F.: Efficient checkpointing of java software using contextsensitive capture and replay. In: Proceedings of the the 6th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering (ESEC-FSE 2007), pp. 85–94. ACM, New York (2007)

288

A. Lienhard, J. Fierz, and O. Nierstrasz

16. Furnas, G.W.: Generalized Fisheye View. In: Proceedings of CHI 1986 (Conference on Human Factors in Computing Systems), pp. 16–23. ACM Press, New York (1986) 17. Fierz, J.: Compass: Flow-centric back-in-time debugging. Master’s thesis, University of Bern (January 2009) 18. Richner, T.: Recovering Behavioral Design Views: a Query-Based Approach. Ph.D thesis, University of Bern (May 2002) 19. Lienhard, A.: Dynamic Object Flow Analysis. Ph.D thesis, University of Bern (December 2008)

A Classification Framework for Pointcut Languages in Runtime Monitoring Karl Klose and Klaus Ostermann University of Aarhus, Denmark {klose,ko}@cs.au.dk Abstract. In runtime monitoring and aspect-oriented programming, the execution of a program is monitored to check whether a property – formulated in a pointcut language – holds at some point in the execution of the program. Pointcut languages differ significantly in their expressiveness and the amount of information they utilize about the state of the program’s execution, and the relation between different pointcut languages is often not clear. We propose a formal framework that provides the common abstractions of these languages and identifies the points of variability, such that pointcut languages can be compared and classified with regard to their expressiveness and cost. Besides its usage as a common frame of reference for pointcut languages, our framework also gives a precise model of the design space of pointcut languages and can hence help to design future pointcut languages in a more principled way.

1

Introduction

There are many different languages that are used to describe conditions on the state or structure of a running program. These conditions are used to react on specific events or states in the software system. Examples are pointcut languages in aspect-oriented programming [9], break-point conditions in debugging systems [15], or specification languages in runtime verification and security enforcement (e.g., [10]). We will use the terminology from aspect-oriented programming and use the term pointcut languages to subsume all of these languages in the subsequent. Depending on the application domain, the program may be halted or additional functionality can be executed (so called advice) if a pointcut matches the current point in execution. In contrast to general purpose programming languages, pointcut languages differ significantly in two important dimensions. First, the expressive power of the languages ranges from simple forms of pattern matching to Turing-complete languages. Second the model of the program execution available in the pointcut language is quite different between systems. Examples for the different flavors of pointcut languages range from simple line numbers used in debuggers via sophisticated domain-specific languages such as the AspectJ [8] pointcut language to the usage of various general computation models, such as regular expressions or context-free grammars over callstacks or execution history [17], or Datalog/Prolog/XQuery queries over some representation of the program execution [1,14] or the static program structure [4,5,7]. M. Oriol and B. Meyer (Eds.): TOOLS EUROPE 2009, LNBIP 33, pp. 289–307, 2009. c Springer-Verlag Berlin Heidelberg 2009 

290

K. Klose and K. Ostermann

The aim of this work is to bring order into the huge and so far quite heterogenous design space of pointcut languages. This heterogeneity is multi-dimensional, and it includes the kinds of information the pointcut language operates on, and the granularity and expressiveness of the language on this information. This makes comparisons between pointcut languages and principled design of new pointcut languages quite difficult, which is an important task, because the differences between the expressiveness of the language and the richness of the execution model have significant impact on the performance of the resulting system. Using a formal model we can identify groups of pointcut languages which have similar runtime and memory performance and can make use of the same implementation and optimization techniques. Finally, a formalization allows to prove whether or not two pointcut languages are equal, and what the differences are, if they are not. At the moment, the lack of a formalization makes analyses very ad-hoc and difficult. The aim of this work is to improve the current, undesirable situation by the following contributions: – We give precise formal definitions of pointcut-related terminology as mathematical structures based on the base language’s semantics. – We present a multi-dimensional decomposition of the pointcut language design space and discuss both the significance of the dimensions and classifications of existing languages with regard to these dimensions. In particular we make a clear distinction between the richness of the underlying data model and the expressiveness of the pointcut language itself. This distinction is often not made or left implicit when comparing pointcut languages, leading to confusion about how to compare expressiveness at all. – We give a methodology on how to compare pointcut languages, based on the formal definitions. – We discuss how our framework can be used a basis for the formal description of static analysis and program transformations for aspect oriented programs. The remainder of this paper is structured as follows: First we introduce our notion of joinpoint abstractions and models (Sec. 2). We give examples for models and present a small case study showing two different pointcut models. Next we define pointcuts and pointcut languages (Sec. 3). Building on these prerequisites, we show how pointcut languages can be compared in Sec. 4. In Sec. 5 we discuss language implementation issues in the terms of our framework. Finally, we discuss related work and conclude (Sec. 6 and 7).

2

Semantics and Joinpoint Models

Our first step towards a formal framework for pointcut languages is to define the data model upon which joinpoints and pointcuts can be defined. In the following we introduce small-step semantics as the base notation for the semantics of the underlying base language and show how properties of execution traces in this semantics can be described.

A Classification Framework for Pointcut Languages

2.1

291

The Semantics of the Base Language

There are several alternative formalism to model the semantics of programming languages, such as denotational semantics, big-step operational semantics, and small-step operational semantics. Of these semantics frameworks, small-step operational semantics is the most suitable framework to talk about pointcuts, since it has a precise and direct notion of a computation step, which allows to talk about traces, execution order, etc. Hence in the following we assume that the base language is defined by a small-step operational semantics. We will model the small-step semantics of our base languages by a binary relation −→ on a set Σ of states: −→⊂ Σ × Σ. A state σ ∈ Σ can often be split into several parts, such as environment, expression, heap etc., but for our purposes it is sufficient to assume that the state contains all relevant information that is necessary to compute the respective next computation step. In some situations it is desirable to change the usual small-step semantics in such a way that information relevant for pointcuts can be extracted more easily. For example, in a Featherweight Java [6] trace, it is not easy to determine the lexical origin of an expression that is about to be reduced, or to see when a method call ends. It is possible to extract this information from a full execution trace, but it is easier to modify the semantics such that these kinds of information are directly available, e.g., by adding labels to expressions in the first example or introducing a “return” expression in the second example. We will see an example for this situation later on. 2.2

Joinpoint Models

Pointcut languages usually do not refer to the state of evaluation directly, because it contains both too much and too little information: While, say, the exact shape of the heap may be irrelevant for a pointcut language, it may need other information which is not included in the current state, such as information about states in the history of the computation. For this reason, pointcut languages are usually based on joinpoint models, which contain the information that is necessary to evaluate pointcuts. This information may be an abstraction of the evaluation state, static program information, and even information unrelated to the evaluation, or a mixture thereof. To this end, we define joinpoints to be abstractions over the current state1 . States that are of no interest to the joinpoint model (such as entering a for loop in AspectJ) are mapped to a special value . Definition 1 (Joinpoints and Joinpoint Abstraction). A joinpoint abstraction (JPA) J = (J, αJ ) over Σ is a set J of joinpoints together with a mapping αJ : Σ → J ∪ {}, from states to the corresponding abstraction. Using a joinpoint abstraction, a concrete trace can be turned into an abstract trace. The actual information that is available to a pointcut, however, is a kind of summary of the abstract trace, which we call joinpoint model : 1

In terms of Masuhara et al., we use the point-in-time model for joinpoints in this work, because it is more flexible than the region-in-time model [11].

292

K. Klose and K. Ostermann

Definition 2 (Joinpoint Model). A joinpoint model M = (M, TM ) over a JPA (J, αJ ) is a set M of model values together with a joinpoint transfer function: TM : M × J → M . The joinpoint transfer function defines how the joinpoint model value for a state σ is calculated using the previous model value and the state abstraction α(σ), so eventually the current model value is a kind of fold over the abstract trace. This means that an initial value has to be provided to calculate the model value corresponding to the first state of an evaluation. 2.3

Example: A simple OO Language

In this section we show how the joinpoint abstraction and the joinpoint models of two simple pointcut languages can be expressed in terms of our framework. The joinpoint models of these two pointcut languages we consider are similar to the joinpoint models found in the AOP languages AspectJ and Alpha [14]. We focus on a minimal OO language in the style of Featherweight Java (FJ) [6], a simple calculus that models a minimal core of Java and is defined in terms of a small-step semantics. FJ features classes that may contain fields and methods, and can inherit declarations from another class. We give the syntax of expressions E in this language by defining its set of redexes (R) and evaluation contexts (E[·]): R ::=v.f | v.m(¯ v ) | return(v) E ::=[·] | new C(¯ v , E, e¯) | E.f | E.m(¯ e) | v.m(¯ v , E, e¯) | E; e | return(E) where v is a value, e is an expression, C is the name of a class, f the name of a field, m the name of a method, and e¯ is a list of expressions.2 The basic terms are field accesses (v.f ), method calls v.m(¯ v ) and method returns return(v). Values are objects of the form new C(¯ v ), where the arguments to the constructor are in the same order as the fields of the class. This definition differs from the original FJ definition in two regards: First, we use an evaluation context [18] to conveniently identify the redex in an expression. Second, we introduce a new return redex, which is used to identify the end of a method call. This is an example for a modification of the operational semantics as discussed at the end of Sec. 2.1. We will identify the set E of all expressions in our language with contexts from E with a redex or a value placed in the hole [·] and write E[r] for the expression that has context E and redex r. Following [6], the states of the semantics have the form Σ = P × E, where P is a representation of the program that contains the class declarations. The reduction rules are identical to that of FJ, except that there is one new reduction rule E[return(v)] −→ E[v], which discards a return wrapper once the wrapped expression has been reduced to a value. 2

We write x ¯ for a list x1 · . . . · xn , [] for the empty list and x · x ¯ for list construction.

A Classification Framework for Pointcut Languages

293

An AspectJ-style Joinpoint Model. The joinpoint model in this section is meant to capture the core of the model of AspectJ: the current joinpoint and the callstack abstraction, as required for the cflow pointcuts. The first step is to define a joinpoint abstraction as a basis for the joinpoint model. The states which are of interest for a callstack-based joinpoint model are method calls, i.e., redexes of the form v.m(¯ v ), where v is a value and v¯ is a list of values, and returning from method calls, i.e., redexes of the form return(v), where v is a value. In AspectJ, it is also possible to react to field reads, hence we could add redexes of the form v.f , too, but we want to illustrate the case how to deal with redexes that are not of interest to pointcuts, hence we assume that in this pointcut language field reads can not be addressed by pointcuts. In AspectJ, the joinpoint model does not include receiver and argument values but only their types3 . To create the joinpoint abstraction we have thus to replace the receiver and argument values in the method call redex by their respective types. Since values in FJ have the form new C(¯ v ) we can give a simple value-type mapping by type(new C(¯ v )) = C. Similar to AspectJ we do not treat the return joinpoint as a “joinpoint event”, i. e., as a entity directly addressable by a construct of the pointcut language. Instead, we only need them to maintain the callstack model. Thus we can safely ignore the return value in our joinpoint model. Based on these considerations, joinpoint abstractions in the callstack based ¯ return}, where C ranges over class model are of the form JAJ = {C.m(C), names. The abstraction mapping αAJ simply removes the unnecessary information from the states in the semantics: ⎧ ¯ ⎪ ⎨type(v).m(C)) αAJ (σ) = return ⎪ ⎩ 

if σ = (p, E[v.m(¯ v )]), Ci = type(vi ) if σ = (p, E[return(v)]) otherwise

The joinpoint model MAJ consists of a list of joinpoint abstractions represent∗ ing the callstack: MAJ = JAJ . The corresponding transfer function maintains the call stack and the current joinpoint:  TAJ (¯j, j) =

¯ j · ¯j if j = C.m(C) k¯ if j = return and ¯j = k · k¯

An Alpha-like Joinpoint Model. In Alpha, not only the types but also the dynamic values of the objects involved in a state are available when formulating pointcuts. Hence the joinpoints in Alpha are more fine-grained4 : JAlpha = {v.m(¯ v ), return(v)} 3 4

The actual values may be needed for the execution of an advice in AspectJ, but this work focuses on pointcuts only. Again we omit field reads to illustrate how reductions can be ignored.

294

K. Klose and K. Ostermann

Furthermore, not only the callstack but the complete execution history is available for pointcuts. Accordingly, the abstraction mapping αAJ is the projection on the current redex: ⎧ ⎪ v) if σ = (p, E[v.m(¯ v )]) ⎨v.m(¯ αAlpha (σ) = return(v) if σ = (p, E[return(v)]) . ⎪ ⎩  otherwise The model values in this model are meant to keep the complete (abstracted) trace of the execution history. We will model this again as a list of joinpoints: ∗ , together with a transfer function which accumulates all releMAlpha = JAlpha vant joinpoint abstractions in the list: TAlpha (¯j, j  ) = j · ¯j. In contrast to the AspectJ-like model, return joinpoint abstractions are stored and not discarded. The reason is that Alpha pointcuts frequently reconstruct former callstacks from the history, hence this information is needed to evaluate such pointcuts. 2.4

Classification of Joinpoint Models

In the previous section, we saw two different joinpoint models for a small base language. In order to illustrate the design space of the combination of joinpoint abstraction and model definition, we will now look at some of typical cases of joinpoint models. The space dimension. The first axis of the design space of joinpoint models is concerned with how much information about the past can be stored in a model value. The simplest such joinpoint model is the constant model, where Tconst (m, j) = m. In this model, the initial model value is present at every point in the program. This kind of model can be used to represent configuration values in a system or to pass the AST of the program or parts of it around. A constant model alone is not very useful, since the model value never changes, but it is useful if combined with other models, see later discussion about model constructions. In contrast, global models are not restricted in the way they collect information about the execution. Given a fixed JPA, the model values of a global model are lists representing a view on the past execution and the transfer function is of the form Tglobal (m, j) = f (j) · m for some function f . The idea behind this definition is that the joinpoint model accumulates information (possibly transformed by f ) from every state in the execution in a list. The most general global model is the full trace model, which has f = idJ . The Alpha joinpoint model falls in this category. A slightly more restricted model is a bounded model, where the size of the model (according to some size metric) is restricted by a function on the current state. The AspectJ joinpoint model falls in this category: The length of the stored list is bounded by the depth of the current callstack. Finally, a local model is a joinpoint model that is restricted in the amount of information that it can keep in one model value. The canonical example is a

A Classification Framework for Pointcut Languages

295

model with Tlocal (m, j) = j, where the model only has access to the most recent state. The joinpoint value can of course be transformed by any function instead of the identity in Tlocal . The AspectJ pointcut language without the cflow-style pointcuts could be evaluated in terms of a local joinpoint model. The same holds for the breakpoint conditions typically found in debuggers. Static and dynamic. The other axis of the design space of joinpoint models is the kind of information that can be accessed. Here we distinguish joinpoint models whose only information about the current state is the lexical position of the current redex from joinpoint models that also refer to other dynamic information. Henceforth, a static model is one whose corresponding joinpoint abstraction function α has the form α(σ) = loc(σ), where loc is a function that retrieves the lexical position of the current redex. Such pointcut languages are particularly easy to implement, since every pointcut can be directly mapped to a set of locations in the code. An example is the pointcut language by Eichberg et al. [4]. In contrast, dynamic models are not restricted in the way they information about the state. Product Models. In most pointcut languages the joinpoints models are combinations of the classes of models described above. For example, even if a very generic trace based model may be available, there may be static source code information that is not part of the trace, like the subtype relation. This means that models will usually be cartesian products of different kinds of models whose components can be classified using the categories above. The product model of two joinpoint models M = (M, TM ) and M = (M  , TM  ) is MP = M × M , where the product of the transfer functions is defined componentwise as TMP ((m, m ), j) = (TM (m, j), TM  (m , j)). Of course, the product construction can easily be generalized by replacing the pair constructor with an arbitrary function. Such a construction can be used to model dependencies between the joinpoint models in such a way that one model value influences the other one. The categorization of joinpoint models (or their components) is not only useful when comparing different joinpoint models and pointcut languages but also in design and implementation of aspect oriented systems: the kind of mapping indicates which information must be stored or calculated in the runtime system when augmenting or implementing a system for a pointcut language.

3

Pointcuts

So far we have defined what we mean by joinpoint models and how these models are connected to the underlying operational semantics by abstraction of states into joinpoints. Based on these definition we will now describe what pointcuts are in our framework and how pointcut languages can be modeled. 3.1

Pointcuts and Matching

We have already stated that pointcuts are a means to identify points in the execution of a program that share a certain property. In our framework, the

296

K. Klose and K. Ostermann

states of the program are not directly available but through the abstraction of joinpoint models. Thus pointcuts are modeled by stating on which joinpoint model values they match. Definition 3 (Pointcut). Let M = (M, TM ) be a joinpoint model. A pointcut P is a set of model values, i. e., P ⊆ M. 5 Pointcut P is said to match a model value m, if m ∈ P . The definition of pointcut matching can be extended to execution traces, i. e., lists of the form t = σ1 , . . . , σn of states with σi −→ σi+1 . To this end we have to define how model values are transformed over traces of states. We begin by ∗ : M × (J ∪ {})∗ → M × (J ∪ {}) lifting the transfer function to a function TM that maps the transfer function over abstract traces, that is, over lists of joinpoint ∗ (m, []) = (m, ) and abstractions by TM ⎧ ⎪ ⎨(TM (m, j), j) ∗ ∗ ¯ TM (m, j · j) = TM (m, ¯j) ⎪ ⎩ ∗ TM (TM (m, j), ¯j)

if j =  and length(¯j) = 0 if j =  if j =  and length(¯j) > 0

This is a glorified fold of the transfer function over the abstract trace, which is a little more complex than a normal fold due to the fact that abstract traces may contain  values, which are no valid arguments for the model transfer function TM . However, we cannot simply ignore them; if a trace finishes on , then the model value assigned to this trace is the result of an earlier state in the execution, and a pointcut should match only at that earlier state and not on each subsequent state, that is mapped to . Thus we keep track of the last joinpoint that has been used to calculate the model value and use  to indicate, that no pointcut should match this trace. ∗ operates on lists of joinpoint abstractions, we need to lift traces, which As TM consist of states, to abstract traces, which consist of the corresponding joinpoint σ ) = map(αJ , σ ¯ ). Based on this we abstractions. We define this operation α∗J (¯ define what it means that a pointcut matches a trace. Definition 4 (Pointcut Matching on Traces). Let M be a joinpoint model with joinpoint abstraction J. A pointcut P matches a trace t with initial model ∗ value m0 , if TM (m0 , α∗J (t)) = (m, j) with j ∈ J and m ∈ P . 3.2

Pointcut Languages

The means by which a pointcut is described is defined by a pointcut language. A pointcut languages consists of syntax – a set of valid expressions – and a semantics, mapping syntactical constructs to their meaning in terms of pointcuts. 5

Pointcuts can be identified with predicate functions by looking at the characteristic function χP .

A Classification Framework for Pointcut Languages

297

Definition 5 (Pointcut Language). Let M = (M, TM ) be a joinpoint model. A pointcut language L over M is a set L of language expressions called pointcut designators together with a semantics function [[·]]L : L → P(M ), which maps pointcut designators to their extension, i.e., the set of model values on which the corresponding pointcut matches. We define the extension of a pointcut language as [[L]] = {[[π]] | π ∈ L}. In any non-trivial pointcut language, there will be multiple pointcut designator with the same extension – in most cases even infinitely many. For example, in AspectJ one can find arbitrary many pointcut designators matching all possible method calls: call(*.*(..)), call(*.*(..)) && call(*.*(..)), and so forth. It is further noteworthy that some of the extension sets will be infinitely large, because we do not fix a program in the definition of pointcut languages. For example, if an identifier – like a method name – is part of the model values and a pointcut designator does not impose any restrictions on this identifier, the extension of this pointcut designator will comprise model values for each identifier expressible in the language at hand. It is, however, an interesting property of a pointcut language, which extensions are finite for any or some fixed program. For example, in the case of identifiers, extensions will be finite (at least with respect to the identifier) because the set of identifiers is finite for any given program. On the other hand, in pointcut languages over joinpoint models containing the whole execution history, many pointcut designators will have infinite extensions, because there are infinitely many possible traces satisfying the constraints given in the pointcut designator. 3.3

Example Pointcut Languages

We have already given the definition of the pointcut model of Alpha and AspectJ like languages defined over a small OO base language. We will discuss how to compare pointcut languages next and for that purpose we want to discuss briefly the pointcut languages of these two approaches. Due to place constraints we will not give a complete definition of the syntax and semantics in terms of our framework. AspectJ pointcuts are basically built from a set of primitive pointcut expressions using boolean the operators && (and), || (or) and the higher-order pointcut cflow, which is true if its argument denotes a joinpoint on the callstack. For our purpose, only the primitive pointcut call(R C.m(a)) is important. In this pointcut R is the return type, C the class name, m the method name and a argument. For each of these parameters, wildcards can be used. Alpha pointcuts are basically Prolog queries over a representation of the execution trace. This representation contains facts for each joinpoint that occurred in the execution, paired with a time stamp. These time stamps can be used to relate the joinpoints and to define abstractions like cflow. Since the programmer can also provide his/her own predicates, Alpha pointcuts can compute arbitrary functions over the trace representation.

298

K. Klose and K. Ostermann

Both of these approaches make the quantification over the joinpoint model implicit. In order to come up with a semantics in the form of Def. 5, we would have to make this relation explicit by presenting the semantics as predicate functions over the joinpoint model and then use these predicate functions as characteristic functions for the pointcut extension.

4

Comparing Pointcut Languages

In this section we will develop a methodology to systematically compare pointcut languages. Since we introduced three dependent layers – joinpoint abstractions, joinpoint models and pointcut languages – we have to define comparison on the lower layers first, to make constructs on the higher layers comparable. Thus we start by describing, what it means that one of two joinpoint abstractions is more abstract. Then we talk about the relation between joinpoint models over the same joinpoint abstraction. Finally we show how pointcut languages can be compared and how constructive proofs can be constructed for one language being more/less expressive than an other. 4.1

Comparison of Joinpoint Abstractions

We begin by describing how joinpoint abstractions can be compared. We assume the same base language semantics for both abstractions, because we want to talk about the way that the abstractions act on the same states. We define a joinpoint abstraction to be more abstract than another (more detailed) joinpoint abstraction, if it identifies joinpoints of the more detailed model. Definition 6 (Abstractness of Join Point Abstraction). Let J1 = (J1 , αJ1 ) and J2 = (J2 , αJ2 ) be joinpoint abstractions over Σ. Then J2 is more abstract than J1 if there exists a surjective mapping δ : J1 → J2 , such that δ ◦ αJ1 = αJ2 . Consider the joinpoint abstractions JAJ and JAlpha from Sec. 2.3. According to Def. 6, the joinpoint abstraction JAJ is obviously more abstract than JAlpha , because its elements contain only type names, while the elements of JAlpha contain values. We show this formally by constructing δ : JAlpha → JAJ as follows: ⎧ ¯ ⎪ v ) and Ci = type(vi ) ⎨type(v).m(type(C)) if j = v.m(¯ δ(j) = return if j = return (v) ⎪ ⎩  otherwise and showing that δ satisfies the condition given in Def. 6: v )]) = δ(v.m(¯ v )) = type(v).m(type(¯ v )) δ ◦ αAlpha (E[v.m(¯ = αAJ (E[v.m(¯ v )]) δ ◦ αAlpha (E[return(v)]) = δ(return(v)) = return = αAJ (E[return(v)]).

A Classification Framework for Pointcut Languages

299

All other expressions are mapped to  by both joinpoint abstraction mappings, so these are the only two cases to consider. Thus we have shown that δ ◦ αAlpha = αAJ , and therefore that the call stack based model is more abstract than the trace model. 4.2

Comparison of Joinpoint Models

When comparing joinpoint models, one important problem is that the underlying joinpoint abstractions may be different. In order to find a common joinpoint abstraction to compare both models, we can use the joinpoint abstraction translation δ to compare both models on the more abstract joinpoints. Given this common joinpoint abstraction, the key idea is that values of the more expressive (richer) joinpoint model contain enough data to reconstruct values of the less expressive model. This means that there exists a mapping between the model values which is “compatible” with the model transfer function. Definition 7 (Sub-Model). A joinpoint model M1 = (M1 , T1 ) is a sub-model of M2 = (M2 , T2 ), if both have the same joinpoint abstraction J and there exists a surjective function β : M2 → M1 , such that ∀j ∈ J, m ∈ M2 . β(T2 (j, m)) = T1 (j, β(m)). The mapping β identifies all elements of M2 that can not be distinguished in M1 by mapping them to the same element. The condition on β requires the mapping to be consistent with the model transfer function. If β is a bijection then both models are equal (up to renaming), otherwise M1 is a proper sub-model of M2 . For example, consider again the call stack and trace based models from Sec. 2.3. We define β : MAlpha → MAJ as follows:  δ(j) · β(¯j) if j = v.m(¯ v) ¯ β([]) = [], β(j · j) = ¯ k if j = return(v) and β(¯j) = k · k¯ It is easy to prove that the result of β is indeed the corresponding call stack. That β is correct with respect to the definition can be seen from the fact, that δ is correct and by simulating the semantics for one step. 4.3

Criteria for the Comparison of Pointcut Languages

In the following we compare pointcut languages, and similar to the joinpoint model comparisons, we assume that both languages are defined over the same joinpoint model, since we can translate the richer joinpoint model into the less expressive and compare the translated languages over the less expressive model. Since pointcut languages consist of both a syntax and a semantics we can compare pointcut languages with respect to both. First, we ignore syntax and concentrate on the extensions of the languages, that is, the sets of model values that are extensions of pointcuts in the languages. Later we describe how languages

300

K. Klose and K. Ostermann

can be compared with respect to syntax and finally we give a third formulation based on the the ability of a language to distinguish program executions. Using the definition of the extension of a pointcut language (Def. 5), we can formalize our first comparison criterion. The idea is that a language L1 is a sub-language of L2 , if the extension of L2 contains every pointcut that is in the extension of L1 : Definition 8 (Expressiveness of Pointcut Languages). Let L1 and L2 be two pointcut languages over the same model M . Then, L1 is less expressive, if [[L1 ]] ⊆ [[L2 ]] and L1 is proper less expressive than L2 , if [[L1 ]] ⊂ [[L2 ]]. This definition means that if there is a pointcut designator π1 ∈ L1 (i. e. [[π1 ]] ∈ [[L1 ]]) then there exists a pointcut designator π2 ∈ L2 such that [[π1 ]] = [[π2 ]]. Consider, for example, the Alpha pointcut calls(R, ’m’, V), typeof(R, ’C’), odd(V) 6 , which matches all traces ending on a call of a method m on instances of class C, where the argument is an odd integer. To compare the extension of this pointcut to AspectJ pointcuts, we need to embed the AspectJ pointcut language into the Alpha pointcut model. To this end we identify a pointcut P ⊆ MAJ with the Alpha pointcut β −1 (P ) = {m ∈ MAlpha | β(m) ∈ P }. Since we cannot reason about values in AspectJ, β −1 (P ) contains traces ending in calls with every possible argument value. Hence, there is no AspectJ pointcut that corresponds to our Alpha pointcut and thus AspectJ is not equally expressive as Alpha. However, since it is possible to reason about types of values in Alpha, every AspectJ pointcut can be expressed in Alpha. This means that AspectJ is a proper less expressive language than Alpha. The expressiveness of a pointcut language can also be expressed on the level of syntax. On this level, a less expressive or sub-language is characterized by a mapping between the pointcut designators of the two languages. This mapping maps each pointcut designator of the less expressive language to one of the more expressive language: Definition 9 (Sub-Language). Let L1 and L2 be two pointcut languages over the same model M . L1 is a sub-language of L2 , if there exists a mapping of pointcut designators ι : L1 → L2 with: ∀π ∈ L1 . [[π]] = [[ι(π)]]. Of course, the extension of the pointcuts must be invariant under the syntactic mapping, which is what the condition imposed on ι ensures. In a sense, this comparison criterion is more useful, because it requires to construct a concrete syntactical mapping. In practice, this mapping can be used for the emulation of the sub-language in an implementation of the super-language. We have already mentioned that there is an Alpha pointcut for every possible AspectJ pointcut, which means that we can construct a mapping ι : LAspectJ → LAlpha by translating the primitive and cflow AspectJ pointcuts into the corresponding Prolog terms and translate the boolean operators into logical connectives in Prolog. 6

We assume that some representation of integers is available and write them as numbers to keep the examples short.

A Classification Framework for Pointcut Languages

301

On the other hand it is often desirable to have a constructive proof that a language is not a sub-language of another language. With the above criteria, this can only be established by proving the non-existence of such a mapping. The definition of pointcut matching over traces (Def. 4) allows for another characterization of pointcut languages. A pointcut can be identified with the set of traces on which it matches for a given initial model value. Thus we can classify pointcut languages with the traces that are separable by any pointcut designator belonging to the language. First note that this set of traces is restricted by the joinpoint abstraction: each abstract trace can be the abstraction of several traces. The set of traces that correspond to an abstract trace tα is: (α∗J )−1 (tα ) = {t | α∗J (t) = tα }. Thus pointcuts can only be compared by the abstract traces that they are able to distinguish. We will call two traces t1 and t2 separable by a pointcut π, if there is a model value ∗ ∗ m on which TM (α∗J (t1 ), m) matches π iff TM (α∗J (t2 ), m) does not match π. This notion of separability can be used to define an equivalence relation on traces that identifies all traces which can be separated by a given pointcut language: Definition 10 (Separability Equivalence). Let L be a pointcut languages over a model M = (M, TM ). The separability equivalence relation ≡L for the language L is defined as follows t1 ≡L t2 :⇔ ∗ ∗ ∀π ∈ L, m ∈ M. TM (m, α∗J (t1 )) ∈ ([[π]] × J) ⇔ TM (m, α∗J (t2 )) ∈ ([[π]] × J).

Thus we can give an alternative definition for pointcut language expressiveness based on the pointcut languages’ ability to separate sets of model values: Definition 11 (Precision of Pointcut Language). plusLet L1 and L2 be two pointcut languages over the same model M . L1 is less precise than L2 , if ≡L2 ⊆≡L1 . L1 is proper less precise than L2 , if ≡L2 is a proper subset of ≡L1 . From the previous examples we see that Alpha is much more precise than AspectJ, because it is possible to refer to runtime values in the pointcut. This is, however, not due to the lack of values in our modeling of AspectJ: Even if we had values instead of types, AspectJ cannot distinguish traces that differ in arbitrary computable properties, like the sum of all arguments, the structure of the heap (reachability) and so forth. 4.4

A Methodology for Language Comparison

Using the criteria above we propose the following methodology for comparing pointcut languages: First, we have to ensure, that the two pointcut languages we want to compare are defined over the same joinpoint model and thus over the same joinpoint abstraction. The latter can be achieved by using δ to translate detailed joinpoints into abstracted ones. Then we construct the β mapping from the more expressive model to the less expressive and use it perform the comparison either on the less expressive model or on the more expressive model using β −1 as in the examples.

302

K. Klose and K. Ostermann

To show that one language is a sub-language of/less-expressive than another language, we construct a mapping from pointcut designators of the sub-language to pointcut designators of the richer language. To prove that a language L1 is not a sub-language of another language L2 , two traces have to be provided that the language L1 can separate, but L2 can not. In our example comparison we have seen that there are two different reasons, why AspectJ is less expressive than Alpha: The lack of values in the joinpoint model and the simpler computational expressiveness. Care must be taken not to confuse these different sources of expressiveness by comparing the languages with respect to both the more expressive and the less expressive model.

5

Discussion

Finally we will give an outlook on how our formalization of pointcuts and models is connected to typical issues found in the implementation of programming languages and runtime monitoring systems. 5.1

Shadows and Optimization

We have identified pointcuts with sets of joinpoint model values. By the joinpoint abstraction, each of these model values can be identified with one or more traces in the program execution. To be precise, we can identify each model value m ∈ M with the set Tm of pairs of traces and initial values that produce m given by: ∗ {(t, m0 ) | (∃j ∈ J) TM (m0 , α∗J (t)) = (m, j)}. For a pointcut π, the set Tm =  Sπ = m∈[[π]] {t | (∃m0 ) (t, m0 ) ∈ Tm } is the set of traces that may lead to a model state that the pointcut matches. This set is of particular interest, if there is a function loc mapping states to elements of the program representation. For example, when using labeled expressions, the labels can be used to identify the source code (or, program) element that is currently evaluated. These source code locations are typically called shadows of the pointcut and the set of residues Resπ (l) = {(σ0 , . . . , σn ) ∈ Sπ | loc(σn ) = l}, can be thought of as defining the dynamic test to be performed at the labeled element to decide if the pointcut matches. Shadows and the corresponding dynamic checks are important concepts in the implementation and optimization of pointcut languages. However, without further information about the structure of states (and, therefore, of traces), the elements of Resπ (l) are not a very good descriptions of the dynamic checks, because they simply enumerate all matching traces. But if it is possible to exploit the semantics of the base language and knowledge about the joinpoint model to compute a set Tl of approximated traces for the location l, we can eliminate all traces from Resπ (l) which are no approximations, possibly yielding a much smaller set of traces. In the extreme case the remaining set could be empty. In this case we know that the pointcut can never match an execution trace reaching

A Classification Framework for Pointcut Languages

303

this location and we do not have to check for matching at runtime. But even in the case of a non-empty result, an optimization is possible if we are able to find a “more efficient” pointcut expression π  with Resπ (l) ∩ Tl = Resπ (l) ∩ Tl . These optimizations are too dependent on the exact details of the base language, joinpoint model and pointcut language to be described in this work, but a specialization of our framework for a particular language is likely to yield concrete and useful optimization strategies. 5.2

Observational Equivalence

Two expressions are observationally equal, if they cannot be distinguished by any context in which they are inserted. Observational equivalence is obviously important both in terms of program understanding and optimization. If we consider a runtime monitoring system over a fixed language, the observational equivalence relation will in general become smaller, because a pointcut observing the program execution can suddenly distinguish expressions that would otherwise be observationally equivalent. For example, a call to a method that returns a constant cannot be replaced with this constant, because a trace model could distinguish these terms. Let us assume that the expressions M and N are observationally equivalent in the unobserved program. We can now look at the modified observational equality relation on different levels. If we allow arbitrary models, we have to consider all traces that the evaluation of the terms may produce. If the traces in all contexts are equal for both expressions, then the expressions are also observationally equal with respect to the runtime monitoring system. If we fix the model, we have more observationally equal expressions, but we still have to check all possible traces in all contexts. Finally, with a fixed pointcut language, we have to check if the terms have separable traces in at least one context, otherwise they are observationally equivalent. This means that we can find better approximations for observational equivalence, if we have a compact description of separability for the given pointcut language. A typical application of identifying observationally equivalent expressions is in program optimization. Like with shadows, in this generality we cannot give concrete optimizations. However, for a fixed base and pointcut language, the separability relation may have a simple approximation that allows to derive simple static analyses to approximate observational equivalence. 5.3

Advice and Context Binding

Using pointcuts to trigger additional functionality is only one possible application of pointcuts, but it is useful in many contexts, like debugging and tracing, enforcements of trace and security properties, and aspect-oriented modularization. In this section we show how a “aspect-aware” semantics can be defined on top of a base semantics and a pointcut language over this semantics. To make this elaboration a bit less abstract, we use the convention from our former examples and and define the states of the base semantics as pairs (p, E[e]) of the program representation and a context E with a redex e.

304

K. Klose and K. Ostermann

We give a semantics which augments the original semantics by adding the required context, a syntactical element for proceeding with the evaluation of the original state of execution, and derivation rules for pointcut matching and advice evaluation to the semantics of the underlying language. These extensions describe when and how pointcuts are evaluated and how they trigger the evaluation of advice functionality. The additional functionality is specified in aspects, which bind a piece of code to the pointcut which describes when to execute that code. There are different ways to introduce the additional functionality: before, after or instead (often called around ) the original expression. In our model we use the around strategy, because both other strategies can be expressed in terms of around. It is not sufficient, however, to simply use a pair of a pointcut and an expression to model aspects, because aspects may bind information from the model value and they may choose to proceed with a modified expression instead of the original one. Taking this possibilities into account we model aspects as the set A = L × (M → E) × (M → E → E) .

    bind

target

The bind function describes how the context is used to define the additional ¯ → print(m); proceed functionality. For example, the bind function C.m(C) prints the method name of the intercepted method before proceeding with the call. The target is used to modify how to proceed with the evaluation. It takes the model value at the point proceed is about to be evaluated and the original expression and produces the expression that has to be evaluated instead of proceed. To model this augmented semantics, we fix a pointcut language P with execution model M and joinpoint abstraction J. The syntax of the base language (E) is extended by the expression proceed, which can be used in advice to resume the evaluation of the expression that lead to the advice activation. Since the definition of the base language’s syntax will in most cases be a recursive constructor, the definition of the extended syntax depends on the base syntax and can not be defined in a general way. We assume that such a syntax E  can be constructed from E and that this set contains all the base expressions and additionally – at least – the expression proceed. We define the lifted semantics as follows: the new states are of the form Σ  = M × [A] × [e] × Σ. ¯ An element (m, a ¯, e¯, σ) ∈ Σ  consists of the current model value m, a list a of aspects, a list e¯ of expressions whose evaluation has been “shadowed” by invocation of an aspect, and the original state σ. The evaluation relation −→ ⊂ Σ  × Σ  is defined by the rules in Figure 1. The (Advice App) rule explains how matching advice is applied. Rule (Cong) allows to use other aspects than the head of the list. This rules introduces a nondetermism for advice application. Note that we did not model multiple aspects to match at the same state. The third rule describes the semantics of the new expression proceed, and the final rule embeds the base semantics.

A Classification Framework for Pointcut Languages a = (π, b, t)

m = TM (αJ (σ), m)

m ∈ [[π]]



(m, a · a ¯, e¯, (p, E[e])) −→ (m, a · a ¯, e · e¯, (p, E[b(m )])) (m, a ¯, e¯, σ) −→ σ 

e = proceed

(m, a · a ¯, e¯, σ) −→ σ  ∀(π, b, t) ∈ a ¯. TM (αJ (σ), m) ∈ / [[π]] (m, a ¯, e · e¯, (p, E[proceed])) −→ (m, a ¯, e¯, (p, E[t(m, e)])) σ −→ σ 

σ = (p, E[e])

e = proceed

305

(Advice App)

(Cong)

(Proceed)

∀(π, b, t) ∈ a ¯. TM (αJ (σ), m) ∈ / [[π]]

(m, a ¯, c¯, σ) −→ (TM (σ, m), a ¯, c¯, σ  ) (Base) Fig. 1. Evaluation Rules for the Lifted Semantics

6

Related Work

Many formal definitions of the semantics of aspect oriented languages have been proposed to clarify the semantics concepts like advice application, aspect precedence, and proceed. For example, Masuhara et al. [13] present a model that explains compilation and optimization of an AspectJ-like language. The model is based on an interpreter which is partially evaluated to explain issues like shadow finding and removal of runtime checks which are unnecessary. The resulting model is not as general as our formalization, because it is tailored to AspectJ. This model (and similar ones for other pointcut languages) can be used to describe the semantic of AspectJ-pointcuts in our formal model, because it defines the characteristic functions for pointcut extension. This and similar approaches use fixed base and pointcut languages; a general exploration and organization of the pointcut language design space is not in the scope of these works. Another branch of related work is the development of meta-models and ontologies for pointcut-advice (PA) languages [2,3]. These models are used to identify the parts in PA-languages in a very general way that can be used as a basis for implementations of aspect-oriented systems. Although these models contain informal representations of pointcuts, pointcut context (similar to our joinpoint models), as well as a distinction between static and dynamic context, these entities do not have a formally specified meaning and are hence not amenable to constructive comparisons between pointcut languages. St¨ orzer and Hanenberg [16] give a categorizations of pointcut language constructs with respect to a fixed set of three different model classes: specification based pointcut languages, which are similar to our constant models, state-based constructs, which are similar to our local models, and so-called progress-based pointcut constructs, which are a subset of our global (trace-based) models. In contrast to our work, there is no notion of joinpoint abstraction or pointcut language expressiveness, and no methodology to compare pointcut languages.

306

K. Klose and K. Ostermann

Masuhara and Kiczales [12] present a comparison framework in which they evaluate four different AOP languages. The focus of that work is to characterizing and comparing the weaving process that these languages use. This is in contrast to our framework, which focuses only on the language part and ignores advice and weaving, thus making the framework applicable in “non-AOP” systems.

7

Conclusions

We have presented a formal framework for the classification and comparison of pointcut languages. This model gives precise meaning to joinpoints, pointcuts and pointcut languages. The most important usage of this framework is a methodology for the comparison of pointcut languages, which is based on giving constructive proofs for sub-language relations. These proofs can be used to construct embeddings of pointcut languages into more expressive ones, or to characterize the reason why one language is less expressive than another. Furthermore, our framework illustrates the design dimensions and corner cases of pointcut language design and implementation, and hence we hope that it can be used as guidance in both the design of new pointcut languages and the development of efficient compilation techniques for pointcut languages. We consider our framework both as a starting point to improve the modeling, design and implementation of pointcut languages and as way to combine the existing approaches to give semantics to particular pointcut languages.

References 1. Allan, C., Avgustinov, P., Christensen, A.S., Hendren, L., Kuzins, S., Lhot´ ak, O., de Moor, O., Sereni, D., Sittampalam, G., Tibble, J.: Adding trace matching with free variables to AspectJ. In: Johnson, R.E., Gabriel, R.P. (eds.) OOPSLA 2005: Proceedings of the Conference on Object Oriented Programming, Systems, Languages, and Applications, pp. 345–364. ACM Press, New York (2005) 2. Bockisch, C., Mezini, M.: A flexible architecture for pointcut-advice language implementations. In: VMIL 2007: Proceedings of the workshop on Virtual machines and intermediate languages for emerging modularization mechanisms. ACM Press, New York (2007) 3. Bockisch, C., Mezini, M., Gybels, K., Fabry, J.: Initial definition of the aspect language reference model and prototype implementation adhering to the language implementation toolkit architecture. Technical report, AOSD Europe Deliverable, Technische Universit¨ at Darmstadt (2007) 4. Eichberg, M., Mezini, M., Ostermann, K.: Pointcuts as functional queries. In: Chin, W.-N. (ed.) APLAS 2004. LNCS, vol. 3302, pp. 366–381. Springer, Heidelberg (2004) 5. Hajiyev, E., Verbaere, M., de Moor, O.: codeQuest: scalable source code queries with datalog. In: Thomas, D. (ed.) ECOOP 2006. LNCS, vol. 4067, pp. 2–27. Springer, Heidelberg (2006) 6. Igarashi, A., Pierce, B., Wadler, P.: Featherweight Java: A minimal core calculus for Java and GJ. In: Berman, A.M. (ed.) OOPSLA 1999: Proceedings of the Conference on Object-Oriented Programming, Systems, Languages and Applications, NY, pp. 132–146 (1999)

A Classification Framework for Pointcut Languages

307

7. Janzen, D., De Volder, K.: Navigating and querying code without getting lost. In: Aksit, M. (ed.) AOSD 2003: Proceedings of the International Conference on AspectOriented Software Development, pp. 178–187. ACM Press, New York (2003) 8. Kiczales, G., Hilsdale, E., Hugunin, J., Kersten, M., Palm, J., Griswold, W.G.: An Overview of AspectJ. In: Knudsen, J.L. (ed.) ECOOP 2001. LNCS, vol. 2072, pp. 327–353. Springer, Heidelberg (2001) 9. Kiczales, G., Lamping, J., Mendhekar, A., Maeda, C., Lopes, C.V., Loingtier, J.M., Irwin, J.: Aspect-Oriented Programming. In: Aksit, M., Matsuoka, S. (eds.) ECOOP 1997. LNCS, vol. 1241, pp. 220–242. Springer, Heidelberg (1997) 10. Martin, M., Livshits, B., Lam, M.S.: Finding application errors and security flaws using PQL: a program query language. In: Ralph, R.P.G., Johnson, E. (eds.) OOPSLA 2005: Proceedings of the 20th annual ACM SIGPLAN conference on Object oriented programming, systems, languages, and applications, vol. 40, pp. 365–383. ACM, New York (2005) 11. Masuhara, H., Endoh, Y., Yonezawa, A.: A Fine-Grained Join Point Model for More Reusable Aspects. In: Kobayashi, N. (ed.) APLAS 2006. LNCS, vol. 4279, pp. 131–147. Springer, Heidelberg (2006) 12. Masuhara, H., Kiczales, G.: Modeling crosscutting in aspect-oriented mechanisms. In: Cardelli, L. (ed.) ECOOP 2003. LNCS, vol. 2743, pp. 219–233. Springer, Heidelberg (2003) 13. Masuhara, H., Kiczales, G., Dutchyn, C.: A compilation and optimization model for aspect-oriented programs. In: Hedin, G. (ed.) CC 2003. LNCS, vol. 2622, pp. 46–60. Springer, Heidelberg (2003) 14. Ostermann, K., Mezini, M., Bockisch, C.: Expressive Pointcuts for Increased Modularity. In: Black, A.P. (ed.) ECOOP 2005. LNCS, vol. 3586, pp. 214–240. Springer, Heidelberg (2005) 15. Pothier, G., Tanter, E., Piquer, J.: Scalable omniscient debugging. In: Bacon, D.F. (ed.) OOPLSA 2007: Proceedings of the International Conference on ObjectOriented Programming, Systems, Languages and Applications, pp. 535–552. ACM Press, New York (2007) 16. Stoerzer, M., Hanenberg, S.: A classification of pointcut language constructs. In: SPLAT 2005: Proceedings of the workshop on Software-Engineering Properties of Languages and Aspect Technologies (2005) 17. Walker, R.J., Viggers, K.: Implementing protocols via declarative event patterns. In: Taylor, R.N., Dwyer, M.B. (eds.) FSE 2004: Proceedings of the International Symposium on Foundations of Software Engineering, pp. 159–169. ACM, New York (2004) 18. Wright, A.K., Felleisen, M.: A syntactic approach to type soundness. Information and Computation 115, 38–94 (1994)

Fast Simulation Techniques for Design Space Exploration Daniel Knorreck, Ludovic Apvrille, and Renaud Pacalet System-on-Chip Laboratory (LabSoC), Institut Telecom, Telecom ParisTech, LTCI CNRS, 2229, Route des Crˆetes, B.P. 193, F-06904 Sophia Antipolis, France {daniel.knorreck,ludovic.apvrille,renaud.pacalet}@telecom-paristech.fr

Abstract. This paper addresses an open-source UML based toolkit named TTool - for performing efficient system-level design space exploration of Systems-On-Chip. Main modeling, verification and simulation capabilities of TTool are first presented, and exemplified by an MPEG2 application. Then, an innovative simulation strategy to significantly reduce simulation time is introduced. The basic idea is to take benefit from high level descriptions of applications by processing transactions spanning potentially hundreds of clock cycles as a whole. When a need for inter task synchronization arises, transactions may be split into smaller chunks. The simulation engine is therefore predictive and supports backward execution thanks to transaction truncation. Thus, simulation granularity adapts automatically to application requirements. Emphasis is more particularly put on procedures taking place under the hood after having pushed the TTool simulation button. Finally, the new simulation strategy is assessed and compared to an earlier cycle-based version of the simulation engine. Keywords: System-On-Chip, Design Space Exploration, System Level Modeling, UML, DIPLODOCUS, TTool, Fast Simulation Techniques.

1

Introduction

System-level design space exploration in a System-on-Chip (SoC) design cycle is an issue of great concern in today’s rapidly growing and heavily constrained design processes. The increasing complexity of SoC requires a complete re-examination of design and validation methods prior to final implementation. We believe that the solution to this problem lies in developing an abstract model of the system intended for design, on which fast simulations and formal static analysis could be performed in order to test the satisfiability of both functional and non functional requirements. In this context, we have previously introduced a UML-based environment named DIPLODOCUS. The strength of our approach relies on formal verification capabilities and fast simulation techniques ( [7] [18] ). DIPLODOCUS design approach is based on the following fundamental principles: M. Oriol and B. Meyer (Eds.): TOOLS EUROPE 2009, LNBIP 33, pp. 308–327, 2009. c Springer-Verlag Berlin Heidelberg 2009 

Fast Simulation Techniques for Design Space Exploration

– – – –

309

Use of a high level language (UML) Clear separation between application and architectural matters Data abstraction. Use of fast simulation and formal static analysis techniques, both at application and mapping levels

The designer is supposed to model in an orthogonal fashion the application and the architecture of the targeted system. Thereafter, a mapping stage associates application and architectural components. The strength of our approach relies in simulation and formal proofs techniques that can be applied to modeled systems at all methodological stages. Also, our environment totally hides knowledge of simulation or formal proofs techniques: knowledge of our UML profile is the only asset for engineers. Today, this methodology is supported by an open-source toolkit named TTool [3]. The first sections of the paper are dedicated to the presentation of TTool. Simulation is an integral part of the DIPLODOCUS methodology and enables the designer to get first estimates of the performance of the final system. The first version of our simulation environment relied on the standard SystemC kernel and took advantage of the integrated discrete event simulator. In that simulator, a SystemC process is assigned to each active hardware node, and a global SystemC clock is used as a means of synchronization. Abstract communication and computation transactions defined in the application model are broken down to the corresponding number of wait cycles. Unfortunately, the inherent latency of the SystemC scheduler due to frequent task switches made the simulation suffer from low performance. However, simulation speed is an issue of concern as it represents one argument to justify accuracy penalties as a result of abstractions applied to the model. In order to achieve a better performance, a new simulation strategy is introduced in the paper. It leverages these abstractions by processing high-level instructions as a whole whenever possible. Furthermore, its simulation kernel incorporates a slim discrete event scheduler so that SystemC libraries are not necessary any more. The paper is composed of seven sections. Section 2 gives an overview of the DIPLODOCUS methodology and points out the features of the integrated development environment TTool. Section 3 elaborates on the modeling stages and illustrates involved models with an MPEG2 case study. Section 4 introduces the main concepts of the simulation strategy and provides an insight into some important algorithmic aspects. Section 5 benchmarks the new simulation engine and compares its performance to the previous simulator version. Section 6 puts our approach in context with related works. Section 7 finally concludes the paper and draws perspectives.

2

The DIPLODOCUS Environment in a Nutshell

DIPLODOCUS is a UML profile targeting the design of System-on-Chip at a high level of abstraction. A UML profile customizes UML [12] for a given domain, using UML extension capabilities. Also, a UML profile commonly provides a methodology and is supported by a specific toolkit.

310

2.1

D. Knorreck, L. Apvrille, and R. Pacalet

Methodology

DIPLODOCUS includes the following 3-step methodology (see Figure 1): 1. Applications are first described as a network of abstract communicating tasks using a UML class diagram. The latter represents the static view of the application. Each task behavior is described with a UML activity diagram based on the given functional requirements. 2. Targeted architectures are modeled independently from applications as a set of interconnected abstract hardware nodes. UML components have been defined to model those nodes (e.g. CPUs, buses, memories, hardware accelerators, bridges). 3. A mapping process defines how application tasks can be bound to execution entities and also how abstract communications between tasks are assigned to communication and storage devices.

Fig. 1. Global view of our Design Space Exploration Approach

One main DIPLODOCUS objective is to help designers to find a hardware architecture that satisfies functional and non-functional requirements. To achieve this, DIPLODOCUS relies on fast simulation and formal proof techniques. The application models are designed independently of targeted underlying architectures. Due to the high abstraction level of both application and architecture, simulation speed can be increased significantly with regards to simulations usually performed at lower abstraction level (e.g. TLM level, RTL level, etc.). Additionally, efficient formal static analysis techniques may be applied before and after mapping. Within a SoC design flow, Design Space Exploration is supposed to be carried out at a very early stage, right after a specification document has been issued. In the scope of hardware/software partitioning, several appropriate architectures could be assessed by means of our methodology. Thus, it should be emphasized that DIPLODOCUS designs are settled above more detailed but still abstract models like TLM. Indeed, the main difference between DIPLODOCUS and TLM models relies in data (and not only signal) abstraction and the modeling of

Fast Simulation Techniques for Design Space Exploration

311

computational complexity. Hence, with DIPLODOCUS, a designer can already get a first estimate of the behavior of final applications even if some algorithmic details have not yet been stipulated. 2.2

Toolkit

The DIPLODOCUS UML profile - including its methodology - has been implemented in TTool [3]. TTool is an open-source toolkit that supports several UML2 / SysML profiles, including TURTLE [6] and DIPLODOCUS [5]. The main idea behind TTool is that all modeling may be formally verified or simulated. In practice, UML diagrams are first automatically translated into an intermediate specification expressed in a formal language, which serves as starting point for deriving formal specification and simulation code. TTool has the following main modeling capabilities (see Figure 2): – Modeling of applications using either a UML class and activity diagrams or a specific language called TML. From both formats, it is possible to generate simulation code or formal verification specifications from which formal proofs may be performed. – Modeling of architectures using UML deployment diagrams (or a textual format named TARCHI). – Modeling of mappings using UML deployment diagrams (or a textual format named TMAP). A mapping scheme associates applications and architectures. Based on these three building blocks (mapping, application, architecture), TTool automatically generates simulation code or formal specifications at the push of a button. From DIPLODOCUS application or mapping models, TTool may perform simulations or formal proofs: – For simulation purpose, TTool generates SystemC or C++ code. Indeed, TTool is currently equipped with two simulators: one based on the SystemC simulation engine, and another and more recent one implemented in pure C++. That generated code is linked with libraries - included in TTool before being executed. Also, The C++-based simulator is the main contribution described later on in the paper. This C++ simulator outperforms its SystemC counterpart in terms of simulation speed, but it is yet not compatible with third-parties SystemC-based components. – For formal proof purpose, TTool generates LOTOS or UPPAAL code. Formal verification of application or mapping models is out-of-scope of this paper. Before being translated into either SystemC, UPPAAL or LOTOS, models are first translated into intermediate languages. Figure 2 refers to this process.

312

D. Knorreck, L. Apvrille, and R. Pacalet

Fig. 2. Modeling and verification capabilities of TTool

3 3.1

Modeling SoCs with DIPLODOCUS Application Modeling

An application model is the description of functions to be performed by the targeted SoC. One has to bear in mind that, at application modeling level, there is no notion of time but only a partial ordering between actions. As mentioned before, functions are modeled as a set of abstract tasks described within UML class diagrams. One UML activity diagram per tasks represents its behavior and the diagram is built upon the following operators: control flow and variable manipulation operators (loops, tests, assignments, etc.), communication operators (reading/writing abstract data samples in channels, sending/receiving events and requests), and of computational cost operators. This section briefly describes a subset of the aforementioned operators as well as their semantics and provides definitions for Channels, Events and Requests: – Channels are characterized by a point-to-point unidirectional communication between two tasks. The following Channel types exist: • Blocking Read-Blocking Write (BR-BW) • Blocking Read-Non Blocking Write (BR-NBW) • Non Blocking Read-Non Blocking Write (NBR-NBW) – Events are characterized by a point-to-point unidirectional asynchronous communication between two tasks. Events are stored in an intermediate FIFO between the sender and the receiver. This FIFO may be finite or infinite. In case of an infinite FIFO, incoming events are never lost. When adding an event to a finite FIFO, the incoming event may be discarded or the oldest event may be dropped if the FIFO is full. Thus, a single element

Fast Simulation Techniques for Design Space Exploration

313

FIFO may be used to model hardware interrupts. In tasks, events can be sent (NOTIFY), received (WAIT) and tested for their presence (NOTIFIED). – Requests are characterized by a multi-point to one point unidirectional asynchronous communication between two tasks. A unique infinite FIFO between senders and the receiver is used to store all incoming requests. Consequently, a request cannot be lost. As some of the constituting operators of activity diagrams are referred to in later sections, a brief survey is provided which is not meant to be exhaustive (see Table 1). Table 1. Commands for modeling of task behavior Icon:

Command: Semantics: READ x

Read x samples in a channel

WRITE x

Write x samples in a channel

NOTIFY e

Notify event e with 3 parameters

WAIT e

Wait for event e carrying 3 parameters

REQUEST Ask the associated task to run EXECI x

Computational complexity of x execution units

IF

Branch command, can be deterministic or random Sequence command

Let us consider an example to illustrate our modeling approach: The class diagram depicted in Figure 3 is taken from a case study that we carried out recently. In the scope of that case study, we conceived a model of an MPEG decoder based on the DIPLODOCUS methodology and mapped it on several different architectures. The star structure of the diagram reveals that the task called Parser directs four other tasks by means of requests. The four remaining tasks are dedicated to the processing stages of an MPEG decoder: Variable Run Length Decoding (VLC), Inverse Quantization, Inverse Discrete Cosine Transform (IDCT) and Motion Compensation respectively. A chain of channels accounts for data dependencies between processing stages. As the parsing process and the run length decoding process are closely coupled, a synchronization becomes necessary. Thus, the VLC task is able to send an event to the Parser after having completed the decoding procedure.

314

D. Knorreck, L. Apvrille, and R. Pacalet

An excerpt of an activity diagram describing the behavior of the Parser task is given in Figure 4. The first branch of the sequence operator is intended to assign values to task variables depending on the picture type which is treated. Within the remaining three branches requests carrying parameters are sent to processing tasks in order to launch them. The whole structure is nested in a loop which is run through for each macro block, the second branch comprises a loop iterating through the coded blocks. These examples should give the reader a rough idea of how more complex models can be realized based on our approach. In addition to simulation, formal verifications could be conducted in order to verify the absence of deadlock situations and liveness properties.

Fig. 3. MPEG class diagram

Fig. 4. Extract of the activity diagram of the parser task

Fast Simulation Techniques for Design Space Exploration

3.2

315

Architecture Modeling

A DIPLODOCUS architecture is built upon the following parameterized hardware nodes: – Computation nodes. Typically, an abstract CPU model merges both the functionality of the hardware component and its operating system. The behavior of a CPU model can be customized by the following parameters (amongst others): data size, pipeline size, cache miss ratio and scheduling algorithm. – Communication nodes. A communication node is either a bus or a bridge. The bus model exhibits the following parameters: data size, latency and scheduling policy. Note that links which are established during the mapping stage are meant to interconnect a hardware node - except for buses - with a bus. A link may be annotated by a priority if the respective bus has a priority-based scheduling policy. – Storage nodes. Typically, a memory. Memories are parametrized by two measures: latency and data size. A DIPLODOCUS architecture is modeled in terms of a UML deployment diagram where DIPLODOCUS links and DIPLODOCUS nodes are depicted by their corresponding UML counterparts. 3.3

Mapping of Applications on Architectures

A DIPLODOCUS mapping is meant to describe the association of application elements - i.e. tasks, channels, requests and events - and hardware nodes. Thereby the following rules apply: – Abstract tasks must be mapped on exactly one computation node. – Abstract communication entities must be mapped on communication and storage nodes. For the time being, the simulation engine stipulates that a channel is mapped on n buses, n-1 bridges and exactly one storage element. Furthermore, all connected communication links have to form a continuous path without loops. A future version of the simulator should support several memory elements per channel. Depending on the mapping semantics, additional parameters may become necessary. For example, when mapping a task on a CPU node having a priority-based scheduling policy, task priorities have to be defined. The mapping stage is carried out based on previously created DIPLODOCUS architecture diagrams: symbols representing tasks and channels are simply bound to hardware components in a drag and drop fashion. Post-mapping formal specifications contain less traces than pre-mapping formal specifications since a mapping is intended to resolve shared resource allocations; i.e. the application model does not stipulate any temporal order of concurrent actions. Finally, traces obtained after mapping are supposed to be a subset of traces obtained before mapping.

316

D. Knorreck, L. Apvrille, and R. Pacalet

To illustrate the notions of architecture and mapping, let us come back to our MPEG decoder case study. Figure 5 illustrates an interesting subset of an architecture on which the decoder application is mapped. Two CPUs are in charge of evaluating the MPEG stream (CPU Controller) and reading back output images (CPU DisplaySim) for display purposes respectively. Three channels shall be considered exemplarily in the following: The Parser task reads samples from channel inputBuffer which is mapped on memory Mem Input and Bus Input. The DisplaySim task reads samples from channel outputBuffer which is mapped on memory Mem Output and Bus Output. The hardware accelerators executing task VLC and task Motion Compensation which communicate by means of channel decMotVec are omitted for the sake of simplicity. The latter channel is shown anyway to justify the presence of the bridge. Thanks to simulation capabilities of the DIPLODOCUS environment, the designer is capable of quickly comparing the given architecture to another one which for instance does not comprise a bridge (and thus the two buses converge to one). Without modifying the application, the designer will discover that in case of a bus utilization of 0.2 (Bus Output) and 0.07 (Bus Input), the average latency of bus transactions issued by a specific component is not significantly improved when introducing a bridge. For example the average latency per transaction seen by the hardware accelerator executing the VLC task amounts to 1.8 time units without bridge and to 0 time units when considering a bridge.

Fig. 5. Extract of the MPEG architecture diagram

In the remaining part of this paper, emphasis is put on post-mapping simulation of tasks mapped onto a given hardware architecture. More precisely, we introduce our new simulation scheme. It is indeed especially suited to exploit characteristics of our high level task and architecture descriptions. The two next sections of the paper are more particularly focused on the architecture of the simulation engine, and on its performance.

Fast Simulation Techniques for Design Space Exploration

4

317

Fast Simulation Techniques

Before going into details, it should be clearly emphasized what we understand by fast simulation. Our fast simulation approach is centered around two basic principles: – A modeling methodology which allows for conceiving abstract application models by applying both data and functional abstractions – A simulation strategy which exploits efficiently these characteristics of the high level model. The granularity of the simulation thereby matches the granularity of the application model. This implies that the gain in terms of simulation speed can hardly be expressed in cycles/per second as it highly depends on the application model. Indeed, in theory the simulator could attain any ratio of cycles/second if transactions were sufficiently large as we will see in the following sections. But in this case the respective model would be far from reflecting key characteristics of the real system due to the lack of expressiveness. The trade-off between detailedness and simulation speed is still subject to our research and can be varied smoothly thanks to our simulation environment. Nevertheless we tried to obtain some key figures based on several special simulation scenarios (refer to section 5) on the one hand and based on our case study of an MPEG decoder on the other hand. 4.1

Basic Principles

The simulator detailed in this paper is transaction-based. A transaction refers to a computation internal to a task, or a communication between tasks. Those transactions may obviously last from one to hundreds of clock cycles. Transaction durations are initially defined according to the application model, that is to say their maximum duration is given by the length of corresponding operators within tasks description. At simulation runtime, a transaction may have to be broken down into several chunks of smaller size, just because for example, a bus is not accessible and so the task is put on I/O wait on its CPU. Transaction cutting is more likely to happen when the amount of inter task communication is high and hence the need for synchronization arises. The aggregation of cycles may slightly impact simulation semantics when incorrect branch predictions arise on CPUs. The exact point in time of their occurrence cannot be resolved and thus the resulting delay is merely determined on a per-transaction base. Unlike a conventional simulation strategy where all tasks are running in lockstep, a local simulation clock is assigned to each active hardware component. Thus, simulation granularity automatically adapts to application requirements as abstract measures for computational, and communicational costs of operations are specified within the application model. The coarse granularity of the high level description is exploited in order to increase simulation speed.

318

D. Knorreck, L. Apvrille, and R. Pacalet

A problem at stake when considering transaction-based simulation is the scheduling of transactions themselves. This procedure embraces mainly four elements (see Figure 6): tasks, CPUs, buses and the main scheduler. The sequence of the entities of the aforementioned list also reflects their hierarchy during the scheduling process: tasks are settled at the top layer and the main scheduler constitutes the lowermost layer. Based on the knowledge of their internal behavior, tasks are able to determine the next operation which has to be executed within their scope. This operation is encapsulated subsequently in a transaction data structure and forwarded consecutively to hardware components being in charge of its execution. The basic idea is that a transaction carries timing information taken into account by hardware components to update their internal clock. Moreover, a hardware component may delay transactions and modify their duration according to the execution time needed by that specific component. By doing so, the simulation algorithm accounts for the speed of CPUs, the data rate of buses, bus contention and other parameters characterizing the hardware configuration.

Task

Task

CPU

Bus

Task

Task

Task

CPU

Task

CPU

Task

Scheduling Trans . length calculation

Scheduling Add bus delay to trans . length nextTransaction proposal

Main scheduler

Selects the transaction which Terminates first

Transaction to schedule

Fig. 6. Object collaboration during scheduling process

More precisely, on time management, tasks tag each transaction with its earliest start time (which is equal to the finish time of their previously executed transaction) and an abstract measure of the computational complexity or of the amount of data to transfer respectively. CPUs in turn have to recalculate the start time based on their internal schedule and are able to convert the abstract length to time units. Buses are in charge of handling communication transactions which are treated in a similar way. If several devices are involved in the execution process, transactions are simply passed from one device to another device and modified accordingly. Correct values for start time and duration are hence determined incrementally by forwarding transactions to dedicated hardware nodes. Finally, the main scheduler acts as a discrete event simulator and assures the causality of the simulation.

Fast Simulation Techniques for Design Space Exploration

4.2

319

Example Illustrating the Main Scheduler

To illustrate the cooperation of entities discussed previously and the main scheduler, we consider the following example (Figure 7): there are two CPUs, referred to as CPU1 and CPU2. A CPU model merges both an abstract hardware component and an abstract real time operating system. Both CPUs (CPU1 and CPU2) have already executed a transaction of a task T11 and T21 respectively. The transaction on CPU1 finished at tSI , the one on CPU2 finished at tSII . The schedulers of both CPUs subsequently propose the next transaction to execute. The simulation algorithm selects the transaction which will terminate first because its execution could have an impact on other tasks (making them runnable for instance). This decision is in line with traditional mechanisms used for discrete event simulation. As we have to be sure that during the execution of the selected transaction no events will occur, special care has to be taken when considering communication-related commands (read to channel, write to channel, wait on event, notify event, ...). The length of these transactions are therefore calculated based on the number of samples to read, the number of samples to write, the content and the size of the channel as well as the size of atomic burst transfers on the bus. After T12 has been scheduled, tSI is set to the end time of T12 , referred to as tSInew . If the completed transaction T12 causes a task to become runnable on CPU2, T22 is truncated at tSInew and the remaining transaction is scheduled on CPU2. As a channel always links only two tasks, revealing dependencies becomes trivial. If a portion of T22 has been added to the schedule, tSII is changed accordingly (tSII = tSInew ). After this, all schedulers of CPUs which have executed a transaction are invoked. The algorithm has now reached its initial state where all schedulers have selected a potential next transaction. Again, the transaction which finishes first is scheduled...

Fig. 7. Scheduling scenario

4.3

Transactions and the Timestamp Policy

This chapter elaborates on the attributes which characterize transactions. One has to discern two time scales within a transaction: a virtual one and an absolute

320

D. Knorreck, L. Apvrille, and R. Pacalet

one. The virtual time scale bases on the number of execution units (for commands modeling computational complexity) and the amount of data to transfer (READ/WRITE commands for abstract channels) respectively. Thus, the virtual time scale does not depend on architecture specific parameters (such as the speed/data rate of devices). The absolute time is calculated as a function of the virtual time and device parameters. The main attributes of a transaction are tabulated below: – startTime: Stands for the absolute point in time when the transaction starts. – length: Indicates the number of time units needed to execute the transaction. – virtualLength: Specifies the amount of data to read/write or the amount of processing units to carry out (example: for a READ 3 and a channel having a width of 2, the corresponding transaction spanning the whole operation would have a virtualLength of 6). – runnableTime: Indicates the point in time when the transaction gets runnable, that is when the dedicated task is ready to execute it. As stated in the introductory part of this paper, timestamps contained in transactions are exploited by hardware components as well as by the main scheduler. This paragraph elaborates on the contribution of each entity which modifies transactions - including commands and channels which have not been introduced yet. Transactions are initiated by commands as the latter keep track of their internal progress. Our modeling methodology provides abstract channels which have their representation within the simulation environment. Different semantics like Blocking Read-Blocking Write, Non Blocking Read-Non Blocking Write and Blocking Read-Non Blocking Write have to be respected. Each object depicted in Figure 8 is thus able to (re)calculate one or more transaction attribute(s) based on its internal state. Arrows denote the data flow of transaction pointers. 1. TMLCommand initially instantiates a transaction. The amount of remaining processing units (the difference between the virtualLength of the command and its progress) is assigned to virtualLength. Furthermore, a first estimate of the runnableTime is stored in the dedicated variable. The value simply amounts to the time when the last transaction of the command terminated. 2. Due to its inner state (i.e. filling level, references to transaction attempting to read/write,...), a channel is able to deduce the definite value of runnableTime. The time determined by the command is delayed if the channel is blocked due to an overflow or an underflow of data. TMLChannel recalculates virtualLength depending on the value proposed by TMLCommand and the content and the size of the channel (a detailed description of this procedure given later in this paper). In other words: a channel may truncate a transaction if its inner state does not allow for executing the transaction as a whole.

Fast Simulation Techniques for Design Space Exploration

321

3. The CPU class finally computes the start time of the transaction and its real duration. To determine the startTime, runnableTime has to be deferred to the end of the schedule (EndLastScheduledTransaction). The absolute length of a transaction depends on the product of the processor frequency and virtualLength as well as on possible task penalties (such as incorrect branch predictions, wake-up delays of CPUs, task switching times). If the objective of the transaction is a bus transfer, length is calculated within the bus class. 4. This last step is only carried out for transactions necessitating bus access. Possibly startTime is delayed once again due to bus contention and the real length of a transaction is calculated based on the bus frequency.

Fig. 8. Data flow of a transaction

4.4

Simulation Phases

The structure of the simulation algorithm is such that each command has to be prepared before it is able to execute. Thus, simulation enters alternately prepare and execution phases of commands. When the simulator is launched initially, the first command of each task is prepared. During simulation, only commands which were able to execute their current transaction have to prepare a new one. At the end of the prepare phase, transactions which might be processed by CPUs are known, only bus scheduling decisions are pending. That way, the prepare phase is crucial for the causality of the simulation. Following tasks are accomplished during the prepare phase: 1. It must be checked if the current command has been processed entirely. In this case, the progress of the command is equal to its virtual length and the following command is activated. 2. The number of remaining virtual execution units is calculated. 3. A new transaction object is instantiated. 4. If the concerned command tends to read/write samples from/to channels, it must be checked if the current state of the channel permits this operation. Therefore, read and write transactions are registered at the channel.

322

D. Knorreck, L. Apvrille, and R. Pacalet

5. Task variables may be evaluated or modified. The objective of the execution phase is to update state variables of the simulation after a transaction has been carried out. To achieve this, the following actions have to be taken: 1. Issue read/write operations on channels definitely so that state variables of channels are updated. 2. Update the progress of a command achieved by a completed transaction. 3. Add transactions to the schedule of CPUs and buses. Now, the basic phases have been introduced and attention should be drawn to the way they are arranged to conduct the simulation procedure: 1. At first, the prepare phase of the the first command of all tasks is entered. 2. Schedulers of all CPUs are invoked to determine the next transaction. Each CPU may have its own scheduling algorithm. 3. CPU schedulers register their current transaction at the appropriate bus in case the transaction needs bus access. 4. The main scheduler is in charge of identifying the runnable transaction t1 having the least end time. Therefore, the main scheduler queries all CPUs which may check in turn if bus access was granted for their communication transaction. Bus scheduling is triggered in a lazy manner when the scheduling decision is required by CPUs. 5. The execution phase for transaction t1 is entered. 6. If transaction t1 makes another task T runnable which is in turn able to execute a new transaction t3 , transaction t2 currently running on the CPU on which T is mapped is truncated at the end time of transaction t1 . The execution phase t2 is entered. During the next scheduling round, the scheduler of the aforementioned CPU is able to decide whether to switch to t3 or to resume t2 . 7. Commands which have been executing are prepared. 8. CPUs which have carried out a transaction are rescheduled. 9. Return to step 3

5

Experimental Results

Two important cases are considered to evaluate the performance of our simulation engine: event notification/reception on the one hand and communication via a data channel on the other hand. Therefore, two task sets have been executed on both simulators - the SystemC one, and the C++ one - and the execution time has been captured. The measurements have been performed under the following conditions: 1. Output capabilities of both simulators are disabled in order to measure the pure simulation time. No trace files are created. 2. The time consumed by the initialization procedure of both simulation environments is not taken into account for the same reason.

Fast Simulation Techniques for Design Space Exploration

323

3. All measurements are subject to noise caused by the multi tasking operation system and interfering tasks running on the CPU which executes the simulation. Therefore, the average of several measured values has been taken into account for all different tests. Thus the noise should distort similarly the results for both simulators. 4. As the analysis has been carried out on a less powerful machine, only the comparison of both simulators is meaningful in Figures 9 and 10, and not the absolute execution times. According to this, Figure 11 merely reflects the qualitative behavior of the new simulator; it performs much faster on recent machines. The first task set is composed of two tasks communicating by means of a Blocking Read-Blocking Write channel with the width parameter set to 1 and the length parameter set to 100. All commands are nested in a 1 million-loop structure. This way, the ratio of execution time of the simulator and noise due to the multitasking operating system (task switches,...) is increased. Let us now have a closer look at the example tasks. Task 1 writes x samples into channel ch1, performs x integer operations and finally reads x samples from channel ch2. Task 2 carries out the complementary operations, it reads x samples from channel ch1, performs x integer operations and writes x samples to channel ch2. The second task set is configured similarly: the only difference is that channels are replaced by events based on infinite FIFOs. Figure 9 and Figure 10 depict the execution time as a function of x which varies from 1 to 10. It can be deduced that the execution time of the old simulator grows more or less linearly with the command length. For the new simulator however, the execution time does not depend on the command length as one command corresponds to one transaction in this simulation. The execution time of the new simulator is only impacted by the number of transactions (linear growth, see Figure 11). But even in the worst case, when all transactions have a length of 1, the new simulator outperforms the old version by approximately a factor of 10. Depending on the respective model, the gain can be much higher given that a transaction comprises several cycles. Thus, the new simulator automatically adapts to the granularity of the application model: if a task issues commands having a length of one cycle, the simulator falls back on a cycle based consideration. If transactions are long, simulation efficiency increases accordingly.

6

Related Work

There are many approaches in practice today which tackle the Design Space Exploration issue. In the following, a few of the most relevant and widely practiced approaches are discussed very briefly. Some of the key features and limitations are highlighted and a comparison with our approach is provided: The Ptolemy environment [11] is based on a simulator that offers different levels of abstraction for jointly simulating - or verifying - hardware and software components. Ptolemy is rather focused on embedded systems than specifically on design space exploration.

324

D. Knorreck, L. Apvrille, and R. Pacalet

70 Old simulator New simulator 60

time [sec]

50

40

30

20

10

0

1

2

3

4

5 6 command length

7

8

9

10

Fig. 9. Data channel performance for old and new simulator

70 Old simulator New simulator 60

time [sec]

50

40

30

20

10

0

1

2

3

4

5 6 command length

7

8

9

10

Fig. 10. Event performance for old and new simulator

In POLIS environment, applications are described as a network of Codesign Finite State Machines (CFSMs) [9]. Each element in the network can be mapped onto either a hardware or a software entity. This approach differs from our work because it deals with application models mainly and there is no separation of architecture and application models. However, it is also intended for formal analysis. Metropolis is a design environment for heterogeneous systems [8]. It offers various models of computation (MoCs). However, both application and architecture are modeled in the same environment. It uses an Instruction Set Simulator (ISS) to generate execution traces which relies on executable application code. Hence, there is no abstraction at application level.

Fast Simulation Techniques for Design Space Exploration

325

22 20 18

t [sec]

16 14 12 10 8 6 4

6

8

10

12

14

16 18 MTransactions

20

22

24

26

Fig. 11. Transactions per second, new simulator (less powerful machine!)

In SPADE/SESAME, applications are modeled as Kahn Process Networks (KPN) which are mapped onto the architecture models and the performance is analyzed through simulation [13] [14] . There is separation of application and architecture. This approach utilizes symbolic instructions (abstractions of RISC instructions) and the models are not generic. In ArchAn (Architecture Simulation Environment), the architecture is modeled at the cycle-accurate level using a mixture of synchronous language Esterel and C. ArchAn focuses on the optimization of hardware architecture by early performance analysis [16] [10]. However the approach in ArchAn does not describe the application and architecture in an orthogonal fashion. It is not intended for formal analysis as well. [17] proposes a methodology to speed up simulations of multi-processor SoCs at TLM level with additional timing information. The idea of time propagation through transaction passing bears some resemblance with our new simulation approach. However, our environment is settled at a higher abstraction level so that the procedure of transaction passing has been extended to optimally support high level models. As our aim is fast simulation, our simulation engine is not based on the SystemC kernel any more. [15] also follows the Y-Chart approach. Mapping of applications to architectures is performed on graph-based descriptions and based on side-information provided by the system designer. The mapping methodology uses abstract information such as cycle counts, and as opposed to our approach the control flow of algorithms cannot be refined. The aforementioned framework is not intended for formal analysis. System level simulators like [4] and [2] are typically platform based, they rely on a bottom-up modeling methodology and require specific models of architecture components (processors, buses,...) to perform transaction level simulation. In that context, ISS may constitute an adequate means to provide cycle accurate behavior of CPU models. [1] yet goes a step further by introducing generic

326

D. Knorreck, L. Apvrille, and R. Pacalet

HW components being parameterizable by the designer and a partially graphical way to describe application functionality. The latter approach bears resemblance with our architecture model in the way that it is also generic and that architecture exploration is accomplished on application level. But in addition to that, the DIPLODOCUS methodology is based on data and functional abstractions which may further reduce simulation time. Due to that functional abstraction, all involved models are completely graphical and no line of code has to be produced.

7

Conclusions and Future Work

The paper first focuses on the description of TTool, a toolkit for performing efficient design space exploration at a high-level of abstraction. Formal verification and simulation techniques are integrated into TTool to assist designers in SoC architecture decisions. Modeling techniques are more particularly presented using an MPEG2 case study. A second important contribution introduced in the paper is the description of a lightweight discrete event simulation engine which is especially optimized for DIPLODOCUS high level application are architecture models. Simulation granularity automatically adapts to application models thanks to our transaction-based approach. When experimenting with models such the one of an MPEG2 decoder, we experienced performance gains in terms of execution time up to factor 30 as compared to a cycle-based SystemC simulator. It should be reemphasized that the gain depends on the application model, it could potentially be much higher. Several improvements are currently under development. At first, hardware components model could be enhanced. For example, the bus model shall reflect modern communication architectures comprising several possible communication paths (e.g. Network-on-Chip). Furthermore, latency due to buffers in buses and bridges has been omitted. Thus, model realism versus simulation complexity is still under study for hardware components. Second, we expect to perform profiling of our simulation engine so as to identify time-consuming sections of code and recode them (e.g. in assembly language). Also, multiprocessor / core architectures could improve simulation performance significantly. At last, more advanced simulation capabilities shall be introduced. Indeed, we expect the simulation engine to be able, at run-time, to check for functional requirements (e.g. if a client requests the bus, access is granted within 10ms). Also, it would also be very interesting to automatically explore several branches of control flow in order to enhance the coverage of simulations.

References 1. 2. 3. 4. 5.

CoFluent Studio, http://www.cofluentdesign.com Coware Virtual Platforms, http://www.coware.com TTool, the Turtle Toolkit, http://labsoc.comelec.enst.fr/turtle Vast System Engineering Tools, http://www.vastsystems.com Apvrille, L.: TTool for DIPLODOCUS: An Environment for Design Space Exploration. In: Proceedings of the 8th Annual International Conference on New Technologies of Distributed Systems (NOTERE 2008), Lyon, France (June 2008)

Fast Simulation Techniques for Design Space Exploration

327

6. Apvrille, L., de Saqui-Sannes, P., Pacalet, R., Apvrille, A.: Un environnement UML pour la conception de syst`emes distribu´es. Annales des T´el´ecommunications 61(11/12), 1347–1368 (2006) 7. Apvrille, L., Muhammad, W., Ameur-Boulifa, R., Coudert, S., Pacalet, R.: A umlbased environment for system design space exploration. In: 13th IEEE International Conference on Electronics, Circuits and Systems, ICECS 2006, December 2006, pp. 1272–1275 (2006) 8. Balarin, F., Watanabe, Y., Hsieh, H., Lavagno, L., Passerone, C., SangiovanniVincentelli, A.: Metropolis: an integrated electronic system design environment. Computer 36(4), 45–52 (2003) 9. Balarin, F., Chiodo, M., Giusto, P., Hsieh, H., Jurecska, A., Lavagno, L., Passerone, C., Sangiovanni-Vincentelli, A., Sentovich, E., Suzuki, K., Tabbara, B.: Hardwaresoftware co-design of embedded systems: the POLIS approach. Kluwer Academic Publishers, Norwell (1997) 10. Chatelain, A., Mathys, Y., Placido, G., La Rosa, A., Lavagno, L.: High-level architectural co-simulation using esterel and c. In: Proceedings of the Ninth International Symposium on Hardware/Software Codesign, CODES 2001, pp. 189–194 (2001) 11. Eker, J., Janneck, J.W., Lee, E.A., Liu, J., Liu, X., Ludvig, J., Neuendorffer, S., Sachs, S., Xiong, Y.: Taming heterogeneity - the ptolemy approach. Proceedings of the IEEE 91(1), 127–144 (2003) 12. Object Management Group. UML 2.0 Superstructure Specification, Geneva (2003), http://www.omg.org/docs/ptc/03-08-02.pdf 13. Pimentel, A.D., Polstra, S., Terpstra, F.: Towards efficient design space exploration of heterogeneous embedded media systems. In: Deprettere, F., Teich, J., Vassiliadis, S. (eds.) SAMOS 2001. LNCS, vol. 2268, pp. 57–73. Springer, Heidelberg (2002) 14. Pimentel, A.D., Erbas, C., Polstra, S.: A systematic approach to exploring embedded system architectures at multiple abstraction levels. IEEE Transactions on Computers 55(2), 99–112 (2006) 15. Ristau, B., Limberg, T., Fettweis, G.: A mapping framework for guided design space exploration of heterogeneous mp-socs. In: Design, Automation and Test in Europe, DATE 2008, March 2008, pp. 780–783 (2008) 16. Silbermintz, M., Sahar, A., Peled, L., Anschel, M., Watralov, E., Miller, H., Weisberger, E.: Soc modeling methodology for architectural exploration and software development. In: Proceedings of the 2004 11th IEEE International Conference on Electronics, Circuits and Systems, ICECS 2004, December 2004, pp. 383–386 (2004) 17. Viaud, E., Pecheux, F., Greiner, A.: An efficient tlm/t modeling and simulation environment based on conservative parallel discrete event principles. In: Design, Automation and Test in Europe, DATE 2006. Proceedings, March 2006, vol. 1, pp. 1–6 (2006) 18. Waseem, M., Apvrille, L., Ameur-Boulifa, R., Coudert, S., Pacalet, R.: Abstract application modeling for system design space exploration. In: 9th EUROMICRO Conference on Digital System Design: Architectures, Methods and Tools, DSD 2006, pp. 331–337 (2006)

PyGirl: Generating Whole-System VMs from High-Level Prototypes Using PyPy Camillo Bruni and Toon Verwaest Software Composition Group University of Bern – Switzerland

Abstract. Virtual machines (VMs) emulating hardware devices are generally implemented in low-level languages for performance reasons. This results in unmaintainable systems that are difficult to understand. In this paper we report on our experience using the PyPy toolchain to improve the portability and reduce the complexity of whole-system VM implementations. As a case study we implement a VM prototype for a Nintendo Game Boy, called PyGirl, in which the high-level model is separated from low-level VM implementation issues. We shed light on the process of refactoring from a low-level VM implementation in Java to a high-level model in RPython. We show that our whole-system VM written with PyPy is significantly less complex than standard implementations, without substantial loss in performance. Keywords: PyPy, Python, RPython, whole-system virtual machine, translation toolchain, high-level language, compile-time meta-programming Game Boy, 8-bit CPU.

1

Introduction

The research field revolving around virtual machines (VMs) is mainly split up in two large subfields. On the one hand we have the whole-system VM (WSVM) domain focusing on new ways to build and optimize emulators for hardware devices. These VMs mimic closely the actual hardware which they are emulating. On the other hand we have the language domain focusing on building high-level language VMs (HLLVM). These VMs only exist virtually. There are no hardware counterparts which natively understand the code running on those VMs. Although both domains share conceptual and implementation similarities, only recently has awareness been growing about the overlap of ideas and acknowledgement that the two fields can enforce each other. As a clear example of this fact we see that modern VM books discuss both fields [12]. Historically, the two fields developed independently of each other. Therefore the tools and techniques that are used in each of them are very different. Although concepts for performance enhancement, like just-in-time compilation, are used in both fields, especially the infrastructure and tools used to realize the systems are very different. An important progress that recently emerged in the field of language virtual machines is the use of higher-level models for describing virtual machines. Final M. Oriol and B. Meyer (Eds.): TOOLS EUROPE 2009, LNBIP 33, pp. 328–347, 2009. © Springer-Verlag Berlin Heidelberg 2009

PyGirl: Generating Whole-System VMs from High-Level Prototypes

329

VMs are then generated from these prototypes and enhanced by specific, low-level optimization techniques. This approach has been realized in the PyPy project. The PyPy project aims at building a complete high-level Python VM in Python rather than in a low-level language like C. All VM implementation details such as garbage collection and JIT compilation are excluded from this prototype. Performance and VM specific details are then reintroduced by applying several model transformation steps from the Python sources to a highly optimized target VM. In this paper we report on our experience using the PyPy translation toolchain1 to prototype WSVMs in a high-level language without sacrificing too much performance in the resulting VM, resulting in only about 40% slowdown compared to a similar WSVM in Java. By separating the high-level VM prototype from low-level implementation details we reduce code complexity. The high-level transformations provided by the PyPy toolchain ensure that the performance of the resulting VM is preserved. Our case study is a custom high-level VM prototype similar to the Squeak VM SPy [3]. SPy is written in RPython as a clean high-level implementation and uses the PyPy toolchain to reintroduce all VM implementation details. Rather than implementing a HLLVM we concentrate on emulating a Game Boy. We port an existing Java implementation of the virtual machine, Mario [5], to RPython. We focus on building a high-level and abstract but executable prototype rather than an inflexible and early optimized system. We then show that by using the PyPy toolchain we are able to generate performant low-level virtual machines from those prototypes. The main contributions of this paper are: – We show how the execution and implementation details of WSVMs are separated in the same way as those of HLLVMs. – We show how the use of preprocessing-time meta-programming minimizes the code and decreases the complexity. – We provide a sample implementation of a WSVM prototype for PyPy which exhibits a simplified implementation without substantial loss of performance (about 40% compared to a similar WSVM in Java). The remainder of this paper is structured as follows. In Section 2 we give an introduction to the PyPy project. Section 3 covers the technical details of the Game Boy, followed by the actual implementation details of PyGirl in Section 4. In Section 5 we compare the performance of the different WSVMs implementations. In Section 6 the future work is discussed. Finally in Section 7 we provide a brief overview of our achievements.

2

PyPy in a Nutshell

In this section we describe the PyPy project, which produced the toolchain that we use to translate our VM model. The PyPy toolchain transforms high-level 1

http://codespeak.net/pypy/

330

C. Bruni and T. Verwaest

prototypes into highly optimized executable binaries that incorporate all needed general VM features. The initial goal of PyPy was to write a full-featured, customizable and fast interpreter for Python written in Python itself, in order to have the language described in itself, i.e. a meta-circular interpreter. Running an interpreter on top of another interpreter results in execution so slow as to be almost useless. PyPy addresses this by providing a “domain specific compiler”, a toolchain that translates high-level VM prototypes in Python down to executables for different back ends, such as C/Posix [10]. Just like the interpreter, the toolchain itself is written in Python. The effort of generating a VM from a model in the language itself is similar to other self-sustaining systems such as Squeak where the VM is written in Slang. Slang is a subset of Smalltalk [4] which can be directly translated to C. The major difference between Slang and PyPy is that Slang is a thinly veiled Smalltalk-syntax on top of the semantics of C, whereas PyPy focuses on making a more complete subset of the Python language translatable to C. This difference is clearly visible in the level of abstraction used by programs written for the respective platforms [6]. For instance, in the Squeak VM exception handling is manually added to check some bit flags each time after returning from a function call. Not only is this a tedious task, but can also easily result in bugs by omitting a manual check. Using PyPy simplifies this task, since it is possible to use highlevel exception handling in the VM prototype which eventually gets translated down automatically to something similar in the executable. There are many examples of high-level language virtual machines (HLLVM) that are realized using higher-level languages. Jikes [7] and earlier Jalapeño [1] realized a complete, modern optimizing Java virtual machine in Java. Inspired by the idea of Squeak, the Squawk2 Java VM [11] realizes a Java virtual machine in Java. The virtual machine is implemented in Java and then translated aheadof-time to an executable. Klein [14] was a research project to explore how to realize and especially bootstrap a virtual machine for the dynamic prototype based language Self [13]. In all these examples the virtual machines realized were language virtual machines rather than virtual machines simulating real hardware. To our best knowledge none of the used frameworks and tools have been evaluated in the context of hardware VMs. 2.1

The Interpreter

The starting point for PyPy is to create a minimal but full interpreter for Python written in Python itself. By minimal we mean that all interpreter implementation details such as garbage collection and optimizations are not implemented but provided by the environment running the interpreter. This results in a very clean and concise implementation of the interpreter, modelling how the language works without obscuring it with implementation details. 2

http://research.sun.com/projects/squawk/

PyGirl: Generating Whole-System VMs from High-Level Prototypes

2.2

331

The Translation Toolchain

In this subsection we describe PyPy’s translation toolchain, a “domain specific compiler” that translates VM prototypes to executables binaries. The translation of high-level VM prototypes to low-level back ends is necessary since the prototypes do not run fast by themselves. The Python interpreter written in Python running on top of standard CPython runs code about 2000 times slower than CPython.3 The PyPy translation toolchain is designed as a flexible toolchain where front and back ends can be replaced so that it can generate VMs for different languages running on different platforms. Not only the front and back end can be changed, but also the set of transformations applied during the translation process. This results in fast and portable VMs. The following figure shows how prototype VM models can be translated to different back ends:

Prolog

Python Interpreter

Scheme Other interpreters…

JavaScript

Type and Flow Analysis

Specialize for object oriented environment

CLI backend

pypy.net

JVM backend

js.net

prolog.net

JS backend

JPyPy

js-jvm

Specialize for lowlevel environment

pypy-c

prolog-js

C backend

LLVM backend

scheme-c

prolog-c

…-llvm

pypy-llvm

Fig. 1. PyPy translation toolchain architecture

This overall architecture of PyPy is shown in Figure 1. On top we have the front end or VM prototype which is the input to the translation toolchain. As output we get a self-containing VM which contains all the required implementation details. This VM is compatible with one of the many back ends which PyPy targets. From here on we will discuss all the steps which the toolchain undergoes to go from prototype to back end specific VM. In short, the PyPy toolchain builds a dynamically modifiable flow graph from the target program’s sources. Then this prototype is transformed in several steps 3

http://codespeak.net/pypy/dist/pypy/doc/faq.html

332

C. Bruni and T. Verwaest

until a final binary results. Using the same intermediate representation for a large part of the translation and applying transformations in small steps allows us to customize every translation aspect. Translation Steps. In the first step the translator loads the source-code it translates. Unlike standard compilers that start by parsing the source code, the PyPy toolchain never accesses the Python source code. The toolchain can use its hosting Python interpreter to load its input files. This is because the input code for the translator is RPython code (which is a subset of Python code), and the toolchain itself is running on top of a Python interpreter. While loading, the Python interpreter evaluates all top-level statements and adds the loaded method and class definitions to the global Python memory. Afterwards the toolchain uses the the globally loaded main function as entry point for its input graph. This setup allows the input prototype to apply meta-programming in plain Python code at preprocessing time. Only the object graph resulting from metaprogramming has to be RPython compatible (Listing 2 shows an example of this feature). Because PyPy mostly targets statically typed back ends the graph is annotated with inferred types. Starting with the specified entry point, the type inference engine works its way through the object flow graph and tries to infer most specific types. If multiple types are possible for a certain node, the type inference engine tries to select the least common superclass as type. If the least upper bound degenerates to Object, an error is thrown to show that no specialization is possible. The same happens when two types have no common superclass, like booleans and objects. If such type-errors arise, they have to be fixed by the programmer. These problems are often solved by introducing a new common superclass or moving a method higher up the hierarchy. In other cases they really are semantic errors and require restructuring. Section 4.3 covers some aspects of resolving errors discovered by the toolchain. Here we see that the compiling process has an impact on the structure of the input source code. It effectively limits the expressiveness of the input language. For this reason we call the restricted input language understood by the PyPy toolchain RPython instead of Python. RPython is described in more detail in Section 2.3. The annotation step is followed by the conversion from a high-level flow-graph into a low-level one. Up to now there are two converters, one which specializes towards low-level back ends like C and one which specializes towards objectoriented back ends like CLI. To the low-level flow-graph optional back end optimizations are applied. These optimizations are rather similar to optimizations found in standard compilers, like function inlining and escape analysis. After the low-level flow-graph is optimized, it gets specialized for a specific back end. The preparation for code generation covers the following steps:

PyGirl: Generating Whole-System VMs from High-Level Prototypes

333

– Insertion of explicit exception handling. – Adding memory management details. Different garbage-collection strategies are available4. Note that these garbage collectors themselves are also written in Python code. They also get translated and woven into the VM definition. – Creation of low-level names for generated function and variables. Eventually the language-specific flow-graph is transformed into source files. These source files are then again processed by the back end, which can perform further domain-specific optimizations. For example generated C source files are compiled with GCC using the -O3 flag. 2.3

RPython

The PyPy translation toolchain is designed to boost the performance of the Python interpreter written in Python. More than just that, it translates general VM prototypes to fast executable binaries. The translation from a dynamicallytyped language like Python to a statically-typed language like C is not straightforward however. As mentioned before, in order to be able to preserve the semantics we are forced to limit the expressiveness of the input language. For this reason, when we talk about the language accepted by the translation toolchain, we do not refer to Python but rather to RPython or restricted Python. The language is defined implicitly by the translation toolchain5 The main differences between the full Python language and RPython are summarized in the following list: – – – –

Variables need to be type consistent, Runtime reflection is not supported, All globals are assumed to be constants, Types of all variables in the code must be inferable.

Although these restrictions seem to be substantial for a dynamic language such as Python, it is still possible to use high-level features like single inheritance, mixins and exception handling. More importantly, since RPython is a proper subset of Python, it is possible to test and debug the input programs with all Python tools before trying to translate it. Since VM prototypes we build for PyPy are executable by themselves, there is a great development speedup against a classical compile-wait-test cycle.

3

Game Boy Technical Details

As a case study we implement PyGirl, an executable VM prototype of a Game Boy. We will then translate this prototype using the PyPy translation toolchain 4 5

http://codespeak.net/pypy/dist/pypy/doc/garbage_collection.html http://codespeak.net/pypy/dist/pypy/doc/coding-guide.html# restricted-python

334

C. Bruni and T. Verwaest

presented in the previous section. In this section we list the technical details of the gaming hardware. The official documentation is available on Nintendo’s website6 Cartridge

CPU

Sound

ROM

Device

Video

RAM

3.1

JoyPad

Hardware Pieces

The Game Boy system is composed of six essential pieces which are accessible through shared memory. External events are supported through an 8 bit maskable interrupt channel. There are two kind of 8 bit opcodes: – First-order opcodes are executed directly – Second-order opcodes fetch the next instruction for execution. The combined opcode doubles the range of possible instructions at the cost of execution speed. The second-order opcodes are mostly used for bit testing and bit setting on the different registers. The following list shows some more specific details of the different parts of Nintendo’s gaming device. – The 8 bit CPU is a slightly modified version of the Zilog 80 with a speed of 4.19 MHz. The CPU supports two power-saving mechanisms both working in a similar way. After a certain interrupt, the CPU is put into a low power consumption mode which is left only after another interrupt has occurred. – The cartridge contains ROM with the embedded game and possibly additional RAM and/or other devices. The size of the RAM depends on the type of cartridge. Some types of cartridges support an additional battery to store game-state. Switchable memory banks are used to extend the 8 bit limited address range. A checksum in the header and a startup procedure are used to guarantee that the device is working correctly and the cartridge is not corrupted. – The supported resolution is 160×144 pixels with 4 shades. It is possible to show maximally 40 sprites of 8×8 or 8×16 pixels at the same time. The video chip has two tile-map memory regions, one for the background and one for the foreground. The actual background and foreground drawn on the screen 6

http://www.nintendo.co.uk/NOE/en_GB/support/game_boy__pocket__color_ 559_562.html

PyGirl: Generating Whole-System VMs from High-Level Prototypes

335

are cropped versions of the data available in the tile maps. This allows the Game Boy to easily refresh the tile maps. – A serial connection can be used to communicate with another device. – The sound chip supports stereo sound and has four internal mono sound channels. Sound can be either read directly from the RAM, thus creating arbitrary samples at the cost of its calculation, or it can be produced via a noise-channel or via two different wave-pattern generators.

4

PyGirl Implementation

In this section we highlight the most relevant implementation details of the PyGirl VM. We present a list of refactorings that we applied while going from a source implementation to our own model. We especially stress the details regarding the application of preprocessing-time meta-programming. Instead of starting to implement from a formal specification for the Game Boy we adapt an existing stereotypical VM implementation to PyPy. This allows us to show the differences of both implementation styles more easily. We then compare the complexity of our resulting prototype to various other Game Boy VM implementations: Mario which we refactor to our own implementation, JavaBoy7 and AEPgb8 . Table 1 shows how our final prototype differs from the other Game Boy VM implementations in terms of McCabe cyclomatic code complexity (MCC) [8]. It assigns a number to a piece of code corresponding to the number of possible traces through the code. Table 1. McCabe cyclomatic complexity of the Cartridge related classes Cartridge JavaBoy AEPgb Mario PyGirl r54984 r63242 KLoC 1.0 0.5 1.2 0.8 0.8 Number of methods 28 24 84 89 90 MCC Sum over all methods 220 103 268 217 211 Methods over MCC > 10 3 1 5 0 0 Max MCC 70 17 23 9 8 average 7.86 4.29 3.19 2.44 2.34

4.1

Source Implementation

In this section we present the important details of the Game Boy VM written in Java from which we started. The Game Boy VM Mario [5] is developed in a portable manner by abstracting out platform-specific details from certain components. Hence it provides variants of the emulator for the different versions of Java architectures like the Java Standard Edition and Applets for the web. 7 8

http://www.millstone.demon.co.uk/download/javaboy/ http://sourceforge.net/projects/aepgb/

336

C. Bruni and T. Verwaest

The application is structured by providing one class for each physical piece of hardware. Platform-specific parts are factored out by providing a set of abstract driver interfaces handling input and output. These drivers are then implemented for each architecture separately, adapting to the platform-specific requirements. Even though this is a fairly abstract and portable design, this already is an indicator that without a toolchain VM implementations are bound to be cluttered with back end specific details. While at first glance the code appears to be written in an object-oriented manner, many parts of Mario strictly follow the low-level execution details of the hardware. On top of this, the implementation is cluttered with local optimizations. Two types of optimization strategies clearly stand out: manual inlining of code and manual unrolling of loops. Both strategies result in an overly expanded code-base, obscuring the overall design and semantics. For example, the CPU class is cluttered with such speed optimizations. The reason is that a CPU is a very low-level general-purpose device which does not provide many possibilities for abstraction. However, even the video chip is implemented in a non-abstract procedural way. This is so even though there are more conceptual components ready for abstract representation, such as sprites, background and foreground. PyGirl uses high-level abstractions for these components resulting in less complex code. Table 2 shows that our Video classes are less complex than the ones from Mario. Table 2. McCabe cyclomatic complexity of the Video related classes Video JavaBoy AEPgb Mario PyGirl r54984 r63242 KLoC 1.0 1.1 1.2 0.6 1.1 Number of methods 53 108 68 76 182 MCC Sum over all methods 212 288 223 201 293 Methods over MCC > 10 5 5 5 3 0 Max MCC 36 24 20 15 8 average 4.00 2.67 3.28 2.64 1.61

4.2

From Java to Python

Now we show how we migrated the source VM from to Java to Python. In a first step we ported the code one-to-one, in order to easily track the upcoming refactoring progress. During the whole process we keep the overall structure of the existing system because it directly corresponds to the hardware. The following sections cover different refactorings we apply and abstractions introduced to go from a low-level detailed implementation to a high-level prototype of the VM. Memory Usage Considerations. The Java code is cluttered with type-casts between bytes and integers. Bytes are used to represent the 8 bit hardware

PyGirl: Generating Whole-System VMs from High-Level Prototypes

337

architecture, whereas integers are used for all sorts of arithmetic operations. Instead of using integers whenever possible, the Java version focuses on reducing the memory footprint of the running emulator and focuses on the implementation details of the original hardware. Only at very few places in the code is the use of bytes justified by the resulting two’s-complement interpretation of the numbers. In our prototype type-casts are removed wherever possible to improve readability and maintainability. Firstly we consider the memory footprint of the device we emulate too small to justify optimization of memory usage. Even if we use four to eight times as much memory as the original device would have used, corresponding to an expansion from 8 to 32 or 64 Bit, this would mean that we end up with about 20Mb memory usage. Running PyGirl on a 64 Bit machine results in a total memory usage of 24Mb. This is a negligible amount for modern computers. Secondly and more importantly it is, hypothetically speaking, possible to plug an additional transformation into the toolchain converting all integers to bytes. By doing so the memory footprint of the final VM would again be equal to the one of the original device. Metaprogramming. As described in Section 2.2 we can use the full Python language at preprocessing-time for meta-programming. The most prominent candidate for refactoring is the CPU class. It is packed with duplicated and inlined code and has a typical opcode dispatch switch. Table 3 shows the MCC for the CPU class in Mario, the two other Game Boy VM implementations AEPgb and JavaBoy and two snapshots of PyGirl. Table 3. McCabe cyclomatic complexity of the CPU related classes CPU JavaBoy AEPgb Mario PyGirl r54984 r63242 KLoC 2.7 2.9 4.2 1.2 1.0 Number of methods 22 111 372 173 180 MCC Sum over all methods 801 704 996 272 252 Methods over MCC > 10 5 1 2 0 0 Max MCC 415 536 513 10 9 average 36.41 6.34 2.28 1.57 1.4

PyGirl’s CPU class has half of the number of methods that Mario has. The sum of the code complexity over all methods is drastically reduced. One reason for the high complexity sum of Mario’s CPU class is its size. The original class is around 4000 lines long, featuring an unpleasant 1700 line switch delegating the incoming opcodes. On top of that there is a nested switch of 800 LoC handling the second-order opcodes. An excerpt of this dispatch switch is given in the following listing:

338

C. Bruni and T. Verwaest

public void execute(int opcode) { switch (opcode) { case 0x00: this.nop(); break; ... case 0xFF: this.rst(0x38); break; default: throw new RuntimeException(ERR); } }

In both switches we identify patterns which can be used as basis for abstractions. In the following excerpt from the original Java code we see how bytecodes directly encode their semantics in a structured way: public void execute(int opcode) { ... case 0x78: this.ld_A_B(); case 0x79: this.ld_A_C(); ... } public final void ld_A_B() { this.a = this.b; this.cycles -= 1; } Listing 1. Java: Grouped opcode mappings

The Java code covers all these switch cases by manually specifying them and by encoding the logic in one function per opcode. Since all these operations are symmetrical in terms of semantics and use of cycles the code can be compacted by using meta-programming. Even while there are only few lines of code per operation there is quite some redundancy. Instead of separate functions PyGirl uses a single load function for all register combinations. We reuse the function for multiple opcodes by applying the load function to each time two register objects. Every combination of the load function with two registers is related to a single opcode encoding its meaning. def load(self, register1, register2): register1.set(register2.get())

PyGirl: Generating Whole-System VMs from High-Level Prototypes

339

To refactor Listing 1, we create such reusable functions for all the different types of operations. Then we replace the switch with a compact lookup in an opcode table generated from the abstract functionality descriptions. def execute(self, op_code): OP_CODES[op_code](self)

Instead of hard-coding the mapping to the respective functions, we use metaprogramming to compute the definition at translation time. The mapping of opcodes to functions in the example Listing 1 can be replaced with a more compact definition. We specify the connected opcodes in a set of entries, each consisting of a starting opcode, an offset, a function and a set of registers. At runtime a sequence of opcodes is mapped to such a function. A corresponding register out of the register set is passed as an argument to this function. An example of this compact opcode definition is given in the following listing: REGS = [CPU.get_bc, CPU.get_de, CPU.get_hl, CPU.get_sp] SET = [ (0x01, (0x03, (0x09, (0x0B,

0x10, 0x10, 0x10, 0x10,

CPU.fetch_double_register, REGS), CPU.inc_double_register, REGS), CPU.add_hl, REGS), CPU.dec_double_register, REGS),

... (start, step, func, registers) ... ] OP_CODE_TABLE += create_op_codes(SET) Listing 2. RPython: Compacted definitions of opcodes

In the first line we see that we do not directly specify the register. Instead we use getter methods of the CPU class returning these registers at runtime. The first line of the SET specifies that the opcode 0x01 is mapped to fetch_double_register passing in the register returned by get_bc(). The opcode is 0x11 using the result of get_de() as argument to the function. Since there are 4 registers in the SET the last opcode in this sequence is 0x31. Next we have to create the concrete methods from this definition. Helper function take the opcode definition set at preprocessing time and create corresponding closures. The create_op_codes method in the following listing creates such closures: def create_op_codes(table): op_codes = [] for entry in table: op_code = entry[0] step = entry[1]

340

C. Bruni and T. Verwaest function = entry[2] for getter in entry[3]: op_codes.append( (op_code, register_lambda(function, getter))) op_code += step return op_codes

Since Python handles the scope of variables at the function rather than the block level we introduced a further helper method register_lambda for creating the closures. In the following excerpt you can see how this method creates a specific closure depending on the incoming registerOrGetter argument: def register_lambda(function, registerOrGetter): if callable(registerOrGetter): return lambda s: function(s, registerOrGetter(s)) else: return lambda s: function(s, registerOrGetter)

We create opcode table entries at preprocessing time not only for most of the register operations such as loading and storing, but also for nearly all other register operations. In total we apply meta-programming to generate about 450 out of all 512 opcodes. As a note on performance, when translating this code to C, PyPy is able to take the preprocessed opcode table and translate it back into an optimized switch. So the source code stays compact and maintainable without substantial loss in performance. Even better, future versions of PyPy are expected to automatically optimize running bytecode interpreters dynamically towards the bytecode they evaluate, i.e. JIT compiling the bytecodes. PyPy can also inline small methods. Although there are no inlined methods in Mario, it would be the next logical step for manually optimizing the code. By letting PyPy handle the optimizations, we maintain a clean implementation. 4.3

Translation

Directly trying to translate our VM prototype, which is initially full Python code, to a low-level back end raises conflicts. This is due to the fact that the PyPy translation toolchain does not take full Python code as input, but rather a restricted subset of Python called RPython. The restrictions imposed by RPython only become apparent when you try to translate prototypes with PyPy. Most bugs come from the fact that while Python is fully polymorphic, RPython enforces the static types of all variables to be correct. Every variable needs to be inferable to a specific type. The most generic type in the system, i.e. Object, is not allowed as type for any variable. All messages sent to instances of a class must be declared in that class, or in any of its superclasses.

PyGirl: Generating Whole-System VMs from High-Level Prototypes

341

The easiest translation bugs are straightforward syntactic bugs, like typos. Such bugs often go unnoticed in dynamic programming languages since they are not statically enforced and are only a problem at runtime. After fixing these, you typically encounter type conversion errors. Some of these problems are related to assignments of objects of different types to the same variable. One possible solution for static type inconsistencies is to introduce common abstract superclasses. This ensures that objects assigned to a single variable have this superclass as a common type. Call Wrappers. We handle operations on registers and other CPU functions elegantly by passing function closures around. This allows us to reuse methods for different actions. A very common example is the following load function: def load(self, getter, setter): setter(getter()) load(self.flag.get, self.a.set) load(self.fetch, self.a.set)

The load function is called with different arguments. In the first example the function is used to copy the values from the flag-register into the register a. The second example loads the next instruction into register a. When we first translated this code fragment the toolchain was unable to transform the code into a typed counterpart. PyPy was unable to create a strict typed function due to the different origins of the passed function closures. We resolve this by creating call-wrappers with a common superclass. Introducing call-wrappers for the passed function closures adds some overhead, but it is a simple way to keep the code minimal and close to the original idea of passing around closures. The following code shows the same function calls using wrappers for each passed type of closure: class CallWrapper(object): def get(self, use_cycles=True): raise Exception("called CallWrapper.get") def set(self, value, use_cycles=True): raise Exception("called CallWrapper.set")

class RegisterCallWrapper(CallWrapper): def __init__(self, register): self.register = register def get(self, use_cycles=True): return self.register.get(use_cycles) def set(self, value, use_cycles=True): return self.register.set(value, use_cycles)

342

C. Bruni and T. Verwaest

class CPUFetchCaller(CallWrapper): def __init__(self, CPU): self.CPU = CPU def get(self, use_cycles=True): return self.CPU.fetch(use_cycles)

RegisterCallWrapper takes a register and calls get or set on it. The CPUFetchCaller is an abstraction for the fetch method of the CPU that allows it to be used as an argument for the load function. These are two out of the five total callwrappers we use to handle closure passing. The common superclass CallWrapper makes it possible for PyPy to infer a common type for the argument of every method. Note that return-types of methods are also required to be type-correct. For our wrappers this implies that all get methods are required to return a value of the same type. Thus all get methods return integers. To support the callwrappers we replaced the closure calls. Instead of directly invoking the argument as in Listing 4.3 we call get or set on the call wrappers. Applied to the previously presented load method this resulted in the following code: def load(self, getCaller, setCaller): setCaller.set(getCaller.get())

Then the method-calls from the original example look now like load(RegisterCallWrapper(self.flag), RegisterCallWrapper(self.a)) load(CPUFetchCaller(self), RegisterCallWrapper(self.a))

While it is possible to do the same in Java, this imposes a huge runtime overhead. If we create the CallWrappers in Java, this form of meta-programming is not handled at preprocessing time. Instead the objects will be around at runtime, which implies a performance as well as a memory overhead.

5

Performance Evaluation

In this section we show that using the PyPy toolchain to reduce complexity of WSVMs does not result in substantial performance loss. To do so we compare the performance of our Game Boy VM PyGirl to the performance of our source implementation, Mario. We run benchmarks on three different versions of the Game Boy VM. The original Java emulation Mario, the interpreted variant of PyGirl and finally the translated binary version of PyGirl. We benchmark the interpreted PyGirl by running it on top of CPython whereas the translated version is built from those sources. Each test shows the average execution time over 100 runs using Java 1.6.0_10, Cacao 0.97 and CPython 2.5.2 (additionally using Psyco 1.6-1) on a 64 Bit Ubuntu 8.04.1 server machine with an Intel Xeon CPU QuadCore 2.00 GHz processor. We use revision number r63242 of the PyPy project.

PyGirl: Generating Whole-System VMs from High-Level Prototypes

5.1

343

Benchmark Details

In this section we discuss the benchmark parameters. The binaries for the RPython benchmark are created using the default arguments for PyPy resulting in C code which is then transformed to an executable binary via GCC. PyPy uses the following GCC optimizations when creating the binary executable: -c -O3 -pthread -fomit-frame-pointer In order to show the difference and impact of a JIT and dynamic optimization for the interpreted PyGirl we use Psyco [9], a just-in-time specializer for Python. Since the current version of Psyco (1.6-1) only emits machine code for 32 bit intel-based systems, we benchmark our code using Psyco on a similar Ubuntu installation with the appropriate 32 bit processor. Since those results are less important than the comparison with Java, we simply scaled them to fit the 64 Bit machine. Those results should only display the possible performance gain by using a JIT. Still, it highlights that running the emulator on top of Psyco equally results in unacceptable performance. For those who are missing the -server switch in the Java 1.6 benchmark, we can say that this option did not result in any significant performance gain for the tests. The speedup is less than a percent and about the order of the standard deviation, thus the results are merely distinguishable from the standard configuration. 5.2

Runtime Optimization Comparison

In this section we compare the performance of Mario and PyGirl. In order to test performance we let both systems run a ROM which exercises the video output9 but produced no sound10 . The ROM simply prints hello world! on the background while a smiley moves over the screen as a moving sprite, a simple but complex enough benchmark. In the Table 4 you can see the actual CPU time used by the emulators in relation to the actual game time. By game time we mean the actual time that a user spends playing a game at normal speed. By CPU time we mean the time spent by the processor of the host actually running the emulator. Notice that as game time increases, the difference in CPU time between Mario and the translated PyGirl emulator shrinks. Eventually Mario gets faster than PyGirl due to the runtime optimizations of the JVM. Figure 2 shows the performance for both emulators running on different platforms. It is clearly visible that the translated PyGirl runs at linear speed, whereas the Mario behaves differently. The JVM’s JIT is only warmed up after about 30 to 40 seconds of actual game time. The Cacao VM performs somewhat differently from the standard JVM implementations. Cacao’s JIT already runs Mario optimized after very little game time, but ends up being even a bit 9 10

The pixels are drawn in internal buffers but are not actually written to a screen. In both systems we fully disabled the sound drivers so that they cannot have any impact on performance.

344

C. Bruni and T. Verwaest

Table 4. Influence of the JIT on the benchmark results. Average execution in seconds over 100 runs per test. Game Time RPython CPU time 1 0.03 4 0.12 10 0.29 15 0.43 30 0.86 60 1.73 300 8.60 600 17.23 1800 51.81 3600 103.62

Java 1.6 CPU time Ratio 0.19 0.15 0.25 0.46 0.37 0.78 0.46 0.93 0.74 1.17 1.28 1.36 5.47 1.57 10.73 1.61 31.65 1.64 63.08 1.64

CPU time for emulation [Seconds]

100

10

1

0.1

0.01

RPython r63242 Java 1.6.0.10 Java Cacao 0.97 CPython 2.5.2 CPython 2.5.4 Psyco 10

100

Game time on the Game Boy [Seconds]

Fig. 2. This graph shows the CPU time of the host needed to emulate an example ROM, in relation to the game time on the Game Boy. It compares the performance of PyGirl and Mario emulating all device parts excluding sound. The results are discussed in Section 5.2.

slower than PyGirl. While Mario on the standard JVM has the advantage of the availability of a JIT compiler and dynamic optimizations, it is only about 60% faster than PyGirl.

PyGirl: Generating Whole-System VMs from High-Level Prototypes

345

Simple benchmarks running on simple language interpreters which are compiled with PyPy to its CLI-back end11 have shown that a JIT compiler can also be used here to get significant performance gains. We strongly believe that future versions of PyPy will directly improve the performance of the PyGirl VM, thanks to a generated JIT compiler, by the same order of magnitude. Future versions of PyPy featuring dynamic optimizations for the C-back end might even help us outplay the performance of Mario running on the standard JVM.

6

Future Work

In this section we discuss the future work needed for PyGirl and related tasks for PyPy. 6.1

Future Work for the Game Boy VM PyGirl

While most hardware parts already have a fully functioning software counterpart, this is not the case yet for the sound unit. This is mostly the case since it is the least important piece of hardware for the Game Boy emulator to be immediately usable. For it to work it still needs to be fully ported and refactored. 6.2

Future Work for PyPy

PyGirl’s current state of implementation only allows the code to be translated using the C-back end. The current implementation of the I/O drivers are based on libSDL12 , thus it is not yet compatible with other back ends. The fact most VMs only need very basic graphics support, like simple bitblitting, makes enabling graphics support for the other back ends a straightforward task. Translating PyGirl with the JVM as target makes it possible to directly compare the performance of the original Java implementation with our approach. Since the Java Virtual Machine is widely available and compatible with many different platforms, this would eventually even allow us to run PyGirl on mobile devices [2].

7

Conclusion

In this paper we have shown that the use of high-level prototypes for the definition of whole-system virtual machines and the application of meta-programming reduces code complexity significantly without substantial loss of performance. By using the translation toolchain PyPy we keep our high-level WSVM model free from low-level implementation details, such as garbage collection and exception handling. When we build a final WSVM from out prototype, those details are reintroduced by the translation toolchain. 11 12

http://morepypy.blogspot.com/2008/11/porting-jit-to-cli-part-1.html http://www.libsdl.org/

346

C. Bruni and T. Verwaest

We have supported our claims by comparing the performance and complexity of two VM implementations for the Game Boy, PyGirl and Mario. We found that for specific classes the average McCabe cyclomatic complexity is reduced to less than the half compared to other Game Boy VMs. In other cases, like the CPU class, we even reduced the maximum MCC from over 500 down to 9. In order to strengthen our case about the reduced complexity, we compared the complexity of our prototype with yet another two Game Boy emulators, exhibiting the same complexity issues as Mario. In Section 5 we have shown that using PyPy does not result in a significant performance loss. PyGirl only runs about 40% slower than the Game Boy VM Mario on the standard JVM. Acknowledgments. We would like to thank Marcus Denker, Adrian Kuhn, Jorge Ressia, Oscar Nierstrasz and the anonymous reviewers for kindly reviewing this paper. We gratefully acknowledge the financial support of the Swiss National Science Foundation for the project “Bringing Models Closer to Code” (SNF Project No. 200020-121594, Oct. 2008 - Sept. 2010).

References 1. Alpern, B., Attanasio, C.R., Cocchi, A., Lieber, D., Smith, S., Ngo, T., Barton, J.J., Hummel, S.F., Sheperd, J.C., Mergen, M.: Implementing Jalapeño in Java. In: Proceedings of the 14th ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications (OOPSLA 1999), pp. 314–324. ACM, New York (1999) 2. Aubry, L., Douard, D., Fayolle, A.: Case study on using PyPy for embedded devices. Technical report, PyPy Consortium (2007) 3. Bolz, C.F., Kuhn, A., Lienhard, A., Matsakis, N.D., Nierstrasz, O., Renggli, L., Rigo, A., Verwaest, T.: Back to the future in one week — implementing a Smalltalk VM in PyPy. In: Workshop on Self-sustaining Systems (S3) 2008, page TBA (to appear) (2008) 4. Goldberg, A., Robson, D.: Smalltalk 80: the Language and its Implementation. Addison Wesley, Reading (1983) 5. Hasan, C.: Mario (2007) 6. Ingalls, D., Kaehler, T., Maloney, J., Wallace, S., Kay, A.: Back to the future: The story of Squeak, a practical Smalltalk written in itself. In: Proceedings of the 12th ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications (OOPSLA 1997), November 1997, pp. 318–326. ACM Press, New York (1997) 7. The Jikes research virtual machine, http://jikesrvm.sourceforge.net/ 8. McCabe, T.J.: A measure of complexity. IEEE Transactions on Software Engineering 2(4), 308–320 (1976) 9. Rigo, A.: Representation-based just-in-time specialization and the psyco prototype for Python. In: PEPM 2004: Proceedings of the 2004 ACM SIGPLAN symposium on Partial evaluation and semantics-based program manipulation, pp. 15–26. ACM, New York (2004) 10. Rigo, A., Pedroni, S.: PyPy’s approach to virtual machine construction. In: Proceedings of the 2006 conference on Dynamic languages symposium, OOPSLA 2006: Companion to the 21st ACM SIGPLAN conference on Object-oriented programming systems, languages, and applications, pp. 944–953. ACM, New York (2006)

PyGirl: Generating Whole-System VMs from High-Level Prototypes

347

11. Simon, D., Cifuentes, C., Cleal, D., Daniels, J., White, D.: Java on the bare metal of wireless sensor devices: the squawk java virtual machine. In: VEE 2006: Proceedings of the 2nd international conference on Virtual execution environments, pp. 78–88. ACM Press, New York (2006) 12. Smith, J.E., Nair, R.: Virtual Machines. Morgan Kaufmann, San Francisco (2005) 13. Ungar, D., Smith, R.B.: Self: The power of simplicity. In: Proceedings OOPSLA 1987, ACM SIGPLAN Notices, December 1987, vol. 22, pp. 227–242 (1987) 14. Ungar, D., Spitz, A., Ausch, A.: Constructing a metacircular virtual machine in an exploratory programming environment. In: OOPSLA 2005: Companion to the 20th annual ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications, pp. 11–20. ACM, New York (2005)

Using Grammarware Languages to Define Operational Semantics of Modelled Languages Daniel A. Sadilek and Guido Wachsmuth Humboldt-Universit¨ at zu Berlin, Unter den Linden 6, 10099 Berlin, Germany {sadilek,guwac}@informatik.hu-berlin.de

Abstract. Abstract State Machines, Prolog, and Scheme are wellestablished for language engineering in the technological space of grammarware. In this paper, we show how they can be integrated into the technological space of modelware to describe operational semantics of modelled languages. Integration is based on three bridges: a physical, a logical, and a pragmatical bridge between grammarware language and modelling framework. We applied our approach to integrate Abstract State Machines, Prolog, and Scheme as description languages in EProvide 2.0, an extensible, unified framework for prototyping operational semantics of modelled languages. In this paper, we discuss in detail the bridging of Abstract State Machines as well as an industrial case study based on this.

1

Introduction

Technological spaces. This paper is concerned with describing operational semantics of computer languages. For this task, a language engineer needs a description language. Our goal is to enable language engineers to use grammarware description languages for defining the operational semantics of modelled computer languages, i.e., of languages that are, apart from their operational semantics, defined with modelware technologies. Modelware and grammarware are technological spaces. The term was initially proposed by Kurtev et al. to name a “working context with a set of associated concepts, body of knowledge, tools, required skills, and possibilities” [9]. The grammarware technological space is concerned with grammars, grammar-based description languages, and associated tools. The modelware technological space is concerned with metamodels, model-based description languages, and associated tools. Operational Semantics. In the grammarware technological space, operational semantics as pioneered by Plotkin [13] is a common way for specifying language semantics in terms of sequences of computational steps. A transition system Γ, → forms the mathematical foundation, where Γ is a set of configurations and → ⊆ Γ × Γ is a transition relation. This approach has several benefits: First, we can describe the operational semantics in terms of the language’s structure. Thus, we stay in the language’s domain so that experts of that domain can understand these structures and can explain the expected effects of execution. M. Oriol and B. Meyer (Eds.): TOOLS EUROPE 2009, LNBIP 33, pp. 350–358, 2009. c Springer-Verlag Berlin Heidelberg 2009 

Using Grammarware Languages to Define Operational Semantics

Puzzle + iDimension: int 1..* cells

Cell

1..* fields

Field

fields 1..*

cells + iCellValue: int 1..* + iPossibleValues: int[0..*]

351

Run(s:Sudoku) =def forall f in s.fields do RunElimination (f) forall f in s.fields do RunSingleCell (f) forall f in s.fields do RunHiddenCell (f) forall f in s.fields do RunLockedCandidates (f) RunElimination(f:Field) =def forall c in f.cells with c.iCellValue0 do forall cc in f.cells do delete c.iCellValue from cc.iPossibleCellValues RunSingleCell(f:Field) =def forall c in f.cells with c.iCellValue = 0 and c.iPossibleCellValues.size = 1 do choose v in c.iPossibleCellValues do c.iCellValue := v

Row

Column

Box

...

Fig. 1. Simplified Sudoku metamodel and ASM semantics description from [7]

Second, if expressed in an appropriate language, an operational semantics description can be executed and, thus, be used as a prototypical interpreter. Certain grammarware languages are commonly used for this purpose, e.g., Abstract State Machines (ASMs), Prolog, or Scheme. In the modelware technological space, the standard way to prepare the execution of models is to translate them (by model transformation or code generation) into an executable target language. However, several proposals were made that allow to define operational semantics directly in the modelware technological space [11,14]. Their unifying idea is to model Γ , the possible runtime states of language instances, with a metamodel. But they use different description languages to model the transition relation →. In [19], we employ QVT Relations [12] for this purpose. We combine this approach with existing model-driven editor creation technology in order to support rapid prototyping of animated visual interpreters and debuggers [15]. We called the implementation of that approach EProvide. Motivating example. As described above, there are established languages for prototypically implementing operational semantics in grammarware. People are used to these languages and there are semantics descriptions already expressed in them. We find this, for example, when Gjøsæter et al. model a language to express Sudokus, the very famous mathematical puzzles [7]. A Sudoku consists of 9x9 cells, some of them containing numbers from 1 to 9, some of them empty. The Sudoku is solved by filling the empty cells with numbers from 1 to 9 in such a way that in each row, column, and each of the nine (non-overlapping) 3x3 sub-areas each number is unique. Besides defining a Sudoku metamodel and some other artefacts, Gjøsæter et al. also describe the step-wise Sudoku solving process in terms of an ASM in an operational style. Though the description was only created for humans to read, an ASM was chosen because it is precise and concise. Fig. 1 shows the metamodel and excerpts from the ASM description.

352

D.A. Sadilek and G. Wachsmuth

Notably, the ASM description uses the class and attribute names from the metamodel. However, we have to translate the ASM into a modelware description language in order to derive an interpreter from it that works on instances of the metamodel. For example, we could translate it to QVT Relations, and EProvide would provide us with a visual interpreter. But using the ASM directly would be less expensive and much more elegant. The aim of this paper is, therefore, to directly use grammarware languages for implementing operational semantics of modelled languages. Structure of the paper. In the following section, we present a pattern for integrating a grammarware description language into modelware. We implemented this pattern in bridges between ASMs, Prolog, and Scheme on the one side and the Eclipse modelling framework EMF on the other side. In Sec. 3, we explain the implementation of the ASM bridges and their usage for the Sudoku example and for an ongoing industrial case study. The paper is concluded in Sect. 4.

2

General Approach

In each description language that should be used to express →, one needs to query and modify models. This is naturally supported by languages from the modelware technological space. Supporting model access in a grammarware description language requires to bridge the language with a modelling framework. A typical bridging approach is to translate models into a representation in the grammarware language [4,14,18] (usually by generating text). This is adequate when one only wants to query models. But it is insufficient if one also wants to modify models because then one needs a back-propagation that merges the changes made in the grammarware language back into the model. Because this merging problem cannot be solved generically without a unique identifier for model objects [2], we opted for another approach. Our bridging approach is to use proxy representations of the model objects in the grammarware language. Any modification on the proxy representation is directly applied to the model, thus, avoiding the merging problem. Although the concrete implementation of such a bridge depends on the description language, we identified three typical facets that each must have: 1. Physical bridge: We must connect the runtime environments of the modelling framework and the description language physically to enable communication. 2. Logical bridge: We must integrate both environments logically to represent models with the data structures provided by the description language. 3. Pragmatical bridge: We must integrate models into the description language pragmatically. This means we can work with models in a similar way as with the description language’s native data structures. In the rest of this section, we discuss these facets in detail.

Using Grammarware Languages to Define Operational Semantics

(d) (b)

Semantics Description

(a)

Semantics Description

353

Semantics Description Proxy Representation

Proxy Representation

Model Description Language Interpreter

Description Language Interpreter

Proxy Repr.

JVM

(c) Semantics

Description

CPU Legend: Interpreted data Runtime environment

Program

Model or its Proxy Repr.

Transformation works on representation

Logical bridge

Fig. 2. Four cases (a – d) of description and model environment nesting

Physical Bridge. Description language and modelling framework can have different runtime environments, which we call description environment and model environment, respectively. In order to execute a semantics description, data from the model environment must be accessible in the description environment. This may require a physical bridge for communication. Whether such a bridge is necessary and how it looks like depends on how the environments are nested. We can distinguish four cases (a – d), which are visualised in Fig. 2. To discuss them, we assume the JVM as model environment. (a) Description and modelling framework share a runtime environment. A physical bridge is unnecessary. Java as a description language, for example, is executed on the JVM. Therefore, semantics descriptions written in Java can work directly with the Java representation of models. (b) Description environment is embedded into the model environment. A physical bridge is necessary, which consists of a foreign function interface (FFI). An FFI of a language allows one to call functions of the environment surrounding the language’s runtime environment. ASMs and Scheme are examples for this case. For these description languages, we use interpreters implemented in Java (SISC [10] and CoreASM [6]). Scheme already provided an appropriate FFI to Java, for ASMs we were able to adapt the interpreter. (c) Model environment is embedded into the description environment. A physical bridge is necessary. Description languages that get compiled to native code, e.g. C++, account for this case. Here, the Java Native Interface must be used as a bridge from the native description environment to the Java model environment.

354

D.A. Sadilek and G. Wachsmuth

(d) Both model and description environments are embedded into a surrounding runtime environment. A physical bridge is necessary. In this case, a natively implemented interpreter executes the semantics descriptions. For example, we use a native interpreter for Prolog (SWI-Prolog [20]). If both description environment and model environment provide FFIs, we can use them to implement the bridge. Logical Bridge. In each description language, the language engineer works on a model representation that uses the data structures available in that language (Fig. 2). A logical bridge is needed to map this representation to the representation provided by the modelling framework. In a dedicated model-transformation language, such as QVT Relations, the native data structures are already models. In contrast, for Java, for example, an object-oriented API for model access is necessary. It is already provided by the modelling framework EMF we use. In other description languages, model representation requires substantial work. This is especially the case with languages whose native data structures are designed for holding trees and cannot directly hold graphs. Models, which are typically graphs, would have to be encoded as trees with additional symbol tables. Fortunately, the three grammarware languages for which we implemented bridges fall not into this category. Pragmatical Bridge. One motivation to support several description languages is to allow a language engineer to select a language he is familiar with. Therefore, it is important to not only provide the language engineer with some interface for working with models in his language of choice but to provide him with one that integrates naturally into the language. There should be a pragmatical bridge that allows the language engineer to query and modify models in a programming style that ideally is indistinguishable from the style used to work with the language’s native data structures.

3

Integrating ASMs into Modelware

In the last section, we identified the three facets of each grammarwaremodelware-bridge. In this section, we explain their implementation for an ASMEMF-bridge. Physical Bridge. To execute ASMs, we use CoreASM [6], a Java-based interpreter (case b). For CoreASM, there is already a plugin available that provides data types to hold references to Java objects and an FFI to call Java methods. However, from a pragmatical point of view, this interface is not optimal because querying and modifying the model must be encoded in uses of the FFI. Furthermore, CoreASM does not provide metaprogramming facilities that would enable us to generate a better interface at runtime. Therefore, we decided not to use the existing FFI but to extend the CoreASM interpreter. We added additional interpreter classes that adapt existing ASM concepts to represent models as described in the following.

Using Grammarware Languages to Define Operational Semantics

355

Logical Bridge. In ASMs, data is represented as an algebraic structure, or algebra for short, that conforms to a certain signature. An algebra consists of a set of all its elements, called the superuniverse, and a number of functions. The functions can denote special elements (0-ary functions), the membership of elements to universes/domains (1-ary characteristic functions), or relations between elements (n-ary functions). A signature defines a vocabulary of function and universe names. To “program” an ASM, one writes guarded update rules that update the functions. For example, one may use a 0-ary function as an analogue to a variable in traditional programming. ASM execution is performed step-wise according to the rules of the ASM. In each step, the algebra is updated. (That is why ASMs were formerly called “evolving algebras”.) To use ASMs for describing operational semantics, we must represent models as algebras. For this, we derive an ASM signature from the metamodel: for each class in the metamodel, a universe name is defined; for class features (attributes and associations), function signatures are defined. Instances of metamodel classes can then be represented as universe elements; feature values are represented with function values. Pragmatical Bridge. The structures used in object-oriented modelling and in ASMs are surprisingly similar. In modelling, we deal with models that contain objects. A model conforms to its metamodel. An object is an element of the instance set of its class, and it can be related to other objects via links and attribute values that are instances of the features declared by its class. Analogue in ASMs, we deal with algebras that contain abstract entities. An algebra conforms to its signature. An abstract entity is an element of a universe with a universe symbol, and it can be related to other entities via functions with function signatures. Because of these structural similarities, we are able to map the modelling concepts directly to ASM concepts and, thus, to provide a very natural model representation in ASMs. Using the ASM-EMF Bridge for the Sudoku Example. With the ASMEMF bridge, we are able to copy the ASM description of the Sudoku solving semantics from [7] and execute it. We only have to make some minor syntactic modifications. For example, we must replace the dot-notation for accessing attributes with a function notation, e.g., we change “c.iCellValue” to “iCellValue(c)”. Furthermore, our implementation does not support the delete primitive, so we replace “delete c.iCellValue from cc.iPossibleCellValues” with “iPossibleValues(cc, iCellValue(c)) := false”. With an additional editor for Sudoku models, we can now create Sudokus and solve them step-wise. Using the ASM-EMF bridge, the ASM works directly on a model representation. This allows us to animate a Sudoku’s solving process using the Sudoku model editor.1 Industrial Case Study. We use the ASM bridge in an ongoing industrial case study conducted in co-operation with the Fraunhofer FOKUS. The goal of this 1

Full source code of the Sudoku example is available under http://eprovide.svn.sf.net/svnroot/eprovide/trunk/no.uia.sudoku*

356

D.A. Sadilek and G. Wachsmuth

case study is to develop a prototypical interpreter for Continous TTCN-3 [17], a proposed extension of the test specification language TTCN-3. Continous TTCN3 extends TTCN-3 with concepts for testing embedded control systems, e.g., control systems in cars. To create a prototypical interpreter, we create a metamodel for Continuous TTCN-3 and describe its operational semantics with an ASM. Presenting the case study in detail is out of scope of this paper. Therefore, we just report about the necessary work for the first prototype: The metamodel we created consisted of 17 classes. We could keep the metamodel so small because we modelled expressions as strings. Normally, expressions account for the most classes in a language’s metamodel. The ASMs that describe the semantics were (without empty lines or comments) around 130 lines with 650 tokens. We could also keep the ASMs quite small because we delegated the interpretation of the string expressions to an external (stub) interpreter. Creating the metamodel and the ASMs took us one day. We spent another day of work for creating (with a generative technique [16]) a textual editor for Continuous TTCN-3 instances that could also visualise the runtime states. This means, the presented approach allowed us to create and animatedly execute the first Continous TTCN-3 example after just two days.

4

Conclusion

Contribution. We described a general pattern for the integration of grammarware languages into modelware based on a physical, a logical, and a pragmatical bridge. We demonstrated the integration of ASMs with the modelling framework EMF. We implemented our approach for ASMs, Prolog, and Scheme within EProvide 2.02 , an extensible, unified Eclipse-based framework for describing and executing operational semantics of modelled languages. EProvide 2.0 supports operational semantics description languages from both modelware and grammarware. Thus, it offers language engineers the flexibility to choose the description language that suits their skills and requirements best. Languages like ASMs, Prolog, and Scheme are well-established in grammar-based language engineering with a vast body of knowledge. Language engineers familiar with these languages are now able to use their knowledge in a model-driven language engineering process—exploiting the available tool support. Related Work. There are many description languages and corresponding tools concerning operational semantics in the modelware space. We discuss these approaches in [19,15]. In [15], we additionally discuss related work for prototyping visual interpreters. Related work about bridges between grammarware and modelware can be classified into approaches that use formalism mapping, transformations, and proxy representations. The idea of formalism mapping is to bridge the formalisms underlying grammarware and modelware, i.e. context-free grammars and metamodels. In [3], Alanen and Porres provide a bidirectional mapping between EBNF grammars and 2

E clipse plugin for PROtotyping V isual I nterpreters and DE buggers; available at http://eprovide.sf.net

Using Grammarware Languages to Define Operational Semantics

357

MOF metamodels. Additionally, they suggest a corresponding bidirectional mapping between syntax trees and models. A similar approach can be found in [21]. Mapping approaches pose restrictions on the metamodels, e.g., multi-valued features must be ordered and non-compositional associations are forbidden, which means one can handle only trees, not graphs. In contrast, we do not restrict the expressiveness of metamodels because we keep models in the modelware technological space and provide access to them in the grammarware technological space. The most common way of bridging is to use transformations. For example, this is the approach of MoMaT [18], with which Prolog can be used for model querying. It is also done for describing the semantics of modelled languages with ASMs, e.g., semantic anchoring by Chen et al. [4] and an extension of AMMA by Di Ruscio et al. [14]. These approaches transform models (language instances) to a grammarware (Prolog or ASM) representation, but they do not solve the back-propagation merging problem. With our approach of proxy representations, we circumvent this problem. There are also bridges that use proxy representations like we do. Similarly to our approach, in the Moses tool suite [5] one can write an ASM that works directly on an attributed graph (which plays the role of a model). But Moses does only provide a generic and not a typed model interface: It misses a pragmatical bridge. For example, to access a list of a Sudoku’s cells, one would write “V(G, ‘Cell’)” in Moses (“V” stands for vertices and “G” for the whole graph) and simply “Cell” with our approach. Future Work. As we mentioned, there are several approaches to describe operational semantics. Most of them come with their own description languages, which could be integrated into EProvide 2.0. Due to their formal foundations and popularity, we are particularly concerned with integrating graph transformations. Furthermore, we are working on testing the equivalence of operational semantics. Different descriptions work on the same type of data structure, namely EMF models. Thus, we have a comparable basis for doing a bisimulation of some example language instance with different semantics descriptions. To backtrack over runtime states of non-deterministic languages, we could use EProvide 2.0’s step-back support [8]. Acknowledgement. We thank Markus Scheidgen, Falko Theisselmann, Stephan Weißleder, and the reviewers for comments on preliminary versions of this paper. Terje Gjøsæter and Andreas Prinz explained us their Sudoku example. J¨ urgen Großmann explained us Continous TTCN-3. Roozbeh Farahbod helped us integrating the CoreASM interpreter. This work is supported by grants from the Deutsche Forschungsgemeinschaft, Graduiertenkolleg METRIK (GRK 1324/1).

References 1. Schieferdecker, I., Hartman, A. (eds.): ECMDA-FA 2008. LNCS, vol. 5095. Springer, Heidelberg (2008) 2. Alanen, M., Porres, I.: Difference and union of models. In: Stevens, P., Whittle, J., Booch, G. (eds.) UML 2003. LNCS, vol. 2863, pp. 2–17. Springer, Heidelberg (2003)

358

D.A. Sadilek and G. Wachsmuth

3. Alanen, M., Porres, I.: A relation between context-free grammars and Meta Object Facility. Technical report, TUCS (2004) 4. Chen, K., Sztipanovits, J., Abdelwalhed, S., Jackson, E.: Semantic anchoring with model transformations. In: Hartman, A., Kreische, D. (eds.) ECMDA-FA 2005. LNCS, vol. 3748, pp. 115–129. Springer, Heidelberg (2005) 5. Esser, R., Janneck, J.: Moses: A tool suite for visual modeling of discrete-event systems. In: HCC 2001, pp. 272–279. IEEE Computer Society Press, Los Alamitos (2001) 6. Farahbod, R., Gervasi, V., Gl¨ asser, U.: CoreASM: an extensible ASM execution engine. Fundamenta Informaticae 77(1-2), 71–103 (2007) 7. Gjøsæter, T., Isfeldt, I.F., Prinz, A.: Sudoku – a language description case study. In: Gaˇsevi´c, D., L¨ ammel, R., Van Wyk, E. (eds.) SLE 2009. LNCS, vol. 5452, pp. 305–321. Springer, Heidelberg (2009) 8. Hartmann, T., Sadilek, D.A.: Undoing operational steps of domain-specific modeling languages. In: Proceedings of the 8th OOPSLA Workshop on Domain-Specific Modeling (DSM 2008), University of Alabama at Birmingham (2008) 9. Kurtev, I., B´ezivin, J., Aksit, M.: Technological spaces: An initial appraisal. In: CoopIS, DOA 2002 Federated Conferences, Industrial track (2002) 10. Miller, S.G.: SISC: A complete scheme interpreter in Java. Technical report, Indiana University (January 2002) 11. Muller, P., Fleurey, F., J´ez´equel, J.: Weaving executability into object-oriented meta-languages. In: Briand, L.C., Williams, C. (eds.) MoDELS 2005. LNCS, vol. 3713, pp. 264–278. Springer, Heidelberg (2005) 12. Object Management Group. Meta Object Facility 2.0 Query/View/Transformation Specification (2007) 13. Plotkin, G.D.: A Structural Approach to Operational Semantics. Technical Report DAIMI FN-19, University of Aarhus (1981) 14. Ruscio, D.D., Jouault, F., Kurtev, I., B´ezivin, J., Pierantonio, A.: Extending AMMA for Supporting Dynamic Semantics Specifications of DSLs. Technical Report 06.02, LINA Laboratoire D’Informatique De Nantes Atlantique (April 2006) 15. Sadilek, D.A., Wachsmuth, G.: Prototyping visual interpreters and debuggers for domain-specific modelling languages. In: Schieferdecker, I., Hartman, A. (eds.) ECMDA-FA 2008. LNCS, vol. 5095, pp. 63–78. Springer, Heidelberg (2008) 16. Scheidgen, M.: Textual modelling embedded into graphical modelling. In: Schieferdecker, I., Hartman, A. (eds.) ECMDA-FA 2008. LNCS, vol. 5095, pp. 153–168. Springer, Heidelberg (2008) 17. Schieferdecker, I., Großmann, J.: Testing hybrid control systems: an overview on cont. ttcn-3. Int. J. Softw. Tools Technol. Transf. 10(4), 383–400 (2008) 18. St¨ orrle, H.: A prolog-based approach to representing and querying software engineering models. In: Proceedings of the VLL 2007 workshop on Visual Languages and Logic. CEUR-WS, vol. 274, pp. 85–96 (2007) 19. Wachsmuth, G.: Modelling the operational semantics of domain-specific modelling languages. In: L¨ ammel, R., Visser, J., Saraiva, J. (eds.) GTTSE 2007. LNCS, vol. 5235, pp. 506–520. Springer, Heidelberg (2007) 20. Wielemaker, J.: SWI-Prolog 5.6 Reference Manual (2008), http://www.swi-prolog.org 21. Wimmer, M., Kramler, G.: Bridging grammarware and modelware. In: Bruel, J.-M. (ed.) MoDELS 2005. LNCS, vol. 3844, pp. 159–168. Springer, Heidelberg (2006)

Automatic Generation of Integrated Formal Models Corresponding to UML System Models Helen Treharne, Edward Turner1 , Richard F. Paige, and Dimitrios S. Kolovos2 1

Department of Computing, University of Surrey {H.Treharne,Edward.Turner}@surrey.ac.uk 2 Department of Computer Science, University of York {paige,dkolovos}@cs.york.ac.uk

Abstract. This paper presents a complex model transformation that takes two related UML models as input (class diagrams and state machines) and outputs an integrated formal model. The transformation is achieved using the Epsilon model management framework and has involved the definition of new CSP and B metamodels. The target model is a CSP || B specification (a fusion of CSP and B) that can be used to analyse the source UML. Through a comparative study we conclude that the code written using Epsilon is more concise than the equivalent model-to-text transformation achieved using the iUML toolset. Keywords: Model transformation, Epsilon, UML, CSP, B.

1

Introduction

Model transformations are at the heart of Model-Driven Engineering (MDE) [1]. Transformations of various kinds (e.g., model-to-model, model-to-text, updatein-place) support MDE processes, allow incremental elaboration of models, and support interoperability between tools and languages. Model transformation also enables analysis of models, through transformation of system models (e.g., in UML, SysML, or domain-specific languages) to languages more amenable to rigorous analysis. There are powerful formal languages and tools to support rigorous analysis, ranging from finite-state model checkers, to automated or semi-automated theorem provers. Examples of such languages and tools include CSP [2], B [3], and Alloy [4]. However, these languages are not as widely used by engineers as less mathematical languages such as UML or SysML. Model transformation provides systematic mechanisms to bridge the gap between well-known system modelling languages, and languages well suited to formal analysis. We provide a specific application of model transformation to bridge the gap between UML modelling and formal modelling languages. There has been previous work on using model transformation to generate formal models, which we discuss in Section 5. This paper contributes a novel application of model transformation to support formal analysis, and does so M. Oriol and B. Meyer (Eds.): TOOLS EUROPE 2009, LNBIP 33, pp. 357–367, 2009. c Springer-Verlag Berlin Heidelberg 2009 

358

H. Treharne et al.

in a novel way. The application of model transformation is novel because it simultaneously takes two types of UML diagrams as input, and produces formal models in an integrated language as output. The integrated language is a fusion of CSP and B. The context in which this transformation has been developed is also novel: it has been developed in the context of a safety-critical systems development project, where the model transformation that has been developed has to be shown to provide improvements over the traditional means by which formal models have been produced from UML models. Indeed, we are able to show improvements via use of state-of-the-art model transformation languages and tools and we discuss this in more detail in the paper. The model transformation framework that we use is part of Epsilon [5], which is a platform of integrated task-specific model management languages; we use the languages that focus on model-to-model (M2M) transformation (ETL) [6], and model-to-text (M2T) transformation (EGL) [7]. They both build on a common OCL-based model navigation and manipulation language (EOL). We will demonstrate how ETL can be used to support transformations between UML and CSP || B [8] models. To achieve this we define two metamodels; one for CSP and one for B. The reason for having two models, and not one, is that CSP and B are different languages and we wanted to retain their separation at the meta-level. This will mean that the metamodels can be re-used in the future as a basis for other transformations. Separation of concerns is an underlying principle of the CSP || B modelling approach. A CSP || B model comprises a separate CSP specification and a B specification; together they describe the behavioural and data aspects of UML state machines and class diagrams. Their separation enables us to use the existing formal analysis tools of Fdr [9] and ProB [10] when analysing the CSP and B models respectively. We demonstrate how EGL is used to generate machine readable CSP and B scripts that can be executed in their respective analysis tools. Our previous work [11] has targeted transformations specifically from Executable UML (xUML) [12], which is currently our industrial collaborator’s favoured UML modelling language. The code generator we previously implemented was developed using the iUML tool-set [13], which uses a single, low-level language called ASL [14]. ASL is used to define all model-management tasks, including UML model actions. We defined a suite of M2T transformations, and the output was CSP || B. In this paper, we produce a similar style of CSP || B using transformations in Epsilon (derived from our previous work) and we highlight that our new approach requires a significantly reduced amount of code. The paper continues in Section 2 with an overview of the CSP || B approach; the transformations and metamodels are presented in Section 3. Our experiences of using Epsilon is summarised in Section 4 and we conclude with a comparison of related work and future directions.

2

CSP || B

CSP  B is a formal methodology that combines the event-based CSP notation [2] with the state-based B notation [3] to aid the specification of systems

Automatic Generation of Integrated Formal Models

359

containing both complex flows of control and structured data. Thus, CSP || B is suited to specifying UML state machines and data-rich class diagrams. CSP process expressions can be constructed from a range of operators. Those used in this paper include external choice (2), local definitions (let . . . within, enabling the definition of processes of processes) and event sequencing, denoted prefix (→). The set of all events for a process is denoted its alphabet. B specifications are structured using machines. Each machine contains some variables describing the objects of interest, and operations to manipulate the state. The machine invariant declares state predicates, and specifies what must be preserved by the execution of operations. Functions and relations are used to model complex state. A B operation takes the form PRE P THEN S END where P is a predicate and S represents the statements that update variables. In CSP  B, the events of a CSP process trigger operation calls of a B machine, and we therefore refer to CSP processes as controllers. Structured events are used to pass values between a controller and a B machine, in both directions. For example, the event ev !x ?y, denotes a communication over the channel, ev , which outputs (!) x and binds y to an input value (?), and this event corresponds to a B operation ev that accepts x as input, and outputs y 1 . We combine collections of process/machine pairs using the architecture identified in [15], where collections of controllers represent the behaviour of instances of classes, and the machines record and update the instances’ attributes.

3

Generating CSP||B from UML

The generation of the formal model comprises two main phases, and has required the definition of new CSP and B metamodels. In the first phase, we use ETL to perform an M2M transformation from the source UML model into one instance of both the CSP model and the B model. These newly created models provide the source models for an EGL M2T transformation, which traverses the meta-classes and prints them appropriately to both a CSP file and a B file. 3.1

Transformations to CSP

The abstract syntax for the CSP scripts is given in Fig. 1. As a basis for its development, we referred to the compact CSP metamodel by Bisztray et al. [16], who make use of it to verify behavioural equivalences between refactored UML systems comprising component, structure and activity diagrams. Indeed, it is possible to see similarities between the two metamodels, e.g., the Event class. There are however significant differences. A notable difference is that they model CSP conforming to Hoare’s definition of the language [2], which states that every process P has its own, intrinsic alphabet, αP . Moreover, only a single parallel operator is defined where P  Q is a process comprising the synchronous execution of any event, ev ∈ αP ∩ αQ , and the concurrent execution of all other 1

It is not necessary for a corresponding CSP event/B operation pair to include parameters.

360

H. Treharne et al.

Fig. 1. CSP metamodel

events in P and Q . In terms of machine readable CSP, called CSP-M, which is the syntax required by Fdr, the authors model Hoare’s parallel using the interface parallel operator: P [| αP ∩ αQ |]Q . We have chosen to model directly a wide range of CSP-M notions not represented in [17], to allow for flexibility in our current/future transformations. Therefore, we also support the definition of CSP alphabetised parallel, of the form P [αP || αQ ]Q , which also exhibits synchronous execution of any event, ev ∈ αP ∩ αQ , but which refuses the execution of events not specified in the alphabets. In fact, our experience is that alphabetised parallel improves the clarity of CSP-M, making it more simple for users to experiment and construct new CSP refinements assertions. Another major difference is that we choose to represent the data that can reside in CSP, e.g., using the classes Datatype and Set. Thus, instead of having an Event class that represents all notions of events, our definition is more finegrained. We model all event parameters and their types, e.g., the event ev !a?b corresponds to an instance of Event for the Channel called ev , which has two EventParameters, a and b. Modelling this aspect is especially important for our use which requires communication of CSP events and parameters and corresponding B machine operations parameters.

Automatic Generation of Integrated Formal Models

Fig. 2. File class diagram

361

Fig. 3. File state chart

File SCTRL(self ) = let File Created ENTRY = File Created STATE File Created STATE = remove!self ? !File initialisedSignal → File Idle ENTRY File Idle ENTRY = File Idle STATE File Idle STATE = remove!self ? !File writeSignal → File Writing ENTRY 2 remove!self ? !File selfCheckSignal → File Idle ENTRY File Writing ENTRY = File Writing STATE File Writing STATE = . . . within File Created STATE Fig. 4. Generated CSP for the File state chart

Note that Fig. 1 shows only the main classes and association labels of the metamodel necessary for the purposes of this paper. There also exist classes that model all variations of the Choice operator (internal, external and asymmetric choice), namespaces, process interrupts, Skip, Stop, sequential composition, process assertions, renaming and the derivatives of Concurrent processes: interleaved/indexed and alphabetised/interface parallel. Figures 2 and 3 provide a source model for our transformations. The class diagram contains a File class with the attributes inUse and size, and the state machine represents the states of this file: Created, Idle and Writing. Fig. 4 presents part of the CSP script generated by the ETL and EGL transformation for the File class and state chart. The format of this CSP follows the style we used in our xUML-to-CSP || B code generator [11]. The key components of the script include definitions for modelling class instances, and channels that enable attribute manipulations/queries, operation calls, object creations/deletions and signal processing. The dynamic behaviour of each instance of each class is defined through a repeatable set of process equations. The File SCTRL(self ) process represents the behaviour of the File state chart, where the self parameter is a File instance handle. This localised process (a let . . . within CSP construct) comprises a pair of process equations for each state. The first process, e.g., File Idle ENTRY , models the state’s entry actions, none of which are defined in our toy example. The second process captures the processing of signals that have arrived at a state, and its subsequent transition(s) to successor

362

H. Treharne et al.

states. For example, File Idle STATE includes the deterministic choice (2) of two events modelling the removal of two signals from the objects’ conceptual signal queue [12]. These events are followed by the entry state process of their respective successor states. Finally, observe that File SCTRL(self ) begins in the File Created STATE process equation, as specified by the terminal, ‘within’ clause; this information is derived from the initial state of the state chart. Listing 1 presents the core ETL rule used in our transformation workflow that produces CSP notation such as that given in Fig. 4. The StateMachine2CTRL Process rule is responsible for transforming a UML state machine into a localised (let . . . within) process.This primarily involves the creation of new instances of ProcessID, ProcessAssignment and ProcessParameterList. In lines 6 to 8, we link together the ProcessAssignment and ProcessID, and the ProcessID and ProcessParameterList. Line 9 sets the name of the process, e.g., File SCTRL, and lines 10 through to 13 add the controller process parameter ‘self’. We create a localised process (line 14), and on line 15, we iterate through each state of the state machine and apply the rule Sm2ProcPair. Each application of Sm2ProcPair returns a pair of ProcessAssignments, which are added to the localised process; these new instances represent the ENTRY and STATE processes described previously. Finally, we compute the initial state of the class diagram using the getStartState operation and we link the new localised process to the original ProcessAssignment, pa. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

17 18 19 20

rule StateMachine2CTRLProcess transform sm : StateMachine to p : CSP!ProcessID, pa : CSP!ProcessAssignment, ppl : CSP!ProcessParameterList { pa.procID := p; ppl.size := 1; p.paramList := ppl; p.name := sm.name+‘_SCTRL’; var procParam : new CSP!ProcessParameter; procParam.name := ‘self’; ppl.first := procParam; ppl.item.add(procParam); var root : new CSP!LocalisedProcess; for(state in StateMachine!State.all) { root.process := root.process + state.equivalents(‘Sm2ProcPair’). select(st | st.isKindOf(CSP!ProcessAssignment)); } root.first := getStartState(sm); pa.procExpr := root; } Listing 1. CSP transformation rule

Automatic Generation of Integrated Formal Models

363

Fig. 5. B metamodel

3.2

Transformations to B

The abstract syntax for the B we use is depicted in Fig. 5. The main classes defining entities of a B machine are Machine, Operation, Variable and Expression. The Expression class is used as a basis from which specific B expressions are derived, such as Ifs, and Actions (e.g., assignments). To our knowledge, this metamodel is a novel contribution to the existing literature. Fig. 6 presents the B machine generated by the ETL and EGL transformation for the File class. We represent the set of all possible instance handles of the class by the B set, fileIH , and we define a subset of this, fileObj , to capture the file instances already created in the system. Functions are used to represent attributes, e.g., size ∈ fileObj → INTEGER, and we also generate simple operations to query or modify their values. Operations are also defined to support the dynamic creation and deletion of objects, e.g., file create. Other user defined operations of a class are transformed to skeleton B operations that specify the typing of input variables. We also create a single B machine to store the relationships of the class diagram; however, Fig. 5 does not include any. Listing 2 presents the core ETL rule used in our transformation workflow that produces B notation, such as that given in Fig. 6. The Class2Machine rule is responsible for transforming a UML class to a newly created B machine, Machine instance. First, a guard is specified to constrain the application of the rule to only non-UML (primitive) classes. The procedure sets the name of the machine to that of the source class, File, and creates for it a Set representing all instances of the class. The operation on line 7 getInstanceHandleSetName() is used to format the name of the set correctly; fileIH. Lines 9 to 11 create the variable representing the active objects in the system, where getActiveInstanceHandleSetName() is used to set its name to fileObj. A Predicate is then created to define the type of fileObj, on line 12. This predicate is set as a machine invariant, and we

364

H. Treharne et al.

MACHINE File SEES Bool TYPE SETS fileIH = { fifoIH0 , fifoIH1 } VARIABLES fileObj , inUse , size INVARIANT fileObj ⊆ fileIH ∧ inUse ∈ fileObj → BOOL size ∈ fileObj → INTEGER INITIALISATION fileObj := ∅  inUse := ∅  size := ∅

OPERATIONS  file create ( ih ) = PRE ih ∈ fileIH ∧ ih ∈ llObj THEN fileObj := fileObj ∪ { ih }  inUse ( ih ) := false END val ←− file get inUse ( ih ) =  PRE ih ∈ llObj THEN val := inUse ( ih ) END file set inUse ( ih , val ) =  PRE ih ∈ llObj ∧ val ∈ BOOL THEN inUse ( ih ) := val END /* Also defined: file delete, file get size, file set size */ END

; ; ;

Fig. 6. Generated B machine for the File class

also reference it from the variable to assert that this variable indeed has a type definition defined. We call the attribute2variable operation so that we can identify for each attribute of the class (line 16) a new Variable and Predicate specifying its type. We also create a ‘get’ and ‘set’ operation for querying and manipulating the value of the attribute. 1 2 3 4 5 6 7 8 9 10 11 12 13

14 15 16 17 18 19 20 21 22 23

rule Class2Machine transform c : SourceUML!Class to mch : B!Machine { guard : ensureNonUmlClass(c) mch.name := c.name; var ihSet : new B!‘‘Set’’; ihSet.name := mch.name.getInstanceHandleSetName(); mch.sets.add(ihSet); var activeIHSet : new B!Variable; activeIHSet.name := mch.name.getActiveInstanceHandleSetName(); mch.variable.add(activeIHSet); var activeIHSet_type : new B!Predicate; activeIHSet_type.expression := activeIHSet.name+’