Lecture Notes in Computer Science Commenced Publication in 1973 Founding and Former Series Editors: Gerhard Goos, Juris Hartmanis, and Jan van Leeuwen
Editorial Board David Hutchison Lancaster University, UK Takeo Kanade Carnegie Mellon University, Pittsburgh, PA, USA Josef Kittler University of Surrey, Guildford, UK Jon M. Kleinberg Cornell University, Ithaca, NY, USA Alfred Kobsa University of California, Irvine, CA, USA Friedemann Mattern ETH Zurich, Switzerland John C. Mitchell Stanford University, CA, USA Moni Naor Weizmann Institute of Science, Rehovot, Israel Oscar Nierstrasz University of Bern, Switzerland C. Pandu Rangan Indian Institute of Technology, Madras, India Bernhard Steffen TU Dortmund University, Germany Madhu Sudan Microsoft Research, Cambridge, MA, USA Demetri Terzopoulos University of California, Los Angeles, CA, USA Doug Tygar University of California, Berkeley, CA, USA Gerhard Weikum Max Planck Institute for Informatics, Saarbruecken, Germany
6520
Roland Kaschek Lois Delcambre (Eds.)
The Evolution of Conceptual Modeling From a Historical Perspective towards the Future of Conceptual Modeling
13
Volume Editors Roland Kaschek Gymnasium Gerresheim Am Poth 60, 40625 Düsseldorf, Germany E-mail:
[email protected] Lois Delcambre Portland State University Maseeh College of Engineering and Computer Science Computer Science Department P.O. Box 751, Portland, OR, 97207-0751, USA E-mail:
[email protected] Library of Congress Control Number: 2010941310 CR Subject Classification (1998): H.2-4, I.2.3-4, F.4.1, H.1, J.1, D.2, C.2 LNCS Sublibrary: SL 3 – Information Systems and Application, incl. Internet/Web and HCI ISSN ISBN-10 ISBN-13
0302-9743 3-642-17504-X Springer Berlin Heidelberg New York 978-3-642-17504-6 Springer Berlin Heidelberg New York
This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. springer.com © Springer-Verlag Berlin Heidelberg 2011 Printed in Germany Typesetting: Camera-ready by author, data conversion by Scientific Publishing Services, Chennai, India Printed on acid-free paper 06/3180
Preface
The origin of this volume is the Dagstuhl Seminar on The Evolution of Conceptual Modeling (Schloss Dagstuhl, April 2008) organized by Roland Kaschek with Lois Delcambre and Heinrich C. Mayr as co-organizers. Their motivation for organizing the seminar was that they felt the field of conceptual modeling, though clearly very important for computing, is not very well organized, developed, or recognized. Although a number of researchers, instructors and teachers have a serious interest in conceptual modeling and often include conceptual modeling topics in a wide range of courses, in a typical curriculum, conceptual modeling may only be covered in a brief or peripheral manner, e.g., as part of another course. The objectives for the seminar were: (1) to look into conceptual modeling from a historical perspective with a view toward the future of conceptual modeling and (2) to achieve a better understanding of conceptual modeling issues in several different domains of discourse using a perspective that goes beyond individual (modeling) projects. Note that participation in a Dagstuhl Seminar is by invitation only and permission to conduct a Dagstuhl Seminar is granted by a scientific advisory board with the University of Saarbr¨ ucken, Germany, based on the seminar proposal as well as the list of invitees. The seminar, attended by 33 experts from the field of conceptual modeling, consisted primarily of the following presentations:1 • Valeria De Antonellis: Conceptual Modeling for Service-Oriented, Collaborative Systems • Karen C. Davis: Data Warehousing Modeling and Its Evolution • Lois M. L. Delcambre: Multiple Conceptual Models in Superimposed Information Systems • David W. Embley: Turning the Web into a Web of Knowledge • Nicola Guarino: Ontological Foundations of Conceptual Modeling • Sven Hartmann: Conceptual Modeling and Natural Language Grammar • Brian Henderson-Sellers: Meta-modeling • Wolfgang Hesse: Evolution in Software Engineering from Using Models to Ontologies • Dietmar Jannach: Error Recovery in Business Processes • Klaus P. Jantke: Conceptual Modeling for Computer Games • Roland Kaschek: Sociological Turn in Conceptual Modeling • Christian Kop: Templates and Glossaries in Conceptual Modeling • Gunther Kreuzberger: Entertainment Engineering for Digital Games 1
Please note that some of these speakers were presenting cooperatively produced work. We were not able to invite all authors to attend the seminar.
VI
Preface
• Heinrich C. Mayr: Extreme Non-programming • Oscar Pastor: Conceptual Modeling Techniques Used in Human Genome Engineering • Klaus-Dieter Schewe: Specifying Data-Centric Services with Abstract State Machines • Michael Schrefl: Ontological Multi-level Modeling • Vladimir A. Shekhovtsov: Using Simulation for Quality Requirements Elicitation • Stefano Spaccapietra: Modeling Scientific Hypotheses • Markus Stumptner: Modeling Web Service Composition • Yuzuru Tanaka: A Memetics View on Web Evolution • Bernhard Thalheim: A Token Model of Process Semantics • Susan D. Urban: A Historical Account of Semantic and Current Data Models • Gottfried Vossen and Gunnar Thies: Issues of Conceptual Modeling for Web 2.0 • Tatyana Welzer: Evaluation of Conceptual Modeling • Roel Wieringa: Conceptual Modeling of Social Domains as Opposed to Physical Domains Near the end of the seminar, the participants decided to produce this edited volume, based in large part on presentations from the seminar. With this volume we wish to contribute to increasing the role of conceptual modeling as a discipline in research, teaching, and practice. Our process was as follows. The participants2 in the seminar were invited to contribute a chapter to this volume. In addition to the quality assurance provided by the Dagstuhl process (resulting in the initial invitation of experts), the submitted papers were carefully reviewed in a peer review process, in the first round. In the second round, we, the editors, carefully reviewed all resubmitted papers. We are very pleased to present 14 chapters in this volume, offering significant coverage of the topics from the seminar. The book is organized into three sections. The first, “The Evolution of Conceptual Modeling Techniques,” includes six papers, namely: a discussion of real-world semantics of conceptual models (Roel Weiringa), a survey of the use of templates in domain modeling (Christian Kop and Heinrich C. Mayr), an analysis of the use of integrity constraints in conceptual database models (Bernhard Thalheim), a survey of methods for modeling abstraction at multiple levels (Bernd Neumayr, Michael Schrefl, and Bernhard Thalheim), a discussion of multi-level conceptual models (Brian Henderson-Sellers), and an account of the evolution of modeling software quality in conceptual models (Vladimir A. Shekhovtsov).
2
In a few cases, colleagues who were invited to participate in the seminar but were not able to attend were invited to contribute chapters to this volume. Also, based on their role as editors, Roland Kaschek and Lois M. L. Delcambre decided not to submit chapters to be considered for the book.
Preface
VII
The second section,“The Extension of Conceptual Modeling” includes the following papers: an account of how Web pages can be (nearly automatically) processed to produce a “Web of knowledge” (David W. Embley, Stephen W. Liddle, and Cui Tao), a description of a technique to produce “semantic overlays” in a peer-to-peer system (Devis Bianchini, Valeria De Antonellis, and Michele Melchiori), a survey and new proposal for dynamic Web service composition using conceptual modeling (Georg Grossmann, Rajesh Thiagarajan, Michael Schrefl, and Markus Stumptner), the presentation of a new approach to dynamic constraint checking, exception handling, and recovery in contemporary distributed applications using rules (Susan D. Urban, Le Gao, Rajiv Shrestha, and Andrew Courter), a discussion of past, current, and future approaches to exception handling in Web service processes (Dietmar Jannach, and Alexander Gut), and the presentation of a framework for model-based tool integration based on conceptual modeling techniques (Gerti Kappel, Manuel Wimmer, Werner Retschitzegger, and Wieland Schwinger). The third and final section of this volume, “New Directions for Conceptual Modeling,” includes two papers: the presentation of a new conceptual data model for scientific models and hypotheses (Fabio Porto and Stefano Spaccapietra) and the description and suggestions for further enhancement of the application of conceptual modeling techniques, namely, model-based software engineering, to new applications – with a particular focus on the interpretation of the human genome (Oscar Pastor, Ana M. Levin, Matilde Celma, Juan Carlos Casamayor, Aremy Virrueta, and Luis E. Eraso).
October 2010
Lois M.L. Delcambre Roland Kaschek
Introduction Issues of a Conceptual Modeling Agenda3
This introduction is intended to present to this volume’s audience some of the reasoning that led to organizing the Dagstuhl seminar this volume originates from. It aims at providing general insights into the field of conceptual modeling that are not necessarily discussed in any of this volume’s chapters but nevertheless aid in conceptualizing the inner structure and coherence of the field. The evolution of mankind can be considered as the evolution of how human beings know, what they know, what they use their knowledge for, and the knowledge media they use. Conceptual modeling is a rather recent approach to creating knowledge. It has emerged in response to the computer revolution which started in the middle of the twentieth century. Computers in the meantime have turned out to be a major new knowledge media. Conceptual modeling is an answer to the difficulties experienced throughout the development of computer applications and aims at creating an effective, reasonably priced, and sharable knowledge about using computers in business. It turns out, however, that conceptual modeling has the potential to exceed the boundaries of business and computer use. Humans share knowledge by developing a culture, i.e., a system of codified behaviors such that knowledge can be constituted as a system of meaningful differences. Knowledge can then be structured on a micro-level according to logical schemes such as “A is true” and “if A is true then B is true” and the like. On higher levels, humans structure knowledge into models and more or less basic and more or less confirmed theories. We do so because that way checking knowledge is reduced to checking the logic it arises from, the premises of that logic, and intended use the knowledge. Structuring knowledge, moreover, simplifies the process of knowledge acquisition and sharing as the knowledge may be broken down into small, manageable pieces. The knowledge acquired and maintained that way is not necessarily the only kind of knowledge that needs to be or can be considered. Other kinds of knowledge may be addressed by what we call intuition or revelation. The mechanisms or techniques to share them may be different from the mechanisms that play a role for modeling. Modeling is the use of a substitute rather than the original, i.e., the entity one is actually interested in. Investigating a substitute rather than an original may be a good thing to do because the original is inaccessible, unavailable or too expensive for study, or nonexistent. The structure of the modeling situation is peculiar because it is often difficult to establish the model’s suitability for the intended purpose. In particular if the original is natural and the model is an
3
I thank Lois Delcambre for helping me to improve the readability of this text.
X
Preface
artifact then an analytical justification of the model’s suitability might be out of reach. Models may even be used if an analytical justification is known to be nonexistent. What is needed is the empirical evidence (or in some cases only the prospect of it) of the model’s suitability. In monetary terms, the justification of a model’s suitability may be expressed as: using the model is cheaper than not using it. It goes without saying that the related cost estimations are by no means a simple task. Since modeling is the use of a substitute rather than the original it creates two classes of knowledge: generic and case-specific. The generic knowledge is encoded in models and the case-specific knowledge is used for deciding which model to use, when and how. Throughout the young history of computing the importance of models, their creation and use has continuously increased. This is why in this book we discuss a number of more or less fundamental aspects of models and modeling. In this book, however, we are not concerned with all kinds of modeling. Rather, we ignore bodily and physical modeling. We focus on what has come to be known as conceptual modeling. Bodily models are material objects used to study aspects of the original. For example, bodily models have been used intensively in the industry constructing ships, trains, cars, aircraft, or spacecraft. This kind of modeling also has a long history in architecture. For bodily models, it is often important that they are significantly smaller than their original, much easier to deal with, much cheaper to produce, retire or similar. On the other hand, physical models are models that are encoded in a language with key terms that refer to characteristics of a given computer or computational device such as a PDP/11, an IBM 360, 390, a Wintel PC, a Mac or similar. The difference between physical and conceptual modeling is the language used for encoding the model. While in physical modeling a low-level computer jargon is used, in conceptual modeling a high-level jargon is used. That high-level jargon tends to focus on the discourse domain to which the problem to be solved with a computer application belongs rather than to the computer itself. Conceptual modeling thus brings about a situation in which humans can use computers as a tool in problem-solving language games4 without having to understand computers. Usually one can rely on IT professionals to keep the computers functioning reliably and with satisfactory performance. Human knowledge is commonly classified as being either procedural (“know how”) or declarative (“know what”). As is well known, the distinction between these two kinds of knowledge is a fundamental one. For example, the largest part of any human individual’s knowledge is procedural, and acquiring proce-
4
With the use of Wittgenstein’s term, language game, I want to recall two things: First, when people say something, what they mean with their utterance usually strongly depends on what they do while talking, i.e., the context of the actual utterance. Second, when an adult talks to someone else they are usually not entirely free to say whatever they might want. Rather, there are types of situations in which certain utterances are expected of them and others are considered as inappropriate. To me this implies that in many situations a pattern of codified behavior applies that at least partially gives meaning to what people say.
Preface
XI
dural knowledge on average takes much less time than acquiring declarative knowledge. Conceptual modeling links declarative knowledge and procedural knowledge. From a business point of view, conceptual modeling thus is a key activity because it enables working out the formalized declarative knowledge that is required for implementing the computer applications that are needed to back up the current or intended business practices. Obviously the link between computer applications and business practices can also be employed the opposite way: technology can be used to enable new business practices. Today’s computing landscape is very different from the computing landscape of the 1940s. Back then, mathematicians, physicists, and engineers were the most prominent computer users. Moreover, the relative importance that is attributed to conceptual modeling nowadays was not attributed to it at the begining of the computing age. The metaphor of “evolution” is applicable to computing as a whole and to software development in particular because a trend can be perceived, a trend toward more abstract and hardware-independent descriptions of computational processes and structures. That trend, however, is not the result of an individual agent’s decision. Rather it results from all those forces impacting the whole area of computing and in particular from its status as a major new branch of business. In the middle of the twentieth century when modern computers and computer science entered the stage, computers actually were already quite old. Charles Babbage had designed mechanical computers between 1840 and 1850. In doing that he extended the work of the French loom maker Joseph-Marie Jacquard, who in 1805 presented a program-controlled loom. Other sources of mechanical computing seem to date back to much earlier in history as is evidenced by the early European mechanical calculators such as the 1623 calculator built by Wilhelm Schickard and the Antikythera mechanism, the construction of which has been dated back to about 100 B.C. Construction and practical use of analog computers is commonly dated back much further in history. Examples of such early computers are the Goseck observatory (about 5000 B.C.), Stonehenge (about 3000 B.C.) and the Nebra Sky Disk (about 2000 B.C.). While in the nineteenth century the driving forces might not have been sufficient for triggering a largescale interest in computers and computing, the situation was quite different in the middle of the twentieth century. The war on Nazi Germany, the unfolding cold war, and the capability to make money using computers changed the situation. At the end of the 1970s the observation was made that in software engineering, database engineering, and artificial intelligence, differences in the development of computer applications were beginning to show that were less a consequence of the subject than of the arbitrary differences between these fields. It was then natural to try to overcome aspects of overspecialization by focusing on what was common to these sub-disciplines of computing, i.e., conceptual modeling. More recently further stimuli of conceptual modeling have started impacting the field: First, successful software systems have achieved a lifetime beyond the lifetime of their hardware or software platform and thus require to have
XII
Preface
specifications independent of these platforms. Second, the variety of problems that can be solved without paying attention to characteristics of the basic hardware and software has increased significantly and so high-level specifications make more sense. Third, new branches of computing have emerged such as gaming, social networks, simulators, or Web systems. It is desirable for these new branches to exploit application specification expertise from the more traditional branches of computing. From an abstract point of view, during conceptual modeling such as ER, UML, system, Petri net, state chart modeling or similar, a network of concepts is worked out. The concepts in this network mutually define each other. This means that the related concepts together create a language that constitutes a domain of discourse and that within that discourse each of the concepts has a unique role. In this way conceptual modeling is very similar to mathematics. Three interesting differences between the two, however, should be stated. First, in mathematics, there is a much stricter adherence to a given standard modeling language (i.e., the Zermelo-Fraenkel axiomatic system explicating the set concept) than in conceptual modeling, where we find in use many more or less different basic modeling languages. Second, in conceptual modeling, the precise description of computational processes and data structures is enforced by the need to establish program control of the computer, i.e., the computational agent. In mathematics, the computational agent was initially and still is a human individual. Programmed control of that agent has not yet achieved a mature state. In computing, moreover, some of the computational processes and structures have turned out to be useful on levels abstracting from the hardware that executes programs. Third, in mathematics, the basic modeling language is formally defined (by the axiomatic system). In conceptual modeling, however, this is not the case. In computing, formalization is important but is often considered later, namely, when specified solutions need to be implemented on a computer. It should be noted finally that the idea of describing “real-world” domains as nets of concepts was put forward as early as 1928 by Rudolf Carnap in his book Der logische Aufbau der Welt and thus has a relevant prehistory outside computing. Probably the most important point about conceptual models is that they are utterances specifically tailored for a particular audience, a given purpose, a particular technology to be used and are often valid only for a particular period of time. One can define the quality of a model as its suitability for the intended audience and usage circumstances. Models are subject to classification with regard to occurrence in language games and speech acts within such games. For example, early during systems development, requirements engineers talk to anticipated stakeholders to find out what they want, do not want, should want, and should not want. This purpose turns requirements engineering into a language game with (in principle) specific rules. It would, for example, not be that important if the requirements initially were imprecise, incomplete, or inconsistent and in that case not even implementable. What counts is that, finally, it will be figured out what can or should be accomplished given the policy, resources, time and budget constraints of a project.
Preface
XIII
Since models are utterances they are also signs. Like any other sign they represent for an audience a cultural unit, i.e., a thing that within a given culture can meaningfully be referred to. By means of the sign, its user refers to the cultural unit. Different from arbitrary signs, the models are supposed to be processed and that way enable the sign user to create information about the cultural unit. The distinction between sign and model to some extent is arbitrary and there is an overlap, as signs in general need to be processed for the sign user to be capable of referring to the cultural unit. Moreover, signs may also give more information about the cultural unit than the one of its mere existence. For example, a traffic sign may not only indicate that a crossing is ahead but also tell the distance between the current location and that crossing. However, the key functionality of the sign is the mere reference to the cultural unit, while the key feature of a model is to create further information about it. In the case where a sign is a model, its cultural unit obviously is the model’s original. A modeler can use a number of different relationships between a model and its original. As early as the Middle Ages, with regard to models, a descriptive reference mode was distinguish from a prescriptive one. Roel Wieringa, in his PhD thesis, has suggested using what he has called the “direction of fit,” for showing the difference between these two relationships. I use the term mode or reference mode to denote the kind of relationship with which a modeler refers from a model to its original. For example, if a model is used in the descriptive mode, then a non-tolerable difference between a model and its original suggests changing the model. If, however, a model is used in a prescriptive mode then a non-tolerable difference between that model and its original suggests changing the original. Design models are typically used in a prescriptive mode while analysis models are typically used in a descriptive mode. Further reference modes have been identified for better understanding further kinds of modeling. For example in software process modeling and ontologies, the idealizing and the constituting reference modes are used, respectively. The former mode means that differences between a software process model and its instantiation are tolerated only if they are due to the deviation of the actual development conditions from the ideal ones. Otherwise corrective action has to be considered that brings the actual behavior closer to the ideal one. The constitutive reference mode, however, means that the original, for purposes of technical discourse, is created by the model. As a cultural unit, of course, the original must predate its use as a constitutive model. The kind of corrective action to be taken in the presence of intolerable differences between a model and the original depends on the employed reference mode. It therefore is one of the key tasks of conceptual modeling to identify the prime reference modes and work out appropriate techniques for effective and efficient resolution of such differences. It is, moreover, quite possible that a model user with a given model refers to a variety of originals, maybe even employing different reference modes. Therefore, such a modeler might want to use a number of versions of that model so that appropriate changes can be made without confusion. Typically, for example, throughout the software development process, an analysis model will be turned into a design model. Similarly, a model created
XIV
Preface
from a legacy application in a reverse engineering step might be turned into a prescriptive model when a reimplementation of that application becomes necessary. These complexities of the model–original relation play a role regardless of the particular philosophy the model’s creator might follow, as in many cases the model users would not even know or share the modeler’s more basic views. In this text I have tried to abstain from taking any strong position toward a particular worldview. Obviously any such worldview may have far-reaching impact on any stance toward modeling that one may take. This preface, however, is not the place to go into more related detail. With the requirements for a system (at least in a preliminary version) being elicited, developers start the system design in which a system architecture and prototypical implementation of required functionality of required quality are the issues. Any given system under development (SUD) is therefore subject to speech acts in different language games. While in requirements engineering stakeholders say that they want such and such a SUD, in systems design the developers demonstrate that they can build an SUD with such and such properties under the given constraints. It is well-known that the two mentioned language games may have to be carried out several times and may even take turns. I am, however, not going into further detail here. Rather, I only want to establish that there are a number of language games played in systems development and that a given SUD will occur in a variety of speech acts within these language games. From a holistic view on software development, one cannot, once and for all, give precedence to one of the occurring language games over all others. Therefore formal semantics of modeling languages and thus models cannot, once and for all, be considered as more important than what has been called apriori semantics of models. There are further language games to be considered with typical speech acts and uses of models in them such as system use, system maintenance, system retirement, and system deployment. Conceptual modeling naturally extends to these language games. It is well known that models often are abstractions from their original. This means that some of the original’s characteristics are ignored and do not make it into the model characteristics. The model therefore in some way is a simplification of the original. However, often the model can be more specific than its original. The model often has characteristics that are not characteristics of the original. These characteristics are frequent chosen such that the information about the original can be obtained in the intended way. For example, in mathematics models are frequently built such that an equation plays a prominent role. In computing, ER models are built such that a database easily can be defined. Similarly, state charts are worked out because we know how to create (or even generate) code from them. Obviously in “reality” there are usually neither equations, database structures, nor state charts. Therefore modeling and abstraction are very different processes. In modeling, frequently a “guiding metaphor” is used to work out the model. It sort of “pollutes” the conceptualization of the original in a systematic way so that the resulting model can be processed in the intended way. Modeling not only often ignores something, i.e., simplifies or leaves
Preface
XV
out unimportant details; it also often adds something. What it adds, moreover, can be quite problematic (at least for some model users). For example, the concept of limit in many mathematical models from calculus is used when it comes to figuring out the value a variable eventually takes. Such a variable value may quantify any parameter of a technical or natural process. Obtaining the related limit typically presupposes the ability to check out the parameter’s value for points in time arbitrarily far into the future or to let the parameter’s value grow beyond any boundary. Both presuppositions are, however, inconsistent with beliefs held by many regarding characteristics of the universe. For example, many regard the universe as finite in time and only consisting of finite number of particles. A model thus not only might have more characteristics than its original. Some of these surplus characteristics might contradict basic views of a modeler. If one considers design models a little close, then one finds that things actually are even more complicated than that. For example, a conceptual database model, at the time the database it describes does not even exist, is more detailed than that database. The database thus could be considered as an abstraction of its model. However, after the database has been implemented things change. The database is more detailed than its conceptual model and thus the former may be considered as an abstraction of the latter. Obviously, models used in the prescriptive mode in architecture also rely on this inversion of the abstraction relationship. Problems of this kind certainly may be worrying, however, as rumor has it: “All models are wrong. Some are useful.”5 While further interesting things could be said about all, this introduction needs to be concluded and I cannot go into more related detail. I hope that you will enjoy reading this book. Roland Kaschek
5
For this and other nice aphorisms see, e.g., http://www.chebucto.ns.ca/Science/AIMET/models/.
Organization
List of Reviewers Valeria De Antonellis Devis Bianchini Shawn Bowers Lois M. L. Delcambre J¨ org Desel David W. Embley Ulrich Frank Georg Grossmann Brian Henderson-Sellers Wolfgang Hesse Dietmar Jannach Gerti Kappel Roland Kaschek Christian Kop
Bernd Neumayr Andreas Oberweis Christine Parent Oscar Pastor Fabio Porto Michael Schrefl Vladimir A. Shekhovtsov Stefano Spaccapietra Markus Stumptner James Terwilliger Bernhard Thalheim Susan D. Urban Roel Wieringa
Table of Contents
Section I: The Evolution of Conceptual Modeling Techniques Real-World Semantics of Conceptual Models . . . . . . . . . . . . . . . . . . . . . . . . Roel Wieringa
1
Templates in Domain Modeling – A Survey . . . . . . . . . . . . . . . . . . . . . . . . . Christian Kop and Heinrich C. Mayr
21
Integrity Constraints in (Conceptual) Database Models . . . . . . . . . . . . . . . Bernhard Thalheim
42
Modeling Techniques for Multi-level Abstraction . . . . . . . . . . . . . . . . . . . . . Bernd Neumayr, Michael Schrefl, and Bernhard Thalheim
68
Random Thoughts on Multi-level Conceptual Modelling . . . . . . . . . . . . . . Brian Henderson-Sellers
93
On the Evolution of Quality Conceptualization Techniques . . . . . . . . . . . . Vladimir A. Shekhovtsov
117
Section II: Extensions of Conceptual Modeling A Web of Knowledge: A Conceptual-Modeling Perspective . . . . . . . . . . . . David W. Embley, Stephen W. Liddle, and Cui Tao Service-Based Semantic Collaboration in Networked Systems: A Conceptual Perspective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Devis Bianchini, Valeria De Antonellis, and Michele Melchiori Conceptual Modeling Approaches for Dynamic Web Service Composition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Georg Grossmann, Rajesh Thiagarajan, Michael Schrefl, and Markus Stumptner The Dynamics of Process Modeling: New Directions for the Use of Events and Rules in Service-Oriented Computing . . . . . . . . . . . . . . . . . . . . Susan D. Urban, Le Gao, Rajiv Shrestha, and Andrew Courter Exception Handling in Web Service Processes . . . . . . . . . . . . . . . . . . . . . . . . Dietmar Jannach and Alexander Gut
137
161
180
205
225
XX
Table of Contents
Leveraging Model-Based Tool Integration by Conceptual Modeling Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Gerti Kappel, Manuel Wimmer, Werner Retschitzegger, and Wieland Schwinger
254
Section III: New Directions for Conceptual Modeling Data Model for Scientific Models and Hypotheses . . . . . . . . . . . . . . . . . . . . Fabio Porto and Stefano Spaccapietra Model-Based Engineering Applied to the Interpretation of the Human Genome . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Oscar Pastor, Ana M. Levin, Matilde Celma, Juan Carlos Casamayor, Aremy Virrueta, and Luis E. Eraso Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
285
306
331
Real-World Semantics of Conceptual Models Roel Wieringa University of Twente, Enschede, The Netherlands
[email protected] www.cs.utwente.nl/~roelw
Abstract. Conceptual modelling is the addition of more real-world semantics to the computations performed by a computer. It is argued that in a proper engineering approach to computing, three kinds of conceptual modelling need to be distinguished, (1) modelling a software solution, (2) modelling the domain in which it operates, and (3) modelling the impact of the software solution on the domain. Nearly 40 years of research in conceptual modelling has yielded a wealth of concepts and notations for conceptual modelling of the second variety, which we call domainoriented. A summary and framework of these results are presented. Conceptual modelling of the third variety, which we call impact-oriented, has been developed less. A framework for this is provided too, and promising directions for impact-oriented conceptual modelling are identified.
1
Introduction
Conceptual modelling as a discipline in computer science started in the 1970s with landmark papers by Abrial [1] and Chen [2] on data semantics. The goal of conceptual modelling is to describe the set of concepts used in a domain in order to define a conceptual schema for the memory that a software system should have of that domain [3, page 11]. This contains at least two subgoals, namely to specify the conceptual schema of an information system and to describe the concepts used in a domain. Any conceptual modelling notation can be used for either of these two subgoals but, as will be argued in section 2, the resulting conceptual models will be different, to the extent that a correct model of the domain can be an incorrect model of a software system and vice versa. A conceptual model of a software system is a specification of a proposed solution to some problem in a domain. Executable conceptual modelling, for example, focusses on a conceptual modelling of software systems to be realized [4,5]. But when a solution is proposed, it must be justified. This justification must show that the proposed solution, when inserted in a problem domain, will produce effects that satisfy stakeholder goals to some acceptable extent. Analysis of this justification in section 2 motivates a distinction between two kinds of conceptual modelling, one that we will call domain-oriented and focusses on entities, communication and and behavior in the domain, and one which we call impactoriented and focusses on the impact of the software solution on the domain. A review of the results of nearly 40 years of research in conceptual modelling Kaschek/Delcambre (Eds.): The Evolution of Conceptual Modeling, LNCS 6520, pp. 1–20, 2011. c Springer-Verlag Berlin Heidelberg 2011
2
R. Wieringa
(section 3) reveals a mature set of insights into domain-oriented conceptual modelling, that are summarized in section 4. Impact-oriented conceptual modelling has received less attention, and in section 5 we identify promising directions for further conceptual modelling research in this area.
2 2.1
Conceptual Modelling as Domain Modelling The Engineering Argument
Computers, as all other artifacts, are designed to help people achieve their goals, and do this by offering useful interactions with their environment. The crucial difference with all other artifacts is that computers are programmable and therefore real-world problem solving by computer means designing programs that are executed by computers, rather than designing the computers themselves. Nevertheless, the logic of engineering software for human goals is the same as the logic of any other engineering activity, and consists of the following reasoning pattern: (1) Problem domain D & solution S lead to effects E, (2) Effects E satisfy stakeholder goals G to some acceptable extent. For example, a cruise control (S) embedded in a car (D) leads to effects (maintaining speed) that satisfies a goal of the driver (easy driving). And an order information system (S) embedded in an organization with customers and suppliers (D) leads to efficient order processing (E), which satisfies the business goal of operational excellence (G). The engineering argument is the logic of industrial product design as analyzed by Roozenburg and Eekels [6]. For example, interaction of a user and a water tap (D) with a kettle (S) triggers a mechanism in the kettle, namely heating, by which the water in the kettle heats up (E), which satisfies the intention of the user to boil water (G) [7]. In software engineering, Gunter et al. [8] have identified the same reasoning pattern. The solution S is then the specification of a software program; the domain D consists of other software, people or hardware devices, and the interaction of these two should have properties that satisfy goals G. 2.2
Domain Models versus Software Specifications
The engineering argument makes a clear distinction between a solution and the domain in which it is inserted. This distinction is trivial if the solution S is a physical system, but it can be very confusing when S involves software. For example, we may describe a conceptual model of books and borrowers where one book cannot be borrowed by more than one person at the same time (Fig. 1). (The following examples are taken from my book [9], where they are discussed in detail.) In the entity models of figure 1 and 2, we use a snapshot interpretation, meaning that the model represents an arbitrary state of the world (not a history of the world) [9, page 80]. This means that Fig. 1 says that in any state of the
Real-World Semantics of Conceptual Models
3
world a book has a loan relationship to at most one person and that a person can have any number of loan relationships (including 0) to books. Viewed as a conceptual model of the domain, this is a description, and depending on the physical and legal properties of the domain, it could be violated. For example, the information analyst could discover that in a particular library, a book can be borrowed by organizations, and so she would have to correct her model by generalizing Person to include legal as well as biological persons.
Fig. 1. A simple conceptual model of snapshots of a domain
Viewed as a conceptual model of a database solution, by contrast, the diagram in Fig. 1 is a specification, not a description. In this case it can be violated too, because there are possible states of the database in which there may be any number of instances of the Loan relationship between the same document and same person. But our response to these violations would to correct the database, not the conceptual model, because the database should be a correct implementation of the model. The direction of fit between a specification and an implementation is opposite to that between a conceptual model and its subject domain [10,11]. Suppose now that we have a historical database, which accumulates Loan relationship instances, but never deletes them once created. In such a database any number of instances of Loan can exist, each representing a real-world Loan instance that existed at a different time. When specifying the conceptual structure of such a database, we must allow that many historical Loan instances are represented in the database at the same time, as shown in Fig. 2, still using a snapshot interpretation. As a model of the domain this is false, as a book cannot be borrowed by several persons at the same time; but as a model of a historical database it is correct.
Fig. 2. A simple conceptual specification of snapshots of a historical database
Fig. 3 clarifies the situation. A software system, such as a database, contains a model of the domain, and the domain and its model are described by different conceptual models. In our historical database example, Fig. 1 is a conceptual model (CM) of the domain, Fig. 2 is a CM of the database, which contains a historical model of the domain.
4
R. Wieringa CM of the model
CM of the domain
specifies
describes
Software system Model of the domain
represents
Domain
Fig. 3. Conceptual models may describe structures in the domain or may specify conceptual structures of a model in a computer, that in turn represents the domain
Fig. 3 reveals a possible confusion in our terminology concerning conceptual modelling. Conceptual models are actually descriptions of conceptual structures in a domain, as on the right-hand side of Fig. 3, or specifications of conceptual structures to be implemented in software, as on the left-hand side of Fig. 3. (Gonzales-Perez & Henderson-Sellers [11] refer to this as backward and forward looking models, respectively, but the distinction is an old one in the philosophy of language [10].) These descriptions or specifications may be written down in the form of text or diagrams. A model inside a software system, by contrast, is a set of symbol occurrences, physically realized in a computer, that represents a domain [12]. The simplest of such models is a single symbol occurrence in the software, called a surrogate by Hall [13], that represents a single entity in the domain. I will follow general practice and continue to speak of conceptual models, but we should be careful to remember that conceptual models of the domain are actually descriptions of the domain. The model of the domain inside the software system in Fig. 3 is a model in the classical sense that it a duplicates some, but not necessarily all, properties of its subject, just as a scale model duplicates some but not all properties of its subject [14]. We can use the model to answer questions about the subject. To distinguish this kind of model from a conceptual model, we will call it a representation model. Software systems that interact with a problem domain in order to achieve desired effects, i.e. achieve human goals, must contain a representation model of at least some properties of the problem domain in order to interpret stimuli and determine responses [9]. The representation model may be historical, in which case it constitutes a memory of past domain states, or it may be a snapshot, in which case at all times it represents a single recent state of the domain. Object-oriented software is based on the design principle that software contains a representation model of the domain, as in Fig. 3, but sometimes objectoriented methodologists tend to confuse the distinction between domain and software solution. For example, in the Fusion method this confusion was elevated to a central design principle [15]: After a class diagram of the domain was drawn, the analyst drew a rectangle enclosing part of the diagram and declared that this “was” the software solution. What was intended is that the software solution would contain a representation model of this part of the domain, in
Real-World Semantics of Conceptual Models
5
the same way as the object-oriented method avant la lettre Jackson System Development (JSD) recommended building up a software system from an initial representation model of the domain [16]. But when Fusion recommended declaring part of the domain class diagram to be the software solution, it left a gaping hole in the domain, and introduced confusion about which objects, existence constraints and cardinality constraints needed to be specified for this softwareand-domain model. Figures 1 to 3 illustrate how this confusion can arise.
3
Conceptual Modelling Notations Since the 1970s
Since the 1970s and arguably even earlier [17], conceptual modelling research has aimed at defining the semantics of a computation in terms of the problem to be solved rather than in terms of the infrastructure that will run the computation. In terms of the engineering argument, at first this amounted to providing a more abstract specifications of the software solution S. In the 1970s, several people made the move to conceptual modelling of the domain D, and this is where we start of our mini-history of conceptual modelling. We classify the notations into three groups, entity-oriented, communication-oriented and behavior-oriented. As elaborated in detail elsewhere [18], this means that we view real-world domains as consisting of entities that interact through communication channels and whose behavior consists of ordering of these interactions in time. This framework has proven to be general enough to classify, compare, disassemble and reassemble structured and object-oriented specification methods known in the later 1990s [19,9]. Here, we apply it to notations for conceptual modelling of the domain. 3.1
Entity-Oriented Approaches
In databases, a significant abstraction from computational structures took place in 1970, when Codd introduced the relational model as an abstraction from pointer structures [20]. This introduced a logical view on data from which storage structures had been abstracted; what remained were variables, called attributes, and cartesian products of these variables, called relations. A few years later, the move to conceptual modelling of the domain rather than of the software solution was made by Abrial [1] and, very influentially, by Chen, who modelled the domain in terms of entities, attributes and relationships [2], usually referred to as the entity-relationship (ER) approach. These developments matched with the three-level view of databases described by the ANSI-SPARC in a reference architecture [21], in which the storage structure view including the pointer structures is called “physical”, the relational view is called “logical” and the user views defined on top of this are called ”external”. This model contains two confusions (Fig. 4): First, there is nothing physical about the “physical” database view. Pointers and other storage structures are abstractions from a physical computer, and all ANSI-SPARC views are conceptual models in the broad sense that they express conceptual structures in
6
R. Wieringa
a computer. In this chapter a physical domain is a domain that can be described by physical measures such as meters, kilograms, seconds, ampere etc. This is to be distinguished from software domains, which consist of symbols and symbol manipulations, and which may contain many abstraction layers, of which the ANSI “physical” and logical layers are only two. Second, the ANSI-SPARC logical view is also often called the ”conceptual” view [22, page 184] and then defined as the collection of objects of interest in the domain, as in the approaches of Abrial and Chen. This confuses software models with domain models. The ANSI-SPARC models describe conceptual storage or data structures in a computer, and these must be distinguished from conceptual structures in a domain of interest to stakeholders of the system. Fig. 4 distinguishes conceptual software and data models from conceptual domain models, and adds the physical computer as a domain of interest. This allows us to think about the ways in which the physical computer impacts the domain.
Fig. 4. The three-level ANSI-SPARC model replaced by one in which software systems are distinguished from domains, and the real physical computer is included
Numerous extensions to the ER notation have been defined, notably extensions of cardinality constraints [23,24] extensions that deal with time and history [25,26], extensions with aggregation structures [27,28,29,30,31] and, more recently, sets [32], and extensions with taxonomic structures [27,33,34,35]. Temporal extensions of the ER notation offer ways of expressing constraints on how entities, their relationships and their cardinality constraints can evolve over time. Historical extensions allow one to express constraints on the history of entities or relationships. Extensions with aggregation allow the expression of various encapsulation and aggregation structures, and extensions with taxonomic structures allow the description of various static or changeable subclass relationships. Much of this research took place in the 1980s and by the end of that decade surveys started to appear [36,37,38]. By the early 1990s, ER approaches had stabilized and textbooks on ER modelling started to appear [39]. In parallel with the development of the ER approach, an approach was developed variously referred to as “Nijssen’s Information Analysis Method”, “Natural language Information Analysis Method” (both with acronym NIAM), binary
Real-World Semantics of Conceptual Models
7
information modelling and Object-Role Modeling (ORM) [40,41]. The core idea of NIAM/ORM is to describe a domain in elementary sentences of subject–verb– object structure and represent this as a set of binary relationships. NIAM/ORM recognizes taxonomic structures too. A third approach to modelling taxonomic domain structures arose in knowledge representation, using notations motivated from linguistics such as semantic networks [42,43]. This line of research has always been closely linked to various logic specification languages and, today, ontology languages based on description logic have been developed to describe the semantics of data found on the World Wide Web. Ontology languages allow one to define conceptual models of domain concepts and their taxonomic relations and they facilitate automated reasoning about taxonomic structures [44]. Analysis and design of these languages tends to ignore the entity modelling approaches listed above but uses results from philosophical logic and formal logic [45,46]. I will not describe these developments in detail but simply point out that all approaches mentioned here view the domain as a collection of actual and possible entities that may have properties and that play roles with respect to each other. Entity-oriented approaches make a number of important ontological assumptions about the domain: – They all assume that we can classify parts of the domain into discrete entities. – Classes can have taxonomic relationships with respect to each other. – In addition, entities play roles with respect to each other, and they can be counted independently of the roles they are playing. – Many approaches define a notion of aggregation, where one entity can be a part of another, and some distinguish several kinds of aggregation, such as set composition or part-whole relationships. 3.2
Communication-Oriented Approaches
The 1970s also saw the rise of communication-oriented conceptual modelling approaches. Just as in the case of databases, this started with an abstraction from the computing infrastructure. Structured programming was a significant step towards structuring software according to the problem to be solved rather than according to the computing infrastructure that would run the program [47]. However, the “problem to be solved” was still the computation to be performed. In the second half of the 1970s an additional layer of abstraction was added by structured analysis, which contained techniques to model the tasks and data flows needed to solve a problem independently from the logic of problem-solving steps. The notation introduced to represent these tasks was the data flow diagram. Several versions were proposed, including those of Ross [48,49], DeMarco [50], Gane & Sarson [51] and Yourdon & Constantine [52]. In the 1980s, techniques to represent control were added, but we will discuss these in the next section on behavior-oriented approaches. Here, two remarks about structured analysis are relevant.
8
R. Wieringa
First, structured analysis still maintained a focus on the solution (“the problem to be solved”) rather than the domain with which the solution would interact, but it removed attention from the steps to be performed. Structured analysis modelled the solution as a set of tasks that, by communicating with each other and with the domain, would solve the problem. Secondly, the domain was then truncated to the entities with which the solution would interact, which were then called “external entities”. So the focus was on communication within and across the interface of the solution, not on communication in the domain. In the 1980s this changed by the introduction of such methods as essential systems analysis [53] and information engineering [54], in which organizational tasks performed in the domain were represented and redesigned by means of data flow modelling. The central idea is that a conceptual model of the flow of information through an organization would show opportunities for improving the structure of the operational tasks, the business processes, that made the information flow this way. Next to this system-oriented approach to communication modelling, a more people-oriented communication modelling approach of the domain arose in the early 1970s in Scandinavia, where Langefors, Lundeberg and others proposed the ISAC method [55,56,57]. This now nearly-forgotten method started with the identification of stakeholders in the domain, catalogued and analyzed their problems, proposed a mitigation of these problems by improving business processes, and then used IT to implement these improvements. The domain model made in ISAC shows tasks performed in the domain and the information flow between tasks, just like data flow modelling. Unlike structured analysis, however, (1) ISAC made a communication model of stakeholders in the domain rather than of the software only, (2) it modelled problems to be solved, and (3) it did not truncate the domain model once it turned to software specification. ISAC was an information system development method, in which the domain was an organization, and it had a strong focus on analyzing human communication in the domain. In the 1980s a number of researchers, many again from Scandinavia, proposed analyzing not only the information flow but the speech acts performed by stakeholders in the domain [58,59,60,61]. A speech act is not just a flow of information, but it is a flow of information produced by human actions with a specific purpose, such as to request, to inform, to command, to declare, to express, to commit, etc. [62]. Analyzing speech acts in the domain with a view to identifying opportunities for supporting these acts by IT came to be known as the language/action perspective on information analysis [63]. Software tools for supporting speech acts have not caught on, and the language/action perspective has never become mainstream in IS research. However, there is still an active research community working on it [64]. We will return to the language/action approach when we discuss conceptual modelling of the impact of software systems in a social domain (section 5). Viewed from the point of view of conceptual modelling of a domain, the important difference between communication-oriented and entity-oriented approaches is that entity-oriented approaches are concerned with what stakeholders in the
Real-World Semantics of Conceptual Models
9
domain communicate about, whereas communication-oriented approaches focus on who they talk with (information flow) and for the achievement of which goal (speech act). Communication-oriented approaches make a number of important ontological assumptions about a domain and about the interactions of a software system with a domain. – From a communication-oriented perspective, the domain consists of actors that perform speech acts by which they exchange information with each other to achieve certain purposes; from an entity-oriented point of view, this information will be about entities that are classified and can be counted, and that can have aggregation and functional relationships to each other. – If a software system is to communicate with such a domain, it should contain a representation model, not only of the entities communicated about, but also of the actors communicated with. This gives us a two kinds of links with the domain, as represented in Fig. 5. Henceforth, we will refer to entities in the domain that can communicate (“talk”) as actors. Actors can be people or other software systems.
Fig. 5. A software system that interacts with a domain by talking with some actors in the domain and by containing a representation model of some of the entities talked about
3.3
Behavior-Oriented Approaches
In the 1980s structured analysis was extended by Ward & Mellor [65] and Hatley & Pirbhai [66] with notations for behavior to represent actual or desired behavior in domains to be controlled by software. This introduced a third group of notations in conceptual modelling that are behavior-oriented. Where communicationoriented notations represent who is interacting with whom, behavior-oriented notations allow representation of the temporal order in which these interactions take place. In structured analysis, the state transition diagrams and tables developed by Mealy and Moore in the 1940s to represent discrete behavior were used. In the 1980s, Harel combined and extended the Mealy and Moore notations into a very powerful notation called statecharts, that allows specification of complex reactive behavior [67,68]. This could be behavior of a software solution, or behavior of entities in the domain.
10
R. Wieringa
In the early 1980s Jackson proposed a behavior-oriented system development method (JSD) [16]. JSD starts with making a conceptual model of domain entities and events, and allocates events to the entities involved in them. It then models the software solution by specifying a conceptual model (i.e. a specification of the conceptual structure) of a representation model in the software, called an initial system model, which is the model of the domain in the software system in Fig. 3. The software specification is then elaborated by adding components to register, respond, and possibly control domain events according to the desired functions of the system. In the 1990s this approach was adopted and partly formalized in the object-oriented design method Syntropy [69]. All of these methods described behavior of domain entities by means of some form of state transition diagram or table. There is another class of notations, that is used to describe tasks to be performed by people: Workflow modelling notations like Petri nets [70,71], event-process chains [72], UML activity diagrams and the Business Process Modelling Notation (BPMN). There is nothing in these notations that prevents them from being used to model software solutions, but here we consider them as notations to describe the behavior of actors and entities in the domain. This also means that these notations are also used to describe speech acts (register, request, order, etc.) performed by people. Behavior-oriented approaches make a number of important ontological assumptions about the domain. – The domain consists of sequences, choices, and parallel occurrences of events and actions in time. Some notations allow the expression of the real time at which events and actions occur. – Some notations also allow the expression of information flow, such as the transfer of data objects from one task to another. Those notations then allow expression of communication in addition to behavior. – In addition, state transition diagrams usually allow the expression of reactive behavior, which tells us how events that occur to an entity cause the entity to respond by performing some action. For example, a description of a sensor may tell us that it produces a signal (effect) when it is blocked (event), and a specification of a business process may state that a confirmation must be sent (effect) when an insurance claim is received (cause). – Communication and reactive behavior are examples of causal behavior. Causality goes beyond simple temporal ordering, because events and actions that are temporally ordered need not be causally connected. Causality includes a kind of internal necessity which says that, all other things being equal, if the cause would not have occurred, the effect would not have occurred either [73]. The ceteris paribus clause “other things being equal” indicates that many factors may interact to produce an effect and if any one of those changes, the effect of any of the others may change too. Causal interaction is the second way software systems interact with a problem domain, in addition to communication by speech act (Fig. 6).
Real-World Semantics of Conceptual Models
11
Fig. 6. A software system that interacts with a domain by talking with some actors in the domain and causally interacting with entities in the domain
4
Real-World Semantics: Domain Structure
We have seen that entity-oriented approaches are concerned with what actors in the domain communicate about, communication-oriented approaches focus on who they talk with (information flow) and for the achievement of which goal (speech act), and behavior-oriented approaches express how entities and actors behave in order to achieve these goals. For example, in the domain of insurance claim handling, the claim handling workflow specifies how actors (damage experts, claim handling personnel) jointly perform a process that involves some entities (the claim itself and the damaged object) to make a decision about the insurance claim (goal). Fig. 7 provides a simple image of this. The claim handling process consists of activities listed in one column of the table, and each activity involves one or more rows of the table. For example, a damage expert (actor) may assess the damaged object (entity), and an evaluator (actor) may decide that the claim (entity) is accepted. One activity may be split into parts, one part for each entity involved in the activity. Each cell then lists the activities that an entity or actor is involved in within one business process. Finally, the row of an entity or actor contains all activities of all processes that this entity is involved in and the analyst may decide to express the life cycle of this entity or actor in a state transition diagram. For example, we may define the life cycle of an insurance claim or of a policy owner by means of a state transition diagram.
Fig. 7. Domain processes consist of the interaction of entities and actors in the domain. Each row lists the events and actions an entity or actor is involved in. Each column lists the events and actions that occur in a domain process.
12
R. Wieringa
In the previous sections we have learned the following about the real-world semantics of notations to express different aspects of domain structure. – Classification and counting. ER diagrams allow the analyst to represent the classes of entities that can exist in a domain. This includes the entities that can perform speech acts (actors), for entity notations do not distinguish this subcategory of entities. ER notations also allow the expression of taxonomic relationships between these classes, and of cardinality properties of these classes, such as how many instances of a class can exist at the same time or how many entities of one class are play a role with respect to an entity of another class. – Aggregation. Where classification is a relationship between classes, aggregation is a relationship between entities in which one entity somehow encapsulates another. Within the entity approach different kinds of aggregation have been recognized. Fig. 7 shows one important kind of aggregation, the aggregation of the domain out of constituent entities, but the situation may be more complex with intermediate levels of aggregation and with aggregations of different kinds. – Communication. Information about entities or actors is passed between actors by means of speech acts. Notations derived from data flow diagrams in structured analysis or information engineering all represent the flow of data between actors but they do not give more details about the speech acts performed by these actors. – Behavior. Activities in the domain are ordered in time, and they may have causal or logical dependencies. This is expressed in behavioral notations. In terms of Fig. 7, behavioral notations are used to express processes across the domain, such as business processes, workflows, or production control processes, or to express the life cycles of entities or actors. These structures have been analyzed at length elsewhere [74,9] and I will not discuss them in more detail in this high-level overview.
5
Real-World Semantics: Solution Impact
To attempt to solve a problem by computer is to insert a proposed automated solution as an additional actor in the problem domain (Fig. 8). The solution then interacts with other domain entities and actors, which should impact domain processes in such as way that the goals stated in the engineering argument are achieved. Impact-oriented conceptual modelling consists of building a conceptual model of how this happens. To understand this, it is useful to distinguish three kinds of domains.
Real-World Semantics of Conceptual Models
13
Fig. 8. The solution has been introduced in the domain as an additional actor (last row of the table). This impacts some domain processes, who now should contribute to the problem solving goal.
5.1
Physical, Social and Digital Domains
A physical domain is one described in terms of time, space, energy and mass measured by kilograms, meters, second, Amperes, etc. Physical domains can be described by borrowing terms from the physical sciences, such as physics, chemistry, biology, geology, etc. Note that this is a very restrictive view of physical domains. Roughly, a physical domain is a domain seen by sensors or keyboards, not by people. A social domain consists of social constructs such as money, commercial transactions, value, business processes, goals, job roles, responsibility, accountability, etc. The characteristic feature of a social domain is that it contains people who have a shared conceptual model of this domain. Many domain entities and events, such as organizations, job roles and money, are social constructions that would not exist of there were no people who share a conceptual model of these entities and events [75]. At the interface of physical and social domains, and part of both, are digital domains. A digital domain consists of symbols and their physical occurrences. Digital domains are physical, because symbol occurrences are physical: Paper, ink, magnetic disk states, signals travelling through a wire, etc. are physical. But at the same time symbol occurrences have a meaning for people, and this meaning is defined by a convention chosen by a group of people. The meaning of symbols can always be defined in a dictionary, which is part of a social world of shared concepts, but it may also be implicit, not described, but nevertheless shared. Examples of dictionaries are a natural language dictionary, a programming language definition, and a standard for exchanging data between companies. A characteristic feature of dictionaries are that from a physical point of view they are social conventions. The relation between a physical symbol
14
R. Wieringa
occurrence and its meaning is a social convention that from a physical point of view is arbitrary and could have been defined differently. This sets apart a photo from a JPG file describing the photo. People who see the photo do not interpret it according to an artificial convention but see whatever is depicted on the photo by means of a physical (biological) process. By contrast, a JPG file describing the photo is a (highly structured) symbol occurrence that must be interpreted according to an artificial convention, namely the JPEG standard. So the JPG file is part of the digital domain. Digital entities have been called lexical entities in NIAM [40], which emphasizes the presence of a dictionary, and workpieces by Jackson [12], which emphasizes the fact that they can manipulated. The popular term for them is virtual entities, which emphasizes the fact that they are simulated on a computer. In this chapter I use the term “digital entity”. The implementation of a digital domain in a software system is actually executable conceptual modelling, which is out of the scope of this chapter. However, describing the domain to be simulated is within the scope of this chapter. In terms of the first part of the engineering argument (1) domain D & solution S lead to effects E, simulation is the production of desired effects. For example, people (D) who want to view a photo (E) use viewing software (S) to do so. The effect (E) is created by interaction of the user (D) with the viewing software (S). And if a person (D) wants to edit a document (E) then he or she uses an editor (S) to do so. This justifies the description of the domain to be simulated independently from any software solution. This is exactly what is done in Jackson’s problem frame approach [12]. We will call a simulated domain an effect domain to indicate its role in the engineering argument. In some cases, the effect of inserting a software system in some domains is the creation of an effect domain, which can be physical (e.g. photos) or digital (e.g. documents). In other cases, the effect of inserting a software system into a physical domain is the imposition of desired behavior on an existing domain, e.g. a production process. And in yet other cases, the effect of inserting a software system into a social domain can be the creation of new possibilities for action, such as the answering of queries about the domain. 5.2
Physical Modelling
In physical domains, such as in production control or in medical applications, the solution interacts with the problem domain by means of causal interactions. In addition to the communicative interaction shown earlier (Fig. 5) we now have a causal interaction, that we elaborate as shown in Fig. 9. Physical events occur in the domain, which may lead to a chain of causes and effects to a physical stimulus at the interface of the system. Conversely, the system may respond to such a stimulus, or to some communication received from an actor, which leads through a causal chain of events to a change in the domain that we will call an action.
Real-World Semantics of Conceptual Models
15
Fig. 9. A software system that interacts with a domain by communicating with actors and causally interacting with entities. It contains a theory about the domain that is used to interpret stimuli and predict the effect of responses.
From a physical point of view the system now not only needs a model of the domain but also a theory: When it receives a stimulus it needs to explain which event caused it, and when it produces a response it needs to predict what action will then occur. Explanation and prediction are functions of scientific theories. For example, an embedded controller of a copier should use a theory to predict the angular speed of a rubber roller when paper is passing over it. The theory needed by the system, even if it is very simple, has the same structure as that of a scientific theory, namely a set of concepts and a set of empirically valid statistical or causal relationships among these concepts [76,77]. This means that there are two kinds of physical modelling activities for physical domains: – Conceptual modelling, which is the definition, analysis and formalization of a set of concepts in a domain. In the philosophy of science defined concepts are often called constructs or variables. – Empirical modelling, which is the description and validation of statistical or causal relationships among concepts. For example, suppose that a continuous process control system contains tubes with valves that can be opened and closed. To make a physical model of such a domain we need to make (1) a conceptual model that defines what a tube is, how different tubes are distinguished, what a valve is, in what states it can be etc. and (2) to make an empirical model of the behavior of these entities, such as the impact of fluid pressure on the processes of opening and closing the valve, fluid leakage through valves, failure behavior of valves, the effect of temperature, etc. This opens up interesting new avenues for conceptual modelling research, including not only the definition of constructs and measurement procedures but also the description of domain theories in a way that can be used for producing the engineering argument. This is domain-specific research because, for example, process control and car control will contain different theories of their domains[78,79]. In addition, there is also domain-independent research to be done, such as in the investigation of ways to operationalize and measure the constructs that we define in conceptual models, such as has also been done for software properties [80], and in the development and study of languages to represent dynamic systems [81].
16
5.3
R. Wieringa
Performative Modelling
In social domains, software systems can create effects by performing speech acts: answering questions, posing questions, giving commands, making commitments, etc. When inserted in a social domain, software systems become actors just as people are actors in this domain. This also means that a software system can perform declarative speech acts, which are speech acts that declare an event or state to exist by saying that it exists [82,83,84]. If done in the proper situation and according to the proper procedures as defined in a shared conceptual model, then these acts are successful. For example, in the Netherlands, software systems have been used to declare someone eligible for military service or for a housing subsidy. Using a software system to perform speech acts requires that the conceptual model of the impact of the software system in the domain be shared with people in the domain. If no one in the domain knows or believes that the output of a software system is an answer to a question, or is a command, or a declaration, then the software system is not really part of the domain and it cannot play its role, and there is a real sense in which its output in that case is not an answer, command or declaration. As pointed out before, there is a small but active community of researchers investigating software systems from a language/action perspective, but so far this has not had an impact on mainstream information systems development practices. An explanation for this is that practitioners do not experience a big problem in specifying speech acts to be performed by software systems. An interesting avenue for research here is not to develop yet another notation for speech act modelling, but to find out empirically what actually is the structure of automated speech acts in practice, how they are in fact defined, and how the speech acts are embedded in a social domain. For example, the evolution of language in a social domain tends to be frozen when part of the language gets automated in software systems [85] and it is interesting and relevant to know how people actually deal with this.
6
Summary and Conclusions
The system engineering argument justifies a solution by showing how it creates desired effects in a domain. Conceptual modelling research has focussed on the concepts and notations for modelling entities, communication and behavior in the domain but has neglected modelling the impact of the software solution on the domain that is to create the desired effects. Performative modelling research has gone some way in this direction but more empirical work needs to be done in this area, and this needs to be extended with research in physical modelling. Impact-oriented conceptual modelling contains some challenging research questions that deserve more attention than they have received so far. These questions include the study of techniques for operationalization and measurement in physical domains, the description of causal and statistical relationships in conceptual models of physical domains, and the way people deal with frozen conceptual models in social domains.
Real-World Semantics of Conceptual Models
17
Acknowledgments. This chapter benefited from constructive and insightful comments by the reviewers.
References 1. Abrial, J.R.: Data semantics. In: IFIP Working Conference Data Base Management, pp. 1–60. North-Holland, Amsterdam (1974) 2. Chen, P.S.: The entity-relationship model – Toward a unified view of data. ACM Transactions on Database Systems 1, 9–36 (1976) 3. Oliv´e, A.: Conceptual modeling of information systems. Springer, Heidelberg (2007) 4. Mellor, S., Balcer, M.: Executable UML: A Foundation for Model-Driven Architecture. Addison-Wesley, Reading (2002) 5. Pastor, O., Molina, J.C.: Model-Driven Architecture in Practice: A Software Production Environment Based on Conceptual Modeling. Springer, Heidelberg (2007) 6. Roozenburg, N., Eekels, J.: Product design: Fundamentals and Methods. Wiley, Chichester (1995) 7. Roozenburg, N.: On the pattern of reasoning in innovative design. Design Studies 14(1), 4–18 (1993) 8. Gunter, C., Gunter, E., Jackson, M., Zave, P.: A reference model for requirements and specifications. IEEE Software 17(3), 37–43 (2000) 9. Wieringa, R.: Design Methods for Reactive Systems: Yourdon, Statemate and the UML. Morgan Kaufmann, San Francisco (2003) 10. Wieringa, R., Meyer, J.J.C., Weigand, H.: Specifying dynamic and deontic integrity constraints. Data and Knowledge Engineering 4, 157–189 (1989) 11. Gonzalez-Perez, C., Henderson-Sellers, B.: Modelling software development methodologies: A conceptual foundation. The Journal of Systems and Software 80, 1778–1796 (2007) 12. Jackson, M.: Problem Frames: Analysing and Structuring Software Development Problems. Addison-Wesley, Reading (2000) 13. Hall, P., Owlett, J., Todd, S.: Relations and entities. In: Nijssen, G. (ed.) Modelling in Database Management Systems, pp. 201–220. North-Holland, Amsterdam (1976) 14. Apostel, L.: Towards a formal study of models in the non-formal sciences. In: Freudenthal, H. (ed.) The Concept and Role of the Model in the Mathematical and the Natural and Social Sciences, pp. 1–37. Reidel, Dordrecht (1961) 15. Coleman, D., Arnold, P., Bodoff, S., Dollin, C., Gilchrist, H., Hayes, F., Jeremaes, P.: Object-Oriented Development: The FUSION Method. Prentice-Hall, Englewood Cliffs (1994) 16. Jackson, M.: System Development. Prentice-Hall, Englewood Cliffs (1983) 17. Young, J., Kent, H.: Abstract formulation of data processing problems. The Journal of Industrial Engineering 9(6), 471–479 (1958) 18. Wieringa, R.: Requirements Engineering: Frameworks for Understanding. Wiley (1996) Also available at http://www.cs.utwente/nl/~ roelw/REFU/all.pdf 19. Wieringa, R.: A survey of structured and object-oriented software specification methods and techniques. ACM Computing Surveys 30(4), 459–527 (1998) 20. Codd, E.: A relational model of data for large shared data banks. Communications of the ACM 13, 377–387 (1970) 21. ANSI: Ansi/x3/sparc study group on DBMS’s interim report. SIGMOD FDT Bulletin of ACM 7(2) (1975)
18
R. Wieringa
22. Tsichritzis, D., Klug, A. (eds.): The ANSI/X3/SPARC DBMS framework report of the study group on database management systems. Information Systems 3, 173–191 (1979) 23. Thalheim, B.: Fundamentals of cardinality constraints. In: Pernul, G., Tjoa, A.M. (eds.) ER 1992. LNCS, vol. 645, pp. 7–23. Springer, Heidelberg (1992) 24. Liddle, S., Embley, D., Woodfield, S.: Cardinality constraints in semantic data models. Data & Knowledge Engineering 11(3), 235–270 (1993) 25. Elmasri, R., El-Assal, I., Kouramajian, V.: Semantics of temporal data in an extended er model. In: Kangalasso, H. (ed.) Entity-Relationship Approach: The Core of Conceptual Modelling, pp. 239–254. Elsevier, Amsterdam (1991) 26. Tansel, A., Clifford, J., Gadia, S., Jajodia, S., Segev, A., Snodgrass, R. (eds.): Temporal Databases. Benjamin/Cummings (1993) 27. Smith, J.M., Smith, D.: Database abstractions: Aggregation and generalization. ACM Transactions on Database Systems 2, 105–133 (1977) 28. Storey, V.: Meronymic relationships. Journal of Database Administration 2(3), 22– 35 (1991) 29. Motschnig-Pitrik, R.: The semantics of parts versus aggregates in data/knowledge modeling. In: Rolland, C., Cauvet, C., Bodart, F. (eds.) CAiSE 1993. LNCS, vol. 685, pp. 352–373. Springer, Heidelberg (1993) 30. Henderson-Sellers, B., Barbier, F.: Black and white diamonds. In: France, R.B. (ed.) UML 1999. LNCS, vol. 1723, pp. 550–565. Springer, Heidelberg (1999) 31. Opdahl, A., Henderson-Sellers, B., Barbier, F.: Ontological analysis of whole–part relationships in OO–models. Information and Software Technology 43, 387–399 (2001) (Erratum on page 577) 32. Ram, S., Khatri, V.: A comprehensive framework for modeling set-based business rules during conceptual database design. Information Systems 30(2), 89–118 (2005) 33. Mylopoulos, J., Bernstein, P., Wong, H.: A language facility for designing databaseintensive applications. ACM Transactions on Database Systems 5, 185–207 (1980) 34. Borgida, A., Mylopoulos, J., Wong, H.: Generalization/specialization as a basis for software specification. In: Brodie, M., Mylopoulos, J., Schmidt, J. (eds.) On Conceptual Modelling, pp. 87–114. Springer, Heidelberg (1984) 35. Wieringa, R., de Jonge, W., Spruit, P.: Using dynamic classes and role classes to model object migration. Theory and Practice of Object Systems 1(1), 61–83 (1995) 36. Urban, S., Delcambre, L.: An analysis of the structural, dynamic, and temporal aspects of semantic data models. In: Proceedings of the International Conference on Data Engineering, pp. 382–387 (1986) 37. Hull, R., King, R.: Semantic database modeling: Survey, applications, and research issues. ACM Computing Surveys 19(187), 201–260 38. Peckham, J., Maryanski, F.: Semantic data models. ACM Computing Surveys 20, 153–189 (1988) 39. Batini, C., Ceri, S., Navathe, S.: Conceptual Database Design: An EntityRelationship Approach. Benjamin/Cummings (1992) 40. Nijssen, G., Halpin, T.: Conceptual Schema and Relational Database Design. Prentice-Hall, Englewood Cliffs (1989) 41. Vermeir, D., Nijssen, G.M.: A procedure to define the object type structure of a conceptual schema. Information Systems 7(4), 329–336 (1982) 42. Brachman, R.: What IS-A is and isn’t: an analysis of taxonomic links in semantic networks. Computer 16(10), 30–36 (1983) 43. Woods, W.: What’s in a link: Foundations for semantics networks. In: Bobrow, D., Collins, A. (eds.) Representation ans Understanding: Studies in Cognitive Science, pp. 35–82. Academic Press, London (1975)
Real-World Semantics of Conceptual Models
19
44. W3C: OWL Web Ontology Language Overview (2004) 45. De Almeida Falbo, R., Guizzardi, G., Duarte, K.: An ontological approach to domain engineering. In: Proceedings of the 14th International Conference on Software Engineering and Knowledge Engineering (SEKE 2002), pp. 351–358. ACM Press, New York (2002) 46. Guizzardi, G.: Modal aspects of object types and part-whole relations and the de re/de dicto distinction. In: Krogstie, J., Opdahl, A.L., Sindre, G. (eds.) CAiSE 2007 and WES 2007. LNCS, vol. 4495, pp. 5–20. Springer, Heidelberg (2007) 47. Dahl, O.J., Dijkstra, E.W., Hoare, C.A.R.: Structured Programming. Academic Press, London (1972) 48. Ross, D.T.: Structured analysis (SA): A language for communicating ideas. IEEE Transactions on Software Engineering SE-3(1), 16–34 (1977) 49. Ross, D.T., Schoman, K.E.: Structured analysis for requirements definition. IEEE Transactions on Software Engineering SE-3(5), 6–15 (1977) 50. DeMarco, T.: Structured Analysis and System Specification. Yourdon Press/Prentice-Hall (1978) 51. Gane, C., Sarson, T.: Structured Systems Analysis: Tools and Techniques. PrenticeHall, Englewood Cliffs (1979) 52. Yourdon, E., Constantine, L.L.: Structured Design: Fundamentals of a Discipline of Computer Program and Systems Design. Prentice-Hall, Englewood Cliffs (1979) 53. McMenamin, S.M., Palmer, J.F.: Essential Systems Analysis. Yourdon Press/Prentice Hall (1984) 54. Martin, J.: Information Engineering. Prentice-Hall, Englewood Cliffs (1989), Three volumes 55. Langefors, B.: Information systems theory. Information Systems 2, 207–219 (1977) 56. Lundeberg, M., Goldkuhl, G., Nilsson, A.: A systematic approach to information systems development - I. Introduction. Information Systems 4, 1–12 (1979) 57. Lundeberg, M., Goldkuhl, G., Nilsson, A.: A systematic approach to information systems development - II. Problem and data oriented methodology. Information Systems 4, 93–118 (1979) 58. Flores, F., Ludlow, J.: Doing and speaking in the office. In: Fick, G., Sprague, R. (eds.) Decision Support Systems: Issues and Challenges, pp. 95–118. Pergamon Press, Oxford (1980) 59. Goldkuhl, G., Lyytinen, K.: A language action view on information systems. In: Ginzberg, M., Ross, C. (eds.) Proceedings of the Third International Conference on Information Systems, Ann Arbor (1982) 60. Lehtinen, E., Lyytinen, K.: Action based model of information systems. Information Systems 11, 299–317 (1986) 61. Auram¨ aki, E., Lehtinen, E., Lyytinen, K.: A speech-act-based office modeling approach. ACM Transactions on Office Information Systems, 126–152 (1988) 62. Searle, J.: Speech Acts. An Essay in the Philosophy of Language. Cambridge University Press, Cambridge (1969) 63. Winograd, T.: A language/action perspective on the design of cooperative work. Human-Computer Interaction 3(1), 3–30 (1987–1988) 64. Weigand, H.: Two decades of the language-action perspective: Introduction. Communications of the ACM 49(5), 44–46 (2006) 65. Ward, P.T., Mellor, S.J.: Structured Development for Real-Time Systems. PrenticeHall/Yourdon Press (1985), Three volumes 66. Hatley, D., Pirbhai, I.: Strategies for Real-Time System Specification. Dorset House (1987)
20
R. Wieringa
67. Harel, D.: Statecharts: a visual formalism for complex systems. Science of Computer Programming 8, 231–274 (1987); Preliminary version appeared as Technical Report CS 84-05, The Weizmann Institute of Science, Rehovot, Israel (February 1984) 68. Harel, D.: On visual formalisms. Communications of the ACM 31, 514–530 (1988) 69. Cook, S., Daniels, J.: Designing Object Systems: Object-Oriented Modelling with Syntropy. Prentice-Hall, Englewood Cliffs (1994) 70. van der Aalst, W., van Hee, K.: Business process redesign: a Petri-net-based approach. Computers in Industry 29(1-2), 15–26 (1996) 71. van der Aalst, W., van Hee, K.: Workflow Management: Models, Methods, and Systems. MIT Press, Cambridge (2004) 72. Scheer, A.W.: Business Process Reengineering: Reference Models for Industrial Enterprises. Springer, Heidelberg (1994) 73. Shadish, W., Cook, T., Campbell, D.: Experimental and Quasi-experimental Designs for Generalized Causal Inference. Houghton Mifflin Company (2002) 74. Wieringa, R.: Postmodern software design with NYAM: Not yet another method. In: Broy, M. (ed.) RTSE 1997. LNCS, vol. 1526, pp. 69–94. Springer, Heidelberg (1998) 75. Searle, J.: The Construction of Social reality. The Free Press, New York (1995) 76. Gregor, S.: The nature of theory in information systems. MIS Quarterly 30(3), 611–642 (2006) 77. Kaplan, A.: The Conduct of Inquiry. Methodology for Behavioral Science. Transaction Publishers (1998); First edition 1964 by Chandler Publishers 78. Jackson, M.: Specialising in software engineering. IEEE Software 16(6), 119–121 (1999) 79. Zave, P.: Formal methods are research, not development. Computer 29(4), 26–27 (1996) 80. Fenton, N., Pfleeger, S.: Software Metrics: A Rigorousn and Practical Approach, 2nd edn. Thomson (1997) 81. Sterman, J.: Business Dynamics: Systems Thinking and Modeling for a Complex World. McGraw-Hill, New York (2000) 82. Kimbrough, S., Lee, R., Ness, D.: Performative, informative and emotive systems: The first piece of the PIE. In: Maggi, L., King, J., Kraenens, K. (eds.) Proceedings of the Fifth Conference on Information Systems, pp. 141–148 (1983) 83. Kimbrough, S.: On representation schemes for promising electronically. Decision Support Systems 6, 99–121 (1990) 84. Wieringa, R.: Three roles of conceptual models in information system design and use. In: Falkenberg, E., Lindgreen, P. (eds.) Information System Concepts: An In-Depth Analysis, pp. 31–51. North-Holland, Amsterdam (1989) 85. Hoppenbrouwers, S.: Freezing Language: Conceptualisation processes across ICT supported organisations. PhD thesis, University of Nijmegen (2003), http://osiris.cs.kun.nl/iris/web-docs/research/index.html
Templates in Domain Modeling – A Survey Christian Kop and Heinrich C. Mayr Alpen-Adria-Universität Klagenfurt, Department of Applied Informatics / Application Engineering Austria {chris,mayr}@ifit.uni-klu.ac.at
Abstract. Conceptual modeling is often strongly related to a graphical language. Since the 80s, template-based approaches have also been proposed. However, they seldom reached the same popularity as graphical approaches. Nevertheless, template-based approaches are also important for collecting and representing information. This chapter will give a survey of approaches that used and use templates. It will be shown, how these templates are used and why their role is important. Keywords: template-based approaches, conceptual modeling, predesign.
1 Introduction Conceptual modeling is often strongly related to using a graphical language. The Unified Modeling Language (e.g., UML) and other languages like ER, ORM, Data Flows, etc. are good examples of this relationship. Many parts of UML are diagrams for specific purposes during software engineering (e.g., class diagrams, activity diagrams, use cases, state charts, object interaction diagrams). Class diagrams help to model the data structure of information systems. Use cases model the functionality the information system must provide to its users. Finally activity diagrams, state charts and object interaction diagrams help to model behavior (i.e., behavior of the system, components or single objects). Class Diagrams, use case and activity diagrams are used in the late stage of requirements engineering (domain modeling). Class diagrams, state charts, object interaction diagrams are typically used during the design phase in software engineering. Although graphical languages like UML are very popular, they only show parts of information needed to develop a system. For more information, template-based approaches have been also proposed. However, they were mostly seen as an “add on” to graphical approaches. This chapter will give a survey of approaches that used templates which are not based on a graphical (diagrammatic) representation. It will show how these templates are used and why their role is important. In order to do this, the chapter will be structured as follows. Section 2 gives an introduction into the several categories of templates. Notions are be defined and a categorization is made. Section 3 firstly provides some thoughts why graphical approaches are so popular, but afterwards it also discusses why template-based Kaschek/Delcambre (Eds.): The Evolution of Conceptual Modeling, LNCS 6520, pp. 21–41, 2011. © Springer-Verlag Berlin Heidelberg 2011
22
C. Kop and H.C. Mayr
approaches are important and thus should not be ignored. Section 4 gives a survey on existing template-based approaches. The next section provides the description and historical summary of a special approach which was developed by the authors (Section 5). Section 6 concludes with a visionary idea and scenario of how templatebased approaches can be integrated into the whole software development cycle in the future. The chapter is summarized in Section 7.
2 Types of Templates Before starting with the description and history of template-based approaches, it is necessary to define what is meant here by the term template. A template here is any kind of textual information which follows a structure. This means, it is possible to know the meaning of some information according to its position within the structure. Templates can be further divided by their purpose and usage into forms and controlled language sentences pattern. A form is a template that has predefined information and empty slots for each of the predefined information. These empty slots have to be filled out. The process of filling the empty slots is supported by the predefined information. In other words according to the predefined information to which an empty slot belongs, the reader of the form can conclude about the “semantic” of the slot’s content and knows what to do with it (i.e., what kind of information must be entered and/or checked). A form can be further refined to attribute-value-pair lists, tables, matrices, cubes, n-dimensional tables and glossaries. If the form is provided in a tabular format with two columns (attribute, value) and in each row the attribute column is filled with predefined information and the value column can be filled out, then it is an attribute-value-pair list. If the form information is repeated several times and the predefined slots can be extracted and used as column header information because they remain invariant in this repetition then the term table will be used. If a second dimension is added to the table, then it is called a matrix. Further dimensions can be added. Then this becomes a cube (3 dimensions) or a multidimensional table, in general (n-dimensions). In this case the header information is only invariant with respect to a certain dimension. Cubes and Multidimensional Tables are only mentioned for the sake of completeness and they will not be mentioned further. They are not commonly used as a mean for communication in Requirements and Software Engineering since two dimensions can be easily described and visualized on a sheet of paper or in standard software products (e.g., Word, PowerPoint, Excel etc.). The term glossary is commonly defined as follows. A glossary is a list of terms together with their definition. From the point of view of a representation, a glossary consists at least of two columns, the column with the term and the column with the definition of terms. Hence a glossary is a table. Beyond this, a glossary has a primary column which contains the terms which must be described. The rows in the other columns support the description of these elements.
Templates in Domain Modeling – A Survey
23
A controlled natural language sentence pattern is a template in which all information is given, but the semantic of a certain kind of information can be derived from its position within the structure (i.e., the grammar). Each sentence follows this very restrictive grammar. Controlled languages are: “Subsets of natural languages whose grammars and dictionaries have been restricted in order to reduce or eliminate both ambiguity and complexity”1 Although the grammar in a controlled language is restrictive, they have a main benefit. The information expressed as a sentence is unambiguous. Furthermore, according to the syntactical structure of a sentence a writer of a controlled language sentence knows which correct combination of word categories he must use in order to get a well formed sentence. For instance, if the pattern is followed that a well formed sentence should have the form subject, predicate object he will certainly not forget the subject or object. Furthermore if he knows that a verb needs an agent (someone/something that initiates execution of a process) he will not forget and the model becomes much more complete. Furthermore, templates can be divided according to their level of abstraction into • templates representing the schema level and • templates representing the instance level. 2.1 Templates for the Schema (Model) Level In a form that describes the schema, the predefined information belongs to the metamodel. Each of the empty slots belongs to a certain schema (model) element. Figure 1 shows some examples for a general form and an attribute value pair list.
precondition
precondition main flow exceptional flow
main flow
postcondition
exceptional flow
postcondition
Fig. 1. Example for a form and an attribute-value pair list
1
See: http://sites.google.com/site/controllednaturallanguage/ (last access: Sept., 27 2010)
24
C. Kop and H.C. Mayr
A controlled natural language sentence might be “The customer orders many products”. Based on the grammar conclusions can be made about certain words in the context of modeling (e.g., nouns become classes or attributes, verbs become relationships, verbs with an agent become candidates for actions in scenario steps etc.). 2.2 Templates for the Instance Level Templates can also be used on an instance level. Instead of specifying the meta-model in the predefined information and the schema in the empty slots, the predefined information is on the schema level and the empty slots must contain the instances of this schema. The previously mentioned notions form, table, matrix and glossary can be found here as well. Typical examples for forms, tables and matrixes are user interfaces, Excel-Sheets, forms for invoices, tabular reports etc. They are also useful for design since the predefined information is useful for the construction of the schema. The information which has to be entered into the empty slots must be managed by the schema and therefore is useful for concrete test cases.
First name Last name Address
Phone Fax e-Mail
Fig. 2. Example of a form at the instance level
Controlled language pattern can also provide information on the instance level. In this case, instead of common nouns, word categories like proper nouns and numbers are used in combination with common individual nouns. These proper nouns and numbers represent the instance level.
3 Graphical Modeling Languages vs. Template-Based Approaches Before talking about template-based approaches, graphical approaches will be discussed.
Templates in Domain Modeling – A Survey
25
3.1 An Advocacy for Graphical Modeling Languages? There are a number of reasons why graphical approaches are so popular: • Humans are visual beings. They perceive and explore the world through their eyes. Other senses - like the sense to hear, taste, feel, and smell something - are often used secondarily. • Pictures, graphical symbols (e.g., cave paintings), statues and totem-poles were the first models of mankind. • Pictures and graphical representations can provide us a spatial overview and an overall description of a physical thing, a phenomenon or a problem. This is why it is often said that a “picture is worth a thousand words”. Due to the good visual comprehension of human, they can better “read” a fact from an overall picture than from a sequence of letters (e.g., the Latin alphabet). • Pictures and graphical representations (if it does not belong to graphical art) is often made exactly just for the purpose to offer a summary (e.g., bar charts in business domains). • A conceptual model in informatics is often compared with a blue print for the construction of a material object (e.g., house, car, bridge etc.). To be trained to think in that way during the education also has an influence on the skills to read and interpret certain model representations. • Finally, it is often argued that according to the above mentioned advantages, these graphical representations are a good basis for later steps. Even model to model transformation is possible. The class diagrams strongly used in model driven architecture (MDA) are representatives for that argument. 3.2 Limitations of Graphical Modeling Languages According to the above subsection, a graphical representation for a modeling system offers advantages. However, a more detailed look to graphical representation techniques also shows that they are not optimal in every situation and for every reader. In order to read or interpret the graphical description, it is necessary that the “reader” understands the semantics of notions. If a person does not understand an underlying notion (e.g., the general idea of a class, an attribute or association) then a graphical representation (e.g., a UML class diagram) is useless for a reader who does not have the necessary skills. If the “reader” does not know that * means “many” in a detailed description of an association, he will not understand in which way the concepts are related to each other. The same happens if we see today a cave painting which was created ten thousand years ago. Although we see different kinds of painted animals we do not understand anymore what the meaning of the complete picture was. We can only make vague assumptions or guesses. Even if the modeling concepts of the graphical representation are semantically understandable, there is still a question concerning the number of concepts visible in the graphic or picture. If there are only a dozen, then the human being of course can
26
C. Kop and H.C. Mayr
realize and interpret all of them at once. However, if there are many (e.g., hundreds) of them, the reader will get lost and the graphical representation looses its ability to be an overview which can be interpreted immediately. Imagine, for example, a class diagram with 100 classes or a Use Case diagram with 100 use cases and actors related to these use cases. In such cases, human beings must introduce abstraction mechanisms (clustering of classes, use cases) to structure the picture. The different kinds of notions represented in a graphical design are the second problem. If there are too many, the reader may get confused. Imagine for example a class diagram that shows everything: classes, attributes, associations, visibilities, data types of attributes, multiplicities, roles, stereotypes of classes and associations, comments on all modeling concepts and much more. Such a class diagram will also likely become unreadable since the reader will not know on which concepts he has to focus. He will be overwhelmed with too much information which he cannot interpret at once. Conceptual modeling tools therefore often offer features to hide information (e.g., to hide attributes, roles, visibilities etc.). Beside the skills of the reader, often the intended purpose of usage has a great influence on the type of model that is used. If the overview character that is provided very well by a picture or graphic is of less importance, then the graphical representation may have less importance. Imagine for example a situation where a questionnaire or pattern is needed. Typically the structure of a questionnaire (i.e., pre defined questions) gives hints to the person who uses the questionnaire regarding which questions are important. Although many software engineers like to see themselves as architects of new systems, if they work in the field of requirements engineering, this “self description” is only conditionally valid. During the early phase of software engineering the relationship between a software (requirements) engineer and another stakeholder (e.g., end user) is like the relationship between a medical doctor and his patient. It is the responsibility of the doctor to extract the causes and later on find solutions for the causes. From the end user he will only get symptoms. Without questionnaires and patterns (at least in mind) he will not be able to find the causes. If requirements engineering experts do not use these questionnaires and patterns explicitly then this does not mean that they are never used since these experts already have internalized the questionnaires and can use them very flexibly, based on the state of communication with the end user. The final argument for graphical approaches (i.e., that they provide a good basis for subsequent steps) is not always true. Consider for instance a use case diagram. The core of the diagram consists of use cases, actors, relationships between use cases, and relationships between use cases and actors. With these information alone transformation to a subsequent step is not possible. It is a template-based approach – the uses case description – that allows a transformation (at least manually). Only with information about preconditions, exceptional flows, main flow and other parts of the use case description, the designer gets an imagination of the details and hence an understanding about the behavior of the future software system. In general a graphical representation is not responsible for the success of a model transformation. The underlying modeling notions of the source and target model are responsible for the success of a transformation. If a match between the notions of the source and target model is possible, then also a transformation is possible.
Templates in Domain Modeling – A Survey
27
3.3 Why Template-Based Approaches? According to Section 3.1 it seems that template-based approaches cannot compete with graphical approaches and graphical representation are better for human beings. According to Section 3.2 graphical approaches have limitations. Hence, templatebased approaches and graphical approaches can complement each other and there is no reason why graphical approaches alone should dominate. The next figure shows that a template in the form of a glossary can be used as a questionnaire. The columns represent hints for questions which must be asked if a cell of a column and row is empty.
ID
Name
120
ICD10
123
SSN
201
hospital
Do we have to consider synonyms for this notion – which ones?
Can you give me some examples for this notion?
Can you define/describe „ICD10“?
Description
?
Examples
?
Synonyms
ValueConstraint
?
?
Quantity description
?
Social Security No. of a patient
Is there any restriction on the values of the examples?
How many examples can exist (minimum, maximum, average)?
Fig. 3. A KCPM Glossary as a questionnaire
If these and other information would appear in a class diagram, then it would be confusing. Therefore, a glossary can complement a diagram in such a way that it provides the involved stakeholders with information that might be necessary for software development. Use cases are another example for such a symbiosis. The use case diagram is a nice picture to describe a certain aspect of the future software, however the most essential information are hidden in the use case descriptions. Thus the relationship between some graphical approaches and template approaches can be seen as an “iceberg”. Only a small percentage of the information appears graphically. The rest, which is represented by templates, is hidden under the “water”. 3.4 Template-Based versus Diagrams – Comparison Studies In the previous subsections pros and cons for diagram-based and template-based modeling languages were mentioned. In this subsection some studies found in literature are presented. These results are only exemplary for specific kinds of templates.
28
C. Kop and H.C. Mayr The visible (graphically represented) information, e.g., use cases, actors, classes, associations. attributes …
The hidden (not graphically represented) information, e.g., use case descriptions, visibilities of attributes, comments on classes …
Fig. 4. Iceberg of schema information
A study [44] was presented which compared textual use cases (TUC) and sequence diagrams (SSD) in the domain of safety system analysis. Textual use cases were tables with the following columns: user actions, system response, threats and mitigation. In each row of the table a piece of text described the user action, a possible system response as well as the threats and mitigations. For the system’s sequence diagram (SSD) the UML notation was used. The textual use case (TUC) was used for interaction between the system and the user. The SSD was used for the interaction between the system and the user and also for internal system communication. The main feature of safety systems is to guarantee safety and avoid failures. During analysis of such systems it is necessary to detect risky situations (failures which might occur). For the comparison of the two presentation techniques, two groups of people (one in Norway, the other in France) were asked to find failures. Statistical methods (t-test, proportional test) were used for answering the following research questions: Is a Textual Use Case (TUC), in general, better than SSD for identifying failure modes? If it is better, are there problem areas where SSD is better? If TUC is better, are there specific failure modes where SSD is better? According to the tests, the following were found: TUC is better to identify failure modes related to functionality of the system and the operator. However, for failure modes which occur within the system, SSD is better. Another study [9] focused on the comparison of tabular representation and bar charts in order to represent production statistics or income statistics. In this experiment people with two different educations (business and engineering) were asked to answer questions concerning income or production based on tabular and graphical representations (bar charts). The time which was needed to get the answer and the number of correct and incorrect answers were measured. The following results were found. There is no difference between two educations concerning the
Templates in Domain Modeling – A Survey
29
time which is needed to get the answer. If the questions become more complex, the time grows to answer it. Independent of their education however, all persons working with tables could answer the question much faster. Concerning the right answers however, there was an educational difference. People with an engineering background were much better using the graphical representation. On the other hand, people with a business background were better at using the tabular representation. Another study [4] came to the conclusion that graphical approaches are better for displaying instructions. A flow of instructions was presented in five formats: flowchart, logical tree, yes/no tree, decision table and list. Furthermore a control panel in a current state was presented. In total, the control panel was able to have eight states. The subjects who did the experiment had to find out the instruction path which leads to a certain state. This was done by pressing one of the eight button (each button represented a state) in each of the formats. According to the representation, the buttons were located at the end of a path (flow chart) beneath a leaf (decision tree) or in a row (in the decision tables). The number of correctly pressed buttons was measured. The results showed that subjects who worked with decision trees make much more errors. Furthermore, it took longer to find the solutions if decision tables and lists are used instead of the three other graphical representations (flow chart, logical tree, yes/no tree). However it also turned out that subjects who worked with a certain kind of format also preferred this format after switching to another format. The only exceptions were subjects who used lists. To summarize, the pros and cons of these different representations are still debated. The same study [9] which focused on graphs and tables for statistical information, listed in their related work many other results which reported the superiority of graphic representation and studies which reported the reverse.
4 A Historical Survey of Important Template-Based Approaches The main stream of techniques belongs to graphical methodologies. In the 70s and 80s methods like SADT, dataflow diagrams, entity relationship diagrams were used to model different aspects of a software system. The latest representative, UML covers static (class diagrams) as well as dynamic aspects (sequence diagrams, state charts, activity diagrams and use cases) of a system using graphical models. 4.1 Forms, Tables, Matrices, Glossaries Apart from graphical representations, template-based approaches were also used since the 70. Parnas [39] used tabular representations for the description of software functions. The rows and columns represented value ranges of two possible input parameters x and y. The cells described the result depending on the value range combinations of the two input parameters. In the 80s, the DATA ID approach [7] used glossaries as a central concept in their methodology. The generation of glossaries was embedded in an engineering process. Firstly the universe of discourse was divided into organizational units. A characteristic of an organizational unit is the homogenous usage of terms (i.e., no homonyms or synonyms exist in an organizational unit). Secondly users and their tasks which must be provided by the information system were identified. A matrix
30
C. Kop and H.C. Mayr
was generated which showed which user can tell information about which tasks. This matrix was the basis for interviews. The designers asked each user details about his tasks. The results of the interviews were written down as requirements statements. These requirements statements were categorized into data, operation and event statements. Finally the information in each statement was more refined and collected into entries of data, event and operation glossaries. The glossaries were the basis for traditional conceptual schemas (ER diagrams and Petri nets). KCPM adopted these glossaries as a modeling methodology. A detailed survey of KCPM is given in Section 5. In the 80s and in the 90s, object-oriented analysis and design were the buzz words of software engineering. Graphical languages like the Object Modeling Technique (OMT), Booch’s method, Jacobson’s Object Oriented Software Engineering (OOSE) became very popular. In the mid nineties, the previously mentioned languages were merged to form UML (Unified modeling language). UML once again was a graphbased modeling language (with some exceptions e.g., OCL). Though object oriented graphical languages gave good spatial overview, one deficiency was detected. What is the best granularity for a class? How large should a class be? Does an end user have the same understanding of the size and features of a class as the designer has? Do all designers share the same understanding of a certain class? It is interesting that these questions were not answered with the graphical opportunities given in OMT, the Booch method or UML. Instead Class Responsibility Cards were introduced in [3] and [51] as a mean to promote better communication between several stakeholders. For each class, an easily understandable template was generated. This template consists of the class name and a list of responsibilities. A responsibility is a short natural language description (i.e., one short sentence) of what the class offers to it’s environment. A responsibility is not a class method. It is only a general description of something that will be handled by one or more class methods. Together with these templates, guidelines were given (e.g., how many responsibilities a class should have in the best case). According to the number of responsibilities a certain class has, the stakeholders made decision if a class should be divided into more fine-grained classes or not. Although the graphical representation of use cases introduced in OOSE were adopted in UML and became very popular, there was a need to complement them with a template [10]. Cockburn added additional, very important information to use cases and also explained how use case descriptions should be applied during requirements engineering. Hence, someone interested in the detailed model could learn much more from use case descriptions than they could learn from the graphical representation of a use case. A use case description is a form that has the following items which must be filled out: Use case name, preconditions for the use case, primary actors, secondary actors, main (normal flow), alternative flows, exceptional flows, post conditions. Today use case descriptions are widely accepted as templates which store detailed information for a certain use case. The latest approach using form templates is NDT [15], [16]. NDT classifies requirements in storage, actor, functional, interaction and non functional requirements. For each of these requirement types, NDT provides a special template
Templates in Domain Modeling – A Survey
31
that has to be filled out during requirements elicitation. Particularly if there is a certain storage requirement (e.g., customer) or functional requirement (e.g., DVD rental) then a template is made for it. The two templates are typical examples of storage requirements and functional requirements, respectively. A template for functional interaction is similar to a use case description. Table 1. Template for a NDT storage requirement (SR) according to [15] SR-01 Description Specific Data
Customer Natural person who has the right to rent DVDs. Name and description Nature Customer id: identification String for the customer Name: the field stores the String customer’s name Address: the field stores the String postal address
Table 2. Template for a NDT functional requirement (FR) according to [15] FR-01 Description Actors Normal sequence
Exception
rent DVD Renting process of a DVD which is offered and executed by a clerk in the DVD rental store. Clerk AC-01: Clerk Step Action 1 The system provides possibilities to search for DVDs 2 The clerk searches for a certain DVD 3 The clerk enters the Customer ID 4 The clerk selects DVD and mark it as rented for customer with given Customer ID Step Action 2 DVD not available or all the requested DVDs are rented – restart with 1 or stop. 3 Customer ID not valid – stop the process 3 Customer is not yet a registered customer, continue with FR02 (customer registration)
4.2 Forms on the Instance Level On the instance level, forms were used as input for schema design [2]. They took forms used in offices and analyzed their structure. They identified several types of structures which can appear on a form (e.g., parametric text, structured frame, descriptive parts). In combination with their previously mentioned glossary approach (see DATA ID in Section 4.1) they proposed a design consisting of three steps. • Form analysis and area design, • Form design, • Interschema integration.
32
C. Kop and H.C. Mayr
In the first step, the concepts which appear in certain areas in the form and their relationships to other concepts are examined. This knowledge is stored in glossaries. During area design, knowledge about the form areas is taken to map each area to a corresponding conceptual schema. During the second step, the derived schemata are integrated for each form. Instead of several schemata which belong to the several areas of one form only one integrated schema for the whole form remains. The last step is another integration step. The schemata for the several forms are once more integrated to an overall first draft conceptual schema. Whereas the analysis of forms was done manually [2], other work [11] presented a computer supported tool which derives a conceptual schema from a user interface. Further works on user interfaces (forms) and their influence on database schemas were made [14] [47]. Forms and user interfaces were used as a user centered mean to query the database. 4.3 Controlled Language Sentences Though, controlled natural language now is a buzzword, the idea behind it was introduced in the 80s. Synonymous terms for controlled natural language are sentence patterns and structured language. For a better understanding of his Entity Relationship (ER) approach, Chen [8] proposed 11 heuristics to map sentences to an equivalent ER schema. He used 11 structured English sentences and showed on the basis of their structure how the elements of such a sentence can be mapped to elements of the ER schema (i.e., entity types, relationship types or attributes). Other research results (e.g., [5], [6], [30], [36], [37], [46], [48]) complemented and refined these initial results. Some of them provided automatic transformation from controlled language sentences by using parsers. In [12] the functional grammar defined in [13] was used as the linguistic basis. Dik’s verb patterns describe semantic roles which nouns can play in combination with a certain verb. The result of these studies was the Color-X model [49]. It is a graphical approach, but it is based on linguistic templates. In [19] the term controlled language is explicitly used. There a controlled language is applied for paraphrasing of ontologies. In [32] an artificial controlled language (“Orthosprache” / “Normsprache”) was developed. Natural language was the basis as well as logic. The artificial language only contains words which are needed to describe the domain (e.g., nouns and verbs). Words like prepositions and articles do not appear in this language. All notions of such a language must be well defined. For instance if the term customer is used then all stakeholders know what is meant. The same holds if another term e.g. “to order” (customer order products), is used. During requirements analysis, the main aim of the language is to support the deeper understanding of semantics of concepts (“Fachbegriffe”) which are used in a certain domain. Since controlled language avoids ambiguities, it is not only used as a basis for conceptual modeling but also during requirements engineering. It is the task of the requirements engineer to break down complex and ambiguous sentences to controlled language requirements statements [38].
Templates in Domain Modeling – A Survey
33
4.4 Controlled Language Sentences on the Instance Level A typical representative for a controlled language on the instance level is NIAM [31]. Structured language generation and analysis were the proposed procedure to get a graphical schema. Once again templates were important and were seen as a support. The idea was that the stakeholders should describe the domain with simple example facts (e.g., The student with student id XYZ visits the course with course number ABC; The course ABC can be visited by the student with student id XYZ). From the set of facts, a graphical model consisting of object types (e.g., student, course) with connected roles (student visits; course can be visited) can be generated. This model evolved and is now called ORM [21].
5 The Story of KCPM KCPM (Klagenfurt Conceptual Predesign Model) is a template-based modeling language that was initiated and developed in the Application Engineering research group at Klagenfurt University. 5.1 The Beginnings The idea to use glossaries was proposed by one of the authors according to his practical experiences [28]. The DATA ID approach, which was published a few years earlier, was introduced as an effective approach to communicate with the end users. 5.2 Research Three master theses examined different aspects of the DATA ID approach. One focused [17] on the transformation rules between glossaries and natural language sentences. Another thesis [40] extended the model of the DATA ID by improving the concept of event type (later called co operation type). The last one [23] focused on the structural and functional aspects (thing types, operation types). The DATA ID data glossaries were refined to thing glossaries (later called thing type glossaries). Also the DATA ID operation glossary was refined (later called operation type glossary). Since the representation was intended to be applied to any modeling concept of the new approach, also a connection (type) glossary was introduced. This master thesis also combined the glossary representation approach of DATAID with the fact-oriented approach used in NIAM. Therefore an algorithm was developed which maps thing types and connection types into entity types, value types, attributes and relationships of an entity relationship model. During that thesis also the name and acronym (KCPM) was born: KCPM = Klagenfurt Conceptual Predesign Model. With this name it is pointed out that the approach must be applied in between requirements analysis and conceptual modeling. It is a model that helps the software engineer to ask the right questions during requirements engineering but also has the advantage that the conceptual model can be generated easily from these working results. It is “conceptual” since there must be an agreement upon the language notions. Since it supports the development of the final conceptual design model it was called a predesign model.
34
C. Kop and H.C. Mayr
A few years later a PhD thesis [24] started with the aim to integrate the results of the first research studies. The outcome was a first, core, lean model based on a small set of notions namely: • Thing types (concepts): Notions/Object types which are important for the domain. • Connection types: Relationships between thing types. • Operation types: Services provided by the future system or components of the system. • Cooperation types with pre- and post conditions: Behavior that triggers operation types. Another aim of the PhD thesis was to build bridges between KCPM and natural language sentences. Therefore, KCPM was embedded into the NIBA2 project. NIBA was a German acronym for natural language based requirements analysis. Computer linguists developed a grammar model for natural languages sentences called NTMS (Naturalness Theoretical Morphosyntax) [18], [27]. The NTMS was taken as the basis to examine the relationships between KCPM glossaries and natural language text. There is still ongoing research on the KCPM topic. Meanwhile there is a PhD project that examines the possibilities of schema integration using the modeling notions of KCPM [50]. Another PhD project focuses on the questions of user centered visualization of web services. Template-based visualization is seen as one good possibility [20]. There are also research connections to other research groups. In 2006 a study started with the aim to map between KCPM thing types and connection types to CBEADS smart business objects [26]. Two master thesis [29],[52] focused on different mapping aspects from KCPM to the OLIVANOVA conceptual model [33], [34] and [35]. Another master thesis [41] analyzed the usage of thing types and connection types for teaching modeling in informatics education at a high school. There is ongoing research with the University of Kharkiv on aspectual and qualitative predesign [22],[42]. Furthermore KCPM is adopted as an ontology representation in software engineering [1] in a joint project with the University of Marburg/Lahn. 5.3 Practical Studies The research was also complemented by practical studies. After the first master theses on this topic were finished, a student working in a local publishing company used the approach to collect their requirements. It was demonstrated, that a domain expert could validate the requirements very well using the collected information in the glossaries. In another master thesis [45], a further practice study with KCPM was done. The primary goal of this master thesis was to investigate possible usage of KCPM in the domain of business process modeling. Since the student already worked in a software development enterprise which needed this information, the student was told to ask the involved stakeholders in which situations glossaries can be preferred over graphical representations. This study pointed out that glossaries are very well understood and preferred in situations where a listing is necessary, or information can be presented as check lists. Hence thing type glossaries, connection type glossaries as well as 2
NIBA was funded by the Klaus Tschira Stiftung Heidelberg.
Templates in Domain Modeling – A Survey
35
operation type glossaries were seen as the best candidates for a glossary representation (i.e., list of terms = thing type glossary, list of relationships = connection type glossaries, list of services of the system = operation type glossaries). The interviewed persons only had problems with cooperation type glossaries because of their complexity. A cooperation type glossary contains the set of operation types together with their preconditions and postconditions. The persons argued that a graphical representation is more suitable. This study was a very interesting hint for further research. Other practical experiences were gathered in a medical domain. The content of this requirements engineering project were data driven. New software was necessary to manage the data, to get statistics from the database and for decision support of the users. We found that thing type and operation type glossaries could be applied very successfully for the collection of all the requirements. To summarize, parts of the KCPM approach could be applied in several practical projects. The results were twofold. On the one hand it could be shown that there is a need for template approaches. On the other hand, the feedback and experiences from the practical studies were used to further improve the approach. 5.4 Beyond Glossaries – Current State of KCPM The arguments for glossaries and graphical approaches in Section 2 and the practical studies of Section 5.3 showed that glossaries are important but graphical approaches must always be considered. It would be a mistake to ignore graphical approaches. The reasons are simple: • The different skills of different users must be always considered. Some of them like templates and others like graphical approaches. • The situation (purpose) is always changing in a typical requirements engineering project. At one point in time, the requirements engineer must act like a medical doctor extracting the causes from the symptoms of the patients (end user). In the very next moment, he must give an overview to the end user or he needs the overview for his own better understanding. Because of these reasons, in one study [25] the main research goal was not to discuss why and in which situation glossaries are better rather to think of how combining different kinds of representations. Particularly: How could a graphical representation be combined with template representations? The conclusion was: Graphical and template-based representations must be seen as equally valuable views within in a toolset.
6 Conclusion and Future Vision 6.1 Conclusion A recent paper [43] describes how to switch among different aspects of a model (e.g., the dynamic aspects, static aspects etc.) within a meta-model. However, this was presented once again for a graphical representation. Particularly, based on the metamodel a tool provides a view with a class diagram and a view with the use case
36
C. Kop and H.C. Mayr
diagram, the meta-model guarantees consistency between the views. In other words, a use case is not longer seen as an independent model within UML where relationships to classes or other UML concepts are not checked. The idea to switch between different aspects of a model consistently must be combined with different representation techniques. It must not only be possible to switch between classes and use cases but also to switch between a graphic representation and a template-based representation. 6.2 Templates in SW Development – Future Vision In order to describe our view of the future, firstly we comment on the past - in particular something on the evolution of programming languages. At the beginning of software development, assembler languages were used to program the computer. In these assembler languages important and very commonly used combinations of low level operations were clustered into macros. However these languages had the disadvantage that they were close to the machine. The developers had to keep in mind to which memory address and registers they have to store values. Third generation languages like Cobol and Fortran were the first step towards a more human readable programming language. These programming languages were used in specific domains. Cobol was developed and used for implementing business solutions. Fortran was mainly used in mathematical domains. During those days scientists also tried to understand what is essential for good programming. The solutions were programming languages for structured programming (e.g., ALGOL, Pascal etc.). Further style guides (e.g., how to decompose large software into units that can communicate with each other without side effects) lead to further evolutions of 3rd generation programming languages to module and object based languages and then to object oriented languages (e.g., Modula-2, Modula-3, Smalltalk, C++, Java etc.). The idea to make programming languages more user-understandable was also realized in the 4th and 5th generation languages. These languages were developed for certain kind of applications (e.g., SQL for database queries, LISP and Prolog mainly for problemsolving situations). These languages were more human readable since they focused on the “WHAT” and not on the “HOW”. Using these languages the user had to specify WHAT he needed and not how it should be executed. This was once again achieved by hiding some internals (e.g., in SQL the user need not know the access paths to records in a database; in Prolog he need not know the technical details about backtracking but can rely on the Prolog interpreter to fire rules based on the available facts). If we summarize this, then it can be learn learned that during the evolution of programming languages, complexity was hidden and style guides as well as patterns were introduced. During the evolution of 3rd generation programming languages the goal always was to “transform” 3rd generation language code to efficient machine readable code in a 2nd or even 1st generation programming language. With model driven architecture (MDA), 3rd generation programming languages became the final target of transformation processes. Nowadays, MDA is based on the idea that every platform independent model (PIM) will be extended with platform specific features. Doing this, the PIM is transformed to a platform specific model (PSM) which itself can be the PIM for the next transformation step. What is done now in MDA is an
Templates in Domain Modeling – A Survey
37
evolution like the evolution of program languages in the past. Scientists came to an overall understanding about good programming. Therefore programming languages are now the target of transformation. Model driven architecture can be also seen as a next step to make modeling more human understandable. During the evolution of the programming languages, the main focus was to exempt the languages from machinespecific pieces of code. During model driven architecture one of the ideas is that pictures often represent more than 1000 lines of code. Once again certain kinds of macros are introduced. Classes can be specified with its attributes but there is no need to specify their constructors and their get- and set-methods. These methods are automatically derived from the class specifications. Combining this information with the knowledge about templates described in this paper, one vision of the future might be the following. Scientists will come to a better understanding about model driven architectures, based on style guides, design and architectural patterns. They will improve the quality of model driven architecture. At the end we will get a common understanding what a PIM needs in order to be of good quality. Having this, the question will not be any longer to get executable software or a completed source code according to a certain specification. Instead the focus will be on how to get a specification (the first PIM) from requirements. Templates will support this new construction idea. Instead of trying to get a graphical model, requirements engineers will behave like medical doctors to collect all the necessary information and to generate the target graphical model. Such a working step will focus on extracting structured requirements specifications from unstructured requirements specifications. Templates will play an important role within this step. As an intermediate result, for quality checking the stakeholders will mainly work with • • •
template (e.g., glossary) entries and only if really necessary with a graphical model that represents either the final PIM or an intermediate version.
On any of these intermediate results the stakeholders will be able to make corrections. End user and designer together will be able to check the templates. This will also be a further step towards human readability of models. The human reader is now of another kind. It is no longer a technically experienced user but also an application domain expert with little technical knowledge. Thus, in the future modeling a software system will be like going through a specific checklist for this software.
7 Summary This chapter presented a survey on template-based conceptual modeling approaches. Historical approaches were presented. Although some of them are mainly known as graphical modeling techniques, they are based on templates (e.g., forms, glossaries and in most cases linguistic templates). The aim of this chapter was to create awareness, that the usage of graphical representation is not always the best solution. It
38
C. Kop and H.C. Mayr
strongly depends on the stakeholder’s skills and situation. In certain situations, a template is better suited than a graphical representation. However the best usage of graphical and template-based techniques is always a situation depended combination of these techniques. Acknowledgments. The authors would like to thank Dr. h.c. Klaus Tschira for his support of the NIBA project. Without this support the extensive research on this interesting topic would not have been possible. Furthermore, we thank all the colleagues and students which have worked in the area of conceptual predesign. Finally we would thank the reviewers for their helpful hints and comments with which it was possible to improve this chapter.
References 1. Bachmann, A., Russ, A., Vöhringer, J., Hesse, W., Mayr, H.C., Kop, C.: OBSE - an Approach to Ontology-based Software Engineering in the Practice. In: Reichert, M., Strecker, S., Turowski, K. (eds.) Proceedings of the 2nd International Workshop on Enterprise Modeling and Information Systems Architectures. GI Lecture Notes in Informatics (LNI), vol. 119, pp. 129–142. Köllen Verlag (2007) 2. Batini, C., Demo, B., Di Leva, A.: A Methodology for conceptual design of offices data bases. Information Systems 9(2-3), 251–264 (1984) 3. Beck, K., Cunningham, W.: A Laboratory For Teaching Object-Oriented Thinking. In: Conference Proceedings on Object-oriented programming systems, languages and applications, pp. 1–6. ACM Press, New York (1989) 4. Boekelder, A., Steehouder, M.: Selecting and Switching: Some Advantages of Diagrams Over Tables and Lists for Presenting Instructions. IEEE Transaction on Professional Communication 41(4), 229–241 (1998) 5. Buchholz, E., Cyriaks, H., Düsterhöft, A., Mehlan, H., Thalheim, B.: Applying a Natural Language Dialog Tool for Designing Databases. In: Proc. International Workshop on Applications of Natural Language to Databases (NLDB 1995), pp. 119–133 (1995) 6. Buchholz, E., Düsterhöft, A., Thalheim, B.: Capturing Information on Behaviour with the RADD-NLI: A Linguistic and Knowledge Based Approach. In: Riet, v.d., Burg, R.P., Vos, A.J. (eds.) Proceedings of the 2nd Int. Workshop on Applications of Natural Language to Information Systems (NLDB 1996), pp. 185–192. IOS Press, Amsterdam (1996) 7. Ceri, S. (ed.): Methodology and Tools for Database Design. North Holland Publ. Comp., Amsterdam (1983) 8. Chen, P.P.: English Sentence Structure and Entity Relationship Diagrams. Int. Journal of Information Sciences 29, 127–149 (1983) 9. Coll, R.A., Coll, J.H., Thakur, G.: Graphs and Tables a Four-Factor Experiment. Communications of the ACM 37(4), 77–86 (1994) 10. Cockburn, A.: Writing Effective Use Cases. Addison Wesley Publ. Comp., Reading (2000) 11. Choobineh, J., Mannino, M.V., Tseng, V.P.: A form-based approach for Database Analysis and Design. Communication of the ACM 35(2), 108–120 (1992) 12. Dignum, F., Kemme, F., Kreuzen, W., Weigand, H., van de Riet, R.P.: Constraint modelling using a conceptual prototyping language. Data & Knowledge Engineerng 2, 213–254 (1987)
Templates in Domain Modeling – A Survey
39
13. Dik, S.: Functional Grammar. North Holland Publ. Company, Amsterdam (1978) 14. Embley, D.W.: NFQL: The Natural Forms Query Language. ACM Transactions on Database Systems 14(2), 168–211 (1989) 15. Escalona, M.J., Reina, A.M., Torres, J., Mejías, M.: NDT a methodology to deal with the navigation aspect at the requirements phase. In: OOPSLA Workshop: Aspect-Oriented Requirements Engineering and Architecture Design (2004) 16. Escalona, M.J., Koch, N.: Metamodeling the Requirements of Web Systems. In: Proceedings of the 2nd Int. Conf. Web Information Systems and Technologies (WebIST 2006). Lection Notes in Business Information Processing (LNBIP), vol. 1, pp. 267–280. Springer, Heidelberg (2006) 17. Felderer, A.: Zur Tabellarisierung natürlichsprachlicher Anforderungsbeschreibungen. Diplomathesis, Universität Klagenfurt (1992) 18. Fliedl, G.: Natürlichkeitstheoretische Morphosyntax – Aspekte der Theorie und Implementierung. Gunter Narr Verlag, Tübingen (1999) 19. Fuchs, N.E., Höfler, S., Kaljurand, K., Rinaldi, F., Schneider, G.: Attempto Controlled English: A Knowledge Representation Language Readable by Humans and Machines. In: Eisinger, N., Maluszynski, J. (eds.) Reasoning Web. LNCS, vol. 3564, pp. 213–250. Springer, Heidelberg (2005) 20. Gälle, D., Kop, C., Mayr, H.C.: A Uniform Web Service Description Representation for Different Readers. In: Berntzen, L., Smedberg, A. (eds.) Proceedings of the second International Conference on the Digital Society (ICDS 2008), pp. 123–128 (2008) 21. Halpin, T., Bloesch, A.: Data modelling in UML and ORM: a comparison. Journal of Database Management 10(4), 4–13 (1999) 22. Kaschek, R., Kop, C., Shekhovtsov, V.A., Mayr, H.C.: Towards simulation-based quality requirements elicitation: A position paper. In: Rolland, C. (ed.) REFSQ 2008. LNCS, vol. 5025, pp. 135–140. Springer, Heidelberg (2008) 23. Kop, C.: Herleitung von EERM+ Schemata aus Zusammenhangsverzeichnissen, erweiterten Ding- und Operationsverzeichnissen. Diplomathesis, Universität Klagenfurt (1993) 24. Kop, C.: Rechnergestützte Katalogisierung von Anforderungsspezifikationen und deren Transformation in ein konzeptuelles Modell. Doctoral thesis, Universität Klagenfurt (2002) 25. Kop, C.: Visualizing Conceptual Schemas with their Sources and Progress. International Journal on Advances in Software 2(2,3), 245–258 (2009), http://www.iariajournals.org/software/ (last access September 27, 2010) 26. Liang, X., Ginige, A.: Smart Business Object - A New Approach to Model Business Objects for Web Applications. In: ICSOFT 2006, pp. 30–39 (2006) 27. Mayerthaler, W., Fiedl, G., Winkler, C.: Lexikon der Natürlichskeitstheoretischen Morphosyntax. Stauffenburg Verlag, Tübingen (1998) 28. Mayr, H.C., Dittrich, K.R., Lockemann, P.C.: Datenbankentwurf. In: Lockeman, P.C., Schmidt, J.W. (eds.) Datenbank-Handbuch, pp. 486–552. Springer, Heidelberg (1987) 29. Michael, J.: Connecting the dynamic part of KCPM with the OlivaNova Modeler, Masterthesis, Universität Klagenfurt (2010) 30. Moreno, A., van de Riet, R.P.: Justification of the equivalence between Linguistic and Conceptual Patterns for the Object Model. In: Proc. 3rd Int. Workshop on Application of Natural Language to Information Systems, pp. 57–74 (1997) 31. Nijssen, G.M., Halpin, T.: Conceptual Schema and Relational Database Design – A fact oriented approach. Prentice Hall Publ. Comp., Englewood Cliffs (1989)
40
C. Kop and H.C. Mayr
32. Ortner, E.: Methodenneutraler Fachentwurf. B.G. Teubner Verlagsgesellschaft, Stuttgart, Leibzig (1997) 33. Pastor, O., Molina, J.C., Iborra, E.: Automated Production of Fully Functional Applications with OlivaNova Model Execution. ERCIM News (57), 62–64 (2004) 34. Pastor, O., Gomez, J., Insfran, E., Pelechano, V.: The OO-method approach for information systems modeling: from object-oriented conceptual modeling to automated programming. Information Systems 26(7), 507–534 (2001) 35. Pelechano, V., Pastor, O., Insfran, E.: Automated code generation of dynamic specializations: an approach based on design patterns and formal techniques. Data and Knowledge Engineering 40(3), 315–353 (2002) 36. Rolland, C., Ben Achour, C.: Guiding the Construction of textual use case specifications. Data & Knowledge Engineering Journal 25(1-2), 125–160 (1998) 37. Rolland, C.: An Information System Methodology Supported by an Expert Design Tool. In: Pirow, P.C., Duffy, N.M., Ford, J.C. (eds.) Proceedings of the IFIP TC8 International Symposium on Information Systems, pp. 189–201. North Holland Publ. Company, Amsterdam (1987) 38. Rupp, C.: Requirements Engineering und Management. Hanser Verlag (2004) 39. Ryszard, J., Parnas, D.L., Zucker, J.: Tabular Representations in Relational Documents. In: Hoffman, D., Weiss, D.M. (eds.) Software Fundamentials – Collected Papers by David Parnas, pp. 71–85. Addison Wesley Publishing Comp., Reading (2001) 40. Schnattler, M.: Herleitung von Ereignisschemata aus erweiterten Operations- und Ereignisverzeichnissen. Diploma thesis, Universität Klagenfurt (1992) 41. Schein, M.: Moderne Modellierungskonzepte der Informatik in der Schulpraxis, Diplomathesis, Universität Klagenfurt (2010) 42. Shekhovtsov, V., Kostanyan, A., Gritskov, E., Litvinenko, Y.: Tool Supported Aspectual Predesign. In: Karagianis, D., Mayr, H.C. (eds.) 5th International Conference on Information Systems Technology and its Applications (ISTA 2006). LNI, vol. P-84, pp. 153–165. Köllen Verlag (2006) 43. Sinz, E.: Tool-Unterstützung für die SOM-Methodik: Anforderungen und Lösungsstrategien. In: Presentation slides: 1st International Open Models Workshop, Klagenfurt (March 2010) 44. Stålhane, T., Sindre, G., du Bousquet, L.: Comparing Safety Analysis Based on Sequence Diagrams and Textual Use Cases. In: Pernici, B. (ed.) CAiSE 2010. LNCS, vol. 6051, pp. 165–179. Springer, Heidelberg (2010) 45. Stark, M.: Geschäftsprozessmodellierung im konzeptuellen Vorentwurf, Diplomathesis, Univeristät Klagenfurt (2000) 46. Tjoa, A.M., Berger, L.: Transformation of Requirement Specification Expressed in Natural Language into an EER Model. In: Elmasri, R.A., Kouramajian, B., Thalheim, B. (eds.) Proc. 12th International Conference on Entity Relationship Approach, pp. 127–149. Springer, New York (1991) 47. Terwillinger, J.F., Delcambre, L.M., Logan, J.: Queyring through a user interface. Data & Knowledge Engineering 63, 774–794 (2007) 48. Vadera, S., Meziane, V.: From English to Formal Specifications. The Computer Journal 37(9), 753–763 (1994) 49. Van de Riet, R.: Mokum for Correctness by Design in Relation to MDA, In: Kaschek R., Kop, Ch., Steinberger, C., Fliedl, G. (eds.) Information Systems and e-Business Technologies. Lecture Notes in Business Information Processing (LNBIP), vol. 5, pp. 352–364 (2008)
Templates in Domain Modeling – A Survey
41
50. Vöhringer, J., Mayr, H.C.: Integration of schemas on the pre-design level using the KCPM-approach. In: Nilsson, A.G., Gustas, R., Wojtkowski, W.G., Wojtkowski, W., Wrycza, S., Zupancic, J. (eds.) Advances in Information Systems Bridging the Gap between Academia & Industry, pp. 623–634. Springer, Heidelberg (2006) 51. Wirfs-Brock, R., Wilkerson, B.: Object-oriented design: a responsibility-driven approach. In: Conference proceedings on Object Oriented Programming Systems Languages and Applications, pp. 71–75. ACM Press, New York (1989) 52. Yevdoshenko, N.: A Proposal for Model Driven Development Life Cycle: From KCPM Requirements Specifications to OLIVANOVA Conceptual Model, Master Thesis, Universität Klagenfurt (2006)
Integrity Constraints in (Conceptual) Database Models Bernhard Thalheim Department of Computer Science and Applied Mathematics Christian Albrechts University Kiel, Olshausenstr. 40, 24098 Kiel, Germany
[email protected] Abstract. The theory of integrity constraints has led to a large body of knowledge and to many applications such as optimisation of schema behaviour through normalisation to schemata which are easier to maintain, as automatic enforcement of database integrity by triggering or other procedural techniques, as quality improvement of data in databases, and as query recompilations due to the knowledge of validity of constraints in a database. Integrity constraints are often misunderstood, are given in the wrong database context or within the wrong database models, often combine a number of very different facets of semantics in databases and are difficult to specify. At the same time, pearls and hidden treasure of the theory of integrity constraints have been already forgotten. We develop a unifying approach to specification and treatment of integrity constraints. We survey some pearls, nuggets and lessons learned with semantic models.
1 Revisiting Database Constraint Theory Structures and constraints are intertwined and coexisting fundamental concepts in modern database systems. Databases must contain trusted data that are correct and coherent. User want to trust that right answers are given to their queries. The golden rule must be thus that no modification operation to data in a database can cause conflicts with constraints. 1.1 Omnipresence of Database Constraints Database constraints have been introduced for an explicit definition of semantics. Typically a database type is given by a structure and a set of integrity constraints that restrict classes for a type to those for which the constraints are valid. This type theory is a specific form of abstract data type theory. Modification and retrieval operations are assumed to be canonically given for any database type. Similarly, a database schema is specified by database types and schema integrity constraints. Only canonical operations are given. Therefore, integrity constraints play a prominent role during database specification. Constraints also allow one to overcome limitations of the modelling language. For instance, the relational model is a model of low expressivity and thus enhanced by a very large set of different (> 95) integrity constraint classes. Semantical modelling languages such as object-relational (or entity-relationship) languages allow to express some of these constraints through constructs. At present we know at least five application fields of database constraints theory: (1) normalization for a more efficient storage, search and modification; (2) reduction of relations to subsets with the same information together with the semantic constraints; Kaschek/Delcambre (Eds.): The Evolution of Conceptual Modeling, LNCS 6520, pp. 42–67, 2011. c Springer-Verlag Berlin Heidelberg 2011
Integrity Constraints in (Conceptual) Database Models
43
(3) utilization of dependencies for deriving new relations from basic relations in the view concept or in so-called deductive databases; (4) verification of dependencies for a more powerful and user-friendly, nearly natural language design of databases; (5) transformation of queries into more efficient search strategies. A large number of structural and dynamical database constraints have been introduced in the past. We must however acknowledge that a fully fledged theory of database constraints is not yet existing. 1.2 Separation of Integrity Constraints by Their Use and Usage There are several classifications for integrity constraints: – either utilization characteristics are used for classification into domain constraints, key and functional dependencies, referential integrity constraints, join dependencies etc. – or their specific format of the formulas is used for classification into tuple-generating dependencies, equality-generating dependencies, existence constraints, single-table constraints, singleton-tuple constraints, etc. These characterizations are useful whenever constraints are formally defined. Their practical utility is, however, more restricted. Another characterization approach has been used in [Tha00] by relating constraints to the phase of database modelling into design, structural, semantic and representational constraints. We may combine the three approaches by clustering constraints according to their structural properties into – constraints expressing identification or partial identification of values by other values, – constraints stating relative independence of values in a class or within an object, – constraints stating existence (or non-existence) of values in an object, or values in groups of objects, or objects in a class, and – constraints expressing redundancy of values or objects. At the same time we may distinguish constraints according to their utilization in the design process. They might be meaningful at the level of the user, or at the level of the conceptual schema or at the level of the implementation. The following table shows this characterization. Partial identifi- Relative independence Existence depen- Redundancy depencation dency dency identification structure no null elementary facts
Business user level Conceptual functional, multivalued, hierarchilevel equality gener- cal, join dependencies, ating exclusion dependency, tuple generating, horizontal decomposition Implemen- key, uniqueness, decomposition, stored tation level trigger, check procedures, trigger
null-value-free, inclusion constraint, union constraints, exclusion constraint numerical, cardinality constraint no null, stored pro- referential integrity, cedures, trigger surrogate, container
44
B. Thalheim
1.3 Quality Criteria for Constraint Sets Database systems aim in automatic support of quality. There are a number of quality criteria that have classically been considered in many textbooks and papers. Structural quality criteria are structural completeness, satisfiability of the schema, liveness of the database, applicability of automatic control features, explicit exception handling, applicability of operations, executability of operations and framing consistency procedures. The first three conditions are well discussed in the database literature. Automatically generated tests and control conditions are still an open research field. Operations are currently mainly applied based on the transaction approach, i.e., forcing a rollback after problems have been detected. Exception handling and execution control use the same approach. The framing or ramification problem is not yet solved. It requires a separation within a database into data that are not affected by a change and into data that are under potential change. A typical example of non-framed executions are trigger avalanches. Quality control must also consider the abstraction level of the stakeholder involved. Integrity constraints may be ambiguous or may be based on context or ellipses. We therefore need an explicit statement of the abstraction level. For instance, join dependencies are a specific vehicle for structuring the database. They are not used by the requirements engineer. There are however specification constraints at the requirements level that must be mapped to the internal levels. 1.4 Optimisation of Behaviour through Normalisation of Database Structuring Normalisation has been developed as a vehicle for performance improvement of database systems. It addresses at least seven different targets: (A) Redundancy becomes problematic whenever additional actions are required for consistency management of data that are stored within different objects. (B) Blocking of management due to the information capacity of the schema. For instance, the insertion anomaly occurs since units of storage such as schema types do not support insertion of partial information. (C) Information loss after database modification occurs whenever data are eagerly deleted despite the importance of parts of it. The deletion anomaly is observed whenever facts are deleted together with the objects where they are contained despite its importance for the application. (D) Evolution sensitivity and instability of the database whenever changes are applied to the database. (E) Different abstractions are used for the database schema at the same time. For instance, views, derived attributes, logs are stored together with the basic data that are used to derive these values. (F) Performance problems can also be solved through restructuring. Typical performance problems considered are caused by integrity constraint maintenance. Update anomalies have been considered as a prominent example of a performance problem since singleton fact operations resulted in complex bulk operations. Performance problems are however also caused by architectures chosen for the application, by specific behaviour of the application, by retrieval requirements, by generation and maintenance of supporting structures such as indexes, etc. The last set of performance problems is often resolved by denormalisation, i.e., by intentional
Integrity Constraints in (Conceptual) Database Models
45
acceptance of another normalisation. Denormalisation may decrease complexity of retrieval and maintenance operations, may avoid additional join operations and may prepare special derived data for support of repeating computations. It allows us to consider semantically meaningful units instead of normalised structures. Index management is optimised. Denormalisation increases however complexity of some database operations, leads to redundancy problems, may result in inflexibility against evolutionary changes. (G) Expensive maintenance, operating and modification of databases often occurs due to consistency maintenance. Parallel execution of transactions may result in deadlocks. As far as we know there is not yet any theory that integrates the six targets of normalisation. Moreover, (A), (C) and (G) are considered to be the primary issues. 1.5 Outline of the Paper The paper aims to demonstrate the power of constraint theory for advanced database models. There is no textbook that provides a modern view onto the achievements of this theory. Many very useful results are forgotten, neglected or suppressed. We may claim that this state leads to new database models that ignore the achievements of the past and require the reinvention of the wheel or result in less elegant model languages. Some of the difficulties with constraints are based on their limitations in expressivity, are based on their restricted binding to a class, are due to their combinatorial complexity, are due to their partial axiomatisability, or are due to their practical treatment and enforcement. We start first with a systematisation of constraint handling in Section 2. It is surprising that some of the main problems of database theory remains open after almost two decades of research [Tha87]. Some few of them are reviewed in Section 3. At the same time, database theory has also resulted in jewels and background knowledge that here become folklore. We review some of the background results in Section 4 and apply them in Section 5. Since this paper does not intend to survey all papers in the area we refer the reader to [Tha00] for all definitions of notions.
2 Systematic Constraints Deployment In current practice constraints are declared at the conceptual level. These constraints are transformed to constraints at the logical level and then enhanced by control declarations or mapped to procedural representation. The enforcement and the application are left to the DBMS ... and result in a good number of problems. Moreover, structures and constraints are not handled in a holistic manner. We claim that this approach can be improved by an explicit framework and by a deeper consideration of concerns. 2.1 The Constraints Handling Framework Our framework for constraint handling is based on a four level model: 1. The specification level is used for description of the constraints with their schema context. The description consists of a specification of the constraint property, the
46
B. Thalheim
validation, and the policies for evaluation. It can be extended by specific policies for the various development methods such as agile development, by transformations of constraint properties to others, and by associations among constraints. Finally we may derive constraints for the application of the constraint. 2. The control or technical level treats the application of the constraint. It is based on the constraint property portfolio. We develop techniques and methods for applying constraint checks and derive the constraint evaluation plan. 3. The application or technology level handles the management of constraint handling within software projects based on the technology of development. 4. The establishment or organisational level is based on a methodology and may be supported by a constraint maintenance system. Classical research has mainly considered the first level. Implementation approaches have been tackling the second level by explicit specification of policies such as cascade, restrict, no action, defaultify and nullify. Level three is typically handled in an implicit form by some coherence among triggers and transactions. This four-level framework for constraint management can be extended by level five that provides facilities for handling satisfaction of constraints and for predicting changes of satisfaction whenever software is evolving. Level six integrates the constraint management into the optimisation of the software development process. Level seven uses experiences gained for innovation and adaptation of other processes and products that have not yet reached this maturity. Constraint can be oriented towards products of the database and software development or towards the processes of the database and software development. This distinction is reflected by a distinction into product-oriented constraints and process-oriented constraints. The constraints of the first kind are static one, the constraints of the second kind are transitional or dynamic ones. The second kind must be applied to basic or generic activities used within development processes. Most of these processes are refinement steps. Therefore, we may distinguish between faithful (or constraintpreserving) refinement activities and those which must be extended by constraint maintenance or support activities. 2.2 Implicit Model-Inherent Integrity Constraints Each structure used in a database schema is also based on a set of implicit modelinherent integrity constraints: Component-construction constraints are based on existence, cardinality and inclusion of components. These constraints must be considered in the translation and implication process. Identification constraints are implicitly used for the set constructor. Each object either does not belong to a set or belongs only once to the set. Sets are based on simple generic functions. The identification property may be, however, only representable through automorphism groups [BT99]. We shall later see that value-representability or weak-value representability lead to controllable structuring.
Integrity Constraints in (Conceptual) Database Models
47
Acyclicity and finiteness of structuring supports axiomatisation and definition of the algebra. It must, however, be explicitly specified. Constraints such as cardinality constraints may be based on potentially infinite cycles. Superficial structuring leads to representation of constraints through structures. In this case, implication of constraints is difficult to characterize. 2.3 Separation of Concerns Let us discuss three overlooked issues for constraints: (1) Some constraints may combine too many different meanings at the same time. This overburdening limits reasoning on constraint sets. (2) Constraints might have very different meaning for different stakeholders of an application. These different meanings may not be harmonically given within a singleton constraint. (3) Finally, some constraints are much better adopted to some database model languages and unnatural for others. Kinds of Functional Dependencies. Functional dependencies are the most well-known class of database constraints and commonly accepted. They are one of the most important class of equality-generating constraints. Given a type R and substructures X, Y of R. The functional dependency R : X −→ Y is valid in RC if o|Y = o |Y whenever o|X = o |X for any two objects o, o from RC . The following axiomatization is correct and complete for functional dependencies in the EER [HHLS03] for substructures X, Y, Z of the EER type R: Axioms: Rules:
R : X R Y −→ Y R : X −→ Y R : X −→ X R Y
R : X −→ Y, R : Y −→ Z R : X −→ Z
R : X −→ Y R : R [X] −→ R [Y ] The type R denotes a component type of R if R is a relationship type of order i and R is of order i − 1. Functional dependencies carry at least four different and interwoven meanings. The notion of the functional dependency is thus overloaded. It combines different properties that should be separated: Explicit declaration of partial identification: Functional dependencies are typically explicitly declaring a functional association among components of types. The left Ident hand attributes uniquely identify right side attributes, i.e., X −→ Y . Identification can either be based on surrogate or on natural attributes. Tight functional coupling: Functional dependencies may also be numerical constraints [Tha00]. Numerical constraints restrict the number of Y -values for each X-value. N um We denote such constraints by i.e., X −→ Y . Another denotation is based on cardinality constraints. Semantic constraint specific for the given application: Constraints may be stronger than observed in usual life since the application has a limited scope and allows us to strengthen the constraint. In this case, constraints restrict the application only to those
48
B. Thalheim
cases in which the left side has only one associated right side value despite that this Sem restriction may not be valid for any application. We denote this case by X −→ Y . Semantical unit with functional coupling: Semantical units are those reducts of a type that are essential in the given application. Their components cannot be separated without loosing their meaning. Semantical units may have their inner structure. This structure tightly couples dependent object parts to those that are Unit determining them. We denote this coupling by X −→ Y . Structural association among units: Semantical units may allow a separation of concerns for certain elements. Their separation supports a more flexible treatment while requiring that the dependent part cannot exist without the determining part. If Struct this dependence is functional we may represent such by the constraint X −→ Y . Example 1. Let us consider an abstract example with the functional dependencies {A} −→ {B, C} and {B} −→ {D}. TA A
C
(1,1)
TA .to.TB
B
TB D
The functional dependency {A} −→ {C} declares an explicit and direct dependency among the attributes. The functional dependency {A} −→ {B} is an inter-type constraint and leaves the scope of type TA . These two dependencies are completely different and need different support mechanisms. The five cases may be observed for the following instantiations of this example:
Explicit declaration Tight coupling Semantic constraint Semantical unit Structural association
instantiation TB = StudyProgram , B = ProgramCode, D = ProgramName TA = Student, TA .to.TB = MajorProgram, TB = StudyProgram TB = StudyProgram , B = ProgramCode, D = ResponsibleProfessor TB = StudyProgram , B = ProgramCode, D = ProgramDegree TA = Student, TB = RegisteredStudent, A = PersonID, B = StudentNr
Let us consider, for instance, that the type TB represents a study program, and the attributes B represent a program code, D the program name. The functional dependency {B} −→ {D} is an explicit key dependency. If the attribute D represents the professors that are responsible program then we might assume or not assume {B} −→ {D}. This depends on the given application. Therefore the dependency is a semantical constraint. From the other side, if the attribute D represents the program degree then this degree cannot be separated from the values for the program code. In this case the attributes form a semantical unit. Furthermore, in most universities a student is registered for one and only one major degree. The student must have a major program. In this case, the two types Student and StudyProgram are thus tightly coupled. Finally, we can separate the student personal data from the student university data. The person identification number is structurally associated to the student number.
Integrity Constraints in (Conceptual) Database Models
49
This separation results also in a different meaning of derivation rules. For instance, the augmentation rule has a different meaning depending on the kind of functional dependency: Trivilisation of identification
Ident
R : X −→ Y R Y Ident
R : X R X −→ Y
Adaptation of semantics scope
Sem
R : X −→ Y R Y Sem
R : X R X −→ Y
We may also use the Armstrong axiomatisation for derivation of functional dependencies of a given kind. The following two rules are examples of refined rules: Ident
Sem
R : X −→ Y, R : Y −→ Z Ident
R : X −→ Z
Sem
Ident
R : X −→ Y, R : Y −→ Z Sem
R : X −→ Z
Kinds of Key Dependencies. A key dependency or simply key X is a functional dependency R : X −→ R . A key is called minimal if none of its proper substructures forms a key. The set of all minimal keys of R is denoted by Keys(R). We notice that this set type E with n atomic attributes may may be very large. For instance, an entity n have nn minimal keys which is roughly c2√n . 2 The key concept reflects a variety of descriptive meanings depending of the level of consideration. Let us distinguish between external, conceptual and internal levels. The meaning of the key concept can be based on the uniqueness property (each object is unique with this key in the database), the identification property (each object can be identified by its key values), the existence property (for each object there exist key values), the integrity property (the database satisfies a key constraint), and the accessibility property (the properties of an object can be accessed through the key values of the object). These meaning are not equally important at all levels: – At the language definition level, the uniqueness concept is used to express the key property. The identifier, existence, integrity and accessibility functionalities are not considered. – The uniqueness concept is inherited by the external level and expressed mainly via the identification property. The existence property plays a secondary role. The integrity and accessibility concepts are not mainly under consideration. – At the conceptual level, the uniqueness concept of the language definition level is inherited by the identification and the integrity property. In order to express the identification concept, the surrogate concept is introduced. The key dependency is the integrity constraint usually used to express the key property. – At the internal level, the uniqueness concept is completely covered by the identification concept. From the external level the existence and uniqueness concepts are inherited and used in implementation restrictions like the requirement of invariant values in keys. The internal levels also inherit the integrity concept of the conceptual level and uses this concept for the accessibility property of the key. These descriptive meanings of the key concept are pictured in Figure 1. Beside the descriptive meanings we may also use implementation concepts such as pointers and pragmatic meanings such as naming.
50
B. Thalheim language definition level
uniqueness external level
identification
~
conceptual level identification (surrogate)
U integrity constraint (key dependency)
internal level
j identification z accessibility - invariance
existence
Fig. 1. Descriptive meanings of the key concept depending on the levels
Kinds of Multivalued Dependencies. It is often claimed that multivalued dependencies are difficult to model, to teach, to learn and to handle. They are introduced in almost any database book. The classical introduction is based on the tuple-generating definition of the multivalued dependency X → → Y |Z . It requires that whenever two tuples have the same value on the left-hand side then there also exists a tuple in the relation class which matches to the first tuple by the left-hand side and the first element of the right-hand side and which matches to the second tuple by the left-hand side and the second element of the right-hand side. This definition has the clarity of a mathematical definition and the problematic treatment often observed for other mathematical constructs. There are however five other definitions for multivalued dependencies that are easier to understand, to handle, and to represent. We may group these definitions as follows: Derivation constraint: The classical definition is based on the above mentioned tuple generating property. Another definition uses the construction approach for hinged Cartesian products and binds the validity of the multivalued dependency to the equality on query expressions (σX=x (RC ))[Y ∪ Z] = (σX=x (RC ))[Y ] × (σX=x (RC ))[Z] is valid for Y, Z with Y ∩ Z = ∅ for all x ∈ RC [X] . Structure constraint: The structure-oriented definition can be given through – the decomposition of the original class into two classes, i.e., the original class can be reconstructed by applying the natural join to the projection of RC to X ∪ Y and X ∪ Z respectively or – the potential structuring by a nested relation class νY (νZ (RC ))
X {Y } {Z} A1 ... Ak Ak+1 ... Am Am+1 ... An ... ... ... ... ... ... ... ... ...
or
– the potential restructuring in an ER model in the following form Independence concept: The original definition [Cad76] uses relative independence of Y -components from Z-components in a relation class, i.e., the values of the X-
Integrity Constraints in (Conceptual) Database Models
Y
XY
-
X
(1,n)
(1,n)
XZ
-
51
Z
components determine the values of Y-components of objects in RC independently from the values of Z-components, ( (σX=x (RC ))[Y ] = (σ(X=x)∧(Z=z) (RC ))[Y ] for all x-values x ∈ RC [X] and all z-values z ∈ RC [Z]). The generation definition is certainly the most mathematical and the least useful among the definitions. The construction definition provides a facility for checking validity. Both definitions also demonstrate that the relational model language is not appropriate for reasoning on these constraints. The independence notion is the simplest for modelling and for considering. The structure definitions are certainly the best way for the understanding of these constraints. The ER approach is the best suited model language for multivalued dependencies. Example 2. Let us consider a relationship type EmployeeAssociation defined on the entity types: StaffMember, DependentPerson, Project, Supplier, Product with the following multivalued dependencies: { StaffMember } → → { Department, DependentPerson }|{ Project, Product, Supplier } { StaffMember } → → { DependentPerson }|{ Department, Project, Product, Supplier } { Project } → → { StaffMember, Department, DependentPerson }|{ Product, Supplier } { Product } → → { Department, StaffMember, DependentPerson, Project }|{ Supplier } .
The ER model allows us to derive a new structuring by the decomposition of the relationship type. We obtain the schema in Figure 2 for which a new type ‘Working’ reflects the separation of concern to PP roduct, Department Staf f Member and the independence of suppliers and projects according to the multivalued dependencies.
Department
- StaffMember
Working
Dependent
0..1 0..1 Project of Product
-
?
Product
Supplier of Product
Fig. 2. The decomposition of the relationship type based on multivalued dependencies
3 Difficult Unsolved Problems 3.1 Global versus Local Normalisation Normalisation is typically carried out on the basis of one database type. This type is normalised (e.g. decomposed, split or reduced) according to a set of integrity constraints.
52
B. Thalheim
The association and the influence of this normalisation to other types is typically neglected. Therefore, normalisation is typically local. Local normalisation of a singleton database type is well reflected in most database books (e.g., [AHV95, Bis95, Leo92, Yan86]) and publications, most database courses, and in actual database practice. It is considered as one of the pearls of database research and known to almost everybody who knows database technology. The provenance and acknowledgement is based on the facility it provides: keeping as much as possible locally and globally supporting only those processes that are inherently global. Both independence concepts of databases (conceptual independence and implementation independence) are based on localisation. Local normalisation of database structures aims in derivation of such structures of databases that can easily be supported by the DBMS. In the past DBMS have been supporting keys, domain constraints and key-based inclusion constraints. Therefore, it is a goal to derive another equivalent schema to the given one which has an equivalent but supportable set of integrity constraints. This approach can be understood as a procedural approach to optimisation of database structuring depending on the platform for implementation. Normalisation is typically considered to be vertical normalisation. Deductive normalisation and horizontal normalisation are alternatives to vertical normalisation. Horizontal normalisation [PBGG89] is based on selection and union. Horizontal normalisation uses selections based on predicates α1 , ..., αn which n may be pairwise exclusive (αi → ¬αj , i = j) and cover the truth value ½ (( i=1 αi ) → ½). Horizontal normalisation also allows us to separate the part of a set for which a dependency is valid from the part that invalidates a dependency. For instance1 , αX−→Y = (o ∈ RC → ¬∃o ∈ RC (o[X] = o [X]∧o[Y ] = o [Y ])) separates those objects in RC for which the functional dependency is valid from those which invalidate the functional dependency. Deductive normalisation [Tha91] is based on reduction and extended selection. Deductive normalization reduces relations to those elements that cannot be generated from the other elements by generation rules. It is the most storage effective and the best computational method for normalisation as long as the tuple-generating dependency used for decomposition is acyclic. Horizontal and deductive normalisation methods have not yet received a support from the database systems vendors. Local normalisation must however take into account these three kinds of normalisation. Global normalisation aims in normalisation of the schema as a whole. It must take into account the three kinds of local normalisation. Global normalisation has not got an appropriate attention in research despite the interest in implementations. Therefore, a systematic treatment of this normalisation has not yet been given in the literature. 3.2 Optimal Synthesis Algorithm Classical normalisation algorithms usually start from a given set of dependencies and generate one final normalisation of the given schema to a number of schemata in a target normal form. These normalisation suffer from three drawbacks: 1
We use the symbol R for type or class specification and denote the class of elements of the type by RC . Tuples (in the case of object-relational models) or elements of RC are denoted by o. X −→ Y is a functional dependency.
Integrity Constraints in (Conceptual) Database Models
53
Deterministic solution: All known normalisation algorithms produce a different output in dependence of the order of the attributes and of constraints if there are several minimal covers for the same set of integrity constraints. Which order is the best must be determined through trial and error methods. Normalisation is neither confluent nor Church-Rosser. Optimal solution: Since the resulting normalisation depends on the order, the result might not be the optimal normalisation. At present, normalisation is not optimised against behaviour of the database for the given application. Completeness of specification: Normalisation algorithms are very sensitive to incompleteness of constraint specification. Since the number of independent functional dependencies in a relation schema with n attributes is exponential in n the constraint acquisition becomes the major bottleneck for normalisation. It is typically not achievable to collect all constraints that are valid for a certain database. Imagine to collect about 500 independent functional dependencies for a relation schema that has 11 attributes. Unnatural structural clusters: Most synthesis algorithms group all attributes that can be implied from a set of attributes. This collection procedure by left sides of functional dependencies unifies different concerns that are represented by functional dependencies based on the implication rule X → Y , X → Z |= X → Y ∪ Z . The meaning in the application and the treatment in reality of these two functional dependencies might however be completely different. 3.3 Treatment of Almost Valid Constraints Constraints may be valid to a large extent and invalid for exceptional cases. The classical approach to constraints handling proposes to use a holistic constraint that covers both cases. In most cases, this approach results in weakening the original constraint so that the exceptional cases are covered as well. There are however approaches that allow one to handle this case: – Conditional constraints [PBGG89, Tha91] bind the validity of constraints to those cases in which the conditions are valid 2 . – Deontic constraints [Tha00] separate between the normal case and the holistic case. Specific deontic constraints are the exception handling constraints in [BT07]. – State-dependent dynamic constraint sets [Tha00] bind the validity of constraints to general states of the database. 3.4 Consideration of Constraint Sets Instead of Singleton and Isolated Constraints Classically, database constraints are developed one by one. Finally, a constraint set occurs that is difficult to comprehend, to revise, to handle, to extend and to maintain. Moreover, constraints of different kinds cannot be considered together. This impotence is partially caused by non-axiomatisability, e.g. functional and inclusion dependencies. It is also caused by the complexity of constraint sets. Cardinality constraints in 2
These constraints have been reinvented for XML and are also called conditional functional dependencies [BFGM08].
54
B. Thalheim
ER models use an overloaded concept and thus do not have an axiomatisation. They combine a lower bound and an upper bound. For instance, a cardinality constraints card(R, R ) = (1, 1) represents at the same time a functional dependency and an inclusion dependency. It is well known [LL99], that an axiomatisation exists for functional dependencies and inclusion dependencies if functional dependencies and inclusion dependencies do not interact. If they are however inter-twinned then there is no Hilberttype axiomatisation [Tha91]. Moreover, cardinality constraints are specified locally but have global effects [Tha00]. It is thus a nightmare or at least an intellectual challenge to understand a large constraint set. We introduce in Section 4.2 a solution that allows one to handle constraint sets at least for sets of functional dependencies. 3.5 The Expressive Power and Language Relativity Database models are based on a database modelling language that has been selected for specification. Languages may however also restrict modelling. Language expressiveness is an important criterion in language design and it is somewhat different from any of the classical criteria for ‘good’ languages such as availability, understandability, efficiency, verifiability, and portability. The principle of language relativity postulates that languages have their own deficiency, i.e., some concepts of other languages may not be adequately represented in the language that has been chosen. A database modelling language is expressive for a developer or programmer if (s)he can identify a direct representation of concepts with which (s)he reasons about the given application. This language relativity leads to problematic or inadequate models or limits the representation of things. It is not well understood. Typical examples are existence constraints investigated below.
4 Background Theories and Results Research on integrity constraints has resulted in a rich body of knowledge that is known to specialists but has not found its way to practice, to educational books, to implementations, and to a foundation of database culture. At the same time, some results are considered to be so obvious that they are not mentioned in literature nowadays but used at the same time based on cultural background. This section aims in drawing attention to those pearls. Background theorems are currently known among people, are old statements whose authorship is difficult to detect, are combinations and abstractions of a bunch of theorems and form some commonly accepted culture. They are popular and neglected at the same time. 4.1 The Power of Simplicity Functional and multivalued dependencies are a very powerful and simple class of integrity constraints. It is well-known that their reasoning and deduction can be based on propositional formulas [Tha91]. At the same time, a simple and powerful axiomatisation can be given for simple dependencies and their negations. As an example we give an axiomatisation for functional and negated functional dependencies:
Integrity Constraints in (Conceptual) Database Models
55
Axioms XY → Y
Rules (1) (4)
X −→ Y X −→ Y , Y −→ Z X −→ Y , X −→ / Z (2) (3) XV W −→ Y V X −→ Z Y −→ / Z
X −→ / Y XZ −→ / YZ X −→ Z , X −→ / YZ Y −→ Z , X −→ / Z (5) (6) (7) X −→ / YZ XZ −→ / Y X −→ / Y X −→ / Y
4.2 The Power of Graphical Reasoning FD Graphs. Functional dependencies are typically treated in a linear fashion. Graphical representation can however be used for reasoning on constraints as well. It is a folklore approach that deserves more attention. Example 3. Given the set UR = {A, B, C, D, E} of attributes and the set σR = {A −→ E, B −→ E, CE −→ D} of functional dependencies. This set can be represented by the following graph: This graph is easier to survey, simpler for reasoning and A
*
B
E
CE
- D
C
contains the full information on σR . FD graphs can be axiomatised. This axiomatisation mainly follows the rule system of the Armstrong axiomatisation except that right hand side attributes are limited to singleton attribute sets. This axiomatisation [DMT04] provides the first axiomatisation for FD graphs (Y denotes a set; A, B, C denote attributes): (S)
Y →B YC →B
(T )
Y → A, Y A → B Y →B
(P )
Y C → B Y → B
Y → A, Y → B Y A → B, Y → B (R) ()¬(Y → B, Y → B) Y A → B Y → A () is a formalization of ’→’ being the negation of ’→’, i.e., ¬() can be deduced starting with contradictory sets of constraints. (Q)
Theorem 1. [DMT07] The ST implication system over graphs of functional dependencies with rules (S) and (T) and no axioms is sound and complete for functional dependencies. The PQRST implication system over graphs of functional and negated functional dependencies with all the presented rules and the symbolic axiom (), which is used for indicating contradiction, is sound and complete for functional dependencies and their negations if they are not conflicting.
56
B. Thalheim
At the same time reasoning on graphs is more appropriate also for normalisation. Yang [Yan86] has discussed a reduction algorithm that allows a simple generation of canonical covers and of minimal covers for a set of integrity constraints. We can also use graphs of functional dependencies for detection of normal forms that are better than those generated by classical synthesis algorithms. Synthesis algorithms blindly collect all dependencies with the same left side at the final step. Using graphical reasoning we may collect only those dependencies that are not covered by any other group. Example 4. Given: the set UR = {A, B, D, F, G, I} of attributes and the set σR = {A −→ IG, D −→ F G, IAB −→ D, IF −→ AG} of functional dependencies. This set can be represented by the graph in the left picture. The graph in the right picture
I IF
* G 6
* A
- D
ABI F
I
* G 6
* A
IF
- D
AB F
displays its reduction to a minimal cover. Classical synthesis algorithms generate the following relation schemata: R1 R2 R3 R4
= ({A, G, I}, {A −→ GI}) = ({A, F, I}, {A −→ I, F I −→ A}) = ({A, B, D}, {AB −→ D}) = ({D, F, G}, {D −→ F G})
This normalisation is however not minimal. Instead of R1 we can use R1 = ({A, G}, {A −→ G}). The relation schema R1 is obtained through the relation schema R2 in combination with R1 . R2 is not in BCNF. It cannot be split into two relation schemata. The graph in the picture directly allows us to detect this better normalisation since the dependency A −→ I is covered by R2 . This example illustrates the power of graphical reasoning. Normalisation is typically driven by goals such as being dependency preserving, providing the basis for key-based integrity maintenance (BCNF) or being redundancy-free (3NF) at least at the attribute level or avoiding maintenance problems (e.g., insert anomaly) or avoiding data losses (e.g., delete anomaly) or occurrence of performance traps (e.g., update anomaly). It might be extended to such criteria as being redundancy-free at the value level. Textbooks typically dont’t mention that normalisation also introduces additional constraints. Example 5. The last example results in pairwise inclusion constraints such as the multiple value occurrence bindings R1 [A] ⊆⊇ R2 [A] ⊆⊇ R3 [A]. This inclusion dependency set adds an overhead to integrity maintenance. The functional dependencies are
Integrity Constraints in (Conceptual) Database Models
57
directly represented through key dependencies. At the same time, any change to Avalues in one of the relations must be harmonised with changes to A-values in the other relations. FD Set Graphs. Demetrovics, et al., [DMT07] introduce graphical reasoning for sets of constraints. We may represent validity and invalidity of functional dependencies by a graph. Let us illustrate this approach for relational schemata with three attributes. Visualisations may be based on more complex figures, e.g. a hexagon for six attributes. To do so we distinguish two kinds of functional dependencies and display them as follows in Figure 3: Functional dependencies of the form {A} → {B, C} can be decomposed to canonical functional dependencies {A} → {B} and {A} → {C}. They are represented by endpoints of binary edges in the triangle representation. Functional dependencies with two-element left-hand sides {A, B} → {C} cannot be decomposed. They are represented in the triangle on the node relating their right side to the corner. (1)
C
A
(2)
B
C
A
(3)
B
C
A
× × × B
Fig. 3. Examples of the triangle representation. From left to right: (1) The functional dependency {A} → {B} (filled circle at B side (right side of the FD) of the AB edge) and the implied functional dependency {A, C} → {B} (circle around node B in the ABC triangle). (2) The functional dependencies {A} → {B}, {B} → {C} and their implied functional dependencies {A} → {C}, {A, B} → {C}, {A, C} → {B}. (3) The negated functional dependency {A, C} → {B} (crossed filled circle around node B in the ABC triangle) and the implied negated functional dependencies {A} → {B} (crossed circle at B side of the AB edge) and {C} → {B}.
This notation has the advantage that we are supported for reasoning on potential validity of constraints as well. We may represent also candidates for excluded functional dependencies by crossed circles for the case that we know that the corresponding functional dependency is not valid in applications or by small circles for the case that we do not know whether the functional dependency holds or does not hold. Since the ST implication system is sound and complete for non-trivial canonical functional dependencies, rules (S) and (T) can be used for deriving all implied functional dependencies given an initial set. Moreover, the PQRST implication system forms a sound and complete system for both positive and negative (excluded) non-trivial singleton functional constraints, rules (P), (Q) and (R) can be applied as complements of rules (S) and (T) when excluded functional constraints are taken into account. These rules can be interpreted in terms of the graphical representation as well. A deduction step using one of them deals with a node of a higher-dimension object (e.g., a triangle as a two-dimensional object with one of its three vertices) and one or two of its borders (with one dimension lower, e.g., edges of the same triangle as onedimensional objects). Graphical versions of rules are shown on Figure 4 for the triangle
58
B. Thalheim
Fig. 4. Graphical versions of rules (P), (Q), (R), (S) and (T) in terms of the triangle representation. The small black arrows indicate support (necessary context) while the large grey arrows show the implication effects.
representation (case Y = {C}). The large grey arrows indicate the implication effect of each rule. Rule (S) is a simple extension rule and rule (T) can be called the “rotation rule” or “reduction rule”. We may call the left-hand side of a functional dependency the determinant of it and the right-hand side the determinate. Rule (S) can be used to extend the determinant of a dependency resulting in another dependency with one dimension higher, while rule (T) is used for rotation, that is, to replace the determinate of a functional dependency by the support of another functional dependency with one dimension higher (the small black arrow at B indicates support of AC → B). Another possible way to interpret rule (T) is for reduction of the determinant of a higher-dimensional dependency by omitting an attribute if a dependency holds among the attributes of the determinant. For excluded functional constraints, rule (Q) acts as the extension rule (i.e., it needs the support of a positive constraint, i.e., functional dependency) and (R) as the rotation rule (which needs a positive support too). These two rules can also be viewed as negations of rule (T). Rule (P) is the reduction rule for excluded functional constraints, with the opposite effect of rule (Q) (but without the need of support). Rule (Q) is also viewed as the negation of rule (S). 4.3 The Power of Equivalence Class Reasoning Equality Set Characterisation of Functional Dependencies. Validity of functional dependencies in relation classes can also be characterised on the basis of equality sets. Given a relation schema R = (UR , ΣR ) and a relation class RC . Let ERC (A)) = { {t ∈ RC |t =A t } | t ∈ RC } for A ∈ UR and ERC (X)) = { {t ∈ RC |t =X t } | t ∈ RC } for X ⊆ UR for predicates t =A t and t =X t which are true iff t and t have equal A- or X-values, respectively.
Integrity Constraints in (Conceptual) Database Models
59
Lemma 1. Given a relation schema R = (UR , ΣR ) and a relation class RC . RC |= X −→ Y for a functional dependency defined on R iff ERC (X) ⊆ ERC (Y ) . We observe a number of properties for equality sets: (Ru 1) If E1 ⊆ E3 and E2 ⊆ E3 then E1 ∪ E2 ⊆ E3 . (Ru 2) If E1 ∩ E4 ⊆ E2 and E3,1 ∪ ... ∪ E3,k ⊆ E4 then (E3,1 ∪ ... ∪ E3,k ) ∩ E1 ⊆ E2 . (Ru 3) If E1 ⊆ E2 and E2 ⊆ E3 then E1 ⊆ E3 . (Ru 4) If E1 ⊆ E2 and E1 ⊆ E3 then E1 ⊆ E2 ∩ E3 . Equality sets form equivalence relations for tuples of a relation class. They are therefore idempotent, associative for both ∪ and ∩, right-⊆-monotone for ∪ (i.e., E1 ⊆ E1 ∪ E2 ) and left-⊆-monotone for ∩ (i.e., E1 ∩ E2 ⊆ E1 ), and commutative for ∩ and ∪. The equality set system thus forms a lattice for any relation class. The lattice is however not distributive since the following two containments cannot be strengthened to equalities: (E1 ∩ E2 ) ∪ E3 ⊆ (E1 ∪ E3 ) ∩ (E2 ∪ E3 ) (E1 ∪ E2 ) ∩ E3 ⊆ (E1 ∩ E3 ) ∪ (E2 ∩ E3 ) We may now consider a lattice L(RC ) of equality sets of a relation class RC . Each relation class RC on R may have its lattice. These lattices may be incomparable. A lattice L(M, ∩, ∪, , ⊥, ≤) is called sub-cylindric if (Ru 1), (Ru 2), (Ru 3) and (Ru 4) are valid for elements from M. Lattice can be ordered by an embedding relationship . A lattice L1 is smaller than another lattice L2 if a homomorphism from L1 to L2 exist that is a congruence for the lattice operations. Let us consider lattices with M ⊆ 2UR for a relation schema R, ⊥ = ∅, and = UR . The element X denotes the equivalence relation for a subset X of UR . We call theses lattice UR -lattices. Let L(M, ∩, ∪, , ⊥, ≤) be the smallest sub-cylindric lattice that obeys X ≤ Y in LR if ΣR |= X −→ Y . The existence proof is based on the finiteness property of UR . We denote this lattice by LR . We observe a characterisation property of LR . Its proof is straightforward and omitted. Proposition 1. Given a relation schema R. LR L(RC ) for any relation class RC on R. Operations Defined on Singleton Relation Schemata and Validity of Functional Dependencies. One of the main background theorems for functional dependencies is the following statement [Mai83, Tha91]: Theorem 2. Functional dependencies are preserved for join operations, are selectioninvariant, are intersection-invariant, are restricted union-invariant, are not complement-invariant and can be checked on two-tuple relation classes. It is well-known that functional dependencies are not preserved for the projection operation. It is an open question whether there exists a sufficient and necessary condition for extensions of relation classes defined on a projection to the full class. One solution to this problem refers to the existence of a constant complement [AHV95].
60
B. Thalheim
Example 6. Given the set UR = {A, B, C, D, E} of attributes and the set σR = {A −→ E, B −→ E, CE −→ D}, the subset X = {A, B, C, D} of UR . The projec∗ tion of ΣR to X is Σ = {AC −→ D, BC −→ D}. We detect [Mai83] that any projection πX (RC ) of a class RC on R obeys the following curious dependency ∀.(R(a1 , b1 , c1 , d1 ) ∧ R(a2 , b2 , c1 , d2 ) ∧ R(a1 , b2 , c2 , d3 ) ⇒ d1 = d2 ). This dependency does however not characterise those relations in SAT ((X, Σ )) that have an extension in SAT (R). This claim can not be used for the characterization of those relations in SAT ((X, Σ )) for which an extension in SAT (R) exists. For instance, the relation class C RX = {t1 = (1, 3, 6, 9), t2 = (1, 4, 7, 10), t3 = (2, 4, 8, 11), t4 = (2, 5, 6, 12)} obeys the curious dependency but has no extensions in SAT (R). None of the triples matches to the equality condition of the premise. There is another solution that is simpler than the constant complement property. Let us denote the restriction of the UR -lattice L to X by L|X . It is not difficult to prove the following lemma. Lemma 2. Given a relation class RC . Then LR |X L(πX (RC )) is valid for the projection of the UR -lattice. This property is obvious. We thus omit the proof. It turns out that this property is also sufficient. Example 7. Let us first prove the non-extensionality of the relation given above. The relation R obeys Σ since ERCX (A) = {{t1, t2}, {t3, t4}}, ERCX (B) = {{t1}, {t2, t3}, {t4}}, ERCX (C) = {{t1, t4}, {t2}, {t3}}, and ERCX (D) = {{t1}, {t2}, {t3}, {t4}} and ERCX (A) ∩ ERCX (C) ⊆ ERCX (D) , ERCX (B) ∩ ERCX (C) ⊆ ERCX (D). But this relation violates ERCX (A) ∪ ERCX (B)) ∩ ERCX (C) ⊆ ERCX (D). To show that C by E-values ei for each tuple ti (1 ≤ i ≤ 4). Since ERC (A)∪ERC (B) ⊆ we extend RX ERC (E) for any relation class on R we conclude e1 = e2 = e3 = e4 . The requirement ERC (E) ∩ ERC (C) ⊆ ERC (D) is however not fulfilled. Therefore, this relation does not have any extension in SAT (R). The contradiction in the example above has been constructed on the basis of the subset property (E(A) ∪ E(B)) ∩ E(C) ⊆ E(D) for the equality sets. This property can also be represented by a formula that allows us to check the projection: ∀.(R(a1 , b1 , c1 , d1 ) ∧ R(a1 , b2 , c2 , d2 ) ∧ P (a2 , b2 , c3 , d3 ) ∧ R(a2 , b3 , c1 , d4 ) ⇒ d1 = d4 ) . Theorem 3. Given a relation schema R. LR L(RC ) for any relation class RC on R. The proof is based on the completion method used in the example above and the tableau reasoning for equalising introduced values. Corollary 1. For any projection of a relation schema to a subset of its attributes, a set of formulas exist that allows us to check whether a relation on the projection attributes has an extension to the full set of attributes.
Integrity Constraints in (Conceptual) Database Models
61
Compound FD Sets. We can use the equality set approach for an introduction of a new class of equality generating dependencies that characterise those relation classes which have an extension. A set expression on UR is defined by the closure of the recursive construction: (0) Any subset X of UR is a set expression on UR . (1) The expressions S S and S S are set expression on UR for set expressions S, S on UR . A compound functional dependency is a statement S −→ X for a set expression S on UR and X ⊆ UR . C It is valid in a relation class RC if ES ⊆ EY for the equality sets on UR . Compound functional dependencies can be derived from a set of functional dependencies using rules similar to (Ru 1) - (Ru 4). X −→ Y, Z −→ Y X Z −→ Y
X ∪ X −→ Y X X −→ Y
S1 Y −→ Z, S3,1 ... S3,k −→ Y X (S3,1 ... S3,k ) −→ Z
Let us denote the set of all implied compound functional dependencies of a set Σ of functional dependencies by CF D(Σ). The following observations are obvious: Proposition 2. (1) For a given set UR of attributes and a set of functional dependencies ΣR the following relationship is valid: A relation class RC satisfies ΣR iff it satisfies CF D(ΣR ). (2) For any set ΣR of functional dependencies there exists an equivalent set of compound functional dependencies with not more than |UR | elements. The last statement uses the fact that compound functional dependency sets have an equivalent set of compound functional dependencies which right hand side set is a singleton set. It is a simple exercise to show that the following set of derivation rules is sound and complete for compound functional dependencies: (S1 S2 ) (S3 S4 ) −→ Z (S1 S3 ) (S1 S4 ) (S2 S3 ) (S2 S4 ) −→ Z S1 −→ Y, S2 −→ Y S1 S2 −→ Y S −→ Y, Y −→ Z S −→ Z
S1 −→ Y S1 S2 −→ Y
S1 −→ Y S1 −→ Y, S2 −→ Y S1 −→ Y ∪ Z S1 −→ Y
S −→ Y, S −→ Z S −→ Y ∪ Z
Let CF D(ΣR )+ the closure of the set of compound functional dependencies according to the rules above. We can now define the projection πX (S −→ Y ) of a compound functional dependency S −→ Y to a subset X of UR by simply removing those components from the set expression that are not supersets of X and by computing the interaction of X and Y for the right hand side set. We can now summarise the main result of this subsection:
62
B. Thalheim
Theorem 4. Given a relation schema R = (UR , ΣR ), the set of compound functional C dependencies CF D(ΣR )+ and a subset X of UR . A relation RX on X has an extension C + C to a relation R iff the set πX (CF D(ΣR ) ) is valid in RX . This theorem allows us to conclude another interesting property of Boyce-CoddNormal-Forms that is a folklore statement and well-known to people working in practice: Corollary 2. If the relation schema R is in BCNF then any view (or projection) relation class of this schema can be extended to a relation class of the schema R. We note that 3NF schemata that cannot be decomposed into BCNF schemata can often be decomposed may be normalised by applying horizontal normalisation along the additional attribute or by splitting the implied attribute into one that is functionally equivalent to the implied and one that is implied by the key dependency. Example 8. The relation schema R2 in Example 5 can be sliced along F by horizontal decomposition according to the equality set ERC2 (F ). Another resolution of the nontransformability into two BCNF schemata is the split of the attribute A into A1 , A2 with the dependencies IF −→ A2 , A1 −→ I, I −→ A1 .
5 Application of the Background Theories and Results 5.1 BV Constraints and Their Axiomatisation Integrity constraints can be specified based on the B(eeri-)V(ardi)-frame, i.e., by an implication with a formula for premises and a formula for the implication. BV-constraints do not lead to rigid limitation of expressibility. If structuring is hierarchic then BVconstraints can be specified within the first-order predicate logic. We may introduce a variety of different classes of integrity constraints defined: Equality-generating constraints allow to generate for a set of objects from one class or from several classes equalities among these objects or components of these objects. Object-generating constraints require the existence of another object set for a set of objects satisfying the premises. A class C of integrity constraints is called Hilbert-implication-closed if it can be axiomatised by a finite set of bounded derivation rules and a finite set of axioms. It is well-known that the set of join dependencies is not Hilbert-implication-closed for relational structuring. However, an axiomatisation exists with an unbounded rule, i.e., a rule with potentially infinite premises. Since constraint sets may be arbitrary we might ask in which cases an axiomatisation exists. The derivation operator Γ of a deductive system Γ and the implication operator |= may be understood as closure operators Φ, i.e., (0) (i) (+)
Φ0 (Σ) = Σ Φi+1 (Σ) = {α ∈ C ∩ Φ(Φi (Σ))} Φ∗ (Σ) = limi→∞ Φi (Σ)
for any subset Σ from a class C of constraints.
Integrity Constraints in (Conceptual) Database Models
63
The closure operator Φ is called compact for a class C if the property α ∈ Φ∗ (Σ) implies the existence of a finite subset Σ of Σ such that α ∈ Φ∗ (Σ ). It is called closed of Φ∗ (Φ∗ (Σ)) = Φ∗ (Σ) for any Σ ⊆ C. The closure operator is called monotone if Φ∗ (Σ) ⊆ Φ∗ (Σ ∪ Σ ). The operator is reflexive if α ∈ Φ∗ (Σ ∪ {α}) for all formulas and subsets from C. Observation 1. The implication operator Φ∗|= is reflexive, monotone, closed and compact if and only if there exists a deductive system Γ such that ΦΓ and Φ|= are equivalent. If Φ|= additionally has the inference property, the deduction property and is generalization invariant then Φ∗Γ (∅) = Φ∗|= (∅) . If the deduction property fails then the axiomatisation by a deductive system may be based on some obscure rules similar to those for the axiomatisation of PROLOG. The structuring of the extended ER model [Tha00] allows us to deduct a number of properties. As an example we consider the axiomatisation of constraints generalizing those discussed in [Tha91]. We observe first that implication in the hierarchical predicate logic is reflexive, monotone, compact and closed. Let us consider classes of BV-constraints in HERM which form a cylindric algebra [Tsa89]. The order of constraints by Φ|= possibly can be based on the order of of premises and conclusions. In this case the constraint set forms a pair algebra. Observation 2. Cylindric classes are pair algebras. Examples of cylindric classes are the class of functional dependencies, the classes of Hungarian functional dependencies [Tha91], the class of inclusion dependencies and the class of multivalued dependencies. Further, the n-class of all ≥ n-functional dependencies X → Y whose left side contains at least n components and the class of rigid ≤ n-inclusion dependencies T1 [X] ⊆ T2 [X] whose component list contains at most n components form a cylindric constraint set. Usually, union does not preserve cylindric sets. Observation 3. Cylindric constraint classes are axiomatised by reflexivity axioms, augmentation and transition rules. The proof of this statement is based on Section 4.3. If an axiomatisation leads to reflexivity, augmentation and transitivity then union and decomposition rules can be deducted by the other rules. Transitivity may have to consider the specific influence of premises, e.g., transitivity for full multivalued dependencies is based on the root reduction rule [Tha91]. 5.2 Optional, Complex and Null-‘Valued’ Structures Often structures include also optional components. XML schemata and XML DTD allow such kind of open-world-schematalogy. Their design methodology must be different from what we have learned in semantic and object-relational models.
64
B. Thalheim
Let us denote the set of all components of a set O of objects by compon(O) and the set of all optional components of O by componopt (O). Similarly we denote the set of all components used in a constraint α by compon(α). Validity of constraints is either based on strong semantics requiring validity for all object sets independent of whether componopt (O) ∩ compon(O) = ∅ or on weak semantics requiring validity for constraints only for those object sets O for which componopt (O) ∩ compon(O) = ∅ . Classical validity is based on weak semantics which has a severe disadvantage: Observation 4. Weak semantics leads to non-additivity of constraints for object sets O with componopt (O) = ∅ , i.e., it is not true in general that O |= {α1 , ...., αm } is valid if and only if O |= {αi } for each constraint in {α1 , ...., αm } . Observation 5. Strong semantics leads to non-reflexiveness or non-transitivity of constraints for object sets O with componopt (O) = ∅ , i.e., O |= α → α for some constraints α or the validity of O |= α → β and O |= β → γ does not imply O |= α → γ . Constructors used for construction of more complex types are often used for convenience and representing a different structuring. A typical example is the application of the list constructor with the meaning of representing sets. In this case we must add a list-to-set axiom [Tha00] ∀t ∈ compon(o)∀i, j(type(o.i) = type(o.j) = t ⇒ value(o.i) = value(o.j)) . This axiom is often overlooked and not considered. Observation 6. Semantics for structures defined by the list constructor and representing set must be extended by list-to-set axiom. Since attributes are also constructed on the basis of constructors from base types we may ask whether this construction affects the definition of constraints and the axiomatisability. This question is open for most of the constraints. It has, however, been shown [Lin03] that keys and functional dependencies have a similar treatment as in the relational case. Substructures are, however, more complex and represented by the Brouwerian algebra of subcomponents. An example of an existence declaration is the NULL ‘value’. NULL is a special marker used in SQL to indicate that a value for an attribute of an object does not exist in the database. Its aim is a representation of “missing information and inapplicable information”. The NULL marker is one of the most overloaded concepts in database technology. [Tha00] shows that 20 different kinds of null markers might be considered for specification of a database schema. Since this existence declaration is interpreted as a value in SQL, NULL has led to controversies and debates because of its treatment by 3-valued logics, of its special usage requirements in SQL joins, and the special handling required by aggregate functions and SQL grouping operators. The results are: – Unnatural schemata: Values can be missing due to failing generation or can be intentional missing. A value may be not existent or unknown. A value existence depends also on the thing that is represented by an object.
Integrity Constraints in (Conceptual) Database Models
65
– Computational problems: In the presence of NULL markers, the behaviour of predicates in SQL depends on the DBMS. Predicates such as (NOT) IN, (NOT) EXISTS, aggregation functions, and NOT show an anomalous behaviour. Boolean connectives are evaluated to UNKNOWN in the case of NULL markers. – Misuse of logics: SQL follows the semantics of predicate logic also in the case of missing values and thus cause a number of problems: The interpretation of null ‘values’ is not easy to adapt to a particular intuition; the meaning of relational operations when applied to both null values and certain data values is not well-defined; and the treatment of integrity constraints in databases containing null values is not completely defined. NULL markers can however be explicitly incorporated into SQL [Tha10]. Alternatively, we need to overcome the schema deficiency and to define away NULLs from relational schema whenever possible. NULL ‘values’ are representing additional semantical information within a tuple that must be taken into consideration. 5.3 General Normalisation and Optimisation Based on the axiomatisations discussed above we may introduce a general vertical decomposition form: Given a schema structuring S = (ER, ΣS ). A vertical decomposition of S is given as a mapping τ from S to S which is defined by projection functions. The decomposition is lossless if a query q on S can be defined such that for each db on S the equality q(τ (db)) = db is valid. Let further Σ be the set of those constraints from Φ|= (Σ) which are entirely defined on the structures in S . A decomposition based on projection is called C-constraint preserving if Σ ⊆ Φ|= (Σ ) . A classical example of vertical decompositions are decompositions of relations to relations in the third normal form. We may now introduce a general class of C-decomposition algorithms: Construct basic elements which are undecomposable. Derive maximal elements by backward propagation of augmentation. Reduce redundancy in the constraint set by backward propagation of transitivity. Derive a left-right graph by associating conclusions of a constraint with the premise of another constraint. Combine all minimal left sides of constraints which are not bound by another constraint to a group. Derive projections based on all groups in the graph. The first step of the decomposition algorithm is only introduced for convenience. This algorithm is a generalization of the classical synthesis algorithm. Observation 7. The C-decomposition algorithm leads to C-constraint preserving decomposition if the class C is cylindric.
66
B. Thalheim
6 Summary and Conclusion This paper revisits and restructures the theory of integrity constraints. Integrity constraints are widely used for normalisation of database structures, for derivation of integrity maintenance procedures, for prohibition of value inconsistencies in databases and for computational support of database operations. They led to a large body of knowledge that is not entirely covered by surveys, textbooks or monographs. We start with a reconsideration of classical problems and detect that we need a systematic treatment of integrity constraints. Therefore, a framework for integrity constraint handling must be developed. Implicit integrity constraints form an essential part of constraint specification since structures and constraints are intertwined. Although constraint theory is very rich3 , some of the main research issues are still open research problems. The most important problem is the global normalisation problem. It is related to revision of synthesis algorithms and to treatment of almostvalid constraints. Constraint specification typically leads to a specification of sets of constraints from various constraint classes. Research has either considered constraint classes in isolation or interference of entire constraint classes. The set aspect is not yet considered. The constraint literature has also led to combined results and pearls. We discuss in this paper four pearls: simplicity of reasoning, graphical reasoning based on visualisation techniques, reasoning based on equivalence classes, sophisticated specification of constraints based on explicit consideration of their meaning. These pearls allow a general theory of reasoning for constraints in arbitrary database models, provide a treatment of optional structures and support intelligent optimisation and normalisation approaches.
References [AHV95] [BFGM08]
[Bis95] [BT99] [BT07] [Cad76]
[DMT04]
3
Abiteboul, S., Hull, R., Vianu, V.: Foundations of databases. Addison-Wesley, Reading (1995) Bravo, L., Fan, W., Geerts, F., Ma, S.: Increasing the expressivity of conditional functional dependencies without extra complexity. In: ICDE, pp. 516–525. IEEE, Los Alamitos (2008) Biskup, J.: Foundations of information systems. Vieweg, Wiesbaden (1995) (in German) Beeri, C., Thalheim, B.: Identification as a primitive of database models. In: Proc. FoMLaDO 1998, pp. 19–36. Kluwer, London (1999) Berztiss, A., Thalheim, B.: Exceptions in information systems. In: Digital Libaries: Advanced Methods and Technologies, RCDL 2007, pp. 284–295 (2007) Cadiou, J.-M.: On semantic issues in the relational model of data. In: Mazurkiewicz, A. (ed.) MFCS 1976. LNCS, vol. 45, pp. 23–38. Springer, Heidelberg (1976) Demetrovics, J., Moln´ar, A., Thalheim, B.: Graphical reasoning for sets of functional dependencies. In: Atzeni, P., Chu, W., Lu, H., Zhou, S., Ling, T.-W. (eds.) ER 2004. LNCS, vol. 3288, pp. 166–179. Springer, Heidelberg (2004)
More than 40 pages of references in [Tha00] are directly related to constraints.
Integrity Constraints in (Conceptual) Database Models [DMT07]
[HHLS03]
[Leo92] [Lin03]
[LL99] [Mai83] [PBGG89] [Tha87] [Tha91] [Tha00] [Tha10] [Tsa89] [Yan86]
67
Demetrovics, J., Molnar, A., Thalheim, B.: Graphical axiomatisation of sets of functional dependencies in relational databases. In: Alkalmazott Matematikai Lapok, vol. 24, pp. 223–264 (2007) Hartmann, S., Hoffmann, A., Link, S., Schewe, K.-D.: Axiomatizing functional dependencies in the higher-order entity-relationship model. Inf. Process. Lett. 87(3), 133–137 (2003) Leonard, M.: Database design theory. MacMillan, Houndsmills (1992) Link, S.: Consistency enforcement in databases. In: Bertossi, L., Katona, G.O.H., Schewe, K.-D., Thalheim, B. (eds.) Semantics in Databases 2001. LNCS, vol. 2582, pp. 139–159. Springer, Heidelberg (2003) Levene, M., Loizou, G.: A guided tour of relational databases and beyond. Springer, Berlin (1999) Maier, D.: The theory of relational databases. Computer Science Press, Rockville (1983) Paredaens, J., De Bra, P., Gyssens, M., Van Gucht, D.: The structure of the relational database model. Springer, Berlin (1989) Thalheim, B.: Open problems in relational database theory. Bull. EATCS 32, 336– 337 (1987) Thalheim, B.: Dependencies in relational databases. Teubner, Leipzig (1991) Thalheim, B.: Entity-relationship modeling – Foundations of database technology. Springer, Berlin (2000) Thalheim, B.: NULL value algebras and logics. In: Proc. EJC 2010, Jyv¨askyl¨a, pp. 365–376 (2010) Tsalenko, M.S.: Modeling of semantics for databases, Nauka, Moscov (1989) (in Russian) Yang, C.-C.: Relational Databases. Prentice-Hall, Englewood Cliffs (1986)
Modeling Techniques for Multi-level Abstraction Bernd Neumayr1 , Michael Schrefl1 , and Bernhard Thalheim2 1
Department of Business Informatics - Data & Knowledge Engineering Johannes Kepler University Linz, Altenberger Straße 69, 4040 Linz, Austria 2 Department of Computer Science and Applied Mathematics Christian Albrechts University Kiel, Olshausenstr. 40, 24098 Kiel, Germany
Abstract. Employing multi-level abstraction in modeling refers to representing objects at multiple levels of one or more abstraction hierarchies, mainly classification, aggregation and generalization. Multiple representation, however, leads to accidental complexity, complicating modeling and extension. Several modeling techniques, like powertypes, deep instantiation, materialization, m-objects, HERM, and the component model may be used to reduce unnecessary complexity with multilevel abstraction. This chapter compares these modeling techniques using four comparison criteria: (1) compactness (modular and redundancyfree models), (2) query flexibility (number and kind of pre-defined entry points for querying), (3) heterogeneous level-hierarchies, and (4) multiple relationship-abstractions (such as between relationship occurrence and relationship type). Keywords: Multi-Level Modeling, Ontological Metamodeling, Deep Instantiation, Materialization, Powertypes, M-Objects.
1
Introduction
Modeling domain objects at multiple levels of abstraction has received growing attention over the last years. IT-systems cover increasingly broader domains with different applications considering the same entities from different abstraction perspectives. Consider the IT-system of a car manufacturer that supports marketing of and sales of cars. Marketing considers cars from the perspective of car models being instances of a class CarModel, one such instance being for example Porsche911CarreraS. Sales considers cars from the perspective of physical cars, each sold to an individual customer and being instance of some car model such as Porsche911CarreraS. The same object Porsche911CarreraS is once considered as instance and once as class. More generally, cars can be considered not only at two levels but at multiple levels of abstraction, such as brands (e.g., Porsche), models (e.g., Porsche911CarreraS), and physical cars (e.g., myPorsche911CarreraS). Modeling multi-level abstraction has been a research issue in the database and software engineering communities. These communities have introduced different Kaschek/Delcambre (Eds.): The Evolution of Conceptual Modeling, LNCS 6520, pp. 68–92, 2011. c Springer-Verlag Berlin Heidelberg 2011
Multi-level Abstraction
69
but related basic terms for the representation of the application domain vs. for the set of modeling constructs used to build this representation. The database community speaks of database (schema and instance) vs. data model, the software engineering community of ontological vs. linguistic modeling [4]. In this chapter we use the notions of domain model vs. modeling language. For example, such a domain model may be schema and instance of a data warehouse [21]. Several modeling techniques and modeling language constructs have been proposed in recent years to represent a domain concept (e.g, car) with descriptions at multiple abstraction levels (e.g., brand, model, physical entity) and the interplay between these levels. Research in this direction has been motivated by reducing accidental complexity of domain models, ‘that occurs due to mismatches between a problem and the technology used to represent the problem’ [5]. Complexity has to be seen with regard to the domain model and the modeling language. Domain models become more complex if multiple, simple modeling language constructs have to be employed together with explicit constraints to represent some domain concept, such as car with its abstraction levels and their interplay. Frequently, the complexity of a domain model can be reduced if the modeling language provides an appropriate complex language construct that can be employed to directly represent some domain concept. Thereby the complex language construct inherently captures integrity constraints that otherwise have to be stated explicitly. We later introduce several comparison criteria for modeling techniques that address accidental complexity from a wider and more in-depth perspective. Multi-level abstraction is not only associated with modeling at multiple classification levels, but more generally, also with levels in generalization and aggregation hierarchies. Existing modeling techniques focus on different aspects of multi-level abstraction. Ontological metamodeling with potency-based deep instantiation [3,15] provides for multiple levels of classification whereby an object at one level can describe the common properties for objects at each instantiationlevel beneath that level. A powertype [22,12,10] has subclasses of a given class as its instances and allows to describe common properties of these subclasses. Similar, but in our view more related to aggregation, materialization [9,24,8] relates a class of categories (e.g., car models) and a class of more concrete objects (e.g., physical cars). A multi-level object (m-object ) [19] encapsulates the different levels of abstractions that relate to a single domain concept; a multi-level relationship (m-relationship) [19] links two m-objects at multiple levels. The higher-order entity relationship model (HERM) [28] extends the classical entityrelationship model by complex attributes, relationship types of higher order, and cluster types. The component model [29] enhances the HERM by encapsulation based on component schemata. Components may have various ports and views for exchange. Ports are associated through harnesses. In this chapter we present a sample problem and comparison criteria for multilevel abstraction techniques. We will model the sample problem using alternative modeling techniques and highlight strengths and weaknesses of each approach with regard to our comparison criteria. Note, however, that each approach has
70
B. Neumayr, M. Schrefl, and B. Thalheim
been developed with a different focus in mind and that the approaches may complement one another. A related comparison of various modeling techniques to reduce accidental complexity in domain models is given by Atkinson and K¨ uhne [5]. An insightful comparison between powertypes and potency-based deep instantiation is given by Gonzalez-Perez and Henderson-Sellers in the related work section of [10]. These comparisons, however, do not consider all comparison criteria used in this chapter, such as heterogeneous level-hierarchies, query flexibility and multiple relationship-abstractions. The remainder of this chapter is organized as follows: Section 2 defines requirements and comparison criteria for multi-level modeling based on a sample problem. Sections 3 shows how to model the sample problem with each approach and evaluates how each approach fulfills our requirements. Section 4 gives a summary of our comparison and concludes the chapter mentioning application areas and future work.
2
Sample Problem and Comparison Criteria
In this section, we present our running example (taken from [19]), model it using plain UML, and introduce comparison criteria for multi-level abstraction techniques. Example 1 (Sample Problem). The product catalog of an online store is described at three levels of abstraction: physical entity, model, and category. Each product category has associated a tax rate, each product model has a list price. Book editions, i.e., objects at level model that belong to product category book, additionally have an author. Our company keeps track of physical entities of products (e.g., copies of book HarryPotter4), which are identified by their serial number. In addition to books, our company starts to sell cars. For selling cars, it introduces car as a new product category. Cars differ from books in that they are described at an additional level, namely brand, and in that they have additional attributes: maximum speed at level product model and mileage at level physical entity. As our sample online store specializes on selling cars of brand Porsche 911, it wants to be able to register physical entities of this car brand at the Porsche 911 club. Our company further keeps track of companies that produce these products. Companies are likewise described at multiple levels: industrial sector, enterprise, and factory. Producers of cars belong to a specific industrial sector, namely car manufacturer. To track quality problems, our company also associates with each physical entity of product category car the factory at which it was produced. Thereby, this factory must belong to the enterprise, which produces the car model. Using traditional semantic data modeling, e.g., using UML, such multi-level domains can be modeled by representing objects in different abstraction hierarchies, classification, aggregation and generalization. In our reference UML
Multi-level Abstraction
71
model (see Fig. 1) the three-level product catalog is modeled by arranging classes (ProductCategory, ProductModel, ProductPhysicalEntity) and their instances ({Car, Book}, {Porsche911CarreraS, Porsche911GT3, HarryPotter4}, {myPorsche911CarreraS, myCopyOfHP4}, respectively) in an aggregation hierarchy, where each aggregation level represents one level of abstraction (product category, product model, and physical entity).
Product Catalog
ProductCategory - taxRate : Integer
Product Model
ProductCatalog - desc : String
Product Category
Classes
Products: ProductCatalog - desc = 'Our Products'
Singleton BookCategory
ProductModel - listPrice : Integer
Product PhysicalEntity
BookTitle - author : String
CarPhysicalEntity - mileage : Integer
BookPhysicalEntity Product Catalog
Car:CarCategory - taxRate = 20
Singleton CarCategory
CarModel - maxSpeed : Integer
ProductPhysical Entity - serialNr : String
Instances
Product Category
Book:BookCategory - taxRate = 15
Singleton Porsche911CarreraS Model
Porsche911CarreraS: Porsche911CarreraSModel - listPrice = 91.838 - maxSpeed = 293 km/h
Singleton Porsche911GT3 Model
Porsche911GT3: Porsche911GT3Model - listPrice = 108.083 - maxSpeed = 310 km/h
Singleton HarryPotter4 BookTitle
Porsche911CarreraS PhysicalEntity Porsche911GT3 PhysicalEntity HP4-PhysicalEntity
HarryPotter4: HarryPotter4BookTitle - listPrice = 11.50 - author = 'J.K.Rowling' myPorsche911CarreraS: Porsche911CarreraS PhysicalEntity - serialNr = 'C333333' - mileage = 100000 myCopyOfHP4: HP4-PhysicalEntity - serialNr = 'A121212'
Product Model
Fig. 1. Product catalog modeled in plain UML, using generalization, instantiation and aggregation (relationships and level brand are omitted)
Objects at the same level in different subhierarchies may differ from each other. For example, product models have a listprice, but car models (i.e., objects at level model belonging to car, an object at level category) have an additional attribute maxSpeed. Furthermore, physical entities of car additionally have a mileage. In our reference UML model (see Fig. 1) this heterogeneity between objects on the same level is modeled by specializing classes that are part of the aggregation hierarchy below ProductCatalog, i.e. ProductCategory, ProductModel, PhysicalEntity, to corresponding classes, CarCategory, CarModel, CarPhysicalEntity, that describe common properties of objects belonging to product category car at the respective level. Note, that singleton classes like CarCategory are necessary to refine aggregation, e.g., to define that each instance of CarModel belongs to product category Car.
72
B. Neumayr, M. Schrefl, and B. Thalheim
Furthermore, we often need to access attributes defined at a higher level of abstraction, for example, to query the taxRate applicable to myPorsche911CarreraS, defined at its product category, car. Concerning our reference UML model we could navigate from myPorsche911CarreraS to taxRate at car by following the aggregation relationships myPorsche911CarreraS.ProductModel.ProductCategory.taxRate (note, that in order to follow links we use dot-notation and, in the absence of explicit role-names, the names of the classes at the other end of the aggregation relationship). Various modeling techniques —powertypes, potency-based deep instantiation, materialization, m-objects, HERM, and the component model— can be used to reduce the complexity of such multi-level models. Their suitability for multi-level abstraction may be benchmarked against the following comparison criteria. 1. Compactness: A domain model is compact if it is modular and redundancyfree. Modularity means that all model elements that represent a domain concept at different levels of abstraction (e.g., product category car in its instance role as well as in its class roles at levels product model and physical entity) are tied together and can be used and updated as a self-contained part of the model. The common structure of elements at lower levels of abstraction can be specialized without interfering with the rest of the model (e.g., introducing attribute maxSpeed only for car models but not for product models in general). A domain model contains redundancy if information is represented more than once1 . Our reference UML model (see Fig. 1) is neither modular nor redundancyfree: Domain concept car is represented by four separate model elements which are not tied together to a self-contained part of the model, namely by instance Car (labelled Car:CarCategory in Fig. 1) and by classes CarCategory, CarModel, and CarPhysicalEntity. Furthermore, the hierarchic relation between product and car is represented redundantly as aggregation between instances Car and Products and as generalizations between CarCategory and ProductCategory, between CarModel and ProductModel, and between CarPhysicalEntity and ProductPhysicalEntity. 2. Query flexibility: A domain model supports query flexibility if it provides several pre-defined entry points for querying, such as class names or qualified identifiers to refer to various sets of objects. In the context of multi-level abstraction we check whether one may easily refer to the set of objects at one abstraction level that belong to a certain domain concept. To test this criterion, we query (1) all objects belonging to product category car at level product model, (2) all objects belonging to product category car at level physical entity, and (3) all objects belonging to level product physical entity. In our reference UML model these entry points for queries are available through the classes (1) CarModel, (2) CarPhysicalEntity, and (3) ProductPhysicalEntity. 1
Note that domain models of cooperating agents may be redundant; such independent, distributed systems are not addressed here.
Multi-level Abstraction
73
3. Heterogeneous Level-Hierarchies: A modeling technique supports heterogeneous level-hierarchies if an additional abstraction level can be introduced for some domain concept without the need to change the set of abstraction levels for other domain concepts. To evaluate this criterion, we will introduce an additional abstraction level car brand for product category car between levels product category and product model and describe one car brand, namely Porsche 911. If this extension can be made without affecting the existing model, i.e, definitions of other product categories, e.g., book, or generally of products, the modeling technique is considered to support heterogeneous level-hierarchies. Concerning our reference UML domain model the respective extension could be made by introducing a class CarBrand as a component of class CarCategory and as an aggregate class of CarModel.
ProductCategory 1
IndustrialSector
context ProductModel inv: self.enterprise.industrialSector->forall( x | self. productCategory.industrialSector->exists( y | x = y) )
* ProductModel 1
producedBy
producedBy
context ProductPhysicalEntity inv: self.factory.enterprise->forall( x | self. productModel.enterprise->exists( y | x = y) )
* ProductPhysicalEntity
Car : ProductCategory
Porsche911CarreraS : ProductModel myPorsche911CarreraS : ProductPhysicalEntity
producedBy
producedBy
producedBy
producedBy
1
* Enterprise 1
* Factory
CarManufacturer : IndustrialSector PorscheLtd : Enterprise
PorscheZuffenhausen : Factory
Fig. 2. Relationships producedBy at different levels of abstraction modeled with UML and OCL
4. Multiple relationship-abstractions: A modeling technique supports multiple relationship-abstractions if it does not only support modeling of domain objects at multiple levels of abstraction, but equally supports multi-level abstraction of relationships. We are interested in modeling constructs that can be employed to simplify querying relationships at multiple levels and carry implicit constraints that are suitable for generalization, classification, and aggregation of relationships at multiple levels. Consider, for example, that products are produced by factories. Our sample product catalog describes products at three levels of abstraction (physical entity, model and category) and it describes the factories in which products are produced at three levels as well (factory, enterprise, and industrial sector). To test the support for multiple relationship-abstraction, we check how a modeling
74
B. Neumayr, M. Schrefl, and B. Thalheim
technique supports the abstraction of relationship produced by between physical entity and factory to the relationship produced by between product model and enterprise, and the abstraction of the latter to relationship produced by between product category and industrial sector. Support for multiple relationship-abstraction can further be characterized by (a) (Meta-)Classification of Relationships, i.e., relationship occurences into relationship classes and relationship classes into relationship metaclasses. (b) Adressability/Queryability of Relationships and (Meta-)Relationship Classes at different levels (c) Specialization/Instantiation of Domain and Range either explicitly through using (meta-)classes or implicitly through constraints. UML does not provide modeling constructs for multi-level abstraction of relationships; we employ (see Fig. 2) OCL to express extensional constraints that are imposed from a higher to a lower level of abstraction of the producedBy relationship. This only covers (c) specialization of domain and range.
3
Techniques Supporting Multi-level Abstraction
First, we give an overview of how modeling techniques for multi-level abstraction evolved. Then we present and compare several modeling techniques in detail. Note that specific modeling patterns in software engineering also relate to multilevel abstraction, e.g., the type-object pattern. We do not discuss this line of work and refer the interested reader to [5]. The 1970’s and 1980’s saw the emergence of semantic data models [26,13], introducing the abstraction principles of classification (collecting objects with common properties to object types), generalization (abstracting common properties of types into supertypes), and aggregation (abstracting component objects into an aggregate object). Classification recursively applied leads to metaclasses [14]. While Smalltalk supported metaclasses with the restriction that each metaclass has a single class as instance, the Common Lisp Object System (CLOS) provided for metaclasses with multiple classes as instances. Telos and Concept Base supported unbounded classification hierarchies. VODAK introduced the possibility that metaclasses describe not only their instances (which are classes) but also the instances of these classes, a principle which has later been extended to unbounded levels and coined Deep Instantiation [3]. It is described in Section 3.1. Combining classification and generalization, Cardelli [6] introduced the notion of a powertype that represents the metaclass for all subclasses of some class. Later, Odell [22] allowed one class to have several powertypes, each collecting some subclasses of the class. Powertypes are discussed in Section 3.2. Similarly, but in our view more related to aggregation, Pirotte and others [9,24] proposed the abstraction principle of Materialization that ‘relates a class of categories (e.g., models of cars) and a class of more concrete objects (e.g., individual cars)’ [8]. It is discussed in Section 3.3. Most recently, Neumayr et al.[19], introduced multi-level concretization hierarchies not only for objects but also for relationships. This approach is discussed in Section 3.4.
Multi-level Abstraction
75
Various extensions of the entity-relationship model [7] may be also employed to develop domain models capturing multi-level abstraction. As representatives we discuss the Higher-Order Entity Relationship Model (HERM) and the Component Model in Sections 3.5 and 3.6. 3.1
Potency-Based Deep Instantiation
Deep instantiation refers to meta modeling in which an object at some (meta-)level can describe the common properties for objects at each instantiation-level beneath that level. Metamodeling through meta classes that provide for deep instantiation has first been put forward in the area of databases by the VODAK system [14]. VODAK associated several types with an object, the own-type describing the object itself, the instance-type describing objects at one classification level below, and the instance-instance-type describing objects at two classification levels below. In this way, meta classes in VODAK allow to describe not only common properties of instances but also common properties of instances of instances. Later, Atkinson and K¨ uhne [3,4] introduced an approach which we call ontological metamodeling with potency-based deep instantiation. Rather than using different types for each level as in VODAK, attribute definitions within a clabject 2 are annotated with a potency, where this potency denotes the instance level, i.e., attributes with potency 0 describe the clabject itself, attributes with potency 1 describe instances of the clabject, attributes with potency 2 describe instances of instances of the clabject, and so forth. In this way, potency-based deep instantiation supports unbounded classification levels. K¨ uhne also introduced DeepJava [15] for programming with potency-based deep instantiation. Our sample product catalog (see Example 1) consisting of abstraction levels physical entity, product model, and product category, is modeled (see Fig. 3) by representing objects of level physical entity as objects at classification level 0 (individual level). Each physical entity (e.g., myCopyOfHP4 and myPorsche911CarreraS) is modeled as some instance of a product model (e.g., HarryPotter4 and Porsche911CarreraS). Each product model is a clabject at classification level 1 (class level), i.e., is a class of individuals at level 0 as well as an instance of some product category (e.g., Book, Car) at level 2 (metaclass level). Each product category is again a clabject, i.e., a class for objects at classification level 1 and instance of ProductCategory at classification level 3 (metametaclass level). To define that physical entities of product category car additionally have a mileage and car models additionally have a maximum speed, attributes maxSpeed and mileage are introduced with potency 1 and 2, respectively, at clabject Car. To access attributes defined at a higher level (upward navigation) it is possible to follow the instance-of relationships with method type(). The 2
Atkinson [2] coined clabject (short for class/object) for model elements playing both an instance role as well as a class role (also referred to as object facet and class facet, respectively).
76
B. Neumayr, M. Schrefl, and B. Thalheim Level 3 (Metametaclasses)
ProductCategory - taxRate1 : Integer - listPrice2 : Float - serialNr3 : String
Book - taxRate0 = 15 - author1 : String
HarryPotter4 - listPrice0 = 11.50 - author0 = 'J.K.Rowling'
myCopyOfHP4 - serialNr0 = 'A121212'
Car - taxRate0 = 20 - maxSpeed1 : Integer - mileage2 : Integer
Porsche911GT3 - listPrice0 = 108.083 - maxSpeed0 = 310 km/h
Level 2 (Metaclasses)
Porsche911CarreraS - listPrice0 = 91.838 - maxSpeed0 = 293 km/h
Level 1 (Classes)
myPorsche911CarreraS - serialNr0 = 'C333333' - mileage0 = 100000
Level 0 (Individuals)
Fig. 3. Product catalog modeled with potency-based deep instantiation
path from myPorsche911CarreraS to taxRate defined at Car is denoted as: myPorsche911CarreraS.type().type().taxRate. We evaluate this modeling technique against our comparison criteria: 1. Compactness: Potency-based deep instantiation supports modular, redundancy-free modeling. All information concerning all levels of one domain category can be described local to one clabject. For example, all information about domain concept car is encapsulated in a single model element Car which is related to ProductCategory only once (see Fig. 3). 2. Query flexibility: To our knowledge, extensions of classes are not maintained by DeepJava. But to maintain and retrieve the different member sets of our sample queries should be easy in principle, given the set theoretic interpretation of deep instantiation [16]. Every clabject with potency p > 0 can be viewed as having p member sets, one for each level beneath, e.g., Car = {{Porsche911GT3, Porsche911CarreraS},{{},{myPorsche911CarreraS}}}. 3. Heterogeneous Level-Hierarchies: Additional abstraction levels for some domain concept cannot be introduced without requiring global model changes. Consider introducing a new classification level (e.g., to represent that car models are classified by car brands, such as Porsche911) below level n (e.g., classification level 2 containing clabjects such as Car) and above level n-1 (e.g., classification level 1 containing clabjects such as Porsche911CarreraS). Such an additional classification level requires the following global changes: (1) For each clabject at a classification level above the new classification level (e.g., ProductCategory and Book) potencies of attributes that affect objects at classification levels below the new classification level have to be changed (e.g., listPrice and serialNr in clabject ProductCategory and author in clabject Book). (2) For each clabject at level n, a (dummy) clabject at the new classification level has to be introduced, e.g., BookBrandDummy,
Multi-level Abstraction
77
and each clabject at level n-1 has to be re-classified to be instanceof the respective clabject at the new classification level (HarryPotter4 is instance-of BookBrandDummy). Furthermore, (3) upward navigation paths have to be changed: the navigation from myPorsche911CarreraS to its taxRate changes from myPorsche911CarreraS.type().type().taxRate to myPorsche911CarreraS.type().type().type().taxRate. 4. Multiple relationship-abstractions: The constraints modeled in Fig. 2 can be implemented in DeepJava using type parameters [15], which capture aspect (c) Specialization/Instantiation of Range. More recently, Gutheil et al. [11] discussed connectors (relationships) between clabjects in multi-level modeling environments, focusing on their graphical rendering, not discussing querying different class roles of such connectors at different levels. 3.2
Powertypes
The powertype pattern consists of a partitioned type and a powertype. Every instance of the powertype is a subclass of the partitioned type. Such model elements —playing both an instance role as well as a class role— are again referred to as clabjects. For example (see Fig. 4), ProductCategory is powertype of partitioned type ProductModel. Clabject car plays two roles, namely as instance of ProductCategory and as subclass of ProductModel, represented by Car:ProductCategory and CarModel, respectively. Cascaded Powertypes - Basic Pattern. In order to model our sample threelevel product catalog one can use a cascaded setup of powertypes. For example (see Fig. 4), ProductCategory is powertype of ProductModel, which is, in turn, powertype of ProductPhysicalEntity. Thus, each instance of ProductCategory, such as car, is also considered as a subclass of ProductModel, such as CarModel. In turn, each instance of CarModel, such as Porsche911CarreraS, is also considered as a subclass of ProductPhysicalEntity, such as Porsche911CarreraSPhysicalEntity. We evaluate this modeling technique against our comparison criteria: 1. Compactness: The basic pattern for cascaded powertypes (see Fig. 4) partly supports modularity but may lead to redundancies. Domain concept car is represented by a clabject playing a class role (CarModel) and an instance role (Car:ProductCategory). This improves compactness as compared to our reference UML domain model. However, since the class role of car is the class of all car models, there is no class which has all physical cars as instances. Thus, it is necessary to define attribute mileage redundantly, both at Porsche911CarreraSPhysicalEntity as well as at Porsche911GT3PhysicalEntity. 2. Query flexibility: Querying powertypes is not explicitly treated in the literature we are aware of Pre-defined entry points for our “benchmmark queries” can partly be provided by the explicitly represented class roles: (1) All carmodels: members of the class role of clabject car (CarModel). (2) Physical entities of car: no predefined entry point (3) All physical entities of products: members of ProductPhysicalEntity.
78
B. Neumayr, M. Schrefl, and B. Thalheim
isClassifiedAs
ProductCategory - taxRate : Integer
isClassifiedAs
ProductModel - listPrice : Integer
ProductPhysical Entity - serialNr : String
Car:ProductCategory - taxRate = 20 CarModel - maxSpeed : Integer
Book:ProductCategory - taxRate = 15 BookTitle - author : String
Porsche911CarreraS: CarModel - listPrice = 91.838 - maxSpeed = 293 km/h Porsche911CarreraS PhysicalEntity - mileage : String Porsche911GT3: CarModel - listPrice = 108.083 - maxSpeed = 310 km/h
myPorsche911CarreraS: Porsche911CarreraS PhysicalEntity - serialNr = 'C333333' - mileage = 100000
Porsche911GT3 PhysicalEntity - mileage : String HarryPotter4:BookTitle - listPrice = 11.50 - author = 'J.K.Rowling' HP4-PhysicalEntity
myCopyOfHP4: HP4-PhysicalEntity - serialNr = 'A121212'
Fig. 4. Product catalog modeled with cascaded powertypes - basic pattern. Notation as proposed by Henderson-Sellers and Gonzalez-Perez [12].
3. Heterogeneous Level-Hierarchies: The basic cascaded powertype pattern does not support heterogeneous level-hierarchies. 4. Multiple relationship-abstractions: The literature we are aware of does not introduce special techniques for modeling multiple relationship-abstraction with powertypes. The modeling approach shown in Fig. 2, using plain UML and OCL, can be used together with powertypes, which only captures (c) specialization of domain and range. Cascaded Powertypes - Extended Pattern. To overcome the deficiencies of the simple cascaded powertype pattern one can extend it by introducing further subclasses when instantiating a powertype and by specializing relationships which connect partitioned types with powertypes. For example, when instantiating powertype ProductCategory by car, one can additionally introduce a subclass of ProductPhysicalEntity, namely CarPhysicalEntity, which can be used for defining common structure of physical cars as well as an entry-point for querying. The relationship between partitioned type ProductPhysicalEntity and powertype ProductModel is specialized between partitioned type CarPhysicalEntity and its powertype CarModel (which is the class role of clabject car). We evaluate this modeling technique against our comparison criteria: 1. Compactness: Using the extended powertype pattern we end up with a model that is only partly modular and contains redundancies. Using powertypes, representing domain concept car requires to define two separate model elements, namely class CarPhysicalEntity as well as a clabject which consists
Multi-level Abstraction CarBrand - marketLaunch : Date isClassifiedAs
Porsche911:CarBrand - marketLaunch= 1964 Car:ProductCategory - taxRate = 20 CarModel - maxSpeed : Integer
ProductPhysical Entity - serialNr : String
isClassifiedAs
Book:ProductCategory - taxRate = 15 BookTitle - author : String
isClassifiedAs
isClassifiedAs
ProductModel - listPrice : Integer
Porsche911Model isClassifiedAs
isClassifiedAs
ProductCategory - taxRate : Integer
79
Porsche911CarreraS: CarModel - listPrice = 91.838 - maxSpeed = 293 km/h Porsche911CarreraS PhysicalEntity
Porsche911 PhysicalEntity - porsche911Club : Boolean
Porsche911GT3: CarModel - listPrice = 108.083 - maxSpeed = 310 km/h
myPorsche911CarreraS: Porsche911CarreraS PhysicalEntity - serialNr = 'C333333' - mileage = 100000 - porsche911club = true
Porsche911GT3 PhysicalEntity CarPhysicalEntity - mileage : Integer
BookPhysicalEntity
HarryPotter4:BookTitle - listPrice = 11.50 - author = 'J.K.Rowling' HP4-PhysicalEntity
myCopyOfHP4: HP4-PhysicalEntity - serialNr = 'C333333'
Fig. 5. Cascaded powertypes - extended pattern
of instance role (Car) and class role (CarModel). The hierarchic relationship between domain concepts car and Product has to be modeled redundantly by generalization between CarPhysicalEntity and ProductPhysicalEntity, and by classification between Car and ProductCategory (with generalization between classes CarModel and ProductModel being implicit). 2. Query flexibility: Querying powertypes is not explicitly treated in the literature we are aware of. But pre-defined entry points can be easily provided for our “benchmmark queries”. (1) All car-models: members of car’s class role CarModel. (2) Physical entities of car: members of class CarPhysicalEntity. (3) All physical entities of products: members of ProductPhysicalEntity. 3. Heterogeneous Level-Hierarchies: It is possible to introduce an additional level in one sub-hierarchy (e.g., for product category car) without affecting other sub-hierarchies (e.g., product category book). An additional level brand between car category and car model is achieved as follows: One introduces class CarBrand as powertype of CarModel, whereby CarModel forms a clabject with instance Car as before. Then, a particular car brand such as Porsche911 is modeled as an instance of CarBrand, i.e., as a clabject playing an instance role (Porsche911) and a class role (Porsche911Model), where the latter is subclass of CarModel. Furthermore, to describe common properties (e.g., attribute porsche911Club) of objects belonging to car brand Porsche911 at level physical entity, class Porsche911PhysicalEntity is introduced as subclass of CarPhysicalEntity. 4. Multiple relationship-abstractions: The literature we are aware of does not introduce special techniques for modeling multiple relationship-abstraction with powertypes. But the modeling approach shown in Fig. 2, using plain UML and OCL, can be used together with powertypes. This can only capture aspect (c) Specialization of Domain and Range.
80
B. Neumayr, M. Schrefl, and B. Thalheim
3.3
Materialization
In conceptual modeling, materialization3 is a generic relationship type used to relate a class cc of more concrete objects and a class ca of more abstract objects (e.g., ProductModel materializes ProductCategory). At the instance level, each instance oc of the more concrete class materializes exactly one instance oa of the more abstract class (e.g., Porsche911CarreraS materializes Car). Additionally, materialization provides for powerful attribute propagation mechanisms: Object oc both inherits attribute values from object oa (shared values) and instantiates attributes defined at class cc as well as attributes introduced at object oa . In this line, similar to clabjects, oa plays both the role of an object as well as the role of a class. ProductCategory - taxRate : Integer (T1) - modelAttr (T3) - physEntAttr (T1-T3)
model
1 model materialization
materializes
* materialization ProductModel - listPrice : Integer - physEntAttr2 (T3) 1 model
Book:ProductCategory - taxRate = 15 - modelAttr = {author} - physEntAttr = {}
Car:ProductCategory - taxRate = 20 - modelAttr = {maxSpeed} - physEntAttr = {mileage}
-
Porsche911CarreraS: ProductModel listPrice = 91838 maxSpeed = '293 km/h' physEntAttr2 = {} /taxRate = 20 /physEntAttr = {mileage}
-
model materialization
Porsche911GT3: ProductModel listPrice = 108083 maxSpeed = '310 km/h' physEntAttr2 = {} /taxRate = 20 /physEntAttr = {mileage}
materialization
-
model materializes
* materialization ProductPhysicalEntity - serialNr : String
model
materialization
myPorsche911CarreraS: ProductPhysicalEntity - serialNr = 'C333333' - mileage = 100000 - /listPrice = 91838 - /maxSpeed = 293 km/h - /taxrate = 20
HarryPotter4: ProductModel listPrice = 11.50 author = 'J.K.Rowling' physEntAttr2 = {} /taxRate = 15 /physEntAttr = {}
materialization
-
myCopyOfHP4: ProductPhysicalEntity serialNr = 'A121212' listPrice = 11.50 /author = 'J.K.Rowling' /taxrate = 15
Fig. 6. Product catalog modeled with materialization (using a respective UMLnotation by Oliv´e [23]) and applying composite attribute propagation mechanism (T1T3) to mimic deep instantiation
Our sample three-level product catalog (see Example 1) can be modeled by consecutively applying materialization relationships. Class ProductPhysicalEntity materializes class ProductModel which materializes class ProductCategory. Each class represents one level of abstraction (or, in this context, one level of materialization). Definition of shared attribute values at the more abstract object oa that are propagated to the more concrete object oc , and definition of attributes at oa , that are instantiated at oc , are facilitated by a set of attribute propagation mechanisms, introduced by Pirotte et al. [24] and further described by Dahchour et al. [8]. Type 1 propagation (T1) simply propagates values of attributes (monoor multivalued) from oa to oc , e.g., taxRate=20 from Car to Porsche911CarreraS. 3
Not to be mixed up with materialization of database views (materialized views).
Multi-level Abstraction
81
For better readability, we have marked propagated T1-attributes with a “/”character, such as /taxRate=20 at object Porsche911CarreraS. Type 2 propagation (T2) allows to define at ca an attribute, instantiated at oa with a set of possible values. This attribute is instantiated at oc with one value (T2mono) or a set of values (T2multi) from the set of possible values defined at oa (not considered in our example). Type 3 propagation (T3) allows to define a multi-valued attribute at ca , instantiated with a set of values at oa . Each such attribute value at oa is transformed to an attribute in oc and instantiated there. For example, class ProductCategory defines an attribute modelAttr with T3 that is instantiated at object Car with {maxSpeed}, and maxSpeed is instantiated at object Porsche911CarreraS (see Fig. 6) Attribute values defined at objects at higher materialization levels (e.g., taxRate = 20 at Car) can be accessed at objects at lower materialization levels (e.g., myPorsche911CarreraS) either by traversal of materialization links (myPorsche911CarreraS.model.model.taxRate) or, if taxRate is propagated using T1, by directly accessing the propagated attribute value, e.g., myPorsche911CarreraS.taxRate. Composing attribute propagation types, as described in [8], allows one to define attributes that have impact on two materialization levels below and a combination of propagation modes T1 and T3 can be used to mimic potencybased instantiation. The combination of propagation modes T1 and T3 has to the best of our knowledge not been considered in literature, but it can be used as follows to mimic potency-based instantiation. An abstract class ca defines an attribute with T1-T3 propagation. The attribute is instantiated with a set of values at instance oa of class ca and, due to propagation mode T1 (which is first applied), propagated to every materialization oc of abstract object oa . The attribute is propagated further in T3 mode to every materialization occ of oc , i.e., occ receives an attribute for every value of that attribute at oc and instantiates it. In this way, attributes with T1-T3 resemble attributes with potency 2 (see Section 3.1) and attributes with T1-T1-T3 resemble attributes with potency 3, and so forth. We illustrate this approach by the following example. As described in our sample problem statement, product models and physical entities belonging to product category car differ from other product models and physical entities, in that they have attributes maxSpeed and mileage, respectively. To be able to define these peculiarities, attribute modelAttr(T3) and physEntAttr(T1-T3) are introduced at class ProductCategory. As described in the previous paragraph, these attributes serve as placeholders for attributes introduced in instances of ProductCategory. At Car, modelAttr is instantiated with {maxSpeed} to define that there is an attribute maxSpeed in every materialization of Car, i.e. in Porsche911CarreraS and in Porsche911GT3. Furthermore, attribute physEntAttr is instantiated with {mileage} to define that there is an attribute mileage in every materialization of a materialization of Car, i.e. in myPorsche911CarreraS. Alternatively, one could explicitly specialize classes ProductModel and ProductPhysicalEntity by subclasses CarModel and CarPhysicalEntity,
82
B. Neumayr, M. Schrefl, and B. Thalheim
respectively. We do not further consider this alternative, because subclassing of classes which are part of materialization hierarchies is not considered in the literature on materialization and is somewhat counter-intuitive to the idea of materialization. Another powerful composite attribute propagation type, T3-T2, is discussed in [8], but is not needed to model our sample product catalog. We evaluate this modeling technique against our comparison criteria: 1. Compactness: Materialization allows modular and redundancy-free models. All peculiarities of domain concept car, including those concerning objects at level product model and physical entity, can be described local to one model element, namely Car. Thus, materialization effectively reduces accidental complexity. 2. Query flexibility: Materialization hierarchies (as in [8]) provide a pre-defined entry point for all objects at a specific materialization level by the class representing this materialization level, e.g., all objects at level physical entity can be addressed via class ProductPhysicalEntity. Relevant literature does not consider pre-defined query entry points that refer to all objects of one materialization level that directly or indirectly materialize some abstract object. However, one should in principle be able to retrieve such sets of objects by queries like (in SQL-Syntax): SELECT * FROM ProductModel p WHERE p.model = Car or to access all physical cars: SELECT * FROM ProductPhysicalEntity p WHERE p.model.model = Car . 3. Heterogeneous Level-Hierarchies: Literature on materialization does not discuss introducing additional materialization levels local to a specific object in the materialization hierarchy, like level brand local to Car. Since the use of T1-T3-attribute propagation mimics potency-based deep instantiation, additional levels would lead to similar problems as when using potency-based deep instantiation, i.e., lead to global model changes. Using materialization, propagation modes of attributes need to be adapted analogously as described above for potency values of potency-based deep instantiation. 4. Multi-level relationships: Pirotte et al. introduce attribute propagation. Since attributes can be regarded as (directed) relationships, T2 and T3 attribute propagation are a restricted form of multi-level relationships and partly capture (a) (meta-)classification of relationships and capture (c) specialization of range but lacks (b) Adressability/Queryability of Relationships at different levels. They also briefly mention materialization of relationships but do not discuss materialization of relationships in detail. They basically consider a relationship as a class that can be materialized like any other class. Pirotte et al. [24] also sketch the idea of materialization of aggregation, e.g., a product model is composed of various parts (bill of materials) and this bill of materials can be materialized to the level of physical entities. This idea could be elaborated and extended for materialization of relationships in order to support multiple relationship-abstractions.
Multi-level Abstraction
3.4
83
Multi-level Objects
In [19] we introduce multi-level objects (m-objects) and multi-level relationships (m-relationships) for the concretization of objects and relationships along multiple levels of abstraction. The basic ideas of this approach, which builds on the previous approaches discussed above, are (i) to define and describe the different levels of abstraction that relate to a single domain concept (e.g., the level descriptions, catalog, category, model, physicalEntity that relate to single domain concept, product catalog) and to encapsulate them into a single m-object (e.g., Product), and (ii) to combine aspects of the different abstraction hierarchies (aggregation, generalization, and classification) into a single concretization hierarchy. An m-object encapsulates and arranges abstraction levels in a linear order from the most abstract to the most concrete one. Thereby, it describes itself and the common properties of the objects at each level of the concretization hierarchy beneath itself. An m-object specifies concrete values for the properties of the top-level. This top-level describes the m-object itself. All other levels describe common properties of m-objects beneath itself. An m-object can concretize another m-object, which is referred to as its parent. A concretization relationship between two m-objects comprises aggregation, classification, and generalization. This does not mean that one m-object is at the same time component of, instance of and subclass of another m-object as a whole. Rather, a concretization relationship between two m-objects, such as between Car and Product in Fig. 7, is to be interpreted in a multi-faceted way as follows: (1) Aggregation: product category car (represented by the top level of m-object Car) is part of our product catalog (represented by the top-level of m-object Product). (2) Classification: the top-level of Car is instance of the second-top level (category) of m-object Product, it instantiates attribute taxRate by assigning a value to it. (3) Generalization: m-object Car specializes m-object Product by introducing a new level brand and by adding attribute maxSpeed to level model and attribute mileage to level physical entity. The non-top levels model and physical entity of m-object Car are regarded as subclasses of the corresponding levels of m-object Product. M-relationships are analogous to m-objects in that they describe relationships between m-objects at multiple levels of abstraction. M-relationships are bi-directional (in the context of data warehousing [21] we also provide for nary m-relationships with attributes). To facilitate referencing objects involved in binary relationships, we take, however, a (potentially) arbitrary directional perspective by considering one object in the source role and the other object in the target role of the relationship. Each m-relationship links a source m-object with a target m-object. Additionally, it connects one or more pairs of levels of the source and the target. These connections between source- and target-levels constitute the abstraction levels of an m-relationship. They define that m-objects at source-level are related with m-objects at target-level. We note that the generic roles source and target, which we introduced here for simplicity, may be easily replaced by relationship-specific role names.
84
B. Neumayr, M. Schrefl, and B. Thalheim
Product : catalog - desc:String = ‚Our Products’ - taxRate : Integer <model> - listPrice : Float - serialNr Book : category - taxRate = 15 <model> - author : String
Product Category
Car : category - taxRate = 20 - marketLaunch : date <model> - maxSpeed : Integer - mileage : Integer Porsche911 : brand - marketLaunch = 1964 <model> - porsche911club :boolean
HarryPotter4 : model - listPrice = 11.50 - author = ‚J.K.Rowling’
Porsche911CarreraS : model - listPrice = 91.838 - maxSpeed = 293 km/h
myCopyOfHarryPotter4 : physicalEntity - serialNr = ‚A121212’
myPorsche911CarreraS : physicalEntity - serialNr = ‚C333333' - mileage = 100000 - porsche911club = true
Car Brand
Porsche911GT3 : model - listPrice = 108.083 - maxSpeed = 310 km/h
Product Model
Product Physical Entity
Fig. 7. Product catalog modeled with m-objects (inherited attributes not shown)
Like m-objects, m-relationships can be concretized. Concretizing an mrelationship means to substitute its source or its target for a direct or indirect concretization. The concretizes-relationship between two m-relationships expresses instantiation and/or specialization, depending on the levels one looks at. Consider m-relationship producedBy between m-objects Product and Company in Fig. 8, it specifies multiple source- and target-levels. The relationship expresses that product categories are produced by industrial sectors, product models by enterprises and physical entities by factories. This m-relationship is concretized by producedBy between Car and CarManufacturer, which in turn is further concretized between Porsche911CarreraS and PorscheLtd, which in turn is further concretized between myPorsche911CarreraS and PorscheZuffenhausen. We evaluate this modeling technique against our comparison criteria: 1. Compactness: M-objects allow modular and redundancy-free modeling by encapsulating all information concerning one domain concept into one mobject. For example, all information concerning domain concept car is encapsulated into one m-object, Car.
Multi-level Abstraction
Product <model>
producedBy
Company <enterprise>
Car : category <model>
producedBy
CarManufacturer : industrialSector <enterprise>
Porsche911CarreraS : model
producedBy
PorscheLtd : enterprise
myPorsche911CarreraS : physicalEntity
producedBy
PorscheZuffenhausen : factory
85
Porsche911 : brand <model>
Fig. 8. Product catalog modeled with m-objects and their m-relationships [19]
2. Query flexibility: Pre-defined, qualified identifiers can be used to refer to the set of m-objects at a specific level that belong to a specific m-object. The set of all m-objects that are descendants of a given m-object at some specified level is identified by qualifying the name of the m-object by that level. E.g., (1) to refer to all models of product category Car one writes Carmodel, (2) to refer to the physical cars of product category car: CarphysicalEntity, and (3) to refer to all physical entities of products: ProductphysicalEntity. (See [18] for details). 3. Heterogeneous Level-Hierarchies: Hierarchies of m-objects are easy to extend by an additional level (e.g., brand) in one sub-hierarchy (e.g., the subhierarchy rooted by car) without affecting other sub-hierarchies (e.g., the sub-hierarchy rooted by book). Fig. 7 already shows level brand with mobject car. To add this level to cars, but not to books, requires no changes above m-object car or in sibling hierarchies of m-objects (such as book). 4. Multiple relationship abstractions: M-relationships between m-objects are like m-objects described at multiple levels, associating one or more pairs of levels of the linked m-objects. M-relationships are interpreted in a multifaceted way, once as relationship occurrence (or sometimes also called relationship instance or link) and once as relationship class (or sometimes also called relationship type or association), or also as meta-relationship class. They take on these multiple roles depending on what pairs of linked levels one considers and on whether one considers the linked m-objects in an instance or a class-role. As m-relationships may also be concretized along the concretization hierarchy of linked m-objects, they support multiple relationship abstractions.
86
B. Neumayr, M. Schrefl, and B. Thalheim
M-Relationships cover (a) (Meta-)Classification of Relationships, (b) Adressability/Queryability of Relationships at different levels, and (c) Specialization/Instantiation of Domain and Range. 3.5
The Higher-Order Entity Relationship Model
The higher-order entity-relationship model (HERM ) [28] is a high-level data model that builds on the classical entity-relationship model [7] and subsumes most other extended ER models. HERM does not provide special modeling constructs for multi-level abstraction, but we will discuss in the following to what extent our sample problem can be represented by two alternative modeling approaches, through use of a specialization schema and an overlay schema (both explained below) with the general purpose modeling constructs of HERM. One of the distinguishing features of HERM are higher-order relationship types. While a (first-order) relationship type associates one or more entity types (called in HERM: ’components of the relationship type’), a higher-order relationship type also associates relationship types. Also note that in HERM, subtypes are typically modeled by unary relationship types, i.e., relationship types having only one component, e.g., entity type Product is specialized by unary relationship type Car. We first discuss the modeling approach using a specialization schema (see Fig. 9). Domain concepts at the highest abstraction level are modeled as entity types and are specialized by domain concepts at lower abstraction levels which are modeled as unary relationship types. For example, entity type Product is specialized by unary relationship types Car and Book, which are, in turn, specialized by unary relationship types Porsche911 and HarryPotter4, respectively. Relationship types myPorsche911CarreraS and myCopyOfHarryPotter4 define singleton classes (note that singleton classes are not typical for HERM). Specialization may be applied by adding properties, by providing values for properties, or by both (e.g., Book for Product with value assignment for taxRate and adding property author). HERM does not provide modeling constructs for defining abstraction levels; names of abstraction levels (e.g., Product Model in Fig. 9) are rather depicted as annotations for better readability. In order to define that an attribute semantically belongs to a certain abstraction level, like that listPrice belongs to level product model, and that it has to be instantiated at this level, one would additionally have to use explicit constraints. We note that this is a general shortcoming of the ER approach and not peculiar to HERM. Alternatively, one could model the sample product catalog by multiple specialization hierarchies, each rooted in an entity type representing an abstraction level (analogous to our reference UML solution in Fig. 1). HERM requires that each type has a key that may be composed of attributes of the type and attributes of the related types. Relationship types inherit identification from components but inherited identification may be overriden by a key that replaces the inherited key, e.g., ISBN is the identification for type Book.
Multi-level Abstraction
87
H
maxSpeed=310km/h listPrice=108.083 HH Porsche911GT3 H H porsche911club porsche911club=true
H H HH mileage=100000 HH H HH myPorsche911H Porsche911 H H Car Porsche911 HH HHCarreraS HHCarreraS HHH maxSpeed listPrice serialNr HmarketLaunch H =91.838 H =293km/h maxSpeed
mileageHtaxRate=20
ID
desc
taxRate
serialNr=’C33333333’
marketLaunch=1964
Product
I @
author HHH HHBook H
listPrice @
ISBN
product catalog
HH HHH myCopyOf H HarryPotter4 HHarryPotter4 H HH HH serialNr=’A121212’ listPrice=11.50
author=J.K. Rowling
taxRate=15
product category
product brand
product physical entity
product model
Fig. 9. Selected parts of the product catalog modeled with HERM (specialization schema)
We now discuss the modeling approach using an overlay schema (see Fig. 10). Abstraction levels are modeled as entity types (e.g., Catalog, Category, Brand, Model) or unary relationship types (e.g., Entity). The order of abstraction levels is modeled by binary relationship types (e.g., Brand to Model). This overlay schema combines the global schema of the product catalog with specialized schemas of categories car and book and of brand Porsche911. For this purpose, it uses optional attributes which may either be populated or not. The population is constrained by the following null-value constraints: [author]0 binds the population of values for authors in entity type Model to those models whose category kind is Book, [mileage]0 binds the population of values for mileage in relationship type Entity to those physical entities whose category kind is Car, and [clubAccepted]0 binds the population of values for clubs in relationship type Entity to those physical entities whose category kind is Car and whose brand allows club membership. ID
[marketLaunch]0 Brand
H HH Brand to HH Model (0,1) H
[club]0 Catalog ID
Model
[author]0
HH Catalog H to HHCategory H
[clubAccepted]0
ID
listPrice
6
H EntityHH HH H [mileage]0 desc
[maxSpeed]0 (1,1)
HH Category H - of HHModel H
-
Category
taxRate kind ID
desc
Fig. 10. Selected parts of the product catalog modeled with HERM (overlay schema)
88
B. Neumayr, M. Schrefl, and B. Thalheim
We evaluate this modeling technique against our comparison criteria: 1. Compactness: In general, HERM allows a very compact and redundancy-free representation of schemata. However, using specialization types (see Fig. 9) one has to use explicit constraints in order to define that an attribute has to be instantiated at a certain level. The overlay style provides a rather compact way of modeling application domains with many different specializations. It does, however, also not completely fulfill the modularity criterion. 2. Query flexibility: In the specialization schema, abstraction levels are not explicitly modeled, therefore it is not possible to query all objects that belong to a certain abstraction level, e.g., all product models. This is possible in the overlay schema. HERM supports declaration of extended views [28], view towers, and visual queries [30]. It can be combined with aggregation and abstraction functions. The queries and the views must, however, be based on the schema specification. 3. Heterogeneous level-hierarchies: Extended ER models support multi-level hierarchies if the modeling style supports it. The overlay schema in Fig. 10 is not prone to the introduction of additional abstractions whereas the specialization model in Fig. 9 supports multiple and flexible abstraction. 4. Multiple relationship-abstractions: Multiple relationship-abstractions are well supported as long as the relationships among hierarchies are homogeneous. The missing Brand hierarchy layer and changing semantical constraints among hierarchies cause problems. HERM allows one to build relationship types on relationship types and to introduce cluster types. The component diagram in Fig. 11 can be unnested through the introduction of a relationship type ProducedBy. The specialization of this type is not well supported within HERM. 3.6
The Component Model
The component model [29] enhances HERM by the explicit introduction of an encapsulation based on component schemata, views defined for component schemata with or without update propagation through these views, and ports to components that allow an exchange of data among components based on the view assigned to the port. Components may themselves consist of components. Views of inner components may be visible also to outer components. For instance, the Car component in Fig. 11 has its own HERM schema. It has only one view for the outside world that consists of all attributes of this component and the inherited attributes of its inner components. Components may have various ports and views for exchange. Ports are associated through harnesses. Harness producedBy associates products and companies. The harness has a specialisation hierarchy that follows the specialisation hierarchies in component Product and in component Company. It has been proven in [25] that any HERM schema can be transformed into componentized schemata that separate different concerns.
Multi-level Abstraction producedBy
'
◦ desc ◦ taxRate ◦ listPrice ◦ serialNr ' Car $ ◦ maxSpeed ◦ taxRate=20 '◦ marketLaunch ◦ mileage $ Porsche911 ' $
Product Catalog
'
◦ author
Book ◦ taxRate=15 ' HarryPotter4 author = ◦ ’J.K. Rowling’ ◦ listprice = 11.50 '
myCopy OfHP4
Porsche911CarreraS marketLaunch ◦ = 1964 ◦ maxSpeed = 293km/h ◦ porsche911club ◦' listPrice = 91.838 $ $ myPorsche 911CarreraS
serialNr = ’C33333333’ mileage = ◦ 100000 porsche911club ◦ = true ◦
serialNr = ◦ ’A121212’
& & & &
89
& % & % & % &
'
Porsche 911GT3
$ ' $ $Company ' $ Car$ Manufacturer '
PorscheLtd.
$'
$
Porsche Zuffenhausen
listPrice = 108.083 maxSpeed = ◦ 310km/h ◦
% % &
$
% & % & % & % &
% % % %
Fig. 11. Product catalog modeled with components
We evaluate this modeling technique against our comparison criteria: 1. Compactness: The component model inherits the advantages and the quality properties of HERM. It provides additional facilities for zooming out and zooming in depending on the intended level of detail. Components have their ports and views assigned to ports. These ports are connected through harnesses. The harness producedBy has been sketched in Fig. 11. It associates catalog products with companies, car manufacturers with cars, and ‘my Porsche’ to Zuffenhausen. As with HERM, explicit constraints have to be used to define at which level an attribute has to be instantiated. For example marketLaunch and mileage are both introduced at Car. To model that marketLaunch has to be instantiated by objects at level brand, e.g., Porsche911, and mileage has to be instantiated by objects at level physical entity, e.g., myPorsche911CarreraS, one has to use explicit constraints (not shown in Fig. 11). 2. Query flexibility: Query flexibility is similar to the query flexibility of HERM. Visual SQL supports the definition of views. Therefore, any query that can be expressed through Visual SQL can also be defined for component schemata. 3. Heterogeneous level-hierarchies: Handling of heterogeneous level-hierarchies is not restricted to the modeling style for component schemata. Any new level within a hierarchy can be introduced as long as the separation of components within a hierarchy level is based on exclusion constraints for each sub-component. For instance, we assume that Car and Book specify components for exclusive classes of products. 4. Multi-level relationships: The problem of handling multiple and heterogeneous relationships among components is resolved through the introduction of harnesses. Any kind of specialisation and generalisation can be defined also for harness schemata.
90
4
B. Neumayr, M. Schrefl, and B. Thalheim
Conclusion
We discussed the evolution of modeling techniques for multi-level abstraction and analyzed six modeling techniques in detail. The results are summarized in Fig. 12. We wish to stress that our evaluation only addresses the suitability of these techniques for multi-level abstraction as discussed in Section 2, but does not judge their overall quality. Compactness Query Heterogeneous Flexibility Levels Deep Instantiation + + − Powertypes (Simple) ∼ ∼ − Powertypes (Ext.) ∼ + + Materialization + ∼ − M-Objects + + + HERM ∼ −2 /+3 +2 /−3 Component Model ∼ ∼ + Legend: (+) full support (∼) partial support (−) no support 1) using OCL 2) specialization schema, 3) overlay schema
Relationship Abstraction ∼ −/∼1 −/∼1 ∼ + ∼ +
Fig. 12. Summary: evaluation of multi-level abstraction techniques
The discussed techniques provide to some extent support for describing multilevel abstraction, but only m-objects and m-relationships met the requirements benchmarked by our comparison criteria. We want to highlight that the running example was rather simple (among others, we did not consider multi-level abstraction of composite objects) and that in practice to-be represented domains tend to be much more complex. Thus, modeling with multi-level abstraction in the general case is much harder than discussed in this chapter. None of the discussed modeling techniques is capable of fully supporting multi-level abstraction in the general case. But, there are specific complex modeling problems that are well supported by some of the techniques and that could not be concisely modeled otherwise. For example, m-objects and m-relationships, in an extended form [21], may be used to model heterogeneous dimension hierarchies and cubes in data warehouses. This has been demonstrated by a proof-of-concept prototype [27] implemented on top of Oracle DB. Multi-level abstraction is also of interest in the realm of ontologies and the Semantic Web for representing domains where the borderline between individuals and classes is not clear cut [17]. We showed [20] how m-objects and mrelationships can be mapped to the decidable fragment of the web ontology language OWL extended with closed-world integrity constraints. It remains, however, open to future work to evaluate this approach in practice. Aschauer et al. [1] showed how clabjects (based on the ideas of Atkinson and K¨ uhne) can be applied for modeling industrial automation systems in practice. They found that deep instantiation is well applicable for their modeling problem
Multi-level Abstraction
91
if extended by ’gradual instantiation’ (i.e., introducing shared and default values for properties at higher levels). Although first approaches to multi-level abstraction date back 20 years and more, modeling techniques for multi-level abstraction are still in their infancy and are not appropriate for concisely modeling multi-level abstraction in the general case. To overcome these limitations, future work on multi-level abstraction should address the following open issues: – Employment of multi-level abstraction to other modeling primitives beyond properties and simple relationships, such as Composite Objects and Roles, and to common modeling languages, such as UML. – Query Languages for multi-level models, especially for extracting two-level models or retrieving multi-level sub-models. – Multi-Level Constraints, especially in the context of relationships (e.g., cardinality constraints) or for keys (multi-level identifiability).
References 1. Aschauer, T., Dauenhauer, G., Pree, W.: Multi-level Modeling for Industrial Automation Systems. In: EUROMICRO-SEAA, pp. 490–496 (2009) 2. Atkinson, C.: Meta-Modeling for Distributed Object Environments. In: EDOC (1997) 3. Atkinson, C., Auletta, V.: The Essence of Multilevel Metamodeling. In: Gogolla, M., Kobryn, C. (eds.) UML 2001. LNCS, vol. 2185, pp. 19–33. Springer, Heidelberg (2001) 4. Atkinson, C., K¨ uhne, T.: Model-Driven Development: A Metamodeling Foundation. IEEE Software 20(5), 36–41 (2003) 5. Atkinson, C., K¨ uhne, T.: Reducing accidental complexity in domain models. Software and System Modeling 7(3), 345–359 (2008) 6. Cardelli, L.: Structural Subtyping and the Notion of Power Type. In: POPL, pp. 70–79 (1988) 7. Chen, P.P.: The Entity-Relationship Model - Toward a Unified View of Data. ACM Trans. Database Syst. 1(1), 9–36 (1976) 8. Dahchour, M., Pirotte, A., Zim´ anyi, E.: Materialization and Its Metaclass Implementation. IEEE Trans. Knowl. Data Eng. 14(5), 1078–1094 (2002) 9. Goldstein, R.C., Storey, V.C.: Materialization. IEEE Trans. Knowl. Data Eng. 6(5), 835–842 (1994) 10. Gonzalez-Perez, C., Henderson-Sellers, B.: A powertype-based metamodelling framework. Software and System Modeling 5(1), 72–90 (2006) 11. Gutheil, M., Kennel, B., Atkinson, C.: A Systematic Approach to Connectors in a Multi-level Modeling Environment. In: Busch, C., Ober, I., Bruel, J.-M., Uhl, A., V¨ olter, M. (eds.) MODELS 2008. LNCS, vol. 5301, pp. 843–857. Springer, Heidelberg (2008) 12. Henderson-Sellers, B., Gonzalez-Perez, C.: Connecting Powertypes and Stereotypes. Journal of Object Technology 4, 83–96 (2005) 13. Hull, R., King, R.: Semantic database modeling: survey, applications, and research issues. ACM Comput. Surv. 19(3), 201–260 (1987) 14. Klas, W., Schrefl, M.: Metaclasses and Their Application - Data Model Tailoring and Database Integration. Springer, Heidelberg (1995)
92
B. Neumayr, M. Schrefl, and B. Thalheim
15. K¨ uhne, T., Schreiber, D.: Can programming be liberated from the two-level style: multi-level programming with deepjava. In: OOPSLA, pp. 229–244 (2007) 16. K¨ uhne, T., Steimann, F.: Tiefe Charakterisierung. In: Modellierung, pp. 109–119 (2004) 17. Motik, B.: On the Properties of Metamodeling in OWL. J. Log. Comput. 17(4), 617–637 (2007) 18. Neumayr, B.: Multi-Level Modeling with M-Objects and M-Relationships. Ph.D. thesis, Johannes Kepler Universit¨ at Linz (2010), http://www.dke.jku.at/research/publications/PT1002.pdf 19. Neumayr, B., Gr¨ un, K., Schrefl, M.: Multi-Level Domain Modeling with M-Objects and M-Relationships. In: Link, S., Kirchberg, M. (eds.) APCCM. CRPIT, vol. 96, pp. 107–116. ACS, Wellington (2009) 20. Neumayr, B., Schrefl, M.: Multi-level Conceptual Modeling and OWL. In: Heuser, C.A., Pernul, G. (eds.) ER 2009 Workshops. LNCS, vol. 5833, pp. 189–199. Springer, Heidelberg (2009) 21. Neumayr, B., Schrefl, M., Thalheim, B.: Hetero-Homogeneous Hierarchies in Data Warehouses. In: Link, S., Ghose, A. (eds.) APCCM. CRPIT, vol. 110, pp. 61–70. ACS, Brisbane (2010) 22. Odell, J.J.: Power Types. In: Advanced Object-Oriented Analysis & Design Using UML, pp. 23–32. Cambridge University Press, Cambridge (1998); also published as: James Odell: Power Types. JOOP 7(2), 8–12 (1994) 23. Oliv´e, A.: Conceptual Modeling of Information Systems. Springer, Heidelberg (2007) 24. Pirotte, A., Zim´ anyi, E., Massart, D., Yakusheva, T.: Materialization: A Powerful and Ubiquitous Abstraction Pattern. In: VLDB, pp. 630–641, 0605 (1994) 25. Schewe, K.D., Thalheim, B.: Component-driven engineering of database applications. In: APCCM, pp. 105–114 (2006) 26. Schrefl, M., Tjoa, A.M., Wagner, R.: Comparison-Criteria for Semantic Data Models. In: ICDE, pp. 120–125 (1984) 27. Sch¨ utz, C.: Extending data warehouses with hetero-homogeneous dimension hierarchies and cubes – A proof-of-concept prototype in Oracle. Master’s thesis, Johannes Kepler Universit¨ at Linz (2010), http://www.dke.jku.at/research/publications/MT1002.pdf 28. Thalheim, B.: Entity-Relationship Modeling: Foundations of Database Technology. Springer, Heidelberg (2000) 29. Thalheim, B.: Component development and construction for database design. Data Knowl. Eng. 54(1), 77–95 (2005) 30. Thalheim, B.: Visual SQL (2008), http://www.informatik.uni-kiel.de/en/is/miscellaneous/visualsql/
Random Thoughts on Multi-level Conceptual Modelling Brian Henderson-Sellers School of Software Faculty of Engineering and Information Technology University of Technology, Sydney P.O. Box 123, Broadway, NSW 2007, Australia
[email protected] Abstract. Conceptual modelling has a lengthy history. In the context of software engineering, over the last three decades, conceptual modelling has moved from largely being used for data representation, especially in databases, to applications in metamodelling, ontologies, method engineering, standards and business process management. Here, I reflect, somewhat randomly, upon some of these issues and possible future application/research areas for conceptual modelling. Of prime importance must be the synergistic exploration of model quality, a topic poorly represented in the conceptual modelling literature. Keywords: Conceptual modelling, metamodelling, ontologies, standards, methodology, quality.
1 Introduction Modelling has many guises e.g. physical modelling for constructing cars or testing the flight capability of aircraft, for evaluating the stress for dams and reservoirs before construction e.g. [1], for weather forecasting, for representing a business information system e.g. [2], for mathematical modelling of ecosystems e.g. [3] or for representing the more ephemeral characteristics of music e.g. [4]. In information systems and computing in general, many of these modelling approaches are, of course, irrelevant; rather, the subset of modelling known as “conceptual modelling” has been highlighted within the community [5]. In reality, conceptual modelling is just modelling but applied to cognitive artefacts, particular designs for software systems in our context. Conceptual modelling typically uses graphical representations (ER, data modelling, object-oriented modelling languages and so on) as compared to mathematical modelling using partial differential equations, the more likely norm in the physical sciences. In this paper, I discuss some current issues, questions and concerns related to conceptual modelling. Since modelling relies heavily on the notion of abstraction [6], this is discussed first – in Section 2, where several kinds of mapping are highlighted. Abstraction leads naturally to the recognition that modelling may be useful on several cognitive “levels”. In Section 3, multi-level frameworks relevant to IT are explained, introducing specifically the notion of “strict metamodelling” and its problems in modern-day usage together with the mathematically based powertype, which is used in Kaschek/Delcambre (Eds.): The Evolution of Conceptual Modeling, LNCS 6520, pp. 93–116, 2011. © Springer-Verlag Berlin Heidelberg 2011
94
B. Henderson-Sellers
some recent metamodelling standards. The ideas of metamodels are closely related to those of ontologies as discussed in the conceptual modelling and software engineering literature over the last decade (Section 4). How ontological thinking is merged with metamodelling approaches is discussed in detail. All of these discussions lead to the obvious question: “How do we assess model quality?” Some initial observations are found in Section 5 followed by a short evaluation of some of the application areas for conceptual modelling (Section 6). We end with a short indication of some possible future research directions (Section 7).
2 Abstraction Models all have one thing in common: they are representations of conceptualizations of the real world, called here generically the System Under Study or SUS (Figure 1(a)). The conceptualization creates a cognitive or mental model that is said to be an abstraction of the (part) of reality under study e.g. [7-10], which simplifies our conception of the real domain (the SUS) by removing (unnecessary) detail [11,12]. The elements of the cognitive model are then represented in some fashion e.g. using a graphical notation, a natural language, a mathematical equation – which may itself add more detail peculiar to the model of the SUS (see the TI abstraction of Guichiglia and Walsh [13]). This is named the plenty property by Kaschek [14]. Although the link from Conceptualization to Model (Figure 1) is called the representational mapping in ontology (e.g. [8]), many modellers in software engineering and information systems label the “shortcut” (the horizontal “refers-to” relationship in Figure 1(a)) from Model to SUS as “represents” (Figure 1(b)). This can be formulated [9,15,16] as a mapping α given by α=τoπ
(1)
where π represents the abstraction mapping (called “projection” in [9]) and τ the representation mapping of Figure 1(a).
(a)
represents
Model
depicts
abstracts
refers to
Conceptualization
(b)
Conceptualization
SUS
Model
abstracts
represents
Fig. 1. Ogden and Richards’ [17] triangle applied to modelling
SUS
Random Thoughts on Multi-level Conceptual Modelling
95
However, although we acknowledge this two-stage process from SUS to model via conceptualization as being a more appropriate description of the modelling process, clearly differentiating between abstraction and representation, for the remainder of our discussion of the literature, we will adopt the more prevalent view that “a model represents the SUS” (Figure 1(b)). Abstraction is thus crucial for modelling. Informally, Giunchiglia and Walsh [13] propose that 1) abstraction maps a representation of the problem to a new, more “abstract” representation. (This is the essence of modelling); 2) by preserving relevant, desirable properties, the abstraction process allows one to solve the original problem; 3) by throwing away details, the result of the abstraction process provides a simpler problem to be solved than the original; and proceed to formalize the idea of an abstraction function, α, from a notion1 S to a notion A where (e.g. [18]) α: S ⇒ A
(2)
Formally, an abstraction can be defined as a mapping between two formal systems, which may be similar or dissimilar (e.g. [19]). Here, a formal system, Σ, is a set of formulae, Θ, written in a language Λ i.e. Σ = (Λ,Θ)
(3)
Giunchiglia and Walsh [13] then equate the set of formulae, Θ, with the language, Λ. Then, an abstraction is the mapping between two formal systems, Σ = (Λ). The definition (which addresses property 1 above) given in [13] is: f : Σ 1 ⇒ Σ2
(4)
where the mapping is between a pair of formal systems (Σ1, Σ2) with languages Λ1 and Λ2 respectively and there is an effective total function, fΛ, that maps “ground language” Λ1 to “abstract language” Λ2 i.e. fΛ : Λ1 → Λ2
(5)
Equation (5) is given in [11] as the basic definition of abstraction. These authors note that this mapping needs to be both total and surjective so that, respectively, for each element of Λ1, fΛ is defined. Giunchiglia and Walsh [13] introduce three kinds of abstraction, which they call TD, TI and TC (where a TC abstraction is both a TD and a TI abstraction). These are picture in Figure 2. In a TD abstraction, some elements of the base situation are deleted; TI abstractions are when the abstract language has additional axioms or inference rules; and TC abstractions are effectively bijective mappings. 1
Although we do not discuss notions here, we follow Kashek [18] in assuming models to be implicitly systems of notions.
96
B. Henderson-Sellers Base language
Abstract language
TD
TI
TC
Fig. 2. Three kinds of abstraction of Giunchiglia and Walsh [13]
Keet [12] proposes a set of abstractions similar to the TC, TD and TI abstractions of [13] but more relevant to conceptual modelling and ontologies as follows: R-abs: An abstraction is said to be an Rabs abstraction if and only if the more detailed type φ abstracts into its related parent type ψ (6) This makes an isomorphic mapping F-abs: An abstraction is said to be an Fabs abstraction if and only if a more detailed theory T0 (with φ1…φn entity types, relations and constraints among them) abstracts into a single related parent type, ψ, in T1 that is distinct from (7) any φi This makes F-abs an on to or surjective morphism D-abs: An abstraction is said to be a Dabs abstraction if and only if a more detailed theory T0 with φ1…φn entity types, relations and constraints abstracts into theory T1 with φ′1…φ′n entity types, relations and constraints, and with m=Pn.desiredPrice AND Pn.action=”sell” do sellStock(S,Pn);
Fig. 3. Integration Rule Example for a Stock Application [38]
process execution correctness guard as well as a potential rollback point during the recovery process [37,39]. Given that concurrent processes do not execute as traditional transactions in a service-oriented environment, inserting APs at critical points in a process is important for checking consistency constraints and potentially reducing the risk of failure or inconsistent data. An AP also serves as a milestone for backward and forward recovery activities. When failures occur, APs can be used as rollback points for backward recovery, rechecking pre-conditions relevant to forward recovery. The work in [37,39] has developed a prototype of APs using the Process Modeling Language (PML) described in [27,28]. An AP is defined as: AP = , where: - apID is the unique identifier of the AP - apParameters is a list of critical data items to be stored as part of the AP, - IRpre is an integration rule defining a pre-condition, - IRpost is an integration rule defining a post-condition, - IRcond is an integration rule defining additional application rules. In the above notation, * indicates 0 or more occurrences, while ? indicates zero or one optional occurrences. IRpre, IRpost, and IRcond are expressed in the integration rule format introduced in Figure 2, where the eventName is the name of the assurance point that triggers the rule. For IRpre and IRpost, a constraint C is always expressed in a negative form (not(C)). The action of a rule is invoked if the pre or post condition is not true, invoking a recovery action or an alternative execution path. If the specified action is a retry activity, then there is a possibility for the process to execute through the same pre or post condition a second time. In such a case, IRpre and IRpost rules support the specification of a second action to invoke a different recovery procedure the second time through. In its most basic form, the recovery action of an integration rule simply invokes an alternative process. Recovery actions can also be one of the following actions: - APRollback: APRollback is used when the entire process needs to compensate its way back to the start of the process. - APRetry: APRetry is used when a process needs to be backward recovered using compensation to a specific AP. The backward recovery process will go to the first AP reached as part of the compensation process. The pre-condition defined in the AP is re-checked before resuming the execution.
216
S.D. Urban et al.
- APCascadedContingency (APCC): APCC is a backward recovery process that searches backwards through the hierarchical nesting of processes to find a contingent procedure for a failed sub-process. During the APCC backward recovery process, when an AP is reached, the pre-condition defined in the AP is re-checked before invoking a contingent procedure for forward recovery. When the execution of a process reaches an AP, integration rules associated with the AP are invoked. The condition of an IRpost is evaluated first. If the postcondition is violated, the action invoked can be one of the pre-defined recovery actions as described above. If the post-condition is not violated, then an IRpre rule is evaluated before the next service execution. If the pre-condition is violated, one of the pre-defined recovery actions will be invoked. If the pre-condition is satisfied, the AP will check for any conditional rules (IRcond) that may exist. IRcond rules do not affect the normal flow of execution but provide a way to invoke parallel activity based on application requirements. Note that the expression of a pre-condition, post-condition or any additional condition is optional. As an example, consider a subset of an online shopping process, as shown in Figure 4, where two APs are inserted. Both APs have integration rules that must be checked when the process execution reaches the APs. The cop and top in the process indicate the compensation and contingency of the attached activity, respectively. AP1 is orderPlaced, which reflects that the customer has finished placing the shopping order. Before executing the payment activity, the pre-condition at AP1 is checked to guarantee that the store has enough goods in stock. Otherwise, the process invokes the backOrderPurchase process instead. Similarly, the CreditCardCharged AP2 after payment activity has a post-condition that further guarantees that the in-stock quantity must be in a reasonable status (not less than zero) after the decInventory operation. Otherwise, a recovery action APRetry must be invoked to recover the process back to AP1 and re-execute the payment activity. If the post-condition fails after re-execution, then APRollback will be invoked to abort the overall process. 4.3
Dynamic Behavior with Invariants
APs together with integration rules allow data consistency conditions to be checked at specific points in the execution of a process [37], using rule actions to invoke recovery procedures. In some applications, however, stronger condition checking techniques may be needed to monitor data consistency. As a result, an additional way to use rules together with APs is through the use of invariants. An invariant is a condition that must be true during process execution between two different APs. An invariant is designed for use in processes where 1) isolation of data changes in between service executions cannot be guaranteed (i.e., critical data items cannot be locked across multiple service executions), and 2) it is critical to monitor constraints for the data items that cannot be locked. The data monitoring functionality provided by our previous work with DEGS makes it possible to monitor invariant conditions. Invariants provide a stronger way of monitoring constraints and guaranteeing that a condition holds for a specific
The Dynamics of Process Modeling
217
Process
Operation
Add to cart
Select shipping method
Cop
Compensation
Top
Contingency Assurance Point
Payment information input
cop (AbortOrder)
Place an order
AP1
AP2
Rule Invocation
OrderPlaced (orderId, itemID, N)
cop(creditBack)
Charge credit card
top(eCheckPay)
Dec inventory
cop(incInventory)
CreditCardCharged (orderId, cardNumber, amount)
create rule QuantityCheck::pre event: OrderPlaced (orderId) condition: exists(select L.itemId from Inventory I, LineItem L where L.orderId=orderId and L.itemId=I.itemId and L.quantity>I.quantity) action: backOrderPurchase(orderId)
create rule QuantityCheck::post event: CreditCardCharged (orderId, cardNumber, amount) condition: exists(select L.itemId from Inventory I, LineItem L where L.orderId=orderId and L.itemId=I.itemId and I.quantity
As can be seen in the example, WSCI supports similar error-handling concepts as “programming in the large” approaches such as BPEL, i.e., events, transactions and compensation. The difference is that no global view and coordination exists and the descriptions are only defining the behavior of a single service (e.g., which compensating actions are invoked in case of an exception) in a declarative manner. In contrast to WSCI, which only describes the behavioral interface of a single service, WS-CDL aims to extend this model and support a global (but not centrally coordinated) view on the choreography. Since 2005, WS-CDL has the status of a candidate recommendation at W3C. A description of a choreography in WS-CDL has in some respect a similar structure as XPDL or BPEL process definitions. It does not only describe the roles and relationships of the different partners in the process, but also the different activities that the partners can execute along with their associated control flow structures in the form of WorkUnits. This concept of WorkUnits can also be used to describe the error-handling behavior in the choreography. Exceptions that occur at run-time can be caught by so-called ExceptionBlocks and be handled in exception workunits. Discussion. Although WS-CDL can be used to describe similar aspects of a process as BPEL, BPMN or XPDL, the goal of the WS-CDL designers is to have descriptions that are even more independent from the actual implementation below. For example, the individual actions do not have to be web services and could also be Java or C# programs or executable BPEL processes. 13
http://www.w3.org/TR/wsdl
238
D. Jannach and A. Gut
While this independence from technical details is desirable in particular from an academic perspective, choreography approaches suffer from their limited acceptance in practical environments. Even the standardization efforts at W3C have come to a halt in the last years, probably because BPEL in the current version contains the concept of abstract processes, which supports the description of interfaces without requiring a specification of the concrete execution logic. Recently, approaches toward a further enhancement of BPEL (BPEL4Chor) with features required for choreography modeling have been proposed in [DKLW07]. Since BPEL is today’s de-facto standard for modeling web service processes, we expect that languages such as WS-CDL will be replaced in the future by choreography-enhanced process modeling languages.
4
Toward More Intelligent Error Handling Methods
The approaches discussed so far are mostly based on the throw-and-catch error handling pattern as well as on transaction rollbacks or compensation actions, which are explicitly modeled by the designer of the process. In practical applications, such approaches have their limitations in particular when it comes to the problem of recovery or repair, i.e., when the problem is to complete the process despite the presence of an error. If, for instance, one of the involved web services is not reachable, in many domains the option exists to use another service that provides the same functionality; another option in that situation could be to try a different set of activities to reach the same goal (think, e.g., of different payment services). With current approaches it is possible to model such recovery actions only through explicit alternative execution paths or compensation actions. Since, in reality, many or all activities of a process may fail and every single failure may require different recovery activities, the resulting process models soon become too complex. As a result, process designers will only model repair actions for a few of the possible error cases and will accept a rollback for the other situations. In the following, we will summarize recent approaches toward more intelligent error-handling techniques that support the precise identification of the problem, the automated discovery of alternative web services, or the dynamic computation of suitable recovery plans. 4.1
Error Diagnosis and Repair
In order to intelligently handle a problem during process execution, e.g., through the usage of an alternative service that provides the same functionality, one goal could be to find out in a first step, which of the activities could have caused the problem. Only at the first glance, this looks like an easy problem, which can be solved by modeling an exception handler for every single task in the process. Note, however, that a service can also raise an exception due to wrong input data calculated by an erroneous preceding activity. Thus, the goal of diagnosis techniques is to identify a set of already executed activities that could have potentially contributed to the observed problem.
Exception Handling in Web Service Processes
239
Most recent approaches toward advanced error analysis for workflows are based on Model-based Diagnosis(MBD) [Rei87] techniques. Figure 6 shows the basic principle of Model-Based Diagnosis, which was originally developed to diagnose physical systems such as electronic circuits but later on also used to debug software systems, see e.g., [FSW99], [Wot02], or [FFJS04].
Fig. 6. Principle of Model-based Diagnosis
As the name suggests, the approach is based on a model of the real-world system, which is often referred to as SD, the System Description. In an electronic circuit diagnosis scenario, SD contains the list of elements of the circuit (e.g., the different gates), their interconnections and also a description of the relevant physical laws. Based on this model, the system behavior can be simulated. In the domain of electronic circuits, we would for example test different inputs to the circuit and observe how the values are propagated to the output lines. Basically, the diagnosis approach consists of a comparison of the observed behavior of the real system with the expected behavior, which we can determine with the help of a simulation run. If there are discrepancies, we know that that there is some error in the real system (assuming that the model is correct and captures reality properly). The problem now is to find out, which of the components of the system causes the error and should be replaced. In MBDliterature, a set of components of SD that – if assumed to be faulty – explain the observed behavior of the system, is called a diagnosis. Typically, we are only interested in minimal diagnoses, i.e., we do not want to include components in the diagnosis that are not relevant for explaining the error. The set of (minimal) diagnoses for a given problem can for instance be calculated with Reiter’s Hitting Set algorithm[Rei87]. Figure 6 also indicates that finding a diagnosis is normally not the ultimate purpose of model-based problem analysis. Commonly, one is interested in repairing the system, e.g., by replacing individual components. Note that in typical problem settings there are usually several diagnosis candidates that can explain the observed faulty behavior of a process. The (automated) selection of the most probable or appropriate one for a given situation is an open and critical issue in many MBD-based approaches, because also the corresponding repair actions can be different for different diagnoses.
240
D. Jannach and A. Gut
Diagnosing BPEL Process through Dependency Analysis. In [YD07], Yan and Dague propose a method for monitoring and diagnosing web service processes based on the MBD paradigm. In order to apply MBD techniques, a formal model of the web service process has to be developed. Therefore, the existing process definition (given in the BPEL4WS language) is first transformed into a Discrete Event System, that is, into a synchronized automaton, where the states are defined by the values of the variables of the process. State transitions are caused by variable changes that are in turn triggered by actions such as variable assignments, message emissions or receptions. The model of non-abnormal behavior for every activity in usual MBD notation is: ¬ab(A) ∧ ¬ab(A.inputs) ⇒ ¬ab(A.outputs)
(1)
The non-abnormal (correct) model states that if activity A behaves correctly (i.e., is not abnormal ) and the inputs to A are also correct, then also the outputs of activity A, i.e., the resulting variables changes, are correct. Once an exception occurs during run-time execution, the diagnosis phase is initiated and the process starts comparing the observed behavior (given by the execution logs of the engine and the corresponding execution “trajectory”) with the expected behavior (as described by the automata and the given variable dependencies).14 At run-time, the diagnosis engine is called when an exception occurs. The subsequent diagnostic process is based on three pillars. – Dependency analysis: Similar to Program Slicing [Wei81], a software debugging technique, a (static) analysis is made of how variable values can be affected by activities. To that purpose, activities are viewed as functions that accept input variables (defined as global BPEL variables) and produce output variables. These relationships can be derived from BPEL’s correlation sets, which describe how arriving messages are related, and are used to model the non-abnormal behavior of an activity more precisely, i.e. ¬ab(A.input) ⇒ ¬ab(A.output), if U til(A, A.input, A.output)
(2)
The function Util(A,V1,V2) denotes that through activity A output variable V 2 is correlated with the input variable V 1. – Trajectory reconstruction: The execution trajectories are recovered from the BPEL logs by synchronizing the observations (events and exceptions in the log) with the system description, i.e., the formal execution model. Figure 7 shows a simple process model with parallel activity execution (left hand side) and two execution trajectories. One corresponds to a possible successful execution path; in the other, an exception is thrown before the process finishes. 14
Note that to that purpose the BPEL engine must provide additional logging support and e.g., record incoming and outgoing SOAP messages as well as events and exceptions.
Exception Handling in Web Service Processes
241
Fig. 7. Model and two execution trajectories
– Accountability analysis: In this step, the possible “culprits” (i.e., activities that may have caused the problem) of the exception are determined. The set of candidates is limited to those activities that have been already executed and to those who possibly have affected the input variables of the failed activity (which can also be the only responsible activity). This calculation is based on applying responsibility propagation rules through the given trajectory. If two or more exceptions happen simultaneously, the analysis is made in parallel and the resulting diagnoses are then combined. In case of sequential (chained) exceptions, only the first exception is analyzed. Discussion. In principle, the approach is very similar to the well-known Program Slicing technique (with some slight differences). Thus, it applies a proven technique for ordinary programs to the “programming in the large” world. A deeper discussion of the relationship between Model-based Diagnosis for Software Debugging and Program Slicing can be found in [Wot02], who applies an even more elaborate diagnosis technique for Java programs. One limitation of the approach by [YD07] is that the underlying BPEL engine must support the extended logging mechanisms, which are required to spot the problematic activity. In addition, the approach is also limited to certain types of failures. Time-out exceptions, for instance, which are probably caused by problems in a remote system, cannot be analyzed by the proposed technique. Diagnosing Workflow Processes from “First Principles”. In order to overcome the problems of the approach from [YD07], Friedrich and Ivanchenko in [FI08a] propose a more fundamental approach to model-based debugging of workflow executions. Their problem formalization of the workflow semantics is not based on a specific language such as BPEL, but rather on implementation independent Colored Petri Nets (CPN) and they support common workflow control-flow patterns such as sequences, splits, choices, merges and joins. Loops (cycles) in the models are, however, not yet supported by their approach. In order to properly model variable dependencies and process execution states, the authors extend CPNs with so-called (data) objects and call the models Workflow/Data-Petri-Nets (WFD). In addition to these data objects, the authors also make use of guard transitions (activities) that are similar to assertions in programming languages, i.e., at some given points during execution, certain Boolean expressions are checked. In the proposed formalism, a workflow exception therefore corresponds to a situation in which an assertion/guard expression evaluates to false. Together, all these mechanisms (data dependencies, guard transitions and the execution history of individual process instances) serve as a
242
D. Jannach and A. Gut
basis for identifying the potential sources of an exception more precisely in the WFD approach [FI08a]. In order to diagnose workflow executions from “first-principles”, the extended CPN model (WF) and the execution behavior of workflows are expressed in a logical representation, i.e., by a set of First-Order Logic sentences. These descriptions serve as a basis for a precise formal and consistency-based characterization of diagnoses of workflow executions which can be calculated with the help of Reiter’s Hitting Set algorithm [Rei87] and a theorem prover for consistency checking. In a later work, Friedrich et al. [FMS10] reformulate the workflow diagnosis problem using on a different logic-based formalism and encoding. Their approach includes different improvements over the basic model-based debugging method and supports diagnosis-repair settings in which activities are potentially re-executed by the execution engine in order to successfully complete a process despite the existence of a fault. In contrast to dependency-based approaches, their method for identifying incorrect activities in process execution has the advantage that it does not require precise behavioral activities descriptions (which are often not available in loosely coupled systems based on web services). Discussion. When compared with the work in [YD07], which represents more a Program Slicing than a Diagnosis technique, both share the idea of utilizing a formal model of workflow executions and the analysis of data dependencies. The approach of Friedrich and Ivanchenko aims to tackle the diagnosis problem in a more fundamental manner and to characterize diagnoses for workflow execution in general (and not dependent on dependency propagation). While there are some simplifying assumptions of their approach (e.g., loops are allowed in the model but can be dealt with using loop unfolding techniques), the authors show in [FMS10] that a variety of real-world problems can be modeled and solved with their formalism. In addition to the mentioned approach of [YD07], the theory developed in [FI08a] also lays the foundation for the automated computation of possible repair actions for problematic workflow instances, see [FI08b] and [FFM+ 10]. Distributed Diagnosis of Web Services. Yet another approach toward modelbased workflow problem diagnosis that goes beyond today’s standard throw-andcatch scheme, is proposed by Ardissono et al. in [ACG+ 05] and [AFG+ 06]. Their work is driven by the fact that web service processes often span several companies and that the individual elements of the global overall process can be seen as whole sub-processes that are executed at a partner system. Thus, a central point of problem analysis cannot easily be installed, because, e.g., the participating partners may not be willing to share all the internal information required for diagnosis to the coordinating partner. Therefore, the authors propose a model for distributed error handling as shown in Figure 8. It is assumed that every participant of the distributed web service process has a local diagnoser component, which uses the log information of the local web service to diagnose the reasons for the problem. Note however that a failure found in a service is possibly caused by an error in another service. Therefore, a global
Exception Handling in Web Service Processes
243
Fig. 8. Local and global diagnosers (adapted from [AFG+ 06])
diagnoser is introduced, which can combine the information of the individual local diagnosers into a global hypothesis about the reasons of the problem; the communication itself is based again on web service technology and a defined message exchange protocol. Overall, the main advantage of the approach is that the central diagnoser does not have to be aware of the internals of the other participating web services. Even more, the global component does not necessarily need to know in advance how the different services interact with each other. Discussion. Similar to the other works, in the approach of Ardissono et al., activities are viewed as components of a system that transform inputs to outputs and which can have the behavioral modes ok and ab(normal). In particular, every activity has three subcomponents that can fail as the authors distinguish between three types of variables: Variables, whose values should be simply forwarded, variables that are newly created through the activity and variables, whose values are changed by the component. In that respect, the model is a bit more detailed than the more simple input-output model of [YD07]. Details of how activity execution chains in the sub-processes are handled (e.g., based on trajectories as in [YD07]) are not given. On the level of the infrastructure, the approach from Ardissono et al. is based on the assumption that every local service, which is for instance implemented as orchestrated BPEL process, communicates with the local diagnoser. For that purpose, the local web service engine has to be extended and – in case of BPEL processes – the fault handlers of the model have to be modified, so that the diagnoser is appropriately informed whenever a local exception occurs. Overall, techniques from model-based diagnosis appear to be a promising way to deal with exceptions in web service processes, in particular as the basic mechanisms in many cases are to a large extent independent of the protocols and engines that are used in practical environments. What remains open in all discussed approaches is the question of computational costs for real-world web service processes. While it is known that the computation of diagnosis is in general a computationally complex problem, it would be interesting to know
244
D. Jannach and A. Gut
whether the proposed methods are applicable to process definitions of limited complexity that can be found in the real world. Repair Planning / Contingency Planning. Finding out which part of a web service process failed or causes a problem is helpful, when the goal is to exclude a certain service from being used in future process instances or when alternative services that provide the same functionality can dynamically be incorporated into the processes. In many cases, however, this repair scheme, which can be summarized as “find an alternative service and re-try” is not appropriate. It would therefore be desirable to have an agent who can determine dynamically whether a broken workflow instance can still be successfully completed, e.g., by executing a set of alternative activities. In other words, the agent needs to find a repair plan to complete the service or otherwise abort it. In [FI08b], Friedrich and Ivanchenko propose such a method for computing repair plans based on the workflow formalism by [RD98], in which a workflow is represented by a directed acyclic graph of control nodes and activity nodes together with a set of associated objects. A process that terminates normally results in a certain configuration of the output objects. When an error occurs, the problem is to find a repair plan (which is itself a workflow) that leads to a configuration at the endpoint that is equal to the configuration that would appear after normal process execution. The repair approach in [FI08b] is based on the usage of common repair actions such as (re-)execution (suitable, e.g., to deal with transient failures), activity substitution and compensation. In order to deal in particular with transient failures, the concept of time has to be incorporated into the repair model. The starting point for repair planning is therefore a data record that describes which activity failed at which point in time15 . Repair planning itself is based on a logical formalization of the workflow process model, the effects of the different repair actions, the time model and the workflow execution state. The authors of [FI08b] classify their repair planning procedure as “conditional planning with background theories”. The logical formalism was chosen in a way that it can be handled by the dlv disjunctive logic programming system [LPF+ 06]. How this system can be used for knowledge-based planning has first been shown by Eiter et al. in [EFL+ 04]. Later on an extension was developed to support (optimistic) conditional planning was later on developed by [vNEV07]. [FI08b] finally expand these ideas in a way guaranteeing that contingency plans always lead to successful workflow completions. Discussion. An evaluation of various automatically generated test cases of small to medium sized workflow processes indicates that the method is generally applicable to real-world repair problems. The examined processes consisted of up to a few dozen activities. Computation times on a standard desktop PC remained less than a half-minute for most of the test cases, which is acceptable, 15
Note that this information could be derived from a diagnosis step that precedes the repair phase.
Exception Handling in Web Service Processes
245
taking into account that web service processes are typically longer lasting and have no tight time constraints attached. However, what has not been discussed are limitations of the approach, such as the problem of handling loops in the process models. It can be assumed that repair plans for such processes cannot be computed with that method yet (compare the work of the same authors in [FI08a]). Another aspect, which may constrain the applicability of the method in real world concerns the question whether one can assume that an in-depth description of the precise effects of individual activities (web services) is available in practical settings. 4.2
Semantic Web Technology
Replacing a transiently or permanently failing service of a process chain with an alternative service that provides the same functionality was already mentioned as a possible repair action in previous sections. Recent technology requires that the set of alternatives is known in advance and is described explicitly in the process model, e.g., in the exception handler of an individual action. Still, it would be desirable that alternative services could automatically be discovered and used by an intelligent system in case of a problem. Consider, for instance, the problem of booking a travel arrangement automatically which involves various web services and activities such as flight or hotel reservations (compare [MSZ01]). If a particular online hotel booking service fails, the process should not be halted but the booking functionality of another e-tourism platform should be used instead. If we do not want to model the alternative services or the service discovery process explicitly, a more generic mechanism is required and the particular problem is to find a service (or a set of services) that provide the same – or at least a sufficiently similar – functionality as the failing service that caused the problem. Today’s standard technologies for describing web services such as WSDL allow us to model a service and its interface on the syntactical level, i.e., they can be used to list the set of known operations and their parameters. However, these technologies cannot be used to describe the semantics of the service, i.e., the effects of invoking the service. Therefore, in order to allow for automated discovery or – which is even more complex – the automatic composition of service chains to complete an otherwise failing process, mechanisms for precisely describing web service semantics are required. In a broader context, the idea of annotating web resources in a way that they can be located intelligently or combined by a computerized system, is the core vision of the Semantic Web [BLHL01]. In recent years, significant research efforts went into the realization of this vision and lead to the development of a the Web Ontology Language OWL adopted by W3C16 . Shared ontologies are the basis of the Semantic Web and define the set of concepts that can be used to annotate resources in the web, i.e., they help us to insure that the same terms and vocabulary is used and interoperability is 16
http://www.w3.org/2004/OWL/
246
D. Jannach and A. Gut
guaranteed. OWL itself is a formal, XML-based and machine-processible language that can be used to describe specific domain ontologies. An (OWL-based) ontology for the above mentioned travel-planning problem would for instance define the concepts “flight” or “hotel” and probably also describe that a “domestic flight” is a special subtype of a flight. In the context of web service discovery, we could therefore try to extend the existing syntactic WSDL descriptions with semantic annotations, e.g., use the agreed-upon term “Amsterdam hotel booking” to denote that our service provides that functionality. Whenever a service fails in a process chain with this annotation, an alternative service with the same or a comparable annotation – according to a common ontology – could be invoked, if the parameter lists are somehow compatible. If a simple replacement of a service is not sufficient and a different chain of service invocations is required to achieve the original goal, a different way of describing the web service semantics is required. Note that the computation of diagnoses and associated repair plans was also the goal of the approaches of [FI08b] or [YD07] described above. In these approaches, the required execution semantics were described in terms of the variable dependencies or the effects of activities on data objects. However, this representation mechanism is rather technically oriented and not well-suited for sharing execution semantics because modeling is done on the level of variables and their dependencies and not the level of a more abstract domain ontology. A more general and implementation-independent way of capturing web service semantics is to view a service as an action in a STRIPS-style [FN95] planning problem from the field of Artificial Intelligence (AI) and correspondingly model a service through its inputs, outputs, preconditions, and effects (IOPE), see also [MSZ01]. How such a technology can potentially help us to react on workflow exceptions more intelligently, will be described later on. Web Service Discovery. In this subsection we will first sketch the opportunities and open challenges of automated web service discovery, which – as mentioned above – can also be seen as a mechanism to finding alternatives for a transiently or permanently failing web service. UDDI (Universal Description, Discovery and Integration) is a standard that can be used for publishing and locating standard WSDL-based web services in a company-wide or global registry and was originally designed with the concept of service-brokering in mind. Consequently, “matchmaking” between service providers and consumers was a central functionality that should be provided by such an exchange platform. However, the mechanisms provided by UDDI in that context are rather limited and mainly consist of API-functions for locating a service by name, through keyword-matching or category-based search. Beside names and categorizations, web service descriptions can only be given in plain text or keywords. Automatic matchmaking is therefore not easily possible as the information can be incomplete, misleading, contain ambiguous words and so forth. Even the use of classification schemes such as UNSPSC17 may not 17
http://www.unspsc.org/
Exception Handling in Web Service Processes
247
be sufficient because different schemes may be used and these schemes are not compatible. In [TAH07], Tsetos et al. summarize the key innovations and additional components that are required for semantic web service discovery and also propose different possible architectures. The concept of Service Annotation Ontologies (SAO), which can be used to precisely specify the service capabilities, is at the core of their considerations. One proposal for a standardized language for describing web service semantics in this form is OWL-S which was submitted to W3C for standardization in 2004. Note that competing proposals such as WSDL-S or WSMO18 were submitted to W3C. In this chapter, we focus on OWL-S because for different reasons. First, OWL-S is designed in a way that the semantic annotations can contain concepts from domain-specific OWL-based ontologies. At the same time, OWL-S also allows us to “ground” an action and technically bind it to a standard WSDL- and SOAP-based web service. According to the proposal of [TAH07], an extended web service discovery architecture can make use of the existing UDDI infrastructure, but the registry entries should have pointers to the unambiguous semantic descriptions expressed in the annotation language, i.e., the “Service Advertisements” should contain semantic descriptions. If the “Service Requests” are also stated in the same way, i.e., when the desired service capabilities are described in a semantically precise way, advanced matchmaking algorithms can be employed to decide whether a given service matches the requirements. Since exact matches cannot always be expected, usually a measure for the “Degree of Match” (DoM) is introduced. Such a measure can be based on various aspects such as the overlap of concepts used in the advertisement and the request or the placement of the services in a classification hierarchy. Note that the DoM measure can not only be used for service discovery (in the sense of matching) but also for a subsequent ranking step, which is also common in such matchmaking architectures [KSF07]. Finally – besides sophisticated matchmaking algorithms – also modern ontology integration and mediation techniques can be utilized to align ontologies of different user communities; for an overview see, e.g., [ES07]. Technically, different architectures are possible. The semantic web service discovery system can either be centralized (e.g., as an extended UDDI service) or as a peer-to-peer system, in which every participant can act as a provider or requester of a service. For a centralized version, again different architectural choices are possible. In one proposal of [TAH07], the matching algorithms are themselves provided as services. Depending on the type of the request – keywordbased or semantics-based – the corresponding matching service and algorithm is selected dynamically. Web Service Composition and Re-planning. Once semantic descriptions of individual services are available, it may not only be desirable to locate a single 18
http://www.w3.org/Submission/OWL-S/, http://www.w3.org/Submission/WSDL-S/, http://www.w3.org/Submission/WSMO/
248
D. Jannach and A. Gut
service but probably also to find a chain of services to reach a certain process goal or – corresponding to the focus of this chapter – find a repair plan to recover from a workflow exception. This process of deriving execution chains automatically is called web service composition and is often based on AI planning technology, see, e.g., [SPW+ 04] or [HBP07]. Note that although the idea of deriving web service process chains automatically was the driving scenario in the beginning of Semantic Web Services research (see, e.g., the paper of McIlraith et al. in 2001 [MSZ01]), still no reports on practical business implementations can be found until today. One system, in which Semantic Web Services and AI planning were combined successfully in the domain of server-side multimedia adaptation is reported in [JLTH06]. In this work, the goal was to design an open and extensible architecture for a multimedia server which has two basic features: First, it should be capable of integrating multimedia adaptation services and tools of different vendors and provides. Second, the server should be able to combine the different services into an invocation chain that converts a given resource (such as a video) from the given format to the desired format if a single-step adaptation is not possible. In order to achieve these goals, the available adaptation operations (ranging from simple image resizing to complex content- and environment based video transformations) are modeled as Semantic Web Services (SWS) using OWL-S. An action description comprises the required in- and outputs as well as the preconditions for applying a transformation, i.e., the acceptable input formats, and the effects of the transformation that describe how the resource is changed after applying the operator. Thus, given the set of actions, a “start state”, which corresponds to a description of the resource to be transformed, and a “goal state”, that corresponds to a description of how the end user wants to consume the resource, any standard state-space planning engine can be used to calculate a corresponding adaptation chain. Note that in any domain in which Semantic Web Service technology should be exploited, the participants have to agree on a shared domain ontology determining, for example, how the different actions are described. The establishment of such an agreed-upon ontology often is a hampering factor when it comes to applications that should be open to a broad community of participants. In the domain of multimedia adaptation, this problem could be solved by interpreting the existing ISO/MPEG standards (MPEG-7, MPEG-21) as a domain ontology, as these standards exactly prescribe how the different features of a resource (such as color, size etc.) have to be described [JL07]. While the works described in [JLTH06] and [JL07] were not primarily designed for exception handling in web service processes, this example helps us to illustrate how SWS technology can serve as an enabler for more advanced errorrecovery techniques. By design, the system can handle additions and removals of individual services. As long as there exists a possible execution chain that uses the currently available services, the system will find it due to the soundness and completeness of the underlying planning algorithms. If a service transiently or
Exception Handling in Web Service Processes
249
permanently is not available, alternative services can be retrieved automatically based on the semantic annotations of their effects. Moreover, since the composition (and repair) problem is reduced to a wellunderstood AI planning problem, the known techniques for contingency planning [PC96] or re-planning can be applied, see e.g., [Pee05] for a replanning algorithm for web services. The most simple method would be to halt the execution chain when a problem is detected and generate a new plan that leads from the intermediate state to the original goal state. More elaborate replanning and rescheduling techniques that for instance aim to produce a new plan which has minimal deviations from the original plan are also possible, see [KJF07] for an example from the area of process scheduling. Discussion. Semantic Web technology can be seen as a promising base technology for future, more intelligent error-handling techniques in web service processes which are based on automated service discovery or dynamic service composition. However, several challenges remain in practical environments. First, although OWL today is the the de-facto standard for ontology representation, there exists no agreed-upon standard for describing web service semantics. The different proposals (OWL-S, WSMO, WSDL-S) have been submitted for standardization already in 2004 and 2005, but no “winner” has been determined yet. Beside this problem of standardizing the language, most applications domains suffer from the problem that no shared domain ontology exists or that several competing pseudo-standards exist. Consider again the example from the travel planning domain. While there exist several classification schemes such as UNSPSC, the sector is far from having a common ontology that unambiguously describes what the effects of a “flight reservation” or a “hotel booking” are. Thus, semantics-based error recovery is only possible if (a) the involved companies tightly co-operate and agree on a detailed shared ontology or (b) domainspecific standards such as the MPEG-7 specification in the context of the work in [JL07] already exist. Next, as mentioned in [Sri03], there are some fundamental differences between BPEL-based process models and the representation mechanism used in AI planning. BPEL processes describe a global picture of the flow of activities, which may contain complex control structures and so forth. With AI-planning techniques, one needs to model individual activities and think of appropriate preconditions and effects that are required to express rather simple precedence constraints. Another open issue is the question of how to model side-constraints that have to apply to the overall process, which cannot be easily captured by the IOPE-style action descriptions. There might be for example several online airline booking services available, which are basically exchangeable with respect to their functionality. Due to internal business rules, it might however only be allowed to use a certain subset of those booking services in certain situations. Finally, also questions of scalability of the planning and re-planning approaches to web service composition and the computation of repair plans have not been answered so far. Some more practical problems of web service composition (which
250
D. Jannach and A. Gut
are not even semantically-enriched ones) are also discussed in the “reality-check” paper of Lu et al. in [LYRS07].
5
Summary
In contrast to classical workflow processes, today’s web service processes require additional attention with respect to exception handling and recovery because of their inherent distributed and open nature. Problematic cases for instance include situations where a remote service is simply unreachable or where detailed information about a remote exception is simply unavailable. In this chapter, we have discussed various mechanisms to handle exceptions in web service processes. Today’s de-facto process modeling “standard” technology (BPEL and BPMN) is influenced by research in the area of earlier workflow systems and relies on the “catch-and-handle” exception handling pattern known from programming languages as well as on the concepts of transactions and compensating actions. In order to support modular and compact process models that do not mix the regular execution flow with error-handling procedures, some languages such as YAWL provide special model structuring mechanisms and the opportunity to dynamically change the execution flow at run-time. In the final sections, an outline of more intelligent error-handling mechanism of current web service process technology was given. Most recent approaches aim to avoid explicit error-handling models and exploit additional knowledge to calculate problem recovery actions automatically. These recent proposals include for example more advanced debugging aids based on Model-based Diagnosis as well as techniques for the computation of corresponding repair plans. In addition, the opportunities for automated web service discovery and re-planning as a special form of service composition that come along with the emergence of Semantic Web technology have been discussed in this chapter.
References [ACG+ 05]
[AFG+ 06]
[AtHEvdA06]
[AtHvdAE07]
[BLHL01]
Ardissono, L., Console, L., Goy, A., Petrone, G., Picardi, C., Segnan, M., Dupre, D.T.: Enhancing web services with diagnostic capabilities. In: Proc. 3rd Europ. Conf. on Web Services, V¨ axj¨ o, Sweden, p. 182 (2005) Ardissono, L., Furnari, R., Goy, A., Petrone, G., Segnan, M.: Fault tolerant web service orchestration by means of diagnosis. In: Gruhn, V., Oquendo, F. (eds.) EWSA 2006. LNCS, vol. 4344, pp. 2–16. Springer, Heidelberg (2006) Adams, M., ter Hofstede, A.H.M., Edmond, D., van der Aalst, W.M.P.: Worklets: A service-oriented implementation of dynamic flexibility in workflows. In: Proc. CooPIS 2006, pp. 291–308 (2006) Adams, M., ter Hofstede, A.H.M., van der Aalst, W.M.P., Edmond, D.: Dynamic, extensible and context-aware exception handling for workflow. In: Proc. CoopIS 2007, Algarve, Portugal (2007) Berners-Lee, T., Hendler, J., Lassila, O.: The semantic web. Scientific American, 34–43 (2001)
Exception Handling in Web Service Processes [CCPP99]
[CDP08]
[CJ90]
[DKLW07] [EFL+ 04]
[EL96]
[EL97] [ES07] [FFJS04] [FFM+ 10]
[FI08a]
[FI08b]
[FMS10]
[FN95]
[FSW99] [Gla08] [GM06]
[HA00]
251
Casati, F., Ceri, S., Paraboschi, S., Pozzi, G.: Specification and implementation of exceptions in workflow management systems. ACM Transactions on Database Systems 24(3), 405–451 (1999) Combi, C., Daniel, F., Pozzi, G.: XPDL Enabled Cross-Product Exception Handling for WfMSs. In: Fischer, L. (ed.) 2008 BPM and Workflow Handbook, Future Strategies, Inc, Florida, pp. 177–186 (2008) Compton, P., Jansen, R.: Knowledge in context: a strategy for expert system maintenance. In: Proc. 2nd Australian Joint Conference on Artificial Intelligence, pp. 292–306 (1990) Decker, G., Kopp, O., Leymann, F., Weske, M.: Bpel4chor: Extending bpel for modeling choreographies. In: ICWS 2007, Salt Lake City, Utah, pp. 296–303 (2007) Eiter, T., Faber, W., Leone, N., Pfeifer, G., Polleres, A.: A logic programming approach to knowledge-state planning: Semantics and complexity. ACM Transactions on Computational Logic 5(2), 206–263 (2004) Eder, J., Liebhart, W.: Workflow recovery. In: Proc. 1st IFCIS International Conference on Cooperative Information Systems, p. 124 (1996) Eder, J., Liebhart, W.: Workflow transactions. Workflow Handbook, pp. 157–163. John Wiley, Chichester (1997) Euzenat, J., Shvaiko, P.: Ontology matching. Springer, Heidelberg (2007) Felfernig, A., Friedrich, G., Jannach, D., Stumptner, M.: Consistencybased diagnosis of configuration knowledge bases. Artificial Intelligence 152(2), 213–234 (2004) Friedrich, G., Fugini, M., Mussi, E., Pernici, B., Tagni, G.: Exception handling for repair in service-based processes. IEEE Trans. Software Eng. 36(2), 198–215 (2010) Friedrich, G., Ivanchenko, V.: Diagnosis from first principles for workflow executions, Technical Report 2008/002. Institute of Applied Informatics, University Klagenfurt, Austria (2008) Friedrich, G., Ivanchenko, V.: Model-based repair of web service processes, Technical Report 2008/001. Institute of Applied Informatics, University Klagenfurt, Austria (2008) Friedrich, G., Mayer, W., Stumptner, M.: Diagnosing process trajectories under partially known behavior. In: Proc. ECAI 2010, Lisbon, Portugal, pp. 111–116 (2010) Fikes, R.E., Nilsson, N.J.: Strips: a new approach to the application of theorem proving to problem solving. In: Computation & Intelligence: Collected Readings, pp. 429–446. AAAI Press, Menlo Park (1995) Friedrich, G., Stumptner, M., Wotawa, F.: Model-based diagnosis of hardware designs. Artificial Intelligence 111(1-2), 3–39 (1999) Glassey, O.: A case study on process modelling - three questions and three techniques. Decision Support Systems 44(4), 842–853 (2008) Guelfi, N., Mammar, A.: A formal framework to generate XPDL specifications from UML activity diagrams. In: SAC 2006, pp. 1224–1231 (2006) Hagen, C., Alonso, G.: Exception handling in workflow management systems. IEEE Trans. Softw. Eng. 26(10), 943–958 (2000)
252
D. Jannach and A. Gut
[HBP07]
[HH06]
[Hof05] [JL07]
[JLTH06]
[KJF07]
[KSF07]
[LPF+ 06]
[LYRS07]
[MMP06]
[MSZ01] [PC96] [Pee05]
[RD98]
[Rei87] [RvdAtH06] [SPW+ 04]
Hoffmann, J., Bertoli, P., Pistore, M.: Web service composition as planning, revisited: In between background theories and initial state uncertainty. In: Proc. AAAI 2007, Vancouver, BC, pp. 1013–1018 (2007) Halvorsen, O., Haugen, O.: Proposed notation for exception handling in UML 2 sequence diagrams. In: Proc. 18th Australian Software Engineering Conference, Sydney, Australia, pp. 29–40 (2006) Ter Hofstede, A.H.M.: Yawl: yet another workflow language. Information Systems 30, 245–275 (2005) Jannach, D., Leopold, K.: Knowledge-based multimedia adaptation for ubiquitous multimedia consumption. Journal of Network and Computer Applications 30(3), 958–982 (2007) Jannach, D., Leopold, K., Timmerer, C., Hellwagner, H.: A knowledge-based framework for multimedia adaptation. Applied Intelligence 24(2), 109–125 (2006) Kuster, J., Jannach, D., Friedrich, G.: Handling alternative activities in resource-constrained project scheduling problems. In: Proceedings IJCAI 2007, Hyderabad, India, pp. 1960–1965 (2007) Kopeck´ y, J., Paslaru Bontas Simperl, E., Fensel, D.: Semantic web service offer discovery. In: Proc. Workshop on Service Matchmaking and Resource Retrieval in the Semantic Web (SMRR 2007) co-located with ISWC 2007 + ASWC 2007, Busan, South Korea (2007) Leone, N., Pfeifer, G., Faber, W., Eiter, T., Gottlob, G., Perri, S., Scarcello, F.: The dlv system for knowledge representation and reasoning. ACM Transactions on Computational Logic 7(3), 499–562 (2006) Lu, J., Yu, Y., Roy, D., Saha, D.: Web service composition: A reality check. In: Benatallah, B., Casati, F., Georgakopoulos, D., Bartolini, C., Sadiq, W., Godart, C. (eds.) WISE 2007. LNCS, vol. 4831, pp. 523–532. Springer, Heidelberg (2007) Modafferi, S., Mussi, E., Pernici, B.: SH-BPEL: a self-healing plug-in for WS-BPEL engines. In: Proc. Workshop on Middleware for Service Oriented Computing, pp. 48–53 (2006) McIlraith, S.A., Son, T.C., Zeng, H.: Semantic web services. IEEE Intelligent Systems 16(2), 46–53 (2001) Pryor, L., Collins, G.: Planning for contingencies: A decision-based approach. Journal of Artificial Intelligence Research 4, 287–339 (1996) Peer, J.: A pop-based replanning agent for automatic web service composition. In: G´ omez-P´erez, A., Euzenat, J. (eds.) ESWC 2005. LNCS, vol. 3532, pp. 47–61. Springer, Heidelberg (2005) Reichert, M., Dadam, P.: Adeptf lex - supporting dynamic changes of workflows without losing control. Journal of Intelligent Information Systems 10(2), 93–129 (1998) Reiter, R.: A theory of diagnosis from first principles. Artificial Intelligence 32(1), 57–95 (1987) Russell, N., van der Aalst, W., ter Hofstede, A.: Workflow exception patterns. In: Martinez, F.H., Pohl, K. (eds.) CAiSE 2006. LNCS, vol. 4001, pp. 288–302. Springer, Heidelberg (2006) Sirin, E., Parsia, B., Wu, D., Hendler, J., Nau, D.: HTN planning for web service composition using SHOP2. In: Proc. ISCW 2004, Sanibel Island, Florida, pp. 377–396 (October 2004)
Exception Handling in Web Service Processes [SR93] [Sri03]
[STA05]
[TAH07]
[vNEV07]
[Wei81] [Wot02] [YD07]
253
Sheth, A., Rusinkiewicz, M.: On transactional workflows. IEEE Data Engineering Bulletin 16, 3–34 (1993) Srivastava, B.: Web service composition - current solutions and open problems. In: Prof. ICAPS 2003 Workshop on Planning for Web Services, Trento, Italy, pp. 28–35 (2003) Scheer, A.W., Thomas, O., Adam, O.: Process modeling using eventdriven process chains. In: Dumas, M., van der Aalst, W.M.P., ter Hofstede, A.H.M. (eds.) Process-Aware Information Systems, pp. 119– 145. Wiley, Chichester (2005) Tsetsos, V., Anagnostopoulos, C., Hadjiefthymiades, S.: Semantic web service discovery: Methods, algorithms and tools. In: Cardoso, J. (ed.) Semantic Web Services: Theory, Tools and Applications, pp. 240–280. IDEA, USA (2007) van Nieuwenborgh, D., Eiter, T., Vermeir, D.: Conditional planning with external functions. In: Baral, C., Brewka, G., Schlipf, J. (eds.) LPNMR 2007. LNCS (LNAI), vol. 4483, pp. 214–227. Springer, Heidelberg (2007) Weiser, M.: Program slicing. In: Proc. 5th International Conference on Software Engineering, Piscataway, NJ, USA, pp. 439–449 (1981) Wotawa, F.: On the relationship between model-based debugging and program slicing. Artificial Intelligence 135(1-2), 125–143 (2002) Yan, Y., Dague, P.: Modeling and diagnosing orchestrated web service processes. In: Proc. ICWS 2007, Salt Lake City, Utah, pp. 51–59 (2007)
Leveraging Model-Based Tool Integration by Conceptual Modeling Techniques Gerti Kappel1 , Manuel Wimmer1 , Werner Retschitzegger2 , and Wieland Schwinger3 1
2
Business Informatics Group Vienna University of Technology, Austria {kappel,wimmer}@big.tuwien.ac.at Department of Business Information Systems University of Vienna, Austria
[email protected] 3 Department of Telecooperation Johannes Kepler University Linz, Austria
[email protected] Abstract. In the context of model-based tool integration, model transformation languages are the first choice for realizing model exchange between heterogenous tools. However, the lack of a conceptual view on the integration problem and appropriate reuse mechanisms for already existing integration knowledge forces the developer to define model transformation code again and again for certain recurring integration problems in an implementation-oriented manner resulting in low productivity and maintainability of integration solutions. In this chapter, we summarize our work on a framework for modelbased tool integration which is based on well-established conceptual modeling techniques. It allows to design integration models on a conceptual level in terms of UML component diagrams. Not only the design-time is supported by conceptual models, but also the runtime, i.e., the execution of integration models, is represented by conceptual models in terms of Coloured Petri Nets. Furthermore, we show how reusable integration components for resolving structural metamodel heterogeneities, which are one of the most frequently recurring integration problems, can be implemented within our framework.
1
Introduction
With the rise of Model-Driven Engineering (MDE) [25] models become the main artifacts of the software development process. Hence, a multitude of modeling tools is available supporting different tasks, such as model creation, model simulation, model checking, model transformation, and code generation. Seamless exchange of models among different modeling tools increasingly becomes a crucial prerequisite for effective MDE. Due to lack of interoperability, however, it is often difficult to use tools in combination, thus the potential of MDE cannot be fully utilized. For achieving interoperability in terms of transparent model Kaschek/Delcambre (Eds.): The Evolution of Conceptual Modeling, LNCS 6520, pp. 254–284, 2011. c Springer-Verlag Berlin Heidelberg 2011
Leveraging Model-Based Tool Integration
255
exchange, current best practices (cf., e.g. [28]) comprise creating model transformations based on mappings between concepts of different tool metamodels, i.e., the metamodels describing the modeling languages supported by the tools. We have followed the aforementioned approach in various projects such as the ModelCVS1 project [13] focusing on the interoperability between legacy case tools (in particular CA’s AllFusion Gen) with UML tools and the MDWEnet2 project [29] trying to achieve interoperability between different tools and languages for web application modeling. The prevalent form of heterogeneity one has to cope with when creating such mappings between different metamodels is structural heterogeneity, a form of heterogeneity well-known in the area of federated and multi database systems [2,14]. In the realm of metamodeling structural heterogeneity means that semantically similar modeling concepts are defined with different metamodeling concepts leading to differently structured metamodels. Current model transformation languages, e.g., the Object Management Group (OMG) standard Query/View/Transformation (QVT) [21], provide no appropriate abstraction mechanisms or libraries for resolving recurring kinds of structural heterogeneities [18]. Thus, resolving structural heterogeneities requires to manually specify partly tricky model transformations again and again which simply will not scale up having also negative influence on understanding the transformation’s execution and on debugging. In this work, we summarize our work on a framework for realizing model-based tool integration which is based on well-established conceptual modeling techniques. In particular, the framework allows to build so-called metamodel bridges on a conceptual level by using UML component diagrams [19]. Furthermore, such a metamodel bridge allows the automatic transformation of models since for each mapping operator, which is expressed as a component, the operational semantics is specified on basis of Colored Petri Nets [7]. Colored Petri Nets provide a uniform formalism not only for representing the transformation logic together with the metamodels and the models themselves, but also for executing the transformations, thus facilitating understanding and debugging. To demonstrate the applicability of our approach we apply the proposed framework for defining a set of mapping operators subsumed in our mapping language called CAR. This mapping language is intended to resolve typical structural heterogeneities occurring between the core concepts usually used to define metamodels, i.e., class, attribute, and reference, as provided by the OMG standard Meta Object Facility (MOF) [20]. Finally, we present an inheritance mechanism for reusing applied mappings within a single metamodel bridge. The framework has been applied in several modeling tool integration projects where structural modeling languages as well as behavioral modeling languages have been integrated. The rest of the paper is structured as follows. In Section 2 we introduce our framework for defining mapping operators for establishing metamodel bridges. In Section 3, the mapping language CAR is presented for which an inheritance mechanism for reusing mappings within a single metamodel bridge is introduced 1 2
ModelCVS—Concurrent Versioning System for Models. MDWEnet—Model-Driven Web Engineering Network.
256
G. Kappel et al.
in Section 4. While an evaluation of the framework is presented in Section 5, Section 6 discusses related work. Finally, in Section 7 a conclusion and a discussion on future research issues is given.
2
Metamodel Bridging at a Glance
In this section, we describe the conceptual architecture of the proposed Metamodel Bridging Framework in a by-example manner. The proposed framework provides two views on the metamodel bridge, namely a mapping view and a transformation view as illustrated in Figure 1. On the mapping view level, the user defines mappings between elements of two metamodels (cf. M2 in Figure 1). Thereby a mapping expresses also a relationship between model elements, i.e., the instances of the metamodels [3]. In our approach, we define these mappings between metamodel elements with mapping operators standing for a processing entity encapsulating a certain kind of transformation logic. A mapping operator takes as input elements of the source model and produces as output semantically equivalent elements of the target model. Thus, it declaratively describes the semantic correspondences on a high-level of abstraction. A set of applied mapping operators defines the mapping from a left hand side (LHS) metamodel to a right hand side (RHS) metamodel further on subsumed as mapping model. For actually exchanging models between different tools, the mapping models have to be executed. Therefore, we propose, in addition to the mapping view, a transformation view which is capable of transforming models (cf. M1 in Figure 1) from the LHS to the RHS on basis of Colored Petri Nets [7]. 2.1
The Mapping View
For defining mapping operators and consequently also for building mapping models, we are using a subset of the UML 2 component diagram concepts. With this formalism, each mapping operator can be defined as a dedicated component, representing a modular part of the mapping model which encapsulates an arbitrary complex structure and behavior, providing well-defined interfaces to the environment. The resulting components are collected in a mapping operator library which can be seen as a domain-specific language for bridging metamodels. The user can apply the mapping operators expressed as components in a plug&play manner, i.e., only the connections to the provided and required interfaces have to be established manually. Our motivation for using UML 2 component diagrams for the mapping view is the following. First, many software engineers are likely to be familiar with the UML component diagram notation. Second, the provided and required interfaces which can be typed, enable the composition of mapping operators to resolve more complex structural heterogeneities. Third, the clear separation between black-box view (i.e., the component’s externals) and white-box view (i.e., the component’s internals) of components allows switching between a high-level mapping view
Leveraging Model-Based Tool Integration
LHS
M Mapp ingView
1
MappingModel
TargetMM
C2C
Class
M2
RHS
Bridging
SourceMM
name : String
EntityType A2A
ownedAttributes
name : String attributes
R2R
Attribute
Property name : String
name : String
C2C
A2A
Transform
2
Transform 4
SourcePlaces
EntityType 2
history
Traansforrmatio onVie ew
Class_name
2
TargetPlaces
C2C
Class
M1 1+M M2
Transform
TransformationLogic
2
A2A
EntityType_name
2
2
history
Class_ownedAttributes
R2R
EntityType attributes EntityType_attributes
2
2
history
Property
Attribute C2C
2
2
history
Property_name
Attribute_name
A2A
2
2
history
6 5 UMLModel
TokenModel o1
3
o3:Property name=“ssn”
name=“Student”
o4:Property name=“studentnr”
o2
7
TokenModel o1
ERModel
o2
“Student“
“Student“
8 o1:EntityType
“Professor“
o3
o esso “Professor“
o4 “studentnr“ studentnr
“ssn“
o3
o4 studentnr “studentnr“
Trransform
name=“Professor”
o2:Class
Transform m
M1
o1:Class
Export
Execution
Import
name=“Professor”
o3:Attribute name=“ssn”
“ssn“
Fig. 1. Metamodel Bridging Framework by-example
o2:EntityType name=“Student”
o4:Attribute name=“studentnr”
257
258
G. Kappel et al.
and a detailed transformation view, covering the operational semantics, i.e., the transformation logic, of an operator. Anatomy of a mapping operator. Each mapping operator (as for example shown in the mapping model of Figure 1) has input ports with required interfaces (left side of the component) as well as output ports with provided interfaces (right side of the component). Because each mapping operator has its own trace model, i.e., providing a log about which output elements have been produced from which input elements, an additional providedContext port with a corresponding interface is available on the bottom of each mapping operator. This port can be used by other operators to access the trace information for a specific element via requiredContext ports with corresponding interfaces on top of the operator. In the mapping view of Figure 1 (cf. step 1), an example is illustrated where a small part of the metamodel of the UML class diagram (cf. source metamodel) is mapped to a part of the metamodel of the Entity Relationship diagram (cf. target metamodel). In the mapping view, source metamodel elements have provided interfaces and target metamodel elements have required interfaces. This is due to the fact that in our scenario, models of the LHS are already available whereas models of the RHS must be created by the transformation, i.e., the elements of the LHS must be streamed to the RHS according to the mapping operators. Consequently, Class and Property of the source metamodel are mapped to EntityType and Attribute of the target metamodel with Class2Class (C2C ) operators, respectively. In addition, the C2C operator owns a providedContext port on the bottom of the component which shall be used by the requiredContext ports of the appropriate Attribute2Attribute (A2A) and Reference2Reference (R2R) operators to preserve validity of target models. In particular, with this mechanism it can be ensured that values of attributes are not transformed before their owning objects has been transformed and links as instances of references are not transformed before the corresponding source and target objects have been transformed. 2.2
The Transformation View
The transformation view is capable of executing the defined mapping models. For this, so called transformation nets [12,23,34] are used which are a special kind of Colored Petri Nets consisting of source places at the LHS and target places at the RHS. Transitions between the source and target places describe the transformation logic located in the bridging part of the transformation net as shown in Figure 1. Transformation nets provide a suitable formalism to represent the operational semantics of the mapping operators, i.e., the transformation logic defined in the white-box view of the component due to several reasons. First, they enable the execution of the transformation thereby generating the target model out of the source model, which favors also debugging of a mapping model. Second, it allows a homogeneous representation of all artefacts involved in a model transformation (i.e., models, metamodels, and transformation logic) by means of a simple
Leveraging Model-Based Tool Integration
259
formalism, thus being especially suited for gaining an understanding of the intricacies of a specific metamodel bridge. In the next paragraphs, we discuss rules for assembling metamodels, models, and mapping models into a single transformation net and how the transformation can actually be executed. Places represent Metamodels. First of all, places of a transformation net are used to represent the elements of the source and target metamodels (cf. step 2 in Figure 1). In this respect, we currently focus on the three major building blocks of metamodels (provided, e.g. by meta-metamodels such as MOF), namely class, attribute, and reference. In particular, classes are mapped onto one-colored places whereby the name of the class becomes the name of the place. The notation used to visually represent one-colored places is a circle or oval as traditionally used in Petri Nets. Attributes and references are represented by two-colored places, whereby the name of the containing class plus the name of the reference or of the attribute separated by an underscore becomes the name of the place (cf. e.g. Class name and Class ownedAttributes in Figure 1). To indicate that these places contain twocolored tokens, the border of two-colored places is double-lined. Tokens represent Models. The tokens of the transformation net are used to represent the source model which should be transformed according to the mapping model. Each element of the source model is expressed by a certain token, using its color as a means to represent the model element’s identity in terms of a String (cf. step 3 in Figure 1). In particular, for every object, a one-colored token is produced, whereby for every link as an instance of a reference, as well as for every value of an attribute, a two-colored token is produced. The fromColor for both tokens refers to the color of the token that corresponds with the containing object. The toColor is given by the color of the token that corresponds with the referenced target object or the primitive value, respectively. Notationally, a two-colored token consists of a ring (carrying the fromColor) surrounding an inner circle (depicting the toColor). Considering our example shown in Figure 1, the objects o1 to o4 of the UML model shown in the M1-layer are transformed into one-colored tokens. Each one-colored token represents an object identity, pointed out by the object name beneath the token. E.g., the tokens with the inner-color ”Student” and ”Professor” have the same outer-color as their containing objects and the token which represents the link between object o1 and o3 has the same outer-color as the token representing object o1 and the inner-color corresponds to the onecolored token representing object o3. Transitions represent Mapping Models. The mapping model is expressed by the transformation logic of the transformation net connecting the source and the target places (cf. Step 4 in Figure 1). In particular, the operational semantics of the mapping operators are described with transitions, whereby the behavior of a transition is described with the help of preconditions called query-tokens (LHS of a transition) and postconditions called generator-tokens (RHS of a transition). Query-tokens and generator-tokens can be seen as templates, simply visualized
260
G. Kappel et al.
as color patterns, describing a certain configuration of tokens. The pre-condition is fulfilled and the transitions fires, if the specified color pattern described by the query-tokens matches a configuration of available input tokens. In this case, the postcondition in terms of the generator-tokens produces the required output tokens representing in fact the necessary target model concepts. In the following, the most simple mapping operators used in our example are described, namely C2C, A2A, and R2R. C2C. The white-box view of the C2C operators as shown in the transformation view of Figure 1 ensures that each object instantiated from the class connected to the input port is streamed into the mapping operator, the transition matches a single token from the input port, and streams exactly the same token to the output port. This is expressed in the transition by using the most basic querytoken and generator-token combination, both having the same color pattern. In addition, every input and output token combination is saved in a history place representing the trace model which is connected to the providedContext port and can be used as trace information by other operators. A2A. The white-box view of the A2A operator is also illustrated in the bridging part of the transformation view in Figure 1. Two-colored tokens representing attribute values are streamed via the input port into the mapping operator. However, a two-colored token is only streamed to the output port if the owning object of the value has been already transformed by a C2C operator. This is ensured in that the transition uses the same color pattern for the one-colored query-token representing the owning object streamed from the requiredContext port and for the outer color of the two-valued query-token representing the containing object of the attribute value. Only, if a token configuration matches this pre-condition, the two-colored token is streamed via the generator-token to the output port. Again, the input tokens and the corresponding output tokens are stored in a history place which is connected to the providedContext port. R2R. The white-box view of the R2R operator shown in the transformation view of Figure 1 consists of three query-tokens, one two-colored query-token representing the link and two one-colored query-tokens for accessing trace information from C2C operators. The two-colored query-token must have the same inner and outer colors as provided by the C2C trace information, i.e., the source and target objects must be already transformed. When this precondition is satisfied by a token configuration, the two-colored token representing the link is streamed via the generator-token to the output port. Execution of the transformation logic. As soon as the metamodels are represented as places, which are furthermore marked with the respective colored tokens depicting the concepts of the source model (cf. step 5 in Figure 1), the transformation net can be started. Now, tokens are streamed from the source places over the transitions into the target places (cf. step 6 in Figure 1). Considering our running example, in a first step only the transitions of the C2C operators are able to fire due to the dependencies of the A2A and R2R operators. Hence, tokens from the places Class and Property are streamed to
Leveraging Model-Based Tool Integration
261
the appropriate places of the RHS and all combinations of the queried input and generated output tokens are stored in the trace model of the C2C operator. As soon as all necessary tokens are available in the trace model, dependent operators, i.e., the A2A and R2R operators, are also able to fire. Generation of the target model. After finishing the transformation, the tokens from the target places can be exported (cf. step 7 in Figure 1) and transformed back into instances of the RHS metamodel (cf. step 8 in Figure 1). In our example, the one-colored tokens o1 to o4 contained in the target places are transformed back into objects of type EntityType and Attribute. The twocolored tokens which represent attribute values, e.g., ”Professor” and ”Student”, are assigned to their containing objects, e.g., o1 and o2 whereas ”ssn” and ”studentnr” are assigned to o3 and o4. Finally, the two-colored tokens which represent links between objects are transformed back into links between o1 and o3, as well as between o2 and o4.
3
Mapping Operators of the CAR Mapping Language
3.1
Motivating Example
Based on experiences gained in various interoperability projects [10,11,35,36] it has been shown that although most meta-metamodels such as MOF offer only a core set of language concepts for defining metamodels, numerous structural heterogeneities occur when defining modeling languages. LHS-MM
c
Package
Package
name : String
name : String {id} d superClasses Cl *
Class
* Class name : String *
name : Stringg package : String {idref}
*
subClasses *
Generalization
* superClasses Cl
Attribute
*
name : String
Attribute name : String isID : Boolean minCard : Integer maxCard : Integer
RHS-MM
: kind
a
DesAtt b
IdAtt
1..1
Multiplicity minCard : Integer maxCard : Integer
Fig. 2. Structural Heterogeneities Between Metamodels - Example
As an example for structural metamodel heterogeneity consider the example shown in Figure 2. Two MOF-based metamodels represent semantically equivalent core concepts of the UML class diagram in different ways. Whereas the LHS metamodel uses only a small set of classes, the RHS metamodel employs a much larger set of classes thereby representing most of the UML concepts which are in the LHS metamodel implicitly defined as attributes or references explicitly as
262
G. Kappel et al.
first class citizens. More specifically, four structural metamodel heterogeneities can be found (cf. Subsection 3.3 - 3.6) which require mapping operators going beyond the simple one-to-one mappings provided by the mapping operators in Section 2. 3.2
CAR Mapping Language at a Glance
For resolving structural metamodel heterogeneities, we provide nine different core mapping operators as depicted in Figure 2. These nine mapping operators result from the possible combinations between the core concepts of metametamodels, namely class, attribute, and reference, which also led to the name of the CAR mapping language. These mapping operators are designed to be declarative and bi-directional and it is possible to derive executable transformations based on transformation nets. One important requirement for the CAR mapping language is that it should be possible to reconstruct the source models from the generated target models, i.e., any loss of information during transformation should be prevented. In Figure 3, the mapping operators are divided according to their functionality into the categories Copier, Peeler, and Linker which are explained in the following. Class
Attribute
Reference
Class
C2C
C2A
C2R
Att ib t Attribute
A2C
A2A
A2R
Reference
R2C
R2A
R2R
Legend
… Copier … Peeler … Linker
Fig. 3. CAR Mapping Operators and their categorization
Copier. The diagonal of the matrix in Figure 3 depicts the symmetric mapping operators of the CAR mapping language which have been already discussed in Section 2. The term symmetric means that the input and outport ports of the left side and the right side of the mapping operators are of the same type. This category is called Copier, because these mapping operators copy one element of the LHS model into the RHS model without any further manipulations. Peeler. This category consists of mapping operators which create new objects by ”peeling”3 them out of values or links. The A2C operator bridges heterogeneities which are resulting from the fact that a concept is expressed as an attribute in one metamodel and as a class in another metamodel. Analogously, a concept can be expressed on the LHS as a reference and on the RHS as a class which can be bridged by a R2C operator. Linker. The last category consists of mapping operators which either link two objects to each other out of value-based relationships (cf. A2R and R2A operator) 3
Note that the term ”peeling” is used since when looking at the white-box view the transformation of an attribute value into an object requires in fact to generate a one-colored token out of a two-colored token.
Leveraging Model-Based Tool Integration
263
or assign values or links to objects for providing the inverse variants of the A2C and R2C operators (cf. C2A and C2R operator). To resolve the structural heterogeneities depicted in Figure 2, in the following subsections the necessary mapping operators are discussed in detail, comprising besides a variation of the C2C operator mainly mapping operators falling into the above mentioned peeler and linker category. The solution for the integration example is shown in Figure 4 and Figure 5 on the mapping view and on the transformation view, respectively.
LHS
RHS
Bridging
SourceMM
MappingModel
TargetMM Package
C2C
Package
name : String
A2A
name : String A2R C2C
superClasses classes
Class *
A2A
name : String package : String
*
subClasses
Class name *: String St i
R2C
* *
Generalization superClasses
R2R
{inv: true} C2C
attributes ib
*
Attribute attributes
*
Attribute
name : String C2C
:kind
isID=false
name : String isID : Boolean minCard : Integer maxCard : Integer
A2A
C2C
1 1 1..1
Multiplicity minCard : Integer maxCard : Integer
DesAtt isID=ftrue
multiplicity
IdAtt
A2C
Fig. 4. Integration Example solved with CAR (Mapping View)
3.3
Conditional C2C Mapping Operator
Problem. In MOF-based metamodels, a property of a modeling concept can be expressed via a discriminator of an inheritance branch or with an additional attribute. An example for this kind of heterogeneity can be found in Figure 2(a), namely between Attribute.isID on the LHS and the subclasses of the class Attribute on the RHS. This heterogeneity is not resolvable with a primitive C2C operator per se, because one class on the LHS corresponds to several classes on the RHS whereby each mapping is only valid under a certain condition. On the model level, this means that a set of objects has to be splitted into several subsets based on the object’s attribute values.
264
G. Kappel et al.
LHS
Bridging
SourcePlaces Package
RHS
TransformationLogic
TargetPlaces Package
C2C
historyy
A2A
Package_name
Package_name historyy C2C
Class hi t history
Class
Class_name A2A
Class_name historyy
Package_classes
A2R
Class_package _p g
Generalization_subClasses
history R2C
Generalization
Class_superClasses
Generalization_superClasses R2R
history
Class_attributes Class_attributes history
C2C
Attribute
Attribute
C2C
DesAtt IdAtt
history
Attribute_name
isID=false s a se
C2C
Attribute_name history
Attribute_isID
isID=true
Attribute multiplicity Attribute_multiplicity
A2A
Multiplicity
Attribute_minCard history A2C
Multiplicity_minCard
Attribute_maxCard _ Multiplicity_maxCard history
Fig. 5. Integration Example solved with CAR (Transformation View)
Leveraging Model-Based Tool Integration
265
Solution. To cope with this kind of heterogeneity, the C2C operator has to be extended with the capability of splitting a set of objects into several subsets. For this we are annotating the C2C operator with OCL-based preconditions assigned to ports as depicted in Figure 6(a). These preconditions supplement the querytokens of the transitions by additionally allowing to specify constraints on the source model elements. The reason for introducing this additional mechanism is that the user should be able to configure the C2C operator without having to look into the white-box view of the operator, realizing its basic functionality. Example Application. In the example shown in Figure 4, we can apply two C2C mapping operators with OCL conditions, one for mapping Attribute to DesAtt with the precondition Attribute.isID = false, and one for mapping Attribute to IdAtt with the precondition Attribute.isID = true. In addition, this example shows a way how mappings can be reused within a mapping model by allowing inheritance between mappings. This mechanism allows to define certain mappings directly between superClasses and not for each subClass combination again and again (cf., e.g., the A2A mapping between the Attribute.name attributes), as described in more detail in Section 4. 3.4
A2C Mapping Operator
Problem. In Figure 2(b), the attributes minCard and maxCard, which are part of the class Attribute at the LHS, are at the RHS part of a dedicated class Multiplicity. Therefore, on the instance level, a mechanism is needed to ”peel” objects out of attribute values and to additionally take into account the structure of the LHS model in terms of the attribute’s owning class when building the RHS model, i.e., instances of the class Multiplicity must be connected the corresponding instances of class Attribute. Solution. The black-box view of the A2C mapping operator as illustrated in Figure 6(b) consists of one or more required interfaces for attributes on the LHS depending on how many attributes are contained by the additional class, and has in minimum three provided interfaces on the RHS. The first of these interfaces is used to mark the reference which is responsible to link the two target classes, the second is used to mark the class that should be instantiated, and the rest is used to link the attributes of the LHS to the RHS. Additionally, an A2C operator has a required interface to a C2C, because the source object is splitted into two target objects, thereby only one object is created by the A2C, the other has to be generated by a C2C operator which maps the LHS class to its corresponding target RHS class. The white-box view of the A2C operator shown in Figure 6(b) comprises a transition consisting of at least two query-tokens. The first query-token guarantees that the owningObject has been already transformed by a C2C operator. The other query-tokens are two-colored tokens representing the attribute values which have as fromColor the same color as the first query-token. The
266
G. Kappel et al.
object : Class
object : Class
C2C pre: OCL-Exp c2c
Black-Box View
Black-Box View
owningObject : c2c link2OwningObj : Reference value1 : Attribute
…
valueObject : Class
A2C
value1 : Attribute
…
valueN : Attribute
valueN : Attribute a2c
C2C object : Class
object : Class
History pre: OCL-Exp c2c
White-Box View
White-Box View
owningObject: c2c
A2C
link2OwningObj : Reference
value1 : Attribute
valueObject : Class
…
…
value1 : Attribute
…
valueN : Attribute
valueN : Attribute
History a2c
link2SourceObj : Reference link : Reference
R2C
link2TargetObj : Reference r2c sourceObject : c2c
targetObject : c2c
White-Box View
linkObject : Class
Black-Box View
sourceObject : c2c
targetObject : c2c
(b) A2C operator
R2C
link2SourceObj : Reference linkObject : Class
link : Reference
History
link2TargetObj : Reference
r2c
(c) R2C operator
targetObject : c2c
sourceObject : c2c keyValue : Attribute
link : Reference
A2R
keyRefValue : Attribute
a2r targetObject : c2c
sourceObject : c2c
White-Box View
Black-Box View
(a) Conditional C2C operator
A2R keyValue : Attribute keyRefValue : Attribute
link : Reference
History a2r
(d) A2R operator
Fig. 6. Black-box and white-box views of CAR mapping operators
post-condition of the transition consists of at least three generator-tokens. The second generator-token introduces a new color, i.e., this color is not used in the pre-condition part of the transition, and therefore, the generator-token produces a new object with a unique identity. The first generator-token is used for linking the newly created object appropriately into the target model and the other twocolored generator tokens are used to stream the values into the newly generated object by changing the fromColor of the input values. Example Application. In Figure 4, the attributes minCard and maxCard are mapped to attributes of the class Multiplicity. Furthermore, the reference between the classes Attribute and Multiplicity is marked by the A2C mapping as well as the class Multiplicity. To assure that the generated Multiplicity objects can be properly linked to Attribute objects, the A2C mapping is in the context of the C2C mapping between the Attribute classes.
Leveraging Model-Based Tool Integration
3.5
267
R2C Mapping Operator
Problem. In Figure 2(c), the reference superClasses of the LHS metamodel corresponds to the class Generalization of the RHS metamodel. This kind of heterogeneity requires an operator which is capable of ”peeling” an object out of a link and to additionally preserve the structure of the LHS in terms of the classes connected by the relationships at the RHS. Solution. The black-box view of the R2C mapping operator, as depicted in Figure 6(c), has one required interface on the left side for pointing to a reference. On the right side it has three provided interfaces, one for the class which stands for the concept expressed as reference on the LHS and two for selecting the references which are responsible to connect the object which has been peeled out of the link of the LHS into the RHS model. To determine the objects to which the peeled object should be linked, two additional required interfaces on the top of the R2C operator are needed for determining the corresponding objects of the source and target objects of the LHS. The white-box view of the R2C mapping operator, as illustrated in Figure 6(c), consists of a pre-condition comprising three query-tokens. The input link is connected to a two-colored query-token, the fromColor corresponds to the query-token standing for the source object and the toColor corresponds to a query-token standing for the target object. The post-condition of the transition introduces a new color and is therefore responsible to generate a new object. Furthermore, two links are produced by the other generator-tokens for linking the newly generated object with the corresponding source and target objects of the LHS. Example Application. In Figure 4, the reference superClasses in the LHS metamodel is mapped to the class Generalization by an R2C operator. In addition, the references subClasses and superClasses are selected for establishing an equivalent structure on the RHS as existing on the LHS. For actually determining the Class objects which should be connected via Generalization objects, the R2C operator has two dependencies to C2C mappings. This example can be seen as a special case, because the reference superClasses is a reflexive reference, therefore both requiredContext ports of the R2C operator point to the same C2C operator. 3.6
A2R Mapping Operator
Problem. The attribute vs. reference heterogeneity shown in Figure 2(d) resembles the well-known difference between value-based and reference-based relationships, i.e, corresponding attribute values in two objects can be used to ”simulate” links between two objects. Hence, if the attribute values in two objects are equal, a link ought to be established between them. Solution. For bridging the value-based vs. reference-based relationship heterogeneity, the A2R mapping operator as shown in Figure 6(d) provides on the LHS two interfaces, one for marking the keyValue attribute and another for marking
268
G. Kappel et al.
the keyRefValue attribute. On the RHS, the operator provides only one interface for marking the reference which corresponds to the keyValue/keyRefValue attribute combination. The white-box view of the operator comprises a transition which has four query-tokens. The first two ensure that the objects which are referencing each other on the LHS have been already transformed. The last two are the keyValue and keyRefValue query-tokens whereby the inner-color (representing the attribute values) is the same for both tokens. The generator-token of the transition produces one two-colored token by using the outer-color of the keyRefValue query-token as the outer-color and the outer-color of the keyValue query-token as the inner-color. Example Application. In Figure 4, the A2R operator is used to map the Package.name attribute as the key attribute and the Class.package attribute as the keyRef attribute of the LHS metamodel to the reference between Package and Class on the RHS metamodel. After presenting how common structural metamodel heterogeneities are resolved with CAR mapping operators, we proceed with discussing how mapping reuse within a single mapping model may be achieved. This is especially needed for large metamodels which consists of extensive class hierarchies such as the UML 2 metamodel. For example, as reported in [16], the number of single inheritance meta-classes in UML 2 is 209. Therefore, in the next section, an inheritance mechanism for CAR mapping operators for achieving reuse within a single mapping model is presented.
4
An Inheritance Mechanism for CAR Mappings
Why is an inheritance mechanism needed? Metamodeling languages such as MOF allow to define class hierarchies in metamodels through generalization which is frequently used in practice. Consequently, when one defines mappings between metamodels which heavily use generalizations leading to huge taxonomies, it should be possible to reuse previously defined mappings between general classes for mappings between subclasses. Required reuse mechanisms. To facilitate the definition of mapping models between MOF-based metamodels, reuse may be achieved in three ways: (1) Reuse of feature mappings: Mappings between features of superclasses may be defined once for the superclasses and may be inherited by mappings between subclasses which may define additional feature mappings between the subclasses. This reuse mechanism is comparable with code reuse through implementation inheritance supported by common object-oriented programming languages. (2) Reuse for indirect instances: The following mapping situation frequently occurs when two metamodels have to be integrated. Assume, we have on the LHS a superclass which has a multitude of subclasses. In contrast, on the RHS the class hierarchy on the LHS is collapsed into a single class inhibiting all features of the LHS class hierarchy at the RHS, which is equivalent to the LHS
Leveraging Model-Based Tool Integration
269
superclass and also to its subclasses. To avoid that for each subclass a mapping to the RHS class must be defined, it should be possible to apply the mapping between the LHS superclass and the RHS class also for indirect instances of the LHS superclass. (3) Refinement of mappings: A refinement mechanism is needed to define for certain subclasses specific mappings, which refine the mapping between the superclasses. For example, instances of a LHS subclass should be transformed into instances of a RHS subclass and not into instances of the RHS superclass. Of course, the feature mappings between the superclasses should be also applied for such submappings. In the following subsections, we elaborate on how the CAR mapping language is extended in order to provide these three reuse features. 4.1
Inheritance for C2C Mappings
For reusing existing feature mappings (first reuse mechanism), we introduce the possibility to define generalization relationships between C2C mappings. This means, the user defines general mappings between superclasses called supermappings and more specific mappings between subclasses called submappings which may be used to refine the supermappings (third reuse mechanism). As concrete syntax for generalization relationships between C2C mappings, we reuse the notation of UML generalization relationships between classes, i.e., a line with a hollow triangle as an arrowhead. Concerning the second reuse mechanism, it has to be noted that there are also cases in that the mentioned behavior of applying a supermapping for indirect instances is not desired. Sometimes it is required that only direct instances should be transformed and not indirect instances. Therefore, certain configuration parameters for mappings are required in order to express such integration details in the mapping model.
CAR R
superMappings
* C2C
* isAbstract: boolean isApplicable4Subclasses: boolean subMappings
MOF
LHSClass
1
1 Class
isAbstract: boolean superClasses
RHSClass subClasses
* *
Fig. 7. C2C Operator Extended with Generalization Relationships
We allow generalization relationships only for C2C mappings for inheriting feature mappings which are dependent on C2C mappings such as symmetric mappings (A2A, R2R), or asymmetric mappings (A2C, R2C, A2R, and their inverse operators). This is due to the fact that C2C operators are responsible for providing the context information for all other CAR mapping operators. The introduction of generalization relationships between C2C mappings
270
G. Kappel et al.
results in an extension of the C2C operator as shown in Figure 7. In this figure, the class C2C, representing the C2C mapping operator, is extended for defining generalization relationships by setting the references superMappings and subMappings accordingly. Furthermore, for allowing different kinds of supermappings, i.e., if a mapping is itself executable or if it is applicable for unmapped subclasses, two additional boolean attributes, namely C2C.isAbstract and C2C.isApplicable4SubClasses, are defined for the C2C class. These two attributes enable the user to configure the behavior of the mappings in more detail. One important constraint for generalization relationships between C2C operators is that if a generalization between two C2C operators is defined, the participating LHS classes of the supermappings and the submappings must be either in a generalization relationship or it must be actually the same class. Of course, the same constraint must hold on the RHS. These two constraints must be ensured, because the submappings inherit the feature mappings of the supermappings and therefore, the features of the superclasses must be also available on instances which are transformed according to the submappings. The OCL constraints shown in Listing 1.1 validate a mapping model with respect to the correct usage of generalization relationships between C2C mappings. Listing 1.1. Well-formedness Rules for C2C Generalizations context C2C inv : s e l f . su p e r M a p p i n g s −> f o r A l l ( supMap | supMap . LHSClass . s u b C l a s s e s −> u n i o n ( supMap . LHSClass ) −> c o n t a i n s ( s e l f . LHSClass ) ) ; inv : s e l f . su p e r M a p p i n g s −> f o r A l l ( supMap | supMap . RHSClass . s u b C l a s s e s −> u n i o n ( supMap . RHSClass ) −> c o n t a i n s ( s e l f . RHSClass ) ) ;
The default configuration of the C2C operator for each mapping situation specifies that the supermapping itself is executable and applicable for indirect instances. In order to give the user more possibilities to explicitly define other interpretations of supermappings, we furthermore allow three non-default supermapping configurations, thereby the first configuration allows to define abstract supermappings with the capability to be applied for indirect instances, and the other two configurations allow reuse of depending mappings of supermappings without applying the supermappings on indirect instances. For further explanations how to use generalization between C2C operators, we assume that the mapping problem is symmetric, i.e., the same generalization structure is available on the LHS and on the RHS, and that only single inheritance is used for defining the metamodels. In particular, we assume that on the LHS and on the RHS a superclass with various subclasses exists. For asymmetric mapping problems, i.e., one side has a taxonomy and the other has not, and integration scenarios where metamodels use multiple inheritance the interested reader is kindly referred to [32]. 4.2
Representing Inheritance within Transformation Nets
In this subsection we discuss how C2C generalization relationships influence the generation of transformation nets and consequently the execution of the
Leveraging Model-Based Tool Integration
271
transformation logic. An overall design goal is naturally to express new language concepts at the black-box view – such as mapping generalizations in this case – as far as possible by means of existing transformation net mechanisms. Basic Idea. When we take a closer look on supermappings with a standard configuration, we see that these mappings must provide the context, i.e., the trace model information, for all dependent mappings. This means, the supermappings must also provide context information about the transformation of indirect instances, e.g., for assigning attribute values of indirect instances when the attribute is contained by the superclass. Consequently, for a supermapping a transformation component is derived which contains the union of its own trace model for logging the transformation of direct instances of the superclass and the trace models of its submappings for logging the transformation of indirect instances. Therefore, the corresponding transformation components of the submappings are nested into the transformation component of the supermapping. For constructing the union of trace models of nested transformation components, each nested component gets an arc from its own trace model to the union trace model of the outer component. Mappings which depend on the supermapping are connected to the union trace model available on the outer component and mappings which are dependent on submappings are directly connected to the individual trace models of the nested components. Transformation Net
Mapping Model
C2C1 C2C1
history
C2C2.1 C2C3.1 C2C2.1
C2C2.2
C2C3.2
… C2C3.1
C2C3.2
…
C2C2.2
union(2.1, 3.1, 3.2, …)
union(1, 2.1,2.2)
Fig. 8. Representing Inheritance Structures with Nested Transformation Components
Figure 8 illustrates the derivation of generalization relationships into transformation net components. For describing the basic mapping rule how generalization relationships are represented in transformation nets, it is assumed that all mappings are concrete mappings and it is not considered if a mapping is applicable for subclasses or not. The mapping C2C1 of the mapping model shown on the LHS of Figure 8 is transformed into the outer component C2C1 , which consists of a transition for transforming direct instances and of two subcomponents C2C2.1 and C2C2.2 . In addition, the outer component provides a union trace model of the transformation components C2C1 , C2C2.1 , and C2C2.2 . Because
272
G. Kappel et al. pre: oclIsTypeOf(C1|C3)
C2C1 history
Config 1: C2C1.isAbstract = false C2C1.isApplicable4SubClasses = true
Rule1
C2C2
C2C1
pre: oclIsTypeOf(C3)
Config 2:
Mapping Model LHS_MM
C1
C3
C2
RHS_MM C2C1
C1‘ C1
C2C2
C2‘ C2
C2C1.isAbstract = true C2C1.isApplicable4SubClasses = true
history
Rule2
C2C2
C2C1
pre: oclIsTypeOf(C1)
Config 3: C2C1.isAbstract = false C2C1.isApplicable4SubClasses = false
history
Rule3
C2C1
Config 4: C2C1.isAbstract i Ab t t = true t C2C1.isApplicable4SubClasses = false
C2C2
R l 4 Rule4
C2C2
Fig. 9. Representing Supermapping Configurations in Transformation Components
the mapping C2C2.1 has two submappings, the corresponding transformation component has also two subcomponents C2C3.1 and C2C3.2 . In addition, the component C2C2.1 provides a union trace model of itself and the subcomponents C2C3.1 and C2C3.2 . Mapping Rules for Supermapping Configurations. In addition to the derivation of inheritance structures to nested transformation components, specific derivation rules for the configuration variants of the supermappings are needed to represent abstract and concrete mappings in transformation nets as well as the applicability of supermappings for subclasses. In particular, the following four rules, which are summarized in Figure 9, are sufficient to generate transformation nets for all possible supermapping configurations. The mapping model shown in Figure 9 is used as an example input mapping model for describing the mapping rules and comprises a mapping between the superclasses C1 and C1’ of the LHS and RHS metamodels and between the subclasses C2 and C2’, whereby the subclass C3 of the LHS remains unmapped. Rule 1 - Concrete/Applicable Supermapping: When a supermapping is concrete, a transition is available in the outer transformation component for transforming direct instances of the superclass and indirect instances for which no specific mappings are available. Because only direct and indirect instances of subclasses without specific mappings should be transformed by the transition of the outer component, an OCL condition is attached on the inputPort which leads to the transition in order to reject tokens for which more specific mappings are available. Such constraints can be defined with the OCL function oclIsTypeOf
Leveraging Model-Based Tool Integration
273
which gets as parameters the superclass and all subclasses for which no specific mappings have been defined in the mapping model (cf. OCL condition oclIsTypeOf(C1 |C3)). If there is a more specific mapping between subclasses, a nested component is produced and the tokens are not streamed via the superclass mapping, instead the subplace generated from the LHS subclass gets an additional arc which leads to a more specific transformation component. Rule 2 - Abstract/Applicable Supermapping: Although the supermapping is abstract, a transition resides directly in the outer component, which is not applicable for direct instances but for transforming all indirect instances for which no specific mapping has been applied (cf. OCL condition oclIsTypeOf(C3)). Rule 3 - Concrete/Non-Applicable Supermapping: If a supermapping is defined as concrete and non-applicable for unmapped subclasses then an outer component is produced which consists of a transition for transforming direct instances of the superclass (cf. OCL condition oclIsTypeOf(C1)). Rule 4 - Abstract/Non-Applicable Supermapping: When a supermapping is abstract and non-applicable for unmapped subclasses only the outer component is generated for providing a union trace model for its submappings. This is sufficient, because neither direct instances nor indirect instances have to be transformed by such a component. Design Alternatives. The following three design alternatives exist for transformation nets to model the applicability of the supermapping transition on subPlaces. First, we could extend the place modeling constructs with tags such as “superTransition is applicable”. However, the introduction of such a transformation net feature would violate our design goal that the transformation net formalism should not be changed. The second possibility is to generate for each unmapped class an additional arc from the corresponding source place to the outer component generated for the supermapping. This variant would lead to much more complicated transformation nets and to a huge amount of duplicated arcs, which simply does not pay off the information gain for the user. Therefore, we decided for a third variant, namely the usage of OCL constraints as explained for Rule 1 to 3. Example. To summarize this section, a concrete integration example, as shown in Figure 10, is discussed on the mapping view and on the transformation view. In the LHS metamodel, a class Person is specialized into Supplier, Employee, and Customer classes. The RHS metamodel consists also of a superclass Person, and of Client, Staff, and ShareHolder subclasses. Each LHS class can be mapped to a RHS class, except the class Supplier. Hence, the LHS class Person is mapped with a C2C mapping operator to the RHS class Person. The properties of this C2C are set to isAbstract=FALSE and Applicable4SubClasses=TRUE. Consequently, each instance of the LHS class Person is transformed into an instance of the RHS class Person, as well as each instance of a subclass which has no further refinement mapping is also transformed into an instance of the RHS Person class. For example, each instance of the class Supplier becomes an instance of the class Person. Additionally, the name attribute of the LHS class Person is
274
G. Kappel et al.
LHS
RHS
Bridging MappingModel
SourceMM
TargetMM
M Mapping gView
C2C
Person
Person
A2A
name:String
name:String C2C
Client
Customer cuNr:int
Supplier suNr:int
C2C
clNr:int
Employee p y
Staff
A2A
emNr:int
Share Holder
stNr:int
shNr:int
A2A
Transform SourcePlaces
Transform
Transform
TransformationLogic
TargetPlaces
C2C
Person Supplier
Employee
Person
history
Customer
Client
C2C
ShareHolder Staff
Transformation nView
history
C2C history
Person_name
Person_name
A2A
Employee_emNr
Staff_stNr history
A2A
Customer_cuNr
Client_clNr history
Supplier_suNr
ShareHolder_shNr
A2A
history
Fig. 10. Inheritance between C2C Mappings - Example
mapped by an A2A mapping operator to the name attribute of the RHS class Person. The subclasses Employee and Customer of the class Person on the LHS are mapped by C2C mappings to Staff and Client of the RHS, respectively. Additionally, the attributes of these classes, namely Customer.cuNr, Employee.emNr, and Client.clNr, Staff.stNr, are mapped by A2A mappings, respectively. Due to the fact that each of the subclasses inherit the attributes of the superclass – the attribute Person.name – the A2A mapping between the superclasses is also inherited by the C2C mappings by setting the superMappings reference to the C2C mapping which resides between the Person classes.
Leveraging Model-Based Tool Integration
275
The corresponding transformation net for the presented mapping model is depicted in the Transformation View of Figure 10. The Person classes become places which comprise for each subclass an inner place. As subclass places are nested in superclass places, the inheriting submappings are nested in the transformation component which corresponds to the supermapping. The outer transformation component, corresponding to the supermapping, contains a transition, because the isAbstract property of the C2C mapping is set to FALSE. Furthermore, due to the isApplicable4SubClasses property of the C2C mapping, which is set to TRUE, the outer transformation component of the transformation net owns an additional OCL constraint, namely oclTypeOf(Person|Supplier). Due to readability purposes, we refrain from displaying these features in Figure 10. Consequently, each direct instance of type Person from the LHS is transformed into an instance of class Person on the RHS. Furthermore, this OCL constraint ensures that each instance of subclasses of the class Person, which has no refined mapping, is also transformed by the supermapping into an instance of type Person on the RHS. The attribute Person.name can be transformed only if the containing instance which can be of type Person, Employee, or Customer has been already transformed. Consequently, the A2A transformation component for name values must be in the context of three C2C transformation components. This is achieved by the trace model provided by the black port in the middle of the bottom of the outer C2C transformation component. This trace model unifies the individual trace models of the C2C transformation components. The other A2A operators are connected to the gray ports which link directly to individual trace models of the nested components. This example shows the value of using an explicit notion of trace models together with a union of trace models for automatically synchronizing transformation net components. With the automatic synchronization provided by the Petri Net semantic, the streaming of dependent elements such as attribute values and links comes for free. Thus, no additional control structures and OCL constraints are required, and the derivation of transformation nets from mapping models is straightforward.
5
Evaluation
In this section, we discuss the applicability of our approach by (1) elaborating on how additional tool integration requirements, namely roundtrip capabilities, may be supported by our approach and by (2) comparing the efforts of building such roundtrip transformations using the proposed mapping approach and using model transformation languages. Roundtrip Transformations. In case modeling languages of two different tools are not entirely overlapping, i.e., some modeling concepts are available in one modeling language which cannot be expressed in the other modeling language, a transformation may be lossy. Thus, although transformations from tool A to tool B and vice versa are available, the initial model Ma of tool A may be
276
G. Kappel et al.
different from the roundtrip result Ma which is computed by translating Ma into Mb via the transformation Ta2b and the application of Tb2a on Mb to produce Ma . The main reason for not roundtripping transformations is the fact that bijective mappings are not always possible to establish between metamodels as for example reported in [27]. Here the challenging question arises, how to deal with such cases in order to ensure roundtripping transformations. In the ModelCVS project4 , besides other integration efforts, we have bridged the Domain Specific Language (DSL) for defining database schemas of the AllFusion Gen5 (AFG) modeling tool to the class diagram concepts of the UML Rational Software Modeler6 tool. Thereby, the first case study was to bridge structural modeling, i.e., the AFG Data Model with the UML Class Diagram. The first attempt was to bridge AFG with plain UML. The resulting bridge was not appropriate for using the tools in combination. Because, although we have defined for each AFG modeling concept a specific mapping to UML, a lot of information was lost during roundtrip or even though after the first step when moving from the AFG to UML. Table 1 summarizes the roundtrip scenario by depicting some model metrics for each step in the roundtrip. Table 1. Model Metrics for Data Model/Class Diagram RoundTrip (RT) Metrics Initial AFG Model UML Model AFG Model after RT Diff in % #Objects #Values #Links #Containment Links File Size
156 1099 44 155 32,8 KB
165 156 54 164 16 KB
156 156 36 155 14,6 KB
0 85,8 18,2 0 55,5
The main reason for the massive loss of information was the fact that on the attribute level only a minimal overlap between the languages exists. In most cases, only the name attribute of the modeling concepts may be bridged, but all platform specific attributes of the AFG modeling language such as database optimization information may not. When we take a look at the model metrics in Table 1, the initial AFG model and the generated UML model have nearly the same amount of objects and containment links, only some derived objects are additionally instantiated in the UML model. This means, the same model structure can be reproduced on the UML side. However, when we compare the amount of values, we see that a huge amount of information gets lost in the first step. In particular, when comparing the number of values of the initial AFG model and the resulting AFG model after roundtrip, 85,8 % of values are lost during the roundtrip. For links which are not containment links we see that more links exist in the generated UML model compared to the initial AFG model. This is due to the fact, that also derived links are generated for the aforementioned 4 5 6
http://www.modelcvs.org http://ca.com/us/products/product.aspx?ID=256 http://www-306.ibm.com/software/awdtools/modeler/swmodeler
Leveraging Model-Based Tool Integration
277
additionally derived objects. Therefore, even though we have more links and objects on the UML side, less information is expressed and some links cannot be reconstructed when transforming the UML models back to AFG models. Finally, the information loss has of course an effect on the file size, namely the resulting file after roundtrip has only half the size of the initial file. Table 2. Model Metrics for Data Model/Class Diagram Roundtrip Revisited Metrics Initial AFG Model UML Model AFG Model after RT Diff in % #Objects #Values #Links #Containment Links File Size #Annotations #Key/Value Pairs
156 1099 44 155 32,8 KB
165 156 54 164 58,5 KB
156 1099 44 155 32,8 KB
0 0 0 0 0
-
156 951
-
-
In Table 2, the model metrics are again presented, however, now for unmapped features (i.e., attributes and references) dedicated annotations are created on the UML side by using an enhanced transformation generation from mapping models tailored to preserving information during roundtrip. As one can see in the most right column of the table, no difference regarding the number of model elements between the initial AFG model and the AFG model after roundtrip exists. The information can be preserved by applying for each corresponding object an appropriate annotation allowing to use key/value pairs for saving attribute values which otherwise would be lost. This specific logic can be easily introduced in the transformation generation thanks to the declarative nature of the mapping models. Furthermore, a comparison of the initial AFG model and the resulting AFG model using the comparison facility of EMF Compare7 demonstrated that both were equivalent, thus no information has been lost during roundtrip. Metrics for Mapping Model and corresponding Model Transformations. In addition to the evaluation of the roundtrip capability of the generated bridge, the number of model elements of the manually created mapping model compared to the number of elements needed for the corresponding model transformations defined in the de-facto model transformation standard in Eclipse, namely the ATLAS Transformation Language (ATL) [8], is presented. This comparison should give an indication on how much effort requires the creation of the mapping model in contrast to building the integration artifacts manually from scratch by using a textual model transformation language such as ATL. Table 3 summarizes some metrics for the mapping model and for the corresponding ATL model transformations. We decided to use as metrics for the mapping model, first, the number of applied MOPs, and second, how many non-default property values (e.g., for setting inheritance relationships between MOPs) have to be set by the user, because this is exactly the work the user has 7
www.eclipse.org/modeling/emft
278
G. Kappel et al.
to do for creating the mapping model. For the transformation code we simply use lines of code (LOC) as metric, just to give an indication how much effort the manual implementation of the transformations for both directions (AFG2UML, UML2AFG) would cause. Table 3. Metrics Overview: Mapping Model vs. ATL Code Mapping Model Metrics Mapping Operator User Actions C2C A2A R2R Properties
6 1 7 7
Model Transformation Metrics ATL File Lines of Code AFG2UML
UML2AFG
120 overall 75% declarative 25% imperative 100 overall 86% declarative 14% imperative
For realizing model exchange between AFG and UML, in total 220 lines of ATL code are necessary, where most parts are declarative rules. However, for using annotations within the transformation for saving unmapped values, imperative code is needed, e.g., for applying annotations and for setting key/value pairs when moving from AFG to UML as well as for assigning key/value pairs as attribute values when we are going back from UML to AFG. Instead, using the presented mapping approach, we are able to develop the same bridge using only 14 MOPs and setting 7 properties of the applied MOPs from which the transformations in both directions are derived. For dealing with annotations for setting unmapped values, the generator for the transformations is capable of producing this dedicated logic without requiring the user to define mappings for this aspect. Only the generator has been extended with one additional generation rule which can be turned on/off as required. Compared to the manual authoring of model transformations where this aspect is intermingled with the rest of the code, the presented mapping approach allows for a faster and more systematic development of tool integrations.
6
Related Work
With respect to our approach of defining reusable mapping operators for resolving metamodel heterogeneities as a kind of mapping between metamodels we distinguish between three broad categories of related work: first, related work concerning our goal to design a framework for building reusable mapping operators in the field of MDE, and second, related work concerning our solution approach in the field of ontology integration. In addition, we elaborate on related approaches which employ Petri Nets as conceptual modeling language for defining model transformations. 6.1
Reusable Model Transformations
Generic Model Transformations. Typically model transformation languages, e.g., ATL [8] and QVT [21], allow to define transformation rules based on types
Leveraging Model-Based Tool Integration
279
defined as classes in the corresponding metamodels. Consequently, model transformations are not reusable and must be defined from scratch again and again with each transformation specification. One exception thereof is the approach of Varr´ o et al. [30] who propose a notion of specifying generic transformations within their VIATRA2 framework, which in fact resembles the concept of templates in C++ or generics in Java. VIATRA2 also provides a way to implement reusable model transformations, although it does not foster an easy to debug execution model as is the case with our proposed transformation nets. In addition, there exists no explicit mapping model between source and target metamodel which makes it cumbersome to reconstruct the correspondences between the metamodel elements based on the graph transformation rules, only. Transformation Patterns. Very similar to the idea of generic transformations is the definition of reusable idioms and design patterns for transformation rules described by Karsai et al. [1]. Instead of claiming to have generic model transformations, the authors propose the documentation and description of recurring problems in a general way. Thus, this approach solely targets the documentation of transformation patterns. Realization issues how these patterns could be implemented in a generic way remain open. Mappings for bridging metamodels. Another way of reuse can be achieved by the abstraction from model transformations to mappings as is done in our approach or by the ATLAS Model Weaver (AMW) [6]. AMW lets the user extend a generic so-called weaving metamodel, which allows the definition of simple correspondences between two metamodels. Through the extension of the weaving metamodel, one can define the abstract syntax of new weaving operators which roughly correspond to our mapping operators. The semantics of weaving operators are determined by a higher-order transformation that take a weaving model as input and generates model transformation code. Compared to our approach, the weaving models are compiled into low-level transformation code in terms of ATL which is in fact a mixture of declarative and imperative language constructs. Thus, it is difficult to debug a weaving model in terms of weaving operators, because they do not explicitly remain in the model transformation code. Furthermore, the abstraction of mapping operators from model transformations expressed in ATL seems more challenging compared to the abstraction from our proposed transformation net components. 6.2
Ontology Mapping for Bridging Structural Heterogeneities
In the field of ontology engineering, several approaches exist which make use of high-level languages for defining mappings between ontologies (cf. [9] for an overview). For example, in Maedche et al. [17], a framework called MAFRA for mapping two heterogeneous ontologies is proposed. Within this framework, the mapping ontology called Semantic Bridge Ontology usually provides different ways of linking concepts from the source ontology to the target ontology. In addition to the Semantic Bridge Ontology, MAFRA provides an execution platform for the defined mappings based on services whereby for each semantic
280
G. Kappel et al.
bridge type a specific service is available for executing the applied bridges. In [24], Scharffe et al. describe a library of so called Ontology Mediation Patterns which can be seen as a library of mapping patterns for integrating ontologies. Furthermore, the authors provide a mapping language which incorporates the established mapping patterns and they discuss useful tool support around the pattern library, e.g., for transforming ontology instances between different ontology schemas. The main difference to our approach is that ontology mapping approaches are based on Semantic Web standards, such as OWL and RDFS, and therefore contain mapping operators for typical description logic related mapping problems, e.g., union or intersection of classes. We are bridging metamodels expressed in MOF, a language which has only a partial overlap with OWL or RDFS, leading to different mapping problems. Furthermore, in contrast to the ontology mapping frameworks, we provide a framework allowing to build new mapping operators by using well-known modeling techniques not only for defining the syntax but also for the operational semantics of the operators. 6.3
Petri Nets and Model Transformations
The relatedness of Petri Nets and graph rewriting systems has also induced some impact in the field of model transformation. Especially in the area of graph transformations some work has been conducted that uses Petri nets to check formal properties of graph production rules. Thereby, the approach proposed in [31] translates individual graph rules into a place/transition net and checks for its termination. Another approach is described in [5], which applies a transition system for modeling the dynamic behavior of a metamodel. Compared to these two approaches, our intention to use Petri Nets is totally different. While these two approaches are using Petri Nets as a back-end for automatically analyzing properties of transformations by employing place/transition nets, we are using Colored Petri Nets as a front-end for debuggability and understandability of transformations. In particular, we are focussing on how to represent model transformations as Petri Nets in an intuitive manner. This also covers the compact representation of Petri Nets to eliminate the scalability problem of low-level Petri nets. Finally, we introduce a specific syntax for Petri Nets used for model transformations and integrate several high-level constructs, e.g., colors, inhibitor arcs, and pages, into our language. However, it has to be noted that the higher expressivity gained from high-level constructs comes with a negative impact on the analyzability of the Petri Nets.
7
Conclusion and Future Research Issues
In this paper we have presented a framework allowing the definition of mapping operators and their application for building metamodel bridges. Metamodel bridges are defined by the user on a high-level mapping view which represents the semantic correspondences between metamodel elements and are tested and
Leveraging Model-Based Tool Integration
281
executed on a more detailed transformation view which also comprises the transformation logic of the mapping operators. The close integration of these two views and the usage of models during the whole integration process further enhances the debugging of the defined mappings in terms of the mapping operators. The applicability of the framework has been demonstrated by implementing mapping operators for resolving structural metamodel heterogeneities8 . The proposed framework is expressive enough to define advanced composition mechanisms such as inheritance between mapping operators. This is achieved by the explicit notion of trace models for mapping operators in combination with the automatic synchronization provided by the Petri Net semantic. The presented framework has been applied in several modeling tool integration projects (for more details the interested reader is kindly referred to www.modelcvs.org and www.modeltransformation.net). It has to be mentioned that the presented approach is not only applicable for integrating structural modeling languages, but also for integrating behavioral modeling languages. For example, we have integrated the dialog flow modeling language of CA’s AllFusion Gen with UML state machines as well as with UML activity diagrams. The most interesting point of this case study was that we explored nearly the same metamodel heterogeneities as we explored when integration structural modeling languages. Thus, the presented metamodel heterogeneities seem to be modeling domain independent. The work presented in this chapter leaves several issues open for further research. In the following, we present four research issues that we believe are most important for the success of model-based tool integration. (1) Bridging Technical Spaces. Several modeling languages are not described with MOF-based metamodels as proposed by the OMG. Instead, text-based languages such as EBNF, DTD, or XML schema are employed. In order to use model-based integration frameworks which require MOF-based metamodels, converters are needed. In particular, not only the language definition must be converted into a MOF-based metamodel, also the models have to be transformed into instances conforming to the generated metamodels. This raises the question of how to produce such converters for bridging different technical spaces [15] with reasonable effort also in the light of the evolution of these languages. (2) Automatic Creation of Mapping Models. Another issue is the automatic creation of mapping models between two metamodels. With the rise of the semantic web and the emerging abundance of ontologies, several matching approaches and tools for automatically creating mapping models have been proposed, for an overview see [22,26]. The typical output of such tools are simple one-to-one correspondences. However, these correspondences cannot cope with structural heterogeneities between MOF-based metamodels as presented in this work. Therefore, a novel matching approach is needed which is capable to automatically generate mappings expressed with a more powerful mapping language. 8
For more details about the implementation, we kindly refer the interesting reader to www.modeltransformation.net
282
G. Kappel et al.
(3) Formal Verification of Mapping Models. As the correctness of the automatically generated target model fully depends on the correctness of the specified mapping model, formal underpinnings are required to enable verification of mapping models by proving certain properties like confluence and termination, to ease debugging of mapping models. The formal underpinning of CPNs enables simulation of mapping models and exploration of the state space, which shows all possible firing sequences of a CPN. In the future, it has to be determined how generally accepted behavioral properties, characterizing the nature of a certain CPN, e.g., with respect to confluence or termination, as well as custom functions, e.g., to check if a certain target model can be created with the given transformation logic, can be applied for interactive debugging and automatic verification of mapping models [33]. (4) The Role of Semantics. An open research problem in MDE is how to explicitly and formally specify the semantics of modeling languages. In the last decade several diverse approaches inspired from programming language engineering have been proposed. However, in contrast to syntax, currently there is no commonly approved or standardized approach as well as tool support for defining the semantics of modeling languages. Thus, often the semantics are only informally specified in terms of natural language—the most prominent example is UML—or the semantics are hard-coded in code generators, simulators, or interpreters. However, for bridging modeling tools, an explicit notion of semantics would be of paramount importance. For example, when integrating tools for modeling state machines, it can happen that the modeling tools provide completely the same syntax definition, but the execution of the same model in tool A can differ from the execution in tool B, as is for example reported in [4]. Thus, one of the biggest challenges in MDE is how to provide explicit semantic definitions for modeling languages and how to use these definitions to build and verify integration solutions.
References 1. Agrawal, A., Vizhanyo, A., Kalmar, Z., Shi, F., Narayanan, A., Karsai, G.: Reusable Idioms and Patterns in Graph Transformation Languages. In: Proceedings of the International Workshop on Graph-Based Tools (GraBaTs 2004) (2004) 2. Batini, C., Lenzerini, M., Navathe, S.B.: A Comparative Analysis of Methodologies for Database Schema Integration. ACM Computing Survey 18(4), 323–364 (1986) 3. Bernstein, P.A., Melnik, S.: Model management 2.0: manipulating richer mappings. In: Proceedings of the ACM SIGMOD International Conference on Management of Data, China (2007) 4. Crane, M.L., Dingel, J.: UML vs. classical vs. rhapsody statecharts: not all models are created equal. Software and System Modeling 6(4), 415–435 (2007) 5. de Lara, J., Vangheluwe, H.: Translating Model Simulators to Analysis Models. In: Fiadeiro, J.L., Inverardi, P. (eds.) FASE 2008. LNCS, vol. 4961, pp. 77–92. Springer, Heidelberg (2008) 6. Fabro, M.D.D., B´ezivin, J., Jouault, F., Breton, E., Gueltas, G.: AMW: a generic model weaver. In: Proceedings of the 1re Journe sur l’Ingnierie Dirige parles Modles, IDM 2005 (2005)
Leveraging Model-Based Tool Integration
283
7. Jensen, K.: Coloured Petri Nets: Basic Concepts, Analysis Methods and Practical Use. Springer, Heidelberg (1992) 8. Jouault, F., Kurtev, I.: Transforming Models with ATL. In: Bruel, J.-M. (ed.) MoDELS 2005. LNCS, vol. 3844, pp. 128–138. Springer, Heidelberg (2006) 9. Kalfoglou, Y., Schorlemmer, W.M.: Ontology mapping: The state of the art. In: Dagstuhl Seminar Proceedings: Semantic Interoperability and Integration (2005) 10. Kappel, G., Kapsammer, E., Kargl, H., Kramler, G., Reiter, T., Retschitzegger, W., Schwinger, W., Wimmer, M.: Lifting Metamodels to Ontologies: A Step to the Semantic Integration of Modeling Languages. In: Wang, J., Whittle, J., Harel, D., Reggio, G. (eds.) MoDELS 2006. LNCS, vol. 4199, pp. 528–542. Springer, Heidelberg (2006) 11. Kappel, G., Kargl, H., Kramler, G., Schauerhuber, A., Seidl, M., Strommer, M., Wimmer, M.: Matching Metamodels with Semantic Systems - An Experience Report. In: Workshop Proceedings of Datenbanksysteme in Business, Technologie und Web (BTW 2007) (2007) 12. Kappel, G., Kargl, H., Reiter, T., Retschitzegger, W., Schwinger, W., Strommer, M., Wimmer, M.: A Framework for Building Mapping Operators Resolving Structural Heterogeneities. In: Proceedings of 7th Int. Conf. on Information Systems Technology and its Applications (2008) 13. Kapsammer, E., Kargl, H., Kramler, G., Kappel, G., Reiter, T., Retschitzegger, W., Schwinger, W., Wimmer, M.: On Models and Ontologies - A Semantic Infrastructure Supporting Model Integration. In: Proceedings of Modellierung 2006 (2006) 14. Kashyap, V., Sheth, A.P.: Semantic and schematic similarities between database objects: A context-based approach. VLDB Journal 5(4), 276–304 (1996) 15. Kurtev, I., Aksit, M., B´ezivin, J.: Technical Spaces: An Initial Appraisal. In: Meersman, R., Tari, Z., et al. (eds.) CoopIS 2002, DOA 2002, and ODBASE 2002. LNCS, vol. 2519. Springer, Heidelberg (2002) 16. Ma, H., Shao, W.-Z., Zhang, L., Ma, Z.-Y., Jiang, Y.-B.: Applying OO Metrics to Assess UML Meta-models. In: Baar, T., Strohmeier, A., Moreira, A., Mellor, S.J. (eds.) UML 2004. LNCS, vol. 3273, pp. 12–26. Springer, Heidelberg (2004) 17. Maedche, A., Motik, B., Silva, N., Volz, R.: MAFRA – A MApping FRAmework for Distributed Ontologies. In: G´ omez-P´erez, A., Benjamins, V.R. (eds.) EKAW 2002. LNCS (LNAI), vol. 2473, p. 235. Springer, Heidelberg (2002) 18. Olsen, G.K., Aagedal, J., Oldevik, J.: Aspects of Reusable Model Transformations. In: Proceedings of the 1st European Workshop on Composition of Model Transformations (CMT 2006) (2006) 19. OMG: UML Superstructure Specification, version 2.0 formal/05-07-04 edition (2005) 20. OMG: Meta Object Facility (MOF) 2.0 Core Specification, version 2.0 formal/200601-01 edition (2006) 21. OMG: Meta Object Facility (MOF) 2.0 Query/View/Transformation Specification, version 1.0 formal/2008-04-03 edition (2008) 22. Rahm, E., Bernstein, P.A.: A survey of approaches to automatic schema matching. VLDB Journal 10(4), 334–350 (2001) 23. Reiter, T., Wimmer, M., Kargl, H.: Towards a runtime model based on colored Petri-nets for the execution of model transformations. In: 3rd Workshop on Models and Aspects, in conjunction with ECOOP 2007 (2007) 24. Scharffe, F., de Bruijn, J.: A language to specify mappings between ontologies. In: Proceedings of the 1st International Conference on Signal-Image Technology & Internet-Based Systems (SITIS 2005) (2005)
284
G. Kappel et al.
25. Schmidt, D.C.: Guest editor’s introduction: Model-driven engineering. IEEE Computer 39(2), 25–31 (2006) 26. Shvaiko, P., Euzenat, J.: A survey of schema-based matching approaches. In: Spaccapietra, S. (ed.) Journal on Data Semantics IV. LNCS, vol. 3730, pp. 146–171. Springer, Heidelberg (2005) 27. Stevens, P.: Bidirectional Model Transformations in QVT: Semantic Issues and Open Questions. In: Engels, G., Opdyke, B., Schmidt, D.C., Weil, F. (eds.) MODELS 2007. LNCS, vol. 4735, pp. 1–15. Springer, Heidelberg (2007) 28. Tratt, L.: Model transformations and tool integration. Software and System Modeling 4(2), 112–122 (2005) 29. Vallecillo, A., Koch, N., Cachero, C., Comai, S., Fraternali, P., Garrig´ os, I., G´ omez, J., Kappel, G., Knapp, A., Matera, M., Meli´ a, S., Moreno, N., Pr¨ oll, B., Reiter, T., Retschitzegger, W., Rivera, J.E., Schauerhuber, A., Schwinger, W., Wimmer, M., Zhang, G.: MDWEnet: A Practical Approach to Achieving Interoperability of Model-Driven Web Engineering Methods. In: Workshop Proceedings of 7th International Conference on Web Engineering (ICWE 2007) (2007) 30. Varr´ o, D., Pataricza, A.: Generic and Meta-transformations for Model Transformation Engineering. In: Baar, T., Strohmeier, A., Moreira, A., Mellor, S.J. (eds.) UML 2004. LNCS, vol. 3273, pp. 290–304. Springer, Heidelberg (2004) 31. Varr´ o, D., Varr´ o-Gyapay, S., Ehrig, H., Prange, U., Taentzer, G.: Termination Analysis of Model Transformations by Petri Nets. In: Corradini, A., Ehrig, H., Montanari, U., Ribeiro, L., Rozenberg, G. (eds.) ICGT 2006. LNCS, vol. 4178, pp. 260–274. Springer, Heidelberg (2006) 32. Wimmer, M.: From Mining to Mapping and Roundtrip Transformations - A Systematic Approach to Model-based Tool Integration. PhD thesis, Vienna University of Technology (2008) 33. Wimmer, M., Kappel, G., Kusel, A., Retschitzegger, W., Sch¨ onb¨ ock, J., Schwinger, W.: Right or Wrong? - Verification of Model Transformations using Colored Petri Nets. In: Proceedings of the 9th OOPSLA Workshop on Domain-Specific Modeling (DSM 2009) (2009) 34. Wimmer, M., Kusel, A., Reiter, T., Retschitzegger, W., Schwinger, W., Kappel, G.: Lost in Translation? Transformation Nets to the Rescue! In: Proceedings of 8th Int. Conf. on Information Systems Technology and its Applications (2009) 35. Wimmer, M., Schauerhuber, A., Schwinger, W., Kargl, H.: On the Integration of Web Modeling Languages: Preliminary Results and Future Challenges. In: Workshop Proceedings of 7th International Conference on Web Engineering (ICWE 2007) (2007) 36. Wimmer, M., Schauerhuber, A., Strommer, M., Schwinger, W., Kappel, G.: A Semi-automatic Approach for Bridging DSLs with UML. In: Workshop Proceedings of 7th OOPSLA Workshop on Domain-Specific Modeling (DSM 2007) (2007)
Data Model for Scientific Models and Hypotheses Fabio Porto1 and Stefano Spaccapietra2 1
LNCC – National Laboratory of Scientific Computing, Petr´ opolis, Brazil
[email protected] 2 EPFL-IC – Database Laboratory, Lausanne, Switzerland
[email protected] Abstract. New instruments and techniques used in capturing scientific data are exponentially increasing the volume of data consumed by insilico research, which has been usually referred to as data deluge. Once captured, scientific data goes through a cleaning workflow before getting ready for analysis that will eventually confirm the scientist’s hypothesis. The whole process is, nevertheless, complex and takes the focus of the scientist’s attention away from his/her research and towards solving the complexity associated with managing computing products. Moreover, as the research evolves, references to previous results and workflows are needed as source of provenance data. Based on these observations, we claim that in-silico experiments must be supported by a hypotheses data model that describes the elements involved in a scientific exploration and supports hypotheses assessment. Adopting a data perspective to represent hypotheses allow high-level references to experiments and provides support for hypotheses evolution. The data model drives the proposal of a data management system that would support scientists in describing, running simulations and interpreting their results.
1
Introduction
The availability of important experimental and computational facilities nowadays allows many large-scale scientific projects to produce a never before observed amount of experimental and simulation data. This wealth of data needs to be structured and controlled in a way that readily makes sense to scientists, so that relevant knowledge may be extracted to contribute to the scientific investigation process. Current data management technologies are clearly unable to cope with scientists’ requirements [1]. In particular, traditional database technology, which has been designed for supporting business-oriented applications, is poorly suitable for scientific applications. Only a few scientific projects, such as the Sloan Digital Sky Survey, claim to successfully use relational databases [2]. More frequently, science applications require a complex data model [1] to represent meshes, sequences and other scientific abstractions, support for knowledge evolution, and close integration with Kaschek/Delcambre (Eds.): The Evolution of Conceptual Modeling, LNCS 6520, pp. 285–305, 2011. c Springer-Verlag Berlin Heidelberg 2011
286
F. Porto and S. Spaccapietra
simulators and scientific workflow engines. Globally, the list of scientific data management requirements is long and variable, but would also include support for intense world-wide collaboration, and reasoning with uncertain knowledge and uncertain models for an incremental elaboration of scientific knowledge. A promising approach to better scientific data management is based on the concept of scientific models. Building on the informal introduction to the concept given in [3], we define a scientific model as a formal representation that 1) synthesizes the understanding about a studied phenomenon, entity or process, and 2) allows scientists to eventually simulate the behaviour of the real world processes they investigate and compare the expected behaviour against experimental results. Both goals, understanding and simulating, build upon the formalization of the process at hand, e.g. as a set of mathematical formulae or as logic programs. Formalization, indeed, ensures unambiguous interpretation of what is being stated and precisely describes what the implementation must achieve in order to run experiments that will support or contradict their assumptions. The formalization stands at the core of the scientific model. The latter, however, includes much more than formal definitions. Its aim is to include all the knowledge that in one way or another is related to or has contributed to the elaboration and assessment of the formalization. This is a fundamental requisite for collaboration between different teams in the elaboration of the scientific model and for its reuse by other groups. Scientists indeed do not adopt a scientific result without examining how and by whom it has been established, what were the underlying hypotheses and supporting previous works, what were the experimental frameworks that allowed the results to be produced and assessed. It is this holistic view of scientific work that the scientific model concept intends to support. From a data management perspective, scientific models encompass all the information used and produced during a scientific exploration process. This information is contextualized by including a link to the corresponding scientific domain, bibliographic references, provenance data, and a description of the observed phenomenon that is represented by the formalization. Part of a scientific model is its computational model, which we define here as an implementation-oriented specification that translates the formalization into a computable process, including the details about the software and hardware resources. Experiments, essential to the scientific process, are run as defined by the computational model to simulate the behaviour of the studied phenomenon and its interaction with the interfacing environment, according to the specifications in the scientific model. The elaboration of the computational model is a complex but necessary task. It allows scientists to confront their simulation hypotheses with experimental results, reiterate in different experimental settings, and with possibly updated specifications reflecting changes in the hypotheses, eventually and hopefully to an accurate representation of the studied phenomenon. This retry and refine process, known as model tuning, includes both modifying programs (i.e. changing the behaviour) and fitting new input parameter and set-up values. In many cases when experimental data are available, regression analysis provides formal procedures
Data Model for Scientific Models and Hypotheses
287
for model fitting, i.e. the process of fitting the model to match with existing experimental results [4]. It is a fact that the complexity of specifying and running a computational model and managing all the resources produced during a scientific endeavour takes the scientist attention off the phenomenon being investigated onto implementation concerns [5]. State of the art approaches to computational simulation resort to scientific workflows languages, such as SCUFL [6] and BPEL [7], to specify workflows and to their corresponding running environments [8–10] for models evaluation. Tailored environments like MATLAB [11] and Neuron [12] can be used as support for small-scale, non data-intensive simulations. While some of these have attracted a large user community [13], they fail to provide an integrated scientific model environment as the one proposed in this paper, which aims at providing an integrated management of all the facets of the studied phenomenon, the derived models and data therein produced. Having said that, the scientific model management system proposed in this chapter does not substitute theses simulation tools; neither does it with respect to scientific workflow systems. Rather it may integrate them as part of an environment to support the scientists. This work intends to support scientists in specifying, running, analysing and sharing scientific models and data produced by running simulations. Adopting a data-oriented perspective for scientific model representation, we provide formal definitions for the scientific model concept and its components, as informally proposed in [3]. Simulations are expressed using a high-level query language enabling runtime optimizations. The simulation therein specified may refer to software implemented in traditional programming languages as C or Fortran, or using a simulation tool. The data model is grounded in XML and tightly related to domain ontologies that provide formal domain descriptions and uniform terminology. Our intention is to use the proposed model as the basis for a computational environment designed to help scientists manage the resources associated to insilico scientific exploration. Using such an environment, scientists may search for existing scientific models, specify and run simulations. The results therein produced generate complex data to be analyzed and metadata to base queries. A by-product of managing the metadata and data is the provenance information [14] gathered during the research exploration. Initially, a framework for modeling and simulating single scientific models is discussed. Next, the basic framework is extended in two directions. In one direction, support for complex scientific models is added. A complex scientific model integrates other scientific models (in full or in part) in its definition, leading to a compositional network of scientific models. Higher-level models can then be specified, e.g. through views, that export a unified representation of underlying scientific models. A second direction concerns the extension to support scientific hypothesis and its integration with scientific models. Finally, we sketch a software architecture for the computational environment to be built based on the proposed to which we coin the name of scientific model management system.
288
F. Porto and S. Spaccapietra
The remainder of this chapter is structured as follows. Section 2 discusses related work. Section 3 presents a running example extracted from the neuroscience domain to be explored throughout the chapter. Section 4 covers scientific model-driven modeling and Section 5 details the simulation language. Section 6 extends scientific models with hypotheses and Section 7 discusses some advanced model elements. Finally, Section 7 introduces a general architecture for a scientific model management system. The conclusion discusses our achievements so far and future work.
2
Related Work
Data and knowledge management supporting in-silico scientific research is a comprehensive topic that has appeared under the eScience or computational science label. It encompasses the semantic description of the scientific domain, the experiment evaluation through scientific workflow systems, and results analysis through a myriad of different techniques, among other in-silico related tasks. Given the broad class of application domains that may benefit from eScience related data management techniques, it has been postulated that there is little chance that a single solution would cover the diverse set of requirements coming from these domains [1]. The semantic description of scientific domains through ontologies [15] is one exception that has attracted the attention of the scientific community as a means to support collaboration through common conceptual agreement. In this line, GeneOntology [16] is probably the most well-known and successful example of practical adoption of ontologies in the scientific domain. Similarly, scientific workflows have become the de facto standard for expressing and running in-silico experiments, using execution environments, such as [8–10]. The problem of systematically capturing and managing provenance information for in-silico experiments has also attracted significant attention. According to [14], provenance information helps interpreting and understating results by examining the sequence of steps that led to a result. The analysis of provenance data helps with verifying that experiments were conducted according to acceptable procedures and eventually supports reproducibility of the experiment. Provenance information has been classified into two categories: prospective and retrospective [17]. The former refers to description of the computational tasks involved in the experiment, whereas the latter refers to all the information used and produced during the evaluation of the experiment. Some scientific workflow systems give support for gathering and publishing both prospective and retrospective provenance information, such as Vistrails [18] and REDUX [19]. The combination of prospective and retrospective information potentially supports the whole exploratory in-silico experiment with data and metadata. Querying provenance data retrieves experiment results and inform about experiment metadata. Conceptually, our work introduces hypotheses definition and its integration with experiments into prospective provenance. We believe that hypotheses enrich provenance information both qualitatively and quantitatively. Integrating scientific hypotheses in a data model is, however, not trivial.
Data Model for Scientific Models and Hypotheses
289
Hypotheses modeling was introduced in databases back in the 80’s [20]. In that context, one envisions a hypothetical database state, produced by delete and insert operations, and verifies whether queries are satisfied on that hypothetical state. This approach is, however, far from the semantics needed in the experimental settings that we are interested in. Closer to our objective is the logical model proposed in the context of the HyBow project [21, 22] for modeling hypotheses in the biology domain. Hypotheses (H) are represented as a set of first-order predicate calculus sentences with free quantifiers. In conjunction with an axiom set specified as rules that models known biological facts over the same universe, and experimental data, the knowledge base may contradict or validate some of the sentences in H, leaving the remaining ones as candidates for new discovery. As more experimental data is obtained and rules identified, discoveries become positive facts or are contradicted. In the case of contradictions, the rules that caused the problems must be identified and eliminated from the theory formed by H. The approach adopted by Hybrow supports hypotheses validation in the spirit of what we aim to represent, i.e. as a formal definition to be confronted with experimental results. According to what has been discussed above concerning modeling of hypotheses in a scientific framework, the Hybrow model does not offer any support for simulations. Critical is the observation that the adopted modeltheoretical approach for hypotheses validation is not sufficient for representing simulation based research. This is due to the fact that in a model-theoretical approach, the validation of hypotheses only considers the satisfiability of the logical implications defined in the model with respect to an interpretation. This is fine for qualitative assessment of hypotheses but it is insufficient for simulationbased research, in which validation is decided based on the quantitative analysis between the simulation results and the observations. In this context, our work aims at integrating hypotheses with the simulation environment bridging the gap between qualitative and quantitative representation.
3
A Neuroscience Scientific Model
In this section, we introduce a running example taken from scientific models developed for the neuroscience domain. According to Kandel [23], “the task of neural science is to explain behavior in terms of the activities of the brain”. Numerous computational neuroscience groups investigate scientific models that aim at explaining some specific behavior. A classical example from these scientific models is the “axon membrane action potential”, a model proposed by Hodgkin and Huxley [24] that describes how action potentials traverse the cell membrane. An action potential is a pulse-like wave of voltage that can traverse certain types of cell membranes, such as the membrane of the axon of a neuron. The model quantitatively computes the potential between the interior and the extracellular liquid, based on the flow of ions of sodium (N a+ ), potassium (K + ) and a leakage flow representing all other channel types. The formalization in the model is given by the following mathematical equation:
290
F. Porto and S. Spaccapietra
I = m3 h gN a(E − EN a ) + n4 gk (E − Ek ) + gL (E − EL )
(1)
In (1), gi,i={N a,K,L} is a time-dependent variable that represents the membrane conductance for sodium, potassium and leakage; Ei models the equilibrium potential for each ion channel; E is the membrane potential; and, n, m and h are parameters controlling the probability of the sodium or potassium gates to be opened. The total ionic current across the membrane of the cell is modeled by the variable I.
Fig. 1. Single Neuron Model using the Hodgkin&Huxley model domain ontology
This action potential model simulates the variation of voltage in the cell when applied to a neuron compartment. It has been used, for example, in the formulation of the scientific model of a single neuron. Fig. 1 shows the domain ontology that supports the conceptual representation of a single neuron with its various compartments and the membrane behaviour given by the Hodgkin&Huxley (HH) model. The Hodgkin&Huxley class in the ontology represents the corresponding model with the input and output parameters conforming to equation (1). From the scientific model specification, a computational neuroscientist will conceive programs that implement the behaviour defined by equation (1). Next, when running a simulation, the program receives input values for the parameters identified in equation (1) and produces the total ionic current across the membrane (variable I).
4
Data Models for Scientific Models
During in-silico scientific investigation, scientists explore, produce and register data and metadata describing a phenomenon and associated simulations. Different scientific products are generated and need to be managed. Scientific model metadata covers provenance as well as contextual and descriptive information that drive scientific model searching and querying. Similarly, computational model metadata are used as the basis for the automatic instantiation and evaluation of simulations and serve as context to qualify input and output data. To structure this wealth of information, we propose a data model for scientific models data and metadata that is presented in this section.
Data Model for Scientific Models and Hypotheses
4.1
291
The Observed Phenomenon
The starting point of an in-silico scientific investigation is the clear specification of the phenomenon one attempts to explain. The formal description of a phenomenon includes a domain ontology, setting the formal conceptual representation of the domain in which the phenomenon is inserted, a phenomenon title, and a informal textual description. A Phenomenon is represented in the data model as in equation (2), where P hid is the phenomenon identification and Oph refers to the domain ontology URL: P h =< P hid , Oph , T itle > 4.2
(2)
Scientific Model
A scientific model provides a comprehensive description of the scientists’ interpretation of the observed phenomenon. Of prime importance is a formal representation of the scientific phenomenon interpretation, possibly using mathematical formulae, and a reference to the phenomenon it attempts to explain. Once a scientific model is inserted into the same semantic domain as the phenomenon it refers to, the Phenomenon ontology covers its domain description. In addition, the scientific model (SM) description includes bibliography references and other metadata supporting model presentation. Formally, a scientific model is expressed as a sextuplet, as presented in (3). In equation (3), SMid is the scientific model identification and P Hid denotes the phenomenon identifier. The URL in OMF references the ontology describing the scientific model formalization. Finally, a list of bibliographic references, images and annotations are pointed by B, I, and A, respectively. SM =< SMd , P hid , OMF , B, I, A >
(3)
The Hodgkin&Huxley model, presented in Section 3, can be depicted as a scientific model. Its data view is illustrated in Fig. 2. It provides metadata to support basic search queries over a scientific model database and reasoning capabilities on the theory specified by the scientific model ontology. 4.3
Computational Model
The definition of the scientific model suggests an explanation for a phenomenon using some formal language. In order to run in-silico experiments, a scientist develops a computational representation of the scientific model. Although desirable, an automatic mapping from a formal description of a SM to its computational model is still not feasible, requiring engineering efforts from the scientific group. Nevertheless, once a computation model has been specified and built, an engine may read such specification and automatically instantiate an execution if input data is provided. The computational model (CM) description according to the proposed data model identifies required metadata for such automatic instantiation.
292
F. Porto and S. Spaccapietra
Fig. 2. The Hodgkin&Huxley scientific model representation
In this context, the Environmental and Domain ontologies contribute to disambiguate CM specifications. The Environmental ontology describes the execution environment associated to the CM, including: programming language specification, programming environmental parameters, libraries, documentation, input and output parameters, initialization procedure, and executing programs. Computational model input/output parameters are structured into two formats: XML and attribute-value pairs. Parameters that are read or produced by the simulation are structured into a XML document, whereas set-up values are expressed as attribute value pairs. In order to be able to capture output produced by the underlying program, an outputWrapper class must be provided that transforms the output into a set of attribute value pairs mapped into a XML structure. A CM is formally defined as a 7-tuplet CM =< CMid , SMid , XOE , XOP h , Mi , Mo , A >
(4)
In (4), CMid is the CM resource identification; SMid refers to the associated scientific model, XOE and XOP h are the XML serializations of the environmental ontology, and of the Phenomenon domain ontology, respectively1 . Mi and Mo are the mappings between the underlying program input and output parameters and the corresponding domain ontology properties (XML tree leave nodes). Finally, A corresponds to annotations identifying authoring information. Fig. 3 illustrates the representation of a computational model implementing the scientific model SM01 from Fig. 2. 4.4
Simulation
The previous elements of the model present metadata used in scientific model formalization and querying; they also support automatic instantiation of the 1
The serialization of ontologies into an XML structure follows a detailed technique not presented here. The main intuition is to form a tree structure having concepts as nodes and edges represent associations between concepts.
Data Model for Scientific Models and Hypotheses
293
Fig. 3. A Hodgkin-Huxley Computational Model
computational model. We now turn to the expression of simulations. Simulations are in-silico experiments run to assess the scientific model against the observed phenomenon. By analogy with databases, where users’ data is intentionally expressed in queries, we call the specification of a simulation a simulation query. Let us define a simulation database DB S = {VCM 1 , VCM 2 , ..., VCM m }, where VCM i , 1 ≤ i ≤ m, are n-ary data views on the computational model. Given a computational model CMi , a corresponding n-ary data view VCM i abstracts the software program behavior associated to the CM, by exposing its input and output parameters as data attributes and completely hiding its implementing programs. This is similar to modeling user-defined functions as relations in databases [25]. Consider, for instance, the data view below corresponding to the CM in Fig. 3: HodgkinHuxley(i : (m, n, h, gN a , gK , gL , EN a , EK , EL ), o : (I))
(5)
The HodgkinHuxley data view presents one attribute for each input/output CM parameter. Querying a data view requires binding attributes in the input parameter set (prefixed with i:) to input values and retrieving results associated to output parameters (prefixed with o:). In this context, a simulation query (S) interrogates a data view VCM i by providing binding values and obtaining results from output attributes. Section 5 formally presents the simulation query language.
5
Simulation Language
In-silico experiments are commonly expressed using workflow or some sort of scripting languages. We aim to leverage the expression of simulations by providing a high-level query language with which scientists may express a large class of workflows, notably those that can be modeled as directed acyclic graphs.
294
F. Porto and S. Spaccapietra
In this context, a simulation query is specified as an expression in nonrecursive Datalog [26] comprising a head and a body. The body is a boolean expression composed of a conjunction of predicates, whereas the head specifies a predicate holding variables containing the expected simulation results, necessarily appearing in one of the predicates in the body. Users interface with simulation queries by providing the input parameters and set-up values needed for the evaluation of the predicates, and getting in return the output values bound to the variables defined in the head. We start with presenting the syntax and semantics of simulation query predicates. 5.1
Simulation Predicate
A simulation query predicate is specified as: Si ((Vi , Wi ); (Xi , Xo ); (Ii , Oi ), IS )
(6)
In (6), Si labels the simulation query predicate and associates it to the corresponding VCM resource identification. Vi and Wi are the two sets of variables defined to refer to values provided as input or produced as output when running the underlying CM program. The set of input and output parameter’s values are provided by the XML documents Xi and Xo , respectively. Note that the associated CM definition specifies the schemas for Xi and Xo . For example, using the CM in Fig. 3, the Xi document can be obtained from the result of the XPath expression “/CM/DomainOntology” over the Hodgkin&Huxley CM element XOP h , and by filling its leaf nodes with the input values. Thus, /Neuron/Axon/Hogking-Huxley/m = 0.1 illustrates a possible value assignment for the input parameter m. Ii and Oi are the mappings defining the correspondence (see Definition 1) between the input and output variables in Vi and Wi and the input and output parameter values in Xi and Xo . Finally, IS represents simulation set-up parameters. Definition 1. Correspondence assertions in Ii and Oi are specified as $x = Path, where $x is a variable in {Vi ∪ Wi } and Path is an XPath [27] expression pointing to a data element in Xk , k={i,o}, whose leaf node is either an input parameter value or an output value. Having described the syntax for individual simulation predicates, we can proceed to define the semantics of body expressions. We start with defining the semantics of a single simulation predicate. The definition for complete body expressions follows. 5.2
Semantics of a Single Simulation Predicate
A single simulation predicate returns a boolean value of its evaluation according to the definition in Definition 2 below with respect to its syntax in (6) and the CM specification in (4).
Data Model for Scientific Models and Hypotheses
295
Definition 2. A simulation predicate Si evaluates to true iff given an Xi holding the set of parameter values input to the program implementing the corresponding CM, according to Mi , there exists a Xo whose leaf values are produced by the evaluation of the referenced program and that is built from the mappings in Mo . 5.3
Semantics of the Body of a Simulation Expression
More elaborate conjunctive expressions can be composed from single simulation predicates to form the body of a simulation. The semantics of a conjunction of simulation predicates in the body of a simulation is defined in Definition 3. Definition 3. A conjunction of simulation predicates s = s1 ∧ s2 ∧ ... ∧ sn , s is considered to hold true if the conjunctive expression on the right evaluates to true. Moreover, if more than one simulation predicates si and sj in s refer to the same variable, for instance $x, then they share a single associated value. In addition, the shared variable must hold a single binding to a value, either provided as input or produced as output by an underlying program computation. Note that the restriction regarding sharing variables among simulation predicates leads to data-dependency relationships, in which the simulation predicate holding the value associated to the shared variable shall precede in evaluation order the remaining simulation predicates sharing that particular variable. Moreover, variable sharing introduces a particular mode of value assignment to data elements in Xi , which replaces that of the node corresponding to its associated path. Finally, given a body that evaluates to true, then the head of the simulation identifies the variables in the query body whose values are returned as the simulation results, such that if K is the set of variables in the head, then K ⊆ (Vi ∪Wi ), for 1 ≤ i ≤ n, with n being the number of simulation predicates in the body. 5.4
Simulation Query
A simulation query combines the head and its body into a clause as illustrated in (7), according to definitions 1, 2 and 3. S(K) := S1 ((V1 , W1 ); (Xi1 , Xo1); (I1 , O1 ), IS1 )∧ S2 ((V2 , W2 ); (Xi2 , Xo2); (I2 , O1 ), IS2 )∧ ...∧ Sn ((Vn , Wn ); (Xin , Xon ); (In , On ), ISn )
(7)
An example of a simulation query is given in (8). This particular query returns the total ionic current across the membrane ($I) according to the parameter values specified in the input document HHCM 01I . As discussed before, the user must provide a mapping from each query variable to the corresponding data element of the domain ontology XML serialization document. In this example, the input and output XML documents, Xi and Xo , are illustrated by documents HHCM 01I and HHCM 01O , respectively, both of type Neuron.
296
F. Porto and S. Spaccapietra
S($I, $z) := CM01 (i : ($m, $h, $n, $gN a, $gK , $gL, $EN a , $EK , $EL ), o : ($I)); (HHCM 01I , HHCM 01O ); ($m = /N euron/Axon/Hodgkin − Huxley/m, ....2 , $I = /N euron/Axon/Hodgkin − Huxley/I)) ∧ CM023(($I, $z); (ACM 02I , ACM 02O ); ($z = /Analysis/result))
6
(8)
Hypothesis Modeling
Up to now, we have focused our discussion on scientific models and the entities that derive from them during a research process. An important missing aspect from a conceptual point of view is the expression of scientific hypotheses, which drives research by proposing an explanation for a studied phenomenon. Indeed, according to wikipedia [28], a scientific hypothesis is used as a tentative explanation of an observation, but which has not yet been fully tested by the prediction validation process for a scientific theory. A hypothesis is used in the scientific method to predict the results of further experiments, which will be used either to confirm or disprove it. A successfully-tested hypothesis achieves the status of a scientific theory. The explanation a scientific hypothesis conveys a relationship between the causal phenomena and the simulated one, namely, that the simulated phenomenon is caused by or produced under the conditions set by the causal phenomena. Thus, by running the simulations defined by the antecedents in the causal relationship, the scientist aims at creating hypothetical simulation of the studied phenomenon. In this context, the scientific hypothesis is an element of the scientific model data model that may replace a phenomenon. When computing a simulation based on a scientific hypothesis, i.e. according to the causal relationship it establishes, the output results may be compared against phenomenon observations to assess the quality of the hypothesis. The scientific hypotheses therein introduce a valuable contribution by bridging the gap between qualitative description of the phenomenon domain and the corresponding quantitative valuation obtained through simulations. Thus, in our modeling approach we aim at coming up with a representation of scientific hypotheses that may be used in qualitative ( i.e., ontological) assertions and with minimum tricks can be quantitatively confronted to phenomenon observations. Having said that, bridging this gap is a challenging job that we are starting to tackle and whose first results we report in this section. 6.1
Running Example
Let us consider a small variation on the scenario presented in Section 3. Suppose we want to feed a scientific visualization application with the temporal variation 2 3
The remaining mappings are not shown due to lack of space. The remaining mappings are not shown due to lack of space.
Data Model for Scientific Models and Hypotheses
297
on the value of the ionic current (I), assuming in other words, the ionic current is a function of time. The result is a time series showing the variation of the ionic current during an interval of time Δt. In addition, we will assume that independent scientific models are conceived to model the ionic current on each gate (i.e. sodium, potassium and leakage). In this revised scenario, the formulae in (1) can be re-written as: I = (1−d) m3 h(E − EN a )gN a (t)dt + (1−d) n4 (E − EK )gK (t)dt+ (9) (E − EL )gL (t)dt (1−d) In (9), d is the duration of the simulation and the membrane conductance gi is a function of the simulation time instant. The ionic current on each gate is modeled by a different scientific model, leading to the following computational models: ionicChannelN a, ionicChannelK and ionicChannelL . Given this new scenario, a scientist may formulate the following hypothesis concerning HodkingHuxley model: The total ionic current on a membrane compartment is a time function of the ionic current on the sodium, potassium and leakage channels. Under the scenario exposed above, the following entities have been identified: – phenomena: 1. total ionic current on a neural membrane compartment; 2. time dependent ionic current on a sodium ionic channel; 3. time dependent ionic current on a potassium ionic channel; 4. time dependent ionic current on a leakage ionic channel. – computational models: 1. ionicChannelN a 2. ionicChannelK 3. ionicChannelL – scientific hypothesis: The total ionic current on a membrane compartment is a time function of the ionic current on the sodium, potassium and leakage channels. 6.2
Hypotheses Model
In order to integrate scientific hypothesis with the scientific model data model, we formally define a Hypotheses Data Model (HDM). A HDM describes an experiment domain and is defined as HDM = {Ph , PhO , H , E , SM , CM , VCM , SQ , V }, where: – – – – – – – – –
Ph - is a set of phenomena, as defined in (1); PhO - is a set of phenomenon observations, as defined in Definition 4; H - is a set of hypothesis, as defined in Definition 5; E - experiments, as described in Definition 6; SM - is a set of scientific models, as defined in (3); CM - is a set of computational models, as defined in (4); VCM - computational model data views, as described in Section 7; SQ - is a simulation querie, is defined in (7); V - scientific model views.
298
F. Porto and S. Spaccapietra
Elements of HDM are n-ary relations with attributes defined in a set of concrete domains D, such as integers, strings, floats, etc. Definition 4. A phenomenon observation (PhO) is a temporal record of a phenomenon, quantitatively described by its attribute values. The specification in (1) for phenomenon observations is extended as: ph i (obid , date, V , U , A ), Ph i ∈ Ph . In PhO , ph i is a phenomenon observation set label, obid is an observation identifier, V =< V 1 , V 2 , ..., V k >, U =< a1 , a2 , ..., al >, with ai ,V j ∈ D m , D n , respectively, D m , D n ⊆ D , for all 1 ≤ i ≤ k, 1 ≤ j ≤ l. V represents a list of initial set-up values and U a list of phenomenon attributes. Finally, A is a list of annotations. Thus, a phenomenon observation models the observable entity in different states and time instants. The observed variation is recorded as PhO attribute values. In the example of Section 6.1, a scientist may register the following observations concerning totalIonicCurrent: totalIonicCurrent(ob1 , date, < m1 , n1 , h1 >, 0.001); totalIonicCurrent(ob2 , date, < m2 , n2 , h2 >, 0.002);
(10)
In the representations above (10), two instances of the phenomenon totalIonicCurrent are depicted. The observations include an identifier, the time instant of the observation, a set of initial state values, specifying the context on which the phenomenon was observed, and an attribute that quantitatively describes it. The latter serves as the basis for assessing hypotheses. Syntactically, we distinguish initial set-up values from observable attributes with an underline on the latter. Thus, in light of the example in Figure 6.1, we would have the following phenomena observation schemas involved in computing the ionic current in a membrane compartment: totalIonicCurrent (obid , date, IN A , IK , IL , totalIonic); ionicChannelN a (obid , date, t, m, h, IN A ); ionicChannelK (obid , date, t, n, IK ); ionicChannelL (obid , date, t, IL ). In the above schema, IN A , IK , IL represent the ionic current on the three gates of our running example. Phenomenon observations are the basis for the modeling activity. A scientist formulates hypotheses that may or may not be validated when compared to observations. Definition 5. A scientific hypothesis is the specification of a possible explanation for a given phenomenon. In this context, a scientific hypothesis is specified as H =< id, title, Ph id , F , SQ id >
(11)
Data Model for Scientific Models and Hypotheses
299
In (11), id refers to the hypothesis identifier; Ph id is the corresponsing phenomenon identifier; F is a comparison function, used in measuring the accuracy of hypotheses with respect to observations; and SQ id is an identifier for the simulation query that sets the causal relationship among the causing phenomena and the simulated phenomenon. Referring to our running example, the total ionic current phenomenon hypothesis schema would be defined as: totalIonicCurrent (hid , P htotalIonicCurrent , fi , SQtic );
Fig. 4. A scientific hypothesis
Observe that by providing the simulation query SQ tic in a hypothesis declaration, we are indicating that hypotheses are associated to a computational representation, expressed by the query pointed to by SQ id , and whose evaluation simulates the phenomenon identified by Ph id . Definition 6. An experiment (E) corresponds to the instantiation of a simulation, including the results obtained by running the simulation query associated with the hypothesis bound to input-values, and forms the basis for quantitative hypothesis validation. Experiments are formally specified as: ei (hj , date, pj , c, q, dist)
(12)
In (12), ei ∈ E ; hj is an experience identifier; pj = p1 , p2 , ..., pn , pk ∈ D k , D k ⊆ D , for all 1 ≤ k ≤ n, is the initial condition’s value set; c = (c1 , c2 , ..., cm ), cl ∈ D l ,D l ⊆ D for all 1 ≤ l ≤ m, is the set of attribute values used in validation against phenomenon observations; q is a set of phenomenon observations, which define the observations comparison set; and dist is a measure of distance, computed by hypothesis’ comparison function f between the experiment results and the explained phenomenon observations phj . Continuing with our running example, we have: Experiment: totalIonicCurrent (hi1 , < m1 , n1 , h1 >, 0.003, < ob1 , ob2 >, dist1 );
300
F. Porto and S. Spaccapietra
Simulation Query: totalIonicCurrent($totalIonic)= totalIonicCurrent ($IN A , $IK , $IL , $totalIonic) ∧ ionicChannelN a(t1 , m1 , h1 , $IN A )∧ionicChannelK (t1 , n1 , $IK )∧ ionicChannelL (t1 , $IL ) An experiment is planned to hold the same initial condition values as the phenomenon observation it attempts to reproduce. The results obtained by evaluating the computational models representing the causal phenomena are compared against phenomenon observation attribute values through hypotheses comparison function. We motivated the need for a scientific data model so as to permit knowledge evolution in sync with the investigation progress. Hypotheses foster evolution by allowing competing hypotheses to be individually specified and assessed against phenomenon observations. Similarly, by modifying a computational model, a new hypothesis may be specified. Finally, each instance of a hypothesis instance set represents a new scientific essay with a different input value set. Thus, hypotheses, experiments and competing hypotheses are three different dimensions through which scientists can register the evolution of the investigation while keeping provenance information about the progress of the research. The proposed model distinguishes three aspects of a scientific investigation: phenomenon experiment, simulation and formal representation. Table 1 presents a synthesis of the hypothesis data model classified according to these groups. Table 1. Main elements to be considered in the HDM Phenomenon
Phenomenon (Ph)
Simulation
Hypothesis (H) Scientific Model (SM)
Formal representaion
7
Phenomenon observation (PhO) Experiment (E)
Computational Model (CM)
Data View (VCM)
Simulation query (SQ)
Complex Scientific Models
In this section we briefly introduce complex scientific models representation to support the composition of computation models in the same vein as database views [29]. For this reason, we call the construct a simulation view (V). It allows simulation queries to be stored so that they can be re-executed later on or included in a more complex simulation. V provides users with an external perspective of a simulation through the set of input parameter values that configure the participants computational models. In addition, a simulation view establishes correspondences between the exported parameters and the ones specified on each simulation predicate taking part in the body of the simulation describing the view. A simulation view is expressed as follows:
Data Model for Scientific Models and Hypotheses Sv ((V, W ); (Xiv , Xov ); (Iv , Ov ); (ISv , Ms )) = S1 ((V1 , W1 ); (Xi1 , Xo1 ); (I1 , O1 ); IS1 )∧ S2 ((V2 , W2 ); (Xi2 , Xo2 ); (I2 , O2 ); IS2 )∧ ...∧ Sn ((Vn , Wn ); (Xin , Xon ); (In , On ); ISn )
301
(13)
The body of the simulation view is like the one in ordinary simulations, expressing a conjunction of simulation predicates. The difference appears in the head of the formula. Indeed, the latter exports an integrated view of the simulation predicates’ input and output parameters that appear in the body of the formula and that are specified in Xik and Xok , 1 ≤ k ≤ n. The two sets of correspondences, Ik and Ok , map the external view in Xiv , Xov to the corresponding parameters in the simulation predicates in the body, Xik , Xok . Thus, a correspondence assertion is expressed as Sv .path/dataelement ≡ Si .path/dataelement, where path is an XPath expression. In the same line as the input/output parameters, ISv expresses the uniform view of set-up parameter values appearing in the body of the formulae and Ms asserts the correspondences between the set-up data elements in ISv and those in the body.
8
Managing Scientific Models
A scientific model management system (SMMS) supports scientists in designing, creating and searching scientific model’s entities, and in managing the results of simulations and analyses. Fig. 5 depicts the main system functions structured into four layers. A user layer provides the interface for scientists to create and edit elements of the model and to request system services, such as running simulations, querying and reasoning. Users may query scientific model metadata as well as hypotheses data and simulation results. The metadata management layer stores scientific model metadata and supports metadata management services. In this work, scientific model metadata is based on a set of ontologies that guarantees uniform terminology among models and scientists. A transformation and selection service allows scientists to map ontology fragments to XML trees, which are then used in data model elements description. The catalog service manages metadata about scientific model and data, as well as supporting information such as ontologies, views (see section 5.5) and transformation rules. The service layer supports simulation evaluation, querying of simulation results and reasoning. We have extended the query processing system QEF [10] to cope with simulation queries evaluation. Finally, a data management layer supports distributed scientific models management and wrappers implementing complex data types, offering access to simulation results, such as graphs and time series. In this paper, the details of the architecture are not further explored. Similarly, the details regarding ontology management, transformation and alignment are left to future work.
302
F. Porto and S. Spaccapietra
Fig. 5. Scientific Model Management System Architecture
9
Conclusion
Managing in-silico simulations has become a major challenge for eScience applications. As science increasingly depends on computational resources to aid solving extremely complex questions, it becomes paramount to offer scientists mechanisms to manage the wealth of knowledge produced during a scientific endeavor. This chapter presented initial results aiming to contribute to this idea. We propose a data-centric semantic based data model with which scientists represent scientific hypotheses, scientific models and associated computational models. Scientific hypotheses are explanations of observable phenomena expressed through the results of computer simulations, which can be compared against phenomena observations. The model allows scientists to record the existing knowledge about an observable investigated phenomenon, including a formal mathematical interpretation of it, if one exists. Additionally, it intends to serve as the basis for the formal management of the scientific exploration products, as well as supporting models evolution and model sharing. Traditionally, scientific computation takes either a mathematical or computational view when modeling is discussed. Scientific workflows are the most practical example of its computational view. By taking a declarative perspective on scientific model we envisage various benefits. Firstly, a higher-level declarative language allows scientists to concentrate on the scientific questions he/she tries to answers saving precious time that otherwise would be expended on workflow definition and evaluation. Secondly, hypotheses are good candidates to bridge the gap between an ontological description of studied phenomena and the simulations that aim at explaining them. Finally, data views on scientific entities allow for querying and searching for models supporting scientific models sharing among different scientific groups.
Data Model for Scientific Models and Hypotheses
303
There are various opportunities for future work. In this chapter we havent introduced the ontological modeling for hypotheses. Different relationships among hypotheses may be expressed, such as: composition, similarity, use, etc. Investigating how to express these and other relationships among hypotheses and the phenomena, they attempt to explain, is an interesting topic. Furthermore, hypotheses evolve and eventually become a theory. It would be interesting to model and infer on such evolution. Taking into account hypotheses validation, the distance between a hypothesis instance and the referred phenomenon may be probabilistically interpreted. Thus, considering probabilistic inference models when running hypotheses formula needs to be investigated. Another path of research explores the use of provenance information to help on the investigation evolution. Considering data management on the results of simulations, there is a huge path to follow. Computational models usually deal with complex data types, such as meshes, grids, temporal series and etc. Integrating these data types into our model is future work. Besides, all the research work involving heterogeneous data are of relevance to allow the automatic communication between computational models involved in a simulation. We developed a first prototype system that implements the data model and the simulation query language on top of the QEF system. The system was designed in the context of scientific model management system architecture with a set of minimal services that scientists may expect from such an environment.
References 1. Stonebreaker, M., Becla, J., DeWitt, D., et al.: Requirements for Science Data Base and SciDB. In: Conference on Innovative Data Systems Research. CIDR, Asilomar, CA, USA (January 2009) 2. Szalay, A., Kunszt, P., Thakar, A., Gray, J., et al.: Designing and Mining MultiTerabyte Astronomy Archives: The Sloan Digital Sky Survey. In: ACM SIGMOD, Dallas, Tx, USA, pp. 451–462 (2000) 3. Hunter, J.: Scientific Models A User-oriented Approach to the Integration of Scientific Data and Digital Libraries. In: VALA 2006, Melbourne (February 2006) 4. Jaqaman, K., Danuser, G.: Linking data to models: data regression, Nature Reviews. Molecular Cell Biology (7), 813–819 (2006) 5. Silvert, W.: Modelling as a discipline. International Journal General Systems 30(3), 1–22 (2000) 6. http://www.gridworkflow.org/snips/gridworkflow/space/XScufl 7. Akram, A., Meredith, D., Allan, R.: Evaluation of BPEL for Scientific Workflows, Cluster Computing and the Grid. In: CCGRID, vol. 1, pp. 269–274 (May 2006) 8. Oinn, T., Greenwood, M., Addis, M.: Taverna: Lessons in creating a workflow environment for the life sciences. Concurrence and Computation: Practice & Experience 18(10), 1067–1100 (2006) 9. Altintas, I., Berkley, C., Jaeger, E., Jones, M., Ludascher, B., Kepler, M.: An Extensible System for Design and Execution of Scientific Workflows. In: Proceedings of the 16th Conference on Scientific and Statistics Database Management, pp. 423–424 (2004)
304
F. Porto and S. Spaccapietra
10. Porto, F., Tajmouati, O., Silva, V.F.V., Schulze, B., Ayres, F.M.: QEF Supporting Complex Query Applications. In: 7th Int. Symposium on Cluster Computing and the Grid, Rio de Janeiro, Brazil, pp. 846–851 (2007) 11. MATLAB, http://en.wikipedia.org/wiki/Matlab (last access 24/06/2008) 12. Neuron, http://www.neuron.yale.edu (last access 24/06/2008) 13. Roure, D., Goble, C., Stevens, R.: Designing the myExperiment Virtual Research Environment for the Social Sharing of Workflows. In: e-Science 2007 - Third IEEE Int. Conf. on e-Science and Grid Computing, Bangalore, India, December 10-13, pp. 603–610 (2007) 14. Freire, J., Koop, D., Santos, E., Silva, C.T.: Provenance for Computational Tasks: A Survey. Computing in Science and Engineering 10 I(3), 11–21 (2008) 15. Gruber, T.R.: Toward Principles for the Design of Ontologies Used for Knowledge Sharing. International Journal of Human-Computer Studies (1995) 16. http://www.geneontology.org/ 17. Clifford, B., et al.: Tracking Provenance in a Virtual Data Grid. Concurrency and Computation: Practice and Experience 20(5), 565–575 (2008) 18. Callahan, S.P., Freire, J., Santos, E., Scheidegger, C.E., Silva, C.T., Vo, H.T.: Vistrails, Visualization meets Data Management. In: Proceedings of ACM SIGMOD, Chicago, Illinois, June 26-29, pp. 745–747 (2006) 19. Barga, R.S., Digiampietri, L.A.: Automatic Capture and Efficient Storage of eScience Experiment Provenance. Concurrency and Computation: Practice and Experience 20(5), 419–429 (2008) 20. Bonner, A.J.: Hypothetical Datalog: Complexity and Expressibility. Theoretical Computer Science 76, 3–51 (1990) 21. Racunas, S.A., Shah, N.H., Albert, I., Fedoroff, N.V.: Hybrow: a prototype system for computer-aided hypothesis evaluation. Bioinformatics 20, suppl. 1, 257–264 (2004) 22. Racunas, S., Griffin, C., Shah, N.: A Finite Model Theory for Biological Hypotheses. In: Proc. of the 2004 IEEE Computational Sytems Bioinformatics Conferences (2004) 23. Kandel, E., Schwarts, J., Jessel, T.: Principles of NeuroScience, 4th edn. McGrawHill, New York (2000) 24. Hodgkin, A., Huxley, A.: A quantitative description of ion currents and its applications to conduction and excitation in nerve membranes. Journal of Physiology 117, 500–544 (1952) 25. Chauduri, S., Shim, K.: Query Optimization in the presence of Foreign Functions. In: Proc. of the 19th Very Large Database Conference, Dublin, Ireland, pp. 529–542 (1993) 26. Ullman, J.: Principles of Database and Knowledge-Base Systems, vol. 1. Computer Science Press, Rockville (1988) 27. http://www.w3.org/TR/xpath (last accessed 26/04/2008) 28. http://en.wikipedia.org/wiki/Scientific_hypothesis (last access 04/09/2009) 29. Elmasri, R., Navathe, S.: Fundamentals of Database Systems, 2nd edn. Benjamin/Cummings (1994) 30. http://lsids.sourceforge.net/ (last accessed 26/04/2008) 31. Grosof, B., Horrocks, I., Volz, R., Decker, S.: Description Logic Programs: Combining Logic Programs with Description Logic. In: Proc. WWW 2003, Budapest (May 2003)
Data Model for Scientific Models and Hypotheses
305
32. Christiansen, H., Andreasen, T.: A Practical Approach to Hypothetical Database Queries. In: Kifer, M., Voronkov, A., Freitag, B., Decker, H. (eds.) Dagstuhl Seminar 1997, DYNAMICS 1997, and ILPS-WS 1997. LNCS, vol. 1472, pp. 340–355. Springer, Heidelberg (1998) 33. Davidson, S., Freire, J.: Provenance and Scientific Workflows: Challenges and Opportunities. In: Proc. 2008 ACM SIGMOD Int’l. Conf. on Management of Data, Vancouver, CA, pp. 1345–1350 (2008)
Model-Based Engineering Applied to the Interpretation of the Human Genome Oscar Pastor, Ana M. Levin, Matilde Celma, Juan Carlos Casamayor, Aremy Virrueta, and Luis E. Eraso Research Center on Software Production Methods (ProS). Universidad Politecnica de Valencia. Camino Vera S/N. 46007, Valencia, Spain
[email protected] Abstract. In modern software engineering it is widely accepted that the use of Conceptual Modeling techniques provides an accurate description of the problem domain. Applying these techniques before developing their associated software representation (implementations) allows for the development of high quality software systems. The application of these ideas to new, challenging domains –as the one provided by the modern Genomics- is a fascinating task. In particular, this chapter shows how the complexity of human genome interpretation can be faced from a pure conceptual modeling perspective to describe and understand it more clearly and precisely. With that, we pretend to show that a conceptual schema of the human genome will allow us to better understand the functional and structural relations that exist between the genes and the DNA translation and transcription processes, intended to explain the protein synthesis. Genome, genes, alleles, genic mutations... all these concepts should be properly specified through the creation of the corresponding Conceptual Schema, and the result of these efforts is presented here. First, an initial conceptual schema is suggested. It includes a first version of the basic genomic notions intended to define those basic concepts that characterize the description of the Human Genome. A set of challenging concepts is detected: they refer to representations that require a more detailed specification. As the knowledge about the domain increases, the model evolution is properly introduced and justified, with the final intention of obtaining a stable, final version for the Conceptual Schema of the Human Genome. During this process, the more critical concepts are outlined, and the final decision adopted to model them adequately is discussed. Having such a Conceptual Schema enables the creation of a corresponding data base. This database could include the required contents needed to exploit bio-genomic information in the structured and precise way historically provided by the Database domains. That strategy is far from the current biological data source ontologies that are heterogeneous, imprecise and too often even inconsistent. Keywords: Modelling, Human Genome. Kaschek/Delcambre (Eds.): The Evolution of Conceptual Modeling, LNCS 6520, pp. 306–330, 2011. c Springer-Verlag Berlin Heidelberg 2011
Model-Based Engineering Applied to the Interpretation
1
307
Introduction
It is widely accepted that in modern Software Engineering the use of conceptual modeling techniques applied to software development creates higher quality systems [1]. This is due to the fact that description and understanding of the problem domain is done before the implementation occurs, thus the work is carried out at higher abstraction levels. Conceptual modeling techniques have been applied successfully to many different Information Systems domains from Business to medical applications. A well known example of that successful use is the one related with Organizational Systems [2]. The main idea is to provide a suitable conceptual framework, including possible, simple, clear and unambiguous definitions of the most fundamental concepts in the information system field and a suitable terminology for them. Notions as information and communication, organization and information system, etc.. are properly defined. The Information Systems (IS) experience –meaning where that all the efforts done by the IS in order to design, develop and manage Organizational Systems- has provided a relevant set of results, and it is still a domain where a huge amount of academic and industrial work is in progress. But what could be considered “next” in Conceptual Modeling? When exploring the world looking for new, challenging and suitable domains for the application of Conceptual Modeling techniques, one specific domain pops up that is surprisingly not included: Genomics, and in particular Human Genome proper interpretation. A main issue of this chapter is to show how well IS concepts and practices can work on this Human Genome domain. It is widely accepted that the existence of a Conceptual Model improves the efficiency of Information Systems since it helps to manage modularity and evolution [1,3]. Traditionally, Software Engineering applied to Genomics field has been more oriented to the design of powerful and efficient search algorithms, working in the solution space rather than in the problem space. The complexity of the domain added to the explainable lack of knowledge that software engineers experience in this field may explain the absence of Conceptual Modeling contributions in the Bioinformatics domain. We mean by that, that too often the Bioinformatics work is based on heterogeneous repositories of information that contain inconsistencies, redundancies, partial information, different representation of the same data. This type of problems are well-known and well-reported in the IS domain, and they have been intensively explored, especially in the database context, where a relevant set of sound solutions has been provided during years. With more than 1300 different databases currently reported to be used and exploited, it is easy to understand how hard searching relevant data to undertake any genomic-oriented activity is. Or how hard data analysis it is to perform successfully impossible. Data integration is essential to answer questions in a simple and effective way. To solve these problems, we are convinced that a Conceptual Schema of the Human Genome is strongly required, for facilitating global research among the various distinct research groups, by fixing a conceptual gamut from which researchers can draw, in order to ensure a ’standard dictionary
308
O. Pastor et al.
of concepts’. This can be achieved by fixing concepts, their properties, behavior and relations, and by disambiguating existing biological concepts. These are the reasons why we have created a multidisciplinary group in which software engineers with a strong IS background in Conceptual Modeling and biologists experts in Genomics worked close together in order to look for novel and effective solutions that illuminate the understanding of Human Genome. The result of this collaborative work has been the first ideas around a Conceptual Schema of the Human Genome [4]. Unifying the knowledge needed to understand the Human Genome is a difficult task for software engineers. The conceptualization of the relevant concepts of these domains implies that software engineers change the way things are done. A big effort is required to fix concepts definitions in a context where even the knowledge associated with basic concepts is continuously evolving and changing. As we will see, even fundamental notions as what defines a gene are discussed in depth when we want to have an exact definition. Often, the functional result of some partially unknown structure is just what characterizes that given structure. The subsequent conceptual challenge here is to model concepts taking into account the behavior of the domain elements. In this context, these elements are identified by their functional participation in the genome and not merely by their attributes and the relations between them. All the ideas mentioned above imply a new way of conceptualize domains, where model evolution is a need, and where we could speak about a “fuzzy conceptual modeling” strategy, because many parts of the domain that are being modeled, are being understood step by step and day by day. By showing how this conceptual discussion evolves before concluding how to model any particular genomic concept, we try to provide new insights to answer the question of how could the human genome be best explained by introducing a conceptual modeling perspective. Our intention is to show how the conceptual discussion followed to understand a particular genomic concept facilitated a complete understanding of the high-level ideas that are behind the corresponding concept as well as the details required to manipulate its relevant information. The Conceptual Schema –whose construction process is reported in this chapter- is the essential component intended to provide the required holistic context that enables reaching the goal of achieving a complete understanding of the human genome. According to these ideas, in this chapter we describe the Conceptual Schema evolution from its starting point to the present model, including the different versions needed to describe the domain precisely. We believe that it is important to present these successive versions to help understand the complexity of finding an agreement between biologists and conceptual modelers when a decision is to be taken for defining a genomic concept with the required precision. Those concepts whose interpretation is more problematic are emphasized. In order to achieve our objectives, the chapter starts with an analysis of the most relevant related work; in Section 3 an initial conceptual schema of the human genome is introduced to launch the discussion of how to build such a conceptual schema. After that, in Section 4, a set of iterations is presented, intended to show and understand how
Model-Based Engineering Applied to the Interpretation
309
the evolution of the most relevant, representative conceptual challenges guide the corresponding schema evolution process. The result of this discussion is the introduction of a current version of the conceptual schema of the human genome, which is ready to be properly exploited in an applied bioinformatic domain. One important consequence of the proposed Model-Driven approach is that the database that exactly corresponds to the Conceptual Schema can be built. This database can then be loaded with relevant data for the set of selected genes, following a bottom-up approach where the genomic data chaos is overcome by manipulating only those data that are considered relevant for the selected genes. In this way, the database will really work as a kind or “structured cupboard” where every piece of information is going to be properly disposed. Concluding remarks and the list of used references close the chapter.
2
Related Work
It is interesting to realize that we have not found too many relevant references where the Human Genome –or any other Genome- is faced from a IS Conceptual Modeling perspective. Even if the idea of applying conceptual modeling techniques on molecular biology has been tackled by some informatics and biologist in the last years, the approach has a lot of space to explore and discuss. The more relevant contributions in this field are those made by Paton et al. [5]. This work is an important reference for the labor developed in this chapter, because it can be considered as a starting point schema. In this proposal a collection of data models for genomic data is presented. These models describe elements involved in transcriptional and translational processes as well as the variant effects generated by them. As we will see through the chapter, the work presented here extends these ideas, and proposes a complete Conceptual Schema intended to be seen as a central, conceptual repository of Genomic information, which was not the objective in the referenced work. Some other attempts to model more viable genomes have been reported. An interesting experience is provided by the e-Fungi initiative [14,15], where a systematic comparative analysis of fungal genomes is supported. The e-Fungi database integrates a variety of data for more that 30 fungal genomes and it provides fungal biologists with a powerful resource for comparative studies of a large range of fungal genomes. This work is developed in a different domain –the fungi genome instead of the more complex human genome- but it shows a clear path of results exploitation that could be perfectly projected to our Human Genome modeling effort. There are some other interesting examples of conceptual modeling techniques used in molecular biology, although with a more specific view on a particular part, as the work of Ram [6] to model the protein. This proposal, even if it is a little bit out of our scope, includes a part related to genome which can be very useful for the development of a conceptual schema of a complete genome. In any case, our attempt goes beyond the particular view of modeling proteins, because we want to provide a whole conceptual schema for the Human Genome.
310
O. Pastor et al.
Additionally, a relevant set of bioinformatic implementations are based in major or minor degree on conceptual modeling techniques, and they have been accepted favorably. One example of that is the work of Kevin Garwood et al. [7] which is a model-driven approach for the partial generation of user interfaces for searching and browsing bioinformatics data repositories. This work demonstrates that conceptual schemas can be used to produce many applications in the future. Again, when compared with our work the use of conceptual modeling techniques focuses on some very specific part of a software production process –the user interface design- in the bioinformatics domain, while we want to provide a whole, unified conceptual view in the tradition of the IS modeling experience. These works are a few of the existing examples about the use of conceptual modeling in bioinformatics applications. They can be used to prove that conceptual modeling is an effective approach to help improve biologic research. It is our belief that the work described in this chapter is an important contribution to the global understanding of the human genome, because only having a Conceptual Schema to characterize it, will storing the right contents, manage them efficiently, and understand the precise relationships existing between phenotype (external manifestation of human properties) and genotype (their corresponding genomic code).
3
An Initial Conceptual Schema for the Human Genome
A conceptual schema of the human genome was proposed by Normal W. Paton in February 2000 [5] (see Fig. 1). In this model, an initial conceptual view of the human genome is described as a set of chromosomes divided in fragments that belong to a certain gene, which is an intragenic fragmentation. Paton [5] proposes to classify a chromosomic fragment as either a Transcribed Region fragment or a Non-Transcribed Region fragment. In this model, a Transcribed Region fragment represents the transcribed sequence of the gene excluding elements that are also involved in the transcription process, like promoter or terminator. A Non-Transcribed Region fragment includes the regulatory sequences and the chromosomal elements that are part of the chromosome but are not involved in the transcription process. Additionally, in this model a set of primary transcripts can be generated from a Transcribed Region fragment. We elaborated a new conceptual schema for the Human Genome, derived from Paton’s model. This model, presented in [4], provides a basis to fix the main features that are considered relevant to characterize the human genome basic components. The new conceptual schema introduces changes in the description of a genome. The main difference is the classification of chromosome segments. These chromosome segments are classified as Genic or Non–Genic, where a genic segment is seen as the composition of one promoter, one transcribed sequence, one terminator and many enhancer sequences in contrast with Paton‘s model where only the transcribed region in the gene is considered a genic segment. These genic components share a functional relation derived from the protein synthesis process, as
Model-Based Engineering Applied to the Interpretation
Fig. 1. Paton’s Model
311
312
O. Pastor et al.
the result of transcriptional and translational processes. Any other chromosomal sequence is considered as a non-genic segment (see Fig. 2). This can be seen as an example of how the need of conceptualizing with precision really helps to crystalize and form ideas while helping to fix important concepts that are not really so clear when we just look at the current biological data banks. In this case, considering that chromosome segments must be characterized as either genic or not genic, the gene structure is defined more clearly, and the information associated with the proposed structure will be adequately stored in the corresponding Genome database.
4
Conceptual Schema Evolution
In this section the evolution “suffered” by the model will be presented. The most relevant changes and conceptual challenges will be discussed to understand the learning process from the initial model to the current stable conceptual schema of the Human Genome. This is an interesting issue when applying Conceptual Modeling in this domain, because we have experimented how complicated fixing precise definitions can be in a context where new knowledge is discovered day after day. When modeling conventional organizational systems, its main components, their relationships and the functional processes are mainly known, and this makes the conceptual schema construction process viable and feasible. When facing the problem of modeling the human genome, we have seen how concepts and their representation can vary as the bio-genomic knowledge increases. In this section we will comment and discuss some relevant conceptual challenges, meaning by that the process followed to decide how to conceptualize some widely-known and used concepts of the human genome domain. As a very basic notion in this context, we will start with the notion of gene. The precise definition of gene is a very interesting topic that biologists still debate. The term has evolved, since it was first proposed by W. Johansen in 1909 based on the concept developed by Gregor Mendel, to fit the different paradigms and theories that have occurred in the history of Genetics (For a complete review see [10]). The classical view of a gene -a linear piece of DNA inherited from the parents who coded for a single protein- has considerably changed reflecting the complexity of the term. In the updated definition proposed by Gerstein in 2007 [10], a gene is “a union of genomic sequences encoding a set of potentially overlapping functional products” and this evolution of the concept carries important implications inside. Trying to explain such a fuzzy term to “conceptual modellers” is a very challenging and difficult task. During the conceptual modeling of the human genome, our group had to deal with many misunderstandings produced by the dramatically different way of handling concepts of biologists and software engineers. The fact that a gene sequence varies in different databases, the idea that two genes may share the same locus or that post-transcriptional events generate multiple products from one genetic locus are some of the concepts that puzzled software engineers. This reflects the changes that the concept had during the evolution
Model-Based Engineering Applied to the Interpretation
Fig. 2. An initial Conceptual Schema for the Human Genome
313
314
O. Pastor et al.
of the model. In early versions the Gene class was associated to GenicSegment class, which was a big DNA segment composed by one promoter, one transcribed sequence and one terminator and regulated by many enhancer sequences. In the following versions GenicSegment class became smaller, generalizing any genic segment with some functionality (promoter, transcribed sequence, etc...). Then the Gene class was associated to TranscriptionUnit class, which was a really rigid structure that combined the genic segments involved in the transcription process. In later versions, the concept of TranscriptionUnit class became broader and multiple compositions appeared for a single transcription unit, in agreement with Gerstein’s updated definition of Gene. This way we can model a concept whose definition is still discussed and assume that some of its characteristics may be ambiguous, unknown or even wrong. The TranscriptionUnit class, as it appears in the third version of the model (see Fig. 3), evolved in such a way that allows for the inexistence of promoter and terminator sequences, represented by cardinality constraints (0,1) in Promoter and Terminator classes. This change is relevant, considering that these transcription unit elements exist but are frequently not indicated in the data repositories. However, at least a transcribed sequence must be present, represented by cardinality constraint (1..*) in TranscribedSequence class. Following with the analysis of the domain, we found a new important concept that implies a change in the conceptual schema. An Enhancer sequence can be bound to certain proteins to enhance the transcription levels of the corresponding genes. This idea implies that enhancers are included in a group of regulator sequences that regulate the DNA transcription process, therefore the Enhancer class is replaced by RegulatorSequence class. Furthermore, if we take into account that a regulator sequence must be related with one or many transcription units then the RegulatorSequence class must be associated with TranscriptionUnit class. Besides, a regulator sequence is regulated by one or many regulator sequences. At this point, the model only included the concept of gene and its structure from a transcriptional point of view. Once the Transcription and the Translation were studied in depth, a new conceptual schema was defined to include concepts and products related to them (see Fig. 4). For a more detailed description of these processes see [11]. The protein synthesis process starts with the transcription: a DNA sequence is transcribed to a complementary RNA sequence. This RNA molecule is known as primary transcript. In the model, this new concept is represented by PrimaryTranscript class, which is associated to a set of instances of a new class: ElementTranscript. The specialization of this class in Exon and Intron classes describes the integration of exons and introns in the molecular structure of the primary transcript and represents the different types of partition fragments. A reflexive association “Prior,Next” for ElementTranscript class is added to indicate the order of exons and introns in a primary transcript sequence. The splicing process consists in the combination of certain exons of the primary transcript resulting in a mature mRNA. The outcome of the splicing
Model-Based Engineering Applied to the Interpretation
Fig. 3. Second version of the conceptual schema
315
316
O. Pastor et al.
process application to a primary transcript is represented by the SplicedTranscript class that is specialized in the classes mRNA and other RNA types. The aggregation relation “splice” between SplicedTranscript class and Exon class allows to identify the exons that are the result of the splicing process. The mRNA class represents the messenger RNA, a molecule that contains the information needed to synthesize a protein. The RNA sequence determines the amino acid order in the protein. The other classes include RNAs that are not necessarily translated into amino acid sequences: snRNA (small nuclear RNA, that participates in important nuclear processes), tRNA (transfer RNA, an important molecule in the transcription process) and rRNA (ribosomal RNA, which is part of the ribosome). The next step in protein synthesis is the migration of mature mRNA from the nucleus to the cytoplasm. There it associates to the ribosome and the translation process starts. The translation is the production of proteins by decoding the mRNA produced in the transcription. In this process, the mRNA molecule acts as a template for the synthesis of the corresponding amino acid chain. The decodification rules are specified by the Genetic Code. Notably, the mRNA molecule is not translated completely; the ORF (Open Reading Frame) is the part of the mRNA sequence used in the translation process. The PrimaryPolypeptide class is created to describe the protein primary structure: the amino acid chain obtained after the translation of an ORF. This amino acid chains suffers some chemical transformations and the final result is a functional protein which is represented in our model as the Protein class. The association between Protein class and PrimaryPolipeptide class is included to conceptualize that a primary polypeptide originates a protein and a protein is synthesized as a primary polypeptide. Besides this new view, some constraints were defined in the TranscriptionUnit class. The Promoter class has an integrity constraint which describes the order of the elements included in the same transcription unit. This constraint specifies that the final position of the promoter sequence is always the initial position of the transcribed sequence and, at the same time, the final position of the transcribed sequence is always the initial position of the terminator sequence. All the added classes and relations between classes listed below provide the bases for a more complete genomic model that, due to the size of the schema should be organized in two different views: the Genome view that includes the concepts from genome to transcription unit and the Transcription and Translation view that enclose all the concepts from the primary transcript to the fully functional protein. As a result of the evolution of the discussions, some more important changes arose. In previous versions of the schema, a gene was modeled taking into account its structure, which was considered fix. But in the new model, depicted in Fig. 5, the conceptualization of the gene is replaced by the idea of modeling a generic gene. At this point, the Allele class was introduced. An allele is each one of the alternative forms a gene may have in nature. These alternative forms differ by their sequences and any of them may produce changes on gene function. One
Model-Based Engineering Applied to the Interpretation
317
of these forms is considered the wild type and is distinguished from all other variants allele for being the most abundant in nature. The generic genic structure is conserved independently of the alternative forms of a gene. A genic segment will be associated with generic genic segments. This idea allows conceptualizing of the relation between the alternative forms of a gene and the gene structure. To explain the variation between alleles, we introduced a classification of the mutations occurred in any of these allele variants. The classification is made following two different criteria: the precision of the variation description and the phenotype that it produces. The first classification of the variations is divided in two categories. The precise variations are those that are described accurately in the conventional data bases. The imprecise variations are those that are described by text in the data sources, therefore it is not possible to automatically detect the exact position of the variation. The second classification of the allelic variants is divided in four types: 1) Genic Mutation, which names the variation that produces a pathologic effect. 2) Chromosomic Mutations, which describes the variation that affects more than one gene. 3) Natural Polymorphism, that characterizes a neutral variation, and 4) Unknown Consequence Changes, referring to those that report a variation with undiscovered consequences. To represent in the model all the newly acquired concepts, a set of new classes were introduced. The first of them is the concept of reference sample. A Reference class is created and the aggregation relation “belongs” is included to describe that a genome has a reference.The idea of modeling a generic gene removes the specialization of GenicSegment class in Promoter, TranscribedSequence, Terminator and RegulatorSequence classes. This change is made because a genic segment is not generic. The GenericGenicSegment class is added to the model to represent a generic genic segment. A genic segment can be associated to a generic genic segment, what is represented by the association relation “corresponds” between GenicSegment class and GenericGenicSegment class. This relation describes the concept that a genic segment always corresponds to a generic genic segment and a generic genic segment can be associated (or not) to one or many genic segments. Once again, we want to emphasize how the application of a CM-based perspective allows for dealing with the evolving knowledge that is so characteristic of this domain. With the discussion being focused on the conceptual model level, it is possible to include the novel interpretations in the conceptual model in a evolutionary way, always with a sound control on the semantics of the model. For instance, the distinction that was introduced between Genic Segment and Generic Genic Segment was very significant. Before introducing the Generic Genic Segment, the conceptual schema was not distinguishing between particular genome instances and a generic pattern of reference. While it was understood that individual samples should provide individual genomes, it became clear that the instance level had to be distinguished from the generic reference level.
318
O. Pastor et al.
Fig. 4. Third version of conceptual schema
Model-Based Engineering Applied to the Interpretation
Fig. 5. Fourth version of conceptual schema
319
320
O. Pastor et al.
This generic level denotes a common template valid for any genome structure, while the instance level represents individual occurrences of this generic, common structure. The introduction of the Generic Genic Segment notion related to but different from the former Genic Segment provides the required connection between the two perspectives, leading to a Conceptual Schema that properly captures this important information. Again, we see here a concrete proof of the benefits obtained by using a CM-based approach to fix understand the genomic domain. This allows also to exploit it correctly through the associated database, where the relevant information is accordingly structured. Additionally, the Allele class is added to the model to store the alternative forms of a gene found on the data repositories. Due to this, the gene is not associated anymore to the transcription unit but to alleles and the association relation between TrascriptionUnit class and Gene class is removed. The aggregation relation “has” between Gene class and Allele class is added to denote that a gene will always have one or many alleles associated. The aggregation relation “has” between TranscriptionUnit class and Allele class is included to specify that an allele is always associated to one or many transcription units. In this context, another aggregation relation “has” between Allele class and GenericGenicSegment class is included in this model to describe that an allele has one or many associated generic genic segments. Allele class is specialized in WildType class and AllelicVariant class. The WildType class represents the natural allele sequences and the AllelicVariant class represents the variant sequences that are not the wild type. The allelic variant groups are specified in the model by the specialization of AllelicVariant class in two groups of classes. The first group is specialized in Imprecise class and Precise class. The second group is specialized in GenicMutant, ChromosomicMutant, NeutralPolimorphism, and UnknownConsequenceChange.
5
The Current Version of Conceptual Schema
After the introduction of the main aspects related to the schema evolution, the current version of the Conceptual Schema of the Human Genome is presented in this section. It can be seen as the intermediate stable state that contains the current relevant information. At the same time the schema is ready for both evolving with the incorporation (or modification) of new conceptual issues, and being properly exploited through loading the adequate biogenomic contents using its corresponding database. The role of the database is vital in terms of driving an efficient and effective storage processing for genomic data. According to the well-known Information Systems principles related to database design, the database that corresponds to the conceptual schema has been built. Having clear the definition of the data structure, the next problem is naturally to load it with the adequate contents. Analyzing the current, diverse and often unstructured biological data repositories, the relevant information for particular genes can be detected, isolated and moved to the database. This gene-by-gene strategy opens the door to having a Genomic database containing just the selected,
Model-Based Engineering Applied to the Interpretation
321
correct and required information, instead of the tons of any type of information that we currently find in biological data sources that are most used. A set of experiments are being designed and implemented in the domain of the genomic clinical diagnosis, where the elaboration of a genomic report can be done much more efficiently and accurately using this Model-Driven Engineering approach. For the sake of understandability, the current version of the Conceptual Schema is divided in three main views (Gene-Mutation, Transcription and Genome). Fig. 6 is the general view of the model and is intended to show the interconnections between each one of the different views that will be described in detail in the following subsections. 5.1
Gene-Mutation View
In the Gene-Mutation view presented in Fig. 7, all the knowledge about genes, their structure and their allelic variants based on the different public databases is modeled. The principal entities in this view are the Gene class and Allele class. The Gene class models the concept of generic gene independently of the samples registered in the databases. In this class, we find attributes as Id_Hugo, a numeric code that represents the universal code for the gene according to HGNC (Human Genome Nomenclature Committee) [12]. The name attribute stores the common name of the gene. Chromosome attribute represents the number of the chromosome where the gene is located and the locus attribute represents a gene location into the chromosome according to NCBI information [13]. Finally, a summary of the gene is extracted from NCBI database, and it is stored in the summary attribute. Another main schema class is Allele class, which represents the instances of a generic gene and is the most important class of the model since all the information depends on it. This class contains all the relevant information about alleles such as allele databases, references, variants, generic genes and mRNA and DNA sequences. In the case of AllelicVariant class, all the information about the variation is represented as well as some identification and descriptive attributes. The ord_num attribute is the internal identification number of the allele in our database. Data_bank_source attribute is the external code of the database source. Another important class is AllelicRegion; this class represents the chromosomal region where the allele is, and contains an important attribute: sequence, which will store the complete DNA sequence of the allele. The attributes start_position and end_position will describe the beginning and the end of the allele in reference to the chromosome. The relation between Gene class and Allele class helps to identify any allele of a gene in the information system. The cardinality of this relation (1..1:0..N) allows to represent a gene with no allelic information. To represent external references for gene and allele information, we created certain classes in the conceptual schema. The GeneExternalIdentification class represents the identification of a gene in different public databases. The OthersAlleleDataBank class, is the same representation but for alleles.
322
O. Pastor et al.
As far as we know, an allele might be considered reference or variant. For this reason, the AllelicVariant class and ReferenceType class are specialized classes from Allele class. The ReferenceType class represents the alleles used as references in the existing databases, and the AllelicVariant class represents allelic variations of a reference allele in a database. There is a related association between ReferenceType class and AllelicVariant class, this association allows us to represent a relation between a reference allele and its variations. It is important to note that the WildType class has been replaced by the ReferenceType class, a more suitable term for bioinformatic purposes since it does not have genetical meaning. Once we propose an allelic variant, we determine a specialization hierarchy from the AllelicVariant class. This lead to the classification of allelic variants in two specializations. In G1, four different situations are considered: the allelic variation is specialized in GenicMutant when this variation affects a gene and it is specialized in ChromosomicMutant when the variation affects parts of the chromosome. The ChromosomicMutation class describes the chromosomic variation that affects one or many genes in the same chromosome. NeutralPolimorphisme class is a variation that does not affect the phenotype. And finally the UnknownConsequence class is used to represent the case when the variation consequences are not known yet. In G2, we classified the variation in two types: imprecise and precise. When we do not know details about the variation, we classify it as imprecise. There is a description attribute in the Imprecise class that stores information about the variation in text format. Opposite to that, when a variation is precise we can represent the position where the variation occurred. In this view we also modeled the allele segmentation and structure. We considered that an allele segment has a significant and indivisible sequence of DNA. For this reason we implemented the Segment class, with attributes such as ord_num, (that identifies a segment between all the allele segments), start_position (initial position of the segment in a chromosome) or end_position (the end of the segment). The sequence attribute stores the sequence between start_position and end_position. This class has four specialized classes classified by their function in the transcription process. The first of these specialized classes is Promoter, which represents the region of the DNA sequence that facilitates the transcription of the transcription unit; TranscribedSequence is the sequence of DNA transcribed by the ARN polymerase; Terminator is a fragment of DNA that marks the end of the transcription process. Finally, the RegulatorSequence class is a segment with regulatory functions that regulates one or many transcription units. After the definition of these specialized classes, a new class was created to model the concept of transcription unit. This class – TranscriptionUnit – has one attribute ord_num, which is used for internal identification between all the transcription units of the same allele. The relations between this class and the specialized classes have different means. The relation between TranscriptionUnit class and Promoter class means that a Transcription Unit has a unique promoter; since this one may be unknown, this relation has cardinality 1..1:0..1.
Model-Based Engineering Applied to the Interpretation
Fig. 6. Gene-Mutation View
323
324
O. Pastor et al.
The relation between TranscriptionUnit class and TranscribedSequence class has cardinality 1..1:1..*. This means that many transcript sequences may exist in the same transcription unit starting all at the same position. The relation between TranscriptionUnit class and Terminator class means that a transcription unit may have more than one terminator segment that can be unknown. The relation between TranscriptionUnit class and RegulatorSequence class means that a transcription unit may have many regulator segments, shared by different transcription units belonging to several genes in the most general case. Regarding the precision of the DNA sequences in the schema, it is interesting to remark that initially the concept of allelic region was introduced to alleviate the inaccuracy of the DNA sequences that represent copies of the same gene in different databases. On the other hand, when the same databases were browsed to look for data regarding the fragmentation in different components of a gene (promoter, transcribed sequence and terminator) those data were precise and detailed. The AlellicRegion class indicates the approximate region where a gene is contained, whereas the sequence attribute of the Segment class gives clear and precise data information of the beginning and the end of the transcribed region. 5.2
Transcription View
Another important view of our model is the Transcription view that is showed in Fig. 8. In this view the basic components related to the protein synthesis process are modeled. This is a very important part of the schema, as it enables linking the genotype-oriented features modeled above with the phenotype-oriented manifestations that can be perceived externally. Consequently, it requires to explain the relationships between genes and alleles with their corresponding external expression in terms of behavior. An interesting issue to be discussed in this context is how this transcription view relates to the functional and behavioral perspective of the conceptual model that we are building. The view that we are considering through the chapter is basically structural and data-oriented, meaning by it that what we are fixing is the class arquitecture of the model together with the relationships between classes. This is of course a needed first step, and it is strictly required to set the basis of the conceptual modeling approach defended in this work. While it is completely true that the transcription and translation processes have a complex and relevant behavioral perspective that should be properly described, it is also true that in the context of the description followed in this chapter, we are firstly interested in the consequences of those processes on the relevant classes and relationships that conform the basic data architecture of the system. This is why we leave behavioral and interaction modeling aspects for future versions of the model, once this structural part will be properly fully-specified. Following in that direction, the first class found in this view is PrimaryTranscript class, which represents the RNA copy from the DNA of the transcribed sequence (TranscribedSequence class). This class has an attribute sequence, which is a derived attribute from the Segment class. We implemented the PrimaryTranscriptPath class, in order to model the different splicing factor-driven partitions
Model-Based Engineering Applied to the Interpretation
Fig. 7. Transcription View
325
326
O. Pastor et al.
of the PrimaryTranscript. This class has an attribute ord_num used to identify a partition from all the partitions of a primary transcript. The ElementTranscript class has an ord_num attribute which helps us to identify a specific fragment from all of the partition fragments. The specialized Exon and Intron classes represent the types of the partition fragments. The Produces class represents different exon combinations from a primary transcript that produces a spliced transcript. The result of these combinations is the specialized classes mRNA and othersRNA types. The SplicedTranscript class has the resultant sequence of the exon combinations in the sequence attribute and ord_num attribute for its identification between all the allele spliced transcripts. The mRNA contains nucleotide sequences that could potentially encode a protein; this is what we know as ORF (Open Reading Frame). In our model, the ORF class has an id attribute as internal identification and a sequence attribute that stores codifying sequence. Then we have the PrimaryPolypeptide class, which describes the protein primary structure: the amino acid chain obtained after the translation of an ORF. This amino acid chain suffers some chemical transformations and the final result is a functional protein, which is represented in our model as Protein class. A protein could be formed by one or more primary polypeptides. In the Protein class we find a name attribute which represents the name of the resulting protein and its amino acid sequence in the sequence attribute. We want to close this subsection and focus on the next step to be completed: the phenotype connection. Having characterized where proteins come from, the conceptual schema is ready to include phenotypical information. Relating external, perceived features to the protein or set of proteins involved in their expression, a challenging research for linking genotype and phenotype is really to be analyzed, what is going to be for sure a very relevant direction of research in the next years. Following what we have called in this chapter a “model-driven engineering” perspective, this is going to be possible. As conceptual models become the key software artifact for dealing with the Human Genome, model transformations could be defined both at the upper direction –connecting requirements with the conceptual schema that include how these requirements are expressedand in the lower direction –linking the kwowledge represented in the Conceptual Schema with individual samples to infer potential problem or to identify particular human being characteristiscs. 5.3
Genome View
In Fig. 8, a conceptual schema of a complete individual genome is presented. This view is especially interesting for future applications, since massive parallel sequencing technologies will allow the complete sequencing of individual genomes at a very low price in the near future [8,9]. When this information becomes available we will be able to store it in our database, and it will be stored according to the precise conceptual structure fixed by the Conceptual Schema. First we have ResearchCentre class which represents the labs or research centres where a specific human genome was sequenced. We have name and description attributes
Model-Based Engineering Applied to the Interpretation
Fig. 8. Genome View
327
328
O. Pastor et al.
to record the genome source. Then Genome and Chromosome classes represent a complete genome sequenced by a specific research centre. A genome is a set of chromosomes, this is represented by cardinality 1..* relation between Genome class and Chromosomes class. The number attribute will determine a specific chromosome on a specific genome. The “couple” relation on the Chromosome class represents the concept of homologue pair, which means that every human cell will carry two equivalent chromosomes (one from the father and one from the mother) with different alleles for each gene. The ChromosomeSegment class represents all the segments that form a complete chromosome. This class has as sequence attribute to store the corresponding DNA sequence delimited by start_position and end_position attributes. Another important concept that we have represented in our model is the structure of a chromosome. We identified that a chromosome has two main types of segments: coding-related segments (represented by the GenicSegment class) and not coding-related segments (represented by the NonGenicSegment class); these two classes are specialized from the NonGenicSegment class: IntergenicRegion class, which represents the space between genes and ChromosomalElement class. The last one has three specialized classes that describe other elements of the chromosomes (Centromere class, Telomere class, and ORI class) whose function is to keep the chromosome functional and have little to do with protein production. It is interesting to remark that, at the beginning of the modeling process we expected to have a unique, complete, reference description for each gene. This description should include DNA and RNA sequences, alleles, segmentations, mutations and every detail that could be potentially included in the schema. This is the main reason of the GenericGenicSegment class addition, which tries to manifest the uniqueness of the gene description. Later, conscious of the fact that different databases gave different definitions of the same gene, we chose to drop the idea of a “generic gene” to substitute it by the possibility to include different descriptions of the same gene offered by different databanks. As a result of this OthersAlelleDatabank class was included, as well as the databank attribute in the Allele class. This is another clear example of concept evolution driven by the learning process or what we previously called a Fuzzy Conceptual Modeling Strategy.
6
Conclusions
The way in which the knowledge of the human genome domain is captured and acquired by the IS experts is one of the richest parts of this chapter. When biologists and software engineers interact, the underlying vocabularies and ontologies are too easily too far. To make them become as closer as possible is being a continuous challenge for the group of people working on this conceptual modeling of the human genome effort. In this chapter, we have tried to show how this evolution occurred, and how so basic notions as those of gene, genic segment, alleles, mutations or transcription, were refined step by step. Many times a definition accepted by everybody was changed after some time just because some vague
Model-Based Engineering Applied to the Interpretation
329
detail was missed out in the first rounds. As at the end, a concrete proposal of a conceptual schema is presented as a result of these discussions and interactions, we argue to provide with all the reported experience very valid material for any other interdisciplinary group interested in facing the problem of understanding the human genome. Another contribution of this chapter is directly related to the historical value recognized in conceptual modeling. The modeling benefits that biological systems research should get from IS theory and practice include at least: – working at a higher abstraction level to specify systems easily and more precisely, – enable reasoning to reason about a biological system prior to its construction to foresee consequences in advance, and to facilitate simulation and validation, – the real possibility of automating the subsequent systems development. It is our belief that real practice in the genomic domain is also strongly requiring the IS experts participation. It makes sense from our IS perspective to talk about the chaos of genome data, meaning that there are currently tons of data from the genome publicly available, each one with its corresponding database that is defined with an specific schema, data format, identifications, etc.. and where the integration of the different sources is a very difficult task that sometimes is just not possible. The proper management of this information ecosystem is an urgent need, because the available scientific literature and experimental data, the summaries of knowledge of gene products, the information about diseases and compounds, the informal scientific discourse and commentaries in a variety of forums, and many more genetic-related data is dramatically increasing day after day. To understand the relevant knowledge that is behind that huge amount of information, conceptual models should become the key software artifact. With our proposal of a Conceptual Schema for the Human Genome, including the way in which the model construction has evolved to reach the current situation, we believe to have demonstrated that such a task is feasible and fully useful. This is why we have chosen Model Driven-based Engineering applied to the Understanding of the Human Genome as the chapter title: if models become the key artifact used to interpret and exploit the knowledge that is around the human genome domain, its understanding and subsequent number of successful practical applications will clearly increase. Related with usefulness, we are now in the process of providing the adequate contents to the database that corresponds to the specified conceptual schema. We want to achieve a very simple but ambitious objective: if with Conceptual Models targeted at digital elements we can improve Information Systems Development, with Conceptual Models targeted at life we can directly improve our living. The Conceptual Schema for the Human Genome is our modest contribution in that direction.
330
O. Pastor et al.
References 1. Olivé, A.: Conceptual Modelling of Information Systems. Springer, Heidelberg (2007) 2. Falkenberg, E., Hesse, W., Lindgreken, W., Nilsson, E., Han, J., Rolland, C., Stamper, R., Van Assche, F., Verrijn-Stuart, A., Voss, K.: A Framework Of Information System Concepts. In: IFIP (1998) 3. Pastor, O., Molina, J.C.: Model-Driven Architecture in Practice. Springer, Heidelberg (2007) 4. Pastor, Ó.: Conceptual Modeling Meets the Human Genome. In: Li, Q., Spaccapietra, S., Yu, E., Olivé, A. (eds.) ER 2008. LNCS, vol. 5231, pp. 1–11. Springer, Heidelberg (2008) 5. Paton, W.N., Khan, S., Hayes, A., Moussouni, F., Brass, A., Eilbeck, K., Globe, C., Hubbard, S., Oliver, S.: Conceptual modeling of genomic information. Bioinformatics 16(6), 548–557 (2000) 6. Ram, S.: Toward Semantic Interoperability of Heterogeneous Biological Data Sources. In: Pastor, Ó., Falcão e Cunha, J. (eds.) CAiSE 2005. LNCS, vol. 3520, pp. 32–32. Springer, Heidelberg (2005) 7. Garwood, K., Garwood, C., Hedeler, C., Griffiths, T., Swainston, N., Oliver, S., Paton, W.: Model-driven user interface for bioinformatics data resources: regenerating the wheel as an alternative to reinventing it. Bioinformatics 7(532), 1–14 (2006) 8. Bornberg-Bauer, E., Paton, N.: Conceptual data modelling for bioinformatics. Briefings in Bioinformatics 3(2), 166–180 (2002) 9. Wheeler, D.A., Srinivasan, M., Egholm, M., Shen, Y., Chen, L., McGuire, A., He, W., Chen, Y., Makhijani, V., Roth, G.T., Gomes, X., Tartaro, K., Niazi, F., Turcotte, C.L., Irzyk, G.P., Lupski, J.R., Chinault, C., Song, X., Liu, Y., Yuan, Y., Nazareth, L., Qin, X., Muzny, D.M., Margulies, M., Weinstock, G.M., Gibbs, R.A., Rothberg, J.M.: The complete genome of an individual by massively parallel DNA sequencing. Nature 452, 872–877 (2008) 10. Gerstein, M.B., Bruce, C., Rozowsky, J., Zheng, D., Du, J., Korbel, J., Emanuelsson, O., Zhang, Z., Weissman, S., Snyder, M.: What is a gene, post-ENCODE? History and updated definition. Genome Res. 17, 669–681 (2007) 11. Alberts, B., Johnson, A., Lewis, J., Raff, M., Roberts, K., Walter, P.: Molecular Biology of the Cell. Garland Science, New York (2002), http://www.ncbi.nlm.nih.gov/books/bv.fcgi?rid=mboc4 12. Gene Nomenclature Committee, http://www.genenames.org 13. National Center for Biotechnology Information, http://www.ncbi.nlm.nih.gov 14. Hedeler, C., Wong, H.M., Cornell, M.J., Alam, I., Soanes, D., Rattray, M., Hubbrad, S.J., Talbot, N.J., Oliver, S.G., Paton, N.: e-Fungi: a data resource for comparative analysis of fungal genomes. BMC Genomics 8(426), 1–15 (2007) 15. e-fungi Project, http://www.cs.man.ac.uk/cornell/eFungi/index.html
Author Index
Bianchini, Devis
Neumayr, Bernd
161
Casamayor, Juan Carlos Celma, Matilde 306 Courter, Andrew 205 De Antonellis, Valeria
306
161
Embley, David W. 137 Eraso, Luis E. 306
Jannach, Dietmar
225
Kappel, Gerti 254 Kop, Christian 21
Pastor, Oscar 306 Porto, Fabio 285 Retschitzegger, Werner
254
Schrefl, Michael 68, 180 Schwinger, Wieland 254 Shekhovtsov, Vladimir A. 117 Shrestha, Rajiv 205 Spaccapietra, Stefano 285 Stumptner, Markus 180
Gao, Le 205 Grossmann, Georg 180 Gut, Alexander 225 Henderson-Sellers, Brian
68
93 Tao, Cui 137 Thalheim, Bernhard Thiagarajan, Rajesh
42, 68 180
Urban, Susan D.
205
Levin, Ana M. 306 Liddle, Stephen W. 137
Virrueta, Aremy
306
Mayr, Heinrich C. Melchiori, Michele
Wieringa, Roel 1 Wimmer, Manuel 254
21 161
