Information Modeling Methods and Methodologies

Information Modeling Methods and Methodologies John Krogstie Norwegian University of Science and Technology, Norway SINT...

Author: Valentina Plekhanova

66 downloads 1263 Views 5MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

Information Modeling Methods and Methodologies John Krogstie Norwegian University of Science and Technology, Norway SINTEF Telecom and Informatics, Norway Terry Halpin Northface University, USA Keng Siau University of Nebraska, Lincoln, USA

A release in the “Advanced Topics in Database Research” series

IDEA GROUP PUBLISHING Hershey • London • Melbourne • Singapore

Acquisitions Editor: Senior Managing Editor: Managing Editor: Development Editor: Copy Editor: Typesetter: Cover Design: Printed at:

Mehdi Khosrow-Pour Jan Travers Amanda Appicello Michele Rossi Maria Boyer Jennifer Wetzel Lisa Tosheff Yurchak Printing Inc.

Published in the United States of America by Idea Group Publishing (an imprint of Idea Group Inc.) 701 E. Chocolate Avenue, Suite 200 Hershey PA 17033 USA Tel: 717-533-8845 Fax: 717-533-8661 E-mail: [email protected] Web site: http://www.idea-group.com and in the United Kingdom by Idea Group Publishing (an imprint of Idea Group Inc.) 3 Henrietta Street Covent Garden London WC2E 8LU Tel: 44 20 7240 0856 Fax: 44 20 7379 3313 Web site: http://www.eurospan.co.uk Copyright © 2005 by Idea Group Inc. All rights reserved. No part of this book may be reproduced in any form or by any means, electronic or mechanical, including photocopying, without written permission from the publisher.

Library of Congress Cataloging-in-Publication Data Information modeling methods and methodologies / John Krogstie, Terry Halpin, Keng Siau, Editors. p. cm. ISBN 1-59140-375-8 -- ISBN 1-59140-376-6 -- ISBN 1-59140-377-4 1. System design. 2. Information technology. I. Krogstie, John. II. Halpin, T. A. III. Siau, Keng, 1964QA76.9.S88I528 2004 003.3--dc22 2004003749

Information Modeling Methods and Methodologies is part of the Idea Group Publishing series named Advanced Topics in Database Research (Series ISSN 1537-9299) British Cataloguing in Publication Data A Cataloguing in Publication record for this book is available from the British Library. All work contributed to this book is new, previously-unpublished material. The views expressed in this book are those of the authors, but not necessarily of the publisher.

Information Modeling Methods and Methodologies Table of Contents Preface ............................................................................................................. vi

SECTION I: GENERAL TECHNIQUES FOR COMPARING AND ADAPTING MODELING METHODS Chapter I. Analyzing and Comparing Ontologies with Meta-Models ... 1 Islay Davies, Queensland University of Technology, Australia Peter Green, University of Queensland, Australia Simon Milton, University of Melbourne, Australia Michael Rosemann, Queensland University of Technology, Australia Chapter II. Two Meta-Models for Object-Role Modeling ................... 17 Dave Cuyler, Sandia National Laboratories, USA Terry Halpin, Northface University, USA Chapter III. Evaluating Conceptual Coherence in Multi-Modeling Techniques .................................................................................................... 43 Bart-Jan Hommes, Delft University of Technology, The Netherlands Chapter IV. Assessing Enterprise Modeling Languages Using a Generic Quality Framework ....................................................................... 63 John Krogstie, Norwegian University of Science and Technology, Norway and SINTEF Telecom and Informatics, Norway Sofie de Flon Arnesen, Norwegian University of Science and Technology, Norway

Chapter V. An Approach for Evolution-Driven Method Engineering .. 80 Jolita Ralyté, University of Geneva, Switzerland Colette Rolland, University of Paris 1 - Sorbonne, France Mohamed Ben Ayed, University of Paris 1 - Sorbonne, France

SECTION II: GOAL, REQUIREMENTS, AND PROCESS MODELING Chapter VI. Goal Modeling in Requirements Engineering: Analysis and Critique of Current Methods ........................................................... 102 Evangelia Kavakli, University of the Aegean, Greece Pericles Loucopoulos, University of Manchester, UK Chapter VII. An Empirical Investigation of Requirements Specification Languages: Detecting Defects While Formalizing Requirements ......... 125 Erik Kamsties, University of Duisburg-Essen, Germany Antje von Knethen, Fraunhofer Institute for Experimental Software Engineering, Germany Jan Philipps, Technische Universität München, Germany Bernhard Schätz, Technische Universität München, Germany Chapter VIII. Validating an Evaluation Framework for Requirements Engineering Tools ..................................................................................... 148 Raimundas Matulevièius, Norwegian University of Science and Technology, Norway Chapter IX. A Comparison of the FOOM and OPM Methodologies for User Comprehension of Analysis Specifications ............................ 175 Judith Kabeli, Ben-Gurion University of the Negev, Israel Peretz Shoval, Ben-Gurion University of the Negev, Israel Chapter X. Participatory Development of Enterprise Process Models ......................................................................................................... 195 Reidar Gjersvik, SINTEF Industrial Management, Norway John Krogstie, Norwegian University of Science and Technology, Norway and SINTEF Telecom and Informatics, Norway Asbjørn Følstad, SINTEF Telecom and Informatics, Norway

SECTION III: DATA, ONTOLOGY, AND COMPONENT MODELING Chapter XI. A Taxonomic Class Modeling Methodology for Object-Oriented Analysis ......................................................................... 216 Il-Yeol Song, Drexel University, USA Kurt Yano, Drexel University, USA Juan Trujillo, University of Alicante, Spain Sergio Luján-Mora, University of Alicante, Spain Chapter XII. Comprehension of Hierarchical ER Diagrams Compared to Flat ER Diagrams .............................................................. 241 Revital Danoch, Ben Gurion University of Negev, Israel Peretz Shoval, Ben Gurion University of Negev, Israel Mira Balabaan, Ben Gurion University of Negev, Israel Chapter XIII. Constraints on Conceptual Join Paths ......................... 258 Terry Halpin, Northface University, USA Chapter XIV. Using a Semiotic Framework for a Comparative Study of Ontology Languages and Tools ............................................... 278 Xiaomeng Su, Norwegian University of Science and Technology, Norway Lars Ilebrekke, Norwegian University of Science and Technology, Norway Chapter XV. A Service-Oriented Component Modeling Approach ..................................................................................................... 300 Zoran Stojanovic, Delft University of Technology, The Netherlands Ajantha Dahanayake, Delft University of Technology, The Netherlands Henk Sol, Delft University of Technology, The Netherlands Chapter XVI. Evaluation of Component-Based Development Methods ....................................................................................................... 323 Nicky Boertien, Rabobank Nederland, The Netherlands Maarten W.A. Steen, Telematica Instituut, The Netherlands Henk Jonkers, Telematica Instituut, The Netherlands About the Authors ..................................................................................... 344 Index ............................................................................................................ 354

vi

Preface

The field of information systems engineering includes numerous modeling methods and notations. Even with attempts to standardize (e.g., UML for object-oriented design adopted by the Object Management Group (OMG)), new modeling methods and methodologies are constantly being introduced, many of which differ only marginally from existing approaches. A systematic study of modeling methods and methodologies is needed to understand the strengths and weaknesses of each method, and the appropriate contexts and tasks where each is most suitably applied. This understanding has important consequences for the use of existing methods, evolution of existing methods, and the design of entirely new methods (method engineering) when necessary. The purpose of this book is to disseminate the research results and best practice from researchers and practitioners interested in and working on modeling methods and methodologies. Evaluation of modeling methods remains a challenge in information systems engineering. Though the need for such studies is well recognized, there is a paucity of such research in the literature. There is a clear need for innovative, effective, and efficient techniques for such evaluation. The EMMSAD (Evaluating Modeling Methods for System Analysis and Design) workshops have since 1996 been a meeting ground specifically to attack this problem area. The book is based on extended versions of the best papers submitted to the EMMSAD workshops in 2001, 2002, and 2003. The EMMSAD workshop series has received wide acceptance and support from the modeling community. Each year, we have received submissions from all over the globe, making EMMSAD a truly international workshop, with 30 to 50 participants attending each year, both from research and industry Modeling is used across a number of tasks in connection to information systems, but it is rare to see and easily compare all these uses of diagrammatical models as knowledge representation in the same book, highlighting both commonalities and differences between the underlying principles of, e.g., enterprise modeling, process modeling, requirements modeling, and design modeling. What specifically distinguishes this book is that it looks across these various research domains, to provide an overview of existing approaches and best practices in these conceptually closely related fields.

vii

The chapters in this book are not traditionally covered by traditional textbooks in analysis and design modeling. Textbooks on more advanced modelingoriented topics are mainly focusing on the favorite techniques of the authors, and they often present little coverage on formal evaluation of different modeling techniques. Some books present briefly a number of techniques for a specific domain, but seldom look across domains to see how modeling as a generic technique can be generally applied, and which aspects need to be specifically tailored to the modeling task at hand. The chapters are arranged within three sections: General Techniques for Comparing and Adapting Modeling Methods; Goal, Requirements, and Process Modeling; and Data, Ontology, and Component Modeling.

OVERVIEW OF CHAPTERS Section I focuses on general ways of comparing and extending modeling approaches, including the use of meta-modeling, assessment of quality of models and modeling languages, and method engineering. In the chapter “Analyzing and Comparing Ontologies with Meta-Models,” by Islay Davies, Peter Green, Simon Milton, and Michael Rosemann, the authors propose the use of metamodels for analyzing, comparing, and engineering ontologies. High-level ontologies provide a model of reality and are of increasing popularity for the evaluation of modeling methods. The chapter discusses a methodology using extracts of meta-models for two well-known ontologies that had been used previously in systems analysis and design research. The approach provides a theoretical analysis technique for evaluating these ontologies according to their equivalence, depth of structure, and comprehensiveness of scope. The focus in Chapter II, “Two Meta-Models for Object-Role Modeling,” by Dave Cuyler and Terry Halpin, is also the use of meta-modeling techniques. Although object-role modeling (ORM) has been used for three decades and now has industrial modeling tool support, it has no official, standard meta-model. Because of its extensive capability for expressing business rules, ORM is currently being considered as a possible standard for business rules expression within the object management group (OMG), and for use in ontology standards. To support these initiatives, and the interchange of ORM model data between different software tools, this chapter discusses recent research by the authors to pave the way for a standard ORM meta-model that employs a standard interchange format. Bart-Jan Hommes similarly applies meta-modeling techniques in Chapter III, “Evaluating Conceptual Coherence in Multi-Modeling Techniques.” A metamodel can serve as a basis for quantitative evaluation of methods and techniques. By means of a number of formal metrics based on the meta-model, a quantitative evaluation of methods and techniques becomes possible. Existing meta-modeling languages and measurement schemes do not allow the explicit

viii

modeling of so-called multi-modeling techniques. Multi-modeling techniques are techniques that offer a coherent set of aspect modeling techniques to model different aspects of a certain phenomenon. As a consequence, existing approaches lack metrics to quantitatively assess aspects that are particular to multi-modeling techniques. In this chapter, a modeling language for modeling multi-modeling techniques is proposed, as well as metrics for evaluating the coherent set of aspect modeling techniques that constitute the multi-modeling technique. In Chapter IV, “Assessing Enterprise Modeling Languages Using a Generic Quality Framework,” by John Krogstie and Sofie de Flon Arnesen, an even broader framework of criteria for the goodness of a modeling language is presented and used for the evaluation in a practical setting. The users of the framework have recently wanted to standardize an enterprise modeling language for process modeling for sense-making and communication. To perform the evaluation, a generic framework for assessing the quality of models and modeling languages was specialized to the needs of the company. The work illustrates the practical utility of the overall framework, where language quality features are looked upon as means to enable the creation of models of high quality. It also illustrates the need for specializing this kind of general framework based on the requirements of the specific organization. The last chapter in Section I is titled, “An Approach for Evolution-Driven Method Engineering,” by Jolita Ralyté, Colette Rolland, and Mohamed Ben Ayed. The chapter considers the development of new methods, particularly the evolutionary perspective of method engineering. It presents an approach for method engineering supporting evolution of an existing method, model, or metamodel into a new one satisfying a different engineering objective. This approach proposes several different strategies to evolve from the initial methodology to a new one and provides guidelines supporting these strategies. The approach has been evaluated in the research project around the Lyee methodology. Section II looks at methods and methodologies for behavioral modeling in a broad sense, including both goal-oriented modeling, requirements modeling, and process modeling In the field of requirements engineering, goal modeling approaches have received much attention in recent years by researchers and practitioners alike. In Chapter VI, “Goal Modeling in Requirements Engineering: Analysis and Critique of Current Methods,” Evangelia Kavakli and Pericles Loucopoulos identify the uses of these approaches in different contexts of requirements analysis phases. It provides an analysis of goal modeling approaches in a systematic and consistent manner. The aim of this analysis is to understand the best fit for purpose of different goal modeling approaches and to highlight open issues that provide a foundation for further research in this important area of requirements engineering methodology.

ix

Chapter VII, “An Empirical Investigation of Requirements Specification Languages: Detecting Defects While Formalizing Requirements,” by Erik Kamsties, Antje von Knethen, Jan Philipps, and Bernhard Schätz, presents an empirical study of requirements specification languages, in which two research questions are addressed: Which types of defects are detected by a requirements engineer during formalization? Which types of defects go undetected and what happens to those types in a formal specification? The results suggest looking explicitly for ambiguities during formalization, because they are less frequently detected than other types of defects. If they are detected, they require immediate clarification by the requirements author. Automated support for the requirements engineering process is a recognized research area. However, the mainstream practice still relies on word processors and drawing tools rather than the requirements engineering tools (RETs). The aim of Chapter VIII, “Validating an Evaluation Framework for Requirements Engineering Tools,” by Raimundas Matuleviè ius, is to validate an evaluation framework for RETs. The validation process concerns an RET acquisition process for concrete organizational needs. An observation of maintaining requirements specification shows the important organizational and environmental characteristics for a proper automated support of RE process. The contribution of this work is twofold: first, the validation of the evaluation framework for RETs according to environmental needs in a specific environment, and second, the identification of environmental needs, which emerge from the requirements specification maintenance process. Process modeling is often used for analysis purposes. In Chapter IX, “A Comparison of the FOOM and OPM Methodologies for User Comprehension of Analysis Specifications,” by Judith Kabeli and Peretz Shoval, two approaches to integrated process and OO-modeling is compared. FOOM (functional and object-oriented methodology) and OPM (object-processes methodology) are methodologies used for analyzing and designing information systems. In this study, the authors compare FOOM and OPM from the point of view of both user comprehension of analysis specifications and user preference of specifications. The comparison is based on a controlled experiment that measured: (a) comprehension of the analysis specifications, which includes both structural and behavioral aspects of the system; (b) the time it takes to complete the task of specification comprehension; and (c) the user’s preference of models. The last chapter in Section II follows up on process modeling in practice. In Chapter X, “Participatory Development of Enterprise Process Models,” Reidar Gjersvik, John Krogstie, and Asbjørn Følstad present practical experience from using a technique called Modeling Conference, a method for participatory construction and development of enterprise process models. The Modeling Conference method focuses on broad participation from all actors in the organization, is grounded in a social constructivist perspective, and has its theoretical basis in the method of search conferences and process modeling. In an engi-

x

neering consultancy firm, the Modeling Conference method has been used to develop process models for main common work tasks that have been implemented on an intranet. Independent evaluations show that participation through the Modeling Conference led to significantly more ownership to the process models, and that the actors developed new collective knowledge. Section III looks at techniques for structural modeling, both on an analysis and design level. This includes looking at various data and class-oriented modeling approaches, modeling of ontologies, and modeling of system components. Discovering a set of domain classes during object-oriented analysis is intellectually challenging and time consuming for novice analyzers. In Chapter XI, “A Taxonomic Class Modeling Methodology for Object-Oriented Analysis,” Il-Yeol Song, Kurt Yano, Juan Trujillo, and Sergio Luján-Mora present a taxonomic class modeling (TCM) methodology that can be used for object-oriented analysis in business applications that helps to discover three types of classes: (1) classes represented by nouns in the requirement specification, (2) classes whose concepts were represented by verb phrases, and (3) hidden classes that were not explicitly stated in the requirement specification. The framework integrates the noun analysis method, class categories, English sentence structures, checklists, and other heuristic rules for modeling. Data models of realistic domains often become large and difficult to work with. In Chapter XII, “Comprehension of Hierarchical ER Diagrams Compared to Flat ER Diagrams,” by Revital Danoch, Peretz Shoval, and Mira Balabaan, HERD — a semi-algorithmic method for creating hierarchical ER diagrams from bottom up — is presented. The method is based on packaging operations that are applied in several steps on a given flat ER diagram. The result is a hierarchy of simple and interrelated diagrams (namely ER structures) with external relationships to other such diagrams. The chapter describes the application of HERD method using an example from a hospital domain, and an experiment in which the authors compare the comprehension of HERD diagrams with flat ER diagrams. To ensure that a software system accurately reflects the business domain that it models, the system needs to enforce the business rules (constraints and derivation rules) that apply to that domain. From a conceptual modeling perspective, many application domains involve constraints over one or more conceptual schema paths that include one or more conceptual joins (where the same conceptual object plays roles in two relationships). In Chapter XIII, “Constraints on Conceptual Join Paths,” Terry Halpin contrasts how these join constraints are catered for in object-role modeling (ORM), the unified modeling language (UML), the object-oriented systems model (OSM), and some popular versions of entity-relationship (ER) modeling (ER). Three main problems for rich support for join constraints are identified: disambiguation of schema paths, disambiguation of join types, and mapping of join constraints to implementation code. To address these problems, some notational, meta-model, and mapping extensions are proposed.

xi

Another type of representing structural aspects are the modeling of ontologies. In Chapter XIV, “Using a Semiotic Framework for a Comparative Study of Ontology Languages and Tools,” Xiaomeng Su and Lars Ilebrekke survey and compare different ontology languages and tools by the aid of an evaluation framework (the same framework that was used in Chapter IV). An ontology must be of high quality to enable actors to reach a common understanding of the domain at hand. The notion of “quality” in the context of ontology is discussed, and means to achieve high-quality ontologies are listed. The different quality aspects and means to improve them formulate the template for the comparisons of ontology languages and tools, which are two of the major factors that affect the quality of ontologies. Component-based development (CBD) has received a lot of attention in software engineering literature over the last few years. Awareness has been raised that CBD is the way to go in software development, especially in the domain of e-business, where the benefits of reusing components, i.e., faster time-to-market and quality, are essential. In Chapter XV, “A Service-Oriented Component Modeling Approach,” Zoran Stojanovic, Ajantha Dahanayake, and Henk Sol present a service-oriented component modeling approach focused on the concepts of component and service as the main modeling and design artifacts. In the final chapter, “Evaluation of Component-Based Development Methods,” Nicky Boertien, Maarten W.A. Steen, and Henk Jonkers present an evaluation of five popular methods for component-based development — including Catalysis, the Rational Unified Process, and Select Perspective — on their maturity and fitness-for-use in the context of e-business engineering. The evaluation is done based on their own reference framework for e-business development and a list of objective criteria. The methods each emphasize certain aspects of CBD, but as yet none of them offers a complete solution. The above collection of chapters provides a good mix of applications of modeling techniques across a number of knowledge representation problems within information system analysis and design. The goodness of a modeling technique or language is often highlighted by the proponents of the technique with little substantial evidence. In addition to give an overview of the current state of the art and state of practice within information modeling methods and methodologies, the book focuses on our knowledge of this area through empirical and analytical evaluations of techniques used in practice.

xii

Acknowledgments

The editors would like to acknowledge the help of all involved in the collation and review process of the book, without whose support the project could not have been satisfactorily completed. A further special note of thanks goes also to all the staff at Idea Group Inc., whose contributions throughout the whole process, from inception of the initial idea to final publication, have been invaluable. In closing, I wish to thank all of the authors for their insights and excellent contributions to this book. I also want to thank all of the people who assisted me in the reviewing process. On behalf of the editors, John Krogstie Oslo, Norway December 2003

Section I General Techniques for Comparing and Adapting Modeling Methods

Analyzing and Comparing Ontologies with Meta-Models

1

Chapter I

Analyzing and Comparing Ontologies with Meta-Models Islay Davies, Queensland University of Technology, Australia Peter Green, University of Queensland, Australia Simon Milton, University of Melbourne, Australia Michael Rosemann, Queensland University of Technology, Australia

ABSTRACT High-level ontologies provide a model of reality and are of increasing popularity for the evaluation of modeling methods. Most of the common modeling methods have been studied using ontologies such as the BWW representation model and Chisholm’s ontology. However, at this stage only limited guidance is provided for the selection and evaluation of the appropriate ontology. The aim of this chapter is to propose meta-models for analyzing, comparing, and engineering ontologies. It discusses a methodology using extracts of meta-models for two well-known ontologies that had been used previously in Systems Analysis and Design research. The approach provides a theoretical analysis technique for evaluating these ontologies according to their equivalence, depth of structure, and comprehensiveness of scope. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

2 Davies, Green, Milton, & Rosemann

INTRODUCTION Wand and Weber (2002) have speculated on a research agenda for information systems, as well as Systems Analysis and Design. Their objective is to motivate research that addresses the fundamental question, “How can we model the world better to facilitate our developing, implementing, using, and maintaining more valuable information systems?” Using a theoretical foundation based on ontology could facilitate many of the potential research areas that they identify. Ontology has influenced research in many application areas over the past decade: knowledge representation, natural language processing, knowledge management, enterprise systems, Systems Analysis and Design, and Web services. Ontologies have been extremely popular for the evaluation of modeling methods. Given the important use and potential use of ontologies over the past 10 years, the principal question then becomes: Which ontologies do we use for which purposes? How do we compare and evaluate different ontologies for determining their strengths and weaknesses for the purpose required? The objective of this chapter is to demonstrate the usefulness of meta-models in supporting research opportunities in conceptual modeling that are influenced by ontologies. In particular, our aim is to show the importance of meta-models for selecting, comparing, and evaluating ontologies. In this way, researchers may gain some guidance on which ontology might be useful for their area of interest. A detailed discussion of dealing with semantic and structured diversity in representations is beyond the scope of this research. We are motivated to perform this work for three reasons. First, we can provide practical guidance to researchers and practitioners alike on how to compare and evaluate ontologies. In this way, they will be better able to determine the ontology most applicable for their purposes. Second, we are extending the usefulness of meta-models from understanding individual modeling techniques like process modeling and workflows, to understanding and comparing the theoretical bases (ontologies) on which those techniques can be compared and evaluated. In this way, we are extending the work of Rosemann and Green (2002) where they demonstrated how, through comparing the meta-model of an ontology to that of a modeling technique like Architecture of Integrated Information Systems (ARIS), and using a pattern-matching process, they could evaluate the strengths and weaknesses of the process modeling technique. Finally, we are explaining, and demonstrating using a limited example, how ontologies and meta-models can be very useful in conducting research in many of the areas of conceptual modeling identified by Wand and Weber (2002). Accordingly, the chapter unfolds in the following manner. The next section explains what ontology is and it exemplifies it in the form of the BWW representation model. Moreover, this section introduces the research framework for work in the information systems and conceptual modeling discipline Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


3

presented by Wand and Weber (2002), and it provides an assessment of the usefulness of ontologies to each of those areas. The third section explains what meta-models are, where they have been applied previously, and, in relation to the Wand and Weber (2002) research areas, it assesses their usefulness. The next section presents a section of the meta-model for two popular ontologies — the BWW representation model (Weber, 1997) and Chisholm’s (1976) ontology. It provides guidelines for comparing and evaluating meta-models generally, and then it demonstrates the application of these analytical processes to the two small example meta-models. Differences are highlighted under the categories of ontological equivalence, depth of structure, and comprehensiveness of scope. The chapter concludes with a summary of results and an indication of further work planned.

CHARACTERISTICS AND APPLICATIONS OF ONTOLOGIES What is Ontology? Ontology is a well-established theoretical domain within philosophy dealing with models of reality. Unfortunately, as with most areas of scientific endeavor, over the years many different models of reality — ontologies — have emerged (c.f., Bunge, 1977; Chisholm, 1996; Husserl, 1934). Mylopoulos (1998) suggests that ontologies can be classified into four categories: static, dynamic, intentional, and social. Each of these categories focuses on different concepts in the real world. Ontologies that fall into the static category focus on things and their properties. Dynamic ontologies extend static ontologies to focus on such concepts as events and processes — that is, how concepts in the real world change over time. Intentional ontologies attempt to explain abstract concepts like goals and objectives, while social ontologies emphasize the concepts of values and beliefs. Today, however, interest in and applicability of ontologies extends to areas far beyond metaphysics. As Gruninger and Lee (2002, p. 39) point out: “… a Web search engine will return over 64,000 pages given ‘ontology’ as a keyword…the first few pages are phrases such as ‘enabling virtual business,’ ‘gene ontology consortium,’ and ‘enterprise ontology.’” The usefulness of ontology as a theoretical foundation for knowledge representation and natural language processing is a fervently debated topic at the present time in the artificial intelligence research community (Guarino & Welty, 2002). Holsapple and Joshi (2002), for example, argue the importance of ontologies in the emergent era of knowledgebased organizations and the conduct of knowledge management in those organizations. Kim (2002) shows how ontologies can be engineered to support the first phase of the evolution of the “Semantic Web.” Many of these ontologies are domain specific. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


Two general ontologies have been frequently applied for the evaluation of modeling methods in Systems Analysis and Design. Firstly, much work has focused on a set of ontological models known as the BWW (Bunge-WandWeber) models. Weber (1997) has taken, and extended, an ontology presented by Bunge (1977) and applied it to the modeling of information systems. Their fundamental premise is that any Systems Analysis and Design modeling grammar (set of modeling symbols and their construction rules) must be able to represent all things in the real world that might be of interest to users of information systems; otherwise, the resultant model is incomplete. If the model is incomplete, the analyst/designer will somehow have to augment the model(s) to ensure that the final computerized information system adequately reflects that portion of the real world it is intended to simulate. The BWW models consist of the representation model, the state-tracking model, and the good decomposition model. The work reported in this chapter uses the representation model and its constructs. The representation model defines a set of constructs that, at this time, are thought to be necessary and sufficient to describe the structure and behavior of the real world. A detailed description of all the constructs in the representation model can be found in Green and Rosemann (2000) and Weber (1997). Secondly, Chisholm’s ontology (Chisholm, 1976) has been used to evaluate a representative range of data modeling languages (grammars) with a view to gain insight into those languages (Milton et al., 2001). Chisholm’s ontology contains all necessary elements in one model to represent reality in its facets. In this chapter we concentrate on those elements of Chisholm’s ontology that most closely related to BWW’s representation model. We take these two ontologies, as they are currently defined and perceive the actual evaluation of the completeness and overall quality of these ontologies as outside the scope of this chapter.

Ontologies and Application Areas Wand and Weber (2002) have speculated on a research agenda for information systems and Systems Analysis and Design. In particular, with regard to research on conceptual modeling, they suggest that four elements can be used to structure a framework for the research: conceptual modeling grammar — a set of constructs and their construction rules, conceptual modeling method — a procedure by which the grammar can be used, conceptual modeling script — the product of the conceptual modeling method, and context — the setting in which the modeling occurs. They perceive the highest usefulness of ontologies in the area of conceptual modeling grammars, i.e., the evaluation of ontologies, the evaluation of grammars, assigning real-world semantics to grammars, better use of ontologies, study of ontological issues, empirical testing of theoretical predictions and rules, use of multiple grammars, and implications of grammar deficiencies. Ontologies also appear to be of some actual and potential use in the research areas specified for conceptual modeling scripts and individual contextual factors. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


5

However, it would appear that ontologies have limited actual (and potential) use for research in social agenda factors. Given the above areas of use and potential use of ontologies, which ontologies do we use in which research opportunity? How do we compare and evaluate different ontologies for determining their strengths and weaknesses for the purpose required in the particular research area? The next section of the chapter proposes the use of meta-models of various ontologies for this comparison and evaluation task.

BENEFITS OF META-MODELS FOR ONTOLOGIES Meta modeling is an attempt to adequately model all aspects of any given modeling technique (Smith, 1995). A model is a representation of a relevant part of the real world and is created for the purpose(s) of a subject. On a higher level such a model can also be described in models. These models are called metamodels and specified in meta languages, i.e., a language used for the design of the meta-model. Meta-models have proven popular in explaining and communicating the constructs of some modern modeling techniques, for example, workflow models (Rosemann & zur Mühlen, 1998), object-oriented schemas (Saeki, 1995), and ontologies (Rosemann & Green, 2002). In relation to ontologies, meta-models help to further structure, understand, and analyze an ontology. The appropriateness of the language in which a metamodel is described is of high importance in this context. This language is called the meta-language. It is a characteristic of most ontologies to have a very formalized specification, e.g., based on the Z-language. Though this provides a precise description of the ontology, such a specification has two shortcomings. First, it requires a deep understanding of the meta-language, as this forms a barrier to the application of the ontology. Second, it makes the application of ontologies in the area of conceptual modeling more difficult, as most of the modeling grammars are specified by using meta-languages such as ER-notation or UML. A specification of an ontology in a more widely accepted meta-language will have many benefits (Figure 1). 1. A specification of an ontology in a language that is widely used in the Systems Analysis and Design community facilitates the communication of the ontology. Knowledge in the meta-language would no longer be a critical bottleneck for the access to the ontology. It would be easier to teach ontologies and to distribute the ontology in the interested research community.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


2.

3.

4. 5.

6.

The process of designing the meta-model requires a detailed analysis of all elements, relationships, and attributes of the ontology. This can be regarded as a kind of completeness check and allows a further clarification of inconsistencies and anomalies in the targeted ontology. In particular, such a meta-model for an ontology would streamline the ontological analysis of modeling grammars as they facilitate a direct mapping of elements and relationships between the ontology and the modeling grammar (as long as there is a preexisting meta-model for the modeling grammar in the same meta-language). The meta-model can be used to derive new modeling grammars (that is, ontology-based method engineering). The comparison of ontologies is streamlined, if they are specified in the same meta language. Such an analysis would be a pattern matching exercise between the involved models. It would be obvious, where ontologies overlap, but also where they are more detailed. This is the focus of this chapter. Finally, it will be possible to use the meta-model to engineer new ontologies or re-engineer and further develop existing ontologies.

Figure 1. Application areas of a meta-model for an ontology

1a) Facilitates communication about the ontology 2) Clarifies inconsistencies and anomalies

1b) Simplifies teaching the ontology 5) Streamlines the comparison of ontologies

Q

Ontology B

Meta Model for Meta Model for ontological constructs ontological constructs

3) Streamlines the ontological analysis of grammars

Ontology C Grammar A

Grammar B

4) Enables ontology-based method engineering New Grammar

6)

Enables ontology engineering

New Ontology



7

The next section of the chapter shows, as an example, how meta-models can facilitate research into the comparison and evaluation of ontologies. The assumption of the presented research is that the selected ontologies are comparable in terms of their purpose and their underlying philosophical view of the world. This is a prerequisite before the presented methodology can be applied and requires substantial knowledge about the ontologies.

A META-MODEL-BASED APPROACH FOR THE COMPARISON OF ONTOLOGIES The meta-models in this chapter are designed using extended EntityRelationship models (ERMs) (Chen, 1976). This approach has been chosen because it is well grounded, comprehensively taught, and used as a metalanguage for many modeling grammars. Generalizations are established for reasons of clarity. For every generalization a description of the disjointness constraint (d (disjoint) or n (not disjoint)) and the completeness constraint (p (partial) or t (total)) is given (Elmasri & Navathe, 1995, p. 618). Prior to the evaluation of meta-models, it must be ensured that possible syntactical, semantic, and structural conflicts between the models such as naming conflict have been resolved (Rosemann & zur Mühlen, 1998). Various conflict-resolution strategies have been developed in the area of view integration. These conflicts have to be resolved before two or more meta-models of ontologies can be compared. At the same time, they highlight the need for sound modeling guidelines, covering, for example, naming and layout conventions. A separate repository has to capture all transformations that are made to one model in order to resolve these conflicts, e.g., renaming of an entity type in order to avoid synonyms. Corresponding meta-models are called conflict-free when these conflicts have been resolved. When two conflict-free meta-models are compared, the following information objects can provide useful information: • Entity Types. The comparison of the number and kind of entity types provides the most essential information for the comparison of meta-models. Within a given degree of abstraction, the width of an ontology increases with the number of entity types in the meta-model. • Relationship Types. Another metric concerning the integration within an ontology is the number of relation-ship types. The structural density of an ontology increases with the number of relationship types if the number and kind of entity types stay the same. An example can be found in the relationships between thing (individual) and property (attribute) shown in Figures 2 and 3.



Beyond entity and relationship types, the comparison of cardinalities and attributes typically provides further information. However, in the context of meta-models for ontologies, attributes are rather the exception. Thus, they are not further analyzed here. Independent from entity types and relationship types, three different situations can be distinguished when comparing conflict-free meta-models for ontologies. (a) Between two corresponding elements in two ontologies might be a 1-1 relationship. This case describes ontological equivalence. (b) It might also be the case that one element in an ontology is further specified by two or more elements in the other ontology. In that case, the other ontology has a deeper structure. A special case of this situation is the case in which one element in one ontology subsumes another corresponding element in the other ontology, i.e., a subsumption relationship between the two. (c) Finally, it might be the case that one element in one ontology does not have any correspondence in the other ontology at all. If it can be ensured that all heterogeneous representations, including semantic, syntactic, and structural diversities, have been fully analyzed, it can be stated that one ontology has a more comprehensive scope. The following section elaborates on these differences using the BungeWand-Weber model (Wand & Weber, 1997) and Chisholm’s ontology (as described in Milton & Kazmierczak, 1999). We have translated a portion of both the Bunge-Wand-Weber model and Chisholm’s ontology into ERM-based meta-models in order to clearly depict the key elements and constructs of each. Furthermore, by using a common metalanguage, we are able to easily compare the elements and constructs of each model. The names given to the entity and relationship types within each model closely follow the wording used by the original sources. In order to assure the correctness of the meta-model for the BWW ontology, we discussed the main parts of this model with Professor Ron Weber, one of the originators of the BWW models. It is impossible to describe each model completely in this chapter, although the key elements are covered for the purposes of this work. Interested readers are directed to more detailed discussion of both the BWW ontology (Bunge, 1977; Weber, 1997) and Chisholm’s ontology (Chisholm, 1996; Milton, 2000).

Description of the BWW Meta-Model The BWW (Bunge-Wand-Weber) model (Figure 2) is based on the fundamental elements of things, which exist in the real world, and their properties. Every thing in the real world possesses at least one property. Conversely, every Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


9

property belongs to at least one thing. The generalization symbol depicts things as being either composite or simple, the “d” symbolizes a disjoint constraint (one or the other) and the “t” symbolizes a total constraint (all subtypes which exist are depicted). A composite thing is associated with at least two other things (composite or simple), which combine to comprise the composite thing. A simple thing, on the other hand, is comprised of only itself. Properties can be further divided into sub-properties or types of properties. A disjoint, total constraint shows that a property can be either a property in general or a property in particular. Furthermore, a property in particular is an instance of a property in general. Other subtypes may be used to describe the characteristics of a property. An intrinsic property is inherent in an individual thing. A mutual property is a property that is shared by two or more things. Mutual properties can be either binding, affecting the things involved, or nonbinding, not affecting the things involved, for example, order relations or equivalence relations. A hereditary property is a property of a composite thing that belongs to a component thing. An emergent property is a property of a composite thing that does not belong to a component thing. The generalization constraint n,p used to categorize these subtypes describes them as being nondisjoint (a property may be characterized by more than one of these subtypes) and partial (there may be further subtypes not depicted in the model) (Rosemann & Green, 2002; Weber, 1997). We use attributes to name and represent the properties we wish to model. Often we can assign a single attribute to a particular property directly, however at times it is not so easy to determine or fully understand some properties. In this case we can model a combination of properties, of which we are aware but do not fully understand, via a single attribute (Weber, 1997). Figure 2. Meta-model of BWW ontological constructs associated into

2,n

Composite Thing d,t d,t

Simple Thing

Property in general

0,n

observed as

0,n

Thing

1,n

possesses

1,n

Property in particular

Property

0,1

0,n

modelled as

n,p

Intrinsic Property

Binding Mutual Property

1,n

Attribute Mutual Property

d,t Non-binding Mutual Property



Description of the Chisholm Meta-Model Chisholm’s ontology has at its core the elements of individuals, which are contingent entities, and the attributes they exemplify (exhibit) (Figure 3). Individuals are described as being transient objects that come into being and pass away. In other words they are created and destroyed, giving them a life-span. Furthermore, they need not be material or physical in nature. Individuals may also be structured into constituents or parts. A constituent is, in itself, an individual. Each individual is identified by one or more attributes that it also exemplifies. Individuals may also exemplify (or exhibit) several more attributes. Some attributes, on the other hand, might never be exemplified. In contrast to the transient nature of individuals, attributes are enduring. In other words they do not come into being or pass away. Rather they continually exist. For many of the reasons above, attributes are loosely coupled with individuals. Attributes are also described as being either compound or simple. The meta-model shows the constraints of this attribute generalization as being disjoint (one or the other) and total (all subtypes which exist are depicted). Chisholm is not prescriptive about how this coverage is provided and a number of mechanisms would suffice. Chisholm gives one where a compound attribute results from the conjunction or disjunction of several other compound or simple attributes. Two or more attributes may share the relationship of conceptual entailment. This is a construct that infers when an individual exhibits one attribute, it necessarily exhibits the other, making two or more attributes equivalent. Additional to the core elements of individuals and attributes, the elements of relations and classes/sets also play a prominent role in the ontology. Individuals are related to others through two mechanisms. Additional to constituents of individuals (described above), individuals may have transitory relations to other individuals that are not constituents. Relations of this sort require that each

Figure 3. Meta-model of Chisholm’s ontology 1,1

1,n

Constituent

is an

has 0,n

0,n 1,n

Individual

consists of identified by

0,n

0,n

0,n

exemplifies

0,n

0,n

d,t

Attribute 0,n

conceptual entailment

Compound

0,n

consists of

0,n

Simple

0,n



11

individual involved in the relation be identified by a unique attribute (that could be compound) so that they can be individuated in the relation. Each relation is an ordered pair of these individuating attributes and is unidirectional. Finally, classes and sets are defined by identifying attributes that typify the class or set. Those attributes then allow for selecting members for a specific state of affairs.

Comparison of the BWW and Chisholm Ontologies Before comparing the two meta-models, we need to resolve any naming, type, or structural conflicts between them, as discussed earlier. When developing each meta-model, it was decided to name each element with the wording used by the source of the models as closely as possible. As a result, we need to identify some synonyms and homonyms between the models. Firstly, we can establish that thing and property in the BWW model are essentially the same concepts as individual and attribute in Chisholm’s model. The names expressing the relationship between these two entities, possesses and exemplifies, are also considered synonyms. One homonym that can be identified is attribute. Attribute is described in both models, but has a slightly different meaning in each. In the BWW model, attribute is described as being the name we use to model one or more properties. In Chisholm’s model, on the other hand, attributes directly identify what the BWW model calls properties. The same group of people developed the two meta-models, therefore there are no type or structural conflicts to resolve. Comparisons can be made by focusing on the difference between the number and nature of the entities described in each of the models, the number and nature of the relationships between comparable entities, and the cardinality of the comparable entity relationships. Comparing the two models using the three defined situations results in the following analysis: (a) Ontological Equivalence. Ontological equivalence can be established between a number of constructs in the BWW and Chisholm models. Comparing in the direction from the BWW model to the Chisholm model, we assert that thing is essentially equivalent to individual, property is equivalent to attribute, and possess is equivalent to exemplifies. However, when taking cardinality into account, a difference between property in the BWW model and attribute in the Chisholm model can be seen. The BWW model stresses that properties can only exist with things. Chisholm’s ontology on the other hand asserts that attributes (properties in the BWW model) are enduring and can exist even if not exemplified by any particular individual (Milton & Kazmierczak, 1999). Attribute in Chisholm’s ontology also assumes a greater scope than that from BWW when one considers the temporally oriented nature of attributes necessitated by other aspects of Chisholm’s ontology that are not part of this chapter.



(b) Deeper Structure. The BWW model takes the concept of thing further by breaking it down by way of generalization to being either a composite thing or a simple thing. This situation might suggest a deeper structure to the BWW model. However, such a structure is implicit in Chisholm’s ontology whereby a simple individual is one that has no constituents. This fact could be explicitly presented in the model. Furthermore, property in the BWW model is generalized into numerous subtypes, each categorizing a property in a particular way. In a slightly different way, attribute in the Chisholm model is also broken down showing structure. Chisholm structures attributes into compound and simple classifications to enable different levels of expressiveness. This situation may imply that Chisholm’s ontology also has a deep structure. Another way in which the Chisholm model could be viewed as having a deep structure is the further clarification of the relationship between individual and attribute, one relationship being identified by and the other being exemplified. The BWW model relates thing and properties via only one relationship — possesses. Milton and Kazmierczak (1999) suggest that Chisholm views attributes as being fundamental to his ontology, second only to individuals, which may explain his efforts in further structuring the attribute element and distinguishing further relationships. The number of relationships associated with attribute and the level of effort to describe the construct illustrate this view. Attributes are further used in Chisholm’s ontology to describe sets, classes, and relations. However, many other elements in Chisholm’s ontology do not appear to be broken down to the same extent as the attribute element. (c) More Comprehensive Scope. It is difficult to illustrate comparisons between the scope of each model, considering the limited portion of each model chosen and depicted in this chapter. Within the restricted boundaries of the models represented, however, there is a relationship— conceptual entailment — described in the Chisholm model which is not apparent in the BWW model. This additional element could be grounds to argue that the Chisholm model is more comprehensive in scope. However, when we look at both models in their entirety, we can see that the BWW model appears to describe more constructs than Chisholm’s ontology as a whole. Green and Rosemann (2000) identify 28 main constructs in the BWW model, whereas Milton and Kazmierczak (1999) identify only 12 categories described in Chisholm’s ontology, and with several other terms defined and mapped back to fundamental categories. The following table shows a part of this comparison. Relation and class/set are examples relevant to this chapter. This finding suggests that the BWW model is in fact more comprehensive in scope. It could also suggest that BWW is more detailed in its conceptualization, whereas Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


13

Table 1. Comparison of BWW and Chisholm Information Object Entity Type:

BWW

Thing Property Attribute Property subtypes: - in General - in Particular - Intrinsic - Mutual - Binding Mutual - Non-binding Mutual - Hereditary - Emergent

TOTAL Entities

11

Chisholm

Equivalent?

Individual Attribute

Yes Yes No No

Attribute subtypes: - compound - simple 4

No

2

Chisholm’s ontology is comparatively terse with respect to BWW, with Chisholm’s descriptive power and comprehensiveness being hidden in concepts such as those concerning attribute, event, and state. This is a potential focus for future research (Milton, 2000). We will also aim for more advanced measures. Previous research on complexity measures (Rossi & Brinkkemper, 1994; Siau & Cao, 2001) might provide a promising starting point.

Issues to Consider with this Method of Comparison Having employed the above method to compare two ontological metamodels, some issues that may carry implications for further comparison activities involving meta-models for ontologies have become apparent. 1. To ascertain a comparison of ontological equivalence, it may be more beneficial to establish one set of naming conventions for each comparable element in the two ontological models. Although we considered naming conflicts and resolved them through descriptive comparison, it may be easier to refer to the same concept in each model in subsequent sections of a paper using one common term for each concept. For example, we could decide at the outset to label the equivalent construct of thing (in BWW model) and individual (in Chisholm model) as thing, when referring to this concept in further comparison discussions. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


2.

3.

Deeper structure may be difficult to ascertain when each model has a different focus. For example, in Model A, a particular element may be described in more detail than in Model B, while in Model B a different but still comparable element may be described in more detail than in Model A. A comparison of the deeper structure of models, therefore, needs to be performed on the models as a whole, following a more complex form of analysis. This is compounded when one ontology presents all major concepts in one meta-model, where the other uses three different metamodels to present the same. For a similar reason as in point #2, the entire models, rather than a chosen portion, need to be compared in order to establish an accurate comparison of the comprehensiveness of scope of the models.

CONCLUSIONS The findings presented in this chapter need to be placed into the context of the philosophical heritage of the ontologies used in this study. Firstly, both BWW and Chisholm’s ontologies take a realistic position. Chisholm’s ontology is one of common sense realism and is designed to account for both science (where applicable), but more importantly for human relationship with and understanding of reality. Moreover, its allowance for science must be general enough to be robust in the light of paradigm shifts such as the post-Newtonian world ushered in by Einstein. BWW in contrast is a realism that tends towards scientific realism with a fine-grained systematic view of reality and one that is in need of revision as our fundamental scientific understanding of reality changes. We have reason to believe that BWW is more comprehensive in scope than Chisholm and that Chisholm has a deeper structure than BWW. This may be explained by the differences between scientific realism and common sense realism, and by their mutual relationship. Scientific realism requires a deep and detailed theoretical understanding of reality, as contemporary science permits. BWW is a good example of that. In contrast, common sense realism must simultaneously allow for a rigorous but scientifically naïve or human understanding of reality, and have mechanisms for ‘drilling down’ to a detailed, scientifically rigorous understanding of reality. Consequently, one would expect the ontology that belongs to scientific realism to be more comprehensive than that belonging to common sense realism (Smith, 1995). Further we would also expect that aspects of the common sense realism would encompass elements that are outside the scope of scientific realism and instead are within a naïve or human understanding of reality. This study suggests this is the case for these ontologies. It remains for future research to make more concrete the relationship between BWW and Chisholm’s ontology.



15

These findings must be viewed mindful of the limitation that only portions of the meta-model of the full BWW representation model and Chisholm’s ontology were used in this example. Further work will entail completing a meta-model for the full Chisholm ontology (as it is relevant to information systems research) and then using the comparison mechanism described, performing a full evaluation of the two ontological models. Future work could potentially explore other metamodeling techniques and further ontologies. Another direction of related future work would be to investigate empirical insights of the quality of the proposed methodology. Moreover, we will pursue our work, investigating the usefulness of meta-models of ontologies in the other relevant areas of research opportunity described by Wand and Weber (1997).

REFERENCES Batini, C., & Lenzerini, M. (1984). A methodology for data schema integration in the entity-rela-tionship model. IEEE Transactions on Software Engineering, 10(6), 650-664. Batini, C., Ceri, S., & Navathe, S.B. (1992). Conceptual database design: An entity-relationship approach. Redwood City: Benjamin/Cummings Publishing. Bunge, M. (1977). Treatise on basic philosophy: Volume 3: Ontology 1: The furniture of the world. Boston, MA: Reidel. Chen, P.P.-S. (1976). The entity-relationship model: Toward a unified view of data. ACM Transactions on Database Systems, 1(1), 9-36. Chisholm, R. (1996). A realistic theory of categories—an essay on ontology. Cambridge University Press. Chisholm, R.M. (1976). Person and object: A metaphysical study. London: G. Allen & Unwin. Elmasri, R., & Navathe, S.B. (1994). Fundamentals of database systems (2nd ed.). Redwood City: Benjamin/Cummings Publishing. Green, P., & Rosemann, M. (2000). Integrated process modeling: An ontological evaluation. Information Systems, 25(2), 73-87. Gruninger, M., & Lee, J. (2002). Ontology: Applications and design. Communications of the ACM, 45(2), 39-41. Guarino, N., & Welty, C. (2002). Evaluating ontological decisions with OntoClean. Communications of the ACM, 45(2), 61-65. Holsapple, C.W., & Joshi, K.D. (2002). A collaborative approach to ontology design. Communications of the ACM, 45(2), 42-47. Husserl, E. (1934). Lebenslauf. In A.D. Osborn (Ed.), The philosophy of Edmund Husserl: In its development from his mathematical interests to his first conception of phenomenology in logical investigations (p. 110). New York: International Press. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


Kim, H. (2002). Predicting how ontologies for the semantic Web will evolve. Communications of the ACM, 45(2), 48-54. Milton, S., & Kazmierczak, E. (1999). Enriching the ontological foundations of modeling in information systems. In CNG Dampney (Ed.), Proceedings of the Information Systems Foundations Workshop—Ontology, Semiotics and Practice (pp. 55-65). Sydney: Department of Computing, Macquarie University. Milton, S.K. (2000). Ontological studies of data modeling languages. Unpublished PhD Dissertation, The University of Tasmania, Australia. Milton, S.K., Kazmierczak, E., & Keen, C. (2001). Data modeling languages: An ontological study. Proceedings of the 9th European Conference on Information Systems (volume 1, pp. 304-315). Bled, Slovenia. Mylopoulos, J. (1998). Information modeling in the time of the revolution. Information Systems, 23(3-4), 127-155. Rosemann, M., & Green, P. (2002). Developing a meta-model for the BungeWand-Weber ontological constructs. Information Systems, 27(2), 75-91. Rosemann, M., & zur Mühlen, M. (1998). Evaluation of workflow management systems—A meta-model approach. Australian Journal of Information Systems, 6(1), 103-116. Rossi, M., & Brinkkemper, S. (1994). Complexity metrics for systems development methods and techniques. Information Systems, 21(2), 209-227. Saeki, M. (1995). Object-oriented meta modeling. In M.P. Papazoglou (Ed.), Proceedings of the 14th International Conference on Object-Oriented and Entity-Relationship Modeling, LNCS 1021, (pp. 250-259). Gold Coast: Springer. Siau, K., & Cao, Q. (2001). Unified Modeling Language (UML)—A complexity analysis. Database Management, 12(1), 26-34. Smith, B. (1995). Formal ontology, commonsense and cognitive science. International Journal of Human Computer Studies, 43(12), 641-667. Steele, P.M., & Zaslavsky, A.B. (1993). The role of meta-models in federating system modeling techniques. In R.A. Elmasri, V. Kouramajian, & B. Thalheim (Eds.), Proceedings of the 12th International Conference on the Entity-Relationship Approach—ER ‘93, LNCS 823, (pp. 315326). Arlington: Springer. Wand, Y., & Weber, R. (2002). Information systems and conceptual modeling: A research agenda. Information Systems Research, 13(4), 363-378. Weber, R. (1997). Ontological foundations of information systems. Coopers and Lybrand Accounting Research Methodology, Monograph No. 4, Melbourne, Australia.


Two Meta-Models for Object-Role Modeling

17

Chapter II

Two Meta-Models for Object-Role Modeling Dave Cuyler, Sandia National Laboratories, USA Terry Halpin, Northface University, USA

ABSTRACT For conceptual information analysis, the object-role modeling (ORM) approach is arguably more suitable than entity-relationship modeling and the class modeling technique within the Unified Modeling Language. Although ORM has been used for three decades and now has industrial modeling tool support, it has no official, standard meta-model. Because of its extensive capability for expressing business rules, ORM is currently being considered as a possible standard for business rules expression within the Object Management Group (OMG), and for use in ontology standards. To support these initiatives and the interchange of ORM model data between different software tools, this chapter discusses recent research by the authors to pave the way for a standard ORM meta-model that employs a standard interchange format. Two different ways of meta-modeling ORM features are presented, design trade-offs are examined, and extensions to ORM are proposed. As proof of concept, a working prototype that is compliant with the OMG’s Meta-Object Facility is also discussed. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

18 Cuyler & Halpin

INTRODUCTION The analysis phase of information systems development employs a conceptual schema of the application domain to enable the information structures and business rules to be easily validated by domain experts. A validated conceptual schema can then be mapped to logical/physical/external schemas using automated and/or manual processes. For the persistent (database) and transient (inmemory) structures, the high-level data modeling is often performed using entityrelationship (ER) modeling (Chen, 1976) and the Unified Modeling Language (UML) (OMG UML RTF, 2003), respectively. Recently, the fact-oriented approach exemplified by object-role modeling (ORM) (Halpin, 1998) is becoming a popular alternative to ER and UML at the conceptual analysis phase. Because of its attribute-free approach, ORM models are inherently more stable than ER models or UML class models. The ORM graphical language can formally capture many more business rules than the diagram notations supported by industrial ER tools or UML. ORM textual languages enable business rules to be expressed naturally using mixfix predicates of any arity (unary, binary, ternary, etc.). Moreover, ORM’s constraint primitives are orthogonal, and they work properly with n-ary associations. In contrast, industrial ER is confined to binary associations, while UML forbids unary associations, fails to support value-based identification schemes, and its main constraint primitive (multiplicity) fails to scale properly to n-ary associations (Halpin, 2001a). Moreover, ORM models can be automatically transformed to ER and UML class models when desired. For such reasons, ORM is being considered as a possible standard for business rules expression at the computation independent model (CIM) level within the Business Rules Special Interest Group recently formed within the Object Management Group (OMG). In addition, ORM is being investigated as a possible foundation for ontology specification (Demey, Jarrar, & Meersman, 2002; Spyns, Meersman, & Jarrar, 2002). Although ORM has been used productively in industry for three decades and now has industrial modeling tool support, it has no official, standard meta-model. If ORM is to be used as a standard for business rule or ontology specification, a standard meta-model for ORM is essential. Such a meta-model would also facilitate the interchange of ORM model data between different software tools. Historically, the fact-oriented modeling approach exemplified by ORM has been known under different names, including Natural Information Analysis Method (NIAM) (Wintraecken, 1990), Predicator-Set Model (PSM) (ter Hofstede, Proper, & Weide, 1993), Natural Object Relationship Method (NORM) (De Troyer & Meersman, 1995), and Fully Communication Oriented Information Modeling (FCO-IM) (Bakema, Zwart, & van der Lek, 1994). Although many publications exist on the fact-oriented approach, to our knowledge the only



19

significant ORM meta-models that exist in the public literature are works by the current authors (Cuyler, 2000; Halpin, 2000, 2002a). This chapter discusses more recent research by the authors to pave the way for a standard ORM meta-model. Different ways of meta-modeling ORM features are presented, design trade-offs are examined, and extensions to ORM are proposed. The next section discusses how ORM object types, roles, and predicates may be meta-modeled in ORM, including component reuse. The section after that examines ways to meta-model business rules in ORM. We then show how to meta-model instance data in ORM. As proof of concept, we then discuss a prototype of an ORM meta-model that is compliant with the OMG’s Meta-Object Facility. The conclusion summarizes the main results, suggests topics for future research, and lists references for further reading. Two short appendices further clarify the proof of concept. Appendix A describes the XML Schema for ORM XMI, while Appendix B demonstrates the automated ORM to XMI transformation using a simple example.

OBJECT TYPES, ROLES, PREDICATES, AND REUSE ORM concepts used are briefly defined in this chapter as needed. An overview of ORM that includes a symbol glossary is available online (Halpin, 1998), and a detailed treatment may be found in Halpin (2001b). To highlight alternatives, this chapter includes two different meta-models for ORM that were developed independently by each author. For ease of reference, we name these Meta-Models A and B. Just as English may be properly used as a meta-language to specify English grammar, we use ORM to specify each ORM meta-model. If desired, any ORM model may be recast as a UML model, as shown later for Meta-Model B. Each meta-model specifies the grammar of a syntactically valid ORM model. A tool that supports editing of ORM models should allow storage of in-progress ORM models that violate many of these grammatical rules, but be capable of checking compliance with these rules when a model error check is requested. Figure 1 portrays Meta-Model A’s view of the main ORM metatypes. An object type is either an entity type (non-lexical) or a value type (lexical), and is displayed as a named ellipse (dotted ellipse if value type). If an entity type has a simple reference scheme, this may be abbreviated by a reference mode in parentheses. A role is a part in a relationship type (association) and is depicted as a box. The arity of an association is its number of roles. An association may be unary, binary, or longer, and is composed of a logical predicate (open proposition with placeholders for objects) and the object types that play the roles. Predicates are displayed as sequences of role boxes,


20 Cuyler & Halpin

Figure 1. Main types in ORM meta-model A "SubtypeConnection" applies to Subtyping Constraint

(ac,it)

>=2

is primary

is a subtype of

Role (Id)

{ 'ET', 'VT' }

ObjectType

ObjectTypeKind (Code)

is of

is played by / plays

is derived

objectifies Primitive ObjectType

Value Type

Subtype

is independent is inactive *

DataType (Id)

Predicate

>=2

is based on

has

Subtype Definition

Entity Type

has virtualUnnested EntityType

has

Nested EntityType

has

Derivation Rules: Subtype is inactive iff Subtype is an ObjectType that plays no Role Subtype Definitions:

RefMode Name

RefModeType (Name)

Derived Predicate

has / defines

Derivation Rule

Each PrimitiveObjectType is an ObjectType that is a subtype of no ObjectType Each Subtype is an ObjectType that is a subtype of an ObjectType Each ValueType is an ObjectType that is of ObjectTypeKind 'VT' Each EntityType is an ObjectType that is of ObjectTypeKind 'ET' Each NestedEntityType is an EntityType that objectifies a Predicate Each UnnestedEntityType is an EntityType that objectifies no Predicate Each DerivedPredicate is a Predicate that is derived

and have one or more readings depending on the order in which the roles are traversed. Arrow-tipped bars over roles depict internal uniqueness constraints, indicating which roles or role combinations may have only unique entries. External uniqueness constraints apply to roles from different predicates and are depicted as a circled “u” (no examples shown here). A black dot on a role connector depicts a mandatory constraint, indicating the role is mandatory for its object type. A circled black dot is a disjunctive-mandatory (inclusive-or) constraint and applies to two or more roles (no examples here). Possible values for an object type may be specified as a value constraint in braces. For example, ObjectTypeKind has two possible codes: ‘ET’ (Entity Type) and ‘VT’ (Value Type). A frequency constraint restricts the number of



21

times an entry may appear in any given population of a role or role sequence. For example, each subtyping constraint applies to at least two (>= 2) subtype connections. Set-comparison constraints may apply between compatible role-sequences, to constrain their populations, and are of three kinds: subset (depicted as a circled “⊆”), equality (depicted as a circled “=”), and exclusion (depicted as a circled “×”). For example, virtual predicates cannot be objectified, and an unnested entity type has a reference mode if and only if it has a reference mode type. If two roles in a predicate are played by compatible types, they may be constrained by a ring constraint. For example, the subtyping relationship is both acyclic (oac) and intransitive ( oit). Subtyping relationships are depicted as solid arrows from subtype to supertype. Subtypes in ORM are dynamic (intensional rather than extensional), and their meaning is declared by a subtype definition. An object type either is primitive or is a subtype. A primitive object type is independent if its instances can exist independently of playing any role in a fact type. Subtyping in ORM allows multiple inheritance. If subtypes collectively exhaust their supertype, this may be displayed as a circled dot. If subtypes are mutually exclusive, this may be displayed as a circled “X”. These subtyping completeness and exclusion constraints may be overlaid to form a “lifebuoy” or partition symbol, as shown. Ideally, such subtyping constraints are derivable from formal subtype definitions and other constraints. For rule and query navigation purposes, an injective, virtual, read-only predicate “is/is” is implicitly created for each subtype connection. Subtype exclusion and completeness constraints are equivalent to exclusion and inclusive-or constraints over the supertype roles of these predicates. This is ignored here. In traditional ORM, each subtype must be active (play a specific role). To enable use of subtypes to merely display taxonomies, we remove this restriction, allowing subtypes to be inactive. A predicate is either primitive or derived. Derived predicates are marked with an asterisk “*”. Ideally, subtypes and derived predicates should be well defined by rules. In Figure 1 subtype definitions and derivation rules are declared in a formal ORM textual language. In ORM an association may be objectified. For example, the meta-fact type ObjectType is a subtype of ObjectType is objectified as the entity type SubtypeConnection. In this meta-model, objectification is modeled by the 1:1 association EntityType objectifies Predicate. This is partly motivated by a desire to minimize the amount of multiple inheritance in the metamodel itself, to ease implementation in languages like Java and C# that support single inheritance only. This is one of three alternative ways to model objectification. The second alternative treats ObjectType and Association as overlapping subtypes Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

22 Cuyler & Halpin

of something else. This multiple inheritance approach is used in UML and also the ORM Meta-Model B, discussed shortly. With this approach, one considers the association to be the very same thing as the object type obtained by the objectification process. The third alternative treats Association as a subtype of ObjectType. Further analysis is required to determine which of these three alternatives is the best approach. For rule and query navigation purposes, mandatory, functional, virtual predicates are implicitly created to connect nested entity types to their component object types. These predicates are by default assigned the readings “involves/is involved in.” These predicate readings are editable by the user, partly to cater for the case where there is only one component object type. Traditional ORM allows an association to be objectified only if it has a fully spanning uniqueness constraint or is a 1:1 binary association. We now remove this restriction, thereby allowing objectification of any association. This decision is in harmony with UML, and has two advantages. First, it enables a simpler representation for certain cases (e.g., a mandatory constraint might be used instead of a set of equality or subset constraints). Secondly, it provides greater stability for cases where the uniqueness constraint on an objectified predicate may vary over time (the predicate can remain objectified throughout this process). To compensate for this relaxation, a modeling tool may transform such objectifications into equivalent non-objectified structures as a preprocessing step before mapping, and issue warnings when a user objectifies an nrole (n > 1) predicate with only one uniqueness constraint, where the constraint spans n-1 roles. Meta-Model A makes no reuse of elements in the UML meta-model, because its main purpose is to clarify the semantics first using ORM. Once this clarification has been performed, the UML 2.0 meta-model (expected to be approved in March 2003) will be examined to determine which model elements are worth reusing in the ORM meta-model. In contrast, ORM Meta-Model B has already reused several elements from the UML 1.4 meta-model (OMG UML RTF, 2001). Figure 2 shows the main types in Meta-Model B. Here NestedEntityType is treated as a subtype of both EntityType and FactType (equivalent to association in Meta-Model A). SubtypeConnection is inherited from Generalization in the UML meta-model. Derived predicates are added to show how SubtypeConnection can also be modeled in terms of co-referencing (alternative to nesting). Role is inherited from UML’s StructuralFeature, and derived predicates are used to relate it to the usual ORM view. Other elements are reused as shown. As discussed earlier, ORM associations may have multiple readings. Forward and inverse readings are shown for some binaries we have already met. In general, an n-ary predicate has n! ways of traversing its roles, and for each Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


23

Figure 2. Main types in ORM meta-model B

of these ways, one or more readings may be given. This extreme flexibility in association readings is supported by both A and B meta-models, but in different ways. Optionally, any role may also be given a name. For any given object type, names of its far roles must be unique in order to support unambiguous role path navigation using attribute-style notation (where role names act as attribute names). Figure 3 shows the basic approach adopted in Meta-Model A. Here, the term “predicate” denotes an unordered set of roles, covering all the roles in a sentence type. A predicate traversal is a permutation (ordered set) of these roles, so it corresponds to a predicate in the logical sense. An association is a sentence type, formed by adding to the predicate the object-type(s) that play the roles. This model fragment allows any number of readings for all possible ways of traversing an association. A much simpler model can be obtained by restricting the number of readings and/or restricting allowed arities. By default, constraints on derived predicates are also derived. Although not marked, the derived fact types are all mandatory in the sense of must-be-known (implied by derivation rule). Their internal uniqueness constraints are implied.


24 Cuyler & Halpin

Figure 3. Naming of ORM roles, predicates, and associations in metamodel A is played by / plays

, in which: § Definition 2: S is a set of aspect modeling techniques. § Definition 3: C is a set of generic modeling concepts. § Definition 4: K is a set of generic knowledge primitives.

•

S is the set of aspect modeling techniques. In the example that was depicted in Figure 5, this would imply that S={DataFlowModel, EntityRelationshipModel}. C is the set of all the generic modeling concepts in the language, regardless of the aspect modeling technique in which they appear. In the example above, this would imply that C = {Process, Flow, Entity, Relationship}. K is the set of knowledge primitives, e.g., K={OutputProcess, InputProcess, SubProcess, FlowEntity, Role} in line with the example above. To denote structures of concepts (consisting of both concepts and knowledge primitives we will use), we will use auxiliary set CSt = C ∪ K. Definition 5: r : K → ℘(C×(N\{0})) is a total function called the referent mapping function. It maps the generic knowledge primitives to a the set of Cartesian products, specifying the concepts to which the knowledge primitive refers and the number of times it refers to the concept. For example, the element < Marriage, { <Man,1>, <Woman,1> } > of r indicates that the generic knowledge primitive Marriage refers once to the generic concept Man and once to the generic concept Woman. In another example, r(Marriage) = {<Man,1>, <Woman,1>}. The function is surjective since distinct knowledge primitives can be mapped to the same value. Also, r(Marriage)=r(Engagement)= {<Man,1>, <Woman,1>}.

For the example above, this would imply that R={, , <SubProcess, Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Evaluating Conceptual Coherence in Multi-Modeling Techniques

51

{}>, }>, , }>}. E.g. r(FlowEntity)={, <Entity, 1>}. • Definition 6: o : K → C is a partial function called the objectification function. It maps generic knowledge primitives to generic concepts that are objectifications of these knowledge primitives. The function is partial since not all k∈K need to be objectified by a generic concept. The function is bijective, therefore it has an inverse function o-1. • Definition 7: G ⊆ C × C is called the generalization relation. For all elements of G, the concept a is the super type of b and the other way round, b is the subtype of a. • Definition 8: A ⊆ CSt × S is called the concept aspect relation. It defines in which aspect model the generic concepts and knowledge primitives appear. Distinct concepts and knowledge primitives can be mapped to the same aspect model, e.g., the generic concepts ‘state’ and ‘transition’ appear both in the aspect model ‘state-transition-model’. On the other hand, there are concepts that appear in several aspect models, such as the UML concept ‘class’ that appears both in the aspect model ‘class-model’ as well as in the aspect model ‘sequence-model’ (OMG, 2001). The last element of the tuple is the set O that may contain coherency relationships between modeling constructs in different aspect models. In order to define this set, some auxiliary sets and functions will first be introduced. • Definition 9: cofa: S → ℘(CSt) is an auxiliary function. It is a total function that maps aspect models to the set of concepts and knowledge primitives that appear in the aspect model. The definition of cofa(s)={ c ∈ CSt | ∈ A }. The next five auxiliary sets (Definition 0-10 - Definition 0-14) contain pairs of concepts or knowledge primitives that are somehow connected, either by a subtype relationship, referent mapping, or objectification. • Definition 10: CESub = { | ∈ G ∨ ∈ G } is an auxiliary set containing all pairs of concepts that are connected by means of a subtype relationship. • Definition 11: CERef = { | ∃x ∈ N\{0} : ∈ r(b) } is an auxiliary set containing all pairs of concepts and knowledge primitives that are connected by means of a referent mapping. • Definition 12: CERef-1 = { | ∃x ∈ N\{0} : ∈ r(a) } is an auxiliary set containing all pairs of knowledge primitives and concepts that are connected by means of a referent mapping. • Definition 13: CEObj = { | o(a) = b } is an auxiliary set containing all pairs of knowledge primitives and concepts that are connected by means of an objectification. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

52

• •

•

•

•

Hommes

Definition 14: CEObj-1 = { | o-1(a) = b } is an auxiliary set containing all pairs of concepts and knowledge primitives that are connected by means of an objectification. Definition 15: CE = CESub ∪ CERef ∪ CERef-1 ∪ CEObj ∪ CEObj-1 is an auxiliary set. It is the union of the five sets that were defined above, therefore containing all pairs of concepts or knowledge primitives that are somehow connected. Note that CE ⊆ CSt × CSt. Definition 16: SC = { | ∃x∈a, y∈b : <x, y> ∈ CE} is an auxiliary set. The elements of this set are pairs of sets containing concepts and knowledge primitives of which at least one element of a set is connected to an element of the other set. Definition 17: D = {d ∈℘(CSt)| ∀e⊂d : <e, d\e> ∈ SC} is an auxiliary set. It contains all coherent concept structures. The elements of D are sets of concepts and knowledge primitives. For each set that is an element of D, all possible partitionings into two sets will result in sets of concepts and knowledge primitives that are connected to each other by at least concept or knowledge primitive; in other words, of the sets of concepts and knowledge primitives that are elements of D, there is no partitioning possible such that there is no connection between the two partitioned sets. Therefore, the elements of D are coherent sets of concepts and knowledge primitives. They will be referred to as modeling constructs. Definition 18: O ⊆ { | b ≠ d ∧ a ⊆ cofa (b) ∧ c ⊆ cofa (d) } is called the set of coherency relationships. Elements of this set are 4-tupes in which a is a modeling construct that is part of aspect model b, and c is a modeling construct that is part of aspect model d, of which the modeling constructs represent the same knowledge. O is a subset of all possible pairs of modeling constructs in different aspect models.

As previously mentioned, the elementary coherency relationship consists of one generic concept that is shared by two different aspect models. In this case, |a| = |c| = 1 holds for the sets in a tuple of O. The coherence relationship in Figure 6 is an example of such a relationship: ∈ O. For complex coherence relationships |a| > 1 or |c| > 1. Examples follow in the next section.

Formal Coherency Metrics The formulation of formal metrics based on a formal description of a technique is a known and proven approach (Rossi et al., 1996). Although all kinds of metrics are thinkable, of which complexity metrics are most often thought of, the focus in this chapter is to propose metrics especially suitable for measuring conceptual coherence. Based on the formal description above, two formal Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


53

metrics are proposed to measure both the coherence between individual aspect modeling techniques and the level of coherence of a multi-modeling technique as a whole: • AMC (Aspect Model Coherence). AMC i,j∈S measures the conceptual coherence between two aspect modeling techniques i,j ∈ S. The idea is to count those elements of O in which i and j appear as aspect models. This gives a measure for the number of coherency relationships between the two. The definition is therefore as follows: AMCi,j∈S = | { ∈ O | (b=i ∩ d=j) ∪ (b=j ∩ d=i) } |. In the example mentioned, AMCDFD,ERD = 1, since there is only one coherency relationship between the two aspect modeling techniques. • WMC x (Whole Model Coherence). WMC x measures the conceptual coherence of a multi-modeling technique as a whole, and is defined as follows:

W MCx =

{ 1,0, ifelseA M C ∑ i,j∈S

i,j

> x,

½ |S|(|S|-1)

The WMC0 measures the total coherence between all aspect modeling techniques in a multi-modeling technique. It counts all pairs of aspect modeling techniques that have at least one coherence relationship between them and divides this sum by the total number of possible coherency relationships between aspect modeling techniques. This yields a measure between [0…1]. In the example above, WMC0 = 1: Since the technique contains only two aspect modeling techniques, (|S|=2), the total number of possible coherency relationships is 1 (only between DFD and ERD). And, in fact there is a coherency relationship between the two since AMCDFD,ERD = 1. Therefore, the coherence is maximal. A multi-modeling technique offers per definition a set of coherent aspect modeling techniques. The WMC0 gives an indication of the level of this coherence. The WMC1 measure is meant to determine unwanted coherency relationships between aspect models. WMC1 counts all pairs of aspect modeling techniques that have more than one coherency relationship between them. It is presumed that the application of aspect modeling techniques that have more than one coherency between them results in models that are more difficult to keep consistent since so much of the same knowledge is expressed in both models. More details follow regarding the interpretation of the values that these metrics yield.


54

Hommes

APPLICATION Description of the UML for Business Modeling The UML for Business Modeling is an extension of the UML, specifically suitable for modeling business processes (Eriksson et al., 2000). It will be referred to as the UBM in the remainder of the chapter. The conceptual coherence between three aspect modeling techniques within the UBM has been studied, using example diagrams as a starting point. The example diagrams belong to the main example that is worked out in the book of Eriksson et al. (2000). First of all there is the UBM Goal modeling technique, of which Figure 6 is a resulting diagram. Next there is the UBM Process modeling technique (a resulting diagram is depicted in Figure 7), and a finally a UBM Class modeling technique (a resulting diagram is depicted in Figure 8). Firstly, the example diagrams have served as the basis for deriving the CMe aspect model depicted in Figure 7. Due to the limited space in this chapter, an elaboration on the process of constructing these C-Me models out of the description will not be given. Summarized, the approach taken is to first describe all individual knowledge primitives expressed in a diagram and to later classify these individual expressions into generic concepts and knowledge primitives (this is according to the FCO-IM approach described in Bakema et al., 1996).

Figure 7. Example of a UML process model (Eriksson et al., 2000)



55

Figure 8. Example of a UML conceptual model (Eriksson et al., 2000)

Examples of individual expressions that can be derived out of Figure 9 are “Many market shares depends on high profit,” “Many market shares depends on motivated staff,” “High profit and many transactions are contradictory,” etc. Noun phrases (italicized in the example) are classified into generic concepts, e.g., the concept ‘Goal’ and the language functions of phrases are classified into generic knowledge primitives, e.g., “<x> depends on ” becomes the knowledge primitive ‘goal dependency’, and “<x> and are contradictory” becomes the knowledge primitive ‘contradictory goal’ (Figure 10). The described procedure is carried out for each individual aspect model, resulting in the three C-Me aspect diagrams presented in Figure 10.


56

Hommes

Figure 9. Example of a UML goal model (Eriksson et al., 2000)



57

Figure 10. C-Me aspect diagrams of the UML for business modeling Goal Model Complete Goal Decomposition

Goal Dependency

Contradictory Goal Incomplete Goal Decomposition

Goal

Goal Value Number Current Value

Qualitative Goal

Quantitative Goal Unit Of M easurement

M easurement Unit

Process Model

Class Model O b ject C om p ositio n

A sso ciatio n C om p ositio n

A gg reg ate R elatio nsh ip

Su b P ro cess

G oal

R eso urce C on sum ptio n

C on strain t K ind

Pro cess C on strain t C overag e

C on strain t K ind

G oal A llo cation

A sso ciatio n

M u ltiplicity K in d M u ltiplicity

R eso urce Pro d uction

R eso urce

C on strain t U nid irectio nal A sso ciatio n

D ep en den cy R elatio nsh ip

R eso urce C on tro l

C lass G en eralizatio n

Pro cess Su pp ly

Describing Conceptual Coherence in the UML for Business Modeling After having constructed the C-Me aspect diagrams, the next step is to determine coherency relationships between the aspect modeling techniques and the nature of these relationships. The first step is to identify the elementary coherency relationships by checking the extension of each generic concept in an aspect model with the extension of each generic concept in another aspect model, again making use of the example diagrams. The elementary coherency relationships that can be found are depicted by relationships 1 to 5 in Figure 14, called the C-Me detailed coherence diagram. The nature of these relationships is as follows: 1. Intersection Relationship. Some of the goals specified in the goal model appear as classes in the conceptual model. On the other hand, some of the classes in the conceptual model appear as goals in the goal model. In other words, the extensions of both concepts intersect. An example is the goal/ class ‘Many market shares’ that appears in both goal model and conceptual model. 2. Equality Relationship. Goals that appear in the goal model also appear in the process model and the other way round.


58

Hommes

Figure 11. Coherence between processes and classes

Process Model

3.

4. 5.

6.

Conceptual Model

Intersection Relationship. Some of the goals mentioned in the process model appear also in the conceptual model. On the other hand, some of the classes in the conceptual model appear as goals in the process model. This relationship is comparable to the intersection relationship between goal model and conceptual model. Subset Relationship. All resources mentioned in the process diagram appear as classes in the conceptual model. Subset Relationship. Some of the processes in the process model appear as classes in the conceptual model as depicted in Figure 11. The next step is to consider structures of concepts and knowledge primitives and to consider whether there is semantic overlap. This is done by revisiting the example diagrams and identifying where the same knowledge is expressed. This yields the relationships 6 and 7 (Figure 14). Complex Coherence Relationship. Both in the goal model and conceptual model, there are structures of concepts and knowledge primitives that overlap semantically. Figure 12 shows an example. In the goal model it is

Figure 12. Coherence between goal and conceptual model

Conceptual Model

Goal Model Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


59

Figure 13. Coherence between dependencies and associations

Conceptual Model

Process Model

Figure 14. C-Me detailed coherence diagram of UML G oal Model G o al D ep en den cy

Class Model 6 G o al

1 U nidirection al A sso ciation D ep end enc y R elation sh ip

C lass

2 5 3 Process Model

4 G o al

7

R eso urce C on su m p tio n P ro cess R eso urce P ro du ction

R eso urce

R eso urce C on trol

P ro cess Su pp ly


60

Hommes

specified that realization of the goal ‘many market shares’ is dependent on the realization of the goal ‘many transactions’. In the conceptual model it is specified that many ‘transactions’ lead to ‘many market 7. Complex Coherence Relationship. Some dependencies between resources and processes appear as associations in the conceptual model as depicted in Figure 13. The found coherency relationships can be depicted in a C-Me detailed coherence diagram as is Figure 14.

Formal Measures Now that the technique and coherency relationships have been described, it is possible to apply the proposed metrics. In total there are three aspect modeling techniques, and thus three pairs of possible combinations to consider. Values for the respective AMCs are: AMCgoal, concept = 2, AMCgoal, process = 1 and AMCprocess, concept = 4. Especially in between the process model and conceptual model, we see many coherency relationships. It is expected that it becomes difficult to construct consistent process and goal models using these aspect modeling techniques. There is at least one coherency relationship between each conceivable pair of aspect modeling techniques within the technique, so each possible AMCi,j∈S > 0. Therefore, the WMC0 = 1: the coherence between the aspect modeling techniques within the UBM is maximal. Two of the three AMCi,j∈S > 1, viz. AMCgoal, concept and AMCprocess, concept. Therefore, the WMC1 = 0.66: two of the three pairs of aspect modeling techniques have more than one coherency relationship between each other. Application of the metrics shows that there is maximal coherence in the UBM. The value for WMC1 shows that there is a lot of unwanted overlap between the techniques. Especially the AMCprocess, concept is high. A question that should be addressed is what a high or low value actually means and what, for instance, threshold values are. At this moment, the question can be answered referring to: (1) the WMC and AMC values found for three other business process modeling techniques, and (2) on empirical research where 40 modelers made business models using UML for business modeling, and the kind and number of errors they made were compared to the WMC and AMC values. Both studies have been carried out and are planned to be published in 2004 as part of a PhD thesis. When the AMC values for UML are compared to three other techniques, it is striking that the AMCprocess, concept value, being four, is the highest of all evaluated modeling techniques. An average value is one. However, the empirical study showed that this high value did not result in the expected number of consistency errors. This is due to the non-restrictive nature of these coherency relationships. A better estimate can be given when also the nature of the Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


61

relationships is taken into account. Furthermore, the empirical results showed that the variance of the outcome of the UBM conceptual models was rather high. The 40 modelers in the experiment came up with quite different UBM conceptual models. This can be prescribed to the number of coherency relationships. A large number of concepts modeled in the other aspect models may also be used in the conceptual model. Since there are no restrictions or guidelines on which of these concepts to use in the conceptual model, the variance in the outcomes is high.

CONCLUSIONS & FURTHER RESEARCH In this chapter I have proposed a modeling technique that is capable of describing aspect modeling techniques as well as multi-modeling techniques. To the best of my knowledge, there are no languages that explicitly distinguish the aspect modeling techniques within a multi-modeling technique. The proposed modeling language contributes to a better understanding of those techniques. Along with describing aspect modeling techniques within multi-modeling techniques comes the possibility to describe coherency relationships within the technique. The proposed modeling language allows the description of those coherency relationships. The formal description of the C-Me language allows the application of various metrics. Although it offers a good basis for defining, e.g., complexity metrics, the focus in this chapter was on the measurement of conceptual coherency. Two metrics have been proposed and their measurement schemes have been anchored in the formal description of the modeling language, allowing a precise measure for coherency not only between individual aspect modeling techniques, but also for the multi-modeling technique as a whole. The usefulness of the proposed modeling language has been illustrated by means of applying it to evaluate the UBM. Unfortunately, the application has been limited to only one evaluation. Promising research has been carried out where the metric values for various multi-modeling techniques other than UBM were compared. Further research focuses on the development and validation of a complete suite of metrics based on the C-Me description language. A promising experiment has been carried out where empirical data on the quality of models has been collected by means of the application of four state-of-the-art business process modeling techniques by 40 professional modelers on a single case. Some results were already mentioned in this chapter. The C-Me framework metrics suite is validated by investigating the correlation between the values obtained by applying the theoretical metrics and the empirical data on the quality of the resulting models for each of the four business process modeling techniques.


62

Hommes

REFERENCES Aalst, W.M.P. van der, & Hee, K.M. van (2002). Workflow management: Models, methods, and systems. Boston, MA: MIT Press. Bajaj, A. (2000). SMMM: A metric-based framework to evaluate the scalability of multiple modeling methodologies. Proceedings of EMMSAD 2000, Stockholm, Sweden. Bakema, G., Zwart, J.P., & Lek, H. van der. (1996). Volledig communicatiegeorienteerde informatiemodellering (FCO-IM) (Dutch). Dordrecht: Kluwer Bedrijfsinformatie. Eriksson, H.-E., & Penker, M. (2000). Business modeling with UML: Business patterns at work. John Wiley & Sons. Halpin, T. (2001). Information modeling and relational databases. Morgan Kaufmann. Hommes, B.-J., & Dietz, J.L.G. (2001). Understanding concepts and their relationship to reality. Proceedings of EJC2001, Maribor, Slovenia. Object Management Group (OMG). (2002). UML specification. Available online at: www.omg.org/uml. Rossi, M., & Brinkkemper, S. (1996). Complexity metrics for systems development methods and techniques. Information Systems, 20. Scheer, A.-W., & Hoffman, M. (1999). From business process model to application system — developing an information system with the House of Business Engineering (HOBE). In M. Jarke & A. Oberweis (Eds.), Advanced Information Systems Engineering, 11th International Conference CAiSE’99. Springer. Yourdon, E., & Constantine, L. (1979). Structured design: Fundamentals of a discipline of computer program and systems design. Englewood Cliffs, NJ: Prentice-Hall.


Assessing Enterprise Modeling Languages

63

Chapter IV

Assessing Enterprise Modeling Languages Using a Generic Quality Framework John Krogstie, Norwegian University of Science and Technology, Norway and SINTEF Telecom and Informatics, Norway Sofie de Flon Arnesen, Norwegian University of Science and Technology, Norway

ABSTRACT Statoil, one of Norway’s largest organizations, recently wanted to standardize an enterprise modeling language for process modeling for sense-making and communication. To perform the evaluation, a generic framework for assessing the quality of models and modeling languages was specialized to the needs of the company. Five different modeling languages were evaluated according to the specialized criteria. Two languages were, through this, found as candidate languages, and further criteria related to tool and process support for using the languages in actual modeling were used to decide the language to choose for future standardization. This work illustrates the practical utility of the overall framework, where language quality features are looked upon as means to enable the creation Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

64 Krogstie & Arnesen

of models of high quality. It also illustrates the need for specializing this kind of general framework based on the requirements of the specific organization.

INTRODUCTION A large number of modeling languages exist for enterprise modeling. Deciding which modeling language to use for a specific task is often done in an ad-hoc fashion by different organizations. Statoil, one of Norway’s largest companies, has over the years developed and used several different modeling languages for modeling enterprise processes. Last year the company embarked on a process for evaluating and selecting a standard modeling language enterprise process modeling. We earlier developed a general framework for assessment of quality of models, where one type of means to support quality goals at different levels is criteria for the language to be used for modeling, also called language quality (Krogstie, 2001). This chapter presents an example of using and specializing the quality framework for the evaluation and selection of a modeling language for enterprise process modeling for Statoil.

Chapter Structure The next section describes the general quality framework, with a focus on language quality. We then describe the Statoil case in more detail and followed with results of the evaluation. The conclusion highlights some of our experiences from using and specializing the quality framework for evaluating modeling languages for enterprise modeling.

FRAMEWORK FOR QUALITY OF MODELS We use the model quality framework (Krogstie & Sølvberg, 2003; Krogstie, 2001) as a starting point for the discussion on language quality. The main concepts of the framework and their relationships are shown in Figure 1. We have taken a set-theoretic approach to the discussion of model quality at different semiotic levels, which has been defined as the correspondence between statements belonging to the following sets: • G, the (normally organizational) goals of the modeling task. • L, the language extension, i.e., the set of all statements that are possible to make according to the graphemes, vocabulary, and syntax of the modeling languages used. • D, the domain, i.e., the set of all statements which can be stated about the situation at hand. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


65

Figure 1. Main parts of the quality framework S o cia l a cto r e xp licit kn o w le d g e KS

M o de ller e xp licit kn o w le d g e KM

S o cia l a c to r in te rp re ta tio n I

G o a ls of m o d elling G P h ysica l qu ality

M o d e lin g d o m a in D

P e rceived se m antic qu ality

Se m an tic qu ality

O rgan iza tio nal qu ality

M o de l exte rn a liza tion M

S o cial q uality

S o cia l p rag m atic qu ality

S yn tactic qu ality

L an g u a ge e xte n sio n L

E m pirica l qu ality Te ch nical p ra gm atic qu ality

T ech n ical a c to r inte rp re ta tio n T

• •

• •

•

• •

M, the externalized model, i.e., the set of all statements in someone’s model of part of the perceived reality written in a language. Ks, the relevant explicit knowledge of the set of stakeholders being involved in modeling (the audience A). A subset of the audience is those actively involved in modeling, and their knowledge is indicated by KM. I, the social actor interpretation, i.e., the set of all statements which the audience thinks an externalized model consists of. T, the technical actor interpretation, i.e., the statements in the model as ‘interpreted’ by different model activators (for example modeling tools). Solid lines between the sets in Figure 1 indicate the model quality types: Physical Quality. The basic quality goals on the physical level are externalization, that the knowledge K of the domain D of some social actor has been externalized by the use of a modeling language, and internalizability, that the externalized model M is persistent and available, enabling the audience to make sense of it. Empirical Quality. Deals with predictable error frequencies when a model is read or written by different users, coding (e.g., shapes of boxes), and HCI-ergonomics for documentation and modeling-tools. Syntactic Quality. The correspondence between the model M and the language extension L of the language in which the model is written.



•

•

• • •

•

Semantic Quality. The correspondence between the model M and the domain D. The framework contains two semantic goals — validity means that all statements made in the model are correct relative to the domain, and completeness means that the model contains all the statements found in the domain. Perceived Semantic Quality. The similar correspondence between the audience interpretation I of a model M and his or her current knowledge K of the domain D, and is what can actually be checked at model validation. Pragmatic Quality. The correspondence between the model M and the audience’s interpretation of it (I). Social Quality. The goal defined for this quality is agreement among audience members’ interpretations I. Organizational Quality. All statements in the model directly or indirectly contribute to fulfilling the goals of modeling (organizational goal validity), and all the goals of modeling are addressed through the model (organizational goal completeness). Language Quality. Relates the modeling languages used to the other sets. It is distinguished between two types of criteria: (1) criteria for the underlying (conceptual) basis of the language (i.e., what is represented in the abstract language model (meta-model) of the language), and (2) criteria for the external (visual) representation of the language (i.e., the notation).

As illustrated in Figure 2, five quality areas for language quality are identified, with aspects related both to the meta-model and the notation.

Figure 2. Language quality related to the quality framework M odeller explic it know ledge KM

S ocial actor interpretation I K n o w le dg e e xte rn a liza b ility a p p ro p ria te n e ss / P a rticip a n t la n gu a g e kn o w le d ge a p p ro p ria te n es s

M odeling dom ain D

M odel externalization M

C o m p re h e n sib ility a p p ro p ria te n e ss

Language extension L

D o m a in a p p ro p ria te n es s

T echnic al actor interpretation T

T e ch nica l a cto r in te rp re ta tio n a p p ro p ria te n e ss



67

Domain Appropriateness Ideally, the conceptual basis must be powerful enough to express anything in the domain, i.e., not having construct deficit (Wand & Weber, 1993). On the other hand, you should not be able to express things that are not in the domain, i.e., what is termed construct excess (Wand & Weber, 1993). The only requirement to the external representation is that it does not destroy the underlying basis. One approach to evaluating domain appropriateness is to look at how the modeling perspectives found useful for the relevant modeling tasks are covered. For instance in information systems modeling, seven general modeling perspectives have been identified (Krogstie & Sølvberg, 2003): structural, functional, behavioral, rule-oriented, object-oriented, language-action-oriented, and roleand actor-oriented. Another approach is to base an evaluation on an ontological theory (see, e.g., Opdahl, Henderson-Sellers, & Barbier, 1999) that uses the ontology presented by Wand and Weber (1993). Domain appropriateness is primarily a means to achieve physical quality, and through this, to achieve semantic quality.

Participant Language Knowledge Appropriateness This area relates the knowledge of the modeler to the language. The conceptual basis should correspond as much as possible to the way individuals perceive reality. This will differ from person to person according to their previous experience, and thus will initially be directly dependent on the modeler. On the other hand, the knowledge of the modeler is not static, i.e., it is possible to educate persons in the use of a specific language. In that case, one should base the language on experiences with languages for the relevant types of modeling, and languages that have been used successfully earlier in similar tasks. Participant language knowledge appropriateness is primarily a means to achieve physical and pragmatic quality.

Knowledge Externalizability Appropriateness This area relates the language to the participant knowledge. The goal is that there are no statements in the explicit knowledge of the modeler that cannot be expressed in the language. Knowledge externalizability appropriateness is primarily a means to achieve physical quality.

Comprehensibility Appropriateness This area relates the language to the social actor interpretation. For the conceptual basis: • The phenomena of the language should be easily distinguishable from each other (vs. construct redundancy, Wand & Weber, 1993). Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


•

• •

• • •

• •

•

The number of phenomena should be reasonable. If the number has to be large, the phenomena should be organized hierarchically and/or in sublanguages of reasonable size linked to specific modeling tasks, making it possible to approach the modeling framework at different levels of abstraction or from different perspectives or viewpoints. The use of phenomena should be uniform throughout the whole set of statements that can be expressed within the language. The language must be flexible in the level of detail. As for the external representation, the following aspects are important: Symbol discrimination should be easy. It should be easy to distinguish which of the symbols in a model any graphical mark in the model is part of (what Goodman (1976) terms syntactic disjointness). The use of symbols should be uniform, i.e., a symbol should not represent one phenomenon in one context and another one in a different context. Neither should different symbols be used for the same phenomenon in different contexts. One should strive for symbolic simplicity. One should use a uniform writing system: All symbols (at least within each sub-language) should be within the same writing system (e.g., nonphonological such as pictographic, ideographic, logographic, or phonological such as alphabetic). The use of emphasis in the notation should be in accordance with the relative importance of the statements in the given model.

Comprehensibility appropriateness is primarily a means to achieve empirical and, through that, pragmatic quality.

Technical Actor Interpretation Appropriateness This area relates the language to the technical actor interpretation. For the technical actors, it is especially important that the language lends itself to automatic reasoning. This requires formality (i.e., both formal syntax and semantics; the formal semantics can be operational, logical, or both), but formality is not sufficient, since the reasoning must also be efficient to be of practical use. This is covered by what we term analyzability (to exploit the mathematical semantics) and executability (to exploit the operational semantics). Different aspects of technical actor interpretation appropriateness are a means to achieve syntactic, semantic, and pragmatic quality (through formal syntax, mathematical semantics, and operational semantics, respectively).



69

A number of subareas are identified for each of the five areas of language quality, and in Østbø (2000) approximately 70 possible criteria were identified. We will return to how this extensive list has been narrowed down and specialized for the task at hand. In addition one can look upon the description of a language as a language model, which is what Prasse (1999) covers under documentation. Language quality of the language used for meta-modeling can be a further point to evaluate according to the five criteria above. In this chapter we have concentrated on the primary language quality aspects.

DESCRIPTION OF THE CASE AND THE EVALUATION APPROACH Statoil is one of the largest companies in Norway. It is primarily in the energy sector, specifically within upstream and downstream of oil and gas. Although the main work of Statoil is connected to Norway and the North Sea, Statoil also has an increasing international presence.

Goals for Business Process Modeling in Statoil Before discussing the needs of Statoil specifically, we outline the main uses of enterprise process modeling. Four main categories for enterprise modeling can be distinguished: 1. Human-Sense Making and Communication. To make sense of aspects of an enterprise and communicate this with other people. 2. Computer-Assisted Analysis. To gain knowledge about the enterprise through simulation or deduction. 3. Model Deployment and Activation. To integrate the model in an information system and thereby make it actively take part in the work performed by the organization. Models can be activated in three ways: • Through people guided by process ‘maps’, where the system offers no active support. • Automatically, where the system plays an active role, as in most workflow engines. • Interactively, where the computer and the users cooperate in interpreting the model. The computer makes decisions about prescribed parts of the model, while the users resolve ambiguities. 4. To give the context for a traditional system development project, without being directly deployed and activated. An orthogonal dimension to these four are the temporal dimension, i.e., if one is to model the past, the present (as-is), or the future (to-be). Another key differentiator is to what extent the focus is on internal processes of a company, Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


or to support inter-organizational cooperation. Finally one can differentiate between process models on a type level and on an instance level.

Statoil’s Requirements The detailed requirements to the modeling language were established in discussion with Statoil. The main person responsible for this process at Statoil, being responsible, e.g., for methodological issues in the company, had both extensive experience with several enterprise modeling tasks within different parts of Statoil, and a good overview of modeling and modeling techniques in general, with a PhD in the area. Our discussions were primarily with him, and he communicated further with different parties within the company. Statoil had the following overall requirements: • It should be a language for sense-making and communication (category 1 above). • The language should be usable by leaders on top and medium levels, and also others that were not used to model with graphical languages. • The language should be simple, but still have sufficient expressiveness. • The language should be independent of any specific modeling domain. • It should be possible to use the language to describe work processes across organizational areas. • It should be possible to use the modeling language both for modeling routine and non-routine work. • It should be possible to model processes both on a type and on an instance level.

1. 2. 3. 4. 5.

The following concepts were regarded as mandatory to be able to model: Processes, including decomposition of the process Activities, indicating the lowest level of a process model Roles Decisions and decision points Flow between processes, activities, and decision points

The following concepts were regarded as recommended (but not mandatory) to be able to model: 1. Deliverables (process results, e.g., in the form of documents) 2. System resources (here with a focus on information systems) It was not a requirement that all the concepts be expressed with a separate symbol. It should be possible to develop the model incrementally. More concretely this meant:



• • •

71

It should be possible to model only processes and flows, and independently model these concepts. It should be possible to model activities and flows, and independently model these concepts. It should be possible to model roles, decision points, system resources, and deliverables independently.

A general set of requirements to a modeling language, based on the discussion of language quality in this chapter, is outlined in Østbø (2000). As already mentioned, this amounted to around 70 sub-requirements. These were looked at relative to the requirements of the Statoil case, and their importance was evaluated as indicated below. Grade

Explanation

0-3

Requirement has no or very limited relevance

4-6

Generally relevant requirements, but not specifically important for the case Specifically relevant requirements for the specific needs of the case An absolutely necessary requirement

7-9 10 (Mandatory)

Only requirements given a grade of seven or more were included. In addition, requirements on domain appropriateness were detailed further compared to the general framework, including the mandatory and recommended concepts mentioned above. This resulted in the following evaluation-criteria: No. 1 2 2.1 2.2 2.3 2.4 2.5 2.6(r)

Requirement The language should be independent of business domain.

Type of req. Domain appropriateness, Underlying basis “

The language should be able to express the following concepts: Processes: A process can consist of several sub“ processes or activities, i.e., process decomposition Activities: The lowest level of a process model “ Roles (of persons involved in the process) “ Decision points/decision rules “ Flow between processes, activities, and decision “ points Deliverables (results) “ continued on following page


72 Krogstie & Arnesen 2.6(r) 2.7(r)

Deliverables (results) System resources (information systems used in the process)

“ “

3

It must be possible to decompose processes in as many levels as necessary. Also the process-symbols should be decomposable.

“

4

5

The terms used of concepts must be same as terms used for these concepts in Statoil

6 7

It must be easy to learn the language. The external representation must be intuitive, meaning that the symbol represents the concept better than another symbol would.

8

The different concepts must be easily distinguishable.

9

The number of concepts must be at a reasonable level. The language must be flexible in the level of detail. Symbol discrimination should be easy.

10 11

12 13 14

The use of symbols should be uniform. One should strive for symbolic simplicity, both in the individual symbols, and for how they are related. The use of emphasis in the notation should be in accordance with the relative importance of the concept.

Domain appropriateness, External representation Participant language knowledge appropriateness, Underlying basis “ Participant language knowledge appropriateness, External basis Comprehensibility appropriateness, Underlying basis “ “ Comprehensibility appropriateness, External representation “ “ “

Statoil was in this connection not interested in automatic reasoning and execution/simulation, and did not want requirements on technical actor interpretation appropriateness to decide on the choice. It was mandatory that the syntax of the language was well defined.

EVALUATION The overall approach to the evaluation was the following: First a short-list of relevant languages was identified by us and the customer in cooperation. The chosen languages were then evaluated according to the selected criteria. To look upon this in more detail, all languages were used for the modeling of several real cases (including both models on an instance and a type level) using an



73

‘independent’ modeling tool (which in our case was Visio). By showing the resulting models and evaluation results to the persons from Statoil, we got feedback and corrections both on the models and our grading. The overall result identified two candidates, where aspects such as available tool support in connection to supporting all aspects of model quality were instrumental to come up with a final choice. Based on discussions with Statoil and experts on enterprise modeling, five languages were selected on a short-list of relevant languages. These will be briefly described; for a longer description see the report (Arnesen, 2001) and the cited references.

Language Used in Modeling Conference (Gjersvik, Krogstie, & Følstad, 2005) Gjersvik has developed a very simple process modeling language to use in so-called participatory modeling conferences, which had been used in Statoil earlier. The language has only three symbols: Process, Products (intermediate and final), and Flow between processes and products. The use of this language is described in further detail in another chapter of this book.

EEML (Extended Enterprise Modeling Language) (EXTERNAL, 1999) EEML is developed in the EU-project EXTERNAL as an extension of APM (Carlsen, 1997). The language has constructs to support all modeling categories previously described, and not only for human sensemaking and communication. The following main concepts are provided: • Task with input and output ports (which are specific types of decision points) • General decision points • Roles (Person-role, Organization-role, Tool-role, Object-role) • Resources (Persons, Organizations and groups of persons, Tools (manual and software), Objects (material and information)) A flow links two decision points and can carry a resource. A task has several parts: an in-port and an out-port, and potentially a set of roles and a set of sub-tasks. Roles ‘is filled by’ resources of the corresponding types. Figure 3 provides a meta-model of the main concepts.

Current Language for Enterprise Modeling in Statoil (Arnesen, 2001) Through earlier enterprise modeling and reengineering projects, Statoil has developed a language that is similar to role-oriented flowlanguages such as Rummler-Brache. The following concepts are provided:



Figure 3. Main concepts of EEML has-part has-part

1:n

1:n

Task

Decision Point

flows-to n:m

1:n

has-resource-role

flows-to n:m

is-filled-by n:1

Person

1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12.

Role

is-filled-by n:1

Organisation Unit

is-filled-by is-filled-by n:1

Information Object

n:1

Tool

Role Input (Start) Output (Product) Process Activity Decision point Delivery flow Internal connector External connector QA checkpoint Document Information system

A work process can be decomposed into several subprocesses, which can be decomposed further. On the lowest level, one has activities, which cannot be decomposed.

UML Activity Diagrams (Booch, Rumbaugh, & Jacobson, 1999) An activity diagram can have the following symbols: • Start, End, Activity, Flow (between activities, either as control or as objectflows), Decision points, and Roles using swimlanes

IDEF-0 (Integration Definition language 0) (IDEF-0, 1993) IDEF-0 diagrams have three main symbols: Functions (boxes), Flow (arrows) of different types between functions, and different types of ports on the functions (input, output, control, and mechanisms). Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


75

OVERVIEW OF EVALUATION RESULTS Below, the main result of the evaluation is summarized. For every language, every requirement is scored according to the scale below (which was also used in Østbø, 2000). Grade

Explanation

0-3

There is no, or very limited support of the requirement.

4-6

The requirement is partly supported.

7-9

There is satisfactory support of the requirement.

10

The requirement is very well supported.

The reasoning behind the grading can be found in Arnesen (2001) and is not included here due to space limitations. The two last rows summarize the results. The last row only includes the mandatory requirements. No 1 2 2.1 2.2 2.3 2.4 2.5 2.6 2.7 3 4 5 6 7 8 9 10 11

Requirement Domain independence Expressiveness Processes Activities Roles Decision points Flow Deliverables (results) System resources Decomposable processes Decomposable symbols Equal naming of concepts and domains Language easy to learn Intuitive external representation Easy to separate symbols Reasonable number of concepts Flexible in precision Easy to differentiate

Grading per language Gjersvik EEML

Statoil

10

10

10

UML - AD 10

IDEF -0 10

10 6 0 0 5 8 0 0

10 6 10 7 10 10 8 10

10 10 10 7 10 10 7 7

10 6 10 7 10 5 0 7

10 6 0 0 10 10 0 10

0 9

10 6

7 8

7 9

10 7

10 7

6 8

7 9

8 9

6 10

10

6

10

10

10

4

5

7

9

4

4 10

10 5

10 7

5 9

0 10

continued on following page



10 11 12 13 14

Flexible in precision Easy to differentiate different symbols Consistent notation Symbolic simplicity Use of emphasis Sum including recommended requirements 2.6 and 2.7 Sum excluding recommended requirements 2.6 and 2.7

4 10

10 5

10 7

5 9

0 10

5 9 7 114

10 5 7 159

7 10 9 172

10 10 10 161

3 10 10 136

106

141

155

156

126

Comparison Table with all the Evaluations Collected

Based on the evaluation, two of the languages were clearly inappropriate: IDEF-0 and Gjersvik’s language. The internal language developed/adapted by Statoil has the highest sum, taking into account also the recommended requirements, whereas UML activity diagrams were slightly ahead when only including the mandatory requirements. EEML comes third using both summations. EEML was regarded as too complex when only looking at the support of modeling category one (which is not surprising, since it is meant to be used across all categories), with too many concepts, symbols, and constraints for inexperienced modelers. Thus looking only at the language quality, two languages were found as candidates for further investigation. Based on earlier critique of activity diagrams (see, e.g., Krogstie, 2001), it was somewhat surprising that this language scored as high as it did. On the other hand, when using this only for sense-making and communication, one could ignore the somewhat alienating official state-oriented semantics defined in UML 1.4, and use the activity diagram more or less as a traditional flowchart. When looking at these languages in connection to other aspects in the model quality framework, e.g., including tool support, it appeared that, for instance, even if activity diagrams as defined in UML 1.4 do have decomposition, the available UML-tool in Statoil (Rational Rose) did not support decomposition of activities properly at the time. It neither supports the more intuitive semantics of activity diagrams (which is introduced in UML2.0), but rather the official semantics. The final choice was made by Statoil, which in this case decided to keep and extend their existing language, differentiating between two versions of the language, and providing further tool and organizational support for using these variants. The choice was based both on the language appropriateness, and the availability and cost of wide tool support for the language.



77

CONCLUSIONS AND FURTHER WORK We have in this chapter described the use of a general framework for discussing the quality of models and modeling languages in a concrete case of evaluating enterprise process modeling languages. The case illustrates how our generic framework can (and must) be specialized to a specific organization and type of modeling to be useful, which it was also found to be by the people responsible for these aspects in Statoil. Since the scope of modeling in the case was quite constrained, general enterprise modeling languages such as EEML was found to be overly expressive since it is meant to be used across a larger set of modeling tasks (including simulation, and as a basis for manual and interactive activation). It can be argued that the actual valuation is somewhat simplistic (flat grades on a 1-10 scale that is summarized). We did not use the first classification on importance as the criteria for weighting. An alternative to flat grading is to use pairwise comparison and AHP on the alternatives (Krogstie, 1999). The weighting between expressiveness and understanding can also be discussed. For later evaluations of this sort, we would like to use several variants of grading schemes to investigate if and to what extent this would impact the result. This said, we should not forget that language quality properties are never more than means for supporting the model quality (where the modeling task typically has specific goals on its own). Thus instead of only evaluating modeling languages ‘objectively’ on the generic language quality features of expressiveness and comprehension, it is very important that these language quality goals are linked to model quality goals to more easily adapt such a generic framework to the task at hand.

REFERENCES Arnesen, S. (2001). Evaluering av språk for arbeidsprosessmodellering (in Norwegian). Unpublished project report, IDI, NTNU, Trondheim. Norway. Booch, G., Rumbaugh, J., & Jacobson, I. (1999). The Unified Modeling Language: User Guide. Addison-Wesley. Carlsen, S. (1997). Conceptual modeling and composition of flexible workflow models. Unpublished PhD Thesis, Information Systems Group, Department of Computer and Information Science, Faculty of Physics, Informatics and Mathematics, Trondheim, Norway, NTNU-Norwegian University of Science and Technology. EXTERNAL. (1999). EXTERNAL— Extended enterprise resources, networks and learning, EU project, IST-1999-10091, new methods of work and electronic commerce, dynamic networked organizations. Partners:



DNV, GMD-IPSI, Zeus E.E.I.G., METIS, SINTEF Telecom, and Informatics, 2000-2002. Gjersvik, R., Krogstie, J., & Følstad, A. (2004). Participatory development of enterprise process models. In Information modeling methods and methodologies. Hershey, PA: Idea Group Publishing. Goodman, N. (1976). Languages of art: An approach to a theory of symbols. Indianapolis: Hackett. IDEF-0: Federal Information Processing Standards Publication 183. (1993, December 21). Announcing the Standard for Integration Definition for Function Modeling (IDEF-0). Krogstie, J. (1999). Using quality function deployment in software requirements specification. In A.L. Opdahl, K. Pohl, & E. Dubois (Eds.), Proceedings of the Fifth International Workshop on Requirements Engineering: Foundations for Software Quality (REFSQ’99) (pp. 171-185), Heidelberg, Germany, June 14-15. Krogstie, J. (2001). Using a semiotic framework to evaluate UML for the development of models of high quality. In K. Siau & T. Halpin (Eds.), Unified Modeling Language: Systems analysis, design, and development issues. Hershey, PA: Idea Group Publishing. Krogstie, J., & Sølvberg, A. (2003). Information systems engineering: Conceptual modeling in a quality perspective. Trondheim, Norway: Kompendiumforlaget. Krogstie, J., Lindland, O.I., & Sindre, G. (1995). Defining quality aspects for conceptual models. In E.D. Falkenberg, W. Hesse, & A. Olive (Eds.), Proceedings of the IFIP8.1 Working Conference on Information Systems Concepts (ISCO3); Towards a Consolidation of Views (pp. 216-231), Marburg, Germany, March 28-30. Opdahl, A., Henderson-Sellers, B., & Barbier, F. (1999). An ontological evaluation of the OML meta-model. In E. Falkenberg, K. Lyytinen, & A. Verrijn-Stuart (Eds.), Proceedings of the IFIP8.1 Working Conference on Information Systems Concepts (ISCO4): An Integrated Discipline Emerging (pp. 217-232), Leiden, The Netherlands, September 20-22. Østbø, M. (2000). Anvendelse av UML til dokumentering av generiske systemer (in Norwegian). Unpublished master’s thesis, Høgskolen i Stavanger, Norway, June 20. Prasse, M. (1998). Evaluation of object-oriented modeling languages. A comparison between OML and UML. In M. Schader & A. Korthaus (Eds.), The Unified Modeling Language—Technical aspects and applications (pp. 58-78). Heidelberg: Physica-Verlag. Wand, Y., & Weber, R. (1993). On the ontological expressiveness of information systems analysis and design grammars. Journal of Information Systems, 3(4), 217-237. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


79

Wieringa, R. (1998). A survey of structured and object-oriented software specification methods and techniques. ACM Computing Surveys, 30(4), 459-527.


80 Ralyté, Rolland, & Ayed

Chapter V

An Approach for Evolution-Driven Method Engineering Jolita Ralyté, University of Geneva, Switzerland Colette Rolland, University of Paris 1 - Sorbonne, France Mohamed Ben Ayed, University of Paris 1 - Sorbonne, France

ABSTRACT This chapter considers the evolutionary perspective of method engineering. It presents an approach for method engineering supporting evolution of an existing method, model, or meta-model into a new one satisfying a different engineering objective. The authors hope that this approach could be helpful for a project-specific method construction. This approach proposes several different strategies to evolve from the initial paradigm model to a new one and provides guidelines supporting these strategies. The approach has been evaluated in the Franco-Japanese research project around the Lyee methodology. A new model called Lyee User Requirements Model has been obtained as an abstraction of the Lyee Software Requirements Model. The chapter illustrates this evolution case. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

An Approach for Evolution-Driven Method Engineering

81

INTRODUCTION To manage the increasing complexity of information systems (IS), IS engineers ask for new methods, taking into account the specific situation of each IS development project. This problem is considered by the situational method engineering (SME) discipline. Instead of looking for universally applicable methods that was the idea of traditional method engineering (ME), SME proposes to develop project-specific methods or to adapt existing ones to specific project situations (Kumar & Welke, 1992). Therefore, each IS development project starts with the definition of its proper method that best fits its situation. It is clear that traditional method construction techniques are too expensive and time-consuming, and are not appropriate to tackle the project-specific method construction. As a consequence, the aim of SME is to provide fast and simple method construction and adaptation techniques and tools. In the next section we survey the research achievements in this domain. In this work we consider method engineering from the evolutionary point of view. In other words, we look for an approach supporting evolution of an existing method, model, or meta-model in order to obtain a new one better adapted for a given engineering situation and /or satisfying different engineering objective. We consider such a method evolution as situation driven and relate our work to the area of SME. The approach that we propose in this chapter is based on some initial model or meta-model that we call the ‘paradigm model’ and supports the evolution of this paradigm model into a brand-new model satisfying another engineering objective. That is why we call this approach “evolution-driven method engineering.” We embedded in this approach our method engineering experience and especially that gained in the meta-modeling domain. The hypothesis of this approach is that a new method is obtained either by abstracting from an existing model or by instantiating a meta-model. We have evaluated our approach in the Franco-Japanese collaborative research project Lyee dealing with the Lyee methodology improvement. Lyee, which stands for GovernmentaL MethodologY for SoftwarE ProvidencE, is a methodology for software development used for the implementation of business software applications. Lyee was invented by Negoro (2001a, 2001b). The aim of this project was to develop a methodology supporting software development in two steps: requirements engineering and code generation. The latter was already supported by the LyeeALL CASE tool proposed by Negoro (2001a, 2001b) in order to generate programs, provided a set of well-formatted software requirements are given. The Lyee Software Requirements Model (LSRM) expresses these requirements in rather low-level terms such as screen layouts and database accesses. Moreover they are influenced by the LyeeALL internals such as the Lyee identification policy of program variables, the generated program structure, and the Lyee program execution control mechanism. ExpeCopyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


rience with LyeeALL has shown the need to acquire software requirements from relatively high-level user-centric requirements. For this reason, we have decided to make the Lyee methodology evolve. We have used the existing LSRM as a baseline paradigm model for the more abstract Lyee User Requirements Model (LURM) construction. In the next section we review the existing SME approaches in order to better situate our approach. Then, we outline our process model for evolution-driven ME and detail the abstraction strategy for method product model construction and the pattern-based strategy for method process model definition. Both strategies are illustrated by the LURM product and process models creation respectively. Finally, we finish this chapter by some conclusions and discussions about our future work.

RELATED RESEARCH A number of SME approaches have already been proposed in the literature. Most of them use an assembly technique based on the reuse of existing method parts in the construction of new methods or in the enhancement of existing ones. The principal preoccupations of such approaches are the definition of reusable method components, the construction of repositories for their storage, the definition of guidelines for their selection and assembly, and the development of computer-aided method engineering (CAME) tools supporting assembly process. For this purpose, Harmsen, Brinkkemper, and Oei (1994) introduce the notion of method fragment as a reusable part of a method. They propose two kinds of fragments: product and process fragments in order to capture the corresponding method perspectives. Plihon et al. (1998) propose the notion of method chunk, which is refined in Rolland, Plihon, and Ralyté (1998) and Ralyté and Rolland (2001a). In contrary to the method fragment, method chunk couple method product and process perspectives into the same module in order to emphasize their coherency and autonomy. Both of these notions, method fragment and method chunk, represent the basic blocks for constructing ‘onthe-fly’ methods. Van Slooten and Hodes (1996) combine method fragments into route maps. A complete route map represents a system development method. It is not always simple to extract reusable method components from existing methods. Ralyté and Rolland (2001b) propose a process model for method reengineering into collection of method chunks, which could be stored in a method repository. Different method repositories are given in Saeki, Iguchi, Wen-yin, and Shinohara (1993), Van Slooten and Brinkkemper (1993), Harmsen (1997), and Ralyté (1999). Following assembly-based approaches, new methods can be constructed by selecting method fragments/chunks from a method repository in such a way that they must fit project method requirements. Van Slooten and Hodes (1996) Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


83

specify project situation by using a set of contingency factors; Punter and Lemmen (1996) use a specific framework to characterize problem situation, whereas Ralyté (2002) provides a process model for method requirements definition that has the form of a requirements map. Most assembly-based SME approaches provide guidelines for the nonoverlapping method fragments assembly (Brinkkemper, Saeki, & Harmsen, 1998; Punter & Lemmen, 1996), whereas Ralyté and Rolland (2001a) enrich the assembly process by a new strategy allowing to assemble overlapping method chunks that have similar objectives but provide different manners to fulfill them. Song (1997) proposes a slightly different method assembly approach, advising two kinds of method components integration: function-driven and quality-driven. The first one is similar to the assembly-based approaches introduced above and is based of the integration of components providing complementary functionalities for system modeling, whereas the second one helps to improve the existing method quality by adding new properties, principles, notations, metrics, etc., borrowed from other methods. CAME environments, such as Decamerone (Harmsen & Brinkkemper, 1995), MetaEdit+ (Kelly, Lyytinen, & Rossi, 1996), and MViews (Grundy & Venable, 1996) provide support for method engineering process. They use method engineering languages such as MEL (Brinkkemper et al., 1998; Saeki, 2003) and CoCoA (Venable, 1993) for method fragments/chunks specification. Another kind of SME approach uses generic conceptual patterns for method construction and extension. Rolland and Plihon (1996) and Rolland and Prakash (1996) introduce the notion of method construction pattern to capture generic lows governing the construction of different but similar methods. A pattern models a common behavior in method construction. Decision-making patterns capturing the best practices in enterprise modeling are proposed by Rolland, Nurcan, and Grosz (2000) to support enterprise knowledge development process. Deneckere and Souveyet (1998) propose domain-specific patterns for existing method extension. In 1998, Tolvanen proposed an approach for incremental method engineering based on the existing method refinement. The principle of this approach is to capture experience of method practice in different projects, and to refine its meta-model and corresponding tool if some problems have been detected in method use or if it is required by the new project situation. A generic product model to construct methods for different application domains is provided in Prakash and Bhatia (2002). Ralyté, Deneckère, and Rolland (2003) propose a generic process model for SME, allowing to combine different SME approaches. The evolution-driven ME approach that we propose in this work is similar to the Tolvanen’s incremental ME approach, as it also allows improving an existing model or meta-model and adapting it to a given situation. But our Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


approach is not limited to such kinds of evolution; it also helps to construct other models or meta-models satisfying different engineering objectives than the one of the initial paradigm model.

PROCESS MODEL FOR EVOLUTION-DRIVEN METHOD ENGINEERING We use the map formalism proposed in Rolland, Prakash, and Benjamen (1999) to express the process model of our approach for evolution-driven method engineering. Map provides a representation system allowing one to combine multiple ways of working into one complex process model. It is based on a nondeterministic ordering of two fundamental concepts, intentions and strategies. An intention represents a goal that can be achieved by the performance of the process. It refers to a task (activity) that is a part of the process and is expressed at the intentional level. A strategy represents the manner in which the intention can be achieved. Therefore, the map is a directed labeled graph with nodes representing intentions and labeled edges expressing strategies. The directed nature of the map identifies which intention can be done after a given one. A map includes two specific intentions, start and stop, to begin and end the process respectively. There are several paths from start to stop in the map for the reason that several different strategies can be proposed to achieve the intentions. A map therefore includes several process models that are selected dynamically when the process proceeds, depending on the current situation. An intention achievement guideline is associated to every triplet <source intention, target intention, strategy> providing advice to fulfill the target intention following the strategy, given the source intention has been achieved. Furthermore, this guideline can be refined as an entire map at a lower level of granularity. Our approach for evolution-driven ME uses meta-modeling as its underlying method engineering technique. Meta-modeling is known as a technique to capture knowledge about methods. It is a basis for understanding, comparing, evaluating, and engineering methods. One of the results obtained by the metamodeling community is the definition of any method as composed of a product model and a process model (Prakash, 1999). A product model defines a set of concepts, their properties, and relationships that are needed to express the outcome of a process. A process model comprises a set of goals, activities, and guidelines to support the process goal achievement and the action execution. Therefore, method construction following the meta-modeling technique is centered on the definition of these two models. This is reflected in the map representing the process model for evolution-driven ME (Figure 1) by two core



85

Figure 1. Process model for evolution-driven method engineering Start Instantiation Utilisation strategy strategy

Abstraction strategy

Refinement strategy

Construct a product model Simple strategy

Stop

Completeness strategy

Adaptation strategy

Context-driven

Construct a process model

Patterndriven

Strategy-driven

intentions (the nodes of the map), construct a product model and construct a process model. A number of product meta-models (Grundy & Venable, 1996; Hofstede, 1993; Prakash & Bhatia, 2002; Saeki et al., 1994; Plihon, 1996) as well as process meta-models (Jarke, Rolland, Sutcliffe, & Domges, 1999; Rolland, Souveyet, & Moreno, 1995; Rolland et al., 1999) are available, and our approach is based on some of them. This is shown in Figure 1 by several different strategies (the labeled edges) to achieve each of the two core intentions. The construction of the product model depends of the ME goal that could be to construct a method: • by raising (or lowering) the level of abstraction of a given model, • by instantiating a selected meta-model, • by adapting a meta-model to some specific circumstances, • by adapting a model. Each of these cases defines a strategy to construct a product model, namely the abstraction, instantiation, adaptation, and utilization strategies. Each of them is supported by a guideline that consists of defining various product model elements such as objects, links, and properties in a different manner. In our example, we use the Lyee Software Requirements Model (LSRM) model as a baseline paradigm model for the more abstract Lyee User Requirements Model (LURM) construction. In this case, the abstraction strategy is the more appropriate one to construct a product model, as the ME goal is to raise the level of abstraction of the LSRM. For this reason, in the next section we detail



and illustrate the guideline supporting product model construction following the abstraction strategy. This guideline is based on the abstraction of different elements from the paradigm model (product and/or process model) into elements in the new product model and the refinement of the obtained elements until the new product model became satisfactory. Process model must conform to the product model. Process steps, activities, and actions always refer to some product model parts in order to construct, refine, or transform them. This is the reason why in the map of Figure 1, the intention to construct a process model follows the one to construct a product model. We know that a process model can take multiple forms. It could be a simple informal guideline, a set of ordered actions or activities to carry out, a set of process patterns to be followed, etc. In our evolution-driven process model (Figure 1), we propose four strategies: simple, context-driven, pattern-driven, and strategy-driven to construct a process model. • The simple strategy is useful to describe an uncomplicated process model that can be expressed as a textual description or a set of actions to execute. • The context-driven process model is based on the NATURE process modeling formalism (Jarke et al., 1999; Rolland et al., 1995). According to this formalism, a process model can be expressed as a hierarchy of contexts. A context is viewed as a couple <situation, intention>. The situation represents the part of the product undergoing the process, and the intention reflects the goal to be achieved in this situation. • The process model obtained following the pattern-driven strategy takes the form of a catalogue of patterns. Each pattern identifies a generic problem, which could occur quite often in the product model construction, and proposes a generic solution applicable every time the problem appears. A generic solution is expressed as set of steps allowing to resolve the corresponding problem. • Finally, the strategy-driven process model, also called the map (Rolland et al., 1999; Benjamen, 1999) (see the first paragraph of this section), permits one to combine several process models into one complex process model. The process model of the LURM was defined following the pattern-driven strategy. A set of patterns has been defined to take into account different situations in the user requirements definition. Each pattern provides advice to capture and formulate requirements. In this chapter we present in detail and illustrate the guideline supporting the pattern-driven strategy for the process model construction.



87

ABSTRACTION-BASED PRODUCT MODEL CONSTRUCTION The abstraction strategy for the product model construction consists of defining a new product model representing the level of abstraction higher than the one of its paradigm model. As a consequence, the objective of the corresponding guideline is to support the construction of a product model as an abstraction of another model (product or process or both of them). This guideline is also expressed by the map shown in Figure 2. As the product model construction consists of the definition of its elements (objects, properties, links), there is only one core intention in this map called define product element. The achievement of this intention is supported by a set of strategies. Two strategies named product-driven abstraction and processdriven abstraction are provided to start the construction process. The first one deals with the paradigm product model, whereas the second one is based on the paradigm process model. The product-driven abstraction consists of analyzing the paradigm product model, identifying elements that could be represented in a more abstract form in the new model, and defining these abstract elements. The process-driven abstraction proposes to analyze the paradigm process model and to abstract some of its activities into the upper-level ones. The product elements referenced by these more abstract activities must be integrated into the product model under construction. The concepts obtained following this strategy have to match concepts (or a collection of concepts) of the paradigm product model. The top-down mapping strategy can be applied to assure it. The generalization, specialization, aggregation, and decomposition strategies are used to refine the model under construction, whereas the linking strategy helps to connect different elements of this model obtained by applying different abstraction strategies. Figure 2. Abstraction-based product model construction Start

Product-driven abstraction

Process-driven abstraction

Top-down mapping

Linking strategy

Define product element

Decomposition strategy Aggregation strategy

Specialisation strategy Generalisation strategy

Completeness strategy

Stop



Figure 3. Lyee software requirements model (LSRM) PNTR

PNTA

PNTN

Logical Unit

InterSF

PNTE

Condition

IntraSF

PNTM

Routing Word

LogicalID Device

L3 – condition L4 – formula Name Domain

PNTC PNTD

Word

PRD1

WordID

NextpalletID PRD PRDName

SFID

POP1

Action Word

1..* 1

W04

1..*

Scenario Function

Domain Word

1

W02

1

Pallet PalletI

1

Word in Pallet /Unit

PCL1 PCR1

PCR2

PBOX PWT1

W03

In order to illustrate the abstraction-based product model construction, we present first our paradigm model, which is the Lyee Software Requirements Model depicted in Figure 3. The central concept in the LSRM is called a word. A word corresponds to a program variable: input words represent values captured from the external world, whereas output words are produced by the system by applying specific formulae. Lyee Software Requirements processing mechanism applies a formula to obtain output words from the given input words. The execution of formulae is controlled by the Process Route Diagram (PRD). A PRD is composed of scenario functions (SFs), composed of pallets that are made of vectors. In order to carry out the generated program control, the function generates its own words, such as the action words and routing words. Action words are used to control physical input/output exchanges in a Lyee program; they implement application actions such as reading a screen, submitting a query to a database, opening or closing a file, etc. Routing words are used to distribute the control over various SFs of a PRD. In order to comply with the LSRM paradigm, the LURM should be centered on a notion that abstracts from the concept of word. Obviously words required by the Lyee processing mechanism are not relevant at this level. On the contrary, the concern is only with domain words. For that reason, the LSRM concept domain word is abstracted into LURM concept item following the productdriven abstraction strategy. The specialization strategy is applied in order to specialize the item into output and input to match the LSRM, which makes the difference between input and output words used in its processing mechanism. An output is produced by the system, whereas the input is captured from the user. In the same manner, the input is specialized into active and passive. The former triggers the system actions, whereas the latter represents values captured from the user. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


89

Next we analyze the LSRM process model. The paradigm process model deals with the generation of the Lyee program structure. The result of the obtained program execution must fit user requirements. In other words, it must allow the user to satisfy one of its goals. For that reason, in the upper user requirements level, we need to reason with concepts that allow identification of these user goals and express how the user interacts with the system in order to achieve them. The process-driven abstraction strategy allows us to define the notion of interaction, representing the exchanges between the user and the system from the user’s viewpoint. An interaction is goal driven in the sense that the user asks the system to achieve the goal he/she has in mind without knowing how the system will do it. As a result, we associate an interaction goal to each interaction. The complexity of the interaction goal defines the complexity of the corresponding interaction. If the interaction goal can be decomposed into several atomic goals, the corresponding interaction can also be decomposed. Consequently, we specialize the interaction into atomic and compound thanks to the specialization strategy. Now we need to define how the interaction concept could be mapped to the concepts defined at the lower LSRM product model. Any of the LSRM concepts does not correspond the interaction of the LURM directly. However, the top-down mapping strategy suggests that an Interaction could be expressed as a combination of items that match the LSRM Domain word concept. An atomic interaction delineates a number of input and output data: the user provides some input and receives the output that corresponds to the expected result. Therefore, the decomposition strategy helps us to decompose every interaction into four kinds of items that we call W input, Woutput, W result, and Wend. Each of them represents: • Winput: the input provided in the interaction • Wresult: the result of the goal achievement • Woutput: the output displayed to the user • Wend: the item to end the interaction Then, we consider the concept of logical unit (from LSRM) that represents a coherent set of words used in the same processing (reading or writing) and constrained by the same physical device (database, file, screens, etc.) used by the program. The concept of defined abstracts this notion in order to aggregate logically related items processed together and constrained by the same conceptual device. One defined can be specialized into one or more logical units. For example, one defined corresponding to a conceptual screen can be implemented by two physical screens requiring four logical units. To sum up, the productdriven abstraction strategy followed by the linking strategy allows us to create the defined concept and to connect it with the items composing it.



Similarly, the concept of PSG, the Precedence Succedence Graph, was obtained by abstraction of the PRD concept from the paradigm product model. A PSG specifies the ordering conditions between defineds, as the PRD do with words. The decomposition strategy was applied to represent the structure of the PSG as a graph composed of links and nodes. Following the top-down mapping strategy, we recognize that the link matches the LSRM InterSF concept that captures different links between the scenario functions in a PRD, whereas the node corresponds the scenario function concept. Thanks to the specialization strategy, the link was specialized into duplex, continuous, and multiplex, whereas the node was specialized into begin, end, and intermediate. Every defined is an intermediate link in at least one PSG. Figure 4 summarizes the abstraction process from the lower LSRM into upper LURM.

Figure 4. Lyee product models for software requirements and for user requirements 1

Interaction goal

Lyee User Requirements Model

Interaction

Winput

End

1..*

Name Type

Node NodeID

PSG

Name 1..* Domain

1

source

{complete, or}

0..*

PSGName 0..*

target

Link

Condition Formula

{complete, or}

Continuous PNTR

PNTA

Multiplex PNTN

Logical Unit

L3 – condition L4 – formula Name Domain

PNTC PNTD

PNTM

Word

Routing Word

WordID

Action Word Word Action

NextpalletID

PRD PRDName

Active

Domain Word

LogicalID Device

IntraSF

Condition

{complete, or}

Passive

InterSF

PNTE

Input

Output

1..* Condition

Duplex

1..* 1

W04

1..*

Scenario Function SFID

1

W02

Wend

Item

Defined

Intermediate 1..*

1..*

Wresult

Atomic

Compound Begin

Woutput

1

Word in Pallet /Unit

PRD1 POP1 PCL1

PCR1

PCR2

PBOX

PWT1

Pallet PalletI

1

W03

Lyee Software Requirements Model



91

PATTERN-BASED PROCESS MODEL CONSTRUCTION The pattern-based process model construction strategy is based on the concept of pattern, which was introduced by Alexander (1977) in architecture and borrowed by IT engineers to capture software design knowledge (Gamma, Helm, Johnson, & Vlissides, 1994; Coad, North, & Mayliefd, 1996; Coplien & Schmidt, 1995; Fowler, 1997), as well as method engineers to capture reusable method knowledge (Rolland & Prakash, 1996; Deneckère & Souveyet, 1998). According to Alexander, a pattern refers to “a problem that occurs again and again in our environment and describes the core of the solution to that problem, in such a way that you can use this solution a million times over, without ever doing it the same way twice.” The key idea of a pattern is thus, to associate a problem to its solution in a well-identified context. Figure 5 shows the pattern meta-model. The problem refers to the situation in which pattern can be applied and the goal to achieve in this situation. The situation is characterized by a set of product elements. The solution is represented by a set of steps to realize in order to resolve the problem. A pattern can be simple or compound. The solution of a compound pattern contains steps which call other patterns and are named pattern steps in the contrary to stand alone steps which are executed. The process model for pattern construction is defined by a map based on two core intentions: identify a pattern and construct a pattern (Figure 6). To identify a pattern means to identify a generic problem. As shown in Figure 6, the problem identification can be based on the discovery of a typical situation or a generic goal in the method context. The two cases are respectively supported by two strategies: situation-based and goal-driven. The aggregation strategy allows one to combine several patterns into a compound one in order to propose Figure 5. Pattern meta-model

1 1

1..*

1

Problem

1

Pattern 1

Compound 1

Solution

1

Situation

Simple solution

Product element

*

Application example

1

Simple 1

*

Goal

0..1

1

*

Complex solution

1..*

Product Structure Step

1 calls 1..*

Stand alone step

Pattern step

*



Figure 6. Pattern-based process model construction Start Goal-driven strategy Precedence strategy Succedence strategy

Stop Completeness strategy

Construct a pattern

Situation-based strategy

Identify a pattern

Aggregation strategy

Decomposition strategy

Product-driven strategy Goal-driven strategy

solutions for complex problems, whereas the decomposition strategy deals with the identification of subproblems, which could also be considered generic problems. The identification of a new pattern situation advises us to consider that there must be another pattern creating this situation. This case is supported by the precedence strategy. To construct a pattern means to formalize its problem (the situation and the goal), to define the solution to its problem as a set of steps to execute, to define its product structure, and to give some examples of its application. Two strategies named product-driven and goal-driven are provided for this purpose (Figure 6). The guideline supporting the product-driven strategy is based on the transformation of the product elements from the pattern situation into the product element defined as the pattern target (pattern goal target). The goal-driven strategy deals with the pattern goal reduction into a set of atomic actions to be carried out in order to achieve this goal. The succedence strategy considers that the result product obtained by applying an already defined pattern can be considered as a potential situation for the definition of another pattern. In order to define the patterns supporting LURM construction, we need to identify typical situations (the problem) in the Lyee user requirements capture (the context) and to define the corresponding guidelines (the solution) assisting in the requirements elicitation and formulation. As shown in Figure 6, we can start the pattern identification process following one of two strategies: goaldriven or situation-based. The guidelines supporting these two strategies supplement each other and there is no preestablished order to realize them. In our case, we start the pattern identification process following the goal-driven strategy, and we consider the core LURM objective ‘to define user requireCopyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


93

ments’. As stated in the previous section, the LURM defines user requirements as user-system interactions. Therefore, we found our reasoning on the notion of atomic interaction and investigate the possibility to identify generic activities for requirements capture within this context. We deduce that the requirements capture related to an atomic interaction comprises four activities that can be considered as four potential pattern goals: • to start the interaction (formulate to start requirements), • to perform the action (formulate to act requirements), • to prepare the output (formulate to output requirements), and • to end the interaction (formulate to end requirements). Each of these activities is linked to the item typology introduced in the previous section, as each activity is associated to one type of item: • the formulate to start requirements deals with the capture of Winput, • the formulate to act requirements is concerned by the calculation of W result , • the formulate to output requirements shall help eliciting and defining Woutput, and • the formulate to end requirements considers Wend. Each requirement activity is concerned with the elicitation and definition of these items, their grouping in defineds, and the positioning of those in the interaction PSG. Next, we select the situation-based strategy to identify a pattern (Figure 6) and consider the possible situations in which these goals are relevant. For instance, we distinguish two different situations dealing with the capture of Winput: either the input value does not exist and is directly captured from the user, or it exists in a database or a file and is captured from this container. As a consequence, we identify two patterns having the same goal, formulate to start requirement, but dealing with different situations, input capture from the user and input capture form the internal device. We call these two patterns respectively immediate start and prerequisite for start. In the same manner we identify two generic situations for each of the four generic goals and identify eight generic patterns. Table 1 characterizes the discovered patterns. Each of these eight patterns deals with one single requirement activity, whereas to get the complete set of requirements for a given problem, the requirements engineer has to perform one of each type of activity. The complete set of requirements requires that each of the following be performed once: to start, to act, to output, and to end. To obtain advice on this, a new pattern, Pattern P9, is introduced thanks to the composition strategy. The succedence strategy for pattern identification suggests that we think about the construction of a compound interaction that could be based on the Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


iteration of an atomic interaction creation guided by the pattern P9. As a result, we identify a new pattern for a compound interaction formulation that we call P10 Complex Composition (Table 1). Let’s illustrate now the construction of a pattern solution. In our example, the pattern solution takes the form of a sequence of rules to be applied by the engineer. Each of them mentions an action to perform like ‘construct a hierarchy of intermediate words involved in the calculation of the result word’. Most of these actions are identifying a requirement, i.e., referring to an element of the meta-model (LURM): Defined, item, node, and link in the PSG, as for example ‘introduce a defined of type screen’.

Table 1. Characterization of the identified patterns Goal Formulate To Start requirements Formulate To Start requirements Formulate To Act requirements

Situation Characterization W input captured directly from the user Winput retrieved from a database or a file Wresult calculated by simple formulae, which do not require the calculation of the intermediate words Formulate To Act Wresult calculated by complex requirements formulae, which require the calculation of the intermediate words and possibly the access to the data in a file or a database Formulate To Output There is no obstacle, neither in the capture of Winput nor in the requirements production of Wresult. Formulate To Output A number of different cases of requirements output production shall be considered due to possible obstacles either in the capture of Winput or in the production of Wresult. Formulate To End The interaction ends normally requirements without additional internal activity. Formulate To End Some internal activity shall be performed such as storing part requirements or the totality of Woutputs. Formulate requirements for an The interaction goal is atomic. atomic interaction Formulate requirements for a The interaction goal is compound interaction compound.

Pattern Name P2 Immediate Start P3 Prerequisite for Start P1 Simple Word

P8 Complex Word

P6 Single Output

P7 Multiple Output

P4 Simple End

P5 Compound End

P9 Simple Composition P10 Complex Composition



95

As an example we propose the construction of the pattern P2 following the product-driven strategy. The objective of this pattern is to prepare a usersystem interaction. The product-driven strategy advises instantiation of the meta-model elements necessary to achieve the pattern goal. In this case we need to instantiate the meta-model elements: defined, item, and PSG, which are necessary for the input values capture. As a consequence, the actions to perform should be: • to create the defined for the necessary input values capture, • to define an item to each input value, • to link the items to the defined, • to type items as input and passive, and • to create the relevant part of the PSG. Next, we need to define the pattern product structure. The pattern product structure is an instance of the meta-model representing the configuration of concepts to be instantiated in any application. In the case of the pattern P2, a PSG must be created containing a begin node, a continuous link, and an intermediate node corresponding to the defined of type screen (called Sinput) composed of the elicited items. Figure 7 shows the pattern P2, its problem, solution, and product structure.

Figure 7. Pattern P2: Immediate start Pattern P2 : Immediate Start Problem: < goal: Formulate ‘To Start’ Requirement > < situation: Winput Solution: 1. Create a Defined Sinput of type screen. Determine its name 2. Elicit Items accociated to Winput 3. Link these Items to the Defined. Determine for each Item its name and domain 4. Type Items as Input and Passive 5. Create a PSG with the Defined as Intermediate node and link from the Start node with a Continuous link Product Structure: Defined :PSG

Type

Name Type

PsgName

(Type = Screen) source

Begin Node1

1

1

Continuous Condition

target 1 1

Intermediat Node2e

S input Defined

1..*

Passive Name Domain



In the same manner we construct all the patterns from P1 to P8. The pattern P9 can be constructed following the goal-driven strategy, which advises decomposition of the principal goal into sub-goals until the atomic actions have been obtained. Thus, the objective of the pattern P9 ‘formulate requirements for an atomic interaction’ can be decomposed into four sub-goals ‘formulate to start requirements’, ‘formulate to act requirements’, ‘formulate to output requirements’, ‘formulate to end requirements’, in this order. As there are always two patterns that are candidates to help achieve the goal, it is necessary to examine the situation first. As pattern situations are exclusive, the choice of the relevant pattern to apply is easy. The obtained pattern is a compound one. It is shown in Figure 8. Finally, the pattern P10 deals with the compound interaction. The goal to be achieved is to get a complete and coherent requirement formulation for a compound interaction. This pattern should give advice on how to decompose a compound interaction into atomic interactions to which the pattern P9 should be applied. In fact, the pattern helps in recognizing that the interaction is not an atomic one in the first place. Each of 10 patterns captures a requirement situation and guides the formulation of the requirement in compliance with the requirement meta-model. The 10 patterns will be applied again and again in the different software projects using Lyee. Even though actual situations are different from one project to another, each of them should match one pattern situation and the pattern will bring the core solution to the requirements capture problem raised by this situation.

Figure 8. Pattern P9: Simple composition Pattern P9 : Simple Composition Problem: < goal: Formulate requirement for an atomic interaction > < situation: The interacion goal is atomic > Solution: 1. Formulate To Start equirement Determine Apply the situation pattern to Start Apply P2 Apply P3

Formulate requirement for an atomic interaction 2. Formulate To Act requirement Determine Apply the situation pattern to Act Apply P1 Apply P8

3. Formulate To Output requirement Determine Apply the situation pattern to Output Apply P6

4. Formulate To End requirement Determine the situation to End

Apply P7

Apply pattern

Apply P1 Apply P8



97

CONCLUSIONS In this chapter we propose an approach for evolution-driven method engineering. Evolution in this case means that we start method engineering with an existing paradigm model (or meta-model) and we obtain a new model (or meta-model) by abstracting, transforming, adapting, or instantiating this paradigm model. Our process model for evolution-driven ME captures these various evolution ways as different strategies to create the product part of the model under construction. The corresponding process part construction is also supported by a set of strategies, the selection of which depends on the process nature and complexity. Every strategy is supported by a guideline, assisting the method engineer in his or her method evolution task. In this chapter we present the evaluation of our approach by the Lyee User Requirements Model construction as evolution of the Lyee Software Requirements Model. The abstraction strategy has been used to construct a product model, while the pattern-driven strategy was applied to construct a process model. In this chapter we present these two strategies in more detail and illustrate their application. Our future preoccupation is to improve, specify, and evaluate all the proposed method evolution strategies. We also think about other possible ways to evolve methods. For example, use of ontologies can be considered as another manner to help method engineers discover the right concepts and to define them in the method under construction. The flexibility offered by the map formalism that we use to express our evolution-driven ME process model allows us to include new guidelines to improve, transform, or extend existing models and meta-models in a rather simple manner. They can be integrated as different strategies to satisfy the intentions construct a product model and construct a process model. Besides, we would like to validate this method engineering approach through other real projects requiring specific methods.

REFERENCES Alexander, C., Ishikawa, S., Silverstein, M., Jacobson, M., Fiksdahl-King, I., & Angel, S. (1977). A pattern language. New York: Oxford University Press. Benjamen, A. (1999). Une approche multi-démarches pour la modélisation des démarches méthodologiques. Unpublished doctoral dissertation, University of Paris 1–Sorbonne, France. Brinkkemper S., Saeki, M., & Harmsen, F. (1998). Assembly techniques for method engineering. Proceedings of the 10th International Conference on Advanced Information Systems Engineering (CAiSE’98), Pisa, Italy, pp. 381-400, Springer, LNCS Volume 1413. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


Coad, D., North, D., & Mayliefd, M. (1996). Object models—strategies, patterns and applications. Yourdon Press (Computing Series). Coplien, J.O., & Schmidt, D.O. (Eds.). (1995). Patron languages of program design. Reading, MA: Addison-Wesley. Deneckère, R., & Souveyet, C. (1998). Patterns for extending an OO model with temporal features. Proceedings of the International Conference on Object-Oriented Information Systems (OOIS’98), Paris, France. Fowler, M. (1997). Analysis patterns: Reusable objects models. Reading, MA: Addison-Wesley. Gamma, E., Helm, R., Johnson, R., & Vlissides, J. (1994). Design patterns: Elements of reusable object-oriented software. Reading, MA: AddisonWesley. Grundy, J.C., & Venable, J.R. (1996). Towards an integrated environment for method engineering. In W.W. Cotterman & J.A. Senn (Eds.), Challenges and strategies for research in systems development (pp. 45-62). Chichester, UK: John Wiley & Sons. Harmsen, A.F. (1997). Situational method engineering. Moret Ernst & Young. Harmsen, A.F. & Brinkkemper, S. (1995). Design and implementation of a method base management system for situational CASE environment. Proceedings of the 2nd APSEC Conference (pp. 430-438). IEEE Computer Society Press. Harmsen, A.F., Brinkkemper, S., & Oei, H. (1994). Situational method engineering for information system projects. In T.W. Olle & A.A. Verrijn Stuart (Eds.), Methods and associated tools for the information systems life cycle — Proceedings of the IFIP WG8.1 Working Conference (CRIS’94) (pp. 169-194). Amsterdam: North-Holland. Hofstede, A.H.M. ter. (1993). Information modeling in data intensive domains. Dissertation, University of Nijimegen, The Netherlands. Jarke, M., Rolland, C., Sutcliffe, A., & Domges, R. (1999). The NATURE requirements engineering. Aachen: Shaker Verlag. Kelly, S., Lyyttinen, K., & Rossi, M. (1996). Meta Edit+: A fully configurable, multi-user and multi-tool CASE and CAME environment. Proceedings of the CAiSE’96 Conference, Heraklion, Crete, Greece. Berlin: Springer Verlag, LNCS Volume 1080. Kumar, K., & Welke, R.J. (1992). Method engineering, a proposal for situationspecific methodology construction. In Cotterman & Senn (Eds.), Systems analysis and design: A research agenda (pp. 257-268). John Wiley & Sons. Negoro, F. (2001a). Methodology to determine software in a deterministic manner. Proceedings of ICH, Beijing, China. Negoro, F. (2001b). A proposal for requirement engineering. Proceedings of ADBIS, Vilnius, Lithuania. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


99

Plihon, V. (1996). Un environnement pour l’ingénierie des methods. Unpublished doctoral dissertation, University of Paris 1–Sorbonne, France. Plihon, V., Ralyté, J., Benjamen, A., Maiden, N.A.M., Sutcliffe, A., Dubois, E., and Heymans, P. (1998). A reuse-oriented approach for the construction of scenario-based methods. Proceedings of the International Software Process Association’s 5th International Conference on Software Process (ICSP’98), Chicago, Illinois. Prakash, N. (1999). On method statics and dynamics. Information Systems, 34(8), 613-637. Prakash, N., & Bhatia, M.P.S. (2002). Generic models for engineering methods of diverse domains. Proceedings of CAISE’02 (p. 612), Toronto, Canada. Berlin: Springer, LNCS Volume 2348. Punter, H.T., & Lemmen, K. (1996). The MEMA model: Towards a new approach for method engineering. Information and Software Technology, 38(4), 295-305. Ralyté, J. (1999). Reusing scenario-based approaches in requirement engineering methods: CREWS method base. Proceedings of the First International Workshop on the Requirements Engineering Process — Innovative Techniques, Models, Tools to Support the RE Process, Florence, Italy, September. Ralyté, J., & Rolland, C. (2001a). An assembly process model for method engineering. Proceedings of the 13th International Conference on Advanced Information Systems Engineering (CAISE’01), Interlaken, Switzerland, pp.267-283, Springer LNCS Volume 2068. Ralyté, J., & Rolland, C. (2001b). An approach for method reengineering. Proceedings of the 20th International Conference on Conceptual Modeling (ER2001), Yokohama, Japan, pp.471-484, Springer LNCS Volume 2224. Ralyté, J., Deneckère, R., & Rolland, C. (2003). Towards a generic model for situational method engineering. Proceedings of the 15th International Conference on Advanced Information Systems Engineering (CAISE’03) (pp. 95-110), Klagenfurt/Velden, Austria. Berlin: Springer, LNCS Volume 2681. Rolland, C., & Plihon, V. (1996). Using generic chunks to generate process models fragments. Proceedings of the 2nd IEEE International Conference on Requirements Engineering (ICRE’96), Colorado Springs. Rolland, C., & Prakash, N. (1996). A proposal for context-specific method engineering. Proceedings of the IFIP WG 8.1 Conference on Method Engineering (pp. 191-208), Atlanta, Georgia. Chapman and Hall. Rolland, C., Nurcan, S., & Grosz, G. (2000). A decision-making pattern for guiding the enterprise knowledge development process. Journal of Information and Software Technology, 42, 313-331.



Rolland, C., Plihon, V., & Ralyté, J. (1998). Specifying the reuse context of scenario method chunks. Proceedings of the 10th International Conference on Advanced Information System Engineering (CAISE’98), Pisa, Italy. Rolland, C., Prakash, N., & Benjamen, A. (1999). A multi-model view of process modeling. Requirements Engineering Journal, 4(4), 169-187. Rolland, C., Souveyet, C., & Moreno, M. (1995). An approach for defining ways-of-working. Information Systems Journal, 20(4), 337-359. Saeki, M. (2003). Embedding metrics into information systems development methods: An application of method engineering technique. Proceedings of the 15th International Conference on Advanced Information Systems Engineering (CAISE’03) (pp. 374-389), Klagenfurt/Velden, Austria. Berlin: Springer, LNCS Volume 2681. Saeki, M., & Wen-yin, K. (1994). Specifying software specification and design methods. Proceedings of the International Conference on Advanced Information Systems Engineering (CAISE’94) (pp. 353-366). Berlin: Springer, LNCS Volume 811. Saeki, M., Iguchi, K., Wen-yin, K., & Shinohara, M. (1993). A meta-model for representing software specification & design methods. Proceedings of the IFIPWG8.1 Conference on Information Systems Development Process (pp. 149-166). Song, X. (1997). Systematic integration of design methods. IEEE Software. Tolvanen, J.-P. (1998). Incremental method engineering with modeling tools: Theoretical principles and empirical evidence. PhD dissertation, University of Jyväskylä, Finland. Van Slooten, K., & Brinkkemper, S. (1993). A method engineering approach to information systems development. In N. Prakash, C. Rolland, & B. Pernici (Eds.), Information systems development process. Amsterdam: Elsevier Science Publishers B.V. (North-Holland). Van Slooten, K., & Hodes, B. (1996). Characterizing IS development project. Proceedings of the IFIP WG 8.1 Conference on Method Engineering (pp. 29-44). Chapman and Hall. Venable, G.R. (1993). CoCoA: A conceptual data modeling approach for complex problem domains. PhD dissertation, SUNY, Binghamton.



101

Section II Goal, Requirements, and Process Modeling


102

Kavakli & Loucopoulos

Chapter VI

Goal Modeling in Requirements Engineering: Analysis and Critique of Current Methods Evangelia Kavakli, University of the Aegean, Greece Pericles Loucopoulos, University of Manchester, UK

ABSTRACT In the field of requirements engineering, goal modeling approaches have received much attention in recent years by researchers and practitioners alike. This chapter identifies the uses of these approaches in different contexts of requirements analysis phases. It examines goal modeling in terms of five methodological orientations and defines a framework for their analysis. Using this framework it provides an analysis of goal modeling approaches in a systematic and consistent manner. The aim of this analysis is to understand the best fit for purpose of different goal modeling approaches and to highlight open issues that provide a foundation for further research in this important area of requirements engineering methodology.


Goal Modeling in Requirements Engineering

103

INTRODUCTION Since the mid-1970s when requirements engineering (RE) was established as a distinct field of investigation and practice, its definition has evolved from initially being concerned with software systems (IEEE-Std.’729’, 1983; IEEEStd.’830’, 1984) to a broader perspective that extends to also incorporate aspects of systems and organizations (Greenspan, Mylopoulos, & Borgida, 1994; Loucopoulos & Karakostas, 1995; Pohl, 1996; Yu, 1997; Zave, 1997). In the last few years, the relation of RE to the organizational context has attracted much interest from the software engineering community. Research in this area is based on the premise that in designing software systems, requirement engineers aim to ‘improve’ organizational situations that are seen as problematic — or, at least, as less than perfect. Hence as a discipline, RE brings software engineering concerns closer to problems experienced in organizational settings. RE addresses the problems associated with business goals, plans, processes, etc., and systems to be developed or to be evolved in order to achieve organizational objectives (Loucopoulos & Karakostas, 1995; van Lamsweerde, 2001). Coupled to the goal-driven nature of software systems is the way RE itself is performed. RE projects require the involvement of multiple stakeholders (the sponsor organization, the system developers and users, external regulators, etc.). The question here is how do stakeholders coordinate their actions in order to provide a common result. Research in the areas of process modeling, work-flow analysis, and CSCW (Ellis & Wainer, 1994; Nurcan & Rolland, 1997; Schedin, 1995) endorses a goaldirected view. This view is based on the premise that in collaborative work situations, people do not strictly follow rules or procedures; rather they are aware of the personal and group goals and act accordingly (Smith & Boldyreff, 1995). This is especially true when people are not faced with well-structured, repetitive processes, rather they tackle ill-structured problems where both the intended outcome and the possible routes that can be followed to reach this outcome need be specified, which is usually the case in RE (Bubenko, 1995; Loucopoulos & Kavakli, 1997). This goal-driven view of the RE process is also suggested in design problem solving (the research field that deals with the creation and transformation of systems). Research in this area indicates that unsatisfied goals or needs motivate and inform design activity, instigating design efforts and providing criteria for the evaluation of design products (Smith & Browne, 1993). In addition, research in goal-driven RE has a cognitive basis in psychological research. This research has established much evidence for the influence of goals on human behavior, and for the use of strategic and goal-driven processes in many kinds of activities that humans perform (Dasgupta, 1994; Rasmussen, Pejtersen, & Goodstein, 1994).


104


It could be argued therefore that all these activities have given rise to a goaldriven RE philosophy due to their channeling of RE-related activities through the concept of goal. From a methodological perspective, this has resulted in many approaches that encourage the modeling of goals in order to understand or describe problems associated with business structures and processes and their supporting systems. We term all these efforts goal-oriented approaches. In most cases goal analysis itself is not the main focus of the approach however; explicit identification and management of goals is an integrated part of its method and plays an essential role in achieving the aims of the approach. This chapter provides an overview of the use of goal analysis methods within the context of the four RE activities: requirements elicitation, negotiation, specification, and validation. Consequently it presents a critique of these methods based on a methodological framework and a set of evaluation criteria. The aim of this analysis is threefold: (a) to understand the coverage of the area and identify possible extensions; (b) to understand the best fit for purpose of different goal-driven approaches; and (c) to highlight the potential for integration and collaboration between different approaches. Finally, the chapter concludes with a discussion of the current state of the art and insights for future developments.

GOAL-ORIENTED APPROACHES IN RE Despite the fact that there is no common definition of the RE process, four tasks to be performed have been identified (Loucopoulos & Karakostas, 1995; Nuseibeh & Easterbrook, 2000; Pohl, 1996): • requirements elicitation, • requirements negotiation, • requirements specification, and • requirements validation. Requirements elicitation is about understanding the organizational situation that the system under consideration aims to improve, and describing the needs and constraints concerning the system under development. The relevant knowledge about the problem (system) is typically distributed among many stakeholders. The objective of negotiation is to establish an agreement on the requirements of the system among the various stakeholders involved in the process. Requirements specification involves a mapping of real-world needs onto a requirements model. The wider view on RE suggests that requirements specification goes beyond the traditional approach, which concentrates on functional requirements modeling and involves the modeling of the enterprise context that provides the purpose of the intended system. Finally, the validation task intends



105

to ensure that the derived specification corresponds to the original stakeholder needs, and conforms to the internal and/or external constraints set by the enterprise and its environment. Examination of current research has highlighted a large number of cases where goal analysis techniques have proven successful in different RE contexts. In more detail, modeling of goals has been proposed during requirements elicitation in order to describe the current organizational behavior [Goal-Based Workflow (Ellis & Wainer, 1994), i* (Yu, Liu, & Li, 2001), EKD (Kavakli & Loucopoulos, 1999), GOMS (Card, Moran, & Newell, 1983)] and set the objectives for change [ISAC (Lundeberg, 1982), F3 (Bubenko, 1994)]. Equally, goal analysis techniques have been used in the context of requirements negotiation in order to assist reasoning about the need for organizational change and to provide the context within which deliberation occurs during RE [SIBYL (Lee, 1997), the reasoning loop model (Louridas & Loucopoulos, 2000), REMAP (Ramesh & Dhar, 1992)]. Modeling of goals has also been used in requirements specification to describe how organizational change can be implemented in terms of the new system’s components by relating business goals to functional and nonfunctional system specifications [KAOS (Dardenne, Lamsweerde, & Fickas, 1993), GBRAM (Anton, 1996), the NFR framework (Mylopoulos, Chung, & Nixon, 1992), the goal-scenario coupling framework (Rolland, Souveyet, & Ben Achour, 1998)]. Finally, in the context of requirements validation, goal analysis techniques have been used to define the stakeholders’ criteria against which the fitness of system components is assessed [GSN (Wilson, Kelly, & McDermid, 1995), GQM (Basili, 1993)]. The role of goal-oriented approaches in relation to the four RE activities is summarized in Table 1. An overview of the goal-oriented approaches is provided in the remainder of this section.

Table 1. Role of goal-analysis in relation to RE activities RE Activity • requirements elicitation • requirements negotiation • requirements specification • requirements validation

Goal-Analysis Contribution 1. understanding the current organizational situation 2. understanding the need for change 3. providing the deliberation context of the RE process 4. relating business goals to functional and non-functional system components 5.

validating system specifications against stakeholders’ goals

Goal-Oriented Approach GOMS, Goal-Based Workflow, i*, EKD ISAC, F3 SIBYL, REMAP, The Reasoning Loop Model KAOS, GBRAM , The NFR Framework, The Goal-Scenario Coupling Framework GSN, GQM


106


Approaches concerned with the understanding of current organizational situation focus on conceptual techniques and tools for explicitly capturing and representing, in a structured way, domain knowledge to be subsequently used to drive the system development phases. This falls into two broad categories: enterprise modeling and cognitive task analysis. Techniques in enterprise modeling describe the business environment as cooperation among different organizational actors (e.g., human individuals, IT systems, workgroups, etc.) based on the assumption that these actors share common goals and act towards their fulfillment. Enterprise models, implicitly or explicitly, represent the goals of individuals, groups, or organizations, whereby a goal is a desired condition potentially attained at the end of an action (or a process). Goals are considered a potential motivator to action, and are distinct of plans, procedures, or other means of attaining the goal. Techniques in cognitive task analysis are focused on human tasks. In this context, a goal (also called an external task) is defined as a state of a system that the human wishes to achieve. A goal is achieved using some instrument, method, agent, tool, techniques, skill, or generally, some device that is able to change the system to the desired state. Work in the area of understanding the need for change focuses on methodologies for planning, organization, and control of enterprises. Discussion of goals in this context is considered not at an individual level, but at a broader organizational level. The purpose of the analysis is to ensure that the business problems to be solved are identified and that these problems are diagnosed correctly. Approaches tackling the issue of deliberation within the RE process aim at providing appropriate conceptualizations of the RE process, as well as supplying methods for improving activities such as problem solving and decision making during RE. In this context, goals have been used in order to document and subsequently trace the history of the rationale of decisions concerning either the system that is being designed or the design process itself. Work in the area of functional and nonfunctional requirements is based on the premise that systems components satisfy some higher goal in the larger environment (Loucopoulos & Kavakli, 1995). By putting emphasis on goal analysis, these approaches explicitly link business needs and objectives to a system’s functional or nonfunctional components. The relationship between business goals and the intended functionality of a system and its quality has been addressed in terms of three broad categories: goal elaboration, scenario definition, and nonfunctional requirements definition. Finally, system validation aims at certifying that the produced system specification is in accordance with the users’ needs. The objective is to ensure a solution that is right for the user needs rather than a correct (i.e., consistent and unambiguous) specification. Validation is performed through the construction of Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


107

a safety case, a collection of documents, and data which together present clear, comprehensive, and defensible arguments that an application will be acceptably safe throughout its life. In this context the modeling of goals has been suggested in order to: (a) give safety cases a better structure; (b) explicitly link safety goals to analysis results and evidence; and (c) make rationale, assumptions, and justifications explicit.

ANALYSIS AND CRITIQUE OF CURRENT APPROACHES The previous section has presented an overview of the different research areas within RE where goals are considered as important constructs. It demonstrated that goals have been used within different RE activities, aiming to achieve a number of different objectives (indicated in Table 1). This section provides a critical analysis of the different strands of goal-driven RE research, based on a common framework for understanding goal-oriented approaches. The proposed framework is loosely based on the four worlds framework originally proposed for system engineering (Mylopoulos, 1992). Using this framework we look at goal modeling in a comprehensive manner, and reflect on current investigations and open issues that provide a foundation for further research. The usage view concerns the objectives of using goal modeling in RE. As seen previously, these can be differentiated into the following: (1) understanding the current organizational situation, (2) understanding the need for change, (3) providing the deliberation context within which the RE process occurs, (4)

Figure 1. Framework for understanding goal-oriented approaches What d oes goal modelling achieve? Usage

How are go als exp ressed?

Representation

Goal-o riented approach

Subject

What is is the nature o f goals?

Development How are go al models d eveloped and us ed?


108


relating business goals to functional and nonfunctional system components, and (5) validating system specification against stakeholders’ goals. The subject view looks at the notion of a goal and its nature. Initially, it seems difficult to discern a uniform notion of goal in RE. Indeed, the term goal has been used in different approaches to convey several meanings, including human tasks, problem-solving outcomes, desired states of the world, target concepts of human behavior, policies and orientations for acting, and so on. However, if we look at the different goal concepts in light of their suggested usage, it is possible to abstract three types of goals, namely: enterprise goals, process goals, and evaluation goals. In more detail, when goal modeling is used to understand the existing organizational situation, then the focus is on current enterprise goals, which describe what the organization currently wishes to achieve. These goals may refer to individual actors’ goals or they can describe business-wide objectives. In a similar way, approaches using goal modeling to describe how business goals relate to functional and nonfunctional system components focus on future enterprise goals that define a desired situation one wishes to reach in the future. Approaches that use goal analysis in order to understand the need for change or to provide the deliberation context within the RE process mainly put emphasis on process goals. Process goals designate any demand to be satisfied, problem to be resolved, issue to be discussed, in general anything that may arise during the RE process and requires effort to be effected, anything that can act as a goal of the RE process. Goal-oriented approaches aiming to assist RE validation focus on evaluation goals that signify the stakeholders’ criteria against which a system specification can be assessed. The notion of evaluation goals is orthogonal to the other two types of goal in the sense that they may refer both to the evaluation of the outcome of RE (the organizational system into consideration) as well as the evaluation of the RE process. The representation view concerns the way goals are expressed. Goals can be expressed in a variety of formats, using more or less formally defined notations. We differentiate between informal, semiformal, and formal approaches. Informal approaches generally use natural language text to express goals; semiformal use mostly box and arrow diagrams; finally, in formal approaches goals are expressed as logical assertions in some formal specification language. Finally, the development view concerns the way that goal models are generated and evolve. This view considers the dynamic aspects of goal-driven approaches, i.e., the proposed way-of-working and the tool support provided for enacting this way-of-working.



109

Using the framework of usage, subject, representation, and development dimensions, it is possible to evaluate each approach against a set of criteria as shown in Table 2. The results indicate how each approach fits the framework and provide a comprehensive overview of current threads in goal-oriented research. Regarding the usage perspective, current goal-oriented approaches have demonstrated that the use of goal analysis offers benefits and advantages in almost every stage involved in the RE process. This broad applicability of goal concepts in RE suggests that the elucidation and manipulation of goals is a natural and inherent part of doing RE, even though earlier RE methods have not made this explicit, and have not provided the associated support. This interpretation is plausible since requirements by their very nature represent a target to be reached, a wish to be fulfilled, a vision to be materialized. Nevertheless, Table 2 also indicates that research in the area is fragmented. Coverage of the area tends to focus each time on specific RE issues, while no research has so far taken place in order to define the overall role that goals could play in RE. Furthermore, contributions from different frameworks seem to complement each other, thus by putting the various goal-based approaches together, one could obtain a stronger framework that takes advantage of the contributions from the many streams of goal-oriented research. Regarding the subject view, the majority of goal-oriented research puts emphasis on enterprise goals and how they shape the system (organization) into consideration; that is the product of RE. Fewer approaches stress the need to model process goals that drive the RE process, while only two approaches (GSN and GQM) put emphasis on the role of goals in system evaluation. Almost none of the approaches seek to understand the interdependencies between the various notions of goal within the field. Goal-oriented research so far has mainly focused on the representation aspects of goal analysis. Indeed, a number of formats have been proposed for expressing goal concepts. Semiformal is the most widely used technique for goal model representation. Semiformal models are imprecise in the sense that: (1) the meaning of goal modeling entities is described solely by the name given to it in the diagram, and (2) the relationships between entities are loosely defined. Nevertheless, these models do provide an adequate basis for discussion between stakeholders and they also establish a framework for further analysis. Using these models stakeholders can confirm their shared view of the situation and agree on the boundary within which a more detailed analysis will be performed. Three out of the five formal approaches considered (REMAP, NFR, i*) use the Telos language to formally define their models. The popularity of Telos is due to its ontological extensibility. This allows the capture of the semantics of one level at upper meta-levels inside Telos itself, thus allowing the definition of a customized conceptual language (Greenspan et al., 1994). In KAOS, goals are Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

110


Usage

understand current org. situation understand the need for change provide the deliberation context within which RE occurs relate business goals to system components evaluate system specs against stakeholder goals

ü ü

enterprise goals

ü ü

Subject

ü ü ü ü

ü ü

ü ü

ü ü ü ü ü ü ü ü ü ü ü

ü ü

üü ü ü ü

tool support

ü ü

ü ü ü

ü ü

ü

informal

way-ofworking

GQM

ü ü ü

evaluation goals

semi-formal

GSN

NFR Framework

Goal-Scenario Coupling

GBRAM

KAOS

REMAP

The Reasoning Loop Model

SIBYL

ü ü

Formal Representation

ISAC

ü ü

process goals

Development

F3 (OM)

EKD

Goal-Based workflow

Framework Components

Cognitive Task Analysis

Goal-Oriented Approaches

i*( strategic dependency model)

Table 2. Overview of current goal-oriented research in RE

u M M F

u

u

M M M M M G G

u u u u M M M M M M M F F G G F

G

u = suggest a number of steps and associated strategies M = support for model construction, F = formal reasoning support, G = process guidance



111

formalized according to the pattern of behavior they require, using temporal logic. In particular four generic goal patterns have been defined: •

Achieve:

G⇒◊Q

Q holds in some future state.

•

Cease:

G⇒◊¬Q

There will be some point in the future that Q will not hold.

•

Maintain:

G⇒£ Q

Q holds in all future states.

•

Avoid:

G⇒£¬Q

Q will never hold in the future.

Often approaches (e.g., the NFR framework, i*) combine diagrammatic with formal techniques. The most commonly used notation for representing goal models is that of a goal decomposition tree (or graph) much in the spirit of AND/ OR trees. Structured language has also been proposed as an alternative to formal assertions. For example goal-scenario coupling and GQM propose a faceted template in order to express goals. The idea is that a structured language is formal enough to allow for formal manipulation of goals (for example it will allow for indexing and retrieval of goals, or for checking for goal similarities or conflicts), while still allowing the flexibility of using a ‘natural’ language and not mathematical assertions. Formal approaches generally intend to provide consistent, unambiguous, and precise representations of goals. They also provide the basis for useful tool support for goal analysis activities since they have well-defined formal semantics. However, they lack the freedom necessary to adequately support goal elicitation (e.g., to allow conflicts and inconsistencies among goals), and they lack the simplicity, flexibility, and ease of use of semiformal representations. Thus, semiformal and formal representations are best seen as complementary, contributing to an evolving framework for expressing goals. While goal-oriented research pays much attention to specification language issues, it only roughly addresses the way of producing a specification (i.e., the development world). In fact, none of the approaches describes an explicit wayof-working model. However, a few approaches (namely, i*, EKD, ISAC, KAOS, GBRAM, NFR framework, and goal-scenario coupling) do prescribe a number of steps and associated techniques that can be applied when performing goal modeling. Regarding tool support, a number of approaches supply modeling tools, which can be used to specify and store goal modeling concepts and their relationships (e.g., the i* Organizational Modeling Environment, the EKD Delos software, the NFR Assistant, the KAOS GRAIL environment, etc.). In approaches that support formal representation of goals, the tools can also offer Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

112


some reasoning capabilities regarding, for example, the consistency and satisfiability of goal structures, goal correctness, etc. However, none of the approaches offer adequate methodological support in order to deal with the complexity of the goal analysis process. Goal analysis is assumed to be a well-structured process based on the analysis of existing documents or interviewing experts. However, enterprise goals are more the outcome of the interaction between stakeholders rather than preexisting in documents or even the stakeholders’ minds. Moreover, due to the different interests and roles of stakeholders, several often-conflicting goal views may exist. Despite the fact that goals pertain to stakeholders, very few approaches support stakeholder involvement in the goal modeling process. In particular, there is a lack of techniques for identifying potential stakeholders, organizing stakeholder workshops, or facilitating stakeholder cooperation. Furthermore, very few approaches provide any assistance to RE participants during process enactment. Even when they do (ISAC, F3, GBRAM, GQM), this assistance takes the form of heuristics rather than systematic guidance. The only exception is the goal-scenario coupling method (Ben Achour et al., 1998), where goal analysis guidelines are embedded as rules in the supporting CREWS-L’écritoire tool. Finally, it should be noted that although there is a great interest in goal analysis methods for RE, there is still not enough practical evidence from applying goal analysis methods in real-life applications (though a number of industrial case studies have been reported in Chung & Nixon, 1995; Kavakli & Loucopoulos, 1999; Lamsweerde et al., 1995; Rolland, Grosz, & Regis, 1999; Rolland, Loucopoulos, Kavakli, & Nurcan, 1999; van Lamsweerde, 2001). Comprehending the practical implications involved in using goal analysis methods in practice is a real issue.

DISCUSSION The need to develop an overall view of goal concepts and goal-oriented approaches has also been argued in Yu and Mylopoulos (1998) and van Lamsweerde (2001). The former investigates the use of goal analysis in terms of different RE activities (requirements acquisition, relating requirements to organizational context, clarifying requirements, dealing with conflicts, and deriving design). On the other hand, van Lamsweerde (2001) compares different goal-oriented methodologies on the basis of their goal modeling and specification approaches. In addition it provides an overview of different goal modeling strategies (goal refinement, goal decomposition, analogical reuse, goal operationalization, goal conflict management, and selection between alternatives).



113

Rather than providing a comprehensive framework for analyzing the contribution of alternative approaches, the objective of these works has been mainly to stress the significance of goal concepts in RE and to draw the attention of the research community to goal-driven RE. In this sense, they focus on specific views of the four worlds framework while neglecting others. In particular, Yu and Mylopoulos (1998) focus on the usage view, while in van Lamsweerde (2001) the emphasis is on the representation and development views. The advantage of the analysis framework presented in this chapter is that it provides an overall picture of goal-oriented research, enabling researchers to identify possible extensions (e.g., the need for methodological support). It also assists requirements engineers in understanding and accordingly selecting the best fit for usage goal modeling method. For example, let us consider a BPR project concerning the reorganization of an electricity distribution company due to market deregulation. In order to meet the conditions in the competitive market, the company needs to re-examine and improve the way of servicing its existing customers, as well as to adopt new ways of working for servicing eligible customers. The implication of these forces on this organization is that any reform requires — prior to (re-)designing business processes and support information systems — a clear understanding (and a sharing of this understanding between many stakeholders) of the current enterprise situation. Thus, a goal modeling approach, such as the i* strategic dependency modeling method or EKD that focus on understanding the current organizational situation, should be used. Alternatively, a software engineering project concerning the development of a corporate knowledge repository may start with a ready plan to introduce a particular information system, i.e., the need for change has already been analyzed and the future business goals have been identified. In this case, the objective of the requirements engineer would be to relate these business goals to the system functional and nonfunctional requirements, and therefore, a method such as KAOS or goal-scenario coupling would be appropriate. The applicability of a particular method is, however, not entirely determined by the functionality that is needed, but also by other nonfunctional or situational factors, relevant to a particular RE project. For example, in the above BPR project, if the organizational actors are not used to working in groups in a participative way, they will probably feel awkward and will not contribute as intended if the selected method requires their active participation in workshops with presentations or discussions in a cooperative manner. In this case, an approach based on interviews and consultations of enterprise experts would better fit the particular organizational culture. Similarly, in the software engineering project, if the vision of the desired software solution is not clear, then an explorative method based on the examiCopyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

114


nation of alternative scenarios may be preferable to an approach where the solution is prescribed by experts. Other situational factors that affect the applicability of a method may include the use of appropriate tools that facilitate method execution and the familiarity of requirement engineers with the applied strategies and supporting technologies. The implication of these observations is that the selection of a particular method cannot be fully prescribed. Furthermore, even when one follows a certain goal modeling method, the situational factors dominating the project may cause a number of adaptations to it. Another observation is that additional benefits can be gained by integrating different methods. Such integration may be usage-driven. For example, let us consider the i* strategic dependency modeling approach. i* focuses on the ‘early’ stages of RE in that it emphasizes the need to understand the enterprise goals and how they shape the behavior of organizational actors. On the other hand, the KAOS approach assumes adequate knowledge about the current organizational state, and focuses on relating enterprise goals to the system functional and nonfunctional requirements. The combination of the two approaches can lead to a more complete methodology. An example of this type of integration is found in Dubois, Yu, and Petit (1998). Method integration could also be development-driven. Indeed, even if two methods serve the same RE function they differ in the strategies that they use in order to perform goal analysis. For example, the ISAC approach suggests a prescriptive strategy, whereby knowledge about goals is founded on the prescriptions given by organizational experts. The EKD approach advocates a descriptive strategy whereby knowledge about goals is abstracted from current practice. The KAOS approach and the NFR framework suggest a reuse strategy, which supports the reuse of knowledge acquired in similar domains. Alternatively, the goal-scenario coupling approach advocates an explorative strategy whereby knowledge is made ‘visible’ by means of systematic investigation of alternative scenarios. By combining alternative strategies, it is possible to improve the usability of a method and/or the quality of the produced goal models. It should be noted that any type of integration should also take into consideration the system and representation views in order to ensure compatibility between different methods and consistency between the different goal representation formalisms. Initial analysis of the goal concepts used in different approaches (Anton, 1996; Pratt, 1997) shows that integration of goal models resulted from different methods is feasible; additional work is required, however, in order to efficiently manage different formalisms and notations used in different approaches for expressing goal concepts.



115

CONCLUSIONS AND FUTURE TRENDS This chapter reviewed and analyzed the use of goal analysis techniques within different RE activities. The main conclusions are: • There is a variety of purposes and uses of goal models in RE. • Goal models deserve to be treated as important design artifacts. • Further research is necessary in order to be able to understand the role of goal analysis across different RE activities and offer better methodological support for performing goal-driven processes. Stakeholder goals and their role in defining and solving design problems are topics of long-standing interest in the field of requirements engineering. Goal analysis approaches emphasize the use of the goal notion in order to understand or describe aspects of the real world, in an attempt to rationally find better ways of coping with the complexity of human affairs. Much of the work has been confined to the research domain, and it is indeed a challenge to researchers, method engineers, and practitioners to facilitate the transition of these techniques from the research laboratory to practical use. Some tentative efforts are beginning to come forth (Castro, Silva, & Mylopoulos, 2003), but much more is needed in terms of education and training, as well as adoption of the goal modeling paradigm in mainstream development methods. In terms of further research directions, there are a number of areas in which goal modeling could facilitate interesting developments, and the remainder of this section elaborates on these themes. According to Norman (1989), people act by forming goals. Norman defines a goal as simply something that one wishes to achieve, something that needs to get done. But this is too abstract for most purposes: while one may have an idea about what needs to be done, this idea needs to be translated into specific actions. But again, this is usually not enough since one must ensure that the actions performed resulted in the anticipated outcome. This gives rise to a requirement of evaluating the results of the actions with reference to the original goal, checking if it has been achieved, and adjusting the performance according to the evaluation results; this in turn may result in the formulation of new goals. The cycle therefore comprises the following stages: • Goal setting • Goal execution • Goal evaluation In terms of the RE process, these three tasks are perceived as cognitive tasks that are mirrored in the conceptual models employed by the designers. These models become the design artifacts, the results of the designing. This top-down goal decomposition view of design involves a teleological view on the Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

116


whole process, the term teleological being used to convey the fact that whatever is done during the process, is done in order to identify the means by which the ends stated will be satisfied. The concept of ‘goal’ in the requirements specification process therefore could simultaneously reflect an enterprise or stakeholder requirement and the goal of the designer in attempting to meet these requirements. The task of someone using goal modeling is to determine the means by which an ultimate goal, let us call it G0, will be realized. In attempting to achieve this, the process is governed by causal relationships between goals in a network of goals. That is, a directed edge from a goal Gi to another goal Gj implies that the achievement of Gi depends on the achievement of Gj. At every step of the process, the process is controlled or driven by the goal on hand. Every node represents successively refined design goals. The actions chosen for attaining the goals represent working hypotheses. As goals and subgoals are established, these are tentative, at least until they are tested about their fitness of purpose, i.e., the satisfiability of higher goals. This observation has its roots in the Principle of Rationality (Newell, 1982), which states that “if an agent has knowledge that one of its actions will lead to one of its goals, then the agent will select that action.” However, an agent may not have complete knowledge to make the appropriate action, or the set of actions may be so complex that it is impossible to determine the single correct set of actions that will lead to the achievement of the goal. In this more general sense, the Principle of Bounded Rationality (Simon, 1982), which states that “given a goal, an agent may not possess perfect or complete knowledge of, or be able to economically compute or access, the correct action (or sequence of actions) that will lead to the attainment of the goal” will apply. Therefore, any actions chosen by an agent in order to achieve a given goal will, in general, be a hypothesis that the actions will lead to the achievement of the goal. A practical implication of hypothesis formulation is the presence of design constraints. Requirements constrain the design solution, and the exact balance of satisfying these requirements cannot be known in advance of producing a design. Consider for example an Air Traffic Control (ATC) system that is being commissioned for design. The client, the airport authority, may have a goal of ‘increasing throughput of aircrafts’; the users, the air traffic controllers, may have as a goal ‘reducing stress at work’; and the legislators ‘conform to safety regulations’. These three goals, three requirements, are at least to some extent in conflict, they interact negatively. How should they be satisfied, if at all? Should they be considered at the outset and arbitrarily assigned some priority? How do they interact with other goals of other stakeholders (or even of the same stakeholders)? The designer will need to explore various possibilities for the balance of satisfaction of the above three requirements. Some requirements, such as that of the legislator, will be value-free, in other words they are not open Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


117

to discussion and negotiation. Others, however, would need a value judgment in order to ascertain the degree of satisfiability offered by different designs. Design constraints arise from required or desired relationships between two or more elements. These relationships may refer either: (1) entirely to elements of the object being designed, in our example the air traffic control system for an airport; or (2) to elements between the object being designed and its environment, in our example this might be a compatibility relationship between the designed system and other air traffic control systems. The former are referred to as internal and the latter as external constraints. Whereas the former are under the direct control of the designers, the latter are normally outside their influence. Constraints may be generated from different categories of ‘stakeholder’ such as client, user, legislator, and even the designer. The need for hypothesis formulation and testing and the role of design constraints in a development process highlight the need for appropriate mechanisms to facilitate these two important design activities. To date the majority of goal-driven approaches in RE focus on the conceptualization of the problem space with the assumption that the models thus produced are sufficient for the evaluation of alternative designs. While most approaches recognize that the value of goal modeling is in tackling synergistically concerns relating to organizational, system, and design process issues, little attempt is made in tackling the issue of system complexity. This complexity is not so much a result of voluminous components (although this could indeed be true in some cases), but rather because of the behavioral characteristics of business processes. Even in systems with a relatively small number of parts, changes that involve the simultaneous change of many variables, some of which may be distant in space and time, can be difficult or impossible to understand without appropriate support mechanisms (Brehmer, 1989; Paich & Sterman, 1993; Sterman, 1989). The advantages of conceptual modeling over informal, natural language descriptions are well documented (Bubenko, 1979; Loucopoulos & Zicari, 1992; Mylopoulos, 1992). Current goal modeling approaches deal well with the need to externalize knowledge that in general is personal, subjective, and situation dependent (Nonaka, 1994; Polanyi, 1966; Stenmark, 2001), and through the common medium of the conceptual model, to assist stakeholders gain insights into the problem and through this to arrive at an agreed set of requirements. However, eliciting and developing maps of stakeholders’ mental models (in the so-called qualitative models) are not normally sufficient by themselves for achieving stakeholders’ agreement. Models need to be subjected to ‘testing’ in order to understand the implications of changes to a system component on the overall behavior of the system. Stakeholders experience difficulties in understanding the dynamics of the system even for the simplest of structures (Loucopoulos, Zografos, & Prekas, 2003). The absence of parameters, inputs, initial conditions, and generally of factors that are needed for testing these Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

118


qualitative models, greatly diminish their value as tools to understanding phenomena of the world. For without testing of the models, it is impossible to comprehend their implications by merely observing, walking through, and debating their contents. Nor is it always feasible to test them through observations from experimentation in the real world. Often the only practical option to test these models is the use of simulation. Testing qualitative models through simulation has been approached with some caution by some authors (c.f. Lane, 1994), but on the whole the rigorous testing offered by simulation has proved to be of indispensable value (Barlas & Kanar, 2000; Eddins, Crosslin, & Sutherland, 1991; Homer & Oliva, 2001). Furthermore, Lang shows that qualitative and quantitative properties are not mutually exclusive, and both facets are required to support business scenario analysis (Lang, 2000). Simulation imposes rigorous testing that removes ambiguity, exposes alternatives to stakeholders, and effectively removes any affectation toward the models driven by personal biases or political factors. Through simulation it is possible to develop scenarios of system behavior for a variety of concerns, such as human computer interaction (Carroll, 2000), software development (Abdel-Hamid & Madnick, 1988), and requirements engineering (Carroll, 2002; Filippidou & Loucopoulos, 1997; Potts, Takahashi, & Anton, 1994). A common feature of scenarios in all these domains is their use in examining alternative future situations. According to Carroll, scenarios support the way of experts working on ill-structured problem settings such as planning and design (Carroll, 2002). Scenarios encourage group brainstorming through which participants could focus on alternative solutions and envision potential behavior of the system prior to its implementation. This design paradigm, often referred to as ‘solution-first strategy’, has been defined as “…a strategy that involves generating a provisional design solution as a method of identifying requirements” (Carroll, 2002). This encourages stakeholders to first define what they consider as important aspects of the problem. They may then develop tentative designs in their scenario analysis sessions to ascertain whether anything else can be discovered about the problem.

REFERENCES Abdel-Hamid, T.K., & Madnick, S.E. (1988). Lessons learned from modeling the dynamics of software development. Cambridge, MA: Center for Information Systems Research, Sloan School of Management, Massachusetts Institute of Technology. Anton, A. (1996). Goal-based requirements analysis. Paper presented at the 2nd International Conference on Requirements Engineering (ICRE ‘96), Colorado Springs, Colorado, April 15-18.



119

Anton, A., Carter, A., Dempster, J., & Siege, D. (2001). Deriving goals from a use-case based requirements specification for an electronic commerce system. Requirements Engineering Journal, 6, 63-73. Barlas, Y., & Kanar, K. (2000). Structure-oriented behavior tests in model validation. Paper presented at the 18th International Conference of the System Dynamics Society, Bergen, Norway. Basili, V.R. (1993). Applying the goal/question/metric paradigm in the experience factory. In N. Fenton, R. Whitty, & Y. Lizuka (Eds.), Software quality assurance and measurement: A worldwide perspective (pp. 2344). Chapman and Hall. Basili, V.R., & Rombach, D.D. (1988). The TAME project: Towards improvement-oriented software environments. IEEE Transactions of the SE, 14(6), 758ff. Ben Achour, C., Rolland, C., & Souveyet, C. (1998). A proposal for improving the quality of the organization of scenario collections. Paper presented at the Fourth International Workshop on Requirements Engineering: Foundation for Software Quality (REFSQ’98), Pisa, Italy. Brehmer, B. (1989). Feedback delays and control in complex dynamic systems. Paper presented at the Computer-Based Management of Complex Systems: International System Dynamics Conference, Stuttgart. Bubenko, J. (1994). Enterprise modeling. Ingenierie des Systems d’ Information, 2(6), 657-678. Bubenko, J. (1995). Challenges in requirements engineering. Paper presented at the Second IEEE International Symposium on Requirements Engineering (RE’95), York, UK. Bubenko, J.A. Jr. (1979). On the role of understanding models in conceptual schema design. Paper presented at the 5th International Conference on Very Large Data Bases (VLDB), Rio de Janeiro, Brazil. Card, S., Moran, T., & Newell, A. (1983). The psychology of human-computer interaction. Hillsdale, NJ: Erlbaum. Carroll, J.M. (2000). Making use: Scenario-based design of human-computer interactions. Cambridge, MA: MIT Press. Carroll, J.M. (2002). Scenarios and design cognition. Paper presented at the IEEE Joint International Conference on Requirements Engineering (RE’02), Essen, Germany. Castro, J., Kolp, M., & Mylopoulos, J. (2002). Towards requirements-driven information systems engineering: The Tropos project. Information Systems, 27, 365-389. Castro, J.F.B., Silva, C.T.L.L., & Mylopoulos, J. (2003). Modeling organizational architectural styles in UML. Paper presented at the 15th International Conference on Advanced Information Systems Engineering (CAiSE 2003), Klangenfurt/Velden, Austria. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

120


Chung, L., & Nixon, B.A. (1995). Dealing with non-functional requirements: Three experimental studies of a process-oriented approach. Paper presented at the International Conference on Software Engineering (ICSE’95), Seattle, Washington. Chung, L., Nixon, B.A., Yu, E., & Mylopoulos, J. (2000). Non-functional requirements in software engineering. Kluwer Academic. Cockburn, A. (1995). Structuring use cases with goals. Retrieved May 19, 2003, from: http://members.aol.com/aocburn/papers/usecases.htm. Dardenne, A., Lamsweerde, A.V., & Fickas, S. (1993). Goal-directed requirements acquisition. Science of Computer Programming, 20, 3-50. Darimont, R. (1995). Process support for requirements elaboration. Unpublished PhD Thesis, Universite Catholique de Louvain, Louvain-la-Neuve. Dasgupta, S. (1994). Creativity in invention and design—computational and cognitive explorations of technological originality. Cambridge: Cambridge University Press. Dubois, E., Yu, E., & Petit, M. (1998). From early to late formal requirements: A process-control case study. Paper presented at the Ninth IEEE International Workshop on Software Specification and Design (IWSSD-9), Isobe, Japan. Eddins, W.R., Crosslin, R.L., & Sutherland, D.E. (1991). Using modeling and simulation in the analysis and design of information systems. In H.G.a.K.M. vanHee (Ed.), Dynamic modeling of information systems (pp. 89-119). Amsterdam: Elsevier. Ellis, C.A., & Wainer, J. (1994). Goal-based models of collaboration. Collaborative Computing, 1, 61-86. Filippidou, D., & Loucopoulos, P. (1997). Using scenarios to validate requirements in a plausibility-centered approach. Paper presented at the 9th International Conference on Advanced Information Systems Engineering (CAiSE’97), Barcelona, June 16-20. Greenspan, S., Mylopoulos, J., & Borgida, A. (1994). On formal requirements modeling languages: RML revisited. Paper presented at the 16th International Conference on Software Engineering (ICSE-94), Sorrento, Italy. Hoffmann, M., Birk, A., van Els, F., & Kempkens, R. (1996). GQMaspect v1.0 User Manual. Kaiserslautern: Fraunhofer Einrichtung, Experimentelles Software Engineering. Homer, J., & Oliva, R. (2001). Maps and models in system dynamics: A response to Coyle. System Dynamics Review, 17(4). IEEE-Std.’729’. (1983). IEEE Standard 729. Glossary of Software Engineering Terminology. New York: The Institute of Electrical and Electronics Engineers. IEEE-Std.’830’. (1984). IEEE Guide to Software Requirements Specifications. New York: The Institute of Electrical and Electronics Engineers. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


121

Kavakli, V., & Loucopoulos, P. (1999). Goal-driven business process analysis— application in electricity deregulation. Information Systems, 24(3), 187207. Kelly, T.P., & McDermid, J.A. (2001). A systematic approach to safety case maintenance. Reliability Engineering and System Safety, 71(2001), 271284. Lamsweerde, A., Darimont, R., & Massonet, P. (1995). Goal-directed elaboration of requirements for a meeting scheduler: Problems and lessons learnt. Paper presented at the Requirements Engineering Workshop, York, UK. Lane, D.C. (1994). With a little help from our friends: How system dynamics and soft OR can learn from each other. System Dynamics Review, 10(2/3), 101-134. Lang, K. (2000). Simulation of qualitative models to support business scenario analysis. Paper presented at the 18th International Conference of the System Dynamics Society, Bergen, Norway. Lee, J. (1997). Design rationale systems: Understanding the issues. IEEE Expert Intelligent Systems and Their Applications, 12(3), 78-85. Lee, J., & Lai, K.-Y. (1991). What’s in design rationale? Human-Computer Interaction, 6(3,4), 251-280. Leite, J., & Haumer, P. (1997). Enhancing a requirements baseline with scenarios. Paper presented at RE’97, Annapolis, Maryland. Letier, E., & van Lamsweerde, A. (2002). Agent-based tactics for goal-oriented requirements elaboration. Paper presented at the 24th International Conference on Software Engineering (ICSE’02), Orlando, Florida. Loucopoulos, P. (1995). The F3 (from fuzzy to formal) view on requirements engineering. Inginierie des Systèmes d’Information, 2(6), 639-655. Loucopoulos, P., & Karakostas, V. (1995). System requirements engineering (1st ed.). London: McGraw-Hill. Loucopoulos, P., & Kavakli, E. (1995). Enterprise modeling and the teleological approach to requirements engineering. International Journal of Intelligent and Cooperative Information Systems, 4(1), 45-79. Loucopoulos, P., & Kavakli, V. (1997). Enterprise knowledge management and conceptual modeling. Paper presented at the International Symposium on ‘Past, Present and Future of Conceptual Modeling’, Los Angeles, California. Loucopoulos, P., & Zicari, R. (Eds.). (1992). Conceptual modeling, databases and CASE: An integrated view of information systems development. John Wiley & Sons. Loucopoulos, P., Kavakli, V., Prekas, N., Rolland, C., Grosz, G., & Nurcan, S. (1997). Using the EKD approach: The modeling component. Research Report (ELEKTRA project) No. ELEKTRA/WP2/T2.1/UMIST/3. Manchester: UMIST. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

122


Loucopoulos, P., Louridas, P., Kavakli, V., Filippidou, D., & Tzanaki, A. (1996). The enterprise requirements analysis (ERA) approach. Technical Report No. ISEG93. Manchester: ISE Group, UMIST. Loucopoulos, P., Zografos, K., & Prekas, N. (2003). Requirements elicitation for the design of venue operations for the Athens 2004 Olympic Games. Paper presented at the 11th IEEE International Requirements Engineering Conference, Monterey Bay, California. Louridas, P., & Loucopoulos, P. (2000). A generic model for reflective design. ACM Transactions on Software Engineering (TOSEM), 9(2), 199-237. Lundeberg, M. (1982). The ISAC approach to specification of information systems and its application to the organization of an IFIP working conference. In T.W. Olle, H.G. Sol, & A.A. Verrijn-Stuart (Eds.), Information systems design methodologies: A comparative review (pp. 273-234). North-Holland: IEEE Computer Society Press. Mylopoulos, J. (1992). Conceptual modeling and Telos. In P. Loucopoulos & R. Zicari (Eds.), Conceptual modeling, databases and CASE: An integrated view of information systems development (pp. 49-68). New York: John Wiley & Sons. Mylopoulos, J., Chung, L., & Nixon, B. (1992). Representing and using nonfunctional requirements: A process-oriented approach. IEEE Transactions on Software Engineering, SE-18(6), 483-497. Newell, A. (1982). The knowledge level. Artificial Intelligence, 18, 87-127. Nonaka, I. (1994). A dynamic theory of organizational knowledge creation. Organization Science, 5(1), 14-37. Norman, D. (1989). The design of everyday things. New York: Currency Doubleday. Nurcan, S., & Rolland, C. (1997). A meta-model for goal-driven cooperative work processes. Paper presented at the Workshop on the many facets of Process Engineering (MEPE’97), Gammarth, Tunis. Nuseibeh, B., & Easterbrook, S. (2000). Requirements engineering: A roadmap (2000). Paper presented at the 22nd International Conference on Software Engineering (ICSE 2000), Limerick, Ireland. Paich, M., & Sterman, J.D. (1993). Boom, bust, and failures to learn in experimental markets. Management Science, 39(12), 1439-1458. Pohl, K. (1996). Process-centered requirements engineering. Taunton, Somerset, England: Research Studies Press Ltd. Polanyi, M. (1966). The tacit dimension. New York: Doubleday. Potts, C., Takahashi, K., & Anton, A. (1994). Inquiry-based requirements analysis. IEEE Software, 11(2), 21-32. Pratt, N. (1997). Goal formalization and classification for requirements engineering. Paper presented at the 3rd International Workshop on Requirements Engineering: Foundations of Software Quality (REFSQ’97), Barcelona, Spain. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


123

Ramesh, B., & Dhar, V. (1992). Supporting systems development by capturing deliberations during requirements engineering. IEEE Transactions on Software Engineering, 18(6), 498-510. Rasmussen, J., Pejtersen, A.M., & Goodstein, L.P. (1994). Cognitive systems engineering. New York: John Wiley & Sons. Rittel, H., & Weber, M. (1973). Dilemmas in a general theory of planning. Policy Science, (4), 155-169. Rolland, C., Grosz, G., & Regis, K. (1999). Experience with goal-scenario coupling in requirements engineering. Paper presented at the IEEE International Symposium on Requirements Engineering, Limerick, Ireland. Rolland, C., Loucopoulos, P., Kavakli, V., & Nurcan, S. (1999). Intention-based modeling of organizational change: An experience report. Paper presented at the Fourth CAiSE/ IFIP8.1 International Workshop on Evaluation of Modeling Methods in Systems Analysis and Design (EMMSAD’99), Heidelberg, Germany. Rolland, C., Souveyet, C., & Ben Achour, C. (1998). Guiding goal modeling using scenarios. IEEE Transactions on Software Engineering, 24(12), 10551071. Schedin, K. (1995). Goals in business modeling. MSc Thesis No. 95-11-DSVSU, Kungl Tekniska Hogskolan. Simon, H.A. (1982). Models of bounded rationality. Cambridge, MA: MIT Press. Smith, G., & Browne, G. (1993). Conceptual foundations of design problem solving. IEEE Transactions on Systems, Man, and Cybernetics, 23(5), 1209-1219. Smith, S., & Boldyreff, A. (1995). Towards an enterprise method for CSCW systems. Paper presented at the Second International Symposium on Autonomous Decentralized Systems (ISADS ‘95), Phoenix, Arizona. Stenmark, D. (2001). Leveraging tacit organizational knowledge. Journal of Management Information Systems, 17(3), 9-24. Sterman, J.D. (1989). Misperceptions of feedback in dynamic decision making. Paper presented at the Computer-Based Management of Complex Systems: International System Dynamics Conference, Stuttgart. van Lamsweerde, A. (2001). Goal-oriented requirements engineering: A guided tour. Paper presented at the 5th IEEE International Symposium on Requirements Engineering (RE’01), Toronto, Ontario. Wilson, S.P., Kelly, T.P., & McDermid, J.A. (1995). Safety case development: Current practice, future prospects. Paper presented at the 1st ENCRESS/ 5th CSR Workshop. Yu, E. (1997). Towards modeling and reasoning support for early phase requirements engineering. Paper presented at the 3rd IEEE International Symposium on Requirements Engineering (RE’97), Washington, DC. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

124


Yu, E., & Mylopoulos, J. (1998). Why goal-oriented requirements engineering. Paper presented at the Fourth International Workshop on Requirements Engineering: Foundation for Software Quality (REFSQ’98), Pisa, Italy. Yu, E., Du Bois, P., E., D., & J., M. (1995). From organization models to system requirements—a “cooperating agents” approach. Paper presented at the 3rd International Conference on Cooperative Information Systems (CoopIS-95), Vienna, Austria. Yu, E., Liu, L., & Li, Y. (2001). Modeling strategic actor relationships to support intellectual property management. Paper presented at the 20th International Conference on Conceptual Modeling (ER-2001), Yokohama, Japan. Zave, P. (1997). Classification of research efforts in requirements engineering. ACM Computing Surveys, 29(4), 315-321.


An Empirical Investigation of Requirements Specification Languages

125

Chapter VII

An Empirical Investigation of Requirements Specification Languages: Detecting Defects While Formalizing Requirements Erik Kamsties, University of Duisburg-Essen, Germany Antje von Knethen, Fraunhofer Institute for Experimental Software Engineering, Germany Jan Philipps, Technische Universität München, Germany Bernhard Schätz, Technische Universität München, Germany

ABSTRACT A well-known side-effect of applying requirements specification languages is that the formalization of informal requirements leads to the detection of defects such as omissions, conflicts, and ambiguities. However, there is little quantitative data available on this effect. This chapter presents an empirical study of requirements specification languages, in which two research questions are addressed: Which types of defects are detected by a requirements engineer during formalization? Which types of defects go undetected and what happens to those types in a formal specification? The results suggest looking explicitly for ambiguities during formalization, Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

126 Kamsties, von Knethen, Philipps, & Schätz

because they are less frequently detected than other types of defects. If they are detected, they require immediate clarification by the requirements author. The majority of ambiguities tend to become disambiguated unconsciously, that is, the correct interpretation was chosen, but without recurring to the requirements author. This is a serious problem, because implicit assumptions are known to be dangerous.

INTRODUCTION The use of a requirements specification language (RSL) in requirements engineering (RE) has manifold benefits. Precise requirements models allow better communication among various stakeholders; checks of completeness and consistency, and proofs of safety properties can be automated; and the dynamic behavior of the requirements models can be simulated. Furthermore, they make the RE process more repeatable than if ad hoc techniques were applied. According to Sommerville and Sawyer (1997), RSLs are a “vehicle for the analyst to add clarity to the fuzzy picture provided by the stakeholder requirements, domain constraints…They are concerned with imposing a structure on the vague notion of a system.” It is this characteristic that leads to a frequently reported side-effect of the application of RSLs: defects in the initial requirements are detected during the development of requirements models (see, e.g., Wing, 1990; Sommerville & Sawyer, 1997; Easterbrook & Callahan, 1997). We subsume under the term requirements specification languages both requirements modeling languages and formal methods for describing requirements. A requirements modeling language offers a graphical language with a formal syntax, that is, a set of diagram elements, and a semi-formal semantics, which is typically stated in natural language. One example of a requirements modeling languages is the Unified Modeling Language (UML) (OMG, 1999). A formal method offers a language with a formal syntax and formal semantics. In most cases, this language is mathematical, but also graphical and tabular languages have been proposed. A formal method allows describing requirements rigorously and analyzing them extensively. Examples of formal methods include SCR (Heitmeyer, Jeffords, & Labaw, 1996), SDL (ITU, 1993), VDM (Jones, 1990), and Z (Spivey, 1992). A requirements model is a set of requirements that is represented using a single RSL. It is a formalized statement of requirements. In contrast, the term requirements document denotes in this chapter an informal statement of requirements given in natural language. This chapter reports on an empirical study aimed at answering two research questions about the defects spotted in informal requirements during formalization. A defect is a product anomaly in a requirements, design, or code document that leads to a misbehavior of a software system. We focus on conflicts,



127

incompleteness, and ambiguities in requirements documents. We address the following questions: Are there differences in the numbers of conflicts, incompleteness, and ambiguities: • that are found during creation of a requirements model? • that are not found and, thus, are contained in the final requirements model? We expected differences, because an RSL forces the requirements engineer to be precise, that is, to resolve ambiguities before creating a requirements model. Thus, ambiguities become unambiguously right or wrong statements in a requirements model. Only a few types of ambiguities become conflicts, which are detected by the consistency checker of a CASE tool. On the other hand, a requirements model can be inconsistent or incomplete. Incompleteness and, in particular, conflicts can violate syntactic or semantic rules of an RSL and then are detected by a CASE tool. Seventeen graduate computer science students from the University of Kaiserslautern (UKL) and the Technische Universität München (TUM) participated in the study. The task was to develop a requirements model of a consumer electronics product, namely the Tamagotchi toy (Bandai, 1997). The most popular RSLs, e.g., UML and SDL, as well as research prototypes such as Focus (Broy & Stølen, 2001; Huber et al., 1998), were used together with CASE tools. This chapter is structured as follows. First, the RSLs that were used in the study are briefly described. Second, the previous empirical research is reviewed. Then, the evaluation framework is discussed, along with the design of the study. Finally, the results are presented, threats to validity are discussed, and conclusions are drawn.

REQUIREMENTS SPECIFICATION LANGUAGES The objects of the study were seven RSLs, namely Focus, SCR, SDL, OCTOPUS, ROOM, Statemate, and UML. The selection of these languages was driven by the availability of CASE tools, availability of experts for supervising the subjects, and practical relevance of languages. Furthermore, the languages should represent a good balance between emerging object-oriented RSLs and traditional structural RSLs. The RSLs and the used CASE tools are described briefly in the following. Focus (Broy & Stølen, 2001) is a formal method for modeling distributed systems. Its semantics is based on an extension of the theory of stream processing functions (Kahn, 1974). AutoFocus is a prototype CASE tool



developed by the TU München, which implements the semantics of Focus (Huber et al., 1998). The SCR (Software Cost Reduction) requirements specification language was developed by the Naval Research Laboratory (NRL) of the U.S. Navy in 1978 (Heninger, 1980). The NRL defined a formal semantics of SCR (Heitmeyer, Jeffords, & Labaw, 1996) and developed a CASE tool called SCR* (Heitmeyer, Bull, Gasarch, & Labaw, 1995). SDL (Specification and Description Language) is standardized by the International Telecommunication Union. It was developed in the late 1960s. Object-oriented concepts have been introduced into the version SDL 92 (ITU, 1993). SDL is considered today a formal description technique. We used the CASE tool ObjectGEODE from Verilog. OCTOPUS (Awad, Kuusela, & Ziegler, 1996) was developed in 1996 by Nokia. It is an object-oriented method based on OMT (Rumbaugh, Blaha, Premerlani, Eddy, & Lorensen, 1991) and Fusion (Coleman et al., 1993) for all phases of the development of embedded systems. There is no CASE tool for OCTOPUS; however, Awad and colleagues recommend among others Software through Pictures/OMT (StP) from Aonix, which was applied in the case study. ROOM (Real-time Object-Oriented Modeling) is an object-oriented method for developing distributed systems created by Bell Northern Research, Ltd. in 1994 (Selic, Gullekson, & Ward, 1994). We used the CASE tool ObjecTime Developer from ObjecTime Systems. Statecharts is a state-machine-based description technique developed by David Harel (1987). The CASE tool Statemate MAGNUM from I-Logix extends the behavioral statecharts view with a structural and a functional view (Harel et al., 1990). The Unified Modeling Language (UML) was presented by Booch, Rumbaugh, and Jacobson in 1997, the applied version was 1.3 (OMG, 1999). The CASE tool Rhapsody by I-Logix was used which allows generating executable code from a subset of UML diagrams. Statecharts as offered by the UML and supported by Rhapsody differs from statecharts as supported by Statemate. The main difference is the underlying modeling paradigm. Rhapsody is object oriented, while Statemate sticks to an extended structured analysis paradigm. That is, the different variants of statecharts do not differ largely in their syntax, but in the way they are used. A UML/Rhapsody statechart relates to a class and, thus, can be inherited. A Statemate statechart relates to a block (i.e., a module), which is semantically different from a class. All employed CASE tools, except StP/OMT, offer simulation of requirements models or full code generation. That is, there is some sort of formal semantics behind every RSL, except OCTOPUS.



129

PREVIOUS EMPIRICAL RESEARCH This section discusses empirical research that covers similar research questions, i.e., empirical research that addresses formalization of informal requirements. Feather, Fickas, Finkelstein, and von Lamsweerde (1997) suggested that empirical studies on RSLs should focus on processes rather than products, i.e., they should focus on causes rather than symptoms of strength and weaknesses of RSLs. One process aspect is the previously mentioned side-effect of RSLs to help spot defects during formalization of informal requirements. This side-effect was studied to our knowledge only by Wing (1988) a posteriori in specifications written for the fourth IWSSD workshop. Wing analyzed ambiguity and incompleteness in informal requirements, in total 11 defects, and checked their interpretation in 12 requirements models. That is, she investigated the requirements models and reported the selected interpretation. Wing made two conclusions: First, RSLs do not radically differ from one another in this respect. Second, the RSLs can be used to identify many, but not all, deficiencies in a set of informally stated requirements. Wing does not provide quantitative data.

EVALUATION FRAMEWORK We extended the above-mentioned evaluation framework of Wing (1988). In addition, we propose to track: (1) the number of defects revealed in informal requirements during creation of a requirements model, and (2) the number of defects that slip into the requirements model. In particular, we suggest distinguishing several cases why defects are resolved or not resolved. Figure 1 shows all paths that a defect in the informal requirements can take. First, a specifier may observe a defect during formalization and report it to the customer. Then, the specifier receives a solution to the defect. Thus, such a defect is removed in the requirements model. For simplification we assume that a specifier does not introduce a defect into the requirements model after knowing the solution. Defects that are removed in the requirements model because they were reported during formalization are called customer-resolved defects. Second, it is possible that a specifier does not report a defect, for instance because the fault is not recognized. Nevertheless, such a defect can be removed by chance. Defects that are removed in the requirements model, but were not reported during formalization, are called self-resolved defects. Third, a defect that was not reported often remains in the requirement document. Residual defects are further distinguished into forwarded and transformed defects. Some defects in the informal requirements are neither removed nor residual in the requirements model, because they are outside the scope of the requirements model. Therefore,



Figure 1. Classification of defects Informal Requirements

Formalizaton

Requirements Model Customer-resolved Defects Removed Defects

Reported Defects

Self-resolved Defects Defects out of scope

Defects Unreported Defects

Forwarded Defects Residual Defects Transformed Defects

the set of reported defects is not necessarily equal to the set of customerresolved defects. In the following, we explain in more detail the defect classes that are of particular interest in the context of our study: • Reported Defect. The defect was recognized by a specifier and was reported to the experimenters while reading an informal requirement or while formalizing an informal requirement. These two activities usually are intertwined, and we did not make an attempt to distinguish them. • Removed Defect. The requirements model is incorrect with respect to the informal requirements, but correct with respect to the customer’s expectations. • Self-Resolved Defect. The defect has been removed, but it has not been reported, e.g., it has been removed by the specifier using his or her background knowledge. • Forwarded Defect. The same defect of the informal requirements is included in the requirements model. The requirements model is correct with respect to the informal requirements, but incorrect with respect to the customer’s expectations. For instance, an incomplete informal requirement has not been recognized and has become an incomplete statement in the requirements model. • Transformed Defect. A defect in the informal requirements has been transformed into another type of defect in the requirements model. The requirements model is incorrect with respect to the informal requirements, as well as with respect to the customer’s expectations. For instance, an ambiguous requirement has been misinterpreted and has become an incorrect statement in the requirements model. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


•

131

Defect Out of Scope. The requirements model is incomplete with respect to the informal requirements as well as with respect to the customer’s expectations. For example, if a defect is concerned with a timing requirement, and timing requirements cannot be specified with the applied RSL, e.g., because of lack of notational power of the RSL, then the defect is considered out of scope.

Defects newly introduced into the requirements models were not investigated in this study, because the effort would have been too high to inspect the requirements models in detail. We limited the types of defects considered in the study to defects that can be identified without knowledge of the application domain, because we did not expect the subjects to have deep knowledge about the application domain. In particular, we were interested in incompleteness (only those detectable without domain knowledge), conflicts, and ambiguities. For the same reason, we did not consider incorrect, unrealistic, or extraneous requirements. A requirements document is incomplete if information is missing such as a function or a definition of a response to particular input data. A requirement is ambiguous if it has several interpretations. Ambiguities include not only linguistic ambiguities such as an uncertain pronoun reference, but also ambiguities about the actual system and its behavior as defined by Schneider, Martin and Tsai (1992). An ambiguous requirement can be considered potentially deficient, because it also conveys the right meaning. Incompleteness defects and ambiguities can be clearly distinguished by the type of required correction activity. The former require adding information, while the latter just require rephrasing the present information so that a requirement unambiguously conveys its meaning. Two requirements are inconsistent if they state facts that cannot both be true, or if they express actions that cannot be carried out at the same time. This type of defect is also called conflict.

EXPERIMENTAL STUDY This section describes hypotheses, design, instruments, preparation, execution, and data validation.

Hypotheses The hypotheses followed the research questions, which are stated in the beginning of this chapter. We assume that there are no significant differences between the investigated RSLs in spotting defects, because they all address behavioral requirements and provide some state-machine-based language to describe them. Rather, we expect differences between the defect types. An RSL forces the requirements engineer to be precise. However, we expect that Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


a specifier does not necessarily recognize an ambiguity as such and misinterprets it, while the structure imposed by the RSL on the requirements helps detecting inconsistencies and incompleteness (recall that we limited the considered kinds of incompleteness to those that are detectable without domain knowledge). Therefore, we hypothesize that fewer ambiguities and more incompleteness and conflicts are reported than one would expect based on the overall numbers of those defects in the requirements document. Our first (alternative) hypothesis and the according null hypothesis are: H 1: There is a difference between the observed and expected numbers of incompleteness, conflicts, ambiguities reported during formalization using an RSL. H 01: There is no difference between the observed and expected numbers of incompleteness, conflicts, ambiguities reported during formalization using an RSL. Following the above argumentation, we hypothesize that fewer ambiguities and more incompleteness and conflicts are removed than one would expect based on the overall numbers of those defects in the requirements document, because ambiguities are less frequently recognized. Rather, one portion of the ambiguities that was not recognized is misinterpreted. Thus, we hypothesize that more ambiguities and fewer incompleteness and conflicts are transformed than one would expect based on the overall numbers of those defects in the requirements document. The other portion of ambiguities that was not recognized is correctly interpreted with the help of other requirements and real-world knowledge. Thus, we hypothesize that more ambiguities and fewer incompleteness and conflicts are self-resolved than one would expect, based on the overall numbers of those defects in the requirements document. Finally, we hypothesize that fewer ambiguities and more incompleteness and conflicts are forwarded than one would expect, based on the overall numbers of those defects in the requirements document. A requirements model can be inconsistent and incomplete to some degree, even if it is checked by a CASE tool. However, ambiguities must be removed or transformed, thus we expected virtually no forwarded ambiguities in the requirements model. Our second (alternative) hypothesis and the according null hypothesis are: H 2: There is a difference between the observed and expected numbers of incompleteness, conflicts, and ambiguities that are {removed, self-resolved, forwarded, and transformed} in a requirements model. H 02: There is no difference between the observed and expected numbers of incompleteness, conflicts, and ambiguities that are {removed, self-resolved, forwarded, and transformed} in a requirements model. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


133

In order to save space, Hypothesis H2 is parameterized, which is indicated by the parentheses. That is, it subsumes four similar hypotheses. The first hypothesis concerns removed defects (H2.1; H 02.1), the second self-resolved defects (H2.2; H02.2), the third forwarded defects (H2.3; H02.3), and the fourth transformed defects (H2.4; H02.4). There was a difference in the customer participation between the sites UKL and TUM. At UKL, the customer was involved from the beginning of the formalization process. At TUM, the customer was involved only at the end of the formalization process, when the final requirements model was evaluated by an interview with the team. This was a deviation of the actual course of the empirical study from the plan, which allows us to investigate a third hypothesis regarding ambiguities. We expect a significant difference between UKL and TUM in the numbers of removed and transformed ambiguities. Humans are naturally skilled in resolving ambiguity. Thus, the ambiguities that were reported at UKL are those that need clarification. If there is no customer participation, as in the case of TUM, the likeliness of misinterpretations raises. H 3 : There is a difference between the UKL and TUM teams in the numbers of removed and transformed ambiguities. H 03 : There is no difference between the UKL and TUM teams in the numbers of removed and transformed ambiguities.

Subjects The empirical study was performed at the University of Kaiserslautern (UKL) and the TU München (TUM). Ten computer science students from UKL and nine from TUM participated in the empirical study. All students were enrolled in a joint seminar. The students were from the third year and above, and had knowledge of the principles underlying the RSLs such as finite state machines and object-orientation, but no experience with the particular languages or CASE tools. The students worked together in teams of two or three students.

Design The RSLs were examined in a replicated project scope according to Basili’s classification of experimental scopes (Basili, Selby, & Hutchens, 1986). This means that each team performed the same task, the development of a requirements model based on a set of informal requirements, but using a different RSL. Six teams were formed such that there is a one-to-one relation between team and RSL. This design was a compromise, because the effort per person and week is required to be low for a seminar (four hours). Ideally, each team should apply each RSL. However, the required resources were not available.



Teams were formed according to the students’ preferences. We decided to assign teams non-randomly to RSLs, because of the distributed nature of the seminar: Due to the location of the supervisors experienced with the RSL, the UKL students only could choose from OCTOPUS, SCR, Statecharts, and UML, while the TUM students only could choose from ROOM, SDL, and Focus.

Instrumentation The teams received an informal requirements document of about nine pages, which was written by the authors of this book chapter. This document described a consumer electronics product, the Tamagotchi toy, which is an event-driven system (note that all selected RSLs are well suited for specifying event-driven systems). The requirements document had two parts: a problem description of four pages that defines the background of a fictional software development project, and the customer requirements of five pages that describe the desired behavior of the Tamagotchi toy. The customer requirements consist of 42 textual requirements; each requirement has on average two or three sentences. Some requirements were derived from a book describing the Tamagotchi (Bandai, 1997), others were reverse engineered from the original toy, and some were invented. The requirements document contained 57 known defects of which 38 were ambiguities, 13 incompleteness defects, and 6 conflicts. The document may contain more defects. However, the experimenters agreed upon these 57 defects. Only these defects were considered in the data collection. Our experience with previous empirical studies in RE has shown that is not practical to seed defects in a requirements document after it was written. Defects in requirements documents are so diverse that “invented” defects could only cover a small portion of the defects actually occurring in practice. Furthermore, a requirements document written in natural language always contains some redundancy. This does not mean necessarily that requirements are mentioned twice, but also that contextual information is given, which backs up the requirements. To introduce a defect “consistently”, these redundant parts need to be changed as well. Doing this change consistently and uniformly has proven not feasible. Instead, we seeded defects during the creation of the requirements. To this end, we relied on the observation that the first versions of a requirements document contain many defects even if they are written carefully. This observation was made by Martin and Tsai (1990), who found 92 defects in a wellwritten, 16-page requirements document. Our experience from a previous empirical study confirms this observation (von Knethen, Kamsties, Reussner, Bunse, & Shen, 1998). We have identified the defects through an intensive review and through the questions posed by the students.



135

Data Collection Data collection was performed in several steps. The teams were required to write a brief report about each issue they encountered in the informal requirements. A solution to the issue was sent in reply by the author of the requirements document. This solution was not sent to the other teams. At the end of the seminar, we interviewed each team about its requirements model using the list of 57 known defects in the informal requirements. For each defect, it was checked whether the defect has been removed, forwarded, transformed, or is out of the scope of requirements model.

Preparation The preparation phase took four weeks. During this phase, the students read material about their RSL and produced a tiny requirements model of the ESFAS (Engineered Safety Feature Actuation System) (Courtois & Parnas, 1993). Then, each team wrote a one- to two-page essay about the RSL. At the end of this phase, the students had an opportunity to discuss with their supervisor all the problems they encountered with the RSL or CASE tool. The outcome of this phase was the requirements model and the essay. Based on these two deliverables, the supervisor got an impression of the students’ current understanding of the treatment. In case a team’s understanding was poor, the team could have discussed the problems with the supervisor. However, this case actually did not occur.

Execution The execution phase took eight weeks and comprises the development of a requirements model of the Tamagotchi toy. All issues and defects that were detected during the formalization of informal requirements were reported to the supervisor. The questions were forwarded to and answered by the author of the requirements document, i.e., the customer, in such a way that if two teams encountered the same issue or defect, they received the same answer. Each team spent a minimum of 100 hours on the development of the requirements model.

Data Validation Several activities were performed to validate the data. First, the validity of the reported issues was checked by the author of the requirements document. There were cases in which the description of an issue that was submitted by a team was not clear. In such a case, the issue was clarified in a discussion between the author and the team. Second, the validity of the collected data regarding removed, forwarded, and transformed defects was checked by simulating and inspecting the requireCopyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


ments models. During the interviews in which the data were collected, each team was asked to show certain behaviors of the requirements model by using the CASE tool’s simulation feature. After the seminar was over, further validation was performed by inspection of the requirements models as part of a diploma thesis.

RESULTS Due to different university holidays at TUM, the schedule for the execution phase was tighter at TUM than at UKL. Thus, the TUM teams did not have enough time for writing defect reports and waiting for answers. Effectively, this means that the processes at UKL and TUM differed in customer participation. At UKL, the customer was involved from the beginning of the formalization process. At TUM, the customer was involved only at the end of the formalization process, when the final requirements models were evaluated by an interview with the team. This difference in customer participation has an effect on the analysis results and makes it necessary for us to block the analysis of this study with respect to the site (UKL, TUM). The fact that customer and user participation can influence the RE process and the quality of its outcome was shown by El Elmam, Quintin, and Madhavji (1996) in an empirical study. We tested the hypothesis H1 by a Chi-square test, which allows testing whether a set of observed frequencies significantly departs from a set of theoretical frequencies. We demanded a significance level α=0.05, which is most common in software engineering experiments. Concretely, we tested whether the reported numbers of ambiguities, incompleteness, and conflicts depart significantly from the expected numbers of those defects. Based on the numbers of known defects in the requirements document (38 ambiguities, 13 incompleteness defects, and 6 conflicts), the theoretical probabilities of detecting an ambiguity, incompleteness, and conflict were 0.67 (38 ambiguities divided by 57 defects in total), 0.23, and 0.1, respectively. The expected number of defects of a particular type fe(j) is defined as fe(j) = n × pj, where n is the total number of reported defects and pj, is the probability of detecting a defect of type j. For example, if a team reported 20 defects, we would expect 13.4 ambiguities (20 reported defects × 0.67 probability of detecting an ambiguity). The underlying assumption is that the difficulty of detecting a defect is homogenous for each defect type. We were able to show this assumption (Kamsties, 2001, pp. 47-48) for the given requirements document. We also tested the hypotheses H2 by a Chi-square test. Table 1 summarizes the observed (“O”) and the expected (“E”) numbers of reported, removed, self-resolved, forwarded, and transformed defects for each site and the results the Chi-square tests. For reasons of space, the provided numbers for incompleteness, conflicts, and ambiguities are average team Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


137

Table 1. Aggregated data and results of Chi-square tests Defect Class

Site

Reported Defects

UKL

Removed Defects

UKL TUM

Self-Resolved Defects Forwarded Defects

UKL UKL TUM

Transformed Defects

UKL TUM

Obs./Exp.

Incompl.

Confl.

Ambig.

O

5.0

1.7

5.3

E O E O E O E O E O E O E O E

2.8 9.4 9.2 7.3 7.0 3.6 6.3 2.6 1.9 4.0 2.4 1.0 2.0 1.6 3.7

1.2 3.3 4.0 2.3 3.1 1.6 2.7 2.3 0.8 3.3 1.0 0.3 0.8 0.3 1.6

8.0 27.3 26.8 21.0 20.5 22.0 18.2 3.3 5.5 3.0 6.9 14.0 5.8 14.0 10.6

Chi-square Test (p-level) .009985 .997092 .959297 .037973 .004681 .000009 .021563 .023047

results. The individual team results are given in Appendix A. The Chi-square test was performed on the individual team results. However, a precondition of this test is that the expected frequencies are not below five. Because most expected frequencies are below five, we test the hypotheses H1 and H2 on aggregated data of the UKL and of the TUM teams, respectively. We used the procedures described in Ferguson and Takane (1989) to aggregate data. Essentially, lower frequencies are summed up until they are above five, which leads to a loss of data points of course. There were differences among the teams in the absolute numbers of incompleteness defects, conflicts, and ambiguities that were reported, removed, and so on (cf., Appendix A). For example, the SCR team reported eight incompleteness defects, two conflicts, and eight ambiguities, while the OCTOPUS team reported only four incompleteness defects, two conflicts, and three ambiguities (cf., Table 3). Because of the similarity of the applied RSLs and based on our personal judgment of the capabilities of the teams, we believe that these differences cannot be attributed to the RSLs. On the other hand, the single team results show a consistent profile of incompleteness, conflicts, and ambiguities that were reported, removed, etc., except for the UML team. For example, each of the SCR, Statecharts, and OCTOPUS teams reported more incompleteness defects and fewer ambiguities than one would expect based on the total numbers of these defects in the requirements document. However, the UML team behaved differently from the other UKL teams. The UML team reported a number of defects quite close to the expected number (cf., Table 3). The members of this team said in the final interview that they did not report every Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


issue they became aware of, but only the ones that they believed they could not solve themselves. Because of this lack of conformance with the experimental process, we treated the UML team as an outlier in two tests. The OCTOPUS requirements model was not very detailed. Therefore, it is omitted from further analysis. The results of the statistical tests are discussed in the remainder of this section. We do not discuss conflicts in detail, because their numbers are too low. For this reason, conflicts were pooled with incompleteness defects in the analysis.

H1: Observed and Expected Numbers of Defects Reported We can reject the null hypothesis H01 for the numbers of reported incompleteness defects, ambiguities, and conflicts, as Table 1 and Table 3 show. The observed numbers differ significantly from the expected ones. The application of an RSL leads to higher numbers of detected incompleteness defects and conflicts, and lower numbers of detected ambiguities, as one would expect based on the defect numbers in the document. A UKL team reported on average 14% of the known ambiguities, but 39% of the known incompleteness defects. This result is noticeable. It shows that ambiguities are not detected just because the informal requirements are formalized. If the requirements engineer is not aware of an ambiguity while developing a requirements model, then an RSL does not help to detect the ambiguity. On the other hand, an RSL seems to help detect incompleteness defects and conflicts, because they were reported more frequently than expected.

H2.1: Observed and Expected Numbers of Defects Removed We cannot reject the null hypothesis H02.1 for the numbers of removed incompleteness defects, ambiguities, and conflicts both at UKL and at TUM, as Table 1 and Table 4 show. When an RSL is applied, there is no difference between the numbers of removed incompleteness defects, conflicts, and ambiguities and what one would expect based on the defect numbers in the document. Seventy-two percent (72%) of ambiguities and incompleteness defects were removed on average by a UKL team; 56% of ambiguities and incompleteness defects were removed on average by a TUM team.

H2.2: Observed and Expected Numbers of Self-Resolved Defects We can reject the null hypothesis H02.2 for the numbers of self-resolved incompleteness defects, ambiguities, and conflicts at UKL, as Table 1 and Table 5 show. There is a significant difference between the numbers of defects that are self-resolved and their expected numbers. On average, a UKL team (except for the UML team) resolved 57% of the known ambiguities, but it resolved only 16%



139

of the known incompleteness defects without asking the customer. During the final interviews, it became apparent that the teams often did not recognize ambiguities as such. Therefore, we conclude that ambiguities are more often unconsciously removed than are other types of defects. Unconscious disambiguation is a serious problem, because implicit assumptions are more likely than in our study to be wrong when the system is more complex.

H2.3: Observed and Expected Numbers of Forwarded Defects We can reject the null hypothesis H 02.3 for the numbers of forwarded incompleteness defects, ambiguities, and conflicts both at UKL and at TUM, as Table 1 and Table 6 show. The observed numbers differ significantly from the expected ones. In accordance with our expectation, the application of an RSL leads to higher numbers of forwarded incompleteness defects and conflicts, and to a lower number of forwarded ambiguities, as one would expect based on the defect numbers in the document. On average, a UKL team forwarded only 9% of the known ambiguities, but it forwarded 21% of the known incompleteness defects. In the case of the TUM teams, this difference is even larger. On average, a TUM team forwarded only 8% of the known ambiguities, but it forwarded 31% of the known incompleteness defects. This result confirms that the applied RSLs significantly reduce the level of ambiguity, however they do not eliminate ambiguity.

H2.4: Observed and Expected Numbers of Transformed Defects We can reject the null hypothesis H02.4 for the numbers of transformed incompleteness defects, ambiguities, and conflicts at UKL and at TUM, as Table 1 and Table 7 show. The observed numbers differ significantly from the expected ones. The application of an RSL leads to more transformed ambiguities and fewer transformed incompleteness defects than one would expect based on the defect numbers in the document. On average, a UKL team (except for the UML team) transformed 20% of the known ambiguities, but it transformed only 4% of the known incompleteness defects. Again, the difference is larger for the TUM teams. On average, a TUM team transformed 37% of the known ambiguities, but it transformed only 13% of the known incompleteness defects. If not detected and not removed, incompleteness defects and conflicts tend to become forwarded, while ambiguities tend to become transformed (i.e., misinterpreted). This behavior of ambiguities is a serious problem, since such a misinterpretation can slip through undetected, because of the customers’ reluctance to read requirements written in artificial language. Simulation, the other way of validating formal requirements, can show only the presence of misinterpretations but not their absence. Disastrous software failures may be the consequence.



Table 2. Effect of customer participation Removed Ambiguities

Forwarded Ambiguities

Transformed Ambiguities

UKL

27.3

3.3

7.3

TUM

21.0

3.0

14.0

0.49543

.512695

0.49543

p-value

H3: Customer Participation Table 2 shows the average numbers of reported, forwarded, and transformed ambiguities for UKL and TUM teams, and the results of a MannWhitney U test a nonparametric variant of t test (Ferguson & Takane, 1989). We can reject the null hypothesis H03. There is a significant difference between the numbers of removed ambiguities at UKL and TUM. The UKL teams removed 72% of the known ambiguities, while the TUM teams removed only 55%. Consequently, there is also a significant difference between the number of transformed ambiguities at UKL and TUM. The TUM teams resolved twice as many ambiguities, 37% the wrong way, as did the UKL teams. The fact that there are no significant differences between UKL and TUM in the numbers of forwarded ambiguities shows the homogeneity of the two groups. Recall that a forwarded defect is a defect that was not observed. Therefore, the customer participation should not have an effect on the numbers of forwarded ambiguities. We analyzed the single ambiguities that were removed by the two groups. Interestingly, each ambiguity that was reported and removed by a UKL team was also recognized and removed by a TUM team. There were no ambiguities that could not be solved. The difference lies in the frequency: more UKL teams were able to remove an ambiguity, because they had access to the customer, than did the TUM teams. This observation confirms a key characteristic of ambiguity. Ambiguity, if it is noticed, needs immediate clarification. Any ambiguity that is removed by one team without a report can be misinterpreted unconsciously by another team and can raise a question for a third team. If this question is not answered, the number of transformed ambiguities grows.

THREATS TO VALIDITY The following possible threats to the validity of this study have been identified:



•

•

•

• •

•

141

The one-to-one mapping between team and RSL makes it difficult to prove that the observed differences between the UKL and TUM teams are really caused by the different customer participation (see hypothesis H3). It could be the case that the differences are caused by inherent differences among the RSLs, i.e., between SCR, Statecharts, and UML on one hand, and ROOM, SDL, and Focus on the other hand. However, as mentioned previously, all RSLs are based on finite state machines. Therefore, the second explanation is unlikely. Note that the results regarding H1 and H2 are not affected by this problem, because we did not test for differences among the RSLs. It was possible for the teams to exchange information. However, we told the teams not to do so and there was no exam at the end; thus, cheating made little sense. Since each team applied a different RSL, it was not possible for a team to copy the requirements model of another team in order to save effort. The requirements document might not be representative in terms of size, complexity, and numbers of defects. The Tamagotchi system already exists, therefore the requirements were well understood and the requirements document might expose a different defect profile compared to one describing a completely new system. However, we strongly believe that our results can be generalized to other requirements documents, as far as ambiguities and incompleteness defects are concerned. The number of conflicts in the Tamagotchi requirements document is too low to draw significant conclusions on them. The results regarding incompleteness defects are valid only for incompleteness defects that can be detected without domain knowledge. The RE process that we followed in this case study assumes a relatively complete and detailed requirements document, before an RSL is used. However, if requirements models are created, they are usually created from a sketchy requirements document, in order to avoid describing things twice. The RE process that we followed is typical for safety-related domains such as the aerospace domain. The subjects who participated in the case study are unlikely to be representative of professional requirements engineers. Therefore, we cannot generalize the results to that population. However, this type of study could not be done in industry because of the high effort for developing the same requirements model using different RSLs several times. We believe that student experiments are useful as a pilot for later industrial experiments. For example, we can test hypotheses in a student setting in order to decide whether it is worth investigating them further in industrial settings.



CONCLUSIONS This chapter presents the results of an empirical study with RSLs. The participants applied RSLs to develop an executable requirements model from a given set of informal requirements whose defects were known to the experimenters. The students were told to report every defect they encountered in the informal requirements during formalization. We analyzed the reported issues, and we analyzed the requirements models based on what happened to those defects in the informal requirements that slipped through undetected. The results of our study are: (1) ambiguities are less frequently reported than other types of defects; (2) ambiguities are removed unconsciously more often, i.e., are correctly resolved without being reported, than other types of defects; (3) ambiguities are misinterpreted more often than other types of defects; and (4) ambiguities, if noticed, require immediate clarification. There are two main lessons learned from this empirical study. First, conflicts and certain types of incompleteness defects can easily be detected by formalization and automated checking of the resulting requirements models. However, requirements engineers should not rely on the formalization of informal requirements helping to spot ambiguities in informal requirements; only some ambiguities are detected. Second, ambiguities cannot be considered potential defects, because they also convey the right meaning, but they are real defects. As our study shows, a considerable number of ambiguities tend to become misinterpreted (20% to 37% depending on customer participation). This number is likely to rise if the domain is more complicated than a simple consumer electronics product such as the Tamagotchi toy. Based on these results, we make two recommendations for the use of RSLs in RE processes: 1. Inspection of informal requirements before their formalization. Since RSLs enforce precision, an ambiguity can become an unambiguously wrong formal requirement, which can slip through undetected because of the customers’ reluctance to read requirements written in artificial language and because of the theoretical limitations of simulation. We recommend the inspection of informal requirements for ambiguities to avoid these problems. An inspection technique for spotting ambiguities is introduced in Kamsties (2001). 2. Participation of customers and users during formalization. The development of requirements models from informal requirements is a task of requirements engineers, not customers or users. Nevertheless, we recommend participation of customers and users during the development of these models, not afterwards, in order clarify observed ambiguities as soon as possible. Spotted ambiguities that cannot be clarified immediately with the customer tend to become misinterpreted. If those misinterpretations are clarified later, costly rework of models may be required. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


143

The phenomenon of ambiguity was investigated empirically only in psycholinguistics. We would like to encourage researchers to perform empirical research in requirements engineering to further investigate the impacts of ambiguity during formalization of requirements. Requirements specification languages are unambiguous, but customer requirements are usually stated in natural language first, i.e., the problem of ambiguity is inherent to the early stages of software development.

ACKNOWLEDGMENTS We would like to thank Martin Glinz for his comments on the design of this study, and we thank Dan Berry and Barbara Paech for their comments on previous versions of this chapter. Moreover, we thank the students from the University of Kaiserslautern and the TU München for their willingness to participate in this experiment. Finally, we gratefully acknowledge the constructive and detailed comments of the anonymous referees on the previous versions of this chapter.

REFERENCES Awad, M., Kuusela J., & Ziegler, J. (1996). Object-oriented technology for real-time systems: A practical approach using OMT and fusion. Englewood Cliffs, NJ: Prentice-Hall. Bandai. (1997). Das original Tamagotchi Buch. Tamagotchi & Bandai. Basili, V.R., Selby, R.W., & Hutchens, D.H (1986). Experimentation in software engineering. IEEE Transactions on Software Engineering, SE-12(7), 733-743. Broy, M., & Stølen, K. (2001). Focus on streams, interfaces, and refinement. Springer. Coleman, D., Arnold, P., Bodoff, S., Dollin, C., Gilchrist, H., Hayes, F., & Jeremaes, P. (1993). Object-oriented development: The fusion method. Englewood Cliffs, NJ: Prentice-Hall. Courtois, P.-J., & Parnas, D.L. (1993). Documentation for safety critical software. Proceedings of the 15th International Conference on Software Engineering (pp. 315-323), Baltimore, Maryland. Computer Society Press. Easterbrook, S., & Callahan, J. (1997). Formal methods for V&V of partial specifications: An experience report. Proceedings of the 3rd IEEE International Symposium on Requirements Engineering (pp. 160-168), Annapolis, Maryland.



El Elmam, K., Quintin, S., & Madhavji, N.H. (1996). User participation in the requirements engineering process: An empirical study. Requirements Engineering Journal, 1(1), 4-26. Feather, M.S., Fickas, S., Finkelstein, A., & van Lamsweerde, A. (1997). Requirements and specification exemplars. Automated Software Engineering, 4(4), 419-438. Ferguson, G.A., & Takane, Y. (1989). Statistical analysis in psychology and education. McGraw-Hill. Harel, D. (1987). Statecharts: A visual formalism for complex systems. Science of Computer Programming, 8, 231-274. Harel, D., Lachover, H., Naamad, A., Pnueli, A., Politi, M., Sherman, R., ShtullTrauring, A., & Trakhtenbrot, M. (1990). STATEMATE: A working environment for the development of complex reactive systems. IEEE Transactions on Software Engineering, 16(4), 403-414. Heitmeyer, C.L. , Bull, A., Gasarch, C., & Labaw, B.G. (1995). SCR*: A toolset for specifying and analyzing requirements. Proceedings of the 10th Annual Conference on Computer Assurance (pp. 109-122), Gaithersburg, Maryland. Heitmeyer, C.L., Jeffords, R.D., & Labaw, B.G. (1996). Automated consistency checking of requirements specifications. ACM Transactions on Software Engineering and Methodology, 5(3), 231-261. Heninger, K.L. (1980). Specifying software requirements for complex systems: New techniques and their application. IEEE Transactions on Software Engineering, SE-6(1), 2-13. Huber, F., Molterer, S., Rausch, A., Schätz, B., Sihling, M., & Slotosch, O. (1998). Tool supported specification and simulation of distributed systems. In B. Krämer, N. Uchihira, P. Croll, & S. Russo (Eds.), Proceedings of the International Symposium on Software Engineering for Parallel and Distributed Systems (pp. 155-164). ITU. (1993). Recommendation Z.100, Specification and Description Language (SDL). Jones, C.B. (1990). Systematic software development using VDM. Englewood Cliffs, NJ: Prentice-Hall. Kahn, G. (1974). The semantics of a simple language for parallel programming. Information Processing, 74, 471-475. Kamsties, E. (2001). Surfacing ambiguity in natural language requirements. PhD Thesis, University of Kaiserslautern, Germany. Martin, J., & Tsai, W.T. (1990). N-fold inspection: A requirements analysis technique. Communications of the ACM, 33(2), 225-232. OMG. (1999). Unified Modeling Language. Technical Report, Rational Software Corporation. Version 1.3.



145

Rumbaugh, J., Blaha, M., Premerlani, W., Eddy, F., & Lorensen, W. (1991). Object-oriented modeling and design. Englewood Cliffs, NJ: PrenticeHall. Schneider, G.M., Martin, J., & Tsai, W.T. (1992). An experimental study of fault detection in user requirements documents. ACM Transactions on Software Engineering and Methodology, 1, 188-204. Selic, B., Gullekson, G., & Ward, P.T. (1994). Real-time object-oriented modeling. New York: John Wiley & Sons. Sommerville, I., & Sawyer, P. (1997). Requirements engineering—a good practice guide. New York: John Wiley & Sons. Spivey, J.M. (1992). The Z notation—a reference manual. Englewood Cliffs, NJ: Prentice-Hall. von Knethen, A., Kamsties, E., Reussner, R., Bunse, C., & Shen, B (1998). A comparative case study with industrial requirements engineering methods. Proceedings of the 11th International Conference on Software Engineering an Its Applications (Vol.3: Preprints), Paris, France. Wing, J.M. (1988). A study of 12 specifications of the library problem. IEEE Software, 66-76. Wing, J.M. (1990). A specifier’s introduction to formal methods. IEEE Computer, 23(9), 8-24.



APPENDIX A The following tables provide the raw data on reported, removed, selfresolved, forwarded, and transformed defects for each team/RSL. Note that the expected frequencies (“E”) are rounded. Table 3. Reported defects RSL

Site

Observ./ Expect.

SCR

UKL

Statecharts

UKL

OCTOPUS

UKL

UML

UKL

O E O E O E O E

Incompl.

Confl.

Ambig.

8 4 6 3 4 2 2 2

2 2 2 1 2 1 1 1

8 12 5 9 3 6 5 5

Incompl.

Confl.

Ambig.

10 9 9 9 9 9 9 7 8 8 5 6

4 4 3 4 3 4 2 3 3 4 2 3

27 26 27 26 28 27 19 20 23 22 21 19

Table 4. Removed defects RSL

Site

Observ./ Expect.

SCR

UKL

Statecharts

UKL

UML

UKL

ROOM

TUM

SDL

TUM

Focus

TUM

O E O E O E O E O E O E

Table 5. Self-resolved defects RSL

Site

Observ./ Expect.

SCR

UKL

Statecharts

UKL

UML

UKL

O E O E O E

Incompl.

Confl.

1 5 3 6 7 7

2 3 1 3 2 3

Ambig. 20 15 23 18 23 22



147

Table 6. Forwarded defects RSL

Site

Observ./ Expect.

SCR

UKL

Statecharts

UKL

UML

UKL

ROOM

TUM

SDL

TUM

Focus

TUM


Incompl.

Confl.

Ambig.

2 1 4 3 2 1 2 2 3 2 7 3

2 1 3 1 2 1 3 1 3 1 4 2

3 5 4 7 3 5 3 5 3 6 3 9

Incompl.

Confl.

Ambig.

1 2 0 2 2 2 2 4 2 3 1 3

0 1 0 1 1 1 1 2 0 2 0 2

8 6 7 4 7 7 16 13 12 9 14 10

Table 7. Transformed defects RSL

Site

Observ./ Expect.

SCR

UKL

Statecharts

UKL

UML

UKL

ROOM

TUM

SDL

TUM

Focus

TUM



148 Matuleviè ius

Chapter VIII

Validating an Evaluation Framework for Requirements Engineering Tools Raimundas Matuleviè ius, Norwegian University of Science and Technology, Norway

ABSTRACT Automated support for the requirements engineering (RE) process is a recognized research area. However, the mainstream practice still relies on word processors and drawing tools rather than the requirements engineering tools (RETs). The aim of this chapter is to validate an evaluation framework for RETs. The validation process concerns an RET acquisition process for concrete organizational needs. An observation of maintaining requirements specification shows the important organizational and environmental characteristics for a proper automated support of RE process. The contribution of this work is twofold: first, the validation of the evaluation framework for RETs according to environmental needs in a specific environment, and second the identification of environmental needs, which emerge from the requirements specification maintenance process. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Validating an Evaluation Framework for Requirements Engineering Tools

149

INTRODUCTION The usefulness of automated support for the software development process is recognized in the literature (Kotonya & Sommerville, 1998; Harrison, Ossher, & Tarr, 2000; Nuseibeh & Easterbrook, 2000; Kaindl et al., 2002), but the mainstream of requirements engineering practice still relies on word processors and drawing tools, rather than targeted tools provided by various researchers and practitioners. Current commercial off-the-shelf (COTS) tools for requirements engineering provide capabilities for documenting requirements and facilitating requirements management. As considered in Kaindl et al. (2002) and Karlsson et al. (2002), the tools are well suited for managing large amounts of requirements written in natural language, but not for engineering the requirements. Requirements engineering tools are usually described as CASE (computer-aided system engineering) tools. CASE technologies are defined as tools that provide automated assistance for software development (Lending & Chervany, 1998). In the early 1980s requirements engineering seemed to be a relatively simple task, and existing CASE tools were expected to provide task-related support for software developers. But as discussed in Kelly, Lyytinen, and Rossi (1996), the weaknesses of CASE tools had become apparent, so product and process quality improvement by using CASE tools remains questionable. Kaindl et al. argue that one plausible reason for this is the lack of maturity to adopt tools. Kelly et al. stress the apparent cost of adopting, using, and maintaining a tool, and the inadequate technological sophistication of the CASE tools. Fitting the requirements engineering tools to meet customer requirements remains problematic because companies employ different software analysis, modeling, and engineering methods. Requirements engineering tools vary in their level of support for requirements engineering activities. The evaluation for selection purposes has to be performed before buying any tools. A company cannot base evaluation on its own long-term tool use. Instead, it can only rely on tool surveys, commercial reports, which are unreliable because in many cases they depend on vendor information and become quickly out of date. Evaluation of requirements engineering tools differs depending on the environment, needs, and purposes for tool usage. Botella et al. (2002) investigate two questions during the evaluation of available software tools. First, how are the tools of a given domain described in order to make their comparison feasible? Second, how may the features of the tool be reconciled with respect to requirements for tools? Evaluation and acquisition of a requirements engineering tool to organizational needs should fall into the feasibility limits of an organization. Requirements engineering tools could be evaluated from a theoretical point of view, practical experience from industry could be gathered and evaluated, or tools could be tried out on some realistic examples under organizational settings. The evaluation and comparison would be more complete and structured if an evaluation framework, which targets the questions, is applied. A framework is Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

150 Matuleviè ius

a skeleton structure, which supports the evaluation and comparison process. For vendors of requirements engineering tools, the evaluation framework might help to pinpoint aspects where their tools are weak and should be improved. For a buyer of requirements engineering tools, the evaluation framework might provide a systematic evaluation and help during the decision-making process. A framework to evaluate the requirements engineering tools according to functional requirements was presented by Matuleviè ius and Strašunskas in 2002. The evaluation framework is based on analytical arguments, but not on an empirical investigation. The purpose of the current work is twofold. First, we show how the framework is applied for organizational needs in a particular environment and provide results of the requirements engineering tools evaluation. Second, analysis of the requirements specification and maintenance shows requirements for an automated requirements specification support. The chapter is structured as follows. First, the related works are analyzed. Next, the evaluation framework (Matuleviè ius & Strašunskas, 2002) and its coverage by a semiotic quality framework are considered. The next section offers analysis of the process of framework validation in several steps. First, the environment and the problem, for which the evaluation framework is considered, are described. Then, the preparation phase, which consists of finding the most important organizational requirements for the acquisition of requirements engineering tools and the survey of the tools candidates, is presented. Finally, the maintenance process of the requirements specification is analyzed. Discussion shows important issues for the evaluation framework validity. The final section of the chapter provides the conclusions and future work.

RELATED WORK This section provides definitions for RE and requirements engineering tools used in this chapter. Next, the section analyzes related works, which include the existing tool taxonomies, tool surveys, and frameworks to evaluate and compare requirements engineering and modeling tools. Zave (1997) describes requirements engineering (RE) as the part of software engineering that concerns real-world goals, functions, and constraints of software systems. RE is also a relationship between these factors and a precise specification of software behavior. Requirements engineering tools (RETs) are software tools that provide automated assistance during the RE process and support the RE activities. According to the RETs definition, only tools that support the entire RE process are considered to be RETs. For example, word processors support requirements documentation, and modeling tools provide facilities for requirements modeling. However, these tools are not RETs because they do not maintain the whole RE



151

process. Such software tools could be used for different RE activities, but their output should be included into the RE process. RETs affect: (1) the process quality, because the tools support a large part of the software engineering process, in particular RE; and (2) the product quality, because output of the RE is a requirements specification, which in itself should be of high quality for subsequent software engineering stages. Kelly, Lyytinen, and Rossi (1996), Kotonya and Sommerville (1998), and Ferdinandi (2002) describe requirements management as a part of the RE process and manage changes of system requirements. We use the term requirements engineering tools instead of requirements management tools, as vendors usually call these tools. The functionality of those tools covers RE activities, such as elicitation, analysis, negotiation, and validation, not only management of project changes. Botella and colleagues (2002) emphasize the use of taxonomies for the problem domain description. The INCOSE1 requirements working group suggests taxonomy (Figure 1) for RETs based on functional characteristics. The taxonomy differentiates between requirements generation tools, requirements traceability tools, requirements classification tools, requirements capture tools, requirements identification tools, and requirements elicitation tools. Requirements generation tools (e.g., Criterium DecisionPlus, DecisionPro, Ergo, and TopRank) utilize system simulation results, performance allocations, mission scenarios, and design constraints to generate lower level requirements in an organized and traceable manner. Requirements traceability tools (e.g.,

Figure 1. SE tools taxonomy — Requirements engineering tools Requirements engineering tools

Requirements management tools

Requirements classification tools

Requirements generation tools

Requirements capture and identification tools

Textual requirements capture tools

Requirements traceability tools

Tools for elicitation of requirements


152 Matuleviè ius

Caliber-RM, CORE, DOORS, Cradle, and RDT) enable software engineers to link requirements to their source, to changes in requirements, and to modeling elements that satisfy the requirements. Requirements classification tools (e.g., CARD, DecisionLink, and TechPlan) help software engineers classify the requirements based on work to be done so that the requirement analysis activity can be scheduled and tracked. They help software engineers to classify requirements on how they will be used in modeling so that completeness of traceability can be monitored. Requirements capture tools assemble the information and assist software engineers in finding relationships among entities in the information and in moving among the entities. Requirements identification tools aid software engineers in highlighting requirements in the information before them from extraneous information. Requirements elicitation tools (e.g., QFD 2000, Invention Machine, and QualWlZard) assist requirements engineers in drawing out requirements from system stakeholders. Tools include survey and interview techniques, methods, and functionality. However, INCOSE taxonomy could mislead during the tool selection process. First, the taxonomy classifies tools only in one category, while the RET usually has a broad functionality and could be classified in several categories. Second, the classification of requirements management tools contradicts the definition provided to such tools, because it includes requirements engineering activities. The INCOSE requirements working group also provides a framework for evaluation of RETs and an RET survey. However, the survey is based on vendor information and in such a way is not verified for the purpose. At certain time intervals RET surveys are provided by LaBudde (1997) and Wiegers (1999). But static tool surveys have little long-term value, as new tools are created and features of existing ones are continuously improved. Nikiforova and Sukovskis (2002) describe a framework for comparison of general modeling tools. Authors use experts to validate the approach and to find out important requirements for an evaluation during modeling tool acquisition. But usability and adaptability of the INCOSE working group, Nikiforova and Sukovskis (2002) frameworks are questionable, since there are no descriptions or coordination for applying the evaluation frameworks for particular organizational needs. A methodology for describing the quality factors of software packages using the ISO/IEC quality standard as a framework was introduced by Franch and Carvallo (2002), who demonstrated that selection of packages could be ameliorated by transforming user quality requirements into requirements expressed in terms of quality model attributes. Lang and Duggan (2001) suggest a list of functional requirements for requirements management, communication, and cooperative work systems. But the requirements are not systematically organized and they are not complete for Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


153

the RET evaluation. Pohl (1994) provides the framework of the RE process. The framework considers RE process in a three-dimensional space of requirements representation, requirements agreement, and requirements specification, and in such a way it corresponds to semantic, syntactic, and pragmatic aspects of the RE process. A semiotic quality framework, described by Lindland, Sindre, and Sølvberg in 1994, and extended by Krogstie in 1998, identifies the main quality types of conceptual modeling. The framework is linked to linguistic and semiotic concepts, and it describes the constructivistic world view, which assumes that the relationship between reality and models of this reality is subject to negotiation among community of observers and may be adapted from time to time. The process of preparing a requirements model is a construction of organizational reality through the joint actions of social actors, where the requirements model serves as the artifact in the organizational environment. During the RE process, individual participants externalize their local reality to organizational reality. The internalization is the process of making sense out of the actions and artifacts in the organization and making the organizational reality part of local and individual reality. The main quality types, analyzed by the semiotic quality framework, are physical, empirical, semantic, syntactic, pragmatic, perceived semantic, and social ones. The process of knowledge externalization and then internalization should be supported by software tools. However, Pohl (1994) and the semiotic quality frameworks (Lindland et al., 1994; Krogstie, 1998) are abstract and do not detail the requirements for the evaluation of RETs. This chapter considers the evaluation framework (Figure 2) for RETs provided by Matuleviè ius and Strašunskas (2002). The evaluation framework is constructed according to an analytical literature study (Pohl, 1994; Krogstie, 1998; Lang & Duggan, 2001). The purpose of this work is the empirical investigation and validation of the evaluation framework. The chapter shows characteristics for an automated support of the RE processes. The next section describes the evaluation framework in detail.

EVALUATION FRAMEWORK FOR REQUIREMENTS ENGINEERING TOOLS A framework for evaluation of RET functional requirements is shown in Figure 2. It is based on Pohl’s (1994) three-dimensional orthogonal framework and Land and Duggan (2001) requirements. The framework features are requirement categories, which should be analyzed during the RET evaluation and acquisition. Each category of requirements is followed with a list of activities that should be tested during the RET evaluation process. The framework focuses on evaluation of RET functional features, which express how the tool behaves. The


154 Matuleviè ius

Figure 2. Framework for evaluation of functional requirements for RET

activities do not include nonfunctional ones, like costs of the tool and acquisition, project participant learning, and time consumed for tool adoption. Framework features and activities are derived from the different tasks, which should be supported by an RET during the RE process. Framework features are adapted from Lang and Duggan’s (2002) requirements list, which after refinement was fit in the three-dimensional space of the RE process (Pohl, 1994). Different activities were discovered through investigation of the RETs from a theoretical point of view (Matuleviè ius & Strašunskas, 2002), and through analysis of the quality types and means of semiotic quality framework (Lindland et al., 1994; Krogstie, 1998). The selection of the evaluation activities and the tool features depends on an organizational profile — what kinds of software development practices and techniques are used in an organization, what kinds of problem domains an organization deals with. The list of activities could be extended and improved according to evaluator experience and organizational needs. Table 1 describes the requirements representation activities. The main emphasis is on representation techniques, like informal, semiformal, and formal requirements model definitions, and the traceability relationships between them. The RET should also support extensions of modeling techniques and facilities like import and export of different requirements representations. Table 2 shows the activities, which describe the requirements agreement dimension. The RET should support different groups of users, including (1) people skilled in the usage of tools, and (2) people not skilled in the RE process, Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


155

Table 1. Activities of evaluation framework for RETs (representation dimension) Features

Activities How does the RET…

FEF1.1

FEF 1.1.1 provide natural language description. FEF 1.1.2 allow specifying unique identification (ID) for each separate requirement. FEF 1.1.3 allow importing of requirements and their description from textual document.

FEF1.2

FEF 1.2.1 provide tools for semiformal language description (i.e., ER diagrams, UML diagrams, DFD, OMT). FEF 1.2.2 provide forward/backward traceability between semiformal, and informal, formal descriptions.

FEF 1.3

FEF 1.3.1 provide tools for formal language description (i.e., Z-schemes, algebraic specifications, action semantics). FEF 1.3.2 provide forward/backward traceability between formal and informal, semiformal descriptions.

FEF 1.4

FEF 1.4.1 provide functions for testing traceability between informal, semiformal, and formal requirement description. FEF 1.4.2 create parent-child traceable relations between requirements.

Representation Dimension

FEF 1.4.3 maintain peer-to-peer traceable relations between requirements. FEF 1.4.4 maintain traceable relation between various related information. FEF 1.4.5 maintain forward/backward traceability between a source of requirements, the requirements, and design. FEF 1.5

FEF 1.5.1 allow importing/exporting requirements description from/to textual documents. FEF 1.5.2 allow importing/exporting requirements description from/to graphical documents.

but who know the problem domain and the needs of the proposed software system. The tool should support communication means and collaborative work facilities. Activities for the requirements specification dimension are described in Table 3. The requirements specification should be supported by documentation and reports during the whole RE process. A requirements specification is prepared more effectively and efficiently if it is possible to reuse knowledge and experience for other similar problem domains. RETs should provide the standards in order to ensure the quality level of requirements specification. According to Pohl’s (1994) framework, the final product of the RE process is commonly Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

156 Matuleviè ius

Table 2. Activities of evaluation framework for RETs (agreement dimension) Features


FEF 2.1

FEF 2.1.1 maintain user authentication to the system (i.e., user name, password). FEF 2.1.2 allow grouping users into different groups. FEF 2.1.3 allow creating different views (according to documents, requirements, attributes) for different groups of stakeholders. FEF 2.1.4 register agreement/rationale/discussion/negotiation/changes/history of requirements and by how it was achieved. FEF 2.1.5 call the earlier requirement description/versions and register them into history context.

FEF 2.2

FEF 2.2.1 allow specifying attributes/properties of the requirement. FEF 2.2.2 provide sorting according to different attributes/properties. FEF 2.2.3 provide filtering according to different attributes/properties.

FEF 2.3

FEF 2.3.1 provide WWW-based interface for geographically distributed users. FEF 2.3.2 allow making a copy for modification of an already approved version of requirements description in different abstract levels (document, requirement).

Agreement Dimension

FEF 2.3.3 provide a change approval cycle for multiple change negotiation and approval before posting into common repository. FEF 2.4

FEF 2.4.1 provide the single repository or data and concept dictionary. FEF 2.4.2 provide separate data dictionaries for non-technical users and technical users. FEF 2.4.3 provide the help system to the users.

agreed among all stakeholders, represented in a formal language, and completes requirements specification. The tool evaluator has to determine the evaluation scheme, which would be used to evaluate the tool features and activities. Different evaluation schemes could be used, like {High, Medium, Low}, which indicate correspondence between the framework feature and the tool functionality. For example, if the tool provides the functionality to import requirement, but does not support the exporting, the feature could be evaluated as Medium. On the other hand, some features and activities could be evaluated using predefined measures, for example, time limits to perform a certain function, a number of steps to export or import a textual description. The RET evaluation framework is covered by the semiotic quality framework (Lindland et al., 1994; Krogstie 1998), which, as discussed in the previous section, distinguishes between goals and means to achieve these goals. The main Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


157

Table 3. Activities of evaluation framework for RETs (specification dimension) Features


FEF 3.1

FEF 3.1.1 enable selection and extraction of common domain requirements. FEF 3.1.2 incorporate common requirements to a concrete project. FEF 3.1.3 adapt/spread changes in domain requirements to concrete projects within domain. FEF 3.1.4 provide comparison of domain requirements feasibility.

FEF 3.2

FEF 3.2.1 provide wizards for report generation. FEF 3.2.2 provide possibility to print report according to views and sorting.

Specification Dimension

FEF 3.2.3 provide possibility to print results of rationale, brainstorm, etc. FEF 3.2.4 provide techniques for error checking. FEF 3.3

FEF 3.3.1 correspond to standards of software documentation. FEF 3.3.2 correspond to standards, defined by an organization. FEF 3.3.3 support formal languages for complete, commonly agreed requirements specification.

quality types include physical, empirical, semantic, syntactic, pragmatic, perceived semantic, and social ones. There are two basic quality means on the physical level: externalization and internalizeability. An RET should support basic database functionality using a repository solution for the internal representation of the requirements model. It should deal with functionality such as version control, configuration management, and an advance concurrency control mechanism. Empirical quality deals with error frequencies when a model is read or written by various users; it also applies to the coding and ergonomics of computer-human interaction for modeling tools. The RET evaluation framework distinguishes a variety of elements; looks for error frequency; and checks layout in documents, reports, graphs, and diagrams. RETs have to prevent and detect such errors in order to cover empirical quality goals. Syntactic quality has the goal of syntactic correctness. Requirements descriptions should be completed according to the syntax and vocabulary of the language. An RET should provide the means for error prevention and error


158 Matuleviè ius

detection, which may help to prevent syntactic invalidity and incompleteness errors. Semantic quality is the correspondence between the model and the domain. An RET should provide the means to reach semantic goals — feasible validity and completeness. Some of them could be consistency checking based on a logical requirements description and constructivity, the use of driving questions or baselines to improve completeness of requirements specification. Perceived semantic quality is the similar correspondence between the participant’s interpretation of a model and the participant’s current explicit knowledge. Its goals are perceived validity and perceived completeness. To achieve these goals an RET should provide the means for participant training, discussions, statement insertion and deletion. Pragmatic quality is the correspondence between the model and audience’s interpretation of it. Its goal is feasible comprehension. The comprehensive common repository would allow better understanding of the domain. RET should also provide the means for requirements inspection, visualization, filtering, explanation, execution, simulation, and prototyping. Social quality deals with participant knowledge, including social and technical audience interpretation. The main activities for achieving the feasible agreement goal are model integration and conflict resolution, i.e., pre-integration, viewpoint comparison and conforming, merging, and restructuring. Table 4. Coverage of RET evaluation framework by semiotic quality framework Quality Framework

Specification Agreement Dimension Dimension

Representation Dimension

RET evaluation framework

Physical quality

Empirical quality

Syntactic quality

Semantic quality

Perceived semantic quality

Pragmatic quality

Social quality

Ext.

Int.

Min. err. freq.

Correct.

Valid.

Comp.

Perc. valid.

Perc. compl.

Compr.

Agr.

FEF 1.1

=

=

=

=

FEF 1.2

=

=

=

=

FEF 1.3

=

=

=

=

=

FEF 1.4

=

=

=

FEF 1.5

=

=

=

FEF 2.1

=

=

=

=

FEF 2.2

=

=

FEF 2.3

=

=

FEF 2.4

=

=

=

FEF 3.1

=

=

=

FEF 3.2

=

=

=

=

FEF 3.3

=

=

=

Ext.—externalization, Int.—internalizability, Min. err. freq.—minimal error frequency, Correct.— syntactic correctness, Valid.—validity, Comp.—completeness, Perc. valid.—perceived validity, Perc. compl.—perceived completeness, Compr.—comprehension, Agr.—agreement. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


159

The semiotic quality framework for the RET evaluation framework is summarized in Table 4. The requirements specification should be of high quality for further software development stages. Davis et al. (1993) summarized the work on quality attributes for a software requirements specification, giving the most comprehensive list of the properties. Thus, Krogstie (2001) demonstrated relationships between the semiotic quality framework and the specific quality attributes, described by Davis and colleagues.

EMPIRICAL VALIDATION OF THE EVALUATION FRAMEWORK FOR RETs This section provides the empirical validation process for the evaluation framework, when the framework is applied in the university environment. The validation process consists of two phases (Figure 3): the preparation phase and the execution phase. The preparation phase consists of problem and environmental descriptions, discovery of the environmental needs, and evaluation of Figure 3. Validation process of the evaluation framework


160 Matuleviè ius

RET candidates. The goal of problem description is to characterize the task for which the framework validation process is applied. Environment description discusses the organizational settings. If the problem and environment descriptions are done before, it is possible to start directly with discovering environmental needs. The overall goal of the preparation phase is to find the appropriate RET for maintenance of requirements specification for the information system, used for teaching purposes. The evaluation framework, described in the previous section, is applied in order to discover the most important aspects of the environment. Then the RET candidates are evaluated. The goal of the execution phase is to prepare and maintain the requirements specification for an information system, used during the educational training course to evaluate students’ practical exercises. The RET evaluation process did not select the RET, but it showed the list of RETs’ weaknesses. Because of that, the editing, drawing, modeling, and communication tools, but not targeted RET, were used for maintenance of requirements specification. The observation of the requirements specification maintenance shows requirements for the automated RE process support, which are missing while using tools not targeted for RE.

Problem Description The purpose of the study was to prepare the requirements specification for an information system that is used for teaching purposes. Figure 4 represents the Figure 4. Representation of the problem — Static view



161

static description of the system, and Figure 5 gives the representation of dynamic relationships between problem elements. The system registers two types of users: students and student assistants. Students submit their solutions to the system. Student assistants evaluate the solutions and form reviewer groups. The reviewer groups consist of three to five students whose solutions are accepted by student assistants. The following step is the review process. The reviews are done according to the semiotic quality framework by Lindland et al. (1994). If the review results are essentially different, the student assistant rejects the review, and the reviewer group should evaluate the work again. Otherwise, the review is accepted and the results are sent to the author—the student who delivered the solution. The system was used for the educational training course, in which 216 graduate students of the Norwegian University of Science and Technology Figure 5. Representation of the problem — Dynamic relationships


162 Matuleviè ius

participated. The case also included six undergraduate students of the same university, and they played the role of student assistants.

Environment Description The requirements specification for the system is developed in a university. Different stakeholders participate during the preparation of requirements specification. They perform different roles at various system development stages. The RE process included the following stakeholder groups: organizing actors such as teachers and supervisors of the educational training course; leading actors such as teaching assistants responsible for coordinating and maintaining different system development phases; and developing actors such as undergraduate students responsible for the system developing and testing. At the different project development phases, the actors performed different roles, for example, the leading actors took the role of requirements engineer in the early phase and project manager in the late phase; the developing actors were designing, implementing, and testing the system. The major research interest and experience of the participants includes information systems and processes, workflow analysis and the semantic Web, information management, maintenance and utilization of information resources, and implementation of decision support systems and intelligent agents.

Discovering Environmental Needs In order to discover environmental characteristics, a quantitative analysis, which included a survey, was carried out. Fifteen researchers were invited to participate in a discussion over important characteristics of the environment, and to fill in a questionnaire for discovering the needs of the environment. The quantitative analysis included a questionnaire, containing questions about the importance of the evaluation framework features and activities. Before starting the questioning process, the motivating discussion, which provided definitions of the problem domain and the problem itself, was held. The needs and purposes of the quantitative analysis were explained to the participants. The respondents were asked to evaluate the general requirements for RETs. The importance of evaluation framework features and activities were evaluated on a scale of 0 to 10 (0, a feature or an activity is not important at all; 10, a very important and useful feature or activity). The overall importance for each feature and activity is evaluated as a mean measure: Mj =

1 n ∑ eij n i =1



163

where i is a participant index, j is a feature index, eij is an evaluation of i participant for j feature, n — a number of participants. The agreement about a single feature or activity is evaluated as the variance measure: Vj =

1 n ∑ (eij − M j ) 2 n − 1 i =1

If the variance is relatively low, it means strong agreement about a single feature or activity. And if the variance is relatively large, it indicates disagreement between participants on a feature or an activity. The agreement ranges from 1,4 to 8,8. In order to determine the agreed features and activities, the threshold (t=5) for agreement (variance) is defined. The threshold removes features and activities about which the respondents do not agree (Table 3). Participants were also asked to suggest the features and activities to the evaluation framework, which they feel are needed to evaluate and which are useful for an RET. These features and activities are shown in Table 6 and discussed in the following sections.

Evaluation of RETs Investigation of the RET candidates was done by two evaluators. After individual evaluations a discussion was held in order to achieve the common agreement for the final scores of RET functionality features. The survey of the evaluation is presented by Matuleviè ius and Strašunskas (2002), where RET features and activities are evaluated as: High=3 (very good), Medium=2 (average), Low=1 (poor). The list of RET candidates for evaluation was selected from commercial requirements engineering tools and includes: Caliber-RM Web v.4.0,2 Core 3.1 Trial, 3 Cradle-4, 4 DOORS 5.2, 5 RequisitePro, 6 RDT Version 3.0, 7 Vital Link,8 and XTie-RT 3.1.9 Trial, demonstration, and evaluation versions were evaluated according to manuals and documentation. RETs were also tried out on small examples. The overall evaluation Ej of the RET j may be calculated as the sum of multiplications between two values: feature evaluation Cij (where i is a number of the feature), decided during evaluation of the tool, and feature importance Mi, discovered during the quantitative analysis:

E j = ∑ Cij M i i

The overall evaluation of the RETs is shown in Table 5.


164 Matuleviè ius

Table 5. Overall evaluation of RETs according to organizational needs in particular environment Features

Agreement

Importance

RETs RET1

RET2

RET3

RET4

RET5

RET6

RET7

RET8

FEF1.6

-

10

2

1

2

1

2

2

2

2

FEF1.7

-

10

1

1

1

1

1

1

1

1

FEF1.2

1,4

8,3

3

2

2

2

1

1

1

2

FEF2.3

1,9

8,3

1

2

2

2

1

1

1

1

FEF3.2

1,9

8,3

2

3

2

3

3

2

2

2

FEF2.1

2,6

8,2

2

2

3

3

2

2

2

2

FEF1.1

2,8

8,1

3

2

2

2

3

2

3

2

FEF1.5

3,0

7,9

2

2

2

2

1

2

2

3

FEF3.3

3,4

7,9

1

1

1

2

1

2

2

3

FEF2.4

3,6

7,6

1

3

2

2

2

1

2

1

FEF1.4

3,6

7,5

3

2

1

2

2

2

3

3

FEF2.2

4,3

7,4

2

3

2

3

3

3

3

2

FEF3.1

4,6

7,3

1

2

2

2

1

2

2

3

FEF1.3

8,8

6,5

-

-

-

-

-

-

-

-

196,4

209

196,4

217,5

187,7

186,8

210

218,3

Overall evaluation:

FEF1.6 Define and maintain requirements constraints. FEF1.7 Allow requirements definition in the abstract level. FEF1.6 and FEF1.7 are features suggested by participant during the quantitative analysis. Agreement of the feature FEF1.3 is larger than the threshold t=5. Because of that, the feature was not considered during tool evaluation.

Evaluation Results The results of the quantitative analysis and the RET evaluation demonstrated how to adapt the evaluation framework to the environment. First, the quantitative analysis allocates weights to evaluate the functionality of the RETs. Weights highlight more important RET characteristics against less important, in a way that evaluation would correspond to the organizational activities better. Second, the quantitative analysis validates the features and activities of the RET evaluation framework. The performed investigation shows correspondence between analytical and practical studies. Third, the quantitative analysis discovers features and activities, which are important in an environment, but not mentioned in the evaluation framework. The suggested features and activities are shown in Table 6. Finally, the quantitative analysis shows the usability issues of the evaluation framework. After the motivating discussion, which took about half an hour, the questioning was performed in a relatively short period of time (13 minutes in average). The framework is easily applicable to the environment. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


165

Table 6. Features and activities for the evaluation framework suggested by respondents during the quantitative analysis

Features

Description FEF1.6 Define and maintain requirements constraints. FEF1.7 Allow requirements definition on the abstract level.

Allow import/export of requirements structure.

Allow viewing requirements before acceptance.

Activities

Allow writing comments to requirements and their properties.

Correspond to standards deployed in a software-developing company.

Comments Requirements constraints specify rules and structure according to which the requirements representation is prepared. The feature was assigned to the requirements representation dimension, but it also contributes to the specification dimension, because it defines basis for specification reuse. The activity was added to the FEF1.5 feature list. The respondents made a prediction that it was not possible to have all the representation possibilities in a single RET. And that additional tool support was needed. The activity extends the feature FEF2.3, which describes the collaborative work facilities. They are dealing with maintaining the rationale behind requirements’ elements and suggest the means for agreement of requirements. The activity was suggested to extend the feature FEF2.3. But it may also be added to FEF2.2, which describes the functionality related to requirements attributes and properties. It is important to have a unified requirements specification form subject to the environment characteristics and the working practice. This activity suggests a definition of such a standard in an operating company.

The evaluation results suggest some RETs as possible candidates (RET4, RET8, RET7, and RET2). However, the evaluation showed the limitations of the RETs to solve RE problems, analyzed by Kotonya and Sommerville (1998): • Business needs are not considered. The RE process is seen as a technical process rather than a business process, and it is dominated by technical concerns. RETs are designed for use by skilled specialists, proficient both in engineering methods and the functionality of the tool. The complex functionality of RETs is not suitable for non-proficient stakeholders. For example, FEF3.3, functionality for requirements specification preparation, comprises a complex sequence of functions, and the tools do not suggest any templates to the document. • Stakeholder communication problems. Most RETs are stand-alone applications and do not provide any (or provide weak) possibilities for collaboration work and communication between stakeholders. According to FEF2.3 evaluation, RETs lack brainstorming, discussion, and negotiation functionality, which would support the rationale behind the requirements. • Lack of stakeholder involvement. None of the available RETs suit ideally for use by a multidisciplinary, distributed team where the stakeholders have diverse skills and needs. For example, the FEF2.3 shows that the RETs do not provide means for collaborative work, and FEF2.1 has limitations in defining separate functionality for different user groups. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

166 Matuleviè ius

•

•

Lack of defined responsibilities. RETs should provide possibilities to define activity scenarios for each individual participant of the RE process and let people understand their individual responsibilities. Maintaining an audit trail of the RE process, such as engaging a mechanism for authentication and change approval request, is important for RETs, because people having different educational and practice background are involved in the RE process and may not understand their individual responsibilities. Maintenance of requirements repositories (FEF3.1) would help to keep the predefined scenarios and reuse the guidelines for different user groups (FEF2.1). Help systems and data dictionaries (FEF2.4) should support and help the user to perform operations. Lack of requirements management. If the RE process does not include effective techniques or methods, it may be introduced in an ad hoc way. But RETs like CASE tools operate according to the method, which is defined as a set of rules guiding the use of an RET. Ad-hoc functionality would provide means to react to frequent changes of project requirements and needs during its development.

RETs usually deal only with informal (in some cases semiformal) representations of RE processes and software requirements (FEF1.1-FEF1.2). The functionality to specify abstract requirements definitions and requirements restrictions is poor as well (FEF1.7). The evaluation results could suggest use of the RET with a highest evaluation result. However, the study of the evaluation results shows that the automated RE support is insufficient both for separating activities of RE and for management of the RE processes. Because of the RET limitations, none of the evaluated RETs was chosen, but the editing, drawing, modeling, and communicating tools were used to prepare and maintain the requirements specification.

Maintaining Requirements Specification The requirements specification of the system for educational training purposes was maintained by using standard office tools (word processors and spreadsheets), modeling tools (Rational Rose, rmledit), communication tools (ICQ, MSN messengers, e-mail), and graphical packages (paint). The observation of the preparation and maintenance process of requirements specification was performed. The observation shows the list of shortcomings needed for an automated support of the RE processes. The shortcomings of RE using standard modeling and editing tools are covered by the evaluation framework, and they contribute to validation of the features and activities for the evaluation framework. The observed shortcomings are: • Lack of automatic generation of standard requirements specification (validates the feature FEF3.3). The requirements specification should Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


•

•

•

167

correspond to standards (e.g., IEEE std 830-1998) that should be maintained by an RET. Requirements specification should separate between different RE phases, like requirements analysis and documentation. It should support different software development methodologies and life cycles, which usually depend on organizational policy. One of the RET candidates (RET8) is supposed to cover this feature quite well, because it provides means to define organizational standards for documentation. Requirements analysis and requirements specification are not separated (validates the feature FEF3.2). Requirements analysis is the activity of learning the aspects of the problem domain to determine how to solve a specific set of user needs. Requirements analysis should be followed with different reports, agreement, negotiation, and documentation. Requirements specification is the activity of documenting a software requirements specification. The FEF3.2 is one of the most important features, as the quantitative analysis has demonstrated. Lack of requirements grouping (validates the feature FEF2.2). The project involves different groups of requirements, for example, requirements for time constraints, functionality, usability, reliability, information storage, source code. The FEF2.2 is quite well supported by the RET candidates (RET2, RET4, RET5, RET6, and RET7), but the tools have many shortcomings concerning different modeling perspectives and participant viewpoints. Requirements grouping activities were carried out manually in our case. This is observed as the shortcoming, taking into account a potential development size of the project. Lack to represent requirements model in different techniques, including informal, semiformal, formal representations (validates the features FEF1.1, FEF1.2, and FEF1.3). The requirements model includes one logical requirements model, but the variety of project participants demands different representation techniques and requirements groups during all the RE process activities. Different representations are used during elicitation, analysis, validation, and other RE activities. In the project two techniques were used for informal requirements representation — natural language and use case templates (Kulak & Guiney, 1998). Natural language requirements specification was the most suitable for the initial phases of the RE process. The use case templates provided a way for specification of the structured information (in natural language), which contributed to requirements communication and requirements understandability. Two techniques were used for semiformal requirements representation: state transition diagrams and reference modeling language diagrams (Sølvberg, 2000). State transition diagrams represented the dynamic behavior of the system. The requirements model, prepared with reference modeling language (Figure 4), aims to produce semantically sound models


168 Matuleviè ius

•

•

•

of the real world of the real phenomena, thus separating the real-world modeling and traditional data-modeling. In order to show intentional relationships of the problem elements, the requirements representations were extended with a formal requirements description (Figure 5). The set theory notations were used. They allowed the description of the requirements model using rules and predicates. Cooperative work among multiple users is not supported (validates the feature FEF2.3). The RETs should provide means for the collaborative work support. The RET should include multidisciplinary teamwork, which could be geographically distributed. The tool should include not only the means for users working in an organizational environment, but also for a multidisciplinary teamwork, which could be geographically distributed. RE is best performed by a cross-functional requirements team that provides an adequate experience base to capture all the requirements and to iterate them in a timely fashion. The FEF2.3 is supported quite weakly, in general only by providing a Web-based user interface. During the project, the requirements specification was distributed to different stakeholders by email. That resulted in shortcomings for maintaining replication and specification versions. Lack to provide means for communication and maintenance of rationale (validates the feature FEF2.1). It is important to be able to recreate the rationale behind some requirements specification items, as analysis by Loucopoulos and Karakostas (1995) shows. The communication means provide awareness to the participants and allow the collaborative work through the entire RE process. It was quite a challenging task of the project. First, due to the different communication tools (e-mail, MSN, and ICQ messenger programs), it is difficult to support and argue appropriately different issues of requirements. Second, the rationale needs to be related to each element of requirements specification. Finally, the maintenance of rationale was kept as the track of e-mails sent about the different RE and system development questions among project participants. None of the RET candidates provide adequate support for argumentation. However, some of them (RET3, RET4) support means for a baseline and maintain requirements version control (e.g., RET3). Lack of maintaining traceability relationship among different requirement elements (validates the feature FEF1.4). It is important to keep traceable relationships among all the related information during the RE process. Traceability is needed to relate requirements, their rationale, source, requirements representation, and requirements specification versions. One of the ways to ensure traceable relationships is to maintain requirements repositories. Some of the RET candidates (RET1, RET7, and



•

•

•

169

RET8) provide adequate support for the FEF1.4. In the project traceability between related requirements, requirements properties and other traceable information was kept manually, and this again makes it difficult to perform when the project develops and its requirements specification expands. Lack of repository for storing data of a requirements specification (validates the feature FEF3.1). An RET should support storage of requirements in a requirements repository instead of a chapter document. The requirements repository stores requirements-related information such as: individual requirements, requirements metadata, different requirements representations, and requirements models. It should include different information formats like diagrams, tables, and formal representations of requirements. The requirements repository benefit is to set traceable relations between various elements of the requirements specification. The repository would provide version control and reuse of already agreed common domain requirements. This feature was very poorly supported by the RET candidates. Lack of maintaining different data formats according to modeling techniques and tools used (validates the feature FEF1.5). A tool should allow export and import of requirements models, prepared with other modeling tools. This would benefit the means to specify requirements using different paradigms and various modeling techniques. It would be beneficial to adopt requirements data interoperability between different tools. Difficult to maintain flexible requirements management. Nguyen and Swatman (2003) show that the RE process should not be characterized by a smooth and incremental evolution, but by occasional “crisis” points where the requirements models are reconceptualized and simplified. An RET needs to promote design creativity and support reconceptualization of the requirement model for restructuring the requirements specification. The maintenance of requirements specification shows the important RE aspects, which are missing while using editing, drawing, and modeling tools. The observation was performed along the features and activities of the evaluation framework, and the results of the observation show the limitations of performing these activities without the adequate automated support. The execution phase demonstrates validation issues for the evaluation framework. It is easy to notice that the features and activities on the evaluation framework cover the shortcomings, arising during preparation and maintenance of requirements specification using standard editing, modeling, and drawing tools.


170 Matuleviè ius

DISCUSSION The usability and adaptability of the evaluation framework is one of the key issues during the evaluation of RETs. The important needs of stakeholders, who are going to work with an RET, are explained. Further, the adaptability and usability studies show the validity of features and activities for the evaluation framework. The quantitative analysis was performed in the university, in the academic environment. The majority of the participants include researchers, with different computer science backgrounds and experiences. First, the participants describe the organizational reality from their own perspectives. Second, the different educational background is the potential threat that could affect the interpretation and understandability of the questions and reliability of the answers. In order to maintain the uniform interpretation of the questions, the discussion about the project was held, where the project objectives were presented. Participants’ experience and knowledge could produce a large variance of agreement for the features and activities of the evaluation framework during the quantitative analysis. The problem solution could be the setting of flexible threshold in order to remove the most non-agreed features and activities, but still leaving a sufficient list of features and activities. However, non-agreed features and activities aren’t unimportant, as the observation showed. Non-agreed features and activities need further investigation of the environment and the problem during the RE process. For example, the feature FEF1.3, which says that the requirements model should be described using formal techniques, was selected as a non-agreed one after the quantitative analysis (Table 5). However, observation of the maintenance of the requirements specification has shown that the formal requirements model is important for some groups of requirements. The case showed that formal representation was useful to describe intentional relationships between requirements elements (Figure 5). After the evaluation framework highlights the most important environmental needs, it becomes possible to describe RETs and to express the quality requirements and needs of an environment. The evaluation of the tools was performed by the same researchers who had defined the evaluation framework in the first place, and it is important to recognize that the evaluation of the RET candidates relies on the subjective opinions of evaluators. But the proposed techniques for the evaluation framework acquisition contribute towards the objective evaluation method, because different organizational representatives are involved during the framework acquisition and the comparison of the RET candidates. The quantitative analysis showed that it is not enough to analyze functional requirements of the tools for their acquisition. Functional requirements describe the RE process, but for tools acquisition it is important to consider nonfunctional Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


171

requirements such as cost of the tool, cost of the tool acquisition, project participant learning, time consumed for tools evaluation, and tool adaptation to the environment. The collected information of the quantitative research should be reused for the other RET evaluation phases, where tool candidates could be tested with practical engineering examples. The quantitative analysis provided useful knowledge for the maintenance of requirements specification, using editing, drawing, communication, and modeling tools. The observation of the maintenance of the requirements specification showed that the evaluation framework could be used to highlight the important features of the RE process, and to evaluate them and the RE process itself according to a predefined way. The observation shows the critical requirements needed for the automated support of RE processes. The results of the observation during the execution phase contribute to validation of the evaluation framework presented by Matuleviè ius and Strašunskas (2002) to a high degree.

Threats to Validity • •

•

The following threats to the framework validity have been identified: the evaluation of the tools was performed by the same researchers who had defined the evaluation framework in the first place; the preparation and maintenance of the requirements specification was done by one of the same researchers who are assumed to be friendly to the evaluation framework, and who selected the observations according to the framework features and activities; the analysis of the importance of the framework features and activities consists of academics, and not of RE practitioners with RE experience or experience in using RETs.

CONCLUSIONS This work considers the evaluation framework for requirements engineering tools presented by Matuleviè ius and Strašunskas (2002). The aim of the work is to show the validation activities for the evaluation framework. The process of the evaluation framework validation consists of two phases: the preparation phase and the execution phase. It is important to recognize that any evaluation depends on the environmental characteristics and the organizational working profile. The preparation phase includes the description of the environment, where validation processes are carried on, along with the discovery of the environmental needs and the survey of the commercial RETs. The evaluation framework adaptability and usability tests help to find out the important characteristics of the environment and the needs for an RET acquisition. The execution phase consists of the preparation and maintenance of the Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

172 Matuleviè ius

requirements specification for the educational training course. The observation of the maintenance for requirements specification using editing, drawing, communication, and modeling tools highlights the requirements for the proper automatic support of the RE process. The results of both investigations — the quantitative analysis, executed during the preparation phase, and observation, carried out during the execution phase — contribute to validation of the evaluation framework for RETs.

Future Work The evaluation framework presented by Matuleviè ius and Strašunskas (2002) concentrates on the evaluation of functional requirements for RETs. In practice most tool selections are affected not only by functional attributes, but also by nonfunctional ones. The framework should be expanded with additional dimensions, features, and activities. For example, the nonfunctional dimension would enable evaluation and validation of such features and activities as purchase and training costs, vendor support, vendor and tool reliability, usability, robustness, and stability. Adaptation of the framework to organizational needs depends on organizational profile, executives’ experience, and an organizational practice. The evaluation framework was applied in an academic environment. It would be beneficial to explore the framework in an industrial environment. Thus, the analysis and observation concerning RE practitioners should be carried out in order to investigate the evaluation framework features and activities from the industrial point of view.

REFERENCES Botella, P., Burgués, X., Carvallo, J.P., Franch, X., & Quer, C. (2002). Using quality models for assessing COTS selection. In O.P. Lopez & J.S. Diaz (Eds.), Proceedings of the 5th Workshop on Requirements Engineering (WER 2002) (pp. 263-278), Valencia, Spain. Davis, A.M et al. (1993). Identifying and measuring quality in a software requirements specification. Proceedings of the First International Software Metrics Symposium (pp. 141-152), Baltimore, Maryland. Ferdinandi, P.L. (2002). A requirements pattern. Succeeding in the Internet economy. Addison-Wesley. Franch, X., & Carvallo, I. (2002). A quality-model-based approach for describing and evaluating software packages. Proceedings of the IEEE Joint International Conference on Requirements Engineering (RE’02) (pp. 104-111), Essen, Germany.



173

Harrison, W., Ossher, H., & Tarr P. (2000) Software engineering tools and environments: A roadmap. In A. Finkelstein (Ed.), The future of software engineering. ACM Press. Kaindl, H. et al. (2002). Requirements engineering and technology transfer: Obstacles, incentives, and improvement agenda. Requirements Engineering, 7(3), 113-123. Karlsson, L., Dahlstedt, A.G., Natt, och Dag, J., Regnell, B., & Persson, A. (2002). Challenges in market-driven requirements engineering — an industrial interview study. Proceedings of the Eighth International Workshop on Requirements Engineering: Foundation for Software Quality (REFSQ2002), Essen, Germany. Kelly, S., Lyytinen, K., & Rossi, M. (1996). MetaEdit+ a fully configurable multiuser and multi-tool CASE and CAME environment. In P. Constantopoulos, J. Mylopoulos, &Y. Vassiliou, (Eds.), Advances in Information System Engineering, 8th International Conference (CAiSE’96) (pp. 1-21), Heraklion, Crete, Greece. Springer LNCS. Kotonya, G., & Sommerville, I. (1998). Requirements engineering: Process and techniques. John Wiley & Sons. Krogstie, J. (1998). Integrating the understanding of quality in requirements specification and conceptual modeling. Software Engineering Notes, 23(1), 89-91. Krogstie, J. (2001). A semiotic approach to quality in requirements specifications. Proceedings of the IFIP 8.1 Working Conference on Organizational Semiotics, Montreal, Canada. Kulak, D., & Guiney, E. (1998). Use cases: Requirements in context. AddisonWesley. LaBudde, E.V. (1997). Finding the Right Off-the-Shelf Requirements Management Tool. Available online at: http://www.devicelink.com/mddi/ archive/97/10/013.html. Lang, M., & Duggan, J. (2001). A tool to support collaborative software requirements management. Requirement Engineering, 6(3), 161-172. Lending, D., & Chervany, N.L. (1998). CASE tools: Understanding the reason for non-use. ACM SIGCPR Computer Personnel, 19(2), 13-26. Lindland, O.I., Sindre, G., & Sølvberg, A. (1994). Understanding quality in conceptual modeling. IEEE Software, 11(2), 42-49. Loucopoulos, P., & Karakostas, V. (1995). System requirements engineering. McGraw-Hill. Matuleviè ius, R., & Strašunskas, D. (2002). Evaluation framework of requirements engineering tools for verification and validation. Proceedings of the First International Workshop on Conceptual Modeling Quality (IWCMQ’02) (pp. 9-22), Tampere, Finland.


174 Matuleviè ius

Nguyen, L., & Swatman, P.A. (2003). Managing the requirements engineering process. Requirements Engineering, 8, 55-68. Nikiforova, O., & Sukovskis, U. (2002). Framework for comparison of system modeling tool. Proceedings of the Fifth IEEE International Baltic Workshop on DB and IS (BalticDB&IS’2002), Tallinn, Estonia. Nuseibeh, B., & Easterbrook, S. (2000). Requirements engineering: A roadmap. In A. Finkelstein (Ed.), The future of software engineering. ACM Press. Pohl, K. (1994). The three dimensions of requirements engineering: A framework and its applications. Information Systems, 19(3), 243-258. Sølvberg, A. (2000). Introduction to concept modeling for information systems. IDI-NTNU. Wiegers, K.E. (1999). Automating requirements management. Available online at: http://www.processimpact.com/articles/rm_tools.html. Zave, P. (1997). Classification of research efforts in requirements engineering. ACM Computing Surveys, 29(4), 315-321.

ENDNOTES 1

2 3

4

5

6

7 8

9

INCOSE: Tools Survey: Requirements Management (RM) Tools by International Council on Systems Engineering (INCOSE). Available online at: http://www.incose.org/tools/tooltax.html. Caliber-RM. Available online at: http://www.starbase.com/. CORE. A Guided Tour. Release 3.0. 12.2000. Available online at: http// www.vtcorp.com/productline.html. Cradle: Cradle User Guide & Tutorial. Available online at: http://www. threesl.com/. DOORS. Using DOORS, 12.06.2001. Available online at: http://www. telelogic.com/. RequisitePro. Rational RequisitePro v2002. Evaluators Guide with a Requirements Management Overview. Available online at: http:// www.rational.com/. RDT. Product Overview. Available online at: http://www.igatech.com/rdt/. VitalLink. Vital Link Tutorial & Help. Available online at: http:// www.complianceautomation.com/. XTie-RT. Cross Tie, Version 3.1.03 Tutorial. Available online at: http:// www.tbe.com/.


A Comparison of the FOOM and OPM Methodologies

175

Chapter IX

A Comparison of the FOOM and OPM Methodologies for User Comprehension of Analysis Specifications Judith Kabeli, Ben-Gurion University of the Negev, Israel Peretz Shoval, Ben-Gurion University of the Negev, Israel

ABSTRACT FOOM (Functional and Object-Oriented Methodology) and OPM (ObjectProcesses Methodology) are methodologies used for analyzing and designing information systems. Both integrate functional and objectoriented approaches, but differ in that the analysis specification of FOOM utilizes OO-DFDs (Data Flow Diagrams with object classes that replace traditional data-stores) and a class diagram, while OPM defines a new notational model for specifying the system’s structural and procedural requirements, which combines processes and classes in a unified diagrammatic notation. In this study, we compare FOOM and OPM from the point of view of both user comprehension of analysis specifications and user preference of specifications. The comparison is based on a controlled experiment that measured: (a) comprehension of the analysis specifications, Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

176 Kabeli & Shoval

which includes both structural and behavioral aspects of the system; (b) the time it takes to complete the task of specification comprehension; and (c) the user’s preference of models. Our results show that FOOM performed better than OPM and that the users preferred FOOM to OPM.

INTRODUCTION FOOM is a methodology used for analyzing and designing information systems. It combines two essential software-engineering paradigms: the functional/data approach (or process-oriented) and the object-oriented (OO) approach (Shoval & Kabeli, 2001). FOOM utilizes known methods and techniques such as DFDs (Data Flow Diagrams), and provides simple visual modeling and notations. It covers the structural and the behavioral aspects of a system through the analysis and design phases, and provides a natural and smooth transition from one stage to the other. Since users express their information needs in terms of both functions and data structure — rather than in terms of object classes and their behavior — it seems to us that an appropriate method to carry out the analysis task is to combine functional and data analysis. For designing an information system, however, OO design should be applied since it has been shown to be a better approach to implement software. The Object-Processes Methodology (OPM) is used for analysis and design of information systems. It combines process and object approaches, and provides a unified notation for the structural and behavior aspects of a system (Dori, 2001). OPM utilizes a single graphic tool, the Object-Process Diagram (OPD), in all of its development phases. However, since OPD defines a new notation that combines DFD and OO diagrams, it incorporates many symbols and rules. In our opinion, such diagrams are not very easy to construct and comprehend for large-scale systems. In our mind, reality should be modeled with simple notations that are easy to comprehend and utilize. A single hybrid notation like OPM must be very rich in order to elicit all points of view, thus leading to a complex, perhaps misunderstood model of reality. On the other hand, multiplicity of models and corresponding diagramming tools, as in UML (Unified Modeling Language), may also be too complicated. Too many diagram types may hamper coherent understanding and lead to the production of erroneous models and systems. Methodologies can be evaluated and compared for various dimensions, e.g., quality of the analysis and design products, comprehensibility, learning ability, ease of use, preference by users or professional developers, and others. In this chapter, we present an experimental comparison of FOOM and OPM methodologies from the point of view of end-user comprehension of specifications, time to complete tasks of comprehension, and user preference of models to determine which methodology is better comprehendible. OPM was selected to be compared with FOOM since it also combines and integrates the two essential Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


177

software-engineering approaches: the functional-process oriented and the object oriented. It utilizes “DFD-like” diagrams, applies hierarchical decomposition, and creates class diagrams. However, the methodologies differ in that OPM diagrams (namely OPDs) include more symbol types — mainly various links — than the FOOM equivalent diagrams (namely OO-DFDs and an initial class diagram). The rest of this chapter is structured as follows: the next section provides a short overview of related studies that evaluate and compare various analysis and design methodologies, in particular from the perspective of user comprehension. FOOM and OPM methodologies are then briefly described, with a concentration on the analysis stage, which is the main subject of this study. The comparative experiment is then described, followed by an analysis of the results, our conclusion, and future research topics.

RELATED STUDIES Numerous studies have dealt with experimental comparisons of information systems development methods. Most concentrate on comparing various data models, such as ER diagrams, normalization of relations, and object-oriented schemas (Batra, 1990; Shoval & Frumerman, 1994). Some of these studies compare models or methods from a user perspective in an attempt to find out which of the compared models or methods is easier to comprehend or apply. Others compare models/methods from a designer perspective where the emphasis is on determining which model/method yields better products, or requires less time to complete the analysis or design task. Following is a brief survey of some of these works. Batra and Antony (1994) compared ER with relational modeling from the point of view of analyst performance. Two groups of analysts received a textual description of user requirements of a certain information system, and were asked to map the description to ER diagram and to normalized relations, respectively. The analyst performance was measured according to the correctness and completeness of the produced schemas. The authors found that ER diagram reached significantly higher grades compared to the relations. Shoval and Frumerman (1994) compared EER and OO schemas also for user comprehension. Two groups of users were given equivalent EER and OO schema diagrams. Comprehension of schemas was based on a questionnaire consisting of statements concerning various constructs of the two data models. Subjects of the experiment were students of behavioral science and management, who had been trained to “read” each of the diagrams. The authors found that in terms of comprehending ternary relationships, the EER model performed better, and there was no significant difference in comprehension of other constructs (e.g., binary relationships). Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

178 Kabeli & Shoval

In a follow-up study, Shoval and Shiran (1997) compared the same EER and OO models for quality, namely correctness of schema specification by designers. They also measured the time to complete the design tasks and the designers’ preferences of the models. The subjects of this experiment were students of information systems. Subjects were divided into two groups and given the same design tasks, each group using a different model. Performance was measured according to the number of correct/incorrect constructs created with each model. The authors found that the EER model specified unary and ternary relationships better than the OO model, with no significant differences in specifying other constructs. They also found that it takes less time to create EER schemas and that designers prefer the EER to the OO. Peleg and Dori (2000) compared OPM/T, a variant of OPM for real-time systems, with OMT/T, a similar variant of OMT (Rumbaugh, Balaha, Premerlani, Eddy, & Lorenson, 1991), for specification quality and comprehension. The subjects were students of information systems engineering. They were divided into two groups, where members of each group utilized one of the methods. For evaluating specification comprehension, the subjects in each group were given the specifications of the same example with one of the methodologies. They were asked to demonstrate comprehension using a questionnaire consisting of statements that were classified according to different model constructs (similar to Shoval & Frumerman, 1994). The authors found OPM/T specifications to be significantly more comprehensible than those of OMT/T. In this chapter we perform a comparative experiment for comprehending specifications for the FOOM and OPM methodologies. We begin with a brief description of the two methodologies.

ESSENTIAL OF FOOM METHODOLOGY We only briefly describe FOOM, and demonstrate its analysis specifications using the IFIP Conference example (Mathiasssen, Munk-Madsen, Axel Nielsen, & Stage, 2000). For a more detailed description of FOOM, see Shoval and Kabeli (2001). FOOM is a methodology for the developing business-oriented information systems that combines the functional approach and the OO approach. It can be understood as an extension of ADISSA (Shoval, 1988), a functional-oriented analysis and design methodology. In FOOM, there is a clear distinction between the analysis and the design phases. In the analysis phase, user requirements are presented in terms of two main products: hierarchical OO-DFDs (ObjectOriented Data Flow Diagrams) and an initial class diagram. In the design phase these products are used to create the following products: a complete class diagram, the system transactions, the user-interface, the inputs and outputs, and



179

a detailed behavior schema expressed in pseudo-code and message diagrams. The two phases are described in some detail in what follows.

The Analysis Phase The analysis phase consists of two main activities: data analysis and functional analysis. The main products of this stage are a data model, in the form of an initial class diagram, and a functional model, in the form of hierarchical OO-DFDs. The initial class diagram consists of “data” classes (also termed “entity” classes), namely classes that are derived from the application requirements and

Figure 1. Initial class diagram—IFIP conference C6 Person -Key ID -Name -Address {Street, No.} -Phone# -Email -Set Skills -Set Interests -Set Participate in [Program]

C9 Regular Participant

C8 Author -Set Present [Accepted Paper] -Set Write [Paper]

1,N Participate in 0,N C1 Program

C10 Performing Participant -Status (Sug, Ass, Acc, Rej) -AssDate

-Key Name -Location -Time {Start Date , End Date} -Call for Papers -Call for Participants and Roles -PrelimDescription -Final Program -Set Participate in [Person] -Set Consist Of [Session]

1,N 1

1 ,2

C12 C 13 Panelist Chair -Set Participate [Panel] -Set Control [Session]

Write

C7 Reviewer -Set Papers {Deadline , RevText, Status(Ass, Rev), [Paper]}

1 ,N

Controlling

Present

2,4 0,N

0,N

Participate

0,N Review

1,1

1,N

C11 Speaker -Set LectureAt [Lecture]

C3 Panel -Description -Set Participants [Panelist]

1,N

C2 Session -Key Name -Topic -Date -Hour {Start, End} -Hall Name -Set Facilities -Controlled By [Chair] -Belong To [Program]

x

C 14 Paper -Key Title -Set Keywords -Paper text -Status (Sub, Ass, Rev, Rej, Acc) -Set Authors [Author] -Set Reviewers {Deadline , RevText, Status(Ass, Rev), [Reviewer]}

1,1

Lecturing

0,N

C4 Lecture -Abstract -PresentedBy [Speaker]

0,N

C15 Accepted Paper -Presented In [Papers Presentation] -Set Presented By [Author]

1,N 1,1

C5 Papers Presentation -Set Presented In {StartTime, EndTime, [Accepted Paper]}

Presented In


180 Kabeli & Shoval

contain “real-world” data. (Other classes are added at the design stage.) Each class includes various types of attributes (e.g., atomic, multi-valued, tuples, keys, sets, and reference attributes). Association types between classes include “regular” (namely 1:1, 1:N, and M:N) relationships, with proper cardinalities, generalization-specialization (is-a, or inheritance) hierarchies, and aggregationparticipation (is-part-of) links. (Regular relationships are signified not only by links between respective classes, but also by reference attributes of the respective classes.) Class methods are not included in the initial class diagram, but are added at the design phase. An example of an initial class diagram is shown in Figure 1. (The method names at the lower compartments of the classes should be disregarded, as they do not belong to the initial class diagram.) The hierarchical OO-DFDs specify the functional requirements of the system. Each OO-DFD consists of general (decomposable) or elementary functions, external entities — mostly user-entities, but also time and real-time entities, object-classes (instead of data-stores, which appear in the “traditional” DFDs), and data flows among them. Examples of OO-DFDs are shown in Figures 2 and 3. The analysis phase can start by creating an initial class diagram, which is followed by specifying the OO-DFDs, or vice versa. When the performing order starts with the functional analysis, the analyst first produces a hierarchy of OOFigure 2. OO-DFD-0—IFIP conference O O -D F D -0 : T h e c o n fe re n c e p la n n in g s y s te m (IF IP ) E1 O rganization C om m itee M em ber E2 Program C om m itee M em ber E4 R egular Participant

E5 Perform ing Participant

E6 R eview er

E7 A uthor

C2 S es s ion p rog ra m d e t a ils

C1 0 P e rf o rm in g P a rt ic ipa n t s e s s io n d et ails

p art ic ipa n t de t a ils

pr og ra m d et ails c a ll f o r p ap e rs c all f o r p a ric ip an t s a nd ro le s

1 P r og ra m M a n ag em e nt

p a rt ic ip a nt d et ails

E3 Person

pa rt ic ipa n t de t a ils re v iew e r de t a ils

r ev ie w er de t a ils

E2 Program C om m itee M em ber

C7 R e v iew e r

p ap er /a ut h o r/r ev ie w e r de t a ils

re v iew e r de t a ils

re v ie w e r de t a ils

rem ind er / p a pe rs de t a ils

a ut ho r/ pa pe r de t a ils

2 P ap er s S e lec t ion

E6 R eview er

a c c e pt an c e m e s s ag e

ev er y w e e k

ac c e p t e d pa pe r d e t a ils

T1 pa pe r/rev ie w er de t a ils

E7 A uthor

a ut ho r d et ails C8 A u t h or

C 15 A c c ep t e d P a p er

E2 Program C om m itee M em ber s e s s io n de t a ils


a ut ho r s es s ion de t a ils

pr es e nt at ion d et ails

p re s e nt at ion d e t a ils

s e s s io n de t a ils an d ac c ep t/r eje c t a c k n ow lw d ge

3 S es s ion s an d P a rt ic ipa nt s H a nd lin g

e v e ry d a y

T2

pa rt ic ipa n t de t a ils

C 10 P e rf or m ing P ar t ic ip an t

re qu e s t f o r a c k n ow le d ge

de t a ils o f s es s ion a nd m is s ing p ar t ic ip an t s

s e s s io n d et ails

E7 A uthor


E2 Program C om m itee M em ber

C2 S e s s io n



181

Figure 3. OO-DFD-2—Paper selection OO-DFD-2 : PapersSelection

E7 Author

paper details C14 Paper

E2 Program Com m itee M em ber

C14 Paper

paper details

reviewers details

reviewer details


C7 Reviewer

E6 Reviewer

assigned paper details 2.5 Checkif last reviewer of paper,and update paper statusif yes

paper details

paper details

paper status=Rev

C14 Paper late reviewer details

reviewers review status=Rev paper details 2.6 Send Reminders to Late Reviewers

everyweek

paper details


papers review status=Ass

2.4 Collect Reviewer Recommendation and Update

papers review status =Rev

C7 Reviewer

T1

reviewers details

2.3 Assign Reviewers for papers

reviewsfor paper

E6 Reviewer

appropriate reviewers details for papers details

paper and reviewer details

reviewers review status=Ass


arrived papers details and their authors details

2.2 Present Paper and Reviewer Details

reviewers details

C8 Author

author details

2.1 InputSubmited Paper and Author Details

author details and papers details

2.7 Display Paper's Reviews

reminder

paper and reviews reviewed paper and reviews C14 Paper

paper and reviews

decision

E6 Reviewer


status=accept/ reject 2.8 InputAcceptance Decision paper detsils C15 AcceptedPaper

acceptedpaper details

reject/ accept 2.10 Produce Accept/ RejectMessage 2.9 create presentationif paper accept

accept/reject message and reviews

E7 Author

author details C8 Author

DFDs, which includes classes rather than “traditional” data-stores. Based on these OO-DFDs, he creates an initial class diagram, which includes classes that already appear in the OO-DFDs. This amounts to mainly defining proper associations among the classes and their attributes. However, it is not always easy to properly define the classes in the course of constructing the OO-DFDs, and many changes may be required in the OO-DFDs after the initial class diagram is constructed. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

182 Kabeli & Shoval

An alternative, more effective performing order of the analysis phase is to first create an initial class diagram (based on a description of user requirements), and then to create the OO-DFDs using the already defined classes. The initial class diagram can be created either by first constructing an ERD and then mapping it to an initial class diagram, or by constructing the initial class diagram directly (not from an ERD). While the latter alternative seems to be more direct, some analysts prefer the “indirect” way (namely, from the ERD to the class diagram), since more correct schemas can be created by using the ERD rather than the class diagram (Shoval & Frumerman, 1994; Shoval & Shiran, 1997). Furthermore, a simple algorithm can be used to map an ERD to an equivalent class diagram (e.g., Elmasari & Navathe, 2000). One of our objectives is to investigate the advantages and disadvantages of the alternative performing orders.

THE DESIGN PHASE Since this chapter concentrates on the evaluation of analysis specifications, we describe the design activities only briefly, without showing examples.

Definition of Basic Methods Basic methods are attached to each data class in the initial class diagram. We distinguish between two types of basic methods: elementary methods and relationship/integrity methods. (Application-specific methods for performing various user needs will be added at later stages of design.) Elementary methods include construct (add) object, delete object, get (find) object, and set (change) attributes of object. Relationship/integrity methods are derived from structural relationships between classes. They are intended to perform referential integrity checks, depending on the relationship types between the classes and on cardinality constraints on those relationships. Generally, for each relationship between classes, we can define an integrity method for operations of construct, delete, connect, disconnect, and reconnect.

Top-Level Design of the Application Transactions This stage is performed according to ADISSA methodology, where the application transactions are derived from DFDs (for more details, see Shoval, 1988). According to ADISSA, a transaction is a process that supports a user of the system who performs a business function, and is triggered as a result of an event (like a “use case” in UML terminology). The application transactions are identifiable in the DFDs: a transaction consists of one or more chained elementary functions, and of classes and external entities that are connected to those functions. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

182 Kabeli & Shoval

An alternative, more effective performing order of the analysis phase is to first create an initial class diagram (based on a description of user requirements), and then to create the OO-DFDs using the already defined classes. The initial class diagram can be created either by first constructing an ERD and then mapping it to an initial class diagram, or by constructing the initial class diagram directly (not from an ERD). While the latter alternative seems to be more direct, some analysts prefer the “indirect” way (namely, from the ERD to the class diagram), since more correct schemas can be created by using the ERD rather than the class diagram (Shoval & Frumerman, 1994; Shoval & Shiran, 1997). Furthermore, a simple algorithm can be used to map an ERD to an equivalent class diagram (e.g., Elmasari & Navathe, 2000). One of our objectives is to investigate the advantages and disadvantages of the alternative performing orders.

THE DESIGN PHASE Since this chapter concentrates on the evaluation of analysis specifications, we describe the design activities only briefly, without showing examples.

Definition of Basic Methods Basic methods are attached to each data class in the initial class diagram. We distinguish between two types of basic methods: elementary methods and relationship/integrity methods. (Application-specific methods for performing various user needs will be added at later stages of design.) Elementary methods include construct (add) object, delete object, get (find) object, and set (change) attributes of object. Relationship/integrity methods are derived from structural relationships between classes. They are intended to perform referential integrity checks, depending on the relationship types between the classes and on cardinality constraints on those relationships. Generally, for each relationship between classes, we can define an integrity method for operations of construct, delete, connect, disconnect, and reconnect.

Top-Level Design of the Application Transactions This stage is performed according to ADISSA methodology, where the application transactions are derived from DFDs (for more details, see Shoval, 1988). According to ADISSA, a transaction is a process that supports a user of the system who performs a business function, and is triggered as a result of an event (like a “use case” in UML terminology). The application transactions are identifiable in the DFDs: a transaction consists of one or more chained elementary functions, and of classes and external entities that are connected to those functions. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


183

The products of this stage include transactions diagrams, which are extracted from the OO-DFDs (namely they are partial DFDs), and top-level descriptions of the transactions. The “main” part of each application transaction will eventually belong to a “Transactions class,” which will be added to the initial class diagram. This virtual class will not contain objects—only methods, which are the “main” parts of the transactions, with messages to other methods of other/related classes (as will be elaborated later on). A top-level transaction description is provided in a structured language (e.g., pseudo-code or flowchart), and it refers to all components of the transaction: every data-flow from or to an external entity is translated to an “Input from…” or “Output to…” line; every data-flow from or to a class is translated to a “Read from…” or “Write to…” line; every data flow between two functions translates to a “Move from… to…” line; and every function in the transaction translates into an “Execute function…” line. The process logic of the transaction is expressed by standard structured programming constructs (e.g., if… then… else…; do-while…). The analyst and the user who presents the application requirements determine the process logic of each transaction. This cannot be deducted “automatically” from the transaction diagrams alone, because a given diagram can be interpreted in different ways, and it is up to the user to determine the proper interpretation. The top-level transaction descriptions will be used in further stages of design, namely Input/Output design and Behavior design, to provide detailed descriptions of the application programs and the application-specific class methods.

Interface Design — The Menus Class This stage too is performed following the ADISSA methodology (Shoval, 1988, 1990). A menu-tree interface is derived in a semi-algorithmic way from the hierarchy of OO-DFDs. Generally, a general function connected to a user-entity becomes a menu, and an elementary function connected to user-entity becomes a menu-item within the menu generated from its parent general function. Hence, there is a correspondence of the menus and their menu items to the respective general functions and their elementary functions in the OO-DFDs. All menus of the application will belong to a “Menus class” that is added to the class diagram. The instances (objects) of the Menus class are the individual menus, and the attribute values of each object are the menu items. Note that, at run time, a user who interacts with the menu of the application system actually works with a certain menu object. He/she may select a menu item that will cause the presentation of another menu object (a sub-menu), or invoke a transaction, whose “main” part will be implemented as a method of the Transactions class.


184 Kabeli & Shoval

Design of the Inputs and Outputs — The Forms and Reports Classes This stage is also performed according to ADISSA methodology, and is based on the input and output lines appearing in each of the transaction descriptions. Hence, for each “Input from…” line, an input screen/form will be designed, and for each “Output to…” line, an output screen/report will be designed. Depending on the process logic of each transaction, some or all of its input or output screens may be combined. Consequently, two new classes are added to the class diagram: “forms class” for the inputs, and “reports class” for the outputs. Obviously, the instances (objects) of each of these classes are the input screens and output screens/reports, respectively.

Design of the System Behavior In this stage, the top-level descriptions of the transactions are converted into detailed descriptions of the application programs and application-specific methods. A detailed description of a transaction may consist of various procedures. A certain procedure may be identified as a basic method of some class. Another procedure may be defined as a new, application-specific method, to be attached to a proper class. Remaining procedures (which are not identified as basic methods or defined as application-specific methods) will be defined as a method of the Transactions class; this method is actually the “main” part of the transaction’s program, which includes messages to other, application-specific or basic methods that are attached to proper classes. Hence, at run time, the user — via proper menus selections — triggers a Transactions method, which may call (namely, send messages to) other methods of respective classes, depending on the process logic of the transaction. The detailed description of a transaction is expressed in two complementary forms: pseudo-code and message diagram. A pseudo-code is a structured description that details the process logic of the Transactions method, as well as any other class method. The transition from a top-level description of a transaction to its detailed pseudo-code description is done as follows: Every “Input from…” and “Output to…” line in the top-level description is translated to a message calling an appropriate method of the Forms/Reports class; every “Read from…” or “Write to…” line is translated to a message calling a basic method of the appropriate class; every “Execute-Function…” line is translated to messages calling one or more basic methods or application-specific methods, or to procedures that become the “main” part of the Transactions method. A message diagram shows the classes, methods, and messages included in a transaction, in the order of their execution. It is actually a partial class diagram that includes only the classes involved in the transaction, the method names (and parameters) included in that transaction, and message links from calling to called



185

classes. Message diagrams supplement the pseudo-code descriptions of transactions, but they are created only for non-trivial transactions. To summarize, the products of the design phase include: (a) a complete class diagram, including Data, Menus, Forms, Reports, and Transactions classes, each with the various attribute types and method names (and parameters), and various associations among the classes; (b) detailed menu objects of the Menus class, each menu listing its items (selections); (c) detailed form and report objects of the Forms and Reports classes, each detailing its titles and data fields; (d) detailed transactions descriptions in pseudo-code; (e) message diagrams — only for non-trivial transactions. At the implementation stage, the programmers use the above design products to create the software with any common OO programming language, such as C++ or Java.

ESSENTIALS OF OPM METHODOLOGY OPM (Dori, 2001; Dori & Goodman, 1996) is a systems development method that combines function, structure, and behavior within a single graphic and textual model, in which both objects and processes are represented without suppressing each other. This approach contrasts with object-oriented systems development methods, notably UML, which require several models to completely specify a system. To avoid model multiplicity, OPM incorporates the static-structural and behavioral-procedural aspects of a system into a single, unifying graphic-textual model. In the OPM ontology, objects are viewed as persistent, state-preserving “things” (entities) that interact with each other through processes — another type of “things”; “thing” is a generalization of an object and a process; processes are patterns of behavior that transform objects; and a transformation is a generalization of effect, consumption, and generation. Hence, transforming objects implies affecting them (i.e., changing their states), generating new objects, or consuming existing objects. OPM uses Object-Process Diagrams (OPDs) for the graphic specification. The OPDs are elaborate workflow-like hyper-graphs that model the system or parts of it at various levels of detail. In OPM, objects and processes are connected by procedural links, which can be either enabling links or transformation links. These two different kinds of links are used to connect objects to processes, depending on the roles that the objects play in the process to which they are linked. Objects may serve as enablers — instruments or intelligent agents that are involved in the process without changing their state. Objects may also be transformed (change their state, generated, consumed, or affected) as a result of a process acting on them.


186 Kabeli & Shoval

An enabling link connects an enabler to the process that is enabled. Enabler is an object that needs to be present in order for the process to occur, but it does not change as a result of the process. An enabling link can be an agent link or an instrument link. An agent link denotes that relative to the enabled process; the enabler is an intelligent agent — a human or an organizational unit that comprises humans, such as a department or an entire enterprise. An instrument link is an enabling link denoted by a white circle at the process end, which denotes that the enabler is an instrument — a nonhuman physical or informational object (machine, file, etc.) that must be present for the process to take place, but is not affected by the process. The consumption link is a transformation link denoted as a unidirectional arrow from the consumed object to the consuming process. OPM uses Object-Process Language (OPL) for the textual specification. Based on a constrained context-free grammar, a textual description in a naturallike language can be automatically extracted from the diagrammatic description in the OPD set. Figures 4 to 8 present OPDs of the same IFIP Conference example that was used earlier. Figures 4 to 6 show OPM’s object-class model (equivalent to FOOM’s class diagram shown in Figure 1). Note that Figure 5 is an explosion of the object “Paper,” and Figure 6 is explosion of the object “Program.” Figure 7 shows the main OPD (equivalent to FOOM’s OO-DFD-0 in Figure 2), and Figure 8 details the “Paper Selection” process (which is equivalent to FOOM’s OO-DFD-2 in Figure 3).

Figure 4. OPD—Class model—IFIP conference



187

Figure 5. Explosion of “paper” object

Figure 6. Explosion of “program” object


188 Kabeli & Shoval

Figure 7. Main OPD—IFIP conference

Figure 8. Blowing of papers selection



189

RESEARCH HYPOTHESIS AND EXPERIMENTAL DESIGN Research Hypothesis At the analysis stage, FOOM and OPM seem to have much in common. Both combine and integrate the functional and structural views, utilize “DFDlike” diagrams, apply hierarchical decomposition, and create class diagrams. However, they differ in that the OPM diagrams (namely OPDs) include more symbol types — mainly various links — than the FOOM-equivalent diagrams (namely OO-DFDs), as illustrated by comparing Figures 2 to 3 with Figures 7 to 8. Since reality can be modeled equivalently by different models, the model that consists of less symbol types is easier to learn, apply, and comprehend. Therefore, we hypothesize that the analysis specifications of FOOM might be easier to comprehend than those of OPM.

Experimental Design We compared FOOM and OPM analysis specifications from the point of view of end-user comprehension of analysis specifications of business-oriented information systems, by conducting a controlled experiment. The subjects were fourth-year undergraduate students from the Department of Industrial Engineering & Management. In the course “Information Systems Analysis,” they studied the two methods, mainly in terms of “reading” and comprehending the analysis specifications. The experimental model is presented in Figure 9, and the experimental design in Table 1. We prepared two case studies: IFIP Conference (Mathiasssen et al., 2000) and Greeting Cards (Dori & Goodman, 1996). For each case study we prepared two analysis specifications: one using FOOM, including a set of OO-DFDs and an initial class diagram; and the other using OPM, including an equivalent set of OPDs and object model. We divided the subjects randomly into two groups. Each subject in each group received a set of specifications in FOOM for one case study, and a set of specifications in OPM for the other case study (one at a time). More specifically, subjects in Group A received FOOM specifications of Greeting Cards and OPM specification of IFIP Conference. Subjects in Group B received OPM specifications of Greeting Cards and FOOM specifications of IFIP Conference. To avoid any bias from the ordering effect, the subjects in each group were further divided into two subgroups: in each subgroup they started to work with a different case study and model (Table 1). Along with the specifications, each subject received a questionnaire in which they had to respond to 40 “true/false” statements about facts appearing in the diagrams (see example in Table 2). The start and stop times of the task were recorded (to enable measurement of the time it took to complete the comprehension tasks). We classified the statements according to structure and Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

190 Kabeli & Shoval

Figure 9. The research model

OPM OPDs, including Objects model

Statements on Structural aspects

Statements on Behavioral aspects

Subjects Group A A-1

Comprehension

Statements

Of analysis specifications

Greeting Cards

Methodology

IFIP Conference

Preference of

Case Studies

Group B

A-2

B-1

B-2

Performance

FOOM OO-DFDs, Class Diagram

Time

Methodologies

Table 1. Experimental design Groups

Subgroup A-1

Subgroup A-2

Subgroup B-1

Subgroup B-2

No. of subjects st 1 Task

30

30

33

33

FOOM Greeting Cards OPM IFIP Conference

OPM IFIP Conference FOOM Greeting Cards

OPM Greeting Cards FOOM IFIP Conference

FOOM IFIP Conference OPM Greeting Cards

2nd Task

Table 2. Example of true (T) and false (F) statements for IFIP conference Structure:

(T) A chairperson can chair more than one session. (F) A reviewer must review at least two and no more than four papers.

Behavior:

(T) A “Call for Participants and Rules” document is prepared by a PC member. (F) A PC member sends reminders to late reviewers.

behavior, in order to distinguish differences in comprehensibility of these two categories for each methodology. In order to avoid bias arising from the order of statements in the questionnaire, we prepared four different sets of questionnaires for each case study, each with a different ordering of the (same) Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


191

statements. At the end of the experiment, each subject was asked to express his subjective preference of each method, using a seven-point scale. Model comprehensibility was measured according to the following criteria: grade on structure category, grade on behavior category, overall grade, time to complete the comprehension task, and preference of method. We computed comprehensibility grades by counting the number of correct answers (whether “true” or “false”) given to the statements by each subject on each category. Based on this, we computed the average grade for category, methodology, and case study, and then we tested the significance of differences in average grades. In addition, we computed the average time to complete a task for each methodology and case study. Based on that, we tested the significance of time differences. Finally, we computed the average preference of models as expressed by the subjects at the end of the experiment.

RESULTS AND DISCUSSION Table 2 presents a summary of all effects between the factors (namely methodologies, case studies, and task order), according to two dependent variables — overall grade and time — using three-way ANOVA/MANOVA. The values of the three factors are: OPM and FOOM methodologies, Cards and IFIP case studies, and 1st and 2nd order of tasks. The results show significant differences in the main effects (1, 2, 3 in Table 3) and insignificant difference in their interactions (12, 13, 23, 123). The result for the methodology main effect (1-Method) shows that FOOM is much more comprehensible than OPM: users of FOOM achieved higher overall grades, and it took them less time to complete the tasks. The result for the case study main effect (2-Case) shows that Greeting Cards was easier to comprehend than IFIP Conference, as its overall grade of comprehension was higher, and it took less time to complete this task. Because of the difference observed in the two case studies, we compared the methodologies for each case study separately as shown below. The result for the task order main effect (3Task) shows that subjects spent more time on the first task regardless of the case study or methodology, and achieved marginally higher grades on comprehension of this task (due to no interactions, as seen in Table 3, the interactions do not present in Tables 4 and 5) Table 4 presents the results for each case study separately. For each case study, FOOM scored higher for both comprehension and time. Subjects achieved higher overall grades, and it took them less time to complete the tasks with FOOM compared to OPM. (Table 3 does not show the Task effect and its interactions, because there was not a significant difference.) Table 5 shows the mean grades of comprehension for the structure and behavior categories separately. The results reveal that in the more complex case Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

192 Kabeli & Shoval

Table 3. Results of 3-way ANOVA Factor No.

Factor Name

Values of Factor

1

Method

2

Case

3

Order of Task Interacti on Interacti on Interacti on Interacti on

12 13 23 123

OPM FOOM CARDS IFIP st 1 nd 2

F

p-level (á=0.05)

11.43

0.000029

27.60

0.000000

5.346

0.006004

1.147

0.234654

0.123

0.884464

1.055

0.351353

0.842

0.433656

Mean Overall Grade (%) 68.413 71.452 73.743 66.122 70.328 69.537

Mean Time (minutes) 73.358 60.054 58.976 74.435 72.394 61.018

Table 4. Results for each case study Case

Category

Methodology

Mean

F

(á=0.05)

Grade Greeting

Overall

OPM

72.80

Cards

Grade (%)

FOOM

74.42

Time (min.)

OPM

63.12

FOOM

53.87

IFIP

Overall

OPM

63.83

Conference

Grade (%)

FOOM

68.26

Time (min.)

OPM

84.00

FOOM

65.76

p-level

3.297

0.0438

9.430

0.000275

Table 5. Results of structure and behavior categories Case

Category

Methodology

Greeting Cards

Structure

OPM FOOM OPM FOOM OPM FOOM OPM FOOM

Mean Grade 73.48 77.5 72.5 73.09 63.09 65.15 64.23 69.93

OPM FOOM

4.000 4.734

Behavior IFIP Conferen ce

Structure Behavior

Preference of Method (7-Point scale)

F 1.176

p-level (á=0.05) 0.282

0.061

0.806

0.365

0.548

4.84

0.032

-2.15 (paired t-test)

0.0368



193

study (IFIP Conference), FOOM scored significantly higher than OPM for behavior and insignificantly higher for structure. In the simpler case study (Greeting Cards), FOOM scored somewhat, but insignificantly higher than OPM for both structure and behavior. The insignificant difference for structure can be explained by the fact that the objects model of OPM is not much different from the class diagram of FOOM. Note that the grade for the behavior was significantly higher when using FOOM with the more complex case study (IFIP Conference). This shows that while in small/simple systems there may not be significant differences in comprehension, in complex systems FOOM is more comprehensible. The last line of Table 5, representing the subjects’ preference of methodology, shows a significant preference for FOOM (mean grade 4.734) over OPM (mean grade 4).

SUMMARY AND FURTHER RESEARCH We found that overall, the analysis specifications of FOOM are significantly more comprehensible than those of OPM, and that with FOOM it takes less time to complete the comprehension task. Moreover, users prefer the specifications of FOOM. Both case studies show that there are no significant differences between the methodologies in comprehension of structure. However, the comprehension of behavior of FOOM is significantly better when used for a complex case study (IFIP Conference). This study suggests therefore that for large-scale applications, a model like FOOM with fewer symbols and simple notations is more comprehensible and is preferable to a model with more symbols and complex notations. The experiment presented in this chapter has several limitations. For one, we concentrate on the domain of business-oriented information systems, but not on other domains (e.g., real-time systems). Another limitation is that the authors, who developed FOOM, were the same who taught and tested the methodologies. (On the other hand, OPM was taught for a longer time than FOOM — four, three-hour meetings compared to two, three-hour meetings.) In future research, we plan to conduct two more experimental evaluations of FOOM. In one experiment we will compare the analysis and design specifications of FOOM and OPM (or perhaps another OO methodology) from the point of view of analysts and designers. Subjects will be given tasks to analyze and to design systems with the two methodologies for the purpose of measuring quality (namely correctness) of the products. In another experiment we plan to investigate the preferred order of analysis specification in FOOM.


194 Kabeli & Shoval

REFERENCES Batra, D., & Antony, S.R. (1994). Effect of data model and task characteristics on designer performance — a laboratory study. Information Journal of Human-Computer Studies, 41(4), 481-508. Batra, D., Hoffer, J., & Bostrom, R. (1990). Comparing representations with relational and EER models. Communication of the ACM, 33(2), 126-139. Dori, D. (2001). Object-Process Methodology applied to modeling credit card transactions. Journal of Database Management, 12(1), 4-14. Dori, D., & Goodman, M. (1996). From object-process analysis to objectprocess design. Annals of Software Engineering, 2, 25-50. Elmasari & Navathe (2000). Fundamentals of databases systems. Reading, MA: Addison-Wesley. Mathiasssen, L., Munk-Madsen, A., Axel Nielsen, P., & Stage, J. (2000). Object-oriented analysis and design. Aalborg, Denmark: Marko Publishing. Peleg, M., & Dori, D. (2000). The model multiplicity problem: Experimenting with real-time specification methods. IEEE Transactions on Software Engineering, 26(6), 1-18. Rumbaugh, J., Balaha, M., Premerlani, W., Eddy, F., & Lorenson, W. (1991). Object-oriented modeling and design. Englewood Cliffs, NJ: Prentice-Hall. Shoval, P. (1988). ADISSA: Architectural design of information systems based on structured analysis. Information System, 13(2), 193-210. Shoval, P. (1990). Functional design of a menu-tree interface within structured system development. International Journal of Man-Machine Studies, 33, 537-556. Shoval, P., & Frumerman, I. (1994). OO and EER conceptual schemas: A comparison of user comprehension. Journal of Database Management, 5(4), 28-38. Shoval, P., & Kabeli, J. (2001). FOOM: Functional- and object-oriented analysis and design of information systems—an integrated methodology. Journal of Database Management, 12(1), 15-25. Shoval, P., & Shiran, S. (1997). Entity-relationship and object-oriented data modeling—an experimental comparison of design quality. Data & Knowledge Engineering, 21, 297-315.


Participatory Development of Enterprise Process Models

195

Chapter X

Participatory Development of Enterprise Process Models Reidar Gjersvik, SINTEF Industrial Management, Norway John Krogstie, Norwegian University of Science and Technology, Norway and SINTEF Telecom and Informatics, Norway Asbjørn Følstad, SINTEF Telecom and Informatics, Norway

ABSTRACT In this chapter we present practical experience from using a technique we call Modeling Conferences, a method for participatory construction and development of enterprise process models. Process models are an important way to support communication, coordination, and knowledge development within an organization. The Modeling Conference method focuses on broad participation from all actors in the organization, is grounded in a social constructivist perspective, and has its theoretical basis in the method of search conferences and process modeling. In an engineering consultancy firm, the Modeling Conference method has been used to develop process models for main common work tasks that have been implemented on an intranet. Independent evaluations show that participation through the Modeling Conferences led to significantly more ownership to the process models, and that the actors have developed new collective knowledge. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

196 Gjersvik, Krogstie & Følstad

INTRODUCTION The Modeling Conference is a method for participatory construction and development of enterprise models. In this, it takes as a starting point the business processes approach to understanding how organizations work. However, while most approaches to the mapping and “re-engineering” of business processes tend to be expert and management focused, the Modeling Conference technique focuses on participation from all the related parties, and the link between organizational learning and institutionalization through technology. The focus on participation stems from a constructivist approach to understanding organizations. Organizations are seen to be a continuous construction and reconstruction of an organizational reality as individuals and groups enact their own local reality through everyday practice (Berger & Luckman, 1966; Gjersvik, 1993). In order to introduce change in the organizational construction processes, the method of change should reflect the joint participation in the everyday construction processes. Because of this, we have developed a method that has at its core the method of Search Conferences (Emery & Purser, 1996). In the next section, we will present the background to our approach. In the third section the Modeling Conference method is outlined, and in the fourth section the application of the method on a specific case is presented. The results from independent evaluations of the longer-term results from using the approach are presented, and finally, general experiences from applying the approach in different settings are summarized, and further work is pointed out.

BACKGROUND Processes have been a key concept in management and organization for the last decade, especially related to Business Process Reengineering. A process has been defined as: “(…) a structured, measured set of activities designed to produce a specified output for a particular customer or market. It implies a strong emphasis on how work is done within an organization.” (Davenport, 1993, p. 5) Process orientation today is most related to organizational thinking, highlighting the importance of information technology. The methods used to map and visualize processes are also very similar to the models being used by various information systems and software engineering approaches. This may constitute a problem when broad participation is desirable, as the expressiveness of traditional modeling languages becomes a barrier when laypersons from many different fields try to use them to map their work processes. When deciding on



197

the suitability of the modeling language to be used, there is a general tradeoff between expressiveness and formality of the one hand, and the suitability for the active use of the language and the comprehension of models developed in the language by the participant on the other hand (Krogstie, 2001). Enterprise Modeling (EM) (Fox, 2000; Loucopoulos, 2000) as a holistic concept was first used by the U.S. aircraft industry in the late ’80s. The term may have been used earlier, but then only to denote any kind of model, ranging from mathematical models to IT architecture models, to data models, geometric models, and even physical mock-up models. The most common kind of enterprise models are process models, showing the transformation from input to output, and the tools, controls, and resources necessary to do this. On the other hand enterprise process modeling in particular is not done for one specific goal only, which partly explains the great diversity of approaches found in literature and practice. Four main categories for enterprise modeling are proposed: 1. Human sense-making and communication. The main purpose of the enterprise modeling is to make sense of aspects of an enterprise and communicate with other people. 2. Computer-assisted analysis. The main purpose of the enterprise modeling is to gain knowledge about the enterprise through simulation or mathematical deduction. 3. Model deployment and activation. The main purpose of enterprise modeling is to integrate the model in an information system and thereby actively take part in the work performed by the organization. Models can be activated in three ways: • through people guided by process ‘maps’, where the system offers no active support or enforcement; • automatically, where the system plays an active role in enforcing the ‘script’, as in most traditional workflow engines; • interactively, where the computer and the users cooperate in interpreting the model in the situations that arise. The computer makes decisions about prescribed fragments, while the users resolve ambiguities. 4.

The enterprise model is a basis and gives the context for a traditional system development project, without being directly implemented.

An orthogonal dimension to these four is the temporal dimension, i.e., if one is to model the past, the present (as-is), or the future (to-be). Another key differentiator is to what extent the focus is on processes internal to a company, or on inter-organizational cooperation. Finally one can differentiate between process models on a type level and on an instance level. A number of modeling frameworks have been developed (ARIS, CIMOSA, GRAI, IEM, PERA, GERAM) that provide business process modeling lanCopyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


guages, allowing description of business processes with various degrees of details and for different points of view on the process itself. Enterprise Modeling is a capability for externalizing, making, and sharing enterprise knowledge. The making and sharing is key to why modeling has value to the enterprise. The model must be more than the sum of individual known views. Enterprise Modeling happens when knowledge workers apply their knowledge and software tools in a creative and purposeful process. The tools can either be used stand-alone to produce various kinds of enterprise knowledge model views, be integrated as front-ends to other systems, or be part of an environment providing a contextual user environment. In the ICG case presented in this chapter, in addition to being a common frame of reference for human sense-making and communication, the process models are also used for model deployment and activation through people, by making process maps available on an intranet. The core of the Modeling Conference method has been adopted from the Search Conference method (Emery & Purser, 1996). The Search Conference is a method for participatory, strategic planning in turbulent and uncertain environments. It has been used in various settings, i.e., community development, organization development, the creation of research initiatives, etc. It has also been done with a number of different designs. The method is however based on the following basic ideas: open systems thinking, active adaptation, genuine democracy, and learning. The result of a Search Conference is a set of action plans, addressing various challenges that the conference has prioritized and that people at the conference have committed themselves to implement. The plans may not always be congruent or coordinated, but there is a shared understanding among the participants on why each of the plans is important for parts of the system.

MODELING CONFERENCE METHOD The Modeling Conference combines process modeling and search conferences, by doing process modeling in a structured conference environment, promoting broad participation. The argument for participation is primarily based on the social construction view of the organization described briefly in the introduction. A set of principles lies at the heart of the Modeling Conference. At the core of these principles are the ones listed for the Search Conference above, but a few are added due to the special purpose and techniques of the Modeling Conference: • Open systems thinking. The unit of development (organization, community, enterprise) is viewed as an open system, interacting with its environment. At the conference, both the whole system itself and the main parts



•

•

•

•

•

•

•

•

199

of the environment should be modeled. The process is always in a context, interlinked with other processes and the rest of the contextual environment. Active adaptation. A further consequence of the open systems view is that the system needs to adapt to the environment. However, in a turbulent environment, passive adaptation is not enough. The organization needs to influence and interact with its environment, to actively create a context in which it can develop. Genuine democracy. As in a search conference, the Modeling Conference is based on the assumption that all human actors in a system or process are experts on how the system/process works as seen from their point of view. All local realities are valid and important in constructing the common model. Given a suitable structure, the participants are jointly able to analyze and understand the situation, and create suitable action plans. Simplicity: Modeling languages, methods, and concepts should be simple so that it is possible for actors with various local realities to express themselves, and thus make real participation possible (Gjersvik & Hepsø, 1998). Pragmatism. An important issue in the design of the conference is to find a structure and a mix of methods that will work for all participants, and that are useful in producing a satisfactory outcome for the actors in the organization (Greenwood & Levin, 1998). The use of the process model as a communicative and reflective device. The models are, in addition to being the product of the conference, the main device driving the conference process. The use of large physical process visualizations encourages dialogue among the participants within a common frame of reference (Gjersvik & Hepsø, 1998). Learning. The conference should create conditions under which the participants can learn from each other, but also from the way they work at the conference. Learning should not be related to the process model, but to leading a discussion about the process and to the development of an understanding regarding what constitutes knowledge and truth about the process and the organization. We have used the ideas of triple-loop learning (Flood & Romm, 1996), stressing that the conference is only one event in a continuous, multi-level learning process. The Modeling Conference is performed according to the following rules: The whole process is performed in one room. All relevant actors in the process should be present or represented in the modeling tasks. In many cases, this also includes outside actors, like users, owners, customers, and governmental and municipal authorities. The tasks alternate between group work and plenary work.



• • • •

•

The participants primarily represent themselves, but are jointly responsible for the content of the conference. The staff facilitates the work, and is responsible for the method used during the conference. The modeling language, tools, and the overall method must be simple, so that the participants may focus on the content. The main outcome of the conference is a process model, which names the key processes, products, and roles. Additional results are related to this process model. The following concepts and notations are used: Process: A series of tasks that produce a specific product.

Process

•

Product: The result of a process, and in demand by a customer. A process may have several products. We distinguish between end products and intermediate products.

•

Customer: Someone who demands and uses the product of a process. Often, the customer is another process. For instance, the process “Install technical applications” is a customer of the process “Draw technical installations,” and demands the product “Drawings of technical installations.”

The conference preferably lasts one-and-a-half days at least. Every group has a large sheet of paper on the wall, on which they work. All symbols are precut and can be attached to the sheet of paper. Through these simple symbols and physical way of working together, one gets great flexibility and intensive learning, but they also limit the form of work. The results of the group work are presented in plenary sessions for discussion and joint construction of consolidated models. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


201

The documentation from a Modeling Conference is a report and a process model. The most important outcome of the conference is the ownership that the participants develop through the construction process, which makes the model an important common reference for further, more detailed development. The conference agenda is designed so that the actors of the conference should develop models based on their own local reality before they enter a discussion with actors having (presumably) different local realities. We always start with homogenous groups, where people with the same background develop their process models. After this, the participants are more comfortable with the modeling language and tools, and have more self-confidence about their own point of view. This is especially important in organizations where there is a high risk of some groups of actors (i.e., management, experts) having model power over other participants through having a previously developed model available (Bråten, 1973). We subsequently mix the participants in heterogeneous groups, where the whole modeling starts over again. The difficult part of the agenda is after the second modeling task, where the models of several groups are to be merged into one. This is done in a plenary session. The conference leader needs to be very attentive to the logic of the different groups, so that he or she is able to combine the elements from different models into one coherent whole. It is important that this plenary session is allowed to take the time it needs to obtain a consensus about the model. This participatory technique has some commonalties with what is found within the field of Participatory Design (Schuler, 1993), but focuses as we have seen primarily on enterprise modeling and not on the design of information systems in general.

CASE STUDY: DEVELOPMENT OF COMMON PROCESS MODEL FOR THE WORK TASKS OF ICG AS A BASIS FOR A NEW INTRANET ICG is an engineering consultancy company with 700 employees. Most of the employees are found in three major cities in Norway (Oslo, Fredrikstad, and Trondheim), but there are also local offices spread throughout Norway and 100 employees abroad (in Africa and Eastern Europe). ICG is the result of a merger between three different companies, each a specialist within an engineering field. The merger was effective at the start of 1999.



Knowledge Infrastructure Project and Preparations for the Modeling Conferences In 1999, ICG started the development of their intranet. As opposed to the existing intranet, the new ICG intranet was meant to be a real support and coordination of the actual work the engineers and consultants in ICG do. One of the ambitions in the ICG merger had been to develop synergy effects through new ways of working across engineering disciplines, and an understanding of engineering work processes was seen as instrumental to that. We decided to use the Modeling Conference as the method to construct the processes. In addition, to create a process-oriented intranet, we wanted the project to focus on both organizational and technological change. There had to be participation, both to create ownership and in order to take seriously that this was a social construction process. We wanted to reflect the continuous reconstruction of local and organizational realities, both in the process construction (the modeling conferences) and through an iterative system development process (not to be described in this chapter). The process construction started with a top management meeting, determining the key processes of ICG. This was done through a group discussion, in which the processes were divided into business processes and support processes. The result is shown in Figure 1. The most important outcome from this discussion was the decision that the process “to initiate, execute, and complete a project 1” (Project Execution) was to be the first process to be constructed and supported by the intranet. Process owners and change agents were appointed for each process. Figure 1. ICG key processes Builing planning process: Plan buildings and installations

Plans for the devlopment of buildings and technical installations

Counsellingprocess: Give advice and develop knowledge

Advice on strategy, concepts and processes

Product development process:

Project execution process: Start, carry through and close off projects

Concepts, software, and methods

Plan and build complete buildings

Operate and maintain plants

Complete buildings

Efficiently operated plants

Performed projects

Business processes Support processes

Strategy and market analysis

Human resources

Develop and maintain strategy and market analysis

Hire, develop, and model competency

Technology and infrastructure

Information

Capital

Procure, develop Create, structure Assure access to and maintain and make available financial capital technology information

New knowledge and methods

Create, structure and develop knowledge



203

The Modeling Conferences We performed four modeling conferences on the Project Execution process: one in each of the three main cities, where people from engineering tended to dominate, and one in which we focused on people from consulting and from smaller, local offices. We would have preferred to mix people more within each conference, but had to take into consideration the costs of the conferences. The participants at each conference were chosen by the process owner and the change agents. We stressed the importance of the project, and that the participants thus should be representative of the whole organization (both experienced and inexperienced, and within various fields). Although we might have had a better method for this, it worked well in practice, and the conferences (apart from the one in Trondheim2) had both a good attendance and a good mix of participants. The process owner and the change agents, assisted by the conference leader, also put together the homogenous groups for the first group work. Mostly, these groups were formed based on skill area. The program for each conference was pretty similar. The learning we developed from conference to conference was mostly related to the content of the model and the way we managed the discussions. The program for the conferences is shown in Figure 2.

Figure 2. Program for the modeling conferences in the modeling of the project execution process Introduction • Welcome. The goals of the conference and of the Project Execution process. The Process Owner. • Enterprise modeling, processes, and the Modeling Conference. The Conference Leader. • Presentation of the initial model. The Conference Leader. Group Work 1: Goals for the Project Execution Process. Construction of a Process Model. (90 min.) Homogenous groups. Plenary presentation of results. Lunch Group Work 2: Construction of a Process Model II. (75 min.) Heterogeneous groups. Plenary presentation of results. Plenary: Construction of a Joint Process Model. Group Work 3: What Information and Tools Are Needed in the SubProcesses? (50 min.) Same groups as in group work 2. Plenary presentation of results. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


Due to cost considerations, we had to limit the conferences to one workday. This meant that we could not run the modeling conferences the way we originally intended to, where we let the participants create the first model from scratch. We had to give them a head start, thus we designed an initial model of project execution where the basic steps were included (Figure 3). In the introduction that the conference leader gave to the participants at the opening, the initial model was used to describe the principles of process modeling. They were also told that their task was to evaluate and validate the model, and preferably to change it completely so that it fitted the way they executed a project. In the first group work, the participants were also asked to come up with goals for the Project Execution process. The Process Owner gave an introduction to his and ICG’s goals at the beginning of the conference, but based on our constructivist view, the goals had to be run through the participative process of the conference before they could acquire meaning for the participants. The conferences went through the two-step process modeling described earlier. At all conferences, there was great enthusiasm and intense group discussions. It was a challenge to get each group to visualize the whole Project Execution process, as some groups got caught up in detailed discussions about a minor part of the process. Managing this was a difficult balance, because it was important to have good discussions about what words and expressions to use, and what sequence the sub-processes had. The conference leader had an important role here, as he observed all the group discussions, and intervened if the discussions seemed to be stuck on a non-productive issue. Figure 3. Initial model for the project execution process Market project

Prospect

Initiate and register project

Registered project

Develop bid and negotiate contract

Signed contract

Execute project work

Product as ordered by the customer

Evaluate project

Evaluation report

Progress Invoice Manage project Money in the bank



205

Another challenge was to keep the participants from thinking in terms of screens and user interfaces. Their task was to construct a good process. Once this was done, it was the job of the process group and the systems development group to find ways to create images of this process that could be used on the intranet. The most demanding part of the conferences was the construction of a joint process model. Even if we had started out with a common initial model, there were significant dissimilarities between the resulting models of each group after group work 2. These models had to be merged into one model. The Conference Leader started out by asking the plenary whether there were any of the three models that seemed to be a good place to start. We particularly asked whether any of the models seemed to contain much of what the other two models also had tried to achieve. After finding such a starting point, we gathered all the participants in front of that model. We then picked elements (processes/ products) from the other models to replace parts of or add elements to the model we were working on. The Conference Leader also helped the participants question the contents of the model. This made sure that all parts of the process seemed to be covered, that terms and concepts matched, that all products had a process and vice versa, and that the end products made sense in terms of who the customers were. As the process model was to be used as a basis for the design of an intranet, it was important that the participants could also say something about what kind of information or tools they required in each sub-process in order to perform good and effective work. We held on to the same heterogeneous groups that we had in group work 2, and divided the sub-processes of the resulting joint process model between the groups. Each group used yellow stickers to attach the required information and tools to the process model on the wall. They could also signal if they required information that had been produced in earlier processes, or produced information that could be used later. This group work did not generate a very systematic input on information and tools, but the sum of the four conferences created valuable input to the further work. Besides, it gave valuable information on what meaning the participants attached to each sub-process in the joint model. The outcome of each of the modeling conferences was documented in a report, which was distributed to all the participants. Thus they could check that the input they had given to the project was correctly represented. The final task was to merge the resulting models from the four conferences into one ICG model. This was done at a change agent training seminar, where we gathered the process owners and change agents from all the four prioritized processes in order to give them a basic training in what the project, process thinking, and modeling conferences were all about. We put all the processes next to each other on one paper, and asked them to find a common model that would Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


Figure 4. Process execution process — Final model (Sub-processes are not shown.) Position and sell project

Offer

Negotiations

Contract

Plan project work

Project plan

Perform project work

Project results

Manage project

Project performed according to plan

Evaluate project performance

Completed project with evaluation report

do justice to all the models. The resulting models were rather dissimilar in visual image and complexity. But as one would expect, the main flow of the process was rather similar. This time, the participants were asked to take into consideration that the model needed to fit on a computer screen, and therefore it would be desirable to create several levels of processes, where some processes are sub-processes and others are sub-sub-processes. The resulting top-level model is shown in Figure 4. The final model of the Project Execution process was turned over to the process group. They continued the work of making the model consistent and turning the model into an intranet tool, both with the intention of simplifying work and supporting the continuous reconstruction of a joint Project Execution process.

EVALUATIONS OF THE RESULTS OF USING THE TECHNIQUE Two different research projects investigated the use and results of the process models developed through the modeling conferences. Følstad (2000, 2002) used a survey method to map the relationships between direct participation in the modeling conferences, and the dependent variables of acceptance, ownership, and use of the resulting process models. Håkonsen and Carlsen (2000) performed group interviews in a study focusing on communities of practice in ICG. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


207

Følstad’s investigation was performed as two surveys, at two different points in time. The first survey was conducted right after the modeling conference, where only the participants of the modeling conference were invited to respond. The second survey was distributed six weeks after the general deployment of the process model to the whole organization. In the second survey, both participants of the modeling conference and other workers in the company were invited to respond, in order to get a broad sample. The respondents of the two groups were matched in regard to work experience and number of years of employment in the company. This was done in order to minimize chances of spurious effects in the between groups analysis. The surveys included measurements on the three dependent variables: 1. Acceptance of the model 2. Ownership of the model 3. Use of the model

Acceptance and Ownership Acceptance of the model was defined as the employee’s evaluation of the model as an appropriate description of her own work (Følstad, 2002). If the employee regarded the model as a highly appropriate description, the acceptance score should be correspondingly high and vice versa. Ownership of the model was understood as the employee’s enthusiasm and feeling of responsibility in regard to the process model. High ownership scores indicate high enthusiasm and a feeling of responsibility. The measurements of the variables were constructed specifically for this task, according to the guidelines of DeVellis (1991). The measurements were based on Likert scale items, ranging from 1 (strongly disagree) to 7 (strongly agree). Negatively phrased questions were transposed. Scores were developed as mean values. The measurement for acceptance included four items in the first survey and six similarly worded items in the second survey. The measurement for ownership included the same four items in both surveys. Acceptance and ownership of the model was measured in both surveys, but in the first survey the respondents were asked about their acceptance and ownership of the preliminary model developed at the modeling conference they had participated in themselves. In the second survey all respondents were asked to state their acceptance and ownership of the intranet-ready process model, as it had been developed by the process group. Thus it was possible to investigate whether the acceptance and ownership changed as the model was developed further, outside the control of the participants of the modeling conferences. The results from the first survey, distributed to the participants of the modeling conference only, showed that the respondents had a mean acceptance score of M=4.8 (SD=1.0, N=34) and a mean ownership score of M=5.0 (SD=1.0, N=34). Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.


In the second survey (N=78), including both participants of the modeling conference (n=23) and others (n=55), the modeling conference participants had a mean acceptance score of M=5.4 (SD=0.8) and a mean ownership score of M=5.0 (SD=1.0). The other respondents had a mean acceptance score of M=4.5 (SD=0.9) and a mean ownership score of M=4.6 (SD=1.3). Differences in the modeling conference participants’ scores on acceptance and ownership across the two surveys were investigated through independent samples T-tests. The acceptance score of the second survey was significantly higher than that of the first survey (t(50), p

Information Modeling Methods and Methodologies

Information Modeling Methods and Methodologies (Advanced Topics of Database Research)

Intelligent Music Information Systems Tools and Methodologies

Information Criteria and Statistical Modeling

Optimal Information Modeling Techniques

Building Information Modeling

Information Security and Ethics: Concepts, Methodologies, Tools and Applications

Modeling Decisions: Information Fusion and Aggregation Operators

Information and complexity in statistical modeling

Information and complexity in statistical modeling

Information Modeling and Relational Databases, Second Edition

Modeling Decisions: Information Fusion and Aggregation Operators

Information Modeling the EXPRESS Way

Information Modeling for Internet Applications

Conceptual Modeling of Information Systems

Methods of Information Geometry

Methods of Information Geometry

Information Communication Technologies: Concepts, Methodologies, Tools, and Applications

Enterprise Information Systems: Concepts, Methodologies, Tools and Applications

Information Communication Technologies: Concepts, Methodologies, Tools, and Applications

Technologies and Methodologies for Evaluating Information Technology in Business

Analytical Methods in Fuzzy Modeling and Control

Face Processing: Advanced Modeling and Methods

Mathematical Modeling and Methods of Option Pricing

Mathematical modeling and methods of option pricing

Mathematical Modeling and Methods of Option Pricing

Information-spectrum methods in information theory

Information-Spectrum Methods in Information Theory

Martingale methods in financial modeling

Martingale Methods in Financial Modeling

Information Modeling Methods and Methodologies